Log In Sign Up

staging.mtggoldfish.com
robots.txt

Robots Exclusion Standard data for staging.mtggoldfish.com

Resource Scan

Scan Details

Site Domain staging.mtggoldfish.com
Base Domain mtggoldfish.com
Scan Status Ok
Last Scan2024-10-29T21:35:03+00:00
Next Scan 2024-11-28T21:35:03+00:00

Last Scan

Scanned2024-10-29T21:35:03+00:00
URL https://staging.mtggoldfish.com/robots.txt
Domain IPs 34.225.155.6, 54.225.131.207
Response IP 54.225.131.207
Found Yes
Hash 391b56ed0568fbae6c4cb7bbfe4a59f00ea6adb440dfb196d6d1080abae2701b
SimHash a085cb8d745c

Groups

*

Rule Path
Disallow /ebay_listings
Disallow /tcgplayer/price_widget
Disallow /embed/decklist
Disallow /widgets/price
Disallow /widgets/price_compact
Disallow /widgets/price_compact/js
Disallow /deals/ad_js
Disallow /deck/download*
Disallow /articles/footer/js
Disallow /health
Disallow /test
Disallow /test_ab

grapeshot

Rule Path
Disallow /q
Back to top

Comments

  • See http://www.robotstxt.org/wc/norobots.html for documentation on how to use the robots.txt file
  • To ban all spiders from the entire site uncomment the next two lines:
Back to top