Log In Sign Up

petstablished.com
robots.txt

Robots Exclusion Standard data for petstablished.com

Resource Scan

Scan Details

Site Domain petstablished.com
Base Domain petstablished.com
Scan Status Ok
Last Scan2024-06-26T21:37:17+00:00
Next Scan 2024-07-03T21:37:17+00:00

Last Scan

Scanned2024-06-26T21:37:17+00:00
URL https://petstablished.com/robots.txt
Domain IPs 100.25.126.195, 100.25.61.17, 3.230.33.121, 34.199.174.241, 54.243.141.95
Response IP 3.230.33.121
Found Yes
Hash 2fe4f076f9ec8eaded5c667ccddde7e878d59d25cc45aed028b43a74593ec724
SimHash 928d0d8f7150

Groups

yandex

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /
Back to top

Comments

  • See http://www.robotstxt.org/robotstxt.html for documentation on how to use the robots.txt file
  • To ban all spiders from the entire site uncomment the next two lines:
  • User-agent: *
  • Disallow: /
  • User-agent: *
  • Disallow: /adoptions/
Back to top