Log In Sign Up

eggstrollers.com
robots.txt

Robots Exclusion Standard data for eggstrollers.com

Resource Scan

Scan Details

Site Domain eggstrollers.com
Base Domain eggstrollers.com
Scan Status Ok
Last Scan2026-01-27T16:13:50+00:00
Next Scan 2026-02-10T16:13:50+00:00

Last Scan

Scanned2026-01-27T16:13:50+00:00
URL https://eggstrollers.com/robots.txt
Domain IPs 104.21.25.60, 172.67.223.50, 2606:4700:3032::6815:193c, 2606:4700:3035::ac43:df32
Response IP 104.21.25.60
Found Yes
Hash fcf0a480123ee5bd58bf9edfbcf3b6e2a70d5ec616b618edfe3e25f381475060
SimHash bb0c47fcde81

Groups

teoma

Rule Path
Disallow /control/
Disallow /report/

*

Rule Path
Disallow /control/
Disallow /report/
Disallow /details/goldenbull2007john/
Disallow /stream/goldenbull2007john/
Disallow /download/goldenbull2007john/
Disallow /14/items/goldenbull2007john/goldenbull2007john_djvu.txt
Back to top

Other Records

Field Value
sitemap http://archive.org/sitemap/sitemap.xml
sitemap http://archive.org/sitemap/sitemap.xml
Back to top

Comments

  • Welcome to the Archive!
  • Please crawl our files.
  • We appreciate if you can crawl responsibly.
  • Stay open!
  • slow down the ask jeeves crawler which was hitting our SE a little too fast
  • via collection pages. --Feb2008 tracey--
Back to top