Log In Sign Up

everything2.net
robots.txt

Robots Exclusion Standard data for everything2.net

Resource Scan

Scan Details

Site Domain everything2.net
Base Domain everything2.net
Scan Status Ok
Last Scan2025-10-19T15:57:56+00:00
Next Scan 2025-10-26T15:57:56+00:00

Last Scan

Scanned2025-10-19T15:57:56+00:00
URL https://everything2.net/robots.txt
Domain IPs 35.155.142.166, 52.11.170.171
Response IP 35.155.142.166
Found Yes
Hash b2b5323edcd02474f697f8b6012cc3b67b95da68324668caffe3f1ee17adca97
SimHash c83cde72a213

Groups

ahrefsbot

Rule Path
Disallow /

yandex

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 2

gptbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

terracotta

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://everything2.com/sitemap/index.xml
Back to top

Warnings

  • 2 invalid lines.
Back to top