isabelnet.com
robots.txt

Robots Exclusion Standard data for isabelnet.com

Resource Scan

Scan Details

Site Domain isabelnet.com
Base Domain isabelnet.com
Scan Status Ok
Last Scan2025-08-15T15:34:44+00:00
Next Scan 2025-08-22T15:34:44+00:00

Last Scan

Scanned2025-08-15T15:34:44+00:00
URL https://isabelnet.com/robots.txt
Redirect https://www.isabelnet.com/robots.txt
Redirect Domain www.isabelnet.com
Redirect Base isabelnet.com
Domain IPs 2001:470:1:3f0::d8da:f813, 216.218.248.19
Redirect IPs 2001:470:1:3f0::d8da:f813, 216.218.248.19
Response IP 216.218.248.19
Found Yes
Hash 4276ffc796c37af0d974287754934867b637d29ce1d1087f3205f71f79f5e169
SimHash b21dd8b35ebb

Groups

*

Rule Path
Disallow /*.cgi$
Disallow /*.sql$
Disallow /*.tar.gz$
Disallow /*.zip$

Other Records

Field Value
crawl-delay 5

ia_archiver

Rule Path
Disallow /

archive.org_bot
itsapic.com_crawler

Rule Path
Disallow /

exabot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

semantic-visions.com crawler

Rule Path
Disallow /

scrapy

Rule Path
Disallow /

r6_commentreader

Rule Path
Disallow /