readytogo.net
robots.txt

Robots Exclusion Standard data for readytogo.net

Resource Scan

Scan Details

Site Domain readytogo.net
Base Domain readytogo.net
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-09-27T07:19:54+00:00
Next Scan 2024-12-26T07:19:54+00:00

Last Successful Scan

Scanned2024-05-31T07:13:48+00:00
URL https://readytogo.net/robots.txt
Redirect https://www.readytogo.net/robots.txt
Redirect Domain www.readytogo.net
Redirect Base readytogo.net
Domain IPs 104.21.23.57, 172.67.209.59, 2606:4700:3030::6815:1739, 2606:4700:3037::ac43:d13b
Redirect IPs 104.21.23.57, 172.67.209.59, 2606:4700:3030::6815:1739, 2606:4700:3037::ac43:d13b
Response IP 104.21.23.57
Found Yes
Hash c5fc4df8eeb177fdea93b2497dad9485501a3f16f23c7534a19542641ede7f93
SimHash 6478ca707377

Groups

*

Rule Path
Disallow cron.php
Disallow search.php
Disallow online.php
Disallow calendar.php

googlebot
googlebot-mobile
bingbot
msnbot

Rule Path
Disallow /smb/members/
Disallow /smb/archive/
Disallow /smb/posts/
Disallow /smb/goto/
Disallow /smb/online/
Disallow /smb/misc/style

fasterfox
baiduspider
baiduspider-video
baiduspider-image
magpie-crawler
yandex
claude-web
twitterbot
facebookbot
gptbot
google-extended
ccbot

Rule Path
Disallow /

proximic

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5