roadsideamerica.com
robots.txt

Robots Exclusion Standard data for roadsideamerica.com

Resource Scan

Scan Details

Site Domain roadsideamerica.com
Base Domain roadsideamerica.com
Scan Status Ok
Last Scan2024-10-31T19:37:56+00:00
Next Scan 2024-11-07T19:37:56+00:00

Last Scan

Scanned2024-10-31T19:37:56+00:00
URL https://roadsideamerica.com/robots.txt
Redirect https://www.roadsideamerica.com/robots.txt
Redirect Domain www.roadsideamerica.com
Redirect Base roadsideamerica.com
Domain IPs 54.193.158.3
Redirect IPs 2600:9000:2175:2600:16:937e:3f80:93a1, 2600:9000:2175:3600:16:937e:3f80:93a1, 2600:9000:2175:7200:16:937e:3f80:93a1, 2600:9000:2175:a600:16:937e:3f80:93a1, 2600:9000:2175:aa00:16:937e:3f80:93a1, 2600:9000:2175:b000:16:937e:3f80:93a1, 2600:9000:2175:e000:16:937e:3f80:93a1, 2600:9000:2175:f000:16:937e:3f80:93a1, 3.160.212.21, 3.160.212.37, 3.160.212.80, 3.160.212.89
Response IP 3.164.206.18
Found Yes
Hash 7604e9ff658b9f6217fee3a7d8b62d1d15b96b9ad636ee4af366cdedc89de7e0
SimHash 7736e9a6e3d2

Groups

*

Rule Path
Disallow /smiler/
Disallow /travel/
Disallow /tipSubmission.html
Disallow /mailus.html
Disallow /mailthisexit.html
Disallow /shared/randomSite.php
Disallow /admin/
Disallow /uploader/

titan

Rule Path
Disallow /

emailcollector

Rule Path
Disallow /

emailsiphon

Rule Path
Disallow /

emailwolf

Rule Path
Disallow /

extractorpro

Rule Path
Disallow /

dialect

Rule Path
Disallow /

webaroobot

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

perplexitybot

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

claude-web

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

applebot-extended

Rule Path
Disallow /private/

Warnings

  • 1 invalid line.