roadsideamerica.com
robots.txt

Robots Exclusion Standard data for roadsideamerica.com

Resource Scan

Scan Details

Site Domain roadsideamerica.com
Base Domain roadsideamerica.com
Scan Status Ok
Last Scan2024-09-19T18:35:03+00:00
Next Scan 2024-09-26T18:35:03+00:00

Last Scan

Scanned2024-09-19T18:35:03+00:00
URL https://roadsideamerica.com/robots.txt
Redirect https://www.roadsideamerica.com/robots.txt
Redirect Domain www.roadsideamerica.com
Redirect Base roadsideamerica.com
Domain IPs 54.193.158.3
Redirect IPs 18.165.183.117, 18.165.183.15, 18.165.183.17, 18.165.183.40, 2600:9000:2175:1800:16:937e:3f80:93a1, 2600:9000:2175:1a00:16:937e:3f80:93a1, 2600:9000:2175:2200:16:937e:3f80:93a1, 2600:9000:2175:2600:16:937e:3f80:93a1, 2600:9000:2175:7600:16:937e:3f80:93a1, 2600:9000:2175:9600:16:937e:3f80:93a1, 2600:9000:2175:c200:16:937e:3f80:93a1, 2600:9000:2175:e00:16:937e:3f80:93a1
Response IP 3.164.206.18
Found Yes
Hash 7604e9ff658b9f6217fee3a7d8b62d1d15b96b9ad636ee4af366cdedc89de7e0
SimHash 7736e9a6e3d2

Groups

*

Rule Path
Disallow /smiler/
Disallow /travel/
Disallow /tipSubmission.html
Disallow /mailus.html
Disallow /mailthisexit.html
Disallow /shared/randomSite.php
Disallow /admin/
Disallow /uploader/

titan

Rule Path
Disallow /

emailcollector

Rule Path
Disallow /

emailsiphon

Rule Path
Disallow /

emailwolf

Rule Path
Disallow /

extractorpro

Rule Path
Disallow /

dialect

Rule Path
Disallow /

webaroobot

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

perplexitybot

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

claude-web

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

applebot-extended

Rule Path
Disallow /private/

Warnings

  • 1 invalid line.