roadonmap.com
robots.txt

Robots Exclusion Standard data for roadonmap.com

Resource Scan

Scan Details

Site Domain roadonmap.com
Base Domain roadonmap.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2025-04-10T02:57:07+00:00
Next Scan 2025-06-09T02:57:07+00:00

Last Successful Scan

Scanned2025-02-10T00:51:41+00:00
URL https://roadonmap.com/robots.txt
Domain IPs 104.21.33.61, 172.67.141.184, 2606:4700:3033::6815:213d, 2606:4700:3033::ac43:8db8
Response IP 172.67.141.184
Found Yes
Hash 4d04bef6e54ef484987f66306e862cdec6dc5920b5e7e64c81cdfa74bf38b0c0
SimHash 560741f0c788

Groups

anthropic-ai

Rule Path
Disallow /

claude-web

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

baiduspider-video

Rule Path
Disallow /

baiduspider-image

Rule Path
Disallow /

sogou spider

Rule Path
Disallow /

youdaobot

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

megaindex.ru/2.0

Rule Path
Disallow /

mail.ru_bot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

psbot

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

npbot-1/2.0

Rule Path
Disallow /

npbot

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

domainstatsbot

Rule Path
Disallow /

seokicks

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

coccocbot-web

Rule Path
Disallow /

linespider

Rule Path
Disallow /

infotigerbot

Rule Path
Disallow /

adsbot

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

zoombot

Rule Path
Disallow /

admantx

Rule Path
Disallow /

serpstatbot

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

yandex

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

neevabot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

amazon-kendra-web-crawler-*

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

applebot

Rule Path
Disallow /

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

*

Rule Path
Disallow /pub/
Disallow /inc/
Disallow /action/
Disallow /policies/