automobile-detail.com
robots.txt

Robots Exclusion Standard data for automobile-detail.com

Resource Scan

Scan Details

Site Domain automobile-detail.com
Base Domain automobile-detail.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2025-03-26T14:51:51+00:00
Next Scan 2025-06-24T14:51:51+00:00

Last Successful Scan

Scanned2024-05-31T14:50:13+00:00
URL https://automobile-detail.com/robots.txt
Domain IPs 104.21.11.239, 172.67.150.177, 2606:4700:3031::ac43:96b1, 2606:4700:3032::6815:bef
Response IP 104.21.11.239
Found Yes
Hash 2340ad7e88fe206cfb5915a40dbeac37f0d0cca36c1be38a3389e9eb94165acc
SimHash 423cc960e732

Groups

*

Rule Path
Allow /*.css
Allow /*.js

*

Rule Path
Disallow /wp-admin/
Allow /wp-admin/admin-ajax.php

spbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

baiduspider

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

proximic

Rule Path
Disallow /

grapeshot

Rule Path
Disallow /

slurp

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

yandex

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

yandexbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

mj12bot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

mediatoolkitbot

Rule Path
Disallow /

linguee

Rule Path
Disallow /

alphaseobot

Rule Path
Disallow /

xenu's link sleuth 1.1c

Rule Path
Disallow /

xenu link sleuth/1.3.8

Rule Path
Disallow /

velenpublicwebcrawler

Rule Path
Disallow /

kingkevinbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

ltx71

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /