paynepreownedmccoll.com
robots.txt

Robots Exclusion Standard data for paynepreownedmccoll.com

Resource Scan

Scan Details

Site Domain paynepreownedmccoll.com
Base Domain paynepreownedmccoll.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-05-04T15:40:05+00:00
Next Scan 2024-08-02T15:40:05+00:00

Last Successful Scan

Scanned2023-12-13T16:41:15+00:00
URL https://paynepreownedmccoll.com/robots.txt
Domain IPs 52.15.161.115
Response IP 52.15.161.115
Found Yes
Hash a2690a032ea19fa3054c0259e0da0bbe9092ced2ea7b0f7ece29a813b3788399
SimHash 531c8c10e294

Groups

mj12bot

Rule Path
Disallow /

aihitbot

Rule Path
Disallow /

tinytestbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

springservebot

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

seekportbot

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

coccocbot-web

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

fidget-spinner-bot

Rule Path
Disallow /

motominerbot

Rule Path
Disallow /

imagesiftbot

Rule Path
Disallow /

serpstatbot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

awariobot

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

tineye-bot-live

Rule Path
Disallow /

brightbot

Rule Path
Disallow /

bingbot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 10

*

Rule Path
Allow /