klr650.net
robots.txt

Robots Exclusion Standard data for klr650.net

Resource Scan

Scan Details

Site Domain klr650.net
Base Domain klr650.net
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-05-05T02:25:50+00:00
Next Scan 2024-08-03T02:25:50+00:00

Last Successful Scan

Scanned2024-01-06T04:03:43+00:00
URL https://klr650.net/robots.txt
Domain IPs 104.21.51.253, 172.67.192.133, 2606:4700:3031::6815:33fd, 2606:4700:3031::ac43:c085
Response IP 104.21.51.253
Found Yes
Hash 5fc6ba1efe09d12640be571dba238e4aa6b1209858caaf3f5ff8fcf5eeb8f99a
SimHash 0d5cc0b2569b

Groups

googlebot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 10

mediapartners-google

Rule Path
Allow /

Other Records

Field Value
crawl-delay 10

adsbot-google

Rule Path
Allow /

Other Records

Field Value
crawl-delay 10

googlebot-mobile

Rule Path
Allow /

Other Records

Field Value
crawl-delay 10

googlebot-video

Rule Path
Allow /

Other Records

Field Value
crawl-delay 10

googlebot-image

Rule Path
Allow /

Other Records

Field Value
crawl-delay 10

googlebot-news

Rule Path
Allow /

Other Records

Field Value
crawl-delay 10

bingbot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 10

bingpreview

Rule Path
Allow /

Other Records

Field Value
crawl-delay 10

msnbot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 10

yahoo-slurp

Rule Path
Allow /

Other Records

Field Value
crawl-delay 10

duckduckbot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 10

duckduckgo-favicons-bot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 10

ia_archiver

Rule Path
Disallow /

ia_archiver-web.archive.org

Rule Path
Disallow /

archive.org

Rule Path
Disallow /

magpie-crawler

Rule Path
Disallow /

hypestat

Rule Path
Disallow /

slurp

Rule Path
Allow /

Other Records

Field Value
crawl-delay 10

yandexbot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 10

baiduspider
baiduspider
baiduspider+
baiduspider-video
baiduspider-image
ahrefsbot
httrack
ezooms
mj12bot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.klr650.net/sitemap.xml