social.kerkour.com
robots.txt

Robots Exclusion Standard data for social.kerkour.com

Resource Scan

Scan Details

Site Domain social.kerkour.com
Base Domain kerkour.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-05-26T20:12:38+00:00
Next Scan 2024-07-25T20:12:38+00:00

Last Successful Scan

Scanned2024-03-05T20:09:40+00:00
URL https://social.kerkour.com/robots.txt
Redirect https://kerkour.com/robots.txt
Redirect Domain kerkour.com
Redirect Base kerkour.com
Domain IPs 216.24.57.253, 216.24.57.3
Redirect IPs 104.21.18.121, 172.67.181.208, 2606:4700:3033::6815:1279, 2606:4700:3036::ac43:b5d0
Response IP 172.67.181.208
Found Yes
Hash 5d6505b685b16d1eb2af86a138d9968caf26f182397614d35aa78092c8c55b0c
SimHash 7014c0708680

Groups

*

Rule Path
Allow /

*

Rule Path
Disallow /__pingoo

ia_archiver

Rule Path
Disallow /

archive.org_bot

Rule Path
Disallow /

baiduspider-video

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

awariorssbot
awariosmartbot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

claude-web

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

facebookbot

Rule Path
Disallow /

magpie-crawler

Rule Path
Disallow /

omgili

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

perplexitybot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://kerkour.com/sitemap.xml