Log In Sign Up

skfo.pro
robots.txt

Robots Exclusion Standard data for skfo.pro

Resource Scan

Scan Details

Site Domain skfo.pro
Base Domain skfo.pro
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2026-02-14T23:00:40+00:00
Next Scan 2026-04-15T23:00:40+00:00

Last Successful Scan

Scanned2025-11-24T22:48:51+00:00
URL https://www.skfo.pro/robots.txt
Domain IPs 104.21.50.86, 172.67.159.157, 2606:4700:3031::ac43:9f9d, 2606:4700:3035::6815:3256
Response IP 172.67.159.157
Found Yes
Hash b7e540854baf5170dce60a2cfd78630b1b4e9d9bc542f06930bd772e912f72ef
SimHash 401ddcb0e593

Groups

googlebot

Rule Path
Allow /

bingbot

Rule Path
Allow /

slurp

Rule Path
Allow /

duckduckbot

Rule Path
Allow /

baiduspider

Rule Path
Allow /

yandexbot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 10

facebookexternalhit

Rule Path
Allow /

twitterbot

Rule Path
Allow /

gptbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

claude-web

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

*

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10
Back to top

Other Records

Field Value
sitemap http://www.skfo.pro/api/sitemap-index
Back to top

Comments

  • Allow major search engines
  • Block AI scrapers
  • Block bad bots
Back to top