Log In Sign Up

webcam.io
robots.txt

Robots Exclusion Standard data for webcam.io

Resource Scan

Scan Details

Site Domain webcam.io
Base Domain webcam.io
Scan Status Ok
Last Scan2025-11-26T23:33:45+00:00
Next Scan 2025-12-26T23:33:45+00:00

Last Scan

Scanned2025-11-26T23:33:45+00:00
URL https://webcam.io/robots.txt
Domain IPs 104.21.13.94, 172.67.167.236
Response IP 104.21.13.94
Found Yes
Hash 9fbd489fcf34c1224eb8ee1a2241daa32fc5665b0d06cf8e9c542dba9d317c3c
SimHash 421dc951e59c

Groups

googlebot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 1

bingbot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 1

slurp

Rule Path
Allow /

Other Records

Field Value
crawl-delay 1

duckduckbot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 1

baiduspider

Rule Path
Allow /

Other Records

Field Value
crawl-delay 1

yandexbot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 1

gptbot

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

claude-web

Rule Path
Disallow /

perplexitybot

Rule Path
Disallow /

youbot

Rule Path
Disallow /

meta-externalagent

Rule Path
Disallow /

facebookbot

Rule Path
Disallow /

applebot-extended

Rule Path
Disallow /

diffbot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

imagesiftbot

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

omgili

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

archive.org_bot

Rule Path
Disallow /

ia_archiver-web.archive.org

Rule Path
Disallow /

*

Rule Path
Allow /

Other Records

Field Value
crawl-delay 2
Back to top

Comments

  • Hauptsuchmaschinen - ERLAUBT
  • KI-Crawler und Datensammler - BLOCKIERT
  • Weitere potentielle KI/Datensammler
  • Internet-Archive - BLOCKIERT
  • Standard
  • Sitemap-Verweis
  • Sitemap: /sitemap.xml
Back to top