Log In Sign Up

av19.is
robots.txt

Robots Exclusion Standard data for av19.is

Resource Scan

Scan Details

Site Domain av19.is
Base Domain av19.is
Scan Status Ok
Last Scan2026-03-02T02:34:19+00:00
Next Scan 2026-04-01T02:34:19+00:00

Last Scan

Scanned2026-03-02T02:34:19+00:00
URL https://av19.is/robots.txt
Domain IPs 104.21.8.66, 172.67.138.161, 2606:4700:3036::ac43:8aa1, 2606:4700:3037::6815:842
Response IP 172.67.138.161
Found Yes
Hash 9e0d22b62cc6f77f998f9d25ec5bdcad9128ffe252107e3bf41d78a3c6f5896d
SimHash 50904a00e575

Groups

googlebot

Rule Path
Allow /

bingbot

Rule Path
Allow /

yandex

Rule Path
Allow /

naverbot

Rule Path
Allow /

yeti

Rule Path
Allow /

gptbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

perplexitybot

Rule Path
Disallow /

*

Rule Path
Allow /
Back to top

Comments

  • Allow all SEO crawlers
  • Block AI training crawlers
  • Default allow
Back to top