cobbcounty.org
robots.txt

Robots Exclusion Standard data for cobbcounty.org

Resource Scan

Scan Details

Site Domain cobbcounty.org
Base Domain cobbcounty.org
Scan Status Ok
Last Scan2025-12-27T02:58:56+00:00
Next Scan 2026-01-26T02:58:56+00:00

Last Scan

Scanned2025-12-27T02:58:56+00:00
URL https://cobbcounty.org/robots.txt
Redirect https://www.cobbcounty.gov/robots.txt
Redirect Domain www.cobbcounty.gov
Redirect Base cobbcounty.gov
Domain IPs 151.101.130.191, 151.101.194.191, 151.101.2.191, 151.101.66.191
Redirect IPs 151.101.130.191, 151.101.194.191, 151.101.2.191, 151.101.66.191, 2a04:4e42:200::703, 2a04:4e42:400::703, 2a04:4e42:600::703, 2a04:4e42::703
Response IP 199.232.114.191
Found Yes
Hash 73e65bd052ae2da59ea82b4edcefd29a8b97d27008971966348133c377d01549
SimHash 419cd9a08615

Groups

*

Rule Path
Disallow /search
Disallow /search/
Disallow /admin
Disallow /admin/
Disallow /wp-admin
Disallow /wp-admin/
Disallow *.pdf
Disallow /sites/
Disallow /user
Disallow /user/
Disallow /login
Disallow /login/
Disallow /health

yandex

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

aspiegelbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

mauibot

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

exabot

Rule Path
Disallow /

omgili

Rule Path
Disallow /
Disallow /

omgilibot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

diffbot

Rule Path
Disallow /

imagesiftbot

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

perplexitybot

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

claude-web

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

facebookbot

Rule Path
Disallow /

amazonbot

Rule Path
Allow /$
Disallow /

slurp

Rule Path
Allow /$
Disallow /

Other Records

Field Value
sitemap https://www.cobbcounty.gov/sitemap.xml

Comments

  • Bad bots
  • AI Bots
  • Busy bots
  • Yahoo

Warnings

  • 1 invalid line.