chatsworth.com
robots.txt

Robots Exclusion Standard data for chatsworth.com

Resource Scan

Scan Details

Site Domain chatsworth.com
Base Domain chatsworth.com
Scan Status Ok
Last Scan2026-03-19T15:17:02+00:00
Next Scan 2026-04-18T15:17:02+00:00

Last Scan

Scanned2026-03-19T15:17:02+00:00
URL https://chatsworth.com/robots.txt
Redirect https://www.chatsworth.com/robots.txt
Redirect Domain www.chatsworth.com
Redirect Base chatsworth.com
Domain IPs 172.66.40.171, 172.66.43.85, 2606:4700:3108::ac42:28ab, 2606:4700:3108::ac42:2b55
Redirect IPs 172.66.40.171, 172.66.43.85, 2606:4700:3108::ac42:28ab, 2606:4700:3108::ac42:2b55
Response IP 172.66.40.171
Found Yes
Hash c48a304aa6673721cf65e25ef2252697bf1c3857b1bdf033d6a990c5c3372124
SimHash 71164149e6d5

Groups

gptbot

Rule Path
Allow /

oai-searchbot

Rule Path
Allow /

chatgpt-user

Rule Path
Allow /

chatgpt-user/2.0

Rule Path
Allow /

anthropic-ai

Rule Path
Allow /

claudebot

Rule Path
Allow /

claude-web

Rule Path
Allow /

perplexitybot

Rule Path
Allow /

perplexity-user

Rule Path
Allow /

google-extended

Rule Path
Disallow /

bingbot

Rule Path
Allow /

amazonbot

Rule Path
Allow /

applebot

Rule Path
Allow /

applebot-extended

Rule Path
Allow /

facebookbot

Rule Path
Allow /

meta-externalagent

Rule Path
Allow /

linkedinbot

Rule Path
Allow /

bytespider

Rule Path
Allow /

duckassistbot

Rule Path
Allow /

cohere-ai

Rule Path
Allow /

ai2bot

Rule Path
Allow /

ccbot

Rule Path
Allow /

diffbot

Rule Path
Allow /

omgili

Rule Path
Allow /

timpibot

Rule Path
Allow /

youbot

Rule Path
Allow /

mistralai-user

Rule Path
Allow /

*

Rule Path
Disallow