curiouskasturi.com
robots.txt

Robots Exclusion Standard data for curiouskasturi.com

Resource Scan

Scan Details

Site Domain curiouskasturi.com
Base Domain curiouskasturi.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonCouldn't establish SSL connection.
Last Scan2025-10-03T06:41:37+00:00
Next Scan 2026-01-01T06:41:37+00:00

Last Successful Scan

Scanned2025-06-05T23:53:03+00:00
URL https://curiouskasturi.com/robots.txt
Domain IPs 104.21.86.164, 172.67.222.27, 2606:4700:3033::6815:56a4, 2606:4700:3035::ac43:de1b
Response IP 172.67.222.27
Found Yes
Hash 6524b808f218a7879f299296a505deb0f9559f7923f9863a2e64a690bec6f747
SimHash 41043a54e413

Groups

*

Rule Path
Allow /

google-extended

Rule Path
Disallow /

oai-searchbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

bingbot (it blocks bing search engine too)

Rule Path
Disallow /

perplexitybot

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

cohere-ai

Rule Path
Disallow /

meta-externalagent

Rule Path
Disallow /

Other Records

Field Value
sitemap https://curiouskasturi.com/sitemap.xml