this.kiji.is
robots.txt

Robots Exclusion Standard data for this.kiji.is

Resource Scan

Scan Details

Site Domain this.kiji.is
Base Domain kiji.is
Scan Status Ok
Last Scan2024-10-27T05:55:44+00:00
Next Scan 2024-11-26T05:55:44+00:00

Last Scan

Scanned2024-10-27T05:55:44+00:00
URL https://this.kiji.is/robots.txt
Redirect https://nordot.app/robots.txt
Redirect Domain nordot.app
Redirect Base nordot.app
Domain IPs 172.253.118.121, 2404:6800:4003:c02::79
Redirect IPs 2001:4860:4802:32::15, 2001:4860:4802:34::15, 2001:4860:4802:36::15, 2001:4860:4802:38::15, 216.239.32.21, 216.239.34.21, 216.239.36.21, 216.239.38.21
Response IP 216.239.34.21
Found Yes
Hash 3dafc2eda6ac470d2ca53a342c48103e9577b73f18af9743a452fa6de07d6c22
SimHash 7a9cd944e593

Groups

ccbot

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

cohere-ai

Rule Path
Disallow /

omgili

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

icc-crawler

Rule Path
Disallow /

applebot-extended

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

claude-web

Rule Path
Disallow /

perplexitybot

Rule Path
Disallow /

perplexity-ai

Rule Path
Disallow /

*

No rules defined. All paths allowed.

Other Records

Field Value
sitemap https://nordot.app/sitemap.xml