topics.or.jp
robots.txt

Robots Exclusion Standard data for topics.or.jp

Resource Scan

Scan Details

Site Domain topics.or.jp
Base Domain topics.or.jp
Scan Status Ok
Last Scan2024-11-07T12:53:11+00:00
Next Scan 2024-11-14T12:53:11+00:00

Last Scan

Scanned2024-11-07T12:53:11+00:00
URL https://www.topics.or.jp/robots.txt
Domain IPs 210.148.177.166
Response IP 210.148.177.166
Found Yes
Hash 585c5b2a6214fa1de7a6947918afce1a06c301561903f19eaae41457bc16d489
SimHash 5b1df950edd3

Groups

ccbot
chatgpt-user
gptbot
google-extended
icc-crawler
claudebot
anthropic-ai
claude-web
perplexitybot
perplexity-ai
applebot-extended
cohere-ai

Rule Path
Disallow /

*

Rule Path
Disallow /search
Disallow /ud/pressrelease/
Disallow /articles/gallery/
Disallow /articles/amp/

Other Records

Field Value
sitemap https://www.topics.or.jp/sitemap.xml