news.web.nhk
robots.txt

Robots Exclusion Standard data for news.web.nhk

Resource Scan

Scan Details

Site Domain news.web.nhk
Base Domain web.nhk
Scan Status Ok
Last Scan2025-12-02T23:26:19+00:00
Next Scan 2025-12-16T23:26:19+00:00

Last Scan

Scanned2025-12-02T23:26:19+00:00
URL https://news.web.nhk/robots.txt
Domain IPs 23.53.118.238, 23.53.118.248, 2600:1413:5000:3::1736:7688, 2600:1413:5000:3::1736:76ac
Response IP 2.17.106.15
Found Yes
Hash d3d1ad4a2cc684de0ef137bbcf433a92303eef45aa8263cc114d8acaf2b0a8fb
SimHash 5014390780d7

Groups

gptbot
chatgpt-user
claudebot
google-extended
applebot-extended
anthropic-ai
cohere-ai
ccbot
icc-crawler
bytespider
oai-searchbot
perplexitybot
meta-externalagent
perplexity-ai
perplexity-user

Rule Path
Disallow /

Other Records

Field Value
sitemap https://news.web.nhk/sitemap.xml