special.nikkeibp.co.jp
robots.txt

Robots Exclusion Standard data for special.nikkeibp.co.jp

Resource Scan

Scan Details

Site Domain special.nikkeibp.co.jp
Base Domain nikkeibp.co.jp
Scan Status Ok
Last Scan2024-06-21T12:03:07+00:00
Next Scan 2024-07-21T12:03:07+00:00

Last Scan

Scanned2024-06-21T12:03:07+00:00
URL https://special.nikkeibp.co.jp/robots.txt
Domain IPs 52.195.127.205, 54.95.109.157
Response IP 54.95.109.157
Found Yes
Hash 721510335fb7c1fd570e88d50392e87ce03fc474a43d54c5953dab06174f944c
SimHash 1a844965a013

Groups

google-extended

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

icc-crawler

Rule Path
Disallow /

omgili

Rule Path
Disallow /

applebot-extended

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

cohere-ai

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

Other Records

Field Value
sitemap https://special.nikkeibp.co.jp/rss/sitemap_index.gz