Log In Sign Up

all.nikkeibp.co.jp
robots.txt

Robots Exclusion Standard data for all.nikkeibp.co.jp

Resource Scan

Scan Details

Site Domain all.nikkeibp.co.jp
Base Domain nikkeibp.co.jp
Scan Status Ok
Last Scan4/2/2025, 5:09:28 PM
Next Scan 5/2/2025, 5:09:28 PM

Last Scan

Scanned4/2/2025, 5:09:28 PM
URL https://all.nikkeibp.co.jp/robots.txt
Domain IPs 3.170.229.114, 3.170.229.127, 3.170.229.89, 3.170.229.97
Response IP 3.170.229.97
Found Yes
Hash d07135000047b41f8a7068d1f1444a92ae24226d8ee14728f8615f1f796e971b
SimHash 7055e901c193

Groups

*

Rule Path
Disallow /

diffbot

Rule Path
Disallow /

facebookbot

Rule Path
Disallow /

omgili

Rule Path
Disallow /

applebot-extended

Rule Path
Disallow /

meta-externalagent

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

cohere-ai

Rule Path
Disallow /

claude-web

Rule Path
Disallow /

piplbot

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

perplexitybot

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

oai-searchbot

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

icc-crawler

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

perplexity-ai

Rule Path
Disallow /

timpibot

Rule Path
Disallow /

webzio-extended

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

webzio

Rule Path
Disallow /
Back to top