natgeo.nikkeibp.co.jp
robots.txt

Robots Exclusion Standard data for natgeo.nikkeibp.co.jp

Resource Scan

Scan Details

Site Domain natgeo.nikkeibp.co.jp
Base Domain nikkeibp.co.jp
Scan Status Ok
Last Scan2024-08-29T09:40:23+00:00
Next Scan 2024-09-28T09:40:23+00:00

Last Scan

Scanned2024-08-29T09:40:23+00:00
URL https://natgeo.nikkeibp.co.jp/robots.txt
Domain IPs 54.64.191.108, 54.65.185.3
Response IP 54.64.191.108
Found Yes
Hash f1cc64fa014928f5d5930cb3bf33205c0ff7d3c8815d5a9ebbf8bed164c2f23f
SimHash 781d594482b2

Groups

*

Rule Path
Disallow /nng/trans/juko/files/
Disallow /nng/trans/juko/april/files/

diffbot

Rule Path
Disallow /

facebookbot

Rule Path
Disallow /

omgili

Rule Path
Disallow /

applebot-extended

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

cohere-ai

Rule Path
Disallow /

claude-web

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

perplexitybot

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

icc-crawler

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

perplexity-ai

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

Other Records

Field Value
sitemap https://natgeo.nikkeibp.co.jp/nng/sitemap.xml