bizgate.nikkei.co.jp
robots.txt

Robots Exclusion Standard data for bizgate.nikkei.co.jp

Resource Scan

Scan Details

Site Domain bizgate.nikkei.co.jp
Base Domain nikkei.co.jp
Scan Status Ok
Last Scan2024-09-06T10:02:57+00:00
Next Scan 2024-10-06T10:02:57+00:00

Last Scan

Scanned2024-09-06T10:02:57+00:00
URL https://bizgate.nikkei.co.jp/robots.txt
Redirect https://bizgate.nikkei.com/robots.txt
Redirect Domain bizgate.nikkei.com
Redirect Base nikkei.com
Domain IPs 52.84.229.109, 52.84.229.22, 52.84.229.45, 52.84.229.89
Redirect IPs 104.18.0.125, 104.18.1.125, 2606:4700::6812:17d, 2606:4700::6812:7d
Response IP 104.18.1.125
Found Yes
Hash 1dd5661791e416ae2ef57a3194e729a3d2bbab85e6a13ebe1c8523c5463173cf
SimHash 70186940a292

Groups

*

Rule Path
Disallow /api/

*

Rule Path
Disallow /login/

ccbot

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

cohere-ai

Rule Path
Disallow /

omgili

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

icc-crawler

Rule Path
Disallow /

applebot-extended

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

claude-web

Rule Path
Disallow /

perplexitybot

Rule Path
Disallow /

perplexity-ai

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

diffbot

Rule Path
Disallow /

facebookbot

Rule Path
Disallow /