xtech.nikkei.com
robots.txt

Robots Exclusion Standard data for xtech.nikkei.com

Resource Scan

Scan Details

Site Domain xtech.nikkei.com
Base Domain nikkei.com
Scan Status Ok
Last Scan2024-05-23T17:51:54+00:00
Next Scan 2024-06-22T17:51:54+00:00

Last Scan

Scanned2024-05-23T17:51:54+00:00
URL https://xtech.nikkei.com/robots.txt
Domain IPs 52.192.137.107, 52.194.206.252
Response IP 52.194.206.252
Found Yes
Hash 0abcc02b6d42937b0586c3e4867898bf90919a59f75a2bae5bcda3dbdae1859c
SimHash 18044941a113

Groups

*

Rule Path
Disallow /data/

google-extended

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

icc-crawler

Rule Path
Disallow /

omgili

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

cohere-ai

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

Other Records

Field Value
sitemap https://xtech.nikkei.com/sitemap.xml