mainichi.jp
robots.txt

Robots Exclusion Standard data for mainichi.jp

Resource Scan

Scan Details

Site Domain mainichi.jp
Base Domain mainichi.jp
Scan Status Ok
Last Scan2024-09-25T17:16:05+00:00
Next Scan 2024-10-02T17:16:05+00:00

Last Scan

Scanned2024-09-25T17:16:05+00:00
URL https://mainichi.jp/robots.txt
Domain IPs 3.165.102.25, 3.165.102.29, 3.165.102.57, 3.165.102.70
Response IP 3.165.102.29
Found Yes
Hash 22450b2365dcea4baa50adc73eb131a81776786052e77e060f4e1a0f8b7b5ff7
SimHash 1a34db008173

Groups

ccbot

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

icc-crawler

Rule Path
Disallow /

googleother

Rule Path
Disallow /