news.tv-asahi.co.jp
robots.txt

Robots Exclusion Standard data for news.tv-asahi.co.jp

Resource Scan

Scan Details

Site Domain news.tv-asahi.co.jp
Base Domain tv-asahi.co.jp
Scan Status Ok
Last Scan2024-10-08T21:52:08+00:00
Next Scan 2024-11-07T21:52:08+00:00

Last Scan

Scanned2024-10-08T21:52:08+00:00
URL https://news.tv-asahi.co.jp/robots.txt
Domain IPs 151.101.1.91, 151.101.129.91, 151.101.193.91, 151.101.65.91
Response IP 199.232.45.91
Found Yes
Hash 141b45ac2ed73d4e1ce3ee28a890bae50b6d54dd769da89ed132b4990070a42a
SimHash 2a14db066353

Groups

googlebot-image

Rule Path
Disallow /articles_img/000267729_640.jpg

*

Rule Path
Disallow /mb/
Disallow /special/recfrom311/parts/
Disallow /special/recfrom311/en/parts/

chatgpt-user

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

icc-crawler

Rule Path
Disallow /

scrapy

Rule Path
Disallow /