47news.jp
robots.txt

Robots Exclusion Standard data for 47news.jp

Resource Scan

Scan Details

Site Domain 47news.jp
Base Domain 47news.jp
Scan Status Ok
Last Scan2024-05-10T16:53:05+00:00
Next Scan 2024-05-17T16:53:05+00:00

Last Scan

Scanned2024-05-10T16:53:05+00:00
URL https://47news.jp/robots.txt
Redirect https://www.47news.jp/robots.txt
Redirect Domain www.47news.jp
Redirect Base 47news.jp
Domain IPs 108.157.254.110, 108.157.254.14, 108.157.254.16, 108.157.254.32
Redirect IPs 108.157.254.110, 108.157.254.14, 108.157.254.16, 108.157.254.32, 2600:9000:2753:0:1d:faa7:c9c0:93a1, 2600:9000:2753:2800:1d:faa7:c9c0:93a1, 2600:9000:2753:3000:1d:faa7:c9c0:93a1, 2600:9000:2753:4600:1d:faa7:c9c0:93a1, 2600:9000:2753:8e00:1d:faa7:c9c0:93a1, 2600:9000:2753:c600:1d:faa7:c9c0:93a1, 2600:9000:2753:e000:1d:faa7:c9c0:93a1, 2600:9000:2753:ec00:1d:faa7:c9c0:93a1
Response IP 108.157.254.110
Found Yes
Hash a1420c9dc863bda34154ccc34fc6674948a54cd65ed740c53f77805c7d268ed8
SimHash 3a1fd94361e0

Groups

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 600

msnbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 600

ccbot

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

*

Rule Path
Disallow /news/column/shasetsu/*.html$
Disallow /news/column/47column/*.html$
Disallow /news/column/47column/*.html$
Disallow /search/
Disallow /search
Disallow /localnews/chiikisaisei/taisho/2019/*.html$
Disallow /localnews/chiikisaisei/taisho/2018/*.html$
Disallow /localnews/chiikisaisei/taisho/2017/*.html$
Disallow /localnews/chiikisaisei/taisho/2016/*.html$
Disallow /localnews/chiikisaisei/taisho/2015/*.html$
Disallow /localnews/chiikisaisei/taisho/2014/*.html$
Disallow /localnews/chiikisaisei/taisho/2013/*.html$
Disallow /localnews/chiikisaisei/taisho/2012/*.html$
Disallow /localnews/chiikisaisei/taisho/2011/*.html$
Disallow /localnews/chiikisaisei/taisho/2010/*.html$
Disallow /localnews/chiikisaisei/taisho/2020/*.html$
Disallow /api/*

Comments

  • 2022-12-23 update