Log In Sign Up

2ch-matome.net
robots.txt

Robots Exclusion Standard data for 2ch-matome.net

Resource Scan

Scan Details

Site Domain 2ch-matome.net
Base Domain 2ch-matome.net
Scan Status Ok
Last Scan2024-06-12T04:35:48+00:00
Next Scan 2024-06-19T04:35:48+00:00

Last Scan

Scanned2024-06-12T04:35:48+00:00
URL http://2ch-matome.net/robots.txt
Domain IPs 160.16.216.58, 2001:e42:102:1810:160:16:216:58
Response IP 160.16.216.58
Found Yes
Hash cd56f0ca3d0feccc36dcb7e040ee8c778a4946eecbbf5c36e9bc0fc7e2918a4a
SimHash c804ca10eb93

Groups

*
covario-ids

Rule Path
Disallow /

yahoocachesystem

Rule Path
Disallow /

*

Rule Path
Disallow /link/

ahrefsbot

Rule Path
Disallow /

bingbot
*

Rule Path
Allow /
Back to top

Other Records

Field Value
sitemap http://2ch-matome.net/sitemap.xml
Back to top

Warnings

  • 2 invalid lines.
  • `clawl-delay` is not a known field.
Back to top