2ch-matome.net
robots.txt
Robots Exclusion Standard data for 2ch-matome.net
Resource Scan
Scan Details
Site Domain | 2ch-matome.net |
Base Domain | 2ch-matome.net |
Scan Status | Ok |
Last Scan | 2024-09-25T21:44:37+00:00 |
Next Scan | 2024-10-02T21:44:37+00:00 |
Last Scan
Scanned | 2024-09-25T21:44:37+00:00 |
URL | http://2ch-matome.net/robots.txt |
Domain IPs | 160.16.216.58, 2001:e42:102:1810:160:16:216:58 |
Response IP | 160.16.216.58 |
Found | Yes |
Hash | cd56f0ca3d0feccc36dcb7e040ee8c778a4946eecbbf5c36e9bc0fc7e2918a4a |
SimHash | c804ca10eb93 |
Groups
*
Rule | Path |
---|---|
Disallow | /link/ |
Other Records
Field | Value |
---|---|
sitemap | http://2ch-matome.net/sitemap.xml |
Warnings
- 2 invalid lines.
- `clawl-delay` is not a known field.