toshiba.co.jp
robots.txt
Robots Exclusion Standard data for toshiba.co.jp
Resource Scan
Scan Details
| Site Domain | toshiba.co.jp |
| Base Domain | toshiba.co.jp |
| Scan Status | Ok |
| Last Scan | 2024-09-01T05:53:28+00:00 |
| Next Scan | 2024-10-01T05:53:28+00:00 |
Last Scan
| Scanned | 2024-09-01T05:53:28+00:00 |
| URL | https://www.toshiba.co.jp/robots.txt |
| Domain IPs | 132.147.114.72, 163.171.210.189, 163.171.211.109, 163.171.211.114, 203.117.159.85 |
| Response IP | 163.171.210.189 |
| Found | Yes |
| Hash | 3fe36515bae5c823d91ea6fa51db9d959c2a4dffa1c5a883ed736d6589555fa8 |
| SimHash | fd204fc00d12 |
Groups
*
| Product | Comment |
|---|---|
| * | All other spiders should avoid |
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | Script files |
| Disallow | /img/ | img files |
| Disallow | /imgdat/ | img files |
| Disallow | /logs/ | img files |
| Disallow | /gbde-prv/ | - |
| Disallow | /image/ | - |
| Disallow | /tanpopo/bbs/cgi-bin/ | - |
| Disallow | /tanpopo/look/ | - |
| Disallow | /recaius/ | Add 20191016 |
Comments