mirror2.extension.netcraft.com
robots.txt
Robots Exclusion Standard data for mirror2.extension.netcraft.com
Resource Scan
Scan Details
| Site Domain | mirror2.extension.netcraft.com |
| Base Domain | netcraft.com |
| Scan Status | Ok |
| Last Scan | 2024-08-29T18:35:59+00:00 |
| Next Scan | 2024-09-28T18:35:59+00:00 |
Last Scan
| Scanned | 2024-08-29T18:35:59+00:00 |
| URL | https://mirror2.extension.netcraft.com/robots.txt |
| Domain IPs | 65.9.112.12, 65.9.112.122, 65.9.112.38, 65.9.112.49 |
| Response IP | 3.160.246.4 |
| Found | Yes |
| Hash | c1bb2eff31e8a471be917bf7cdb57e0ad13543375b7e4104e16c6515eb0b5013 |
| SimHash | e924034503b0 |
Groups
*
| Rule | Path |
|---|---|
| Disallow |
googlebot
| Rule | Path |
|---|---|
| Allow | /site_report$ |
| Disallow | /site_report |
| Disallow | /check_u |
| Disallow | /random_site |
| Disallow | /stats/topsites?s= |
| Disallow | /netblock |