validator.w3.org
robots.txt
Robots Exclusion Standard data for validator.w3.org
Resource Scan
Scan Details
| Site Domain | validator.w3.org |
| Base Domain | w3.org |
| Scan Status | Ok |
| Last Scan | 2025-08-21T22:03:43+00:00 |
| Next Scan | 2025-09-20T22:03:43+00:00 |
Last Scan
| Scanned | 2025-08-21T22:03:43+00:00 |
| URL | https://validator.w3.org/robots.txt |
| Domain IPs | 104.18.22.19, 104.18.23.19, 2606:4700::6812:1613, 2606:4700::6812:1713 |
| Response IP | 104.18.23.19 |
| Found | Yes |
| Hash | 03d193c768e817b3c8df89ddcbd691c0ca0a1d108ea7d3ee088a8dee4e03575e |
| SimHash | c10307c68dd0 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /check |
| Disallow | /feed/check |
| Disallow | /checklink |
| Disallow | /mobile/check |
| Disallow | /mobile/alpha |
| Disallow | /css-validator |
| Disallow | /unicorn/check |
| Disallow | /http-checker |
| Disallow | /i18n-checker/check |
| Disallow | /nu |
Comments