20minutos.es
robots.txt
Robots Exclusion Standard data for 20minutos.es
Resource Scan
Scan Details
| Site Domain | 20minutos.es |
| Base Domain | 20minutos.es |
| Scan Status | Ok |
| Last Scan | 2024-04-28T04:18:21+00:00 |
| Next Scan | 2024-05-05T04:18:21+00:00 |
Last Scan
| Scanned | 2024-04-28T04:18:21+00:00 |
| URL | https://20minutos.es/robots.txt |
| Redirect | https://www.20minutos.es/robots.txt |
| Redirect Domain | www.20minutos.es |
| Redirect Base | 20minutos.es |
| Domain IPs | 99.84.238.116, 99.84.238.138, 99.84.238.174, 99.84.238.214 |
| Redirect IPs | 99.84.238.116, 99.84.238.138, 99.84.238.174, 99.84.238.214 |
| Response IP | 18.165.171.87 |
| Found | Yes |
| Hash | b5a38b03f6429fb6cece20375df63c8579946ff2f565e055743f7d61efc242ed |
| SimHash | bb9ed10a0862 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /view/ |
| Disallow | /view/* |
| Disallow | /buscar |
| Disallow | /busqueda/ |
| Disallow | /busqueda/* |
| Disallow | /imprimir/ |
| Disallow | /mini20 |
| Disallow | /mini20/ |
| Disallow | /home |
| Disallow | /home/ |
| Disallow | /img_validator/ |
| Disallow | /aviso_comentario/ |
| Disallow | /enviar_amigo/ |
| Disallow | /usuarios/ |
| Disallow | /proc/ |
| Disallow | /iphoneapp |
| Disallow | /iphoneapp/ |
| Disallow | /ajax |
| Disallow | /compartir |
| Disallow | /sso/ |
| Disallow | /especial/especial-de-prueba/ |
| Disallow | /especial/pruebas-comercial/ |
| Disallow | /widgets/ |
| Disallow | /boletin/baja/ |
| Disallow | /archivo/ |
| Disallow | /archivo/* |
| Disallow | /*.woff2$ |
| Disallow | /*.ttf$ |
Warnings
- 2 invalid lines.
Comments