20minutos.es
robots.txt
Robots Exclusion Standard data for 20minutos.es
Resource Scan
Scan Details
| Site Domain | 20minutos.es |
| Base Domain | 20minutos.es |
| Scan Status | Ok |
| Last Scan | 2024-11-10T21:40:07+00:00 |
| Next Scan | 2024-11-17T21:40:07+00:00 |
Last Scan
| Scanned | 2024-11-10T21:40:07+00:00 |
| URL | https://20minutos.es/robots.txt |
| Redirect | https://www.20minutos.es/robots.txt |
| Redirect Domain | www.20minutos.es |
| Redirect Base | 20minutos.es |
| Domain IPs | 65.9.112.39, 65.9.112.43, 65.9.112.72, 65.9.112.77 |
| Redirect IPs | 65.9.112.39, 65.9.112.43, 65.9.112.72, 65.9.112.77 |
| Response IP | 18.165.140.104 |
| Found | Yes |
| Hash | 40d9084f68a17ae75a7d9c1ecf0b28b6d49b8b92986f789c19000b7e8e82f4fe |
| SimHash | ba9e510b8066 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /view/ |
| Disallow | /view/* |
| Disallow | /buscar |
| Disallow | /busqueda/ |
| Disallow | /busqueda/* |
| Disallow | /imprimir/ |
| Disallow | /mini20 |
| Disallow | /mini20/ |
| Disallow | /home |
| Disallow | /home/ |
| Disallow | /img_validator/ |
| Disallow | /aviso_comentario/ |
| Disallow | /enviar_amigo/ |
| Disallow | /usuarios/ |
| Disallow | /proc/ |
| Disallow | /iphoneapp |
| Disallow | /iphoneapp/ |
| Disallow | /ajax |
| Disallow | /compartir |
| Disallow | /sso/ |
| Disallow | /especial/especial-de-prueba/ |
| Disallow | /especial/pruebas-comercial/ |
| Disallow | /widgets/ |
| Disallow | /boletin/baja/ |
| Disallow | /archivo/ |
| Disallow | /archivo/* |
| Disallow | /*.woff2$ |
| Disallow | /*.ttf$ |
Warnings
- 2 invalid lines.
Comments