estadao.com.br
robots.txt
Robots Exclusion Standard data for estadao.com.br
Resource Scan
Scan Details
| Site Domain | estadao.com.br |
| Base Domain | estadao.com.br |
| Scan Status | Ok |
| Last Scan | 2024-11-14T20:14:43+00:00 |
| Next Scan | 2024-11-21T20:14:43+00:00 |
Last Scan
| Scanned | 2024-11-14T20:14:43+00:00 |
| URL | https://estadao.com.br/robots.txt |
| Redirect | https://www.estadao.com.br/robots.txt |
| Redirect Domain | www.estadao.com.br |
| Redirect Base | estadao.com.br |
| Domain IPs | 96.17.96.26, 96.17.96.31 |
| Redirect IPs | 23.49.60.42, 23.49.60.64, 2600:1413:b000:13::b857:c194, 2600:1413:b000:13::b857:c19e |
| Response IP | 23.45.207.171 |
| Found | Yes |
| Hash | bc938288faef2bc16c0a86b0472c512164e9d0000cc8e0ffc495d0d68e2b0230 |
| SimHash | 2e14936051b2 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /ewok |
| Disallow | /logs |
| Disallow | /robos-linux |
| Disallow | /shared |
| Disallow | /estadao |
| Disallow | /newsletter/ |
| Disallow | /correcoes/enviar?url=* |
| Disallow | /ext/ |
| Disallow | /eleicoes/2018/busca* |
| Disallow | /eleicoes/2020/busca* |
| Disallow | /feed/$ |
| Disallow | /politica/eleicoes/2024/candidatos-*/*/vice-prefeito/$ |
| Disallow | /politica/eleicoes/2024/candidatos-*/prefeito/$ |
| Disallow | /politica/eleicoes/2024/candidatos-*/vereador/$ |
| Disallow | /politica/eleicoes/2024/candidatos-*/vice-prefeito/$ |
| Disallow | /politica/eleicoes/2024/candidatos/* |
| Allow | /politica/eleicoes/2024/candidatos-*/*/prefeito/$ |
| Allow | /politica/eleicoes/2024/candidatos-*/*/vereador/$ |
| Allow | /ext/.pdf |
| Allow | /estadao-verifica |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.estadao.com.br/arc/outboundfeeds/sitemap-index-by-day/?outputType=xml |
| sitemap | https://www.estadao.com.br/sitemaps/custom/www.estadao |
| sitemap | https://www.estadao.com.br/sitemap/galerias/www.estadao/auto.xml |
Comments