correioweb.com.br
robots.txt
Robots Exclusion Standard data for correioweb.com.br
Resource Scan
Scan Details
| Site Domain | correioweb.com.br |
| Base Domain | correioweb.com.br |
| Scan Status | Ok |
| Last Scan | 2024-09-16T18:52:58+00:00 |
| Next Scan | 2024-09-23T18:52:58+00:00 |
Last Scan
| Scanned | 2024-09-16T18:52:58+00:00 |
| URL | https://correioweb.com.br/robots.txt |
| Redirect | https://www.correioweb.com.br/robots.txt |
| Redirect Domain | www.correioweb.com.br |
| Redirect Base | correioweb.com.br |
| Domain IPs | 23.22.10.20 |
| Redirect IPs | 186.195.65.65 |
| Response IP | 186.195.65.65 |
| Found | Yes |
| Hash | 785d2b1fc41e04a1f05c66f5febaabf5250134a9c00690636e626be248cec06c |
| SimHash | 68484e8231f7 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /autor/* |
| Disallow | /busca/* |
| Disallow | /tags/* |
| Disallow | /related/* |
| Disallow | /_templates/ |
| Disallow | /_temp/ |
| Disallow | /includes/ |
| Disallow | /static/ |
| Disallow | /src/ |
| Disallow | /imagens/ |
| Disallow | /cdn/ |
| Disallow | /assets/ |
| Disallow | /_files/ |
| Disallow | /*.pdf$ |
| Disallow | /*.json$ |
| Disallow | /search/* |
| Disallow | /teste-*.html$ |
| Allow | /*.jpg |
| Allow | /*.JPG |
| Allow | /*.jpeg |
| Allow | /*.JPEG |
| Allow | /*.png |
| Allow | /*.PNG |
| Allow | /*.gif |
| Allow | /*.GIF |
Comments