agrweb.it
robots.txt
Robots Exclusion Standard data for agrweb.it
Resource Scan
Scan Details
| Site Domain | agrweb.it |
| Base Domain | agrweb.it |
| Scan Status | Failed |
| Failure Reason | Scan timed out. |
| Last Scan | 2024-10-31T12:59:22+00:00 |
| Next Scan | 2024-11-14T12:59:22+00:00 |
Last Successful Scan
| Scanned | 2024-10-16T12:58:26+00:00 |
| URL | https://agrweb.it/robots.txt |
| Redirect | https://www.agrweb.it/robots.txt |
| Redirect Domain | www.agrweb.it |
| Redirect Base | agrweb.it |
| Domain IPs | 95.110.129.63 |
| Redirect IPs | 95.110.129.63 |
| Response IP | 95.110.129.63 |
| Found | Yes |
| Hash | 514511eeba44ba4c7f14f0d6223921653c41c1c6b01e0a6456df77f163fed999 |
| SimHash | 693544fc69d3 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /cgi-bin/ |
| Disallow | /admin/ |
| Disallow | /include/ |
| Disallow | /demo/ |
| Disallow | /tmp/ |
| Disallow | /privacy-policy |
| Disallow | /privacy-policy-pop |
| Disallow | /cookies |
| Disallow | /feedrss |
| Disallow | /profilo |
| Disallow | /articoli-pop |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.agronline.it/sitemap_news.xml |
| sitemap | https://www.agronline.it/sitemap_static.xml |
| sitemap | https://www.agronline.it/sitemap_gnews.xml |