midenginecorvetteforum.com
robots.txt
Robots Exclusion Standard data for midenginecorvetteforum.com
Resource Scan
Scan Details
| Site Domain | midenginecorvetteforum.com |
| Base Domain | midenginecorvetteforum.com |
| Scan Status | Failed |
| Failure Stage | Fetching resource. |
| Failure Reason | Server returned a client error. |
| Last Scan | 2024-07-29T08:26:43+00:00 |
| Next Scan | 2024-10-27T08:26:43+00:00 |
Last Successful Scan
| Scanned | 2024-04-01T05:08:30+00:00 |
| URL | https://midenginecorvetteforum.com/robots.txt |
| Redirect | https://www.midenginecorvetteforum.com/robots.txt |
| Redirect Domain | www.midenginecorvetteforum.com |
| Redirect Base | midenginecorvetteforum.com |
| Domain IPs | 52.1.230.221 |
| Redirect IPs | 104.18.207.254, 104.18.208.24, 104.18.223.254, 104.18.224.20, 104.18.224.24, 2606:4700::6812:cffe, 2606:4700::6812:d018, 2606:4700::6812:dffe, 2606:4700::6812:e014, 2606:4700::6812:e018 |
| Response IP | 104.18.224.20 |
| Found | Yes |
| Hash | e9acee9d5c09614d9ec5ea203f418a9c198f1fab684aaf10d7f27103637ffd78 |
| SimHash | 302cdc70a013 |
Groups
*
No rules defined. All paths allowed.
Other Records
| Field | Value |
|---|---|
| crawl-delay | 1 |
*
| Rule | Path |
|---|---|
| Disallow | /admincp/ |
| Disallow | /modcp/ |
| Disallow | /attachments/ |
| Disallow | /images/ |
| Disallow | /members/ |
| Disallow | /search/ |
| Disallow | /new-content/ |
| Disallow | /auth/ |
| Disallow | /register |
| Disallow | /uploader/url |
Other Records
| Field | Value |
|---|---|
| sitemap | /core/xmlsitemap.php |