racingjunk.com
robots.txt
Robots Exclusion Standard data for racingjunk.com
Resource Scan
Scan Details
| Site Domain | racingjunk.com |
| Base Domain | racingjunk.com |
| Scan Status | Failed |
| Failure Stage | Fetching resource. |
| Failure Reason | Server returned a client error. |
| Last Scan | 2024-08-27T11:09:47+00:00 |
| Next Scan | 2024-09-26T11:09:47+00:00 |
Last Successful Scan
| Scanned | 2024-07-29T11:08:21+00:00 |
| URL | https://racingjunk.com/robots.txt |
| Redirect | https://www.racingjunk.com/robots.txt |
| Redirect Domain | www.racingjunk.com |
| Redirect Base | racingjunk.com |
| Domain IPs | 104.18.34.109, 172.64.153.147, 2606:4700:4400::6812:226d, 2606:4700:4400::ac40:9993 |
| Redirect IPs | 104.18.34.109, 172.64.153.147, 2606:4700:4400::6812:226d, 2606:4700:4400::ac40:9993 |
| Response IP | 104.18.34.109 |
| Found | Yes |
| Hash | 014cad3e8cc16de38ac919a6518b1771a39cb11e2c84c2edd2e16b9bc015adca |
| SimHash | 6854fad02a93 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /2922/RacingJunk/ |
| Disallow | /forums/search.php |
| Disallow | /forums/members/ |
| Disallow | /forums/memberlist.php |
| Disallow | /forums/tags.php |
| Disallow | /search |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 3 |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.racingjunk.com/sitemaps/sitemap-index.xml |