frostyrider.com
robots.txt
Robots Exclusion Standard data for frostyrider.com
Resource Scan
Scan Details
| Site Domain | frostyrider.com |
| Base Domain | frostyrider.com |
| Scan Status | Failed |
| Failure Stage | Fetching resource. |
| Failure Reason | Server returned a client error. |
| Last Scan | 2025-11-20T19:25:46+00:00 |
| Next Scan | 2025-11-27T19:25:46+00:00 |
Last Successful Scan
| Scanned | 2025-09-29T19:13:02+00:00 |
| URL | https://frostyrider.com/robots.txt |
| Domain IPs | 45.33.107.164 |
| Response IP | 45.33.107.164 |
| Found | Yes |
| Hash | 7662aa2194cd07e2ce515f4907ca6de5d6112973296920352305f2265c9ef5c2 |
| SimHash | 3908ddc29ee2 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /*blackhole |
| Disallow | /?blackhole |
semrushbot
siteauditbot
semrushbot-ba
semrushbot-si
semrushbot-swa
semrushbot-ct
splitsignalbot
semrushbot-coub
| Rule | Path |
|---|---|
| Disallow | / |
*
| Rule | Path |
|---|---|
| Allow | /wp-content/uploads/ |
| Disallow | /test/ |
| Disallow | /_private/ |
| Disallow | /_overlay/ |
| Disallow | /_themes/ |
| Disallow | /_fpclass/ |
| Disallow | /cgi-bin/ |
| Disallow | /wp-admin/ |
| Disallow | /wp-login.php |
| Disallow | /readme.html |
| Disallow | /refer/ |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 15 |
*
| Rule | Path |
|---|---|
| Disallow | /*blackhole |
| Disallow | /?blackhole |
*
| Rule | Path |
|---|---|
| Disallow | /wp-content/uploads/wpo/wpo-plugins-tables-list.json |
Other Records
| Field | Value |
|---|---|
| sitemap | https://frostyrider.com/sitemap_index.xml |
Comments