autopiter.ru
robots.txt
Robots Exclusion Standard data for autopiter.ru
Resource Scan
Scan Details
| Site Domain | autopiter.ru |
| Base Domain | autopiter.ru |
| Scan Status | Failed |
| Failure Reason | Scan timed out. |
| Last Scan | 2024-10-20T15:51:04+00:00 |
| Next Scan | 2024-11-19T15:51:04+00:00 |
Last Successful Scan
| Scanned | 2024-09-14T13:52:11+00:00 |
| URL | https://autopiter.ru/robots.txt |
| Domain IPs | 37.110.240.102, 84.204.168.2 |
| Response IP | 84.204.168.2 |
| Found | Yes |
| Hash | 9a6b3ea888837f70d14d2739941b4b8ca847b3f03c4b0cfcbb305d89305f5813 |
| SimHash | 054ed4724f23 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /original/* |
| Disallow | /OffLineCat/ |
| Disallow | /offlinecat/ |
| Disallow | */provider?* |
| Disallow | /account/ |
| Disallow | /Account/ |
| Disallow | /404 |
| Disallow | /recovery |
| Disallow | /registration |
| Disallow | /Registration |
| Disallow | /*/*?search* |
| Disallow | /*/*fromNewSite* |
| Disallow | /cart |
| Disallow | /catalog |
| Disallow | /Catalog |
| Disallow | /50x.html |
| Disallow | /gadget/* |
| Disallow | /Gadget/* |
| Disallow | /order/* |
| Disallow | /order/success/* |
| Disallow | /providerNew/* |
| Disallow | /admin/* |
| Disallow | /Order/* |
| Disallow | /Order/success/* |
| Disallow | /ProviderNew/* |
| Disallow | /Admin/* |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 5 |
yandex
| Rule | Path |
|---|---|
| Disallow | /original/* |
| Disallow | /OffLineCat/ |
| Disallow | /offlinecat/ |
| Disallow | */provider?* |
| Disallow | /account/ |
| Disallow | /Account/ |
| Disallow | /404 |
| Disallow | /recovery |
| Disallow | /registration |
| Disallow | /Registration |
| Disallow | /*/*?search* |
| Disallow | /*/*fromNewSite* |
| Disallow | /cart |
| Disallow | /catalog |
| Disallow | /Catalog |
| Disallow | /50x.html |
| Disallow | /gadget/* |
| Disallow | /Gadget/* |
| Disallow | /order/* |
| Disallow | /order/success/* |
| Disallow | /providerNew/* |
| Disallow | /admin/* |
| Disallow | /Order/* |
| Disallow | /Order/success/* |
| Disallow | /ProviderNew/* |
| Disallow | /Admin/* |
Other Records
| Field | Value |
|---|---|
| sitemap | https://autopiter.ru/api/api/sitemap/root.xml |
| sitemap | https://autopiter.ru/api/api/sitemap/root.xml |
Warnings
- `clean-param` is not a known field.
Comments