usedcars.ru
robots.txt
Robots Exclusion Standard data for usedcars.ru
Resource Scan
Scan Details
| Site Domain | usedcars.ru |
| Base Domain | usedcars.ru |
| Scan Status | Ok |
| Last Scan | 2024-11-16T19:32:49+00:00 |
| Next Scan | 2024-11-23T19:32:49+00:00 |
Last Scan
| Scanned | 2024-11-16T19:32:49+00:00 |
| URL | https://usedcars.ru/robots.txt |
| Domain IPs | 5.35.99.80 |
| Response IP | 5.35.99.80 |
| Found | Yes |
| Hash | 7e9e6e481a47bb74e05106ea3ec8006ebf73b7a4168689e320128c8d6c2ea195 |
| SimHash | cd299f780f92 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /?* |
| Disallow | /search |
| Disallow | /users |
| Disallow | /cabinet |
| Disallow | /cars/manage |
| Disallow | /cars/search |
| Disallow | /cars/get_phone |
| Disallow | /articles/*/srochnyj-vykup-automobiley* |
| Disallow | /articles/*/http* |
| Disallow | /*utm_ |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 2 |
Other Records
| Field | Value |
|---|---|
| sitemap | https://usedcars.ru/sitemaps/ |
| sitemap | https://usedcars.ru/sitemaps/articles/1/ |
| sitemap | https://usedcars.ru/sitemaps/articles/2/ |
| sitemap | https://usedcars.ru/sitemaps/articles/3/ |
| sitemap | https://usedcars.ru/sitemaps/articles/4/ |
| sitemap | https://usedcars.ru/sitemaps/articles/5/ |
| sitemap | https://usedcars.ru/sitemaps/articles/6/ |
| sitemap | https://usedcars.ru/sitemaps/cars/1/ |
| sitemap | https://usedcars.ru/sitemaps/rubrics/ |
Warnings
- `host` is not a known field.