avtosale.ua
robots.txt
Robots Exclusion Standard data for avtosale.ua
Resource Scan
Scan Details
Site Domain | avtosale.ua |
Base Domain | avtosale.ua |
Scan Status | Ok |
Last Scan | 2024-09-26T08:38:07+00:00 |
Next Scan | 2024-10-03T08:38:07+00:00 |
Last Scan
Scanned | 2024-09-26T08:38:07+00:00 |
URL | http://avtosale.ua/robots.txt |
Domain IPs | 212.42.83.249 |
Response IP | 212.42.83.249 |
Found | Yes |
Hash | 9ac6acbc2df0cc78cf287c58b8ba3d3c09184683ac46ba706f4627ba324a57b3 |
SimHash | 6d32c702d714 |
Groups
yandex
Rule | Path |
---|---|
Disallow | /car/?page* |
Disallow | /car/$ |
Disallow | *%26Sort%3D* |
Disallow | /goto.php* |
Disallow | /goto_salon.php* |
Disallow | *car/?* |
Disallow | *avtosalon/?* |
Disallow | *zapchasti/?* |
Disallow | *?* |
Disallow | *Perform* |
Disallow | /new_tender* |
Disallow | /response/ |
Disallow | /my_car/ |
Disallow | /jump.php* |
Other Records
Field | Value |
---|---|
crawl-delay | 10 |
*
Rule | Path |
---|---|
Disallow | /car/?page* |
Disallow | /car/$ |
Disallow | *%26Sort%3D* |
Disallow | /goto.php* |
Disallow | /goto_salon.php* |
Disallow | *car/?* |
Disallow | *avtosalon/?* |
Disallow | *zapchasti/?* |
Disallow | *?* |
Disallow | *Perform* |
Disallow | /new_tender* |
Disallow | /response/ |
Disallow | /my_car/ |
Disallow | /jump.php* |
Other Records
Field | Value |
---|---|
crawl-delay | 10 |
Other Records
Field | Value |
---|---|
sitemap | http://avtosale.ua/sitemap.xml.gz |
sitemap | http://avtosale.ua/sitemap.xml.gz |
Warnings
- 2 invalid lines.
- `host` is not a known field.