exkavator.ru
robots.txt
Robots Exclusion Standard data for exkavator.ru
Resource Scan
Scan Details
Site Domain | exkavator.ru |
Base Domain | exkavator.ru |
Scan Status | Failed |
Failure Stage | Fetching resource. |
Failure Reason | Server returned a client error. |
Last Scan | 2024-09-23T22:17:11+00:00 |
Next Scan | 2024-12-22T22:17:11+00:00 |
Last Successful Scan
Scanned | 2023-05-07T18:15:52+00:00 |
URL | https://exkavator.ru/robots.txt |
Domain IPs | 104.26.6.57, 104.26.7.57, 172.67.74.79, 2606:4700:20::681a:639, 2606:4700:20::681a:739, 2606:4700:20::ac43:4a4f |
Response IP | 172.67.74.79 |
Found | Yes |
Hash | 6e0f4b2b94665b0f5630d029b1bc87333835077c062fb444da4ad8ae545d9dd8 |
SimHash | 639ed373c935 |
Groups
*
Rule | Path |
---|---|
Allow | /*.js |
Allow | /*.css |
Allow | /_modules/_cnews/rss/ |
Disallow | /*.php |
Disallow | /other/board/archive |
Disallow | /other/board/images/ |
Disallow | */search* |
Disallow | *startsearch%3D1* |
Disallow | /trade/communication* |
Disallow | /trade/companymanager |
Disallow | /trade/payments |
Disallow | /trade/auth/ |
Disallow | /trade/specials/bestprice |
Disallow | /trade/favourites |
Disallow | /trade/price |
Disallow | /trade/print* |
Disallow | /ajax |
Disallow | /ganalytics/ |
Disallow | /cdn-cgi/ |
Disallow | /we/widget |
Disallow | /other/site/ |
Disallow | /exday |
Disallow | /doosan/ |
Disallow | /hyundai/ |
Disallow | /main/news/inf_news/124281_kak_kupit_ekskavator-pogruzchik_s_probegom_i_ne_popast_v_lovushku.html |
Disallow | /main/news/inf_news/124480_mi_ne_isklyuchaem_rosta_na_rinke_podergannoy_tehniki_rusbiznesavto_o_prodagah_bu_mashin.html |
Other Records
Field | Value |
---|---|
sitemap | https://exkavator.ru/sitemap.xml |
Warnings
- `host` is not a known field.