valdik.ru
robots.txt
Robots Exclusion Standard data for valdik.ru
Resource Scan
Scan Details
| Site Domain | valdik.ru |
| Base Domain | valdik.ru |
| Scan Status | Ok |
| Last Scan | 2024-06-15T16:52:04+00:00 |
| Next Scan | 2024-06-22T16:52:04+00:00 |
Last Scan
| Scanned | 2024-06-15T16:52:04+00:00 |
| URL | https://valdik.ru/robots.txt |
| Domain IPs | 141.8.192.13, 2a0a:2b42:1:3d90:: |
| Response IP | 141.8.192.13 |
| Found | Yes |
| Hash | f1333eb2de92cda6650e0723d8a688e19878334d95ef03db969c608a67ff609c |
| SimHash | 6ee54284897f |
Groups
yandex
| Rule | Path |
|---|---|
| Disallow | */top/ |
| Disallow | *admin/ |
| Disallow | *adm/ |
| Disallow | *start/ |
| Disallow | *good/ |
| Disallow | *start/ |
| Disallow | *good/ |
| Disallow | *money/ |
| Disallow | *helpdesk/ |
| Disallow | *yandexmarket/ |
| Disallow | *cml1c/ |
| Disallow | */vk/ |
| Disallow | */xml/ |
| Disallow | */yml/ |
| Disallow | */turbo/ |
| Disallow | */logs/ |
| Disallow | *multiform/ |
| Disallow | *webasyst/ |
| Disallow | */test/ |
| Disallow | */old/ |
| Disallow | *.html |
| Disallow | *.php |
| Disallow | |
| Disallow | */amp/ |
| Disallow | *search |
| Disallow | *mailer/ |
| Disallow | *login/ |
| Disallow | *signup/ |
| Disallow | *forgotpassword/ |
| Disallow | *order/ |
| Disallow | */cart/ |
| Disallow | *onestep/ |
| Disallow | *checkout/ |
| Disallow | */my/ |
| Disallow | *tel%3A |
| Disallow | *skype%3A |
| Disallow | *mailto%3A |
| Disallow | *tg%3A |
| Disallow | *viber%3A |
| Disallow | *undefined |
| Disallow | *compare/ |
| Disallow | *? |
| Allow | *.*? |
| Allow | *config/? |
| Allow | *?page= |
| Disallow | *?page=*& |
| Disallow | *product*/?page= |
| Disallow | */author/ |
| Disallow | */2015/ |
| Disallow | */2016/ |
| Disallow | */2017/ |
| Disallow | */2018/ |
| Disallow | */2019/ |
| Disallow | */2020/ |
| Disallow | */2021/ |
| Disallow | */2022/ |
| Disallow | */2023/ |
| Disallow | */2024/ |
| Disallow | */2025/ |
| Disallow | */2026/ |
| Disallow | */2027/ |
| Disallow | */2028/ |
| Disallow | */2029/ |
| Disallow | */2030/ |
| Disallow | */2031/ |
| Disallow | */2032/ |
| Disallow | */2033/ |
| Disallow | */2034/ |
| Disallow | */2035/ |
| Allow | /product/20*/ |
| Disallow | /product/20*/? |
| Disallow | */?cart=quick_full |
googlebot
| Rule | Path |
|---|---|
| Disallow | */top/ |
| Disallow | *admin/ |
| Disallow | *adm/ |
| Disallow | *start/ |
| Disallow | *good/ |
| Disallow | *money/ |
| Disallow | *helpdesk/ |
| Disallow | *yandexmarket/ |
| Disallow | *cml1c/ |
| Disallow | */vk/ |
| Disallow | */xml/ |
| Disallow | */yml/ |
| Disallow | */turbo/ |
| Disallow | */logs/ |
| Disallow | *multiform/ |
| Disallow | *webasyst/ |
| Disallow | */test/ |
| Disallow | */old/ |
| Disallow | *.html |
| Disallow | *.php |
| Disallow | |
| Disallow | *search |
| Disallow | *mailer/ |
| Disallow | *login/ |
| Disallow | *signup/ |
| Disallow | *forgotpassword/ |
| Disallow | *order/ |
| Disallow | */cart/ |
| Disallow | *onestep/ |
| Disallow | *checkout/ |
| Disallow | */my/ |
| Disallow | *tel%3A |
| Disallow | *skype%3A |
| Disallow | *mailto%3A |
| Disallow | *tg%3A |
| Disallow | *viber%3A |
| Disallow | *undefined |
| Disallow | *compare/ |
| Disallow | *? |
| Allow | *.*? |
| Allow | *config/? |
| Allow | *?page= |
| Disallow | *?page=*& |
| Disallow | *product*/?page= |
| Allow | *utm_source%3D |
| Disallow | */author/ |
| Disallow | */2015/ |
| Disallow | */2016/ |
| Disallow | */2017/ |
| Disallow | */2018/ |
| Disallow | */2019/ |
| Disallow | */2020/ |
| Disallow | */2021/ |
| Disallow | */2022/ |
| Disallow | */2023/ |
| Disallow | */2024/ |
| Disallow | */2025/ |
| Disallow | */2026/ |
| Disallow | */2027/ |
| Disallow | */2028/ |
| Disallow | */2029/ |
| Disallow | */2030/ |
| Disallow | */2031/ |
| Disallow | */2032/ |
| Disallow | */2033/ |
| Disallow | */2034/ |
| Disallow | */2035/ |
| Allow | /product/20*/ |
| Disallow | /product/20*/? |
| Disallow | */?cart=quick_full |
*
| Rule | Path |
|---|---|
| Disallow | */top/ |
| Disallow | *admin/ |
| Disallow | *adm/ |
| Disallow | *start/ |
| Disallow | *good/ |
| Disallow | *money/ |
| Disallow | *helpdesk/ |
| Disallow | *yandexmarket/ |
| Disallow | *cml1c/ |
| Disallow | */vk/ |
| Disallow | */xml/ |
| Disallow | */yml/ |
| Disallow | */turbo/ |
| Disallow | */logs/ |
| Disallow | *multiform/ |
| Disallow | *webasyst/ |
| Disallow | */test/ |
| Disallow | */old/ |
| Disallow | *.html |
| Disallow | *.php |
| Disallow | |
| Disallow | */amp/ |
| Disallow | *search |
| Disallow | *mailer/ |
| Disallow | *login/ |
| Disallow | *signup/ |
| Disallow | *forgotpassword/ |
| Disallow | *order/ |
| Disallow | */cart/ |
| Disallow | *onestep/ |
| Disallow | *checkout/ |
| Disallow | */my/ |
| Disallow | *tel%3A |
| Disallow | *skype%3A |
| Disallow | *mailto%3A |
| Disallow | *tg%3A |
| Disallow | *viber%3A |
| Disallow | *undefined |
| Disallow | *compare/ |
| Disallow | *? |
| Allow | *.*? |
| Allow | *config/? |
| Allow | *?page= |
| Disallow | *?page=*& |
| Disallow | *product*/?page= |
| Disallow | */author/ |
| Disallow | */2015/ |
| Disallow | */2016/ |
| Disallow | */2017/ |
| Disallow | */2018/ |
| Disallow | */2019/ |
| Disallow | */2020/ |
| Disallow | */2021/ |
| Disallow | */2022/ |
| Disallow | */2023/ |
| Disallow | */2024/ |
| Disallow | */2025/ |
| Disallow | */2026/ |
| Disallow | */2027/ |
| Disallow | */2028/ |
| Disallow | */2029/ |
| Disallow | */2030/ |
| Disallow | */2031/ |
| Disallow | */2032/ |
| Disallow | */2033/ |
| Disallow | */2034/ |
| Disallow | */2035/ |
| Allow | /product/20*/ |
| Disallow | /product/20*/? |
| Disallow | */?cart=quick_full |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 5 |
Other Records
| Field | Value |
|---|---|
| sitemap | https://valdik.ru/sitemap.xml |
| sitemap | https://valdik.ru/sitemap.xml |
| sitemap | https://valdik.ru/sitemap.xml |
Warnings
- `clean-param` is not a known field.
Comments