santehnika-shop.su
robots.txt

Robots Exclusion Standard data for santehnika-shop.su

Resource Scan

Scan Details

Site Domain santehnika-shop.su
Base Domain santehnika-shop.su
Scan Status Ok
Last Scan2024-10-23T08:03:50+00:00
Next Scan 2024-11-22T08:03:50+00:00

Last Scan

Scanned2024-10-23T08:03:50+00:00
URL https://santehnika-shop.su/robots.txt
Domain IPs 188.127.243.32
Response IP 188.127.243.32
Found Yes
Hash e0f02fd2ac055b8b107ed523fc8dff47c7eaa7092740cbada60e71d8e2f80111
SimHash 431ec06bf6db

Groups

*

Rule Path
Allow /bitrix/js/
Allow /api/filter_links/*
Allow /bitrix/components/
Allow /bitrix/cache/
Allow /bitrix/js/
Allow /bitrix/templates/
Allow /bitrix/panel/
Disallow /*?confirm_city
Disallow /basket/
Disallow /bitrix/
Disallow /search/
Disallow /compare/
Disallow /personal/
Disallow /wishlist/
Disallow /*arFilter_859_MAX*
Disallow /*arFilter_859_MIN*
Disallow /*%26del_filter*
Disallow /*register%3Dyes
Disallow /*forgot_password%3Dyes
Disallow /*change_password%3Dyes
Disallow /*login%3Dyes
Disallow /*logout%3Dyes
Disallow /*auth%3Dyes
Disallow /*action%3DADD_TO_COMPARE_LIST
Disallow /*action%3DDELETE_FROM_COMPARE_LIST
Disallow /*action%3DADD2BASKET*
Disallow /*action%3DBUY
Disallow /*backurl%3D*
Disallow /*BACKURL%3D*
Disallow /*back_url%3D*
Disallow /*BACK_URL%3D*
Disallow /*back_url_admin%3D*
Disallow /api/*
Disallow /*?BRAND=*
Disallow /*advert*
Disallow /*?arFilter*
Disallow /*/?set_filter=*
Disallow /*utm_source
Disallow /*_openstat
Disallow /*yclid
Disallow /*roistat
Disallow /*k50id
Disallow /*gclid

yandex

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 2

yandexbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 2

google

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 3

googlebot-image

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 3

screaming frog seo spider

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

slurp

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

seekport crawler

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

geedoproductsearch

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 100

amazonbot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.santehnika-shop.su/sitemap_index.xml