yamart.ru
robots.txt

Robots Exclusion Standard data for yamart.ru

Resource Scan

Scan Details

Site Domain yamart.ru
Base Domain yamart.ru
Scan Status Failed
Failure StageFetching resource.
Failure ReasonCouldn't connect to server.
Last Scan2024-09-04T16:34:54+00:00
Next Scan 2024-12-03T16:34:54+00:00

Last Successful Scan

Scanned2023-11-10T16:13:22+00:00
URL https://yamart.ru/robots.txt
Domain IPs 80.93.188.52
Response IP 80.93.188.52
Found Yes
Hash 704bf0a10ef0fc85dbf6ca52775fd946d13ee6e208efd4e61a4777a4eb7a8eeb
SimHash 635ff8b25780

Groups

*

Rule Path
Disallow /admin/
Disallow /auth/
Disallow /tools/
Disallow /new-product/
Disallow /favorites/
Disallow /forum/
Disallow /moderate/
Disallow /pay/
Disallow /viewed/
Disallow /search/
Disallow /registration/
Disallow /profile/
Disallow /personal/
Disallow /login/
Disallow /russia/
Disallow /catalog/
Disallow /cart/
Disallow /offers/
Disallow /filters/
Disallow /product/
Disallow /model/
Disallow /history/
Disallow *-yo-*
Disallow */page-*
Disallow */?*
Disallow *-ym-*

mail.ru

Rule Path
Disallow /

bingbot

Rule Path
Disallow /

slurp

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

adsbot-google

Rule Path
Disallow /

adsbot-google-mobile

Rule Path
Disallow /

yandexdirect

Rule Path
Disallow /

yandexdirectdyn

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

grapeshot

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

velenpublicwebcrawler

Rule Path
Disallow /

mail.ru

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 30

Warnings

  • `host` is not a known field.