regmarkets.ru
robots.txt

Robots Exclusion Standard data for regmarkets.ru

Resource Scan

Scan Details

Site Domain regmarkets.ru
Base Domain regmarkets.ru
Scan Status Ok
Last Scan2024-09-18T10:52:20+00:00
Next Scan 2024-10-18T10:52:20+00:00

Last Scan

Scanned2024-09-18T10:52:20+00:00
URL https://regmarkets.ru/robots.txt
Redirect https://kupi.ru/robots.txt
Redirect Domain kupi.ru
Redirect Base kupi.ru
Domain IPs 80.93.188.50
Redirect IPs 80.93.188.50
Response IP 80.93.188.50
Found Yes
Hash 4428973a51a3888a3fc652916f307b631571c22d26fa15ca2bfec279838c4368
SimHash 715ffce27688

Groups

*

Rule Path
Disallow /admin/
Disallow /auth/
Disallow /tools/
Disallow /new-product/
Disallow /favorites/
Disallow /forum/
Disallow /moderate/
Disallow /pay/
Disallow /viewed/
Disallow /search/
Disallow /registration/
Disallow /profile/
Disallow /personal/
Disallow /login/
Disallow /catalog/
Disallow /favorites/
Disallow /cart/
Disallow /offers/
Disallow /filters/
Disallow /product/
Disallow /model/
Disallow /company/
Disallow /history/
Disallow /promo/
Disallow /include/
Disallow /bitrix/
Disallow /ajax/
Disallow /bobolink/
Disallow /question-*
Disallow /opinions-*
Disallow /model-opinion-*
Disallow /model-video-*
Disallow /*.kupi.ru
Disallow *-yo-*
Disallow */page-*
Disallow */privacy_policy/*
Disallow */?*
Disallow *-ym-*
Disallow *addComplain*
Disallow *addMessage*
Disallow *ooo*
Disallow /russia-*
Disallow /barnaul/*
Disallow /krasnodar/*
Disallow /nevinnomyssk/*
Disallow /stavropol/*

mail.ru

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 250

bingbot

Rule Path
Disallow /

slurp

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

seekport crawler

Rule Path
Disallow /

mediatoolkitbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

smtbot

Rule Path
Disallow /

adsbot-google

Rule Path
Disallow /

adsbot-google-mobile

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

megaindex

Rule Path
Disallow /

bluechipbacklinks

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

Warnings

  • `host` is not a known field.