rost-holding.ru
robots.txt

Robots Exclusion Standard data for rost-holding.ru

Resource Scan

Scan Details

Site Domain rost-holding.ru
Base Domain rost-holding.ru
Scan Status Ok
Last Scan2024-09-24T21:58:34+00:00
Next Scan 2024-10-24T21:58:34+00:00

Last Scan

Scanned2024-09-24T21:58:34+00:00
URL https://rost-holding.ru/robots.txt
Domain IPs 2a03:6f00:1::5c35:606a, 92.53.123.2
Response IP 92.53.123.2
Found Yes
Hash 0ecfe54113fcbbbc4172042047738f5e33228be10ac569bc51925b6594281745
SimHash 491c8542e7d1

Groups

*

Rule Path
Disallow /admin/
Disallow /404/
Disallow /403/
Disallow /captcha.php
Disallow /download_file.php
Disallow /search/
Disallow /products/cart/
Disallow */producer-*/
Disallow */dinner*/page*/
Disallow */douter*/page*/
Disallow */napravlenienagruzki/page-*/
Disallow */seriapodshipnika/page-*/
Disallow */ryadnost/page-*/
Disallow */manufacture/materials/ftoroplasty/*
Allow /*jquery*
Allow *.css
Allow /hostcmsfiles/css/*
Allow /hostcmsfiles/js/*

mj12bot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

rogerbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

alexibot

Rule Path
Disallow /

surveybot

Rule Path
Disallow /

xenu’s

Rule Path
Disallow /

xenu’s link sleuth 1.1c

Rule Path
Disallow /

mauibot

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

blexbot/1.0

Rule Path
Disallow /

yodao

Rule Path
Disallow /

youdao

Rule Path
Disallow /

zoominfobot

Rule Path
Disallow /

linguee

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

semrushbot/3~bl

Rule Path
Disallow /

velenpublicwebcrawler

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

seekportbot

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

tinytestbot

Rule Path
Disallow /

imagesiftbot

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

go-http-client/1.1

Rule Path
Disallow /

Other Records

Field Value
sitemap https://rost-holding.ru/sitemap.xml

Warnings

  • `host` is not a known field.