rustan.ru
robots.txt

Robots Exclusion Standard data for rustan.ru

Resource Scan

Scan Details

Site Domain rustan.ru
Base Domain rustan.ru
Scan Status Ok
Last Scan2025-12-22T16:42:55+00:00
Next Scan 2026-01-21T16:42:55+00:00

Last Scan

Scanned2025-12-22T16:42:55+00:00
URL https://rustan.ru/robots.txt
Domain IPs 89.108.72.109
Response IP 89.108.72.109
Found Yes
Hash 78d31f7125b7d4e263e0eceeec42e32104fadb15de582bbdd8c48f01d8bd08a8
SimHash 6c2d18ca8f25

Groups

*

Rule Path
Allow /assets/
Allow /sitemap/
Allow /fonts/
Allow *.css
Allow *.js
Allow /*?page=
Disallow /*?*
Disallow *.doc
Disallow *.xls
Disallow /filter/
Disallow /search/
Disallow /site/captcha/
Disallow /node/emb/
Disallow /b/
Disallow /cart
Disallow /node/compare
Disallow /taxonomy/
Disallow /discontinued/
Disallow /stanki-proizvodstva-sssr/
Disallow /sites/default/files/instructions/
Disallow /favorite
Disallow /withdrawn-from-sale/
Disallow /brand/description
Disallow /delivery/emb
Disallow /site/call-request
Disallow /node/comment
Disallow /price/clarify
Disallow /price.htm
Disallow /site/get-cities-list
Disallow /site/login
Disallow /contacts-*

yandex

Rule Path
Allow /assets/
Allow /sitemap/
Allow /fonts/
Allow *.css
Allow *.js
Allow /*?page=
Disallow /*?*
Disallow *.doc
Disallow *.xls
Disallow /filter/
Disallow /search/
Disallow /site/captcha/
Disallow /node/emb/
Disallow /b/
Disallow /cart
Disallow /node/compare
Disallow /taxonomy/
Disallow /discontinued/
Disallow /stanki-proizvodstva-sssr/
Disallow /sites/default/files/instructions/
Disallow /favorite
Disallow /withdrawn-from-sale/
Disallow /brand/description
Disallow /delivery/emb
Disallow /site/call-request
Disallow /node/comment
Disallow /price/clarify
Disallow /price.htm
Disallow /site/get-cities-list
Disallow /site/login
Disallow /contacts-*

googlebot

Rule Path
Allow /assets/
Allow /sitemap/
Allow /fonts/
Allow *.css
Allow *.js
Allow /*?page=
Disallow /*?*
Disallow *.doc
Disallow *.xls
Disallow /filter/
Disallow /search/
Disallow /site/captcha/
Disallow /node/emb/
Disallow /b/
Disallow /cart
Disallow /node/compare
Disallow /taxonomy/
Disallow /discontinued/
Disallow /stanki-proizvodstva-sssr/
Disallow /sites/default/files/instructions/
Disallow /favorite
Disallow /withdrawn-from-sale/
Disallow /brand/description
Disallow /delivery/emb
Disallow /site/call-request
Disallow /node/comment
Disallow /price/clarify
Disallow /price.htm
Disallow /site/get-cities-list
Disallow /site/login
Disallow /contacts-*

ahrefsbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

grapeshot

Rule Path
Disallow /

megaindex

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

mj12bot

Rule Path
Disallow /

riddler (http://riddler.io/about)

Rule Path
Disallow /

python-urllib/2.6

Rule Path
Disallow /

disco pump

Rule Path
Disallow /

wget

Rule Path
Disallow /

teleport pro

Rule Path
Disallow /

psbot

Rule Path
Disallow /

twiceler

Rule Path
Disallow /

netinfobot

Rule Path
Disallow /

ubicrawler

Rule Path
Disallow /

doc

Rule Path
Disallow /

zao

Rule Path
Disallow /

sitecheck.internetseer.com

Rule Path
Disallow /

zealbot

Rule Path
Disallow /

msiecrawler

Rule Path
Disallow /

sitesnagger

Rule Path
Disallow /

webstripper

Rule Path
Disallow /

webcopier

Rule Path
Disallow /

fetch

Rule Path
Disallow /

offline explorer

Rule Path
Disallow /

teleport

Rule Path
Disallow /

linko

Rule Path
Disallow /

httrack

Rule Path
Disallow /

microsoft.url.control

Rule Path
Disallow /

xenu

Rule Path
Disallow /

larbin

Rule Path
Disallow /

libwww

Rule Path
Disallow /

zyborg

Rule Path
Disallow /

download ninja

Rule Path
Disallow /

grub-client

Rule Path
Disallow /

npbot

Rule Path
Disallow /

webreaper

Rule Path
Disallow /

Other Records

Field Value
sitemap https://rustan.ru/sitemap.xml