masterts.ru
robots.txt

Robots Exclusion Standard data for masterts.ru

Resource Scan

Scan Details

Site Domain masterts.ru
Base Domain masterts.ru
Scan Status Ok
Last Scan2025-05-16T17:45:28+00:00
Next Scan 2025-06-15T17:45:28+00:00

Last Scan

Scanned2025-05-16T17:45:28+00:00
URL https://masterts.ru/robots.txt
Domain IPs 176.124.214.165
Response IP 176.124.214.165
Found Yes
Hash 9c932c74d00f0c0f455bd1c07707b4040b3f9dcc2660ae2ad2ff80dbb1184ecf
SimHash fed65c4d8c63

Groups

yandex

Rule Path
Disallow /favorites/
Disallow /comparison/
Disallow /client/
Disallow /uploads
Disallow /organizations/
Disallow /orders/
Disallow /cart/
Disallow /print/
Disallow /admin/
Disallow /search/
Disallow /yandex/
Disallow /thumbnails/
Disallow /print.php
Disallow /files/
Disallow /trimheadinfo/
Disallow */?$
Disallow /index.php
Disallow /news/20*/
Disallow /search-result/
Disallow /*/?$
Disallow /products/*/?
Disallow *checkboxes*
Disallow */app/product/
Disallow *ranges*
Disallow *debug*

Other Records

Field Value
crawl-delay 60

googlebot

Rule Path
Disallow /favorites/
Disallow /comparison/
Disallow /client/
Disallow /search-result/
Disallow /uploads
Disallow /organizations/
Disallow /orders/
Disallow /cart/
Disallow /yandex/
Disallow /print/
Disallow /admin/
Disallow /search/
Disallow /thumbnails/
Disallow /files/
Disallow /trimheadinfo/
Disallow /print.php
Disallow *?view
Disallow *?ranges
Disallow *?checkboxes
Disallow *?sort
Disallow *?category
Disallow *?code
Disallow *?expandable_category
Disallow *?*q=
Disallow *?letter
Disallow *?per-page
Disallow *?*cm_id=
Disallow *?reviews=
Disallow *?city=
Disallow *?orderId=
Disallow *?ref=
Disallow *?clear=
Disallow *?from=
Disallow *?_csrf=
Disallow *?*login=
Disallow *?*phone=
Disallow *?_pjax=
Disallow *?_ym_debug=
Disallow *?clid=
Disallow *?hhtmFrom=
Disallow *?msclkid=
Disallow *?route=
Disallow *?search=
Disallow *?spm=
Disallow *?src_pof=
Disallow *?Yandex-source=
Disallow *?etext=
Disallow *?_ym-debug=
Disallow *?lang=
Disallow *?wptouch_switch=
Disallow *?amp=
Disallow /products/*/?
Disallow *is_request_tag%3D
Disallow *debug*
Disallow *?tag=

Other Records

Field Value
crawl-delay 60

*

Rule Path
Disallow /favorites/
Disallow /comparison/
Disallow /client/
Disallow /search-result/
Disallow /uploads
Disallow /organizations/
Disallow /orders/
Disallow /cart/
Disallow /yandex/
Disallow /print/
Disallow /admin/
Disallow /search/
Disallow /thumbnails/
Disallow /print.php
Disallow /files/
Disallow /trimheadinfo/
Disallow *?view
Disallow *?ranges
Disallow *?checkboxes
Disallow *?sort
Disallow *?category
Disallow *?code
Disallow *?expandable_category
Disallow *?*q=
Disallow *?reviews
Disallow *?letter
Disallow *?per-page
Disallow *?*cm_id=
Disallow *?reviews=
Disallow *?city=
Disallow *?orderId=
Disallow *?ref=
Disallow *?clear=
Disallow *?from=
Disallow *?_csrf=
Disallow *?*login=
Disallow *?*phone=
Disallow *?_pjax=
Disallow *?_ym_debug=
Disallow *?clid=
Disallow *?hhtmFrom=
Disallow *?msclkid=
Disallow *?route=
Disallow *?search=
Disallow *?spm=
Disallow *?src_pof=
Disallow *?Yandex-source=
Disallow *?etext=
Disallow *?_ym-debug=
Disallow *?lang=
Disallow *?wptouch_switch=
Disallow *?amp=
Disallow *?tag=

Other Records

Field Value
crawl-delay 60

riddler (http://riddler.io/about)

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 60

wget

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 60

teleport pro

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 60

psbot

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 60

twiceler

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 60

netinfobot

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 60

ubicrawler

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 60

zao

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 60

zealbot

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 60

sitesnagger

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 60

webstripper

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 60

webcopier

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 60

fetch

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 60

offline explorer

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 60

teleport

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 60

linko

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 60

httrack

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 60

xenu

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 60

larbin

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 60

zyborg

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 60

download ninja

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 60

webreaper

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 60

geedobot

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 60

Other Records

Field Value
sitemap https://masterts.ru/sitemap.xml
sitemap https://masterts.ru/sitemap.xml

Warnings

  • `clean-param` is not a known field.