Log In Sign Up

stroylandiya.ru
robots.txt

Robots Exclusion Standard data for stroylandiya.ru

Resource Scan

Scan Details

Site Domain stroylandiya.ru
Base Domain stroylandiya.ru
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-03-14T11:10:51+00:00
Next Scan 2024-06-12T11:10:51+00:00

Last Successful Scan

Scanned2023-05-09T23:46:50+00:00
URL https://stroylandiya.ru/robots.txt
Domain IPs 104.18.2.18, 104.18.3.18, 2606:4700::6812:212, 2606:4700::6812:312
Response IP 104.18.2.18
Found Yes
Hash e8c5a8e7acdcd3d41bd66ae5a98f7a38cb4d864a83afda0f872406a616f27938
SimHash 32320b509c72

Groups

yandex

Rule Path
Disallow /*noindex.js
Disallow /career/news/
Disallow /news/novosti-kompanii/
Disallow /korporativnym-klientam/
Disallow /ajax/form.php?type=reg
Disallow /api/
Disallow /bitrix/
Disallow /bitrix/admin
Disallow /credit/
Disallow /form/
Disallow /include/php/
Disallow /opt/
Disallow /push/
Disallow /redirect/
Disallow /store/
Disallow /upload/
Disallow /auth/
Disallow /cart/
Disallow /catalog/compare/
Disallow /favourites/
Disallow /order/
Disallow /profile/
Disallow /subscribe-confirm/
Disallow /search/*
Disallow /*/search/
Disallow /catalog/*/filter/*
Disallow /*order%3D
Disallow /*sort%3D
Disallow /*in_stock%3D
Disallow /*limit%3D
Disallow /*bitrix_*%3D
Disallow /*brand_id__in%3D
Disallow /*?parameter_
Disallow /*parameter_*%3D
Disallow /*_price%3D
Disallow /*?amp
Disallow /*?print
Disallow /*index.php
Disallow /*detail.php
Disallow /*.swf
Allow /*.css
Allow /*.js
Allow /*.jpg
Allow /*.JPG
Allow /*.jpeg
Allow /*.JPEG
Allow /*.png
Allow /*.PNG
Allow /*.gif
Allow /*.GIF
Allow /*.svg
Allow /*.SVG
Allow /bitrix/*.css
Allow /bitrix/*.js
Allow /bitrix/*.jpg
Allow /bitrix/*.JPG
Allow /bitrix/*.jpeg
Allow /bitrix/*.JPEG
Allow /bitrix/*.png
Allow /bitrix/*.PNG
Allow /bitrix/*.gif
Allow /bitrix/*.GIF
Allow /bitrix/*.svg
Allow /bitrix/*.SVG
Allow /upload/*.jpg
Allow /upload/*.JPG
Allow /upload/*.jpeg
Allow /upload/*.JPEG
Allow /upload/*.png
Allow /upload/*.PNG
Allow /upload/*.gif
Allow /upload/*.GIF
Allow /upload/*.svg
Allow /upload/*.SVG
Allow /news/novosti-kompanii/orenburg/
Disallow /*?$
Disallow /*?back_url*
Disallow /*?clid*
Disallow /?PAGEN*
Disallow /*?from=*
Disallow /*?ID=*
Disallow /*?site*
Disallow /*?wbraid=*
Disallow /*?subid*
Disallow /*?lang=*
Disallow /*?added=*
Disallow /*?smartleads*
Disallow /*?type=*
Disallow /*?QUANTITY*
Disallow /*?utm_sourse=*
Disallow /*?public=*
Disallow /*?p=*
Disallow /*?m=*
Disallow /*?item=*
Disallow /sale/viber%3A/
Disallow /novinki/?PAGEN_1=*
Disallow /*etext%3D*
Disallow /catalog/*/*?*
Allow /catalog/*/*?PAG*
Disallow /catalog/*/*?PAG*&*
Disallow /local/templates/aspro_next/js/noindex.js
Disallow /local/templates/stroylandiya_2021/js/noindex.js

googlebot

Rule Path
Disallow /*ticket%3D*
Disallow /*noindex.js
Disallow /career/news/
Disallow /news/novosti-kompanii/
Disallow /korporativnym-klientam/
Disallow /ajax/form.php?type=reg
Disallow /api/
Disallow /bitrix/
Disallow /bitrix/admin
Disallow /credit/
Disallow /form/
Disallow /include/php/
Disallow /opt/
Disallow /push/
Disallow /redirect/
Disallow /store/
Disallow /upload/
Disallow /auth/
Disallow /cart/
Disallow /catalog/compare/
Disallow /favourites/
Disallow /order/
Disallow /profile/
Disallow /subscribe-confirm/
Disallow /search/*
Disallow /*/search/
Disallow /catalog/*/filter/*
Disallow /*order%3D
Disallow /*sort%3D
Disallow /*in_stock%3D
Disallow /*limit%3D
Disallow /*bitrix_*%3D
Disallow /*brand_id__in%3D
Disallow /*?parameter_
Disallow /*parameter_*%3D
Disallow /*_price%3D
Disallow /*?*admitad_uid
Disallow /*?*adposition
Disallow /*?*all
Disallow /*?*amp
Disallow /*?*attempt
Disallow /*?*block
Disallow /*?*bxrand
Disallow /*?*category
Disallow /*?*city_name
Disallow /*?*current_fieldset
Disallow /*?*gclid
Disallow /*?*id
Disallow /*?*k
Disallow /*?*keyword
Disallow /*?*matchtype
Disallow /*?*mob
Disallow /*?*news-category
Disallow /*?*openstat
Disallow /*?*page
Disallow /*PAGEN_2
Disallow /*?*position
Disallow /*?*print
Disallow /*%26print
Disallow /*?*q
Disallow /*?*referer1
Disallow /*?*referer2
Disallow /*?*region_id
Disallow /*?*region_name
Disallow /*%26region
Disallow /*?*RID
Disallow /*?*s
Disallow /*?*sessid
Disallow /*?*spush
Disallow /*?*utm_campaign
Disallow /*?*utm_content
Disallow /*?*utm_expid
Disallow /*?*utm_medium
Disallow /*?*utm_referrer
Disallow /*?*utm_source
Disallow /*?*utm_term
Disallow /*?*view_type
Disallow /*?*wid
Disallow /*?*yclid
Disallow /*index.php
Disallow /*detail.php
Disallow /*.swf
Disallow /*?*subid
Disallow /*?*placement
Disallow /*?*digiSearch
Disallow /*?*term
Disallow /*?*params
Disallow /*?*digiPreview
Disallow /*?*clear_cache
Disallow /*?*orderId
Disallow /*?*tagtag_uid
Disallow /*?*MOBILE_APP_VIEW
Disallow /*?*hl
Disallow /*?*sa
Disallow /*?lang
Allow /*.css
Allow /*.js
Allow /*.jpg
Allow /*.JPG
Allow /*.jpeg
Allow /*.JPEG
Allow /*.png
Allow /*.PNG
Allow /*.gif
Allow /*.GIF
Allow /*.svg
Allow /*.SVG
Allow /bitrix/*.css
Allow /bitrix/*.js
Allow /bitrix/*.jpg
Allow /bitrix/*.JPG
Allow /bitrix/*.jpeg
Allow /bitrix/*.JPEG
Allow /bitrix/*.png
Allow /bitrix/*.PNG
Allow /bitrix/*.gif
Allow /bitrix/*.GIF
Allow /bitrix/*.svg
Allow /bitrix/*.SVG
Allow /upload/*.jpg
Allow /upload/*.JPG
Allow /upload/*.jpeg
Allow /upload/*.JPEG
Allow /upload/*.png
Allow /upload/*.PNG
Allow /upload/*.gif
Allow /upload/*.GIF
Allow /upload/*.svg
Allow /upload/*.SVG
Allow /news/novosti-kompanii/orenburg/
Disallow /*?$
Disallow /*?back_url*
Disallow /*?clid*
Disallow /?PAGEN*
Disallow /*?from=*
Disallow /*?ID=*
Disallow /*?site*
Disallow /*?wbraid=*
Disallow /*?subid*
Disallow /*?lang=*
Disallow /*?added=*
Disallow /*?smartleads*
Disallow /*?type=*
Disallow /*?QUANTITY*
Disallow /*?utm_sourse=*
Disallow /*?public=*
Disallow /*?p=*
Disallow /*?m=*
Disallow /*?item=*
Disallow /sale/viber%3A/
Disallow /novinki/?PAGEN_1=*
Disallow /*etext%3D*
Disallow /catalog/*/*?*
Allow /catalog/*/*?PAG*
Disallow /catalog/*/*?PAG*&*
Disallow /*mindbox-message-key%3D*
Disallow /*tagtag_uid%3D*
Disallow /*open%3D*
Disallow /*yadclid%3D*
Disallow /*yadordid%3D*
Disallow /*rs%3D*
Disallow *adrclid%3D
Disallow *ref%3D
Disallow *target%3D
Disallow *_ym_debug%3D
Disallow *_r%3D
Disallow *erid%3D
Disallow *gtm_debug%3D
Disallow *DISCOUNTED%3D
Disallow *SORT%3D
Disallow *utm_doc%3D
Disallow *utm_geo%3D
Disallow *hcb%3D
Disallow *stepNext%3D
Disallow *currentStep%3D
Disallow *phoneNumber%3D
Disallow *Adv%3D
Disallow *sub_id%3D
Disallow *stat-id%3D
Disallow *yandex-source%3D
Disallow *shop_id%3D
Disallow *ENTITY_ID%3D
Disallow /local/templates/aspro_next/js/noindex.js
Disallow /local/templates/stroylandiya_2021/js/noindex.js

*
ahrefsbot
mj12bot

Rule Path
Disallow /*noindex.js
Disallow /career/news/
Disallow /news/novosti-kompanii/
Disallow /korporativnym-klientam/
Disallow /ajax/form.php?type=reg
Disallow /api/
Disallow /bitrix/
Disallow /bitrix/admin
Disallow /credit/
Disallow /form/
Disallow /include/php/
Disallow /opt/
Disallow /push/
Disallow /redirect/
Disallow /store/
Disallow /upload/
Disallow /auth/
Disallow /cart/
Disallow /catalog/compare/
Disallow /favourites/
Disallow /order/
Disallow /profile/
Disallow /subscribe-confirm/
Disallow /search/*
Disallow /*/search/
Disallow /catalog/*/filter/*
Disallow /*order%3D
Disallow /*sort%3D
Disallow /*in_stock%3D
Disallow /*limit%3D
Disallow /*bitrix_*%3D
Disallow /*brand_id__in%3D
Disallow /*?parameter_
Disallow /*parameter_*%3D
Disallow /*_price%3D
Disallow /*?*admitad_uid
Disallow /*?*adposition
Disallow /*?*all
Disallow /*?*amp
Disallow /*?*attempt
Disallow /*?*block
Disallow /*?*bxrand
Disallow /*?*category
Disallow /*?*city_name
Disallow /*?*current_fieldset
Disallow /*?*gclid
Disallow /*?*id
Disallow /*?*k
Disallow /*?*keyword
Disallow /*?*matchtype
Disallow /*?*mob
Disallow /*?*news-category
Disallow /*?*openstat
Disallow /*?*page
Disallow /*PAGEN_2
Disallow /*?*position
Disallow /*?*print
Disallow /*%26print
Disallow /*?*q
Disallow /*?*referer1
Disallow /*?*referer2
Disallow /*?*region_id
Disallow /*?*region_name
Disallow /*%26region
Disallow /*?*RID
Disallow /*?*s
Disallow /*?*sessid
Disallow /*?*spush
Disallow /*?*utm_campaign
Disallow /*?*utm_content
Disallow /*?*utm_expid
Disallow /*?*utm_medium
Disallow /*?*utm_referrer
Disallow /*?*utm_source
Disallow /*?*utm_term
Disallow /*?*view_type
Disallow /*?*wid
Disallow /*?*yclid
Disallow /*index.php
Disallow /*detail.php
Disallow /*.swf
Disallow /*?*subid
Disallow /*?*placement
Disallow /*?*digiSearch
Disallow /*?*term
Disallow /*?*params
Disallow /*?*digiPreview
Disallow /*?*clear_cache
Disallow /*?*orderId
Disallow /*?*tagtag_uid
Disallow /*?*MOBILE_APP_VIEW
Disallow /*?*hl
Disallow /*?*sa
Disallow /*?lang
Allow /*.css
Allow /*.js
Allow /*.jpg
Allow /*.JPG
Allow /*.jpeg
Allow /*.JPEG
Allow /*.png
Allow /*.PNG
Allow /*.gif
Allow /*.GIF
Allow /*.svg
Allow /*.SVG
Allow /bitrix/*.css
Allow /bitrix/*.js
Allow /bitrix/*.jpg
Allow /bitrix/*.JPG
Allow /bitrix/*.jpeg
Allow /bitrix/*.JPEG
Allow /bitrix/*.png
Allow /bitrix/*.PNG
Allow /bitrix/*.gif
Allow /bitrix/*.GIF
Allow /bitrix/*.svg
Allow /bitrix/*.SVG
Allow /upload/*.jpg
Allow /upload/*.JPG
Allow /upload/*.jpeg
Allow /upload/*.JPEG
Allow /upload/*.png
Allow /upload/*.PNG
Allow /upload/*.gif
Allow /upload/*.GIF
Allow /upload/*.svg
Allow /upload/*.SVG
Allow /news/novosti-kompanii/orenburg/
Disallow /*?$
Disallow /*?back_url*
Disallow /*?clid*
Disallow /?PAGEN*
Disallow /*?from=*
Disallow /*?ID=*
Disallow /*?site*
Disallow /*?wbraid=*
Disallow /*?subid*
Disallow /*?lang=*
Disallow /*?added=*
Disallow /*?smartleads*
Disallow /*?type=*
Disallow /*?QUANTITY*
Disallow /*?utm_sourse=*
Disallow /*?public=*
Disallow /*?p=*
Disallow /*?m=*
Disallow /*?item=*
Disallow /sale/viber%3A/
Disallow /novinki/?PAGEN_1=*
Disallow /*etext%3D*
Disallow /catalog/*/*?*
Allow /catalog/*/*?PAG*
Disallow /catalog/*/*?PAG*&*

megaindex.ru/2.0

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

megaindex.ru

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

ahrefsbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 2

linkpadbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 2

baiduspider

Rule Path
Disallow /

sogou spider

Rule Path
Disallow /

yodaobot

Rule Path
Disallow /

sputnikbot

Rule Path
Disallow /

moget

Rule Path
Disallow /

ichiro

Rule Path
Disallow /

naverbot

Rule Path
Disallow /

yeti

Rule Path
Disallow /

obot

Rule Path
Disallow /

addthis

Rule Path
Disallow /

wotbox

Rule Path
Disallow /

embedly

Rule Path
Disallow /

paperlibot

Rule Path
Disallow /

genieo

Rule Path
Disallow /

showyoubot

Rule Path
Disallow /

tweetmemebot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

exabot

Rule Path
Disallow /

surveybot

Rule Path
Disallow /

sistrix

Rule Path
Disallow /

bdcbot

Rule Path
Disallow /

bdcbot/1.0

Rule Path
Disallow /

rogerbot

Rule Path
Disallow /

spbot

Rule Path
Disallow /

linkdexbot

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

haosouspider

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

grapeshot

Rule Path
Disallow /

proximic

Rule Path
Disallow /

sunrise

Rule Path
Disallow /

butterfly

Rule Path
Disallow /

linguee bot

Rule Path
Disallow /

twengabot-2.0

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

amznkassocbot/4.0

Rule Path
Disallow /

nerdybot

Rule Path
Disallow /

trovitbot

Rule Path
Disallow /

shopwiki

Rule Path
Disallow /

siteexplorer

Rule Path
Disallow /

xovibot

Rule Path
Disallow /

smtbot

Rule Path
Disallow /

meanpathbot

Rule Path
Disallow /

mixrankbot

Rule Path
Disallow /

easouspider

Rule Path
Disallow /

riddler

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

swiftbot

Rule Path
Disallow /

qwantify

Rule Path
Disallow /

psbot

Rule Path
Disallow /

crazywebcrawler-spider

Rule Path
Disallow /

hypercrawl

Rule Path
Disallow /

daumoa

Rule Path
Disallow /

coccoc

Rule Path
Disallow /

netseer

Rule Path
Disallow /

backlinkcrawler

Rule Path
Disallow /

alexabot

Rule Path
Disallow /

surdotlybot

Rule Path
Disallow /

yacybot

Rule Path
Disallow /

queryseekerspider

Rule Path
Disallow /

wbsearchbot

Rule Path
Disallow /

deusu

Rule Path
Disallow /

tdjbot

Rule Path
Disallow /

findxbot/1.0

Rule Path
Disallow /

findxbot

Rule Path
Disallow /

y! j-asr

Rule Path
Disallow /

infoweb

Rule Path
Disallow /

nutch bots

Rule Path
Disallow /

python-urllib

Rule Path
Disallow /

ezooms

Rule Path
Disallow /

unwindfetchor

Rule Path
Disallow /

flipboard

Rule Path
Disallow /

voltron

Rule Path
Disallow /

istellabot

Rule Path
Disallow /

silk

Rule Path
Disallow /

wget

Rule Path
Disallow /

uptimerobot

Rule Path
Disallow /

uptimerobot/2.0

Rule Path
Disallow /

screenerbot

Rule Path
Disallow /

searchmetricsbot

Rule Path
Disallow /

wesee

Rule Path
Disallow /

linkdexbot/2.0

Rule Path
Disallow /

linkdexbot/2.1

Rule Path
Disallow /

linkdexbot/2.2

Rule Path
Disallow /

uptimebot

Rule Path
Disallow /

uptimebot/1.0

Rule Path
Disallow /

lipperhey

Rule Path
Disallow /

ltx71

Rule Path
Disallow /

python-urllib/2.7

Rule Path
Disallow /

sogou web spider/4.0

Rule Path
Disallow /

baiduspider/2.0

Rule Path
Disallow /

privacyawarebot/1.1

Rule Path
Disallow /

yoozbot-2.2

Rule Path
Disallow /

wget

Rule Path
Disallow /

tweetmeme

Rule Path
Disallow /

docomo/2.0

Rule Path
Disallow /

ichiro/4.0

Rule Path
Disallow /

ichiro/3.0

Rule Path
Disallow /

twengabot-discover

Rule Path
Disallow /

twengabot/2.0

Rule Path
Disallow /

twengabot

Rule Path
Disallow /

bendercrawler

Rule Path
Disallow /

baidu

Rule Path
Disallow /

bsalsa

Rule Path
Disallow /

phpservermon/3.1.1

Rule Path
Disallow /

gluten free crawler/1.0

Rule Path
Disallow /

gluten free crawler

Rule Path
Disallow /

sogou pic spider/3.0

Rule Path
Disallow /

sogou head spider/3.0

Rule Path
Disallow /

sogou orion spider/3.0

Rule Path
Disallow /

sogou-test-spider/4.0

Rule Path
Disallow /

sogou pic agent

Rule Path
Disallow /

baiduspider+

Rule Path
Disallow /

baiduspider-video

Rule Path
Disallow /

baiduspider-image

Rule Path
Disallow /

youdaobot/1.0

Rule Path
Disallow /

changedetection

Rule Path
Disallow /

surdotlybot

Rule Path
Disallow /

libwww-perl

Rule Path
Disallow /

mojeekbot

Rule Path
Disallow /

sogou web spider

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://stroylandiya.ru/sitemap.xml
Back to top

Warnings

  • 12 invalid lines.
  • `clean-param` is not a known field.
Back to top