market-krovlya.ru
robots.txt

Robots Exclusion Standard data for market-krovlya.ru

Archived Snapshots

Resource Scan

Scan Details

Site Domain	market-krovlya.ru
Base Domain	market-krovlya.ru
Scan Status	Ok
Last Scan	2025-05-21T16:53:04+00:00
Next Scan	2025-06-20T16:53:04+00:00

Last Scan

Scanned	2025-05-21T16:53:04+00:00
URL	https://market-krovlya.ru/robots.txt
Domain IPs	81.200.117.70
Response IP	81.200.117.70
Found	Yes
Hash	d1cdb06afde20592d2f53ea9bbd97792c73b1f5e92c732438196c61a6a50821b
SimHash	4a10cb656901

Groups

*

Rule	Path
Allow	/catalog/view/javascript/bootstrap/css/*.css
Allow	/catalog/view/theme/supermarket-gc/stylesheet/*.css
Allow	/catalog/view/javascript/font-awesome/css/*.css
Allow	/catalog/view/javascript/jquery/owl-carousel/*.css
Allow	/catalog/view/javascript/jquery/*.js
Allow	/catalog/view/javascript/bootstrap/js/*.js
Allow	/catalog/view/javascript/bootstrap/js/*.js
Allow	/catalog/view/javascript/*.js
Allow	/catalog/view/javascript/*.js
Allow	/catalog/view/javascript/jquery/owl-carousel/*.js
Allow	/catalog/view/javascript/jquery/colorbox/*.css
Allow	/catalog/view/javascript/jquery/owl-carousel/*.css
Allow	/catalog/view/javascript/jquery/colorbox/*.js
Disallow	*.gif$
Disallow	*.WEBP$
Disallow	*.PNG$
Disallow	*.jpeg$
Disallow	*.JPG$
Disallow	*.jpg$
Disallow	*.webp$
Disallow	*.JPEG$
Disallow	*.GIF$
Disallow	*.png$
Disallow	/?page=
Disallow	/*route%3Daccount/
Disallow	/*route%3Daffiliate/
Disallow	/*route%3Dcheckout/
Disallow	/*route%3Dproduct/search
Disallow	/index.php?route=product%2Fproduct*&manufacturer_id=
Disallow	/admin
Disallow	/catalog
Disallow	/download
Disallow	/system
Disallow	/*?sort=
Disallow	/?etext=
Disallow	/*%26sort%3D
Disallow	/*?order=
Disallow	/*%26order%3D
Disallow	/*?limit=
Disallow	/*%26limit%3D
Disallow	/*?filter_name=
Disallow	/*%26filter_name%3D
Disallow	/*?filter_sub_category=
Disallow	/*%26filter_sub_category%3D
Disallow	/*?filter_description=
Disallow	/*%26filter_description%3D
Disallow	/*?tracking=
Disallow	/*%26tracking%3D
Disallow	/napolnie-pokritija-*
Disallow	/napolnie-pokritija-/laminat-
Disallow	/napolnie-pokritija-/laminat-/laminat-krono-original-*
Disallow	/napolnie-pokritija-/laminat-/laminat-kronospan-*
Disallow	/politika-konfidencialnosti
Allow	/index.php?route=extension%2Ffeed%2Fgoogle_sitemap

Rule

Path

Allow

/catalog/view/javascript/bootstrap/css/*.css

Allow

/catalog/view/theme/supermarket-gc/stylesheet/*.css

Allow

/catalog/view/javascript/font-awesome/css/*.css

Allow

/catalog/view/javascript/jquery/owl-carousel/*.css

Allow

/catalog/view/javascript/jquery/*.js

Allow

/catalog/view/javascript/bootstrap/js/*.js

Allow

/catalog/view/javascript/bootstrap/js/*.js

Allow

/catalog/view/javascript/*.js

Allow

/catalog/view/javascript/*.js

Allow

/catalog/view/javascript/jquery/owl-carousel/*.js

Allow

/catalog/view/javascript/jquery/colorbox/*.css

Allow

/catalog/view/javascript/jquery/owl-carousel/*.css

Allow

/catalog/view/javascript/jquery/colorbox/*.js

Disallow

*.gif$

Disallow

*.WEBP$

Disallow

*.PNG$

Disallow

*.jpeg$

Disallow

*.JPG$

Disallow

*.jpg$

Disallow

*.webp$

Disallow

*.JPEG$

Disallow

*.GIF$

Disallow

*.png$

Disallow

/*?page=*

Disallow

/*route%3Daccount/

Disallow

/*route%3Daffiliate/

Disallow

/*route%3Dcheckout/

Disallow

/*route%3Dproduct/search

Disallow

/index.php?route=product%2Fproduct*&manufacturer_id=

Disallow

/admin

Disallow

/catalog

Disallow

/download

Disallow

/system

Disallow

/*?sort=

Disallow

/*?etext=*

Disallow

/*%26sort%3D

Disallow

/*?order=

Disallow

/*%26order%3D

Disallow

/*?limit=

Disallow

/*%26limit%3D

Disallow

/*?filter_name=

Disallow

/*%26filter_name%3D

Disallow

/*?filter_sub_category=

Disallow

/*%26filter_sub_category%3D

Disallow

/*?filter_description=

Disallow

/*%26filter_description%3D

Disallow

/*?tracking=

Disallow

/*%26tracking%3D

Disallow

/napolnie-pokritija-*

Disallow

/napolnie-pokritija-*/laminat-*

Disallow

/napolnie-pokritija-*/laminat-*/laminat-krono-original-*

Disallow

/napolnie-pokritija-*/laminat-*/laminat-kronospan-*

Disallow

/politika-konfidencialnosti

Allow

/index.php?route=extension%2Ffeed%2Fgoogle_sitemap

yadirectfetcher

Rule	Path
Disallow	/

Rule

Path

Disallow

yandex

Rule	Path
Allow	/catalog/view/javascript/bootstrap/css/*.css
Allow	/catalog/view/theme/supermarket-gc/stylesheet/*.css
Allow	/catalog/view/javascript/font-awesome/css/*.css
Allow	/catalog/view/javascript/jquery/owl-carousel/*.css
Allow	/catalog/view/javascript/jquery/*.js
Allow	/catalog/view/javascript/bootstrap/js/*.js
Allow	/catalog/view/javascript/bootstrap/js/*.js
Allow	/catalog/view/javascript/*.js
Allow	/catalog/view/javascript/*.js
Allow	/catalog/view/javascript/jquery/owl-carousel/*.js
Allow	/catalog/view/javascript/jquery/colorbox/*.css
Allow	/catalog/view/javascript/jquery/owl-carousel/*.css
Allow	/catalog/view/javascript/jquery/colorbox/*.js
Disallow	*.gif$
Disallow	*.WEBP$
Disallow	*.PNG$
Disallow	*.jpeg$
Disallow	*.JPG$
Disallow	*.jpg$
Disallow	*.webp$
Disallow	*.JPEG$
Disallow	*.GIF$
Disallow	*.png$
Disallow	/?page=
Disallow	/*route%3Daccount/
Disallow	/*route%3Daffiliate/
Disallow	/*route%3Dcheckout/
Disallow	/*route%3Dproduct/search
Disallow	/index.php?route=product%2Fproduct*&manufacturer_id=
Disallow	/admin
Disallow	/catalog
Disallow	/download
Disallow	/system
Disallow	/*?sort=
Disallow	/*%26sort%3D
Disallow	/?etext=
Disallow	/*?order=
Disallow	/*%26order%3D
Disallow	/*?limit=
Disallow	/*%26limit%3D
Disallow	/*?filter_name=
Disallow	/*%26filter_name%3D
Disallow	/*?filter_sub_category=
Disallow	/*%26filter_sub_category%3D
Disallow	/*?filter_description=
Disallow	/*%26filter_description%3D
Disallow	/napolnie-pokritija-*
Disallow	/napolnie-pokritija-/laminat-
Disallow	/napolnie-pokritija-/laminat-/laminat-krono-original-*
Disallow	/napolnie-pokritija-/laminat-/laminat-kronospan-*
Disallow	/politika-konfidencialnosti
Allow	/index.php?route=extension%2Ffeed%2Fgoogle_sitemap

Rule

Path

Allow

/catalog/view/javascript/bootstrap/css/*.css

Allow

/catalog/view/theme/supermarket-gc/stylesheet/*.css

Allow

/catalog/view/javascript/font-awesome/css/*.css

Allow

/catalog/view/javascript/jquery/owl-carousel/*.css

Allow

/catalog/view/javascript/jquery/*.js

Allow

/catalog/view/javascript/bootstrap/js/*.js

Allow

/catalog/view/javascript/bootstrap/js/*.js

Allow

/catalog/view/javascript/*.js

Allow

/catalog/view/javascript/*.js

Allow

/catalog/view/javascript/jquery/owl-carousel/*.js

Allow

/catalog/view/javascript/jquery/colorbox/*.css

Allow

/catalog/view/javascript/jquery/owl-carousel/*.css

Allow

/catalog/view/javascript/jquery/colorbox/*.js

Disallow

*.gif$

Disallow

*.WEBP$

Disallow

*.PNG$

Disallow

*.jpeg$

Disallow

*.JPG$

Disallow

*.jpg$

Disallow

*.webp$

Disallow

*.JPEG$

Disallow

*.GIF$

Disallow

*.png$

Disallow

/*?page=*

Disallow

/*route%3Daccount/

Disallow

/*route%3Daffiliate/

Disallow

/*route%3Dcheckout/

Disallow

/*route%3Dproduct/search

Disallow

/index.php?route=product%2Fproduct*&manufacturer_id=

Disallow

/admin

Disallow

/catalog

Disallow

/download

Disallow

/system

Disallow

/*?sort=

Disallow

/*%26sort%3D

Disallow

/*?etext=*

Disallow

/*?order=

Disallow

/*%26order%3D

Disallow

/*?limit=

Disallow

/*%26limit%3D

Disallow

/*?filter_name=

Disallow

/*%26filter_name%3D

Disallow

/*?filter_sub_category=

Disallow

/*%26filter_sub_category%3D

Disallow

/*?filter_description=

Disallow

/*%26filter_description%3D

Disallow

/napolnie-pokritija-*

Disallow

/napolnie-pokritija-*/laminat-*

Disallow

/napolnie-pokritija-*/laminat-*/laminat-krono-original-*

Disallow

/napolnie-pokritija-*/laminat-*/laminat-kronospan-*

Disallow

/politika-konfidencialnosti

Allow

/index.php?route=extension%2Ffeed%2Fgoogle_sitemap

googlebot-image/1.0

Rule	Path
Disallow	*.png$
Disallow	*.jpg$
Disallow	*.gif$
Disallow	*.JPEG$
Disallow	*.webp$
Disallow	*.JPG$
Disallow	*.gif$
Disallow	*.jpeg$
Disallow	*.WEBP$

Rule

Path

Disallow

*.png$

Disallow

*.jpg$

Disallow

*.gif$

Disallow

*.JPEG$

Disallow

*.webp$

Disallow

*.JPG$

Disallow

*.gif$

Disallow

*.jpeg$

Disallow

*.WEBP$

yandeximages

Rule	Path
Disallow	*.jpeg$
Disallow	*.png$
Disallow	*.jpg$
Disallow	*.GIF$
Disallow	*.webp$
Disallow	*.JPEG$
Disallow	*.JPG$
Disallow	*.PNG$
Disallow	*.gif$

Rule

Path

Disallow

*.jpeg$

Disallow

*.png$

Disallow

*.jpg$

Disallow

*.GIF$

Disallow

*.webp$

Disallow

*.JPEG$

Disallow

*.JPG$

Disallow

*.PNG$

Disallow

*.gif$

baiduspider

Rule	Path
Disallow	*.png$
Disallow	*.jpg$
Disallow	*.JPEG$
Disallow	*.webp$
Disallow	*.GIF$
Disallow	*.jpeg$
Disallow	*.JPG$
Disallow	*.gif$
Disallow	*.WEBP$

Rule

Path

Disallow

*.png$

Disallow

*.jpg$

Disallow

*.JPEG$

Disallow

*.webp$

Disallow

*.GIF$

Disallow

*.jpeg$

Disallow

*.JPG$

Disallow

*.gif$

Disallow

*.WEBP$

bingbot

Rule	Path
Disallow	*.webp$
Disallow	*.JPG$
Disallow	*.GIF$
Disallow	*.jpeg$
Disallow	*.jpg$
Disallow	*.png$
Disallow	*.PNG$
Disallow	*.JPEG$
Disallow	*.gif$

Rule

Path

Disallow

*.webp$

Disallow

*.JPG$

Disallow

*.GIF$

Disallow

*.jpeg$

Disallow

*.jpg$

Disallow

*.png$

Disallow

*.PNG$

Disallow

*.JPEG$

Disallow

*.gif$

yandeximages/3.0

Rule	Path
Disallow	*.jpg$
Disallow	*.png$
Disallow	*.jpeg$
Disallow	*.GIF$
Disallow	*.jpeg$
Disallow	*.JPEG$
Disallow	*.webp$
Disallow	*.WEBP$
Disallow	*.JPG$

Rule

Path

Disallow

*.jpg$

Disallow

*.png$

Disallow

*.jpeg$

Disallow

*.GIF$

Disallow

*.jpeg$

Disallow

*.JPEG$

Disallow

*.webp$

Disallow

*.WEBP$

Disallow

*.JPG$

googlebot-image

Rule	Path
Disallow	*.jpg$
Disallow	*.png$
Disallow	*.gif$
Disallow	*.jpeg$
Disallow	*.JPG$
Disallow	*.PNG$
Disallow	*.WEBP$
Disallow	*.WEBP$
Disallow	*.GIF$

Rule

Path

Disallow

*.jpg$

Disallow

*.png$

Disallow

*.gif$

Disallow

*.jpeg$

Disallow

*.JPG$

Disallow

*.PNG$

Disallow

*.WEBP$

Disallow

*.WEBP$

Disallow

*.GIF$

megaindex.com

Rule	Path
Disallow	/

Rule

Path

Disallow

dataforseobot

Rule	Path
Disallow	/

Rule

Path

Disallow

semrushbot-ba

Rule	Path
Disallow	/

Rule

Path

Disallow

megaindex.ru

Rule	Path
Disallow	/

Rule

Path

Disallow

ahrefsbot2

Rule	Path
Disallow	/

Rule

Path

Disallow

facebookbot

Rule	Path
Disallow	/

Rule

Path

Disallow

scrapy

Rule	Path
Disallow	/

Rule

Path

Disallow

semrushbot-swa

Rule	Path
Disallow	/

Rule

Path

Disallow

seznambot

Rule	Path
Disallow	/

Rule

Path

Disallow

petalbot

Rule	Path
Disallow	/

Rule

Path

Disallow

claude

Rule	Path
Disallow	/

Rule

Path

Disallow

flipboardproxy

Rule	Path
Disallow	/

Rule

Path

Disallow

megaindex

Rule	Path
Disallow	/

Rule

Path

Disallow

dataforseobot/1.0

Rule	Path
Disallow	/

Rule

Path

Disallow

safednsbot

Rule	Path
Disallow	/

Rule

Path

Disallow

ahrefssiteaudit

Rule	Path
Disallow	/

Rule

Path

Disallow

oai-searchbot

Rule	Path
Disallow	/

Rule

Path

Disallow

blackwidow

Rule	Path
Disallow	/

Rule

Path

Disallow

getintent crawler

Rule	Path
Disallow	/

Rule

Path

Disallow

cloudfind

Rule	Path
Disallow	/

Rule

Path

Disallow

flipboardrss

Rule	Path
Disallow	/

Rule

Path

Disallow

mj12bot

Rule	Path
Disallow	/

Rule

Path

Disallow

timpibot

Rule	Path
Disallow	/

Rule

Path

Disallow

velenpublicwebcrawler

Rule	Path
Disallow	/

Rule

Path

Disallow

yak

Rule	Path
Disallow	/

Rule

Path

Disallow

bot

Rule	Path
Disallow	/

Rule

Path

Disallow

chat-user

Rule	Path
Disallow	/

Rule

Path

Disallow

criteobot

Rule	Path
Disallow	/

Rule

Path

Disallow

imagesiftbot

Rule	Path
Disallow	/

Rule

Path

Disallow

semrushbot-ct

Rule	Path
Disallow	/

Rule

Path

Disallow

zoominfobot

Rule	Path
Disallow	/

Rule

Path

Disallow

slurp

Rule	Path
Disallow	/

Rule

Path

Disallow

semrushbot

Rule	Path
Disallow	/

Rule

Path

Disallow

linkpadbot

Rule	Path
Disallow	/

Rule

Path

Disallow

seekport crawler

Rule	Path
Disallow	/

Rule

Path

Disallow

a.pr-cy.ru

Rule	Path
Disallow	/

Rule

Path

Disallow

niraiya.com

Rule	Path
Disallow	/

Rule

Path

Disallow

meta-externalagent

Rule	Path
Disallow	/

Rule

Path

Disallow

Rule	Path
Disallow	/

Rule

Path

Disallow

serpstatbot

Rule

Path

Disallow

claudebot

Rule

Path

Disallow

semrushbot-sa

Rule

Path

Disallow

crawler

Rule

Path

Disallow

ahrefsbot

Rule

Path

Disallow

dataforseobot

Rule

Path

Disallow

fbexternalhit

Rule

Path

Disallow

megaindex.ru

Rule

Path

Disallow

semrushbot-si

Rule

Path

Disallow

semrushbot-bm

Rule

Path

Disallow

rome client

Rule

Path

Disallow

dotbot

Rule

Path

Disallow

cloudfind

Rule

Path

Disallow

flipboardrss

Rule

Path

Disallow

cloud-web

Rule

Path

Disallow

a.pr-cy.ru

Rule

Path

Disallow

gptbot

Rule

Path

Disallow

oai-searchbot

Rule

Path

Disallow

chatgpt-user

Rule

Path

Disallow

Other Records

Field

Value

sitemap

https://market-krovlya.ru/sitemap.xml

Warnings

7 invalid lines.
`clean-param` is not a known field.
`host` is not a known field.
`https` is not a known field.
`ðð¾ð±ð°ð²ñ ð½ð° ð¼ðµññð¾ ð»ð¸ð½ð¸ð¸ ñññð»ðºñ ð½ð° ðð¾ð»ð¸ñð¸ðºñ ðºð¾ð½ñð¸ð´ðµð½ñð¸ð°ð»ñð½ð¾ññð¸ https` is not a known field.
`ð¢ðµðºññ` is not a known field.

market-krovlya.rurobots.txt

Resource Scan

Scan Details

Last Scan

Groups

*

yadirectfetcher

yandex

googlebot-image/1.0

yandeximages

baiduspider

bingbot

yandeximages/3.0

googlebot-image

megaindex.com

dataforseobot

semrushbot-ba

megaindex.ru

ahrefsbot2

facebookbot

scrapy

semrushbot-swa

seznambot

petalbot

claude

flipboardproxy

megaindex

dataforseobot/1.0

safednsbot

ahrefssiteaudit

oai-searchbot

blackwidow

getintent crawler

cloudfind

flipboardrss

mj12bot

timpibot

velenpublicwebcrawler

yak

bot

chat-user

criteobot

imagesiftbot

semrushbot-ct

zoominfobot

slurp

semrushbot

linkpadbot

seekport crawler

a.pr-cy.ru

niraiya.com

meta-externalagent

pinterest

serpstatbot

claudebot

semrushbot-sa

crawler

ahrefsbot

dataforseobot

fbexternalhit

megaindex.ru

semrushbot-si

semrushbot-bm

rome client

dotbot

cloudfind

flipboardrss

cloud-web

a.pr-cy.ru

gptbot

oai-searchbot

chatgpt-user

Other Records

Warnings

market-krovlya.ru
robots.txt