recenzo.pl
robots.txt

Robots Exclusion Standard data for recenzo.pl

Resource Scan

Scan Details

Site Domain recenzo.pl
Base Domain recenzo.pl
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2025-09-20T10:58:49+00:00
Next Scan 2025-12-19T10:58:49+00:00

Last Successful Scan

Scanned2022-10-09T19:56:04+00:00
URL https://recenzo.pl/robots.txt
Redirect https://www.recenzo.pl/robots.txt
Redirect Domain www.recenzo.pl
Redirect Base recenzo.pl
Response IP 104.21.41.28, 172.67.159.57
Found Yes
Hash b75579cdfcecefdb69c70f04e10ef258a34a4453b77f7c8f52f0ecc474fb1757
SimHash 0a1cc962f731

Groups

*

Rule Path
Allow /*.css
Allow /*.js
Disallow /product/click-eshop/
Disallow /eshop-click/

spbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

proximic

Rule Path
Disallow /

grapeshot

Rule Path
Disallow /

slurp

Rule Path
Disallow /

yandex

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

mediatoolkitbot

Rule Path
Disallow /

linguee

Rule Path
Disallow /

alphaseobot

Rule Path
Disallow /

xenu's link sleuth 1.1c

Rule Path
Disallow /

xenu link sleuth/1.3.8

Rule Path
Disallow /

velenpublicwebcrawler

Rule Path
Disallow /

kingkevinbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /
Disallow /admin/
Disallow /auth/