booxy.pl
robots.txt

Robots Exclusion Standard data for booxy.pl

Resource Scan

Scan Details

Site Domain booxy.pl
Base Domain booxy.pl
Scan Status Failed
Failure ReasonScan timed out.
Last Scan2024-11-13T21:11:01+00:00
Next Scan 2024-11-27T21:11:01+00:00

Last Successful Scan

Scanned2024-10-29T21:09:55+00:00
URL https://booxy.pl/robots.txt
Redirect https://www.booxy.pl/robots.txt
Redirect Domain www.booxy.pl
Redirect Base booxy.pl
Domain IPs 185.183.8.127, 2a00:1ed0:1:1800:7:3d:7400:1
Redirect IPs 185.183.8.127, 2a00:1ed0:1:1800:7:3d:7400:1
Response IP 185.183.8.127
Found Yes
Hash bbe8c904a9ed18acff44e23f3d4e5e4a52bf2bc0a376deecb375fb95870de109
SimHash 721d90658a93

Groups

baiduspider

Rule Path
Disallow /

yandex

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

rogerbot

Rule Path
Disallow /

slurp

Rule Path
Disallow /

mail.ru

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

exabot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

screaming frog seo spider

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

sentibot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

slurp

Rule Path
Disallow /

grapeshot

Rule Path
Disallow /

seekportbot

Rule Path
Disallow /

awariobot

Rule Path
Disallow /

*

Rule Path
Allow /

Other Records

Field Value
crawl-delay 5

seznambot

Rule Path
Allow /

Other Records

Field Value
sitemap https://www.booxy.pl/sitemap.xml
sitemap https://www.booxy.pl/sitemap1.xml
sitemap https://www.booxy.pl/sitemap2.xml
sitemap https://www.booxy.pl/sitemap3.xml
sitemap https://www.booxy.pl/sitemap4.xml
sitemap https://www.booxy.pl/sitemap5.xml
sitemap https://www.booxy.pl/sitemap6.xml
sitemap https://www.booxy.pl/sitemap7.xml
sitemap https://www.booxy.pl/sitemap10.xml
sitemap https://www.booxy.pl/sitemap/a/0.xml
sitemap https://www.booxy.pl/sitemap/a/1.xml
sitemap https://www.booxy.pl/sitemap/a/2.xml
sitemap https://www.booxy.pl/sitemap/u/0.xml
sitemap https://www.booxy.pl/sitemap/t/0.xml

Warnings

  • `request-rate` is not a known field.