buythis.pl
robots.txt

Robots Exclusion Standard data for buythis.pl

Resource Scan

Scan Details

Site Domain buythis.pl
Base Domain buythis.pl
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2026-01-05T06:17:40+00:00
Next Scan 2026-04-05T06:17:40+00:00

Last Successful Scan

Scanned2025-03-11T22:44:05+00:00
URL https://buythis.pl/robots.txt
Domain IPs 157.90.182.9
Response IP 157.90.182.9
Found Yes
Hash 305fff15f57b05da8ea8726c12e549492ef3963bd8e426e768975ef9de336260
SimHash 481c880287b4

Groups

*

Rule Path
Disallow /stats/
Disallow /koszyk/
Disallow /regulamin/
Disallow /polityka-prywatnosci/
Disallow /zamowienie/
Disallow /moje-konto/
Disallow /?add-to-cart
Disallow /*?add-to-cart
Disallow /*pwb-brand-filter%3D
Disallow /*orderby%3D
Disallow /*/feed/
Disallow *s%3D
Disallow /tag-*
Disallow /?product-page=*
Disallow /?swcfpc=1

*

Rule Path
Disallow /private/

alphaai

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

applebot-extended

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

claude-web

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

diffbot

Rule Path
Disallow /

facebookbot

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

imagesiftbot

Rule Path
Disallow /

omgili

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

openai

Rule Path
Disallow /

*

Rule Path
Disallow /wp-admin/
Allow /wp-admin/admin-ajax.php

Other Records

Field Value
sitemap https://buythis.pl/sitemap_index.xml

Comments

  • Block AI and LLM bots