Log In Sign Up

hafen.pl
robots.txt

Robots Exclusion Standard data for hafen.pl

Resource Scan

Scan Details

Site Domain hafen.pl
Base Domain hafen.pl
Scan Status Ok
Last Scan2026-02-19T21:21:44+00:00
Next Scan 2026-03-21T21:21:44+00:00

Last Scan

Scanned2026-02-19T21:21:44+00:00
URL https://hafen.pl/robots.txt
Redirect https://www.hafen.pl/robots.txt
Redirect Domain www.hafen.pl
Redirect Base hafen.pl
Domain IPs 104.26.6.181, 104.26.7.181, 172.67.69.116, 2606:4700:20::681a:6b5, 2606:4700:20::681a:7b5, 2606:4700:20::ac43:4574
Redirect IPs 104.26.6.181, 104.26.7.181, 172.67.69.116, 2606:4700:20::681a:6b5, 2606:4700:20::681a:7b5, 2606:4700:20::ac43:4574
Response IP 104.26.6.181
Found Yes
Hash 9396ea6d6e438ad0920dabe483a1f86d11686de48ab596727a2044f08b4733c4
SimHash 60d4019565f6

Groups

gptbot

Rule Path
Allow /

ccbot

Rule Path
Allow /

chatgpt-user

Rule Path
Allow /

claudebot

Rule Path
Allow /

perplexitybot

Rule Path
Allow /

googlebot

Rule Path
Allow /

bingbot

Rule Path
Allow /

*

Rule Path
Disallow /*sort%3D-name*
Disallow /*sort%3Dname*
Disallow /*sort%3Dprice*
Disallow /*sort%3D-price*
Disallow /*sort%3Dcatalog_number*
Disallow /*pageSize%3D*
Disallow /*?promotionsOnly=*
Disallow /*?newOnly=1*
Allow /compiled/app.css$
Allow /compiled/app.js$
Allow /
Back to top

Other Records

Field Value
sitemap https://www.hafen.pl/sitemap.xml
Back to top

Comments

  • Allow all major AI and search engine crawlers to access all main content
  • General user-agent rules
  • Allow static assets for proper rendering
  • Explicitly allow everything else (so ChatGPT and LLMs can see your full content)
  • Sitemap for structured discovery
Back to top