doxa.fm
robots.txt

Robots Exclusion Standard data for doxa.fm

Resource Scan

Scan Details

Site Domain doxa.fm
Base Domain doxa.fm
Scan Status Ok
Last Scan2025-12-12T13:51:52+00:00
Next Scan 2026-01-11T13:51:52+00:00

Last Scan

Scanned2025-12-12T13:51:52+00:00
URL https://doxa.fm/robots.txt
Domain IPs 85.128.253.154
Response IP 85.128.253.154
Found Yes
Hash bd9cb78a46c7e4e63e6d880238d423883201c2a912f4dd3b2381ee48a44cc607
SimHash 5154c639dfb3

Groups

scrapy

Rule Path
Allow /

scrapy

Rule Path
Allow /

*

Rule Path
Disallow /dokumenty/

*

Rule Path
Disallow /wp-admin/
Disallow /wp-login.php
Disallow /cgi-bin/

Other Records

Field Value
crawl-delay 10

mj12bot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

python

Rule Path
Disallow /

curl

Rule Path
Disallow /

no_user_agent

Rule Path
Disallow /

bot

Rule Path
Disallow /

Comments

  • Jacek
  • Blokowanie dzikich botów