static.polityka.pl
robots.txt

Robots Exclusion Standard data for static.polityka.pl

Resource Scan

Scan Details

Site Domain static.polityka.pl
Base Domain polityka.pl
Scan Status Ok
Last Scan2025-10-05T01:14:17+00:00
Next Scan 2025-10-19T01:14:17+00:00

Last Scan

Scanned2025-10-05T01:14:17+00:00
URL https://static.polityka.pl/robots.txt
Domain IPs 35.190.31.207
Response IP 35.190.31.207
Found Yes
Hash 8fc20e9238aef26a671969743727962a335ee197f02f89783e495a018b8a9ffd
SimHash 6b0d1ec5e5b0

Groups

*

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

googlebot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

sentibot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

semrushbot

Rule Path
Disallow /forum

Other Records

Field Value
sitemap http://www.polityka.pl/sitemap.xml
sitemap http://www.polityka.pl/newsSitemap.xml

Warnings

  • 1 invalid line.