ziarenkokawy.pl
robots.txt

Robots Exclusion Standard data for ziarenkokawy.pl

Resource Scan

Scan Details

Site Domain ziarenkokawy.pl
Base Domain ziarenkokawy.pl
Scan Status Ok
Last Scan2024-06-03T20:21:13+00:00
Next Scan 2024-06-10T20:21:13+00:00

Last Scan

Scanned2024-06-03T20:21:13+00:00
URL https://ziarenkokawy.pl/robots.txt
Redirect https://www.ziarenkokawy.pl/robots.txt
Redirect Domain www.ziarenkokawy.pl
Redirect Base ziarenkokawy.pl
Domain IPs 104.21.79.114, 172.67.170.130, 2606:4700:3031::6815:4f72, 2606:4700:3031::ac43:aa82
Redirect IPs 104.21.79.114, 172.67.170.130, 2606:4700:3031::6815:4f72, 2606:4700:3031::ac43:aa82
Response IP 104.21.79.114
Found Yes
Hash e1d133319f09c0730281d8c8d716161d6a73cb178bf738ddc2db43c4d13812a7
SimHash 61009a008bb2

Groups

scrapy

Rule Path
Allow /

*

Rule Path
Disallow /wp-admin/
Allow /wp-admin/admin-ajax.php

Other Records

Field Value
sitemap https://www.ziarenkokawy.pl/sitemap.xml
sitemap https://www.ziarenkokawy.pl/sitemap.rss