prow.si
robots.txt

Robots Exclusion Standard data for prow.si

Resource Scan

Scan Details

Site Domain prow.si
Base Domain prow.si
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2025-10-05T18:31:15+00:00
Next Scan 2026-01-03T18:31:15+00:00

Last Successful Scan

Scanned2025-03-10T02:05:14+00:00
URL https://prow.si/robots.txt
Domain IPs 104.21.31.7, 172.67.174.154, 2606:4700:3035::6815:1f07, 2606:4700:3037::ac43:ae9a
Response IP 104.21.31.7
Found Yes
Hash 94bacaf26dabb0cb62eaccb0906a4e282ea5c0367299151093742af77723f333
SimHash 6d10183061b5

Groups

*

Rule Path
Disallow /pages/

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

bingpreview

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

Other Records

Field Value
sitemap https://prow.si/sitemap.xml

Comments

  • www.robotstxt.org/
  • Allow crawling of all content