unicat.msf.org
robots.txt
Robots Exclusion Standard data for unicat.msf.org
Resource Scan
Scan Details
| Site Domain | unicat.msf.org |
| Base Domain | msf.org |
| Scan Status | Ok |
| Last Scan | 2026-01-29T01:20:09+00:00 |
| Next Scan | 2026-02-28T01:20:09+00:00 |
Last Scan
| Scanned | 2026-01-29T01:20:09+00:00 |
| URL | https://unicat.msf.org/robots.txt |
| Domain IPs | 13.36.16.2, 15.188.131.118, 15.236.176.174 |
| Response IP | 13.36.16.2 |
| Found | Yes |
| Hash | 3eb3bf37cee7d051f48011c58bca8fe9afac038cdef1f222936994c60f16470d |
| SimHash | 6c2dd552ebd3 |
Groups
*
*
| Rule | Path |
|---|---|
| Allow | * |
| Disallow | /web/login |
| Disallow | /shop/wishlist |
| Disallow | /shop/cart |
| Disallow | /shop/checkout |
| Disallow | /search |
| Disallow | /fr/search |
| Disallow | /shop/product/ |
| Disallow | /fr/shop/product/ |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 5 |
Other Records
| Field | Value |
|---|---|
| sitemap | https://unicat.msf.org/sitemap.xml |
Comments