market.unicefusa.org
robots.txt
Robots Exclusion Standard data for market.unicefusa.org
Resource Scan
Scan Details
| Site Domain | market.unicefusa.org |
| Base Domain | unicefusa.org |
| Scan Status | Ok |
| Last Scan | 2025-03-07T12:15:43+00:00 |
| Next Scan | 2025-04-06T12:15:43+00:00 |
Last Scan
| Scanned | 2025-03-07T12:15:43+00:00 |
| URL | https://market.unicefusa.org/robots.txt |
| Domain IPs | 13.249.213.124, 13.249.213.24, 13.249.213.57, 13.249.213.64, 2600:9000:2894:3e00:1a:be49:b340:93a1, 2600:9000:2894:5000:1a:be49:b340:93a1, 2600:9000:2894:7200:1a:be49:b340:93a1, 2600:9000:2894:c800:1a:be49:b340:93a1, 2600:9000:2894:d200:1a:be49:b340:93a1, 2600:9000:2894:d400:1a:be49:b340:93a1, 2600:9000:2894:ee00:1a:be49:b340:93a1, 2600:9000:2894:fc00:1a:be49:b340:93a1 |
| Response IP | 3.170.229.77 |
| Found | Yes |
| Hash | a382da5075ab9245a2629421afecf4ebc26db6e7a7dd92b8618097f943537d06 |
| SimHash | 28419c02af91 |
Groups
*
| Rule | Path |
|---|---|
| Allow | / |
| Disallow | /bag/* |
| Disallow | /*/bag/* |
| Disallow | /checkout/* |
| Disallow | /*/checkout/* |
| Disallow | /myaccount/* |
| Disallow | /*/myaccount/* |
| Disallow | /*/pg-sortby-*/* |
| Disallow | /*/s/* |
| Disallow | /*?searchkeyword=* |
| Disallow | /*%26searchkeyword%3D* |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.market.unicefusa.org/sitemaps/sitemap-index.xml |