msn.com.my
robots.txt
Robots Exclusion Standard data for msn.com.my
Resource Scan
Scan Details
| Site Domain | msn.com.my |
| Base Domain | msn.com.my |
| Scan Status | Ok |
| Last Scan | 2024-09-19T09:36:10+00:00 |
| Next Scan | 2024-09-26T09:36:10+00:00 |
Last Scan
| Scanned | 2024-09-19T09:36:10+00:00 |
| URL | http://msn.com.my/robots.txt |
| Redirect | http://www.msn.com/robots.txt |
| Redirect Domain | www.msn.com |
| Redirect Base | msn.com |
| Domain IPs | 131.253.33.203 |
| Redirect IPs | 204.79.197.203 |
| Response IP | 204.79.197.203 |
| Found | Yes |
| Hash | 584c9dd32d58ecd04e5a4bb3e65baad88f47d428c899afa448b1b755fce1b8c1 |
| SimHash | 4d90db71c526 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /*/health/search/filter |
| Disallow | /spartan |
| Disallow | /pt-ao |
| Disallow | /*preview%3D* |
| Disallow | /*/autos/marketplace/product/* |
| Disallow | /*/cars/marketplace/product/* |
| Disallow | /*?item=* |
| Disallow | /*%26item%3D* |
| Disallow | /codeCoverage |
| Disallow | /*/studio |
| Disallow | /*?loc=*&cvid=* |
| Disallow | /*/shopping/ |
yandex.com/bot
| Rule | Path |
|---|---|
| Disallow | /*/weather/ |
| Disallow | /*/clima/ |
| Disallow | /*/cuaca/ |
| Disallow | /*/eltiempo/ |
| Disallow | /*/el-tiempo/ |
| Disallow | /*/havadurumu/ |
| Disallow | /*/idojaras/ |
| Disallow | /*/meteo/ |
| Disallow | /*/meteorologia/ |
| Disallow | /*/pocasi/ |
| Disallow | /*/pogoda/ |
| Disallow | /*/saa/ |
| Disallow | /*/vader/ |
| Disallow | /*/vejr/ |
| Disallow | /*/weer/ |
| Disallow | /*/wetter/ |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 1 |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.msn.com/sitemaps/casualgames/sitemap-index.xml |
| sitemap | https://www.msn.com/sitemaps/consumption/index.xml |
| sitemap | https://www.msn.cn/sitemaps/consumption/index.xml |
| sitemap | https://www.msn.com/sitemaps/health/health-sitemap-index.xml |
| sitemap | https://www.msn.com/staticsb/statics/latest/0/casualgames/sitemaps/sitemap-index.xml |
| sitemap | https://www.msn.com/staticsb/statics/latest/0/finance/sitemaps/sitemap-index.xml |
| sitemap | https://www.msn.com/sitemaps/weather/sitemap-index.xml |