markets.businessinsider.com
robots.txt
Robots Exclusion Standard data for markets.businessinsider.com
Resource Scan
Scan Details
| Site Domain | markets.businessinsider.com |
| Base Domain | businessinsider.com |
| Scan Status | Ok |
| Last Scan | 2024-11-19T05:44:34+00:00 |
| Next Scan | 2024-11-26T05:44:34+00:00 |
Last Scan
| Scanned | 2024-11-19T05:44:34+00:00 |
| URL | https://markets.businessinsider.com/robots.txt |
| Domain IPs | 151.101.0.64, 151.101.128.64, 151.101.192.64, 151.101.64.64 |
| Response IP | 199.232.44.64 |
| Found | Yes |
| Hash | d94303eb400ea03aa548044b55037f7c5dffd795bfb9cfd4ae0e4f0ca99c56a1 |
| SimHash | ff958c55e784 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /export/ |
| Disallow | /Export/ |
| Disallow | /ajax/ |
| Disallow | /Ajax/ |
| Disallow | /admin/ |
| Disallow | /Admin/ |
| Disallow | /mymarkets |
| Disallow | /mymarkets/* |
| Disallow | /Mymarkets |
| Disallow | /Mymarkets/* |
| Disallow | /rating/ |
| Disallow | /Rating/ |
| Disallow | /news/preview/ |
| Disallow | /News/Preview/ |
| Disallow | /4442842/ |
| Disallow | /sync-domains |
Other Records
| Field | Value |
|---|---|
| sitemap | https://markets.businessinsider.com/sitemap/sitemap.xml |
| sitemap | https://markets.businessinsider.com/sitemap/news_sitemap.xml |
Comments