cnbc.com
robots.txt
Robots Exclusion Standard data for cnbc.com
Resource Scan
Scan Details
| Site Domain | cnbc.com |
| Base Domain | cnbc.com |
| Scan Status | Ok |
| Last Scan | 2024-05-18T14:07:55+00:00 |
| Next Scan | 2024-05-25T14:07:55+00:00 |
Last Scan
| Scanned | 2024-05-18T14:07:55+00:00 |
| URL | https://cnbc.com/robots.txt |
| Redirect | https://www.cnbc.com/robots.txt |
| Redirect Domain | www.cnbc.com |
| Redirect Base | cnbc.com |
| Domain IPs | 50.234.250.31, 50.234.250.32 |
| Redirect IPs | 23.203.79.177 |
| Response IP | 104.103.147.244 |
| Found | Yes |
| Hash | 44b11e04dbf08bc7bba44a953d2cbb19f2ba3e2df1f2fce301293361b5d642dd |
| SimHash | e9871913c104 |
Groups
googlebot
| Rule | Path |
|---|---|
| Disallow | /*native-android-mobile |
| Disallow | /*native-android-tablet |
| Disallow | /*mobile-native |
| Disallow | /preview/ |
| Disallow | /undefined/ |
| Disallow | /proplayer |
| Disallow | /appchart/* |
| Disallow | /search/* |
*
| Rule | Path |
|---|---|
| Disallow | /preview/ |
| Disallow | /undefined/ |
| Disallow | /proplayer |
| Disallow | /appchart/* |
| Disallow | /search/* |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.cnbc.com/sitemapAll.xml |
| sitemap | https://www.cnbc.com/sitemap_news.xml |
| sitemap | https://www.cnbc.com/sitemapvideoAll.xml |
| sitemap | https://www.cnbc.com/SitemapQuotes.xml |
| sitemap | https://www.cnbc.com/sitemapSelectAll.xml |
| sitemap | https://www.cnbc.com/sitemapproAll.xml |
| sitemap | https://www.cnbc.com/sitemapprovideoAll.xml |
| sitemap | https://www.cnbc.com/sitemapinvestingclubAll.xml |
| sitemap | https://www.cnbc.com/sitemapicvideoprodAll.xml |
Comments