m.cnbnews.com
robots.txt
Robots Exclusion Standard data for m.cnbnews.com
Resource Scan
Scan Details
| Site Domain | m.cnbnews.com |
| Base Domain | cnbnews.com |
| Scan Status | Ok |
| Last Scan | 2024-09-28T06:18:03+00:00 |
| Next Scan | 2024-10-28T06:18:03+00:00 |
Last Scan
| Scanned | 2024-09-28T06:18:03+00:00 |
| URL | https://m.cnbnews.com/robots.txt |
| Domain IPs | 222.234.220.52 |
| Response IP | 222.234.220.52 |
| Found | Yes |
| Hash | adb86ab0d2f8cc4eb660b1769ed12d8eca3741e63a00916bb67436cda0785ae0 |
| SimHash | ca089a2444d0 |
Groups
*
| Rule | Path |
|---|---|
| Allow | /news/section.html |
| Allow | /news/section_list_all.html |
| Allow | /news/article_list_all.html |
| Disallow | /news/article_list_writer.html |
| Disallow | /news/search_result.html |
| Disallow | /m/m_search_result.html |
| Disallow | /m/m_section.html |
| Disallow | /m/m_section_list_all.html |
| Disallow | /category/ |
| Disallow | /newsdesk2/ |
| Disallow | /lib/ |
| Disallow | /weblog/ |
| Disallow | /mypage/ |
| Disallow | /member/ |
| Disallow | /info/ |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.cnbnews.com/sitemap.xml |
Comments