crn.com
robots.txt
Robots Exclusion Standard data for crn.com
Resource Scan
Scan Details
| Site Domain | crn.com |
| Base Domain | crn.com |
| Scan Status | Ok |
| Last Scan | 2024-11-12T11:23:51+00:00 |
| Next Scan | 2024-11-19T11:23:51+00:00 |
Last Scan
| Scanned | 2024-11-12T11:23:51+00:00 |
| URL | https://crn.com/robots.txt |
| Redirect | https://www.crn.com/robots.txt |
| Redirect Domain | www.crn.com |
| Redirect Base | crn.com |
| Domain IPs | 104.18.26.190, 104.18.27.190, 2606:4700::6812:1abe, 2606:4700::6812:1bbe |
| Redirect IPs | 104.18.26.190, 104.18.27.190, 2606:4700::6812:1abe, 2606:4700::6812:1bbe |
| Response IP | 104.18.26.190 |
| Found | Yes |
| Hash | da4b2e6b6a7adb5ca8384ee957697fdbef80ab77f26b478712d86d0217e1a4bd |
| SimHash | c118ec210517 |
Groups
*
| Rule | Path |
|---|---|
| Allow | / |
| Disallow | */breakingnews.asp |
| Disallow | */channel-encyclopedia/ |
| Disallow | */channelcommunity/ |
| Disallow | */components/ |
| Disallow | */Components/ |
| Disallow | */contributions/ |
| Disallow | */emailthisarticle.htm |
| Disallow | */encyclopedia/ |
| Disallow | */int/ |
| Disallow | */nl/ |
| Disallow | */print/ |
| Disallow | */printablearticle.htm |
| Disallow | */printableArticle.jhtml |
| Disallow | */printerFriendly.jhtml |
| Disallow | */printmail/ |
| Disallow | */printpdf/ |
| Disallow | */reviews/client-devices/ |
| Disallow | */sections/ |
| Disallow | */Sections/ |
| Disallow | */sponsored/ |
| Disallow | */stock-quotes-financial-data/ |
| Disallow | */tabletablearticle.htm |
| Disallow | */tools/quotes/index.jhtml |
| Disallow | */var/ |
| Disallow | /*.asp |
| Disallow | /*.asp$ |
| Disallow | /*.jhtml |
| Disallow | /*.jhtml$ |
| Disallow | /channel-encyclopedia/ |
| Disallow | /encyclopedia/ |
| Disallow | /search-request.htm |
| Disallow | /slide-shows/channel-programs/240007608/the-top-female-executives-of-the-2012-fast-growth-100.htm |
| Disallow | */tag/ |
| Disallow | /query/related/related |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.crn.com/sitemap-index.xml |
Warnings
- `host` is not a known field.