us.cnn.com
robots.txt
Robots Exclusion Standard data for us.cnn.com
Resource Scan
Scan Details
| Site Domain | us.cnn.com |
| Base Domain | cnn.com |
| Scan Status | Ok |
| Last Scan | 2024-04-27T22:53:26+00:00 |
| Next Scan | 2024-05-04T22:53:26+00:00 |
Last Scan
| Scanned | 2024-04-27T22:53:26+00:00 |
| URL | https://us.cnn.com/robots.txt |
| Domain IPs | 151.101.131.5, 151.101.195.5, 151.101.3.5, 151.101.67.5, 2a04:4e42:200::773, 2a04:4e42:400::773, 2a04:4e42:600::773, 2a04:4e42::773 |
| Response IP | 199.232.47.5 |
| Found | Yes |
| Hash | e2544fca22ce1b8ab12de3ead16def7c9d604a47cf9cfc2d7dc83269ada40a0d |
| SimHash | 2c111b5aa42a |
Groups
anthropic-ai
awariorssbot
awariosmartbot
bytespider
ccbot
chatgpt-user
claudebot
claude-web
cohere-ai
dataforseobot
diffbot
facebookbot
gptbot
google-extended
magpie-crawler
newsnow
news-please
omgili
omgilibot
perplexitybot
scrapy
turnitinbot
| Rule | Path |
|---|---|
| Disallow | / |
*
| Rule | Path |
|---|---|
| Allow | /partners/ipad/live-video.json |
| Disallow | /*.jsx$ |
| Disallow | *.jsx$ |
| Disallow | /*.jsx/ |
| Disallow | *.jsx? |
| Disallow | /ads/ |
| Disallow | /aol/ |
| Disallow | /api/ |
| Disallow | /beta/ |
| Disallow | /browsers/ |
| Disallow | /cl/ |
| Disallow | /cnews/ |
| Disallow | /cnn_adspaces |
| Disallow | /cnnbeta/ |
| Disallow | /cnnintl_adspaces |
| Disallow | /development |
| Disallow | /editionssi |
| Disallow | /help/cnnx.html |
| Disallow | /NewsPass |
| Disallow | /NOKIA |
| Disallow | /partners/ |
| Disallow | /pipeline/ |
| Disallow | /pointroll/ |
| Disallow | /POLLSERVER/ |
| Disallow | /pr/ |
| Disallow | /privacy |
| Disallow | /PV/ |
| Disallow | /Quickcast/ |
| Disallow | /quickcast/ |
| Disallow | /QUICKNEWS/ |
| Disallow | /search |
| Disallow | /terms |
| Disallow | /test/ |
| Disallow | /virtual/ |
| Disallow | /WEB-INF/ |
| Disallow | /web.projects/ |
| Disallow | /webview/ |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.cnn.com/sitemaps/cnn/index.xml |
| sitemap | https://www.cnn.com/sitemaps/cnn/news.xml |
| sitemap | https://www.cnn.com/sitemap/news.xml |
| sitemap | https://www.cnn.com/sitemaps/sitemap-section.xml |
| sitemap | https://www.cnn.com/sitemaps/sitemap-interactive.xml |
| sitemap | https://www.cnn.com/ampstories/sitemap.xml |
| sitemap | https://edition.cnn.com/sitemaps/news.xml |
| sitemap | https://www.cnn.com/sitemap/article/cnn-underscored.xml |
| sitemap | https://www.cnn.com/sitemap/section/cnn-underscored.xml |
| sitemap | https://www.cnn.com/cnn-underscored/money/sitemap.xml |
| sitemap | https://www.cnn.com/cnn-underscored/money/sitemap-news.xml |
| sitemap | https://www.cnn.com/sitemap/section/politics.xml |
| sitemap | https://www.cnn.com/sitemap/article/opinions.xml |
| sitemap | https://www.cnn.com/sitemap/article.xml |
| sitemap | https://www.cnn.com/sitemap/section.xml |
| sitemap | https://www.cnn.com/sitemap/video.xml |
| sitemap | https://www.cnn.com/sitemap/gallery.xml |
| sitemap | https://www.cnn.com/sitemap/markets/stocks.xml |