reviews.cnet.com
robots.txt
Robots Exclusion Standard data for reviews.cnet.com
Resource Scan
Scan Details
Site Domain | reviews.cnet.com |
Base Domain | cnet.com |
Scan Status | Ok |
Last Scan | 2024-05-14T18:12:35+00:00 |
Next Scan | 2024-05-21T18:12:35+00:00 |
Last Scan
Scanned | 2024-05-14T18:12:35+00:00 |
URL | https://reviews.cnet.com/robots.txt |
Redirect | https://www.cnet.com/robots.txt |
Redirect Domain | www.cnet.com |
Redirect Base | cnet.com |
Domain IPs | 34.98.65.244 |
Redirect IPs | 199.232.194.154, 199.232.198.154, 2a04:4e42:4c::666, 2a04:4e42:4d::666 |
Response IP | 146.75.42.154 |
Found | Yes |
Hash | 11176183b18c21924a43a4158b23e62533e46a55d682c0cc275dfe32791b3482 |
SimHash | c807f25ce333 |
Groups
*
Rule | Path |
---|---|
Disallow | */*max-price%3D* |
Disallow | */*min-price%3D* |
Disallow | */Xhr* |
Disallow | */account-information/* |
Disallow | */ad/ad-cookie/* |
Disallow | */ajax/ |
Disallow | */alert/* |
Disallow | */cnet_global_app_redirect.xml |
Disallow | */component/* |
Disallow | */edit/ |
Disallow | */edit/change-email/* |
Disallow | */edit/change-password/* |
Disallow | */email/editor/* |
Disallow | */email/member/* |
Disallow | */email/newsletter/* |
Disallow | */error/* |
Disallow | */ext/make/ingest/* |
Disallow | */feature/* |
Disallow | */finder/*/*/?*filter=*_*_*_*_* |
Disallow | */finder/*/?*filter=*_*_*_*_* |
Disallow | */forums/* |
Disallow | */g00/* |
Disallow | */informacion-de-mi-cuenta/* |
Disallow | */members/* |
Disallow | */products/*?*filter=* |
Disallow | */roadshow/finder/* |
Disallow | */search/* |
Disallow | */user/* |
Disallow | */videos/manifest/* |
Disallow | */videos/secure/* |
Disallow | */videos/selector/* |
Disallow | */wrap-function* |
Disallow | */wrap-xhr* |
Disallow | */xhr* |
Disallow | *?sort=popularity* |
Disallow | *?sort=price* |
Disallow | *?sort=user-rating* |
Disallow | *hub-preview-version%3D* |
Disallow | /.well-known/* |
Disallow | /api/editionurlgenerator/ |
Disallow | /apple-news/* |
Disallow | /au/* |
Disallow | /es/* |
Disallow | /html/story/* |
Disallow | /index.php/* |
Disallow | /products/* |
Disallow | /uk/* |
Other Records
Field | Value |
---|---|
sitemap | https://www.cnet.com/sitemaps/article/index.xml |
sitemap | https://www.cnet.com/sitemaps/best/index.xml |
sitemap | https://www.cnet.com/sitemaps/gallery/index.xml |
sitemap | https://www.cnet.com/sitemaps/how-to/index.xml |
sitemap | https://www.cnet.com/sitemaps/news.xml |
sitemap | https://www.cnet.com/sitemaps/review/index.xml |
sitemap | https://www.cnet.com/sitemaps/static.xml |
sitemap | https://www.cnet.com/sitemaps/updating_story/index.xml |
sitemap | https://www.cnet.com/sitemaps/video/index.xml |
sitemap | https://www.cnet.com/personal-finance/sitemaps/index.xml |
Comments