cnet.fr
robots.txt
Robots Exclusion Standard data for cnet.fr
Resource Scan
Scan Details
| Site Domain | cnet.fr |
| Base Domain | cnet.fr |
| Scan Status | Ok |
| Last Scan | 2024-05-31T22:30:05+00:00 |
| Next Scan | 2024-06-07T22:30:05+00:00 |
Last Scan
| Scanned | 2024-05-31T22:30:05+00:00 |
| URL | https://cnet.fr/robots.txt |
| Redirect | https://www.cnetfrance.fr/robots.txt |
| Redirect Domain | www.cnetfrance.fr |
| Redirect Base | cnetfrance.fr |
| Domain IPs | 34.149.158.179 |
| Redirect IPs | 104.26.12.163, 104.26.13.163, 172.67.70.17, 2606:4700:20::681a:ca3, 2606:4700:20::681a:da3, 2606:4700:20::ac43:4611 |
| Response IP | 104.26.12.163 |
| Found | Yes |
| Hash | 13463f6b96633aa3304941f32194a33b72bfd70674c61c75b1b9ab66220abca1 |
| SimHash | 7908f9c4609e |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /6975/FR/ |
| Disallow | /shopredirect.htm |
| Disallow | /?s= |
| Disallow | /?sort= |
| Disallow | /search |
| Disallow | /newreply |
| Disallow | /printthread |
| Disallow | /sendmessage |
| Disallow | /subscription |
| Disallow | /newthread.php |
| Disallow | /index.php |
| Disallow | /showthread |
| Disallow | /archive/ |
| Disallow | /user/ |
| Disallow | /api/prices.html |
| Disallow | /ajax/api/node/incrementNodeview |
| Disallow | /telecharger/ajax/ |
*
| Rule | Path |
|---|---|
| Disallow | /?s= |
| Disallow | /page/*/?s= |
| Disallow | /search/ |
| Disallow | /wp-json/ |
| Disallow | /?rest_route= |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.cnetfrance.fr/sitemap-news.xml |
| sitemap | https://www.cnetfrance.fr/sitemap_index.xml |
| sitemap | https://www.cnetfrance.fr/sitemap-news.xml |
| sitemap | https://www.cnetfrance.fr/sitemap-contenu-index.xml |
| sitemap | https://www.cnetfrance.fr/sitemap-images.xml |
| sitemap | https://www.cnetfrance.fr/telecharger/telechargements.xml.gz |
| sitemap | https://www.cnetfrance.fr/telecharger/telechargements-inactif.xml.gz |
| sitemap | https://www.cnetfrance.fr/sitemap_index.xml |
Comments