computerweekly.com
robots.txt
Robots Exclusion Standard data for computerweekly.com
Resource Scan
Scan Details
| Site Domain | computerweekly.com |
| Base Domain | computerweekly.com |
| Scan Status | Ok |
| Last Scan | 2024-09-23T21:05:42+00:00 |
| Next Scan | 2024-09-30T21:05:42+00:00 |
Last Scan
| Scanned | 2024-09-23T21:05:42+00:00 |
| URL | https://computerweekly.com/robots.txt |
| Redirect | https://www.computerweekly.com/robots.txt |
| Redirect Domain | www.computerweekly.com |
| Redirect Base | computerweekly.com |
| Domain IPs | 104.18.30.192, 104.18.31.192, 2606:4700::6812:1ec0, 2606:4700::6812:1fc0 |
| Redirect IPs | 104.18.30.192, 104.18.31.192, 2606:4700::6812:1ec0, 2606:4700::6812:1fc0 |
| Response IP | 104.18.30.192 |
| Found | Yes |
| Hash | 84ca8b62056ef4290c7d77e5d3e3b9bd00843c4ef328135c4eecaad7689fe167 |
| SimHash | 4806dd0051b4 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /search/query?q=* |
| Disallow | /search/query |
| Disallow | /es/busqueda/pregunta?q=* |
| Disallow | /es/busqueda/pregunta |
| Disallow | /pt/buscar/pergunta?q=* |
| Disallow | /pt/buscar/pergunta |
| Disallow | /de/suche/frage?q=* |
| Disallow | /de/suche/frage |
bingbot
| Rule | Path |
|---|---|
| Disallow | /search/query?q=* |
| Disallow | /search/query |
| Disallow | /*?s=* |
| Disallow | /es/busqueda/pregunta?q=* |
| Disallow | /es/busqueda/pregunta |
| Disallow | /pt/buscar/pergunta?q=* |
| Disallow | /pt/buscar/pergunta |
| Disallow | /de/suche/frage?q=* |
| Disallow | /de/suche/frage |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 3 |
msnbot
| Rule | Path |
|---|---|
| Disallow | /search/query?q=* |
| Disallow | /search/query |
| Disallow | /*?s= |
| Disallow | /es/busqueda/pregunta?q=* |
| Disallow | /es/busqueda/pregunta |
| Disallow | /pt/buscar/pergunta?q=* |
| Disallow | /pt/buscar/pergunta |
| Disallow | /de/suche/frage?q=* |
| Disallow | /de/suche/frage |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 3 |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.computerweekly.com/siteIndex.xml |
| sitemap | https://www.computerweekly.com/googleNews.xml |
| sitemap | https://www.computerweekly.com/siteIndexVideo.xml |
| sitemap | https://www.computerweekly.com/de/siteIndex.xml |
| sitemap | https://www.computerweekly.com/de/googleNews.xml |
| sitemap | https://www.computerweekly.com/de/siteIndexVideo.xml |
| sitemap | https://www.computerweekly.com/br/siteIndex.xml |
| sitemap | https://www.computerweekly.com/br/googleNews.xml |
| sitemap | https://www.computerweekly.com/br/siteIndexVideo.xml |
| sitemap | https://www.computerweekly.com/es/siteIndex.xml |
| sitemap | https://www.computerweekly.com/es/googleNews.xml |
| sitemap | https://www.computerweekly.com/es/siteIndexVideo.xml |
| sitemap | https://www.computerweekly.com/microscope/siteIndex.xml |
| sitemap | https://www.computerweekly.com/microscope/googleNews.xml |
| sitemap | https://www.computerweekly.com/microscope/siteIndexVideo.xml |
Warnings
- 2 invalid lines.