computerweekly.com
robots.txt
Robots Exclusion Standard data for computerweekly.com
Resource Scan
Scan Details
| Site Domain | computerweekly.com |
| Base Domain | computerweekly.com |
| Scan Status | Ok |
| Last Scan | 2024-11-11T21:07:10+00:00 |
| Next Scan | 2024-11-18T21:07:10+00:00 |
Last Scan
| Scanned | 2024-11-11T21:07:10+00:00 |
| URL | https://computerweekly.com/robots.txt |
| Redirect | https://www.computerweekly.com/robots.txt |
| Redirect Domain | www.computerweekly.com |
| Redirect Base | computerweekly.com |
| Domain IPs | 104.18.30.192, 104.18.31.192, 2606:4700::6812:1ec0, 2606:4700::6812:1fc0 |
| Redirect IPs | 104.18.30.192, 104.18.31.192, 2606:4700::6812:1ec0, 2606:4700::6812:1fc0 |
| Response IP | 104.18.30.192 |
| Found | Yes |
| Hash | b099420dcc6c63c3bbc7bf6f5d037074bbfd710541b1bcd79c6318c6d39bcd7d |
| SimHash | 4807dd0050b8 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /search/query?q=* |
| Disallow | /search/query |
| Disallow | /login |
| Disallow | /register |
| Disallow | /es/busqueda/pregunta?q=* |
| Disallow | /es/busqueda/pregunta |
| Disallow | /pt/buscar/pergunta?q=* |
| Disallow | /pt/buscar/pergunta |
| Disallow | /de/suche/frage?q=* |
| Disallow | /de/suche/frage |
bingbot
| Rule | Path |
|---|---|
| Disallow | /search/query?q=* |
| Disallow | /search/query |
| Disallow | /*?s=* |
| Disallow | /es/busqueda/pregunta?q=* |
| Disallow | /es/busqueda/pregunta |
| Disallow | /pt/buscar/pergunta?q=* |
| Disallow | /pt/buscar/pergunta |
| Disallow | /de/suche/frage?q=* |
| Disallow | /de/suche/frage |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 3 |
msnbot
| Rule | Path |
|---|---|
| Disallow | /search/query?q=* |
| Disallow | /search/query |
| Disallow | /*?s= |
| Disallow | /es/busqueda/pregunta?q=* |
| Disallow | /es/busqueda/pregunta |
| Disallow | /pt/buscar/pergunta?q=* |
| Disallow | /pt/buscar/pergunta |
| Disallow | /de/suche/frage?q=* |
| Disallow | /de/suche/frage |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 3 |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.computerweekly.com/siteIndex.xml |
| sitemap | https://www.computerweekly.com/googleNews.xml |
| sitemap | https://www.computerweekly.com/siteIndexVideo.xml |
| sitemap | https://www.computerweekly.com/de/siteIndex.xml |
| sitemap | https://www.computerweekly.com/de/googleNews.xml |
| sitemap | https://www.computerweekly.com/de/siteIndexVideo.xml |
| sitemap | https://www.computerweekly.com/br/siteIndex.xml |
| sitemap | https://www.computerweekly.com/br/googleNews.xml |
| sitemap | https://www.computerweekly.com/br/siteIndexVideo.xml |
| sitemap | https://www.computerweekly.com/es/siteIndex.xml |
| sitemap | https://www.computerweekly.com/es/googleNews.xml |
| sitemap | https://www.computerweekly.com/es/siteIndexVideo.xml |
| sitemap | https://www.computerweekly.com/microscope/siteIndex.xml |
| sitemap | https://www.computerweekly.com/microscope/googleNews.xml |
| sitemap | https://www.computerweekly.com/microscope/siteIndexVideo.xml |
Warnings
- 2 invalid lines.