tomshw.it
robots.txt
Robots Exclusion Standard data for tomshw.it
Resource Scan
Scan Details
| Site Domain | tomshw.it |
| Base Domain | tomshw.it |
| Scan Status | Ok |
| Last Scan | 2024-09-18T11:15:26+00:00 |
| Next Scan | 2024-09-25T11:15:26+00:00 |
Last Scan
| Scanned | 2024-09-18T11:15:26+00:00 |
| URL | https://tomshw.it/robots.txt |
| Redirect | https://www.tomshw.it/robots.txt |
| Redirect Domain | www.tomshw.it |
| Redirect Base | tomshw.it |
| Domain IPs | 104.26.8.70, 104.26.9.70, 172.67.71.144, 2606:4700:20::681a:846, 2606:4700:20::681a:946, 2606:4700:20::ac43:4790 |
| Redirect IPs | 104.26.8.70, 104.26.9.70, 172.67.71.144, 2606:4700:20::681a:846, 2606:4700:20::681a:946, 2606:4700:20::ac43:4790 |
| Response IP | 104.26.9.70 |
| Found | Yes |
| Hash | 5c35db8e7dc2cc1406f36e7e50f1e689cef52281d4bf99ff7ddbbbc54922e9f0 |
| SimHash | 0d5d77de4f73 |
Groups
acunetix
chinaclaw
dotbot
fhscan
mj12bot
mauibot
npbot*
npbot-1/2.0
teleport
magpie-crawler
piplbot
exabot
| Rule | Path |
|---|---|
| Disallow | / |
*
googlebot
adsbot-google
mediapartners-google
| Rule | Path |
|---|---|
| Disallow | /cms |
| Disallow | /nova-api |
| Disallow | /tag |
| Disallow | /brand |
| Disallow | /brand_tag |
| Disallow | /product |
| Disallow | /gallery |
| Disallow | /*?*keyword= |
| Disallow | /*?*min_rating= |
| Disallow | /*?*platform= |
| Disallow | /*?*genre= |
| Disallow | /*?*year= |
| Disallow | /*?*s= |
| Disallow | /*?*page= |
| Disallow | /*?vertical= |
| Disallow | /notizie-hardware |
| Disallow | /notizie-videogioco |
| Disallow | /notizie-smartphone |
| Disallow | /notizie-culturapop |
| Disallow | /notizie-automotive |
| Disallow | /notizie-business |
| Disallow | /notizie-altro |
| Disallow | /img_vedi.php |
| Disallow | /forum |
| Disallow | /ricerca |
| Disallow | /codici-sconto |
| Disallow | /cont |
| Disallow | /notizie-video |
| Disallow | /tipo-prodotto |
| Disallow | /software.php |
| Disallow | /network.php |
| Disallow | /amp_validated_url |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.tomshw.it/sitemap.xml |
| sitemap | https://www.tomshw.it/google-news-sitemap.xml |
Comments