forum.tomshw.it
robots.txt
Robots Exclusion Standard data for forum.tomshw.it
Resource Scan
Scan Details
| Site Domain | forum.tomshw.it |
| Base Domain | tomshw.it |
| Scan Status | Ok |
| Last Scan | 2024-09-02T04:50:48+00:00 |
| Next Scan | 2024-10-02T04:50:48+00:00 |
Last Scan
| Scanned | 2024-09-02T04:50:48+00:00 |
| URL | https://forum.tomshw.it/robots.txt |
| Domain IPs | 104.26.8.70, 104.26.9.70, 172.67.71.144, 2606:4700:20::681a:846, 2606:4700:20::681a:946, 2606:4700:20::ac43:4790 |
| Response IP | 104.26.8.70 |
| Found | Yes |
| Hash | 204cf0b726138ca18813eb0865f477c2cc9e3a2e03aa721404af032a671c16bc |
| SimHash | 453dcce2efd3 |
Groups
acunetix
chinaclaw
dotbot
fhscan
mj12bot
mauibot
npbot*
npbot-1/2.0
teleport
magpie-crawler
piplbot
exabot
| Rule | Path |
|---|---|
| Disallow | / |
*
| Rule | Path |
|---|---|
| Disallow | /whats-new/ |
| Disallow | /account/ |
| Disallow | /attachments/ |
| Disallow | /goto/ |
| Disallow | /help/ |
| Disallow | /posts/ |
| Disallow | /login/ |
| Disallow | /search/ |
| Disallow | /members/ |
| Disallow | /admin.php |
| Disallow | /deferred.php |
| Disallow | /threads/*?view=reaction_score$ |
| Disallow | /labs/ |
| Disallow | /uix/toggle- |
| Disallow | */unread$ |
| Disallow | */latest$ |
| Allow | / |
| Allow | /search/$ |
| Disallow | /search/* |
| Allow | */threads/post-* |
| Disallow | */post-* |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.tomshw.it/forum/sitemap.php |
Comments