darwinawards.com
robots.txt
Robots Exclusion Standard data for darwinawards.com
Resource Scan
Scan Details
| Site Domain | darwinawards.com |
| Base Domain | darwinawards.com |
| Scan Status | Ok |
| Last Scan | 2024-09-20T19:31:20+00:00 |
| Next Scan | 2024-09-27T19:31:20+00:00 |
Last Scan
| Scanned | 2024-09-20T19:31:20+00:00 |
| URL | https://darwinawards.com/robots.txt |
| Domain IPs | 64.13.139.227 |
| Response IP | 64.13.139.227 |
| Found | Yes |
| Hash | 0e0c67d4b93fc26d4a365f9f5e0763b62865768190a4c163e27405d2e347b10f |
| SimHash | 134b8fc69558 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /*/nav* |
| Disallow | /cgi-share/* |
| Disallow | /cgi/* |
| Disallow | /deutsch/* |
| Disallow | /error/* |
| Disallow | /espanol/* |
| Disallow | /francais/* |
| Disallow | /ikon/* |
| Disallow | /media/* |
| Disallow | /misc/* |
| Disallow | /nav* |
| Disallow | /old/REJECT* |
| Disallow | /old/REMOVE* |
| Disallow | /old/DELETE* |
| Disallow | /russia/* |
| Disallow | /reject/* |
| Allow | /slush/* |
| Disallow | /slush/201701/* |
| Disallow | /slush/201702/* |
| Disallow | /slush/201703/* |
| Disallow | /slush/201704/* |
| Disallow | /slush/201705/* |
| Disallow | /slush/201706/* |
| Disallow | /slush/201707/* |
| Disallow | /slush/201708/* |
| Disallow | /slush/201709/* |
| Disallow | /slush/201710/* |
| Disallow | /slush/201711/* |
| Disallow | /slush/201712/* |
| Disallow | /slush/201801/* |
| Disallow | /slush/201802/* |
| Disallow | /slush/201803/* |
| Disallow | /slush/201804/* |
| Disallow | /slush/201805/* |
| Disallow | /slush/201806/* |
| Disallow | /slush/201807/* |
| Disallow | /slush/201808/* |
| Disallow | /slush/201809/* |
| Disallow | /slush/201810/* |
| Disallow | /slush/201811/* |
| Disallow | /slush/201812/* |
| Disallow | /slush/201901/ |
Comments