davesweb.com
robots.txt
Robots Exclusion Standard data for davesweb.com
Resource Scan
Scan Details
| Site Domain | davesweb.com |
| Base Domain | davesweb.com |
| Scan Status | Failed |
| Failure Reason | Scan timed out. |
| Last Scan | 2025-09-16T06:51:27+00:00 |
| Next Scan | 2025-09-17T06:51:27+00:00 |
Last Successful Scan
| Scanned | 2025-09-09T03:55:26+00:00 |
| URL | http://davesweb.com/robots.txt |
| Domain IPs | 64.22.138.6 |
| Response IP | 64.22.138.6 |
| Found | Yes |
| Hash | fe7dfaed434c4ab845d20e9ad74d0ae7bfd897860d1f3af5fe119fd5f807059d |
| SimHash | 3a00d1216380 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /bin/ |
| Disallow | /global.asax |
| Disallow | /web.config |
| Disallow | /App_Code/ |
| Disallow | /App_Data/ |
| Disallow | /admin.php |
Warnings
- 11 invalid lines.
- `/* crawlers please note` is not a known field.