dwerden.com
robots.txt
Robots Exclusion Standard data for dwerden.com
Resource Scan
Scan Details
| Site Domain | dwerden.com |
| Base Domain | dwerden.com |
| Scan Status | Failed |
| Failure Stage | Fetching resource. |
| Failure Reason | Server returned a client error. |
| Last Scan | 2024-11-14T17:17:52+00:00 |
| Next Scan | 2025-02-12T17:17:52+00:00 |
Last Successful Scan
| Scanned | 2024-04-19T17:00:42+00:00 |
| URL | https://dwerden.com/robots.txt |
| Domain IPs | 20.10.212.212 |
| Response IP | 20.10.212.212 |
| Found | Yes |
| Hash | 32b345aa5572ee482e2e7fbb472c8b853ddcb401ad31d2007cf323ca3212acd8 |
| SimHash | 2c42c346c8f3 |
Groups
*
No rules defined. All paths allowed.
Other Records
| Field | Value |
|---|---|
| crawl-delay | 1 |
*
| Rule | Path |
|---|---|
| Disallow | /detail.cfm |
| Disallow | /board |
| Disallow | /humor.asp |
| Disallow | /humor2.asp |
| Disallow | /rd/ |
| Disallow | /scifi/ |
| Disallow | /Forums/ |
| Disallow | /forum3/ |
| Disallow | /Special/ |
| Disallow | /talk/forum/cfc/ |
| Disallow | /talk/forum/include/ |
| Disallow | /talk/forum/modules/ |
| Disallow | /talk/forum/portal/ |
| Disallow | /talk/forum/i/ |
| Disallow | /talk/forum/messageview.cfm |
| Disallow | /wedding/ |
| Disallow | /_borders/ |
| Disallow | /_derived/ |
| Disallow | /_fpclass/ |
| Disallow | /_overlay/ |
| Disallow | /_private/ |
| Disallow | /_themes/ |
| Disallow | /_vti_bin/ |
| Disallow | /_vti_cnf/ |
| Disallow | /_vti_log/ |
| Disallow | /_vti_map/ |
| Disallow | /_vti_pvt/ |
| Disallow | /_vti_txt/ |
| Disallow | *.lck |
| Disallow | /2004/ |
| Disallow | /allsaints/ |
| Disallow | /AllSaints2005b/ |
| Disallow | /aspnet_client/ |
| Disallow | *.LOG |
| Disallow | *.dwt |
| Disallow | *.mno |
| Disallow | *.psp |
| Disallow | *.tmp |
| Disallow | *.inc |
| Disallow | *.bak |
| Disallow | /Cards/final/ |
| Disallow | /Cimarron/ |
| Disallow | /ColdFusion/_vti_cnf/ |
| Disallow | /Source/ |
| Disallow | /blog3old/ |
| Disallow | /newDWP/ |
| Disallow | /symphoniamembers/ |
| Disallow | /Poll/ |
| Disallow | /cfug2 |
| Disallow | /emg/musicadd.cfm |
| Disallow | /emg/musicpublisheredit.cfm |
| Disallow | /emg/recordingadd.cfm |
| Disallow | /emg/recordingedit.cfm |
| Disallow | /EMG3/ |
| Disallow | /siteobjects/ |
| Disallow | /Resources/*.cfm |
| Disallow | /Resources/*.htm |
| Disallow | /music-videos/browse-brass-ensemble-videos.cfm |
| Disallow | /music-videos/browse-euphonium-videos.cfm |
| Disallow | /music-videos/browse-horn-videos.cfm |
| Disallow | /music-videos/browse-trumpet-videos.cfm |
| Disallow | /music-videos/browse-tuba-videos.cfm |
| Disallow | /music-videos/euphonium-music-vidmusic-videos.cfm |
| Disallow | /forum/member.php |
| Disallow | /forum/attachment.php |
| Disallow | /emg/ColdFusion/ |