506infantry.org
robots.txt
Robots Exclusion Standard data for 506infantry.org
Resource Scan
Scan Details
| Site Domain | 506infantry.org |
| Base Domain | 506infantry.org |
| Scan Status | Ok |
| Last Scan | 2025-03-07T19:56:34+00:00 |
| Next Scan | 2025-04-06T19:56:34+00:00 |
Last Scan
| Scanned | 2025-03-07T19:56:34+00:00 |
| URL | https://506infantry.org/robots.txt |
| Domain IPs | 34.226.77.200 |
| Response IP | 34.226.77.200 |
| Found | Yes |
| Hash | 66ab9c4b660981ee8f4c93200050c3f5fa6561902dfda30b5743405734ca969d |
| SimHash | d0ad8c34e736 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /Captcha/ |
| Disallow | /Admin/ |
| Disallow | /Content/Pictures/PageThumb.ashx |
| Disallow | /Content/Pictures/tempUserPicture.ashx |
| Disallow | /Common/Editor/ExternalLink/ |
| Disallow | /Content/Members/MemberDirectoryWebService.asmx |
| Disallow | /Content/Members/SubscriptionWebService.asmx |
| Disallow | /Sys/MemberDirectory/LoadMembers |
| Disallow | /ScriptResource.axd |
| Disallow | /WebResource.axd |
| Disallow | /CustomUrlHandler.ashx |
| Disallow | /*CheckHttpsAuthentication |
| Disallow | /Sys/FinDocument/ |
| Disallow | /widget/Sys/FinDocument/ |
| Disallow | /Sys/Login |
| Disallow | /Sys/Search |
| Disallow | /Sys/Token/Process |
| Disallow | /Sys/Api/* |
| Disallow | /*%26SelectedDate%3D* |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 10 |
Warnings
- `request-rate` is not a known field.