municipal.codes
robots.txt
Robots Exclusion Standard data for municipal.codes
Resource Scan
Scan Details
| Site Domain | municipal.codes |
| Base Domain | municipal.codes |
| Scan Status | Ok |
| Last Scan | 2024-05-10T08:10:33+00:00 |
| Next Scan | 2024-06-09T08:10:33+00:00 |
Last Scan
| Scanned | 2024-05-10T08:10:33+00:00 |
| URL | https://municipal.codes/robots.txt |
| Domain IPs | 65.9.112.113, 65.9.112.122, 65.9.112.123, 65.9.112.84 |
| Response IP | 18.165.171.69 |
| Found | Yes |
| Hash | ac229c94e893842148908ebc692fd6ff9f26199593f8fa2452dc30369bc20d3d |
| SimHash | a414bb7cc1fb |
Groups
*
| Rule | Path |
|---|---|
| Allow | /*/media/*.pdf$ |
| Allow | /modernize |
| Disallow | /*.html$ |
| Disallow | /*.rtf$ |
| Disallow | /*.pdf$ |
| Disallow | /*.txt$ |
| Disallow | /*/citations |
| Disallow | /*/compare |
| Disallow | /*/preview |
| Disallow | /*/search |
| Disallow | /*/share |
| Disallow | /*/similar |
| Disallow | /*%40 |
| Disallow | /search |
| Disallow | /selections |
| Disallow | /settings |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 5 |