huntsvilleal.gov
robots.txt
Robots Exclusion Standard data for huntsvilleal.gov
Resource Scan
Scan Details
| Site Domain | huntsvilleal.gov |
| Base Domain | huntsvilleal.gov |
| Scan Status | Ok |
| Last Scan | 2024-10-24T10:13:39+00:00 |
| Next Scan | 2024-11-23T10:13:39+00:00 |
Last Scan
| Scanned | 2024-10-24T10:13:39+00:00 |
| URL | https://huntsvilleal.gov/robots.txt |
| Redirect | http://www.huntsvilleal.gov/robots.txt |
| Redirect Domain | www.huntsvilleal.gov |
| Redirect Base | huntsvilleal.gov |
| Domain IPs | 141.193.213.10, 141.193.213.11 |
| Redirect IPs | 141.193.213.10, 141.193.213.11 |
| Response IP | 141.193.213.10 |
| Found | Yes |
| Hash | 08ea3a69b90f7db46bf8f77d074779becd03231afbe77dd570b5d1dbd1656dde |
| SimHash | 1159ea00cfb1 |
Groups
*
| Rule | Path | Comment |
|---|---|---|
| Disallow | /wp-admin/ | - |
| Allow | /wp-admin/admin-ajax.php | - |
| Disallow | *.pdf$ | Block pdf files from all bots. Albeit non-standard, it works for major search engines. |