houstontx.gov
robots.txt
Robots Exclusion Standard data for houstontx.gov
Resource Scan
Scan Details
Site Domain | houstontx.gov |
Base Domain | houstontx.gov |
Scan Status | Ok |
Last Scan | 2024-09-15T08:31:33+00:00 |
Next Scan | 2024-10-15T08:31:33+00:00 |
Last Scan
Scanned | 2024-09-15T08:31:33+00:00 |
URL | https://houstontx.gov/robots.txt |
Domain IPs | 204.235.229.46 |
Response IP | 204.235.229.46 |
Found | Yes |
Hash | e6f0b22e6a1c9994213d1ac7912682ca1a1e176b9e274d87ed3aae2aad958f1d |
SimHash | 900abec24323 |
Groups
*
Rule | Path |
---|---|
Disallow | /awmData-mainmenu |
Disallow | /airportproto |
Disallow | /images |
Disallow | /img |
Disallow | /iwanto |
Disallow | /redirect |
Disallow | /admin |
Disallow | /housing/beta/ |
Disallow | /housing/compliance/pages/ |
Disallow | /housing/archives/ |
Disallow | /housing/Templates/ |
Disallow | /old/ |
Disallow | /*/old/ |
Disallow | /*/*/old/ |