warner.senate.gov
robots.txt
Robots Exclusion Standard data for warner.senate.gov
Resource Scan
Scan Details
| Site Domain | warner.senate.gov |
| Base Domain | senate.gov |
| Scan Status | Ok |
| Last Scan | 2024-11-12T19:36:33+00:00 |
| Next Scan | 2024-12-12T19:36:33+00:00 |
Last Scan
| Scanned | 2024-11-12T19:36:33+00:00 |
| URL | https://warner.senate.gov/robots.txt |
| Redirect | https://www.warner.senate.gov/robots.txt |
| Redirect Domain | www.warner.senate.gov |
| Redirect Base | senate.gov |
| Domain IPs | 23.41.19.178 |
| Redirect IPs | 23.203.72.9, 2600:1413:b000:68f::1fd, 2600:1413:b000:695::1fd |
| Response IP | 23.203.72.9 |
| Found | Yes |
| Hash | a23c89a33d81361bc7224c8269f4f7434225c786bc33e417b9bd1df9a0d950ec |
| SimHash | c81e8e2c1035 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | / |
| Allow | /public/ |
gsa-crawler
| Rule | Path |
|---|---|
| Disallow | / |
| Allow | /public/ |
| Disallow | /*IsLowBandwidth |
| Disallow | /*FuseAction |
| Disallow | /*RSS.Feed |
| Disallow | /*Rss.Feed |
| Disallow | /*start%3D |
| Disallow | /*Group_id%3D |
| Disallow | /*q%3D |
| Disallow | /*print%3D |
| Disallow | /*Print%3D |
| Disallow | /*IsTextOnly%3D |
| Disallow | /*MonthDisplay%3D |
| Disallow | /*YearDisplay%3D |
| Disallow | /*Month%3D |
| Disallow | /*Year%3D |