hiv.gov
robots.txt
Robots Exclusion Standard data for hiv.gov
Resource Scan
Scan Details
Site Domain | hiv.gov |
Base Domain | hiv.gov |
Scan Status | Ok |
Last Scan | 2024-04-16T14:26:59+00:00 |
Next Scan | 2024-05-16T14:26:59+00:00 |
Last Scan
Scanned | 2024-04-16T14:26:59+00:00 |
URL | https://hiv.gov/robots.txt |
Redirect | https://www.hiv.gov:443/robots.txt |
Redirect Domain | www.hiv.gov |
Redirect Base | hiv.gov |
Domain IPs | 15.197.141.195, 3.33.151.151 |
Redirect IPs | 108.157.254.108, 108.157.254.114, 108.157.254.126, 108.157.254.85, 2600:9000:2753:2e00:d:49fb:cd40:93a1, 2600:9000:2753:5a00:d:49fb:cd40:93a1, 2600:9000:2753:6e00:d:49fb:cd40:93a1, 2600:9000:2753:8a00:d:49fb:cd40:93a1, 2600:9000:2753:a400:d:49fb:cd40:93a1, 2600:9000:2753:d200:d:49fb:cd40:93a1, 2600:9000:2753:f000:d:49fb:cd40:93a1, 2600:9000:2753:f400:d:49fb:cd40:93a1 |
Response IP | 108.157.254.114 |
Found | Yes |
Hash | a66d7e9b47eab1c079f316865163252b93ac2de2459f420612ff03364d9118ae |
SimHash | 4548dc504731 |
Other Records
Field | Value |
---|---|
sitemap | https://www.hiv.gov/sitemap-index.xml |
Warnings
- `host` is not a known field.