pubs.usgs.gov
robots.txt

Robots Exclusion Standard data for pubs.usgs.gov

Resource Scan

Scan Details

Site Domain pubs.usgs.gov
Base Domain usgs.gov
Scan Status Ok
Last Scan2024-05-22T22:03:52+00:00
Next Scan 2024-06-21T22:03:52+00:00

Last Scan

Scanned2024-05-22T22:03:52+00:00
URL https://pubs.usgs.gov/robots.txt
Domain IPs 13.33.30.123, 13.33.30.15, 13.33.30.23, 13.33.30.67, 2600:9000:229f:2a00:1c:ab8b:bec0:93a1, 2600:9000:229f:5e00:1c:ab8b:bec0:93a1, 2600:9000:229f:7400:1c:ab8b:bec0:93a1, 2600:9000:229f:8e00:1c:ab8b:bec0:93a1, 2600:9000:229f:9200:1c:ab8b:bec0:93a1, 2600:9000:229f:c000:1c:ab8b:bec0:93a1, 2600:9000:229f:dc00:1c:ab8b:bec0:93a1, 2600:9000:229f:f200:1c:ab8b:bec0:93a1
Response IP 13.33.30.15
Found Yes
Hash df13bf0ac4c68eb5196014d6f05a881f68161b7faeedfdd79fe8770767f86836
SimHash e950d1444790

Groups

*

Rule Path
Disallow /archive/