hathitrust.org
robots.txt
Robots Exclusion Standard data for hathitrust.org
Resource Scan
Scan Details
| Site Domain | hathitrust.org |
| Base Domain | hathitrust.org |
| Scan Status | Failed |
| Failure Stage | Fetching resource. |
| Failure Reason | Server returned a client error. |
| Last Scan | 2025-09-06T05:28:13+00:00 |
| Next Scan | 2025-12-05T05:28:13+00:00 |
Last Successful Scan
| Scanned | 2025-04-17T05:12:25+00:00 |
| URL | https://hathitrust.org/robots.txt |
| Redirect | https://www.hathitrust.org/robots.txt |
| Redirect Domain | www.hathitrust.org |
| Redirect Base | hathitrust.org |
| Domain IPs | 134.68.125.197, 141.213.128.184 |
| Redirect IPs | 162.159.140.37, 172.66.0.37, 2606:4700:7::25, 2a06:98c1:58::25 |
| Response IP | 172.66.0.37 |
| Found | Yes |
| Hash | f0fcf9e8337b4b571ce266bb15acd8b105b4213cfc0c3ba2879a10eb3439c2d9 |
| SimHash | 49010c400993 |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.hathitrust.org/wp-sitemap.xml |