library.vanderbilt.edu
robots.txt

Robots Exclusion Standard data for library.vanderbilt.edu

Resource Scan

Scan Details

Site Domain library.vanderbilt.edu
Base Domain vanderbilt.edu
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-09-09T01:52:35+00:00
Next Scan 2024-12-08T01:52:35+00:00

Last Successful Scan

Scanned2023-07-25T00:46:19+00:00
URL https://library.vanderbilt.edu/robots.txt
Redirect https://www.library.vanderbilt.edu/robots.txt
Redirect Domain www.library.vanderbilt.edu
Redirect Base vanderbilt.edu
Domain IPs 129.59.86.107
Redirect IPs 129.59.86.107
Response IP 129.59.86.107
Found Yes
Hash b8c9d1489a5376b321a96154b416cdb825dd47c20fbc0caa4b2df686c8c3d8e0
SimHash 9d058d06471a

Groups

archive.org_bot

Rule Path
Disallow /_archive/
Disallow /test/
Disallow /test.html
Disallow /test.php

*

Rule Path
Disallow /_archive/
Disallow /_resources/
Disallow /test/
Disallow /restricted/
Disallow /test.html
Disallow /test.php