uaf.edu
robots.txt

Robots Exclusion Standard data for uaf.edu

Resource Scan

Scan Details

Site Domain uaf.edu
Base Domain uaf.edu
Scan Status Failed
Failure ReasonScan timed out.
Last Scan2024-09-21T05:05:35+00:00
Next Scan 2024-09-22T05:05:35+00:00

Last Successful Scan

Scanned2024-08-22T00:58:22+00:00
URL https://uaf.edu/robots.txt
Redirect https://www.uaf.edu/robots.txt
Redirect Domain www.uaf.edu
Redirect Base uaf.edu
Domain IPs 137.229.114.150
Redirect IPs 137.229.114.150
Response IP 137.229.114.150
Found Yes
Hash 511496fe2e35f55d6019f6535a92eee3b3da776f8d3d9a23ab86bc1a621876c8
SimHash 0154f2302f52

Groups

zanrancrawler

Rule Path
Disallow /

*

Rule Path
Disallow /templates/
Disallow /ua-templates/
Disallow /roxen-files/
Disallow /__internal/
Disallow /_internal/
Disallow /__frame/
Disallow /coursefinder/search-results/
Disallow /anla/collections/search/resultDetail.xml
Disallow /hr/