sites.psu.edu
robots.txt
Robots Exclusion Standard data for sites.psu.edu
Resource Scan
Scan Details
Site Domain | sites.psu.edu |
Base Domain | psu.edu |
Scan Status | Ok |
Last Scan | 2024-09-14T22:47:01+00:00 |
Next Scan | 2024-10-14T22:47:01+00:00 |
Last Scan
Scanned | 2024-09-14T22:47:01+00:00 |
URL | https://sites.psu.edu/robots.txt |
Domain IPs | 100.24.182.117, 184.72.224.80, 3.91.109.122, 34.199.202.106, 34.227.238.166, 35.172.73.102 |
Response IP | 34.199.202.106 |
Found | Yes |
Hash | 40989b7cb0fd79483d288ebed83c7a997516571aa3c91352a3032148b5138eff |
SimHash | e0c65fc1812b |
Groups
Other Records
Field | Value |
---|---|
sitemap | https://sites.psu.edu/wp-sitemap.xml |
Warnings
- 6 invalid lines.