bgsu.edu
robots.txt
Robots Exclusion Standard data for bgsu.edu
Resource Scan
Scan Details
Site Domain | bgsu.edu |
Base Domain | bgsu.edu |
Scan Status | Ok |
Last Scan | 2024-08-31T23:13:27+00:00 |
Next Scan | 2024-09-30T23:13:27+00:00 |
Last Scan
Scanned | 2024-08-31T23:13:27+00:00 |
URL | https://bgsu.edu/robots.txt |
Redirect | https://www.bgsu.edu/robots.txt |
Redirect Domain | www.bgsu.edu |
Redirect Base | bgsu.edu |
Domain IPs | 129.1.14.65 |
Redirect IPs | 3.211.79.20, 44.195.131.142 |
Response IP | 44.195.131.142 |
Found | Yes |
Hash | 87b90ee9ff9580af8e3b1ef1d44f7083587ea235da81772868e91e14754b92b2 |
SimHash | cd9db9c4e5b3 |
Groups
*
Rule | Path |
---|---|
Allow | / |
Disallow | /cms-training/online-cms-training/ |
Disallow | /cms-training/testing-area |
Disallow | /cms-training/example-pages |
Disallow | /content/dam/BGSU/cms-training/training-area |
Disallow | /archived-catalog |
Disallow | /content/dam/BGSU/catalog |
Disallow | /content/dam/BGSU/newcatalog |
Other Records
Field | Value |
---|---|
sitemap | https://www.bgsu.edu/sitemap.xml |
Comments