gvsu.edu
robots.txt

Robots Exclusion Standard data for gvsu.edu

Resource Scan

Scan Details

Site Domain gvsu.edu
Base Domain gvsu.edu
Scan Status Ok
Last Scan2024-10-20T05:57:40+00:00
Next Scan 2024-11-19T05:57:40+00:00

Last Scan

Scanned2024-10-20T05:57:40+00:00
URL https://gvsu.edu/robots.txt
Redirect https://www.gvsu.edu/robots.txt
Redirect Domain www.gvsu.edu
Redirect Base gvsu.edu
Domain IPs 104.17.87.18, 104.17.88.18, 2606:4700::6811:5712, 2606:4700::6811:5812
Redirect IPs 104.17.87.18, 104.17.88.18, 2606:4700::6811:5712, 2606:4700::6811:5812
Response IP 104.17.87.18
Found Yes
Hash abf32d6218cc14a12f4540fe64aac8daa34727b188246d25a1cbf3888823c8f4
SimHash 4e685c7cebb0

Groups

vegi bot

Rule Path
Disallow /

msnbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

*

Rule Path
Disallow /menu/
Disallow /script/
Disallow /*_old/
Disallow /_*/
Disallow /studentapps/
Disallow /tools/
Disallow /oasis/
Disallow /cmsadmin/
Disallow /cms4/admin/
Disallow /wgvustore/
Disallow /reference_files/
Disallow /facultygov/online/
Disallow /s/jt
Disallow /it/abuse/
Disallow /it/blocked/
Disallow /it/virus/
Disallow /financialaid/files/scholarshipuploads/
Disallow /CFIDE/
Disallow /includes/
Disallow /reserve/

Other Records

Field Value
sitemap https://www.gvsu.edu/sitemap.xml