oswego.edu
robots.txt

Robots Exclusion Standard data for oswego.edu

Resource Scan

Scan Details

Site Domain oswego.edu
Base Domain oswego.edu
Scan Status Ok
Last Scan2024-09-21T19:22:20+00:00
Next Scan 2024-10-21T19:22:20+00:00

Last Scan

Scanned2024-09-21T19:22:20+00:00
URL https://oswego.edu/robots.txt
Domain IPs 151.101.1.209, 151.101.129.209, 151.101.193.209, 151.101.65.209
Response IP 151.101.193.209
Found Yes
Hash a9f9411268dfb966608ba82cf1b56ec9de59321aeba40085992e5cec95c0abeb
SimHash b52e72704471

Groups

*

Rule Path
Disallow /cgi-bin/
Disallow /faculty-assembly/sites/www.oswego.edu.faculty-assembly/*
Disallow /library/dept/
Disallow /library/library2/
Disallow /library_department/
Disallow /~finaid/
Disallow /~sturr
Disallow /~*
Disallow /~hci
Disallow /~economic/
Disallow /zz_dev/
Disallow /other_campus/
Disallow /giving/shineman_gift.html
Disallow /Documents/*
Disallow /news/search/*
Disallow /news/story/university-police-welcomes-newest-officer-department-remains-committed-helping-campus

semrushbot

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

semanticscholarbot

Rule Path
Disallow /

ut-dorkbot

Rule Path
Disallow /

Comments

  • Keeps all robots from visiting the following directories
  • Previously would only block Google (User-agent: Googlebot).