osrhe.edu
robots.txt

Robots Exclusion Standard data for osrhe.edu

Resource Scan

Scan Details

Site Domain osrhe.edu
Base Domain osrhe.edu
Scan Status Ok
Last Scan2024-09-10T16:22:55+00:00
Next Scan 2024-10-10T16:22:55+00:00

Last Scan

Scanned2024-09-10T16:22:55+00:00
URL http://osrhe.edu/robots.txt
Redirect https://www.okhighered.org/robots.txt
Redirect Domain www.okhighered.org
Redirect Base okhighered.org
Domain IPs 164.58.235.140
Redirect IPs 164.58.235.145
Response IP 164.58.235.145
Found Yes
Hash dbcc8a7f8a5cd041b38d77f8235be4dc75b351f616e0c3ea303b5683ce38f660
SimHash c20cc2a08e93

Groups

baiduspider

Rule Path
Disallow /

baiduspider-video

Rule Path
Disallow /

baiduspider-image

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

semanticscholarbot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

yandex

Rule Path
Disallow /

trendictionbot

Rule Path
Disallow /

*

Rule Path
Allow /wp-content/uploads/
Disallow /wp-content/plugins/
Disallow /wp-admin/
Disallow /current-college-students/complaints/student-complaint-form/
Disallow /osrhe-design-system/
Disallow /okcampuscompact/
Disallow /okcampuscompact/forms/
Disallow /okcampuscompact/members/
Disallow /okcampuscompact/newscenter/
Disallow /okcampuscompact/upcoming-events/
Disallow /okcampuscompact/conf-workshop-archives/
Disallow /okcampuscompact/communicator/
Disallow /6486-2/
Disallow /cc
Disallow /future
Disallow /future/members/
Disallow /future/subcommittees/

Other Records

Field Value
crawl-delay 10

Other Records

Field Value
sitemap https://okhighered.org/sitemap_index.xml

Comments

  • Instructions avaialble here:
  • https://developers.google.com/search/docs/crawling-indexing/robots/create-robots-txt
  • Baiduspider
  • Baiduspider
  • Yandex