Log In Sign Up

www.biokids.umich.edu
robots.txt

Robots Exclusion Standard data for www.biokids.umich.edu

Resource Scan

Scan Details

Site Domain www.biokids.umich.edu
Base Domain umich.edu
Scan Status Ok
Last Scan2025-11-12T13:16:12+00:00
Next Scan 2025-12-12T13:16:12+00:00

Last Scan

Scanned2025-11-12T13:16:12+00:00
URL https://www.biokids.umich.edu/robots.txt
Domain IPs 141.211.76.21
Response IP 141.211.76.21
Found Yes
Hash b42f95784809b77b87d2fa3b4b0ea0d6af4f9c2bc85f032e5ff7773bbfd553cc
SimHash 05a8c670213b

Groups

*

Rule Path
Disallow /mousetrap/
Disallow /skunkworks/
Disallow /mousetrap/courses/
Disallow /mousetrap2k2/
Disallow /workspaces/
Disallow /local/
Disallow /quaardvark/search/
Disallow /quaardvark/user/register/
Disallow /quaardvark/user/login/

*

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 8

yandex

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 16
Back to top

Comments

  • Disallow: /site/accounts/classification/
  • Disallow: /resources/*.jpg
  • Disallow: /media/*/*.jpg
  • Allow: /media/*/*_th.jpg
  • Allow: */thumbnail.jpg
  • Allow: */badge.jpg
  • Allow: */small.jpg
Back to top