Log In Sign Up

duo.uio.no
robots.txt

Robots Exclusion Standard data for duo.uio.no

Resource Scan

Scan Details

Site Domain duo.uio.no
Base Domain uio.no
Scan Status Ok
Last Scan5/19/2025, 5:29:04 PM
Next Scan 5/26/2025, 5:29:04 PM

Last Scan

Scanned5/19/2025, 5:29:04 PM
URL https://duo.uio.no/robots.txt
Redirect https://www.duo.uio.no/robots.txt
Redirect Domain www.duo.uio.no
Redirect Base uio.no
Domain IPs 129.240.118.41
Redirect IPs 129.240.118.41
Response IP 129.240.118.41
Found Yes
Hash 3d5e1a9664eddc2cfa4034ff227d030f7ad6b5d3d95471d4cb34a9c53f28707f
SimHash 783d0db5655c

Groups

*

Rule Path
Disallow /discover
Disallow /search-filter
Back to top

Comments

  • Uncomment the following line ONLY if sitemaps.org or HTML sitemaps are used
  • and you have verified that your site is being indexed correctly.
  • Disallow: /browse
  • You also may wish to disallow access to the following paths, in order
  • to stop web spiders from accessing user-based content:
  • Disallow: /advanced-search
  • Disallow: /contact
  • Disallow: /feedback
  • Disallow: /forgot
  • Disallow: /login
  • Disallow: /register
  • Disallow: /search
Back to top