Log In Sign Up

dbai.tuwien.ac.at
robots.txt

Robots Exclusion Standard data for dbai.tuwien.ac.at

Resource Scan

Scan Details

Site Domain dbai.tuwien.ac.at
Base Domain tuwien.ac.at
Scan Status Ok
Last Scan2025-05-25T05:59:56+00:00
Next Scan 2025-06-24T05:59:56+00:00

Last Scan

Scanned2025-05-25T05:59:56+00:00
URL https://dbai.tuwien.ac.at/robots.txt
Domain IPs 128.131.111.3
Response IP 128.131.111.3
Found Yes
Hash 08eaa8ac9dbd0f41d52b5f54087119b1c88cb25d81db141c7c40e265b8289daf
SimHash 38112d6bcf51

Groups

*

Rule Path
Disallow /cgi-bin/
Disallow /bb/
Disallow /staff/gottlob/papers/
Back to top

Comments

  • Robots Exclusion file - this is used to disallow access
  • to webwalkers conform to the defacto standard
  • <URL:http://web.nexor.co.uk/mak/doc/robots/norobots.html>.
  • webmaster@dbai.tuwien.ac.at
  • Format is:
  • User-agent: <name of spider>
  • Disallow: <nothing> | <path>
Back to top