Log In Sign Up

www.wiwi.uni-muenster.de
robots.txt

Robots Exclusion Standard data for www.wiwi.uni-muenster.de

Resource Scan

Scan Details

Site Domain www.wiwi.uni-muenster.de
Base Domain uni-muenster.de
Scan Status Ok
Last Scan2025-08-16T07:35:38+00:00
Next Scan 2025-09-15T07:35:38+00:00

Last Scan

Scanned2025-08-16T07:35:38+00:00
URL https://www.wiwi.uni-muenster.de/robots.txt
Domain IPs 128.176.86.150
Response IP 128.176.86.150
Found Yes
Hash af304ab7b753d40b56eb493c83eeaf4c0aeddb6e5f8ccca7fa681fa4ebd45361
SimHash 39c25142a354

Groups

*

Rule Path
Allow /*/misc/*.css$
Allow /*/misc/*.css?
Allow /*/misc/*.js$
Allow /*/misc/*.js?
Allow /*/misc/*.gif
Allow /*/misc/*.jpg
Allow /*/misc/*.jpeg
Allow /*/misc/*.png
Allow /*/modules/*.css$
Allow /*/modules/*.css?
Allow /*/modules/*.js$
Allow /*/modules/*.js?
Allow /*/modules/*.gif
Allow /*/modules/*.jpg
Allow /*/modules/*.jpeg
Allow /*/modules/*.png
Disallow /*/includes/
Disallow /*/misc/
Disallow /*/modules/
Disallow /*/profiles/
Disallow /*/scripts/
Disallow /*/themes/
Disallow /*/cron.php
Disallow /*/install.php
Disallow /*/update.php
Disallow /*/xmlrpc.php
Disallow /*/admin/
Disallow /*/comment/reply/
Disallow /*/filter/tips/
Disallow /*/node/add/
Disallow /*/search/
Disallow /*/user
Disallow /*/user/
Disallow /*/user/login?destination=*
Disallow /*/user/register?destination=*
Disallow /*/user?destination=*
Disallow /*/de/admin/
Disallow /*/de/comment/reply/
Disallow /*/de/filter/tips/
Disallow /*/de/node/add/
Disallow /*/de/search/
Disallow /*/de/user
Disallow /*/de/user/
Disallow /*/de/user/login?destination=*
Disallow /*/de/user/register?destination=*
Disallow /*/de/user?destination=*
Disallow /*/en/admin/
Disallow /*/en/comment/reply/
Disallow /*/en/filter/tips/
Disallow /*/en/node/add/
Disallow /*/en/search/
Disallow /*/en/user
Disallow /*/en/user/
Disallow /*/en/user/login?destination=*
Disallow /*/en/user/register?destination=*
Disallow /*/en/user?destination=*
Disallow /*/?q=admin%2F
Disallow /*/?q=comment%2Freply%2F
Disallow /*/?q=filter%2Ftips%2F
Disallow /*/?q=node%2Fadd%2F
Disallow /*/?q=user
Disallow /*/?q=user%2F
Disallow /*/?q=search%2F
Disallow /wiwi/
Disallow /intranet/
Disallow /at/
Disallow /fakultaet/de/anmeldung-edw
Disallow /fakultaet/de/teilnahme-edw
Disallow /fakultaet/en/node/
Disallow /fakultaet/en/news
Disallow /fakultaet/en/news/
Disallow /fakultaet/en/termine
Disallow /fakultaet/en/termine/
Disallow /phpmyadmin/
Disallow /orsee/
Disallow /exam_time/

Other Records

Field Value
crawl-delay 10
Back to top

Comments

  • robots.txt
  • This file is to prevent the crawling and indexing of certain parts
  • of your site by web crawlers and spiders run by sites like Yahoo!
  • and Google. By telling these "robots" where not to go on your site,
  • you save bandwidth and server resources.
  • CSS, JS, Images
  • Drupal-Directories
  • Drupal-Files
  • Drupal-Paths (clean URLs)
  • Drupal-Paths (no clean URLs)
  • Specific Drupal-Site-Paths
  • Other paths
Back to top