Log In Sign Up

iphils.uj.edu.pl
robots.txt

Robots Exclusion Standard data for iphils.uj.edu.pl

Resource Scan

Scan Details

Site Domain iphils.uj.edu.pl
Base Domain uj.edu.pl
Scan Status Ok
Last Scan2025-08-17T03:31:34+00:00
Next Scan 2025-09-16T03:31:34+00:00

Last Scan

Scanned2025-08-17T03:31:34+00:00
URL https://iphils.uj.edu.pl/robots.txt
Domain IPs 149.156.163.133
Response IP 149.156.163.133
Found Yes
Hash ae9b5d0fa3a4b59401b3aa58b346a43afcaf56c7e3c34cb7c5ecf18b31453d22
SimHash 3bc4cb61cb51

Groups

*

Rule Path
Disallow /images/
Disallow /m/
Disallow /glosowanie/
Disallow /all_our_e-mail_addresses

stress-agent

No rules defined. All paths allowed.

Back to top

Comments

  • exclude help system from robots
  • the next line is a spam bot trap, for grepping the logs. you should _really_ change this to something else...
  • but allow htdig to index our doc-tree
  • User-agent: htdig
  • disallow stress test
Back to top