darkshade.homeip.net
robots.txt

Robots Exclusion Standard data for darkshade.homeip.net

Resource Scan

Scan Details

Site Domain darkshade.homeip.net
Base Domain darkshade.homeip.net
Scan Status Ok
Last Scan2025-11-09T15:43:12+00:00
Next Scan 2025-12-09T15:43:12+00:00

Last Scan

Scanned2025-11-09T15:43:12+00:00
URL https://darkshade.homeip.net/robots.txt
Domain IPs 24.246.30.74
Response IP 24.246.30.74
Found Yes
Hash dd292390618c9633cb1989462b5c70bf303a42019dc6da06f508d37f4d2fb288
SimHash 3a44c163cb75

Groups

*

Rule Path
Disallow /manual/
Disallow /manual-2.0/
Disallow /manual-1.3/
Disallow /addon-modules/
Disallow /doc/
Disallow /images/
Disallow /all_our_e-mail_addresses
Disallow /admin/

stress-agent

Rule Path
Disallow /

Comments

  • exclude help system from robots
  • the next line is a spam bot trap, for grepping the logs. you should _really_ change this to something else...
  • same idea here...
  • but allow htdig to index our doc-tree
  • User-agent: htdig
  • Disallow:
  • disallow stress test