Log In Sign Up

fsml-technologies.com
robots.txt

Robots Exclusion Standard data for fsml-technologies.com

Resource Scan

Scan Details

Site Domain fsml-technologies.com
Base Domain fsml-technologies.com
Scan Status Ok
Last Scan2025-09-13T03:50:48+00:00
Next Scan 2025-09-20T03:50:48+00:00

Last Scan

Scanned2025-09-13T03:50:48+00:00
URL http://fsml-technologies.com/robots.txt
Domain IPs 107.180.26.71
Response IP 107.180.26.71
Found Yes
Hash d4f589f287b36331be4e2b644eb78033e7991ea614bb88418a52ce9dd9ecc7a2
SimHash 3a448963cb71

Groups

*

Rule Path
Disallow /manual/
Disallow /manual-1.3/
Disallow /manual-2.0/
Disallow /manual-2.2/
Disallow /addon-modules/
Disallow /doc/
Disallow /images/
Disallow /all_our_e-mail_addresses
Disallow /admin/

stress-agent

Rule Path
Disallow /
Back to top

Comments

  • exclude help system from robots
  • the next line is a spam bot trap, for grepping the logs. you should _really_ change this to something else...
  • same idea here...
  • but allow htdig to index our doc-tree
  • User-agent: htdig
  • Disallow:
  • disallow stress test
Back to top