Log In Sign Up

antique9.com.tw
robots.txt

Robots Exclusion Standard data for antique9.com.tw

Resource Scan

Scan Details

Site Domain antique9.com.tw
Base Domain antique9.com.tw
Scan Status Ok
Last Scan2026-02-14T19:43:48+00:00
Next Scan 2026-02-21T19:43:48+00:00

Last Scan

Scanned2026-02-14T19:43:48+00:00
URL https://www.antique9.com.tw/robots.txt
Domain IPs 34.216.7.181, 35.161.44.177
Response IP 35.161.44.177
Found Yes
Hash f3e403afaf94f6ebca3549c3663494d067c37f988bf8c1a0cfa7f6b1eb5b8fba
SimHash 3a46c162cb71

Groups

*

Rule Path
Disallow /manual/
Disallow /manual-2.0/
Disallow /manual-1.3/
Disallow /addon-modules/
Disallow /doc/
Disallow /images/
Disallow /manage/
Disallow /template/*.htm$
Disallow /getServerTime.php
Disallow /admin/

oai-searchbot

Rule Path
Allow /

stress-agent

Rule Path
Disallow /
Back to top

Comments

  • exclude help system from robots
  • the next line is a spam bot trap, for grepping the logs. you should _really_ change this to something else...
  • Disallow: /
  • same idea here...
  • but allow htdig to index our doc-tree
  • User-agent: htdig
  • Disallow:
  • disallow stress test
Back to top