Log In Sign Up

top-one.com.my
robots.txt

Robots Exclusion Standard data for top-one.com.my

Resource Scan

Scan Details

Site Domain top-one.com.my
Base Domain top-one.com.my
Scan Status Ok
Last Scan2026-02-14T05:47:32+00:00
Next Scan 2026-02-21T05:47:32+00:00

Last Scan

Scanned2026-02-14T05:47:32+00:00
URL https://top-one.com.my/robots.txt
Redirect https://www.top-one.com.my/robots.txt
Redirect Domain www.top-one.com.my
Redirect Base top-one.com.my
Domain IPs 103.6.198.12
Redirect IPs 103.6.198.12
Response IP 103.6.198.12
Found Yes
Hash 7602c1158121b3ba6795a2b6b7c98d23b86ffba4bbd24b21c5c4c91be78dca5b
SimHash 81071a51e6d6

Groups

*

Rule Path
Disallow /admin/
Disallow /login/
Disallow /private/
Disallow /temp/
Disallow /cgi-bin/
Disallow /cart/
Disallow /connect/
Disallow /cms/
Disallow /itinerary/
Allow /
Allow /more/
Allow /itinerary/public/
Allow /images/
Allow /js/
Allow /css/

meta-externalagent

Rule Path Comment
Allow / Allow everything
Disallow /admin/ -
Disallow /login/ -
Disallow /private/ -
Disallow /temp/ -
Disallow /cgi-bin/ -
Disallow /cart/ -
Disallow /connect/ -
Disallow /cms/ -
Disallow /itinerary/ -
Back to top

Other Records

Field Value
sitemap https://www.top-one.com.my/sitemap.xml
Back to top

Comments

  • ==============================
  • Robots.txt for Top-One Travel
  • ==============================
  • 禁止抓取后台、临时文件、敏感信息
  • 允许抓取核心页面
  • 图片、JS、CSS 允许抓取，提高页面渲染和SEO
  • Sitemap
  • ==============================
  • Notes:
  • 1. 禁止抓取后台和用户敏感页面，避免泄露信息。
  • 2. 允许抓取所有公开页面，包括行程、目的地和博客。
  • 3. 图片、JS、CSS 允许抓取，有助于 Google 更好渲染页面。
  • ==============================
Back to top