shop.amway.com.tw
robots.txt

Robots Exclusion Standard data for shop.amway.com.tw

Resource Scan

Scan Details

Site Domain shop.amway.com.tw
Base Domain amway.com.tw
Scan Status Ok
Last Scan2025-12-29T17:07:22+00:00
Next Scan 2026-01-28T17:07:22+00:00

Last Scan

Scanned2025-12-29T17:07:22+00:00
URL https://shop.amway.com.tw/robots.txt
Domain IPs 13.33.88.110, 13.33.88.126, 13.33.88.21, 13.33.88.86
Response IP 13.33.88.126
Found Yes
Hash e28318d3e9e89f476905d3a1241882245503be15e85c74373d16ae513981cb07
SimHash 28565f164ef1

Groups

*

Rule Path
Disallow /cart
Disallow /checkout
Disallow /profile
Disallow /address
Disallow /favorites
Disallow /mylist
Disallow /bulkorders
Disallow /orders
Disallow /privilegeorders
Disallow /switchRecipient
Disallow /a40stamps

cazoodlebot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

dotbot/1.0

Rule Path
Disallow /

gigabot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://shop.amway.com.tw/sitemap-old.xml
sitemap https://shop.amway.com.tw/sitemap.xml

Comments

  • 20220601 Updated
  • For all robots
  • Block access to specific groups of pages
  • Allow search crawlers to discover the sitemap
  • Block CazoodleBot as it does not present correct accept content headers
  • Block MJ12bot as it is just noise
  • Block dotbot as it cannot parse base urls properly
  • Block Gigabot