Log In Sign Up

aximer.ae
robots.txt

Robots Exclusion Standard data for aximer.ae

Resource Scan

Scan Details

Site Domain aximer.ae
Base Domain aximer.ae
Scan Status Ok
Last Scan2026-01-31T13:15:55+00:00
Next Scan 2026-03-02T13:15:55+00:00

Last Scan

Scanned2026-01-31T13:15:55+00:00
URL https://aximer.ae/robots.txt
Domain IPs 104.21.3.203, 172.67.131.43, 2606:4700:3035::6815:3cb, 2606:4700:3035::ac43:832b
Response IP 172.67.131.43
Found Yes
Hash cdefcd6a91c16cc3608adefecfe6ac82bd3a40a9c7fe3e64c1239772037ac1d2
SimHash 4e9f4a436761

Groups

*

Rule Path Comment
Disallow /wp-admin/ -
Allow /wp-admin/admin-ajax.php -
Disallow /wp-login.php -
Disallow /welcomevictoria -
Disallow /wp-signup.php -
Disallow /?author= -
Disallow /*?author= -
Disallow /*%26author%3D -
Disallow /author/*/feed$ -
Disallow /author/*/page/ -
Disallow /*/author/ -
Disallow /*/author/* -
Disallow /author/ -
Disallow /my-account/ -
Disallow /account/ -
Disallow /wishlist/ Remove this line if you want wishlist
Disallow /*.pdf$ -
Disallow /cart/ -
Disallow /checkout/ -
Disallow /order-received/ -
Disallow /*add-to-cart%3D* -
Disallow /*wc-ajax%3D* -
Disallow /*_wpnonce%3D* -
Disallow /cdn-cgi/ -
Disallow /search/ -
Disallow /*filter_* -
Disallow /*/filters/* -
Disallow /*/filter-* -
Disallow /*/pa_* -
Disallow /*/attribute/* -
Disallow /category/ -
Disallow /tag/ -
Disallow /date/ -
Disallow /cgi-bin/ -
Allow /wp-content/uploads/ -
Allow /wp-json/ Important for some themes/plugins

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

yandex

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5
Back to top

Other Records

Field Value
sitemap https://aximer.ae/sitemap_index.xml
Back to top

Comments

  • ==============================
  • robots.txt for E-commerce (WooCommerce/WordPress)
  • Last updated: 2025-09-30
  • Note: Do NOT block CSS/JS, otherwise Google rendering may break.
  • ==============================
  • --- Admin and private sections
  • --- Shop: block transactional/filter/search pages
  • --- Internal search results & filter/sort parameters
  • --- Extra filter protection (added 2025-10-26)
  • Block any filtered or faceted pages in both query and path form
  • --- Archives (optional)
  • If you run a blog and want categories/tags indexed, comment these lines out
  • --- System/sensitive files
  • If your frontend depends on REST API, remove the line above
  • --- Allow critical resources for rendering
  • Do NOT block /wp-includes or /wp-content/plugins
  • --- Sitemap (adjust to your site)
  • If using Rank Math or Yoast, this is usually enough
  • Add additional sitemap URLs if you have subdomains or multilingual sites
  • ==============================
  • Specific bot rules
  • ==============================
  • Bing and Yandex support crawl-delay (Google ignores it)
  • Aggressive crawlers (optional – enable if server load is high)
  • Disallow: /
  • User-agent: MJ12bot
  • Disallow: /
  • User-agent: SemrushBot
  • Disallow: /
  • User-agent: dotbot
  • Disallow: /
  • Host directive (for Yandex) – optional if multiple domains
  • Host: aximer.ae
Back to top