Log In Sign Up

hardwaremart.lk
robots.txt

Robots Exclusion Standard data for hardwaremart.lk

Resource Scan

Scan Details

Site Domain hardwaremart.lk
Base Domain hardwaremart.lk
Scan Status Ok
Last Scan2026-01-27T16:45:12+00:00
Next Scan 2026-02-26T16:45:12+00:00

Last Scan

Scanned2026-01-27T16:45:12+00:00
URL https://hardwaremart.lk/robots.txt
Domain IPs 104.21.10.54, 172.67.131.62, 2606:4700:3033::6815:a36, 2606:4700:3034::ac43:833e
Response IP 172.67.131.62
Found Yes
Hash 380403d72d31b82bdf6c8bcfebac49f2e33577c46786b78c6b4e29a789044ef5
SimHash a74079d7a19b

Groups

*

Rule Path
Disallow /wp-admin/
Disallow /wp-login.php
Disallow /wp-register.php
Disallow /cart/
Disallow /checkout/
Disallow /my-account/
Disallow /order-tracking/
Disallow /thank-you/
Disallow /?s=
Disallow /search/
Disallow /?p=
Disallow /product-category/
Disallow /product/
Disallow /shop/
Disallow /shop/page/
Allow /wp-content/uploads/
Allow /wp-includes/js/

Other Records

Field Value
crawl-delay 20
Back to top

Comments

  • General settings
  • Disallow admin pages to prevent crawling sensitive areas
  • Disallow cart, checkout, and account-related pages
  • Disallow search and query result pages (to avoid duplicate content)
  • WooCommerce
  • Disallow unnecessary WooCommerce pages (product category, product pages, etc.)
  • If you don't want to index product pages, you may leave these disallowed
  • Optionally, disallow shop overview and paginated pages if you don't want them indexed
  • Allow important assets to be crawled (for proper indexing and user experience)
Back to top