speer.com
robots.txt

Robots Exclusion Standard data for speer.com

Resource Scan

Scan Details

Site Domain speer.com
Base Domain speer.com
Scan Status Ok
Last Scan2025-12-24T22:50:23+00:00
Next Scan 2026-01-23T22:50:23+00:00

Last Scan

Scanned2025-12-24T22:50:23+00:00
URL https://speer.com/robots.txt
Redirect https://www.speer.com/robots.txt
Redirect Domain www.speer.com
Redirect Base speer.com
Domain IPs 52.165.135.234
Redirect IPs 104.18.43.16, 172.64.144.240
Response IP 104.18.43.16
Found Yes
Hash 762cfe6096cfc66b5cdf13a5181478c4c9e0ddcaec84da339cd7378e07f224d9
SimHash a522d3086761

Groups

*

Rule Path
Disallow /Account-EditProfile
Disallow /Account-EditPassword
Disallow /Account-PasswordReset
Disallow /Account-SetNewPassword
Disallow /Account-SetNewPasswordConfirm
Disallow /Account-Show
Disallow /Account-StartRegister
Disallow /Account-RegisterProducts
Disallow /Account-Reorder
Disallow /Account-SavedRecipes
Disallow /Address-List
Disallow /Address-AddAddress
Disallow /Cart-Show
Disallow /Checkout-Begin
Disallow /Checkout-Login
Disallow /Compare-Show
Disallow /ContactUs-Landing
Disallow /DealerSupport-Show
Disallow /Default-Offline
Disallow /Default-Start
Disallow /Error-Start
Disallow /EmailPreferences-Manager
Disallow /Home-ErrorNotFound
Disallow /Order-History
Disallow /Order-Details
Disallow /Login-Show
Disallow /PaymentInstruments-List
Disallow /ProductRecall-Show
Disallow /ProductRegistration-Start
Disallow /Search-Show
Disallow /Stores-Find
Disallow /Wishlist-Search
Disallow /Wishlist-Show
Disallow /*prefv*
Disallow /*prefn*
Disallow /*pmax*
Disallow /*pmin*
Disallow /*srule*
Disallow /*dwvar*
Disallow /account*
Disallow /billing*
Disallow /cart*
Disallow /checkout*
Disallow /my-account*
Disallow /my-company*
Disallow /order*
Disallow /payment*
Disallow /register*
Disallow /search*
Disallow /wishlist*

Other Records

Field Value
sitemap https://www.speer.com/sitemap_index.xml

Comments

  • prevent crawling of pipeline urls
  • prevent crawling of raw search refinements, price ranges, sorting rules
  • prevent crawling of dwvar parameter
  • prevent crawling of specific urls
  • prevent crawling demandware.store urls - removed 08252025
  • Disallow: /*demandware.store/*
  • sitemaps