Log In Sign Up

crutchfield.com
robots.txt

Robots Exclusion Standard data for crutchfield.com

Resource Scan

Scan Details

Site Domain crutchfield.com
Base Domain crutchfield.com
Scan Status Ok
Last Scan2024-10-07T01:37:58+00:00
Next Scan 2024-11-06T01:37:58+00:00

Last Scan

Scanned2024-10-07T01:37:58+00:00
URL https://crutchfield.com/robots.txt
Redirect https://www.crutchfield.com/robots.txt
Redirect Domain www.crutchfield.com
Redirect Base crutchfield.com
Domain IPs 205.196.12.74
Redirect IPs 205.196.12.74
Response IP 205.196.12.74
Found Yes
Hash 5a8d6d3dee6786c3c9864ebb07fe0cbfd0730cd229529e8a69f615b1c3356a39
SimHash 823c1d4088d3

Groups

msiecrawler

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

voilabot

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

*

Rule Path
Disallow /Popups/
Disallow /*/Popups/
Disallow /popups/
Disallow /*/popups/
Disallow /secure/account/
Disallow /*/secure/account/
Disallow /*/Secure/Account/
Disallow /Secure/Account/
Disallow /product/glossary.aspx
Disallow /*/product/glossary.aspx
Disallow /Product/Glossary.aspx
Disallow /*/Product/Glossary.aspx
Disallow /product/condition.aspx
Disallow /*/product/condition.aspx
Disallow /Product/Condition.aspx
Disallow /*/Product/Condition.aspx
Disallow /product/recommender.aspx
Disallow /*/product/recommender.aspx
Disallow /Product/Recommender.aspx
Disallow /*/Product/Recommender.aspx
Disallow /product/recommenderproductcarousel.aspx
Disallow /*/product/recommenderproductcarousel.aspx
Disallow /Product/RecommenderProductCarousel.aspx
Disallow /*/Product/RecommenderProductCarousel.aspx
Disallow /handlers/
Disallow /*/handlers/
Disallow /Handlers/
Disallow /*/Handlers/
Disallow /secure/handlers/
Disallow /*/secure/handlers/
Disallow /Secure/Handlers/
Disallow /*/Secure/Handlers/
Disallow /*/Cart/
Disallow /Cart/
Disallow /*/cart/
Disallow /cart/
Disallow /car/RemoveVehicle.aspx
Disallow /*/car/RemoveVehicle.aspx
Disallow /Car/RemoveVehicle.aspx
Disallow /*/Car/RemoveVehicle.aspx
Disallow /product/item/loadtab.aspx
Disallow /*/product/item/loadtab.aspx
Disallow /Product/Item/LoadTab.aspx
Disallow /*/Product/Item/LoadTab.aspx
Disallow /Learn/AuthorBiosPopup.aspx
Disallow /*/Learn/AuthorBiosPopup.aspx
Disallow /product/bundle/
Disallow /*/product/bundle/
Disallow /Product/Bundle/
Disallow /*/Product/Bundle/
Disallow /Product/Accessory/CompatibleProducts.aspx
Disallow /*/Product/Accessory/CompatibleProducts.aspx
Disallow /car/systems/carsystemsmodal.aspx
Disallow /*/car/systems/carsystemsmodal.aspx
Disallow /car/systems/CarSystemsModal.aspx
Disallow /*/car/systems/CarSystemsModal.aspx
Disallow /car/systems/PhotoCarousel.aspx
Disallow /*/car/systems/PhotoCarousel.aspx
Disallow /product/accessory/
Disallow /*/product/accessory/
Disallow /Product/Accessory/
Disallow /*/Product/Accessory/
Disallow /product/bundle/
Disallow /*/product/bundle/
Disallow /Product/Bundle/
Disallow /*/Product/Bundle/
Disallow /*/product/item/substancewarning.aspx
Disallow /product/item/substancewarning.aspx
Disallow /*/car/outfitmycar/mycar.aspx
Disallow /car/outfitmycar/mycar.aspx
Disallow /*/Product/CompareTo.aspx
Disallow /Product/CompareTo.aspx
Disallow /*/Car/Popups/
Disallow /Car/Popups/
Disallow /*/car/popups/
Disallow /car/popups/
Disallow /*/Car/Solution/
Disallow /Car/Solution/
Disallow /*/car/solution/
Disallow /car/solution/
Back to top

Other Records

Field Value
sitemap https://www.crutchfield.com/sitemap.xml
Back to top