rspure.ross-simons.com
robots.txt

Robots Exclusion Standard data for rspure.ross-simons.com

Resource Scan

Scan Details

Site Domain rspure.ross-simons.com
Base Domain ross-simons.com
Scan Status Ok
Last Scan2025-05-24T00:15:31+00:00
Next Scan 2025-06-23T00:15:31+00:00

Last Scan

Scanned2025-05-24T00:15:31+00:00
URL https://rspure.ross-simons.com/robots.txt
Redirect https://www.ross-simons.com/robots.txt
Redirect Domain www.ross-simons.com
Redirect Base ross-simons.com
Domain IPs 104.18.183.107, 104.18.184.107
Redirect IPs 104.18.183.107, 104.18.184.107
Response IP 104.18.183.107
Found Yes
Hash 25e2de78dcb83b0e4a7491d291c6c3ccb68f0253d91ba0b3da9d7f1f1675c144
SimHash 60649c56ceae

Groups

*

Rule Path
Disallow /*prefn3%3D*
Disallow /*prefv3%3D*
Disallow /*prefn4%3D*
Disallow /*prefv4%3D*
Disallow /*prefn5%3D*
Disallow /*prefv5%3D*
Disallow /*prefn6%3D*
Disallow /*prefv6%3D*
Disallow /*prefn7%3D*
Disallow /*prefv7%3D*
Disallow /*pmin%3D*
Disallow /*pmax%3D*
Disallow *%7C*
Disallow *%7C*
Disallow /cart*
Disallow /checkout*
Disallow */demandware.store*
Allow */demandware.store/*/Search-UpdateGrid
Allow /

*

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 2

pinterestbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 0

facebookbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 0

facebookexternalhit/1.1

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 0

facebookexternalhit/1.0

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 0

Other Records

Field Value
sitemap https://www.ross-simons.com/sitemap_index.xml

Comments

  • Prevent indexing of raw search refinements, price ranges, sorting rules
  • Disallow crawling of 3+ Refinements
  • Prevent indexing of checkout and cart sessions
  • Prevent indexing of un-aliased storefront URLs
  • Copy and uncomment to allow access to certain resources
  • Allow: */demandware.store/*/Pipeline-Name
  • Taxonomy Sitemaps
  • Crawl delays for common bots