Log In Sign Up

pryor.com
robots.txt

Robots Exclusion Standard data for pryor.com

Resource Scan

Scan Details

Site Domain pryor.com
Base Domain pryor.com
Scan Status Ok
Last Scan2025-09-25T23:36:06+00:00
Next Scan 2025-10-25T23:36:06+00:00

Last Scan

Scanned2025-09-25T23:36:06+00:00
URL https://pryor.com/robots.txt
Redirect https://www.pryor.com/robots.txt
Redirect Domain www.pryor.com
Redirect Base pryor.com
Domain IPs 13.86.16.31
Redirect IPs 104.18.36.157, 172.64.151.99
Response IP 104.18.36.157
Found Yes
Hash b96fbeef54a90196b7b94a93ae7ac72ce4560a16ff41e44d0608204ddf06d37c
SimHash 61249476c47e

Groups

*

Rule Path
Disallow /*prefn*
Disallow /*prefv*
Disallow /*pmin*
Disallow /*pmax*
Disallow /*srule*
Disallow /cart*
Disallow /checkout*
Disallow */demandware.store*
Allow */demandware.store/*/Search-UpdateGrid
Disallow */Login?*
Disallow /lms.pryor.com*
Disallow /lmscert.pryor.com*
Disallow /lms01.pryor.com*
Disallow /lms02.pryor.com*
Disallow /lmstest.pryor.com*
Allow /

*

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

pinterestbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

facebookbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

facebookexternalhit/1.1

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

facebookexternalhit/1.0

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1
Back to top

Other Records

Field Value
sitemap https://www.pryor.com/sitemap_index.xml
Back to top

Comments

  • Prevent indexing of raw search refinements, price ranges, sorting rules
  • Prevent indexing of checkout and cart sessions
  • Prevent indexing of un-aliased storefront URLs
  • Copy and uncomment to allow access to certain resources
  • Allow: */demandware.store/*/Pipeline-Name
  • Prevent indexing of Query pages
  • Prevent indexing of subdomains
  • Taxonomy Sitemaps
  • Crawl delays for common bots
Back to top