Log In Sign Up

karenmillen.com
robots.txt

Robots Exclusion Standard data for karenmillen.com

Resource Scan

Scan Details

Site Domain karenmillen.com
Base Domain karenmillen.com
Scan Status Ok
Last Scan2024-11-02T09:39:51+00:00
Next Scan 2024-11-16T09:39:51+00:00

Last Scan

Scanned2024-11-02T09:39:51+00:00
URL https://karenmillen.com/robots.txt
Redirect https://www.karenmillen.com/robots.txt
Redirect Domain www.karenmillen.com
Redirect Base karenmillen.com
Domain IPs 34.213.106.51, 54.68.182.72
Redirect IPs 23.215.7.25, 23.215.7.32, 2600:1413:b000:1b::17d7:719, 2600:1413:b000:1b::17d7:720
Response IP 104.88.71.122
Found Yes
Hash 16250ddb07c322b8608fe52aa8c51b36a12c27bcc6254b6455420472b9fdbaaa
SimHash 22293f4b037d

Groups

*

Rule Path
Disallow /account-edit*
Disallow /preferences-edit*
Disallow /checkout-login*
Disallow /order-history*
Disallow /order-confirmation*
Disallow /on/demandware.store/*
Allow /on/demandware.store/*/Product-Show?pid=*
Disallow */Wishlist-Add*
Disallow */wishlist-add*
Disallow */Product-Variation*
Disallow /*sz%3D40%26start%3D*
Disallow /*sz%3D80%26start%3D*
Disallow /*srule%3D*
Disallow /*pmin%3D*
Disallow /*pmax%3D*
Disallow /*prefn*
Disallow /*prefv*
Disallow /*%7C*
Disallow /*search*
Allow /on/demandware.static/*/*.js
Allow /on/demandware.static/*/*.css
Allow /on/demandware.static/*/*.json
Allow /on/demandware.static/*/*.jpg
Allow /on/demandware.static/*/*.jpeg
Allow /on/demandware.static/*/*.webp
Allow /on/demandware.static/*/*.woff2
Allow /on/demandware.static/*/*.png
Allow /on/demandware.static/*/*.gif
Allow /on/demandware.static/*/*.svg

pinterestbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 0.2
Back to top

Other Records

Field Value
sitemap https://www.karenmillen.com/sitemap_index.xml
sitemap https://www.karenmillen.com/us/sitemap_index.xml
sitemap https://www.karenmillen.com/au/sitemap_index.xml
sitemap https://www.karenmillen.com/ie/sitemap_index.xml
Back to top

Comments

  • Pages
  • Product Filter
  • Ordering & Product per page
  • Number of product per page - Default 60
  • Order By
  • Price
  • Faceted Navigation
  • Search
  • Ensure no Static Ressources is blocked
  • Crawl Delay - 5 URL max per second
Back to top