Log In Sign Up

clarks.com
robots.txt

Robots Exclusion Standard data for clarks.com

Resource Scan

Scan Details

Site Domain clarks.com
Base Domain clarks.com
Scan Status Ok
Last Scan2024-10-01T16:01:35+00:00
Next Scan 2024-10-31T16:01:35+00:00

Last Scan

Scanned2024-10-01T16:01:35+00:00
URL https://www.clarks.com/robots.txt
Domain IPs 104.18.30.157, 104.18.31.157, 2606:4700::6812:1e9d, 2606:4700::6812:1f9d
Response IP 104.18.31.157
Found Yes
Hash fbf931f0bc7905d54f87338dfac03cc3815fd4aec1aea585e6895af6ebe48fe6
SimHash e25cd857d5df

Groups

*

Rule Path
Disallow */en-gb/basket*
Disallow */en-ie/basket*
Disallow */checkout*
Disallow */login*
Disallow */reset-password*
Disallow */register*
Disallow */my-account*
Disallow */find-order
Disallow */search*
Disallow *?size*
Disallow *?color*
Disallow *?productType*
Disallow *?productSubType*
Disallow *?styleName*
Disallow *?fit*
Disallow *?upperMaterial*
Disallow *?technologies*
Disallow *?minPrice*
Disallow *?maxPrice*
Disallow *?review*
Disallow *?heelHeight*
Disallow *?brand*
Disallow *?fastening*
Disallow *?gender*
Disallow *?sortBy*
Disallow */newsletter-unsubscribe*
Disallow */my-appointment*
Disallow */appointment-success*
Disallow */appointment-failed*
Disallow */cart*

adsbot-google-mobile

Rule Path
Allow */basket*
Allow */register*
Allow */my-account*
Allow */search*
Allow *?size*
Allow *?color*
Allow *?productType*
Allow *?productSubType*
Allow *?styleName*
Allow *?fit*
Allow *?upperMaterial*
Allow *?technologies*
Allow *?minPrice*
Allow *?maxPrice*
Allow *?review*
Allow *?heelHeight*
Allow *?brand*
Allow *?fastening*
Allow *?gender*
Allow *?sortBy*
Allow */cart*

adsbot-google

Rule Path
Allow */basket*
Allow */register*
Allow */my-account*
Allow */search*
Allow *?size*
Allow *?color*
Allow *?productType*
Allow *?styleName*
Allow *?fit*
Allow *?upperMaterial*
Allow *?technologies*
Allow *?minPrice*
Allow *?maxPrice*
Allow *?review*
Allow *?heelHeight*
Allow *?brand*
Allow *?fastening*
Allow *?gender*
Allow *?sortBy*
Allow */cart*

adidxbot

Rule Path
Allow */basket*
Allow */register*
Allow */my-account*
Allow */search*
Allow *?size*
Allow *?color*
Allow *?productType*
Allow *?styleName*
Allow *?fit*
Allow *?upperMaterial*
Allow *?technologies*
Allow *?minPrice*
Allow *?maxPrice*
Allow *?review*
Allow *?heelHeight*
Allow *?brand*
Allow *?fastening*
Allow *?gender*
Allow *?sortBy*
Allow */cart*

cazoodlebot

Rule Path
Disallow /

ahrefsbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

mj12bot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

gigabot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

ahrefssiteaudit

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

dotbot/1.0

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10
Back to top

Other Records

Field Value
sitemap https://www.clarks.com/en-gb/sitemap.xml
sitemap https://www.clarks.com/en-us/sitemap.xml
sitemap https://www.clarks.com/en-ca/sitemap.xml
sitemap https://www.clarks.com/fr-ca/sitemap.xml
sitemap https://www.clarks.com/en-ie/sitemap.xml
sitemap https://www.clarks.com/en-at/sitemap.xml
sitemap https://www.clarks.com/de-at/sitemap.xml
sitemap https://www.clarks.com/en-be/sitemap.xml
sitemap https://www.clarks.com/nl-be/sitemap.xml
sitemap https://www.clarks.com/fr-be/sitemap.xml
sitemap https://www.clarks.com/en-ba/sitemap.xml
sitemap https://www.clarks.com/en-bg/sitemap.xml
sitemap https://www.clarks.com/en-hr/sitemap.xml
sitemap https://www.clarks.com/en-cy/sitemap.xml
sitemap https://www.clarks.com/en-cz/sitemap.xml
sitemap https://www.clarks.com/en-dk/sitemap.xml
sitemap https://www.clarks.com/en-ee/sitemap.xml
sitemap https://www.clarks.com/en-fi/sitemap.xml
sitemap https://www.clarks.com/en-fr/sitemap.xml
sitemap https://www.clarks.com/fr-fr/sitemap.xml
sitemap https://www.clarks.com/en-ge/sitemap.xml
sitemap https://www.clarks.com/en-de/sitemap.xml
sitemap https://www.clarks.com/de-de/sitemap.xml
sitemap https://www.clarks.com/en-gi/sitemap.xml
sitemap https://www.clarks.com/en-gr/sitemap.xml
sitemap https://www.clarks.com/en-hu/sitemap.xml
sitemap https://www.clarks.com/en-is/sitemap.xml
sitemap https://www.clarks.com/en-it/sitemap.xml
sitemap https://www.clarks.com/it-it/sitemap.xml
sitemap https://www.clarks.com/en-lv/sitemap.xml
sitemap https://www.clarks.com/en-li/sitemap.xml
sitemap https://www.clarks.com/en-lt/sitemap.xml
sitemap https://www.clarks.com/en-lu/sitemap.xml
sitemap https://www.clarks.com/fr-lu/sitemap.xml
sitemap https://www.clarks.com/en-mt/sitemap.xml
sitemap https://www.clarks.com/en-mc/sitemap.xml
sitemap https://www.clarks.com/en-me/sitemap.xml
sitemap https://www.clarks.com/en-nl/sitemap.xml
sitemap https://www.clarks.com/nl-nl/sitemap.xml
sitemap https://www.clarks.com/en-mk/sitemap.xml
sitemap https://www.clarks.com/en-no/sitemap.xml
sitemap https://www.clarks.com/en-pl/sitemap.xml
sitemap https://www.clarks.com/en-pt/sitemap.xml
sitemap https://www.clarks.com/en-md/sitemap.xml
sitemap https://www.clarks.com/en-ro/sitemap.xml
sitemap https://www.clarks.com/en-sm/sitemap.xml
sitemap https://www.clarks.com/it-sm/sitemap.xml
sitemap https://www.clarks.com/en-rs/sitemap.xml
sitemap https://www.clarks.com/en-sk/sitemap.xml
sitemap https://www.clarks.com/en-si/sitemap.xml
sitemap https://www.clarks.com/en-es/sitemap.xml
sitemap https://www.clarks.com/es-es/sitemap.xml
sitemap https://www.clarks.com/en-se/sitemap.xml
sitemap https://www.clarks.com/en-ch/sitemap.xml
sitemap https://www.clarks.com/de-ch/sitemap.xml
sitemap https://www.clarks.com/fr-ch/sitemap.xml
sitemap https://www.clarks.com/it-ch/sitemap.xml
sitemap https://www.clarks.com/en-tr/sitemap.xml
sitemap https://www.clarks.com/hreflang-sitemap.xml
sitemap https://www.clarks.com/editorial-sitemap.xml
sitemap https://www.clarks.com/misc-sitemap.xml
sitemap https://www.clarks.com/content-landing-sitemap.xml
Back to top

Comments

  • Block access to specific groups of pages
  • Sitemap Index Files
  • Hreflang Sitemap Files
Back to top