Log In Sign Up

didriks.com
robots.txt

Robots Exclusion Standard data for didriks.com

Resource Scan

Scan Details

Site Domain didriks.com
Base Domain didriks.com
Scan Status Ok
Last Scan2024-11-02T11:16:58+00:00
Next Scan 2024-12-02T11:16:58+00:00

Last Scan

Scanned2024-11-02T11:16:58+00:00
URL https://www.didriks.com/robots.txt
Redirect https://www.didriks.com/robots_www-didriks-com.txt
Domain IPs 125.56.219.3, 23.32.29.91
Response IP 23.32.29.91
Found Yes
Hash eb6514e7accaa5185efacf3407fd53f989bef5900af4d73bafa0e17c997ab5a7
SimHash 267d7f050453

Groups

*

Rule Path
Allow /
Disallow /cart
Disallow /*?order=
Disallow /*?page=
Disallow /*?lang=
Disallow */search*
Disallow */api/*
Disallow *?show=*
Disallow *?item-category=*
Disallow *?item-size=*
Disallow */price/*
Disallow */onlinecustomerprice/*
Disallow */pricelevel5/*
Disallow */itemsize/*
Disallow /*/*/*/designer*
Disallow /*/*/*/brand*
Disallow */item-category/*
Disallow */brand/*
Disallow */capacity/*
Disallow */collections/*
Disallow */induction-ready/*
Disallow */drink-type/*
Disallow */material/*
Disallow */pattern/*
Disallow */scent-type/*
Disallow */shape/*
Disallow */type/*
Disallow */weave/*
Disallow */on-clearance/*
Disallow */item-color/*
Disallow */features/*
Disallow */on-promo/*
Disallow */finish/*
Disallow */designer/designer/*
Disallow */decore/decore/*
Back to top

Other Records

Field Value
sitemap https://www.didriks.com/sitemap_www.didriks.com_Index.xml
Back to top

Comments

  • These entries in the robots.txt file let the search engine know to ignore any URL that includes these facets.
  • By using the *wildcards*, the pages are not indexed, regardless of the position of the facet in the URL.
  • Updated 12-102-2018 to use the NS native sitemap
  • Allow all robots to spider everything by disallowing nothing
  • Disallow cart related
  • Disallow sort and language related
  • Disallow facet related
  • Sitemap: https://www.didriks.com/didriks-sitemap.xml
Back to top