Log In Sign Up

shepher.com
robots.txt

Robots Exclusion Standard data for shepher.com

Resource Scan

Scan Details

Site Domain shepher.com
Base Domain shepher.com
Scan Status Ok
Last Scan2024-08-26T03:16:12+00:00
Next Scan 2024-09-25T03:16:12+00:00

Last Scan

Scanned2024-08-26T03:16:12+00:00
URL https://www.shepher.com/robots.txt
Domain IPs 125.56.219.18, 23.32.29.89
Response IP 23.215.7.18
Found Yes
Hash 9f042a84e0d72cab6776e85959aa01279bcb81e91a4a92a8d64b6e107bbe7cc6
SimHash 2c38cd574c03

Groups

*

Rule Path
Allow /
Disallow /cart
Disallow */c.4440160/*
Disallow */secure/*
Disallow /*?order=
Disallow /*?page=
Disallow /*?lang=
Disallow */api/*
Disallow */undefined%26origin%3Dhome%26origin_hash
Disallow /*undefined%26origin%3Dhome%26origin_hash
Disallow */price/*
Disallow */onlinecustomerprice/*
Disallow */pricelevel5/*
Disallow */commercecategory/*
Disallow */pricing-group/*
Disallow */age-group/*
Disallow */brand/brand/*
Disallow */item-category/*
Disallow */shop/*
Disallow */provider/*
Disallow */new-item/*
Disallow */Yes/*
Disallow */lower-price/*
Disallow */newreview/*
Disallow */newReview/*
Back to top

Other Records

Field Value
sitemap http://www.shepher.com/sitemap_www.shepher.com_Index.xml
Back to top

Comments

  • These entries in the robots.txt file let the search engine know to ignore any URL that includes these facets.
  • By using the *wildcards*, the pages are not indexed, regardless of the position of the facet in the URL. Updated 04-10-2021
  • Allow all robots to spider everything by disallowing nothing
  • Disallow cart related
  • Disallow facet related
  • Disallow reviews related
Back to top

Warnings

  • 1 invalid line.
Back to top