pantene.com
robots.txt

Robots Exclusion Standard data for pantene.com

Resource Scan

Scan Details

Site Domain pantene.com
Base Domain pantene.com
Scan Status Ok
Last Scan2024-09-23T10:39:56+00:00
Next Scan 2024-10-07T10:39:56+00:00

Last Scan

Scanned2024-09-23T10:39:56+00:00
URL https://pantene.com/robots.txt
Domain IPs 13.107.246.59
Response IP 13.107.246.58
Found Yes
Hash 85f73fbe457c1ab518df9a86a479e5914efb0b3ffeafd045df1fa06557affd19
SimHash 2b444bf3a740

Groups

*

Rule Path
Allow /
Disallow /en-us/product/miracle-rescue-curl-define-shine-leave-on-conditioner
Disallow /en-us/product/keratin-miracle-repair-treatment
Disallow /en-us/product/miracle-rescue-regenerative-hair-oil
Disallow /en-us/product/regenerative-oil-set
Disallow /en-us/product/extreme-damage-care-shampoo
Disallow /en-us/product/extreme-damage-care-conditioner
Disallow /en-us/product/miracle-bond-repair-shampoo-conditioner
Disallow /en-us/404
Disallow /en-us/ittakesscience
Disallow /es-us/producto/miracle-rescue-curl-define-shine-leave-on-conditioner
Disallow /es-us/producto/miracle-rescue-regenerative-hair-oil
Disallow /es-us/producto/regenerative-oil-blend-set
Disallow /es-us/producto/keratin-miracle-repair-treatment
Disallow /es-us/producto/miracle-rescue-10-in-1-multitasking-spray-leave-in-conditioner
Disallow /es-us/producto/extreme-damage-care-shampoo
Disallow /es-us/producto/extreme-damage-care-conditioner
Disallow /es-us/producto/miracle-repair-dual-pack
Disallow /es-us/producto/miracle-repair-conditioner
Disallow /es-us/producto/miracle-repair-shampoo
Disallow /es-us/producto/infinite-lengths-dual-pack
Disallow /es-us/producto/infinite-lengths-conditioner
Disallow /es-us/producto/infinite-lengths-shampoo
Disallow /es-us/producto/mix-in-shine
Disallow /es-us/producto/mix-in-anti-frizz
Disallow /es-us/producto/mix-in-lengthen
Disallow /es-us/producto/mix-in-strengthen
Disallow /es-us/producto/mix-in-hydrate
Disallow /es-us/producto/mix-in-curl

Other Records

Field Value
sitemap https://pantene.com/sitemap.xml

Warnings

  • `host` is not a known field.