catlifestyle.pe
robots.txt

Robots Exclusion Standard data for catlifestyle.pe

Resource Scan

Scan Details

Site Domain catlifestyle.pe
Base Domain catlifestyle.pe
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-09-22T01:43:21+00:00
Next Scan 2024-10-22T01:43:21+00:00

Last Successful Scan

Scanned2024-08-01T01:42:44+00:00
URL https://www.catlifestyle.pe/robots.txt
Domain IPs 13.33.88.120, 13.33.88.20, 13.33.88.57, 13.33.88.77, 2600:9000:223b:4c00:15:222:6000:93a1, 2600:9000:223b:5200:15:222:6000:93a1, 2600:9000:223b:6800:15:222:6000:93a1, 2600:9000:223b:6e00:15:222:6000:93a1, 2600:9000:223b:7200:15:222:6000:93a1, 2600:9000:223b:ba00:15:222:6000:93a1, 2600:9000:223b:e600:15:222:6000:93a1, 2600:9000:223b:ec00:15:222:6000:93a1
Response IP 13.33.88.57
Found Yes
Hash 38fc6410f82eb4defd971660c314cd33b7f4d4a98e11951590cedd9956b9797f
SimHash ac282d96c9d9

Groups

*

Rule Path
Disallow /account*
Disallow /login*
Disallow /busca*
Disallow /quick-view*
Disallow /espiar*
Disallow /buscapagina*
Disallow /sale/*
Disallow /cart*
Disallow /search-result*
Disallow /checkout/*
Disallow /Sistema/*
Disallow /*?*PS*
Disallow /*?*O*
Disallow /*?*map*
Disallow /*?*q*
Disallow /*?*map*
Disallow /*?*ProductLinkNotFound=*
Disallow /*?sku*
Disallow /*?*filter*

Other Records

Field Value
sitemap https://www.catlifestyle.pe/sitemap.xml
sitemap https://caterpillarpe.vtexassets.com/arquivos/sitemap-caterpillar.xml

Comments

  • Disallow all crawlers access to certain pages.
  • Disallow URL Parameters
  • Disallow: /*?*utm*