Log In Sign Up

colgate.co.uk
robots.txt

Robots Exclusion Standard data for colgate.co.uk

Resource Scan

Scan Details

Site Domain colgate.co.uk
Base Domain colgate.co.uk
Scan Status Ok
Last Scan2025-12-04T12:26:09+00:00
Next Scan 2026-01-03T12:26:09+00:00

Last Scan

Scanned2025-12-04T12:26:09+00:00
URL https://colgate.co.uk/robots.txt
Redirect https://www.colgate.com/robots.txt
Redirect Domain www.colgate.com
Redirect Base colgate.com
Domain IPs 34.230.254.189
Redirect IPs 23.220.203.42, 23.220.203.49, 2600:1413:5000:13::1737:27ca, 2600:1413:5000:13::1737:27d0
Response IP 23.215.7.23
Found Yes
Hash ed57e082d735f6280fc67f2024a2c9bb655aee111d963a5c4e90cbe02671ceeb
SimHash 6921fb31c300

Groups

*

Rule Path
Allow */search
Allow */content/
Disallow */cp-sites/
Allow */cp-sites/oral-care*
Disallow */smartlabel
Disallow */smiles/
Allow */smiles/special-offers
Disallow *?thumbparam*
Disallow *?filter=*
Disallow *?occ_tag=*
Disallow *?wcmmode=*
Disallow *?bvstate=*
Disallow */b2bcontent/
Disallow */cp-sites-aem/
Disallow /*?*
Disallow */search-results
Disallow /dam/cp-sites/aem*/
Disallow */product-match-tool*/
Disallow */oral-health-quiz*/
Disallow */results-page/

swiftbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5
Back to top

Other Records

Field Value
sitemap https://www.colgate.com/images_sitemap.xml
sitemap https://www.colgate.com/en-us/sitemap.xml
sitemap https://www.colgate.com/es-mx/sitemap.xml
sitemap https://www.colgate.com/es-co/sitemap.xml
sitemap https://www.colgate.com/es-ar/sitemap.xml
sitemap https://www.colgate.com/es-py/sitemap.xml
sitemap https://www.colgate.com/es-bo/sitemap.xml
sitemap https://www.colgate.com/es-ec/sitemap.xml
sitemap https://www.colgate.com/es-ve/sitemap.xml
sitemap https://www.colgate.com/es-cl/sitemap.xml
sitemap https://www.colgate.com/es-pe/sitemap.xml
sitemap https://www.colgate.com/fr-ca/sitemap.xml
sitemap https://www.colgate.com/en-ca/sitemap.xml
sitemap https://www.colgate.com/es-do/sitemap.xml
sitemap https://www.colgate.com/es-uy/sitemap.xml
sitemap https://www.colgate.com/en-gb/sitemap.xml
sitemap https://www.colgate.com/tr-tr/sitemap.xml
sitemap https://www.colgate.com/en-za/sitemap.xml
sitemap https://www.colgate.com/es-us/sitemap.xml
sitemap https://www.colgate.com/th-th/sitemap.xml
sitemap https://www.colgate.com/en-in/sitemap.xml
sitemap https://www.colgate.com/en-sa/sitemap.xml
sitemap https://www.colgate.com/en-ph/sitemap.xml
sitemap https://www.colgate.com/pt-pt/sitemap.xml
sitemap https://www.colgate.com/en-my/sitemap.xml
sitemap https://www.colgate.com/en-sg/sitemap.xml
sitemap https://www.colgate.com/ms-my/sitemap.xml
sitemap https://www.colgate.com/ar-sa/sitemap.xml
sitemap https://www.colgate.com/kz-kz/sitemap.xml
sitemap https://www.colgate.com/es-cr/sitemap.xml
sitemap https://www.colgate.com/es-pa/sitemap.xml
sitemap https://www.colgate.com/es-hn/sitemap.xml
sitemap https://www.colgate.com/es-sv/sitemap.xml
sitemap https://www.colgate.com/es-ni/sitemap.xml
sitemap https://www.colgate.com/es-gt/sitemap.xml
Back to top

Comments

  • Standard Excludes
  • ZG Sitemap Excludes
  • Non-ZG Excludes
  • Sitemap Locations
  • Search Engine Crawl Delays
Back to top

Warnings

  • `noindex` is not a known field.
Back to top