palmolive.com
robots.txt

Robots Exclusion Standard data for palmolive.com

Resource Scan

Scan Details

Site Domain palmolive.com
Base Domain palmolive.com
Scan Status Ok
Last Scan2025-11-08T05:46:25+00:00
Next Scan 2025-12-08T05:46:25+00:00

Last Scan

Scanned2025-11-08T05:46:25+00:00
URL https://palmolive.com/robots.txt
Redirect https://www.palmolive.com/robots.txt
Redirect Domain www.palmolive.com
Redirect Base palmolive.com
Domain IPs 34.230.254.189
Redirect IPs 23.215.7.17, 23.215.7.18, 2600:1413:5000:13::1737:27c5, 2600:1413:5000:13::1737:27d2
Response IP 23.58.144.145
Found Yes
Hash 7b5d3ce09707e1773280095c787b69f20aa7ec2e961add79029ba615349572cc
SimHash 658c8a71c131

Groups

swiftbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

*

Rule Path
Disallow /content/
Disallow /cp-sites/
Disallow /dev/
Allow /en-us/sitemap.xml
Disallow */cp-sites/
Allow */cp-sites/home-care/palmolive-na*

Other Records

Field Value
sitemap https://www.palmolive.com/en-us/sitemap.xml
sitemap https://www.palmolive.com/en-ca/sitemap.xml
sitemap https://www.palmolive.com/fr-ca/sitemap.xml
sitemap https://www.palmolive.com/en-au/sitemap.xml

Comments

  • Images