chrobinson.com
robots.txt
Robots Exclusion Standard data for chrobinson.com
Resource Scan
Scan Details
| Site Domain | chrobinson.com |
| Base Domain | chrobinson.com |
| Scan Status | Ok |
| Last Scan | 2024-11-12T14:15:41+00:00 |
| Next Scan | 2024-12-12T14:15:41+00:00 |
Last Scan
| Scanned | 2024-11-12T14:15:41+00:00 |
| URL | https://www.chrobinson.com/robots.txt |
| Domain IPs | 104.18.39.51, 172.64.148.205, 2606:4700:4400::6812:2733, 2606:4700:4400::ac40:94cd |
| Response IP | 104.18.39.51 |
| Found | Yes |
| Hash | d3116005baafc63da71ebf4bbb80c93d4396d6389ea4ed75b5f0f84ceefecbb1 |
| SimHash | 180184d33190 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /*/downloads/nolink/* |
| Disallow | /*/golf/* |
| Disallow | /*/europe2010/* |
| Disallow | /*/assets/downloads/* |
| Disallow | /*/branchEvents/* |
| Disallow | /*/mobile/* |
| Disallow | /*/default/* |
| Disallow | /*/hardship/* |
| Disallow | /*/cdn-cgi/* |
| Disallow | /*/testimonials/* |
| Disallow | /*/contact/global-services/* |
| Disallow | /en-us/contact-us/global-forwarding-leads/* |
| Disallow | /*/lp/* |
| Disallow | /*/sp/* |
| Disallow | /*/lp-* |
| Disallow | /*/search/?* |
| Disallow | /*/chrfreshglobal/* |
| Disallow | /*/chrglobal/* |
| Disallow | /*/ChRobinsonUSA/* |
| Disallow | /en-US/resources/resource-center/webinars/content-n/* |
| Disallow | /en-US/About-Us/Mexico/Mexico-Capabilities/ |
| Disallow | /en-US/About-Us/Canada/Canada-Supply-Chain-Solutions/ |
| Disallow | /en-us/newsroom/in-the-news/* |
| Disallow | *sitecore* |
| Disallow | *thank-you* |
| Disallow | *error* |
| Disallow | /en-us/carriers/carrier-financial-services/factoring/ |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.chrobinson.com/chrobinson-de-de-sitemap.xml |
| sitemap | https://www.chrobinson.com/chrobinson-en-au-sitemap.xml |
| sitemap | https://www.chrobinson.com/chrobinson-en-gb-sitemap.xml |
| sitemap | https://www.chrobinson.com/chrobinson-en-us-sitemap.xml |
| sitemap | https://www.chrobinson.com/chrobinson-es-ar-sitemap.xml |
| sitemap | https://www.chrobinson.com/chrobinson-es-es-sitemap.xml |
| sitemap | https://www.chrobinson.com/chrobinson-es-us-sitemap.xml |
| sitemap | https://www.chrobinson.com/chrobinson-fr-ca-sitemap.xml |
| sitemap | https://www.chrobinson.com/chrobinson-fr-fr-sitemap.xml |
| sitemap | https://www.chrobinson.com/chrobinson-it-it-sitemap.xml |
| sitemap | https://www.chrobinson.com/chrobinson-ko-kr-sitemap.xml |
| sitemap | https://www.chrobinson.com/chrobinson-pl-pl-sitemap.xml |
| sitemap | https://www.chrobinson.com/chrobinson-zh-tw-sitemap.xml |
| sitemap | https://www.chrobinson.com/chrobinson-zh-cn-sitemap.xml |
| sitemap | https://www.chrobinson.com/chrobinson-en-sg-sitemap.xml |