us.caudalie.com
robots.txt
Robots Exclusion Standard data for us.caudalie.com
Resource Scan
Scan Details
| Site Domain | us.caudalie.com |
| Base Domain | caudalie.com |
| Scan Status | Ok |
| Last Scan | 2025-03-28T06:16:55+00:00 |
| Next Scan | 2025-04-11T06:16:55+00:00 |
Last Scan
| Scanned | 2025-03-28T06:16:55+00:00 |
| URL | https://us.caudalie.com/robots.txt |
| Domain IPs | 2600:9000:21e1:4600:5:7978:35c0:93a1, 2600:9000:21e1:5c00:5:7978:35c0:93a1, 2600:9000:21e1:9c00:5:7978:35c0:93a1, 2600:9000:21e1:b400:5:7978:35c0:93a1, 2600:9000:21e1:b800:5:7978:35c0:93a1, 2600:9000:21e1:c800:5:7978:35c0:93a1, 2600:9000:21e1:d000:5:7978:35c0:93a1, 2600:9000:21e1:d600:5:7978:35c0:93a1, 3.170.229.124, 3.170.229.38, 3.170.229.54, 3.170.229.76 |
| Response IP | 3.170.229.124 |
| Found | Yes |
| Hash | 16e501ee32952ccf82470c050882bb3d751e74cdf7dbbf82836d36a892fae128 |
| SimHash | ef001c54c331 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /*.css$ |
| Disallow | /*.js$ |
| Disallow | /*.php$ |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 5 |
Other Records
| Field | Value |
|---|---|
| sitemap | https://us.caudalie.com/sitemap.xml |