halseytaylor.com
robots.txt
Robots Exclusion Standard data for halseytaylor.com
Resource Scan
Scan Details
| Site Domain | halseytaylor.com |
| Base Domain | halseytaylor.com |
| Scan Status | Ok |
| Last Scan | 2024-10-15T05:43:17+00:00 |
| Next Scan | 2024-11-14T05:43:17+00:00 |
Last Scan
| Scanned | 2024-10-15T05:43:17+00:00 |
| URL | https://www.halseytaylor.com/robots.txt |
| Domain IPs | 2600:9000:2024:200:18:e330:1dc0:93a1, 2600:9000:2024:4600:18:e330:1dc0:93a1, 2600:9000:2024:5400:18:e330:1dc0:93a1, 2600:9000:2024:7c00:18:e330:1dc0:93a1, 2600:9000:2024:c800:18:e330:1dc0:93a1, 2600:9000:2024:cc00:18:e330:1dc0:93a1, 2600:9000:2024:ea00:18:e330:1dc0:93a1, 2600:9000:2024:f600:18:e330:1dc0:93a1, 65.9.112.102, 65.9.112.15, 65.9.112.29, 65.9.112.85 |
| Response IP | 18.165.140.17 |
| Found | Yes |
| Hash | 9c7245fd3aec227e3867e140a31eb86b0564ed918676fa794a79eec029eaf3c0 |
| SimHash | 4c0c9c70c782 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /us/en/components-sampler/ |
| Disallow | /us/en/components-sampler.html |
| Disallow | /us/en/shadow-nav-home/ |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 10 |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.halseytaylor.com/sitemap.xml |