westerndigital.com
robots.txt
Robots Exclusion Standard data for westerndigital.com
Resource Scan
Scan Details
| Site Domain | westerndigital.com |
| Base Domain | westerndigital.com |
| Scan Status | Ok |
| Last Scan | 2024-10-18T21:39:56+00:00 |
| Next Scan | 2024-11-17T21:39:56+00:00 |
Last Scan
| Scanned | 2024-10-18T21:39:56+00:00 |
| URL | https://www.westerndigital.com/robots.txt |
| Domain IPs | 23.50.95.10 |
| Response IP | 23.50.95.10 |
| Found | Yes |
| Hash | 1b54b74ba8a0395ca098423bc02d5e1a5c0224549df2446c96aabbb129b44adb |
| SimHash | ed019970c515 |
Groups
wdcbot
| Product | Comment |
|---|---|
| wdcbot | Western Digital |
| Rule | Path |
|---|---|
| Disallow | /content/dam/ |
| Allow | /content/dam/store/ |
| Allow | /content/dam/western-digital/ |
| Disallow | /etc/ |
| Allow | /etc/designs/ |
| Allow | /etc/clientlibs/ |
| Disallow | /libs/ |
*
| Rule | Path |
|---|---|
| Disallow | /content/dam/ |
| Disallow | /etc/ |
| Disallow | /libs/ |
| Disallow | /etc/ |
| Allow | /content/dam/store/ |
| Allow | /content/dam/western-digital/ |
| Allow | /etc/designs/ |
| Allow | /etc/clientlibs/ |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.westerndigital.com/sitemap-index.xml |