support-en.sandisk.com
robots.txt

Robots Exclusion Standard data for support-en.sandisk.com

Resource Scan

Scan Details

Site Domain support-en.sandisk.com
Base Domain sandisk.com
Scan Status Ok
Last Scan2025-08-16T03:12:22+00:00
Next Scan 2025-09-15T03:12:22+00:00

Last Scan

Scanned2025-08-16T03:12:22+00:00
URL https://support-en.sandisk.com/robots.txt
Domain IPs 130.35.231.8
Response IP 130.35.231.8
Found Yes
Hash 1b1ff347ef03d0b75762f9ef564044cc34408c512db48dca8f22551a088aa28e
SimHash 6febda20d447

Groups

wdcbot

Product Comment
wdcbot Western Digital # CUSTOM
Rule Path Comment
Allow / CUSTOM

algolia crawler

Product Comment
algolia crawler CUSTOM
Rule Path Comment
Allow / CUSTOM

googlebot

Product Comment
googlebot Google # ADDED BY HMS
Rule Path Comment
Disallow ADDED BY HMS

msnbot

Product Comment
msnbot MSN # ADDED BY HMS
Rule Path Comment
Disallow ADDED BY HMS

Other Records

Field Value Comment
crawl-delay 0.2 ADDED BY HMS

slurp

Product Comment
slurp Yahoo! # ADDED BY HMS
Rule Path Comment
Disallow ADDED BY HMS

Other Records

Field Value Comment
crawl-delay 0.2 ADDED BY HMS

teoma

Product Comment
teoma Ask.com # ADDED BY HMS
Rule Path Comment
Disallow ADDED BY HMS

bingbot

Product Comment
bingbot Bing # ADDED BY HMS
Rule Path Comment
Disallow ADDED BY HMS

*

Product Comment
* ADDED BY HMS
Rule Path Comment
Disallow / ADDED BY HMS

Other Records

Field Value Comment
sitemap https://support-en.sandisk.com/ci/sitemap/ ADDED BY HMS

Comments

  • Algolia-Crawler-Verif: 9D7B80177C3BF811 # CUSTOM