tw.taobao.com
robots.txt
Robots Exclusion Standard data for tw.taobao.com
Resource Scan
Scan Details
Site Domain | tw.taobao.com |
Base Domain | taobao.com |
Scan Status | Ok |
Last Scan | 2024-06-02T00:28:10+00:00 |
Next Scan | 2024-07-02T00:28:10+00:00 |
Last Scan
Scanned | 2024-06-02T00:28:10+00:00 |
URL | https://tw.taobao.com/robots.txt |
Redirect | https://world.taobao.com/robots.txt |
Redirect Domain | world.taobao.com |
Redirect Base | taobao.com |
Domain IPs | 163.181.160.213, 163.181.160.214 |
Redirect IPs | 163.181.160.213, 163.181.160.214 |
Response IP | 163.181.160.214 |
Found | Yes |
Hash | 66947f4d581f73e854c8c4227f3689dc9587529c6f525c64631d2d59c1d431b6 |
SimHash | c416f750f329 |
Groups
baiduspider
Rule | Path |
---|---|
Disallow | /search/ |
Disallow | /cart/ |
Disallow | /login/ |
Disallow | /reg/ |
Disallow | /buy/ |
Disallow | /plus/ |
Disallow | /category/ |
Disallow | /product/ |
Disallow | /item/ |
Disallow | /product-amp/ |
Disallow | /category-amp/ |
Disallow | /item-amp/ |
Disallow | /dxxxxxxxx/ |
Disallow | /spu/ |
Disallow | /spu-amp/ |
Allow | / |
googlebot
Rule | Path |
---|---|
Disallow | /search/ |
Disallow | /cart/ |
Disallow | /login/ |
Disallow | /reg/ |
Disallow | /buy/ |
Disallow | /plus/ |
Disallow | /spu/ |
Disallow | /spu-amp/ |
Allow | /product/ |
Allow | /category/ |
Allow | /item/ |
Allow | /product-amp/ |
Allow | /category-amp/ |
Allow | /item-amp/ |
Allow | /dianpu/ |
Allow | /dianpu-amp/ |
Allow | / |
bingbot
Rule | Path |
---|---|
Disallow | /search/ |
Disallow | /cart/ |
Disallow | /login/ |
Disallow | /reg/ |
Disallow | /buy/ |
Disallow | /plus/ |
Disallow | /spu/ |
Disallow | /spu-amp/ |
Allow | /product/ |
Allow | /category/ |
Allow | /item/ |
Allow | /product-amp/ |
Allow | /category-amp/ |
Allow | /item-amp/ |
Allow | /dianpu/ |
Allow | /dianpu-amp/ |
Allow | / |
yahoo! slurp
Rule | Path |
---|---|
Disallow | /search/ |
Disallow | /cart/ |
Disallow | /login/ |
Disallow | /reg/ |
Disallow | /buy/ |
Disallow | /plus/ |
Disallow | /spu/ |
Disallow | /spu-amp/ |
Allow | /product/ |
Allow | /category/ |
Allow | /item/ |
Allow | /product-amp/ |
Allow | /category-amp/ |
Allow | /item-amp/ |
Allow | /spu/ |
Allow | /spu-amp/ |
Allow | /dianpu/ |
Allow | /dianpu-amp/ |
Allow | / |
pinterestbot
Rule | Path |
---|---|
Disallow | /search/ |
Disallow | /cart/ |
Disallow | /login/ |
Disallow | /reg/ |
Disallow | /buy/ |
Disallow | /plus/ |
Disallow | /spu/ |
Disallow | /spu-amp/ |
Allow | /product/ |
Allow | /category/ |
Allow | /item/ |
Allow | /product-amp/ |
Allow | /category-amp/ |
Allow | /item-amp/ |
Allow | /dianpu/ |
Allow | /dianpu-amp/ |
Allow | / |
applebot
Rule | Path |
---|---|
Disallow | /search/ |
Disallow | /cart/ |
Disallow | /login/ |
Disallow | /reg/ |
Disallow | /buy/ |
Disallow | /plus/ |
Disallow | /spu/ |
Disallow | /spu-amp/ |
Allow | /product/ |
Allow | /category/ |
Allow | /item/ |
Allow | /product-amp/ |
Allow | /category-amp/ |
Allow | /item-amp/ |
Allow | /dianpu/ |
Allow | /dianpu-amp/ |
Allow | / |
google-extended
Rule | Path |
---|---|
Disallow | /search/ |
Disallow | /cart/ |
Disallow | /login/ |
Disallow | /reg/ |
Disallow | /buy/ |
Disallow | /plus/ |
Disallow | /spu/ |
Disallow | /spu-amp/ |
Allow | /product/ |
Allow | /category/ |
Allow | /item/ |
Allow | /product-amp/ |
Allow | /category-amp/ |
Allow | /item-amp/ |
Allow | /dianpu/ |
Allow | /dianpu-amp/ |
Allow | / |
yisouspider
Rule | Path |
---|---|
Disallow | /search/ |
Disallow | /cart/ |
Disallow | /login/ |
Disallow | /reg/ |
Disallow | /buy/ |
Disallow | /plus/ |
Disallow | /spu/ |
Disallow | /spu-amp/ |
Allow | /product/ |
Allow | /category/ |
Allow | /item/ |
Allow | /product-amp/ |
Allow | /category-amp/ |
Allow | /item-amp/ |
Allow | /dianpu/ |
Allow | /dianpu-amp/ |
Allow | / |
yandex
Rule | Path |
---|---|
Disallow | /search/ |
Disallow | /cart/ |
Disallow | /login/ |
Disallow | /reg/ |
Disallow | /buy/ |
Disallow | /plus/ |
Disallow | /spu/ |
Disallow | /spu-amp/ |
Allow | /product/ |
Allow | /category/ |
Allow | /item/ |
Allow | /product-amp/ |
Allow | /category-amp/ |
Allow | /item-amp/ |
Allow | /dianpu/ |
Allow | /dianpu-amp/ |
Allow | / |
yeti
Rule | Path |
---|---|
Disallow | /search/ |
Disallow | /cart/ |
Disallow | /login/ |
Disallow | /reg/ |
Disallow | /buy/ |
Disallow | /plus/ |
Disallow | /spu/ |
Disallow | /spu-amp/ |
Allow | /product/ |
Allow | /category/ |
Allow | /item/ |
Allow | /product-amp/ |
Allow | /category-amp/ |
Allow | /item-amp/ |
Allow | /dianpu/ |
Allow | /dianpu-amp/ |
Allow | / |
*
Rule | Path |
---|---|
Disallow | / |