m.shopclues.com
robots.txt

Robots Exclusion Standard data for m.shopclues.com

Resource Scan

Scan Details

Site Domain m.shopclues.com
Base Domain shopclues.com
Scan Status Ok
Last Scan2024-10-04T17:08:03+00:00
Next Scan 2024-10-18T17:08:03+00:00

Last Scan

Scanned2024-10-04T17:08:03+00:00
URL https://m.shopclues.com/robots.txt
Domain IPs 3.6.122.117, 3.6.173.29, 43.205.25.34
Response IP 43.205.25.34
Found Yes
Hash 680163f8681c617b82958ed6f0771b48190669e269fdec6a5c19237fe6e50882
SimHash e6c6d6428822

Groups

*

Rule Path
Disallow /skins/
Disallow /payments/
Disallow /store_closed.html
Disallow /core/
Disallow /lib/
Disallow /install/
Disallow /schemas/
Disallow /welcome-kit-3.html
Disallow /similar_products/*
Disallow /deleted-categories-27/*

baiduspider

Rule Path
Disallow /

baiduspider+

Rule Path
Disallow /

baiduspider+(+http://www.baidu.com/search/spider.htm)

Rule Path
Disallow /

baiduspider/2.0;+http://www.baidu.com/search/spider.html

Rule Path
Disallow /

baiduspider/2.0

Rule Path
Disallow /

mozilla/5.0 (compatible; baiduspider/2.0; +http://www.baidu.com/search/spider.html)

Rule Path
Disallow /

spider

Rule Path
Disallow /

yandex

Rule Path
Disallow /

yandex.com

Rule Path
Disallow /

yandexbot/3.0

Rule Path
Disallow /

sosospider+

Rule Path
Disallow /

ezooms

Rule Path
Disallow /

ezooms/1.0

Rule Path
Disallow /

mozilla/5.0 (compatible; ezooms/1.0; ezooms.bot[at]gmail[dot]com)

Rule Path
Disallow /

sogou spider

Rule Path
Disallow /

wotbox/2.01

Rule Path
Disallow /

dwbot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.shopclues.com/sitemap.xml
sitemap https://www.shopclues.com/blog/sitemap_index.xml

Comments

  • Baiduspider
  • Yandex
  • Sosospider
  • Ezooms
  • Sogou
  • 80legs.com

Warnings

  • 8 invalid lines.