linkshop.com
robots.txt

Robots Exclusion Standard data for linkshop.com

Resource Scan

Scan Details

Site Domain linkshop.com
Base Domain linkshop.com
Scan Status Ok
Last Scan2024-10-02T13:37:38+00:00
Next Scan 2024-11-01T13:37:38+00:00

Last Scan

Scanned2024-10-02T13:37:38+00:00
URL https://www.linkshop.com/robots.txt
Redirect http://www.linkshop.com/robots.txt
Domain IPs 180.163.140.218, 2408:8766:1:1500:3::7f5
Response IP 180.163.141.217
Found Yes
Hash 5137acca9572c3930633d43df7e5be1c602e6c74c82b7c517d5a230080b253ce
SimHash 3e449e4d4676

Groups

baiduspider

Rule Path
Disallow /web/article_news.aspx
Disallow /web/trainList.aspx
Disallow /web/trainCal.aspx
Disallow /members/Read.aspx
Disallow /members/shopping.aspx

slurp

Rule Path
Disallow /job/
Disallow /club/dispbbs.aspx

sogou

Rule Path
Disallow /web/search.aspx

*

Rule Path
Disallow /web/article_news.aspx
Disallow /gif.aspx
Disallow /admin/
Disallow /club/admin/
Disallow /club/adg.aspx
Disallow /club/changeAnn.aspx
Disallow /club/countjs.aspx
Disallow /club/getipcityjs.aspx
Disallow /club/GroupBbsann.aspx
Disallow /club/GroupInvite.aspx
Disallow /club/loginchk.aspx
Disallow /club/login.aspx
Disallow /club/manbbs.aspx
Disallow /club/reannounce.aspx
Disallow /club/search.aspx
Disallow /club/twiter.aspx
Disallow /club/twitershow.aspx
Disallow /club/twiterlist.aspx
Disallow /club/userblog.aspx
Disallow /club/userfriend.aspx
Disallow /club/usersms.aspx
Disallow /fav/
Disallow /job/person/index.aspx
Disallow /job/person/personlist.aspx
Disallow /job/webstat.aspx
Disallow /job/admin/
Disallow /web/Article_Count.aspx
Disallow /web/Article_login.aspx
Disallow /web/createSmallImage.aspx
Disallow /web/get_linkshopAds.aspx
Disallow /members/Read.aspx
Disallow /members/shopping.aspx

Other Records

Field Value
sitemap http://www.linkshop.com/sitemap.xml

Comments

  • robots.txt for http://www.linkshop.com/
  • delay 2 second
  • Crawl-delay: 2