lcbl.amegroups.com
robots.txt

Robots Exclusion Standard data for lcbl.amegroups.com

Resource Scan

Scan Details

Site Domain lcbl.amegroups.com
Base Domain amegroups.com
Scan Status Ok
Last Scan2024-10-08T13:20:12+00:00
Next Scan 2024-11-07T13:20:12+00:00

Last Scan

Scanned2024-10-08T13:20:12+00:00
URL https://lcbl.amegroups.com/robots.txt
Domain IPs 47.74.40.41
Response IP 47.74.40.41
Found Yes
Hash 3d959c1aea15236c8f9b690d9435de2d73e95b173d1627538c500bd80ae4710b
SimHash 885457538771

Groups

dotbot

Rule Path
Disallow /

velenpublicwebcrawler

Rule Path
Disallow /

mozilla/4.0 (compatible; msie 9.0; windows nt 6.1

Rule Path
Disallow /

*

Rule Path
Disallow /js/
Disallow /styles/
Disallow /css/
Disallow /statics/
Disallow /template
Disallow /favicon.ico
Allow /search/authors
Allow /*/search/authors
Disallow /tools
Disallow /user
Disallow /author
Disallow /manager
Disallow /editor
Disallow /content
Disallow /reviewer

Comments

  • -----------------------------------------------------------------------------
  • 禁止爬虫爬取无效URL,提升网站核心静态资源抓取及索引效率。
  • 无效URLåŒ
  • 等各种无需被SE收录的URL。
  • -----------------------------------------------------------------------------
  • ojs

Warnings

  • 1 invalid line.
  • `含` is not a known field.