Log In Sign Up

cdn.amegroups.cn
robots.txt

Robots Exclusion Standard data for cdn.amegroups.cn

Resource Scan

Scan Details

Site Domain cdn.amegroups.cn
Base Domain amegroups.cn
Scan Status Ok
Last Scan2024-06-01T03:24:05+00:00
Next Scan 2024-07-01T03:24:05+00:00

Last Scan

Scanned2024-06-01T03:24:05+00:00
URL https://cdn.amegroups.cn/robots.txt
Domain IPs 163.181.160.248
Response IP 163.181.160.248
Found Yes
Hash afafce968aedeae07db3028cdeb59931f0db27eee602c1132f399cbbd416220b
SimHash 904455528378

Groups

dotbot

Rule Path
Disallow /

velenpublicwebcrawler

Rule Path
Disallow /

mozilla/4.0 (compatible; msie 9.0; windows nt 6.1

Rule Path
Disallow /

*

Rule Path
Disallow /js/
Disallow /styles/
Disallow /css/
Disallow /statics/
Disallow /vendor/
Disallow /template
Disallow /favicon.ico
Back to top

Comments

  • -----------------------------------------------------------------------------
  • 禁止爬虫爬取无效URL，提升网站核心静态资源抓取及索引效率。
  • 无效URLåŒ
  • 等各种无需被SE收录的URL。
  • -----------------------------------------------------------------------------
Back to top

Warnings

  • 1 invalid line.
  • `含` is not a known field.
Back to top