dl-xinde.com
robots.txt

Robots Exclusion Standard data for dl-xinde.com

Resource Scan

Scan Details

Site Domain dl-xinde.com
Base Domain dl-xinde.com
Scan Status Ok
Last Scan2025-07-17T06:46:50+00:00
Next Scan 2025-07-31T06:46:50+00:00

Last Scan

Scanned2025-07-17T06:46:50+00:00
URL http://dl-xinde.com/robots.txt
Domain IPs 107.174.51.89
Response IP 107.174.51.89
Found Yes
Hash a07f3be2ca4023c7950b4c741dfb3bc17d2739655efc4918a3786e4307821216
SimHash e83451fa831a

Groups

*

Rule Path
Disallow /

baiduspider

Rule Path
Disallow

yisouspider

Rule Path
Disallow

googlebot

Rule Path
Disallow /

bingbot

Rule Path
Disallow /

sogou web spider

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

yandex

Rule Path
Disallow /

Comments

  • 默认禁止所有爬虫访问
  • å
  • å
  • 屏蔽谷歌爬虫
  • 屏蔽Bing爬虫
  • 屏蔽搜狗爬虫
  • 屏蔽360搜索爬虫
  • 屏蔽Ahrefs爬虫(站点分析工å
  • 屏蔽Semrush爬虫
  • 屏蔽MJ12bot(SEOå·¥å
  • 屏蔽Yandex(俄罗斯搜索引擎)

Warnings

  • 8 invalid lines.