Log In Sign Up

charfriend.com
robots.txt

Robots Exclusion Standard data for charfriend.com

Resource Scan

Scan Details

Site Domain charfriend.com
Base Domain charfriend.com
Scan Status Ok
Last Scan2024-11-12T17:51:13+00:00
Next Scan 2024-11-19T17:51:13+00:00

Last Scan

Scanned2024-11-12T17:51:13+00:00
URL https://charfriend.com/robots.txt
Domain IPs 34.96.185.251
Response IP 34.96.185.251
Found Yes
Hash 4b8bc10b4d4a7f769d682875511c62fb34efecea2c1d0e2b2947e9646aa9335a
SimHash 8745dd36650b

Groups

baiduspider

Rule Path
Disallow /

baiduspider-image

Rule Path
Disallow /

youdaobot

Rule Path
Disallow /

sogou web spider

Rule Path
Disallow /

sogou inst spider

Rule Path
Disallow /

sogou spider2

Rule Path
Disallow /

sogou blog

Rule Path
Disallow /

sogou news spider

Rule Path
Disallow /

sogou orion spider

Rule Path
Disallow /
Back to top

Comments

  • User-agent: bingbot
  • Disallow: /
  • User-agent: Googlebot
  • User-agent: MSNBot
  • User-agent: ChinasoSpider
  • User-agent: Sosospider
  • User-agent: yisouspider
  • User-agent: EasouSpider
  • User-agent: DuckDuckBot
  • User-agent: archive.org_bot
  • User-agent: crawling
Back to top