donnons.be
robots.txt

Robots Exclusion Standard data for donnons.be

Resource Scan

Scan Details

Site Domain donnons.be
Base Domain donnons.be
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-09-13T08:15:37+00:00
Next Scan 2024-12-12T08:15:37+00:00

Last Successful Scan

Scanned2024-05-17T07:00:29+00:00
URL http://donnons.be/robots.txt
Redirect https://donnons.org/robots.txt
Redirect Domain donnons.org
Redirect Base donnons.org
Domain IPs 176.31.53.248
Redirect IPs 176.31.53.248
Response IP 176.31.53.248
Found Yes
Hash e9bf403d26a347be29a203d1b915b6756b4ec03698450c19b1fc9e07e452769f
SimHash 2a74d3c2d1d0

Groups

google
googlebot
googlebot-news
googlebot-image
googlebot-video
googlebot-mobile
mediapartners-google
mediapartners
adsbot-google
adsbot-google-mobile-apps
bingbot
msnbot
msnbot-media
adidxbot
bingpreview
slurp
orangebot
duckduckbot
facebookexternalhit
pinterest
twitterbot

Rule Path
Disallow
Disallow /weepeggle

yandex

Rule Path
Disallow /

moget
ichiro

Rule Path
Disallow /

naverbot
yeti

Rule Path
Disallow /

baiduspider
baiduspider-video
baiduspider-image

Rule Path
Disallow /

sogou spider

Rule Path
Disallow /

youdaobot

Rule Path
Disallow /

grapeshot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

exabot

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

Comments

  • ------------------------------------------------------------------------------
  • Whitelist
  • ------------------------------------------------------------------------------
  • Googlebot
  • Bing
  • Yahoo
  • Orange
  • DuckDuckGo
  • Social network
  • ------------------------------------------------------------------------------
  • Blacklist
  • ------------------------------------------------------------------------------
  • Yandex
  • Goo
  • Naver
  • Baidu
  • SoGou
  • Youdao
  • GrapeshotCrawler
  • Majestic-12
  • Exabot
  • Alexa