Log In Sign Up

leerwiki.nl
robots.txt

Robots Exclusion Standard data for leerwiki.nl

Resource Scan

Scan Details

Site Domain leerwiki.nl
Base Domain leerwiki.nl
Scan Status Ok
Last Scan2024-09-20T14:58:00+00:00
Next Scan 2024-09-27T14:58:00+00:00

Last Scan

Scanned2024-09-20T14:58:00+00:00
URL https://leerwiki.nl/robots.txt
Redirect https://www.leerwiki.nl/robots.txt
Redirect Domain www.leerwiki.nl
Redirect Base leerwiki.nl
Domain IPs 37.97.149.102
Redirect IPs 37.97.149.102
Response IP 37.97.149.102
Found Yes
Hash 50a9c065294c7578252e22b43a9731a662d092fa147e2339c70194babc57067b
SimHash 4155e7417618

Groups

googlebot-image
googlebot

Rule Path
Allow /images/*
Allow /uploads/*

*

Rule Path
Disallow /images/*
Disallow /uploads/*

adsbot*
baiduspider
barkrowler
blexbot
bingbot
dotbot
mail.ru_bot
mail.ru
megaindex.ru
megaindex.ru/2.0
mj12bot
petalbot
seekport crawler
semrushbot
serpstatbot
sogou spider
yandexbot

Rule Path
Disallow /

vagabondo

Rule Path
Disallow /
Back to top

Comments

  • No images
  • bots
  • wise guys scanning ancient pages Mozilla/4.0 (compatible; Vagabondo/4.0; http://webagent.wise-guys.nl/; http://www.wise-guys.nl/)
Back to top