kennisbank.sparen.nl
robots.txt

Robots Exclusion Standard data for kennisbank.sparen.nl

Resource Scan

Scan Details

Site Domain kennisbank.sparen.nl
Base Domain sparen.nl
Scan Status Ok
Last Scan2024-10-22T16:23:56+00:00
Next Scan 2024-11-21T16:23:56+00:00

Last Scan

Scanned2024-10-22T16:23:56+00:00
URL https://kennisbank.sparen.nl/robots.txt
Domain IPs 104.21.12.172, 172.67.152.213, 2606:4700:3030::6815:cac, 2606:4700:3030::ac43:98d5
Response IP 172.67.152.213
Found Yes
Hash a31ec80c15c1fc21e3846deb7f150f40271ca175253d38db0fa04c1430c1db08
SimHash b21c65608cb0

Groups

*

Rule Path
Disallow

rogerbot

Rule Path
Disallow /

ezooms

Rule Path
Disallow /

webcrawler

Rule Path
Disallow /

bubing

Rule Path
Disallow /

proximic

Rule Path
Disallow /

trendictionbot

Rule Path
Disallow /

vagabondo

Rule Path
Disallow /

bilbo

Rule Path
Disallow /

linkdexbot

Rule Path
Disallow /

exabot

Rule Path
Disallow /

slurp

Rule Path
Disallow /

niki-bot

Rule Path
Disallow /

compspybot

Rule Path
Disallow /

trendictionbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

beetlebot

Rule Path
Disallow /

mail.ru_bot

Rule Path
Disallow /

istellabot

Rule Path
Disallow /

plukkie

Rule Path
Disallow /

melodibot

Rule Path
Disallow /

spbot

Rule Path
Disallow /

queryseekerspider

Rule Path
Disallow /

buddhabot

Rule Path
Disallow /

imrbot

Rule Path
Disallow /

acoonbot

Rule Path
Disallow /

yacybot

Rule Path
Disallow /

gufybot

Rule Path
Disallow /

nesotebot

Rule Path
Disallow /

sogou web spider

Rule Path
Disallow /

seokicks-robot

Rule Path
Disallow /

solomonobot

Rule Path
Disallow /

wbsearchbot

Rule Path
Disallow /

noyabot

Rule Path
Disallow /

flappybot

Rule Path
Disallow /

siteexplorer

Rule Path
Disallow /

proximic

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

crawler4j

Rule Path
Disallow /

nutch

Rule Path
Disallow /

berlin-fu-cow

Rule Path
Disallow /

riddler

Rule Path
Disallow /

python-requests

Rule Path
Disallow /

python-requests/2.8.1

Rule Path
Disallow /

Warnings

  • 4 invalid lines.