kennisbank.sparen.nl
robots.txt

Robots Exclusion Standard data for kennisbank.sparen.nl

Archived Snapshots

Resource Scan

Scan Details

Site Domain	kennisbank.sparen.nl
Base Domain	sparen.nl
Scan Status	Ok
Last Scan	2024-10-22T16:23:56+00:00
Next Scan	2024-11-21T16:23:56+00:00

Last Scan

Scanned	2024-10-22T16:23:56+00:00
URL	https://kennisbank.sparen.nl/robots.txt
Domain IPs	104.21.12.172, 172.67.152.213, 2606:4700:3030::6815:cac, 2606:4700:3030::ac43:98d5
Response IP	172.67.152.213
Found	Yes
Hash	a31ec80c15c1fc21e3846deb7f150f40271ca175253d38db0fa04c1430c1db08
SimHash	b21c65608cb0

Groups

*

Rule	Path
Disallow

Rule

Path

Disallow

rogerbot

Rule	Path
Disallow	/

Rule

Path

Disallow

ezooms

Rule	Path
Disallow	/

Rule

Path

Disallow

webcrawler

Rule	Path
Disallow	/

Rule

Path

Disallow

bubing

Rule	Path
Disallow	/

Rule

Path

Disallow

proximic

Rule	Path
Disallow	/

Rule

Path

Disallow

trendictionbot

Rule	Path
Disallow	/

Rule

Path

Disallow

vagabondo

Rule	Path
Disallow	/

Rule

Path

Disallow

bilbo

Rule	Path
Disallow	/

Rule

Path

Disallow

linkdexbot

Rule	Path
Disallow	/

Rule

Path

Disallow

exabot

Rule	Path
Disallow	/

Rule

Path

Disallow

slurp

Rule	Path
Disallow	/

Rule

Path

Disallow

niki-bot

Rule	Path
Disallow	/

Rule

Path

Disallow

compspybot

Rule	Path
Disallow	/

Rule

Path

Disallow

trendictionbot

Rule	Path
Disallow	/

Rule

Path

Disallow

semrushbot

Rule	Path
Disallow	/

Rule

Path

Disallow

beetlebot

Rule	Path
Disallow	/

Rule

Path

Disallow

mail.ru_bot

Rule	Path
Disallow	/

Rule

Path

Disallow

istellabot

Rule	Path
Disallow	/

Rule

Path

Disallow

plukkie

Rule	Path
Disallow	/

Rule

Path

Disallow

melodibot

Rule	Path
Disallow	/

Rule

Path

Disallow

spbot

Rule	Path
Disallow	/

Rule

Path

Disallow

queryseekerspider

Rule	Path
Disallow	/

Rule

Path

Disallow

buddhabot

Rule	Path
Disallow	/

Rule

Path

Disallow

imrbot

Rule	Path
Disallow	/

Rule

Path

Disallow

acoonbot

Rule	Path
Disallow	/

Rule

Path

Disallow

yacybot

Rule	Path
Disallow	/

Rule

Path

Disallow

gufybot

Rule	Path
Disallow	/

Rule

Path

Disallow

nesotebot

Rule	Path
Disallow	/

Rule

Path

Disallow

sogou web spider

Rule	Path
Disallow	/

Rule

Path

Disallow

seokicks-robot

Rule	Path
Disallow	/

Rule

Path

Disallow

solomonobot

Rule	Path
Disallow	/

Rule

Path

Disallow

wbsearchbot

Rule	Path
Disallow	/

Rule

Path

Disallow

noyabot

Rule	Path
Disallow	/

Rule

Path

Disallow

flappybot

Rule	Path
Disallow	/

Rule

Path

Disallow

siteexplorer

Rule	Path
Disallow	/

Rule

Path

Disallow

proximic

Rule	Path
Disallow	/

Rule

Path

Disallow

blexbot

Rule	Path
Disallow	/

Rule

Path

Disallow

crawler4j

Rule	Path
Disallow	/

Rule

Path

Disallow

nutch

Rule	Path
Disallow	/

Rule

Path

Disallow

berlin-fu-cow

Rule	Path
Disallow	/

Rule

Path

Disallow

riddler

Rule	Path
Disallow	/

Rule

Path

Disallow

python-requests

Rule	Path
Disallow	/

Rule

Path

Disallow

python-requests/2.8.1

Rule	Path
Disallow	/

Rule

Path

Disallow

Warnings

4 invalid lines.

kennisbank.sparen.nlrobots.txt

Resource Scan

Scan Details

Last Scan

Groups

*

rogerbot

ezooms

webcrawler

bubing

proximic

trendictionbot

vagabondo

bilbo

linkdexbot

exabot

slurp

niki-bot

compspybot

trendictionbot

semrushbot

beetlebot

mail.ru_bot

istellabot

plukkie

melodibot

spbot

queryseekerspider

buddhabot

imrbot

acoonbot

yacybot

gufybot

nesotebot

sogou web spider

seokicks-robot

solomonobot

wbsearchbot

noyabot

flappybot

siteexplorer

proximic

blexbot

crawler4j

nutch

berlin-fu-cow

riddler

python-requests

python-requests/2.8.1

Warnings

kennisbank.sparen.nl
robots.txt