sparen.nl
robots.txt

Robots Exclusion Standard data for sparen.nl

Resource Scan

Scan Details

Site Domain sparen.nl
Base Domain sparen.nl
Scan Status Ok
Last Scan2024-09-27T04:55:06+00:00
Next Scan 2024-10-04T04:55:06+00:00

Last Scan

Scanned2024-09-27T04:55:06+00:00
URL https://sparen.nl/robots.txt
Redirect https://www.sparen.nl/robots.txt
Redirect Domain www.sparen.nl
Redirect Base sparen.nl
Domain IPs 104.21.12.172, 172.67.152.213, 2606:4700:3030::6815:cac, 2606:4700:3030::ac43:98d5
Redirect IPs 104.21.12.172, 172.67.152.213, 2606:4700:3030::6815:cac, 2606:4700:3030::ac43:98d5
Response IP 172.67.152.213
Found Yes
Hash f80e05c50931c1d3d80edd12a61129e97568c127807bbe6338a1e1620b9c14c3
SimHash b21c6f6488a2

Groups

*

Rule Path
Disallow /naar-spaarbank
Disallow /vergelijken
Disallow /vergelijken/
Disallow /static/
Disallow /javascript%3Avoid
Disallow /javascript%3A__dopostback

rogerbot

Rule Path
Disallow /

ezooms

Rule Path
Disallow /

webcrawler

Rule Path
Disallow /

bubing

Rule Path
Disallow /

proximic

Rule Path
Disallow /

trendictionbot

Rule Path
Disallow /

vagabondo

Rule Path
Disallow /

bilbo

Rule Path
Disallow /

linkdexbot

Rule Path
Disallow /

exabot

Rule Path
Disallow /

slurp

Rule Path
Disallow /

niki-bot

Rule Path
Disallow /

compspybot

Rule Path
Disallow /

trendictionbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

beetlebot

Rule Path
Disallow /

mail.ru_bot

Rule Path
Disallow /

istellabot

Rule Path
Disallow /

plukkie

Rule Path
Disallow /

melodibot

Rule Path
Disallow /

spbot

Rule Path
Disallow /

queryseekerspider

Rule Path
Disallow /

buddhabot

Rule Path
Disallow /

imrbot

Rule Path
Disallow /

acoonbot

Rule Path
Disallow /

yacybot

Rule Path
Disallow /

gufybot

Rule Path
Disallow /

nesotebot

Rule Path
Disallow /

sogou web spider

Rule Path
Disallow /

seokicks-robot

Rule Path
Disallow /

solomonobot

Rule Path
Disallow /

wbsearchbot

Rule Path
Disallow /

noyabot

Rule Path
Disallow /

flappybot

Rule Path
Disallow /

siteexplorer

Rule Path
Disallow /

proximic

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

crawler4j

Rule Path
Disallow /

nutch

Rule Path
Disallow /

berlin-fu-cow

Rule Path
Disallow /

riddler

Rule Path
Disallow /

python-requests

Rule Path
Disallow /

python-requests/2.8.1

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.sparen.nl/sitemap.xml

Warnings

  • 4 invalid lines.