spaarrente.nl
robots.txt

Robots Exclusion Standard data for spaarrente.nl

Resource Scan

Scan Details

Site Domain spaarrente.nl
Base Domain spaarrente.nl
Scan Status Ok
Last Scan2024-09-22T19:19:29+00:00
Next Scan 2024-09-29T19:19:29+00:00

Last Scan

Scanned2024-09-22T19:19:29+00:00
URL https://spaarrente.nl/robots.txt
Redirect https://www.spaarrente.nl/robots.txt
Redirect Domain www.spaarrente.nl
Redirect Base spaarrente.nl
Domain IPs 2001:16e8:0:211::104, 62.148.167.233
Redirect IPs 2001:16e8:0:211::104, 62.148.167.233
Response IP 62.148.167.233
Found Yes
Hash 30b17d2218efc03560736a1a2f8212497ae528ba59d980b4b819440f515319a3
SimHash b29c67448c90

Groups

*

Rule Path
Disallow /naar-spaarbank
Disallow /vergelijken
Disallow /static/
Disallow /javascript%3Avoid
Disallow /javascript%3A__dopostback
Disallow /1328198/

rogerbot

Rule Path
Disallow /

ezooms

Rule Path
Disallow /

webcrawler

Rule Path
Disallow /

bubing

Rule Path
Disallow /

proximic

Rule Path
Disallow /

trendictionbot

Rule Path
Disallow /

vagabondo

Rule Path
Disallow /

bilbo

Rule Path
Disallow /

linkdexbot

Rule Path
Disallow /

exabot

Rule Path
Disallow /

slurp

Rule Path
Disallow /

niki-bot

Rule Path
Disallow /

compspybot

Rule Path
Disallow /

trendictionbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

beetlebot

Rule Path
Disallow /

mail.ru_bot

Rule Path
Disallow /

istellabot

Rule Path
Disallow /

plukkie

Rule Path
Disallow /

melodibot

Rule Path
Disallow /

spbot

Rule Path
Disallow /

queryseekerspider

Rule Path
Disallow /

buddhabot

Rule Path
Disallow /

imrbot

Rule Path
Disallow /

acoonbot

Rule Path
Disallow /

yacybot

Rule Path
Disallow /

gufybot

Rule Path
Disallow /

nesotebot

Rule Path
Disallow /

sogou web spider

Rule Path
Disallow /

seokicks-robot

Rule Path
Disallow /

solomonobot

Rule Path
Disallow /

wbsearchbot

Rule Path
Disallow /

noyabot

Rule Path
Disallow /

flappybot

Rule Path
Disallow /

siteexplorer

Rule Path
Disallow /

proximic

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

crawler4j

Rule Path
Disallow /

nutch

Rule Path
Disallow /

berlin-fu-cow

Rule Path
Disallow /

riddler

Rule Path
Disallow /

python-requests

Rule Path
Disallow /

python-requests/2.8.1

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.spaarrente.nl/sitemap.xml

Warnings

  • 4 invalid lines.