rente.nl
robots.txt

Robots Exclusion Standard data for rente.nl

Resource Scan

Scan Details

Site Domain rente.nl
Base Domain rente.nl
Scan Status Ok
Last Scan2024-09-24T22:34:16+00:00
Next Scan 2024-10-24T22:34:16+00:00

Last Scan

Scanned2024-09-24T22:34:16+00:00
URL https://rente.nl/robots.txt
Redirect https://www.rente.nl/robots.txt
Redirect Domain www.rente.nl
Redirect Base rente.nl
Domain IPs 104.21.42.171, 172.67.163.180, 2606:4700:3034::ac43:a3b4, 2606:4700:3037::6815:2aab
Redirect IPs 104.21.42.171, 172.67.163.180, 2606:4700:3034::ac43:a3b4, 2606:4700:3037::6815:2aab
Response IP 104.21.42.171
Found Yes
Hash 215219bb0c21387a9f6997a6ae3ec4a9294764fb8fc51430c0aec597bb455b22
SimHash b2986d668893

Groups

*

Rule Path
Disallow /lenen/vergelijken
Disallow /lenen/naar-kredietverstrekker
Disallow /lenen/aanvraag-gegevens
Disallow /lenen/aanvraag-bedankt
Disallow /hypotheekrente/vergelijken
Disallow /hypotheekrente/uitgebreid_vergelijken
Disallow /hypotheekrente/aanvragen
Disallow /hypotheekrente/aanvraag-bedankt
Disallow /static/
Disallow /javascript%3Avoid
Disallow /javascript%3A__dopostback
Disallow /assets/js/
Disallow /assets/fonts/
Disallow /root/static/

rogerbot

Rule Path
Disallow /

ezooms

Rule Path
Disallow /

webcrawler

Rule Path
Disallow /

bubing

Rule Path
Disallow /

proximic

Rule Path
Disallow /

trendictionbot

Rule Path
Disallow /

vagabondo

Rule Path
Disallow /

bilbo

Rule Path
Disallow /

linkdexbot

Rule Path
Disallow /

exabot

Rule Path
Disallow /

slurp

Rule Path
Disallow /

niki-bot

Rule Path
Disallow /

compspybot

Rule Path
Disallow /

trendictionbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

beetlebot

Rule Path
Disallow /

mail.ru_bot

Rule Path
Disallow /

istellabot

Rule Path
Disallow /

plukkie

Rule Path
Disallow /

melodibot

Rule Path
Disallow /

spbot

Rule Path
Disallow /

queryseekerspider

Rule Path
Disallow /

buddhabot

Rule Path
Disallow /

imrbot

Rule Path
Disallow /

acoonbot

Rule Path
Disallow /

yacybot

Rule Path
Disallow /

gufybot

Rule Path
Disallow /

nesotebot

Rule Path
Disallow /

sogou web spider

Rule Path
Disallow /

seokicks-robot

Rule Path
Disallow /

solomonobot

Rule Path
Disallow /

wbsearchbot

Rule Path
Disallow /

noyabot

Rule Path
Disallow /

flappybot

Rule Path
Disallow /

siteexplorer

Rule Path
Disallow /

proximic

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

crawler4j

Rule Path
Disallow /

nutch

Rule Path
Disallow /

berlin-fu-cow

Rule Path
Disallow /

riddler

Rule Path
Disallow /

python-requests

Rule Path
Disallow /

python-requests/2.8.1

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.rente.nl/sitemap.xml
sitemap https://www.rente.nl/sitemap_images.xml

Warnings

  • 4 invalid lines.