poliswijzer.nl
robots.txt

Robots Exclusion Standard data for poliswijzer.nl

Resource Scan

Scan Details

Site Domain poliswijzer.nl
Base Domain poliswijzer.nl
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-09-02T13:31:54+00:00
Next Scan 2024-12-01T13:31:54+00:00

Last Successful Scan

Scanned2024-04-13T11:25:25+00:00
URL https://poliswijzer.nl/robots.txt
Redirect https://www.poliswijzer.nl/robots.txt
Redirect Domain www.poliswijzer.nl
Redirect Base poliswijzer.nl
Domain IPs 52.157.220.210
Redirect IPs 13.107.213.59, 13.107.246.59, 2620:1ec:46::59, 2620:1ec:bdf::59
Response IP 13.107.246.59
Found Yes
Hash 60779a715fbab7e32d2cb791045775332bc5019324cd002edd67fc8613624367
SimHash 539d4d608890

Groups

*

Rule Path
Disallow /zorgverzekering/aanvragen/
Disallow /zorgverzekering/aanvraagformulier/
Disallow /zorgverzekering/turff

ahrefsbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

ezooms

Rule Path
Disallow /

webcrawler

Rule Path
Disallow /

bubing

Rule Path
Disallow /

proximic

Rule Path
Disallow /

trendictionbot

Rule Path
Disallow /

msnbot-media

Rule Path
Disallow /

vagabondo

Rule Path
Disallow /

bilbo

Rule Path
Disallow /

yandex

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

yandeximages

Rule Path
Disallow /

yandexvideo

Rule Path
Disallow /

yandexmedia

Rule Path
Disallow /

yandexfavicons

Rule Path
Disallow /

linkdexbot

Rule Path
Disallow /

exabot

Rule Path
Disallow /

slurp

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

niki-bot

Rule Path
Disallow /

compspybot

Rule Path
Disallow /

trendictionbot

Rule Path
Disallow /

beetlebot

Rule Path
Disallow /

mail.ru_bot

Rule Path
Disallow /

istellabot

Rule Path
Disallow /

plukkie

Rule Path
Disallow /

melodibot

Rule Path
Disallow /

spbot

Rule Path
Disallow /

queryseekerspider

Rule Path
Disallow /

buddhabot

Rule Path
Disallow /

imrbot

Rule Path
Disallow /

acoonbot

Rule Path
Disallow /

yacybot

Rule Path
Disallow /

gufybot

Rule Path
Disallow /

nesotebot

Rule Path
Disallow /

sogou web spider

Rule Path
Disallow /

seokicks-robot

Rule Path
Disallow /

solomonobot

Rule Path
Disallow /

wbsearchbot

Rule Path
Disallow /

noyabot

Rule Path
Disallow /

flappybot

Rule Path
Disallow /

siteexplorer

Rule Path
Disallow /

proximic

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

crawler4j

Rule Path
Disallow /

nutch

Rule Path
Disallow /

berlin-fu-cow

Rule Path
Disallow /

riddler

Rule Path
Disallow /

butterfly

Rule Path
Disallow /

eknip

Rule Path
Disallow /

sqlmap

Rule Path
Disallow /

sqlmap/1.0-dev-nongit-20150818 (http://sqlmap.org)

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.poliswijzer.nl/sitemap.xml

Comments

  • User-Agent: Screaming Frog SEO Spider
  • Disallow: /

Warnings

  • 4 invalid lines.