cdn.finansportalen.se robots.txt

*

Rule	Path
Disallow	/BingSiteAuth.xml
Disallow	/cgi-bin/
Disallow	/robots.txt

Rule

Path

Disallow

/BingSiteAuth.xml

Disallow

/cgi-bin/

Disallow

/robots.txt

abonti

Rule	Path
Disallow	/

Rule

Path

Disallow

/

abonti/0.92

Rule	Path
Disallow	/

Rule

Path

Disallow

/

abot v1.0

Rule	Path
Disallow	/

Rule

Path

Disallow

/

aboutthedomain

Rule	Path
Disallow	/

Rule

Path

Disallow

/

add catalog

Rule	Path
Disallow	/

Rule

Path

Disallow

/

add catalog/2.1

Rule	Path
Disallow	/

Rule

Path

Disallow

/

advbot

Rule	Path
Disallow	/

Rule

Path

Disallow

/

advbot/2.0

Rule	Path
Disallow	/

Rule

Path

Disallow

/

aihitbot

Rule	Path
Disallow	/

Rule

Path

Disallow

/

aihitbot/2.9

Rule	Path
Disallow	/

Rule

Path

Disallow

/

alphabot

Rule	Path
Disallow	/

Rule

Path

Disallow

/

alphabot/3.2

Rule	Path
Disallow	/

Rule

Path

Disallow

/

anonymous/0.0

Rule	Path
Disallow	/

Rule

Path

Disallow

/

arachnida

Rule	Path
Disallow	/

Rule

Path

Disallow

/

arquivo-web-crawler

Rule	Path
Disallow	/

Rule

Path

Disallow

/

aspiegel

Rule	Path
Disallow	/

Rule

Path

Disallow

/

aspiegelbot

Rule	Path
Disallow	/

Rule

Path

Disallow

/

associative spider

Rule	Path
Disallow	/

Rule

Path

Disallow

/

baiduspider

Rule	Path
Disallow	/

Rule

Path

Disallow

/

baidu spider

Rule	Path
Disallow	/

Rule

Path

Disallow

/

barkrowler

Rule	Path
Disallow	/

Rule

Path

Disallow

/

barkrowler/0.7

Rule	Path
Disallow	/

Rule

Path

Disallow

/

battleztar bazinga

Rule	Path
Disallow	/

Rule

Path

Disallow

/

battleztar bazinga/0.01

Rule	Path
Disallow	/

Rule

Path

Disallow

/

bdfetch

Rule	Path
Disallow	/

Rule

Path

Disallow

/

betabot

Rule	Path
Disallow	/

Rule

Path

Disallow

/

bieshu

Rule	Path
Disallow	/

Rule

Path

Disallow

/

bigli seo

Rule	Path
Disallow	/

Rule

Path

Disallow

/

blackboard safeassign

Rule	Path
Disallow	/

Rule

Path

Disallow

/

blazer 1.0

Rule	Path
Disallow	/

Rule

Path

Disallow

/

blexbot

Rule	Path
Disallow	/

Rule

Path

Disallow

/

blexbot/1.0

Rule	Path
Disallow	/

Rule

Path

Disallow

/

blp_bbot

Rule	Path
Disallow	/

Rule

Path

Disallow

/

blp_bbot/0.1

Rule	Path
Disallow	/

Rule

Path

Disallow

/

boia-accessibility-agent/pr 1.0

Rule	Path
Disallow	/

Rule

Path

Disallow

/

bot for jce

Rule	Path
Disallow	/

Rule

Path

Disallow

/

bot/0.1 (bot for jce)

Rule	Path
Disallow	/

Rule

Path

Disallow

/

bpimagewalker

Rule	Path
Disallow	/

Rule

Path

Disallow

/

bpimagewalker/2.0

Rule	Path
Disallow	/

Rule

Path

Disallow

/

bubing

Rule	Path
Disallow	/

Rule

Path

Disallow

/

buck/2.2

Rule	Path
Disallow	/

Rule

Path

Disallow

/

buibui-bot

Rule	Path
Disallow	/

Rule

Path

Disallow

/

buibui-bot/1.0

Rule	Path
Disallow	/

Rule

Path

Disallow

/

ca-crawler

Rule	Path
Disallow	/

Rule

Path

Disallow

/

ca-crawler/1.0

Rule	Path
Disallow	/

Rule

Path

Disallow

/

cakephp

Rule	Path
Disallow	/

Rule

Path

Disallow

/

calypso v/0.01

Rule	Path
Disallow	/

Rule

Path

Disallow

/

calypso

Rule

Path

Disallow

/

cb/nutch-1.7

Rule

Path

Disallow

/

ccbot

Rule

Path

Disallow

/

ccbot/2.0

Rule

Path

Disallow

/

checkbot

Rule

Path

Disallow

/

checkgzipcompression.com

Rule

Path

Disallow

/

chushou

Rule

Path

Disallow

/

clarabot

Rule

Path

Disallow

/

clarabot/1.4

Rule

Path

Disallow

/

cloudservermarketspider

Rule

Path

Disallow

/

cloudservermarketspider/1.0

Rule

Path

Disallow

/

clushbot/3.x-binaryfury

Rule

Path

Disallow

/

cms crawler

Rule

Path

Disallow

/

cms crawler: http://www.cmscrawler.com

Rule

Path

Disallow

/

coccoc

Rule

Path

Disallow

/

coincornerbot

Rule

Path

Disallow

/

coincornerbot/1.1

Rule

Path

Disallow

/

companybook

Rule

Path

Disallow

/

companybook-crawler

Rule

Path

Disallow

/

copyscape

Rule

Path

Disallow

/

couponwcode

Rule

Path

Disallow

/

crawler4j

Rule

Path

Disallow

/

crazywebcrawler 0.9.0

Rule

Path

Disallow

/

crazywebcrawler 0.9.1

Rule

Path

Disallow

/

crazywebcrawler 0.9.7

Rule

Path

Disallow

/

crazywebcrawler

Rule

Path

Disallow

/

crazywebcrawler-spider

Rule

Path

Disallow

/

crowsnest

Rule

Path

Disallow

/

crowsnest/0.5

Rule

Path

Disallow

/

curious george - www.analyticsseo.com/crawler

Rule

Path

Disallow

/

curious george

Rule

Path

Disallow

/

cuwhois

Rule

Path

Disallow

/

cuwhois/1.0

Rule

Path

Disallow

/

dahoms

Rule

Path

Disallow

/

datagnionbot

Rule

Path

Disallow

/

dataxu/1.0

Rule

Path

Disallow

/

deusu/5.0.2

Rule

Path

Disallow

/

digincore

Rule

Path

Disallow

/

digincore bot

Rule

Path

Disallow

/

dispatch/0.11.0

Rule

Path

Disallow

/

dnyzbot

Rule

Path

Disallow

/

dnyzbot/1.0

Rule

Path

Disallow

/

domain re-animator bot

Rule

Path

Disallow

/

domainappender /1.0

Rule

Path

Disallow

/

domainappender

Rule

Path

Disallow

/

domaincrawler/3.0

Rule

Path

Disallow

/

domainsigmacrawler

Rule

Path

Disallow

/

domainsigmacrawler/0.1

Rule

Path

Disallow

/

domnutch

Rule

Path

Disallow

/

domnutch-bot

Rule

Path

Disallow

/

domnutch-bot/nutch

Rule

Path

Disallow

/

domnutch-bot/nutch-1.0

Rule

Path

Disallow

/

dotbot

Rule

Path

Disallow

/

eccp/1.2.1

Rule

Path

Disallow

/

ecommercebot

Rule

Path

Disallow

/

elefent

Rule

Path

Disallow

/

elefent/1.2

Rule

Path

Disallow

/

enlle punto com/nutch-1.9

Rule

Path

Disallow

/

episerver link checker

Rule

Path

Disallow

/

euripbot

Rule

Path

Disallow

/

euripbot/2.0

Rule

Path

Disallow

/

evc/2.0

Rule

Path

Disallow

/

evc-batch

Rule

Path

Disallow

/

evc-batch/2.0

Rule

Path

Disallow

/

everyonedomainsbot

Rule

Path

Disallow

/

everyonedomainsbot/1.0

Rule

Path

Disallow

/

express webpictures

Rule

Path

Disallow

/

extlinksbot

Rule

Path

Disallow

/

extlinksbot/1.5

Rule

Path

Disallow

/

faraday v0.8.8

Rule

Path

Disallow

/

faraday

Rule

Path

Disallow

/

findxbot

Rule

Path

Disallow

/

findxbot/1.0

Rule

Path

Disallow

/

flamingo_searchengine

Rule

Path

Disallow

/

flipboard robot

Rule

Path

Disallow

/

getproxi.es-bot

Rule

Path

Disallow

/

getproxi.es-bot/1.1

Rule

Path

Disallow

/

g-i-g-a-b-o-t

Rule

Path

Disallow

/

gigablastopensource

Rule

Path

Disallow

/

gigablastopensource/1.0

Rule

Path

Disallow

/

girafabot

Rule

Path

Disallow

/

gluten free crawler

Rule

Path

Disallow

/

gluten free crawler/1.0

Rule

Path

Disallow

/

griffinbot

Rule

Path

Disallow

/

grifinbot/0.01

Rule

Path

Disallow

/

gwpimages

Rule

Path

Disallow

/

gwpimages/1.0

Rule

Path

Disallow

/

haiula

Rule

Path

Disallow

/

haiula/1.4

Rule

Path

Disallow

/

haosouspider

Rule

Path

Disallow

/

hivemind

Rule

Path

Disallow

/

hostharvest

Rule

Path

Disallow

/

hostharvest/0.4.28

Rule

Path

Disallow

/

hrcrawler

Rule

Path

Disallow

/

hrcrawler/2.0

Rule

Path

Disallow

/

http://git.io/tl_s2w

Rule

Path

Disallow

/

http://www.checkprivacy.or.kr:6600/rs/privacy_enfaq.jsp

Rule

Path

Disallow

/

hubspot links crawler 1.0

Rule

Path

Disallow

/

hubspot webcrawler

Rule

Path

Disallow

/

hubspot

Rule

Path

Disallow

/

hunchan

Rule

Path

Disallow

/

hybridbot

Rule

Path

Disallow

/

hypercrawl

Rule

Path

Disallow

/

hypercrawl/0.2

Rule

Path

Disallow

/

icap-iod

Rule

Path

Disallow

/

icc-crawler

Rule

Path

Disallow

/

icc-crawler/2.0

Rule

Path

Disallow

/

ichiro robot

Rule

Path

Disallow

/

image.coccoc/1.0

Rule

Path

Disallow

/

image2play

Rule

Path

Disallow

/

image2play/0.1

Rule

Path

Disallow

/

indy library

Rule

Path

Disallow

/

insightscollector

Rule

Path

Disallow

/

insightscollector/0.1

Rule

Path

Disallow

/

insightscollector/0.1beta

Rule

Path

Disallow

/

integrity/5

Rule

Path

Disallow

/

internaetboten

Rule

Path

Disallow

/

internaetboten/0.99

Rule

Path

Disallow

/

irl crawler

Rule

Path

Disallow

/

james bot - webcrawler

Rule

Path

Disallow

/

james bot

Rule

Path

Disallow

/

jamesbot

Rule

Path

Disallow

/

jetbrains 5.0

Rule

Path

Disallow

/

jetbrains

Rule

Path

Disallow

/

joc web spider

Rule

Path

Disallow

/

kocmohabt

Rule

Path

Disallow

/

komodiabot

Rule

Path

Disallow

/

komodiabot/1.0

Rule

Path

Disallow

/

kraken

Rule

Path

Disallow

/

kraken/0.1

Rule

Path

Disallow

/

kyoto-tohoku-crawler/v1

Rule

Path

Disallow

/

larbin

Rule

Path

Disallow

/

lcc

Rule

Path

Disallow

/

lechenie

Rule

Path

Disallow

/

libwww-perl

Rule

Path

Disallow

/

link checker

Rule

Path

Disallow

/

link/1.0

Rule

Path

Disallow

/

linkcheck

Rule

Path

Disallow

/

linkcheckv3.0

Rule

Path

Disallow

/

linkdex

Rule

Path

Disallow

/

linkdex.com/v2.0

Rule

Path

Disallow

/

linkdex.com/v2.1

Rule

Path

Disallow

/

linkdexbot

Rule

Path

Disallow

/

linkdexbot/2.0

Rule

Path

Disallow

/

linkdexbot/2.1

Rule

Path

Disallow

/

linkdexbot-mobile/2.1

Rule

Path

Disallow

/

linkpadbot

Rule

Path

Disallow

/

linkpadbot/1.06

Rule

Path

Disallow

/

linqiascrapebot

Rule

Path

Disallow

/

linqiascrapebot/1.0

Rule

Path

Disallow

/

lipperhey seo service

Rule

Path

Disallow

/

lipperhey

Rule

Path

Disallow

/

lipperhey-kaus-australis

Rule

Path

Disallow

/

lipperhey-kaus-australis/5.0

Rule

Path

Disallow

/

listicka

Rule

Path

Disallow

/

lssrocketcrawler

Rule

Path

Disallow

/

lssrocketcrawler/1.0 lightspeedsystems

Rule

Path

Disallow

/

lssrocketcrawler/1.0

Rule

Path

Disallow

/

ltx71

Rule

Path

Disallow

/

lwnutch/nutch-1.4

Rule

Path

Disallow

/

mail.ru

Rule

Path

Disallow

/

mail.ru_bot

Rule

Path

Disallow

/

mail.ru_bot/2.0

Rule

Path

Disallow

/

mail.ru_bot/fast/2.0

Rule

Path

Disallow

/

mail.ru_bot/robots/

Rule

Path

Disallow

/

mail.ru_bot/robots/2.0

Rule

Path

Disallow

/

mbcrawler

Rule

Path

Disallow

/

mbcrawler/1.0

Rule

Path

Disallow

/

md5sum

Rule

Path

Disallow

/

md5sum\x22

Rule

Path

Disallow

/

meanpathbot

Rule

Path

Disallow

/

mediatoolkitbot

Rule

Path

Disallow

/

megaindex.ru

Rule

Path

Disallow

/

megaindex.ru/2.0

Rule

Path

Disallow

/

mezhpozvonochnoi

Rule

Path

Disallow

/

mike-crawler

Rule

Path

Disallow

/

mixbot

Rule

Path

Disallow

/

mixnodecache

Rule

Path

Disallow

/

mixnodecache/1.8

Rule

Path

Disallow

/

mixrankbot

Rule

Path

Disallow

/

mj12bot

Rule

Path

Disallow

/

monkeybot/0.1

Rule

Path

Disallow

/

my crawler

Rule

Path

Disallow

/

my nutch spider/nutch-1.9

Rule

Path

Disallow

/

mycrowl/nutch-1.9

Rule

Path

Disallow

/

mygreatua/2.0

Rule

Path

Disallow

/

myiptest

Rule

Path

Disallow

/

nameprotect robot

Rule

Path

Disallow

/

nerdybot

Rule

Path

Disallow

/

netcraft spider

Rule

Path

Disallow

/

netestate ne crawler

Rule

Path

Disallow

/

netlyzer fastprobe

Rule

Path

Disallow

/

netpeakcheckerbot

Rule

Path

Disallow

/

netpeakcheckerbot/3.1

Rule

Path

Disallow

/

netresearchserver

Rule

Path

Disallow

/

netresearchserver/4.0

Rule

Path

Disallow

/

nimbostratus-bot

Rule

Path

Disallow

/

nimbostratus-bot/v1.3.2

Rule

Path

Disallow

/

nmap scripting engine

Rule

Path

Disallow

/

node.io

Rule

Path

Disallow

/

node.js

Rule

Path

Disallow

/

node/simplecrawler 0.5.2

Rule

Path

Disallow

/

node/simplecrawler

Rule

Path

Disallow

/

obot/2.3.1

Rule

Path

Disallow

/

omgilibot

Rule

Path

Disallow

/

omgilibot/0.4

Rule

Path

Disallow

/

online domain tools - online website link checker

Rule

Path

Disallow

/

online domain tools - online website link checker/1.2

Rule

Path

Disallow

/

openfind robot

Rule

Path

Disallow

/

openhosebot

Rule

Path

Disallow

/

openhosebot/2.1

Rule

Path

Disallow

/

openstat

Rule

Path

Disallow

/

openstat/0.1

Rule

Path

Disallow

/

optimizationcrawler

Rule

Path

Disallow

/

optimizationcrawler/0.2

Rule

Path

Disallow

/

outclicksbot

Rule

Path

Disallow

/

outclicksbot/1

Rule

Path

Disallow

/

page analyzer v4.0

Rule

Path

Disallow

/

page analyzer

Rule

Path

Disallow

/

pageanalyzer

Rule

Path

Disallow

/

pageanalyzer/1.1

Rule

Path

Disallow

/

pageanalyzer/1.5

Rule

Path

Disallow

/

pagesinventory

Rule

Path

Disallow

/

pagespeed/1.1 fetcher

Rule

Path

Disallow

/

pagespeed/1.1

Rule

Path

Disallow

/

pagespeedbot

Rule

Path

Disallow

/

paracrawl

Rule

Path

Disallow

/

perl lwp

Rule

Path

Disallow

/

phpcrawl

Rule

Path

Disallow

/

phpsitecheck 1.0

Rule

Path

Disallow

/

phpsitecheck

Rule

Path

Disallow

/

pi-monster

Rule

Path

Disallow

/

pilicanbot/nutch-1.15

Rule

Path

Disallow

/

pimeyes.com

Rule

Path

Disallow

/

plukkie

Rule

Path

Disallow

/

pogs/2.0

Rule

Path

Disallow

/

powermarks

Rule

Path

Disallow

/

powerpivot

Rule

Path

Disallow

/

privacy_enfaq.jsp

Rule

Path

Disallow

/

prlog

Rule

Path

Disallow

/

prlog/1.0

Rule

Path

Disallow

/

publiclibraryarchive.org

Rule

Path

Disallow

/

publiclibraryarchive.org/1.0

Rule

Path

Disallow

/

pu_in crawler

Rule

Path

Disallow

/

putin

Rule

Path

Disallow

/

putin spider

Rule

Path

Disallow

/

qingdao

Rule

Path

Disallow

/

qlikview

Rule

Path

Disallow

/

quipu

Rule

Path

Disallow

/

quipu/1.0

Rule

Path

Disallow

/

quipu/2.0

Rule

Path

Disallow

/

r6_commentreader

Rule

Path

Disallow

/

r6_feedfetcher

Rule

Path

Disallow

/

re-re studio

Rule

Path

Disallow

/

researchscan

Rule

Path

Disallow

/

researchscan/t12ca

Rule

Path

Disallow

/

researchscan/t12l

Rule

Path

Disallow

/

researchscan/t13rl

Rule

Path

Disallow

/

riddler

Rule

Path

Disallow

/

rivalseek.com-bot

Rule

Path

Disallow

/

rogerbot

Rule

Path

Disallow

/

rogerbot/1.0

Rule

Path

Disallow

/

rootlink

Rule

Path

Disallow

/

ru_bot/2.0

Rule

Path

Disallow

/

ryowlengine

Rule

Path

Disallow

/

ryowlengine/1.0

Rule

Path

Disallow

/

rytebot

Rule

Path

Disallow

/

rytebot/1.0.0

Rule

Path

Disallow

/

scopia

Rule

Path

Disallow

/

scopia crawler

Rule

Path

Disallow

/

scopia crawler 1.0

Rule

Path

Disallow

/

scopia crawler 1.1

Rule

Path

Disallow

/

scopia crawler 1.2

Rule

Path

Disallow

/

scooperbot

Rule

Path

Disallow

/

scrapy

Rule

Path

Disallow

/

scrapy/0.16.5

Rule

Path

Disallow

/

scrapy/0.24.4

Rule

Path

Disallow

/

scrapy/0.24.5

Rule

Path

Disallow

/

scrapy/0.24.6

Rule

Path

Disallow

/

scrapy/1.0.1

Rule

Path

Disallow

/

scrapy/1.5.0

Rule

Path

Disallow

/

scrapy/1.8.0

Rule

Path

Disallow

/

scrutiny/4

Rule

Path

Disallow

/

searchatlas.com

Rule

Path

Disallow

/

seekport

Rule

Path

Disallow

/

seekport crawler

Rule

Path

Disallow

/

semrushbot

Rule

Path

Disallow

/

semrushbot-sa

Rule

Path

Disallow

/

seodiver/1.0

Rule

Path

Disallow

/

seokicks

Rule

Path

Disallow

/

seokicks-robot

Rule

Path

Disallow

/

seolyticscrawler

Rule

Path

Disallow

/

seolyticscrawler/3.0

Rule

Path

Disallow

/

seoscanners

Rule

Path

Disallow

/

seoscanners.net

Rule

Path

Disallow

/

seoscanners.net/1

Rule

Path

Disallow

/

seoscanners.net/1.0

Rule

Path

Disallow

/

seostats 2.1.0

Rule

Path

Disallow

/

seosys/nutch-2.3

Rule

Path

Disallow

/

serpstatbot

Rule

Path

Disallow

/

serpstatbot/1.0

Rule

Path

Disallow

/

setcronjob/1.0

Rule

Path

Disallow

/

seznambot

Rule

Path

Disallow

/

sheerboredom.experimental.robot

Rule

Path

Disallow

/

sheerboredom.experimental.robot/0.2

Rule

Path

Disallow

/

showyoubot

Rule

Path

Disallow

/

simplecrawler

Rule

Path

Disallow

/

sistrix crawler

Rule

Path

Disallow

/

sistrix

Rule

Path

Disallow

/

sitebot

Rule

Path

Disallow

/

sitebot/0.1

Rule

Path

Disallow

/

siteexplorer

Rule

Path

Disallow

/

siteexplorer/1.0

Rule

Path

Disallow

/

siteexplorer/1.0b

Rule

Path

Disallow

/

siteluxbot

Rule

Path

Disallow

/

siteluxbot/1.0

Rule

Path

Disallow

/

sjuupbot

Rule

Path

Disallow

/

skimbot

Rule

Path

Disallow

/

skimbot/1.0

Rule

Path

Disallow

/

sky nutch crawler/nutch-1.9

Rule

Path

Disallow

/

smtbot

Rule

Path

Disallow

/

smtbot/1.0

Rule

Path

Disallow

/

snacktory

Rule

Path

Disallow

/

snk screenshot bot

Rule

Path

Disallow

/

snk screenshot bot/0.20

Rule

Path

Disallow

/

sogou spider

Rule

Path

Disallow

/

sogou web spider

Rule

Path

Disallow

/

spambayes

Rule

Path

Disallow

/

spambayes/1.1a3+

Rule

Path

Disallow

/

spbot

Rule

Path

Disallow

/

spbot/4.4.2

Rule

Path

Disallow

/

spiderbot

Rule

Path

Disallow

/

spiderling

Rule

Path

Disallow

/

spiderbot/nutch-1.7

Rule

Path

Disallow

/

spray-can

Rule

Path

Disallow

/

spray-can/1.2.1

Rule

Path

Disallow

/

ssg/3.0

Rule

Path

Disallow

/

statastico

Rule

Path

Disallow

/

statastico/4.0

Rule

Path

Disallow

/

steeler

Rule

Path

Disallow

/

steeler/3.5

Rule

Path

Disallow

/

stratagems kumo

Rule

Path

Disallow

/

stratagems

Rule

Path

Disallow

/

studiofaca search

Rule

Path

Disallow

/

studiofaca

Rule

Path

Disallow

/

sukibot

Rule

Path

Disallow

/

sukibot_heritrix

Rule

Path

Disallow

/

sukibot_heritrix/3.1.1

Rule

Path

Disallow

/

supra-bot

Rule

Path

Disallow

/

supra-bot/1.0

Rule

Path

Disallow

/

surveybot

Rule

Path

Disallow

/

synapse

Rule

Path

Disallow

/

synthesio crawler release monalisa

Rule

Path

Disallow

/

tbot-nutch/nutch-1.10

Rule

Path

Disallow

/

traackr.com bot

Rule

Path

Disallow

/

trendictionbot

Rule

Path

Disallow

/

trendiction-bot

Rule

Path

Disallow

/

truebot

Rule

Path

Disallow

/

truebot/1.0

Rule

Path

Disallow

/

tulipchain/5.xx

Rule

Path

Disallow

/

turnitin

Rule

Path

Disallow

/

turnitinbot

Rule

Path

Disallow

/

twmbot/0.1

Rule

Path

Disallow

/

typhoeus

Rule

Path

Disallow

/

ucmore crawler app

Rule

Path

Disallow

/

umbot-ln

Rule

Path

Disallow

/

umbot-ln/1.0

Rule

Path

Disallow

/

updown_tester

Rule

Path

Disallow

/

urlchecker

Rule

Path

Disallow

/

v1.0/1.2

Rule

Path

Disallow

/

w3af.org

Rule

Path

Disallow

/

wappalyzer

Rule

Path

Disallow

/

wasalive

Rule

Path

Disallow

/

wasalive-bot

Rule

Path

Disallow

/

vbseo

Rule

Path

Disallow

/

wbsearchbot

Rule

Path

Disallow

/

wbsearchbot/1.1

Rule

Path

Disallow

/

wearenotevil

Rule

Path

Disallow

/

webalta

Rule

Path

Disallow

/

webalta crawler

Rule

Path

Disallow

/

web corpus crawler

Rule

Path

Disallow

/

webcookies

Rule

Path

Disallow

/

webcookies/1.0

Rule

Path

Disallow

/

webcopier vx.xa

Rule

Path

Disallow

/

webnest 0.9

Rule

Path

Disallow

/

weborama-fetcher

Rule

Path

Disallow

/

webql

Rule

Path

Disallow

/

webscanner

Rule

Path

Disallow

/

webscanner/1.0

Rule

Path

Disallow

/

webscout

Rule

Path

Disallow

/

webscout/1.0

Rule

Path

Disallow

/

web-sniffer

Rule

Path

Disallow

/

web-sniffer/1.1.0

Rule

Path

Disallow

/

webster pro v3.4

Rule

Path

Disallow

/

webtarantula.com crawler

Rule

Path

Disallow

/

wecrawlforthepeace

Rule

Path

Disallow

/

vegebot

Rule

Path

Disallow

/

vegi bot

Rule

Path

Disallow

/

velenpublicwebcrawler

Rule

Path

Disallow

/

velenpublicwebcrawler/1.0

Rule

Path

Disallow

/

welikelinks

Rule

Path

Disallow

/

vericitecrawler

Rule

Path

Disallow

/

vericitecrawler/nutch-1.9

Rule

Path

Disallow

/

whatweb

Rule

Path

Disallow

/

whatweb/0.4.8-dev

Rule

Path

Disallow

/

visited by http://tools.geek-tools.org

Rule

Path

Disallow

/

voila robot

Rule

Path

Disallow

/

voltron

Rule

Path

Disallow

/

woobot

Rule

Path

Disallow

/

woobot/1.1

Rule

Path

Disallow

/

woobot/2.0

Rule

Path

Disallow

/

woorankreview

Rule

Path

Disallow

/

woorankreview/2.0

Rule

Path

Disallow

/

vorboss web crawler

Rule

Path

Disallow

/

vorboss web crawler/nutch-2.3

Rule

Path

Disallow

/

worldbrewbot

Rule

Path

Disallow

/

worldbrewbot/2.1

Rule

Path

Disallow

/

worldwebheritage.org

Rule

Path

Disallow

/

worldwebheritage.org/1.0

Rule

Path

Disallow

/

wscheck.com

Rule

Path

Disallow

/

wscheck.com/1.0.0

Rule

Path

Disallow

/

www.deadlinkchecker.com

Rule

Path

Disallow

/

www.petitsage.fr site detector 0.4

Rule

Path

Disallow

/

www-mechanize

Rule

Path

Disallow

/

www-mechanize/1.74

Rule

Path

Disallow

/

xenu link sleuth

Rule

Path

Disallow

/

xenu's link sleuth

Rule

Path

Disallow

/

xovibot

Rule

Path

Disallow

/

xovibot/2.0

Rule

Path

Disallow

/

xspider

Rule

Path

Disallow

/

yandex robot

Rule

Path

Disallow

/

yandex

Rule

Path

Disallow

/

yetibot

Rule

Path

Disallow

/

yisouspider

Rule

Path

Disallow

/

yoozbot

Rule

Path

Disallow

/

yoozbot-2.2

Rule

Path

Disallow

/

your-website-sucks

Rule

Path

Disallow

/

your-website-sucks/2.0

Rule

Path

Disallow

/

zgrab/0.x

Rule

Path

Disallow

/

zzabmbot

Rule

Path

Disallow

/

zzabmbot/1.0

Rule

Path

Disallow

/

Site Domain	cdn.finansportalen.se
Base Domain	finansportalen.se
Scan Status	Failed
Failure Stage	Fetching resource.
Failure Reason	Couldn't connect to server.
Last Scan	2024-10-02T03:36:43+00:00
Next Scan	2024-12-31T03:36:43+00:00

Scanned	2024-06-05T03:29:26+00:00
URL	https://cdn.finansportalen.se/robots.txt
Domain IPs	13.227.254.108, 13.227.254.25, 13.227.254.30, 13.227.254.61
Response IP	13.227.254.108
Found	Yes
Hash	0ddab57846d62f9c290e76ddd49723d6e0a59326f3dc47ef988d6781273efbfa
SimHash	78c10f17e630

cdn.finansportalen.serobots.txt

Resource Scan

Scan Details

Last Successful Scan

Groups

*

abonti

abonti/0.92

abot v1.0

aboutthedomain

add catalog

add catalog/2.1

advbot

advbot/2.0

aihitbot

aihitbot/2.9

alphabot

alphabot/3.2

anonymous/0.0

arachnida

arquivo-web-crawler

aspiegel

aspiegelbot

associative spider

baiduspider

baidu spider

barkrowler

barkrowler/0.7

battleztar bazinga

battleztar bazinga/0.01

bdfetch

betabot

bieshu

bigli seo

blackboard safeassign

blazer 1.0

blexbot

blexbot/1.0

blp_bbot

blp_bbot/0.1

boia-accessibility-agent/pr 1.0

bot for jce

bot/0.1 (bot for jce)

bpimagewalker

bpimagewalker/2.0

bubing

buck/2.2

buibui-bot

buibui-bot/1.0

ca-crawler

ca-crawler/1.0

cakephp

calypso v/0.01

calypso

cb/nutch-1.7

ccbot

ccbot/2.0

checkbot

checkgzipcompression.com

chushou

clarabot

clarabot/1.4

cloudservermarketspider

cloudservermarketspider/1.0

clushbot/3.x-binaryfury

cms crawler

cms crawler: http://www.cmscrawler.com

coccoc

coincornerbot

coincornerbot/1.1

companybook

companybook-crawler

copyscape

couponwcode

crawler4j

crazywebcrawler 0.9.0

crazywebcrawler 0.9.1

crazywebcrawler 0.9.7

crazywebcrawler

crazywebcrawler-spider

cdn.finansportalen.se
robots.txt