Log In Sign Up

folkbladet.se
robots.txt

Robots Exclusion Standard data for folkbladet.se

Resource Scan

Scan Details

Site Domain folkbladet.se
Base Domain folkbladet.se
Scan Status Ok
Last Scan2025-12-11T22:11:56+00:00
Next Scan 2025-12-18T22:11:56+00:00

Last Scan

Scanned2025-12-11T22:11:56+00:00
URL https://folkbladet.se/robots.txt
Redirect https://www.folkbladet.se/robots.txt
Redirect Domain www.folkbladet.se
Redirect Base folkbladet.se
Domain IPs 151.101.131.42, 151.101.195.42, 151.101.3.42, 151.101.67.42
Redirect IPs 146.75.119.52, 2a04:4e42:9::820
Response IP 151.101.39.52
Found Yes
Hash a2a90c9358e3ebcc76312d1d69dc11791c44286e97f1dec1477b11b1ff421e15
SimHash 7d074351c497

Groups

*

Rule Path
Disallow /logga-in*
Disallow /mina-sidor*
Disallow /nyhetsbrev*
Disallow /TSPD*
Disallow /bli-prenumerant/kampanj*
Disallow /bli-prenumerant/los-in/*
Disallow /bli-prenumerant/artikel/*
Disallow /artikel/*
Disallow /ntm-login*

mb2345browser

Rule Path
Disallow /

liebaofast

Rule Path
Disallow /

zh-cn

Rule Path
Disallow /

micromessenger

Rule Path
Disallow /

zh_cn

Rule Path
Disallow /

kinza

Rule Path
Disallow /

oppo a33

Rule Path
Disallow /

aspeigel

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

twitterbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

deepseekbot

Rule Path
Disallow /

deepseek

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

perplexitybot

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

webizo-extended

Rule Path
Disallow /

cohere

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

pangubot

Rule Path
Disallow /

youbot

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

ai2bot

Rule Path
Disallow /

ai2bot-dolma

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

aspeigel

Rule Path
Disallow /

aspiegelbot

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

applebot

Rule Path
Disallow /

applebot-extended

Rule Path
Disallow /

arquivo-web-crawler

Rule Path
Disallow /

awariorssbot

Rule Path
Disallow /

awariosmartbot

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

claude-web

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

coccocbot-web

Rule Path
Disallow /

cohere-ai

Rule Path
Disallow /

cohere-training-data-crawler

Rule Path
Disallow /

cohere

Rule Path
Disallow /

crawler4j

Rule Path
Disallow /

crawlspace

Rule Path
Disallow /

deepseek

Rule Path
Disallow /

deepseekbot

Rule Path
Disallow /

diffbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

duckassistbot

Rule Path
Disallow /

facebookbot

Rule Path
Disallow /

friendlycrawler

Rule Path
Disallow /

google-cloudvertexbot

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

hada.news

Rule Path
Disallow /

icc-crawler

Rule Path
Disallow /

imagesiftbot

Rule Path
Disallow /

imediaethics.org

Rule Path
Disallow /

img2dataset

Rule Path
Disallow /

isscyberriskcrawler

Rule Path
Disallow /

jenkersbot

Rule Path
Disallow /

kangaroo bot

Rule Path
Disallow /

kinza

Rule Path
Disallow /

liebaofast

Rule Path
Disallow /

livelapbot

Rule Path
Disallow /

magpie-crawler

Rule Path
Disallow /

mauibot

Rule Path
Disallow /

mb2345browser

Rule Path
Disallow /

meltwater

Rule Path
Disallow /

meta-externalagent

Rule Path
Disallow /

meta-externalfetcher

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

mojeek

Rule Path
Disallow /

mojeekbot

Rule Path
Disallow /

moodlebot

Rule Path
Disallow /

neticlebot

Rule Path
Disallow /

netvibes

Rule Path
Disallow /

newsnow

Rule Path
Disallow /

oai-searchbot

Rule Path
Disallow /

omgili

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

oppo a33

Rule Path
Disallow /

pangubot

Rule Path
Disallow /

pangubot

Rule Path
Disallow /

perplexitybot

Rule Path
Disallow /

perplexity-user

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

scrapy

Rule Path
Disallow /

seekportbot

Rule Path
Disallow /

seekr

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

timpibot

Rule Path
Disallow /

turnitin

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

velenpublicwebcrawler

Rule Path
Disallow /

webizo-extended

Rule Path
Disallow /

webzio-extended

Rule Path
Disallow /

yacy

Rule Path
Disallow /

yacybot

Rule Path
Disallow /

yandex

Rule Path
Disallow /

yandexadditional

Rule Path
Disallow /

yandexadditionalbot

Rule Path
Disallow /

youbot

Rule Path
Disallow /

youbot

Rule Path
Disallow /

zh-cn

Rule Path
Disallow /

googlebot

Rule Path
Allow /

bingbot

Rule Path
Allow /

duckduckbot

Rule Path
Allow /
Back to top

Other Records

Field Value
sitemap https://folkbladet.se/sitemap/sitemap-index.xml
Back to top