Log In Sign Up

morgenpost.de
robots.txt

Robots Exclusion Standard data for morgenpost.de

Resource Scan

Scan Details

Site Domain morgenpost.de
Base Domain morgenpost.de
Scan Status Ok
Last Scan2024-05-14T23:14:04+00:00
Next Scan 2024-05-21T23:14:04+00:00

Last Scan

Scanned2024-05-14T23:14:04+00:00
URL https://morgenpost.de/robots.txt
Redirect https://www.morgenpost.de:443/robots.txt
Redirect Domain www.morgenpost.de
Redirect Base morgenpost.de
Domain IPs 18.185.81.127, 18.196.221.37, 3.72.121.83
Redirect IPs 13.33.30.21, 13.33.30.47, 13.33.30.73, 13.33.30.98, 2600:9000:2200:6000:5:150d:dc00:93a1, 2600:9000:2200:6400:5:150d:dc00:93a1, 2600:9000:2200:7200:5:150d:dc00:93a1, 2600:9000:2200:8e00:5:150d:dc00:93a1, 2600:9000:2200:d400:5:150d:dc00:93a1, 2600:9000:2200:e600:5:150d:dc00:93a1, 2600:9000:2200:ec00:5:150d:dc00:93a1, 2600:9000:2200:fe00:5:150d:dc00:93a1
Response IP 13.33.30.21
Found Yes
Hash ee0da2f3fa30153e0a44765cdc42998842a0c43ba237d402d3f4528d5e61cabe
SimHash 5c0fd050c631

Groups

*

Rule Path
Allow /static/*/client.js
Allow /static/*/main.css
Allow /static/*/favicon.png
Disallow /stats/*
Disallow /*?config*
Disallow /*.xmli*
Disallow /*?service=Ajax
Disallow /*?service=ajax
Disallow /config/*
Disallow /test/*
Disallow /Test/*
Disallow /template/*
Disallow /*?*token=*
Disallow /*?*eventId=*
Disallow /static/*
Disallow /migration_import_no_section/*
Disallow /secure/
Disallow /socialmedia/*
Disallow *reader_id%3DREADER_ID*
Disallow /suche/*
Disallow /*?widgetid=
Disallow /newsletter-result/
Disallow *tpcc%3D*
Disallow /resources/
Disallow /bin/
Disallow /downloads/
Disallow /service/newsletter-adconsent
Disallow /pagespeed_static/
Disallow /resources/img/*icon*pagespeed

cliqzbot

Rule Path
Disallow /

sogou spider

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

flamingo_searchengine

Rule Path
Disallow /

audisto

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

archive.org_bot

Rule Path
Disallow /

arquivo-web-crawler

Rule Path
Disallow /

arquivo.pt

Rule Path
Disallow /

backlinkcrawler

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

browsertrix

Rule Path
Disallow /

brozzler

Rule Path
Disallow /

builtwith

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

cincraw

Rule Path
Disallow /

coccocbot

Rule Path
Disallow /

contao/crawler

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

deepcrawl

Rule Path
Disallow /

dmbot

Rule Path
Disallow /

domainstatsbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

fluid

Rule Path
Disallow /

freshbot

Rule Path
Disallow /

haosouspider

Rule Path
Disallow /

happywing

Rule Path
Disallow /

harsilbot

Rule Path
Disallow /

hatena antenna

Rule Path
Disallow /

heritrix

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

imagesiftbot

Rule Path
Disallow /

kazbtbot

Rule Path
Disallow /

kraken

Rule Path
Disallow /

linkchecker

Rule Path
Disallow /

linkdebot

Rule Path
Disallow /

linkfluence yak bot

Rule Path
Disallow /

mail.ru_bot

Rule Path
Disallow /

majestic

Rule Path
Disallow /

majestic12

Rule Path
Disallow /

metajobbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

monsidobot

Rule Path
Disallow /

netestate

Rule Path
Disallow /

ogdwctcxcrawler

Rule Path
Disallow /

onpagebot

Rule Path
Disallow /

openindexspider

Rule Path
Disallow /

openindexspider

Rule Path
Disallow /

optimizer

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

researchbot

Rule Path
Disallow /

riddler

Rule Path
Disallow /

rogerbot

Rule Path
Disallow /

rytebot

Rule Path
Disallow /

semanticbot

Rule Path
Disallow /

semanticscholarbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

semrushbot-si

Rule Path
Disallow /

sentibot

Rule Path
Disallow /

seobility

Rule Path
Disallow /

seodiver

Rule Path
Disallow /

seokicks

Rule Path
Disallow /

seokicks-robot

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

sirdatabot

Rule Path
Disallow /

sistrix

Rule Path
Disallow /

sistrix

Rule Path
Disallow /

sistrix crawler

Rule Path
Disallow /

siteauditbot

Rule Path
Disallow /

spbot

Rule Path
Disallow /

spbot

Rule Path
Disallow /

special_archiver

Rule Path
Disallow /

splitsignalbot

Rule Path
Disallow /

tag-crawler

Rule Path
Disallow /

testcrawler

Rule Path
Disallow /

thinkers-bot

Rule Path
Disallow /

toplistbot

Rule Path
Disallow /

uipbot

Rule Path
Disallow /

urlsuma

Rule Path
Disallow /

user-agent

Rule Path
Disallow /

viennatinybot

Rule Path
Disallow /

vsusearchspider

Rule Path
Disallow /

weborama-fetcher

Rule Path
Disallow /

wiseguys robot

Rule Path
Disallow /

wpbot

Rule Path
Disallow /

yandex

Rule Path
Disallow /

yeti

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://www.morgenpost.de/sitemaps/news.xml
Back to top