thueringer-allgemeine.de
robots.txt

Robots Exclusion Standard data for thueringer-allgemeine.de

Resource Scan

Scan Details

Site Domain thueringer-allgemeine.de
Base Domain thueringer-allgemeine.de
Scan Status Ok
Last Scan2024-06-15T04:07:58+00:00
Next Scan 2024-06-22T04:07:58+00:00

Last Scan

Scanned2024-06-15T04:07:58+00:00
URL https://thueringer-allgemeine.de/robots.txt
Redirect https://www.thueringer-allgemeine.de:443/robots.txt
Redirect Domain www.thueringer-allgemeine.de
Redirect Base thueringer-allgemeine.de
Domain IPs 18.185.81.127, 18.196.221.37, 3.72.121.83
Redirect IPs 108.157.254.10, 108.157.254.110, 108.157.254.74, 108.157.254.98, 2600:9000:2753:1400:0:747c:e140:93a1, 2600:9000:2753:2200:0:747c:e140:93a1, 2600:9000:2753:3000:0:747c:e140:93a1, 2600:9000:2753:6c00:0:747c:e140:93a1, 2600:9000:2753:9600:0:747c:e140:93a1, 2600:9000:2753:c400:0:747c:e140:93a1, 2600:9000:2753:cc00:0:747c:e140:93a1, 2600:9000:2753:f200:0:747c:e140:93a1
Response IP 108.157.254.10
Found Yes
Hash d2ed7c8fc7a41768200563ca12e550046de6ef71be374b0e13298d47da6e7296
SimHash 9015d877cf13

Groups

*

Rule Path
Allow /static/*/client.js
Allow /static/*/main.css
Allow /static/*/favicon.png
Disallow /stats/*
Disallow /*?config*
Disallow /*.xmli*
Disallow /*?service=Ajax
Disallow /*?service=ajax
Disallow /config/*
Disallow /test/*
Disallow /Test/*
Disallow /template/*
Disallow /*?*token=*
Disallow /*?*eventId=*
Disallow /static/*
Disallow /migration_import_no_section/*
Disallow /secure/
Disallow /socialmedia/*
Disallow *reader_id%3DREADER_ID*
Disallow /suche/*
Disallow /*?widgetid=
Disallow /newsletter-result/
Disallow *tpcc%3D*
Disallow /resources/
Disallow /bin/
Disallow /downloads/
Disallow /service/newsletter-adconsent
Disallow /pagespeed_static/
Disallow /resources/img/*icon*pagespeed

cliqzbot

Rule Path
Disallow /

sogou spider

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

flamingo_searchengine

Rule Path
Disallow /

audisto

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.thueringer-allgemeine.de/sitemaps/news.xml