stern.de
robots.txt

Robots Exclusion Standard data for stern.de

Resource Scan

Scan Details

Site Domain stern.de
Base Domain stern.de
Scan Status Ok
Last Scan2026-01-14T20:41:37+00:00
Next Scan 2026-01-21T20:41:37+00:00

Last Scan

Scanned2026-01-14T20:41:37+00:00
URL https://stern.de/robots.txt
Redirect https://www.stern.de/robots.txt
Redirect Domain www.stern.de
Redirect Base stern.de
Domain IPs 207.174.61.1
Redirect IPs 23.32.39.132, 23.32.39.141
Response IP 23.32.39.132
Found Yes
Hash 0983a28a93da3ab43e0d7e8dfaa45c62a58bd1df2b04f39200f20f7953ff4550
SimHash 0b4fcb947393

Groups

msnbot-media/1.1

Rule Path
Disallow /

msnbot-media/2.0b

Rule Path
Disallow /

googlebot-news

Rule Path
Disallow /noch-fragen/
Disallow /action/
Disallow /sn/
Disallow /syndication/
Disallow /elastic/
Disallow /gutscheine/
Disallow /search?
Disallow /vergleich/
Disallow /p-api/*
Disallow /siegel/aerzte/
Disallow /p-user/

scrapy

Rule Path
Disallow /

diffbot

Rule Path
Disallow /

rytebot

Rule Path
Disallow /

proximic

Rule Path
Disallow /*%26ampshare%3D*

gptbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

applebot-extended

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

*

Rule Path
Disallow /action/
Disallow /sn/
Disallow /syndication/
Disallow /elastic/
Disallow /gutscheine/wp-admin/
Disallow /search?
Disallow /vergleich/zum/
Disallow /siegel/aerzte/
Disallow /p-api/
Disallow /p-user/