burda.de
robots.txt

Robots Exclusion Standard data for burda.de

Resource Scan

Scan Details

Site Domain burda.de
Base Domain burda.de
Scan Status Ok
Last Scan2024-05-27T20:21:42+00:00
Next Scan 2024-06-26T20:21:42+00:00

Last Scan

Scanned2024-05-27T20:21:42+00:00
URL http://burda.de/robots.txt
Redirect https://www.burda.com/robots.txt
Redirect Domain www.burda.com
Redirect Base burda.com
Domain IPs 193.26.101.11
Redirect IPs 2a01:4f8:1c1e:a38e::1, 5.75.188.121
Response IP 5.75.188.121
Found Yes
Hash 5ac6ecb641244edacbbd6a83efc47edde28c2900635b6ffd069d21dc9bc36a2a
SimHash ca189cc06e90

Groups

ahrefsbot
blexbot
bytespider
mj12bot
megaindex.ru
semrushbot
semrushbot-sa
megaindex.com
sogou spider
molokaibot

Rule Path
Disallow /

*

Rule Path
Disallow /*/download_media/*/
Disallow /*/pdf/
Disallow /de/karriere/suche/*/apply/
Disallow /en/career/search/*/apply/

Other Records

Field Value
crawl-delay 20

Other Records

Field Value
sitemap https://www.burda.com/sitemap_index.xml