dr.dk
robots.txt

Robots Exclusion Standard data for dr.dk

Resource Scan

Scan Details

Site Domain dr.dk
Base Domain dr.dk
Scan Status Ok
Last Scan2024-11-13T21:12:44+00:00
Next Scan 2024-11-27T21:12:44+00:00

Last Scan

Scanned2024-11-13T21:12:44+00:00
URL https://dr.dk/robots.txt
Redirect https://www.dr.dk/robots.txt
Redirect Domain www.dr.dk
Redirect Base dr.dk
Domain IPs 2600:1413:b000:6::17d5:2bc8, 2600:1413:b000:6::17d5:2bdc, 96.17.96.17, 96.17.96.8
Redirect IPs 2600:1413:b000:6::17d5:2bcd, 2600:1413:b000:6::17d5:2bdf, 96.17.96.11, 96.17.96.22
Response IP 23.32.29.96
Found Yes
Hash cad62417f8c0862d346a13cff1d4b9fe445cc764d0cfc301c4af2c9d1b97097c
SimHash 982bd8408e03

Groups

gptbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

claude-web

Rule Path
Disallow /

omgili

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

*

Rule Path
Disallow /beta/
Disallow /bonanza20_assets/
Disallow /kgf
Disallow /kgf/
Disallow /tjenester/share/
Disallow /tjenester/esi
Disallow /l/
Disallow /nav/tekstttv/
Disallow /NR/exeres*
Disallow /nyheder/webdok/*/zone/*
Disallow /P3/find/
Disallow /programoversigttiltryk
Disallow /staging/
Disallow /stage/
Disallow /Templates/
Disallow /test/
Disallow /login/
Disallow /search/*
Disallow /soeg/*
Disallow /drtv/soeg*
Disallow /lyd/programmer?q*
Disallow /cgi-bin/fttx1.exe/
Disallow /dr-laer/mediebiblioteket/
Disallow /download/Forside/2021/
Disallow /download/Forside/2022/
Disallow /nyhedsbrev/
Disallow /undervisning_flash/
Disallow /auth/

Other Records

Field Value
sitemap https://www.dr.dk/sitemapindex.xml