dr.dk
robots.txt

Robots Exclusion Standard data for dr.dk

Resource Scan

Scan Details

Site Domain dr.dk
Base Domain dr.dk
Scan Status Ok
Last Scan2024-06-18T21:09:06+00:00
Next Scan 2024-07-02T21:09:06+00:00

Last Scan

Scanned2024-06-18T21:09:06+00:00
URL https://dr.dk/robots.txt
Redirect https://www.dr.dk/robots.txt
Redirect Domain www.dr.dk
Redirect Base dr.dk
Domain IPs 23.209.46.145, 23.209.46.155, 2600:1413:b000:1d::17d1:2e91, 2600:1413:b000:1d::17d1:2e9b
Redirect IPs 23.50.232.200, 23.50.232.248, 2600:1413:b000:1d::17d1:2e8f, 2600:1413:b000:1d::17d1:2ea1
Response IP 23.59.168.131
Found Yes
Hash f104d2bab27a413855fd1ce2ed3c0b26fc16f4995aa2b961c793f4a2762cc66b
SimHash 882bd8008e03

Groups

gptbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

claude-web

Rule Path
Disallow /

omgili

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

*

Rule Path
Disallow /beta/
Disallow /bonanza20_assets/
Disallow /kgf
Disallow /kgf/
Disallow /tjenester/share/
Disallow /tjenester/esi
Disallow /l/
Disallow /nav/tekstttv/
Disallow /NR/exeres*
Disallow /nyheder/webdok/*/zone/*
Disallow /P3/find/
Disallow /programoversigttiltryk
Disallow /staging/
Disallow /stage/
Disallow /Templates/
Disallow /test/
Disallow /login/
Disallow /hydra/auth/
Disallow /search/*
Disallow /soeg/*
Disallow /drtv/soeg*
Disallow /lyd/programmer?q*
Disallow /cgi-bin/fttx1.exe/
Disallow /dr-laer/mediebiblioteket/
Disallow /download/Forside/2021/
Disallow /download/Forside/2022/
Disallow /nyhedsbrev/
Disallow /hydra/unilogin/
Disallow /undervisning_flash/

Other Records

Field Value
sitemap https://www.dr.dk/sitemapindex.xml