rtl.lu
robots.txt

Robots Exclusion Standard data for rtl.lu

Resource Scan

Scan Details

Site Domain rtl.lu
Base Domain rtl.lu
Scan Status Ok
Last Scan2024-09-20T07:48:32+00:00
Next Scan 2024-09-27T07:48:32+00:00

Last Scan

Scanned2024-09-20T07:48:32+00:00
URL https://rtl.lu/robots.txt
Redirect https://www.rtl.lu/robots.txt
Redirect Domain www.rtl.lu
Redirect Base rtl.lu
Domain IPs 81.92.238.105, 81.92.238.106
Redirect IPs 81.92.238.105, 81.92.238.106
Response IP 81.92.238.105
Found Yes
Hash 654738163812e512270dee53e993da1b09fa59f76894a542aa6f389c4d972e78
SimHash 809cdc00d974

Groups

twitterbot

Rule Path
Disallow

*

Rule Path
Disallow /__IPL/
Disallow /__IPL_DFP/
Disallow /__IPL_VIDEO/
Disallow /actu/
Disallow /aktiounen/app/
Disallow /news/sponsored-content-en/
Disallow /*/*/ra/*

gptbot

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

Comments

  • OpenAI ChatGPT
  • Google Bard
  • Common Crawl Foundation