Log In Sign Up

rtbf.be
robots.txt

Robots Exclusion Standard data for rtbf.be

Resource Scan

Scan Details

Site Domain rtbf.be
Base Domain rtbf.be
Scan Status Ok
Last Scan2024-11-16T07:00:26+00:00
Next Scan 2024-11-23T07:00:26+00:00

Last Scan

Scanned2024-11-16T07:00:26+00:00
URL https://rtbf.be/robots.txt
Redirect https://www.rtbf.be/robots.txt
Redirect Domain www.rtbf.be
Redirect Base rtbf.be
Domain IPs 151.101.1.63, 151.101.129.63, 151.101.193.63, 151.101.65.63
Redirect IPs 151.101.1.63, 151.101.129.63, 151.101.193.63, 151.101.65.63, 2a04:4e42:200::319, 2a04:4e42:400::319, 2a04:4e42:600::319, 2a04:4e42::319
Response IP 199.232.45.63
Found Yes
Hash 975c6342708daf7d98ed278da589beb6ebe1179997a896b94440facfa22c96de
SimHash ff548161adb3

Groups

*

Rule Path
Allow /

*

Rule Path
Disallow /embed

*

Rule Path
Disallow /embed/

*

Rule Path
Disallow /auvio/embed

*

Rule Path
Disallow /auvio/embed/

gptbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

claude-web

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://www.rtbf.be/sitemap.xml
sitemap https://www.rtbf.be/site-map/archives.xml
sitemap https://www.rtbf.be/site-map/articles.xml
sitemap https://www.rtbf.be/site-map/articles5000.xml
sitemap https://www.rtbf.be/site-map/dossiers.xml
sitemap https://www.rtbf.be/site-map/en-continu.xml
sitemap https://www.rtbf.be/site-map/programs.xml
sitemap https://www.rtbf.be/site-map/history-sitemap.xml
sitemap https://www.rtbf.be/site-map/menu.xml
sitemap https://www.rtbf.be/site-map/elections2024.xml
Back to top

Comments

  • *
  • *
  • *
  • *
  • *
  • GPTBot
  • ChatGPT-User
  • Google-Extended
  • CCbot
  • anthropic-ai
  • Claude-Web
  • Host
  • Sitemaps
Back to top

Warnings

  • `host` is not a known field.
Back to top