Log In Sign Up

gem.cbc.ca
robots.txt

Robots Exclusion Standard data for gem.cbc.ca

Resource Scan

Scan Details

Site Domain gem.cbc.ca
Base Domain cbc.ca
Scan Status Ok
Last Scan2025-12-04T04:06:14+00:00
Next Scan 2025-12-18T04:06:14+00:00

Last Scan

Scanned2025-12-04T04:06:14+00:00
URL https://gem.cbc.ca/robots.txt
Domain IPs 125.252.233.183, 2600:1413:5000:f80::16be, 2600:1413:5000:f84::16be
Response IP 23.202.132.88
Found Yes
Hash a2f3911bad0c786a5adcbd49d647324f449a72cb7949a61213b0a777bf3de3d5
SimHash 74110150a4b0

Groups

anthropic-ai

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

claude-web

Rule Path
Disallow /

cohere-ai

Rule Path
Disallow /

deepseekbot

Rule Path
Disallow /

deepseek

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

oai-searchbot

Rule Path
Disallow /

omgili

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

perplexitybot

Rule Path
Disallow /

perplexity-user

Rule Path
Disallow /

*

Rule Path
Disallow /api/health
Back to top

Other Records

Field Value
sitemap https://gem.cbc.ca/sitemaps/index.xml
Back to top

Comments

  • ___ ___ ___
  • /\ \ /\ \ /\__\
  • /::\ \ /::\ \ /::| |
  • /:/\:\ \ /:/\:\ \ /:|:| |
  • /:/ \:\ \ /::\~\:\ \ /:/|:|__|__
  • /:/__/_\:\__\ /:/\:\ \:\__\ /:/ |::::\__\
  • \:\ /\ \/__/ \:\~\:\ \/__/ \/__/~~/:/ /
  • \:\ \:\__\ \:\ \:\__\ /:/ /
  • \:\/:/ / \:\ \/__/ /:/ /
  • \::/ / \:\__\ /:/ /
  • \/__/ \/__/ \/__/
Back to top