Log In Sign Up

csi.ca
robots.txt

Robots Exclusion Standard data for csi.ca

Resource Scan

Scan Details

Site Domain csi.ca
Base Domain csi.ca
Scan Status Ok
Last Scan2024-10-29T17:06:59+00:00
Next Scan 2024-11-28T17:06:59+00:00

Last Scan

Scanned2024-10-29T17:06:59+00:00
URL https://csi.ca/robots.txt
Redirect https://www.csi.ca/robots.txt
Redirect Domain www.csi.ca
Redirect Base csi.ca
Domain IPs 15.222.118.237, 35.183.152.104
Redirect IPs 15.222.118.237, 35.183.152.104
Response IP 35.183.152.104
Found Yes
Hash af9f91a57642ba3bb1eeb1b70713dc631286347d65c4fc8016b05662ef0bd1d4
SimHash a0119b51d4c3

Groups

chatgpt-user

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

*

Rule Path
Disallow /en/page-sitemap.xml
Disallow /fr/page-sitemap.xml
Disallow /post-sitemap.xml
Disallow /slider-sitemap.xml
Disallow /mega_menu-sitemap.xml
Disallow /webinars-sitemap.xml
Disallow /events-sitemap.xml
Disallow /press_releases-sitemap.xml
Disallow /insights-sitemap.xml
Disallow /da_image-sitemap.xml
Disallow /category-sitemap.xml
Disallow /popular_topics-sitemap.xml
Disallow /content_format-sitemap.xml
Disallow /author-sitemap.xml
Disallow /student/
Disallow /en/slider/
Disallow /fr/slider/
Disallow /en/search/
Disallow /fr/search/
Disallow /en/popular_topics/
Disallow /fr/popular_topics/
Disallow /en/content_format/
Disallow /fr/content_format/
Back to top

Other Records

Field Value
sitemap https://www.csi.ca/page-sitemap.xml
Back to top

Comments

  • Blocks access to auto generated sitemaps that are not being used and can't be removed
  • Blocks access to the Old Website server
  • Blocks access to image sliders on home page where each slider is its own page on the website.
  • Blocks access to search queries
  • Blocks access to popular topics for insights pages. These show up as individual pages
  • Blocks access to content format for insights pages. These show up as individual pages
  • Allows access to updated sitemaps.
Back to top