Log In Sign Up

courses.centsibleketo.com
robots.txt

Robots Exclusion Standard data for courses.centsibleketo.com

Resource Scan

Scan Details

Site Domain courses.centsibleketo.com
Base Domain centsibleketo.com
Scan Status Ok
Last Scan2025-06-19T15:48:54+00:00
Next Scan 2025-07-19T15:48:54+00:00

Last Scan

Scanned2025-06-19T15:48:54+00:00
URL https://courses.centsibleketo.com/robots.txt
Domain IPs 13.35.37.21, 13.35.37.26, 13.35.37.41, 13.35.37.44, 2600:9000:213e:400:f:8d74:f980:93a1, 2600:9000:213e:5000:f:8d74:f980:93a1, 2600:9000:213e:7400:f:8d74:f980:93a1, 2600:9000:213e:7c00:f:8d74:f980:93a1, 2600:9000:213e:ba00:f:8d74:f980:93a1, 2600:9000:213e:ce00:f:8d74:f980:93a1, 2600:9000:213e:d400:f:8d74:f980:93a1, 2600:9000:213e:f600:f:8d74:f980:93a1
Response IP 13.35.37.41
Found Yes
Hash 52882d040b55cf2b3af329a1116a9c64615b8a24306e511fe32003e667e65851
SimHash 5830ed50a0f4

Groups

gptbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

yeti

Rule Path
Disallow /

*

Rule Path
Disallow /wp-admin/
Disallow /wp-login.php
Disallow /wp-content/
Disallow /wp-includes/
Disallow /xmlrpc.php

*

Rule Path
Allow /
Back to top

Other Records

Field Value
sitemap https://courses.centsibleketo.com/sitemap.xml
Back to top

Comments

  • Block known AI scrapers
  • Block WordPress-related paths
  • Optional: Allow all other bots to crawl the public site
Back to top