Log In Sign Up

coursehorse.com
robots.txt

Robots Exclusion Standard data for coursehorse.com

Resource Scan

Scan Details

Site Domain coursehorse.com
Base Domain coursehorse.com
Scan Status Ok
Last Scan2025-04-06T23:17:10+00:00
Next Scan 2025-05-06T23:17:10+00:00

Last Scan

Scanned2025-04-06T23:17:10+00:00
URL https://coursehorse.com/robots.txt
Domain IPs 172.66.40.149, 172.66.43.107, 2606:4700:3108::ac42:2895, 2606:4700:3108::ac42:2b6b
Response IP 172.66.40.149
Found Yes
Hash 183117f25cb33dfef8f941ded4ecc031b1d88481e9f483cc641a0e499782592f
SimHash 6a5ccd82a6d2

Groups

*

Rule Path
Disallow /user/
Disallow /team2/
Disallow /course/checkout/
Disallow /course/checkout/enter-info
Disallow /course/checkout/enter-info?schedule=
Disallow /cart
Disallow /school-admin-wiki
Disallow /gift-card?schedule=
Disallow /gift-card?deal=
Disallow /*?deal
Disallow /course/index/recommendations
Disallow /course/checkout/info-session
Disallow /course/checkout/confirm
Disallow /home
Disallow /goal-pledge
Disallow /*/*/schedule

baiduspider

Rule Path
Disallow /

yandex

Rule Path
Disallow /

megaindex

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

istellabot

Rule Path
Disallow /

exabot

Rule Path
Disallow /

domainappender

Rule Path
Disallow /

vegebot

Rule Path
Disallow /

betabot

Rule Path
Disallow /

smtbot

Rule Path
Disallow /

checkdogbt

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

slurp

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

ahrefsbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

rogerbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

dotbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

nextgensearchbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

amazonbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10
Back to top

Comments

  • robots.txt file for CourseHorse.com
  • Block bots that aren't relevant
  • Slow down bots that are heavy handed
Back to top