careerfoundry.com
robots.txt

Robots Exclusion Standard data for careerfoundry.com

Resource Scan

Scan Details

Site Domain careerfoundry.com
Base Domain careerfoundry.com
Scan Status Ok
Last Scan2025-07-04T17:37:40+00:00
Next Scan 2025-08-03T17:37:40+00:00

Last Scan

Scanned2025-07-04T17:37:40+00:00
URL https://careerfoundry.com/robots.txt
Domain IPs 108.157.150.17, 108.157.150.69, 108.157.150.76, 108.157.150.96, 2600:9000:2804:1a00:e:6233:aa00:93a1, 2600:9000:2804:7200:e:6233:aa00:93a1, 2600:9000:2804:9200:e:6233:aa00:93a1, 2600:9000:2804:ac00:e:6233:aa00:93a1, 2600:9000:2804:dc00:e:6233:aa00:93a1, 2600:9000:2804:e00:e:6233:aa00:93a1, 2600:9000:2804:fc00:e:6233:aa00:93a1, 2600:9000:2804:fe00:e:6233:aa00:93a1
Response IP 3.161.82.106
Found Yes
Hash c4efd8d9065fe70878fed3a4d89bf23aa62e61d8e6f40b4f1e24c60d15ed332f
SimHash f2ad4d83e5e4

Groups

*
*

Rule Path
Disallow /signup$
Disallow /signup/$
Disallow /signin$
Disallow /signin/$
Disallow /login$
Disallow /login/$
Disallow /*settings
Disallow /forgot_password$
Disallow /forgot_password/$
Disallow /*welcome
Disallow /*enroll
Disallow /*spots_available
Disallow /*payment_confirm
Disallow /*dashboard
Disallow /*themes/*
Disallow /*exercise/*
Disallow /*program_checkout/*
Disallow /*programme_checkout/*
Disallow *.pdf
Disallow /*careerhub*
Disallow /*course-plan
Disallow /*contract_billing_profiles/*
Disallow /*mentor_billing_profiles/*
Disallow /*all_notifications/*
Disallow /*submissions/*
Disallow /*user_achievements/*
Disallow /*course_extensions/*
Disallow /*communications/*
Disallow /*en/career/*
Disallow /*faq
Disallow /*course-guidebook
Disallow /*blog/authors/*
Disallow /*blog/?q=*
Disallow /*events/?event_topics=*
Disallow /*en/forms/*/
Disallow /*?*
Allow /dashboards/main

twitterbot

Rule Path
Allow /*?utm*

facebookexternalhit

Rule Path
Allow /*?utm*

Other Records

Field Value
sitemap https://careerfoundry.com/sitemap.xml
sitemap https://careerfoundry.com/en/video-sitemap.xml
sitemap http://app.wistia.com/sitemaps/66891.xml

Comments

  • See http://www.robotstxt.org/wc/norobots.html for documentation on how to use the robots.txt file
  • To ban all spiders from the entire site uncomment the next two lines: