Log In Sign Up

my.yoast.com
robots.txt

Robots Exclusion Standard data for my.yoast.com

Resource Scan

Scan Details

Site Domain my.yoast.com
Base Domain yoast.com
Scan Status Ok
Last Scan2026-02-08T16:22:25+00:00
Next Scan 2026-02-22T16:22:25+00:00

Last Scan

Scanned2026-02-08T16:22:25+00:00
URL https://my.yoast.com/robots.txt
Domain IPs 104.18.32.250, 172.64.155.6, 2606:4700:440c::ac40:9b06, 2a06:98c1:3100::6812:20fa
Response IP 172.64.155.6
Found Yes
Hash a5cd3a3d601ffa98f09e3e0a8652e44d7ccf7890cbd9f3ef2037b3ce6c3823d1
SimHash 22172e40fe82

Groups

*

Rule Path
Disallow /cdn-cgi/bm/cv/
Disallow /cdn-cgi/challenge-platform/
Back to top

Comments

  • If you want to learn about why our robots.txt looks like this, read this post: https://yoa.st/robots-txt
  • Prevent crawling of Cloudflare challenge URLs.
Back to top