Log In Sign Up

retireetoday.com
robots.txt

Robots Exclusion Standard data for retireetoday.com

Resource Scan

Scan Details

Site Domain retireetoday.com
Base Domain retireetoday.com
Scan Status Ok
Last Scan2025-04-02T14:38:26+00:00
Next Scan 2025-04-09T14:38:26+00:00

Last Scan

Scanned2025-04-02T14:38:26+00:00
URL https://retireetoday.com/robots.txt
Domain IPs 104.21.37.81, 172.67.206.32, 2606:4700:3030::6815:2551, 2606:4700:3035::ac43:ce20
Response IP 172.67.206.32
Found Yes
Hash bdaeb282c99b17a648455d2d3684515db2de4e8ff5436829995c640213927bba
SimHash 10149506ec52

Groups

ahrefsbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

semrushbot-ba

Rule Path
Disallow /

semrushbot-si

Rule Path
Disallow /

semrushbot-swa

Rule Path
Disallow /

semrushbot-ct

Rule Path
Disallow /

semrushbot-bm

Rule Path
Disallow /

rogerbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

xenu

Rule Path
Disallow /
Back to top

Comments

  • Block Ahrefs Crawler
  • Block Semrush Crawler
  • Block Moz Crawler
  • Block Majestic Crawler
  • Block Xenu Crawler
Back to top