betterthancash.org
robots.txt

Robots Exclusion Standard data for betterthancash.org

Resource Scan

Scan Details

Site Domain betterthancash.org
Base Domain betterthancash.org
Scan Status Ok
Last Scan2024-06-18T09:46:23+00:00
Next Scan 2024-07-18T09:46:23+00:00

Last Scan

Scanned2024-06-18T09:46:23+00:00
URL https://www.betterthancash.org/robots.txt
Domain IPs 23.22.5.68, 3.226.182.14, 52.21.227.162, 54.237.159.171
Response IP 3.226.182.14
Found Yes
Hash 5b4fb58484e636feb04f0e0fe54a6b1ff5641f5fcf77bbda0521b36d4398683b
SimHash 2420caf04fc1

Groups

twitterbot

Rule Path
Disallow *
Allow /images

Comments

  • Block all site crawlers by default.
  • Comment out the first disallow line to allow everything later.
  • Disallow: /api
  • Disallow: /404
  • Disallow: /406
  • Disallow: /422
  • Disallow: /500
  • Disallow: /503
  • Disallow: /504