Log In Sign Up

ra.org
robots.txt

Robots Exclusion Standard data for ra.org

Resource Scan

Scan Details

Site Domain ra.org
Base Domain ra.org
Scan Status Ok
Last Scan2024-09-27T23:43:23+00:00
Next Scan 2024-10-27T23:43:23+00:00

Last Scan

Scanned2024-09-27T23:43:23+00:00
URL https://ra.org/robots.txt
Redirect https://www.rainforest-alliance.org/robots.txt
Redirect Domain www.rainforest-alliance.org
Redirect Base rainforest-alliance.org
Domain IPs 20.231.98.17
Redirect IPs 104.22.26.184, 104.22.27.184, 172.67.28.47, 2606:4700:10::6816:1ab8, 2606:4700:10::6816:1bb8, 2606:4700:10::ac43:1c2f
Response IP 104.22.26.184
Found Yes
Hash aea27ce4e0cfe61314d34f428b782cc26a62c8eb4573452f71198437971b8666
SimHash b8909d090114

Groups

*

Rule Path
Disallow /wp-admin/
Allow /wp-admin/admin-ajax.php
Disallow /wp-content/plugins/
Disallow /readme.html
Disallow /wp-login.php
Disallow /?s=
Disallow /search/
Allow /wp-content/uploads/
Disallow /*.json
Disallow /*.php
Disallow /assets/
Back to top

Other Records

Field Value
sitemap https://www.rainforest-alliance.org/sitemap_index.xml
Back to top

Comments

  • robots.txt
  • This file is to prevent the crawling and indexing of certain parts
  • of your site by web crawlers and spiders run by sites like Yahoo!
  • and Google. By telling these "robots" where not to go on your site,
  • you save bandwidth and server resources.
  • This file will be ignored unless it is at the root of your host:
  • Used: http://example.com/robots.txt
  • Ignored: http://example.com/site/robots.txt
  • For more information about the robots.txt standard, see:
  • http://www.robotstxt.org/robotstxt.html
  • To test, use https://support.google.com/webmasters/answer/6062598?hl=en&ref_topic=6061961
  • Disable access to WP admin area...
  • ...but enable ajax stuff.
  • Hide specific core files
  • Don't index search results
  • Index the uploads if you find them
  • Hide files of these types
Back to top