Log In Sign Up

arborday.org
robots.txt

Robots Exclusion Standard data for arborday.org

Resource Scan

Scan Details

Site Domain arborday.org
Base Domain arborday.org
Scan Status Ok
Last Scan2024-10-05T21:06:49+00:00
Next Scan 2024-11-04T21:06:49+00:00

Last Scan

Scanned2024-10-05T21:06:49+00:00
URL https://arborday.org/robots.txt
Redirect https://www.arborday.org/robots.txt
Redirect Domain www.arborday.org
Redirect Base arborday.org
Domain IPs 104.17.243.100, 104.17.244.100, 2606:4700::6811:f364, 2606:4700::6811:f464
Redirect IPs 104.17.243.100, 104.17.244.100, 2606:4700::6811:f364, 2606:4700::6811:f464
Response IP 104.17.243.100
Found Yes
Hash 4af75e005764194cf84a9257e3fd1fa0a5ae93b2c0a81632608cce7dee9a2eeb
SimHash a910ddb38f02

Groups

emailcollector

Rule Path
Disallow /

emailsiphon

Rule Path
Disallow /

emailwolf

Rule Path
Disallow /

mail sweeper

Rule Path
Disallow /

*

Rule Path
Disallow /Accounts/
Disallow /Acorn/
Disallow /Cart/
Disallow /programs/environmental-justice-intake/
Disallow /Shopping/Memberships/Renewals/
Disallow /programs/alliance-for-community-trees/environmental-justice/
Disallow /tracking/
Disallow /members/documents/keys/
Disallow /members/docs/keys/
Disallow /media/print/documents/
Back to top

Other Records

Field Value
sitemap https://www.arborday.org/sitemap.xml
Back to top

Comments

  • Robots.txt file.
  • Get rid of some of the big spam collectors
  • Nix out the account and cart
Back to top