Log In Sign Up

aarp.org
robots.txt

Robots Exclusion Standard data for aarp.org

Resource Scan

Scan Details

Site Domain aarp.org
Base Domain aarp.org
Scan Status Ok
Last Scan2024-11-02T08:56:57+00:00
Next Scan 2024-11-09T08:56:57+00:00

Last Scan

Scanned2024-11-02T08:56:57+00:00
URL https://aarp.org/robots.txt
Redirect https://www.aarp.org/robots.txt
Redirect Domain www.aarp.org
Redirect Base aarp.org
Domain IPs 45.60.152.55, 45.60.172.55
Redirect IPs 13.35.210.108, 13.35.210.15, 13.35.210.59, 13.35.210.64
Response IP 13.35.210.59
Found Yes
Hash 138e04d7507365f60083143b3c7034e85870328dd0ae719b122bbae1ff00b88d
SimHash 38a2ffc00d55

Groups

*

Rule Path
Allow /rewards/$
Disallow /*.pdf
Disallow /benefits/
Disallow /content/aarpe/en/home/
Disallow /content/aarpe/es/home/
Disallow /content/aarpe/es/home/vendor/
Disallow /content/campaigns/
Disallow /content/dam/aarp/Member-Benefits/AARP-Auto-Buying-Program-For-Member-Communications-Digital-Fulfillment.pdf
Disallow /content/experience-fragments/
Disallow /content/specialized-membership/staying-sharp/
Disallow /espanol/busqueda/
Disallow /espanol/vendor/
Disallow /everywhere/
Disallow /everywhere-configuration/
Disallow /playground-for-testing-stuff/
Disallow /qa/
Disallow /relationships/caregiving/
Disallow /rewards/
Disallow /search/
Disallow /secure-information/
Disallow /training-aarpe/
Disallow /vendor/
Back to top

Other Records

Field Value
sitemap https://www.aarp.org/sitemap.xml
Back to top

Comments

  • _____ _____
  • /\ /\ | __ \ | __ \
  • / \ / \ | |__) | | |__) |
  • / /\ \ / /\ \ | _ / | ___/
  • / ____ \ / ____ \ | | \ \ | |
  • /_/ \_\ /_/ \_\ |_| \_\ |_|
  • Robots.txt file created by https://www.aarp.org/
  • For domain: https://www.aarp.org/
  • Created 09-12-2017 Raymond Deschenes
  • Updated 10-21-2024 Kasia Lorenc - added /rewards/
  • All robots will spider the domain
Back to top