Log In Sign Up

states.aarp.org
robots.txt

Robots Exclusion Standard data for states.aarp.org

Resource Scan

Scan Details

Site Domain states.aarp.org
Base Domain aarp.org
Scan Status Ok
Last Scan2026-01-28T23:31:59+00:00
Next Scan 2026-02-27T23:31:59+00:00

Last Scan

Scanned2026-01-28T23:31:59+00:00
URL https://states.aarp.org/robots.txt
Domain IPs 13.35.202.106, 13.35.202.112, 13.35.202.115, 13.35.202.117
Response IP 13.35.202.106
Found Yes
Hash a34bd4663c1540567c6f4ac587ab0b2d20855e4bdebee4ebd4204c47c507ce68
SimHash 2bea6bc66dd4

Groups

*

Rule Path
Disallow /*/tag/
Disallow /tag/
Disallow *.pdf
Back to top

Other Records

Field Value
sitemap https://states.aarp.org/sitemap.xml
Back to top

Comments

  • Robots.txt file created by https://www.aarp.org/
  • For domain: https://states.aarp.org/
  • Created 07-21-2021 Jim Huang - Block all tag pages. Remove legacy sitemap.
  • All robots will spider the domain
Back to top