capitalgroup.com
robots.txt

Robots Exclusion Standard data for capitalgroup.com

Resource Scan

Scan Details

Site Domain capitalgroup.com
Base Domain capitalgroup.com
Scan Status Ok
Last Scan2024-11-06T08:29:51+00:00
Next Scan 2024-11-20T08:29:51+00:00

Last Scan

Scanned2024-11-06T08:29:51+00:00
URL https://capitalgroup.com/robots.txt
Redirect https://www.capitalgroup.com:443/robots.txt
Redirect Domain www.capitalgroup.com
Redirect Base capitalgroup.com
Domain IPs 15.197.203.180, 3.33.191.209
Redirect IPs 118.215.80.44
Response IP 118.215.80.44
Found Yes
Hash b35ac3a36a39e2365d1366c9e0aa1532155af645c4f4465d937bc6b6974a3ea8
SimHash 21300a578de8

Groups

atomz/1.0

Rule Path
Disallow

ia_archiver

Rule Path
Disallow /

visweb

Rule Path
Disallow /

*

Rule Path
Disallow /*/error
Disallow /*/preferences
Allow /advisor/preferences/communication-preferences.htm
Disallow /*/accounts
Allow /*/accounts/login.htm
Disallow /*/pardon
Disallow /*/search
Disallow /advisor/merrill-lynch.html
Disallow /design
Disallow /content
Allow /content/dam/cgc/shared-content/sitemap/tcg/esg-sitemap.xml
Allow /content/dam/cgc/shared-content/documents/policies/DEI_Supplier_Code_of_conduct_072523.pdf
Allow /content/dam/*/Images/*
Allow /content/dam/*/images/*
Disallow /content/dam/*/images/*.pdf
Disallow /content/dam/*/Images/*.pdf
Allow /content/*.css
Allow /content/*.js
Allow /content/*.json
Disallow /global-errors/*
Disallow /*/_jcr_content/*
Allow /etc/designs/default/canvas/content/sites/*
Disallow /institutional/investments/fund/*
Allow /institutional/investments/fund/refgx
Disallow *.pdf

ultraseek

Rule Path
Disallow /advisor
Disallow /individual

twitterbot

Rule Path
Disallow

linkedinbot
bingbot

Rule Path
Disallow /pe/investment-strategy/our-value-add.html
Disallow /pe/portfolio/cipef-vi/go-to.html
Disallow /pe/investment-strategy/our-approach.html
Disallow /pe/investment-strategy.html
Disallow /pe/portfolio.html
Disallow /pm/
Disallow /ria/insights/news/article/citywire-price-points-are-too-high-janus-ceo-wary-of-overpaying-for-private-market-shops-a2442074
Disallow /institutional/insights/retirement/alts_in_public_pensions.html
Disallow /ria/insights/articles/what-to-know-about-alternatives.html
Disallow /ria/insights/news/article/citywire-tony-robbins-returns-with-alternative-investing-book-a2435654
Disallow /pe/our-team.html
Disallow /pe/about-us.html
Disallow /pe/

Other Records

Field Value
sitemap https://www.capitalgroup.com/sitemap_index.xml
sitemap https://www.capitalgroup.com/content/dam/cgc/shared-content/sitemap/tcg/esg-sitemap.xml

Comments

  • robots.txt for http://www.capitalgroup.com
  • Allow S&P
  • exclude ia_archiver and VisWeb robots
  • exclude directories
  • index exclusion for all PDF files
  • exclude advisor content from investor index
  • To enable Twitter and Linkedin access
  • Temporary removal for bingbot