mckinsey.com
robots.txt

Robots Exclusion Standard data for mckinsey.com

Resource Scan

Scan Details

Site Domain mckinsey.com
Base Domain mckinsey.com
Scan Status Ok
Last Scan2024-11-05T02:09:42+00:00
Next Scan 2024-11-19T02:09:42+00:00

Last Scan

Scanned2024-11-05T02:09:42+00:00
URL https://mckinsey.com/robots.txt
Redirect https://www.mckinsey.com/robots.txt
Redirect Domain www.mckinsey.com
Redirect Base mckinsey.com
Domain IPs 157.191.4.253
Redirect IPs 23.36.48.47
Response IP 23.208.168.12
Found Yes
Hash 958a3b9d0bb86a8a6cef4bc275d2d79167e909a48e86ecf5a400eb80b0fb24a8
SimHash e3472dbe6233

Groups

*

Rule Path
Disallow /*.axd
Disallow /modules/
Disallow /sitecore/
Disallow /layouts/
Disallow /Sublayouts/
Disallow /sitecore_files/
Disallow /upload/
Disallow /NotFound.aspx
Disallow /InternalError.html
Disallow /*undefined$
Disallow /services/forms/formapi.asmx/jsdebug
Disallow /services/forms/formapi.asmx/js
Disallow /error
Disallow /ClientLink/
Disallow /clientlink/
Disallow /AppData
Disallow /McKinsey/
Disallow /content/
Disallow /tools/Wrappers
Disallow /user_registration/
Disallow /user-registration
Disallow /search
Disallow /RSS/
Disallow /insights/rss.aspx
Disallow /nocookie
Disallow /noaccess
Disallow /html/
Disallow /assets/dotcom/staging/
Disallow /gsa/
Disallow /practice-clients/
Disallow /Practice-Clients/
Disallow /alumniappload/
Disallow /AlumniAppLoad/
Disallow /alumniappload
Disallow /AlumniAppLoad

gsa-crawler

Rule Path
Allow /clientlink/
Allow /gsa/
Allow /practice-clients/

amazon-kendra

Product Comment
amazon-kendra Amazon Kendra Web Crawler
Rule Path Comment
Disallow / disallow access to any pages

Other Records

Field Value
sitemap https://www.mckinsey.com/sitemap.xml

Comments

  • undefined
  • jsdebug
  • Miscellaneous exclusions

Warnings

  • 1 invalid line.