Log In Sign Up

magicleap.com
robots.txt

Robots Exclusion Standard data for magicleap.com

Resource Scan

Scan Details

Site Domain magicleap.com
Base Domain magicleap.com
Scan Status Ok
Last Scan2024-06-09T04:07:00+00:00
Next Scan 2024-07-09T04:07:00+00:00

Last Scan

Scanned2024-06-09T04:07:00+00:00
URL https://magicleap.com/robots.txt
Redirect https://www.magicleap.com/robots.txt
Redirect Domain www.magicleap.com
Redirect Base magicleap.com
Domain IPs 104.18.28.206, 104.18.29.206, 2606:4700::6812:1cce, 2606:4700::6812:1dce
Redirect IPs 76.76.21.164, 76.76.21.9
Response IP 76.76.21.241
Found Yes
Hash 6a55419bbfc12a67ff0c3cc0aecff3f102a2fc6d3cf8e594fc8ee166d91ff6d3
SimHash 1a65081c46f0

Groups

*

Rule Path
Disallow /_hcms/preview/
Disallow /hs/manage-preferences/
Disallow /hs/preferences-center/
Disallow /blog-staging/*
Disallow /previews/*
Disallow /topic/*
Disallow /tag/*
Disallow /author/*
Disallow /*.json$
Disallow /*_buildManifest.js$
Disallow /*_middlewareManifest.js$
Disallow /*_ssgManifest.js$
Disallow /*.js$
Back to top

Other Records

Field Value
sitemap https://www.magicleap.com/sitemap.xml
Back to top

Comments

  • Next.JS Crawl Budget Performance Updates
  • Block files ending in .json, _buildManifest.js, _middlewareManifest.js, _ssgManifest.js, and any other JS files
  • The asterisks allows any file name
  • The dollar sign ensures it only matches the end of an URL and not a oddly formatted url (e.g. /locations.json.html)
Back to top