nextdc.com
robots.txt

Robots Exclusion Standard data for nextdc.com

Resource Scan

Scan Details

Site Domain nextdc.com
Base Domain nextdc.com
Scan Status Ok
Last Scan2025-11-20T11:39:25+00:00
Next Scan 2025-12-20T11:39:25+00:00

Last Scan

Scanned2025-11-20T11:39:25+00:00
URL https://nextdc.com/robots.txt
Redirect https://www.nextdc.com/robots.txt
Redirect Domain www.nextdc.com
Redirect Base nextdc.com
Domain IPs 199.60.103.49
Redirect IPs 199.60.103.228, 199.60.103.28, 2606:2c40::c73c:671c, 2606:2c40::c73c:67e4
Response IP 199.60.103.28
Found Yes
Hash 8bf341e3cb3cdc262d9262d975c78462c62fdbdd493b404a4883a4efb3f11bb3
SimHash 51104961dc93

Groups

*

Rule Path
Allow /
Allow /search
Disallow /anseo-test/*
Disallow /*zlib/*
Disallow /*utils/*
Disallow /*clone/*
Disallow /_hcms/preview/
Disallow /hs/manage-preferences/
Disallow /hs/preferences-center/
Disallow /*?*hs_preview=*
Disallow /*?*hsCacheBuster=*

gptbot

Rule Path
Allow /
Disallow /_hcms/preview/
Disallow /hs/manage-preferences/
Disallow /hs/preferences-center/
Disallow /*?*hs_preview=*
Disallow /*?*hsCacheBuster=*

chatgpt-user

Rule Path
Allow /
Disallow /_hcms/preview/
Disallow /hs/manage-preferences/
Disallow /hs/preferences-center/
Disallow /*?*hs_preview=*
Disallow /*?*hsCacheBuster=*

openaiembeddings

Rule Path
Allow /
Disallow /_hcms/preview/
Disallow /hs/manage-preferences/
Disallow /hs/preferences-center/
Disallow /*?*hs_preview=*
Disallow /*?*hsCacheBuster=*

claudebot

Rule Path
Allow /
Disallow /_hcms/preview/
Disallow /hs/manage-preferences/
Disallow /hs/preferences-center/
Disallow /*?*hs_preview=*
Disallow /*?*hsCacheBuster=*

anthropic-ai

Rule Path
Allow /
Disallow /_hcms/preview/
Disallow /hs/manage-preferences/
Disallow /hs/preferences-center/
Disallow /*?*hs_preview=*
Disallow /*?*hsCacheBuster=*

perplexitybot

Rule Path
Allow /
Disallow /_hcms/preview/
Disallow /hs/manage-preferences/
Disallow /hs/preferences-center/
Disallow /*?*hs_preview=*
Disallow /*?*hsCacheBuster=*

ccbot

Rule Path
Allow /
Disallow /_hcms/preview/
Disallow /hs/manage-preferences/
Disallow /hs/preferences-center/
Disallow /*?*hs_preview=*
Disallow /*?*hsCacheBuster=*

grok

Rule Path
Allow /
Disallow /_hcms/preview/
Disallow /hs/manage-preferences/
Disallow /hs/preferences-center/
Disallow /*?*hs_preview=*
Disallow /*?*hsCacheBuster=*

gleanbot

Rule Path
Allow /
Disallow /_hcms/preview/
Disallow /hs/manage-preferences/
Disallow /hs/preferences-center/
Disallow /*?*hs_preview=*
Disallow /*?*hsCacheBuster=*

geminibot

Rule Path
Allow /
Disallow /_hcms/preview/
Disallow /hs/manage-preferences/
Disallow /hs/preferences-center/
Disallow /*?*hs_preview=*
Disallow /*?*hsCacheBuster=*

google-extended

Rule Path
Allow /
Disallow /_hcms/preview/
Disallow /hs/manage-preferences/
Disallow /hs/preferences-center/
Disallow /*?*hs_preview=*
Disallow /*?*hsCacheBuster=*

youbot

Rule Path
Allow /
Disallow /_hcms/preview/
Disallow /hs/manage-preferences/
Disallow /hs/preferences-center/
Disallow /*?*hs_preview=*
Disallow /*?*hsCacheBuster=*

phindbot

Rule Path
Allow /
Disallow /_hcms/preview/
Disallow /hs/manage-preferences/
Disallow /hs/preferences-center/
Disallow /*?*hs_preview=*
Disallow /*?*hsCacheBuster=*

neevabot

Rule Path
Allow /
Disallow /_hcms/preview/
Disallow /hs/manage-preferences/
Disallow /hs/preferences-center/
Disallow /*?*hs_preview=*
Disallow /*?*hsCacheBuster=*

amazonbot

Rule Path
Allow /
Disallow /_hcms/preview/
Disallow /hs/manage-preferences/
Disallow /hs/preferences-center/
Disallow /*?*hs_preview=*
Disallow /*?*hsCacheBuster=*

quorabot

Rule Path
Allow /
Disallow /_hcms/preview/
Disallow /hs/manage-preferences/
Disallow /hs/preferences-center/
Disallow /*?*hs_preview=*
Disallow /*?*hsCacheBuster=*

applebot

Rule Path
Allow /
Disallow /_hcms/preview/
Disallow /hs/manage-preferences/
Disallow /hs/preferences-center/
Disallow /*?*hs_preview=*
Disallow /*?*hsCacheBuster=*

aliyunbot

Rule Path
Allow /
Disallow /_hcms/preview/
Disallow /hs/manage-preferences/
Disallow /hs/preferences-center/
Disallow /*?*hs_preview=*
Disallow /*?*hsCacheBuster=*

Other Records

Field Value
sitemap https://www.nextdc.com/sitemap.xml

Comments

  • Allow search engines explicitly (redundant but clear)
  • Sitemap for all bots