Log In Sign Up

cdiproducts.com
robots.txt

Robots Exclusion Standard data for cdiproducts.com

Resource Scan

Scan Details

Site Domain cdiproducts.com
Base Domain cdiproducts.com
Scan Status Ok
Last Scan2026-01-01T08:00:10+00:00
Next Scan 2026-01-31T08:00:10+00:00

Last Scan

Scanned2026-01-01T08:00:10+00:00
URL https://cdiproducts.com/robots.txt
Redirect https://www.cdiproducts.com/robots.txt
Redirect Domain www.cdiproducts.com
Redirect Base cdiproducts.com
Domain IPs 199.60.103.168, 199.60.103.68
Redirect IPs 199.60.103.225, 199.60.103.31, 2606:2c40::c73c:671f, 2606:2c40::c73c:67e1
Response IP 199.60.103.225
Found Yes
Hash 65b4cd319ade241218aa9639874a492a71703de2304086b77c2faf5f6308f2c2
SimHash 689408e0e753

Groups

*

Rule Path
Disallow /email/
Disallow /confirmation/
Disallow /www.cdiproducts.com/online-dures-calculator
Disallow /gated-downloads/
Disallow /blog/tag/
Disallow /blog/topic/
Disallow /blog/author/
Disallow /news-and-events/tag/
Disallow /news-and-events/topic/
Disallow /news-and-events/author/
Disallow /_hcms/preview/
Disallow /hs/manage-preferences/
Disallow /hs/preferences-center/
Disallow /*?*hs_preview=*
Disallow /*?*hsCacheBuster=*

bingbot

Rule Path
Disallow /online-dures-calculator
Disallow /_hcms/preview/
Disallow /hs/manage-preferences/
Disallow /hs/preferences-center/
Disallow /*?*hs_preview=*
Disallow /*?*hsCacheBuster=*

duckduckbot

Rule Path
Disallow /online-dures-calculator
Disallow /_hcms/preview/
Disallow /hs/manage-preferences/
Disallow /hs/preferences-center/
Disallow /*?*hs_preview=*
Disallow /*?*hsCacheBuster=*

googlebot

Rule Path
Disallow /online-dures-calculator
Disallow /_hcms/preview/
Disallow /hs/manage-preferences/
Disallow /hs/preferences-center/
Disallow /*?*hs_preview=*
Disallow /*?*hsCacheBuster=*

yandex

Rule Path
Disallow /online-dures-calculator
Disallow /_hcms/preview/
Disallow /hs/manage-preferences/
Disallow /hs/preferences-center/
Disallow /*?*hs_preview=*
Disallow /*?*hsCacheBuster=*

baiduspider

Rule Path
Disallow /online-dures-calculator
Disallow /_hcms/preview/
Disallow /hs/manage-preferences/
Disallow /hs/preferences-center/
Disallow /*?*hs_preview=*
Disallow /*?*hsCacheBuster=*

sogou

Rule Path
Disallow /online-dures-calculator
Disallow /_hcms/preview/
Disallow /hs/manage-preferences/
Disallow /hs/preferences-center/
Disallow /*?*hs_preview=*
Disallow /*?*hsCacheBuster=*

exabot

Rule Path
Disallow /online-dures-calculator
Disallow /_hcms/preview/
Disallow /hs/manage-preferences/
Disallow /hs/preferences-center/
Disallow /*?*hs_preview=*
Disallow /*?*hsCacheBuster=*

facebot

Rule Path
Disallow /online-dures-calculator
Disallow /_hcms/preview/
Disallow /hs/manage-preferences/
Disallow /hs/preferences-center/
Disallow /*?*hs_preview=*
Disallow /*?*hsCacheBuster=*

oai-searchbot

Rule Path
Allow /
Disallow /_hcms/preview/
Disallow /hs/manage-preferences/
Disallow /hs/preferences-center/
Disallow /*?*hs_preview=*
Disallow /*?*hsCacheBuster=*

gptbot

Rule Path
Allow /
Disallow /_hcms/preview/
Disallow /hs/manage-preferences/
Disallow /hs/preferences-center/
Disallow /*?*hs_preview=*
Disallow /*?*hsCacheBuster=*

claudebot

Rule Path
Allow /
Disallow /_hcms/preview/
Disallow /hs/manage-preferences/
Disallow /hs/preferences-center/
Disallow /*?*hs_preview=*
Disallow /*?*hsCacheBuster=*

claude-web

Rule Path
Allow /
Disallow /_hcms/preview/
Disallow /hs/manage-preferences/
Disallow /hs/preferences-center/
Disallow /*?*hs_preview=*
Disallow /*?*hsCacheBuster=*

perplexitybot

Rule Path
Allow /
Disallow /_hcms/preview/
Disallow /hs/manage-preferences/
Disallow /hs/preferences-center/
Disallow /*?*hs_preview=*
Disallow /*?*hsCacheBuster=*

google-extended

Rule Path
Allow /
Disallow /_hcms/preview/
Disallow /hs/manage-preferences/
Disallow /hs/preferences-center/
Disallow /*?*hs_preview=*
Disallow /*?*hsCacheBuster=*

amazonbot

Rule Path
Allow /
Disallow /_hcms/preview/
Disallow /hs/manage-preferences/
Disallow /hs/preferences-center/
Disallow /*?*hs_preview=*
Disallow /*?*hsCacheBuster=*
Back to top

Other Records

Field Value
sitemap https://www.cdiproducts.com/sitemap.xml
Back to top

Comments

  • ============================
  • OpenAI bots
  • ============================
  • For ChatGPT search results
  • For training OpenAI's models
  • ============================
  • Anthropic (Claude)
  • ============================
  • ============================
  • Perplexity
  • ============================
  • ============================
  • Google AI / Extended
  • (content allowed for AI features)
  • ============================
  • ============================
  • Amazonbot (AI & search-related)
  • ============================
Back to top