container-management.cioapplications.com
robots.txt

Resource Scan

Scan Details

Site Domain container-management.cioapplications.com
Base Domain cioapplications.com
Scan Status Ok
Last Scan2025-12-17T22:03:56+00:00
Next Scan 2026-01-16T22:03:56+00:00

Last Scan

Scanned2025-12-17T22:03:56+00:00
URL https://container-management.cioapplications.com/robots.txt
Domain IPs 52.11.180.226
Response IP 52.11.180.226
Found Yes
Hash 1f2734513134168fb57a1f54f230eae5dde1a30622949d71e03f6cd74c13fed4
SimHash 5911db91c691

Groups

*

Rule Path
Disallow /admin/
Disallow /scripts/
Disallow /*.php$
Disallow /zlib/
Disallow /lib/
Disallow /utils/
Allow /magazines/*

googlebot
googlebot-image
googlebot-news
googlebot-video
adsbot-google
mediapartners-google
adsbot-google-mobile-apps
bingbot
msnbot
slurp
yandex
duckduckbot
googlebot-mobile
msicrawler
linkedinbot
twitterbot
facebot
baiduspider

Rule Path
Allow /

screaming frog seo spider

Rule Path
Allow /

ahrefsbot

Rule Path
Allow /

ahrefssiteaudit

Rule Path
Allow /

yandexbot

Rule Path
Allow /

oai-searchbot

Rule Path
Allow /

chatgpt-user

Rule Path
Allow /

openai

Rule Path
Allow /

claude-searchbot

Rule Path
Allow /

perplexitybot

Rule Path
Allow /

perplexity-user

Rule Path
Allow /

claude-web

Rule Path
Allow /

*

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.cioapplications.com/cioapplicationnews_sitemap.xml
sitemap https://www.cioapplications.com/all_in_one_sitemap_main.xml
sitemap https://www.cioapplications.com/cioapplications_newnews_sitemap.xml
sitemap https://www.cioapplications.com/contributors_sitemap.xml
sitemap https://www.cioapplications.com/rankingxml_sitemap_new.xml
sitemap https://www.cioapplications.com/cioapplicationvendor_sitemap_new.xml

Comments

  • Allow major search engine bots
  • Allow specific tools
  • Allow AhrefsBot
  • Allow AhrefsSiteAudit
  • ===== OpenAI / ChatGPT / OAI bots =====
  • Anthropic
  • ===== Perplexity AI =====
  • ===== Other popular AI crawlers (examples) =====
  • Anthropic Claude
  • Block all other bots