Log In Sign Up

ecoonomia.com
robots.txt

Robots Exclusion Standard data for ecoonomia.com

Resource Scan

Scan Details

Site Domain ecoonomia.com
Base Domain ecoonomia.com
Scan Status Ok
Last Scan2026-03-10T08:52:02+00:00
Next Scan 2026-04-09T08:52:02+00:00

Last Scan

Scanned2026-03-10T08:52:02+00:00
URL https://ecoonomia.com/robots.txt
Domain IPs 104.21.55.59, 172.67.145.89, 2606:4700:3034::6815:373b, 2606:4700:3036::ac43:9159
Response IP 104.21.55.59
Found Yes
Hash 3ccc4c201dd65bdcbc54c04758aeff239a807628273bc62645c29afc3f659576
SimHash 6816da00e4ce

Groups

*

Rule Path
Disallow /wp-admin/
Allow /wp-admin/admin-ajax.php
Allow /*.css$
Allow /*.js$
Allow /wp-content/uploads/
Disallow /?s=
Disallow /search/
Disallow /trackback/
Disallow /comments/
Disallow */feed/
Disallow */rss/
Disallow /*?replytocom=
Disallow /*?utm_*
Disallow /*?fbclid=

googlebot

Rule Path
Allow /

googlebot-image

Rule Path
Allow /

googlebot-news

Rule Path
Allow /

google-extended

Rule Path
Allow /

bingbot

Rule Path
Allow /

duckduckbot

Rule Path
Allow /

applebot

Rule Path
Allow /

applebot-image

Rule Path
Allow /

yandexbot

Rule Path
Allow /

baiduspider

Rule Path
Allow /

petalbot

Rule Path
Allow /

yeti

Rule Path
Allow /

seznambot

Rule Path
Allow /

sogou web spider

Rule Path
Allow /

gptbot

Rule Path
Allow /

oai-searchbot

Rule Path
Allow /

perplexitybot

Rule Path
Allow /

claudebot

Rule Path
Allow /

claude-user

Rule Path
Allow /

ccbot

Rule Path
Allow /

meta-externalagent

Rule Path
Allow /

googlebot-image

Rule Path
Allow /wp-content/uploads/
Back to top

Other Records

Field Value
sitemap https://ecoonomia.com/sitemap_index.xml
sitemap https://ecoonomia.com/news-sitemap.xml
Back to top

Comments

  • WordPress robots.txt — allow major search + AI/LLM crawlers
  • Global rules (apply to all crawlers)
  • Keep WP admin private, but allow AJAX
  • Make sure assets and images are crawlable
  • Avoid thin/duplicate pages
  • Popular SEARCH ENGINES — explicit allow
  • Google
  • (AI usage preference bot)
  • Microsoft Bing
  • DuckDuckGo
  • Apple (Apple Search / Spotlight / Siri)
  • Yandex
  • Baidu
  • Huawei Petal Search
  • Naver (Korea)
  • Seznam (Czech)
  • Sogou (China)
  • AI / LLM CRAWLERS — explicit allow
  • OpenAI (web crawler for models)
  • OpenAI (search indexing for ChatGPT)
  • Perplexity AI
  • Anthropic (Claude)
  • On-demand fetches from Claude when users open links
  • Common Crawl (training corpus used by many AI systems)
  • Meta (Facebook/Instagram) AI/content fetcher
  • Images (extra clarity for Google Images)
  • Sitemap
Back to top