hellenictechnologies.com
robots.txt

Robots Exclusion Standard data for hellenictechnologies.com

Resource Scan

Scan Details

Site Domain hellenictechnologies.com
Base Domain hellenictechnologies.com
Scan Status Ok
Last Scan2025-11-06T15:58:22+00:00
Next Scan 2025-12-06T15:58:22+00:00

Last Scan

Scanned2025-11-06T15:58:22+00:00
URL https://hellenictechnologies.com/robots.txt
Domain IPs 104.17.15.101
Response IP 104.17.15.101
Found Yes
Hash cfccd33a02ef6bf041990b59fb1d352617c912d09ff4537b85578ef782a9e6de
SimHash 581cc843ec53

Groups

*

Rule Path
Disallow /?s
Disallow /page*/?s
Disallow /search
Disallow /wp-json
Disallow /?rest_route=
Allow /

gptbot

Rule Path
Allow /

anthropic-ai

Rule Path
Allow /

google-extended

Rule Path
Allow /

ccbot

Rule Path
Allow /

cohere-ai

Rule Path
Allow /

youbot

Rule Path
Allow /

perplexitybot

Rule Path
Allow /

mistralbot

Rule Path
Allow /

andi-agent

Rule Path
Allow /

komo-ai

Rule Path
Allow /

neevabot

Rule Path
Allow /

facebookexternalhit

Rule Path
Allow /

metaai

Rule Path
Allow /

aleph-alpha

Rule Path
Allow /

phindbot

Rule Path
Allow /

*

Rule Path
Allow /

ahrefsbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

semrushbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

rogerbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

mj12bot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

similarwebbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

*

Rule Path
Disallow /private/
Allow /

Other Records

Field Value
sitemap https://hellenictechnologies.com/sitemap_index.xml

Comments

  • ============================
  • START YOAST BLOCK
  • ============================
  • ============================
  • END YOAST BLOCK
  • ============================
  • ============================
  • AI Crawlers Allowlist
  • ============================
  • Catch-all for unidentified AI crawlers
  • ============================
  • SEO & Marketing Bots (Throttled)
  • ============================
  • ============================
  • Default: Allow all but protect private
  • ============================