Log In Sign Up

epicenter.tech
robots.txt

Robots Exclusion Standard data for epicenter.tech

Resource Scan

Scan Details

Site Domain epicenter.tech
Base Domain epicenter.tech
Scan Status Ok
Last Scan2025-12-03T20:22:22+00:00
Next Scan 2026-01-02T20:22:22+00:00

Last Scan

Scanned2025-12-03T20:22:22+00:00
URL https://epicenter.tech/robots.txt
Domain IPs 104.26.12.229, 104.26.13.229, 172.67.71.114, 2606:4700:20::681a:ce5, 2606:4700:20::681a:de5, 2606:4700:20::ac43:4772
Response IP 104.26.12.229
Found Yes
Hash 6022045f59660af84080d639c76e6229a29270b314d7e1a12e9d9393918aa39b
SimHash 48990b9074cd

Groups

*

Rule Path
Allow /

googlebot

Rule Path
Allow /

google-extended

Rule Path
Allow /

googleother

Rule Path
Allow /

bingbot

Rule Path
Allow /

bingpreview

Rule Path
Allow /

applebot

Rule Path
Allow /

applebot-extended

Rule Path
Allow /

xai

Rule Path
Allow /

x.ai

Rule Path
Allow /

grok

Rule Path
Allow /

deepseekbot

Rule Path
Allow /

gptbot

Rule Path
Allow /

oai-searchbot

Rule Path
Allow /

chatgpt-user

Rule Path
Allow /

claudebot

Rule Path
Allow /

claude-searchbot

Rule Path
Allow /

claude-user

Rule Path
Allow /

perplexitybot

Rule Path
Allow /

ccbot

Rule Path Comment
Allow / Common Crawl (feeds many AI datasets)
Back to top

Other Records

Field Value
sitemap https://epicenter.tech/sitemap_index.xml
Back to top

Comments

  • START YOAST BLOCK
  • ---------------------------
  • ---------------------------
  • END YOAST BLOCK
  • Standard Search Engine Bots
  • AI / LLM / Generative Engine Crawlers
Back to top