Log In Sign Up

symmetrical.ai
robots.txt

Robots Exclusion Standard data for symmetrical.ai

Resource Scan

Scan Details

Site Domain symmetrical.ai
Base Domain symmetrical.ai
Scan Status Ok
Last Scan2025-10-06T16:35:39+00:00
Next Scan 2025-11-05T16:35:39+00:00

Last Scan

Scanned2025-10-06T16:35:39+00:00
URL https://symmetrical.ai/robots.txt
Redirect https://www.symmetrical.ai/robots.txt
Redirect Domain www.symmetrical.ai
Redirect Base symmetrical.ai
Domain IPs 3.175.86.111, 3.175.86.115, 3.175.86.77, 3.175.86.86
Redirect IPs 13.203.125.58, 13.233.175.166, 3.109.243.18
Response IP 57.180.160.60
Found Yes
Hash e0a319b0faee0a739e4cb1897cb1c2b6d268303737d99e83304c49668c0e490e
SimHash 641b885157e4

Groups

gptbot

Product Comment
gptbot OpenAI (ChatGPT model training)
Rule Path
Allow /

oai-searchbot

Product Comment
oai-searchbot OpenAI (ChatGPT search integration)
Rule Path
Allow /

google-extended

Product Comment
google-extended Google (Gemini & Search Generative Experience)
Rule Path
Allow /

anthropic-ai

Product Comment
anthropic-ai Anthropic (Claude model training)
Rule Path
Allow /

claudebot

Product Comment
claudebot Anthropic (Claude answer-engine linker)
Rule Path
Allow /

perplexitybot

Product Comment
perplexitybot Perplexity AI (Answer Engine)
Rule Path
Allow /

applebot-extended

Product Comment
applebot-extended Apple (On-device LLM & AI features)
Rule Path
Allow /

cohere-ai

Product Comment
cohere-ai Cohere (Enterprise AI models)
Rule Path
Allow /

bytespider

Product Comment
bytespider ByteDance (TikTok's parent company AI)
Rule Path
Allow /

youbot

Product Comment
youbot You.com (Conversational search engine)
Rule Path
Allow /

*

Rule Path
Allow /
Back to top

Other Records

Field Value
sitemap https://www.symmetrical.ai/sitemap.xml
sitemap https://www.symmetrical.ai/sitemap.xml
Back to top

Comments

  • Symmetrical.ai – robots.txt
  • AEO (Answer Engine Optimization) Gold Standard
  • Last Updated: 16-06-2025
  • ---------------------------------------------------------------
  • 1. AI & LLM CRAWLERS — FULL & EXPLICIT ACCESS
  • We grant full access to all known AI bots to ensure our content
  • is used for training and providing answers.
  • ---------------------------------------------------------------
  • ---------------------------------------------------------------
  • 2. LEGACY & GENERAL CRAWLERS — DEFAULT ACCESS
  • This rule applies to all other bots not specified above,
  • including standard search engines like Googlebot and Bingbot.
  • ---------------------------------------------------------------
  • ---------------------------------------------------------------
  • 3. SITEMAP LOCATION
  • The single most important directive: tells all bots where to
  • find the map of our entire website.
  • ---------------------------------------------------------------
Back to top