Log In Sign Up

goaztecs.com
robots.txt

Robots Exclusion Standard data for goaztecs.com

Resource Scan

Scan Details

Site Domain goaztecs.com
Base Domain goaztecs.com
Scan Status Ok
Last Scan2025-08-28T12:11:24+00:00
Next Scan 2025-09-04T12:11:24+00:00

Last Scan

Scanned2025-08-28T12:11:24+00:00
URL https://goaztecs.com/robots.txt
Domain IPs 34.149.245.161
Response IP 34.149.245.161
Found Yes
Hash 94727dba1243412e8c677eb1b8b3f85c30f7e586c830f1e3d351d4d17d1b5d5e
SimHash 7b1ec150b7f2

Groups

gptbot
oai-searchbot
chatgpt-user
chatgpt-user/2.0

Rule Path
Allow /

anthropic-ai
claudebot
claude-web

Rule Path
Allow /

perplexitybot
perplexity-user

Rule Path
Allow /

google-extended

Rule Path
Allow /

bingbot

Rule Path
Allow /

amazonbot

Rule Path
Allow /

applebot
applebot-extended

Rule Path
Allow /

facebookbot
meta-externalagent

Rule Path
Allow /

ccbot
ai2bot
diffbot

Rule Path
Allow /

bytespider
duckassistbot
cohere-ai
timpibot
youbot

Product Comment
bytespider ByteDance / TikTok
duckassistbot DuckDuckGo
cohere-ai Cohere
Rule Path
Allow /

*

Rule Path
Allow /
Back to top

Other Records

Field Value
sitemap https://goaztecs.com/website-api/sitemap
sitemap https://goaztecs.com/sitemap.xml
Back to top

Comments

  • OPENAI
  • ANTHROPIC / CLAUDE
  • PERPLEXITY
  • GOOGLE GEMINI
  • MICROSOFT BING / COPILOT
  • AMAZON
  • APPLE
  • META / FACEBOOK
  • COMMON CRAWL & RESEARCH
  • OTHER EMERGING BOTS
  • CATCH-ALL (optional)
Back to top