Log In Sign Up

globalsynturf.com
robots.txt

Robots Exclusion Standard data for globalsynturf.com

Resource Scan

Scan Details

Site Domain globalsynturf.com
Base Domain globalsynturf.com
Scan Status Ok
Last Scan2026-01-20T16:53:59+00:00
Next Scan 2026-02-19T16:53:59+00:00

Last Scan

Scanned2026-01-20T16:53:59+00:00
URL https://globalsynturf.com/robots.txt
Redirect https://www.globalsynturf.com/robots.txt
Redirect Domain www.globalsynturf.com
Redirect Base globalsynturf.com
Domain IPs 104.26.2.125, 104.26.3.125, 172.67.69.168, 2606:4700:20::681a:27d, 2606:4700:20::681a:37d, 2606:4700:20::ac43:45a8
Redirect IPs 104.26.2.125, 104.26.3.125, 172.67.69.168, 2606:4700:20::681a:27d, 2606:4700:20::681a:37d, 2606:4700:20::ac43:45a8
Response IP 104.26.3.125
Found Yes
Hash 30849a19c74494cfb8e63f57010e17a1563263fc8abfb363736cab525c712ac5
SimHash 50ac8a40ed72

Groups

googlebot

Rule Path
Allow /

google-extended

Rule Path
Allow /

googleother

Rule Path
Allow /

google-cloudvertexbot

Rule Path
Allow /

perplexitybot

Product Comment
perplexitybot Perplexity AI
Rule Path
Allow /

duckassistbot

Product Comment
duckassistbot DuckDuckGo AI
Rule Path
Allow /

gptbot

Product Comment
gptbot OpenAI
Rule Path
Disallow /

claudebot

Product Comment
claudebot Anthropic
Rule Path
Disallow /

ccbot

Product Comment
ccbot Common Crawl
Rule Path
Disallow /

bytespider

Product Comment
bytespider ByteDance / TikTok
Rule Path
Disallow /

tiktokspider

Product Comment
tiktokspider TikTok
Rule Path
Disallow /

meta-externalagent

Product Comment
meta-externalagent Meta/Facebook/Instagram
Rule Path
Disallow /

applebot

Product Comment
applebot Apple AI indexing
Rule Path
Disallow /

amazonbot

Product Comment
amazonbot Amazon AI
Rule Path
Disallow /

petalbot

Product Comment
petalbot Huawei AI
Rule Path
Disallow /

stabilityai

Product Comment
stabilityai Stability.ai crawlers
Rule Path
Disallow /

*

Rule Path
Allow /
Back to top

Comments

  • ------------------------
  • Allow Google SEO + AI
  • ------------------------
  • Google's AI / SGE crawlers
  • ------------------------
  • Block AI training bots
  • ------------------------
  • ------------------------
  • Default for others
  • ------------------------
Back to top