Log In Sign Up

propertyturkey.com
robots.txt

Robots Exclusion Standard data for propertyturkey.com

Resource Scan

Scan Details

Site Domain propertyturkey.com
Base Domain propertyturkey.com
Scan Status Ok
Last Scan2026-03-14T03:00:40+00:00
Next Scan 2026-04-13T03:00:40+00:00

Last Scan

Scanned2026-03-14T03:00:40+00:00
URL https://propertyturkey.com/robots.txt
Redirect https://www.propertyturkey.com/robots.txt
Redirect Domain www.propertyturkey.com
Redirect Base propertyturkey.com
Domain IPs 104.21.21.101, 172.67.197.239, 2606:4700:3030::6815:1565, 2606:4700:3031::ac43:c5ef
Redirect IPs 104.21.21.101, 172.67.197.239, 2606:4700:3030::6815:1565, 2606:4700:3031::ac43:c5ef
Response IP 172.67.197.239
Found Yes
Hash 85dcf77dafc4b7df7cd65643e1a50fbe1a1d27c014577b5eb469f5296d0fcc6b
SimHash 4d37da54ee37

Groups

*

Rule Path
Disallow /wp-admin/
Disallow /admin/
Disallow /login/
Disallow /cart/
Disallow /checkout/
Disallow /user/
Disallow /api/
Disallow /private/
Disallow /temp/
Allow /

gptbot

Rule Path
Allow /

chatgpt-user

Rule Path
Allow /

google-extended

Rule Path
Allow /

claude-web

Rule Path
Allow /

perplexitybot

Rule Path
Allow /

ccbot

Rule Path
Allow /
Back to top

Other Records

Field Value
sitemap https://www.propertyturkey.com/sitemap.xml
Back to top

Comments

  • robots.txt for https://www.propertyturkey.com
  • -------------------------------------------------
  • 1. GENERAL SETTINGS (Search Engines & Generic Bots)
  • -------------------------------------------------
  • Block standard backend/private areas to save crawl budget
  • Allow everything else
  • -------------------------------------------------
  • 2. AI & LLM SPECIFIC PERMISSIONS
  • Explicitly inviting major AI models to train/index
  • -------------------------------------------------
  • OpenAI (ChatGPT)
  • Google (AI Training/Overviews)
  • Anthropic (Claude)
  • Perplexity AI
  • Common Crawl (Used by many AIs)
  • -------------------------------------------------
  • 3. SITEMAPS (The "Map" for both Google & GPTs)
  • -------------------------------------------------
Back to top