Log In Sign Up

balkanscout.com
robots.txt

Robots Exclusion Standard data for balkanscout.com

Resource Scan

Scan Details

Site Domain balkanscout.com
Base Domain balkanscout.com
Scan Status Ok
Last Scan2025-12-13T07:52:31+00:00
Next Scan 2025-12-20T07:52:31+00:00

Last Scan

Scanned2025-12-13T07:52:31+00:00
URL https://balkanscout.com/robots.txt
Domain IPs 2001:8d8:100f:f000::200, 217.160.0.198
Response IP 217.160.0.198
Found Yes
Hash a3d1da0b07ec2d637b6fface4bbd7fa4d8ef689473e0307d1105dde64ac74e83
SimHash 650287d3c4b0

Groups

*

Rule Path
Allow /

mediapartners-google

Rule Path
Allow /

adsbot-google

Rule Path
Allow /

googlebot-image

Rule Path
Allow /

*

Rule Path
Allow /static/css/
Allow /static/js/
Allow /*.css
Allow /*.js

*

Rule Path
Allow /images/
Allow /static/images/
Allow /*.jpg
Allow /*.jpeg
Allow /*.png
Allow /*.gif
Allow /*.webp
Allow /*.svg
Back to top

Other Records

Field Value
sitemap https://balkanscout.com/sitemap.xml
Back to top

Comments

  • robots.txt for balkanscout.com
  • Real Estate Property Listing Website
  • Allow all search engines to crawl everything
  • Allow Google AdSense bot (important for ad serving)
  • Allow Google AdSense verification
  • Allow Google Images (for property photos)
  • Disallow admin/backend pages (if you have any)
  • User-agent: *
  • Disallow: /admin/
  • Disallow: /wp-admin/
  • Disallow: /dashboard/
  • Disallow search result pages (if you have internal search)
  • User-agent: *
  • Disallow: /search?
  • Disallow: /*?q=
  • Disallow API endpoints (if you have any)
  • User-agent: *
  • Disallow: /api/
  • Disallow: /api/*
  • Disallow duplicate content (if you have filters/sorting)
  • User-agent: *
  • Disallow: /*?sort=
  • Disallow: /*?filter=
  • Disallow: /*&page=
  • Allow CSS and JavaScript files (important for Google PageSpeed)
  • Allow image files (important for property photos)
  • Crawl-delay (optional - only if you have server performance issues)
  • User-agent: *
  • Crawl-delay: 1
  • Sitemap location (update with your actual sitemap URL when you create one)
  • Additional sitemaps (if you create separate ones)
  • Sitemap: https://balkanscout.com/sitemap-properties.xml
  • Sitemap: https://balkanscout.com/sitemap-pages.xml
Back to top