Log In Sign Up

trendsfutures.com
robots.txt

Robots Exclusion Standard data for trendsfutures.com

Resource Scan

Scan Details

Site Domain trendsfutures.com
Base Domain trendsfutures.com
Scan Status Ok
Last Scan2025-12-10T22:13:30+00:00
Next Scan 2025-12-17T22:13:30+00:00

Last Scan

Scanned2025-12-10T22:13:30+00:00
URL https://trendsfutures.com/robots.txt
Domain IPs 34.111.179.208
Response IP 34.111.179.208
Found Yes
Hash 7f16638621a0460ef9897f6b72529d89527b47c1f48316a3a4dba9223791c85e
SimHash ec80da2245f1

Groups

*

Rule Path
Allow /

perplexitybot

Rule Path
Allow /
Allow /about
Allow /why-etfs
Allow /blog
Allow /calculator
Allow /investing-terms
Allow /live-news
Allow /news-archive
Allow /media
Allow /learning-path
Allow /etf-report
Allow /survey
Allow /new-survey
Allow /contact
Allow /privacy-policy
Allow /terms-of-service
Allow /cookie-policy
Disallow /admin
Disallow /admin/
Disallow /admin-login
Disallow /secure-admin
Disallow /admin/internal-enhanced
Disallow /admin/2fa-setup
Disallow /admin/blog-topics
Disallow /admin/blog-queue
Disallow /thank-you
Disallow /contact-thank-you
Disallow /*?ref=*
Disallow /*?utm_*
Disallow /*?fbclid=*
Disallow /*?gclid=*

Other Records

Field Value
crawl-delay 1
Back to top

Other Records

Field Value
sitemap https://trendsfutures.com/sitemap.xml
Back to top

Comments

  • Optimize for Perplexity search engine
  • Sitemap location
  • Allow all pages to be crawled and indexed
  • Disallow admin areas and authentication pages
  • Disallow thank you pages (thin content)
  • Disallow tracking parameters - Google should ignore these
  • Clean URLs preference - canonical domain
  • Crawl delay (optional - be respectful to search engines)
Back to top

Warnings

  • `host` is not a known field.
Back to top