cryptofuture.pro
robots.txt

Robots Exclusion Standard data for cryptofuture.pro

Resource Scan

Scan Details

Site Domain cryptofuture.pro
Base Domain cryptofuture.pro
Scan Status Ok
Last Scan2025-12-09T00:38:38+00:00
Next Scan 2026-01-08T00:38:38+00:00

Last Scan

Scanned2025-12-09T00:38:38+00:00
URL https://cryptofuture.pro/robots.txt
Domain IPs 162.0.235.45
Response IP 162.0.235.45
Found Yes
Hash 536c8017d60b03bac5ebb07d2a3d04cab0f87c49db882374327c2ba03fe18b5e
SimHash 480ece02ec93

Groups

*

Rule Path
Allow /
Allow /index.php
Allow /dashboard.php
Allow /img/
Allow /css/
Allow /js/
Allow /manifest.json
Disallow /config.php
Disallow /api/
Disallow /admin/
Disallow /withdrawals.txt
Disallow /logs/
Disallow /backup/
Disallow /vendor/
Disallow /private/
Disallow /*.sql$
Disallow /logout.php
Disallow /reset-password.php
Disallow /*?*utm_*
Disallow /*?*ref=*&*
Disallow /*?*session*
Disallow /*?*sid=*
Disallow /*?page=dashboard*&*
Disallow /*?*lang=*&*

googlebot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 1

bingbot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 1

slurp

Rule Path
Allow /

Other Records

Field Value
crawl-delay 2

duckduckbot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 1

baiduspider

Rule Path
Allow /

Other Records

Field Value
crawl-delay 2

yandexbot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 1

ahrefsbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://cryptofuture.pro/sitemap.xml
sitemap https://cryptofuture.pro/sitemap-images.xml

Comments

  • Block sensitive areas
  • Block URL parameters that don't add value
  • Block duplicate content
  • Special rules for search engines
  • Block bad bots
  • Sitemap location
  • Host specification (helps with canonicalization)

Warnings

  • `host` is not a known field.