blumefy.com
robots.txt

Robots Exclusion Standard data for blumefy.com

Resource Scan

Scan Details

Site Domain blumefy.com
Base Domain blumefy.com
Scan Status Ok
Last Scan2025-10-23T22:19:09+00:00
Next Scan 2025-11-22T22:19:09+00:00

Last Scan

Scanned2025-10-23T22:19:09+00:00
URL https://blumefy.com/robots.txt
Domain IPs 13.33.183.123, 13.33.183.38, 13.33.183.47, 13.33.183.51
Response IP 13.33.45.90
Found Yes
Hash 6bc5ea150cefd173fa824c40a696b0d1e031a338f4ff219934bca661d4d0331a
SimHash 0b0f9a6366b0

Groups

*

Rule Path
Allow /
Allow /
Allow /about-us
Allow /migration
Allow /privacy-policy
Allow /terms-and-conditions
Disallow /admin/
Disallow /private/
Disallow /api/
Disallow /staging/
Disallow /test/
Disallow /dev/
Disallow /login/
Disallow /dashboard/
Disallow /user/
Disallow /account/
Disallow /profile/
Disallow /settings/
Disallow /*.json$
Disallow /*.xml$
Disallow /*.txt$
Disallow /*.pdf$
Disallow /*.doc$
Disallow /*.docx$
Disallow /*.xls$
Disallow /*.xlsx$
Disallow /search?
Disallow /*?query=
Disallow /*?filter=
Disallow /*?sort=
Disallow /temp/
Disallow /cache/
Disallow /tmp/
Disallow /.git/
Disallow /.env
Disallow /node_modules/
Disallow /analytics/
Disallow /tracking/
Disallow /stats/
Allow /assets/
Allow /images/
Allow /css/
Allow /js/
Allow /fonts/

Other Records

Field Value
crawl-delay 1

googlebot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 0.5

bingbot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 0.5

slurp

Rule Path
Allow /

Other Records

Field Value
crawl-delay 1

ahrefsbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://blumefy.com/sitemap.xml

Comments

  • Sitemap
  • Important pages - explicitly allow
  • Block irrelevant pages and directories
  • Block file types that shouldn't be indexed
  • Block search and filter pages
  • Block temporary and cache files
  • Block analytics and tracking pages
  • Allow important assets
  • Crawl delay for better server performance
  • Additional directives for specific bots
  • Block bad bots
  • Host directive (if using multiple domains)
  • Host: https://blumefy.com