Chapter 31

MixMind System Overview

Learning Objectives

  • Understand the overall architecture
  • See how modules connect
  • Learn about the data flow
  • Understand the tech stack

System Architecture

MixMind is a browser-based DJ application that separates audio into stems and creates automatic mixes. Here's how everything connects:

  • Frontend: HTML, CSS, JavaScript (vanilla)
  • Audio Processing: Web Audio API
  • AI/ML: ONNX Runtime Web + Spleeter model
  • Hosting: Cloudflare Pages (CDN)
💡 Key Design Principle:

Everything runs in the browser! No backend server needed. This means:

  • Privacy: Audio never leaves your device
  • Speed: No network latency
  • Cost: No server costs
  • Offline: Works without internet (after initial load)

Main Components

ComponentFileResponsibility
UI Layerindex.htmlUser interface, controls, displays
Stylingstyles.cssVisual design, layout, animations
App Logicapp.jsMain application, state management
Audio Engineaudio-engine.jsAudio loading, playback, processing
Analysis Engineanalysis-engine.jsBPM, key, phrase detection
Separation Engineseparation-engine.jsAI stem separation
Transition Enginetransition-engine.jsCreating mixes, transitions
Export Engineexport-engine.jsRendering final mix to file

Data Flow

// Typical user workflow:

// 1. User uploads tracks
const track1 = await loadTrack('song1.mp3');
const track2 = await loadTrack('song2.mp3');

// 2. Analyze tracks
const analysis1 = await analysisEngine.analyze(track1);
const analysis2 = await analysisEngine.analyze(track2);
// Returns: { bpm, key, beats, phrases, energy, ... }

// 3. Optional: Separate stems
const stems1 = await separationEngine.separate(track1);
// Returns: { vocals, drums, bass, other }

// 4. Configure mix
const mixConfig = {
    tracks: [track1, track2],
    transitions: [{
        type: 'crossfade',
        duration: 16, // beats
        syncToBeats: true
    }]
};

// 5. Create mix
const mix = await transitionEngine.createMix(mixConfig);

// 6. Export
const audioFile = await exportEngine.render(mix);
// Returns: Blob (WAV file)

// 7. User downloads file

Technology Stack

CategoryTechnologyWhy
LanguageJavaScript (ES6+)Universal, runs in browser
AudioWeb Audio APINative browser audio processing
AI/MLONNX Runtime WebRun neural networks in browser
ModelSpleeter (U-Net)Proven stem separation
ComputeWebAssemblyNear-native performance
ParallelWeb WorkersBackground processing
StorageIndexedDBCache large files
HostingCloudflare PagesFast CDN, free tier

🎵 MixMind Connection

MixMind's Architecture Philosophy

// Design principles:

// 1. Browser-first
// - Everything runs client-side
// - No server dependencies
// - Works offline after load

// 2. Modular
// - Each engine is independent
// - Easy to test and replace
// - Clear separation of concerns

// 3. Progressive enhancement
// - Core features work everywhere
// - Advanced features use modern APIs
// - Graceful degradation

// 4. Performance-focused
// - Web Workers for heavy lifting
// - Chunked processing for large files
// - Streaming for progress feedback

// 5. User-friendly
// - Real-time feedback
// - Progress indicators
// - Error handling with helpful messages

// Architecture diagram:
// User → UI → App.js → [Engines] → Web Audio API → Speakers/Export
//                          ↓
//                    ONNX Runtime → AI Model

Exercise 31.1: Practice

Goal: Apply what you've learned in this chapter

Try the concepts from this chapter in your browser's console or a simple HTML file. Experiment with the code examples and modify them to see what happens.

📝 Quick Reference:

Review the key concepts from this chapter and make sure you understand each one before moving to the next chapter.