Machine Learning Basics

What is Machine Learning?

Machine Learning (ML) is a way to teach computers to perform tasks without explicitly programming them. Instead of writing rules, we show the computer examples and let it figure out the patterns.

Traditional programming vs Machine Learning:

• Traditional: Rules + Data → Answers
• Machine Learning: Answers + Data → Rules

Types of Machine Learning

MixMind uses supervised learning for stem separation—training on audio with known source labels (vocals, drums, bass, other).

Training and Inference

Machine learning has two phases:

1. Training: The computer learns from data (takes hours/days)
2. Inference: Use the learned model to make predictions (takes milliseconds)

// Training phase (done once, offline)
// 1. Collect data: thousands of songs with separated stems
// 2. Feed data to algorithm
// 3. Algorithm adjusts its internal parameters
// 4. Save learned parameters as a "model"

// Inference phase (done in real-time, in browser)
// 1. Load pre-trained model
// 2. Feed new audio to model
// 3. Model outputs predictions (separated stems)
// 4. Fast! (milliseconds per audio chunk)

// Pseudocode
const model = await loadModel('spleeter-model.onnx');

// Inference
function separateStems(audioBuffer) {
    const input = preprocessAudio(audioBuffer);
    const output = model.run(input);
    const stems = postprocessOutput(output);
    return stems; // { vocals, drums, bass, other }
}

Features and Labels

ML algorithms work with numbers, not raw audio. We extract features—numerical representations of the audio.

// Example: classifying instruments
// Features extracted from audio:
const features = {
    spectralCentroid: 2500, // "Brightness"
    spectralRolloff: 4000, // High frequency content
    zcr: 0.15, // Zero crossing rate
    rms: 0.3, // Energy
    mfcc: [12 numbers], // Timbre description
    chroma: [12 numbers] // Pitch classes
};

// Label: what instrument it is
const label = "guitar";

// Training: show thousands of (features, label) pairs
// Inference: extract features from new audio, predict label

// MixMind's ML pipeline
class StemSeparator {
    constructor() {
        this.model = null;
    }
    
    async initialize() {
        // Load pre-trained model (done once)
        this.model = await ort.InferenceSession.create('spleeter.onnx');
    }
    
    async separate(audioBuffer) {
        // 1. Preprocess: convert audio to spectrogram
        const spectrogram = this.audioToSpectrogram(audioBuffer);
        
        // 2. Run model (inference)
        const input = new ort.Tensor('float32', spectrogram.data, spectrogram.dims);
        const feeds = { input_1: input };
        const results = await this.model.run(feeds);
        
        // 3. Postprocess: convert output back to audio
        const stems = {
            vocals: this.spectrogramToAudio(results.vocals),
            drums: this.spectrogramToAudio(results.drums),
            bass: this.spectrogramToAudio(results.bass),
            other: this.spectrogramToAudio(results.other)
        };
        
        return stems;
    }
}

// The model was trained offline on thousands of songs
// Now it runs in your browser in real-time!