Building an AI Subtitle Generator with Whisper & WebAssembly FFmpeg

Video is the dominant online medium. By combining WebAssembly video processing with AI models, we can build subtitle tools that run fast and cheap.

The bandwidth cost of processing video files

Uploading a 500MB video file to extract a 5MB audio track wastes user bandwidth and slows down transcription generation pipelines.

The Solution: Subtitle Generator Tech Architecture

We can load FFmpeg.wasm into the client browser to extract and compress the audio track locally, sending only a small MP3 file to our AI transcription endpoint.

Extract Audio Client-Side using FFmpeg.wasm

// WebAssembly FFmpeg audio extraction example
import { createFFmpeg, fetchFile } from '@ffmpeg/ffmpeg';
const ffmpeg = createFFmpeg({ log: true });

const extractAudio = async (videoFile) => {
  await ffmpeg.load();
  ffmpeg.FS('writeFile', 'input.mp4', await fetchFile(videoFile));
  await ffmpeg.run('-i', 'input.mp4', '-vn', '-acodec', 'libmp3lame', '-ar', '16000', 'output.mp3');
  const data = ffmpeg.FS('readFile', 'output.mp3');
  return new Blob([data.buffer], { type: 'audio/mp3' });
};

Key Insights & Takeaways

Related Articles & Case Studies