Export to MP4 in the browser

After this recipe you can export a project to an MP4 file entirely in the browser, with no server: the engine renders each frame on the GPU in a worker, the browser’s WebCodecs encoder compresses it, and mediabunny writes the container.

This recipe uses the low-level pieces of the alpha2 package. If you are building an editor, @alpha2-sh/editor/browser wraps all of this (import, preview, export) behind bridge.startExport(); see Editor.

The project

A title over a gradient. It has only generated sources, so the export needs nothing decoded, only a font:

{
  "schema_version": 1,
  "id": "browser-export",
  "name": "Browser export",
  "settings": {
    "width": 1280,
    "height": 720,
    "frame_rate": {
      "num": 30,
      "den": 1
    },
    "sample_rate": 48000
  },
  "sources": [
    {
      "id": "backdrop",
      "kind": "generated",
      "generator": {
        "type": "fill",
        "paint": {
          "type": "linear_gradient",
          "start": {
            "x": 0,
            "y": 0
          },
          "end": {
            "x": 1,
            "y": 1
          },
          "stops": [
            {
              "offset": 0,
              "color": {
                "red": 0.17,
                "green": 0.36,
                "blue": 0.99,
                "alpha": 1
              }
            },
            {
              "offset": 1,
              "color": {
                "red": 0.7,
                "green": 0.11,
                "blue": 0.98,
                "alpha": 1
              }
            }
          ]
        }
      }
    },
    {
      "id": "title",
      "kind": "generated",
      "generator": {
        "type": "text",
        "text": "Exported in the browser",
        "style": {
          "font_family": "Hanken Grotesk",
          "font_size": 0.1,
          "position": {
            "x": 0.5,
            "y": 0.5
          },
          "anchor": {
            "x": 0.5,
            "y": 0.5
          }
        }
      }
    }
  ],
  "timeline": {
    "tracks": [
      {
        "id": "back",
        "name": "Backdrop",
        "kind": "video",
        "clips": [
          {
            "id": "backdrop-clip",
            "source_id": "backdrop",
            "timeline_range": {
              "start": 0,
              "end": 240000
            },
            "source_range": {
              "start": 0,
              "end": 240000
            }
          }
        ],
        "transitions": []
      },
      {
        "id": "titles",
        "name": "Title",
        "kind": "video",
        "clips": [
          {
            "id": "title-clip",
            "source_id": "title",
            "timeline_range": {
              "start": 0,
              "end": 240000
            },
            "source_range": {
              "start": 0,
              "end": 240000
            },
            "visual": {
              "opacity": {
                "mode": "keyframes",
                "keyframes": [
                  {
                    "time": 0,
                    "value": 0,
                    "interpolation": {
                      "type": "cubic_bezier",
                      "x1": 0.2,
                      "y1": 0,
                      "x2": 0,
                      "y2": 1
                    }
                  },
                  {
                    "time": 72000,
                    "value": 1,
                    "interpolation": {
                      "type": "hold"
                    }
                  }
                ]
              }
            }
          }
        ],
        "transitions": []
      }
    ]
  }
}

Install

npm install alpha2 mediabunny

mediabunny is an optional dependency of alpha2, used by alpha2/mediabunny. In TypeScript it brings @types/dom-webcodecs, which clashes with TypeScript 5.9’s DOM library: compile with skipLibCheck, or add "overrides": { "@types/dom-webcodecs": "0.1.19" } to your package.json. An app that imports alpha2/mediabunny also needs "ESNext.Disposable" in its lib.

Steps

Start a render worker. The worker owns the GPU device. It loads the engine, registers the font and loads the project, then answers the page’s requests.

import init, { WebEffectRegistry } from "alpha2/wasm";
import { createWasmRenderWorkerHost, installRenderWorker } from "alpha2/worker";

self.onmessage = async ({ data }) => {
  await init();
  installRenderWorker(
    self,
    createWasmRenderWorkerHost(WebEffectRegistry, data.projectJson, {
      fonts: [{ family: "Hanken Grotesk", bytes: data.font }],
    }),
  );
  self.postMessage({ type: "ready" });
};

installRenderWorker replaces self.onmessage with its own request loop, so this first message is the only one the code above handles. Give the export the same font files as the preview, so the text draws the same.

Connect to it from the page.

import { WorkerExportBackend, WorkerRenderBackend } from "alpha2/worker";
import { Alpha2Exporter, exportToSink } from "alpha2/exporter";
import { canEncodeVideoTrack, MediabunnySink } from "alpha2/mediabunny";

const worker = new Worker(new URL("./export-worker.js", import.meta.url), { type: "module" });
const font = new Uint8Array(await (await fetch("/fonts/HankenGrotesk.ttf")).arrayBuffer());
const ready = new Promise((resolve) =>
  worker.addEventListener("message", function onReady(event) {
    if (event.data?.type !== "ready") return;
    worker.removeEventListener("message", onReady);
    resolve();
  }),
);
worker.postMessage({ projectJson: JSON.stringify(project), font });
await ready;

const backend = new WorkerRenderBackend(worker);
// Export renders offscreen at the project's size; this canvas is only the preview surface.
await backend.initialize(new OffscreenCanvas(64, 36));

Schedule the frames. Alpha2Exporter walks the project’s frame grid over a range of ticks. WorkerExportBackend renders each frame through the worker, in the pixel layout the codec needs.

const { width, height, frame_rate } = project.settings;
const rate = { num: BigInt(frame_rate.num), den: BigInt(frame_rate.den) };
const exporter = new Alpha2Exporter(
  new WorkerExportBackend(backend, "h264"),
  rate,
  0n,
  240_000n, // [start, end) in ticks: 2 s
);

Make the sink. Ask the browser whether it encodes H.264 at this size first. canEncodeVideoTrack returns null when it does not, and otherwise the codec string to open the encoder with.

const videoRateControl = { bitrate: 8_000_000, bitrateMode: "variable" };
const encoding = await canEncodeVideoTrack("avc", { width, height, rate: videoRateControl });
if (encoding === null) throw new Error("this browser has no H.264 encoder at this size");

const sink = new MediabunnySink({
  container: "mp4",
  videoCodec: "avc",
  videoRateControl,
  ...(encoding.codecString === undefined ? {} : { videoCodecString: encoding.codecString }),
  frameRate: frame_rate,
  destination: { kind: "memory", maxBytes: 256 * 1024 * 1024 },
});

Pass frameRate: without it the muxer derives frame durations from floating-point deltas and the video judders. The codecs must be ones the container offers: avc, hevc, vp9 and av1 for video, aac and opus for audio (alpha2/containers lists which container takes which).

Run the export and download the file.

const controller = new AbortController(); // controller.abort() cancels
try {
  const result = await exportToSink(
    exporter,
    sink,
    async (waiting) => {
      throw new Error(`no media for ${waiting.missingSources.join(", ")}`);
    },
    { signal: controller.signal },
  );
  console.log(`${result.frames} frames at ${result.fps.toFixed(1)} fps`);
} finally {
  backend.dispose();
}

const output = sink.output; // { kind: "memory", buffer } once finish() has run
const url = URL.createObjectURL(new Blob([output.buffer], { type: "video/mp4" }));
const link = Object.assign(document.createElement("a"), { href: url, download: "title.mp4" });
link.click();
URL.revokeObjectURL(url);

The third argument, refill, is called when a frame needs source frames the worker does not have. This project has none to decode, so it never runs.

exportToSink keeps up to 3 frames and 128 MiB of readback in flight by default (maxInFlightFrames, maxInFlightBytes). exporter.progress gives { completed, total } as bigints while it runs. Aborting the signal rejects with an ExportAbortedError whose code is cancelled. A lost GPU rejects with code device-lost.

Projects with video

When the project has video sources, refill decodes the frames the engine asks for and uploads them to the worker. MediabunnyVideoDecoder decodes a Blob or File:

import { MediabunnyVideoDecoder } from "alpha2/mediabunny";
import { decoderConsumerId } from "alpha2/media";

const decoders = new Map();
for (const source of project.sources.filter((s) => s.kind === "video"))
  decoders.set(source.id, await MediabunnyVideoDecoder.open(source.id, files.get(source.id)));

const refill = async (waiting) => {
  for (const { clipId, readerId, sourceId, mediaTimeTicks } of waiting.missingFrames) {
    const decoded = await decoders.get(sourceId).refill({
      sourceId,
      mediaTimeTicks,
      reason: "prefetch",
      generation: 0,
      consumerId: decoderConsumerId(clipId, readerId),
    });
    await backend.uploadVideoFrame(decoded);
  }
};

Close the decoders (decoder.close()) when the export ends. A frame still waiting after four refills fails the export with RESOURCE_LIMIT.

Large files

A memory destination holds the whole file, and past maxBytes the export fails with RESOURCE_LIMIT. For long exports, stream to a file instead. WritableFileDestination from alpha2/export-destination takes a file that accepts positioned writes, such as a FileSystemWritableFileStream:

import { WritableFileDestination } from "alpha2/export-destination";

const handle = await window.showSaveFilePicker({ suggestedName: "title.mp4" });
const file = new WritableFileDestination(await handle.createWritable());
const sink = new MediabunnySink({
  // ...the options above, with:
  destination: { kind: "stream", writable: file.writable },
});

showSaveFilePicker is the File System Access API, which not every browser has.