Rust

The alpha2 crate is the engine for Rust programs. After reading this page you can open a GPU device, load a project, lend the engine your decoded frames, render a preview frame, and walk an export frame by frame to YUV planes for an encoder.

The engine renders a timeline project on the GPU through wgpu and mixes its audio. It does not read or write media files: your program decodes the media and encodes the output, and the engine renders what the project shows from the frames you lend it.

Add the crate

Cargo.toml

[dependencies]
alpha2 = "0.5"
pollster = "1" # or any executor, to wait for the GPU device

Requirements

The crate needs Rust 1.88 or later. It renders through wgpu: DX12 on Windows, Metal on macOS, Vulkan then GL on Linux.

The crate is a facade over the engine’s crates. Its modules are those crates: audio, engine (playback and export), gpu, media, model (the project document), render and time. The types most programs use are also at the top level: Project, EngineProject, GpuContext, Renderer, FrameProvider, SourceView, Player, Exporter, ExportStep, QualityProfile, RenderSize, MediaTime, FrameRate and TimeRange.

A first render

The project is one second of one image at 1280x720 and 30 fps:

project.json

{
  "schema_version": 1,
  "id": "tutorial-project",
  "name": "Tutorial project",
  "settings": {
    "width": 1280,
    "height": 720,
    "frame_rate": { "num": 30, "den": 1 },
    "sample_rate": 48000
  },
  "sources": [
    { "id": "image-1", "locator": "asset://image-1", "kind": "image" }
  ],
  "timeline": {
    "tracks": [
      {
        "id": "video-1",
        "name": "Main video",
        "kind": "video",
        "clips": [
          {
            "id": "clip-1",
            "source_id": "image-1",
            "timeline_range": { "start": 0, "end": 120000 },
            "source_range": { "start": 0, "end": 120000 }
          }
        ],
        "transitions": []
      }
    ]
  }
}

The program renders one preview frame and then every export frame:

src/main.rs

use alpha2::media::{SourceId, VideoCodec};
use alpha2::render::UploadTexture;
use alpha2::{
    EngineProject, ExportStep, Exporter, FrameProvider, GpuContext, MediaTime, Player, Project,
    QualityProfile, RenderSize, Renderer, SourceView,
};

/// The application's decoded image, lent to the renderer for source `image-1`.
struct Image {
    id: SourceId,
    pixels: UploadTexture,
}

impl FrameProvider for Image {
    fn texture_for(&self, source: &SourceId, _time: MediaTime) -> Option<SourceView<'_>> {
        (source == &self.id).then(|| SourceView::from(&self.pixels))
    }
}

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let project = Project::from_json(&std::fs::read_to_string("project.json")?)?;
    let project = EngineProject::new(project)?;
    let gpu = pollster::block_on(GpuContext::request_product())?;
    let mut renderer = Renderer::new(gpu);
    // The built-in effects and transitions a project can name.
    renderer.register_builtins().expect("the built-in definitions register");

    // Opaque mid gray, premultiplied sRGB RGBA8, at the project's size.
    let size = RenderSize::new(1280, 720)?;
    let gray = [128, 128, 128, 255].repeat(1280 * 720);
    let image = Image {
        id: SourceId::new("image-1")?,
        pixels: renderer.upload_rgba8(size, &gray)?,
    };

    // One preview frame, read back as RGBA8.
    let mut player = Player::new(QualityProfile::preview())?;
    let frame = player.render_at(&project, &renderer, &image, MediaTime::ZERO)?;
    if let Some(texture) = frame.texture {
        let rgba = renderer.read_rgba8(texture)?;
        println!("preview: {} bytes", rgba.len());
    }

    // Every frame of the export, as YUV 4:2:0 planes for an encoder.
    let mut exporter = Exporter::for_project(&project, QualityProfile::export(), VideoCodec::H264)?;
    loop {
        match exporter.render_next(&project, &renderer, &image)? {
            ExportStep::Rendered { readback, .. } => {
                let planes = readback.wait()?;
                // planes.y, planes.u, planes.v: hand them to an encoder.
                println!("frame: Y {} U {} V {} bytes", planes.y.len(), planes.u.len(), planes.v.len());
            }
            ExportStep::Waiting { missing_sources, .. } => {
                return Err(format!("no frame for {missing_sources:?}").into());
            }
            ExportStep::Complete => break,
        }
    }
    Ok(())
}

How the pieces fit

1. Load the project. Project::from_json reads and checks a document. EngineProject::new makes an immutable snapshot of it, indexed for evaluation, which the player and the exporter render. Load a new snapshot after each edit.
2. Open the GPU. GpuContext::request_product() opens the device a host renders on: this OS’s backend, the adapter ALPHA2_GPU_ADAPTER names, else a discrete GPU before an integrated one.
3. Make a renderer. Renderer::new(gpu), then register_builtins() for the built-in effects and transitions. Each is compiled the first time a frame draws it or warmup names it. A frame that draws an effect the renderer has not registered fails.
4. Lend frames. Implement FrameProvider for your decoded media.
5. Render. Player for preview frames, Exporter for export frames.

FrameProvider

A FrameProvider lends the renderer a frame for each source at the exact source time the engine asks for:

fn texture_for(&self, source: &SourceId, time: MediaTime) -> Option<SourceView<'_>>;

time is a tick time on the source’s own timeline, which differs from timeline time for trimmed or retimed clips. Return None when you do not have that frame yet. Renderer::upload_rgba8(size, pixels) turns premultiplied sRGB RGBA8 pixels into an UploadTexture you can lend with SourceView::from.

Player

Player::new(QualityProfile::preview()) makes a player. render_at(&project, &renderer, &frames, time) renders the frame shown at a time and returns it with its texture. renderer.read_rgba8(texture) reads it back.

When a source frame is missing, the result names the source (missing_sources) and, by the player’s missing-frame policy, shows the last frame again (repeated) or none. A frame equal to the last one is not rendered again (unchanged).

Exporter

Exporter::for_project(&project, QualityProfile::export(), codec) walks the project’s frame grid over its whole timeline at export quality. codec (VideoCodec::H264 and the like) places the chroma samples the way that codec expects. Each render_next returns one ExportStep:

Step	Means
Rendered { readback, .. }	A frame rendered. readback.wait() gives its y, u and v planes.
Waiting { missing_sources, .. }	The frame lacked these sources. Supply them and call render_next again: the export continues at the same frame.
Complete	Every frame is done.

effective_range() is the requested range widened to the frame grid; its start is the output’s time zero. audio_window(sample_rate) gives the matching span of audio samples.

When the GPU does not open

When request_product fails, GpuError::step() says why (GpuAcquisitionStep): no adapter, a device refused, a backend that did not start, an ALPHA2_GPU_ADAPTER pin that names no adapter, or no hardware GPU. Its source() gives the platform’s reason where there is one.

Decoding and encoding

The engine does not read or write media files. alpha2-native decodes, probes and encodes with FFmpeg 8, linking the FFmpeg installed where it is built. alpha2-host is a complete program that exports projects with it.

Effects

alpha2::render::builtin_catalog() lists the built-in effects and transitions with their parameters (also in the effects reference). Chroma keying (alpha2.effect.chroma-key) and 3D LUTs (alpha2.effect.lut-3d, with a .cube file registered through Renderer::register_lut3d) are built-in effects a project applies like any other. See effects and compositions.

wgpu interop

The default API names no wgpu type. The wgpu-interop feature adds the items that do, for a program that shares its own wgpu device or textures with the engine:

• GpuContext::from_parts adopts your device.
• SourceView::from_wgpu lends a wgpu::TextureView as a source.
• You can take the wgpu texture of a rendered frame, and render into a hardware encoder’s planes.

alpha2 = { version = "0.5", features = ["wgpu-interop"] }

With the feature on, alpha2::gpu::wgpu is the wgpu crate the engine is built on. Name wgpu types through that path rather than your own wgpu dependency.

Caution

The items behind wgpu-interop follow wgpu’s major version; the rest of the API does not. The documentation on docs.rs is built without the feature and does not show them.

Next

• Project model: what project.json holds.
• Determinism: why every runtime draws the same frames.
• CLI: alpha2-host, the engine with decoding and encoding built in.