Devices

A device is an effect or an instrument that you can plug in, not a widget with audio inside. It comes in four pieces that share one description, so you take the ones you need: a host that has its own audio engine keeps the state and the panel, an agent keeps the state and the tools, and a page that only wants sound takes them all.

npx shadcn@latest add @audio-ui/compressor installs a device's state, engine and panel in components/audio-ui/devices/compressor/. Each piece also installs on its own (@audio-ui/compressor-device, -engine, -panel), so a host with its own engine takes only the state and the panel. The tools come from @audio-ui/core/device. Every device page has a block that wires the pieces to a loop: @audio-ui/block-compressor.

Catalog

Dynamics

EQ and filters

Distortion

Delay and reverb

Modulation

Utility

Instruments

The state

A state is a plain object keyed by parameter id. It holds numbers, booleans and strings, and it is safe to turn into JSON. The device keeps no instance, so you store the state, save it with your project and send it where you like.

Copy
import { compressorDevice } from "@/components/audio-ui/devices/compressor/device";

const state = compressorDevice.defaults();
// { enabled: true, thresholdDb: -18, ratio: 4, attackMs: 5, ... }

compressorDevice.set(state, "thresholdDb", -200); // a new state, brought to -60
compressorDevice.sanitize(JSON.parse(saved)); // a loaded project, repaired

A parameter is a number with a range, a switch or a choice from a list. The id of the device, audioui.compressor, is what a project stores. It is not the panel, which you can restyle without breaking anyone's session.

The panel

A panel is controlled. It keeps no state: it draws the one it is given and returns the whole new state.

Copy
import { CompressorPanel } from "@/components/audio-ui/devices/compressor/panel";

<CompressorPanel onValueChange={setState} value={state} />;

Most panels draw something the state means, such as a transfer curve, a frequency response or a set of echoes, with a handle you drag, over the knobs. The knobs come from the descriptors of the device, with their ranges, steps and readouts.

The engine

Copy
import { createCompressorEngine } from "@/components/audio-ui/devices/compressor/engine";

const engine = createCompressorEngine(ctx, state);
source.connect(engine.input);
engine.output.connect(ctx.destination);

engine.update(nextState); // parameters ramp: no click
engine.dispose();

An engine is a function of an audio context and a state. It gives you an input, an output, an update and a dispose, and it moves every parameter with a ramp so that turning a knob never clicks. The numbers that go on the nodes come from a pure function of the state, such as compressorSettings(state), that you can read and test without an audio context.

Some engines add to this:

A worklet is loaded from a blob, so there is no file to serve. Until ready resolves, a device that needs one passes the signal unchanged. In a context with no worklets it never resolves to true.

Sidechain

A compressor or a gate can follow the level of another signal instead of its own: a kick ducks a pad, a voice ducks the music. Switch Sidechain on, and connect the signal to engine.sidechain. It is only listened to, never heard.

Copy
const engine = createCompressorEngine(ctx, { ...state, external: true });
pad.connect(engine.input);
kick.connect(engine.sidechain); // listened to, not heard
engine.output.connect(ctx.destination);

The sidechain is not a parameter. The state holds the switch and the host routes the signal. A device that takes one says so: compressorDevice.sidechain is true, and so is describe().sidechain, so a host knows to route something to it and an agent can be told. With the sidechain on, what the device does depends on the other signal, so its panel does not draw a transfer curve.

Tempo sync

A delay, a chorus, a phaser and an auto pan can take their time or their rate from a note value, such as 1/8, a triplet 1/8t or a dotted 1/8d, from 1/32 to a whole note. Switch Sync on and pick a Note. The tempo is the host's, not the device's: give it to the engine with the first state or with any update.

Copy
const engine = createDelayEngine(ctx, state, { tempo }); // { bpm: 100, beatsPerBar: 4, beatUnit: 4 }
engine.update(nextState, { tempo: { ...tempo, bpm: 120 } }); // the echo follows
engine.update(otherState); // no context: the last tempo holds

The panel takes the same tempo as a prop, tempo. With Sync on, the knob for the free value is drawn off; with it off, the list of notes is. A tool call can switch Sync on and pick a note, but it never sets the tempo.

Latency

Some devices delay the signal. device.latency(state, sampleRate) returns it in seconds, so a host can compensate. The dry path waits as long as the wet one, so a partial mix does not comb.

Every other device has none. The numbers were measured in Chrome and the specification does not give them, so measure them in another browser with measureLatency.

Tools

The tools are derived from the parameters of the device, with a JSON Schema input, so any agent can use them. They are not tied to an SDK.

Copy
import { deviceTools } from "@audio-ui/core/device";

const [set, reset] = deviceTools(compressorDevice);
// set.name === "audioui_compressor_set"

const { state: next, summary } = set.run(state, { thresholdDb: -300, ratio: "high" });
// summary: "Threshold: -18.0 dB to -60.0 dB (brought into range); ratio: refused, not a number"

set changes parameters and reset puts them back to their defaults. An instrument adds a play tool that gives the host a note to start. The ranges a model is told are the ones the device enforces. A value out of range is brought in and reported, a value of the wrong type is refused by name and never replaced by a default, and a call never throws. The summary is the answer to give back to the model.

Measure your engine

A host that brings its own engine can check it against what the panel draws. device-measure renders an engine in an offline audio context and compares it with the curve of the device. It works for any device whose curve is a frequency response, such as the equalizer, the filter and the channel EQ.

Copy
import { equalizerCurve, equalizerDevice } from "@/components/audio-ui/devices/equalizer/device";
import { measureLatency, measureResponse } from "@/lib/device-measure";

const state = equalizerDevice.defaults();
// Your factory: (ctx, state) => { input, output, dispose }.
const { worstDb, at } = await measureResponse(createMyEngine, state, equalizerCurve(state));
// worstDb: how far your engine is from the panel, in dB, and where (at, in Hz).

const latencyMs = await measureLatency(createMyEngine, state);
// What a host reports to compensate it.

It is a measurement, not a certification: the curve is the device's own and the tolerance is yours.

Your own device

defineDevice is the one extension point, and the core keeps no list of devices. A device you define gets the same panel, tools and bindings as the ones here.

Copy
import { defineDevice } from "@audio-ui/core/device";
import { DevicePanel } from "@/components/audio-ui/device-panel";

const tremolo = defineDevice({
  id: "acme.tremolo",
  name: "Tremolo",
  description: "Moves the level up and down.",
  parameters: [
    { id: "on", kind: "toggle", label: "Active", default: true },
    { id: "rate", label: "Rate", min: 0.5, max: 20, scale: "log", step: 0.5, unit: "Hz", default: 5 },
    { id: "depth", label: "Depth", min: 0, max: 100, step: 1, unit: "%", default: 50 },
  ],
});

<DevicePanel device={tremolo} onValueChange={setState} value={state} />;

DevicePanel makes a knob for each number, a toggle for each switch and a row of buttons for each choice, such as a waveform or a filter type. A face of your own, such as a curve or a spectrum, is optional.

An instrument is the same with kind: "instrument". Its parameters shape the sound, and its engine adds noteOn and noteOff next to update. The pocket synth is the example: its panel is controlled and sends notes out, so it plays through any engine.

Your own engine

A host that has its own audio engine, such as a DAW with one in a worklet, keeps the state and the panel and turns the state into its own messages. A DeviceBinding is just that:

Copy
import type { DeviceBinding } from "@audio-ui/core/device";

const binding: DeviceBinding<CompressorState> = {
  update: (state) => engine.post({ type: "compressor", ...compressorSettings(state) }),
  dispose: () => engine.post({ type: "compressor-off" }),
};

Accessibility

Every control is a native input. A knob is a slider named after its parameter, with its value and unit as the text value ("Threshold, -18.0 dB"), and a switch is a toggle button. A panel is a group named after the device. The curve is decoration around the controls, a handle on it is a slider, and everything takes the keyboard.