Back To Schedule

Code as the Instrument Surface

Source-Backed Synthesis for Humans and Agents

While producing my own music, I kept running into the same awkward split whenever I thought about AI-assisted sound design. The tools that an AI coding agent can read and edit naturally are not the tools producers usually touch while shaping a sound.

On one side are DSP languages such as FAUST and Cmajor. They're built to describe and compile audio algorithms, and that source is exactly the kind of text an agent can pick up and change. Some workflows can already run in realtime or target plugin formats. But the source is still mostly the DSP implementation, not the thing a producer grabs while a note is playing.

On the other side are the commercial synths — Serum, Vital, Spire. They feel right inside a DAW: play a note, drag the cutoff, hear the patch change. But the state that defines the sound lives behind plugin-specific presets, modulation matrices, and proprietary runtime state. There is no obvious source file for an agent to diff, patch, or discuss with the user.

Flows is a small DSL built on one bet: that those two worlds can live on the same surface. It is a synthesizer plugin where the .flow source file is the instrument surface. A user can grab the cutoff: 1.2khz value and drag it right there in the code — no typing — and the text and the sound move together. If an agent proposes a sound-design edit, it changes that same text. Flows reparses the patch, updates the engine, and the probes beside the code show what changed. There is no separate "AI state" and no hidden parameter panel that is more authoritative than the source.

This talk is about the engineering behind that loop — what makes source-backed synthesis feel like an instrument instead of a code generator: SourceSpan as the shared handle for mouse gestures and agent edits, live source rewrites, compile boundaries that respect the audio thread, and probe feedback that stays tied to the code. I will also cover the Producer / Developer split in the language: high-level synth building blocks for patch authoring, with a lower-level DSP layer underneath for extension.

The demo will show a running .flow patch being edited in two ways: by dragging values in the code surface, and by applying an agent-proposed source edit. In both cases, the sound and the visual probes update from the same source file.