Back To Schedule

Code as the Instrument Surface

Source-Backed Synthesis for Humans and Agents

13:20 - 13:40 UTC | Friday 16th October 2026 | ADCx Gather
Intermediate

While producing my own music, I kept running into an awkward split in AI-assisted sound design. Coding agents work naturally with text they can read, edit, and compare. When shaping a sound, I want to play a note, move a control, and hear the result immediately. I wanted both interactions to work on the same instrument.

Flows is an experimental synthesizer built around that idea: the source code defines the instrument, and the interface makes that source playable. Its .flows language describes oscillators, envelopes, filters, and the signals connecting them. A musician can drag a numeric value directly in the editor or adjust a Signal Flow control. An AI assistant can propose a change to the same source, which the user reviews and applies.

The source also gives observation a place in the workflow. Name an intermediate signal, attach a probe, and listen to or inspect that stage of processing. A change to a filter can be explored through its sound, waveform, and spectrum, with the processing and observations defined together.

This talk explores the engineering that makes this loop feel like an instrument: mapping source locations to interactive controls, preserving editing and undo behavior during live changes, keeping compilation outside the audio callback, and observing signals computed by the running DSP engine. It also covers how reusable synthesis building blocks and lower-level signal expressions support different depths of sound design, and how revision-checked agent proposals protect intervening user edits.

The demo follows one instrument through two kinds of change: dragging values in the code editor and applying an agent-proposed source edit. In both cases, the instrument runs from the same source, and probes let us hear and inspect what the change actually did.

View Slides

Zuoming Fu

Zuoming Fu is an independent audio software developer at Shanghai, China. He is an alumnus of Cornell University, where he graduated with a Master's degree in Electrical and Computer Engineering. A passionate music technology enthusiast and electronic dance music (EDM) trance producer, he focuses on combining his engineering background with his love for sound to create innovative audio tools.