Generating Video Game Sound Effects on the Fly
The tiny sounds players barely notice are often the ones that take the most work to create. A single footstep, for instance, has to be recorded, processed, implemented and integrated, then repeated across numerous surfaces and contexts to minimize listener fatigue.
This talk introduces SARGE, a runtime generative-audio prototype capable of creating video game sound effects and their variations on the fly. Along the way, the talk covers each component behind the prototype, including a physics-aware world model, a small LLM that writes custom prompts for Sony’s Woosh generative audio model, and more. The result is an open-source proof of concept, with a Unity demo, that developers can explore and build on.
Tsz Chai Alan Liu
Lead Researcher
Sonoform Labs
Tsz Chai Alan Liu is the Lead Researcher of Sonoform Labs, an independent research lab working at the intersection of music information retrieval, bioacoustics, and machine learning with work accepted for publication and demoing at CLEF, IEEE CoG and NeurIPS. Alan also builds real-time generative audio systems for games, including the prototype behind today’s talk. Prior to his research journey, he had a short stint as an indie game developer, and studied classical piano for several years.