From Trained Model to Real-Time Implementation
A Practical Guide to Neural Network Inference and Common Pitfalls
Training a neural network for audio is a well-documented process. However, running it inside a deterministic callback that must return within a few milliseconds is not. This workshop addresses this issue. Starting with the requirements that a model must satisfy for real-time operation to be possible, such as streamability without boundary artefacts, explicit handling of state and a graph that the runtime can execute, as well as having an inference time below the deadline, the workshop then takes a pretrained model through export, benchmarking, integration and optimisation using ANIRA — an open-source library for real-time safe neural network inference. The session concludes with an open segment and a jam in which the networks are played live.
Outline
- Part I: Introduction talk
- Part II: Pretrained model to C++ implementation
- Part III: Benchmarking the model in C++
- Break
- Part IV: Building the real-time application
- Part V: Optimisation and real-time safety
- Part VI: Open session and closing jam
Requirements
Each participant requires a laptop (macOS, Linux or Windows; with a C++ toolchain, CMake and Python + headphones).
Fares Schulz
Researcher
Technische Universität Berlin
I am a researcher in the Computer Music and Sound Synthesis Team, part of the Audio Communication Group at the Technische Universität Berlin. At present, my particular interest lies in the exploration of novel applications of neural networks for creative audio effects and synthesis, especially in the real-time and mixed-signal domains. Considering neural networks as a tool rather than a one-size-fits-all solution, I am researching how to make them available alongside long-established methods such as DSP algorithms and analog circuitry. I am also working on spatial audio (multi) systems, which require clustered audio servers due to their high computational cost.
Currently in the final stages of my Master's degree in Audio Communication and Technology, my educational background includes two Bachelor's degrees in Physics and Audio Engineering. Throughout this time, it is my passion for electronic music production that has taken me from theoretical mathematical equations and abstract artistic concepts to their development as algorithms and analog circuits. I am always looking for new ways to combine my interests in music, technology, and science, and love to chat with others who share these passions.
Valentin Ackva
Audio Software Developer
INSONE
I am an audio programmer and electronic musician based in Berlin. With a background in computer science, I'm currently working towards my master's degree in Audio Communication and Technology at the Technische Universität Berlin. My passion lies at the intersection of music, programming, and technology, especially where artistry meets innovation.
For the last 4 years, I have been working as an audio software developer at a speech processing startup in Leipzig. At my position there, I am responsible for the development of audio effects for speech enhancement. This role includes research into the real-time implementation of state-of-the-art neural networks for tasks such as denoising, audio super-resolution, and dereverberation.
Last year, I have co-founded a collective that combines the fields of DSP and AI, bringing together a group of audio programmers, machine learning engineers, and artists based in Berlin. In March, we released our first software, "Scyclone". An audio plugin that utilizes neural timbre transfer technology, introducing a new approach to automatic layering. Scyclone's innovative design and interaction of DSP and AI led to it winning the Audio Plugin Competition organised by the Audio Programmer.