Real-Time Inference of Neural Networks
A Practical Approach for DSP Engineers – Part II
Continuing our exploration of implementing neural network inference for real-time audio applications, we have expanded from our initial plugin example to a comprehensive library that simplifies the deployment and integration of neural networks in audio applications.
In this talk, we discuss various aspects of our implementation. Since it is crucial to know whether inference engines exhibit real-time violations, we first quantify real-time violations within inference executions. Subsequently, we explore the integration of these engines in real-time audio environments, specifically addressing the challenges of running multiple instances simultaneously. To accomplish this, we use a static thread pool and, when available, host-provided threads. We also focus on strategies for achieving the lowest possible latency, presenting techniques we have implemented, and open a dialogue on a controversial approach to further reduce latency. Moreover, we share our findings on the performance impact of various factors on inference runtimes. To this end, we have extensively benchmarked different neural network architectures across different inference engines, and can show how differences in input buffer size and model size, as well as previously executed inferences, affect the overall performance.
Fares Schulz
Researcher
Technische Universität Berlin
Fares Schulz leads the Computer Music and Neural Audio Systems research team within the Audio Communication Group at Technische Universität Berlin, and works as a freelance software developer. His current research focuses on novel applications of neural networks for creative audio effects and synthesis, particularly in real-time and mixed-signal contexts. He treats neural networks as one tool among many rather than a one-size-fits-all solution, and is investigating how they can be deployed alongside long-established methods such as DSP algorithms and analog circuitry. He also works on large-scale multichannel spatial audio systems, whose high computational cost calls for clustered audio servers.
Valentin Ackva
Audio Software Developer
tanh-lab.com
Valentin Ackva is an independent audio software developer and consultant working on real-time AI inference for music and audio. He is also an external researcher at TU Berlin's Computer Music and Neural Audio Systems team.
Valentin develops plugins freelance for Baby Audio and runs the audio software agency tanh-lab, which he founded with Fares Schulz, building (neural) audio products for clients like Suture Sound Inc, Tonsturm, meltedsounds and Black Salt Audio. He co-maintains anira, an open-source C++ library for real-time-safe neural network inference.