https://audio.dev/ -- @audiodevcon
---
Real-Time Inference of Neural Networks: A Guide for DSP Engineers - Part II - Fares Schulz & Valentin Ackva - ADC 2024
---
Continuing our exploration of implementing neural network inference for real-time audio applications, we have expanded from our initial plugin example to a comprehensive library that simplifies the deployment and integration of neural networks in audio applications.
In this talk, we discuss various aspects of our implementation. Since it is crucial to know whether inference engines exhibit real-time violations, we first quantify real-time violations within inference executions. Subsequently, we explore the integration of these engines in real-time audio environments, specifically addressing the challenges of running multiple instances simultaneously. To accomplish this, we use a static thread pool and, when available, host-provided threads. We also focus on strategies for achieving the lowest possible latency, presenting techniques we have implemented, and open a dialogue on a controversial approach to further reduce latency. Moreover, we share our findings on the performance impact of various factors on inference runtimes. To this end, we have extensively benchmarked different neural network architectures across different inference engines, and can show how differences in input buffer size and model size, as well as previously executed inferences, affect the overall performance.
---
Slides: https://data.audio.dev/talks/2024/real-time-inference-of-neural-networks/slides.pdf
---
Fares Schulz
I am a researcher in the Computer Music and Sound Synthesis Team, part of the Audio Communication Group at the Technische Universität Berlin. At present, my particular interest lies in the exploration of novel applications of neural networks for creative audio effects and synthesis, especially in the real-time and mixed-signal domains. Considering neural networks as a tool rather than a one-size-fits-all solution, I am researching how to make them available alongside long-established methods such as DSP algorithms and analog circuitry. I am also working on spatial audio (multi) systems, which require clustered audio servers due to their high computational cost.
Currently in the final stages of my Master's degree in Audio Communication and Technology, my educational background includes two Bachelor's degrees in Physics and Audio Engineering. Throughout this time, it is my passion for electronic music production that has taken me from theoretical mathematical equations and abstract artistic concepts to their development as algorithms and analog circuits. I am always looking for new ways to combine my interests in music, technology, and science, and love to chat with others who share these passions.
---
ADC is an annual event celebrating all audio development technologies, from music applications and game audio to audio processing and embedded systems. ADC’s mission is to help attendees acquire and develop new audio development skills, and build a network that will support their audio developer career.
Annual ADC Conference - https://audio.dev/
https://www.linkedin.com/company/audiodevcon
https://facebook.com/audiodevcon
https://instagram.com/audiodevcon
https://www.reddit.com/r/audiodevcon/
https://mastodon.social/@audiodevcon
---
Streamed & Edited by Digital Medium Ltd: https://online.digital-medium.co.uk
---
Organized and produced by JUCE: https://juce.com/
---
Special thanks to the ADC24 Team:
Sophie Carus
Derek Heimlich
Andrew Kirk
Bobby Lombardi
Tom Poole
Ralph Richbourg
Jim Roper
Jonathan Roper
Prashant Mishra
#adc #audiodev #neuralnetworks #deeplearning #multithreading #dsp #audio #audioprocessing #audioproduction #audioprogramming #inference #soundtech #audiotech #audiotechnology
