Tag: AI

GPU Based Audio Processing Platform with AI Audio Effects – Simon Schneider – ADCxGather 2024

https://audio.dev/ -- @audiodevcon​
---

GPU Based Audio Processing Platform with AI Audio Effects - Are GPUs ready for real-time processing in live sound engineering? - Simon Schneider - ADCxGather 2024
---

Parallelising real-time audio effects requires complex task management and synchronisation. GPUs are optimised for parallel processing while still retaining flexible scheduling comparable to a CPU. As a result, it combines the strengths of both DSPs and CPUs in a single device. In addition, recent trends advocate using AI audio processing algorithms, which work best on GPU architectures.

This thesis presents an implementation of an embedded GPU-based audio processing framework on an Nvidia Jetson hardware platform. It can combine neural network inference and other audio effects into signal graphs that process within periods as small as 32 frames (0.667ms).

The signal graph does not restrict the number and combination of parallel and serial audio effects as long as the real-time limit is met. Therefore, the framework has been tested on large numbers of parallel channels, as found in a mixing console, and complex routing options available in high-end audio effect processors, such as the Neural DSP Quad Cortex.

Launching GPU work using the CUDA graph API produces better stability and performance than was observed using the CUDA stream API in a 2017 study. Processing a signal graph that fully utilises the Jetson's resources by mimicking a 64-channel mixing console on a 128-frame (2.67ms) period has a higher than 99% success rate. However, occasional stalling on the GPU can produce worst-case execution times of up to 20ms, regardless of the loaded audio effects. As a result, the framework can not yet be classified as real-time capable.
Further study of the CUDA scheduler and improvements to the operating system and audio driver may be able to achieve real-time capability in the future.
---

Simon Schneider

Simon is a musician based in Winterthur. After completing his Bachelor's Degree in IT he realized that he could combine his passion for music and software engineering in audio development. As a result, he went on to complete a Master's Degree in Electrical Engineering to learn about signal processing and hardware programming. During this time he was working part-time in the ZHAW Institute of Embedded Systems. There he developed an audio effects processing platform on an Nvidia Jetson GPU as part of his Master Thesis.

His bandmates will say, that playing the guitar is just an excuse to justify all the money he spends on music-related gear. While this is not entirely untrue, Simon is the type of programmer who spends days automating or streamlining a repetitive process that takes minutes to complete manually.
As a result, automating guitar effect changes or building a wireless InEar monitoring rack for shorter live show setup and teardown times simply requires a lot of gear.

Furthermore, Simon has a flair for modular and customizable software architecture. Since audio gear often lacks both of these qualities, he endeavours to make them a primary focus in all his audio development projects.
---

ADC is an annual event celebrating all audio development technologies, from music applications and game audio to audio processing and embedded systems. ADC’s mission is to help attendees acquire and develop new audio development skills, and build a network that will support their audio developer career.
Annual ADC Conference - https://audio.dev/
https://www.linkedin.com/company/audiodevcon

https://facebook.com/audiodevcon
https://instagram.com/audiodevcon
https://www.reddit.com/r/audiodevcon/
https://mastodon.social/@audiodevcon
---

Streamed & Edited by Digital Medium Ltd: https://online.digital-medium.co.uk
---

Organized and produced by JUCE: https://juce.com/
---

Special thanks to the ADCxGather Team:

Sophie Carus
Derek Heimlich
Andrew Kirk
Bobby Lombardi
Tom Poole
Ralph Richbourg
Jim Roper
Jonathan Roper
Prashant Mishra

#audioprocessing #gpu #ai #adc #audiodev #dsp #audio #conferenceaudio #audioproduction #audioprogramming #musictech #soundtech #audiotech #audiotechnology

Filed under: UncategorizedTagged with: , , , ,

KEYNOTE: Foundation Models Don’t Understand Me – Lessons From AI Lutherie for Live Performances – Manaswi Mishra

https://audio.dev/ -- @audiodevcon​
---

Keynote: Foundation Models Don’t Understand Me - Lessons From AI Lutherie for Live Performances - Manaswi Mishra - ADC 2024
---

Living in a world of rapidly accelerating synthetic media, the outputs of generative AI often leave us feeling frustrated, amused, and even manipulated. Early examples of creative AI tools struggle to go beyond imitating styles and patterns, producing a context-less blend of borrowed aesthetics from the datasets they’re trained on. This race to the statistically average flattened aesthetic, misunderstands the core goals of creative expression. In contrast, Audio developers and Instrument builders understand the importance of providing a toolkit of exploration, intentional serendipity and discovery to a new age of artists performing with AI. In my work of building AI musical instruments for live Opera, Symphony and Installations, I share a framework of frustrations and guidelines for AI instrument building.
---

Slides: https://data.audio.dev/talks/2024/foundation-models-dont-understand-me/slides.pdf
---

Manaswi Mishra

Manaswi Mishra is a LEGO Pappert Fellow and a current PhD research assistant in the Opera of the Future group, MIT Media Lab. His research explores strategies and frameworks for a new age of composing, performing and learning music using A.I. centered on bespoke human intent. He joined the MIT Media Lab in 2019 and completed his MS in Media Arts and Science, developing his work “Living, Singing A.I”, to empower bespoke A.I. music making with just the human voice. His AI instruments have been featured in Opera and Symphony performances like VALIS (2023), Brain Opera (2022), Flow Symphony (2024). His work has been published and exhibited in the MIT Press, Harvard Tech Review, Washington Post, Boston Globe, Conferences of Computational Creativity, ISEA Brisbane, IFA Stuttgart, CVPR 2024, Burning Man 2023, Copyright Society 2023, Bloomberg Law, Seoul Arts Center, etc.

Prior to joining MIT, he did a Masters in Music Technology at UPF, Barcelona and Bachelors in Technology at the Indian Institute of Technology Madras. He is passionate about a creative future where every individual can express, reflect, create and connect through music. Manaswi is also a founding instigator of the Music Tech Community in India and has organized workshops, hackathons and community events to foster a future of music and technology in his home country. In his free time, Manaswi loves collecting weird sounds.
---

ADC is an annual event celebrating all audio development technologies, from music applications and game audio to audio processing and embedded systems. ADC’s mission is to help attendees acquire and develop new audio development skills, and build a network that will support their audio developer career.
Annual ADC Conference - https://audio.dev/
https://www.linkedin.com/company/audiodevcon
https://twitter.com/audiodevcon
https://facebook.com/audiodevcon
https://instagram.com/audiodevcon
https://www.reddit.com/r/audiodevcon/
https://mastodon.social/@audiodevcon
---

Streamed & Edited by Digital Medium Ltd: https://online.digital-medium.co.uk
---

Organized and produced by JUCE: https://juce.com/
---

Special thanks to the ADC24 Team:

Sophie Carus
Derek Heimlich
Andrew Kirk
Bobby Lombardi
Tom Poole
Ralph Richbourg
Jim Roper
Jonathan Roper
Prashant Mishra

#ai #artificialintelligence #artificialintelligencetechnology #adc #audiodev #audio #conferenceaudio #audioprocessing #audioproduction #audioprogramming #musictech #soundtech #audiotech #audiotechnology

Filed under: UncategorizedTagged with: , , ,