Conference Year: 2024

The Future of Immersive Music

There is a dramatically increasing demand for artists and audio professionals to create content specifically for 3D-immersive listening on headphones and at home plus new audio experiences as demonstrated by venues such as Lightroom (London), Dareshack (Bristol) and Sphere (Las Vegas).

Most immersive music content, however, is re-purposed stereo productions never intended to be heard outside the two-speaker stereo environment with resulting output feeling disconnected and unfulfilling.

Record companies and artists want to avoid paying for post-production immersive audio mixes of original stereo recordings. They are increasingly requesting music studios deliver cost-effective ways for recording music from the outset in both stereo and immersive formats.

In this panel talk, the makers of 'Flow', Real World Studios, Dr Ruth Farrar from Bath Spa University and Adrian Utley from the band Portishead will share knowledge gained during this immersive music project. They will share insights from their trial to develop new immersive audio processing workflows to overcome current challenges of moving between stereo and immersive formats. They will also discuss their process for developing equipment set-up and sound capture approaches to create novel immersive audio compositions, elevating production processes from traditional stereo to being truly immersive.

Panel members include Hans Martin-Buff, Andrew Scheps, Adrian Utley, Theo Kozlowski and Ruth Farrar.

Filed under: Uncategorized

Our Ultra-Processed Interfaces

Many digital musical instruments, both commercial and otherwise, share similar interface design characteristics like rectangular rubber pads, piano interfaces, sliders, and grids of controls. These conventions are useful, but where do they come from, and why are they so ubiquitous?

In this talk I dive deep into how the design conventions of music interfaces have evolved, and how traditionally they're deeply rooted in the culture in which they're created ... until recently. I discuss how this decoupling of musical interfaces from cultural context has developed alongside the larger contemporary trend of "ultra-processing", a way of creating consumable products, from food to media to devices, that are engineered for maximum consumption. I describe how this idea of ultra-processing can be observed in digital musical interfaces, as well as the larger social, economic, and cultural effects of ultra-processing on our musical outputs. Along with a historical and theoretical perspective on interface design, this talk discusses ways that we might adopt an informed and critical standpoint when creating interfaces for ourselves, and for others.

Filed under: Uncategorized

Reinventing the Plugin Editor

Introduction

What happens when you want to build a plugin but you suffer from not-invented-here syndrome and you like things that are minimalistic and self-contained?

Traditional plugin GUIs like JUCE and VSTGUI, are mature, powerful and give the user a lot for free. On the flipside, they also tend to be heavyweight, have many dependencies, and using them often result in a lot of boilerplate code and state synchronisation issues.

Some years ago I discovered that there was a whole class of minimalistic GUI libraries (Nuklear, Dear ImGui, NanoVG/NanoGUI, etc) originating from the game dev world that were vector based, hardware accelerated and often based on a paradigm know as Immediate Mode GUI.

Immediate mode GUIs offer a clean API that does away with widgets and callbacks, allows for rapid prototyping, minimises boiler plate code and state synchronisation and generally can do more with less code. The memory and disk footprints also tend to be small.

This talk will explore how to integrate a GUI framework that was not built with plugins in mind, to work in an audio plugin editor. In the process we will learn about how GUIs work under the hood, all the nitty gritty details of windowing systems on different platforms, and in particular how threading works in them, and how threading works when your GUI runs inside a plugin that runs inside of a host process - Whose bloody message thread is it really?

And then we will see what happens when we bring hardware accelerated graphics like OpenGL into the picture.

The examples in the talk will be based on Dear ImGui, as I found it the most mature and actively maintained project in the genre. But the majority of content in this talk will be useful for any GUI framework. And for those who just want to have a better understanding and appreciation of all the things a well written GUI framework will do for you.

ADC24 - Reinventing the GUI slide deck

Filed under: Uncategorized

Wait-Free Thread Synchronisation With the SeqLock

When developing real-time audio processing applications in C++, the following problem arises almost inevitably: how can we share data between the real-time audio thread and the other threads (such as a GUI thread) in a way that is real-time safe? How can we synchronise reads and writes to C++ objects across threads, and manage the lifetime of these objects, while remaining wait-free on the real-time thread?

This talk is the second in a series of talks about thread synchronisation in a real-time context. In the first episode, given at ADC 2022, we focused on the case where the real-time thread needs to read a sufficiently large, persistent object that is simultaneously mutated on another thread. In this second episode, we focus on the reverse case: the real-time thread needs to write the value while remaining wait-free, and while other (non-real-time) threads are reading it.

The traditional solution for this problem in audio processing code today is double buffering. This strategy works well in certain cases, but like every algorithm it has certain tradeoffs. If we look beyond the audio industry, it turns out there is actually another strategy that has more favourable tradeoffs for some use cases: the SeqLock.

We describe the general idea of the SeqLock, discuss the different parts of the algorithm, and show a working reference implementation. It turns out that in order to implement a SeqLock portably and without introducing undefined behaviour, we need to reconcile the algorithm with the C++ memory model, which presents an interesting challenge. In order to make it work and be efficient, we need to be very careful with our use of memory fences, memory ordering, and atomic vs. non-atomic memory accesses. Along the way we will learn useful things about writing lock-free code in Standard C++.

Finally, we compare the tradeoffs between SeqLock and other approaches to this problem, offer some guidelines on which approach to use when, and present a proposal to add the SeqLock algorithm to the C++ Standard.

Filed under: Uncategorized

Debugging Audio Content With Visuals

A debugger is a crucial tool to any developer. But how much time did you spend on a breakpoint, sitting in front of a hundred of audio samples, with no idea on how to interpret them?

A good way to get information from these samples is to plot a graphical representation : waveforms, spectrograms... but it's not always straightforward to create such
visualizations, especially when working with compiled languages like C/C++

In this talk, I would like to first review all the tools I discovered and tried these past years, with the goal of making my life easier when debugging audio.

Then, most importantly, I will present a tool I have started developing myself these last months with the objective to solve this issue. A debugger extension which adds new commands, to display the audio content directly from one's buffer classes. No need to recompile your code, nor to instrument it with macros or custom libraries.

Filed under: Uncategorized

Elliptic BLEP

BLIT/BLEP/BLAMPs are a useful tool to reduce aliasing in basic waveforms, nonlinearities, and hard-sync synthesis. There are many variations, but most of them require some lookahead and bookkeeping to work properly.

We start by looking at where aliasing comes from in these situations, how BLEP and friends can help, and the compromises of some common variations.

We'll then introduce a new IIR-based approach based on Holters-Parker resampling. This approach has a simple zero-latency implementation, and has extremely good aliasing-reduction based on elliptic filters.

An implementation will also be released as a zero-dependency open-source C++ library (single-file header). This can also be used as an arbitrary-rate resampler for modulated delay lines and sample playback.

Filed under: Uncategorized

A Critique of Audio Plug-In Formats

Audio plug-in formats such as VST, AU, and AAX have revolutionized the audio workflow, empowering a whole industry of creative coders. However, as programming paradigms and user expectations evolve, the design decisions of these established formats must be re-evaluated. This talk critically examines these design choices, exploring how they can be improved, and there is a new contender: CLAP.

The presentation is divided into two sections: a developer-focused critique and a customer-centric analysis. The developer-focused section delves into fundamental design flaws in plug-in APIs, such as stateful lifecycle management, poor single-use interface design, and why the overarching synchronous “process block” design may become quickly outdated with ever-increasing specialized hardware. The talk also discusses more audio plug-in specific issues like ambiguous parameter/state ownership, the limited utility of the “maxSamplesPerBlock” parameter, and what we can learn from Linux about negotiating bus formats.

The second section critiques audio plug-ins from the user’s standpoint. The talk explores major pain points such as lack of portability, cumbersome copy-protection mechanisms, and the archaic practice of plug-in scanning. The section concludes with a call-to-action for the industry to adopt unified standards as the only real solution to these problems.

Filed under: Uncategorized

LLVM’s Real-Time Safety Revolution

"Error: RealtimeSanitizer: call to malloc detected during execution of nonblocking function MyAudioCallback::process!"

"Warning (-Wfunction-effects): nonblocking function `MyAudioCallback::process` must not call blocking function `SketchyCall`"

Real-time programmers working on mission-critical audio, autonomous vehicle, and aerospace code are well-acquainted with the golden rule: “Thou shalt not call time-unbounded operations in your real-time thread.” Despite its importance, tools to enforce this rule have been non-existent—until now!

At ADC 2023, we presented RealtimeSanitizer; and now it's coming to mainline Clang. In the latest version of LLVM, RealtimeSanitizer (now RTSan) forms one of two new features that help uphold real-time guarantees by detecting calls to malloc, system calls, and other user-defined "unsafe" functions at runtime. We introduce the sanitizer, how it works, and how to use it. Next, we explore Clang's new Performance Constraints system, which can prevent blocking function calls from non-blocking functions at compile time. We will compare and contrast these methods and offer recommendations on how to integrate them into your codebase effectively.

By leveraging these new tools, you can ensure your real-time audio systems remain robust, reliable, and ready for any challenge.

Filed under: Uncategorized

Crunching the Same Numbers on Different Architectures

DSP, microcontroller or a general purpose CPU? What are the major factors affecting design choices in embedded systems today, and how do they compare in terms of computational speed?

Since 1978 DSP chips were invented to cope with some limitations of existing CPUs. Audio processing has always been challenging because of inherently high demand for both throughput and low latency processing features. This led the chip manufacturers to develop specific architectures for such challenging tasks.

During the years the increasing demand of computation led also to develop on-chip hardware accelerators for specific tasks like FIR, IIR and DMA, making the machines more and more performant generation after generation. Hardware acceleration is great, but has a drawback: the design then becomes more and more architecture-locked, preventing a faster renewal of the OEM platforms and the embracement of new emerging technologies.

While the DSP world was evolving, also the general purpose CPUs made very big steps forward in terms of computational speed. They typically run with some high end operating systems like Linux that offer a wide set of ready to use features such as networking, USB, BT… and a huge number of available libraries you can use to fasten the audio application development cycle. This makes them very attractive in terms of flexibility.

In the middle, between specialised DSPs and general purpose CPUs there are microcontrollers. They integrate some other hardware that a product usually requires and provide low power operation. We will explore why they are one of the choices available and how they position in terms of computational performance.

The aim of this talk is to give an overview of the different architectures used in embedded audio signal processing, their advantages and disadvantages, and try to compare them with a holistic view. This will take into account computation power, ease of use, BOM cost, power requirements, expandability and give an overview of why all the architectures are still there in 2024 competing after so many years.

We will make use of real code for the comparison that will also highlight how the different chip architectures have an impact on code design.

Will there be any winner?

Filed under: Uncategorized

Perceptual Convolution

Convolution plays a crucial role in modern music production, extending beyond classical reverberation to applications with neural networks and hardware emulation. The demand for efficient computation has long constrained real-time capabilities. Perceptual Convolution aims to address this challenge by computing only the segments of the impulse response (IR) that are perceptible to the human ear.

Identifying these acoustically important segments of the IR is the key challenge. In this presentation, we will explore the computational savings achieved, methods for analyzing existing IRs to enhance their efficiency, and the impact on performance and perceived audio quality. By focusing on perceptually relevant data, we can improve both the efficiency and the overall auditory experience.

Filed under: Uncategorized