Conference Year: 2024

Javascript, WebViews and C++

This talk is an "everything you need to know" about the pros, cons, tricks, tips and gotchas of using a WebView and javascript in your native C++ app or plugin.

Over the last few years, more and more developers have been drawn to web-technology for their GUIs because HTML/CSS/JS is so ubiquitous, has great tools, a huge ecosystem, and is fast to iterate on. However, gaffer-taping a WebView onto some back-end code written in C++ can be complicated.

After 30 years of writing UIs (and UI frameworks) in C++, I've spent the last couple of years migrating to WebViews in several projects. This talk is a brain-dump of what you need to know about this from the perspective of an audio/realtime C++ developer.

I'll cover the essential best-practices of how to create and manage mixed C++/JS projects, lessons learned, gotchas, benchmarks, top tips, and all the pros and cons you might be wondering about if you're a "web-curious" C++ coder.

Filed under: Uncategorized

Branch-Free Oscillators for Fun and Profit

A discussion of techniques and approaches for building parallel branch-free oscillators for VA software synthesizer applications in C++ - exploring the stack from low-level machine instructions and CPU architecture up to signal processing concerns and language-level abstractions.

How can a branch-free design help you build more performant oscillators, how does it work underneath, and what are the advantages, disadvantages and potential pitfalls of this type of design approach?

If you're curious about CPU architecture and what's going on underneath your code, this talk will be an interesting window into that sometimes less-than-obvious world.

Filed under: Uncategorized

Pipewire: The How, What and Why of Audio on (Embedded) Linux

Understanding the audio ecosystem on Linux is not straight-forward.

Alsa, who? Pulse-audio, what? And what is this Pipewire everyone keeps talking about?

In this presentation, we want to provide a brief overview of the audio subsystem on Linux, looking at libraries provided by the kernel and how to work with it when writing user-space applications. Our goal is to clarify some of the confusion around the different libraries and tools and empower people to embrace the ecosystem instead of being intimidated by it.

Lastly, we would like to give a hands-on introduction to Pipewire, a new audio server shipped in many Linux distributions. We will cover how Pipewire works and how we use it professionally on our embedded linux platform. We will also show how to configure, introspect and debug the pipewire daemon, all the way to creating DSP chains by simply writing a small config file.

Filed under: Uncategorized

In Praise of Loudness

Let's talk about the greatest controversy in modern music production: LOUDNESS. Sacred to some, profane to others, the desire for loudness has spawned myriad techniques, products, and narratives throughout our industry. But which narratives are misguided and which are enlightening? How do various techniques stack up and what are the compromises? Why has loudness been vilified so much in the last couple decades? And ultimately, what is so useful and seductive about loudness in the music we hear?

This presentation aims to give you a much deeper understanding of loudness: what it is, how it's measured, how technology enhances it, how myths of its goodness or evil came to be, how regulatory bodies came to their distribution standards, how it fits into our products, and how it changes the way we produce and perceive music. So, if you'd like to sit down in front of a limiter without guilt and get your music heard, take a seat and listen to what loud can do for you.

Filed under: Uncategorized

Symphony of a Thousand

In the last few years, there has been an increasing interest in GPU DSP research. While the GPU brings highly parallel processing to the mix, the problems in implementing the real-time requirements and data flow of a GPU audio pipeline are well known. There are other challenges that stand out, such as the complexities of running GPU apps on consumer hardware, as well as identifying "killer apps" for GPU audio: desirable use cases where the GPU really shines and outclasses a classical CPU implementation for the everyday musician.

The aim of this talk is to cover the design, implementation, tradeoffs and compromises needed to build and run a basic GPU synthesizer on consumer hardware. This talk will be hands-on and suitable for audio developers that are relatively fresh to GPU applications in particular. We will present an algorithm implemented in NVIDIA CUDA that is based on straightforward wavetable synthesis and leverages the GPU's parallelism to outclass an equivalent CPU implementation. We will show how to integrate this algorithm and its CUDA kernels with JUCE to create a proof-of-concept synthesizer app.

The usual real-time challenges such as CPU-GPU copying, latency and buffering are addressed, and more hands-on issues such as interopability with 3D graphics running on the same GPU as well as real-time scheduling and persistent kernels will be covered as well as needed.

We purposely target lower-end hardware to explore the feasibility of a GPU synthesizer on consumer hardware, and to possibly enable a path towards a more extensive standalone "GPU hardware synthesizer" in the future.

Filed under: Uncategorized

Real-Time Inference of Neural Networks

Continuing our exploration of implementing neural network inference for real-time audio applications, we have expanded from our initial plugin example to a comprehensive library that simplifies the deployment and integration of neural networks in audio applications.

In this talk, we discuss various aspects of our implementation. Since it is crucial to know whether inference engines exhibit real-time violations, we first quantify real-time violations within inference executions. Subsequently, we explore the integration of these engines in real-time audio environments, specifically addressing the challenges of running multiple instances simultaneously. To accomplish this, we use a static thread pool and, when available, host-provided threads. We also focus on strategies for achieving the lowest possible latency, presenting techniques we have implemented, and open a dialogue on a controversial approach to further reduce latency. Moreover, we share our findings on the performance impact of various factors on inference runtimes. To this end, we have extensively benchmarked different neural network architectures across different inference engines, and can show how differences in input buffer size and model size, as well as previously executed inferences, affect the overall performance.

Filed under: Uncategorized

CANCELLED

Traditional audio processing often relies on labeled data, hindering its scalability and efficiency. This session delves into Differentiable Digital Signal Processing (DDSP), a revolutionary approach that enables unsupervised learning for various audio tasks. We'll explore the core principles of DDSP and its advantages for audio processing.

The presentation will showcase how DDSP can be harnessed for tasks like feature extraction, audio parameterization, and even audio generation – all without the need for extensive labeled datasets. Crucially, DDSP allows us to learn the inherent structural priors of audio data itself. This refers to the underlying patterns and relationships within audio signals that DDSP can leverage to perform complex tasks without explicit human guidance.

Through real-world examples and case studies, we'll gain insights into how DDSP is pushing the boundaries of unsupervised audio processing.

Filed under: Uncategorized

An Efficient, Open-Source C++ Loop Classifier and Tempo Estimator

An efficient, offline C++ algorithm for loop classification and tempo estimation is presented, alongside its evaluation framework. The framework provides the area under the ROC curve (AUC) of the classifier, facilitating regression-free development and tuning of the algorithm. The AUC is now 0.93 when evaluated against the set of files (publicly available on freesound.org under the Creative Commons license) listed in the framework's source code. By providing computation time measurement, the framework has also been useful for optimizing the algorithm, which is now typically over 2500 times faster than real-time (measurement made on a Windows laptop with a 12th Gen Intel Core i7-12800HX processor and 32 GB of RAM). Furthermore, the framework can be used to set the target false positive rate according to the requirements of your application. Algorithm and evaluation framework are open source, and care has been taken to keep the algorithm easily reusable.

The algorithm can be seen as a "classical" algorithm and reuses ideas described elsewhere in the literature. However, the idea behind the classifier is original. A set of loosely plausible numbers of tatums (or ticks) fitting in the duration of the provided audio file is taken. The likelihood of each tatum count hypothesis is evaluated by measuring the distance of each onset to its closest tatum and using the onset's strength in the weighted average of all distances. The average is then compared to a threshold, and, if below, a disambiguation step is carried out, where the number of tatums is reused to determine the most likely tempo (BPM) and time signature.

As implied above, the input audio must be a loop for its tempo to be detected. This limitation was not deemed critical for the application the algorithm was intended for. On the other hand, it opened possibilities to improve the discriminant factor of the classifier, allowing a higher success rate while keeping the false positive rate low. This choice may explain the originality of the approach despite its simplicity.

Filed under: Uncategorized

Introducing ni-midi2

MIDI 2.0 implementations arrived in recent macOS and Linux versions, and Windows MIDI 2.0 support is expected to arrive end of 2024.

ni-midi2 is a modern C++ library implementing MIDI2 UMP 1.1 and MIDI-CI 1.2. The platform-independent library allows you to easily migrate your code to support MIDI 2, while maintaining compatibility to the traditional MIDI 1 protocol.

The talk will cover basic concepts of the library and real-world examples of how it helps you to achieve protocol-agnostic MIDI 2 support in your codebase.

Filed under: Uncategorized

Synchronised Data for Audio Plugins

Have you ever wanted to have your plugin presets show up on different devices? For example, on an iPad, and in your DAW. Or even on the web?

A brief history of a search for a synchronised database solution that works across plugins, mobile apps and the web.

An introduction to methods and libraries that can be used to store preset and other data for plugins, automatically synchronised across devices. Both for general data as well as user-specific data or presets.

Filed under: Uncategorized