Modular Spectral Audio Processing on FPGAs with C/C++
In this talk, we will present our system for modular spectral audio processing on a field-programmable gate array (FPGA) platform using a high-level programming approach. FPGAs are integrated circuits or chips with a large number of reconfigurable logic units or programmable logic, enabling custom hardware programming. They provide high throughput, ultra-low latency and vast parallelisation possibilities, making them well-suited to audio processing applications, especially spectral processing, which is generally challenging to implement in computationally limited embedded environments. Through our system, we will showcase how these advantages can be leveraged to implement real-time spectral processing as a set of custom hardware modules that can be interconnected with each other to form processing graphs in FPGA hardware. These modules, known as intellectual property (IP) cores, can be designed in C/C++ using high-level synthesis tools, rather than the low-level hardware description languages such as VHDL and Verilog that FPGA design commonly entails and that may be out of reach for most audio developers, musicians and creative technologists. The modules form a part of ModFPGA, our open-source modular audio processing framework, that also provides a base audio system for audio I/O, MIDI and sensor interaction, and takes care of task division between the ARM CPU and Programmable Logic of a traditional FPGA system-on-chip (SoC). Two spectral processing techniques will be covered: streaming spectral processing such as the phase vocoder, which can be used for real-time pitch shifting, time-stretching, spectral granulation and blurring, as well as time-varying convolution for live audio signals, enabling dynamic spectral interaction between two live input signals. We will show how users can build their own spectral processing FPGA designs with the framework, and examine the design and optimization considerations, benefits and constraints of this platform. We will also highlight how these custom hardware modules can integrate and communicate with our bare-metal implementation of the sound and music computing system, Csound, running on the SoC's ARM CPU for co-processing.
Aman Jagwani
PhD Researcher
Maynooth University
Aman Jagwani, from Mumbai, India, is an audio programmer, musician, and researcher. He completed his Bachelor's degree in Electronic Production and Design and Drum-set Performance at Berklee College of Music, Boston, and later earned a Master of Science in Sound and Music Computing from Maynooth University.
Currently, Aman is undertaking his Ph.D. at Maynooth University with supervisor, Prof. Victor Lazzarini, for which he received the Hume Fellow Scholarship Award. His research centres on audio DSP, embedded systems, FPGAs and interactive musical experience design.
Aman is also involved in the Jazz and Electronic music scenes as a composer, producer and performer. He often integrates custom plugins, hardware devices and instruments into his performances, merging his technological insights with his musical expression.
Additionally, Aman has experience with creating custom audio and interactive systems for immersive sound and art installations, having contributed to projects in India, Europe, and the USA.