Fixing the One in Two Billion Audio Glitch
Practical Lessons From Troubleshooting Real Time Professional Audio Systems
Joins us on our recent 8-month long journey to fix multiple issues in a professional real-time audio processor, from the moment the first issues were reported by internal users and QA teams, all the way to the point where we were able to confidently release a fix.
We will show the importance of long automated testing sessions, why making audio software observable is essential, and the mindset we should be in as engineers to squash that remaining once in 12 hours audio glitch.
The techniques shown have allowed us to uncover defects with different classes of root causes such as:
- Audio algorithms implementation issues due to floating point rounding errors;
- Scheduling problems on Linux PREEMPT-RT such as priority inversion;
- Memory management in third-party libraries, including machine learning inference runtimes.
We will showcase how building custom tooling is the only way to gain enough knowledge to fix those bugs, and the challenges that come with acquiring, storing and analysing the very large amounts of data produced by the 12 hours+ test runs required to uncover rarely occurring issues.
At the end of this talk, you will be left with practical ideas on how you can build you own tooling to validate your audio software and improve its quality.
Olivier Petit
Head of Development, Creative Software
L-Acoustics
After an MSc in Integrated Circuit design, I have joined the Software and Creative department of L-Acoustics in 2018 as a C++ software engineer. Since then, I have been part of our mission to better connect the world’s best artists with their audiences, ensuring everyone gets the sound quality they deserve. Every day we are tackling the challenge of finding ways to deploy audio solution at the largest scale without any compromise on sound quality and system reliability. We are always putting the audience and our users in the centre, making sure all our products, from hardware to software, are as intuitive and easy to use as possible.
Unlike most of my peers, I’m ashamed to reveal that I don’t play any instrument... Instead you will find me on a climbing wall, or riding my Brompton folding bicycles outside of its comfort zone!