Machine Learning Was the Easy Part
Engineering Lessons from Building an Adaptive Audio System
Adaptive audio systems promise to move beyond static presets by learning from user interaction. While much attention is given to machine-learning models, our experience suggests that the difficult problems lie not in model selection, but in engineering a system capable of generating meaningful training data.
This talk presents the lessons learned while developing a closed-loop sound personalization system for hearing assistance, in which users generate training data through interaction with an existing audio processing chain. Users teach an existing DSP pipeline to adapt its parameters according to their individual listening preferences.
Before any machine-learning model could be trained, we first had to engineer a dataset worth learning from. Our models learn from the relationship between acoustic features and user adjustments. Each training sample therefore consists of an acoustic context paired with the corresponding user adjustments.
Designing a system capable of generating these reliable paired observations turned out to be the real engineering challenge. Since deployment on embedded hearables was the goal from the outset, feature selection was guided by computational constraints. Interestingly, a carefully selected set of features not only reduced computational cost but also outperformed substantially larger feature sets.
We will also present the outcome of a recent user study conducted in collaboration with Fraunhofer IDMT and Charité Universitätsmedizin Berlin and conclude with four engineering lessons for building adaptive audio systems, including the considerations involved in transitioning from a validated research prototype to embedded hearable deployment.
Although developed for hearing assistance, the engineering principles presented in this talk apply broadly to adaptive audio systems that combine real-time DSP, embedded machine learning, and human-in-the-loop personalization.
Peggy Sylopp
Peggy Sylopp is a computer scientist, founder, and AI Product Lead specializing in adaptive audio systems and human-centered AI. She founded the Fraunhofer spin-off Sinceare, where she explores how adaptive audio systems can learn user preferences from interaction data, combining DSP, machine learning, and human-centered product design.
Her work spans audio signal processing, machine learning, product discovery, and user-centered design, with a focus on translating complex research into real-world software systems. Peggy is particularly interested in how adaptive audio technologies can bridge the gap between perception, engineering, and intelligent system design