Acoustic Guitar as Percussion
A Mel-Spectrogram Blind Spot
This talk examines the caveats of using mel spectrograms as input representations for audio tagging models.
The mel-spectrogram pipeline drops information at several stages: fixed-window STFTs trade temporal for frequency resolution, mel filtering reduces high-frequency detail, and log compression reduces transient contrast. This can make transient-rich sounds harder to distinguish. For example, PANNs (Pretrained Audio Neural Networks) are large-scale pretrained models for audio pattern recognition that use log-mel spectrograms as input and can misclassify acoustic guitar attacks as percussion.
Because mel spectrograms are widely used, these limitations can be inherited by models built on top of them. The talk considers alternatives including multi-resolution STFTs, CQT, and PCEN, and compares their trade-offs in computational cost, robustness, and classification performance to understand where each might be useful.
Shreya Gupta
Free-lance researcher/audio consultant
Audio Machine Learning researcher specialising in Music Information Retrieval (MIR), with four years of prior studio experience as a producer and mix engineer in the Hindi music industry. She is passionate about integerating MIR tools creative music making tools.