GuiBuh – a framework for automated end-to-end plugin tests
Comprehensive Automatic End-To-End Plugin Testing
Continuous delivery is the place to be. But thorough and comprehensive automated testing poses a significant challenge in building audio applications. While backend unit tests for logical and DSP modules are standard practice, true user-side, end-to-end testing, where the plugin is treated as a sandboxed black box within a DAW, remains limited.
GuiBuh (it's a German pun ^^) introduces an automated testing solution that attaches to an audio host (Reaper) to solve this problem. It combines image processing to verify the GUI states of an audio plugin, alongside audio null tests that verify DSP consistency across application versions.
While GuiBuh interacts with Reaper using its built-in API, interactions with the audio plugin itself rely solely on simulated mouse and keyboard inputs to fully mimic the user’s interactions with the plugin. By doing this, an independent representation of the GUI and plugin interactions is built, fully detaching the tool from any backend bias.
Regression testing is ultimately achieved through three pillars: analysing the visual GUI via image processing, evaluating the audio processed by these specific GUI configurations, and verifying the parameters exposed to the DAW.
Jakob Siebenrock
QA Engineer
Sonible GmbH
Jakob Siebenrock is a Master’s student in Audio Engineering in Graz, Austria, specializing in Digital Signal Processing.
Since joining Sonible in 2021 in QA and front-end support, he has focused on software testing workflows. In 2023, he initiated the development of GuiBuh, an automated end-to-end framework designed to validate both GUI interactions and audio output across applications without requiring access to their backend implementation.