Engineering Practices Break Music Interaction (but Can Also Fix It)
Have you ever encountered an interactive system of great technical prowess but with lousy interaction capabilities? And what about one with a big AI sticker up front?
This situation is far more common than we think. Can we unpack the design process of a technical artefact and understand the people and beliefs that drive it?
The engineer's toolbox is a wide array of tools and tricks to get the job done. But we may be getting more than we bargained for: It carries a way to model reality that sometimes attempts to stabilize the messy world we live in, trying to make sense of it in ways that can bizarrely fail beyond the narrow test scenarios we envision.
- What happens when you apply such a model to the task of real-time audio analysis, trying to make sense of the rich and subjective craft of playing a musical instrument?
In what ways can you fail when your tidy model of reality crumbles, as you realize your practices and beliefs are worth very little for this task? That was me two years ago, working in instrumental interaction with artificial intelligence.
But there was light at the end of the tunnel.
Join me as I discuss a three-year journey designing an expressive, low-latency system for musical instrument sound transformation. This system, now available, emerged from a challenging process of unlearning traditional engineering approaches, embracing the potential of ambiguity and expression in neural networks, while discovering new ways of exercising designer agency, walking a thin line between engineering, data science and music playing."
Franco Caspe
Engineer, maker, musician, and a PhD Student at Queen Mary University of London.
Designing low-latency AI algorithms for musical interaction, bridging musical instruments with expressive synthesizers and audio effects, using neural audio, high-performance software, and human-computer interaction principles.