Participants: Alberto Acquilino and Gary Scavone

Period: 2020 - ongoing

This project aims at developing technologies and signal processing methods to assist students in learning to play wind music instruments. Learning to master a musical instrument involves different technical features that still have to be addressed by music technologies, such as facility with timbre, dynamics, steadiness of tone, attack and release precision and refinement, embouchure configuration and variation, finger position and movement, posture and breathing, other than intonation and rhythm. The application of novel signal processing and feature extraction techniques, together with intuitive and user-friendly software interfaces, are proposed to create new tools that can help instrumentalists and teachers in becoming more efficient and effective with their practice and instruction time. In particular, the development of a system that provides real-time feedback to musicians on the correctness of their technique is considered. Such a system is receiving input data from specially-made hardware components applied on the instrument. The purpose of the project is to develop algorithms - by making use of mathematical models and techniques from the fields of audio spectrum analysis, cognitive sciences, AI and Machine Learning - which enable the software to provide a visualization of the sound quality. In this way, musicians can check every day the correctness of their technique; they can therefore objectively track their progress over time and identify strengths and weaknesses for targeted practice. With such tools, beginner and advanced musicians will be able to more efficiently learn to play correctly. The project objectives are as follows and will in some cases be repeated for each family of instruments to be considered, though the focus will be primarily on wind instruments:

1. Feature identification:
   • Meet with expert music pedagogues to discuss needs, common problematic issues for students;
   • Identify the sensor/signal types and associated features that need to be extracted from the signals in order to provide an assessment or feedback to the student.
2. Data / feature extraction methods:
   • Assess accuracy and efficiency of existing time- and/or frequency-based methods; feasibility of developing new information retrieval techniques for those cases that are not currently available;
   • Algorithm design, testing and validation, especially with an eye toward implementations that can work on lower-cost computational systems, such as microprocessors and systems on chip (SoC).
3. Sensor technology development: Identify and develop sensor systems that can be used beyond simple audio microphones to gather useful performance data, including systems to assess embouchure precision, oral-cavity adjustments, finger position and motion, etc…
4. Interface design: Evaluate and develop systems to best present the information to students of different levels, as well as ways to allow teachers to prescribe exercises and evaluate progress.

Throughout the developments, the validity and usefulness of the systems is therefore verified by teachers and students of the Schulich School of Music, creating a constructive collaboration between the Computational Acoustic Modeling Lab and the world of musical pedagogy with the aim of iteratively improving the quality of the implemented systems.