| Human-Computer Interaction: Real-time Gestural Control of Computer Music |
|
|---|
Prof. Marcelo M. Wanderley
e-mail: mwanderley@acm.org
Phone: 398-4535 # 00917
Office: E228
Office Hours: Mondays 2:30-4:30 pm.
| Course Details: |
|---|
Time and place Credits Time:
Thursdays 8:30-11:30 amNumber of credits: 3 Place: LSR3
(Marvin Duchow Library) - 550 Sherbrooke St. W., 11th FloorNEWS: last update on Nov 14, 2002.
Class 10 and class 11 online. Talks on Class 8 available (Wes Hatch and Rob Ferguson). Class 7 online. Complementary Readings for Class 7: Vertegaal's MSc thesis and ICMC94 paper on the evaluation of input devices. Class 6 online here and Cory's presentation. iCube sensors arrived (Monday Oct. 07 2002). Software installed in SPCL control room. Slides from class 5 and Pierre-Yves' presentation. Slides from class 4 available, including Richard McKenzie's presentation on sensors. Joel Ryan's paper (link to html version) Slides from class 1, class 2 and class3 available (powerpoint files and jpeg files) Directions for class presentation.
| Course Description: |
|---|
Computers are already long able to synthesize high quality sound in real-time. The question nowadays is how to play the computer as a real-time instrument. In order to answer this question, the analysis of performer gestures and the design of digital musical instruments using the computer are essential steps towards the definition of the interaction possibilities between the performer and the machine. This seminar aims at presenting the ground notions regarding human-computer interaction (HCI) in complex, multi-parametric contexts such as computer music, and interactive live performance.Specifically, the design of gestural controllers will be analysed in detail through:
The review of basic topics on sound synthesis with respect to real-time control The analysis of the existing literature on the design of input devices in HCI and possible applications of this knowledge to the design and evaluation of new interfaces for musical expression. Possibilities regarding gestural acquisition: direct, indirect and physiological gestural acquisition. Mapping strategies relating gestural variables to synthesis variables and their influence on instrument expressiveness.
| Teaching Methods: |
|---|
The seminar will basically consist on lectures and student presentations, followed by in-class discussions. Lectures will be presented in Powerpoint and examples will be provided by means of videos and demos using jMax and MAX/MSP.Student presentations will be based on selected readings and on their own projects developed during the seminar.
Practical work leading to the definition of the prototype can be performed at the Sound Processing and Control Laboratory (SPCL, pronounced "Special"), at the EMS (Electronic Music Studio), fifth floor, east wing, Strathcona Music Building. Keys to the lab will be available on demand.
| Learning Outcomes: |
|---|
At the successful completion of the seminar, students will have:- A thorough understanding of the major trends in human-computer interaction in musical contexts, including a substantial knowledge about the published works in this area.
- A clear idea of the functioning of several available input devices and will be able to evaluate their fitness to different musical contexts, including mapping strategies;
- Hands-on experience in a HCI related project consisting of preparing a working prototype based on a short research project;
- Knowledge about writing down ideas and results in the form of a publishable conference paper (E.g. for NIME03).
| Evaluation: |
|---|
Seminar evaluation will be based on the following items:- (10%) Research on the state of the art (CMJ format) in the chosen field, with possible publication in the ICMA/EMF Working Group on Interactive Systems and Instrument Design in Music.
- (20%) Class participation and student presentation (1 formal presentation about a research paper) in weeks 5 to 10.
- (30%) A final paper describing the project in detail (in NIME03 format), including substantial discussion on both the technical aspects involved as well as on the human-computer interaction aspects (usability, etc.) (Week 13 - Nov. 28)
- (40%) A working prototype, result of the project, to be demonstrated in week 12 (Nov. 21 2002).
| Class Outline: |
|---|
| Class | Date | Subject | Readings (to be prepared before class!!) | Evaluation |
| 1 | Sept/05 | Introduction to the course
|
- Buchla, Buxton et al.: Round Table from Trends in Gestural Control of Music | ---- |
| 2 | Sept/12 | Gestural Control of Sound Synthesis
Preparing and delivering scientific/
technological presentations |
- Wanderley: Gestural
Control of Music
- Mulder: Towards a Choice of Gestural Constraints for Instrumental Performers - Tanaka: Music Performance Practice on Sensor-based Instruments |
Definition of the different projects |
| 3 | Sept/19 | Gestures and Feedback | - Cadoz and Wanderley: Gesture-
Music
- Iazzetta: Meaning in Musical Gestures - Goldstein: Gestural Coherence and Musical Interaction Design |
---- |
| 4 | Sept/26 | Direct gesture acquisition | - Bongers: Physical
Interfaces in Electronic Arts
- Ryan: Some Remarks on Musical Instrument Design at STEIM |
Bibliography review due |
| 5 | Oct/03 | Gestural Controllers | - Paradiso: Electronic
Music - New Ways to Play
- Cook: Principles for Designing Computer Music Controllers - Wessel and Wright: Problems and Prospects for Intimate Musical Control of Computers |
Beginning of student presentations
- Pierre Yves Fortier
|
| 6 | Oct/10 | Research on input devices in HCI and applications to Music | - Buxton: The Haptic Channel
- Card et al.: A Morphological Analysis of the Design Space of Input Devices - Jacob et al.: Integrality and Separability of Input Devices |
- Cory McKay
- Philippe Zaborowski |
| 7 | Oct/17 | Evaluation of Input Devices for Musical Applications | - Vertegaal et al.: Towards
a Musician's Cockpit: Transducers, Feedback and Musical Function
- Wanderley et al.: On the Choice of Transducer Technologies for Specific Musical Functions - Orio et al.: New Interfaces for Musical Performance |
- Adam Tindale
- Andreas Luesse |
| . | . | Welcome to the middle of the course! | . | . |
| 8 | Oct/25 | Other gestural acquisition possibilities
Writing research papers
|
- Marrin-Nakra: Searching for Meaning in Gestural Data: Interpretive Feature Extraction and Signal Processing for Affective and Expressive Content | - Robert Ferguson
- Wesley Hatch |
| 9 | Oct/31 | Mapping strategies for musical performance. | - Hunt et al.: Towards
a Model of Instrumental Mapping for Expert Interaction
- Hunt and Kirk: Mapping Strategies for Musical Performance |
- Caroline Traube
- Andrew Brouse |
| 10 | Nov/07 | Applications/Mapping | - Garnett and Goudeseune: Performance Factors in Control of High-Dimensional Spaces | End of student presentations |
| 11 | Nov/14 | Expressive performer gestures | - Davidson: Visual Perception of Performance Manner in the Movements
of Solo Musicians
- Wanderley: Quantitative Analysis of Non-Obvious Performer Gestures - Wanderley: Non-Obvious Performer Gestures in Instrumental Music |
---- |
| 12 | Nov/21 | Paper presentation | Wanderley and Depalle: Gesturally-Controlled Digital Audio Effects | Final Paper due! |
| 13 | Nov/28 | Final Project Presentation | ---- | Prototype Demonstration |
| Bibliography: |
|---|
Required:
M. Wanderley and M. Battier (eds.). 2000. Trends
in Gestural Control of Music. IRCAM - Centre Pompidou.
Available from the EMF
- CDemusic
Complementary:
ACM SIGCHI publications (Digital library freely available inside McGill)
Journals:
Computer Music JournalProceedings:
Journal of New Music Research
Organised Sound
NIME proceedingsHundreds of references available from the ICMA/EMF Working Group on Interactive Systems and Instrument Design in Music!
ICMCs (International Computer Music Conference)