This paper describes a system for modeling, recognizing, and classifying sound textures. The described system translates contemporary approaches from video texture analysis, creating a unique approach in the realm of audio and music. The signal is first represented as a set of mode functions by way of the Empirical Mode Decomposition technique for time/frequency analysis, before expressing the dynamics of these modes as a linear dynamical system (LDS). Both linear and non- linear techniques are utilized in order to learn the system dynamics, which leads to a successful dis- tinction between unique classes of textures. Five classes of sounds comprised a data set, consisting of crackling fire, typewriter action, rainstorms, carbonated beverages, and crowd applause, drawing on a variety of source recordings. Based on this data set the system achieved a classification accu- racy of 90%, which outperformed both a Mel-Frequency Cepstral Coefficient based LDS-modeling approach from the literature, as well as one based on a standard Gaussian Mixture Model classifier.

Purpose - The purpose of this paper is to present an embodied view on human/machine co-creation in general, and musical improvisation in particular.
Design/methodology/approach - Questions and propositions are formed by examining personal work in intelligent, interactive system design.
Findings - Proper consideration of gestural representation and intentionality leads to enhanced potential for collective expression in human/machine interaction.
Originality/value - This approach extends ideas of conversation theory to improvisational contexts based on spontaneous, collective expression.
Keywords: Music, Improvisation, Creativity, Gesture, Embodied cognition, Cybernetics Paper type Conceptual paper

The author discusses performance utilizing his GREIS software system, which is built around the principle of a "scrubbing" interaction with roots in the recording industry and the paradigm of scrubbing tape across a magnetic head.

In the majority of discussions surrounding the design of digital instruments and real-time performance systems, notions such as control and mapping are seen from a classical systems point of view: the former is often seen as a variable from an input device or perhaps some driving signal, while the latter is considered as the liaison between input and output parameters. At the same time there is a large body of research regarding gesture in performance that is concerned with the expressive and communicative nature of musical performance. While these views are certainly central to a conceptual understanding of 'instrument', it can be limiting to consider them a priori as the only proper model, and to mediate one's conception of digital instrument design by fixed notions of control, mapping and gesture. As an example of an alternative way to view instrumental response, control structuring and mapping design, this paper discusses the concept of gesture from the point of view of the perception of human intentionality in sound and how one might consider this in interaction design.

The word 'noise' has taken on various meanings throughout the course of twentieth-century music. Technology has had direct influence on the presence of noise, as phenomenon and as concept, both through its newfound ubiquity in modernity and through its use directly in music production - in electroacoustics. The creative use of technologies has lead to new representation systems for music, and noise - considered as that outside of a given representation - was brought into meaning. This paper examines several moments in which a change in representation brought noise into musical consideration - leading to a 'noise music' for its time before simply becoming understood as music.

Carry Groups are composed of finite and infinite cyclic groups with a natural operation that involves a carry. They are simple to understand but their structure is not immediately obvious. The authors present a structure theorem for a class of these finitely generated groups, showing their equivalence to certain direct sums of the integers with finite cyclic groups.

In this paper we discuss aspects of our work in developing performance systems that are geared towards human-machine co-performance with a particular emphasis on improvisation. We present one particular system, FILTER, which was created by the first author and tested by the three authors in the context of their electro-acoustic performance trio. We discuss how this timbrally rich and highly non-idiomatic musical context has challenged the design of the system, with particular emphasis on the mapping of machine listening parameters to higher level behaviors of the system in such a way that spontaneity and creativity are encouraged while maintaining a sense of novel dialogue.

This paper describes a system for separating a musical stream into sections having different textural qualities. This system translates several contemporary approaches to video texture analysis, creating a novel approach in the realm of audio and music. We first represent the signal as a set of mode functions by way of the Empirical Mode Decomposition (EMD) technique for time/frequency analysis, before expressing the dynamics of these modes as a linear dynamical system (LDS). We utilize both linear and nonlinear techniques in order to learn the system dynamics, which leads to a successful separation of the audio in time and frequency.

In this paper we discuss our approach to designing improvising music systems whose intelligence is centered around careful listening, particularly to qualities related to timbre and texture. Our interest lies in systems that can make contextual decisions based on the overall character of the en- tire sound field, as well as the specific shape and contour created by each player. We describe the paradigm of "expanded instrument" systems that we in turn build upon, endowing these with the ability to listen, recognize and transform performer's sound in a contextually relevant fashion. This musical context is defined by our free improvisation trio, which we briefly describe. The different modules of our current system are described, including a new tool for real-time sound analysis and method for sonic texture recognition.

This paper describes the architecture for a new mixing console that was especially designed for telematic live-music collaborations. The prototype mixer is software-based and programmed in Max/MSP. It has many traditional features but also a number of extra modules that are important for telematic projects: latency meter, remote data link, auralization unit, remote sound level calibration unit, remote monitoring, and a synchronized remote audio-recording unit.

This paper describes a novel system that combines machine listening with evolutionary algorithms. The focus is on free improvisation, wherein the interaction between player, sound recognition and the evolutionary process provides an overall framework that guides the improvisation. The project is also distinguished by the close attention paid to the nature of the sound features, and the influence of their dynamics on the resultant sound output. The particular features for sound analysis were chosen in order to focus on timbral and textural sound elements, while the notion of "sonic gesture" is used as a framework for the note-level recognition of performer's sound output, using a Hidden Markov Model based approach. The paper discusses the design of the system, the underlying musical philosophy that led to its construction as well as the boundary between system and composition, citing a recent composition as an example application.

The internet has proven to be an important catalyst in bringing together musicians for remote collaboration and performance. Existing technologies for network audio streaming possess varying degrees of technological transparency in regards to allowable bandwidth, latency, software interface constraints among other factors. In another realm of digital audio, the performance of "laptop music" presents a set of challenges in regards to human-computer and inter-performer interaction - particularly in the context of improvisation. This paper discusses the limitations as well as newfound freedoms that can arise in the construction of musical performance systems that merge the paradigms of laptop music and network music. Several varied creative solutions are presented from personal work created in the past several years which consider the meaning of digital music collaboration, the experience of sound-making in remote physical spaces and the challenge of improvising across time and space with limited visual feedback. These examples include shared audio processing over high- speed networks, shared control of locally-generated sound synthesis, working with artifacts in low-bandwidth audio chat clients and the use of evolutionary algorithms to guide group improvisations.

This paper presents an approach to the analysis of gestural data and extraction of related features from a cloth-based instrument. Issues surrounding the meaning of gesture and intentionality in such a performance environment are discussed, and we present a solution to analyzing and extracting information in a way that leverages the inherent quality of the cloth-as-controller. Other factors are considered in the system design, including the performance context in which the goal is to elicit improvised play from participants who do not possess an a priori model of interaction or vocabulary of acceptable gestural input.

This paper describes musical experiments aimed at designing control structures for navigating complex and continuous sonic spaces. The focus is on sound processing techniques which contain a high number of control parameters, and which exhibit subtle and interesting micro-variations and textural qualities when controlled properly. The examples all use a simple low-dimensional controller - a standard graphics tablet - and the task of initimate and subtle textural manipulations is left to the design of proper mappings, created using a custom toolbox of mapping functions. This work further acts to contextualize past theoretical results by the given musical presentations, and arrives at some conclusions about the interplay between musical intention, control strategies and the process of their design.

This paper presents the Library of Maps toolbox to aid in the mapping of control parameters to sound synthesis parameters via strategies that result from a geometric representation of control. A set of objects have been created for Max/MSP and Jitter that allow the user to map arbitrary high-dimensional data from control to sound parameter space, and to visualize this through the use of Jitter and OpenGL. The mapping implementations are discussed and related to existing work.

This paper presents an implementation of the phase vocoder within a Gaussian state-space framework. Rather than formulate the problem as a deterministic evolution of frequencies centered around a given bin, this evolution is treated stochastically by introducing noise into the dynamics matrix of the recursive state equation. This produces effects on the roughness of the input sound, which vary depending on the position within the matrix where the noise is added, how it is propagated throughout the matrix and further by the variance of the noise input.

The role of mapping as determinant of expressivity is examined. Issues surrounding the mapping of real-time control parameters to sound synthesis parameters are discussed, including several representations of the problem. Finally a study is presented which examines the effect of mapping on musical expressivity, on the ability to navigate sonic exploration and on visual feedback.

The choice of mapping strategies to effectively map controller variables to sound synthesis algorithms is examined. Drawing from underlying mathematical theory, this paper seeks to establish a framework through which these strategies can be compared, with the goal of achieving an appropriate match between mapping and musical performance context. This method of comparison is applied to existing work, while a suggestion is offered on how to integrate and extend this work. Specifically, existing implementations which give control and synthesis parameter spaces a geometric representation are the focus.

Visual communication is an important aspect of music performance, for example, to pick up temporal cues and find the right entries. Visual cues can also be instrumental to negotiate the solo order in improvised music or enable social exchange, for example, by signaling someone that her solo was well received. The problem with visual communication is that one has to catch someone else's attention, and visual cues outside someone's visual field cannot be detected, even more so if the addressee is busy reading a music score or closing his eyes in a Free Music session. Acoustic communication does not encounter these challenges, but of course someone does not want to disturb the music with other acoustic signals. The haptic modality has the advantage that it does not necessarily interfere with the acoustic signal and does not require attention. However, it allows interpersonal communication if both parties are within close proximity. Using telematic interfaces solves the problem of proximity by allowing participants to communicate over any physical distance. In the project presented here, haptic interfaces were explored in connection with an intelligent music system, CAIRA, to examine both the effect of human/machine and inter-human communication. [Work supported by the National Science Foundation, No. 1002851.]

The idea of the project reported here was to design a system that builds on touch to enable people with severe hearing impairments to "listen" to music through a process called sensory substitution. The goal was to transform the auditory parameter space into one that is adequate for haptic perception. The approach reported here builds on (i) the design of a haptic display, a tabletop device with 8-24 actuators that can be driven individually, (ii) machine learning algorithms, and (iii) a psychophysical study to determine which music cues can be perceived through touch. The latter was necessary because vibrotactile perception is not yet well understood in the context of music perception. The double‐blind study analyzes how vibrotactile stimuli contribute to the perception, cognition, and distinction of sounds in human participants who have been trained versus those who have not. In order to ensure that normal-hearing participants could not hear sounds radiated from the haptic display, sound isolating headphones were used to playback pink noise during the experiment

The paradigm of live performance mixing acoustics and electronics has predominantly focused on simple background "tape music", human players performing highly structured sample-based music (e.g., using the ABLETON LIVE software), or reactive systems that respond to player qualities such as timing, pitch, and so on. In this talk I will present my approach to improvised "laptop performance" that focuses on the transformation of acoustic players in real-time. Rather than simply altering the acoustic content in the manner of an effect processor, the goal is to capture notes and phrases in short-term memory and to re-articulate the material so that it presents a new gestural inflection and timbral content that can be completely novel or suggestive of other players' sound. The system presented utilizes a hybrid system combining spectral analysis and feature extraction with block-based temporal processing and a feedback delay network. The interaction paradigm of "scrubbing" the intermediate time-frequency representation is used to generate the ultimate output. The result in an ensemble context is an extended palette that can "keep up" with the musical dialog while eliciting the subtle textural qualities of acoustic players.

The development of automated music transcription systems focuses predominantly on polyphonic musical instruments. At the same time, the analysis of a monophonic instrument is usually much simpler wherein pitch, loudness, and duration of individual notes may be tracked robustly. When using extended techniques, however, many more parameters than the aforementioned three can be meaningful for the performed music. This paper explores the challenges that extended techniques pose for music recognition systems using the example of the saxophone. The goal is to correctly identify extended techniques over the whole range of the instrument, including subtones, multiphonics, growl, and other voice-enhanced tones, as well as tones where the reed is supported by the lower teeth. The feature analysis is based on cepstrum, spectral moments, pitch, and roughness, among other features. A hidden Markov model is used to recognize the trajectory of the various extended techniques based on the given feature space. Finally, it is demonstrated how the recognizer can be integrated into an intelligent live electronics system to control its parameters. For example, the characteristics of a virtual acoustic enclosure (room size, reverberation time, etc.) can be adapted this way.

There has been a considerable amount of research in the area of timbre perception, and specifically attempts at finding acoustic correlates that generate, influence or otherwise contribute to the various related perceptual attributes. Much attention has been paid in the literature to certain acoustic properties that have proven to be perceptually relevant, as has been shown through a combination of user studies, signal analysis/synthesis and various mathematical techniques - most notably multidimensional scaling. Contrary to this, the subjective notion of "texture" has seen little attention in the area of sound perception. A primary difficulty in dealing with sound texture is that it is not immediately clear how one defines it and further develops classification schemes for comparison and grouping. While timbre may not be ordinal, it has a clear perceptual structure that can lead to groupings and associations such as instrumental families and acoustical source properties. In a musical setting, I would argue, texture relates more to one's phenomenological experience of a sound result. In considering sonic qualities, texture can be seen to differ from timbre in terms of degree of separability: timbre being perceived as a unified sound object and texture perception relating to micro-variations in spectro-temporal sound properties over larger time scales. Coming from a perceptual signal processing point of view, I will present different models for texture analysis and synthesis that have been proposed in the literature. Rather than focusing on the deeper mathematics of such models, I will present the underlying assumptions and concepts dealing with questions of determinism vs. stochasticity, spectral vs. temporal models, time scales, etc. and how they relate to implicit models of perception. I will further discuss the manner in which texture can be related to both timbre and melodic intervals (differing in regards to degree of temporal separation), and will conclude with suggestions for how texture might be considered from the perspective of electroacoustic composition in constructing a sense of movement, density and tension. Existing examples from the EA literature will be employed to emphasize this use of texture in composition, and an analysis will be given that links with the associated discussion of perceptual/signal modeling.

Il a été démontré que l'association entre le donnés de sortie provenant d'un controller aux parametres de son, autrement dit "mapping", a un effet important sur le retour exercer par l'instrument electronique. Cette presentation examine le role du mapping comme facteur determinant dans l'expression musicale relié a de tel instruments. Certains aspects du mapping entre les paramêtres de contrôle en temps-reel et les paramêtres de la synthèse sonore sont abordés, notamment les notions d'interpolation et de representation fonctionelle. Plus specifiquement, l'interpretation geometrique de contrôle en temps-reel est etudiée. Plusieurs implementations sont presentées aussi bien que l'analyse des benefices et inconvenients pour chacun des cas. Finalement, je presente les resultats d'une etude qui compare les differents mappings basé sur l'interpolation. Les critères de comparisons étant les effet sur l'expression musicale, la possibilité de naviguer dans l'espace sonore, la reproducibilité et enfin leurs potentiel d'utilisation pour le feed-back visuel.

The term "noise" has taken on disparate meanings in its use as sonic referent. It is wholly subjective and context dependent, and yet noise as concept has been a recurrent thread in the development of experimental musics of the past 100 years in general and electroacoustic music in particular. Without attempting to define noise in absolute terms, this paper will examine its conceptualization in certain periods of 20th century music, and relate this to a more current trend that one might consider as a contemporary production of noise: the subversion of a musical system and the (mis)use of technology to achieve this.

From before the time we are born, we are exposed to the infinite possibilities of our sound environment. Through our interaction with society at large, we begin to experience sound not as itself but as an object given meaning through a dominant representation that we call music. As a result, an analytic mode of listening is privileged over a holistic one, producing a hierarchy wherein different "levels" of listening exist. Not only do certain sounds and relations to sound get relegated to the background, but listening identities that do not rely on analysis become marginalized as well. This notion of "listening identity" refers here to one's unique way of listening that is informed by their larger social identity. We maintain that this privileged analytic mode of listening reflects a bourgeois ideal of individualism in contrast to holistic listening, which can be seen as contextual and relational. For the purpose of this presentation we will be focusing on gendered and classed identities as they relate to music, sound and modes of listening. One way to de-emphasize the aforementioned standard for interacting with a musical work is to introduce noise as an element of composition. Noise is not (and can not be) directly defined here, as it refers to a subjective quality that is rooted in one's process of listening. Nevertheless, as there exists a dominant musical representation there also exists an outside: one that cannot be understood through the symbols that define time and frequency (pitch) relationships. In the context of electroacoustic music, this noise is the de-structuring of the tools and systems of production themselves - the sound that results from the misuse of technology. Not only does this subvert the accepted means to a musical end, but it is subversive in that its reception does not rely on one established sonic viewpoint. Within this space of noise as music, the individual is allowed to navigate their own listening identity - be it analytic, holistic or otherwise - and create their own meaning that is not solely determined by dominant ideology. In this presentation we will address issues surrounding the role that noise can play in the formation of alternative listening identities, specifically in the context of electroacoustic music practice.