Interactivity, where to from here

Interactivity, where to from here
of 14

Please download to get full document.

View again

All materials on our website are shared by users. If you have any questions about copyright issues, please report us to resolve them. We are always happy to assist you.
Share
Tags
Transcript
  Interactivity, where to from here? Dr Garth Paine garth@activatedspace.com.auhttp://www.mti.dmu.ac.uk and http://www.activatedspace.com.au Abstract This article raises some points of interest and marks out some pointers to alternative approaches to thedesign and execution of interactive music systems and artworks, which pursue interaction that: • does not include any pre-defined pathways, • takes dynamic morphology as its foundation, and • implements dynamic software infrastructures, built on the object-oriented model, providingdynamic instrument instantiation, orchestration and timbrel   control.It is intended that such design would be a precursor to a new approach to interactivity that respondsmore directly and uniquely to those who engage in the work, and in so doing, rewards then more richlyfor their time, energy and enthusiasm. Introduction Over the last six or so years, I have created a number of large scale audio-visual environments thathave been controlled, ‘performed’, or engaged with, by people through the dynamic of their movementand the patterning of their behavior. This installation practice has fueled a consideration of the natureof interactivity. The terms interactive and interactivity are broadly applied in the new media arts,however, the diversity of application, has, lead to a lack of focus. It will be argued that the terminteractivity is therefore widely abused, and in line with Bongers (2000), that most systems are notinteractive, but simply reactive or responsive, because they lack a level of cognition.In order to strip away the influences of current usage and artistic practice, dictionary definitions of theterm interaction are introduced as a starting point in an attempt to establish a solid understanding of thesemantics of the term, and subsequently, explore how such a definition might influence system design,artistic applications and a working perception of what an interactive system might be.It will also be argued that streamed approaches to data representation and interpretations for, digital(interactive) instruments, and, responsive (interactive) sound installations is preferential to event basedsystems, because it more accurately reflects an ongoing inter-relationship between the user and thesystem, and as such provides a more flexible, responsive, and artistically rewarding outcome than asystem based on the triggering of pre-made, finite content (audio samples for example).Dynamic morphology, ( Wishart:1996 ) is explored as a conceptual framework for dealing with streameddata that facilitates an exploration of dynamic timbre in interactive, responsive music systems, andmore broadly as a conceptual framework for the design of truly interactive systems, covering human-computer interface and sound synthesis applications. Smalley’s spectro-morphology ( Smalley: 1986  )follows a similar line of enquiry, but from an analytical perspective. Both theories deal with theevolution of sound over time and provide a model for dealing with all sounds as compositionalmaterial. A detailed treatment of the nature of the relationship between the excitation source and theevolution of the sound is absolutely critical if those engaged in an interactive, responsive sound systemare to perceive a correlation between the quality of their input and the quality of the resulting sound(s).  Interactivity, where to from here? Dr Garth Paine   Interaction In the technology age, interactivity has become a major consideration in the development of acontemporary artistic practice that engages with the proliferation of computer-based technologies. Thisproliferation has seen a revolution in the fields of animation and image generation as well as sound artand music composition. It is the harbinger of an entirely new artistic praxis, giving rise to time basedgenres that apply everything from ‘artificial intelligence’ to tactile engagement as a means to activelyengage the ‘spectator’/audience, and in so doing, make the work vary dynamically. Such a work isdefined in the moment, by the nature of the direct or indirect influence(s) brought to bear. The audio-visual output is either made up of a collection of pre-defined material, triggered on the basis of distinctconditions, or generated algorithmically in realtime. Much of this work is called interactive. However,what do we mean by interactive? So many things are said to be interactive that the common usage of the term is suffering from a lack of focus.Before detailing the current approaches to interactive musical systems, the semantics of the terminteractive must be set on a firm foundation, and so I turn to the following dictionary definitions.The Oxford English Dictionary (2000) defines interaction as follows:The prefix inter- [meaning] Between, among, mutually, reciprocally.interact [meaning to], act reciprocally or on each otherinteraction a noun, [meaning to] blend with each otherThe Collins English Dictionary (1992) contains the following definitions:interact is a verb, [meaning] to act on or in close relation with each otherinteraction is a noun, [meaning]1.A mutual or reciprocal action or influence2.2. Physics, The transfer of energy between elementary particles, between a particle and afield or between fields.interactive adjective, [meaning]1.Allowing or relating to continuous two-way transfer of information between a user and thecentral point of a communication system, such as a computer or television.2.(of two or more person, forces etc.) acting upon or in close relation with each other;i.e. interacting.The prefix of the word interaction, inter- is defined as something between, among, mutual, reciprocal.This definition implies that the two parties act upon each other; that the parties exchange something,they act upon each other in a way that is reciprocal. The Collins English Dictionary definition of interaction outlines an action that involves reciprocal influence. In the field of physics, it leads us tounderstand that an exchange of energy takes place.How then do these definitions translate into the area of interactive music systems, and the new mediaarts? Does an exchange of energy occur when one is viewing a CD-ROM? That is an exchange of energy between the viewer and the product? An exchange of information certainly occurs, but anexchange of energy, probably not. The user requests a piece of information, and the computer,through the programming of the CD-ROM, delivers that information to a screen in such a way that theuser can comprehend it.One could argue that a transfer of energy takes place when someone is playing a computer game thatrequires a racing car driving wheel, a gear changer and brake and acceleration pedals to be used (manyof these can be seen in amusement arcades). In this case the user is directly transferring energy throughthe interface by turning the steering wheel, changing gears and possibly operating the accelerator andbrake pedals. This energy is certainly transferred to the interface, we know this from the “OUT OFORDER” signs that regularly grace the amusement arcades. The variation in condition of the interfaceis transferred as data to the computer program. This data is used by the computer to construct ascenario that is then drawn to the screen, to which the user responds. There is clearly a causal loop  Interactivity, where to from here? Dr Garth Paine   here. A causal loop being a scenario in which all parties require the other for their survival, and wherethe interaction of all parties maintains a balanced system. However; • Does the racetrack alter in response to the behaviour of the driver? and, • is there actually a reciprocal energy transfer taking place?The answer to both these questions is no, the user is simply attempting to maintain a state that isacceptable to the criteria of the game, that is, to keep the car moving forward and on the track. Thecomputer program defines the conditions to be met, and these conditions do not change as a result of users input. They are the same every time the game is played. Music and Interaction In Interactive Music Systems ( 1994 ), Robert Rowe defines an interactive computer music system as“one whose behavior changes in response to musical inputs”. In this case, we can assume the term‘musical’ implies a common understanding. Winkler ( 1998 ) expresses this ‘musical’ understanding asbeing made up of a “huge number of shared assumptions and implied rules based on years of collectiveexperience.”Rowe continues by discussing the interpretation of low-level musical signals into structured high-levelrepresentations. Rowe suggests that interactive music systems:... interpret the input by evaluating human musical understanding (Rowe:1994:3) … In theirinterpretation of musical input, interactive systems implement some collection of concepts,often related to the structure musicians commonly assume. Each interactive system alsoincludes methods for constructing responses, to be generated when particular input constructsare found. As methods of interpretation approach the successful representation of humanmusical concepts, and as response algorithms move towards an emulation of humanperformance practices, programs come increasingly close to making sense of andaccomplishing an instruction such as broaden the end of the phrase. (ibid:4).Clearly Rowe’s position only remains true while the input is of a instrumental nature. If the systeminput is a human gesture, be it a dance troupe or a solo music performer, or for that matter, a memberof the public (as is true of most interactive, responsive sound installation works) the definition becomesproblematic.Here we face the first challenge in searching for a definition of what interactivity means when appliedto digital music systems. The Rowe definition is founded on pre-existing musical practice, that is ittakes chromatic music practice, focusing on notes, time signatures, rhythms and the like as itsfoundation; it does not derive from the inherent qualities of the nature of engagement such an‘interactive’ system may offer.Todd Winkler ( 1998 ) expands the Rowe definition with four distinctions. He suggests that interactivesystems present differing levels of interaction, and specifies the following models:The Conductor Model, exemplified by a symphony orchestra, where everyone is controlled from asingle source. The musical interpretation and execution is dictated by a master controller, theconductor.The Chamber Music Model, exemplified by the string quartet, where control may be passed fromplayer to player at different moments within a performance. Winkler says, “Intonation, phrasing andtempo are constantly in flux, with control often passed around to the musician with the most prominentmusical material.” (Winkler:1998:25)The Improvisation Model, exemplified by the jazz combo, where the musicians operate within adefined framework, frequently pass control, and vary the musical score with improvised interjections,  Interactivity, where to from here? Dr Garth Paine   interplay and improvised solos passages. Winkler comments “Musicians trade off taking control of themusic, fashioning their solos into spontaneous personal statements that alter and influence thesurrounding accompaniment.” (ibid:26) The outcome is perceived as musical because it has a sharedform and concurs with Winkler’s definition of musical understanding above. This behavior establishesa kind of musical intelligence.An extension of the improvisation model above is free improvisation, where a broad range of oftenchaotic interchanges, governed by a common musical understanding (the musical intelligence justsummarized), encourage a homogenous musical outcome from a vast amalgam of inputs. No formalstructure is agreed in advance, and so the musicians continuously respond to each other ad infinitum.Whilst the Winkler categories form a useful platform for discussion, it should be noted that the notionof interaction is not well defined in musical theory. For instance Bongers (2000) points out in PhysicalInterfaces in the Electronic Arts, Interaction Theory and Interfacing Techniques for Real-timePerformance that many interactive music systems are in fact reactive systems, due to the absence of cognition. These systems represent Winkler’s conductor model, failing to exhibit the dynamicintelligence of the other models.As was true of Rowe’s definition of an interactive music system, discussed above, these definitions areall coaxed in terms of existing musical practice. Interactivity may offer an entirely new approach tomusic making, and so in order to avoid getting stuck in the current musical paradigms, we shouldquestion, not only the nature of the system input (such as musical notes, tempi, rhythms, or humangestures, dance movement, or a conductor’s gestures), but we should pay equal attention to the outputof the system, and the qualitative relationship between the two.In Interactive Music Systems, Robert Rowe outlines three stages of an interactive music systema.Sensing, which includes pitch and rhythmic pattern detectorsb.Processing, which includes the scheduling of tasks that create musical events inresponse to the sensed inputsc.Response, where the constructed audible response is delivered back to the interactiveagent.In a situation where the system is designed to accompany or improvise with a musician theconstruction of the responses within an agreed musical aesthetic makes sense, however, this approachdoes nothing to further our exploration of the inherent qualities of an interactive music system, itsimple squeezes interaction into a known template.I would like to proffer one further model based on the process of human conversation. Winkler alsoraises this conceptual model, commenting that human conversation, like any good interaction is a “two-way street … two people sharing words and thoughts, both parties engaged. Ideas seem to fly. Onethought spontaneously affects the next.” (ibid:3). The human conversation model represents a numberof other characteristics as well: The conversation is • unique and personal to those individuals, • unique to that moment of interaction, varying in accordance with the unfolding dialog, but is • maintained within a common understood paradigm ( both parties speak the same language, andaddress the same topic).Within such an interaction the starting point is known by one of the parties, and whilst in somediscussions there is a pre-existing agenda, in general, the terrain of the conversation is not know inadvance. It is a process of exchange, of the sharing of ideas. It is a cybernetic-like product of thewhole, a product that is unique to the journey they share during their discourse.This process of interaction is extremely dynamic, with each of the parties constantly monitoring theresponses of the other and using their interpretation of the other parties input to make alterations totheir own response strategy, picking up points of personal interest, expanding points of commoninterest, and negating points of contention.This kind of interchange is much more in line with the kind of public/artist/system relationship thatbecomes apparent when an interactive system is exhibited in a public space as an interactive,responsive sound installation or immersive environment. In this scenario, the systems creator(s) cannot  Interactivity, where to from here? Dr Garth Paine   expect those engaging with the system to have knowledge of formal musical paradigms. Furthermore,when the input to the interactive system is a human gesture, it is questionable whether a musicalconstruct, constrained by the precedents of historical musical practice (chromatic music for instance), isan appropriate response. The appropriateness of response, that is a perceivable relationship betweenthe gestural input and the system output, is a central issue in the design of interactive systems. Thisrelationship should express the unique characteristics of each person’s engagement and commitment tothe instrument/art work/system, and is, I suggest, best represented by a system based on streamed datatechniques rather than triggered, pre-defined events. The mapping of sensed input data to processingalgorithms is the most complex and subjective aspect of system design.The mappings must be such that there is extensive scope for exploration and the discovery of newoutcomes, but where the outcomes prove repeatable to the extent that they confirm the cognitive mapthat the interactor is developing as their relationship with the interactive system deepens. The mappingshould reflect the nuance of the individual’s engagement with the piece, and must therefore be made inrealtime, an issue discussed in more detail later in this paper.I propose the public exhibition of interactive, responsive sound installations and environments, is agood platform for the investigation of mappings that may be inherent to the process of interaction. Of course the interface design dictates the nature and the scope of all interaction to some extent, but publicexhibition exposes the work to an untrained and inquisitive audience, who are prepared to invest timein the development of a relationship with the interactive system. They have no prior knowledge of therules of engagement, and therefore set out to develop a cognitive map of possible relationships with thesystem, a map that deepens over time.In this situation, the designer must consider why, for instance, would one wish to convert themovement of an arm to a musical chord, which in turn embeds itself within a chromatic musicstructure, but does not necessarily address the weight and subtleties of the interactive gesture? Whatrole or relevance does a sense of tonality have to the experience of interaction? Equally one has tocontemplate the role of other historically ingrained aspects of musical composition, rhythm, melody,musical form and structure, which so clearly constrain the Rowe and Winkler definitions.In order to address these issues, the interactive input must closely represent natural physical activities.Video sensing allows continuous tracking of human movement, and provides a continuous stream of data that represents a qualitative indication of the movement currently taking place. The followingqualities can be derived from that data; • direction of movement, • speed of movement, • size of a moving object, • proximity of movement to other moving objects, • inertia, • consistency of movement, and the • cartesian co-ordinates for both the momentary position, that is per video frame, and the position of rest.Whilst direction and speed of movement may be applied to existing musical parameters, they makemuch more sense when equated to the characteristics of naturally occurring sounds. A consideration of the concepts of dynamic morphology (a conceptual model developed by Trevor Wishart (1996) fordescribing all sounds, through the change of timbre, pitch and time in a 3D space) and spectralmorphology (an analytical model developed by Dennis Smalley (1986) for analysing how spectrainfluence timbre) as an alternative approach to system output becomes relevant.If interactivity is predicated on the ability of both parties to change in a way that reflects the developingrelationship or discourse between them (as discussed above by way of the racing car computer game),we have to accept that multimedia systems that do not evolve their behaviour in relation toaccumulated patterns of input (as described in the human conversation model above), are therefore notinteractive, but simply responsive, a pattern of engagement that has been prevalent since the inventionof the Theremin.In order for the system to represent an interaction, it must be capable of changing and evolving. Theprocess of evolution must promise continually new outcomes that are based upon the nature of a
Advertisement
Related Search
We Need Your Support
Thank you for visiting our website and your interest in our free products and services. We are nonprofit website to share and download documents. To the running of this website, we need your help to support us.

Thanks to everyone for your continued support.

No, Thanks