Sound studies is still tuning in

‘If you got ears, you got to listen’ sang Captain Beefheart in 1980. His statement alludes to the immersive nature of sound, and its taken-for-grantedness, which may explain the relative lack of studies of sound in culture and the media. The emerging discipline of sound studies struggles to deal with the ephemeral acoustic culture in the same systematic way that humanists and sociologists have for a long time dealt with the cultures of books, paintings, films, and so on. It is not an easy task, and it requires the development of a concerted tradition, with common methods, topics and theoretical foundations. Indeed, many publications that can be labelled ‘sound studies’ refer to the same theoretical and historical traditions, with Rick Altman and Michel Chion as prominent theoreticians, and James Lastra (film), Susan Douglas (radio) and Michael Chanan (music recording) as prominent historians. As I am suggesting, there is a need for more basic knowledge, and there are ontological questions and historical developments that sound studies must work with for decades. Two new books have been published in this essentially Anglo-American academic field; they both refer to the suggested canon and can reasonably be considered as works in sound studies.

Karen Collins’ Game Sound is a practical introduction to the history and production of sound in the computer games industry since the 1970s. A book about this topic is sorely needed in sound studies. Karen Collins is a researcher at the University of Waterloo in Canada, where she teaches game design and sound, and also designs software for interactive audio applications. She has edited From Pac-man to Pop Music: Interactive Audio in Games and New Media (Collins, 2008).

Frances Dyson’s Sounding New Media is a complex theoretical monograph that discusses conceptions of sound in art and avant-garde culture from the beginning of the 20th century to the present day. Frances Dyson is an Associate Professor at the University of California at Davis, where she teaches film, new media, audio art and technocultural theory. She has exhibited installations and performance works in the USA, Canada and Japan, and is a contributor to the Australian Broadcasting Corporation’s audio arts program ‘The Listening Room’.

Reading these books, I was curious about whether they would contribute to a concerted build-up of the sound studies tradition. Both contain great arguments about the function of sound in the media. Dyson makes a good point when she says that with new media we are navigators instead of listeners or viewers. She explains that ‘the interactive compo- nent of VR allows the virtual scene to change as the head moves’ and ‘the data glove allows the user to point in the direction the user wants to go, which then changes the scene to simulate movement toward that place’ (p. 14). This type of equipment introduces a very important element of individual control absent from traditional mass media listening practices. In 2010 it is appropriate also to mention smart phones with GPS, accelerometers and gyros, as well as Wii-consoles with games based on visual interactivity (dancing, fighting with swords, etc.). Collins contributes to the clarification of this practice by pointing to the possibility for unique sounds being triggered by the player of a computer game. Although the ensuing sounds will have been determined in advance by the producers’ instructions, they have never been heard before the player triggers them; this kind of novel sound experience at the very least puts into perspective traditional forms of communication. Also, there are sounds that inform a net surfer of the presence of an advertisement or alert a game player to danger or opportunity in a game like Doom (p. 130). These are also experienced in a setting where minute movements by the user influence the types of sound heard. They are the result of a combination of advanced audio-computing and proprioceptive perception; the authors who wrote about film, radio, music and modern art in the 1960s and 1970s naturally had little to say about them. Participatory features of sound communication must surely become one of the core topics in the future, and they are discussed in insightful ways in both books.

However, when it comes to the value of these two books for sound studies, I have come to the conclusion that they do not have the pedagogical energy that the emerging tradition needs. They seem to demonstrate what the sociologist C. Wright Mills (2000: 66) lamented as ‘the blindness of empirical data without theory and the emptiness of theory without data’. My impression is that Collins’ book is so concrete about computer schematics and musical scores that it sometimes loses direction, while Dyson’s book is so lofty in its conceptualization of the metaphors of auditory culture that it sometimes loses real-world relevance.

My approach to reading these books can be called realistic. Its rhetorical purpose is to present the world as directly as possible, with little verbal flourish and a limited amount of technical detail. Instead of philosophical or technical jargon, I want straightforward and hands-on answers to what characterizes communication in sound, especially its production and listening practices. An important ingredient in realistic approaches is systematic empirical histories of technology and experience of sound media in different eras and countries.

Both Collins and Dyson are inspired by structuralist and post-structuralist thought, and therefore I find it necessary to point out that the structuration of communication by language has a restricted role in my approach. Indeed, I believe that communication in sound has four dimensions that are of equal importance: time (live, recorded), space (enclosed, outdoors, etc.), person (formal, informal, etc.) and language (narrative, genre, content). These dimensions will be configured in different material ways through history, and sound studies must catalogue and describe them in much greater detail than what has been accomplished so far. This position is established in direct opposition to forms of cultural study that consider language, narrative structures and genres to be primary in that it considers the historical development of sound materiality and auditory perception to be primary. My contribution can be found in the book Sound Media (Nyre, 2008). I am unable to take up any other position in reading the two books, and it influences my judgement of their value for sound studies.

As mentioned previously, Game Sound is a production-oriented presentation of sound in the games industry. I will first deal with her historical narrative and afterwards discuss her ontology. The book provides a solid description of the technology and aesthetics of games, and its focus on American history and development is systematic and informative. It contains a range of technical figures and musical scores; because it uses industry language, it resembles traditional handbooks in radio, TV or film production. The target group for this book would seem to be undergraduates who study programming, new media, and interactive design. Game Sound has a solid historical grounding. It deals with the history of video games in three chapters: the sound of arcade games and home consoles, the emergence of big industry players like Nintendo, Sega and Amiga, and the mature audio for home consoles from the 1990s and onward. After these historical chapters follow four chapters about game audio today, dealing with the audio production process, the use of pop music in games, the interactive function of audio in gameplay, and compositional approaches to game audio. The whole book is based on empirical evidence, and is as such highly valuable for sound studies.

However, Collins’ approach is not truly introductory. It soon becomes evident that readers will learn much more from this book if they already know something about computer programming and can read musical scores of some complexity. This combination of skills is not typical of students or scholars, and although her requirements increase the sensitivity of the descriptions, they also restrict the likely readership of the book.

Game Sound does not contain an ontology. Although Collins has provided a careful and well-reasoned theoretical discussion of narratives, genres and the complexity of diegetic and non-diegetic sounds (p. 125), she does not have a distinct position on the question of what sound communication really is. She says that she will draw on James Lastra’s distinction between devices, discourses, practices and institutions in the analysis of sound media (p. 6), but she does not go through with this in a systematic way, and none of the four concepts are listed in the index (while Nintendo has 27 listings). The aforementioned lack of theory is found in the questions that Collins does not ask. There is no overall position, and she presumes that the workings of auditory culture are obvious and need not be spelled out in clear text. A book for the humanities and creative arts must deal with grander, overarching questions, especially since the tradition is so immature.

Above, I mentioned some of the ontological and historical questions that sound studies should always address. What characterizes communication in sound? What are its production practices and its listening practices? Let me continue to present my position. The phenomenologist Maurice Merleau-Ponty argues that perception of the world is primary, and that the senses of sight, hearing, touch, smell and taste work as a system, with bodily movement and engagement being its vehicle. I agree with this description, most strikingly formulated in Phenomenology of Perception (Merleau-Ponty, 1962), not least because it fits well with the realistic descriptions of sound perception as immersive and mood-driven in Don Ihde’s Listening and Voice (1976) and Walter Ong’s Orality and Literacy (1982). Even more interestingly, many of Marshall McLuhan’s eclectic state- ments in Understanding Media (1964) and The Medium is the Massage (McLuhan and Fiore, 1967) fit well with Merleau-Ponty’s approach, especially the claim that electronic media are primarily an acoustic space or a media environment in which we all live. In the 1920s, the acoustic media environment was mechanically driven with its attendant limitations. In the 1960s it was electrically driven. Since the 1990s it has been digitally driven, again with its attendant limitations. This combination of phenomenology and technology theory is well suited to the study of sound in the mass media, but it is not taken up by Dyson, although she discusses many more theoretical issues than Collins.

A blurb on the jacket of Sounding New Media tells us that the book examines the ‘long-neglected role of sound and audio in the development of new media theory and practice’. But Dyson explains that she will not write about the realistic dimension of media, and indeed she writes little about the material history of different sound media, the production and listening practices, and the differences from other media like film, television and the internet. Note that this is a highly conscious choice on her part. She has written a precise introduction to the material dimension which shows that she under- stands it well (pp. 10–11). Instead, she promises to deal with ‘the terms, metaphors, and rhetorical manoeuvres that outline the discursive fields of sound and new media’ (p. 6). For example, she discusses ‘voice’ (chapter 1), ‘vibration’ (chapter 3), ‘silence,’ ‘noise,’ (chapter 4) and ‘atmosphere’ (chapter 7). Hereby she signals her allegiance to what I consider to be a postmodernist, relativist approach.

Dyson states that she will contribute to an ‘aural ontology in our visually oriented technoculture’ (p. 7). There is no doubt that she writes an ontology that deals with sound communication, but it is not an ontology that clarifies the unique contribution of sound in modern communication characterized by new media like the internet, the mobile phone and computer games. Rather, it is an almost mystical ontology of mediation in general terms. Along with her tendency towards obscurantism is the fact that she does not contribute to the empirical history of technology and experience in the field of sound media. In the end, it turns out that her seeming focus on new media is also a focus on media in general. She uses ‘new media’ as a relative term which includes the telegraph and telephone as well as the internet, and they are only defined as ‘new’ in relation to what came before. In this regard the title of the book is misleading.

As suggested, Dyson’s interest in historical narrative is less evident than her interest in ‘aural ontology’. The historical grounding of her argument comes across as sporadic instead of systematic. Indeed, there is a suggested timeline in that she discusses the telephone before the wireless radio and the tape recorder before virtual reality; she also links each of these technologies with artists writing at approximately the time of the introduction of these sound media. But there is no proper historical narrative, and I don’t think Dyson wants there to be one either. She wants to discuss the ideas and visions of classical writers such as Marinetti, Artaud, Arnheim, Cage, Schaeffer and Schafer, all of whom created works of art, music or journalism, and who also reflected on media sound, and wrote profusely about it too. But there is a need for greater precision in her narrative. For example, the discussions of Cage are not put into the American cultural context of his active period, and the very different cultural context of Antonin Artaud is not dealt with either. Dyson has chosen to discuss authors she agrees with, and only sparingly offer words of critique as when she uses Jonathan Sterne to critique R. Murray Schafer as a sonic essentialist (p. 74).

In laying out her aural ontology, Dyson fills her prose with references to the theorists and artists that I have mentioned above, and also many others from modernism and onwards. She does not explain her aural ontology in a pedagogical way. It doesn’t help that many of the writers are well known for their jargon, like Martin Heidegger and Jacques Derrida. The discussions of Heidegger’s Dasein (p. 88) and Derrida’s différance (p. 99) are so difficult to understand for non-philosophers that they don’t really contribute anything to the understanding of the world of sound. For example, Dyson states that Heidegger freed the cultural association of music from emotion, and aligned it instead with ‘the spirito-acoustic trope of vibration – which is the essence of being tuned’ (p. 88). If I were in the mood to read Dyson as she wants to be read, I might see the interesting thing about spirito-acoustic tropes, but the emerging tradition of sound studies needs clarity, and it doesn’t help much to be told that sound communication is a process of being tuned into vibrations, and achieving a certain Stimmung. Although I acknowledge the academic virtue of carrying on the discussion in this classical tradition, I dispute its fruitfulness for the discipline of sound studies. At the very least these interpretations would have had to be a foundational for the book, so that they could be unwrapped and put into a historical context chapter by chapter and presumably be corroborated as fruitful perspectives in sound studies. Dyson’s abstract form of argumentation limits the likely readership for this book to PhD students and professors who can see the implications of Heidegger, Derrida and the other philosophers in a larger perspective, based on substantial previous knowledge.

Another example of her non-realistic knowledge interest is the way she deals with the term (and experience of) ‘immersion’. Dyson discusses immersion as an ‘all-embracing, all enveloping condition’ and ‘the desire to elide the distance that technological mediation imposes – between the user and the apparatus, the real and the simulated, the natural and the artificial, the human and the technological’ (pp. 6–7). She wants the term to conceal ‘the social and technological interventions and delimitations that go into producing digital media’ (p. 7). In chapter 5, which is simply called ‘Immersion’, she makes much out of Char Davies’s interactive artwork Osmose, which creates a virtual reality experience for its users by applying 3D sound, head-mounted display and real-time motion tracking based on breathing and balance. In a detailed analysis Dyson argues that this artwork is a critique of ocularcentrism and subversive of the dominant visual emphasis of VR (p. 132); she argues that it contains an ethics of stillness and an aesthetic of ephemerality (p. 129). In the end it becomes highly unclear what Dyson means by the term immersion.

As already suggested, I find Dyson’s discursive approach relativist and quite fanciful, and I believe her discussions of immersion are not fruitful for sound studies at large. Karen Collins is much more down-to-earth in this regard. She describes immersion by reference to IMAX movies and Nintendo’s Wii, where the producers use all technological means at hand to construct a ‘suspension of disbelief’ (p. 134). Collins stresses that the most common rhetorical objective is to create realism, whether it is in a first-person shooter game or a sports game. The immersive experience can be created with subwoofers and surround sound, with 360 degree imaging, and with bodily movements creating responses in the controlled environment. In Collins’ approach it is clear that immersion is a specific technologically constructed experience, while Dyson makes so much more of it that the clarity vanishes. Collins helps the reader to understand that immersion is something different than non-immersive experiences like writing and reading on paper, while Dyson’s argument (see above) can be interpreted to mean that all kinds of media experience are equally immersive.

In this review article I am defending the interests of a young tradition that is still struggling to define itself. There is no doubt that both Collins and Dyson contribute substantially to the build-up of this tradition, the former by presenting game sound in great technical detail and the latter by continuing the philosophical discussion of media sound, but they both have flaws as I have pointed out. Sound studies needs books that have the right balance between ontological theory and empirical detail, between dialogue with authoritative thinkers and original historical narratives. More than anything, the discipline needs exemplary analyses of actual sound, in the same way that literary studies always needs more analyses of poems, plays and novels, and film studies needs analyses of new films.

My final complaint is that neither Collins nor Dyson illustrate their many facts and formulations with real sound. In the present decade it is not difficult to include sound examples in a book project. They can be appended to the printed book in a CD or made available on a website as mp3 files. Maybe they have thought about it and chose to make the arguments maximally conceptual (Dyson), or decided it would be too time-consuming and expensive to deal with rights holders (Collins). In material and experiential terms many things will be missing in books unaccompanied by CDs, especially the aesthetic talents of artists and producers, the acoustics of well-placed microphones and sophisticated editing, and for the reader of the book the strange and stimulating experience of shifting between being a reader of words and a listener to sounds. By not including sound they leave out the core evidence, and the discussions of sound get more abstract and less tied to reality than the emerging discipline needs.

References:

Collins Karen (2008) From Pac-man to Pop Music: Interactive Audio in Games and New Media. Aldershot: Ashgate.
Ihde Don (1976) Listening and Voice: A Phenomenology of Sound. Athens: Ohio University Press.
McLuhan Marshall (1964) Understanding Media: The Extensions of Man. New York: McGraw-Hill.
McLuhan Marshall and Fiore Quintin (1967) The Medium is the Massage. New York: Bantam Books.
Merleau-Ponty Maurice (1962) Phenomenology of Perception. New York: Humanities Press.
Mills C.W. (2000) The Sociological Imagination (40th anniversary edition). Oxford: Oxford University Press.
Nyre Lars (2008) Sound Media: From Live Journalism to Music Recording. London: Routledge.
Ong Walter (1982) Orality and Literacy: The Technologizing of the Word. London: Methuen.