Computers have revolutionized the composition, production and recording of music. However, they have not yet revolutionized music education. While a great deal of educational software exists, it mostly follows traditional teaching paradigms, offering ear training, flash cards and the like. Meanwhile, nearly all popular music is produced in part or in whole with software, yet electronic music producers typically have little to no formal training with their tools. Somewhere between the ad-hoc learning methods of pop and dance producers and traditional music pedagogy lies a rich untapped vein of potential.
This paper will explore the problem of how software can best be designed to help novice musicians access their own musical imagination with a minimum of frustration. I will examine a variety of design paradigms and case studies. I will hope to discover software interface designs that present music in a visually intuitive way, that are discoverable, and that promote flow.
Interaction Design Principles
Any user interface, musical or otherwise, must operate within the limits of our ability to draw metaphorical connections between visual images and abstract concepts. A software interface is not a neutral intermediary between the user and the work. It guides and shapes the final product, especially when the user is inexperienced. Software interfaces do a great deal of implicit teaching, and indeed may be the only instructor that some musicians ever encounter. A given interface’s constraints can be obstacles to creativity, but sometimes constraints can be drivers of creativity in their own right.
Visualizing Music
Computer-based music production involves the eyes as much as the ears. The representations in audio editors like Pro Tools and Ableton Live are purely informational, waveforms and grids and linear graphs. Some visualization systems are purely decorative, like the psychedelic semi-random graphics produced by iTunes. Any investigation into digital music pedagogy must begin with an examination of music visualization systems.
Notation is the oldest (known) method of visualizing music. Western notation is a very neat and compact system, unambiguous and digital. It can be effortlessly converted to and from MIDI data. But western notation has some limitations, especially for contemporary music. The key signature system works well for diatonic major and minor keys, but is less helpful for modal music and is fairly worthless for the blues. After one hundred years of jazz, there is no good way to notate swing other than to write the word “swing” at the top of the score. The further an instrument departs from the piano, the more difficult it is to map it to a notation system. How would one notate the music of Eric B and Rakim, or Aphex Twin, or Girl Talk?
The MIDI “piano roll” has a mostly straightforward one-to-one mapping to western notation. MIDI therefore has some of the limitations mentioned above. However, it offers several advantages. The piano roll shows not only which notes are being played and when, but exactly how long they’re held. Note velocity is indicated via color or vertical lines. Pitch bend data can convey microtonal nuance. Rhythmic nuance can be conveyed through the precise positioning of events against the grid.
As software becomes more sophisticated in its ability to extract pitch data from audio recordings, we can begin to manipulate the audio with the same ease as MIDI. The pitch-correction program Melodyne enables MIDI-style editing of polyphonic audio recordings, both in piano roll view and in traditional notation. The actual pitch contour is shown as a series of lines, and the abstracted “score” is a corresponding series of orange waveform blobs. The blobs’ thickness shows dynamics.
Notation conventionally shows time moving from left to right. But this is not necessarily the most intuitive way to visualize time. Rhythm games like Dance Dance Revolution and Guitar Hero show time as a track projecting into the screen, along which the player moves forward, as if on a train. The pioneering game FreQuency shows the song as a polyhedral tube, with each face of the tube standing for a different instrumental track.
Which visualization scheme is the best? To answer that, we must examine the nature of metaphors and how we make sense of them.
Bodily Image Schemas in Music
In their book Metaphors We Live By, George Lakoff and Mark Johnson observe that we frame metaphors in terms of states of our bodies. Indeed, body states are the only basis for abstract thought that we possess. The closer a metaphor is to a state of the body, the easier it is for us to understand. Metaphors that are several layers of abstraction removed from bodily states will be difficult to learn and remember. A good software interface must hew closely to our need for body metaphors cognition.
Music has a broad set of commonly used bodily metaphors. The most basic bodily experiences we relate to music are rhythm and repetition. We experience these phenomena throughout our inner experience, from our heartbeats and respiration to our gait. We also use a variety of spatial metaphors for music, referred to by Wilkie et al as image schemas. Such musical image schemas include containers, cycles, verticality, balance, the notion of center-periphery, and (in the case of western melodies) a narrative of source-path-goal.
When we conceive melodies, we think of the line moving over a metaphorical landscape, with altitude standing in for pitch. We might say, “The melody starts on F, goes up to Bb, down to A, and then lands back on F.” The pitch-as-height metaphor is muddied by the circularity of pitch class, and by the fact that we feel ascending pitch movement differently from ascending. We may use alternative image schemas; it is common to visualize higher pitches as brighter, and lower pitches as darker.
We are on more consistent metaphorical footing with the notion of the tonic as “home base.” We imagine a piece that modulates through different keys as going out on a journey and then returning home. Both a piece of music and a physical journey are events ordered in time, so the metaphorical connection here is effortless.
Our metaphors acquire an additional layer of abstraction when we conceive of music not as a sequence of events, but a nested collection of objects. When we say that a piece of music is in the key of F minor, we imagine F minor as a container from which the music draws notes, like Legos from a box. We think of chords as vertical stacks of notes, again evoking Legos.
We approach software equipped with our familiar bodily image schemas, learned and innate. The highest praise we can give to an interface is that it is “intuitive,” meaning that we can apply prior knowledge and familiar image schemata: innate, sensory-motor, embodied, cultural, or expert. Wilkie et al have gathered a comprehensive list of familiar bodily metaphors for music. These include:
- A piece of music is an object.
- A whole piece of music is constructed from a number of parts.
- Harmonic and melodic progression is movement along a path from a source to a goal.
- Musical repetition is a pattern.
- Keys and chords are containers for notes.
- Unexpected musical change is a diversion, a change in spatial direction.
- High pitch is up; low pitch is down.
- Intervals have sizes or lengths.
- A rest or silence is an object.
- Musical silence is a blockage to motion.
- Key/chords are related to the tonic in the familial sense.
An interface that employs these metaphors will be easier to learn, and an interface that does not will resist learning. For example, on the piano keyboard, pitches are arranged left-right rather than high-low. I have played basic piano for twenty years and still have difficulty immediately recalling whether the higher pitches are on the right or left. Similarly, beginning guitarists struggle with the fact that the lower-pitched strings are physically located above the higher ones. Software designers, unconstrained by ergonomic considerations, can and should hew closer to intuition.
Tool as Participant
Music has always been shaped in part by the tools used to make it. Music written for piano will be more concerned with dynamics than music for harpsichord. The active role played by tools has never been more critical than in the digital realm. The presets and defaults in a popular piece of software will leave indelible stamps on the music they produce. When Propellerheads’ Reason attained widespread popularity a few years ago, its default drum loops became ubiquitous in television commercials. The most dedicated musicians will find ways to produce the sounds in their heads regardless of the tools. But most musicians are not that dedicated, especially the beginners. For the casual musician, a tool’s affordances will have a strong impact on their work.
Electronic musicians tend to begin their work by playfully experimenting with a piece of equipment or software, a period of open-ended “knob-twiddling.” The discoveries made during this period, particularly those not intended by the musician or the software’s designers, are crucial raw materials for the more formal composition and editing that follows. One subject interviewed by Gelineck and Serafin described his tools as “having a life of their own.” In general, there is a feeling that the electronic musician’s tools are active, even intelligent participants.
A central attraction of packages like Logic or Ableton Live is their large libraries of music samples, instruments and effects presets. These prefabricated building blocks give qualms to some music educators. Why learn “creativity” when the student can simply throw loops together from a folder and call it a day? If a track is primarily built from presets, can its creator really be said to have created anything? Should the creators of the presets be considered the “real” composers?
The question of electronic music authorship becomes even more vexed when the software generates ideas ex nihilo. Many MIDI sequencers offer the ability to generate notes semi-randomly within particular scales or chords. Ableton Live has an effect called Beat Repeat that semi-randomly shuffles pieces of audio, sometimes with startlingly musical results. Beat Repeat’s output can be strongly reminiscent of classical embellishments or jazz improvisation. If I apply Beat Repeat in a track, should I consider the computer a co-composer? Or the programmers who created Beat Repeat?
Music software steers its user toward the sounds that can be produced most expediently. The availability of features and the ease with which they can be learned and accessed convey implicit lessons to the inexperienced user. In this sense, all music software is educational. What lessons does it teach? Are they the right ones? Will the software-as-teacher guide and inspire students or stultify and limit them?
Constraints
Audio programming environments like SuperCollider and Max/MSP offer the skilled musician virtually unlimited sonic freedom. Rather than liberating creativity, however, these tools can just as easily stifle it under the weight of option paralysis. Music made with such tools frequently never makes it past the experimentation stage into fully-realized works. Performances can all too easily take the form of technical demos.
Simple, limited interfaces have two major virtues. A small feature set can be learned quickly, and its most obvious uses will quickly become tiresome. Magnusson speaks approvingly of interfaces that “proscribe complexity in favor of a clear, explicit space of gestural trajectories and musical scope.” If presented with a finite feature set, users are encouraged to move quickly past the knob-twiddling stage and into finding musically expressive uses for the tool.
How do interface designers balance the need for complexity and surprise with the need for constraint and simplicity? The following section addresses a variety of approaches to meeting this challenge.
Pedagogical Strategies For Music Technology
Beginner musicians face a frustrating situation. They can hear the desired sounds in their heads, but it is many months or years before they can produce those sounds with their instruments. Many would-be musicians are unable or unwilling to persist through the tedium of rote practice to reach the stage where expression and pleasure begin. The computer has a lower barrier to entry than most acoustic instruments. Producing music that sounds good is practically effortless. For that reason, software can be an invaluable motivator, bridging the gap between conception and execution. Rather than supplanting traditional musicianship, software can motivate beginners who would otherwise give up in the early stages of learning.
Computers have primarily been in music education as a more convenient delivery system for traditional pedagogy: reading materials, drills, and multimedia storage and playback. Treating the computer as a valid expressive tool in its own right, worthy of a central role in the curriculum, will take some adjustment among educators. The computer is fundamentally unlike other music-making tools — producing flawlessly executed performances and infinite loops is effortless, while expressiveness, improvisation and idiosyncrasy are challenging. The following sections list some sources of inspiration for successful use of software for music education.
Learning from Repetition
Western classical tradition takes the linear narrative as its defining metaphor. Electronic dance and pop music are based on a very different basic image: the endless loop. Copy and paste is the defining gesture of digital editing tools, and infinitely looping playback their signature sound.
The cyclic nature of pop, dance and hip-hop music are obvious (and much lamented by “sophisticated” musicians.) Upon examination, however, this loop-centric bias appears throughout contemporary art music as well, with African-American dance music as the fundamental cultural transmission vector. Susan McClary observes:
The proliferation of [cyclic repetition] across genres has not been noticed very much, in large part because they do not share the same audiences. The devoted fans of Goldie or Missy “Misdemeanor” Elliot don’t usually attend Steve Reich concerts, nor do many of the symphony subscribers who admire the works of John Adams involve themselves in the dance-club scene or participate in raves… Yet the genres often sound astonishingly similar, especially in their ways of structuring time… Given its ubiquity, black pop music would seem to be the element most clearly responsible for converting our collective sense of time from tortured heroic narratives to cycles of kinetic pleasure. As Prince sings, “There’s joy in repetition!”
Software is ideally suited to producing endlessly repeated loops. Creating a two-bar loop using a computer is significantly easier than building a long, linear, narratively structured melody. This fact is of great benefit to the music educator, because the loop is an excellent teaching tool. There is no better way to deeply internalize a musical concept than to hear it repeated on a loop several hundred times.
Repetition is fundamental to all forms of human learning. Neuroscientists use the word “rehearsal” to describe the mental repetition of information as we transfer it from short-term into long-term memory, where repeated firings of certain networks of neurons strengthen their connections until a persistent pattern is firmly wired in place. The connection between this sense of rehearsal and the musical sense is not at all coincidental.
Loops can, of course, be any length. A loop might encompass a few notes, a few measures, a few phrases or an entire piece. The key to effective music learning is “chunking,” breaking a long piece into short, tractable segments and then building those segments into larger meta-segments. The length of looped chunks can be customized based the level of the students — beginners will work with looped single measures or phrases, while more advanced students can loop through passages and sections. Chunking helps get students to musical-sounding results quickly. Rather than struggling painfully and haltingly through a long passage, students can attain mastery over a shorter one, which builds confidence. Furthermore, chunking can help make feedback more immediate and thus more effective.
Saville cites the music educator’s truism that “accurate feedback may be the single greatest variable for improving learning.” The longer the delay between the performance and the feedback, the less effective it is. It is best to give feedback in the moment, while the student plays along with the loop, “in the heat of battle.” The loop can continue to run indefinitely, so students need not lose the flow when they drop a note or receive feedback.
Dance music tools like Ableton Live make it easy to isolate sections of recordings, loop them, and alter their tempo or key independently. Melodies can also be entered via MIDI, though these will not have the organic feel of a recording of real musicians. Still, MIDI has the advantage of nearly infinite flexibility. Also, any public domain work can be found in MIDI format on the internet with a simple Google search. MIDI is also highly amenable to copy-and-paste methods of composition. In their study of teachers’ use of music technology in the classroom, Stuart Wise, Janinka Greenwood and Niki Davis describe a class that used Sibelius to teach western notation. Students quickly discovered its potential as a MIDI sequencer, rather than just a way to prepare scores for eventual human performance. One young student created a series of complex arpeggios by means of simple copying and pasting, a compositional idea that almost certainly would not have come to fruition via pencil on paper.
In an ideal world, the loop would completely replace the metronome for music practice. The deadening tick-tock of the metronome demoralizes students, and is difficult to relate to a lively beat. Practicing over an actual nuanced groove offers a vastly richer and more engaging practice experience. Furthermore, simply listening to the loop itself can create a gratifyingly trancelike, meditative feeling, as fans of repetitive electronic dance music will attest. This meditative state is highly conducive to flow, and it can turn repetitive drilling into a pleasurable act.
Learning from the Studio
Outside of the classical world, the canonical version of a piece of music is a recording rather than a score or performance. The interpretation on a well-known recording can become coextensive with the work itself. This is especially true in improvised forms like jazz, and in aural traditions like country and blues. In the current musical culture, we experience live music through the lens of intense familiarity with the canonical recordings. Given the primacy of the recording, it is an uphill battle to get music students to engage emotionally with notes on a page. The score becomes much more tractable and approachable after a close study of a recording. As described above, the ability to effortlessly loop a recorded passage in any key at any tempo supplements the score with much-needed aural context. Scores cannot represent timbres, sonic effects or performative nuance the way recordings do. Students of jazz or country or blues will not have much in the way of scores to begin with, making the digital manipulation of recordings that much more valuable.
Thus far, we have considered only recordings of live performances. But recordings need not be a document of a contiguous series of physical events. Tape editing made it commonplace to spice together ideal performances from many takes. Audio editing tools make such splicing trivially easy. More significantly, they also make it possible to build recordings without an actual “performance” ever taking place. The tradition of concert realism that prevailed in the first half of the 20th century has been supplanted by hyperrealism or outright surrealism. We no longer expect a recording to relate back to a group of people in a room playing instruments at the same time. Producers and recordists have become musicians in their own right, often acting as the central or sole creative figures in the making of recordings.
While theorists from Walter Benjamin onwards have lamented the alienation brought about by our recorded music culture, Mark Katz gives us reason to be more sanguine. An individual with a laptop in a bedroom can create fully formed musical works that a few decades ago would have required a small army of musicians and engineers. If I download an mp3 posted by such a bedroom producer on the internet, I have an almost perfectly unmediated view into the artists’ creative intent. Such immediacy and intimacy would have been a rare privilege indeed not so long ago.
As the price of hardware and software plummet, schools are increasingly able to build full-featured digital recording studios. Matthew Thibeault urges music teachers to think of the studio not just as a collection of gear that can be used to document the “real” performance, but as a musical instrument in its own right, carrying with it an entire philosophy of music-making. Educators should recognize that the studio is the locus for most of the musical creativity of the past several decades, and that composition, recording and editing have collapsed into a single act. Even “realistic” performances are usually highly digitally processed. For pop songs, the distinction between songwriting, performance, recording, editing and mixing is blurring to the point of meaninglessness. To meaningfully participate in the musical world, students must become familiar with the studio’s particular demands and affordances.
The skills needed in the studio are quite different from the skills needed on the stage. Digital audio editing diminishes the need for flawless virtuoso performances. It increases the need for creative sequencers and editors, who can work together with performers at any skill level to produce a musical-sounding result. A casually tossed-off improvisation or playful throwing together of sounds can produce better music than hours of sweating over a score. The performer need not even be aware of or involved in the creation of the musical idea; it is common to create a recording using sampled and otherwise decontextualized performances. Such recordings can be more affecting than if the performer had deliberately delivered the finished product.
Thibeault describes his own exposure to the studio as instrument:
I have now recorded myself harmonizing with my voice, recorded multiple takes on different instruments to overlap them, as Stevie Wonder did with his classic albums of the 1970s. For music educators who wish to create a recording studio, it’s worth remembering that the studio can create these pathways, taking students down roads that concerts never can. As students learn to use the studio as an instrument, it’s also possible for them to dream in new ways, imagining music that they would not be able to imagine without these different pathways in.
The studio expands the definition of the word “musician” beyond traditional performers and composers to include anyone with the patience and the will to learn the software and explore its possibilities. Brian Eno, one of the most prominent and influential musicians in the world, is at best a rudimentary instrumentalist and singer; he earned his place in the history of music with his virtuoso “playing the studio.”
Music education should be serving would-be Brian Enos, not just would-be Yo-Yo Mas and Wynton Marsalises. There are emotional aspects to studio work that are just as important for students to learn as the technical aspects. Live music demands flawless, error-free performance. The studio is highly tolerant of mistakes, and unintended sounds are frequently the most valuable. The studio releases musicians from the anxiety of an audience, but creates an entirely different anxiety in the face of the microphone and its clinical surroundings, and the seemingly mysterious workings of the software that is recording the sound. It would be unfair to prepare music students for one situation and not the other.
Learning from Dance Music Producers
Musicians who wish to learn hip-hop, techno and other electronic dance music (EDM) styles are left mostly to their own devices. While software tutorials are widely available, it is difficult to find relevant learning materials about the music itself. The diatonic harmony taught in introductory music theory classes sounds out of place in electronic dance music. The blues, modal and exotic harmonies that are idiomatic to EDM only become available to most music students at advanced levels. EDM producers are left to stumble across their desired chords and scales by trial and error. Dance musicians’ ad hoc learning strategies have nevertheless been proven remarkably effective, honed through decades of practical usage. Rather than trying to supplant such bottom-up learning, formal music instruction should incorporate its best practices.
Some music educators have begun to fill the vacuum. Michael Hewitt’s Music Theory for Computer Musicians is a fairly typical introductory-level theory text, but with an interesting twist: it presumes familiarity with production and sequencing software, and no familiarity with classical notation or terminology. For example, Hewitt explains standard Western notation in terms of the MIDI piano roll. It is fascinating to me to see MIDI treated as the baseline standard, and notation as an alien visualization scheme. As the use of production software spreads, students are likely to enter their first music class with some exposure to MIDI; meeting them on familiar territory will ease their musical growth.
Hewitt takes a similarly forward-looking approach to his text Composition for Computer Musicians. Again, the book presumes familiarity with Logic, Ableton Live, Reason and similar software. He introduces scales and modes that are useful to EDM, presenting the major scale as no more central or important than any of a number of exotic scales. He then presents harmony using the copy-and-paste paradigm comfortable for MIDI users. He begins by duplicating a melodic line and transposing it up a perfect fifth, observing approvingly that parallel fifths are idiomatic to the music of Indonesia. This kind of admirable decanonization extends a welcome mat to those students coming from non-western cultural traditions. Only later does Hewitt introduce full chords and “proper” western voice leading. While this approach would produce dreadful classical music, it gives students of EDM exactly the sounds they are seeking.
Another virtue of Hewitt’s approach is his implicit Afrocentricism. He addresses drum programming early and deeply, covering a variety of rhythmic styles in detail. He also moves quickly from the drum sounds’ placement in the bar to a discussion of shaping their timbre. He recognizes that musical content and sonic content are inseparable, and that timbre is as easily manipulated in software as any other musical parameter. Furthermore, he gives the impression that melody and harmony are decorations of the rhythm. This is a welcome corrective to western classical pedagogy’s neglect of the groove.
The significance of Hewitt’s approach extends far beyond hip-hop and techno producers. His method has promise for music education generally. Computer-literate students will find music production software much easier to learn than music itself, the same way that they find Microsoft Word easier to learn than good writing. Music educators can take advantage of production software to make learning composition and theory more meaningful, interactive and engaging for students working in any style. Software is in many ways a more attractive learning environment than the classroom. Feedback is instantaneous, the musical results of your choices are obvious, and your ears can be the judge of the validity of your ideas. Afrocentricism, multiculturalism and the copy-and-paste aesthetic are similarly valuable to any music student, not just those learning the conventions of EDM.
There is a strong analogy to be made between the role of computers in music education and math education. In both cases, the computer is much better at executing algorithms than people are. The focus of education should shift from executing algorithms to understanding how the computer executes them, and how to express them in computer-readable terms. Hewitt’s germane example is the arpeggiator function included in most sequencers. The computer can effortlessly spin out repetitive symmetrical note patterns; the student needs to understand the meaning and content of those patterns, not how to execute them manually. There is also the strong analogy between composition and computer programming. Both can be clearly visualized using flowcharts. Hewitt makes good use of the flowchart model in his discussion of song structure.
Many musicians and educators will be alarmed by Hewitt’s embrace of electronic music as a teaching tool. There is the widespread fear that relying on software will diminish interest in traditional instruments, and will eventually undermine the very notion of the performer. Schloss and Jaffe address this anxiety in their investigation of “intelligent musical instruments” that decouple the musicians’ gestures from the resulting sounds. While the authors recognize that electronic music poses a grave threat to instrumental virtuosity, they do not regard this as a negative development. They point out that the notion of the virtuoso individual performer and genius composer are specifically western constructions. Other world cultures see music as a collective activity belonging to everyone and no one. Schloss and Jaffe see intelligent musical instruments as opening up western musicians to a similar perspective.
The music world at large can learn further from the listening habits of sample-oriented EDM musicians. The DJs in Thompson’s study speak of listening at three different levels. There is the song level, used for selecting tracks in a club setting. There is the sample level, closer listening while searching for particularly exciting short phrases. And there is the sound level, the most granular of all, used while searching for a particular kick or snare drum sound. These three listening levels could be easily applied to jazz, classical or anything else.
The most important learning resource for the EDM community is the music itself. Jazz musicians have similarly benefitted from easy access to recordings; as the jazz aphorism goes, “All the answers are in your record collection.” A dance musician can move beyond studying recordings, and through sampling and remixing, actively engage with them. The multitrack stems for a variety of well-known pop songs from the Beatles through Kanye West are widely available on the world wide web. In an ideal world, these stems would be standard tools in any music curriculum, for remixing, transcribing or simple listening and enjoying.
User Interface Case Studies
There is no shortage of beginner-oriented, “approachable” music software. However, not all of it succeeds. The following section lists different interface approaches and evaluates their merits.
The keyboard as metaphor
The keyboard has dominated electronic music interfaces since the Telharmonium, and for good reason. It offers a straightforward and intuitive mapping of one key to one pitch. The idea of hitting the key harder to play the sound louder mirrors the familiar analog world. The MIDI standard has entrenched the keyboard metaphor even more strongly. The keyboard paradigm also extends into environments where there is no keyboard involved, for example in the scale entry interface in Auto-tune.
The problem with the keyboard, real or metaphorical, is its strict pitch quantization. The finite pitch set is gentle on beginners, perhaps, but it restricts expressiveness. The blue notes and microtonal nuances we have come to expect from a century of American vernacular and pop music are simply unavailable to the keyboard player. The pitch bend wheel on some keyboards is a step in the right direction, but it is a primitive affordance at best. A guitarist or violinist can effortlessly bend different notes within a chord by different amounts. Electronic music interfaces are sorely in need of a similarly nuanced fine pitch control.
The touchscreen offers some hope in this direction. The iOS app Nodebeat has an exceptionally expressive touch keyboard. Notes played in the center of the keys will sound standard pitches, but the player can also play the entire pitch continuum by dragging from one key region to the next.
Non-Keyboard Interfaces
Not every digital music interface uses keyboard input. Electronic instruments and MIDI controllers have been based on non-keyboard instruments like guitar, saxophone, violin, drums and even the accordion. Like the keyboard, these interfaces hew closely to analog reality in their parameters and mappings. It does not take much imagination to have your MIDI violin map fingerboard position to pitch. The ubiquitous Akai MPC sampler, mimicked in many subsequent devices, is essentially a set of small drums played with the fingertips. The relationship between hitting a sampler pad and hearing a sample played back is obvious, visceral and appealing to our intuition.
What if you want to leave the acoustic instrument paradigm entirely? You are suddenly confronted with a serious difficulty: you must determine the mappings from gesture to sound entirely from scratch. The past century has seen a lot of fascinating experiments in non-traditional control schemes, from the Theremin onward, but none of those devices has found widespread use. The hegemony of the keyboard (and other acoustic instrument metaphors) remains substantially unchallenged.
This is not for a lack of trying among experimental interface designers. They have tried motion sensors, touch-based controls, piezoelectric pickups attached to every conceivable object, and even, in Alvin Lucier’s case, direct readings of brainwaves. However inventive these interface schemes are, however, any musician wishing to learn such a scheme is faced with a heroic learning challenge. Before any expression is possible, the musician must understand the device’s idiosyncratic mappings between gesture and sound. Furthermore, the audience must go through this learning process as well; it is quite unsatisfying to watch a performance without being able to connect the performer’s actions to the sounds produced. Morton Subotnick laments that once the gestures in his stage pieces became excessively abstracted from the sounds they triggered, it would have made no difference if he had just played a tape of the desired sounds and had the performer mime along. It is no wonder that interface designers keep returning to drums and keyboards.
Interface designers remain undaunted. Currently, most of their creative energies are directed at various motion-capture systems, helped by the availability of the inexpensive Microsoft Kinect. But as interesting as motion capture is as a technological novelty, it is doubtful that it well ever displace the keyboard or sampler pad. The difficulty is the lack of haptic feedback. In a nutshell, motion capture systems don’t push you back. Without physical resistance, it requires extraordinarily disciplined muscle control to maintain accurate positioning. Motion capture works best in dance-oriented video games, where the player is imitating a specific set of movements. As a music generation method, however, motion is largely an unsolved problem.
Perhaps a motion capture standard will someday catch on the way MIDI did, and Kinect-like controllers will become as ubiquitous as keyboards. I am not optimistic. Motion capture defies bodily intuition. There is no instinctive causal relationship between empty-handed movement and sound generation. Usually it is the sound that inspires the movement. But perhaps I am excessively pessimistic, and that some future music technologist will prove me wrong.
Graphical Scores
The graphical score long predates software abstractions of music. Iannis Xenakis was one of the major innovators in the field. Though the music itself is difficult, Xenakis’ drawing of “Metastasis” is a beautiful work of art in its own right. Brian Eno’s “Music For Airports” includes an attractive graphical score on the back of the album cover. Since a DAW or MIDI sequencer display is effectively a form of graphical score, it seems logical that drawing and graphic design could become a direct form of musical expression. Thus far, however, there have been few successes.
The earliest graphic score concept to have been incorporated into a working electronic instrument was the ANS synthesizer, invented by Evgeny Murzin in 1938. The initials stand for Alexander Nikolayevich Scriabin, the mystical synaesthetic composer to whom Murzin dedicated his invention. Four decades later, the Fairlight CMI introduced a light pen that allowed the user to draw audio waveforms freehand directly onto a screen. These waveforms could then be played back as synthesizer tones from the keyboard. The system was clunky at best, but it did capture the imagination of musicians like Herbie Hancock and Quincy Jones.
The advent of the touchscreen has resulted in an explosion of graphical music interfaces. Ning and Zhou are the designers of a tabletop music creation system, The Music Pattern, a system similar to the Reactable, but with more of a drawing component. The user draws patterns that map to particular pitches and rhythms, which can then be arranged in rows to form melodies. The system translates patterns into a pitch and timing information, with real-time audio feedback to guide the user’s finger movements. Intriguing though this approach is, it ultimately suffered from the same difficulties as motion capture: the mapping between image and sound was necessarily arbitrary and had to be learned laboriously. While Ning and Zhou aimed to make music creation more accessible to novices, they instead inadvertently created an alternative visual interface for an ordinary MIDI keyboard. While their interface may be attractive and entertaining, it is less approachable than a MIDI keyboard would be.
The idea of controlling electronic sounds using graphics suffers from score-centrism. Writing and drawing come from different parts of the brain than singing and dancing, where music originates. It is counterintuitive to abstractly conceive the music in visual form first, and only hear it afterwards. Furthermore, visual appeal and musical appeal are not necessarily coextensive. As Gérard Pape observes:
Sometimes the pages that are the most attractive visually don’t sound well – beauty on a visual level does not necessarily correspond to interest on a sonic or musical level.
Indeed, why should we expect otherwise? Being disappointed that a beautiful drawing does not translate into good music is like being disappointed that it tastes or smells bad. Mappings between graphics and audio must strictly defer to bodily intuition if they are to be intelligible.
User Interface Case Study: the Electric Guitar
The most successful and ubiquitous electronic music interface of all time is the electric guitar, and it has many lessons for the physical interface designer. The electric guitar’s intended purpose was to amplify traditional playing styles. However, this purpose has been almost entirely supplanted by uses undreamt of by the instrument’s inventors. The electric guitar’s lightweight strings encourage wildly microtonal embellishments and multistring bends that would be difficult or impossible on an acoustic instrument. Furthermore, some electric guitars come equipped with a whammy bar that enables the player to effortlessly detune the entire instrument with very fine and nuanced control.
The true source of the electric guitar’s musical impact is not the guitar itself; it is the amplifier. Because the player usually stands close to the amplifier, feedback is a constant danger. Jimi Hendrix and others of his generation had the remarkable insight that amplifier feedback could be a form of musical expression in its own right. Hendrix treated the amplifier itself as the instrument, effectively an analog synthesizer amplifying a modulated electrical signal. While the guitar is the most significant control surface for modulating this signal, it is only one of many — other control surfaces include amplifier settings and effects, expression and effects pedals, e-bows, talk boxes, loop pedals, and MIDI pickups.
The example of the guitar poses questions for other electronic music interface designers: how can a tool fail productively? How can designers build a functioning tool that still leaves the door open to unpredictable use cases? How can space be left for the emergent, the serendipitous, the flaw turned into a virtue?
Dennett argues that virtual environments like composition software need artificial collision detection. While the “real” world is full of rough edges, entropy and chaos, these things need to be inserted into computer programs laboriously and by hand. Computer music lacks the “spontaneous intrusions” of human music — there is no amplifier feedback unless the programmer puts it there. An acoustic instrument is slowly, constantly going out of tune. Even hitting the same piano key at the same velocity produces subtly different sounds each time. Analog synthesizers are sensitive to temperature, humidity and the power coming out of the wall. A MIDI piano note is always the same, unless the programmer goes to considerable effort to write quasi-random algorithms to perform pitch-shifting.
A satisfying musical interaction system has to be noisy and unpredictable, because the noise contains potential new signals. A major part of our creativity is not just creating patterns, but discerning patterns in noise that are not really there. Generations of unschooled rock guitarists have expanded the sonic palette of their instruments because the instrument rewards playful experimentation. The best software should do the same.
A promising example of guitar-like open-endedness can be found in Janet McDowall’s field research in the use of software to teach music to young children. She focuses specifically on a program called MidiPads, which uses a touchpad controller similar to an Akai MPC. The pads can be configured to play tonal sounds or percussion. McDowall notes enthusiastically that the program engages children “in the full gamut of ‘modes of expression’ … listening, singing, playing instruments, composing, notating and conducting.” Furthermore, children find uses for MidiPads beyond conventional “music.” Children are observed acting out stories accompanied by sounds triggered by the MidiPads, effectively turning a music tool into a cinematic sound design tool.
I will conclude this section with the story of Paul DeMarinis‘ attempts to design a touch-sensitive “guitar” controller for synthesizers. Each iteration of his design kept moving closer and closer to the construction of actual electric guitars. DeMarinis even started using leather straps, because the smell evoked the rock and roll fantasy so integral to the guitar experience. How many electronic interface designers think about smell? Perhaps more of them should.
User Interface Case Study: Digital Audio Workstations
The Digital Audio Workstation (DAW) is as ubiquitous a software paradigm as the keyboard is a hardware paradigm. However, like the keyboard, the DAW imposes serious limitations out of the box. DAWs universally employ the metaphor of the multitrack tape recorder. Each voice in a recording occupies its own track, and each track persists for the entire duration of the song. This paradigm is familiar to engineers trained on multitrack tape, which helped ease the adoption of Pro Tools and its ilk into the marketplace. However, younger musicians who never learned tape recording in the first place find it to be an unhelpful metaphor.
A contemporary pop song might accrue dozens of tracks, some containing a single sample or phrase. The management of all of these tracks becomes a significant task unto itself. Producers have been known to hire assistants whose sole job is to keep track of their DAW tracks. Furthermore, the tape-recorder metaphor presumes a clear division between recording and performance. A few experimentalists have created compositions directly from tape, but the process is far too laborious to have achieved much mainstream use. For the most part, tape exists to document the sound of a person playing an instrument in real time.
Digital music production operates quite differently. Many electronic musicians are “performing” by manipulating repeats and effects parameters improvisationally rather than playing instruments. DAWs make such performances awkward by requiring playback to be stopped for many editing operations, and by making it difficult to manipulate loops in real time. Ableton Live is somewhat better than the rest of the field in this regard, since it has a mode specifically for such performances. However, moving between “recording” mode and “performance” mode is still awkward, usually requiring that loops be rendered to raw audio first. DAWs further complicate matters by offering limited control of the granularity of voice aggregation. Jumping from a high level of abstraction (sections) to a low level (audio waveforms) and back can take considerable work and organization on the musician’s part. Yet another novel job description is the “project manager” who manages a producer’s audio bounces, versions of sessions and other associated files.
The DAW’s highest level of abstraction is the song. However, live performers and DJs need to work at the level of the set list. Dancers like seamless flows from one song to the next, and a good DJ will keep the groove flowing continuously for hours on end. Closing a DAW session and opening another takes time, and such breaks in the flow are out of the question for dance music DJs and performers. These artists are obliged to assemble enormous and cumbersome single sessions including all of the tracks from the entire performance’s series of songs.
Even those musicians who work solely in the studio continually share elements between songs. It is tedious to have to continually close and open session files just to duplicate a sample from one to another. Duignan, Noble and Biddle offer an especially astute diagnosis of DAWs’ limitations around theme and variation.
DAWs… provide only primitive abstraction mechanisms in this regard, forcing our participants into a false choice between creating blocks of material that are simply references pointing to other blocks (so that no variation from the original is possible), and creating copies, where any variation is possible (but subsequent changes to the original cannot be propagated to derived blocks).
Using file pointers is convenient and reduces overhead, but it makes keeping track of dependencies quite complex.
Imagine the unfortunate beginner, confronted with the tangled metaphorical history of the DAW. Before one can master Pro Tools or Logic, it is first necessary to substantially master multitrack tape recording, at least at a conceptual level. Then one must account for the differences between the gear being emulated onscreen with the reality of the software “under the hood.” Imagine if a Microsoft Word user had to learn how to set lead type. This is the predicament faced by beginner DAW users. Unfortunately, no one has yet devised a fully workable alternative. Ableton Live’s performance mode is a step in the right direction, but much work remains to be done.
User Interface Case Study: Harmony Space
The network has rich potential as visual metaphor for representing music. The flowcharts employed by Hewitt to illustrate “control flow” of a song are an excellent example. Another inspiring idea comes from Leonhard Euler, who devised created a network representation of tonal harmony. He discovered that if one connects the diatonic thirds and fifths into a lattice, the resulting network takes on the topology of a torus. Harmonic proximity and physical proximity on the torus are one and the same. One could not ask for a more body-friendly metaphor.
Harmony Space is a software extension of Euler’s Tonnetz idea. It uses a flat plane rather than a torus, but the principle is the same. Harmony Space organizes the diatonic pitches onto a hexagonal grid, once again showing harmonic relatedness by means of spatial proximity. In Harmony Space, adjacent notes form diatonic thirds and triads. Chords and scales form distinctive geometric shapes. The user can transpose chords and other patterns by simply moving the shapes around on the grid.
While Harmony Space is an elegant didactic tool, it is only partially useful — by design, it totally neglects rhythm. The authors discuss the difficulty of designing a visualization scheme for rhythm that is as elegant as the tone grid. One ingenious solution to the rhythm dilemma is offered by the Shape of Song visualization project by Martin Wattenburg. His method shows the piece of music as a horizontal line, with arcs connecting repeated passages. Narrower arcs show repeated phrases, while wider arcs show repeated sections. One can get a rich and nuanced sense of musical structure and organization simply by glancing at one of Wattenburg’s diagrams. Perhaps in the future, a clever designer will find a way to marry the Shape of Song with Harmony Space, and an intuitive multidimensional music visualization scheme will be born.
User Interface Case Study: Garageband
Apple’s Garageband software has become the de facto beginner-level music production environment simply because it is included free with all of their desktop computers. Garageband is a feature-limited version of Logic, using the same multitrack tape recorder metaphor as most other DAWs. In addition to the basic recording and mixing functionality, Garageband includes a variety of appealing audio and MIDI loops and software instruments. The loops can be altered by the user in a full-fledged MIDI editor.
Garageband is relatively versatile and accessible, but it has several shortcomings as a tool for beginner self-teaching. The pre-installed loops are well-recorded and diverse, but Garageband offers no suggestion as to how to make good musical use of those materials. The interface does not suggest, for example, that by western pop tradition, loops sound best when repeated two, four, eight or sixteen times. Also, Garageband makes no attempt at showing harmonic relationships. Users are left to trial and error to find musical chord/scale combinations. The program does enable the user to set a global key center, which loops will fit into automatically. But when it comes time to enter individual MIDI notes, users are on their own. Ideally, Garageband’s MIDI editor would suggest to the user which notes would actually sound good, perhaps by means of color-coding, or by snapping to consonant chord tones.
A ten-year-old guitar student of mine is a vivid example of both Garageband’s strengths and weaknesses. After only his third lesson, he tried writing his own song with Garageband, mostly by throwing loops together. While I admire his initiative, his song was jagged and disjointed, lacking any kind of structural logic. It is natural that a first effort would be a mess, of course, but I felt a missed opportunity. Watching other beginners struggle with Garageband has made me realize that the software is not really for novices after all. As the name suggests, it seems more intended toward adult amateurs with previous musical experience, perhaps dads in cover bands, who have some half-remembered knowledge of chord progressions and song forms and who just need a minimum of prodding to start putting together tracks on the computer. Complete beginners need to learn musical fundamentals elsewhere before Garageband gives satisfying results.
The iPad version of Garageband is a better experiential learning tool for the beginner. Its new touch-specific interfaces encourage playful exploration. The program is not trying to be particularly pedagogical; nevertheless, its presets and defaults give valuable implicit guidance. And while iPad Garageband is quite a bit more limited than the desktop version, those limitations are strengths for a beginner’s purposes.
The iPad Garageband guitar interface is an exemplary piece of user interface design, one that deserves to be widely imitated. You can tap out notes on the graphical fretboard, complete with string bends. A profoundly non-musical friend of mine took to the virtual guitar immediately, doing huge multistring bends that would be impossible on a real guitar (and that it therefore would not have occurred to me to try.) In addition to the real fretboard layout, you can also play the guitar in Scales mode. You select a scale, major or blues or Mixolydian mode or what have you. The scale tones will then be the only notes available to you on the fretboard. With no wrong notes possible, you are free to effortlessly explore melodic ideas.
The guitar interface also offers a chord strumming mode, and here the touch interface begins to feel truly magical. The strings are overlaid with vertical rectangles, each representing a chord. Brushing your fingertip across the strings within a given rectangle sounds the corresponding chord. It feels very much like strumming with a pick, except that all of the notes are automatically correct. For whatever key you have selected, Garageband presents an assortment of chords that are either diatonic or part of common rock/pop/blues practice. There is also a slot reserved for the user’s choice of chord. The options are surprisingly varied, including several sophisticated jazz chords. Once you have strummed your chords, you can go into the MIDI piano roll and edit individual notes. An advanced user can rough out a guitar part by strumming, and then refine the harmonies one note at a time in the piano roll.
The keyboards work very much like the guitar. You can play individual notes on a regular keyboard, or you can use the equivalent of the guitar’s strumming mode, where moving your fingertip on a vertical strip produces arpeggios or melodic figures. While the prefabricated patterns are banal, they can be easily customized in the MIDI editor. The violin, cello and upright bass interface has a similar arpeggio/pattern interface, along with a fretboard mode allowing both bowing and pizzicato. The fretboard allows satisfying microtonal slides and vibrato. Scales mode is available as well.
From a pedagogical standpoint, the best aspect of iOS Garageband is the way it handles song structure. By default, song sections are eight bars long. You are free to use any length, but if you fall back on the defaults, all of your sections will be eight bars. Duplicating a section is effortless, which encourages symmetrical song structure. More advanced users are free to introduce idiosyncratic phrase lengths if they wish.
There is certainly room for improvement. The scales mode that is so useful in the instruments would be beneficial in the MIDI editor as well. Beginners would benefit from being locked into a single scale globally by default — they could always switch scale mode off if they wanted chromaticism. Also, it would be useful if the program organized the scales and gave descriptions of how they sound and what they’re useful for. The word “Mixolydian” is meaningless to beginner musicians. It would be better to say, “this scale sounds good in rock, blues, country, and a variety of world musics.” Similarly, it would be useful if the chords gave some indication of their function. Perhaps future versions will include better pedagogical guidance. For now, though, iPad Garageband is perhaps the best entry-level digital music tool on the market.
User Interface Case Study: Propellerhead Figure
Propellerhead is a pioneering software synthesizer company. Their first release, 1998’s Rebirth, was designed to emulate the classic Roland TB-303 bass synthesizer and Roland’s TR-808 and TR-909 drum machines. Propellerhead reproduced the sound of these devices with remarkable fidelity. Unfortunately, they also chose to reproduce the original devices’ impenetrable user interfaces, with primitive step sequencing and clumsy editing tools.
The company’s next major release, Reason, was a substantial improvement because it included a MIDI editor. But the design still relied too heavily on skeuomorphism. As DAWs presume familiarity with multitrack tape, so Reason presumes familiarity with rack-mounted hardware synthesizers. Its onscreen graphics are larded with nonfunctional “realistic” decoration: screws, power cords, labels, vents. It is difficult to distinguish the functional elements from the decorative ones. Furthermore, the functional interface elements are modeled after hardware knobs, buttons and LED displays. While this aesthetic is appealing at first glance, it swiftly becomes an obstacle. The skeuomorphisms occupy valuable screen space, making the usable elements smaller and harder to read. Turning fake knobs with the mouse is needlessly difficult and imprecise. And if, like me, you used software for years before ever even seeing a vintage synthesizer or analog effects unit, the hardware metaphor is totally unhelpful. A simple boxes-and-arrows interface with black text on a white background would work better.
Propellerhead has taken an admirable step forward with its first mobile offering, an iOS app called Figure. It has an appealing simplicity: one lead synth, one bass synth, and one drum machine, with a variety of settings and effects for each. Figure uses no skeuomorphism whatsoever. Its interface is comprised solely of flat-colored polygons and large, friendly text. Everything on the screen is functional; nothing is decorative. Smartphone software forces these kinds of minimalist design choices just by virtue of the limited screen real estate, and Propellerhead has wisely chosen to turn that limitation into a strength.
Figure is intended to give the novice user a maximally intuitive and effortless input scheme. Loops play automatically, and the user alters and manipulates them by dragging across the touchscreen. A loop can be broken up into more or fewer rhythmic elements by dragging up or down on a broken wheel graphic. You can select the number of scale degrees available through a similar wheel interface. Selecting the actual scale degree, modifying its loudness, and manipulating effects parameters are all performed by simple dragging within large rectangular screen regions.
This interface, while visually appealing and approachable, has its flaws. Continuous x-y pads map awkwardly onto discrete drum hits and synth notes. It is needlessly difficult to specify a particular beat or pitch — Propellerheads assumes that you will be content with the preset patterns. In fairness, their goal with Figure is different from a designer of pedagogical software. Figure aims to move the user quickly past the sequencing stage quickly and into the filters and effects, where most of the expression in techno music takes place. They succeed in this goal, but it is a shame that the choice of notes and drum beats is so needlessly opaque. Still, Figure’s friendly minimalist aesthetic is inspirational. By limiting the amount of information on the screen, Figure invites the novice to explore, free from option paralysis.
Music Games
Outside of the academy, the richest source of novel interface schemes is the video game industry. The past few generations have been controlling software with joysticks, thumbsticks, buttons, touchpads, motion controls and voice command since childhood. Game controllers are quickly making their way into electronic music, helped along by music-specific games like Guitar Hero. The surprise of seeing a Wiimote or Kinect onstage is rapidly wearing off, and we can expect such controllers to eventually become commonplace in music.
Games like Guitar Hero, Dance Dance Revolution and the like are electronic music interfaces, albeit very simple ones. These games may not enable much creative music-making, but they have a well-documented ability to teach players how to listen actively, like musicians. McDowall and Gower praise these games for having “progressive levels of difficulty and rewards for success and persistence,” which are crucial psychological motivators. The pop-music orientation of music games is invaluable for drawing students into music-learning generally. The authors also observe that in addition to being an enticement into “real music,” pop is worthy of inclusion in the curriculum in its own right. Studies have shown that the skills learned in Guitar Hero or SingStar transfer readily to non-game settings. The authors therefore give an unequivocal endorsement of these games’ inclusion in the music classroom.
Jacob Smith observes that music video games are effective tools for immersing the player in music as an active listener. “[P]laying these games can feel like a genuinely musical experience: the controller is no longer a trigger but a percussion instrument, and the player stops thinking in terms of locking on targets and instead tries to feel the groove.” The “easy” mode in such games is an abstraction of the major melodic and rhythmic events in the song, similar to the reductions performed by music theoreticians. At more advanced levels, the simplified abstractions are progressively filled in until the player must replicate every note or event in the song. This progressively more granular attention to the music is an excellent way to learn to listen like a musician. An avowedly non-musical friend told me that until he played through Beatles Rock Band as Paul McCartney, he had never paid attention to a song’s bassline. Now he hears those familiar Beatles songs with an entirely new informational dimension. No one could have taught him better.
With digital music production taking on many of the qualities of computer games, Smith believes that games offer the best way forward for new and more intuitive interfaces. While the “instrument” interfaces in most music games are simplified, they need not be. It is now possible to play Guitar Hero-like games using a full-fledged MIDI guitar. The game goes beyond simulation into actual guitar pedagogy, using the game framework’s effective psychological motivations.
There is one crucial difference between music games and real music-making, however, and that is the absence of improvisation. The player moves through the song like a train on a track, and the games penalize any variation from the prescribed notes. Not all real-life music is improvisational either, but there is usually some element of personal expressiveness. Not so in Rock Band or Guitar Hero. Mimicry is the only way to play. Joshua Rosenstock recognizes this shortcoming, and has devised a game to try to address it. Working with students at the Worcester Polytechnic Institute, he developed iGotBand, an experimental video game that incorporates improvisation. Players need not reproduce the given note sequences exactly; they are free to use any rhythm, and can interject notes of their choice.
Rosenstock’s game is an admirable attempt at incorporating improvisation into a music game, but he fails to address some basic problems. The improvisation in iGotGame has no bearing on the player’s success or failure. This makes it a nice but meaningless feature. Rosenstock readily admits this to be a problem, and discusses the challenges inherent in turning musical improvisation into a game. Games and music share the verb “to play.” But in both domains, the word play has several distinct meanings. Rosenstock pithily equates play with freedom, and games with rules. He introduces the term paidia, meaning childlike play: spontaneous and unruly. The musical equivalent would be free jazz and other radical improvisational forms. By contrast, there is play as ludus: games with ordered rules, like chess or basketball, and indeed, nearly all video games. The musical equivalent of ludus is classical composition and more formally bound jazz styles like bebop.
Like most other music video games, iGotBand is an example of ludus. The improvisation aspect is a dash of paidia, but this aspect of the game has no bearing on the win condition, and cannot therefore be said to be intrinsic to the experience. We can hardly blame Rosenstock for this shortcoming. How would one possibly devise an unambiguous system of rules for judging improvisation that meet the requirements of ludus? Thelonious Monk’s improvisation is unequivocally better than mine, and mine is unequivocally better than my guitar students. But how could one quantify our relative levels of ability? Rosenstock attempts to address this problem by suggesting that users vote on the quality of others’ improvisation. This, however, does not address the problem, since the votes would be perfectly subjective and arbitrary.
Improvisation might superficially resemble a game, but Rosenstock inadvertently demonstrates how fundamentally incompatible it is with a competitive video game. A better direction for music games would be to remove the win condition entirely. Instead of music games, we could create music tools with game-like interfaces. The Guitar Hero interface could work well as a beginner-friendly production and composition tool. It could present familiar song forms like twelve-bar blues and some suggested riffs that the player could alter at will. The pioneering music game FreQuency included a mode where the player could remix the game’s song library. A further convergence between the gentle learning curve of the game world with the open-endedness of music software like Logic or Ableton Live would invite a great many people into making their own music, rather than just passively consuming it.
Conclusion
Electronic dance and hip-hop musicians are mostly self-taught, learning in ad-hoc peer-to-peer networks or alone. Yet learn they do — global popular culture is dominated by the output of musicians with little or no formal musical training of any kind.
Because music production software is somewhat easier to learn than the violin or guitar, the major challenge of producing electronic music is learning how to listen. This is a skill that generalizes to any kind of music-making. Electronic musicians have the advantage of being able to immediately hear flawless renderings of their ideas, with instant feedback for changes. As with good video games, the dynamic and interactive music software experience is its own motivator and its own psychic reward.
I anticipate that the next generation of beginner-oriented production software will draw not on the tape recorder metaphor, but on the sampler. I could imagine a simplified version of the Session View in Ableton Live, allowing the user to build songs out of musical Legos, dragging and dropping in real time. The user could then open and customize the individual Legos, manipulating smaller Legos to any desired degree of granularity. The customized Legos could then be re-used, transformed, pitch-shifted, time-stretched and so on.
Better visual metaphors for music have yet to be implemented. For example, while the “container” for chords is intuitive, it is also misleading, since the chord is comprised of tones, not a box for them. A better image would be tones as atoms and chords as molecules built from those atoms. The molecular network gets at the relational nature of musical elements. As the molecule becomes a more familiar image, it will become available as an “intuitive” image schema.
The challenge with digital music production is not having ideas or being able to realize them. Spinning out ideas is practically effortless once you master the software. The real problem is option paralysis. Beginners would benefit from a limited set of musical options, gradually unfolding into the full universe of possibilities. Such limitations can guide novices into the best rhythms, melodic structures and harmonic progressions known to traditionally educated musicians. The challenge now is to design software constraints that stimulate creativity, rather than frustrating it.
Bibliography
Blackwell, Alan and Collins, Nick. “The Programming Language as a Musical Instrument.” In P. Romero, J. Good, E. Acosta Chaparro & S. Bryant (Eds). Proc. PPIG 17, pp. 120-130.
Chadabe, Joel. Electric Sound. Pearson, 1996.
Collins, Karen, ed. From Pac-Man to Pop Music: Interactive Audio in Games and New Media. Ashgate, 2008.
- Jesper Kaae. “Theoretical Approaches to Composing Dynamic Music For Video Games.” pp 75 – 91.
- Norbert Herber. “The Composition-Instrument: Emergence, Improvisation and Interaction in Games and New Media.” In Collins 2008, pp 103 – 123.
- Peter Schultz. “Music Theory In Music Games.” In Collins 2008, pp 177 – 188.
Daniel Dennett. “Collision Detection, Muselot, and Scribble: Some Reflections on Creativity.” In Cope, David, Virtual Music, The MIT Press 2001.
Duignan, Matthew; Noble, James; and Biddle, Robert. “Abstraction and Activity in Computer-Mediated Music Production.” Computer Music Journal, Volume 34 Issue 4, Winter 2010, pp 22 – 33.
Gelineck, Steven and Serafin, Stefania. “From Idea to Realization – Understanding the Compositional Processes of Electronic Musicians.” Proc. Audio Mostly 2009, Glasgow, Scotland, 2009.
Gower, Lily and McDowall, Janet. “Interactive Music Video Games and Children’s Musical Development.” British Journal of Music Education, Volume 29, Issue 1, March 2012, pp 91 - 105.
Hewitt, Michael. Composition for Computer Musicians. Course Technology, 2009.
- Ch 5 “Dance Music Drum Programming” pp 59 – 74
- Ch 9 “Melody, Bass and Harmony” pp 113 -122
- Ch 14 “Arpeggiation” pp 161 – 168
- Ch 17 “Approaching Structure” pp 185 – 190
Hewitt, Michael. Music Theory for Computer Musicians. Course Technology PTR, 2008.
Katz, Mark. Capturing Sound: How Technology Has Changed Music. University of California Press, 2010.
- “Introduction.” pp 1 – 9.
- “Music in 1s and 0s: the Art and Politics of Digital Sampling.” pp 146 – 176.
- “Listening in Cyberspace.” pp 177 – 210.
Kindall-Smith, Marsha, et al. “Challenging Exclusionary Paradigms In The Traditional Musical Canon: Implications For Music Education Practice.” International Journal of Music Education, 2011.
Lähdeoja, Otso; Navarret, Benoît; Quintans, Santiago; and Sedes, Anne. “The Electric Guitar: An Augmented Instrument and a Tool for Musical Composition.” Journal of Interdisciplinary Music Studies, Fall 2010, volume 4, issue 2, art. #10040203, pp. 37-54.
Lakoff, George and Johnson, Mark. Metaphors We Live By. University Of Chicago Press, 2003.
Magnusson, Thor. “Designing Constraints: Composing and Performing with Digital Musical Systems.” Computer Music Journal, Volume 34, Number 4, Winter 2010, pp 62 – 73.
McClary, Susan. “Rap, Minimalism and Structures of Time in Late Twentieth-Century Culture.” in Daniel Warner, ed, Audio Culture, Continuum International Publishing Group, 2004, pp 289 – 298.
McDowall, Janet. “Music Technology: a Vehicle for Young Children’s Learning.” Australian Journal of Music Education, no. 2, 2008.
Ning, Cong and Zhou, Steven. “The Music Pattern: a Creative Tabletop Music Creation Platform.” Computers in Entertainment, Volume 8, Issue 2, December 2010, Article 13.
Rosenstock, Joshua Pablo. “Free Play Meets Gameplay: iGotBand, a Video Game for Improvisers.” Leonardo Music Journal, Vol. 20, pp. 11–15, 2010.
Saville, Kirt. “Strategies for Using Repetition as a Powerful Teaching Tool.” Music Educators Journal, 2011 98: 69
Schloss, Andrew and Jaffe, David. “Intelligent Musical Instruments: The Future of Musical Performance or the Demise of the Performer?” Interface, 22:3, 1993, pp 183-193.
Smith, Jacob. “I Can See Tomorrow In Your Dance: A Study of Dance Dance Revolution and Music Video Games.” Journal of Popular Music Studies, Volume 16, Issue 1, April 2004, pp 58–84.
Snyder, Bob. Music and Memory: An Introduction. MIT Press, 2001
Thibeault, Matthew. “Wisdom for Music Education from the Recording Studio.” General Music Today, 20 October 2011.
Thompson, Paul. “An Empirical Study Into The Learning Practices And Enculturation Of DJs, Turntablists, Hip-Hop And Dance Music Producers.” Journal of Music, Technology & Education, Volume 5, Number 1, pp 43 – 58.
Vickery, Lindsay. “The Evolution of Notational Innovations from the Mobile Score to the Screen Score.” Organised Sound 17, 2012, pp 128 – 136.
Wilkie, Katie; Holland, Simon; and Mulholland, Paul. “What Can the Language of Musicians Tell Us about Music Interaction Design?” Computer Music Journal, Vol. 34, No. 4, Pages 34-48
Wise, Stuart; Greenwood, Janinka; and Davis, Niki. “Teachers’ Use of Digital Technology in Secondary Music Education: Illustrations of Changing Classrooms.” British Journal of Music Education, Volume 28, Issue 2, July 2011, pp 117 - 134.