The first time I played PaRappa the Rapper was a struggle. I was with my parents at a novelty deli where each table was outfitted with a television and a PlayStation video-game console. I didn’t have one at home, but a paper-thin, hip-hop cartoon dog named PaRappa that goes on some sort of adventure in a 3-D world seemed like the kind of game I would enjoy. I assumed it was an action-jumping affair. What other games were there? The shooting kind, the sword-swinging kind, the puzzle kind, the punching kind and, my favourite, the googly-eyed tortoise-stomping kind. PaRappa isn’t like any of those.
PaRappa is infatuated with a talking flower named Sunny. The little pup’s quest to impress the plant begins with learning kung fu from an onion named Chop Chop Master Onion. But this absurdity isn’t the reason why PaRappa the Rapper is unlike all the popular games that came before it.
“Kick! Punch! It’s all in the mind,” falsettos the onion. “If you wanna test me, I’m sure you’ll find / The things I’ll teach ya is sure to beat ya / But nevertheless, you’ll get a lesson from the teacher.
“Now kick!”
To beat the level, you have to keep up with Chop Chop Master Onion’s lyrics, repeat them to show you’re worthy of having “rap” in your name. When you can’t keep up, rubber ducks start quacking over the song.
PaRappa was hip, delightful, and trailblazing. Published by Sony Computer Entertainment (SCE) in Japan in 1996 and the rest of the world the following year, the game won the top awards for interactive design and sound design at the first annual Interactive Achievement Awards in 1998. Since its release, it has sold over a million copies. In a world with more fake plastic Xbox 360 guitars collecting dust than ever, games that focus on playing music aren’t a novelty anymore, they’re a genre. Studios have found it easy to produce addictive games about moving to the beat, from Dance Dance Revolution to Beatles: Rock Band. But to say PaRappa is nothing more than a charismatic version of Simon Says misses one of its most interesting elements.
“The best use of music in a game would be to implement a total design strategy that allows the player and the music to interact with one another,” Masaya Matsuura, the Japan-based creator of PaRappa, tells me. “I think it’s okay for music to be used to support elements of the story, but if that’s all it’s used for, this isn’t a usage that piques my interest, really.”

To beat PaRappa, you need to complete each song with a rating of GOOD (the two ranks below are BAD and AWFUL). After finishing the game, you unlock a new rank, COOL, which you can’t achieve just by following the lyrics or tracing shadows. You have to rhyme on your own, pressing buttons to scat back the words in a dazzling way. When you reach COOL, your mentor (the cartoon feeding you lines) congratulates you, and for the rest of the track, playing the game means playing your own song. Matsuura explains: “Making players feel the importance of imagination is one of my biggest goals, and even now I am still trying to figure out good ways of doing that.”

Since PaRappa, Matsuura has released several albums of his own music, most recently Beyooond!!!, which came out in March 2014. And he’s created more musically inclined games, notably the cult, minimal beat-tripper Vib-Ribbon (SCE), which was one of fourteen initial games selected in 2012 by the Museum of Modern Art (MOMA) in New York for a new section of its permanent collection.
Matsuura has no interest in first-person shooters. The games that make him happy are the ones he calls “interactive ground music,” a term, he says, that is “meant to contrast the idea of background music by implying that music plays more than just a secondary role—I think it would be safe to say that there is no widely accepted term for this just yet!”
The term aptly describes the Toronto-made Sound Shapes (SCE), released in 2012, through which players construct songs by pop-music experimentalist Beck, as well as Toronto-based artists Deadmau5, Jim Guthrie, and I Am Robot and Proud. This is not done by pressing certain buttons to prescripted beats, but rather through the elegant, natural agency of playing. Like PaRappa, Sound Shapes—winner of various U.S. and Canadian video game awards for its music, mobility, and innovation— is part of an ambitious, growing array of works that experiment with the ways that we interact with our music.
The first time I played Sound Shapes, Jon Mak and Shaw-Han

Liem, its creators, were standing next to me at a Sony preview showcase, which was being held in a very wide, loud hall. Mak, a small, quiet person, handed me a thick pair of headphones and hovered around as I started playing the Mechanica level, featuring music composed by Liem under his stage name I Am Robot and Proud. I was struck by how little sound there was at the beginning. Mechanica takes place in an alien-looking factory; in between the pounding sounds from a few heavy, crushing obstacles, you could hear a pin drop. Once you pick up one of the floating orbs, you begin to hear something different. Things begin to compile and overlap, and the once-disjointed sounds of deathtraps and machinery now look and sound like a sly Portishead rendition of Modern Times’ most famous scene. Each level is a living song: Beck’s Cities is an empty metropolis under a rocket assault; Jim Guthrie’s Personnel is a neverending Kafkaesque office complex; Deadmau5’s Break-a-noids is a rave inside a Space Invaders arcade machine. Playing the game and playing the songs are one and the same—which makes sense, given the philosophy of its creators.

Mak once told me that playing a guitar is similar to playing Street Fighter: both skills are very technical and internalized, and both are mastered through creative expression. And Liem believes that all music is, in a way, interactive. “Music is about communication, always about a creator making something—an audience responding to it, interacting in that way,” he says. “The simplest thing is going to a show, seeing someone performing—you’re hearing someone make music, but you’re also seeing it deconstructed. You’re seeing a part of the production process, and you’re responding to it by dancing, clapping, doing whatever. Musicians feed off whatever energy is happening in the room. It’s not a special thing that games do. Or at least, it’s part of the conversation of how someone listening to music responds to it.”
Mak began attending Liem’s I Am Robot and Proud shows around the same time as he wrapped up his first release, Everyday Shooter, a Robotron-esque sound voyage. It similarly has the player survive alongside the progression of an album, blasting abstract enemies, letting each piece, beat, or part of a score be a reward for success. In Sound Shapes, the deconstruction of these melodies amplifies the fact that players create their own stages, using the sounds like building blocks. The perils aren’t just there in order to create a thoughtfully designed level, but to synchronize with an ongoing harmony, which is, more than anything, what excited the game’s creators.

“Even as a musician, making music for Sound Shapes opened up a lot of compositional ideas that I hadn’t thought about before,” says Liem. “It’s like a completely different visualization—visualizing pitch, rhythm. It opens up possibilities. Being sort of a gateway for people who don’t make music was definitely one of our major goals.” You can play Sound Shapes like a lot of other video games. There’s no repercussions of neglecting the music or of not picking up those spheres. But it’s like watching Fantasia on mute: as inspiring as the visuals are, or as engaging as the play might be, it’s the way the game sings that makes it so spectacular.

Music for games, although often greatly appreciated, has mostly existed in its own enclosures. Japanese composer Nobuo Uematsu, who scored most of the Final Fantasy series, and British composer David Wise, the creator of Donkey Kong Country’s haunting melodies, both have passionate cult followings. To some people, the theme from The Legend of Zelda is a more powerful ballad than Stairway to Heaven. But this music is typically complementary to the game, a nostalgic anchor on top of something already en route to being beloved.
The games being designed today are far different from those of decades past. The production teams responsible for great games don’t need vaults of resources and acres of coders; they can be as small as ten, four, or two people—or even one person. So it follows that games have become more ambitious, artistic, focused, and avant-garde in scope. And it’s only a matter of time before more musicians, akin to Liem or David Kanaga, get in on making games themselves.
Before creating the Naked Gun-inspired spy spoof Jazzpunk (Adult Swim), Toronto-based Luis Hernandez, cofounder of Necrophone Games, created a game incarnate of doom metal called Void One, which pummels players with heavy and disorienting environments. Toronto-based experimental-music and performance-art collective Yamantaka // Sonic Titan—whose second album, UZU, was a shortlist nominee for the 2014 Polaris Music Prize—has supplied music for Severed (Toronto’s DrinkBox Studios), scheduled for release in 2015, and the band has been developing its own interactive space–rock-opera video game, YOUR TASK // SHOOT THINGS.
But what Matsuura means by “interactive ground music” is more than a game that merely puts music in the spotlight. It’s a game that creates music: sounds and rhythms map the functions of a keyboard where the jump-and-fire commands would otherwise be, noise-coated landscapes become a sound installation of infinite possibilities.
Electroplankton (Nintendo) is a strange toy of a game created by Toshio Iwai, codesigner of the Tenori-on, an electronic musical instrument. Over the course of Electroplankton’s ten different diorama-like stages, players can manipulate the setting, which changes the trajectory and behaviours of the little musical plankton that live inside the game, this loose sense of control leading to the fascinating sensation of composition. “Electroplankton and Tenori-on have a similar first-hour experience,” says Liem, one of several artists who performed on the 2008 world tour that introduced the Tenori-on. “You pick it up, start messing around, but it doesn’t tell you very much about itself—you discover it by experimenting. Experimenting itself is part of the fun. The fun is built into the learning.”
The only thing you can do in the abstract Mountain, the first game from famed artist–animator David O’Reilly, is play a small, simple chiming xylophone to see what effects, if any, it holds over a floating, talking mountain. For some, the interpretive, ambient game needs little else: the soothing pricks of sound are a delight in themselves. The game’s enjoyment lies in how personal it feels to the user—or so says, Robin Arnott, whose own game, SoundSelf, offers a similar satisfaction, though on a much larger scale.
The first time I played SoundSelf was in a hotel room during a Toronto games festival. Arnott, its Austin-based

creator, wasn’t in the city, and the developer attending the event wasn’t around. The volunteers had very little to say about what I would be playing, but as it happened, being told to jump without a bungee cord proved to be the best direction anyway. The visuals were like a slow plasma globe being projected on the wall like a spirit in the room. The patterns, colours, and shapes don’t change when you mash on a controller, but, rather, when you begin to sing to it. I began chanting from my gut. I guess you could call this my short throat-singing phase. After a few seconds, SoundSelf did something even more shocking: it began to sing back. SoundSelf “relies on a feedback relationship with the player and their own voice,” says Arnott. “I wanted to have a visualization intimately connected to the sound that you hear, the sound you identify as your own voice, because it begins with your own voice. I could have a very high bandwidth portal into the player’s perception of who and what they are.”

SoundSelf was inspired by an LSD-enhanced encounter with a six-chambered structure called reOnion, an art installation presented at the 2011 Burning Man event in Nevada. Once inside, Arnott was compelled to chant. The installation filled with the sound of human voices, and he found the experience sublime. It was exactly the eureka moment he needed, struggling with where to take his next project.
Arnott previously created Deep Sea, a high-concept game which forces the player to put on a blind-eyed gas mask and, using only ambient sounds, annihilate an abyss-lurking creature before it annihilates the player. I’ve tried it myself: The creature can detect your breathing and track you, and the headgear’s rubber collar squeezes your neck, making you feel oxygen-impoverished. Critics rave about the encompassing, claustrophobic experience. Arnott was happy with the game, but not with the fact that it relied so much on terror. He thinks fear, in games, is a crutch.
While his reOnion trip was chemically induced, the experience and sobriety are not mutually exclusive. SoundSelf is mysterious, ominous, but not intimidating—something Arnott likens to a dance partner. You don’t control it, but it moves with you. “It’s candy for your brain,” he says. “Breaks it up, looks for patterns, tries to rationalize it. It’s your brain being overwhelmed by the complexity, but it’s still a reflective space. It allows your mind to reflect on itself. It’s not designed to sound good; it’s designed to feel good.”
Robin ended up taking SoundSelf to Burning Man, turning the game into an installation—an experience that can result in more than just physical effects.

 The first time I played Proteus I was, I must admit, incredibly high; David Kanaga, the California-based musician–composer who created the game with independent British game designer and programmer Ed Key, was charmed when I told him this—so that’s somewhat vindicating. The game—which won several awards during its development and was released for various platforms throughout 2013—begins with you standing in the ocean, looking towards a procedurally generated island (meaning it was created from scratch, and starting again will give you a differently constructed environment). Other than using the joystick, there’s not much the controller can do. One button will close your eyes, to reset the island. “Keeping it mysterious was an important thing throughout,” says Kanaga. “It felt important to keep a proper sense of mystery that even we didn’t understand.” In Proteus, you explore the landscape of rough pixels and pastel colours. But it’s not just sights to behold. Every item on the island—stones, animals, arbour—make music. Whispering flowers, croaking frogs, humming boulders, all cozy up to each other in a composition you create through traversing. Stranger things can, and do, happen for the inquisitive player, though.

When I told Kanaga that I had followed my gut, he said that that’s the proper way to do things. Places on the island can seem peculiar, totems, formations. While you cannot interact with them, you can wait to see what happens, and as day turns to night, as summer turns to fall, the game begins to unravel. One daunting moment for me was during the fall. I was riding the exploratory high of hopping between seasons, an exodus triggered by following certain strange phenomena around the isle. While night usually blossoms with bird howls and star hymns, I found myself being trailed by an intense red, glowing light. A little spooked, I hoped that by warping into the next season I would escape the ominous hue. But I met with winter—cold, grey, vision-impairing, and most devastatingly of all, without sound. The island, usually alive with melody, was muted. Winter made me feel incredibly sad: all the changing life on this little land had vanished. I was all that was left, and I eventually was lifted out of there by the Northern Lights.
“You had the red light just before the winter transition? That’s so cool,” says Kanaga, “I’ve never seen that happen. A lot of things can happen.
“(Games) are like really wet clay, and you’re scooping something out of it. Really wet, smooshing. I like to think that’s how the music works in these games. It’s really the same stuff as the landscape—same material, but a different reflection of it.”
Kanaga regards his games and others like it as akin to freeform jazz, bricolage created by player agency. Kanaga has also composed the score for the musically centric Canadian game Dyad (Right Square Bracket Left Square Bracket), which led to an album on the Mexican Summer label. The music as track-by-track album, and the music as affected by a player’s interaction feel like different works to him. He says listening to the soundtrack is like watching someone else play the game. “Dyad was more like a mix CD,” says Kanaga, “but Proteus had a pretty tight structure— four seasons, four movements. Pretty easy for a classic or symphonic form. One single piece.”
Kanaga is currently working with Fernando Ramallo and Canadian developer Phil Fish, creator of Fez and a key subject in the documentary Indie Game: The Movie. In this new game, called Panoramical, the player swiftly swoops over a changing, singing landscape, using a keyboard and mouse or a MIDI controller. It’s an inverse, sort of spiritual sequel to Proteus. “It feels like a middle ground to me,” says Kanaga, “between playing a song in iTunes and playing a keyboard. Pressing ‘Play,’ playing music on an instrument, pressing ‘Play’ many times—Panoramical’s fun because that is approaching the instrumental side of things.”
Games can be a lot of things. After their long history as a technically advancing toy, it’s safe to say their name now feels pretty loaded. Today’s games don’t need traditional game structures, they don’t need end goals as much as conclusions, they don’t have to be competitive as much as endearing. Today there are more games than just the shooting kind, the sword-swinging kind, the puzzle kind, the punching kind, and the turtle-stomping massacre kind.
We now have games that are not the game kind of game. We have the installation kind, the locative kind, the experimental kind. We have games that start to sound more like music. We have games that essentially are music: music with a shape, a form—one you can interact with and can yourself alter the progression of its sound. Interactive ground music, if that’s what we’ll call these games (and it’s hard to deny one of the pioneers the pleasure of having his term used) show how once again technology can expand the methods in which we listen to music.
It’s music you can play, instead of just pressing play.

AudioApollo Dux (2011), composed and performed by David Kanaga for the video game ProteusHomepage slider: Graphic created by Atanas Bozdarov. Images (top to bottom)PaRappa the Rapper creator Masaya Matsuura performs in Kyoto in 2014 (by Yoshihiko Kawai); Chop Chop Master Onion; Sound Shapes creators Jonathan Mak (left) and Shaw-Han Liem in Toronto (by Mark Rabo); Beck's Cities is an empty metropolis under rocket assault in Sound ShapesSoundself creator Robin Arnott; screenshot from Proteus, created by musician-composer