Someone in Vietnam with the screen name “yipiehk” is picking out “Amazing Grace” with the Ocarina 2 app on a mobile-device-turned-musical-instrument, and so am I. It’s easy; just blow into the microphone and finger four “keys” on the screen (lights guide your fingers), like a digital flute. We’re both terrible, to be honest, but in addition to making music, the app shows me yipiehk’s location on a globe; I click a button marked “Love” to send him or her some encouragement.
As I stumble to the finish, Ocarina tells me that I have earned 12 “breath points” and played 35 notes. So far, I am 41 percent of the way to mastering “Amazing Grace,” but I may move on to “Twinkle, Twinkle, Little Star” or “Claire de Lune,” purchase a more difficult song (anything from Mozart’s “Ave Verum Corpus” to Justin Bieber’s “Baby”), or just noodle on my own. When I am done, I can share my performances with friends via Facebook or Twitter.
Ocarina may have selected my performance to broadcast to anyone else using the app, just as it sent yipiehk’s performance to me. Meanwhile, a feature on the app shows me that “michaelreid1994” in Scotland has picked up “Amazing Grace,” as has “darengasa” in Italy. Within a few seconds, 33 Ocarina users around the world “love” darengasa’s rendition. It’s an international group hug.
Ocarina, a 99-cent app downloaded more than 8 million times, was created by Ge (pronounced “Guh”) Wang *08, the co-founder, chief technology officer, and chief creative officer of Smule, which also makes nearly a dozen other music apps. They include Magic Piano (a kind of Guitar Hero in which anyone with a smartphone can be a piano prodigy), Glee Karaoke (which puts the fun of a karaoke club in your pocket), and I Am T-Pain (which lets you distort your voice like the famous rapper). The company’s name comes from Sonic Mule, a character in Isaac Asimov’s sci-fi Foundation Trilogy who is able to bend the minds of others. (“What’s in a name?” Wang asks. “A lot.”) He now considers himself an “accredited entrepreneur.”
In addition to running Smule, Wang, who emigrated to the United States from China when he was 9, is an assistant professor at Stanford University’s Center for Computer Research in Music and Acoustics (CCRMA), referred to by everyone as “karma.” He directs Stanford’s Laptop Orchestra (SLOrk) and recently formed an adjunct Mobile Phone Orchestra (MoPhO) to capitalize on the new handheld technology. Both are modeled after the Princeton Laptop Orchestra (PLOrk), which relies on software Wang wrote as a Ph.D. student. Teams of students type diligently on their laptops or phones to create unusual sounds, which the conductor builds into a musical whole. Many of these compositions, which can include weird fugues, clicks, hums, or sounds like running water, are much more eclectic than an old gospel classic such as “Amazing Grace.” To the uninitiated, it may well sound odd, yet it can be hypnotizing, if not necessarily toe-tapping.
For a small, cutting-edge department, CCRMA occupies palatial real estate on the Stanford campus. Its home, known as The Knoll, was designed to be the president’s mansion. When a visitor asks directions outside the music department on the main quad, a faculty member turns wistfully and says, “Ah, yes, the fools on the hill.”
Wang hears this story and laughs. “That’s a compliment — I guess.” The music department’s view of CCRMA, he says, ranges from detached amusement to genuine curiosity.
The Knoll is an old building full of new gadgets. Wang occupies a small pentagonal office that opens onto a garden where roses still are blooming in early October; the office is cluttered with a huge computer monitor, a hard hat, a Rocket Man helmet, and a stack of 40 MacBooks, still in their boxes, for the laptop orchestra. A poster on his door shows Wang wearing a red unitard, his maestro’s mane of black hair flowing over his shoulders, and he eagerly shows off a Smule video in which he runs along a sunlit road in slow motion while someone plays the theme from Chariots of Fire on Magic Piano. Ge Wang, in case this hasn’t gotten across, is quite a showman.
Ajay Kapur ’02, a professor at the California Institute of the Arts, develops robotic instruments. He stands beneath MahaDeviBot, which strikes 12 percussion instruments from India. The bouncing head can relay tempo to the human performer.
Computer technology might seem like the ultimate science and music the ultimate art, but Princetonians are in the vanguard of blending the two in a new and arresting harmony. About a dozen faculty members and alumni are changing the way we listen to music, the way we think about music, and, to hear them tell it, maybe even the way we learn and interact with one another socially.
Among them is Ajay Kapur ’02, director of the Program in Music Technology at the California Institute of the Arts in Valencia. Kapur plays the sitar, but it is unlike any traditional sitar you may have seen. His instrument, which he designed himself, is equipped with a USB jack and microsensors. As for his accompanists ... well, there’s Tammy, a 6-foot robot made of bells, strings, and marimba pieces; GanaPatiBot, a collection of drum heads backed by two speakers playing sounds from an iPod mini; and MahaDeviBot, which uses its 12 mechanical arms to strike bells, drums, and finger cymbals and can keep tempo by bobbing its, er, head. These and other robots are programmed to “listen” to what Kapur plays and respond to it: As he changes the notes or tempo on his sitar, their accompaniment changes as well, enabling man and bot to jam together.
Kapur started with a question at the heart of computer music — how do you enable a computer to improvise with a human? — and then founded a company, KarmetiK (a fusion of the words “karma” and “kinetic”) Technologies, to design and program robotic instruments that might provide an answer. What he has come up with so far gives classical Indian music (as his website puts it) “a modern electronic and experimental groove.” Last April, his KarmetiK Machine Orchestra show “Samsara” — which comes from the Sanskrit word for the cycle of birth, life, death, and reincarnation — played two sold-out concerts in Los Angeles’ Walt Disney Concert Hall.
The show was a multimedia robotic adaptation of Indian fables known as the Panchatantra. Working with CalArts theatrical director Michael Darling and Curtis Bahn *98, Kapur and six other human musicians not only played their instruments, but sent commands to the 10 remote-controlled robots in the Machine Orchestra. Meanwhile, dancers whirled and an animated film illustrated the story on a huge backdrop. The result was an extravaganza that used technology to retell an ancient story in a new and arresting way, and which the Los Angeles Times likened to “‘The Jetsons’ meets ‘Spinal Tap.’”
Kapur joined the CalArts faculty (he also teaches at the New Zealand School of Music in Wellington) after earning his Ph.D. at the University of Victoria in British Columbia. His biography describes an eclectic mix of coursework in computer science, electrical engineering, mechanical engineering, music, psychology, and media technology; his desk, which is strewn with polished wood, wires, and computer circuits — the guts of a harmonium he is building — suggests he draws on it all. He revamped the school’s computer-music curriculum and introduced a master’s program in music technology. The undergraduate department has 38 students, more than three times as many as when he arrived in 2008. “It feels like Princeton here now,” he boasts, “because all our kids are so amazing.”
Princeton’s influence in the world of computer music is wide and deep. Ocarina, SLOrk, PLOrk, and most other computer-music applications run on ChucK, a software language Wang developed for his dissertation with Perry Cook, a Princeton professor emeritus in computer science. “ChucK lets you program on the fly,” explains Cook, who also taught Kapur.
Over the years, Cook’s former students have helped spread computer music around the country. Kapur ticks off a long list, among them: Georg Essl *02 at the University of Michigan; Tae Hong Park *04 at New York University; Colby Leider *07 at the University of Miami; George Tzanetakis *02 at the University of Victoria, British Columbia; and Bahn at Rensselaer Polytechnic Institute. “We all come from Perry,” Kapur says.
The Princeton Laptop Orchestra rehearses in spring 2009.
Cook co-founded PLOrk, the first laptop orchestra in the country, in 2005, along with Professor Dan Trueman *99, Wang, and Scott Smallwood *08, now an assistant professor of composition at the University of Alberta. Today there are more than 60 laptop orchestras in colleges and even high schools around the country.
In a typical laptop orchestra, a dozen or more students control “instruments” consisting of the laptops and a variety of controllers and multichannel hemispheric speakers made from wooden salad bowls with car speakers inserted in precut holes. Depending on the musical composition, students might produce sound by tapping on their keyboards, waving handsets from Wii game consoles, or pulling strings. But the laptop’s (or phone’s) capabilities also are used in other ways: Rolling the device from side to side, for example, distorts the sound by triggering the accelerometer, a built-in component meant to protect the hard drive.
Student performers write code and program their laptops as well as play them, following a conductor’s lead but also communicating with each other visually, aurally, or electronically. Wang even has experimented with compositions in which the performers play in different cities, synthesizing their sound and communicating with each other by modem.
As Wang’s website puts it, a laptop orchestra “fuses the irreplaceable human aspects of music making with the computer’s precision, possibility for new sounds and for the fantastical automations to provide a radically new type of ensemble, research platform, and classroom.” Or as Perry Cook says: “The main purpose of PLOrk and SLOrk was to ask a lot of questions.”
Almost everyone making music today relies on computers, whether for recording, editing, amplifying, distributing, or listening. Hip-hop and R&B artists use software to distort their voices, while programs such as Auto-Tune digitally correct the human voice to produce sounds cleaner than any vocalist could make them. If humans are social, expressive, emotional, and intentional, Wang suggests, computers are precise, consistent, repeatable, networkable, logical, and tireless. They also have much greater storage capacities than we do.
Assistant professor Rebecca Fiebrink *11 works in the Sound Lab in Princeton’s computer science building.
From a composition standpoint, Rebecca Fiebrink *11, an assistant professor of computer science at Princeton, co-director of PLOrk, and another of Cook’s former students, is unapologetic in her desire to use computers to make new kinds of music, with new kinds of instruments to play it. Her students might develop an instrument out of a gaming joystick. They might write code approximating what a 20-foot-long flute would sound like. Or a flute under water. If you can conceive it, you can try to build it.
“No one is interested in making computers do better Beethoven,” she reasons. “Beethoven does Beethoven really well.”
The possibilities these new instruments offer can be both intimidating and inspiring. “Schubert works well within well-established musical conventions,” Fiebrink continues. “The rich palette, the major and minor keys, the final resolutions — all that fits with our culture. In computer music, people consciously decided to work without these rules. There are no conventions on harmony or rhythm. So, for example, how do you convey emotion? Suddenly, all the constraints have been removed. Composers’ jobs are completely open-ended now. They have to deal with new questions. What is the performer doing? What is the computer doing? Are people communicating by sending musical signals? Text messages?”
And how do you turn those possibilities into something people want to hear?
Using technology to push musical boundaries isn’t new. In the 18th century, John Broadwood, a Scottish piano maker working in London, designed grand pianos with a longer key range and a stronger frame that produced a deep, rich sound. Beethoven took Broadwood’s new piano and ran with it, writing works such as the Emperor Concerto with tonal ranges and subtleties that would have been impossible on tinny pianofortes. “Ludwig,” writes music historian Roger Neill, “was Broadwood’s beta tester.”
In the early 1950s, Princeton professors Milton Babbitt [later *92] and Roger Sessions helped found the Columbia-Princeton Electronic Music Center, which sought to innovate musical composition using reel-to-reel audiotape, the high tech of the day. The universities stopped collaborating in the 1980s, but by then Princeton had a well-established relationship with Bell Labs in Murray Hill, N.J. Paul Lansky *73, the William Shubael Conant Professor of Music, remembers typing punch cards as a graduate student, converting them to magnetic tape, and then sending each day’s work up to Bell Labs for overnight processing. For the rest of the century, a talented faculty that included Babbitt, Lansky, Kenneth Steiglitz, Godfrey Winham ’56 *65, and George Perle pushed the bounds of theory and continued to experiment with ways to harness computers to music.
Princeton professor emeritus Perry Cook in his state-of-the-art home studio in Oregon. The rubber chickens contain sensors and are wired to produce different sounds when pulled.
The current era in Princeton’s program might be said to date to 1996, when Perry Cook arrived from Stanford with a joint appointment in computer science and music. Cook introduced new classes such as “Transforming Reality by Computer” and directed the Princeton Sound Lab, a research group run by the two departments, while also doing his own research in fields ranging from voice synthesis to designing computer-music controls. In 2005, he and Trueman founded PLOrk; three years later, the MacArthur Foundation awarded PLOrk a $238,000 grant to further explore ways to develop networked computer music.
Many of Cook’s former students speak of him as a guru, and as befits a guru he now lives on a mountain. His house in southern Oregon affords a view of the Applegate River and a neighboring vineyard full of champagne grapes, but the really unusual sights are inside. Besides a solar-powered rain stick (an ancient South American instrument), Cook has built a 1,200-square-foot state-of-the-art music studio.
The walls are not parallel (to improve the acoustics), and the space is filled with monitors, control panels, speakers, amplifiers, and the many instruments Cook plays, including acoustic and electric guitars, conch shells, and an electrified variant of the aboriginal didgeridoo, which he cleverly calls a digitaldoo. Imagine the musical possibilities of another current project, the Choke-o-phonic FowlHarmonic, which Cook is co-developing with his longtime collaborator, McGill University music neuropsychologist Daniel Levitin. It’s a row of colored rubber chickens with sensors inside, each wired to produce a different sound when pulled. Program them one way and they sound like snare drums. Program them differently and they sound like a pipe organ. Or like chickens.
Cook’s sense of whimsy, Trueman thinks, may be his greatest contribution to computer music. The field, he says, “has sometimes been poisoned by pretension. [Yet] here is this musical guy who says we can do these great things and make this great art, but it will be fun and we can learn from it at the same time.”
Kapur, like Wang, credits Cook for pulling him into computer music. Born in San Francisco and raised in Connecticut, Kapur had entered Princeton planning to study computers, but by his junior year he knew that the “computer-science major wasn’t working for me.” Interested in music, he took a course with Cook, who became Kapur’s thesis adviser and also recruited him as a jazz drummer.
“Perry showed me why I wanted to learn engineering,” Kapur says. “He made it fun.” Today, Kapur tries to do the same with his own students, in courses ranging from “Introduction to Programming for Digital Artists” to seminars blending traditional Indian and Indonesian music with 21st-century technology. Students in Kapur’s composition class write a 30-second song each week using ChucK software; in his classroom, illustrating the odd juxtapositions common in computer-music programs, large computer monitors sit alongside hammers and a power drill. Tammy the robot and her fellow members of the Machine Orchestra smile benignly from hooks on the ceiling.
Kapur believes it is important to blend computers and art in the classroom as well as on stage. This past April, he and Cook received a three-year, $110,000 grant from the National Science Foundation to teach computer science to digital artists. He and Wang also have received a large gift from Sony, including a dozen 55-inch computer monitors, to support a new program at CalArts that they hope will extend the laptop-orchestra model of learning to the broader curriculum.
From the time of Socrates, Kapur says, the traditional model of education has been one teacher addressing many students. Technology, he believes, can drive a new model in which instructors and students interact with and teach each other. In Kapur’s classroom, the students sit in a large circle and their large computer monitors face toward the center, so everyone can see what everyone else is doing. Whatever one types instantly is shared with other members of the class. Creative decision-making, Kapur has written, “takes place from doing, experimenting, and peer learning.”
A purist might ask if any of this really is music — or whether laptop orchestras or individuals blowing into Ocarinas or tapping on Magic Piano apps aren’t just playing around with toys.
“Of course it’s music,” snaps Michael Pratt, conductor of the University Orchestra, with just a hint of irritation in his voice. “Why does it have to come out of a violin in order to be music? It would be really weird if people weren’t using computers to make music.”
Scott Burnham, the Scheide Professor of Music History, concurs, although he adds that Cook and others have succeeded in adding a playful element to computer music that makes the word “toy” not completely inapposite. Not that it matters. “There are many ways to make music,” Burnham says. “If music is organized sound, then we’re talking about music here.”
Wang has written that the Ocarina was designed to be “an instrument that also feels like a toy (something that everyone feels comfortable starting/ playing).” He suggests calling it an “expressive musical toy,” but quickly adds, “I believe people shouldn’t have to think too hard about making music!”
There, it seems, may lie the key. Though some computer-based compositions seem inaccessible, it is worth considering that computers have the potential to do for creating music what they already have done for listening to it: make it instantly available to anyone.
“People love the violin because you spend 18 or 20 years so you can be really good at it,” Fiebrink says. “In the hands of an expert, it is the pinnacle of human expression. But a violin doesn’t sound good if you’ve never played it before. Computer music can lower the barrier for people without musical training. It gives people some freedom, but doesn’t make them study for 20 years.”
Going a step further, Wang muses on changes that the mobile revolution already is causing, referring to machines, as he often does, like old friends. “If I’m forced to use this guy,” he says, pointing to his desktop monitor, “I have to come to its world. This guy” — here he pulls out his phone — “comes with me. I can use it to make music if I’m waiting in line to buy milk.”
Cook thinks that Wang’s apps and Kapur’s shows have added a welcome social dimension to computer music, not only bringing it to a larger audience, but making it fun. Wang hopes that cellphone or iPad Ocarinas someday might revive the lost pastime of family sing-alongs around the parlor piano — or at least a modern counterpart. “We’re missing out a ton by being so passive,” he says. “There is a special joy in picking something up and making something with it.”
Wang speaks as someone who has done just that, recalling a night he was testing new code for Magic Piano on the San Francisco subway. A group of tipsy Giants fans coming home from a ballgame heard him and spontaneously joined in singing. “Something magical happens. People look at you funny — but in a good way.”
“You don’t have to be Mozart on this,” Wang says, waving his Ocarina. “You don’t have to be Franz Liszt. It’s a [bleeping] phone.” Then he bursts into a laugh that can only be described as musical.
Mark F. Bernstein ’83 is PAW’s senior writer.
Related News
What Now? Princeton Faculty React to Donald Trump Victory
Many ponder how dangerous a second term will be. “I think he needs to be taken seriously, but not literally,” says Robert P. George.
Will Venable ’05 Hired To Manage Chicago White Sox
Former major league outfielder sees “the opportunity to build a foundation’ with his new team
0 Responses