Current Issue

Jan. 16, 2013

Vol. 113, No. 6


Musical machines

Are we ready for orchestras composed of computers and robots?

By Mark F. Bernstein ’83
Published in the January 16, 2013, issue

Ajay Kapur ’02, a professor at the California Institute of the Arts, develops robotic instruments. He stands beneath MahaDeviBot, which strikes 12 percussion instruments from India. The bouncing head can relay tempo to the human performer.
Ajay Kapur ’02, a professor at the California Institute of the Arts, develops robotic instruments. He stands beneath MahaDeviBot, which strikes 12 percussion instruments from India. The bouncing head can relay tempo to the human performer.

Computer technology might seem like the ultimate science and music the ultimate art, but Princetonians are in the vanguard of blending the two in a new and arresting harmony. About a dozen faculty members and alumni are changing the way we listen to music, the way we think about music, and, to hear them tell it, maybe even the way we learn and interact with one another socially.

Among them is Ajay Kapur ’02, director of the Program in Music Technology at the California Institute of the Arts in Valencia. Kapur plays the sitar, but it is unlike any traditional sitar you may have seen. His instrument, which he designed himself, is equipped with a USB jack and microsensors. As for his accompanists ... well, there’s Tammy, a 6-foot robot made of bells, strings, and marimba pieces; GanaPatiBot, a collection of drum heads backed by two speakers playing sounds from an iPod mini; and MahaDeviBot, which uses its 12 mechanical arms to strike bells, drums, and finger cymbals and can keep tempo by bobbing its, er, head. These and other robots are programmed to “listen” to what Kapur plays and respond to it: As he changes the notes or tempo on his sitar, their accompaniment changes as well, enabling man and bot to jam together.

Kapur started with a question at the heart of computer music — how do you enable a computer to improvise with a human? — and then founded a com­pany, KarmetiK (a fusion of the words “karma” and “kinetic”) Tech­nologies, to design and program robotic instruments that might provide an answer. What he has come up with so far gives classical Indian music (as his website puts it) “a modern electronic and experimental groove.” Last April, his KarmetiK Machine Orchestra show “Samsara” — which comes from the Sanskrit word for the cycle of birth, life, death, and reincarnation — played two sold-out concerts in Los Angeles’ Walt Disney Concert Hall.

The show was a multimedia robotic adaptation of Indian fables known as the Panchatantra. Working with CalArts theatrical director Michael Darling and Curtis Bahn *98, Kapur and six other human musicians not only played their instruments, but sent commands to the 10 remote-controlled robots in the Machine Orchestra. Meanwhile, dancers whirled and an animated film illustrated the story on a huge backdrop. The result was an extravaganza that used technology to retell an ancient story in a new and arresting way, and which the Los Angeles Times likened to “‘The Jetsons’ meets ‘Spinal Tap.’”

Kapur joined the CalArts faculty (he also teaches at the New Zealand School of Music in Wellington) after earning his Ph.D. at the University of Victoria in British Columbia. His biography describes an eclectic mix of coursework in computer science, electrical engineering, mechanical engineering, music, psychology, and media technology; his desk, which is strewn with polished wood, wires, and computer circuits — the guts of a harmonium he is building — suggests he draws on it all. He revamped the school’s computer-music curriculum and introduced a master’s program in music technology. The undergraduate department has 38 students, more than three times as many as when he arrived in 2008. “It feels like Princeton here now,” he boasts, “because all our kids are so amazing.”

Princeton’s influence in the world of computer music is wide and deep. Ocarina, SLOrk, PLOrk, and most other computer-music applications run on ChucK, a software language Wang developed for his dissertation with Perry Cook, a Princeton professor emeritus in computer science. “ChucK lets you program on the fly,” explains Cook, who also taught Kapur.

Over the years, Cook’s former students have helped spread computer music around the country. Kapur ticks off a long list, among them: Georg Essl *02 at the University of Michigan; Tae Hong Park *04 at New York University; Colby Leider *07 at the University of Miami; George Tzanetakis *02 at the University of Victoria, British Colum­bia; and Bahn at Rensselaer Poly­technic Institute. “We all come from Perry,” Kapur says.

The Princeton Laptop Orchestra rehearses in spring 2009.
The Princeton Laptop Orchestra rehearses in spring 2009.

Cook co-founded PLOrk, the first laptop orchestra in the country, in 2005, along with Professor Dan Trueman *99, Wang, and Scott Smallwood *08, now an assistant professor of composition at the University of Alberta. Today there are more than 60 laptop orchestras in colleges and even high schools around the country.

In a typical laptop orchestra, a dozen or more students control “instruments” consisting of the laptops and a variety of controllers and multichannel hemispheric speakers made from wooden salad bowls with car speakers inserted in precut holes. Depending on the musical composition, students might produce sound by tapping on their keyboards, waving handsets from Wii game consoles, or pulling strings. But the laptop’s (or phone’s) capabilities also are used in other ways: Rolling the device from side to side, for example, distorts the sound by triggering the accelerometer, a built-in component meant to protect the hard drive.

Student performers write code and program their laptops as well as play them, following a conductor’s lead but also communicating with each other visually, aurally, or electronically. Wang even has experimented with compositions in which the performers play in different cities, synthesizing their sound and communicating with each other by modem.

As Wang’s website puts it, a laptop orchestra “fuses the irreplaceable human aspects of music making with the computer’s precision, possibility for new sounds and for the fantastical automations to provide a radically new type of ensemble, research platform, and classroom.” Or as Perry Cook says: “The main purpose of PLOrk and SLOrk was to ask a lot of questions.”

Almost everyone making music today relies on computers, whether for recording, editing, amplifying, distributing, or listening. Hip-hop and R&B artists use software to distort their ­voices, while programs such as Auto-Tune digitally correct the human voice to produce sounds cleaner than any vocalist could make them. If humans are social, expressive, emotional, and intentional, Wang suggests, computers are precise, consistent, repeatable, networkable, logical, and tireless. They also have much greater storage capacities than we do.

Post Comments
Tell us what you think about
Musical machines
Enter the word as it appears in the picture below
By submitting a comment, you agree to PAW's comment posting policy.
CURRENT ISSUE: Jan. 16, 2013
Web Exclusives
Ge Wang *08’s Magic Piano app
Watch and listen to Ajay Kapur ’02’s creations
Music Issue playlist
Download music by artists featured in PAW
The Music Issue