The Brain Opera will premiere at the Lincoln Center Summer Festival in New York, from July 23 to August 3. Shows will occur hourly from 1pm to 8pm, each day. Information on how to get tickets will soon appear here. Between now and our Lincoln Center debut, we will be constantly adding new activities to this Web Site, enabling you to:

• prepare and send us music, sounds, text and images
• play realtime interactive music games especially designed for the Net
• shape, stretch manipulate and personalize "Brain Opera" performances, both on the Net and at theaters and concert halls around the world.

We are attempting a lot of things in this project that are pretty new - at least WE are still figuring out how to do them - and that's part of the excitement of the "Brain Opera." A group of about 50 of us - musicians, designers, visual artists, hackers, inventors, etc. - are working on the project here at the MIT Media Lab.

We are building an artistic experience that we hope turns out to be a lot more than the sum of our individual contributions. We'd really like you to join us in contributing to the "Brain Opera" from now until Summer 1996, and then beyond to the millenium. Let's discover how friends and strangers, near and far, online and face-to-face, can make surprisingly beautiful music together!

Tod Machover

2. Experience Space

FOREST

"The developing brain is like a forest within which many different creatures grow, in conflict and in harmony"

GESTURE WALL

Five larger-than-life-size Gesture Walls turn body gestures into sound and image. By stepping onto a plate mounted invisibly in the floor, a low-voltage electric signal is transmitted through the player's shoes and then broadcast through the body to a set of four sensors located on goosenecks around the perimeter of a giant projection screen. These sensors measure body movement very accurately, allowing the slightest twitch or most dramatic dance to modify and "perturb" ongoing music and moving images displayed on the wall.

HARMONIC DRIVING

Of all Brain Opera activities, this will be most like a traditional video game.

Three separate "driving" units will be contained in a circular, translucent kiosk. Players are seated in front of a graphic screen, with a steering wheel, joystick, and foot pedal as controls.

Using specially-designed software, the player literally "drives" through a piece of music in one of three Harmonic Driving units. Like playing a traditional video game, the user negotiates onrushing paths and trajectories on a video screen, using a specially designed, spring-mounted steering column that measures turn and tilt and bend. Branches in the road - signaled by blue and orange barber poles - cause the music to become "cooler" or "hotter," and the player's micro-steering - whether rhythmic and precise or sinuous and meandering - makes the music become sharp-edged or atmospheric. Both path and music are updated on the fly as the player makes choices, and the consistency of these choices - or "conviction of the interpretation," if you will - is rewarded by a skill rating at the end of the run/piece.

MELODY EASEL

There are three Melody Easels, each designed to accomodate one player and two observers. Seated at a table-like structure, one plays the Melody Easel with the touch of a finger, creating a feeling of subtle articulation and phrasing usually associated with single-line melodic instruments like the violin or saxophone. The movement and touch of the finger as it travels across a specially designed transparent surface not only creates music -- melodic variations and timbral filigrees based on the voice of mezzo-soprano Lorraine Hunt -- but also a beautiful graphic trace.

RHYTHM TREE

The Rhythm Tree is a sculpture with more than 300 networked drum pads mounted on seven giant, circular pods which stretch throughout the Mind Forest. Each individual pad is hand-molded in translucent rubber, and equipped with a special microprocessor and LED display. There is enough intelligence on each pad to allow variation in touch - from the slightest tap to the most powerful pounding - to produce a wide array of sonic results, all controllable with familiarity and skill. Each pad is also connected to many others, like branches on a tree (or synapses in the brain). As the player strikes them, different signals are sent ricocheting through the connected circuits, creating an ever-changing variety of sounds and images.

Audio Submission
This is a call for your input to the Brain Opera! We are collecting as many sound-bites as we can from a wide range of sources, and would like to invite you to participate as well. We will incorporate the material you send into the third movement of the live Brain Opera performances, as well as in future on-line incarnations. You can prepare your material in advance and send it to us at any time.

Performance Space

Visitors can stay in the maze-like lobby space as long as they like. However, every hour there will be a "performance" of the Brain Opera, which everyone will be invited to attend. Each performance will have certain common features, but will also be unique because of the contributions and "personality" of that particular audience group. It is here that the separate fragments of the Brain Opera come together to form a coherent artistic experience.

The Performance Space will be set up like a night club: people will be able to sit at the periphery of the room, or move and dance at the center; images and sounds will be projected on a curved surface at the front of the room, as well as on the floor and ceiling.

Each performance will last approximately 45 minutes and will have three distinct parts, one flowing into the next:

• Part 1 will be a Sonic Blitz, as recorded voices of audience members are mixed into a dense sound collage, turning words into beautiful music.

• Part 2 will inject snippets of music produced by audience members, colliding and overlapping, and finally combining and aligning to create fascinating musical mini-pieces.

• Part 3 will bring together all of the musical themes and layers of the Brain Opera, getting ever faster and more intense, as the music builds to a crashing and cathartic climax.

In addition, sensors in the room's carpet and radar beams at head level will measure audience movement and energy level during the performance, using this information to automatically influence the musical build-up and formal shape of the piece.

The Brain Opera is a project founded on interactivity between all its performers. Each and every single one of you out there is a performer!

We welcome any comments or suggestion ! Feel free to write to us!
Please join our mailing list so you can stay up to date on every new development at the Brain Opera.

"What magical trick makes us intelligent? The trick is that there is no trick. The power of intelligence stems from our vast diversity, not from any single, perfect principle. Our species has evolved many effective although imperfect methods, and each of us individually develops more on our own. Eventually, very few of our actions and decisions come to depend on any single mechanism. Instead, they emerge from conflicts and negotiations among societies of processes that constantly challenge one another."
-- Marvin Minsky, from The Society of Mind

The Brain Opera will be loosely based on Marvin Minsky's seminal book, The Society of Mind, in which he proposes that human personality is not controlled by a centralized "conductor" in the brain, but rather emerges from seemingly unintelligent and unconnected mental processes, or "agents." With Minsky's theory as a metaphor, participants will reach a new sensitivity to the many different parts of the mind that are engaged when we enjoy and respond to music.

And we hope that a new kind of music will emerge from this project, representing a middle ground between the rigid rules of the classical tradition and the anarchy of the Cageian avant-garde.

The Brain Opera is not structured around a "story," as in traditional opera, but around an all-encompassing emotional and psychological experience in which the audience - both real and online - will be drawn into the mysteries of music and the human mind, to discover and explore the interplay of sensory perception, musical structure, language, memory, thinking and emotion, and actively take part in creating a multimedia work of art in which the composer's musical concepts will be enhanced and transformed by material emerging from the individual and collective contributions of the audience.

We expect the audience/participants to experience this opera at many levels (engaging the diversity of Minskian "agents" in their minds):

• as a powerful and engrossing artistic experience

• as a stimulating set of interactive music games

• as an exploration of the mind, and of the nature of artistic and creative experience

• as a demonstration of the positive capacity of technology to enhance personal expression, to encourage interpersonal collaboration, and to build community

• as a powerful, joyous and unifying journey, with cohesion emerging from complex fragments

• as a new synthesis of art, technology, psychology and science

Below are several short text examples, taken from interviews with Marvin Minsky which were conducted by Tod Machover between 1993 and 1996. These texts, about the mind and music, have provided much of the textual material for Tod Machover's vocal compositions in the Brain Opera, and therefore comprise a kind of libretto. These text examples are provided below:

1. The mind is too complicated to summarize.
2. There are probably several hundred little pieces of brain that each of us uses.
3. There's no instruction manual and no way to debug it.
4. You never do anything in just one way.
5. It wasn't designed very well by any organized process.
6. That comes from 400 million years of evolution.
7. The things that look simple to us are usually very complicated, and the things that look complicated are usually much simpler.
8. Music is a very mysterious thing.
9. Scientifically there don't seem to be any good theories of it.
10. We don't know where it came from.
11. You don't see animals tapping their feet to rhythm.
12. Its probably one of the most intricate activities we have.
13. It seems like its meaningful.
14. You can't help relating it to things.
15. What I'm saying is its not a natural thing.
16. Every part of your brain has to make sense of it.
17. Its got phrases and clauses and sentences.
18. What story is this like?
19. Your memory has to find something!
20. Sometimes the associations are very compelling.
21. What right does Schumann have to order me around?
22. How come people don't get angry when music affects their emotions?
23. Some people use it for influencing crowds and other people use it to make people think more and get away from the crowd.
24. You only live a hundred years.
25. People have no respect for their minds.
26. Especially from thinking about music.
27. ....how to think, how to use the time, how to recognize how much time you're using......
28. One thing that music does is it makes you able to think three or four things at a time.
29. There's a lot happening.
30. This is very important.
31. The things that we call intelligence are nothing more than the ability to manage time better.
32. Thinking about the mind might sensitize people to how silly it is to spend a day thinking nothing at all.
33. Are there people who are more creative than others?
34. What a genius is, is just someone who has been very lucky.
35. To be a really good composer you need ten or fifteen tricks.
36. But everybody has most of these things.
37. Everyone is immensely creative.
38. It just seems a lot better because you can't do it.
39. We know a lot about emotions.
40. We don't know much about thinking at all.
41. Each emotion is just a kind of thinking.
42. Music changes your mood very directly.
43. No one knows; no one studied this very well.
44. What's going on?!
45. People don't study this; people don't ask that question.
46. Why do we like music?
47. Its taboo.
48. Isn't it strange?!

The Story of the Brain Opera is about its history as a project, and the ideas and texts on which it is based. Its roots in the thought of Marvin Minsky, its progression during the past three years, the successive stages of its iterative design process, and the team of people who have shaped it along the way. From this page, you can link to information about Marvin Minsky's "Society of Mind," a welcome from our director, bios and web pages of the individuals on our team, the Brain Opera "libretto," and an archive of material related to our project. The contents of the archive include the work of related research groups at the Media Lab, architectural drawings of the Brain Opera set, technical designs for the experiences, articles by Tod Machover, and theses by his students.

The following is excerpted from "The Brain Opera and Active Music," by Tod Machover:

Thirty years ago, the great pianist and essayist Glenn Gould published an article on the future of music recording (High Fidelity, April 1966) in which he said: "In the best of all possible worlds, art would be unnecessary. Its offer of restorative, placative therapy would go begging a patient. The professional specialization involved in its making would be presumption...The audience would be the artist and their life would be art."

As someone with many of the usual "professional" music credentials, I was surprised on rediscovering Gould's article recently that my own work in music and technology has evolved in exactly this direction. In fact, I now believe that the highest priority for the coming decade or two is to create musical experiences and environments that open doors of expression and creation to anyone, anywhere, anytime. To accomplish this without producing numbing background music - but music that enhances the senses and stimulates the mind - is the real trick! I believe that such "active music" could be one of our most powerful tools for discovering the unity and coherence that underlies the chaos and complexity of everyday life.

My view of technology has always been that it should respond to human intentions, rather than simulate or replace them, and I started developing Hyperinstruments at the MIT Media Lab in 1985 towards this end. The first generation of hyperinstruments was designed for virtuosic professional musicians, such as Yo-Yo Ma. These hyperinstruments measured many nuances of performance expression, using this information to enhance and expand the instrument's capabilities. Starting in 1991, we began building hyperinstruments for non-professional music lovers. Our Joystick Music system allows a piece of music to be steered, modified, and shaped by manipulating two videogame joysticks. A Sensor Chair, designed for magicians Penn & Teller, uses an invisible electric field to detect body motion and turn it into sound. Such instruments are easy to learn but difficult to master, with enough depth to make them worth practicing and exploring.

Central to the Brain Opera idea is the work of Marvin Minsky, a colleague of mine at the MIT Media Lab, and someone I have known since my student days at Juilliard in the mid 1970s. Although there are obvious similarities to be found on every level of the Brain Opera to Minsky's philosophy of mind -- and indeed the "libretto" of the opera is taken from interviews I have done with him over the past two years -- the connection to Minsky is deeper and more subtle than that. Marvin Minsky is the first person I ever met who dared to ask questions about music so basic that they seemed naive, yet so perceptive that no one has yet answered them. Why do we like music? Why do we spend so much time with an activity that has little or no practical benefit? Why does music make us feel? And think? And are feeling and thinking the same? Is music the activity that most deeply unifies our complex selves?

With Minsky, such questions lead to a whirlwind of speculations about where music comes from and what it tells about us as human beings.

Perhaps music is one of our capacities acquired latest in evolution, and so is "messiest," having had to share its footprint with numerous mental agents already entrenched. Perhaps because of this, music has no brain center all its own, but rather touches a very large number of other mental functions ("gut" feeling, storytelling, mathematics, movement, speech processing, etc.) and somehow synthesizes them. Perhaps music allows us to experiment with thinking, in a liberated fashion because the results of such "musical thinking" have no repercussions in the "real world." Perhaps -- as John Cage might have said -- music is a way of preparing our minds and our personalities -- to finally throw away music altogether and experience the world directly.

We have designed our Brain Opera experiences to stimulate audiences to reflect on such questions, and on how the independent fragments and layers of music come together to form complex yet unified sonic images. And one of our deepest hopes for the Brain Opera is that it will encourage people to be excited by their own minds, and by the desire to "look inside and hear what is going on" (Minsky).

It is this kind of audience involvement -- not the mere manipulation of our hyperinstruments -- that makes the Brain Opera truly an "opera." Although the work does not have a linear narrative, which I have avoided at every step of the design process, it certainly has LOTS of voices - professional and amateur, singing and speaking, individual and communal - and the whole texture is actually very vocal, even "operatic." More significantly, the Brain Opera does have a significant dramatic progression, which is the voyage of each audience member through the maze of fragments, thoughts and memories, to collective and coherent experience. Just the process of understanding the scenario of each instrument -- how it is played and what it means -- and seeing how these turn into full musical structures in the performance, is a very rich and involving story in itself.