October 1995
by Tod Machover
My deepest goal has always been to create art - or to support anyone creating art - that has a chance of transforming people's lives for the better. This kind of transformation can be something simple (like becoming more stimulated, enthusiastic, or especially aware and sensitive), or something more comprehensive (like a reevaluation of what is fundamentally important in life, and a rejection of the superficial). This may sound a bit naive these days, when most art has become "entertainment," but I have always believed that it is possible to stimulate such reaction, and that one should strive for no less.
I have also always believed - since I was a kid - that
the deepest and most meaningful experiences are those
that somehow combine and synthesize EVERYTHING
that is part of being alive, exclusive of nothing. In this
way, I am part of the very American musical tradition
of people like Charles Ives, John Cage, Elliott Carter,
John Coltrane, and Bob Dylan: people who tried to
convey the complexity and richness of life as directly
as possible, without oversimplifying it.
I grew up with such ideas around me, and with a
natural test bed for studying how to put diverse
life-views together: my mother is a pianist, humanist,
and embodiment of the European fine art and
intellectual tradition, who has spent her career
developing methods for teaching musical creativity -
and love of music - to children (her book on this
subject will be released in 1996 by Oxford University
Press); my dad is one of the pioneers in computer
graphics, an engineer who early understood the ability
of images to simplify complex information, and who
also is a deep believer in popular culture and the
positive, transformative power of technology.
So my work has always tried to come to terms with
these paradoxes, striving for unity by using the most
diverse sounds and emotions. I have always tried to
bring together worlds that don't seem to quite belong:
popular and serious, acoustic and electronic,
straightforward and mysterious, active and
contemplative, humanistic and technological. In fact,
ever since I was a kid I have believed that there is more
similarity than difference between people, and that it
is possible to convey the most serious and significant
messages if you just talk to people in a simple,
unpatronizing, confident and intelligent way. As our
world becomes more complex, with increasingly
dazzling amounts of information to absorb and the
resurgence of petty misunderstandings and
intolerances between people, I find this theme ever
more important and essential.
Technology has always been for me a way to exaggerate
the differences between things, as well as to create a
larger context - a kind of viscous, amniotic liquid - in
which diverse ideas and feelings could find a context
and a medium in which to communicate. I got to know
technology as a kid, visiting my dad's company and
watching the early experiments in man-machine
interface.
As a teenager (after years as a die-hard classical
cellist), I formed a rock band and experimented with
tape-recorder-type manipulations of sound (I had
also grown up listening to an incredible assortment of
music: from Cage to Subotnick to Boulez to Beethoven
to Coltrane to Bikel......no pop, no opera, everything
else). At Juilliard in the mid-70's, I became interested
in computer music and got someone to teach me, a
VERY unpopular thing to do at the time, as the 60's
synthesizer era was long over, and computers were
decidedly un- hip, at least at orchestral-oriented
Juilliard. But I learned FORTRAN, wrote music with
punch cards, and heard the result of my programs
after a week's delay!
At the beginning, my interest in computers was to
achieve greater musical complexity, in fact to "hear"
instrumental music I had been composing that was
unbelievably difficult to perform (like everyone in a
different tempo). In 1978, I got invited to Pierre
Boulez's IRCAM in Paris, first as a composer-in-
residence, and then to stay on as director of musical
research.
Being at IRCAM was a technological revelation to me
in two ways. First, I arrived there - luckily for me -
just at the time when the first real-time digital
synthesizers were being tested by Giuseppe de Giugno,
the first machines fast enough to give musical results
immediately with no wait whatsoever, and I was
fortunate to become the kind of musical guinea pig for
this project. As a serious performer (cellist and
conductor), I had always felt that computer music
would not soar until the computer could be
manipulated like an instrument, responsive to the
subtlest gestural and emotional response, and the
questions raised in 1978 about how to make such a
thing possible have motivated my work ever since.
Second, I had been trained in the Western classical
tradition, capped off at Juilliard, where the ideal to
strive for is Beethoven....deafness and all! That is to
say that a composer was meant to imagine all musical
ideas inside the head, in total perfection, with no input
from the outside world; composers who plinked out
tunes on the piano were condescended to as sort of tin
pan alley types..."real" composers did not NEED the
piano. The idea was to finish this perfect, imagined
score, write it out briskly a la Mozart in pen and ink,
deliver it 10 minutes before the performance to the
orchestra, and sit back as the premiere unfolded
impeccably, reproducing exactly what one had
imagined in the inner ear. No revisions afterwards, of
course!!
Now I am really lucky that I DID receive such a
rigorous training, and I still do much of my best
musical thinking sitting in an armchair at home, eyes
closed, dreaming and imaging what I want my music
to be. But at IRCAM, for the first time I had access to
di Giugno's real-time synthesizers which allowed me
to TEST what I had in mind, to sculpt my orchestra
with my fingers and hands, to play with my
composition as it developed, and -perhaps most
importantly - to then change my musical materials
(the instruments, sounds, interactions, whatever!) as
my ideas developed - simply by programming and
reprogramming the computer.
These ideas - "playing" a machine with physical
gesture and intuitive action, moving back-and-forth
between imagination and materials, and
experimenting with an "orchestra" while inventing
and composing - remain fundamental changes that
computers have brought to music, accessible in 1978 in
only the most arcane research institutions, but now
available to anyone with a MIDI studio.
Between 1979 and 1982, I composed the first music for
real-time digital synthesizer, culminating in a piece
called "Fusione Fugace" premiered at the Venice
Biennale. From 1982 to 1985, I experimented with
greatly expanding the musical palette of available
sounds, creating pieces that combined acoustic and
electronic instruments seamlessly, attempting to
replace traditional harmony with tension and release
patterns based on sound color changes, from melody
to harmony to spectrum to noise and back.
I came to the MIT Media Lab in Fall 1985, right when
it opened, and got active in the MIDI world as
suddenly computer music technology became widely
available. At this point, I started the Hyperinstrument
project. The idea of hyperinstruments was to develop
computer systems that could monitor and eventually
"understand" every nuance of musical performance,
so that the musician's interpretation and feeling could
lead to an enhanced and expanded performance. My
idea was always to try to capture the most complete
and integrated sense of the musician's meaning and
intention, rather than to collect a set of unrelated
"parameters" from performance which could then be
"mapped" to independent features of a synthesizer or
automated composition system. I always want the
musician to imagine a musical result in its totality, to
use highly developed musical skills and talents, and
then to have the machine do the work to translate this
into a desired and predictable result.
For this reason, my hyperinstruments between 1986 and
1991 were all designed for highly skilled performers
who could master all the nuances and subtleties
needed to control such systems. I designed
instruments, and wrote music, for virtuosi such as
Yo-Yo Ma, Kim Kashkashian, the Los Angeles
Philharmonic, and various rock and pop musicians.
Hyperinstruments were designed for keyboards, guitars,
string instruments, conducting and hand gesture, etc.
One of main interests in this work was to explore
HOW an individual's expressive intentions could be
enhanced in a meaningful way, as if the computer was
reading one's mind and filling in the blanks in a
totally natural way. I have always believed that
humans should be in CONTROL of such systems, and
that the computer result should reflect our goals and
desires, either our conscious ones, or - increasingly -
the subtler ones that we may not even be totally aware
of!
Since 1991, I have become increasingly interested in
developing hyperinstruments that could allow
"ordinary" people to participate actively in
music-making, or at least in listening. My goal here
has been consonant with my beliefs stated at the
beginning of these comments: that any normal,
intelligent person is capable of far more sensitivity and
creativity than she is normally given credit for. This is
not a demogogic or naive argument that any person
could be a Beethoven if given the proper "tools" -
such a statement, all too common in this era of
interactivity, is total nonsense.
It is very difficult to create things, and only a few
people have the insight and talent to make things that
are truly unique and universal. However, everyone is
capable of expressing their own unique qualities and
special experiences, and doing so enriches life. Also, we
have all become passive consumers of art and
entertainment, a far cry from the 19th century when
all played piano and tried out new scores by
sight-reading on Saturday evenings, or even earlier
when everyone sang in weekly religious services.
A first goal of interactivity today is to restore this sense
of participation and direct experience to audiences, for
music and others activities. Music is a great place to
start, since it actually has such a long tradition of
amateur community music-making, found in
cultures around the world.
The goal of our recent hyperinstruments for amateurs
is to enable such active music participation to take
place in a significant way. We have designed a whole
series of such instruments. "Drum-Boy" allows
people to make complex rhythm compositions, by
having the computer automatically "answer" and
suggest music based on what you play, and also by
allowing you to "talk" to the computer by using to
adjectives to change the music in the way you want.
Unlike other similar systems, this one really seems to
work, and actually uses these adjectives to analyze any
music already playing, and to change it in the desired
way immediately....people seem to recognize their
adjectives! Yamaha has already integrated main
features of Drum-Boy into its new RY-20 series of
drum machines.
The next series of amateur hyperinstruments extended
similar principles to all aspects of music, including
melody, harmony, "texture," and orchestration.
Using the "seed music" concept, such systems allow
you to give the computer a fragment or "seed" of any
music you want, which is then automatically analyzed.
The computer then makes up more music -
indefinitely - similar to, but different from - what you
fed it. The trick is that the system is designed so that
the user can then "shape" and "steer" (much like a
driving game) the music as it develops, changing its
qualities and direction with intuitive commands, from
interfaces like videogame joysticks. This kind of system
is very interesting as a new kind of gestural
instrument, maybe as a kind of "music game," and
even as an automated soundtrack to a movie or
videogame, where the music must change
automatically in response to unpredictable turns in
the action.
Another goal of mine has been to make the interaction
with such sophisticated systems/instruments as
intuitive and natural as possible for the general public.
I feel that it is very important now to develop
interfaces that rethink the necessity of forcing people
to study a particular musical instrument for 10-20
years (so make easy PHYSICAL interfaces), but that
do NOT eliminate the necessity of concentration, skill
and imagination (so make challenging MENTAL
interfaces). Far too many interactive music systems
right now are, in my view, kinds of toys - they take no
skill to play, make you feel like a "master" after 5
minutes, but then don't allow you to get any better.
Such systems are a bad idea because they become
boring very fast, but also because they encourage
people to be lazy and complacent - feeling that they
are doing things that they aren't really doing - rather
than more sensitive and aware, the goals stated at the
outset here.
With our current hyperinstruments, I am trying to
prove that the latter is possible. It means, in my view,
simplifying the physical interface while redefining
completely the mental and imaginative control that
people can have over a music system. To do this, we are
trying to improve interfaces, while putting more
intelligence and sensitivity in the machine. With my
colleague Neil Gershenfeld, a physicist at the Media
Lab, we have developed a new technique of magnetic
field sensing that gives a very accurate measurement
of physical movement with no wires or funny gadgets,
the most sensitive technique developed so far for
allowing amateur conducting, danse- to-music
conversion, etc. We are developing systems that
measure the emotion in acoustic signals, allowing
anyone's voice to be used to control an interactive
environment. And we are moving towards systems
which measure not only instantaneous feeling or
mood, but have a sense of your goals and intentions,
allowing you to sketch out a musical "story," whose
details are filled in by your computational assistant.
Such instruments are requiring a totally new
definition of the concept of interactivity, and since
computer musicians have been working with
interactive systems for over ten years now, it is not
surprising that we are somewhat in the forefront in
helping to define this new field of interactive
entertainment. If you compare what is needed now
with my "Beethoven-the- ideal-creator" model
above, it is clear that what is needed now is a new role
for artists where a work is imagined not as a perfect,
totally finished creation which is sent out into a world
of passive consumers, but rather a conception of
creating works and situations which are partially
finished, partially opened, where the audience is
invited to explore, complete, and maybe totally change
what is offered to them. Such an idea implies that
listeners should be actively invited to SEARCH for
meaning, rather than to accept ready-made answers.
Such an interactive experience is VERY hard to
achieve, and I think that most experiments so far fall
short of the mark, either offering unsupervised
wandering (like the encyclopedia model), egocentric
fanzine attention (the
"learn-everything-you-never-wanted-to-know-about-the-artist"
school), or only superficial interaction (as in remixing
a couple of tracks of some rock song). One of my goals
this year is to create a whole series of short pieces, each
with a different interactive model, to find out what
kinds of things work best, and which give the user the
most sophisticated sense of interaction.
These separate interactive music pieces, which I
sometimes think of as "music games," will form the
core of my "Brain Opera" project (for which you will
find a project description tomorrow). The idea of the
"Brain Opera" is not only to present such a collection
of different kinds of interactive musical experiences to
the public (ranging from those that encourage
listening, to performance, to creation), but also to
organize all of them in a coherent, directed whole, a
total experience that allows a kind of cohesion to
emerge out of the actions of many individuals (the
most recent expression of my ideas stated at the
outset). Now, I am planning to present the first
version of the "Brain Opera" at the new Lincoln
Center Festival in Summer 1996, and then on to
Japan and Europe in 1997. In addition, the "Brain
Opera" will be an official project of the 1996 Internet
World's Fair, thus making it available online to people
around the world.
As for the "professional" hyperinstruments, I
completed my hyperstring trilogy last year, with
"Begin Again Again..." for solo hypercello, "Song of
Penance" for hyperviola and chamber orchestra, and
"Forever and Ever" for hyperviolin and orchestra. I
am starting work with Yo-Yo Ma to make a CD- Rom
of "Begin Again Again..." for Sony Classical release
next year.
I am currently working on a hyper-string quartet for
the Kronos Quartet, to be premiered next year. This
work will push the hyperstring technology much
further (hopefully involving technology which would
allow any acoustic string instrument to become a
"hyperstring"), and will also explore my theme of
contrasting and combining different cultures, using
the hyper- quartet as the transformative medium.
The quartet will be given a lot of interactive freedom,
and we are starting to design an interactive version of
the piece for the public where they will be able to
explore the piece and use its elements to create
original music.
Orchestras have been slow to find ways to integrate
technology in a comprehensive way - little budget for
experiments, conservative atmosphere. But I am
discussing a project with the National Symphony in
Washington, D.C., to create an orchestral fanfare
which the public can play with and modify during
intermission. And I am composing a totally
non-technological, acoustic orchestral work for the
Halle Orchestra, and conductor Kent Nagano, in
Manchester, England.
One of my more fun projects this past year was a short
25-minute opera that I wrote for magicians Penn &
Teller, called "Media/Medium". This piece
incorporates music, magic, technology, and
interactivity into something which is pretty unusual.
With my colleagues at the Media Lab, we designed a
"sensor chair," using the magnetic field sensors
described above. I turned this into a totally new
musical instrument that measures all body
movements and turns them into sound while you are
sitting in the chair. Movements are measured very
precisely, and can control anything from very delicate
"finger painting with sound" to a wild percussion
performance on 400 different drums. During the
opera, premiered at MIT in October and now a part of
Penn & Teller's touring show, this technological
marvel becomes a mysterious means for conjuring up
spirits - as mediums did in the 19th century - thus
exploring the fine line between magic and
technological marvel, between hype and plain truth.
This sensor chair instrument has attracted a lot of
attention, and discussions are underway with various
people, from symphony orchestra managements to
major rock stars to furniture companies, on
collaborations for the next version!
In closing, I would say that music is at a major
crossroads right now. Amazingly, technology -
through MIDI first and then "multimedia" - has put
music back at the center of attention. In fact, music
has developed faster than most other art/technology
fields, and this is cause for celebration. There is no
doubt in my mind that our main media for expression
in the future will be digital - there is no turning back
(even though now we are experiencing a kind of
shocked pre-millenial reaction to major change).
However, there are many dangers. Music is more a
part of our lives than it has ever been - we listen in the
car, while we are eating, while we are working, as
background to movies, CD-Roms, everywhere. But
therein lies a BIG problem: music has become
background, and many people have lost the
concentration to listen to music without doing
something else at the same time! In fact, I think that
one big task for composers in our generation is to
reinvigorate music - in a sense to reinvent it - so that
people are attracted to it for its own virtues, and not
just because it subliminally supports something else.
This is connected with another problem. Art and
entertainment swing pendulum-like between periods
of complexity and periods of simplicity; the most
interesting art always comes from periods somewhere
in the middle, when the package is as simple as possible
but the message is deep and demanding. In my view,
the pendulum has been swinging since the mid- 1980's
towards a kind of undemanding, simplistic art and
entertainment, originally a necessary antidote to the
public-ignoring "who cares if you listen" experiments
of the 1950's and 1960's, but now turned into a kind of
boring pablum, afraid to engage and challenge the
audience, and more and more peripheral to people's
lives. Pop culture has been much quicker in
significantly embracing new technology than
"serious" culture, and my fear is that this rift is
growing, and fast! - again emphasizing an outgrown
distinction between serious and popular - instead of
shrinking.
So I think that we must strive now for a music which
can stand on its own against the other media -
emphasizing its ability to create mysteriously deep
emotional and mental experiences, encouraging
listeners to "fill in the blanks" - while combining a
seriousness of purpose and depth of content with a
colloquial and non-elitist expression.
Our interactive technology should encourage this
goal, by creating situations which invite people to
participate in significant ways through eliminating
every unnecessary barrier, while making sure that the
experience offered enhances and expands life
experience instead of being a mindless drug.
So, the situation is in fact paradoxical. We DO have to
find increasing ways to make music a part of everyday
life, by building "instruments" of expression into our
everyday environment, into furniture, clothing, toys,
walls, everything - but must do this so that our lives
become richer, more REAL, and not synthetically
plastic and "virtual."
My fear is that people will prefer the latter until they
are shown that the former is possible.
To demonstrate this possibility - in our instruments,
in our environments, in our music - is one of the
principal goals of my work for the coming decade.
And while technology is incredibly important in our
future arts and entertainment, it is also important to
NOT mistake the tool for the meaning. Too many
people nowadays see technology itself as the
fundamental issue, becoming interested in a project
because of some fancy new techniques, or inversely
turning away from that same project because of an
aversion to technology.
This technology is there to help us deal with deeper
human issues - in my case, the establishment of
community between very different kinds of peoples,
and the encouragement of everyone to grow and be
stimulated, to face life with enthusiasm and optimism
and generosity. My most fundamental wish for the
future is that we do in fact use our technology this
way, and that more and more people view it as such,
and not as an end in itself.