Resume Reading — Ingenious: Jonathan Berger


You've read 1 of 2 free monthly articles. Learn More.


Ingenious: Jonathan Berger

The composer on transforming noise into music.

I was electrified by Jonathan Berger’s music before I knew he wrote about music. His chamber works arise out of a lightning storm…By Kevin Berger

I was electrified by Jonathan Berger’s music before I knew he wrote about music. His chamber works arise out of a lightning storm of modernist angles, dramatic and startling, though anchored to melodies that sail like a swallow, as one of his string quartets is called. His one-act operas Theotokia and The War Reporter, performed together in concert, match taut musical brocades to the hallucinations of, respectively, a schizophrenic, hearing voices of various mothers, and a photojournalist, based on Paul Watson, who won the 1994 Pulitzer Prize for his image of the corpse of an American soldier being dragged through the streets of Mogadishu.

A few years ago, I read some of Jonathan’s academic writing about music, which had a sharp focus on neurology and acoustics. He is a professor of music at Stanford, where he teaches composition, music theory, and cognition at the Center for Computer Research in Music and Acoustics. On a hunch that he could connect with a popular audience, I asked him to write an essay for Nautilus about how composers upend expectations to keep listeners off guard and engaged. That article, “Composing Your Thoughts,” and his next one for Nautilus, “How Music Hijacks Our Perception of Time,” which contain musical clips to illustrate his points, have been among our most popular articles.

There’s a certain amount of problem solving that happens in the context of a band of noise.

For this month’s issue I called Jonathan and was delighted to learn he had thought a lot about noise. Noise, as it does with all of us, drives him around the bend; his worst-case scenario is airports. But he had also thought a lot about how composers had folded environmental sounds and noise into their works, reflecting their times, and remolding music itself. It’s a practice Jonathan himself uses. Swallow recreates the ways the swift birds communicate, which sounds to humans, Jonathan points out, like chirps, whines, and gurgles. As we talked, Jonathan and I got to wondering if what composers have done with noise is similar to what our brains do with it—transform it into something tolerable and even pleasurable. Jonathan was excited by the idea and the result is his essay, “How Noise Makes Music,” again featuring a bevy of musical clips, this time from Verdi to Chinese pop.

Jonathan’s as busy as ever these days. This year he was awarded the prestigious Rome Prize, and a Guggenheim Fellowship, and is soon off to Rome to compose a work for voices and orchestra. Before he left, we sat down to talk about music and the brain and how we manage to get by in our increasingly noisy world.

The video interview plays at the top of the screen.

View Video

The Necessity of Musical Hallucinations

During the last months of my mother’s life, as she ventured further from lucidity, she was visited by music. In collusion with her dementia, her hearing loss filled her consciousness with musical hallucinations. Sometimes welcome, more often not, her musical...READ MORE

Interview Transcript

Why does noise annoy us so much?

That’s a complex question. Let’s first distinguish between listening to white noise under conditions where you would prefer to have quiet, which is sort of a normal state; or being in Yosemite, and being near Vernal Fall, and hearing the same noise coming from the waterfall, and thinking, “This is magnificent. This is so calming.” There’s a huge amount of emotional valence, and a huge amount of content that defines and distinguishes between how we deal with noise.

I think there’s a temporal aspect to it also, that noise, again, very quick, very short outbursts of noises are incredibly upsetting, and very long, prolonged noises that obscure clarity are very upsetting. There’s a lot of in between, though. There’s a lot of play in between that.

Acoustically speaking, what is noise?

Well, noise is a confluence of, or a convergence of all, of many wavelengths. Depending on how it’s filtered, it can be some more than others, but it’s a confusion, because the signal is so complex that we can’t distinguish periodicity. Now, periodicity, the way we distinguish periodicity happens further down the line, once the sound gets into the cochlea, but what happens at that outer ear is a resonance that amplifies certain parts of the sound.

Do we avoid the anxiety of noise by trying to find something pleasurable in it?

Let’s go back to Yosemite for a moment. I’m in a tent, and it’s very windy outside, and I hear the crunching of the leaves, which has just added noise to it, and I need to distinguish whether that’s the footsteps of my partner, or whether that’s the footsteps of something that’s going to eat me alive. We’re good at that. We’re actually remarkably good at that. So yes, I think this idea of being able to analyze and contextualize what’s happening with a noise is important.

Noise is music if it’s used in an organized way.

The other aspect is speech recognition. Our auditory systems are so fine-tuned toward speech. If you think about it, the ear is such an amazingly complex and yet simple mechanism. The first, the outer ear is what directionalizes sound. Its angularity and its differences pose filters, and they pose temporal distortions, but then the next part of the outer ear is the canal. The canal is just the length needed to amplify 3 kilohertz. Now, 3 kilohertz is a high, 3 to 5 kilohertz, that range, that’s high frequency. So there’s a little resonating tube, so it’s as if we had a little toilet paper roll. Before anything happens, before it turns into, before the signal gets transferred into the fluid-based cochlea, before the cochlea does its work, certainly before it turns into neural impulses, pre-cortical, before anything mechanical happens, sound goes through this little tube that gets, where a certain resonator happens, and that resonance amplifies just that critical area—which is way, I mean it’s higher than most musical sound, certainly higher than any speech. It’s the area that’s critical for understanding, for distinguishing between phonemes. It’s a speech critical analyzer that happens in its simplest way before there’s any mechanics. I think that tells us a lot.

Is how we deal with noise in everyday life anything like what a composer does with music?

Let me point out that noise is, in the sense of drumming or banging on things, probably precedes any other, “musical signal,” except for the voice itself, so we’ve had noise as a component of musical communication as long as anything.

One answer is, what I said before, that I think that there’s a certain amount of problem solving that happens in the context of a band of noise. We’re trying hard to analyze it and figure it out, and if we manipulate it, if composers manipulate it in the right way, that becomes a sort of pleasurable experience because we’re trying to link things, we’re trying to read what’s happening underneath it, we’re inferring what it’s leading to—so it’s part of this chain of events that’s all about anticipation and building expectations.

The other piece of it, though, is that the ear is mysteriously and magically comprised of two components. We talk about the cochlear component, the component that’s actually dealing with the transformation of sound into a signal that goes to the auditory cortex, but the vestibular mechanism, that gives us this sense of balance, also has an auditory component to it, and so when we listen to very loud, dramatically loud music, very low basses, driving basses, our ear is working on both, both mechanisms are at play.

What do you mean that noise is about anticipation?

Yesterday, as I was trying to clarify things for myself, the phrase “drum roll, please” came into my head, and that was actually, that was an “a-ha” moment because, you think about a drum roll. A drum roll, and particularly with the cliché of “drum roll, please,” which is anticipatory. It’s a sound that leads into, that prepares you for something. It usually prepares you for something dramatic. Drum rolls are there before the guillotine. Drum rolls are there before prizes are awarded on TV. Drum rolls are there in classical music before some big change happens that leads you into something. There’s this anticipatory drum roll, and the interesting thing about the drum roll is that it can be arbitrarily long. It can’t go on forever, but drum rolls can be pretty long. They go (drum roll sound), and you can go on a long time before you make that final hit, and what’s happening during that period is this very obtrusive noise. That’s the consummate noise band, a snare drum, maybe with cymbals or gongs being rolled there. It doesn’t get noisier and more complex and confusing than that, and that’s what we do to lead us in, to build our anticipation, toward something. I think that musically, that might be the key to how this all works.

What’s the difference between noise and music?

The philosophical answer is Varèse’s definition of music, which is organized sound, so that noise is music if it’s used in an organized way. The Cagean answer, the John Cage answer, is noise is music, period—whether it’s structured or not. Actually I think that Cage actually brings us back to Leonardo, which is you listen to static on the radio, or you listen to 3 minutes and 14 seconds of, 4 minutes and 33 seconds of silence, and that becomes the piece itself, right? So, philosophically, there’s not necessarily a distinction. Now, old fart composer that I am, for me, music is really about expressivity and communication and lyricism, and so noise becomes a partner in the scene. It is actually a component of it all.  It’s used to distort, to willfully distort sound. It’s used to bring in a sense of confusion. It’s used to bring in a sense of anticipation. It’s used to punctualize and punctuate but it’s an element of music rather than music in itself, as my colleagues would say.

What questions are you asking as a composer?

I was asking, at first, big questions. The two big questions that plagued me—I still to this day call them the black holes of music theory: One is musical expectations. Musical expectations seem to be so fundamentally central to western music at least, if not most music. There’s no language. There’s no parlance. There’s no metric. There’s no way to quantify or qualify musical expectations. This has been plaguing me.

Noise is in the ear of the beholder.

The other one, and this gets into the acoustic side, is timbre. The color of music. Here again, it seems so central to what western music is about and indeed, most music. The only terminology we have for timbre is adjectival. It’s cross-modal. We think of sounds that are cold or hot or rough or smooth. We currently have no metric for it. A lot of my grad students are working on trying to quantify differences in timbre. That was my entry way to acoustics.

What do you mean by “musical expectations?”

When we listen to music, we’re thinking ahead. We’re constantly anticipating what’s going to happen next. That’s the way our minds work. Daniel Dennett said the mind is an “anticipation machine.” In my mind, music is really a way to test that, a way to sharpen and hone in on that. It’s a lot of the reason music exists. Composers are constantly playing off of building expectations and processing those expectations. Realizing the expectations or violating the expectations. When they violate the expectations in different ways. Again, there’s no real language about how expectations work. I’ve come to think about expectations in terms of the strength and specificity of an expectation.

If you think about the correlations between these two, it opens up this world of thinking about how a composer or a rhetorician will pose an argument, will build it, will present clarity, and then interrupt that clarity. In fact, one of the big ah-ha takeaways that I’ve had—I think I actually I wrote a bit about this in another Nautilus article—is that we think of expectations, we think of western music rhetorically. We think of it in terms of, here’s this clarity and here’s where I’ll violate the clarity. Here’s where I’ll throw a monkey wrench into the works and confuse you a little bit. That confusion is always aimed at strengthening the argument at the end. That’s sort of the idea of rhetoric. Music, of course, in the 17th, 18th century was the embodiment of the purest state of rhetoric.

How is music an embodiment of rhetoric?

Certainly as the 19th century goes along, and it started well before that, composers played with the idea of starting without clarity, of starting in a state of vagueness. Starting in a state of not giving enough information or giving too much information. This is where noise comes in. Having this sense of floating, of not quite knowing what you’re doing; and then what we as listeners do is we try to build scenarios. The antithesis of this rhetorical model where I give you a sense of clarity and then I distort is this other thing. I don’t even have a good name for it. I’ve tried calling it other names but the concept is, it’s a puzzle-solving approach. You start with a problem, you start with vagueness, and then over time you control how clarity comes into play. There are many composers, Brahms was the master of starting ambiguously. Bruckner with his beginnings of long tremolo that don’t settle in on a chord, that don’t settle in on a direction, that have no meter. That leaves you hanging for this long period of time.

What inspired your puzzle-solving approach to composing?

There was a period when I started teaching and I was composing. I had a long commute and I was rereading all of the books that I should’ve read in high school but always skirted around. One of them was The Sound and the Fury. Faulkner starts The Sound and the Fury with this—for me it was a life-changing moment of reading; I remember reading it in the bus going ... Because it starts very much like Bruckner. It starts by giving this description of a golf game as seen through the eyes of a mentally disabled person. You hear about flags. Nothing is clear. Over time it becomes clear. It takes pages. As it goes on, you realize that this isn’t really central to the story. These are the pieces of the story that come together. It occurred to me that if anything that the 19th century did as we enter the 20th and now the 21st century, it’s distorting; it’s moving away from this set concept that music is about rhetoric. Music is about giving you an argument, posing a problem, resolving it.

We tend to loosely ascribe noise to anything that we don’t have the patience to sit and comprehend.

What sparked your interest in science?

I think, like many people of my generation, Gödel, Escher, and Bach was this spark. It was a moment that sprung forward here. Again, I’m a latecomer to music, so a lot of these answers are actually transposed from my youth to much later in life. The other was my mentor, John Chowning, here at Stanford, who became fascinated by applying simple mathematical or arithmetical principles, like the Fibonacci series, to music at all levels. What it made him do, and what it made me do looking over his shoulder, was look at what it means to take a series of numbers that have specific traits and values, both structurally on the large scale and in terms of acoustically when you bring them out and synthesize them, what it means to tie these things together, how a structure or a series or data can instantiate itself on any level of cognition.

How have you transformed noise in your own music?

Really, you can find an example in any of my music, but another example that comes to mind is my opera Theotokia, which is about auditory hallucinations. It’s about a person suffering from schizophrenia, who has these deep hallucinations and the way the piece works ... The protagonist is never heard until the very end of the opera and what you’re seeing are his hallucinations. What the audience is hearing are ... I mean they’re hearing the hallucinations some but they’re also hearing through an electronic, an ambient electronic situation. They’re actually hearing a mapping, a realization of the actual brain scans of the hallucination, of the onset of auditory hallucinations. So, they’re actually prepared with this noise that says a hallucination is coming, although I don’t spoon-feed that. I don’t tell them that and at the key moment of the opera where I wanted the protagonist to emerge for the first time and make his first sound, I didn’t want it to be vocal. He vocalizes only at the very end and so it turns into this massive percussion solo where he then does this rhythmic drumming on stage and it’s this band of noise but what comes out of it is his obsession at the end. So, again, it’s noise used dramatically as opposed to noise used anticipatorily.

Why does some music sound like to noise to listeners?

That’s a fantastic question. Right, so, that actually brings up the third issue of noise, which is, noise is in the ear of the beholder very often. If we think about noise as irresoluble, difficult to comprehend, incomprehensible, difficult to pull meaning out of, the first audiences who heard Beethoven’s first symphony reportedly objected to it as unfathomable noise; and throughout the history of music, if you look at this wonderful book by Slonimsky or the Lexicon of Musical Invective, he has an anthology of responses to first performances by critics, by audience members, and noise comes up there all the time, and [in] works that today we think of as part of the central canon of western music.  We tend to loosely ascribe noise in its negative veilings, to anything that just, we don’t have the patience to sit and comprehend. And patience is a key aspect to it.

How can music help us think about science?

Music, it’s a symbolic language. Mathematics is a symbolic language. Chemistry is a symbolic language. Anything about thinking are symbolic languages. These symbolic systems intersect in sometimes very direct ways. Science has informed or permeated my music, whether it’s a mathematical principle that I’m playing with and trying to turn into sound, or whether it’s DNA strings that I’m trying to find structure and find pattern and use sound to do that. There’s this whole world of sonifying and pulling information out of, pulling sound, meaningful sound, out of data.

I think that sometimes in a very mundane way that’s useful and informative to scientists, but I think in a much deeper and more important way there’s an inspirational circularity here, where I hope, I think, that scientists, physicians, engineers are interested in music partly because they walk away and ask new questions.

You notated your mother’s singing at the end of her life. Can you tell us about that?

Absolutely, yeah. I think that the real answer—you asked before about what science really did to me—actually happens at that point in my life. There were two parts of my family. One is my brother-in-law, who is severely schizophrenic. Although he doesn’t overtly have auditory hallucinations that I can tell, he gets into these obsessive phonological loops. He starts saying the same thing again and again and again, and audition really has a play with it.

Let me tell you about the two elder women in my life. My mother-in-law, who suffered from Alzheimer’s, at the very end of her life, my wife, God bless her, would Skype with her every day, and she would go into these loops. She would repeat the same word—my mother-in-law—would repeat the same word again and again and again and again. I don’t remember what it was that made her figure it out, but my wife realized that there was this one song from an old Bollywood film from the 1960s that somehow brought her back. She would start singing it and my mother-in-law would suddenly click into the song and sing it for a while, and then you would see it disentangle and she’d go back to this noisy inner state.

The kicker was my mother. She was bedridden for the last few years of her life. Toward the end of her life she began to hear music. Now, she was not a musician. She liked music, but we had no classical music in my house as an upbringing. The two “classical” composers in my life were John Philip Sousa, my father would wake me up with marches every day, and Gilbert and Sullivan, God helps us. Our musical world was Mitch Miller. This was her musical world. She started hearing these sounds. I remember the last time we brought her out of the house into the hospital on a gurney it was around Christmas time. There was a harpist playing Christmas carols in the entry to the hospital. She looked at me and she said, “Isn’t that music beautiful?” She was singing something entirely different. They were sort of snippets of something. I said stupidly ... I tried to make her realize but that was a mistake. In retrospect that was a mistake.

I began to transcribe what she was singing. She was singing things in ... She had a nice voice but it was not precise. I would transcribe these snippets. One of the big technologies in our world is queried by humming, is trying to find songs by ... As a game I fed these in. Low and behold the song that came out that she in fact was singing was the number 1 hit single in January, 1948, which was the week my brother was born.

By this time in her life she barely recognized us. She was out of it completely. This song that somehow stuck, that was remarkably poignant. That was incredibly compelling to me. I think that two of these stories are stories about auditory hallucinations and speech in schizophrenia and musical hallucinations, particularly in this sad convergence of hearing loss and memory loss, I found incredibly inspiring. I plunged into the science of hallucinations.

What comes out of it is, what are these really plaguing questions? I suffer from what’s called ear worms all the time. I’ll get a little snippet in my head and I can’t for the life of me pull it out. What’s the boundary line between that and a hallucination? It’s clearly hallucinatory. In fact, what is a composer if not someone who’s hallucinating all the time?

Join the Discussion