What Color Is This Song?

Suppose you’re at a concert with a friend who leans over and whispers in your ear, “What color was that music?” It may seem like a strange question, but there are some people for whom the answer is entirely self-evident, and perhaps your friend is among them. Such individuals have a neurological condition called “sound-to-color synesthesia,” or “chromesthesia,” in which they effortlessly and spontaneously experience their own personal light show while hearing music and other sounds.¹ Interestingly, many chromesthetes grow up assuming that everyone has the same visual responses to sounds as they do, and are shocked when they discover this is not so!

Chromesthesia is relatively rare, occurring in only about 1 in 3,000 individuals. Nevertheless, a remarkable number of famous visual artists and musicians are members of this select group, including Vincent Van Gogh, Wassily Kandinsky, David Hockney, Nikolai Rimsky-Korsakov, Leonard Bernstein, and Duke Ellington. Kandinsky actually used his chromesthesia in creating his paintings. A formative experience, he wrote, was hearing Wagner’s Lohengrin: “I saw all my colors in spirit, before my eyes. Wild, almost crazy lines were sketched in front of me.”

Most of us don’t experience colored light shows when we listen to music. But recent scientific evidence shows that many non-synesthetes do have music-to-color associations similar to the cross-modal experiences of chromesthetes. In my laboratory at the University of California, Berkeley, we have been seeking answers to questions about the nature of music-to-color correspondences in both synesthetes and non-synesthetes.^2,3 Our results have been eye-opening. They are uncovering the remarkably associative power of the human brain, and perhaps above all, underscoring the centrality of emotion in our mental lives.

To see what I mean, let’s start with a mini-experiment. I invite you to listen to five different music clips and choose the three colors that you feel go best with each one. I won’t name the tunes yet because I don’t want that to influence the colors you choose.

Start with Example A, featured below. Listen to the sample by clicking on the triangular arrow on the sound bar. While listening, choose the three colors that you feel go best with this music from the array to the left, picking the best one first. Follow the same procedure for each music clip. After the fifth clip, you will see 15 color arrays that summarize everybody’s choices, including yours.

Example A is from Bach’s Brandenburg Concerto No. 2. It’s in a major key and has a lively, dance-like tempo, featuring a trumpet solo bouncing along over an accompanying chamber orchestra. As you see, people picked a variety of colors, but they have strong similarities in being bright, vivid, and dominated by yellow hues.

Example B is from a different part of the same Brandenburg Concerto. Here, a violin and oboe trade a much slower melodic line in a minor key over the same chamber orchestra. The colors our participants chose are obviously different from the ones they chose for the other part of the same composition, being strikingly more dark, gray, and blue.

Example C is an instrumental excerpt of rock music by the 1990s power trio BBM. Its dominant features are prominent drum rhythms, fast-paced guitar riffs, and a loud, high-energy sound. The colors chosen are notably different from those for the two Brandenburg excerpts. In this case they are predominantly reds, black, and other dark colors.

Example D is “easy-listening” music, characterized by its laid-back tempo, minor key, and the quiet sound of the simple piano melody. The chosen colors are muted, cool colors dominated by various shades of blues and grays.

Finally, Example E is an excerpt from a salsa song. It is a fast, high-energy piece in a minor key that features lots of brass and catchy rhythms in the drums and other percussion. The colors are overwhelmingly shades of red and other “hot, spicy” colors, such as oranges and yellows.

Why, then, do we choose the colors we do? An important part of the answer comes from certain features of the music itself. Fast-paced music in the major mode, such as in Example A, tends to elicit bright, vivid, warm colors, such as intense yellows. Slow-paced, quieter selections in the minor mode, such as in Examples B and D, are more likely to evoke darker, grayer, cooler colors, such as shades of blue, purple, and gray.

Fast-paced, complex, high-energy music with a driving beat and distorted sounds, such as Example C, brings out reds and blacks more than any of the others. Example E is a bit unusual in that it is fast-paced and energetic, but in a minor key, and with a complex, distinctive, Afro-Cuban beat. These features led people to choose vivid “hot” colors that might even be termed “spicy.”

As these brief descriptions imply, the musical features that matter most in determining the colors people choose include tempo (slow to fast), energy (low to high), volume (soft to loud), mode (minor to major), pitch (low to high), harmonic and melodic content (consonant to dissonant), complexity (simple to complex), and distortion (clear to distorted).

But there is another way to look at these choices, one that centers on emotion. Consider how you might describe the emotional qualities of the first four musical selections. Roughly speaking, Example A sounds happy and strong, Example B sounds sad and weak, Example C sounds angry and strong, and Example D sounds calm and a bit sad.

Why do we choose the colors we do? An important part of the answer comes from certain features of the music itself.

If colors have analogous emotional qualities, people might simply pick the colors whose emotion best matches that of the music. Look again at the color arrays and consider which one looks happiest, which looks saddest, which angriest, and which calmest. We had participants rate the happiness, sadness, angriness, and calmness of each of the 37 colors and then computed the average value of each emotion in the color choices they made for different musical clips. It turned out that array A contains the happiest colors, B the saddest colors, C the angriest colors, and D the calmest colors.

Such findings suggest that non-synesthetes perform this music-to-color association task by choosing the colors whose emotional associations best match the emotional qualities of the music. We computed a statistical measure of how well the emotional ratings of the musical selections predict the emotional ratings of the colors people chose as going best with that music (and vice versa). For the four emotional dimensions we studied—happy/sad, angry/calm, strong/weak, and lively/dreary—the correlations were extremely high. They showed that if you know how happy (or sad) a given piece of music sounds, you can predict with great accuracy how happy (or sad) the colors they chose as going best with it will be. The colors may differ in many ways, but they will tend to be “happy” ones.

We further tested our intuition about emotions by investigating how participants responded to faces expressing various emotions. We asked participants to perform the same test again, but this time, they were given pictures of faces expressing various emotions. We found that the correlations between the emotional ratings of the expressive faces—happy, sad, angry, calm—and the emotional ratings of the 37 colors were very high. Strong connections also emerged between emotional ratings of musical selections and emotional faces. These results further support the claim that emotions are involved in the links we discovered from music to color.

We repeated the same experiments with collaborators in Mexico using the same music and the same colors to examine the nature of cultural differences in music-to-color associations. The Mexican and United States results were virtually identical, with equally high emotional correlations for music-to-color associations. These findings suggest that such emotion-based mappings in non-synesthetes are cross-culturally robust and might even be universal.

The existence of emotional effects in music-to-color associations of non-synesthetes provides a window into the nature of chromesthesia. Given that signals from the five senses—sight, hearing, touch, taste, and smell—enter the human brain in different anatomical regions, you might ask why auditory and visual experiences interact with each other at all. One of the most important reasons is to enable our brains to sort out sensory events from different modalities that correspond to the same event—such as when our eyes register the sight of a glass shattering to pieces as it hits the floor at the same time that our ears register the sound of it breaking. Cross-modal connections between the senses enable our brains to integrate and make sense of the complex multisensory inputs that bombard us all the time.

Scientists do not fully understand the biological mechanisms that underlie chromesthesia. Some propose that synesthesia results from neural “cross-wiring” between different sensory areas of the brain, such as direct neural connections from the auditory cortex to the visual cortex in the case of chromesthesia.⁴ Others propose that the neural circuits responsible for synesthesia are more complex, including mediation through activity in other brain regions.^5,6 According to this view, neural activity in the auditory cortex would cause activation in some other mediating brain area, which would, in turn, cause synesthetic activity in the visual cortex.

If chromesthetes show effects of emotion similar to those for non-synesthetes, it would suggest that chromesthesia is more likely to rely on neural activity through mediating brain regions, such as the amygdala or other structures of the limbic system involved in emotional processing, than on direct connections between the auditory and visual cortexes. (Because the two possibilities are not mutually exclusive, it is also possible that both direct and indirect connections are in effect.)

To study the relation between synesthetic experiences and non-synesthetic associations, we used the same paradigm described above. We found 11 chromesthetes near Berkeley who experience colors to music. We also tested 11 non-synesthetes who were matched with the synesthetes in age, gender, musical training, and color training.

Everyone first chose three colors for 18 pieces: six each by Bach, Mozart, and Brahms. The only difference was their task. The non-synesthetes chose the colors that “went best” with the music, as you did in our mini-experiment. The synesthetes, however, chose the colors that were “most similar to the colors they experienced while hearing the music.” After performing this music-to-color task, everyone rated each musical selection and each color for their emotional qualities. The primary question of interest was how the choices of the synesthetes would compare to those of the non-synesthetes in terms of their emotional correlations.

Figure 2 below shows the first-choices of the 11 music-to-color synesthetes and the 11 matched control participants for six selections of classical music: three selections of fast, major-key music by Bach, Mozart, and Brahms, which sound happy and strong, versus three selections of slow, minor-key music by the same composers, which sound sad and weak.

As you see, the colors reported by the chromesthetes as most similar to their synesthetic experiences (Figure 2B) are remarkably like the colors that non-synesthetes chose as going best with the same music (Figure 2A). The fast, major music produced similarly vivid, light, yellowish colors (on the left side of Figures 2A and 2B), whereas the slow, minor music produced similarly dark, drab, bluish colors (on the right side of Figures 2A and 2B).

We also analyzed the data from the synesthetes and non-synesthetes for evidence of emotional effects by computing the correlations between the emotional ratings of the music with the emotional ratings of the colors that were chosen as going best with that music. The results show that the non-synesthetes’ correlations were all high and statistically above chance for all five emotional dimensions measured (Figure 3). The synesthetes’ emotional correlations were similarly high and reliable for happy/sad, active/passive, and strong/weak, but lower and unreliably different from chance for calm/agitated and angry/not-angry.

These findings show that synesthetes exhibit emotional effects in the colors they spontaneously experience when listening to complex music. They appear to be as strong in synesthetes as in non-synesthetes for some emotions. That similarity implies that some of the neural mechanisms underlying music-to-color synesthesia are influenced by activation from brain regions that process emotion. The fact that it is weaker in synesthetes than non-synesthetes for certain emotional dimensions suggests that direct neural connections between the auditory and visual cortexes may also influence synesthetes’ color experiences by diluting some of these emotional effects.

So what color is a given piece of music? The answer depends on both the features of the music (slow/fast, major/minor, loud/quiet, high-pitched/low-pitched, clear/distorted) and the emotions those features elicit in the listener. Fast, loud, high-pitched, music sounds agitated, for example, whereas slow, quiet, low-pitched music sounds calm.

We don’t fully know why that is. But one intriguing possibility is a hypothesis that I like to call “musical anthropomorphism.” Perhaps we perceive music as if it were a person behaving in ways that express emotion. Fast, loud music at a high pitch might be perceived as agitated because when people are agitated, they move more quickly and raise their voices both in pitch and volume. Similarly, slow, quiet, low-pitched music may sound sadder or calmer because when people are sad or calm, they move more slowly and speak more quietly in a lower register.

It’s not yet clear why music in a major key sounds happier to most people than the same music in a minor key. Musical anthropomorphism suggests that it might result from songs in minor keys “drooping” a bit more emotionally than the same songs would in major keys, given that all the standard notes in a minor scale that differ from those in a major scale are a half-tone lower in pitch, and lower pitched vocalizations in the same speaker are associated with saddened affect.

The human brain is an incredibly powerful, all-purpose associative engine for discovering relations among our senses.

Perhaps the most important message about music-to-color connections in synesthetes and non-synesthetes is the central role emotion plays in our mental lives, even when it is not immediately obvious that it is relevant. Music and colors have few, if any, sensory properties in common: Music is auditory and has properties such as tempo, pitch, timbre, and rhythm. Color is visual and has the properties of lightness, vividness, and hue. But both music and color map aspects of emotion.

Color has its own form of anthropomorphism. The strong connection between red and anger is likely due to the fact that blood is red, which makes angry faces redder than calm faces. Anger also tends to lead to violence, which can, in turn, lead to red blood being spilled. The links from bright yellows to happiness and from dark blues and grays to sadness are less clear, but may arise from people’s emotional responses to different weather conditions. People are generally happier on sunny days with a bright, warm, yellowish light and sadder on cloudy days when the illumination is darker, gray, and tinged with blue. We often speak of people who are depressed as “feeling blue” or “living under a cloud.”

Emotion connects us to so many different things in our lives that we are left to wonder about its evolutionary purpose. It seems likely that emotional responses funnel sensory information into tendencies to behave in particular ways. Happiness most often leads to approach and engagement, sadness to withdrawal and inaction, anger to aggression and violence, and fear to fright and flight.

Our experiments continue. Currently, for instance, we are studying the emotional effects on music-to-color associations in Turkey and the U.S., using traditional Turkish music that differs from western music not only in the instruments used, but in that it has many more modes (makams) than the western modes of major and minor.

So far we have been endlessly intrigued by our results. The human brain is an incredibly powerful, all-purpose associative engine for discovering systematic relations among disparate sensory inputs. Emotions are potentially relevant to almost everything we experience, even when those experiences arise from different kinds of information in different brain regions. I have a feeling that as scientists study people’s responses to different kinds of sensory input more fully, the importance of emotion in creating integrated, meaningful experiences will strike an even deeper, more resonant chord than it does now.

Stephen E. Palmer is a professor at the Graduate School in Psychology at the University of California, Berkeley. He is the author of Vision Science: Photons to Phenomenology, and co-editor of the recent Aesthetic Science: Connecting Minds, Brains, and Experience.

References

1. Cytowic, R.E., & Eagleman, D. Wednesday Is Indigo Blue: Discovering the Brain of Synesthesia MIT Press (2009).

2. Palmer, S.E., Schloss, K.B., Xu, Z., & Prado-Leon, L.R. Music-color associations are mediated by emotion. Proceedings of the National Academy of Sciences 110, 8836–8841 (2013).

3. Palmer, S.E., Langlois, T.A., & Schloss, K.B. Music-to-color associations of single-line piano melodies in non-synesthetes.  Multisensory Research (2015).

4. Ramachandran, V.S. & Hubbard, E.M. Synaesthesia—a window into perception, thought and language. Journal of Consciousness Studies 8, 3–34 (2001).

5. Cytowic, R.E. Synesthesia: Phenomenology and neuropsychology. Psyche, 2, (1995).

6. Ward, J. Emotionally mediated synaesthesia. Cognitive Neuropsychology 21, 761-772 (2004).

Get the Nautilus newsletter

Cutting-edge science, unraveled by the very brightest living thinkers.

NL – In Page Mobile

Email: *

Captcha

Sign up for free