Can you hear sound when you’re looking at colors?
Well, that’s the phenomenon known as synesthesia, a neurological condition where one’s senses can become blended as your brain processes data in one gigantic information dump. You might hear sounds when you experience certain hues or shapes, taste words, or feel particular emotions when you touch certain textures. For instance, I cry uncontrollably when I wear velvet dungarees, vomit when I see the word “authentic,” and hear a sad trombone whenever I see a Rothko painting.
Pioneering abstract painter Vassily Kandinsky was one such synesthete and was said to hear symphonies of sound when working on any number of his paintings.
Now, a new interactive project from Google Arts & Culture—which I’m assuming is nestled somewhere next to my Google Drive and Docs on that dropdown in my Gmail—called Play a Kandinsky lets you explore how the painter experienced sound as he painted. By coupling machine learning with Kandinsky's color theories, they’ve imagined what the painter might have heard while working on his iconic work from 1925, Yellow Red Blue.
Kandinsky believed that colors and shapes could whip up all kinds of emotions in us, and to recreate some of those sounds, Google worked with musicians Antoine Bertin and NSDOS to study the painter’s many thoughts. First, the interactive game highlights the sounds of specific colors—when Kandinsky painted with pink shades, he might have heard something similar to a music box. If it was blue, he might have become overwhelmed by the soothing sounds of an organ. Also, according to Google’s tool, he was a wee bit ahead of his time and getting those glitchy Autechre shits when he used blacks and greys. Moving on to the segment with Yellow Red Blue, when you listen to the painting, the viewer can experience seven movements attached to different portions of the canvas, a wide-ranging variety of sounds from the bright and joyful to the complex and moody.
This new experience is from both Google Arts & Culture and The Centre Pompidou, part of an ongoing initiative to preserve and celebrate the work of Kandinsky.