Facebook
Twitter
Pocket
Reddit
Email
In the warming days of early New England summer, I started lying to my fiancé. Every morning, my alarm clock would go off at six o’clock and I would start my day by making two espressos—the first for her; the second for me—getting dressed and telling my partner that I had to head out to the office, where I would catch up on some reading and get ready for a busy day. Our wedding was only two months away, on Aug. 18, 2023, and we each had overflowing to-do lists as we prepared to take the next step in our partnership. The only problem was that my early morning destination was a lie: I was not headed to the office, and each day I was becoming more worried that my gambit would be a bad omen for my marriage.
The ruse was simple: In the months leading up to our wedding, I decided to demonstrate my love for my fiancé by composing a short piece for the piano. I had trained as a pianist when I was younger, and had yearned to be a composer, too. But while I loved trying to master the devilishly hard works of Chopin, Liszt, and Rachmaninoff, my attempts at composing my own music had always been an utter disaster. I could never finish a piece. I had ideas, and I would start writing, but inevitably I always ended up hating what I wrote, and, plagued by doubt, would scrap most of those attempts. The few pieces that survived my self-critical eye were all “commissions” for the music theory or composition courses I took in college. What’s worse, the last time I even tried to write was 15 years ago.
AI produced something that resembled a drunkard’s rendition of Mozart.
I spent the summer heading to the practice rooms at a nearby university in the morning, playing around, scribbling notes into music notation software. In the evenings, at home, I sat in the kitchen, trying to make sense of those scattered ideas, organizing them, and listening back to the synthesizer version through my headphones. I was hoping to write a short piece, just a few minutes long. Something melodic. A modest serenade that I would infuse with sounds from my native Russia and those of her South India. I spent many nights trying to understand Carnatic music, ragas, and how their fluid sounds could be translated to the rigid keys of a piano. As the summer progressed, my manuscript grew in length, but not in coherence.
I have spent more than a decade researching how the brain works, having trained as a neuroscientist. My work was adjacent to AI, and I hoped that if we could reverse-engineer the brain, this would allow us to not only understand it, but build strong AI tools. By the summer of 2023, having transitioned out of academia, I was infected by the excitement around generative AI developments like ChatGPT and Stable Diffusion. If such generative models were good for language, images, and potentially designing new proteins for better drugs, shouldn’t they also be good at music composition, too? What if I could get help from a music composition program that employed artificial intelligence?
I had been amazed to see how fluent and flexibly intelligent ChatGPT was for language and basic reasoning tasks, and I knew that the latest AI music composition programs were based on the same underlying technology. I had come up with some melodic and pianistic bits—passages that leveraged whatever latent technique and finger dexterity I could still muster—myself. What I had no faith in was putting it all together into a coherent musical story: something with a beginning, middle, and end that progressed and jived together.
I turned to OpenAI’s MuseNet for help. Released in 2019, it uses a transformer architecture—the same neural network as ChatGPT—trained to fill in blanks in a vast corpus of music. MuseNet can take prompts and generate four-minute-long audio. The brilliance of machine learning in general is that we no longer need to specify the rules by which something works to extract meaningful insights or generate examples; instead, we need huge datasets and powerful algorithms that can learn the high-dimensional distributions of the data. For music, this means that a model like MuseNet could learn directly from the data and infer all the underlying rules—and ways in which they are often broken—to generate new music.
I realized I wanted my piece to be original, to uniquely represent my feelings.
And yet, the results aren’t exactly impressive. On OpenAI, I selected one of its sample prompts: “Compose in the style of Chopin starting with Mozart’s ‘Rondo alla Turca.’” It produced something that resembled a drunkard’s rendition of the Rondo and had no relation to Chopin whatsoever. Asking for the Rondo in the style of Mozart himself ironically produced something that sounds like a Liszt transcription. I also experimented with OpenAI’s JukeBox and Google’s MusicLM, but found the quality of the generated music lacking, and the user interface—entering text descriptions of the desired music—was not how I wanted to compose the music. After a few nights of feverishly trying to find a high quality and usable AI music model, I abandoned the effort.
Moreover, as bad as my piece might have turned out, I realized I wanted it to be original, to uniquely represent my feelings and ideas. In some ways I feel lucky that I did not have access to an easy to use and powerful software that could have “helped” my composition. Would the resulting piece have been truly mine in the way that a creative output should be?
At any rate, I struggled with the composition through the months of June and July, but things were starting to make sense. I was beginning to see the structure of my piano piece more clearly in my mind, and details were falling into place. I had a main theme: a melodic line in D-flat Major. I had an ending in mind: a return to the main theme, expressed fortissimo in large chords. Having forced myself to meditate on the theme—say it again, but differently; start again, but end up somewhere else; move and morph—I now had a few variations that progressed the story. My fingers even stumbled upon a couple of bravado passages that would glue the elements together. Despite the progress, by the beginning of August, I had a problem: the middle section, which I wanted to blend elements of a Carnatic raga with those of the Slavic theme, morphed into an unplayable monstrosity that resembled neither.
Still, I was having fun: attempting to express my love through music, practicing piano, coming up with cool chords and a melody. These were all invigorating, like simultaneously creating and solving a puzzle. The middle section was the final piece of this puzzle. There’s a lot to the Indian ragas, which are often sung, that a piano simply can’t capture. But there was one thing my percussive string instrument could do. Some of the Carnatic ragas resemble a harmonic minor scale, with the addition of some minor and augmented seconds. That’s something Slavic music loves to use, and a piano can express easily. I realized that this had to be the middle section, the core of my piece: a melody that combined the harmonic roots of our two cultures.
There is nothing like love—and a looming deadline—to motivate one to take responsibility for their creation. By the second week of August, the various snippets I had been jotting down were forming a coherent whole. I spent the last week before the wedding practicing, tinkering with a few passages that my fingers had not yet accepted, designing a cover graphic, and hoping my fiancé would appreciate the gesture. Finally, “Jesseba’s Serenade” premiered on the evening of our wedding rehearsal, and I knew that my summer ruse had not been in vain: The feedback was positive. 
Lead image: Africa Studio / Shutterstock
Get the Nautilus newsletter
Cutting-edge science, unraveled by the very brightest living thinkers.
