An AI wrote a Christmas song, and it’s… filled with flowers?
Researchers used a new technology called “neural karaoke,” which they then used in order to make an AI write a Christmas song.
When it comes to Christmas songs, they are frequently not the most complex of musical compositions. Older Christmas songs generally talk about Jesus or Santa, while newer ones talk about helping people or getting sexy. And then you have bands like Manheim Steamroller that do… things.
There are some exceptions, and some Christmas songs are just as complex as any, of course. But this isn’t one of those songs.
Researchers from the University of Toronto recently introduced a new technology known as “neural karaoke” that AI can use to create a unique song. It’s not the first time an AI has written a song, but this is a little different.
The way the technology works is that an AI is shown a specific image. The AI then processes what it sees, and creates a song that it believes will be a crowd pleaser. At least in theory. The result is a little… Well, it’s not good, but it’s an interesting start for something bigger in the future.
“We are used to thinking about AI for robotics and things like that. The question now is what can AI do for us?” said Raquel Urtasun, an associate professor in machine learning and computer vision at Toronto’s computer science lab, told The Guardian.
“You can imagine having an AI channel on Pandora or Spotify that generates music, or takes people’s pictures and sings about them,” her colleague Sanja Fidler said. “It’s about what can deep learning do these days to make life more fun?”
The neural karaoke program is actually one piece of a bigger effort that would allow AI to make music, write lyrics, create dance routines, and generally entertain its human overlords.
As a starting point, a PhD student at the Toronto lab named Hang Chu created and trained a neural network using 100 hours of music found online. The program was then able to take a musical scale and melodies, then produce a basic 120-beats-per-minute melody, then add additional instruments.
Once the music was set, the team took hours of footage from the Ubisoft game Just Dance, in order to teach the AI to move to the music. That allowed it to track human poses. The researchers then added 50 hours of random song lyrics to teach the AI how to put words to the music. The program eventually built up a vocabulary of just under 3400 words.
The team then added several images with captions so the program could learn how words could be linked to visual patterns. The goal is to show the AI any image and have it churn out an enjoyable song based on that image – whether that image is a person, place, or even holiday picture. For the Christmas song, the researchers gave the AI an image of a fully decorated Christmas tree.
You can hear the results below, but it still has a ways to go. For now, it just sounds like GLaDOS’s sad cousin.
Check out the clip. While it’s not exactly a modern classic, but it’s not as rapey as “Baby, It’s Cold Outside,” or as accidentally racist as “Don’t They Know It’s Christmas,” so it’s a start. But what’s up with all the flowers?