Meta has launched a groundbreaking open-source AI code called AudioCraft, enabling users to effortlessly craft music and sounds using generative AI.
AudioCraft comprises three distinct AI models, each specializing in different aspects of sound generation. First, there’s MusicGen, which skillfully generates music from text inputs. The model’s impressive capabilities stem from being trained on an extensive dataset of “20,000 hours of music” exclusively owned by Meta or specifically licensed for this innovative purpose.
Next up is AudioGen, designed to produce lifelike audio based on written prompts. Whether it’s mimicking barking dogs or the sound of footsteps, this AI model draws from a vast collection of publicly available sound effects.
To further enhance the user experience, AudioCraft includes an upgraded version of Meta’s EnCodec decoder. This enhancement minimizes artifacts, the unintended distortions that can occur when manipulating audio excessively, allowing users to create seamless and pristine sounds with ease.
The company offered the media an opportunity to listen to sample audio created using AudioCraft. The generated sounds, such as whistling, sirens, and humming, exhibited a remarkably natural quality. However, despite the realism in the guitar strings of the songs, there was still a discernible artificial touch to them.
Meta is just one of the latest companies delving into the fusion of music and AI. Google, for instance, introduced MusicLM, a substantial language model that can generate minutes of sounds based on text prompts, but it is accessible exclusively to researchers. Additionally, an “AI-generated” song featuring voice likenesses of Drake and The Weeknd went viral before being taken down due to its controversial nature.
More recently, notable musicians like Grimes have been advocating for the utilization of AI to create songs incorporating people’s voices, leading to intriguing developments in the realm of AI-made music.
Certainly, musicians have been exploring electronic audio experimentation for a considerable time; the emergence of EDM and festivals like Ultra bears witness to this fact. However, computer-generated music has often sounded like modified versions of existing audio.
The advent of AudioCraft and other generative AI-powered music changes this landscape, as they can create sounds purely from text inputs and an extensive library of sound data.
Presently, AudioCraft appears best suited for elevator music or stock songs to create ambiance, rather than becoming an instant sensation as the next big pop hit.
Nevertheless, Meta holds a strong belief that its innovative model has the potential to spark a transformative wave of songs, much like synthesizers revolutionized music once they gained popularity.
“We envision MusicGen evolving into a new kind of instrument, akin to the impact synthesizers had when they initially emerged,” stated the company in a blog post. Meta openly acknowledges the challenges in crafting AI models capable of generating music, considering that audio involves millions of points where actions are taken, unlike written text models such as Llama 2, which operate with only thousands of data points.
The company emphasizes that open-sourcing AudioCraft is essential to enhance the diversity of the training data it relies on.
“We acknowledge that the datasets used to train our models lack inclusivity. Notably, the music dataset predominantly comprises Western-style music and solely consists of audio-text pairs with text and metadata written in English,” Meta stated. “By making the code for AudioCraft openly accessible, we aim to enable other researchers to explore new methods to mitigate or eliminate potential biases and prevent misuse of generative models.”
Concerns about AI’s implications have already been raised by record labels and artists. Many worry that AI models may incorporate copyrighted material during training, which has historically led to legal disputes.
Examples like the Napster case serve as reminders, and more recently, Spotify faced a billion-dollar lawsuit based on a law dating back to the era of player pianos. In addition, this year witnessed a court ruling on whether Ed Sheeran’s “Thinking Out Loud” copied from Marvin Gaye’s work.