Nvidia unveils Fugatto AI model for music and audio generation
Nvidia NVDA.O. unveiled on Monday a new artificial intelligence model for generating music and audio that can modify voices and generate novel sounds. This technology is aimed at the producers of music, films, and video games.
Nvidia, the world’s biggest supplier of chips and software used to create AI systems, said it does not have immediate plans to publicly release the technology, which it calls Fugatto, short for Foundational Generative Audio Transformer Opus 1.
Fugatto joins other technologies shown by startups such as Runway and larger players such as Meta Platforms META.O that can generate audio or video from a text prompt.
According to the company, this model can process and modify existing audio. For example, it can transform a piano melody into a human voice singing the same line, or take a spoken word recording and alter the accent and mood expressed.
“If we think about synthetic audio over the past 50 years, music sounds different now because of computers, because of synthesisers,” said Bryan Catanzaro, vice president of applied deep learning research at Nvidia. “I think that generative AI is going to bring new capabilities to music, to video games, and to ordinary folks that want to create things.”
Nvidia’s new model was trained on open-source data, and the company said it is still debating whether and how to release it publicly.
“Any generative technology always carries some risks, because people might use that to generate things that we would prefer they don’t,” Catanzaro said. “We need to be careful about that, which is why we don’t have immediate plans to release this.”
Creators of generative AI models have yet to determine how to prevent abuse of the technology, such as a user generating misinformation or infringing on copyrights by generating copyrighted characters.
OpenAI and Meta META.O similarly have not said when they plan to release their models that generate audio or video.
