Nvidia has demonstrated a new artificial intelligence model called Fugatto (Foundational Generative Audio Transformer Opus 1) that can generate sound effects, create music and change voice using textual cues. This research project has the potential to revolutionize industries such as music, entertainment, and translation services. Despite the technology’s potential, Nvidia has yet to announce its commercial launch.
And despite the technology’s potential, Nvidia has yet to announce its commercial launch.
New Horizons Audio Technology
As explained by Brian Catanzaro, Nvidia’s vice president of applied deep learning research, Fugatto combines the capabilities of several separate models. It can synthesize speech, add sound effects to music, and create entirely new compositions. This approach makes Fugatto analogous to generative models for images and video, such as Stable Video Diffusion or Sora.
Fugatto’s approach makes it an analog to generative models for images and video, such as Stable Video Diffusion or Sora.
«We can synthesize sound with language, which opens up new possibilities for creating unique audio»” Catanzaro noted.
“We can synthesize sound with language, which opens up new possibilities for creating unique audio,” Catanzaro said.
.
Fugatto also has «emergent properties» meaning it can combine trained elements and execute complex instructions. For example, you can load an audio file with a voice and translate the text into another language, preserving the original intonation. Or turn a simple melody into an orchestral composition.
Potential for creativity and controversy
The model can not only read text in a given voice, but also convey emotion, making the sound more expressive. However, as Catanzaro noted, Fugatto’s results are not always perfect, and quality can vary.
The use of such technologies raises questions about the impact on creative professions. For example, Hollywood studios have already faced protests from screenwriters and actors over concerns that AI could replace their labor. However, Catanzaro is confident that Fugatto will become a tool to empower musicians and sound designers.
Fugatto will be a tool to empower musicians and sound designers.
«I hope this leads to new tools for artists. Audio has always been an interesting area for experimentation»” he added.
He added.
.
Fugatto opens the door to new forms of creativity, as the guitar once did for rock music and turntables for hip-hop. But its impact on the industry remains a matter of debate for now.
Fugatto’s impact on the industry is still a matter of debate.
By ForGeeks 
