Voice-Swap has collaborated with the world-famous soprano Asmik Grigorian by creating an AI voice model of the opera star. Alongside our co-producers CultTech Association, Voice-Swap has spent the past few months executing this project, from working with the rights holders to developing Asmik’s voice model. Now, we are delighted to preview the results of our team’s work, the first-ever voice model of Asmik Grigorian which debuted at the Salzburg Festival in Vienna as part of an all-star panel on Thursday, August 15th, 2024.  

You can hear her voice model in action in the SoundCloud embed below. Read on to find out how we made the model and the challenges we faced. 

Creating a voice model of a soprano singer was a big challenge for us and our team. The dynamic nature of operatic performance and the dynamic octaves and ranges explored by opera singers meant that the model had to capture and re-create a much more varied style than other genres. To obtain the required amount of audio to train the model, we had to isolate acapella from Asmik’s existing recordings. Unlike other genres, classical music tends to be recorded in one space, with one take, without multi-tracked vocals or acapella. This led to a new challenge – how could we access the clean and dry vocals required to achieve a high-quality voice model without isolated acapellas? The solution came from AudioShake, a stem separation company that’s worked with Disney, De La Soul, Greenday and the estate of Nina Simone. Their high-quality stem separation tech allowed us to isolate Asmik’s vocals, revealing enough suitable audio to create and fine-tune Asmik’s AI voice. 

The next task was to find a song that Asmik had never recorded or performed live to demonstrate the model’s capabilities. We also wanted to find a song recorded in a different language to what Asmik usually records in to separate it from her existing repertoire. Given Asmik’s wide range of performances, it was a challenge, but the soprano settled on a recording of Komitas’s ‘Krunk’ by Ruzanna Nahapetjan. 

After obtaining permission from Ruzanna, we set about once again using stem separation to isolate the piano – played by Roderigo Robles De Medina – before replacing Ruzanna’s vocals with Asmik’s AI voice model. We then mixed the two recordings back together using traditional EQ and reverb to place the audio in the same ‘space’, as if it was recorded at the same time. The result was Asmik ‘singing’ a song she’d never performed or recorded before. 

The combination of working with a global superstar soprano, a myriad of technical challenges and the complex nature of operatic performance meant the creation of the model has been one of the most challenging, and rewarding, Voice-Swap projects to date.

The goal of this project was not to create a replacement but rather to demonstrate how this technology could create discussion, debate and even disagreement. AI is still in its infancy, and while we can clone timbres, tones and textures, we were not attempting to entirely replace the lifetime of emotion and the disciplined training that goes into an operatic performance.

Performance is entirely human, and as this technology continues to develop, it is in the merging of human artistic and creative endeavours with revolutionary technology where new ideas flourish. From sampling to synthesizers, drum machines and auto-tune, it is always the human behind the tech that defines its impact. Long may that continue.

Thanks to the Salzburg Festival, CultTech, Asmik Grigorian, Alpha Classics, AudioShake and Ruzanna Nahapetjan for permission to use her recording of ‘Krunk’. You can buy Ruzanna’s CD featuring her recording of ‘Krunk’ here. 

Read more about the event on CultTech’s blog here. 

If you want to create your own voice model, email us at info [at] voice-swap.ai