Respeecher startup creates artificial voices for Hollywood – KyivPost
When young Luke Skywalker appeared in the final episode of “The Mandalorian,” a spinoff series of the Star Wars films, viewers were left speechless. Skywalker looked like himself at 28, although he was played by 68-year-old American actor Mark Hamill, who has played the character since 1977.
To rejuvenate the iconic Jedi, director Jon Favreau used a visual effect called de-aging. âWhat people didn’t realize is Skywalker’s voice wasn’t real either,â Favreau said. It was artificially synthesized by the Ukrainian startup Respeecher.
Respeecher’s nearly 20-strong team worked for a Hollywood series from their small office in Kiev. Ukrainian technicians obtained recordings of 40-year-old Hamill’s voice, analyzed them using artificial intelligence (AI), and generated the voice of young Skywalker. Then, they swapped the voice of an actor who read the script for the synthetic one they created.
âThe fact that everyone was surprised that the voice wasn’t real means that we did everything right,â said Oleksandr Serdiuk, co-founder of Respeecher.
Besides Lucasfilm, the production company behind Star Wars, Respeecher works with other big clients in Hollywood and smaller studios around the world. The startup is revealing very few names due to nondisclosure agreements.
While many startups, including US giants such as Replica Studios, Descript, and Modulate, use voice modification technologies, only a few have managed to produce sound that satisfies the demanding film industry.
âThe Respeecher team is always ahead of the game,â said Bas Godska, general partner at Acrobator Ventures, the Dutch investment fund that has invested in the Ukrainian company.
The voice technology industry has great potential, Godska said. It is expected to grow at a rate of 17% per year, attracting investment of nearly $ 27 billion by 2025.
âIn 3 to 5 years, it (voice conversion) will become a common tool in post-production,â said Sergii Soldatov, executive producer of Ukrainian studio Eve Production.
The Respeecher team believes its chances of gaining a large share of this market are high. âNo company in Hollywood produces the same quality of sound as we do,â said Serdiuk.
Serdiuk and his partner, Dmytro Bielievtsov, started working on voice conversion, the technology that makes one voice sound like another, in 2016, when no other company offered such a service, they say. At that time, Serdiuk and Bielievtsov were working as data analysts and were passionate about the possibilities of AI. Bielievtsov, a music enthusiast who plays four musical instruments, decided to apply his technical knowledge to work with sounds.
At one of the AI ââconferences in Kiev, Serdiuk and Bielievtsov met Grant Reaber, an American tech enthusiast. Reaber studied computer science, machine learning, and math for almost 15 years, and he loved the idea of ââvoice conversion. The three founded Respeecher in 2018.
Unlike most other voice tech startups, which turn written text into speech, Respeecher has trained their algorithms to process vocal sounds only. It’s more complicated because it requires analyzing the voices of real people, including the different accents.
One of Respeecher’s first major projects was to make the voice of the 37th US President Richard Nixon heard. They created it to express an unspoken speech that Nixon prepared in case the Apollo 11 mission in 1969 failed. The clip was used in the American documentary short “In Event of Moon Disaster”.
âWe created a very realistic film, in large part thanks to the work of Respeecher,â director Halsey Burgund said, according to the Respeecher website.
Nixon gives a speech he prepared in case the 1969 Apollo 11 mission fails. This speech was artificially created for the film âIn Event of Moon Disasterâ by Ukrainian startup Respeecher.
Since its founding three years ago, Respeecher has worked on nearly 50 projects. The startup has collaborated with Hollywood studios and produced commercials for major events.
For Super Bowl 2021, the annual championship game of the United States National Football League, Respeecher recreated the voice of legendary coach Vince Lombardi, who died in 1970. As Digital Lombardi gave a motivational speech, his voice rang. exactly like 50 years ago.
Working on Lombardi’s audio media was difficult – they were old and damaged. But the ability to convert even low-quality sound into realistic speech is what makes Respeecher attractive to big customers, Serdiuk said.
To date, the startup is working with its clients on a project-by-project basis, but Ukrainians want to make the technology more automatic, according to Bielievtsov. This could have a big impact on the film industry, Respeecher investors said.
Ukrainian startup Respeecher recreated the voice of the legendary American National Football League coach for the Super Bowl commercial that aired in February 2021.
When audio turns out to be faulty, it is expensive and time-consuming to bring actors back to the studio to re-record it, said Dionis Akulov, creative producer at To Be Production, the Ukrainian post-production company. With Respeecher, studios could repair the audio layer remotely.
“One day, Respeecher could be part of a large production company,” said Roman Nikitov, co-head of venture capital at ICU, the fund that invested in Respeecher in 2020.
Respeecher has already received takeover offers, but the startup turned them down. âThere are still a lot of things that we want to improve on our own,â said Serdiuk.
Despite the technology’s many benefits, some experts in the film industry fear that it is also dangerous. People could use voices without the permission of the actors or modify them to damage someone’s reputation.
To avoid this, Respeecher obtains written permission from the actors whose voices are changed. He also wants to add watermarks to his audio, so sound specialists can detect changes that the human ear cannot recognize.
âIt’s new technology, so people are scared,â Serdiuk said. âBut it’s true for all innovations: we see the threats first, then the possibilities. “
Praise for investors
Voice conversion is an expensive technology, according to Serdiuk. But there are still many investors who want to invest in Respeecher. Since 2018, the startup has attracted more than $ 2 million.
“We can imagine how far this development will go, so it was obvious for our fund to invest,” said Godska.
Respeecher does not disclose its income and market value, but Serdiuk said the startup is on the cusp of becoming profitable. It has not yet become profitable as it reinvests all income in further technological development.
Investors continue to support the company due to high profile clientele such as Lucasfilm and the US TV channel Telemundo. âWe see their product is in demand,â Nikitov said.
To date, Respeecher’s technology is too expensive for small studios. But the company plans to make it more affordable. This could also allow its use in the translation of foreign films.
Instead of dubbing, Respeecher can use an original actor’s voice and have them speak a foreign language.
âWe can make Brad Pitt speak the perfect Ukrainian actor or Ukrainian – perfect English,â Serdiuk said.
The startup is also working on the introduction of other more affordable services. He just launched the so-called Voice Marketplace which sells copyrighted voices – real or artificially generated – that small businesses can use in their own games or movies.
âWe are investing a lot in this technology,â Serdiuk said. âIt will help all creative companies compete with the big studios. We will therefore have a competition of ideas, not of budgets.
The video shows Respeecher’s voice conversion technology, which allows one person to speak with another’s voice.
Kyiv Post technology coverage is sponsored by Ciklum, ELEKS, Intellias, Intetics, Itera, Parimatch Tech, SimCorp, SoftServe and TECHIIA. The content is independent of the donors.