Deepfakes have been used to swap one movie star for another, put words into politicians’ and celebrities’ mouths, create pornography starring public figures, and other dubious deeds. But as a recent article reveals, the technology behind them could also revolutionize mainstream entertainment and communication - in a good way.
India’s International Institute of Information Technology Hyderabad has developed elaborate technology that translates audio from one language to another, and then makes movie and TV stars’ mouths appear to move to the new language. This means that when films and shows are dubbed, the result would be highly believable, instead of the strange, out-of-sync look that often occurs when an audio track in another language is matched as best as possible to the original.
This technology is especially useful for audiences who use video not only for entertainment, but also for information. For instance, 85% of YouTube videos watched in India are in local languages.
Read on to learn more about this new development in efficient translation and highly realistic lip synching, and why it’s important. You’ll also be able to see some videos of dubbed originals and their new AI counterparts.
A diagram explaining how this innovative speech and lip synching technology works
(Image source)
Contact Our Writer – Alysa Salzberg
