Rask AI, an innovative video and audio localization tool, is excited to introduce its latest feature: Multi-Speaker Lip-Sync. Now, over 750,000 users can effortlessly translate their content into 130+ languages, ensuring it sounds as natural as if spoken by a native speaker.
For quite some time, dubbing has faced a challenge – the mismatch between lip movements and voices, particularly in English-speaking countries. This disparity has contributed to the relatively lower popularity of dubbing in these regions. However, experts argue that aligning lip movements significantly enhances the realism of localized content, making it more appealing to audiences.
Linguistics professor Yukari Hirata’s study emphasizes the importance of focusing on lip movements for perceiving challenging phonemic contrasts in a second language. Moreover, lip reading plays a fundamental role in general language learning.
Enter Rask AI’s groundbreaking feature. Now, dubbed content gets a significant upgrade as the AI automatically refines the lower face based on references, considering both the speaker’s appearance and their speech. This results in remarkably realistic dubbed videos, taking the viewing experience to a new level of authenticity and engagement.
How it works:
- Upload a video with one or more people in the frame.
- Translate the video into another language.
- Press the ‘Lip Sync Check’ button and the algorithm will evaluate the video for lip sync compatibility.
- If the video passes the check, press ‘Lip Sync’ and wait for the result.
- Download the video.
According to Maria Chmir, founder and CEO of Rask AI, the new feature will help content creators expand their audience. The AI visually adjusts lip movements to make a character appear to speak the language as fluently as a native speaker.
Built on generative adversarial network (GAN) learning, this technology comprises a generator and a discriminator engaged in a continuous competition. The generator creates content, specifically lip movements, while the discriminator ensures quality control, each striving to outpace the other in this dynamic interplay.
• The beta release is available to all Rask subscription customers.
Source:- https://www.rask.ai/
