Audio dubbing is an alternative that enables content providers to add language-specific content to the original audio.
Manik Gupta is Associate Director of Engineering at Interra Systems.
With the globalisation of content, providers are looking for new ways to streamline video delivery and ensure a good viewing experience for consumers. While creating content for different geographies is an opportunity to reach new audiences, preparing content in various languages can be challenging. Roughly 6,500 different languages are spoken around the world today, each with its own distinct differences.
For years, content providers relied on the use of captions and subtitles to deliver content in regional languages. However, these come with certain limitations. Subtitles make it challenging to deliver long dialogue scenes within limited screen times. They can be distracting and often fail to convey the emotions in a dialogue.
Audio dubbing is an alternative that enables content providers to add language-specific content to the original audio. Audio dubbing translates a foreign language programme into the audience’s native language. When content creators produce dubbed audio tracks, however, they must take care to ensure high quality, as synchronisation issues are among the biggest challenges with dubbing.
Any noticeable lead or delay between audio and video can negatively affect the viewing experience. The time taken to communicate the same message in different languages can vary drastically, thereby creating synchronisation problems. Sync issues also occur if a frame on the dubbed track is lost.
Ideally, a QC solution can efficiently check synchronisation between the dubbed track and the master track, and determine if the audio track was changed in any way after the dubbing stage. A mismatch in timing between the original and dubbed tracks causes synchronisation problems that can quickly lead to viewer dissatisfaction.
By automating QC workflows and performing comprehensive checks, content providers can streamline audio dubbing. Advancements in automated and AI-/ML-based technologies are driving new efficiencies for QC workflows, allowing content providers to rapidly and accurately detect audio dubbing issues with reduced manual supervision. There are five particular ways in which technology is transforming audio dubbing for content providers:
- Automated QC solutions make the verification of complex dubbing packages, including multiple MXF and .wav files, more efficient.
- By leveraging an automated QC solution, content providers can certify that metadata package structures are precise, while also checking that the number of audio tracks, channel configuration of dubbed tracks, and duration of the original audio track compared with dubbed audio tracks are correct.
- Content providers can rapidly confirm synchronisation between video and dubbed tracks, as well as between original and dubbed tracks. An automated QC solution can detect whether synchronisation issues exist between the video and dubbed tracks, by verifying the presence of black frames in video tracks when there is silence, as well as by detecting colour and test tones in dubbed audio tracks. Checking for sync loss between the original and dubbed audio tracks is a little tricky, since the audio data are vastly different. The master track and dubbed track have common background music or effects, which can be separated from the audio track using techniques such as bandpass filter. An automated QC solution that can check for localised correlation between background beds of dubbed audio tracks and the original audio track, and compare loudness curves, ultimately helps content providers identify any out-of-sync audio.
- Language identification is an important part of the audio dubbing process. Over the last few years, AI/ML algorithms have become so intelligent that automated QC systems can detect language in any audio track with more than 90% accuracy. It only takes a few hours to train AI/ ML models before they’re capable of predicting the language spoken in the audio track. Using metadata, content creators can verify that it is correct.
- Many countries have regulations requiring that regular television programming and commercials have the same level of loudness. These measures are put in place to ensure that commercials are not substantially louder. With an automated QC system, content creators can compare loudness values between audio tracks, based on standard CALM or ITU algorithms.
In short, with automated QC solutions and the use of AI/ML technologies, content providers can create dubbed packages with greater speed and accuracy, ensuring a high quality of experience for global viewers.
Manik Gupta is Associate Director of Engineering at Interra Systems.