YouTube is putting new features in I see. Pretty sure that's AI generated.
@alyx tried this today with an Italian video, better than nothing.
I've seen really good AI dubs of videos, but I think those are too costly to run at scale for free. The YouTube one right now is not amazing, but being able to understand the content at all is good.
I imagine they're getting a transcript from the audio, translating that and then putting a text-to-speech on it. A paid app can do much better, and instead of working on some videos it would work on all videos you pick as a user.
@leo
Honestly, I'd be satisfied if they just gave people better translated captions. Last time I tried even just regular auto transcribed (not translated) captions, they were still bad. I don't need full dubbing if there's some foreign language thing I'm interested in. Good subtitles are more than enough.
@alyx I think their transcriptions are the main problem. Which is weird because these days I'm used to perfect or almost perfect transcriptions.
This might just come down to the same cost issue, YouTube is free and hundreds of hours of video is uploaded every minute. On top of encoding and hosting all of that, I can't imagine they have a lot of compute budget left.
They probably have an absurdly cheap and optimized transcription model. If they improve that, AI dubs should improve too.
@leo
Had a quick look just now, cause I haven't tested them in a while. I think English transcriptions have gotten noticeably better. Only tried 1 video right now, but it was on point. No complaints.
But I tried one in my native language too, and... not good enough to try to translate them. You kinda need perfect transcripts if you want to put into a translation model. Still, a bit better than I expected.
I expect more popular languages like German and French to be worked on more though, and they're probably at good enough level to do the dubbing. But I don't speak other languages to be able to make a judgement on how good things are.
