I'm just gonna ramble about codecs for a bit.

Codecs like H.264, AAC, PNG, JPEG, GIF, and even MP3, MPEG-2, and AC-3 are never going to go away. They're going to be with us forever.

Plenty of codecs have come and gone a la RealVideo (.rm), WMA, WMV, FPS1, H.263, DivX/Xvid/MPEG-4 Part 2, VP6, RealAudio (.ra), etc.

Then there are codecs that were pretty much dead on arrival like FLIF.

The image format wars are so fucking fucked right now. On the Chrome (and Firefox because they have no significant market share anymore so they just have to follow whatever Google decides) side you have WebP (AVIF is also starting to gain some traction) and on the Apple/Safari side you have JPEG2000. Safari has enough market share that CDNs will encode JPEG2000 images just for Safari.

The best solution is JPEG XL. However, adoption isn't looking so good.
https://caniuse.com/jpegxl

It looks like AVIF has won the image format wars.
https://caniuse.com/avif

Well, it certainly could have been worse.

I also just found out about Google's Lyra speech codec:
https://en.wikipedia.org/wiki/Lyra_(codec)

And Microsoft's new speech codec, Satin:
https://techcommunity.microsoft.com/t5/microsoft-teams-blog/satin-microsoft-s-latest-ai-powered-audio-codec-for-real-time/ba-p/2141382

Those are both really cool!

It makes me wonder what could be possible as we get more powerful NPUs and TPUs. Would be neat to use a neutral net to encode video. I guess it is already kind of possible. You could use a neutral net to examine a video and dynamically adjust the encoding parameters on the fly to get the best possible quality.

Actually, I wonder if that's what NVEC2 is doing. Once NVIDIA added tensor cores we got NVENC2. Encoding speed was almost halved compared to previous generations but the quality dramatically went up. At content delivery bitrates like 720p 30FPS at constrained VBR of 3.5 Mbps it'll deliver a higher quality video than x264 on the placebo preset. I bet that's what they're doing. Well, at least it's part of what they're doing. Would be nice to have an open source implementation tho.

Speaking of new possibilities for video encoding, I wonder if VVC/H.266 will learn their lesson from HEVC/H.265 licensing fiasco. Even if licensing is exactly the same as AVC/H.264 I wonder if the industry will adopt it the same as they did with AVC/H.264 or if they'll just embrace royalty free codecs like AV1 because they don't want the possibility of having to deal with all of that shit again with the next generation.

I mean, it seems the industry got burned pretty badly from the HEVC/H.265 licensing mess. I think they've kind of screwed themselves. The industry seems to be embracing royalty free codecs. You needn't look any further than Opus to see that. xHE-AAC outperforms Opus. Especially at very low bitrates. However, the industry sees Opus as good enough and it has received wide adoption while xHE-AAC has revived very limited adoption. And now for speech we apparently have neural net codecs that are royalty free and can work at absurdly low bitrates.

So, even though VVC/H.266 outperforms AV1 I could see the industry saying, "that's cool but this is good enough. We're not doing that again. We'll be making our next codec ourselves to."

I think our future is going to be filled with royalty free codecs that are standard and widely supported.

Thank you for reading my blog post.
internet.jpg

@sjw
I'm curious why you think specifically MPEG 2 and AC3 aren't going away.

AVIF is amazing. Did some testing with it, and it performs great. I am so god damned happy to see a good JPEG replacer. I should look at browser support, last time I checked it was still lacking. JPEG2000 is already old as fuck itself and I never saw any reviews of it saying it makes a difference. As for WEBP, I want it to be good, but it's not really. Sure it gets smaller files, but it obliterates image quality. I've looked at it as a GIF replacer but even there it's lacking sometimes, as lossless WEBP can't match a GIF if the animation is low palette, and sometimes you do want it to be lossless.

Only heard about LYRA2 yesterday, and had no idea H266 or xHE-AAC was a thing. I'm curious if LYRA or LYRA2 work well for other content besides voice (i.e. music) like opus does. If yes, I could see it as a benefit to help it gain traction. Otherwise, seeing how it's Google, I think it will be as ignored as WEBP.
Unless xHE-AAC is compatible with previous AAC hardware, I don't give a shit.
As for H266, AV1 already has hardware encoders in modern GPUs. I think that alone will decide this battle.

I loved your codec ramble. 10/10 would read again.