💀
I've got a Linux question. But first some context.
I'm running Manjaro, on a btrfs filesystem with zstd compression activated.
Since Manjaro is an Arch distro, I get my software updates through pacman, which gets packages also compressed with zstd.
Ever since I switched to compressed btrfs I noticed something: pacman downloads take just as long as before, but the installation itself is basically instant.
So my question is: does my system know that, since the pacman packages have the same compression applied to them as my btrfs filesystem, it doesn't need to decompress the software packages only to recompress them again for btrfs? Does it actually just skips decompressing during install and just plops the files compressed as they are to the disk?
I can't figure out why installs have been so fast for me lately, and this is the only explanation that makes sense to me.
https://wiki.archlinux.org/title/btrfs#Compression
@lewdthewides
>Btrfs and zstd have mechanisms that determine if data should be (re)compressed or not
That's what I would expect as a basic feature of filesystem level compression in general, it just feels unreal that somehow this mechanisms can intervene all the way to pacman, so the system doesn't need to decompress and then recompress zstd.
I expected that pacman would just do it's thing as usual, decompressing the packages, and then btrfs receives the uncompressed files and notices they can compressed back again.
@lewdthewides
When I had ext4, I was still getting my packages compressed with zstd. So it's not from that. Whatever it is, it is tied my filesystem.
@lewdthewides
That could actually help explain it A LOT.
I'm also wondering about the deduplication going on with btrfs. How much of the actual bytes of the updates files are gonna be different? I imagine even some individual files might be the same. With btrfs, I can basically take a 1TB folder of stuff, make a copy on the same disk, and it happens instantly, without even taking extra space, because btrfs is smart enough to know they're the same file, so it just points the new copied folder to the same data already on disk.
"zstd and xz trade blows in their compression ratio. Recompressing all packages to zstd with our options yields a total ~0.8% increase in package size on all of our packages combined, but the decompression time for all packages saw a ~1300% speedup"