@Moon @matrix and also shouldn't compressing a database be seen as a last resort to just expanding the partition?

@lewdthewides
That costs money
@Moon

@lewdthewides @matrix @Moon cost per gig buying at 4tb is ridiculously cheap. It’s never been easier to digitally hoard; use coupons and get a 4TB nas HDD (only ~$100/ ~$120) for basically nothing

@lewdthewides @matrix if you've got cpu cycles to spare, why not? Also pg uses a compression they call TOAST, when you use ZFS compression you turn it off so you're not doing it twice.

@Moon
How big is it a CPU hit? My Masto VPS only really gets pinned when someone posts a bigger non x264 video and it starts transcoding
@lewdthewides

@matrix @lewdthewides I haven't done it but I doubt it's a problem.

So basically if you were doing this you'd be doing it to save space. I can't say what the performance implications are. I've ready typically that copy on write filesystems are bad for databases but there are multiple guides on hosting your postgres db on zfs. Either it's tolerable, or these people aren't having performance problems because they're not taxing the db. pleroma will tax the db. So, I just don't know.

@Moon
Mastodon seems to be more efficient with DB usage than Pleroma, however since server has NVME all the queries are mostlikely CPU bound which means it could perform worse.
I might copy the DB to my PC, spin up VM to test it first.
I have a NVME drive and the same CPU architecture so it should show conclusive results.
@lewdthewides

@matrix @lewdthewides pleroma storing jsonb doesn't seem to be efficient but it's good for other reasons

@Moon
I assume it's better for CPU usage when serving AP activities because it doesn't have to create it, just queries the db and serves it?
@lewdthewides

@matrix @lewdthewides I mean that it's more easily expandable to additional activitypub object types.

@matrix @Moon @lewdthewides could do what I did and get a slab and setup a writeback cache with a partition on the NVME. 99% of the time I got NVME performance.

sjw@bae.st @matrix @Moon that's pretty much what I'm doing here

@Moon
Thanks for that. I'm going to look it up. Normally I wouldn't touch Postgres at all.

@rin @matrix @feld wow maybe I need to set that up

@Moon @rin
ah gosh yes, very interest also; how do?

@rin
100GB Masto backup takes up 17GB and 40GB Misskey takes up 5GB. I'm pretty sure pg_dump doesn't include indexes though.
@feld

@matrix @feld oh, I do backups via barman, so it's more like backing up the postgresql folder with indexes and all that

@matrix @rin @feld it does not

@solarsailor
I have 3 3900X threads. I don't think it should handle it ok.

@matrix@gameliberty.club It's not about CPU usage, it's about write latency. PostgreSQL and other in memory DBs really really really want to have writes that commit to disk quickly, because they are essentially single threaded while writes are in progress. FS compression increases write latency to the degree that its compression is effective - a little bit for LZO, a lot for GZip. The whole reason Intel created NVDIMMs was to reduce database I/O latency, for example.

@solarsailor
I thought it should be usef to some latency since the initial release was all the way back when SSDs weren't a thing (I think)

@matrix@gameliberty.club Half the reason for adoption of SSDs was to make MySQL and PostgreSQL run faster.

@solarsailor @matrix this is why zstd is kinda the best for something like this

@sjw @solarsailor lz4 is probably still the fastest though
https://indico.fnal.gov/event/16264/contributions/36466/attachments/22610/28037/Zstd__LZ4.pdf

@matrix @solarsailor eh, no. LZO-RLE if it supports it. Faster and slightly higher compression.

@matrix @solarsailor block size plays a big role too

@sjw
100GB Mastodon and 40GB Misskey

@matrix Wow, mastodon is that big? Interesting. I wonder why.

I'd still say you're better off with a writeback cache on XFS.

Here's a video that explains it:
https://youtu.be/WEvCxFLHOM0

Basically, you just make a partition on NVME drive, and set it to be a writeback cache. You'll read and write to the NVME and it'll be lazily synced back to the slab.

A disadvantage is a drive failure of the cache drive will almost always result in data loss. NVME is RAID 10 tho so no real worries there.

@sjw @matrix I would also stay away from bcache. Me and zero got burned by filesystem corruption because no one caught a bug when kernel 5.16 dropped. Probably all the qa is focused on bcachefs these days.

Lvmcache should be safer from uncaught bugs just because it's more widely used

@sjw It's not that big. It's 76974356 statuses over 3 years indexed with rum.
I don't really wanna spend more money so I want to better utilize the space I have.
It would probably make sense for that setup to put WAL on a separate drive.

@sjw @matrix LVM snapshots suck tho

@Moon @matrix and they're a fuck when caching

@matrix

dayum, disk space is super expensive with my discount VPS, what are you using for hosting?

@sjw

@ademan @sjw BuyVM
It is expensive. I'm paying $45 for it.

@matrix @ademan @sjw yes, but it's nice knowing that if someone is gonna be pulling the rug from underneath my instance, it won't be buyvm. Also I think it's the best deal if you're planning to remain a small instance, and any future needs will be for more storage.

@lewdthewides

if someone is gonna be pulling the rug from underneath my instance, it won’t be buyvm

Can you elaborate? Are they free speech absolutists or something?

TBH their pricing looks pretty similar to my host, though mine’s more generous with CPU.

@sjw @matrix

@ademan @sjw @lewdthewides Yeah, BuyVM is pretty hands off. Daily Stormer even used them as a proxy.

@matrix @ademan @lewdthewides
BuyVM.png

@sjw

who tf is francisco? missing some lore here

is colocrossing pozzed? I mean we’re pretty mild over here but I don’t want to run any unnecessary risks… maybe I’ll start a more resilient instance this year now that I know some things to look out for, I think .social was a mistake, and I’ll make sure to avoid colocrossing…

@matrix @lewdthewides

@sjw

nvm, finished the thread, Francisco seems like a pretty cool guy eh remains neutral on his customers and doesn’t afraid of anythin

@lewdthewides @matrix

@ademan @matrix @lewdthewides nop he's also been in the hosting game longer than Digital Ocean

@ademan @matrix @lewdthewides Fran runs Frantech which runs BuyVM
get_baest-e8O7.mp4

@sjw @ademan @lewdthewides @matrix wheres the rest, fucker

@Moon @sjw @ademan @lewdthewides @matrix

@Moon @matrix @lewdthewides @ademan that's the whole video she posted

@Moon make her yours and put her in a qipao

@sjw @matrix @lewdthewides

@Moon @ademan @lewdthewides @matrix
get_baest-9KDt.mp4

@Moon @ademan @lewdthewides @matrix
get_baest-G7O6.mp4

@Moon @ademan @lewdthewides @matrix
get_baest-qr9M.mp4

@Moon @ademan @lewdthewides @matrix She seems like you type @Moon
get_baest-Q7fF.mp4

@sjw

in all seriousness though, don’t date egirls

@matrix @lewdthewides @Moon

@ademan @sjw @matrix @lewdthewides @Moon
Why not?
As long as it's all in my head, I don't see the problem XD

@sjw @matrix @lewdthewides @ademan < me after about a dozen rewatches

@ademan @sjw @matrix I wouldn't call them absolutists, but they're more along the lines of "don't make me have to forward info to the cops or lawyers, and there won't be any issue"