Hey @lanodan how long should I expect the 20200602125218_backfill_notification_types migration to take?

@sjw Uuh… quite a long time but IIRC that's not merged in develop you absolute mad waifu.

@lanodan Got added a few hours ago
https://git.pleroma.social/pleroma/pleroma/-/commit/7aa6c82937090ca6f2298dee0ef894954ca2f129

@sjw Aaah different migration my bad…

I think it should be long but feel free to report once it's done.

@lanodan Been going for about an hour and a half now

@sjw whaaaat.

@lanodan Yeah... However, my database is over 100 GB these days.

@sjw Well I looked at the migration again and it's not that surprising given a very active instance.

Can you file an issue?

@lanodan For what exactly? Just "this migration takes too long!"? I don't really know if it's much of an issue. Maybe add a warning or something but still kinda unavoidable assuming it's actually working.

@sjw Well it’s just that I’ll probably forget about it in few minutes and you probably have more information than me for this.

Also can you do something like:

SELECT count(id) FROM notifications;

or the size of notifications with a \d+ so we have an idea of the size of your table.

@lanodan Got 846638.
Output of \d+ notifications:

@sjw I meant \d+ alone but the count should be enough.

@lanodan Oh! It's 130 MB

@lanodan Still going...

@sjw Holy shit…

@lanodan I'm wondering if there's something wrong... Surely it shouldn't take this long should it?

@sjw

I'm quite bad at knowing how much a SQL command takes, I often try normal programming modifications and then EXPLAIN.

@lanodan Well, postgres isn't really using any CPU.

@sjw Does this htop has detailed CPU time? (so you can see iowait)

@lanodan Yeah. I've also got netdata running on the server
https://neckbeard.xyz/netdata/#menu_system;theme=slate;help=true;update_always=true

@lanodan Postgres logs are just filled with stuff about @igel's broken accountJun 10 15:40:43 neckbeard.shittyurl.org postgres[2520292]: 2020-06-10 15:40:43.220 PDT [2520292] STATEMENT: INSERT INTO "user"("id", "createdAt", "updatedAt", "lastFetchedAt", "username", "usernameLower", "name", "followersCount", "followingCount", "notesCount", "avatarId", "bannerId", "tags", "avatarUrl", "bannerUrl", "avatarColor", "bannerColor", "isSuspended", "isSilenced", "isLocked", "isBot", "isCat", "isAdmin", "isModerator", "emojis", "host", "inbox", "sharedInbox", "featured", "uri", "token") VALUES ($1, $2, DEFAULT, $3, $4, $5, $6, DEFAULT, DEFAULT, DEFAULT, $7, $8, $9, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, $10, $11, $12, DEFAULT, DEFAULT, DEFAULT, $13, $14, $15, DEFAULT, $16, DEFAULT) RETURNING "followersCount", "followingCount", "notesCount", "tags", "isSuspended", "isSilenced", "isLocked", "isBot", "isCat", "isAdmin", "isModerator", "emojis"
Jun 10 15:42:58 neckbeard.shittyurl.org postgres[2809194]: 2020-06-10 15:42:58.450 PDT [2809194] ERROR: duplicate key value violates unique constraint "IDX_5deb01ae162d1d70b80d064c27"
Jun 10 15:42:58 neckbeard.shittyurl.org postgres[2809194]: 2020-06-10 15:42:58.450 PDT [2809194] DETAIL: Key ("usernameLower", host)=(igel, jaeger.website) already exists.
Jun 10 15:42:58 neckbeard.shittyurl.org postgres[2809194]: 2020-06-10 15:42:58.450 PDT [2809194] STATEMENT: INSERT INTO "user"("id", "createdAt", "updatedAt", "lastFetchedAt", "username", "usernameLower", "name", "followersCount", "followingCount", "notesCount", "avatarId", "bannerId", "tags", "avatarUrl", "bannerUrl", "avatarColor", "bannerColor", "isSuspended", "isSilenced", "isLocked", "isBot", "isCat", "isAdmin", "isModerator", "emojis", "host", "inbox", "sharedInbox", "featured", "uri", "token") VALUES ($1, $2, DEFAULT, $3, $4, $5, $6, DEFAULT, DEFAULT, DEFAULT, $7, $8, $9, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, $10, $11, $12, DEFAULT, DEFAULT, DEFAULT, $13, $14, $15, DEFAULT, $16, DEFAULT) RETURNING "followersCount", "followingCount", "notesCount", "tags", "isSuspended", "isSilenced", "isLocked", "isBot", "isCat", "isAdmin", "isModerator", "emojis"
Jun 10 15:42:58 neckbeard.shittyurl.org postgres[2809194]: 2020-06-10 15:42:58.612 PDT [2809194] ERROR: duplicate key value violates unique constraint "IDX_5deb01ae162d1d70b80d064c27"
Jun 10 15:42:58 neckbeard.shittyurl.org postgres[2809194]: 2020-06-10 15:42:58.612 PDT [2809194] DETAIL: Key ("usernameLower", host)=(igel, jaeger.website) already exists.
Jun 10 15:42:58 neckbeard.shittyurl.org postgres[2809194]: 2020-06-10 15:42:58.612 PDT [2809194] STATEMENT: INSERT INTO "user"("id", "createdAt", "updatedAt", "lastFetchedAt", "username", "usernameLower", "name", "followersCount", "followingCount", "notesCount", "avatarId", "bannerId", "tags", "avatarUrl", "bannerUrl", "avatarColor", "bannerColor", "isSuspended", "isSilenced", "isLocked", "isBot", "isCat", "isAdmin", "isModerator", "emojis", "host", "inbox", "sharedInbox", "featured", "uri", "token") VALUES ($1, $2, DEFAULT, $3, $4, $5, $6, DEFAULT, DEFAULT, DEFAULT, $7, $8, $9, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, $10, $11, $12, DEFAULT, DEFAULT, DEFAULT, $13, $14, $15, DEFAULT, $16, DEFAULT) RETURNING "followersCount", "followingCount", "notesCount", "tags", "isSuspended", "isSilenced", "isLocked", "isBot", "isCat", "isAdmin", "isModerator", "emojis"
Jun 10 15:42:58 neckbeard.shittyurl.org postgres[2849948]: 2020-06-10 15:42:58.945 PDT [2849948] ERROR: duplicate key value violates unique constraint "IDX_5deb01ae162d1d70b80d064c27"
Jun 10 15:42:58 neckbeard.shittyurl.org postgres[2849948]: 2020-06-10 15:42:58.945 PDT [2849948] DETAIL: Key ("usernameLower", host)=(igel, jaeger.website) already exists.
Jun 10 15:42:58 neckbeard.shittyurl.org postgres[2849948]: 2020-06-10 15:42:58.945 PDT [2849948] STATEMENT: INSERT INTO "user"("id", "createdAt", "updatedAt", "lastFetchedAt", "username", "usernameLower", "name", "followersCount", "followingCount", "notesCount", "avatarId", "bannerId", "tags", "avatarUrl", "bannerUrl", "avatarColor", "bannerColor", "isSuspended", "isSilenced", "isLocked", "isBot", "isCat", "isAdmin", "isModerator", "emojis", "host", "inbox", "sharedInbox", "featured", "uri", "token") VALUES ($1, $2, DEFAULT, $3, $4, $5, $6, DEFAULT, DEFAULT, DEFAULT, $7, $8, $9, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, $10, $11, $12, DEFAULT, DEFAULT, DEFAULT, $13, $14, $15, DEFAULT, $16, DEFAULT) RETURNING "followersCount", "followingCount", "notesCount", "tags", "isSuspended", "isSilenced", "isLocked", "isBot", "isCat", "isAdmin", "isModerator", "emojis"

@sjw Might explain few things…

@lanodan I think that's just kinda incoming federation while it's running
https://git.pleroma.social/pleroma/pleroma/-/issues/1716

@lanodan Migration just finished.

@sjw yeah, saw it, OOF at the time it took.

@lanodan Looks like 20200527163635_delete_notifications_from_invisible_users is going to be another 20200602125218_backfill_notification_types

@sjw Ouch, could you report it? (I moved my Sunday to today so dayoff)

@lanodan I'll let it go for at least 30 minutes to be sure it's a long boi

@lanodan Nevermind. Only took 08m34s