@matrixThis is all njudeas fault for being stingy with the vram. With The way they fuck normal people when it comes to that I don't want to imagine how much profit they make per GB on ai cards and systems for businesses.
@special-boy@matrix my understanding, possibly wrong, is that the HBM memory can't just be "increased" without all kinds of issues, possibly having to do with fanout or timing.
LLMs would already be close to AGI if they could remember larger contexts