Whisper CPP is now fully CUDA accelerated and it absolutely chews through audio while not using a lot of memory even with the largest model.