ShoutFlow's local mode stands on two open-source projects. WhisperKit turns speech into text on the Neural Engine; MLX runs the language model that turns that raw text into clean prose. Both run entirely on your Mac. This note is a tour of what each layer does and why it is fast enough to feel instant. [1] [2]
WhisperKit: speech to text on the Neural Engine
WhisperKit takes OpenAI's Whisper models and compiles them for Core ML, Apple's inference stack. The heavy lifting lands on the Neural Engine, which is designed exactly for this shape of work, leaving the CPU and GPU free for everything else you are doing. Real-time transcription of a quantised model is comfortably achievable on any recent M-series chip. [1]
Streaming, not batch
Because inference is local, there is no batch boundary to wait behind. WhisperKit can begin emitting tokens a fraction of a second after you start speaking, which is why ShoutFlow's overlay can show a live preview while you are mid-sentence.
MLX: the clean-up pass
Raw transcription is accurate but literal: filler words, false starts, spoken punctuation. The clean-up stage is a small language model running on MLX, Apple's array framework for machine learning on its own silicon. MLX shares memory between CPU and GPU, which keeps a multi-gigabyte model responsive on a laptop. [2]
| Layer | Input | Output |
|---|---|---|
| WhisperKit | 16 kHz microphone audio | Literal raw transcript |
| MLX clean-up | Raw transcript plus tone | Punctuated, de-fillered prose |
| ShoutFlow | Clean prose plus app context | Text inserted at the cursor |
Why open source matters here
A privacy claim is only as good as your ability to check it. Both WhisperKit and MLX are public, audited and maintained in the open: WhisperKit by Argmax, MLX by Apple's own machine learning research team. ShoutFlow's privacy story does not depend on trusting a black box. [3]
The best privacy architecture is the one a sceptical engineer can verify over a weekend.
Honest limits
On-device models are not magic. Very rare vocabulary and heavy accents still favour the largest cloud models, which is precisely why BYOK exists as an opt-in. The full local pipeline requires Apple silicon, and Intel Macs are not currently supported. [4]