Problem
Mainstream voice assistants send everything you say to someone else's servers. AURA answers the same kinds of questions with the language model and the speech synthesis both running on the user's own machine.
What made you build this — a specific privacy concern, or curiosity about what local models could do? The honest answer is more interesting than the generic one.
Constraints
- Everything runs locally — no cloud inference, so model size is bounded by what Apple Silicon can hold and run at conversational speed.
- Sub-second speech — a voice assistant that pauses before speaking stops feeling like a conversation.
- Two runtimes to bridge — a FastAPI backend serving models and a Next.js interface, which have to stay in step during streaming responses.
Which Mac, how much unified memory, and which Ollama model size? Local-inference claims live or die on the hardware you state.
Architecture
A Next.js interface talks to a FastAPI service that runs the language model through Ollama and synthesises speech with Kokoro-82M — the full loop from speech in to speech out stays on the machine.
Show the request path including where streaming starts, and say how speech-to-text is handled on the way in.
Key Decisions
Why Kokoro-82M for TTS specifically? Why Ollama rather than running the model directly? Both are real choices with alternatives worth naming.
Measured Outcome
External data transmission dropped by 90% against a cloud-assistant baseline, with local text-to-speech responding in under a second.
State how the 90% was measured — traffic captured over a fixed set of interactions, compared against which assistant? A measured number needs its method to survive scrutiny.
What I'd Improve Next
What can't it do that a cloud assistant can? Where does the local model's quality show its limits?