Apollo — Persistent Voice Agent

Phase 3 · Built 2026-10-14

Architecture

Local voice loop: STT → LLM → TTS with barge-in.

Microphone → faster-whisper (STT) → LLM endpoint → edge-tts (TTS) → ffplay
                    ↑                                              |
                    └────────── barge-in detection ────────────────┘

Stack

Component	Tool	Why
STT	`faster-whisper` (tiny)	Local, fast, no API key, English-optimized
LLM	Configurable endpoint	Defaults to `http://127.0.0.1:1920/v1/chat/completions` (LFM2.5)
TTS	`edge-tts` 7.2.7	Free, 6 high-quality neural voices, local
Audio capture	`sounddevice`	Cross-platform, low latency
Playback	`ffplay` (ffmpeg)	Killable subprocess for barge-in
State	`apollo_state.json`	Live status read by dashboard

Voices

Name	Edge TTS ID
Aria (default)	`en-US-AriaNeural`
Jenny	`en-US-JennyNeural`
Guy	`en-US-GuyNeural`
Ana	`en-US-AnaNeural`
Christopher	`en-US-ChristopherNeural`
Eric	`en-US-EricNeural`

Files

File	Purpose
`~/Projects/hermes-dashboard/apollo/apollo_voice.py`	Voice loop script
`~/Projects/hermes-dashboard/apollo/apollo_state.json`	Live state (auto-generated)
`~/Projects/hermes-dashboard/apollo/apollo_output.mp3`	Last TTS output (auto-generated)

Launch

From dashboard

Toggle "▶ Start Apollo" in the Apollo card. Select voice first.

From terminal

cd ~/Projects/hermes-dashboard/apollo
/path/to/hermes/venv/Scripts/python apollo_voice.py --voice Aria

Stop with Ctrl+C or dashboard "Stop Apollo" button.

How it works

Continuous listening mode: Captures mic until 1.5s of silence detected.
Transcription: faster-whisper tiny model, CPU int8 (fast enough for real-time).
LLM: POST to configured endpoint with system prompt for concise voice responses.
TTS: edge-tts generates MP3, played via ffplay.
Barge-in: During playback, checks mic energy every 300ms. If above threshold, kills ffplay and listens again.

Current limits

LLM requires a running endpoint. Without LFM2.5 or similar running on port 1920, Apollo echoes "(No LLM endpoint)" and skips TTS. Start the model first or update llm_endpoint in apollo_state.json.
Wake word: Uses continuous VAD (silence detection), not wake-word. Sherpa "hey Jimmy" runs separately in Hermes and is not integrated here.
Streaming: TTS generates full MP3 before playback starts (not streaming synthesis). Latency is ~2-4s for short responses.
GPU: faster-whisper runs on CPU (int8). Tiny model is fast enough for real-time use (~200ms for short utterances).

State file schema

{
  "status": "idle|listening|transcribing|thinking|speaking|stopped",
  "voice": "Aria",
  "mode": "continuous",
  "last_utterance": "...",
  "last_response": "...",
  "last_active": "2026-10-14T...",
  "llm_endpoint": "http://127.0.0.1:1920/v1/chat/completions"
}

Wiring to dashboard

The Streamlit dashboard Apollo card reads apollo_state.json every render and shows live status. Start/stop buttons spawn/kill the Python process directly.