vault-sync: 2026-10-14 11:24 (leonard)

2026-10-14 11:24:10 -07:00
parent 1c08293902
commit d3a3086e21
30 changed files with 2942 additions and 518 deletions
@@ -0,0 +1,89 @@
+# Apollo — Persistent Voice Agent
+
+**Phase 3** · Built 2026-10-14
+
+## Architecture
+
+Local voice loop: STT → LLM → TTS with barge-in.
+
+```
+Microphone → faster-whisper (STT) → LLM endpoint → edge-tts (TTS) → ffplay
+                    ↑                                              |
+                    └────────── barge-in detection ────────────────┘
+```
+
+## Stack
+
+| Component | Tool | Why |
+|-----------|------|-----|
+| STT | `faster-whisper` (tiny) | Local, fast, no API key, English-optimized |
+| LLM | Configurable endpoint | Defaults to `http://127.0.0.1:1920/v1/chat/completions` (LFM2.5) |
+| TTS | `edge-tts` 7.2.7 | Free, 6 high-quality neural voices, local |
+| Audio capture | `sounddevice` | Cross-platform, low latency |
+| Playback | `ffplay` (ffmpeg) | Killable subprocess for barge-in |
+| State | `apollo_state.json` | Live status read by dashboard |
+
+## Voices
+
+| Name | Edge TTS ID |
+|------|------------|
+| Aria (default) | `en-US-AriaNeural` |
+| Jenny | `en-US-JennyNeural` |
+| Guy | `en-US-GuyNeural` |
+| Ana | `en-US-AnaNeural` |
+| Christopher | `en-US-ChristopherNeural` |
+| Eric | `en-US-EricNeural` |
+
+## Files
+
+| File | Purpose |
+|------|---------|
+| `~/Projects/hermes-dashboard/apollo/apollo_voice.py` | Voice loop script |
+| `~/Projects/hermes-dashboard/apollo/apollo_state.json` | Live state (auto-generated) |
+| `~/Projects/hermes-dashboard/apollo/apollo_output.mp3` | Last TTS output (auto-generated) |
+
+## Launch
+
+### From dashboard
+Toggle "▶ Start Apollo" in the Apollo card. Select voice first.
+
+### From terminal
+```bash
+cd ~/Projects/hermes-dashboard/apollo
+/path/to/hermes/venv/Scripts/python apollo_voice.py --voice Aria
+```
+
+Stop with Ctrl+C or dashboard "Stop Apollo" button.
+
+## How it works
+
+1. **Continuous listening mode**: Captures mic until 1.5s of silence detected.
+2. **Transcription**: `faster-whisper` tiny model, CPU int8 (fast enough for real-time).
+3. **LLM**: POST to configured endpoint with system prompt for concise voice responses.
+4. **TTS**: `edge-tts` generates MP3, played via `ffplay`.
+5. **Barge-in**: During playback, checks mic energy every 300ms. If above threshold, kills ffplay and listens again.
+
+## Current limits
+
+- **LLM requires a running endpoint**. Without LFM2.5 or similar running on port 1920, Apollo echoes "(No LLM endpoint)" and skips TTS. Start the model first or update `llm_endpoint` in `apollo_state.json`.
+- **Wake word**: Uses continuous VAD (silence detection), not wake-word. Sherpa "hey Jimmy" runs separately in Hermes and is not integrated here.
+- **Streaming**: TTS generates full MP3 before playback starts (not streaming synthesis). Latency is ~2-4s for short responses.
+- **GPU**: faster-whisper runs on CPU (int8). Tiny model is fast enough for real-time use (~200ms for short utterances).
+
+## State file schema
+
+```json
+{
+  "status": "idle|listening|transcribing|thinking|speaking|stopped",
+  "voice": "Aria",
+  "mode": "continuous",
+  "last_utterance": "...",
+  "last_response": "...",
+  "last_active": "2026-10-14T...",
+  "llm_endpoint": "http://127.0.0.1:1920/v1/chat/completions"
+}
+```
+
+## Wiring to dashboard
+
+The Streamlit dashboard Apollo card reads `apollo_state.json` every render and shows live status. Start/stop buttons spawn/kill the Python process directly.