# Apollo — Persistent Voice Agent

**Phase 3** · Built 2026-10-14

## Architecture

Local voice loop: STT → LLM → TTS with barge-in.

```
Microphone → faster-whisper (STT) → LLM endpoint → edge-tts (TTS) → ffplay
                    ↑                                              |
                    └────────── barge-in detection ────────────────┘
```

## Stack

| Component | Tool | Why |
|-----------|------|-----|
| STT | `faster-whisper` (tiny) | Local, fast, no API key, English-optimized |
| LLM | Configurable endpoint | Defaults to `http://127.0.0.1:1920/v1/chat/completions` (LFM2.5) |
| TTS | `edge-tts` 7.2.7 | Free, 6 high-quality neural voices, local |
| Audio capture | `sounddevice` | Cross-platform, low latency |
| Playback | `ffplay` (ffmpeg) | Killable subprocess for barge-in |
| State | `apollo_state.json` | Live status read by dashboard |

## Voices

| Name | Edge TTS ID |
|------|------------|
| Aria (default) | `en-US-AriaNeural` |
| Jenny | `en-US-JennyNeural` |
| Guy | `en-US-GuyNeural` |
| Ana | `en-US-AnaNeural` |
| Christopher | `en-US-ChristopherNeural` |
| Eric | `en-US-EricNeural` |

## Files

| File | Purpose |
|------|---------|
| `~/Projects/hermes-dashboard/apollo/apollo_voice.py` | Voice loop script |
| `~/Projects/hermes-dashboard/apollo/apollo_state.json` | Live state (auto-generated) |
| `~/Projects/hermes-dashboard/apollo/apollo_output.mp3` | Last TTS output (auto-generated) |

## Launch

### From dashboard
Toggle "▶ Start Apollo" in the Apollo card. Select voice first.

### From terminal
```bash
cd ~/Projects/hermes-dashboard/apollo
/path/to/hermes/venv/Scripts/python apollo_voice.py --voice Aria
```

Stop with Ctrl+C or dashboard "Stop Apollo" button.

## How it works

1. **Continuous listening mode**: Captures mic until 1.5s of silence detected.
2. **Transcription**: `faster-whisper` tiny model, CPU int8 (fast enough for real-time).
3. **LLM**: POST to configured endpoint with system prompt for concise voice responses.
4. **TTS**: `edge-tts` generates MP3, played via `ffplay`.
5. **Barge-in**: During playback, checks mic energy every 300ms. If above threshold, kills ffplay and listens again.

## Current limits

- **LLM requires a running endpoint**. Without LFM2.5 or similar running on port 1920, Apollo echoes "(No LLM endpoint)" and skips TTS. Start the model first or update `llm_endpoint` in `apollo_state.json`.
- **Wake word**: Uses continuous VAD (silence detection), not wake-word. Sherpa "hey Jimmy" runs separately in Hermes and is not integrated here.
- **Streaming**: TTS generates full MP3 before playback starts (not streaming synthesis). Latency is ~2-4s for short responses.
- **GPU**: faster-whisper runs on CPU (int8). Tiny model is fast enough for real-time use (~200ms for short utterances).

## State file schema

```json
{
  "status": "idle|listening|transcribing|thinking|speaking|stopped",
  "voice": "Aria",
  "mode": "continuous",
  "last_utterance": "...",
  "last_response": "...",
  "last_active": "2026-10-14T...",
  "llm_endpoint": "http://127.0.0.1:1920/v1/chat/completions"
}
```

## Wiring to dashboard

The Streamlit dashboard Apollo card reads `apollo_state.json` every render and shows live status. Start/stop buttons spawn/kill the Python process directly.