# Producer Brief — Episode 004: Mixture of Agents

## Topic
Mixture-of-Agents (MoA): orchestrating a *team* of LLMs that refine each other's outputs in layered stages, instead of relying on one "jack-of-all-trades" model. The claim — and the 2024-2025 results backing it — is that a layered council of models (often open-weight) can match or beat a single frontier proprietary model on benchmarks.

## Why It Matters
This is the architectural bet behind "combine models, don't pick one." If it holds at scale, it changes procurement (cheap local models + a smart aggregator > one big API bill), it changes reliability (different failure modes cancel out), and it changes what "a model" even means to a builder. But it also multiplies latency, token cost per call, and orchestration complexity. The open question isn't *can* it beat a single model — it's *when is the overhead worth it*.

## Seed Question (goes to Leonard first)
"Everyone's excited about Mixture-of-Agents — instead of one model, you stack a layer of models that read each other's drafts and refine. The paper shows open-source models in a council beating a single proprietary frontier model on benchmarks. So is this the future of how we build, or is it a benchmark hack that falls apart the moment you care about cost-per-token and latency in production?"

## Context Notes (background depth — NOT a script)
- MoA paper (arXiv 2406.04692, June 2024): layered architecture, each layer = multiple LLM agents; each agent takes all previous-layer outputs as auxiliary context. Reported surpassing GPT-4-class models on several benchmarks using open models.
- Trend into 2025-2026: Mixture-of-LLMs active learning — ensembles matching/surpassing single-LLM and human annotation accuracy at lower compute, especially with lightweight locally-deployable models (Jan 2026).
- Failure modes to actually probe: layer-after-layer token cost, latency stacking, and whether "refinement" is real improvement or just stylistic convergence (all agents sound the same after 3 layers).
- Real builder angle: Hermes itself fans out to sub-agents — is that MoA or just parallelism? The distinction (refinement vs. independent delegation) is worth them arguing about.