| 2026-10-15 | Business voice = **Gemini Kore** (B6), style "friendly, polished product explainer, conversational, upbeat but credible" | User picked B6 from B1-B10 ([[Test Scorecards]]) |
| 2026-10-15 | Political voice = **Gemini Charon** (P1), newscaster style ("measured, authoritative newscaster, confident, serious") | User picked P1; authoritative male fits political ads |
| 2026-10-15 | Local fallbacks: Chatterbox cloning the user's own recording (`C:\tts\voices\user_announcer_ref.wav`) and Qwen3-TTS | Free on the 3060; real human reference beats Kokoro refs |
| 2026-10-15 | **HyperFrames** default renderer, Remotion fallback | Rendered Prop 1 first try in 31 s; one plain HTML file per scene is easier for local models |
| 2026-10-15 | Music ~12 dB under the voice with ducking; final mix -16 LUFS | Voice clarity; consistent loudness |
| 2026-10-15 | User samples and approves voice and music (and script) before any build | Avoid wasted builds; user taste drives choices |
| 2026-10-15 | Political ads: serious civic style, no cheesy stings | User: "This is a political video. The music does not work." |
| 2026-10-15 | Only ONE llama-server on TonyB (Qwen, port 1922) | 12 GB 3060 VRAM; TTS must fit alongside |
| 2026-10-15 | Only tonyjbala Gitea credentials | User rule |
| 2026-10-15 | Each video gets a unique design | Avoid template sameness |
| 2026-10-15 | Political stack kept separate from business stack | Different rules (sourcing, disclaimers, tone) |
# STACK.md — Everything Hermes needs to produce Grok-Bot-quality explainer videos
Audience: the Hermes **`explainer`** profile on **TonyB** (Windows 11, RTX 3060 12 GB), running a cheap model (e.g. `gpt-6-luna`).
This is the single source of truth. When another doc disagrees with this one, **this one wins**.
Repos: `tonyjbala/explainer-video-stack` (business explainers) and `tonyjbala/political-explainer-stack` (ballot/political ads).
Worked example to copy: `C:\video\examples\prop1ad` (HyperFrames, 45 s, 9:16).
Last updated: 2026-10-15.
---
## 1. Overview and the quality bar
A "Grok-Bot-quality" video is a **30–90 s explainer or ad (default 45 s, 9:16, 1080x1920)** that passes all six tests:
| # | Bar | What it means in practice |
|---|---|---|
| 1 | **Verified facts** | Every number and claim has an opened, primary source (LAO, official voter guide, SOS, agency data, product docs). Opinion is labelled or phrased as opinion. A short source line sits under every stat on screen. |
| 2 | **Natural VO** | Written for the ear (Section 5.1). No misreads (Whisper-checked), no odd pauses, no read-aloud stage directions. Political = authoritative male (Gemini **Charon**); business = firm, friendly female (Gemini **Kore**). |
| 3 | **Music that adds feel** | Timed to the beats: pulse/tension under the problem or cost, a swell under the CTA, resolves on the cut. A generic bed that "doesn't do anything" is a fail (Prop 1 lesson). |
| 4 | **Kinetic but clean design** | One idea per scene, huge numbers, staggered entrances, a palette made for the topic, nothing inside the platform UI safe zones, no dead half-screens. |
| 5 | **Perfect captions** | Burned-in, 3-word groups, active word highlighted, timed from Whisper word timestamps, spelled correctly ("$500M", "Prop 1", not "Prop.", not "$5"). |
| 6 | **Broadcast loudness** | Music ~12 dB under voice with sidechain ducking; final **-16 LUFS integrated, true peak ≤ -1.5 dBTP**, AAC 192k 48 kHz; last word ends ~2 s before the cut. |
Plus: three approval gates (script, voice, music) **before** any build, and nothing is ever posted or sent by you.
---
## 2. Hardware and software inventory (verified on TonyB 2026-10-15 unless marked)
| Item | Version / path | Notes |
|---|---|---|
| GPU | NVIDIA GeForce RTX 3060, **12288 MiB**, driver 610.88 | Qwen alone uses ~6.4 GB baseline (≈11.5 GB with KV in use). TTS models need 3.9–5.0 GB. |
| LLM | `C:\llama\llama-server.exe`, **Qwen3.8-35B-A3B-Q4_K_M on 127.0.0.1:1922** | The only model the video work assumes. Health: `http://127.0.0.1:1922/health`. |
| Qwen3-TTS | venv **`C:\tts\qwen3tts`** (uv, torch 2.6 cu124, `qwen-tts`) → `C:\tts\qwen3tts\Scripts\python.exe`; 1.7B VoiceDesign + 1.7B Base | Second local option; voice-from-description. Slow (~2.5–5 min per 15–30 s), 4.3–5.0 GB, no flash-attn on Windows. Runner: `C:\tts\vt\vt.py` (env `VREF`/`VTAG`; engines `cb|qwen|qclone`). |
| Whisper | **faster-whisper `small.en`, int8, CPU** | Built into `tts.py`; produces `words.json``{"1": [[word,start,end],...]}`. |
| Kokoro | kokoro-onnx (box/legacy) | Fast fallback voice for drafts only. **Never use a Kokoro clip as a clone reference** (refs in `C:\tts\voices\kokoro_*_ref.wav` are legacy; don't use them). |
| Human voice ref | `C:\tts\voices\user_announcer_ref.wav` (Anthony, recorded with consent; 20.65 s mono 24 kHz) | The only approved clone reference. |
-`script.json` = JSON list of lines (or a .txt, one sentence per line). Default: one call per sentence, joined with 200 ms gaps. `--each DIR` also saves `DIR\s01.wav…`.
-`--whole` = **one API call for the whole script** (use this by default; it saves quota).
- On a 429 quota error it retries automatically on `gemini-3.8-flash-lite-tts` (`--fallback-model ""` disables).
- Business (sample B6): `--voice Kore --style "friendly, polished product explainer, conversational, upbeat but credible" --whole`.
- **Quota (free tier):** flash-tts = **10 requests/day**; flash-lite-tts = 3 requests/minute. A per-sentence political ad can eat a whole day. Paid tier costs about $0.01–0.03/min.
- **Key:** `GEMINI_API_KEY` from the environment. Put `GEMINI_API_KEY=...` in `C:\Users\fight\AppData\Local\hermes\profiles\explainer\.env` (or `setx GEMINI_API_KEY "..."`). Never print it, echo it, or commit it.
Political: use `political-explainer-stack\templates\brief.md`.
**Done when:** every field filled or explicitly "ask user".
### Step 2 — Research and fact-check → `vo_review.md`
- Open every source you cite. Prefer primary: LAO ballot analysis, CA SOS Quick Guide / Official Voter Guide, UC Berkeley IGS, agency data, the product's own docs.
- Table columns: beat | original line | suggested accurate line | status (✅/⚠️) + why | source URL. Then a "Sources" list.
- Watch for the Prop 1 traps: a figure that applies to only part of the measure ($500–600M/yr is for the $10B part only), "inflation-adjusted" qualifiers, endorsements you cannot verify (EDC GOP's own slate was not online; attribute only to a guide you actually have).
- Label opinion lines ("doesn't change the rules that limit supply").
**Done when:** every number has a URL, every ⚠️ has a fix.
### Step 3 — Script → `script.json` (2 variants)
Rules in Section 5.1. Structure (political, 5 beats): **hook (0–7 s) → breakdown → cost/stakes → concerns/proof → vote call**. Business: hook → problem → how it works → proof/benefit → CTA.
~**105 words max for 45 s** (≤150 wpm, ~2 s spare). One JSON string per beat/scene. Numbers spelled as spoken.
**GATE 1:** send both variants (plain text, with word counts and estimated seconds). Stop until the user picks or edits.
### Step 4 — Voice sample → **GATE 2**
Render line 1 (or the first ~15 s) with the preset voice (Section 4.4). If the user wants options, render 2–3 voices of the same line.
**Done when:** the user approves one voice by name.
### Step 5 — Music previews → **GATE 3**
Write a timed music brief first (e.g. "0–6 s sparse tension; 15–30 s driving low pulse under the cost; 38–45 s resolute swell, resolve on the cut"). Make **3 distinct options**, 20 s previews of the most important section, mp3. One-line description each.
**Done when:** the user picks one (or asks for changes; iterate).
### Step 6 — Full VO + Whisper check → `vo\v1..vN.wav`, `vo\words.json`
Render the whole script with the approved voice, one file per line (or `--whole` then Whisper). Check transcript vs script; re-roll any misread or awkward line.
**Done when:** every line matches (digit-vs-word differences are fine), no pause > ~0.6 s inside a sentence, total VO ≤ length − 2 s.
### Step 7 — Scene design
Pick palette, type, motion for **this topic** (Section 5). Plan one scene per VO line, with start = line start − 0.1 s.
Safe zones 9:16: **top 250 px, bottom 430 px, right 90 px** clear. Content box x 80–910, y 380–1260. Captions at y≈1350.
### Step 8 — HyperFrames build
Scene-per-`<div>` pattern from `prop1ad\build.py`: one Python script writes `hf\ad\index.html` from `words.json` + scene HTML. Lint, snapshot stills at key timestamps, look at them, fix, then render silent.
**Done when:** lint clean, stills at hook / each scene / final 2 s look right, `out\silent.mp4` is exactly the target length.
### Step 9 — Mix
Exact ffmpeg in Section 4.9. **Done when:** I = -16 ±1 LUFS, TP ≤ -1.5 dBTP, voice clearly on top, music audible in gaps and swelling at the CTA.
### Step 10 — Final QA → deliver → **GATE 4**
Run the checklist (Section 4.10). Deliver `out\<project>_45s_vertical.mp4` (H.264 + AAC). If a 16:9 version is wanted, **re-lay out** the scenes for 1920x1080 (new positions, side-by-side panels); never just crop or pad the vertical. Send the path + report to the user; never post it.
Done: every HEARD line matches WANT (digits vs words OK). Join lines with 200 ms silence for `vo\voice.wav` if you rendered per line (gemini_tts.py does this already).
### 4.5 Music previews
```powershell
# code-synth options (edit the arc/section times to the brief first)
If the VO doesn't start at 0 s, delay it first: `-af "adelay=200|200"` on the voice or build `voice.wav` with the exact line offsets used in `build.py` (`ST` list).
Done: I -16 ±1, Peak ≤ -1.5, listen once end to end.
### 4.10 Final QA checklist
- [ ] Every fact sourced; source line on screen; "Paid for by …" + "Demo, not official" if political/demo.
- [ ] Gates 1–3 approved (script, voice, music).
- [ ] Whisper matches script; listened end to end; no awkward pauses.
- [ ] Captions match audio word for word, 3-word groups, active word highlighted, inside safe zones.
- [ ] Exact duration; last word ends ≥ ~2 s before the cut; music resolves on the cut.
- [ ] -16 LUFS ±1, TP ≤ -1.5 dBTP; music audible but never masking VO.
- [ ] H.264 + AAC, 1080x1920, file name ends in `.mp4`, plays on a phone.
- [ ] Exactly one llama-server (1922) still running; no leftover scheduled tasks.
- [ ] Score the run in `scorecard.md` (copy, voice, music, visuals, speed, cost; 1–5).
---
## 5. Design system
### 5.1 Copywriting (write for the ear)
1. Sentences ≤ 12 words. One idea per line; one line = one breath.
2. Numbers as spoken: "ten billion dollars", "twenty twenty-six", "Prop One". Never two numbers in one sentence if avoidable (Prop 1 line 3 had four → awkward).
3. No comma chains or read-aloud lists; split them.
4. Contractions (it's, don't, you'll). Lead with the stake in the first 2 s.
5. 1 s breath before the CTA. ≤150 wpm; ~105 words per 45 s.
6. Two variants every time. Read aloud (TTS) and time against words.json before designing.
### 5.2 Palette, type, motion — chosen per topic
- Start from the topic's emotion: civic/serious (navy, red, cream), local/warm (greens, sand), tech/fintech (deep ink + one electric accent). One dark base, one light text, **one accent**, one highlight color for the active caption word. Check contrast (white on base ≥ 7:1).
- Type: one heavy sans (`"Liberation Sans", Arial` weight 800–900 for numbers/headlines, 700 for subs). Sizes for 1080x1920: kicker 30 px letter-spaced caps; sub 46 px; hero numbers **128–280 px**; notes/sources 26–28 px (≥26 px).
- Motion (GSAP): each scene has `a1/a2/a3` elements entering at **+0.2 / +0.7 / +1.2 s**, `opacity 0→1, y 40→0, 0.5 s, power2.out`. No bounces or spins for political. Business can add a count-up or slide, still short (≤0.6 s).
- Never reuse Prop 1's look unchanged for another topic.
### 5.3 Captions
Built from Whisper `words.json`: group **3 words**, show from first word start to min(next group start, last word end + 0.4 s); active word tweens to the highlight color (`#FFD34D` in Prop 1) over 0.08 s at its timestamp. 60 px, weight 800, centered, `text-shadow:0 4px 12px #000`, box x 80–910 at y 1350. Clean tokens: "Prop." → "Prop", fix currency splits ("$5"+"00" → "$500"), attach "%"/"." to the previous token.
### 5.4 Kinetic numbers and data callouts
One number per beat, hero size; proportional bars (`$10B` 820 px vs `$1.25B` 103 px); `stat` blocks with a 10 px accent rule; "✕" concern cards (max 3); a source note under each.
### 5.5 End cards
Held ≥ 5 s. Political: big **NO/YES** block in the accent color, "ON PROP N", sponsor, close line, bordered box "Paid for by <exact committee name> · Demo, not official / Source: …". Business: logo/name, one-line value, CTA (URL), no clutter.
1.**One llama-server only: Qwen on 1922.** Never start Gemma (1923) or any second model alongside. Don't stop or start models without asking the user. (Don't run `C:\llama\restore_llama_servers.bat` — the TonyB copy still starts Gemma too.)
2.**Long runs via one-time `schtasks`**, then delete the task.
3.**Never post, send, publish, upload or share** without explicit approval. No `hyperframes publish`.
4.**Gitea: tonyjbala account only.** Never use Ty / Ty_Tech credentials.
5.**No Kokoro-made clone references.** Clone only from `C:\tts\voices\user_announcer_ref.wav` (consented) or documented public-domain audio. Never clone an identifiable person without written consent.
6.**Keep tokens low:** reuse templates, read files only as needed, stills instead of test renders, regenerate only the broken line.
7. Never print or commit keys (`GEMINI_API_KEY`, `ELEVENLABS_API_KEY`); no media files (mp4/wav/mp3) in git.
---
## 7. Known pitfalls and fixes
| Pitfall | Fix |
|---|---|
| **Blank HyperFrames scenes** — scenes render empty because the GSAP selector doesn't match (e.g. targeting `.a1` without the scene id, or an id that doesn't exist), so elements stay at the CSS start state / are never animated in. | Every scene div needs a unique `id="sN"` + `class="scene clip"` + `data-start`/`data-duration`; animate with `"#sN .a1"`. Register `window.__timelines["main"]=tl` and `tl.set({},{},<duration>)`. Run `lint`, then `snapshot` mid-scene to confirm content is visible. |
| **Gemini reads directions aloud** ("Say in a calm voice: …" gets spoken) | Text = only words to be spoken. Delivery goes in `--style` (sent as `speech_metadata.style`). systemInstruction is rejected. |
| **Gemini quota (429)** — 10/day on flash-tts | Use `--whole`; script auto-falls back to Flash-Lite (3/min). The daily quota resets once a day (observed around 4:50 PM PT). Fallback: Chatterbox with human ref. |
| **Whisper number formatting** ("350,000" vs "three hundred fifty thousand") | Not a misread. Compare normalized; for captions show digits (they read better on screen). |
| **Awkward VO** (odd pauses, flat delivery) | Shorter sentences, fewer numbers per line; one sentence per call; re-roll seed; never a Kokoro reference. |
| **Tight ending** (slower voice runs into the cut) | Budget ~2 s spare; trim words, or speed VO ≤3% (`atempo=1.03`); never cut the last word. |
| **Windows mp4 extension** — Explorer hides extensions, so a file can end up `name.mp4.mp4` or without `.mp4` and won't play on the phone | Always pass an explicit `-o ...\name.mp4`; verify with `Get-ChildItem out` (shows full names) and `ffprobe`. |
| Remote shell jobs die on disconnect | schtasks (Rule 2). |
---
## 8. How a cheap model should work
1.**One small step at a time.** Do one numbered step from Section 3, write its output to a file, then verify it (read the file back, run the check command) before the next.
2.**Use the "Done when" lines and the QA checklist literally.** If a check fails, fix only that thing.
3.**Copy, don't invent.** Start from `prop1ad\build.py`, the templates, and the exact commands above; change values, not structure.
4.**Stop at every GATE** and ask the user in one short message with the files/paths. Don't build ahead of approvals.
5.**Small context.** Don't paste whole files or logs; use `-Tail`, `Select-String`, and stills.
6.**When unsure, ask.** If a source can't be opened, a tool errors twice, or VRAM is short: report it and ask; don't improvise around a rule.
7.**Report at the end:** mp4 path, duration, LUFS/TP, voice + settings, final script, sources, claims softened, scorecard.
- A Kokoro-made reference makes Chatterbox clones flat with odd pauses. Use a real human reference (the user's recording worked well; leave more headroom, it peaked at -0.1 dBFS).
- Generate one sentence at a time (best of several seeds, ~200 ms gaps) to avoid strange pauses.
- Copywriting: short spoken sentences, numbers written as spoken words, ~105 words per 45 s, leave ~2 s spare. Number-heavy long lines made Prop 1 sound awkward.
- Gemini reads directions aloud unless they go in the style field.
- Music must add feel: civic_tension and piano B were rejected; option A (dark low strings, slow heartbeat pulse) chosen for Prop 1. If code-composed music misses twice, use a richer generator.
- ElevenLabs free tier was blocked for TTS ("unusual activity"); not retried.
- HyperFrames is easier for local models than Remotion (plain HTML per scene).
- Pick an authoritative male voice for political ads.
- `C:\tts\tts.py` - Chatterbox voice script (`--ref <wav>` for cloning)
- `C:\tts\gemini_tts.py` - Gemini TTS; `--style` for directions, `--whole` (whole script in one request), `--fallback-model` (switch to Lite when the daily limit is hit)
- `C:\tts\qwen3tts` - Qwen3-TTS venv (1.7B VoiceDesign / Base clone)
File diff suppressed because it is too large
Load Diff
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
jimmy