Tony_tech/personal-agent-os
6
0
Fork 0
Code Issues 2 Pull Requests Actions Packages Projects Releases Wiki Activity

vault-sync: 2026-10-16 06:01 (jimmy)

Browse Source
This commit is contained in:
jimmy
2026-10-16 06:01:33 -07:00
parent 5083eca6ae
commit 3557c46c7c
8 changed files with 574 additions and 53 deletions
Download Patch File Download Diff File Expand all files Collapse all files
memory-systems/agent-os/Video Stack/Decisions.md
+2 -1
View File
@@ -1,6 +1,6 @@
---
tags: [video-stack, decisions]
updated: 2026-10-15
updated: 2026-10-16
---
# Decisions
Back to [[Video Stack - Overview]]
@@ -18,3 +18,4 @@ Back to [[Video Stack - Overview]]
| 2026-10-15 | Only tonyjbala Gitea credentials | User rule |
| 2026-10-15 | Each video gets a unique design | Avoid template sameness |
| 2026-10-15 | Political stack kept separate from business stack | Different rules (sourcing, disclaimers, tone) |
| 2026-10-16 | Music = **ACE-Step 1.5** (local, MIT) as primary generator via `C:\tts\acestep_music.py`; code-synth `gen.py` fallback; ElevenLabs Music optional. Music approval gate (Gate 3) unchanged | User: include it in the stack; code-synth music "didn't really do anything for the feel"; ACE-Step is free on the 3060 (~25 s per 45 s track) |
memory-systems/agent-os/Video Stack/Full Stack Spec.md
+24 -8
View File
@@ -1,6 +1,6 @@
---
tags: [video-stack, spec]
updated: 2026-10-15
updated: 2026-10-16
source: explainer-video-stack/STACK.md
---
# STACK.md — Everything Hermes needs to produce Grok-Bot-quality explainer videos
@@ -44,8 +44,9 @@ Plus: three approval gates (script, voice, music) **before** any build, and noth
| Kokoro | kokoro-onnx (box/legacy) | Fast fallback voice for drafts only. **Never use a Kokoro clip as a clone reference** (refs in `C:\tts\voices\kokoro_*_ref.wav` are legacy; don't use them). |
| Human voice ref | `C:\tts\voices\user_announcer_ref.wav` (Anthony, recorded with consent; 20.65 s mono 24 kHz) | The only approved clone reference. |
| Gemini TTS | `C:\tts\gemini_tts.py` (stdlib only; repo copy `scripts/tts/gemini_tts.py`) | **Default voice engine.** Section 2.1. |
| Music (code) | `scripts/music/gen.py` (options A–C), `gen2.py` (D–F) in explainer-video-stack; `compose.py` presets in `C:\video\template` | numpy synth, royalty-free. Writes full track + 20 s preview. |
| Music (paid, optional) | `C:\video\template\el_music.py "<prompt>" <seconds> out.mp3` | ElevenLabs Music, needs `ELEVENLABS_API_KEY`; instrumental only. Use if two synth rounds miss. |
| **ACE-Step 1.5** | `C:\Users\fight\Projects\ACE-Step-1.5` (v1.5.0, commit ca1e85f, uv `.venv`, torch 2.7.1 cu128). Checkpoints: `acestep-v15-turbo` (DiT), `acestep-5Hz-lm-0.6B` (+1.7B present, not used on 12 GB), `Qwen3-Embedding-0.6B`. Server: schtasks **`ACEStepServer`** → `start_acestep_api.bat` → Gradio UI + REST API on **http://127.0.0.1:7860** (log `acestep_server.log`). Wrapper **`C:\tts\acestep_music.py`** (repo copy `scripts/music/acestep_music.py`) | **Primary music generator (local, free, MIT).** ~22–30 s per 45 s instrumental track on the 3060. Section 2.2. |
| Music (code, fallback) | `scripts/music/gen.py` (options A–C), `gen2.py` (D–F) in explainer-video-stack; `compose.py` presets in `C:\video\template` | numpy synth, royalty-free. Use only if ACE-Step is down or misses twice. Writes full track + 20 s preview. |
| Music (paid, optional) | `C:\video\template\el_music.py "<prompt>" <seconds> out.mp3` | ElevenLabs Music, needs `ELEVENLABS_API_KEY`; instrumental only. Optional third choice. |
### 2.1 Gemini TTS (`gemini_tts.py`)
```
@@ -62,6 +63,21 @@ python C:\tts\gemini_tts.py script.json out.wav [--voice Charon] [--style "..."]
- Output: 24 kHz mono 16-bit WAV.
---
### 2.2 ACE-Step 1.5 music (`acestep_music.py`)
- **What:** open-source text-to-music model (ACE-Step 1.5, turbo DiT + 0.6B planning LM). Code and weights are **MIT licensed** (`LICENSE`, `pyproject.toml`, checkpoint model card) → commercial use OK, output is ours. Still: never prompt "in the style of <artist>", never upload copyrighted reference audio.
- **Runs as:** a persistent local server (Gradio UI for humans + REST API for scripts) on `127.0.0.1:7860`, launched with `--enable-api --init_service true --config_path acestep-v15-turbo --lm_model_path acestep-5Hz-lm-0.6B --offload_to_cpu true`. Check: `curl http://127.0.0.1:7860/health` → `"status":"ok"`. If down: `schtasks /Run /TN ACEStepServer` (models load in ~80 s). Do not start a second copy.
- **VRAM coexistence:** it shares the 3060 with the one Qwen llama-server (port 1922). Idle ~8.5–10 GB total, **peak ~11.7 / 12 GB during generation** (CPU offload is on). It is not an LLM, so it does not break the one-model rule, but: never run ACE-Step generation at the same time as Qwen3-TTS or Chatterbox (OOM risk) — generate music, then voice, one job at a time. Never start Gemma or a second llama-server.
- **Command (one track = full + 20 s preview, -18 LUFS mp3):**
```powershell
$AP="C:\Users\fight\Projects\ACE-Step-1.5\.venv\Scripts\python.exe"
& $AP C:\tts\acestep_music.py --prompt "serious civic political ad underscore, cinematic orchestral, low strings, steady timpani pulse, building to a resolute swell, no vocals" --duration 47 --bpm 80 --key "D minor" --out C:\video\myproj\music\optA
# -> optA.mp3 (full, -18 LUFS two-pass loudnorm), optA_preview.mp3 (last 20 s; --preview-start N to choose), optA_raw.wav (use raw/full in the mix)
# options: --seed N (repeatable), --steps 8 (turbo default), --thinking (LM planning; slower), --lufs -18
```
Prints JSON with `gen_seconds`. Always instrumental (sends `lyrics="[Instrumental]"`).
- **Prompt tips (ACE-Step expects comma-separated tags, not sentences):** genre/use, mood, instruments, tempo feel, arc, then `no vocals`. e.g. business: `modern, warm, upbeat corporate tech, light synth pulse, piano, no vocals` (`--bpm 110`); political: as above (`--bpm 70-90`, minor key). Put BPM/key in the flags, not the prompt. Keep it to 6–12 tags. For 3 options at Gate 3, vary the tags (e.g. strings vs piano vs brass) or seeds. It doesn't follow exact timestamps, so pick the take whose swell lands near the CTA, or trim/offset the start so it does. Seconds-level timing still comes from the edit.
- **Measured (2026-10-16):** 45 s track in 22–30 s wall time; output 48 kHz stereo.
## 3. The end-to-end pipeline (with gates)
```
@@ -94,7 +110,7 @@ Render line 1 (or the first ~15 s) with the preset voice (Section 4.4). If the u
**Done when:** the user approves one voice by name.
### Step 5 — Music previews → **GATE 3**
Write a timed music brief first (e.g. "0–6 s sparse tension; 15–30 s driving low pulse under the cost; 38–45 s resolute swell, resolve on the cut"). Make **3 distinct options**, 20 s previews of the most important section, mp3. One-line description each.
Write a timed music brief first (e.g. "0–6 s sparse tension; 15–30 s driving low pulse under the cost; 38–45 s resolute swell, resolve on the cut"). Make **3 distinct options** with **ACE-Step** (code synth only as fallback), 20 s previews of the most important section, mp3. One-line description each.
**Done when:** the user picks one (or asks for changes; iterate).
### Step 6 — Full VO + Whisper check → `vo\v1..vN.wav`, `vo\words.json`
@@ -163,7 +179,10 @@ Done: every HEARD line matches WANT (digits vs words OK). Join lines with 200 ms
### 4.5 Music previews
```powershell
# code-synth options (edit the arc/section times to the brief first)
# PRIMARY: ACE-Step (Section 2.2), 3 options with different tags/seeds
$AP="C:\Users\fight\Projects\ACE-Step-1.5\.venv\Scripts\python.exe"
foreach($o in @(@("A","<tags A>, no vocals"),@("B","<tags B>, no vocals"),@("C","<tags C>, no vocals"))){ & $AP C:\tts\acestep_music.py --prompt $o[1] --duration 47 --out C:\video\myproj\music\opt$($o[0]) }
# FALLBACK code-synth options (edit the arc/section times to the brief first)
cd C:\video\myproj\music
python <explainer-video-stack>\scripts\music\gen.py # A–C
python <explainer-video-stack>\scripts\music\gen2.py # D–F
@@ -282,6 +301,3 @@ Held ≥ 5 s. Political: big **NO/YES** block in the accent color, "ON PROP N",
- `C:\video\` — `examples\prop1ad`, `template` (Remotion), `voice_tests` (B1–B10, P1–P4, NOTES.md), `COPYWRITING.md`, `HERMES_PROMPT.md`, `STACK.md` (this file)
- Hermes profile — `C:\Users\fight\AppData\Local\hermes\profiles\explainer\` (`SOUL.md`, `STACK.md`, `.env`, `skills\media\explainer-video\SKILL.md`)
- Obsidian — `C:\Users\fight\Documents\Obsidian Vault\Video Stack\Full Stack Spec.md`
Back to [[Video Stack - Overview]]
memory-systems/agent-os/Video Stack/Tools and Setup.md
+11 -1
View File
@@ -1,6 +1,6 @@
---
tags: [video-stack, setup]
updated: 2026-10-15
updated: 2026-10-16
---
# Tools and Setup
Back to [[Video Stack - Overview]]
@@ -27,3 +27,13 @@ Back to [[Video Stack - Overview]]
- Free tier: 10 requests/day on flash-tts (resets ~4:50 PM PT); flash-lite-tts 3/min. Script auto-falls back to Lite.
- Style must go in `speech_metadata.style` (interactions endpoint), never in the text.
- `GEMINI_API_KEY` goes in the explainer profile `.env` (user step; not confirmed done).
## Music: ACE-Step 1.5 (primary, added 2026-10-16)
- Install: `C:\Users\fight\Projects\ACE-Step-1.5` (v1.5.0, commit ca1e85f, `.venv`). License **MIT** (code + weights): commercial use OK.
- Models: `acestep-v15-turbo` DiT + `acestep-5Hz-lm-0.6B` (1.7B LM also on disk, unused).
- Server: scheduled task `ACEStepServer` -> `start_acestep_api.bat` -> Gradio UI + REST API at http://127.0.0.1:7860 (`/health`, `/release_task`, `/query_result`). Log: `acestep_server.log`. Restart: `schtasks /Run /TN ACEStepServer` (~80 s to load). Does not start at boot.
- Wrapper: `C:\Users\fight\Projects\ACE-Step-1.5\.venv\Scripts\python.exe C:\tts\acestep_music.py --prompt "<tags>, no vocals" --duration 47 [--bpm N --key "D minor" --seed N] --out <path-no-ext>` -> full mp3 + 20 s preview at -18 LUFS.
- Speed: 22-30 s per 45 s instrumental. VRAM: shares the 3060 with Qwen (1922); peak ~11.7/12 GB during generation (CPU offload on). Don't run it at the same time as Qwen3-TTS/Chatterbox.
- Test tracks: `C:\video\music\acestep\acestep_business.mp3`, `acestep_political.mp3`.
- Fallbacks: code-synth `scripts/music/gen.py`/`gen2.py`; ElevenLabs `el_music.py` (optional, paid).