vault-sync: 2026-10-02 18:00 (jimmy)

2026-10-02 18:00:15 -07:00
parent 5b3c5768c8
commit e0cf8059b7
6 changed files with 994 additions and 0 deletions
@@ -0,0 +1,88 @@
+# Trace Review Bot — Status
+
+> Session-trace miner + weekly lesson review for the Hermes fleet. Created 2026-10-02 by **jimmy**
+> (Hermes profile `jimmy`, Windows 11). Purpose: turn raw session traces into evidence-backed
+> lesson proposals and deliver them to Tony's Telegram — instead of leaving failures buried in
+> chat history.
+
+## What it is
+
+- **Miner** (deterministic, no LLM): `C:\Users\fight\AppData\Local\HermesHost\diagnostics\trace_review.py`
+  - Scans `state.db` plus every `profiles/*/state.db` over a `--days` window.
+  - Signals: **S1** repeated-error, **S2** retry-loop, **S3** user-correction, **S4** assistant
+    self-correction, **S5** repeated-request.
+  - Every finding carries evidence (`session`, `message_id`, `timestamp`, snippet) and a
+    **falsifier**; anything it cannot evidence is recorded `NOT_AVAILABLE`, never filled in.
+- **Reports**: `...\diagnostics\trace-review\<UTC-stamp>\` → `trace-review.md`, `trace-review.json`,
+  `findings.jsonl` (ledger used for NEW / RECURRING / ESCALATED).
+- **Lesson pass**: the `trace-lessons-review` skill reads the findings and writes `proposals.md`
+  into the same folder. It proposes; it never applies.
+
+## Jobs
+
+| Job | ID | Schedule | Mode | Script / skill |
+|-----|----|----------|------|----------------|
+| Trace miner (daily) | `6b57b2ed2847` | `30 7 * * *` | no_agent | `profiles\jimmy\scripts\trace_review_run.py` |
+| Trace lessons review (weekly) | `9fb3bd266e71` | `0 8 * * 0` | agent | skill `trace-lessons-review` |
+| Trace digest sender (weekly) | `743bc93e1ff5` | `30 8 * * 0` | no_agent | `profiles\jimmy\scripts\trace_digest_sender.py` |
+
+All three run with **native cron delivery** (`deliver: telegram:5789896722`) — the gateway delivers
+the job's stdout. See [[01_Wiki/hermes-cron-delivery-gateway-config]] for the day the delivery
+looked broken and what it actually was.
+
+- `743bc93e1ff5` is **paused**: it was a fallback sender built while delivery looked broken. Kept
+  on disk, resumable if a report ever needs a guaranteed push.
+
+## Delivery path
+
+- `profiles\jimmy\scripts\send_telegram_digest.py` — reads the bot token and home channel from the
+  profile `.env` at runtime, redacts keys/tokens, chunks to Telegram's limit, posts via the Bot API,
+  prints `['ok']` per chunk.
+- Digests are deliberately **snippet-free** (signatures + counts only): error lines can carry
+  credentials. Full evidence quotes stay on disk in the report.
+- `trace_digest_sender.py` is the safety net: it sends any `proposals.md` / `trace-review.md` newer
+  than its `.last-delivered.json` marker, so a report cannot be written and then never delivered.
+
+## Findings so far
+
+2026-10-02, 7-day window: 39 sessions / 4,225 messages / **13 findings** — S3 user-correction 10
+across 5 sessions; S1 `browser_exec` timeouts (5 sessions); S1 terminal 180 s timeouts (5); S4
+self-correction 5; S1 ripgrep search failures (4); approval-gate `BLOCKED` (3); plus retry-loop,
+`command not found` and path-not-found repeats.
+
+Weekly review 2026-10-02 (`...trace-review\20261002T202641Z\proposals.md`): **5 proposals covering
+6 findings, 7 findings dropped** as noise/low-trust.
+
+| # | Proposal | State |
+|---|----------|-------|
+| P1 | Miner defect: S3 counted diagnostic questions ("why did X break?") as user corrections | **Applied** 2026-10-02 |
+| P2 | `browser_exec` hangs to the 420 s batch deadline ignoring `timeout_s` (7× in one week) | Open — needs Tony |
+| P3 | Shell `timeout N` cannot extend the terminal tool's 180 s default | Open — needs Tony |
+| P4 | `search_files` can fail wholesale and still report `total_count: 0` | Open — needs Tony |
+| P5 | `execute_code` kernel does not inherit the shell environment | Open — needs Tony |
+| P6 | `agent-reach` references call `gh` / `mcporter`, neither installed here (exit 127) | Open — needs Tony |
+
+## Lessons (generalizable)
+
+- **Hermes delivers notices in the user role.** Tool timeouts, delegation results and gateway
+  envelopes arrive as `role=user`. Any signal computed over user messages must exclude them, or
+  `[Command timed out after 180s]` counts as a user correction.
+- **Anchored regexes only match the start of a message.** `^\s*(…|again\b)` never caught "why did
+  telegram break again?"; mid-sentence recurrence complaints needed a separate unanchored pattern.
+- **A signal's own precision is falsifiable.** P1 was applied only after checking the 12 matching
+  messages by hand (9 were questions, not corrections) — 44 markers → 40, all user-authored.
+- **A long-running process goes stale across an update.** This session started 08:26:41 and the
+  update landed 08:45:11; every `from <module> import <name>` for a name the update added then
+  failed with "cannot import name", citing a file that does define it. Restart the session (and
+  the gateway) after an update — the update receipt's "mixed sys.modules" warning is the signal.
+- **An agent job can write its report and still skip its own send step.** When delivery is native
+  that no longer matters; while it looked broken, delivery had to be a separate deterministic job.
+
+## Open items
+
+- P2–P6 awaiting accept/reject.
+- Miner gap: "that is wrong" (no apostrophe) still misses `that'?s (wrong|not)`.
+
+## Owner / status
+
+Owner: **jimmy** (Hermes profile `jimmy`). Status: **active**. Created 2026-10-02.