# Trace Review Bot — Status

> Session-trace miner + weekly lesson review for the Hermes fleet. Created 2026-10-02 by **jimmy**
> (Hermes profile `jimmy`, Windows 11). Purpose: turn raw session traces into evidence-backed
> lesson proposals and deliver them to Tony's Telegram — instead of leaving failures buried in
> chat history.

## What it is

- **Miner** (deterministic, no LLM): `C:\Users\fight\AppData\Local\HermesHost\diagnostics\trace_review.py`
  - Scans `state.db` plus every `profiles/*/state.db` over a `--days` window.
  - Signals: **S1** repeated-error, **S2** retry-loop, **S3** user-correction, **S4** assistant
    self-correction, **S5** repeated-request.
  - Every finding carries evidence (`session`, `message_id`, `timestamp`, snippet) and a
    **falsifier**; anything it cannot evidence is recorded `NOT_AVAILABLE`, never filled in.
- **Reports**: `...\diagnostics\trace-review\<UTC-stamp>\` → `trace-review.md`, `trace-review.json`,
  `findings.jsonl` (ledger used for NEW / RECURRING / ESCALATED).
- **Lesson pass**: the `trace-lessons-review` skill reads the findings and writes `proposals.md`
  into the same folder. It proposes; it never applies.

## Jobs

| Job | ID | Schedule | Mode | Script / skill |
|-----|----|----------|------|----------------|
| Trace miner (daily) | `6b57b2ed2847` | `30 7 * * *` | no_agent | `profiles\jimmy\scripts\trace_review_run.py` |
| Trace lessons review (weekly) | `9fb3bd266e71` | `0 8 * * 0` | agent | skill `trace-lessons-review` |
| Trace digest sender (weekly) | `743bc93e1ff5` | `30 8 * * 0` | no_agent | `profiles\jimmy\scripts\trace_digest_sender.py` |

All three run with **native cron delivery** (`deliver: telegram:5789896722`) — the gateway delivers
the job's stdout. See [[01_Wiki/hermes-cron-delivery-gateway-config]] for the day the delivery
looked broken and what it actually was.

- `743bc93e1ff5` is **paused**: it was a fallback sender built while delivery looked broken. Kept
  on disk, resumable if a report ever needs a guaranteed push.

## Delivery path

- `profiles\jimmy\scripts\send_telegram_digest.py` — reads the bot token and home channel from the
  profile `.env` at runtime, redacts keys/tokens, chunks to Telegram's limit, posts via the Bot API,
  prints `['ok']` per chunk.
- Digests are deliberately **snippet-free** (signatures + counts only): error lines can carry
  credentials. Full evidence quotes stay on disk in the report.
- `trace_digest_sender.py` is the safety net: it sends any `proposals.md` / `trace-review.md` newer
  than its `.last-delivered.json` marker, so a report cannot be written and then never delivered.

## Findings so far

2026-10-02, 7-day window: 39 sessions / 4,225 messages / **13 findings** — S3 user-correction 10
across 5 sessions; S1 `browser_exec` timeouts (5 sessions); S1 terminal 180 s timeouts (5); S4
self-correction 5; S1 ripgrep search failures (4); approval-gate `BLOCKED` (3); plus retry-loop,
`command not found` and path-not-found repeats.

Weekly review 2026-10-02 (`...trace-review\20261002T202641Z\proposals.md`): **5 proposals covering
6 findings, 7 findings dropped** as noise/low-trust.

| # | Proposal | State |
|---|----------|-------|
| P1 | Miner defect: S3 counted diagnostic questions ("why did X break?") as user corrections | **Applied** 2026-10-02 |
| P2 | `browser_exec` hangs to the 420 s batch deadline ignoring `timeout_s` (7× in one week) | Open — needs Tony |
| P3 | Shell `timeout N` cannot extend the terminal tool's 180 s default | Open — needs Tony |
| P4 | `search_files` can fail wholesale and still report `total_count: 0` | Open — needs Tony |
| P5 | `execute_code` kernel does not inherit the shell environment | Open — needs Tony |
| P6 | `agent-reach` references call `gh` / `mcporter`, neither installed here (exit 127) | Open — needs Tony |

## Lessons (generalizable)

- **Hermes delivers notices in the user role.** Tool timeouts, delegation results and gateway
  envelopes arrive as `role=user`. Any signal computed over user messages must exclude them, or
  `[Command timed out after 180s]` counts as a user correction.
- **Anchored regexes only match the start of a message.** `^\s*(…|again\b)` never caught "why did
  telegram break again?"; mid-sentence recurrence complaints needed a separate unanchored pattern.
- **A signal's own precision is falsifiable.** P1 was applied only after checking the 12 matching
  messages by hand (9 were questions, not corrections) — 44 markers → 40, all user-authored.
- **A long-running process goes stale across an update.** This session started 08:26:41 and the
  update landed 08:45:11; every `from <module> import <name>` for a name the update added then
  failed with "cannot import name", citing a file that does define it. Restart the session (and
  the gateway) after an update — the update receipt's "mixed sys.modules" warning is the signal.
- **An agent job can write its report and still skip its own send step.** When delivery is native
  that no longer matters; while it looked broken, delivery had to be a separate deterministic job.

## Open items

- P2–P6 awaiting accept/reject.
- Miner gap: "that is wrong" (no apostrophe) still misses `that'?s (wrong|not)`.

## Owner / status

Owner: **jimmy** (Hermes profile `jimmy`). Status: **active**. Created 2026-10-02.