# 2026-09-24 — Bonsai 2 27B + MTP: verified end-to-end on the 3060

## Status
The Ternary Bonsai 2 / PrismML stack is **already built and serving** on this box. It was
built 2026-09-19 in WSL2 and re-verified 2026-09-24. Nothing needed downloading.

- Repo (fork, `bonsai2` branch = kernel PRs #218 + Hadamard #217): `/opt/llama-prism-sudoingx`
- Model: `/opt/models/Ternary-Bonsai-2-27B-PTQ1_0-mtp.gguf` (7,012,820,512 bytes, MTP head grafted)
- Launcher: `/opt/serve-bonsai-mtp.sh` — listens **:8899** (not 8080)
- Hermes wiring: `providers.bonsai2` + `model_aliases.bonsai2` -> `http://127.0.0.1:8899/v1`

## Measured 2026-09-24 (RTX 3060 12 GB, driver 610.74)
- Decode: **48.6 – 50.8 tok/s** across 4 runs (~50 tok/s as advertised)
- Prefill: **241 tok/s** cold; KV-cache-warm runs report low prefill because the prefix is cached
- VRAM: **11,166 MiB used / 950 free** at `-c 131072` — weights fully resident, no RAM spill
- `--jinja` tool calling: `finish_reason=tool_calls`, correct function name + JSON args
- Sanity: `17 * 23 = 391` correct

## The one real defect found and fixed: empty `content`
The template default reasoning effort is `xhigh`. At `temperature 1.0` reasoning length varies
wildly (observed 238 chars to 2,524 chars). When it ran long, reasoning consumed the entire
`max_tokens` budget and the response came back `finish_reason=length` with **empty `content`**.

Fix: added `--reasoning-effort medium` to `/opt/serve-bonsai-mtp.sh` (backup at
`/opt/serve-bonsai-mtp.sh.bak-20260924-181142`). After the change: 3/3 realistic ~2.3k-token
prompts returned content, reasoning down to 238–319 chars.

This failure mode looks like two other known Bonsai problems but is neither:
- NOT the `/new` context trap (context 131072 matches alias; `/health` 200)
- NOT the post-abort scheduler wedge (decode speed is normal, slots launch fine)

Diagnose it by comparing `reasoning_content` length against `usage.completion_tokens`.

Escalation ladder if `medium` still overruns: `--reasoning-budget N` (hard cap on thinking
tokens; injects a message before the end-of-thinking tag). Per-request overrides also work —
the fork honours both `reasoning_effort` and `chat_template_kwargs.reasoning_effort` in the
request body, so no restart is needed for a one-off.

## Correction to a common (wrong) playbook for this box
Advice circulating for this model recommends the "official PrismML installer" (`Bonsai-demo`,
PQ2_0, port 8080) as Path A. On THIS machine that path is superseded and weaker: the
`Bonsai-demo` PQ2_0 setup at :8080 is the older, slower variant (~26 tok/s baseline, and the
Vulkan build that ships with it is ~32x slower). The sudoingX PTQ1_0 + MTP build at :8899 is
the canonical one.

Also settled long ago and still true: **a Windows-native CUDA build of the fork is impossible
on this PC** — MSVC 14.44 (VS 18) hard-crashes CUDA 12.4 `cudafe++` (ACCESS_VIOLATION), there is
no admin for CUDA 13 / older MSVC, and the pip `nvidia-cuda-nvcc-cu12` wheels ship no `nvcc.exe`
on Windows (ptxas only). The build lives in WSL2 with `gcc-13` + CUDA 12.4 from apt.

## Gotchas re-confirmed
- WSL2 servers must bind `--host 0.0.0.0`; a 127.0.0.1 bind inside WSL is invisible from Windows.
- Keep the GGUF in WSL ext4 (`/opt/models/`), not `/mnt/c` — 9p/drvfs makes the 7 GB mmap crawl.
- `pkill -f llama-server` **kills your own shell** when the wrapper command line contains that
  string (exit 15). Use the bracket trick: `pkill -f '[l]lama-server'`.
- `pkill -f bonsai` does NOT match the model path `Ternary-Bonsai-...` (case-sensitive).
- Switch to this model with `/new` **before** `/model bonsai2` — a grown session prefills
  slowly at 3060 speeds and can time out mid-load.