--- tags: [ops, incident, newsroom, local-web-apps, hermes] created: 2026-10-14 status: resolved category: incident --- # Newsroom launch failure — root cause & fix (2026-10-14) ## Symptom "Newsroom won't launch." The desktop plugin showed a blank / failed frame. ## Root cause **The backing server on port 3520 was simply not running — not a broken app.** The `hermes-newsroom` desktop plugin does not run the app; it only *frames* `http://127.0.0.1:3520/newsroom` in an iframe. When nothing listens on 3520, `curl` returns `000` (connection refused) and the frame renders failed/blank. Three compounding factors made it *look* broken and kept it flapping: 1. **No autostart entry existed.** Nothing started the 3520 server at login, so after any reboot / process exit it stayed down with no way to come back. 2. **Every "fix" I made was a plain background launch.** `bash -lc 'npm run start &'` and `Popen` without a session get **reaped when the `execute_code` kernel dies (or times out)**. Symptom each time: server up, then `000` again a few minutes later with *changed node PIDs*. This is a lifecycle artifact, not an app crash. 3. **Mode was `fixture`.** Even when the server *was* up, it served bundled demo data (22 stories), so the app "worked" but looked empty/sample — `liveSources` defaulted to `false` because no `config/newsroom.local.json` existed. ## Why it *kept* breaking (the real mechanism) - No node/Next.js crash in the Windows Application event log (only an unrelated RazerCortex .NET error). - Each manual launch inherited my kernel's session → killed on kernel reap. - Only a launch in its **own process group** (the autostart `.cmd`'s `start ""` detachment, or `start_new_session=True`) survives. Current listener `node.exe` 22228, parent `cmd.exe` (3988) from the launcher — independent of the kernel. ## Fix applied 1. **Autostart** — wrote `%APPDATA%\Microsoft\Windows\Start Menu\Programs\Startup\hermes-newsroom.cmd`: ``` @echo off cd /d "C:\Users\fight\src\hermes-newsroom" start "" /min "C:\Users\fight\AppData\Local\hermes\tools\node-26.7.0-win32-x64\node.exe" "C:\Users\fight\src\hermes-newsroom\node_modules\next\dist\bin\next" start --hostname 127.0.0.1 --port 3520 ``` Pinning absolute node + next paths (PATH is not resolved at login). Verified by kill → confirm 000 → re-run the `.cmd` → re-probe 200. 2. **Live sources** — wrote `config/newsroom.local.json` (git-ignored) with `liveSources: true`, watchlists, `socialSources.bluesky=false` (Bluesky public API returns **403** from this box — bot-walled/region; left on it'd just show a provider-error line). Reddit + GitHub + HN are all reachable (200). ## Verified end state (live) - Port 3520 → `200`, title "Hermes Newsroom · Command Center". - Mode `live`, stories 71, sources ['github', 'hackernews', 'reddit']. - Providers: hermes-releases:ok(6), hermes-projects:ok(29), hermes-changes:ok(20), hermes-discussions:ok(3), reddit-search:ok(94). - Sample: "Hermes Agent v0.22.0 — The Threshold Release" (github), live HN/Reddit hits. ## Known / open items (not blocking) - **Bluesky 403** — off. To re-enable, fix egress/region or add a proxy; it's the only social source that fails here. - **Hermes gateway not connected** — "My Hermes Daily" / personal feed stays empty until the local Hermes API gateway is started. Independent of Newsroom itself. - **No server log file.** The `.cmd` launches `next start` to a detached console (`/min`, no capture). Consider redirecting to a log for future triage. ## Key lesson (generalized) For any `127.0.0.1:PORT` server behind a Hermes desktop plugin: **test the port, not the plugin.** Blank frame = server down. A durable launch needs its own process group (`start "" /min` in a Startup `.cmd`, or `Popen(start_new_session=True, stdin/out/err=DEVNULL)`). Always prove durability by re-probing the port in a *separate* call, not just immediately. ## Related - [[2026-10-10-trace-analysis-and-fix]] (different project, same "local service down" class) ## Follow-up hardening (same session) Hardened the autostart launcher so future failures are triage-able and idempotent: - **Event log** — every launch appends a timestamped line to `C:\Users\fight\src\hermes-newsroom\logs\events.log` (`autostart triggered` / `started next start` / `already running - skip`). This is the durable trail the incident lacked. - **Idempotency guard** — before spawning, checks `netstat` for a listener on :3520; if one exists it logs "already running - skip" and exits. No duplicate servers. - **Server output** — `next start` stdout/stderr → `logs\server.log` (Next.js buffers the "Ready" line, so it's sparse, but captures crashes/errors). - Verified both branches: kill → launcher → 200 (start); re-run while up → skip, still exactly 1 listener (guard). Both survived the agent kernel (parent = cmd.exe). ## Gateway fix (session 2 — "proceed with fixes") Enabled the Hermes **API Server** so the Newsroom personal feed can reach the local gateway. - **Why it was disconnected:** Newsroom's `src/newsroom/hermes/server.ts` reads `~/.hermes/gateway_state.json` → `platforms.api_server.listener_base` + `API_SERVER_KEY` from the profile `.env`. Neither existed → "Hermes gateway is unavailable". - **Fix:** 1. Appended to `C:\Users\fight\AppData\Local\hermes\.env`: `API_SERVER_ENABLED=true` + a generated `API_SERVER_KEY` (48-hex). 2. Added `platforms.api_server` (`enabled:true, host:127.0.0.1, port:8642`) to `config.yaml` (the `.env` toggle alone didn't survive the gateway's direct-spawn re-ingest). 3. `hermes -p default gateway restart` → `[Api_Server] API server listening on http://127.0.0.1:8642`, `gateway_state.json.listener_base` now set. - **Verified end-to-end:** - `GET http://127.0.0.1:8642/health` → `200` - `POST /v1/chat/completions` (Bearer key) → returned text, real usage (16k prompt tokens) - Newsroom `GET /api/newsroom/hermes` → `{"connected":true,"message":"Hermes connected"}` - **Note:** the API server is per-profile-multiplexed (endpoints `/p//v1` under the shared listener); the base `/v1` + `/health` work for Newsroom. A `Scheduled Task: Hermes_Gateway` autostarts it, so it survives reboots. ## Bluesky 403 — definitive (not fixable from this box) - `public.api.bsky.app` **root → 200**, but **all `app.bsky.*` API routes → 403** (Cloudflare bot-wall on this egress IP/region); `www.bsky.app` unreachable. - No egress proxy available (`hermes egress` binary missing, no `HTTPS_PROXY` set). - **Decision: keep `socialSources.bluesky=false`.** Re-enabling requires a working outbound proxy or a different egress — out of scope here. GitHub/HN/Reddit cover the social/dev news; this is cosmetic. ## Final state - Newsroom: **live**, 75 stories (github + hackernews + reddit), personal feed now reaches Hermes (`connected:true`). - Autostart + event/idempotency logging in place; API server + Newsroom server both durable across reboots.