# 2026-09-19 — Post-mortem: Ashbot Gate 3 / inbox day (CoS + Hermes)

**Scope:** Phoenix Salon Gate 3 pilot → Birdeye send → inbox replies → STATUS watcher.  
**Outcome:** Birdeye email SENT; inbox pass found no returns; several process failures burned time and trust.  
**Author:** CoS (Chief of Staff) for Gatekeeper Anthony.

---

## 1. Timeline (PT, 2026-09-19)

| Time (approx) | Event |
|---|---|
| ~1:20 | Anthony: "Nothing" — expected Ashbot iMessage after Gate 3 assign |
| ~1:21 | First Gate 3 dispatch **failed in ~5s** (Tailscale SOCKS AssertionError). CoS had implied start; folder never created |
| ~1:32–1:36 | Diagnose → redispatch. Detached `start /B` / Start-Process **did not run**. SSH-sync `_launch.py` finally worked (pid 38648) |
| ~1:36–1:52 | Gate 3 pilot **done**: FindGlocal WONT-DO (canonical); hairsalonsinusa Cloudflare WONT-DO; Birdeye draft only (no send_mail in that session). iMessage blocked (Photon free-tier / sidecar) |
| ~2:23–2:28 | Birdeye **SENT** via SMTP from `ashbotphoenix@agentmail.to` (after username fix from rejected `ashbot@`) |
| ~2:40–2:42 | Park Phoenix domain mailbox; lock AshBot Phoenix = AgentMail identity; patch config username |
| ~2:58 | Close-out list for Anthony (IG / YP / Gate 3 leftovers) |
| ~2:59 | Standing rule: CC `tonyjbala@gmail.com` on listing reply threads |
| ~3:00 | Inbox pass dispatched — **oneshot missing AGENTMAIL_API_KEY** |
| ~3:00–3:14 | Ashbot credential-hunt spiral (himalaya missing, safety blocks, 143s source grep). STATUS stuck `working` |
| ~3:08+ | STATUS watcher created (@every 5m) after Anthony demanded visibility |
| ~3:12–3:14 | Trace analysis; Anthony: intervene as superior |
| ~3:15 | CoS killed pid 36384; redispatched **v2** with `.env` inject (`key_present=True`, pid 38988) |
| ~3:26 | Mid-job ping: curling AgentMail inbox |
| ~3:32 | **Inbox v2 done**: 4 msgs scanned, **NONE_FOUND** (no Birdeye/listing returns). ACTIVE_JOB cleared |
| ~3:39–4:16 | Desktop often unreachable; watcher correctly stayed silent (idle) but kept waking CoS every 5m |

---

## 2. What "went off the rails"

### A. False starts / silent failures
- Claimed or implied Ashbot "started" before SOCKS/SSH/folder/pid were real.
- Windows detached launch looked dispatched with empty logs.
- Anthony waited with "Nothing" / "It's been a long time" because CoS did not fail-loud or done-ping on time.

### B. Ashbot oneshot environment gap (root technical failure)
- Method A `_launch.py` did **not** load `profiles/ashbot/.env`.
- Gateway/Desktop sessions had AgentMail; SSH oneshot did not.
- Birdeye send luckily had key in that session; inbox pass did not → multi-minute dead-end hunt.
- Ashbot did not flip STATUS to `blocked` after first KEY_MISSING; CoS was slow to intervene.

### C. Visibility debt
- No reliable mid-job channel until STATUS watcher.
- iMessage contract failed (Photon free-tier + sidecar down) — chat ping is the real guarantee.
- Watcher later correct for idle, but **@every 5m forever** creates noise when no job (and when desktop offline).

### D. Scope / attention scatter
- Retro Revolution paste briefly looked like wrong-thread pollution.
- Desktop Local dropped after inbox done → watcher "offline" ticks (silent to Anthony, still wake tax).

### E. What actually worked
- File-brief + SSH-sync launch (once SOCKS up).
- Birdeye outbound SMTP with `ashbotphoenix@agentmail.to`.
- Intervene rule applied: kill spiral → inject key → v2 completed cleanly.
- Lessons filed in Obsidian; several standing rules locked in CoS memory.

---

## 3. Root causes (ordered)

1. **CoS process immature:** announce before prove; late intervene; late done-ping.
2. **Launcher incomplete:** oneshot must always inject AgentMail (and profile) from ashbot `.env`.
3. **Hermes worker defaults:** will source-dive instead of STATUS=blocked + stop.
4. **Notify path fragile:** Photon not reliable; chat must be primary.
5. **Watcher cadence too aggressive when idle.**

---

## 4. Locked rules (do not regress)

1. Never say Hermes "started" until folder + STATUS pid + launch log LAUNCH line.
2. SSH-sync `_launch.py` + `_cos_brief.txt`; no `start /B` until proven.
3. Inject `AGENTMAIL_API_KEY` from ashbot `.env` before every mail oneshot.
4. Key missing once → STATUS blocked + stop; CoS fixes; no source archaeology.
5. Dead-end path → CoS intervenes immediately (superior).
6. Done/blocked → CoS pings Anthony in chat immediately.
7. Mid-job ~5m STATUS pings only while ACTIVE_JOB points at live work.
8. Ashbot From = `ashbotphoenix@agentmail.to`; CC Anthony on listing reply threads.
9. Phoenix domain mailbox parked; do not block on `assistant@phoenixcameronpark.com`.

---

## 5. Corrective actions (next)

| Action | Owner | Status |
|---|---|---|
| Standardize hermes-relay / `_launch.py` with .env inject | CoS | Partially done (v2 pattern) — promote to default template |
| Pause STATUS watcher when ACTIVE_JOB empty; resume on dispatch | CoS | Do now |
| Optional: daily/idle inbox poll for Birdeye replies (with key inject) | CoS | Not started |
| Anthony: IG + YP confirm (close letter blockers) | Gatekeeper | Open |
| Mirror lessons into Gitea `docs/agents/` | CoS | Pending |

---

## 6. Honest scorecard

| Goal | Result |
|---|---|
| Gate 3 3-URL pilot | Done (2 WONT-DO + Birdeye draft→sent) |
| Birdeye email out | **Success** |
| Inbox replies + CC Anthony | N/A — no returns yet (pass succeeded) |
| iMessage start/done | Failed (infrastructure) |
| Gatekeeper experience | **Poor mid-day** — silence, false starts, late intervene |
| Learning loop | **Strong** — traces → lessons → rules → intervene |

**Bottom line:** The company work mostly completed; the *operating system* for CoS↔Hermes nearly failed Anthony’s trust. Paying attention to traces was correct. Default path forward is thin CoS, injected env, intervene early, chat as source of truth.