Leonard
394d5dae3f
- report_generate.py: REPORT-final.md + VALIDATION.md from findings + raw capture; per-surface status from summary.surfaces_ok; Data Limitations - report_gate.py: §6 gate scripted (count/evidence/identity/material support); non-zero exit blocks delivery - audit_diff.py: before/after capture comparison -> DELTA.md - audit_pipeline.sh: 2 steps -> 4; collision-safe capture copy (re-runs preserve baseline); relative output dir resolved to absolute - beta-audit-process.md: locked v1.0 -> v1.1 (+ dated decision record) - all 9 runs of the 2026-08-15 batch regenerated + gated (9/9 PASS) - live end-to-end proof: Gilmore re-run, DELTA.md 0 changes (same day)
63 lines
2.7 KiB
Markdown
63 lines
2.7 KiB
Markdown
# Decision: Audit pipeline promoted to v1.1
|
||
|
||
**Date:** 2026-08-15
|
||
**Status:** Approved
|
||
**Supersedes:** Locked v1.0 process (2026-08-15) — same date, same day, tighter mechanics
|
||
|
||
## Change
|
||
|
||
`beta-audit-process.md` moves from **Locked v1.0** to **Locked v1.1**.
|
||
The locked process said *what* must be true (report, gate, artifact set).
|
||
v1.1 wires the *how* into the pipeline so the guarantees hold without
|
||
human ritual.
|
||
|
||
## What changed in the toolchain
|
||
|
||
`implementation/auditing/` gains two scripts and the pipeline grows from
|
||
2 steps to 4:
|
||
|
||
```text
|
||
multi_scraper → audit_engine → report_generate → report_gate
|
||
```
|
||
|
||
1. **`report_generate.py`** — builds `REPORT-final.md` + `VALIDATION.md`
|
||
from the run's `findings.json` + raw capture. Per-surface status in the
|
||
Surfaces Reviewed table is derived from `summary.surfaces_ok`
|
||
(✓/✗/⚠ with reason); a Data Limitations section states the known
|
||
ceilings (no review text, no authenticated hours, point-in-time).
|
||
2. **`report_gate.py`** — the §6 pre-delivery gate, scripted. Four checks:
|
||
count match, evidence present (absence-type findings re-derived from
|
||
raw capture instead of requiring quoted evidence), identity match, and
|
||
material support (every material finding re-derived against the raw
|
||
capture). Non-zero exit on failure; result stamped into `VALIDATION.md`.
|
||
3. **`audit_diff.py`** — temporal delta between two raw captures →
|
||
`DELTA.md` in the after-run folder. The proof-of-fix artifact: after the
|
||
client applies recommendations, re-run and the delta is the invoice
|
||
evidence.
|
||
4. **`audit_pipeline.sh`** — steps 3–4 wired in; gate FAIL blocks delivery
|
||
(`exit 1`). Capture copy is collision-safe: a re-run never overwrites
|
||
the prior capture (timestamped sibling file), which is what makes the
|
||
temporal baseline in `audit_diff.py` possible.
|
||
|
||
## Why
|
||
|
||
- The v1.0 gate was manual; a 9-run batch showed the manual gate drifts
|
||
(reports were regenerated by a tmp script outside the repo).
|
||
- Re-runs overwrote prior captures, so before/after proof was impossible.
|
||
- The report builder lived in `/tmp`, not the repo, so "report is a
|
||
pipeline output" was a convention, not a mechanism.
|
||
|
||
## Verification (2026-08-15)
|
||
|
||
- All 9 runs of the 2026-08-15 batch regenerated through
|
||
`report_generate.py`; `report_gate.py` PASS 9/9.
|
||
- Live end-to-end proof: Gilmore Heating, Air and Plumbing re-run through
|
||
the full 4-step pipeline. Baseline capture preserved as a timestamped
|
||
sibling; report regenerated; gate PASS; `audit_diff.py` produced
|
||
`DELTA.md` (0 field changes — same day, as expected).
|
||
|
||
## No change to
|
||
|
||
- Scope (four surfaces), intake, reviewer disposition loop, blocked-surface
|
||
treatment (§4), delivery. v1.1 changes mechanics, not meaning.
|