feat(auditing): pipeline v1.1 — report generation + scripted gate + temporal delta

- report_generate.py: REPORT-final.md + VALIDATION.md from findings + raw
  capture; per-surface status from summary.surfaces_ok; Data Limitations
- report_gate.py: §6 gate scripted (count/evidence/identity/material
  support); non-zero exit blocks delivery
- audit_diff.py: before/after capture comparison -> DELTA.md
- audit_pipeline.sh: 2 steps -> 4; collision-safe capture copy (re-runs
  preserve baseline); relative output dir resolved to absolute
- beta-audit-process.md: locked v1.0 -> v1.1 (+ dated decision record)
- all 9 runs of the 2026-08-15 batch regenerated + gated (9/9 PASS)
- live end-to-end proof: Gilmore re-run, DELTA.md 0 changes (same day)
This commit is contained in:
2026-08-15 16:52:23 +00:00
parent c951f79a18
commit 394d5dae3f
33 changed files with 1681 additions and 228 deletions
@@ -0,0 +1,62 @@
# Decision: Audit pipeline promoted to v1.1
**Date:** 2026-08-15
**Status:** Approved
**Supersedes:** Locked v1.0 process (2026-08-15) — same date, same day, tighter mechanics
## Change
`beta-audit-process.md` moves from **Locked v1.0** to **Locked v1.1**.
The locked process said *what* must be true (report, gate, artifact set).
v1.1 wires the *how* into the pipeline so the guarantees hold without
human ritual.
## What changed in the toolchain
`implementation/auditing/` gains two scripts and the pipeline grows from
2 steps to 4:
```text
multi_scraper → audit_engine → report_generate → report_gate
```
1. **`report_generate.py`** — builds `REPORT-final.md` + `VALIDATION.md`
from the run's `findings.json` + raw capture. Per-surface status in the
Surfaces Reviewed table is derived from `summary.surfaces_ok`
(✓/✗/⚠ with reason); a Data Limitations section states the known
ceilings (no review text, no authenticated hours, point-in-time).
2. **`report_gate.py`** — the §6 pre-delivery gate, scripted. Four checks:
count match, evidence present (absence-type findings re-derived from
raw capture instead of requiring quoted evidence), identity match, and
material support (every material finding re-derived against the raw
capture). Non-zero exit on failure; result stamped into `VALIDATION.md`.
3. **`audit_diff.py`** — temporal delta between two raw captures →
`DELTA.md` in the after-run folder. The proof-of-fix artifact: after the
client applies recommendations, re-run and the delta is the invoice
evidence.
4. **`audit_pipeline.sh`** — steps 34 wired in; gate FAIL blocks delivery
(`exit 1`). Capture copy is collision-safe: a re-run never overwrites
the prior capture (timestamped sibling file), which is what makes the
temporal baseline in `audit_diff.py` possible.
## Why
- The v1.0 gate was manual; a 9-run batch showed the manual gate drifts
(reports were regenerated by a tmp script outside the repo).
- Re-runs overwrote prior captures, so before/after proof was impossible.
- The report builder lived in `/tmp`, not the repo, so "report is a
pipeline output" was a convention, not a mechanism.
## Verification (2026-08-15)
- All 9 runs of the 2026-08-15 batch regenerated through
`report_generate.py`; `report_gate.py` PASS 9/9.
- Live end-to-end proof: Gilmore Heating, Air and Plumbing re-run through
the full 4-step pipeline. Baseline capture preserved as a timestamped
sibling; report regenerated; gate PASS; `audit_diff.py` produced
`DELTA.md` (0 field changes — same day, as expected).
## No change to
- Scope (four surfaces), intake, reviewer disposition loop, blocked-surface
treatment (§4), delivery. v1.1 changes mechanics, not meaning.