feat(auditing): pipeline v1.1 — report generation + scripted gate + temporal delta
- report_generate.py: REPORT-final.md + VALIDATION.md from findings + raw capture; per-surface status from summary.surfaces_ok; Data Limitations - report_gate.py: §6 gate scripted (count/evidence/identity/material support); non-zero exit blocks delivery - audit_diff.py: before/after capture comparison -> DELTA.md - audit_pipeline.sh: 2 steps -> 4; collision-safe capture copy (re-runs preserve baseline); relative output dir resolved to absolute - beta-audit-process.md: locked v1.0 -> v1.1 (+ dated decision record) - all 9 runs of the 2026-08-15 batch regenerated + gated (9/9 PASS) - live end-to-end proof: Gilmore re-run, DELTA.md 0 changes (same day)
This commit is contained in:
@@ -0,0 +1,62 @@
|
||||
# Decision: Audit pipeline promoted to v1.1
|
||||
|
||||
**Date:** 2026-08-15
|
||||
**Status:** Approved
|
||||
**Supersedes:** Locked v1.0 process (2026-08-15) — same date, same day, tighter mechanics
|
||||
|
||||
## Change
|
||||
|
||||
`beta-audit-process.md` moves from **Locked v1.0** to **Locked v1.1**.
|
||||
The locked process said *what* must be true (report, gate, artifact set).
|
||||
v1.1 wires the *how* into the pipeline so the guarantees hold without
|
||||
human ritual.
|
||||
|
||||
## What changed in the toolchain
|
||||
|
||||
`implementation/auditing/` gains two scripts and the pipeline grows from
|
||||
2 steps to 4:
|
||||
|
||||
```text
|
||||
multi_scraper → audit_engine → report_generate → report_gate
|
||||
```
|
||||
|
||||
1. **`report_generate.py`** — builds `REPORT-final.md` + `VALIDATION.md`
|
||||
from the run's `findings.json` + raw capture. Per-surface status in the
|
||||
Surfaces Reviewed table is derived from `summary.surfaces_ok`
|
||||
(✓/✗/⚠ with reason); a Data Limitations section states the known
|
||||
ceilings (no review text, no authenticated hours, point-in-time).
|
||||
2. **`report_gate.py`** — the §6 pre-delivery gate, scripted. Four checks:
|
||||
count match, evidence present (absence-type findings re-derived from
|
||||
raw capture instead of requiring quoted evidence), identity match, and
|
||||
material support (every material finding re-derived against the raw
|
||||
capture). Non-zero exit on failure; result stamped into `VALIDATION.md`.
|
||||
3. **`audit_diff.py`** — temporal delta between two raw captures →
|
||||
`DELTA.md` in the after-run folder. The proof-of-fix artifact: after the
|
||||
client applies recommendations, re-run and the delta is the invoice
|
||||
evidence.
|
||||
4. **`audit_pipeline.sh`** — steps 3–4 wired in; gate FAIL blocks delivery
|
||||
(`exit 1`). Capture copy is collision-safe: a re-run never overwrites
|
||||
the prior capture (timestamped sibling file), which is what makes the
|
||||
temporal baseline in `audit_diff.py` possible.
|
||||
|
||||
## Why
|
||||
|
||||
- The v1.0 gate was manual; a 9-run batch showed the manual gate drifts
|
||||
(reports were regenerated by a tmp script outside the repo).
|
||||
- Re-runs overwrote prior captures, so before/after proof was impossible.
|
||||
- The report builder lived in `/tmp`, not the repo, so "report is a
|
||||
pipeline output" was a convention, not a mechanism.
|
||||
|
||||
## Verification (2026-08-15)
|
||||
|
||||
- All 9 runs of the 2026-08-15 batch regenerated through
|
||||
`report_generate.py`; `report_gate.py` PASS 9/9.
|
||||
- Live end-to-end proof: Gilmore Heating, Air and Plumbing re-run through
|
||||
the full 4-step pipeline. Baseline capture preserved as a timestamped
|
||||
sibling; report regenerated; gate PASS; `audit_diff.py` produced
|
||||
`DELTA.md` (0 field changes — same day, as expected).
|
||||
|
||||
## No change to
|
||||
|
||||
- Scope (four surfaces), intake, reviewer disposition loop, blocked-surface
|
||||
treatment (§4), delivery. v1.1 changes mechanics, not meaning.
|
||||
Reference in New Issue
Block a user