Files

T

Leonard 394d5dae3f feat(auditing): pipeline v1.1 — report generation + scripted gate + temporal delta

- report_generate.py: REPORT-final.md + VALIDATION.md from findings + raw
  capture; per-surface status from summary.surfaces_ok; Data Limitations
- report_gate.py: §6 gate scripted (count/evidence/identity/material
  support); non-zero exit blocks delivery
- audit_diff.py: before/after capture comparison -> DELTA.md
- audit_pipeline.sh: 2 steps -> 4; collision-safe capture copy (re-runs
  preserve baseline); relative output dir resolved to absolute
- beta-audit-process.md: locked v1.0 -> v1.1 (+ dated decision record)
- all 9 runs of the 2026-08-15 batch regenerated + gated (9/9 PASS)
- live end-to-end proof: Gilmore re-run, DELTA.md 0 changes (same day)

2026-08-15 16:52:23 +00:00

2.7 KiB

Raw Permalink Blame History

Decision: Audit pipeline promoted to v1.1

Date: 2026-08-15 Status: Approved Supersedes: Locked v1.0 process (2026-08-15) — same date, same day, tighter mechanics

Change

beta-audit-process.md moves from Locked v1.0 to Locked v1.1. The locked process said what must be true (report, gate, artifact set). v1.1 wires the how into the pipeline so the guarantees hold without human ritual.

What changed in the toolchain

implementation/auditing/ gains two scripts and the pipeline grows from 2 steps to 4:

multi_scraper → audit_engine → report_generate → report_gate

report_generate.py — builds REPORT-final.md + VALIDATION.md from the run's findings.json + raw capture. Per-surface status in the Surfaces Reviewed table is derived from summary.surfaces_ok (✓/✗/⚠ with reason); a Data Limitations section states the known ceilings (no review text, no authenticated hours, point-in-time).
report_gate.py — the §6 pre-delivery gate, scripted. Four checks: count match, evidence present (absence-type findings re-derived from raw capture instead of requiring quoted evidence), identity match, and material support (every material finding re-derived against the raw capture). Non-zero exit on failure; result stamped into VALIDATION.md.
audit_diff.py — temporal delta between two raw captures → DELTA.md in the after-run folder. The proof-of-fix artifact: after the client applies recommendations, re-run and the delta is the invoice evidence.
audit_pipeline.sh — steps 3–4 wired in; gate FAIL blocks delivery (exit 1). Capture copy is collision-safe: a re-run never overwrites the prior capture (timestamped sibling file), which is what makes the temporal baseline in audit_diff.py possible.

Why

The v1.0 gate was manual; a 9-run batch showed the manual gate drifts (reports were regenerated by a tmp script outside the repo).
Re-runs overwrote prior captures, so before/after proof was impossible.
The report builder lived in /tmp, not the repo, so "report is a pipeline output" was a convention, not a mechanism.

Verification (2026-08-15)

All 9 runs of the 2026-08-15 batch regenerated through report_generate.py; report_gate.py PASS 9/9.
Live end-to-end proof: Gilmore Heating, Air and Plumbing re-run through the full 4-step pipeline. Baseline capture preserved as a timestamped sibling; report regenerated; gate PASS; audit_diff.py produced DELTA.md (0 field changes — same day, as expected).

No change to

Scope (four surfaces), intake, reviewer disposition loop, blocked-surface treatment (§4), delivery. v1.1 changes mechanics, not meaning.

2.7 KiB Raw Permalink Blame History Unescape Escape