feat(auditing): pipeline v1.1 — report generation + scripted gate + temporal delta

- report_generate.py: REPORT-final.md + VALIDATION.md from findings + raw
  capture; per-surface status from summary.surfaces_ok; Data Limitations
- report_gate.py: §6 gate scripted (count/evidence/identity/material
  support); non-zero exit blocks delivery
- audit_diff.py: before/after capture comparison -> DELTA.md
- audit_pipeline.sh: 2 steps -> 4; collision-safe capture copy (re-runs
  preserve baseline); relative output dir resolved to absolute
- beta-audit-process.md: locked v1.0 -> v1.1 (+ dated decision record)
- all 9 runs of the 2026-08-15 batch regenerated + gated (9/9 PASS)
- live end-to-end proof: Gilmore re-run, DELTA.md 0 changes (same day)
This commit is contained in:
2026-08-15 16:52:23 +00:00
parent c951f79a18
commit 394d5dae3f
33 changed files with 1681 additions and 228 deletions
+22 -5
View File
@@ -1,6 +1,6 @@
# VeriPath Beta Audit Process
**Status:** Locked v1.0 — 2026-08-15, for beta use
**Status:** Locked v1.1 — 2026-08-15, for beta use (v1.1 decision: `docs/decisions/2026-08-15-audit-pipeline-v1-1.md`)
**Supersedes:** 2026-08-14 draft lock
## Purpose
@@ -39,9 +39,13 @@ Collect:
Run:
```text
multi_scraper → audit_engine
multi_scraper → audit_engine → report_generate → report_gate
```
The pipeline writes the full required artifact set itself. `report_gate`
fails the run (non-zero exit) if the §6 gate fails — a failing run does
not deliver.
### 3. Human validation (reviewer decision loop)
The reviewer must resolve every engine finding to exactly one disposition.
@@ -74,14 +78,21 @@ language or marketing urgency.
### 6. Pre-delivery consistency gate
All three checks pass before the report is sent. Any failure blocks delivery:
Executed by `implementation/auditing/report_gate.py` as pipeline step 4;
result stamped into `VALIDATION.md`. All four checks pass before the
report is sent. Any failure blocks delivery:
1. **Count match:** the executive summary's stated number of findings equals the
number of findings in the Findings section.
2. **Evidence present:** every finding's cited evidence exists in the run's
archived artifacts (raw capture or `findings.json`).
archived artifacts (raw capture or `findings.json`). Absence-type findings
(no website, no description, UTM parameters, unreachable surface) are
re-derived from the raw capture instead of requiring quoted evidence.
3. **Identity match:** report client name, location, and audit date match the
intake record.
4. **Material support:** each material finding is re-derived against the raw
capture (phone cores across surfaces, hours day counts, rating, website
presence, description/OG fields).
### 7. Delivery
@@ -99,6 +110,11 @@ Every run is archived under `docs/validation/<YYYY-MM-DD>-<slug>/` with:
| `findings.md` | engine markdown |
| `REPORT-final.md` | client-facing report (post-review) |
| `VALIDATION.md` | run notes, surface results, reviewer decisions |
| `DELTA.md` | before/after field comparison (re-runs only) |
Re-runs never overwrite the prior raw capture: the pipeline writes a
timestamped sibling (`*__YYYYMMDD-HHMMSS.json`). The prior capture is the
temporal baseline; `audit_diff.py` compares the two into `DELTA.md`.
Runs completed before 2026-08-15 are grandfathered as-is. Every run on or
after this date must complete the set before delivery.
@@ -129,7 +145,8 @@ implementation/auditing/
The standard execution path is:
```text
multi_scraper → audit_engine
multi_scraper → audit_engine → report_generate → report_gate
audit_diff (re-runs: baseline capture vs. new capture)
```
## Deviations