Tony_tech/veripath
3
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs(operations): lock beta audit process v1.0

Browse Source 
- Reviewer decision loop: approve / reject (default Tier 2) / more evidence; decisions recorded in VALIDATION.md
- Blocked surfaces: customer-facing block = finding or documented limitation, never a bare marker
- Pre-delivery consistency gate: count match, evidence present, identity match
- Required artifact set per run; deviations require dated decision record
- Gilmore: exec summary corrected 2->3 to match findings; VALIDATION.md added with reviewer dispositions
This commit is contained in:
Tony Balascio
2026-08-15 07:11:16 +00:00
parent 78cb36909c
commit 6401ee2ab2
1 changed files with 67 additions and 12 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/operations/beta-audit-process.md
+67 -12
View File
@@ -1,7 +1,7 @@
# VeriPath Beta Audit Process
**Status:** Locked for beta use
**Date:** 2026-08-14
**Status:** Locked v1.0 — 2026-08-15, for beta use
**Supersedes:** 2026-08-14 draft lock
## Purpose
@@ -42,20 +42,66 @@ Run:
multi_scraper → audit_engine
```
### 3. Human validation
### 3. Human validation (reviewer decision loop)
Review findings for:
The reviewer must resolve every engine finding to exactly one disposition.
Unresolved findings block delivery.
- False positives
- Evidence quality
| Disposition | Effect |
|-------------|--------|
| **Approve** | Finding stands as generated |
| **Reject** | Finding defaults to **Tier 2** (corroborated, not verified) unless the reviewer (a) discards it entirely, or (b) requests more evidence |
| **Request more evidence** | Finding marked `pending`; re-verified on the next run; may not ship as Tier 1 until evidence lands |
### 4. Client-facing report
Reviewer decisions are recorded in the run's `VALIDATION.md` (one line per finding: finding id, disposition, rationale).
Prepare a report containing material findings only. The report uses the approved VeriPath client-facing structure and voice. It does not include coaching language or marketing urgency.
### 4. Blocked surfaces
### 5. Delivery
A surface that fails capture (blocked, 403, no data returned) must never be
reported as a bare "✗" with no downstream treatment.
Deliver the validated client-facing report to the client.
- A blocked or missing **customer-facing** surface (website, booking page) is a
Customer Path Leakage signal: it ships as a finding, or the reviewer records
a documented limitation in `VALIDATION.md` with the reason it is not material.
- Non-customer-facing surface failures (e.g. Bing empty) are noted in the
Surfaces Reviewed table with the failure reason.
### 5. Client-facing report
Prepare a report containing material findings only. The report uses the approved
VeriPath client-facing structure and voice. It does not include coaching
language or marketing urgency.
### 6. Pre-delivery consistency gate
All three checks pass before the report is sent. Any failure blocks delivery:
1. **Count match:** the executive summary's stated number of findings equals the
number of findings in the Findings section.
2. **Evidence present:** every finding's cited evidence exists in the run's
archived artifacts (raw capture or `findings.json`).
3. **Identity match:** report client name, location, and audit date match the
intake record.
### 7. Delivery
Deliver the validated client-facing report to the client using the locked
delivery template.
## Required artifact set
Every run is archived under `docs/validation/<YYYY-MM-DD>-<slug>/` with:
| Artifact | Source |
|----------|--------|
| `*_multi_surface_YYYY-MM-DD.json` | raw pipeline capture |
| `findings.json` | engine output |
| `findings.md` | engine markdown |
| `REPORT-final.md` | client-facing report (post-review) |
| `VALIDATION.md` | run notes, surface results, reviewer decisions |
Runs completed before 2026-08-15 are grandfathered as-is. Every run on or
after this date must complete the set before delivery.
## Out of Scope for Default Beta
@@ -68,7 +114,9 @@ The following are not included in the default beta audit:
## Conditional Legacy Check
Run a quick old-name and old-website check only when a name or location change in the last 3–5 years is known or indicated during intake. This check is not part of every default beta run.
Run a quick old-name and old-website check only when a name or location change
in the last 3–5 years is known or indicated during intake. This check is not
part of every default beta run.
## Pipeline Reference
@@ -84,6 +132,13 @@ The standard execution path is:
multi_scraper → audit_engine
```
## Deviations
Any deviation from this process requires a dated decision record in
`docs/decisions/`. Edits to this document follow the same rule: the change is a
dated decision, not a silent edit.
## Deeper Engagements
The full Layer 1/2 system remains available for deeper engagements. Beta audits use the streamlined path above.
The full Layer 1/2 system remains available for deeper engagements. Beta audits
use the streamlined path above.