docs(operations): lock beta audit process v1.0
- Reviewer decision loop: approve / reject (default Tier 2) / more evidence; decisions recorded in VALIDATION.md - Blocked surfaces: customer-facing block = finding or documented limitation, never a bare marker - Pre-delivery consistency gate: count match, evidence present, identity match - Required artifact set per run; deviations require dated decision record - Gilmore: exec summary corrected 2->3 to match findings; VALIDATION.md added with reviewer dispositions
This commit is contained in:
@@ -1,7 +1,7 @@
|
|||||||
# VeriPath Beta Audit Process
|
# VeriPath Beta Audit Process
|
||||||
|
|
||||||
**Status:** Locked for beta use
|
**Status:** Locked v1.0 — 2026-08-15, for beta use
|
||||||
**Date:** 2026-08-14
|
**Supersedes:** 2026-08-14 draft lock
|
||||||
|
|
||||||
## Purpose
|
## Purpose
|
||||||
|
|
||||||
@@ -42,20 +42,66 @@ Run:
|
|||||||
multi_scraper → audit_engine
|
multi_scraper → audit_engine
|
||||||
```
|
```
|
||||||
|
|
||||||
### 3. Human validation
|
### 3. Human validation (reviewer decision loop)
|
||||||
|
|
||||||
Review findings for:
|
The reviewer must resolve every engine finding to exactly one disposition.
|
||||||
|
Unresolved findings block delivery.
|
||||||
|
|
||||||
- False positives
|
| Disposition | Effect |
|
||||||
- Evidence quality
|
|-------------|--------|
|
||||||
|
| **Approve** | Finding stands as generated |
|
||||||
|
| **Reject** | Finding defaults to **Tier 2** (corroborated, not verified) unless the reviewer (a) discards it entirely, or (b) requests more evidence |
|
||||||
|
| **Request more evidence** | Finding marked `pending`; re-verified on the next run; may not ship as Tier 1 until evidence lands |
|
||||||
|
|
||||||
### 4. Client-facing report
|
Reviewer decisions are recorded in the run's `VALIDATION.md` (one line per finding: finding id, disposition, rationale).
|
||||||
|
|
||||||
Prepare a report containing material findings only. The report uses the approved VeriPath client-facing structure and voice. It does not include coaching language or marketing urgency.
|
### 4. Blocked surfaces
|
||||||
|
|
||||||
### 5. Delivery
|
A surface that fails capture (blocked, 403, no data returned) must never be
|
||||||
|
reported as a bare "✗" with no downstream treatment.
|
||||||
|
|
||||||
Deliver the validated client-facing report to the client.
|
- A blocked or missing **customer-facing** surface (website, booking page) is a
|
||||||
|
Customer Path Leakage signal: it ships as a finding, or the reviewer records
|
||||||
|
a documented limitation in `VALIDATION.md` with the reason it is not material.
|
||||||
|
- Non-customer-facing surface failures (e.g. Bing empty) are noted in the
|
||||||
|
Surfaces Reviewed table with the failure reason.
|
||||||
|
|
||||||
|
### 5. Client-facing report
|
||||||
|
|
||||||
|
Prepare a report containing material findings only. The report uses the approved
|
||||||
|
VeriPath client-facing structure and voice. It does not include coaching
|
||||||
|
language or marketing urgency.
|
||||||
|
|
||||||
|
### 6. Pre-delivery consistency gate
|
||||||
|
|
||||||
|
All three checks pass before the report is sent. Any failure blocks delivery:
|
||||||
|
|
||||||
|
1. **Count match:** the executive summary's stated number of findings equals the
|
||||||
|
number of findings in the Findings section.
|
||||||
|
2. **Evidence present:** every finding's cited evidence exists in the run's
|
||||||
|
archived artifacts (raw capture or `findings.json`).
|
||||||
|
3. **Identity match:** report client name, location, and audit date match the
|
||||||
|
intake record.
|
||||||
|
|
||||||
|
### 7. Delivery
|
||||||
|
|
||||||
|
Deliver the validated client-facing report to the client using the locked
|
||||||
|
delivery template.
|
||||||
|
|
||||||
|
## Required artifact set
|
||||||
|
|
||||||
|
Every run is archived under `docs/validation/<YYYY-MM-DD>-<slug>/` with:
|
||||||
|
|
||||||
|
| Artifact | Source |
|
||||||
|
|----------|--------|
|
||||||
|
| `*_multi_surface_YYYY-MM-DD.json` | raw pipeline capture |
|
||||||
|
| `findings.json` | engine output |
|
||||||
|
| `findings.md` | engine markdown |
|
||||||
|
| `REPORT-final.md` | client-facing report (post-review) |
|
||||||
|
| `VALIDATION.md` | run notes, surface results, reviewer decisions |
|
||||||
|
|
||||||
|
Runs completed before 2026-08-15 are grandfathered as-is. Every run on or
|
||||||
|
after this date must complete the set before delivery.
|
||||||
|
|
||||||
## Out of Scope for Default Beta
|
## Out of Scope for Default Beta
|
||||||
|
|
||||||
@@ -68,7 +114,9 @@ The following are not included in the default beta audit:
|
|||||||
|
|
||||||
## Conditional Legacy Check
|
## Conditional Legacy Check
|
||||||
|
|
||||||
Run a quick old-name and old-website check only when a name or location change in the last 3–5 years is known or indicated during intake. This check is not part of every default beta run.
|
Run a quick old-name and old-website check only when a name or location change
|
||||||
|
in the last 3–5 years is known or indicated during intake. This check is not
|
||||||
|
part of every default beta run.
|
||||||
|
|
||||||
## Pipeline Reference
|
## Pipeline Reference
|
||||||
|
|
||||||
@@ -84,6 +132,13 @@ The standard execution path is:
|
|||||||
multi_scraper → audit_engine
|
multi_scraper → audit_engine
|
||||||
```
|
```
|
||||||
|
|
||||||
|
## Deviations
|
||||||
|
|
||||||
|
Any deviation from this process requires a dated decision record in
|
||||||
|
`docs/decisions/`. Edits to this document follow the same rule: the change is a
|
||||||
|
dated decision, not a silent edit.
|
||||||
|
|
||||||
## Deeper Engagements
|
## Deeper Engagements
|
||||||
|
|
||||||
The full Layer 1/2 system remains available for deeper engagements. Beta audits use the streamlined path above.
|
The full Layer 1/2 system remains available for deeper engagements. Beta audits
|
||||||
|
use the streamlined path above.
|
||||||
|
|||||||
Reference in New Issue
Block a user