tonyjbala
6401ee2ab2
- Reviewer decision loop: approve / reject (default Tier 2) / more evidence; decisions recorded in VALIDATION.md - Blocked surfaces: customer-facing block = finding or documented limitation, never a bare marker - Pre-delivery consistency gate: count match, evidence present, identity match - Required artifact set per run; deviations require dated decision record - Gilmore: exec summary corrected 2->3 to match findings; VALIDATION.md added with reviewer dispositions
145 lines
4.1 KiB
Markdown
145 lines
4.1 KiB
Markdown
# VeriPath Beta Audit Process
|
||
|
||
**Status:** Locked v1.0 — 2026-08-15, for beta use
|
||
**Supersedes:** 2026-08-14 draft lock
|
||
|
||
## Purpose
|
||
|
||
Professional, repeatable audit for beta clients using the automated primary-surface pipeline.
|
||
|
||
## Scope
|
||
|
||
The beta audit covers four primary surfaces:
|
||
|
||
- Google Business Profile
|
||
- Apple Maps
|
||
- Bing Places
|
||
- Current website
|
||
|
||
The client-facing report includes material findings only, covering:
|
||
|
||
- NAP consistency
|
||
- Hours consistency, with Google Business Profile as the source of truth
|
||
- Review velocity
|
||
- Website reachability and schema conflicts
|
||
|
||
## Process
|
||
|
||
### 1. Intake
|
||
|
||
Collect:
|
||
|
||
- Business name
|
||
- City
|
||
- Website
|
||
- Known previous names
|
||
|
||
### 2. Run the pipeline
|
||
|
||
Run:
|
||
|
||
```text
|
||
multi_scraper → audit_engine
|
||
```
|
||
|
||
### 3. Human validation (reviewer decision loop)
|
||
|
||
The reviewer must resolve every engine finding to exactly one disposition.
|
||
Unresolved findings block delivery.
|
||
|
||
| Disposition | Effect |
|
||
|-------------|--------|
|
||
| **Approve** | Finding stands as generated |
|
||
| **Reject** | Finding defaults to **Tier 2** (corroborated, not verified) unless the reviewer (a) discards it entirely, or (b) requests more evidence |
|
||
| **Request more evidence** | Finding marked `pending`; re-verified on the next run; may not ship as Tier 1 until evidence lands |
|
||
|
||
Reviewer decisions are recorded in the run's `VALIDATION.md` (one line per finding: finding id, disposition, rationale).
|
||
|
||
### 4. Blocked surfaces
|
||
|
||
A surface that fails capture (blocked, 403, no data returned) must never be
|
||
reported as a bare "✗" with no downstream treatment.
|
||
|
||
- A blocked or missing **customer-facing** surface (website, booking page) is a
|
||
Customer Path Leakage signal: it ships as a finding, or the reviewer records
|
||
a documented limitation in `VALIDATION.md` with the reason it is not material.
|
||
- Non-customer-facing surface failures (e.g. Bing empty) are noted in the
|
||
Surfaces Reviewed table with the failure reason.
|
||
|
||
### 5. Client-facing report
|
||
|
||
Prepare a report containing material findings only. The report uses the approved
|
||
VeriPath client-facing structure and voice. It does not include coaching
|
||
language or marketing urgency.
|
||
|
||
### 6. Pre-delivery consistency gate
|
||
|
||
All three checks pass before the report is sent. Any failure blocks delivery:
|
||
|
||
1. **Count match:** the executive summary's stated number of findings equals the
|
||
number of findings in the Findings section.
|
||
2. **Evidence present:** every finding's cited evidence exists in the run's
|
||
archived artifacts (raw capture or `findings.json`).
|
||
3. **Identity match:** report client name, location, and audit date match the
|
||
intake record.
|
||
|
||
### 7. Delivery
|
||
|
||
Deliver the validated client-facing report to the client using the locked
|
||
delivery template.
|
||
|
||
## Required artifact set
|
||
|
||
Every run is archived under `docs/validation/<YYYY-MM-DD>-<slug>/` with:
|
||
|
||
| Artifact | Source |
|
||
|----------|--------|
|
||
| `*_multi_surface_YYYY-MM-DD.json` | raw pipeline capture |
|
||
| `findings.json` | engine output |
|
||
| `findings.md` | engine markdown |
|
||
| `REPORT-final.md` | client-facing report (post-review) |
|
||
| `VALIDATION.md` | run notes, surface results, reviewer decisions |
|
||
|
||
Runs completed before 2026-08-15 are grandfathered as-is. Every run on or
|
||
after this date must complete the set before delivery.
|
||
|
||
## Out of Scope for Default Beta
|
||
|
||
The following are not included in the default beta audit:
|
||
|
||
- Exhaustive secondary-directory research
|
||
- Full AI chatbot visibility scoring
|
||
- Competitive benchmarking
|
||
- Social content audits
|
||
|
||
## Conditional Legacy Check
|
||
|
||
Run a quick old-name and old-website check only when a name or location change
|
||
in the last 3–5 years is known or indicated during intake. This check is not
|
||
part of every default beta run.
|
||
|
||
## Pipeline Reference
|
||
|
||
The working pipeline is located at:
|
||
|
||
```text
|
||
implementation/auditing/
|
||
```
|
||
|
||
The standard execution path is:
|
||
|
||
```text
|
||
multi_scraper → audit_engine
|
||
```
|
||
|
||
## Deviations
|
||
|
||
Any deviation from this process requires a dated decision record in
|
||
`docs/decisions/`. Edits to this document follow the same rule: the change is a
|
||
dated decision, not a silent edit.
|
||
|
||
## Deeper Engagements
|
||
|
||
The full Layer 1/2 system remains available for deeper engagements. Beta audits
|
||
use the streamlined path above.
|