veripath/docs/operations/beta-audit-process.md

# VeriPath Beta Audit Process

**Status:** Locked v1.1 — 2026-08-15, for beta use (v1.1 decision: `docs/decisions/2026-08-15-audit-pipeline-v1-1.md`)
**Supersedes:** 2026-08-14 draft lock

## Purpose

Professional, repeatable audit for beta clients using the automated primary-surface pipeline.

## Scope

The beta audit covers four primary surfaces:

- Google Business Profile
- Apple Maps
- Bing Places
- Current website

The client-facing report includes material findings only, covering:

- NAP consistency
- Hours consistency, with Google Business Profile as the source of truth
- Review velocity
- Website reachability and schema conflicts

## Process

### 1. Intake

Collect:

- Business name
- City
- Website
- Known previous names

### 2. Run the pipeline

Run:

```text
multi_scraper → audit_engine → report_generate → report_gate
```

The pipeline writes the full required artifact set itself. `report_gate`
fails the run (non-zero exit) if the §6 gate fails — a failing run does
not deliver.

### 3. Human validation (reviewer decision loop)

The reviewer must resolve every engine finding to exactly one disposition.
Unresolved findings block delivery.

| Disposition | Effect |
|-------------|--------|
| **Approve** | Finding stands as generated |
| **Reject** | Finding defaults to **Tier 2** (corroborated, not verified) unless the reviewer (a) discards it entirely, or (b) requests more evidence |
| **Request more evidence** | Finding marked `pending`; re-verified on the next run; may not ship as Tier 1 until evidence lands |

Reviewer decisions are recorded in the run's `VALIDATION.md` (one line per finding: finding id, disposition, rationale).

### 4. Blocked surfaces

A surface that fails capture (blocked, 403, no data returned) must never be
reported as a bare "✗" with no downstream treatment.

- A blocked or missing **customer-facing** surface (website, booking page) is a
  Customer Path Leakage signal: it ships as a finding, or the reviewer records
  a documented limitation in `VALIDATION.md` with the reason it is not material.
- Non-customer-facing surface failures (e.g. Bing empty) are noted in the
  Surfaces Reviewed table with the failure reason.

### 5. Client-facing report

Prepare a report containing material findings only. The report uses the approved
VeriPath client-facing structure and voice. It does not include coaching
language or marketing urgency.

### 6. Pre-delivery consistency gate

Executed by `implementation/auditing/report_gate.py` as pipeline step 4;
result stamped into `VALIDATION.md`. All four checks pass before the
report is sent. Any failure blocks delivery:

1. **Count match:** the executive summary's stated number of findings equals the
   number of findings in the Findings section.
2. **Evidence present:** every finding's cited evidence exists in the run's
   archived artifacts (raw capture or `findings.json`). Absence-type findings
   (no website, no description, UTM parameters, unreachable surface) are
   re-derived from the raw capture instead of requiring quoted evidence.
3. **Identity match:** report client name, location, and audit date match the
   intake record.
4. **Material support:** each material finding is re-derived against the raw
   capture (phone cores across surfaces, hours day counts, rating, website
   presence, description/OG fields).

### 7. Delivery

Deliver the validated client-facing report to the client using the locked
delivery template.

## Required artifact set

Every run is archived under `docs/validation/<YYYY-MM-DD>-<slug>/` with:

| Artifact | Source |
|----------|--------|
| `*_multi_surface_YYYY-MM-DD.json` | raw pipeline capture |
| `findings.json` | engine output |
| `findings.md` | engine markdown |
| `REPORT-final.md` | client-facing report (post-review) |
| `VALIDATION.md` | run notes, surface results, reviewer decisions |
| `DELTA.md` | before/after field comparison (re-runs only) |

Re-runs never overwrite the prior raw capture: the pipeline writes a
timestamped sibling (`*__YYYYMMDD-HHMMSS.json`). The prior capture is the
temporal baseline; `audit_diff.py` compares the two into `DELTA.md`.

Runs completed before 2026-08-15 are grandfathered as-is. Every run on or
after this date must complete the set before delivery.

## Out of Scope for Default Beta

The following are not included in the default beta audit:

- Exhaustive secondary-directory research
- Full AI chatbot visibility scoring
- Competitive benchmarking
- Social content audits

## Conditional Legacy Check

Run a quick old-name and old-website check only when a name or location change
in the last 3–5 years is known or indicated during intake. This check is not
part of every default beta run.

## Pipeline Reference

The working pipeline is located at:

```text
implementation/auditing/
```

The standard execution path is:

```text
multi_scraper → audit_engine → report_generate → report_gate
audit_diff (re-runs: baseline capture vs. new capture)
```

## Deviations

Any deviation from this process requires a dated decision record in
`docs/decisions/`. Edits to this document follow the same rule: the change is a
dated decision, not a silent edit.

## Deeper Engagements

The full Layer 1/2 system remains available for deeper engagements. Beta audits
use the streamlined path above.