veripath/docs/operations/beta-audit-process.md

VeriPath Beta Audit Process

Status: Locked v1.1 — 2026-08-15, for beta use (v1.1 decision: docs/decisions/2026-08-15-audit-pipeline-v1-1.md) Supersedes: 2026-08-14 draft lock

Purpose

Professional, repeatable audit for beta clients using the automated primary-surface pipeline.

Scope

The beta audit covers four primary surfaces:

• Google Business Profile
• Apple Maps
• Bing Places
• Current website

The client-facing report includes material findings only, covering:

• NAP consistency
• Hours consistency, with Google Business Profile as the source of truth
• Review velocity
• Website reachability and schema conflicts

Process

1. Intake

Collect:

• Business name
• City
• Website
• Known previous names

2. Run the pipeline

Run:

multi_scraper → audit_engine → report_generate → report_gate

The pipeline writes the full required artifact set itself. report_gate fails the run (non-zero exit) if the §6 gate fails — a failing run does not deliver.

3. Human validation (reviewer decision loop)

The reviewer must resolve every engine finding to exactly one disposition. Unresolved findings block delivery.

Disposition	Effect
Approve	Finding stands as generated
Reject	Finding defaults to Tier 2 (corroborated, not verified) unless the reviewer (a) discards it entirely, or (b) requests more evidence
Request more evidence	Finding marked pending; re-verified on the next run; may not ship as Tier 1 until evidence lands

Reviewer decisions are recorded in the run's VALIDATION.md (one line per finding: finding id, disposition, rationale).

4. Blocked surfaces

A surface that fails capture (blocked, 403, no data returned) must never be reported as a bare "✗" with no downstream treatment.

• A blocked or missing customer-facing surface (website, booking page) is a Customer Path Leakage signal: it ships as a finding, or the reviewer records a documented limitation in VALIDATION.md with the reason it is not material.
• Non-customer-facing surface failures (e.g. Bing empty) are noted in the Surfaces Reviewed table with the failure reason.

5. Client-facing report

Prepare a report containing material findings only. The report uses the approved VeriPath client-facing structure and voice. It does not include coaching language or marketing urgency.

6. Pre-delivery consistency gate

Executed by implementation/auditing/report_gate.py as pipeline step 4; result stamped into VALIDATION.md. All four checks pass before the report is sent. Any failure blocks delivery:

1. Count match: the executive summary's stated number of findings equals the number of findings in the Findings section.
2. Evidence present: every finding's cited evidence exists in the run's archived artifacts (raw capture or findings.json). Absence-type findings (no website, no description, UTM parameters, unreachable surface) are re-derived from the raw capture instead of requiring quoted evidence.
3. Identity match: report client name, location, and audit date match the intake record.
4. Material support: each material finding is re-derived against the raw capture (phone cores across surfaces, hours day counts, rating, website presence, description/OG fields).

7. Delivery

Deliver the validated client-facing report to the client using the locked delivery template.

Required artifact set

Every run is archived under docs/validation/<YYYY-MM-DD>-<slug>/ with:

Artifact	Source
*_multi_surface_YYYY-MM-DD.json	raw pipeline capture
findings.json	engine output
findings.md	engine markdown
REPORT-final.md	client-facing report (post-review)
VALIDATION.md	run notes, surface results, reviewer decisions
DELTA.md	before/after field comparison (re-runs only)

Re-runs never overwrite the prior raw capture: the pipeline writes a timestamped sibling (*__YYYYMMDD-HHMMSS.json). The prior capture is the temporal baseline; audit_diff.py compares the two into DELTA.md.

Runs completed before 2026-08-15 are grandfathered as-is. Every run on or after this date must complete the set before delivery.

Out of Scope for Default Beta

The following are not included in the default beta audit:

• Exhaustive secondary-directory research
• Full AI chatbot visibility scoring
• Competitive benchmarking
• Social content audits

Conditional Legacy Check

Run a quick old-name and old-website check only when a name or location change in the last 3–5 years is known or indicated during intake. This check is not part of every default beta run.

Pipeline Reference

The working pipeline is located at:

implementation/auditing/

The standard execution path is:

multi_scraper → audit_engine → report_generate → report_gate
audit_diff (re-runs: baseline capture vs. new capture)

Deviations

Any deviation from this process requires a dated decision record in docs/decisions/. Edits to this document follow the same rule: the change is a dated decision, not a silent edit.

Deeper Engagements

The full Layer 1/2 system remains available for deeper engagements. Beta audits use the streamlined path above.