169 lines
5.7 KiB
Markdown
169 lines
5.7 KiB
Markdown
# VeriPath Beta Audit Process
|
||
|
||
**Status:** Locked v1.2 — 2026-08-17, for beta use (v1.2: `docs/operations/decisions/DR-003-website-stack-fingerprinting.md`)
|
||
**Supersedes:** 2026-08-14 draft lock
|
||
|
||
## Purpose
|
||
|
||
Professional, repeatable audit for beta clients using the automated primary-surface pipeline.
|
||
|
||
## Scope
|
||
|
||
The beta audit covers four primary surfaces:
|
||
|
||
- Google Business Profile
|
||
- Apple Maps
|
||
- Bing Places
|
||
- Current website
|
||
|
||
The client-facing report includes material findings only, covering:
|
||
|
||
- NAP consistency
|
||
- Hours consistency, with Google Business Profile as the source of truth
|
||
- Review velocity
|
||
- Website reachability, schema conflicts, build stack, and SSR (rendering) signal
|
||
|
||
## Process
|
||
|
||
### 1. Intake
|
||
|
||
Collect:
|
||
|
||
- Business name
|
||
- City
|
||
- Website
|
||
- Known previous names
|
||
|
||
### 2. Run the pipeline
|
||
|
||
Run:
|
||
|
||
```text
|
||
multi_scraper → audit_engine → report_generate → report_gate
|
||
```
|
||
|
||
The pipeline writes the full required artifact set itself. `report_gate`
|
||
fails the run (non-zero exit) if the §6 gate fails — a failing run does
|
||
not deliver.
|
||
|
||
Website capture also fingerprints the build stack and records the SSR signal
|
||
(`website.stack.primary`, `website.stack.ssr_signal` in the raw JSON). When
|
||
`primary` is not `unknown`, the client report states how the website was
|
||
built. When `ssr_signal` is `weak` (fewer than 50 words in the raw HTML), the
|
||
report flags that non-JavaScript AI crawlers cannot read the page body, and
|
||
any capture-based finding about website content must be qualified accordingly.
|
||
|
||
### 3. Human validation (reviewer decision loop)
|
||
|
||
The reviewer must resolve every engine finding to exactly one disposition.
|
||
Unresolved findings block delivery.
|
||
|
||
| Disposition | Effect |
|
||
|-------------|--------|
|
||
| **Approve** | Finding stands as generated |
|
||
| **Reject** | Finding defaults to **Tier 2** (corroborated, not verified) unless the reviewer (a) discards it entirely, or (b) requests more evidence |
|
||
| **Request more evidence** | Finding marked `pending`; re-verified on the next run; may not ship as Tier 1 until evidence lands |
|
||
|
||
Reviewer decisions are recorded in the run's `VALIDATION.md` (one line per finding: finding id, disposition, rationale).
|
||
|
||
### 4. Blocked surfaces
|
||
|
||
A surface that fails capture (blocked, 403, no data returned) must never be
|
||
reported as a bare "✗" with no downstream treatment.
|
||
|
||
- A blocked or missing **customer-facing** surface (website, booking page) is a
|
||
Customer Path Leakage signal: it ships as a finding, or the reviewer records
|
||
a documented limitation in `VALIDATION.md` with the reason it is not material.
|
||
- Non-customer-facing surface failures (e.g. Bing empty) are noted in the
|
||
Surfaces Reviewed table with the failure reason.
|
||
|
||
### 5. Client-facing report
|
||
|
||
Prepare a report containing material findings only. The report uses the approved
|
||
VeriPath client-facing structure and voice. It does not include coaching
|
||
language or marketing urgency.
|
||
|
||
### 6. Pre-delivery consistency gate
|
||
|
||
Executed by `implementation/auditing/report_gate.py` as pipeline step 4;
|
||
result stamped into `VALIDATION.md`. All four checks pass before the
|
||
report is sent. Any failure blocks delivery:
|
||
|
||
1. **Count match:** the executive summary's stated number of findings equals the
|
||
number of findings in the Findings section.
|
||
2. **Evidence present:** every finding's cited evidence exists in the run's
|
||
archived artifacts (raw capture or `findings.json`). Absence-type findings
|
||
(no website, no description, UTM parameters, unreachable surface) are
|
||
re-derived from the raw capture instead of requiring quoted evidence.
|
||
3. **Identity match:** report client name, location, and audit date match the
|
||
intake record.
|
||
4. **Material support:** each material finding is re-derived against the raw
|
||
capture (phone cores across surfaces, hours day counts, rating, website
|
||
presence, description/OG fields).
|
||
|
||
### 7. Delivery
|
||
|
||
Deliver the validated client-facing report to the client using the locked
|
||
delivery template.
|
||
|
||
## Required artifact set
|
||
|
||
Every run is archived under `docs/validation/<YYYY-MM-DD>-<slug>/` with:
|
||
|
||
| Artifact | Source |
|
||
|----------|--------|
|
||
| `*_multi_surface_YYYY-MM-DD.json` | raw pipeline capture |
|
||
| `findings.json` | engine output |
|
||
| `findings.md` | engine markdown |
|
||
| `REPORT-final.md` | client-facing report (post-review) |
|
||
| `VALIDATION.md` | run notes, surface results, reviewer decisions |
|
||
| `DELTA.md` | before/after field comparison (re-runs only) |
|
||
|
||
Re-runs never overwrite the prior raw capture: the pipeline writes a
|
||
timestamped sibling (`*__YYYYMMDD-HHMMSS.json`). The prior capture is the
|
||
temporal baseline; `audit_diff.py` compares the two into `DELTA.md`.
|
||
|
||
Runs completed before 2026-08-15 are grandfathered as-is. Every run on or
|
||
after this date must complete the set before delivery.
|
||
|
||
## Out of Scope for Default Beta
|
||
|
||
The following are not included in the default beta audit:
|
||
|
||
- Exhaustive secondary-directory research
|
||
- Full AI chatbot visibility scoring
|
||
- Competitive benchmarking
|
||
- Social content audits
|
||
|
||
## Conditional Legacy Check
|
||
|
||
Run a quick old-name and old-website check only when a name or location change
|
||
in the last 3–5 years is known or indicated during intake. This check is not
|
||
part of every default beta run.
|
||
|
||
## Pipeline Reference
|
||
|
||
The working pipeline is located at:
|
||
|
||
```text
|
||
implementation/auditing/
|
||
```
|
||
|
||
The standard execution path is:
|
||
|
||
```text
|
||
multi_scraper → audit_engine → report_generate → report_gate
|
||
audit_diff (re-runs: baseline capture vs. new capture)
|
||
```
|
||
|
||
## Deviations
|
||
|
||
Any deviation from this process requires a dated decision record in
|
||
`docs/decisions/`. Edits to this document follow the same rule: the change is a
|
||
dated decision, not a silent edit.
|
||
|
||
## Deeper Engagements
|
||
|
||
The full Layer 1/2 system remains available for deeper engagements. Beta audits
|
||
use the streamlined path above.
|