veripath/docs/agents/audit-playbook-v1.md

Audit Playbook v1

Purpose: Give any agent (especially Leonard) a fixed, Charter-compliant method for performing Layer 1 (Data Inventory) and Layer 2 (Threat Register) work.

Source: Extracted from the Overcome Fitness benchmark process (Director + Leonard iterations).

Governing documents:

• docs/agents/agent-charter-v1.md
• docs/architecture/path-to-poc-sequencing.md
• Templates in docs/templates/

GBP-specific tooling:

• Intake template: docs/templates/gbp-snapshot-intake.md
• Leonard execution card (minimal reasoning): docs/agents/leonard-gbp-execution-card.md
• HTML form (fill → copy/paste to Leonard): tools/gbp-snapshot-form.html

---

Non-Negotiables

1. Evidence Model only — Every fact is Tier 1 (Verified) or Tier 2 (Indicative). No unmarked claims.
2. Agents draft. Humans approve. Nothing is locked without human sign-off.
3. No invented data, numbers, or threats.
4. Order is fixed: Data Inventory (Layer 1) before Threat Register (Layer 2). Never reverse.
5. No solution language in Layer 1 or 2. Diagnosis only. No “fix this,” “claim that,” or workflow suggestions.
6. Stale data must be flagged. If a snapshot is older than ~30–60 days or its date is unknown, mark it and treat findings as provisional.
7. Prefer the more constrained, evidence-bound position when uncertain.

---

Layer 1 — Data Inventory Rules

Goal: Record what exists, where it lives, and verification status. Nothing else.

Required sources (minimum)

• Google Business Profile / GBP (or clear statement that it is missing/unclear)
• Website
• Primary social channel(s)
• Booking / scheduling system (or explicit “not visible”)
• Reviews / reputation surfaces if observable

Required fields per source

• Access Status
• Data Range Available (use date ranges, not vague “12 months”)
• Snapshot Date
• Verification Tier (Tier 1 or Tier 2)
• Notes / Gaps
• Snapshot Location path

Rules

• Public observation = Tier 2 until owner access or stronger verification upgrades it.
• Do not claim “owner access” unless it is confirmed in the current process.
• Missing sources are documented as gaps, not ignored.
• Technical findings (404s, schema, NAP conflicts, etc.) require a dated snapshot. Old findings must be marked for re-verification.
• GBP without owner access: use human snapshot via tools/gbp-snapshot-form.html (or markdown intake). Leonard maps via the execution card only — no free-form GBP reasoning.

Exit criteria (human must confirm)

• All accessible sources recorded
• Every source has Tier + Snapshot Date
• Gaps explicitly listed
• No analysis or threat language present
• Human sign-off obtained

---

Layer 2 — Threat Register Rules

Goal: Identify discrete, evidence-backed failures in customer acquisition or retention. Diagnosis only.

Required fields per threat

• Threat ID (T-001, T-002…)
• Title (short, factual)
• Signal Source (must map to an inventory row)
• Evidence Snippet (specific, observable fact)
• Data Recency (must match or reference inventory snapshot dates)
• Estimated Severity (Critical / Major / Minor)
• Verification Tier
• Notes / Context (optional, still factual)
• Human Decision (Pending until review)

Rules

• One primary fact per threat. Do not bundle multiple unrelated gaps under one ID.
• Evidence Snippet must be concrete (what was observed), not a general claim.
• Severity is preliminary and Tier-linked. Do not escalate severity beyond what the evidence supports.
• Competitor comparisons are secondary context only, never the primary claim.
• If data is stale, say so in Data Recency or Notes.
• No recommendations, fixes, or next actions.
• GBP-related threats: only create/update via execution-card trigger rules R1–R3 when ingesting a GBP snapshot.

Severity guidance (use conservatively)

• Critical: Directly blocks or severely impairs local discovery / conversion and evidence is clear.
• Major: Material friction or visibility gap with solid observation.
• Minor: Real but lower immediate impact, or evidence is thin.

When unsure, choose the lower severity.

Exit criteria (human must confirm)

• Every threat traces to the inventory
• Every threat has Evidence Snippet + Data Recency + Tier
• No solution language
• No invented threats
• Human has recorded decisions (Verified / Downgraded / Rejected)

---

Operating Pattern for Agents

1. Read the Charter and this Playbook first.
2. Work only inside the current layer.
3. Produce drafts that match the templates.
4. List every change made when revising an existing document.
5. Surface blockers and stale data explicitly.
6. Stop at the human review gate. Do not self-lock.

GBP cold-audit pattern

1. Human fills tools/gbp-snapshot-form.html (or markdown intake).
2. Human pastes output to Leonard with: follow docs/agents/leonard-gbp-execution-card.md exactly.
3. Leonard maps fields and fires only R1–R3.
4. Human reviews; Director may score against the rubric below.

---

Training / Evaluation Rubric (simple)

Score a Leonard (or any agent) run 0–2 on each:

Criterion	0	1	2
Evidence grounding	Invented or vague	Mostly grounded	Fully concrete snippets
Tier discipline	Missing or wrong	Present but soft	Correct and consistent
Scope control	Drifted into solutions/requirements	Minor drift	Strict Layer 1/2 only
Recency honesty	Ignored stale data	Partially flagged	Explicitly flagged
Template fidelity	Structure broken	Mostly followed	Clean match

Pass threshold: No zeros, and total ≥ 7/10.
Benchmark quality: Total 9–10 with zero scope violations.

---

Current Benchmark Reference

• Client: Overcome Fitness
• Files: docs/clients/overcome-fitness/data-inventory-v1.md and threat-register-v1.md
• Status: Draft (Benchmark) — not yet Locked; GBP completeness still open

Future Leonard runs should be compared against this package and scored with the rubric above.

---

Version: v1
Status: Active for training
Next upgrade trigger: After first Locked client Layer 1+2 package exists and at least one Leonard scored run is complete.