Files
SlabUploader/docs/TEST-SPEC.md
T
grokbot_ty 8deb18de1b docs(test-spec): encode PRODUCT GATE vs REPORTING locks
Separate fail-closed Results reporting from product-gate readiness.
Thin BVTs and tip/paperwork cross-checks do not unlock UI, walkable, or UAT.
2026-09-07 21:57:12 +00:00

50 KiB
Raw Blame History

TEST-SPEC — Engineer Test Specification

Status: SPEC · engineer bar · SoT gitea-atd main · 2026-09-07

This document defines every affirmative test case for the SlabUploader, from Phase 0 through production. Each case states what must happen, where the rule lives, and how it is verified. The human walkable practice script is at docs/UAT-PRACTICE.md and is left untouched by this file.


1. Case ID naming convention

Case IDs follow the pattern <prefix>-<number>. The prefix identifies the test layer or gate:

Prefix Meaning Examples
TV Test Vector from docs/TECH-SPEC-PIPELINE.md §8 TV-1, TV-10
V Validation rule (shared validation module) V-1, V-3
AES AES-GCM encryption / decryption AES-1, AES-2
H Health endpoint H-1
D Draft store (slab creation / persistence) D-1
GC Gate C case (early Phase 1, claimable draft listing) GC-1, GC-5
W WooCommerce mock integration W-1, W-5
INF Inference endpoint probe / toggle INF-1, INF-4
UI UI behavior (capture, knobs, overlay, ready gates) UI-1, UI-5
UIE UI end-to-end (review, publish, settings, full flow) UIE-1, UIE-10
API API contract (routes, concurrency, error responses) API-1, API-10
TAX Taxonomy sync / anchor / species cache TAX-1, TAX-3
C1 Call 1 (vision) inference call C1-1, C1-3
C2 Call 2 (content/text) inference call C2-1, C2-4
OFF Inference OFF path (full suite without inference) OFF-1, OFF-2
OPS Operations / deployment / backup OPS-1, OPS-5
INS Instrumentation / observability INS-1, INS-4
E2E End-to-end browser test (full user flow) E2E-1, E2E-5
FUND Fundamentals (cross-cutting dependency bar) FUND-1, FUND-25

Other abbreviations used in this document:

  • sqft: square feet
  • bdft: board feet
  • U2Net: Universal 2 Network, a background-removal model (deferred from the proof of concept)
  • API: application programming interface
  • E2E: end-to-end
  • UI: user interface
  • POC: proof of concept
  • SKU: stock keeping unit
  • Woo: WooCommerce, the WordPress e-commerce plugin
  • HTTP/HTTPS: Hypertext Transfer Protocol (plain / secure)
  • AES-GCM: Advanced Encryption Standard in Galois/Counter Mode
  • UUID: universally unique identifier
  • JSON: JavaScript Object Notation
  • LLM: large language model

2. Evidence & process contract

This contract is the mill-owner enforcement bar for test evidence. Soft words do not count. Tables below are the rules.

Evidence class vocabulary

Every case row and every Results row must declare one Evidence class. No class means the row cannot be marked pass.

Evidence class What it proves Honest sources
product-state Real product behavior on real fixtures or the real deployed path Real slab photos, real store draft opened by Ty, real host HTTPS health
deploy The inventoried host and compose path work as specified Compose up, Caddy HTTPS, inventory filled, backup/restore on the real host
e2e A real browser session through the product UI Playwright (or equivalent) against a started stack
human-walk A person walked the product and recorded what they saw Ty or delegate walk notes tied to named case IDs
proxy-synthetic Synthetic doubles only. Unit fixtures, in-process fakes, stub UIs, mocked Woo/inference, wrong-interface curls Vitest/pytest fixtures, respx mocks, stub frontend
api-contract HTTP request/response shape and status through the app API httpx TestClient or equivalent against FastAPI

Extend this list only when an existing table needs a clear new class. Do not invent classes ad hoc in a Results file.

Proxy isolation

Proxy evidence may fill only cells whose declared Evidence class is proxy-synthetic, or a cell that explicitly names a proxy class.

Proxy evidence never fills product-state, deploy, e2e, or human-walk.

Examples of proxy evidence include synthetic image fixtures, in-process fakes, stub UIs, mocked Woo or inference, and curls that hit the wrong interface.

Fail-closed results

Allowed result vocabulary only:

Result Meaning
pass The named case ran, evidence matches the declared Evidence class, and the expected outcome held
fail The named case ran and the expected outcome did not hold. Blocker pointer required
blocked The named case could not be run to a verdict. Blocker pointer required

Soft or ambiguous words do not count as pass. That includes "looks good", "mostly", "deferred", "follow-up", and treating an unrun case as quiet success.

not run is not an allowed result. Use blocked with a blocker pointer.

No Evidence class on a Results row means that row cannot be marked pass.

Gate exit is its own verifiable unit

Three different things:

Unit What it is Who closes it
Coding tip Code or docs landed on the branch Anyone with write access
Test complete Named case rows have Results with honest Evidence class and pass/fail/blocked The test runner report
Gate ready Gate exit checklist is filled with the required Evidence classes, and Ty confirms Ty only

Coding tip is not test complete. Test complete is not Gate ready.

Gate or UAT exit needs a mill-owner Results table whose proof classes match the declared exit checklist for that gate.

Bots never declare Gate B or Gate C green.

Product gate vs reporting

PRODUCT GATE vs REPORTING. Fail-closed Results paperwork is not a fail-closed product gate. A filled Results table is reporting. It is not UI, walkable, or UAT readiness.

Coding tip is not test complete. Test complete is not Gate ready. Report complete is not Gate ready.

Signal What it is Unlocks UI, walkable, or UAT readiness?
Results table filled (pass, fail, or blocked, even fail-closed) Reporting against named case IDs No
Tip or paperwork cross-check of prior layers only Check of prior tips or paperwork only. Forbidden as readiness No
Thin BVT (Select Photos file-inject only, curl health, SvelteKit title check, or similar shallow probe) Shallow smoke No
human-walk or live-host e2e that induces the claimed product state Required proof class for readiness claims Yes, when the proof matches the claim

Before any UI, walkable, or UAT readiness claim, require human-walk or live-host e2e proof. That proof must induce the claimed product state. Do not shrink Done-when to thin proxies.

For mill-owner phone and iPad mask fundamentals, that walk or live-host e2e must show all of the following:

  • Photo visible under the translucent overlay (not a blank canvas, not a green-only washout that hides the wood)
  • Knobs usable without multi-second lag
  • Sheet mode not all-slab washout

Dishonest proxies for UI, walkable, or UAT readiness (never treat as unlock):

  • Filled Results tables, including fail-closed ones
  • Tip cross-checks
  • Select Photos file-inject BVT
  • curl health
  • SvelteKit title checks
  • Soft language that equates "report complete" with "gate ready"

Dual proof for product-state TV cases

Some TV cases list product-state as the case Evidence class because Verified by requires real fixtures (TV-3, TV-6, TV-7, TV-10).

A separate Results row may record a proxy-synthetic pass for the unit runner. That row is honest proxy proof only. It does not satisfy Gate B exit for those cases.

Gate B exit still requires product-state or human-walk evidence on real fixtures for TV-3, TV-6, TV-7, and TV-10.

Build brief required fields

A product or gate Build brief must name all three fields below. If the ticket claims product or gate proof and any field is missing, refuse the brief.

Required field What to write
product state induced Which real product state this case creates or observes
dishonest proxies for this case Which doubles would look like a pass but do not count
does NOT count as pass Explicit non-pass list for this case

SoT mutation permit

Personas, skills, docs/TEST-SPEC.md, and Builds that change Source of Truth need explicit Ty go (or PM relaying that go) recorded before the write.

Field Required
What SoT artifact changes yes
Ty go (or PM relay of Ty go) yes, dated, before the write
Why the change is needed yes

Approval sits before the write, not after the apology.


3. Results recording & report-out

Every report maps to named case IDs from this spec. A filled Results table is reporting, not gate readiness. See §2 Product gate vs reporting.

Allowed results only: pass, fail, blocked. Never invent pass. Soft words do not count.

Every Results row must include Evidence class. Allowed classes are listed in §2.

Evidence is required for every pass or fail. Use a path or paste of real runner output (pytest, httpx, compose logs, curl, walk notes). No evidence means blocked with a blocker pointer.

Report table columns:

Column Required
Case yes
What this proves (mill-owner English) yes for gate Results
Evidence class yes
Result yes (pass | fail | blocked)
Evidence yes for pass/fail; pointer or blocker for blocked
Blocker required when result is fail or blocked

Gitea issues are the only mechanism for reporting issues found during a test pass. Chat, Slack, and PR comments are not the bug log. On fail, open or update a Gitea issue that includes the case id and evidence pointer. The report table blocker column links that issue. Do not invent issues that lack runner evidence.

The Test Bar may only summarize what runners printed. Ty also reads raw results and owns gate exit. Gate or phase exit evidence in the Gates section still requires Ty confirmation where IMPL-PLAN mandates it. Bots never declare Gate B or Gate C green.

When code does not exist yet, Phase 0 and earlier cases stay blocked with a blocker that names the missing surface. Do not speculate.

4. How cases are executed

Test automation lives in the repo under backend/tests/ and frontend/e2e/. The harness uses:

  • Unit tests: backend/tests/unit/ (pytest) and frontend/src/__tests__/ (Vitest). These run against pure modules with no network calls.
  • API contract tests: backend/tests/contract/ (pytest + httpx TestClient). These hit the FastAPI app through its test client, asserting request/response shape, status codes, and revision handling.
  • Integration tests: backend/tests/integration/ (pytest + httpx TestClient + respx WooCommerce mock). These verify Woo create, duplicate SKU, taxonomy sync, stale-field recovery, and settings encryption round-trips without touching a real store.
  • UI end-to-end tests: frontend/e2e/ (Playwright). These exercise the full browser flow: capture, mask knobs, axis confirm, numbers entry, review, and publish. They run against a started Docker Compose stack with the FastAPI test server and mocked Woo endpoints.
  • Ops / deployment tests: deploy/tests/ (shell scripts). These verify compose health, backup/restore, and Caddy routing.

Test dependencies are pinned in backend/tests/requirements-test.txt and frontend/package.json (devDependencies). CI runs the full suite before any merge; the suite must pass with inference disabled.


5. Test pyramid overview

Each row states the test layer, when tests in that layer are written, and what document drives the expected behavior.

Layer When written Primary source of truth
Unit TDD: unit tests written before the feature code exists docs/TECH-SPEC-PIPELINE.md, docs/AGENTS.md
API / contract Alongside the backend feature code docs/OPENAPI.md, docs/DATA-MODEL.md
Integration Alongside the backend feature code docs/CONTENT-WOO.md, docs/ARCHITECTURE.md
UI / E2E Alongside the frontend feature code docs/UX.md, docs/TECH-SPEC-PIPELINE.md
Ops / deployment Alongside deployment configuration docs/DEPLOYMENT.md, docs/IMPL-PLAN.md Phase 6

Test-driven development (TDD): write the failing unit test first, then write the minimum code to pass it. This applies to all deterministic core functions: bdft math, pricing, rounding, image crop, and validation rules.

Tests-with-code: API contract tests, integration tests, and UI end-to-end tests are written in the same commit that implements the feature. Each feature commit includes its tests and shown execution output in the commit message.


6. Fundamentals — cross-cutting bar

These cases are the white-box bar for every dependency that touches the product. They run across all phases.

SvelteKit / Vitest (frontend/src/tests/)

Case What must happen Where the rule lives Verified by Evidence class
FUND-1 Pure logic tests have zero browser, DOM, or network dependencies SvelteKit/Vitest convention Vitest unit test: vi.mock any runtime import, assert no browser globals proxy-synthetic
FUND-2 Component/browser tests use semantic getByRole selectors; class or data-testid alone is not sufficient SvelteKit testing-library convention Vitest component test: assert role-based query succeeds proxy-synthetic
FUND-3 Real Tab key navigates focus; click-only tests are not sufficient for keyboard paths UX.md §Browser compatibility Playwright E2E: page.keyboard.press('Tab') followed by toBeFocused e2e
FUND-4 Shared rune/state changes flush before DOM assertion; stale store values do not pass Svelte rune reactivity Vitest component test: flush/store tick before expect proxy-synthetic

Playwright (frontend/e2e/)

Case What must happen Where the rule lives Verified by Evidence class
FUND-5 Tab order: press Tab, assert toBeFocused after each stop; no keyboard trap Playwright convention; UX.md §Browser compatibility Playwright E2E: sequential keyboard.press('Tab') + toBeFocused e2e
FUND-6 Viewport resize/reflow at ~320 px and phone/PC sizes: layout does not clip, overlap, or trap content UX.md §Browser compatibility Playwright E2E: setViewportSize then assert no clip/overlap e2e
FUND-7 Focus ring visible on touch and keyboard; focus ring is not hidden by outline: none on interactive elements Playwright convention; WCAG 2.4.7 Playwright E2E: assert focus ring CSS present on focused interactive element e2e

FastAPI / pytest TestClient (backend/tests/)

Case What must happen Where the rule lives Verified by Evidence class
FUND-8 App factory pattern creates a fresh app instance per test; no shared mutable state between tests FastAPI testing convention pytest TestClient: each test calls the factory, not a module-level app api-contract
FUND-9 with TestClient(app) manages lifespan; startup and shutdown hooks run and clean up FastAPI testing convention pytest TestClient: context manager exits with clean lifespan api-contract
FUND-10 Clear dependency_overrides after every test; no override bleeds to the next test FastAPI testing convention pytest TestClient: assert dependency_overrides empty after each test api-contract
FUND-11 Override Woo/inference dependencies with mocks; no real network call leaks FastAPI testing convention pytest TestClient: assert zero HTTP calls to real Woo/inference endpoints api-contract

SQLite + Alembic (backend/tests/)

Case What must happen Where the rule lives Verified by Evidence class
FUND-12 PRAGMA foreign_keys=ON enforced in every test database; FK violation rejected SQLite testing convention; DATA-MODEL.md pytest + SQLite: insert orphan row, assert IntegrityError proxy-synthetic
FUND-13 Unique and check constraints enforced; duplicate or out-of-range values rejected SQLite testing convention; DATA-MODEL.md pytest + SQLite: insert violating row, assert IntegrityError proxy-synthetic
FUND-14 Concurrent poll / WAL stale-read risk handled; WAL mode with proper isolation level prevents stale reads SQLite WAL convention; DATA-MODEL.md pytest + SQLite: concurrent reader/writer, assert no stale read proxy-synthetic
FUND-15 Illegal status transitions rejected by the application layer; status machine enforced AGENTS.md §7; DATA-MODEL.md slabs pytest TestClient: attempt illegal transition, assert 422 api-contract
FUND-16 Migrations must not silently drop integrity indexes or constraints; post-migration assertions verify schema integrity Alembic convention; DATA-MODEL.md pytest + SQLite: run migration, assert indexes/constraints preserved proxy-synthetic

AES-GCM (settings)

Case What must happen Where the rule lives Verified by Evidence class
FUND-17 Fresh nonce per encrypt; tampered ciphertext or wrong key/nonce raises authentication failure AES-GCM specification; AGENTS.md §4 Unit test: tamper ciphertext, assert auth failure; encrypt twice, assert nonces differ proxy-synthetic
FUND-18 Ciphertext never appears in any GET response; only the configured username and woo_credentials_configured boolean returned OPENAPI.md §settings; DATA-MODEL.md settings API contract test: assert woo_app_password absent from GET /api/v1/settings api-contract
FUND-19 Missing SLAB_AES_KEY refuses startup; no fallback to plaintext storage AGENTS.md §4; DATA-MODEL.md settings Unit test: unset env var, assert startup rejection proxy-synthetic

Woo REST client (httpx + App Passwords over HTTPS)

Case What must happen Where the rule lives Verified by Evidence class
FUND-20 Basic Auth with App Passwords over HTTPS; consumer keys not implemented AGENTS.md §4; DEPLOYMENT.md §3; OPENAPI.md §settings Unit test: assert HTTP Basic header, HTTPS-only URL proxy-synthetic
FUND-21 401/400/404 response shapes handled; server maps to actionable error, not raw passthrough Woo REST convention; CONTENT-WOO.md §2.6 Integration test with respx: mock 401/400/404, assert mapped error response proxy-synthetic
FUND-22 Pre-create SKU lookup mandatory; Woo concurrent duplicate-SKU race handled by checking before create AGENTS.md §5; CONTENT-WOO.md §2.5 Integration test with respx: create duplicate SKU, assert 409 before create proxy-synthetic
FUND-23 Timeouts and network errors handled gracefully; server returns actionable error, not stack trace Woo REST convention; CONTENT-WOO.md §2.6 Integration test with respx: mock timeout/connection error, assert graceful error proxy-synthetic
FUND-24 Secrets redacted in logs; no Woo username/password, inference key, or AES key in any log output AGENTS.md §4; ARCHITECTURE.md §5 Unit test: assert log output contains no secret material proxy-synthetic

Compose / Caddy (deploy/tests/)

Case What must happen Where the rule lives Verified by Evidence class
FUND-25 HTTPS-only WOO_BASE_URL; health through proxy; inventory/port collision checked at deploy time DEPLOYMENT.md §3; AGENTS.md §9 Shell script: assert HTTPS, health through Caddy, port conflict detection deploy

Phase 0 / Gate B — Hybrid skeleton + deterministic core

Gate B exit checklist

Coding tip is not test complete. Test complete is not Gate ready. Bots never declare Gate B green.

Exit condition Required Evidence class Status owner
TV-3 bdft identity on real fixtures product-state or human-walk Ty
TV-6 crop/PNG rules on real fixtures product-state or human-walk Ty
TV-7 undersized-source warn/retake on real fixtures product-state or human-walk Ty
TV-10 rounding on real fixtures product-state or human-walk Ty
Health returns 200 on the specified HTTPS deploy path deploy Ty
AES credentials round-trip without ciphertext leak api-contract Ty
Draft store create/persist/retrieve api-contract Ty
Unit and API automation recorded for Gate B cases proxy-synthetic and/or api-contract as declared per case Test report

A prior proxy-synthetic pass for TV-3, TV-6, TV-7, or TV-10 is not Gate B product-state proof.

Unit — deterministic core (test-driven)

Case What must happen Where the rule lives Verified by Evidence class
TV-1 Green-sheet synthetic photo: interior green-ish patch that does not touch the border stays after chroma-key; U2Net is not invoked docs/TECH-SPEC-PIPELINE.md §3.1, §3.4 Unit test with synthetic image fixture proxy-synthetic
TV-2 Black-sheet synthetic photo: dark interior wood that does not touch the border stays after threshold; flood-fill isolates border-connected pixels only docs/TECH-SPEC-PIPELINE.md §3.1, §3.4 Unit test with synthetic image fixture proxy-synthetic
TV-3 Slab 96" long, 1.5" thick, mask 0.64 square feet → bdft = 0.96. Identity: 1 square foot at 1" thick equals 1 bdft. No divide-by-12. docs/TECH-SPEC-PIPELINE.md §4.6 Unit test, shown output on real fixtures product-state
TV-4 Width samples at 6" stations on a 96" length axis report min, max, and average perpendicular to the confirmed axis docs/TECH-SPEC-PIPELINE.md §4.4 Unit test with fixture mask proxy-synthetic
TV-5 Slab with one square end and one 45° cut: length axis follows prevailing length; widths stay perpendicular after user confirm docs/TECH-SPEC-PIPELINE.md §4.1, §4.2 Unit test with angled-end fixture proxy-synthetic
TV-6 3:4 crop, slab centered, 80 percent fill at extremes, transparent PNG, shorter side ≥ 1600 when source allows docs/TECH-SPEC-PIPELINE.md §5, §7.1 Unit test, shown output on real fixtures product-state
TV-7 Undersized source: after 80 percent crop shorter side would be below 1600 and source cannot supply it → warn, no upscaling, photo not publishable until retake docs/TECH-SPEC-PIPELINE.md §5, §7.1 Unit test, shown output on real fixtures product-state
TV-8 Pricing: bdft 0.96, species $12.50/bdft → recommended price 12.00; override 150.00 stored as 150.00; no rule → recommended price empty, manual 99.99 stored docs/TECH-SPEC-PIPELINE.md §6, §8 TV-8 Unit test with pricing rules proxy-synthetic
TV-9 Thickness 1" → Woo range rounds up (for example 1"–1 1/2") using synced attribute term, not a hardcoded label docs/TECH-SPEC-PIPELINE.md §8 TV-9, docs/DATA-MODEL.md slabs Unit test with mock taxonomy proxy-synthetic
TV-10 Rounding: 96.04" → 96"; square feet 0.641 → 0.64; bdft 11.525 → 11.53; price 12.345 → 12.35. Half-up to 2 decimals. docs/TECH-SPEC-PIPELINE.md §8 TV-10 Unit test, shown output on real fixtures product-state

Unit — shared validation

Case What must happen Where the rule lives Verified by Evidence class
V-1 SKU charset [A-Z0-9-]{3,24} accepted; values outside the charset rejected docs/TECH-SPEC-PIPELINE.md §2, issue #17 Unit test with valid and invalid SKU strings proxy-synthetic
V-2 Title, short title, description: UTF-8 within WooCommerce maxLength enforced at field exit and at submit docs/TECH-SPEC-PIPELINE.md §2, issue #17 Unit test with boundary-length strings proxy-synthetic
V-3 Validation module version hash: client caches module; submit carries hash; mismatch returns 412 with fresh module, not 409 docs/OPENAPI.md §validation, issue #17 Unit test of validation module hash comparison proxy-synthetic

Unit — AES-GCM settings

Case What must happen Where the rule lives Verified by Evidence class
AES-1 Encrypt then decrypt round-trips for woo_wp_username and woo_app_password; ciphertext never appears in GET settings docs/ARCHITECTURE.md §5, issue #5 Unit test with round-trip encryption api-contract
AES-2 Missing SLAB_AES_KEY environment variable at startup: app refuses to start; no fallback to plaintext docs/AGENTS.md §4, docs/DATA-MODEL.md settings Unit test with missing env var proxy-synthetic

API / contract — health + draft store stub

Case What must happen Where the rule lives Verified by Evidence class
H-1 GET /api/health returns 200 { "status":"ok" } with woo_reachable false when credentials are not configured docs/OPENAPI.md §health, docs/IMPL-PLAN.md Phase 0 API contract test via httpx TestClient api-contract
D-1 POST /api/v1/slabs accepts multipart with data JSON + files + metadata; returns 201 with status: calibrated and server_rev docs/OPENAPI.md §slabs, docs/DATA-MODEL.md slabs API contract test via httpx TestClient api-contract

Gate C — Early Phase 1 (first draft listing, inference disabled)

Exit evidence (Evidence class). Ty opens one SLAB-UAT-* WooCommerce draft listing in the real store admin (human-walk or product-state). Inference OFF. Mocked-store automation is proxy-synthetic only and is not Gate C exit. Bots never declare Gate C green.

API / contract — create + draft

Case What must happen Where the rule lives Verified by Evidence class
GC-1 POST /api/v1/slabs with SLAB-UAT-* SKU creates a WooCommerce product with status: "draft" regardless of the woo_create_status setting docs/AGENTS.md §5, docs/CONTENT-WOO.md §2.4 API contract test with mocked Woo api-contract
GC-2 Duplicate SKU before create: GET /products?sku={sku} finds existing → 409 duplicate_sku with { field:"sku", actions:["edit_sku","open_existing"] } docs/CONTENT-WOO.md §2.5, docs/OPENAPI.md §slabs, issue #3 API contract test with pre-existing SKU in mock api-contract
GC-3 POST /api/v1/settings/test-woo returns ok with populated taxonomy; always resyncs taxonomy on success docs/OPENAPI.md §settings, issue #14 API contract test with mocked Woo taxonomy api-contract
GC-4 WOO_BASE_URL is compose-only HTTPS; Settings GET returns it read-only; PUT rejects write attempts with 422 docs/AGENTS.md §5, issue #11 API contract test with PUT attempt api-contract
GC-5 FastAPI refuses to start when WOO_BASE_URL environment variable is missing or set to http:// docs/AGENTS.md §4, docs/DEPLOYMENT.md §3 Unit test for startup config validation proxy-synthetic

Integration — WooCommerce mock (draft create, SLAB-UAT-*, duplicate SKU, taxonomy)

Case What must happen Where the rule lives Verified by Evidence class
W-1 Woo mock returns a new product with status draft; server stores woo_product_id and purges images docs/CONTENT-WOO.md §2.4, docs/IMPL-PLAN.md Phase 4 Integration test with respx mock proxy-synthetic
W-2 Woo mock returns 409 for duplicate SKU; server maps to { field:"sku", actions:["edit_sku","open_existing"] }; draft retained docs/OPENAPI.md §slabs, issue #3 Integration test with respx mock returning 409 proxy-synthetic
W-3 Taxonomy sync returns cached data; anchor stays unchanged on no-op sync; anchor bumps only when stored-subset hash changes docs/ARCHITECTURE.md §7, issue #14 Integration test with taxonomy mock proxy-synthetic
W-4 test-woo always runs taxonomy resync after successful connection test docs/ARCHITECTURE.md §7, issue #14 Integration test: assert taxonomy resync call proxy-synthetic
W-5 Failed publish: slab status failed, photos retained, sync_log written; retry from same screen without re-photographing → success → photos purged docs/CONTENT-WOO.md §2.6, docs/TECH-SPEC-PIPELINE.md §5, docs/DATA-MODEL.md slabs Integration test with retryable mock failure proxy-synthetic

Integration — inference disabled path + test-inference probe (issue #2 substance)

Case What must happen Where the rule lives Verified by Evidence class
INF-1 With inference_enabled: false: slab create skips Call 1; inference_status is null; full suite passes end-to-end with no inference network calls docs/AGENTS.md §6, docs/IMPL-PLAN.md Phase 5 Integration test asserting zero inference calls proxy-synthetic
INF-2 POST /api/v1/settings/test-inference returns { ok: true, mode: "stateful", vision_capable: true } on a stateful provider docs/OPENAPI.md §settings, issue #2 Integration test with mocked inference endpoint proxy-synthetic
INF-3 test-inference returns { ok: true, mode: "stateless", vision_capable: true } on a stateless provider; Call 2 uses full resend docs/OPENAPI.md §settings, issue #2 Integration test with mocked inference endpoint proxy-synthetic
INF-4 test-inference returns { ok: false, mode: "fail" }: Call 2 enablement blocked; Gate C path remains green with inference disabled docs/OPENAPI.md §settings, issue #2 Integration test: assert Call 2 blocked proxy-synthetic

UI end-to-end — capture knobs, mask, axis, ready gates (phone + PC)

Case What must happen Where the rule lives Verified by Evidence class
UI-1 Four inline knobs (sheet, sensitivity, edge offset, feather) on capture screen: changing any knob immediately re-runs mask and refreshes overlay without leaving the screen docs/TECH-SPEC-PIPELINE.md §3.3, docs/UX.md §Capture, issue #15 Playwright E2E test: knob change → overlay update, no navigation e2e
UI-2 Mask overlay on source photo: user confirms edge; below-threshold edge after tuning → retake; no U2Net control in the proof of concept docs/TECH-SPEC-PIPELINE.md §3.2, §3.5, docs/UX.md §Capture Playwright E2E test: mask overlay visible, confirm/retake paths e2e
UI-3 Length axis overlay: user rotates if skewed; samples and square feet update live; nearly-square slab: user picks length direction docs/TECH-SPEC-PIPELINE.md §4.2, docs/UX.md §Capture Playwright E2E test: axis rotation, live number update e2e
UI-4 Numbers entry: length 1/8" steps, thickness pick, SKU typed; computed square feet/bdft/widths displayed docs/TECH-SPEC-PIPELINE.md §4, docs/UX.md §Capture Playwright E2E test: type inputs, assert computed output e2e
UI-5 Ready-set gates: slab cannot reach ready until full named set is filled (SKU, length, thickness, square feet/bdft/widths, ≥1 PNG, species, wood category, edge, figure, grade, thickness band, moisture, price); inline nudges per empty required field, not a submit-time list docs/AGENTS.md §7, docs/UX.md §Review, issue #9 Playwright E2E test: attempt ready with missing fields, assert nudges e2e

Ops — compose health

Case What must happen Where the rule lives Verified by Evidence class
OPS-1 GET https://${APP_HOSTNAME}/api/health returns ok after compose up with filled inventory docs/DEPLOYMENT.md §8, docs/IMPL-PLAN.md Phase 0 Shell script: compose up → curl health endpoint deploy

Phase 1 remainder — Full API surface

Exit evidence (Evidence class). Agent-written test automation drives a full calibrated to ready cycle via the API (api-contract). OPENAPI responses match. Revision conflict and taxonomy anchor logic verified. Gate-ready still needs Ty where IMPL-PLAN says so.

API / contract — all routes, concurrency, secrets redaction

Case What must happen Where the rule lives Verified by Evidence class
API-1 PUT /api/v1/slabs/{id} updates fields with client_rev; server echoes new server_rev; mismatch returns 409 revision_conflict with current_server_rev docs/OPENAPI.md §conventions, docs/DATA-MODEL.md slabs API contract test with httpx TestClient api-contract
API-2 PUT /api/v1/slabs/{id} stores client-sent derived fields (square feet, bdft, widths); server does not recompute them as source of truth docs/OPENAPI.md §slabs PUT, docs/ARCHITECTURE.md §3 API contract test: assert no server-side recompute api-contract
API-3 GET /api/v1/settings returns SettingsView with woo_credentials_configured boolean; woo_app_password never returned docs/OPENAPI.md §settings, issue #5 API contract test: assert password absent from response api-contract
API-4 POST /api/v1/slabs/{id}/infer-taxon returns 202 { status: "inferring" }; client polls GET /api/v1/slabs/{id} for results; timeout greater than 45 seconds returns 504 docs/OPENAPI.md §inference, issue #8 API contract test with mocked inference api-contract
API-5 POST /api/v1/slabs/{id}/infer-content returns 202; requires test-inference ok; below-threshold Call 1 (confidence below 0.7): fields stay empty docs/OPENAPI.md §inference, docs/AGENTS.md §6, issue #8/#9 API contract test with confidence-gated mock api-contract
API-6 POST /api/v1/slabs/{id}/publish: pre-condition check returns 422 with field keys for missing required fields; UI maps fields, no raw code docs/OPENAPI.md §publish, issue #18 API contract test with incomplete slab api-contract
API-7 POST /api/v1/admin/taxonomy/sync: pulls categories/attributes/tags; rebuilds cache; anchor bumps only on stored-subset hash change; 502 retains previous cache docs/OPENAPI.md §taxonomy, docs/ARCHITECTURE.md §7, issue #14 Integration test with taxonomy mock api-contract
API-8 GET /api/v1/validation/module returns hash and rules; client caches; submit mismatch returns 412 with fresh module docs/OPENAPI.md §validation, issue #17 API contract test: assert 412 on stale hash api-contract
API-9 DELETE /api/v1/slabs/{id}: soft-delete returns 204; unpublished drafts only; published slabs undeletable docs/OPENAPI.md §slabs DELETE API contract test with published slab api-contract
API-10 test-woo fails closed: missing credentials or non-HTTPS URL returns { ok: false, detail: "..." } without crashing docs/OPENAPI.md §settings test-woo API contract test with bad credentials api-contract

Integration — taxonomy sync + anchor

Case What must happen Where the rule lives Verified by Evidence class
TAX-1 Sync on publish when now - taxonomy_last_sync_at > 1 hour; bump anchor only on real change; stale picked id returns 422 with current options docs/ARCHITECTURE.md §7, docs/CONTENT-WOO.md §2.2, issue #14 Integration test with time-skewed sync api-contract
TAX-2 Species picker offers only WooCommerce-synced species; no hardcoded fallback; new species selectable only after sync stores it docs/ARCHITECTURE.md §7, issue #12 Integration test: assert species list matches cache api-contract
TAX-3 Client reads taxonomy anchor; re-pulls full taxonomy only when anchor advances past client's last value; no-per-navigation sync docs/ARCHITECTURE.md §7, docs/OPENAPI.md §taxonomy Integration test with anchor versioning api-contract

Phase 3 — Frontend review + publish UI

Exit evidence (Evidence class). Ty publishes a test slab to the real store as a WooCommerce draft SLAB-UAT-* (human-walk or product-state). Automation covers UI cases (e2e). Ty is the exit.

UI end-to-end — review, field recovery, publish outcome (phone + PC)

Case What must happen Where the rule lives Verified by Evidence class
UIE-1 Review screen: all fields editable; client-computed square feet/bdft/widths shown with manual override; typed override persists across poll and refresh docs/IMPL-PLAN.md Phase 3, docs/UX.md §Review Playwright E2E test: edit, poll, refresh, assert persistence e2e
UIE-2 Call 1 prefill at or above 0.7 confidence: fields pre-filled and mutable; user override sticks; below 0.7 fields empty; inline nudges for every still-required control docs/AGENTS.md §6, issue #8/#9, docs/UX.md §Review Playwright E2E test with mocked Call 1 results e2e
UIE-3 Duplicate SKU at publish: under SKU field, "This SKU already exists in the store (any status)"; edit SKU or open existing; draft retained; no raw 409 docs/UX.md §Publish, docs/CONTENT-WOO.md §2.5, docs/AGENTS.md §5, issue #3/#18 Playwright E2E test with pre-existing SKU e2e
UIE-4 Stale field at publish: field shows "This value is no longer valid. Pick from the current list"; draft retained; no raw 422 docs/CONTENT-WOO.md §2.6, issue #18 Playwright E2E test with removed taxonomy item e2e
UIE-5 Publish success: screen shows listing created, woo_product_id when available; local draft and photos gone; slab status published docs/CONTENT-WOO.md §2.6, docs/DATA-MODEL.md slabs Playwright E2E test: assert published state and cleanup e2e
UIE-6 Publish failure: plain message, no status number, no stack; retry from same screen without re-photographing docs/CONTENT-WOO.md §2.6, issue #18 Playwright E2E test: assert retry path preserves state e2e
UIE-7 Inference disabled: full manual taxonomy + content path works; no inference network calls; Generate text disabled; templates still work docs/IMPL-PLAN.md Phase 3, docs/UX.md §Review Playwright E2E test with inference flag off e2e
UIE-8 Settings UI: WordPress username + application password + test-woo; inference endpoint + test; pricing rules; brand voice/GEO; woo_create_status; store URL read-only docs/IMPL-PLAN.md Phase 3, docs/OPENAPI.md §settings, issue #11 Playwright E2E test: settings form and test buttons e2e
UIE-9 Changing sensitivity slider immediately refreshes mask overlay without leaving screen or server round trip docs/TECH-SPEC-PIPELINE.md §3.3, issue #15, docs/UX.md §Capture Playwright E2E test: slider → overlay update, no HTTP call e2e
UIE-10 Changing edge offset pulls cut in/out; feather changes alpha blend; sheet switch (auto/green/black) re-runs; all persisted in localStorage until reset docs/TECH-SPEC-PIPELINE.md §3.3, issue #15 Playwright E2E test: knob changes, persist, reset e2e

Phase 5 — Inference (after Gate C is green)

Exit evidence (Evidence class). Ty runs species detection on 3 real slabs with judgment on acceptance (human-walk or product-state). Automation covers Call 1/2 cases (proxy-synthetic). Full inference-disabled path proven (e2e and/or proxy-synthetic as declared).

Integration — Call 1/2, portable context, confidence gating

Case What must happen Where the rule lives Verified by Evidence class
C1-1 Call 1 sends all inventory photos at 1024px + WooCommerce taxonomy only + SKU/length/thickness; model constrained to cached taxonomy from WooCommerce cache docs/IMPL-PLAN.md Phase 5, issue #12 Integration test: inspect Call 1 request payload proxy-synthetic
C1-2 Confidence at or above 0.7: species, wood categories, edge, figure, grade, feat-* pre-filled and mutable; user override wins docs/AGENTS.md §6, issue #8/#9 Integration test with high-confidence mock response proxy-synthetic
C1-3 Confidence below 0.7: those fields stay empty; ready blocked until full named set is filled; species plus one figure is not the gate docs/AGENTS.md §6, issue #8/#9 Integration test with low-confidence mock response proxy-synthetic
C2-1 Call 2 triggered only by user tap; portable Call 1 context: full resend default; confirmed versus inferred labels carried docs/ARCHITECTURE.md §9, issue #2 Integration test: assert no auto-trigger, full resend proxy-synthetic
C2-2 Test-inference stateful mode: previous_response_id used when available; stateless mode: full resend; fail mode: Call 2 blocked docs/OPENAPI.md §settings, issue #2 Integration test with three probe modes proxy-synthetic
C2-3 LLM does not invent dimensions; templates inject deterministic numbers into title and description docs/PROMPTS.md §Call 2, docs/AGENTS.md §6 Integration test: inspect Call 2 response for invented numbers proxy-synthetic
C2-4 Audit: inference_log stores request_payload and response_payload per call; prompt file edits apply without restart docs/DATA-MODEL.md inference_log, docs/PROMPTS.md §Management Integration test: verify log entry, edit prompt file proxy-synthetic

UI end-to-end — inference disabled path

Case What must happen Where the rule lives Verified by Evidence class
OFF-1 Full suite (calibrated to ready to publish to success) passes with inference_enabled: false; zero calls to inference endpoint asserted docs/AGENTS.md §6, docs/IMPL-PLAN.md Phase 5 Playwright E2E test with network assertion for zero inference calls e2e
OFF-2 test-inference fail mode blocks Call 2 enablement only; Gate C inference-disabled path unaffected docs/OPENAPI.md §settings, issue #2 Playwright E2E test: assert Call 2 disabled, full suite passes e2e

Phase 6 / Ops — Deployment, user acceptance, hardening

Exit evidence (Evidence class). Deployment smoke automated (deploy). Ty walks docs/UAT-PRACTICE.md and signs off (human-walk). v1 declared done only after Ty sign-off.

Ops — compose health, deploy path, backup/restore

Case What must happen Where the rule lives Verified by Evidence class
OPS-2 INVENTORY.md filled: APP_HOST, APP_HOSTNAME, ports verified free, Caddy location confirmed, DNS reachable from phone on LAN/Tailscale docs/DEPLOYMENT.md §1, docs/IMPL-PLAN.md Phase 6 Shell script: port scan, Caddy check, DNS resolve deploy
OPS-3 Compose health: GET https://${APP_HOSTNAME}/api/health ok; test-woo ok with species populated; woo_create_status=draft; inference disabled for early gates docs/DEPLOYMENT.md §8, docs/UAT-PRACTICE.md §1 Shell script: curl health + test-woo endpoints deploy
OPS-4 Backup: deploy/backup.sh creates WAL-safe .backup of slab.db and tar of images volume; retention 7 daily + 4 weekly docs/DEPLOYMENT.md §6 Shell script: run backup, verify files and retention deploy
OPS-5 Restore: stop compose → restore DB + untar images → start compose → health ok; tested at least once before declaring backup done docs/DEPLOYMENT.md §6, docs/IMPL-PLAN.md Phase 6 Shell script: full restore cycle with health assertion deploy

Ops — instrumentation (issue #6 substance)

Case What must happen Where the rule lives Verified by Evidence class
INS-1 Health non-200 on database or volume failure; woo_reachable false on WooCommerce unreachable docs/OPENAPI.md §health, issue #6 Unit test with simulated DB/volume failure proxy-synthetic
INS-2 Request ID logged on every server request issue #6 Unit test: assert request ID in log output proxy-synthetic
INS-3 Inference request_payload and response_payload recorded in inference_log; payloads contain model name and confidence data docs/DATA-MODEL.md inference_log, issue #6 Integration test: assert log entry after inference call proxy-synthetic
INS-4 Sync_log written with payload_hash on every publish attempt; retains success and failure docs/DATA-MODEL.md sync_log, issue #6 Integration test: assert sync_log entries for success and failure proxy-synthetic

Ops — end-to-end on all target browsers

Case What must happen Where the rule lives Verified by Evidence class
E2E-1 Capture to mask/axis confirm to calibrated upload to review to publish to WooCommerce draft SLAB-UAT-* works on iOS Safari (phone) docs/UX.md §Browser compatibility, docs/IMPL-PLAN.md Phase 6 Playwright E2E on iOS Safari e2e
E2E-2 Same flow works on Android Chrome (phone) docs/UX.md §Browser compatibility, docs/IMPL-PLAN.md Phase 6 Playwright E2E on Android Chrome e2e
E2E-3 Same flow works on desktop Chrome, Edge, Firefox (PC) docs/UX.md §Browser compatibility, docs/IMPL-PLAN.md Phase 6 Playwright E2E on desktop browsers e2e
E2E-4 Manual length override recalculates widths/square feet/bdft correctly on all three browser families docs/TECH-SPEC-PIPELINE.md §4.3, docs/IMPL-PLAN.md Phase 6 Playwright E2E: override length, assert recalculation e2e
E2E-5 Client crop: slab approximately 80 percent frame; shorter side ≥ 1600 when source allows; undersized source warns and requests retake docs/TECH-SPEC-PIPELINE.md §5, docs/IMPL-PLAN.md Phase 6 Playwright E2E: inspect crop output dimensions e2e

When tests are written

Deterministic core (TV-1 through TV-10, validation, AES): test-driven development. Unit tests written before any user interface consumes the functions. Property checks: bdft monotonic in thickness and square feet; fill band; no upscale path.

User interface, FastAPI, WooCommerce/inference adapters: tests-with-code. Each feature commit includes its test and shown execution output.

Integration tests: httpx TestClient with a respx WooCommerce mock covering success, error, duplicate, stale-field, and taxonomy-anchor paths.

Full suite must pass with inference disabled. Call 1/Call 2 cases are gated after Gate C is green.

bdft equals square feet times thickness in inches (no divide-by-12). TV-3 proves the identity. Gate B requires TV-3, TV-6, TV-7, and TV-10 as product-state or human-walk on real fixtures. A proxy-synthetic unit pass alone does not satisfy that exit.


Cross-reference

These cases Source of truth document(s)
TV-1 through TV-10 docs/TECH-SPEC-PIPELINE.md §8
V-1 through V-3 docs/TECH-SPEC-PIPELINE.md §2, docs/OPENAPI.md §validation
AES-1 through AES-2 docs/ARCHITECTURE.md §5, docs/AGENTS.md §4
GC-1 through GC-5, W-1 through W-5 docs/CONTENT-WOO.md §2.4–2.6, docs/OPENAPI.md §slabs/§settings, docs/DEPLOYMENT.md §3
INF-1 through INF-4 docs/OPENAPI.md §settings, docs/AGENTS.md §6
UI-1 through UI-5 docs/TECH-SPEC-PIPELINE.md §3–§4, docs/UX.md §Capture/§Review
UIE-1 through UIE-10 docs/UX.md §Review/§Publish, docs/CONTENT-WOO.md §2.5–2.6, docs/IMPL-PLAN.md Phase 3
C1-1 through C2-4 docs/AGENTS.md §6–§7, docs/PROMPTS.md §Call 1/2
OPS-1 through OPS-5, E2E-1 through E2E-5 docs/DEPLOYMENT.md §1–§8, docs/IMPL-PLAN.md Phase 6
INS-1 docs/OPENAPI.md §health, docs/AGENTS.md §6
INS-2 issue #6
INS-3 through INS-4 docs/DATA-MODEL.md §inference_log/sync_log, docs/AGENTS.md §6
FUND-1 through FUND-25 docs/AGENTS.md §1–§9, docs/ARCHITECTURE.md §5, docs/OPENAPI.md, docs/DATA-MODEL.md, docs/CONTENT-WOO.md, docs/DEPLOYMENT.md

Issue #19 mapping

This spec folds affirmative case substance from issue #19 where the rule exists in a live doc. Issue #19 is not closed by this work.

Issue #19 item Cases Source of truth document(s)
#2 portable multi-turn INF-2, INF-3, INF-4, C2-1, C2-2 docs/OPENAPI.md §settings, docs/ARCHITECTURE.md §9
#3 duplicate SKU wording GC-2, W-2, UIE-3 docs/OPENAPI.md §slabs, docs/CONTENT-WOO.md §2.5
#5 Application Passwords AES-1, AES-2, API-3 docs/ARCHITECTURE.md §5, docs/DATA-MODEL.md settings
#6 instrumentation INS-1, INS-2, INS-3, INS-4 docs/DATA-MODEL.md sync_log/inference_log, issue #6
#8/#9 Call 1 prefill + 0.7 UIE-2, C1-2, C1-3 docs/AGENTS.md §6, docs/PROMPTS.md §Call 1
#11 WOO_BASE_URL GC-4, UIE-8 docs/AGENTS.md §5, docs/DEPLOYMENT.md §3
#12 synced-species-only C1-1, TAX-2 docs/ARCHITECTURE.md §7, docs/CONTENT-WOO.md §2.2
#14 taxonomy_anchor TAX-1, W-3, W-4, API-7 docs/ARCHITECTURE.md §7, docs/OPENAPI.md §taxonomy
#15 four inline knobs UI-1, UIE-9, UIE-10 docs/TECH-SPEC-PIPELINE.md §3.3, docs/UX.md §Capture
#17 shared validation V-1, V-2, V-3, API-8 docs/OPENAPI.md §validation
#18 409/422 recovery UIE-3, UIE-4, UIE-5, UIE-6 docs/CONTENT-WOO.md §2.5–2.6, docs/OPENAPI.md §slabs

Gates and phase map

Phase / Gate Cases Exit evidence (Evidence class)
Phase 0 / Gate B TV-1 through TV-10, V-1 through V-3, AES-1 through AES-2, H-1, D-1, OPS-1, GC-5 See Gate B exit checklist. TV-3/6/7/10 need product-state or human-walk on real fixtures. Health needs deploy. AES and draft store need api-contract. Proxy-only TV passes do not green the gate. Bots never declare Gate B green.
Gate C (early Phase 1) GC-1 through GC-4, W-1 through W-5, INF-1 through INF-4, UI-1 through UI-5 Ty opens one SLAB-UAT-* Woo draft in real store admin (human-walk or product-state). Inference OFF. Mocked-store is not exit. Bots never declare Gate C green.
Phase 1 remainder API-1 through API-10, TAX-1, TAX-2, TAX-3 Full calibrated to ready cycle via API (api-contract). OPENAPI matches. Revision and taxonomy verified.
Phase 3 UIE-1 through UIE-10 Ty publishes test slab as Woo draft SLAB-UAT-* (human-walk or product-state). UI automation is e2e. Ty is exit.
Phase 5 C1-1 through C2-4, OFF-1, OFF-2 Ty judgment on 3 real slabs (human-walk or product-state). Call 1/2 automation is proxy-synthetic. Inference-disabled path proven per case class.
Phase 6 / production OPS-2 through OPS-5, INS-1 through INS-4, E2E-1 through E2E-5 Deploy smoke (deploy). Ty walks docs/UAT-PRACTICE.md (human-walk). v1 done only after Ty sign-off.

Rules that do not change

  • Inference is optional. The full suite passes with inference disabled. Call 1/Call 2 cases are gated after Gate C.
  • bdft equals square feet times thickness in inches. No divide-by-12. TV-3 proves the identity.
  • Client is authoritative for deterministic math; server stores client-sent numbers.
  • No new product locks in this document. All rules reference existing live docs or issue #19 substance.