Docs: Elevate background-removal quality — tuning knobs must be inline in the primary capture workflow with live re-run #15

New Issue

2026-09-01T23:26:05Z

Ty commented

2026-09-01 23:26:05 +00:00

Summary

The plan of record must make explicit that background removal is the load-bearing phase of the image pipeline: the mask defines the slab's pixel footprint, and every downstream measurement (length, widths, sqft, bdft) is derived from it. A poor mask silently corrupts the numbers with no visual cue, so the user must be able to judge and correct the mask in place.

Why this matters

The mask is deterministic (U2Net + fixed pipeline) — no confidence score, no auto-gate. The user's eye is the only validator.
The four controls (sheet mode, sensitivity, edge offset, feather) exist specifically to let the user dial the mask to the exact slab edge.
Because a bad mask changes the geometry the measurement math runs on, the correction path must be immediate and frictionless.

Required behavior

Inline, not in Settings. The four tuning knobs live in the primary capture workflow — on or adjacent to the mask preview — not behind a navigation to Settings. The user never leaves the capture context to adjust them.
Live re-run. Changing any knob re-performs the background-removal operation on the current photo and updates the preview immediately, so the user sees the effect before committing.
Mask overlay on the source photo. The preview shows the mask overlaid on the original image (not just the cropped result), so the user can see exactly what was included or excluded.
Document the failure mode. The docs must state explicitly: a clipped or over-inclusive mask silently shifts the bounding rectangle and width samples; that is the failure the sliders prevent.

POC scope: client-side only — server fallback deferred, not removed

For the POC, background removal is client-side only: sheet detection, chroma-key or black threshold, and the four sliders all run on the phone. The server-side U2Net "try harder" fallback is deferred from the first release, not cut entirely.

The client-side path with the four sliders covers the common case — green or black sheet, clean mill face — which is the controlled capture environment the POC assumes.
The "try harder" path (phone sends one photo to FastAPI, U2Net runs on demand, mask returns) remains in the architecture as a post-POC addition for awkward backgrounds: mixed lighting, patterned floors, outdoor shots.
Docs must state the fallback is deferred, not absent, so no agent rebuilds or removes it from the spec based on the old server-pipeline language.
A retry-the-whole-photo option covers the failure modes the POC would actually hit without the server round trip.

Doc changes required

TECH-SPEC — state that background removal defines the measurement footprint; the four controls are first-class, inline, live-rerun controls in the capture flow; client-side only for POC; U2Net fallback explicitly deferred.
ARCHITECTURE — swimlane shows the knobs adjacent to the mask preview with a re-run loop; no Settings detour; mark the server fallback path as deferred.
IMPL-PLAN — acceptance criteria: user can adjust any knob and see a fresh mask without leaving the screen; mask overlay is visible on the source photo; no server call required for mask tuning in POC.
AGENTS / PROMPTS — agents must not relocate the controls to Settings, remove the live re-run, or treat the U2Net fallback as deleted rather than deferred.

Acceptance criteria

Docs state background removal is the load-bearing phase and the mask defines the measurement footprint.
The four controls are specified as inline in the primary workflow, not in Settings.
Changing a control re-runs background removal live and updates the preview.
Preview shows the mask overlay on the source photo.
The silent-corruption failure mode is documented.
No doc implies the user must navigate away to tune the mask.
Docs state the U2Net "try harder" fallback is deferred from POC, not removed — retained for post-POC awkward-background cases.
Docs state POC background removal is client-side only; no server round trip for mask tuning.

Labels

docs, blocking-for-poc, ux

## Summary The plan of record must make explicit that **background removal is the load-bearing phase** of the image pipeline: the mask defines the slab's pixel footprint, and every downstream measurement (length, widths, sqft, bdft) is derived from it. A poor mask silently corrupts the numbers with no visual cue, so the user must be able to judge and correct the mask *in place*. ## Why this matters - The mask is deterministic (U2Net + fixed pipeline) — no confidence score, no auto-gate. The user's eye is the only validator. - The four controls (sheet mode, sensitivity, edge offset, feather) exist specifically to let the user dial the mask to the exact slab edge. - Because a bad mask changes the geometry the measurement math runs on, the correction path must be immediate and frictionless. ## Required behavior 1. **Inline, not in Settings.** The four tuning knobs live in the primary capture workflow — on or adjacent to the mask preview — not behind a navigation to Settings. The user never leaves the capture context to adjust them. 2. **Live re-run.** Changing any knob re-performs the background-removal operation on the current photo and updates the preview immediately, so the user sees the effect before committing. 3. **Mask overlay on the source photo.** The preview shows the mask overlaid on the original image (not just the cropped result), so the user can see exactly what was included or excluded. 4. **Document the failure mode.** The docs must state explicitly: a clipped or over-inclusive mask silently shifts the bounding rectangle and width samples; that is the failure the sliders prevent. ## POC scope: client-side only — server fallback deferred, not removed For the POC, background removal is **client-side only**: sheet detection, chroma-key or black threshold, and the four sliders all run on the phone. The server-side U2Net "try harder" fallback is **deferred from the first release**, not cut entirely. - The client-side path with the four sliders covers the common case — green or black sheet, clean mill face — which is the controlled capture environment the POC assumes. - The "try harder" path (phone sends one photo to FastAPI, U2Net runs on demand, mask returns) remains in the architecture as a post-POC addition for awkward backgrounds: mixed lighting, patterned floors, outdoor shots. - Docs must state the fallback is **deferred**, not absent, so no agent rebuilds or removes it from the spec based on the old server-pipeline language. - A retry-the-whole-photo option covers the failure modes the POC would actually hit without the server round trip. ## Doc changes required - **TECH-SPEC** — state that background removal defines the measurement footprint; the four controls are first-class, inline, live-rerun controls in the capture flow; client-side only for POC; U2Net fallback explicitly deferred. - **ARCHITECTURE** — swimlane shows the knobs adjacent to the mask preview with a re-run loop; no Settings detour; mark the server fallback path as deferred. - **IMPL-PLAN** — acceptance criteria: user can adjust any knob and see a fresh mask without leaving the screen; mask overlay is visible on the source photo; no server call required for mask tuning in POC. - **AGENTS / PROMPTS** — agents must not relocate the controls to Settings, remove the live re-run, or treat the U2Net fallback as deleted rather than deferred. ## Acceptance criteria - [ ] Docs state background removal is the load-bearing phase and the mask defines the measurement footprint. - [ ] The four controls are specified as inline in the primary workflow, not in Settings. - [ ] Changing a control re-runs background removal live and updates the preview. - [ ] Preview shows the mask overlay on the source photo. - [ ] The silent-corruption failure mode is documented. - [ ] No doc implies the user must navigate away to tune the mask. - [ ] Docs state the U2Net "try harder" fallback is deferred from POC, not removed — retained for post-POC awkward-background cases. - [ ] Docs state POC background removal is client-side only; no server round trip for mask tuning. ## Labels docs, blocking-for-poc, ux

Ty added this to the POC milestone 2026-09-01 23:49:09 +00:00

grokbot_ty referenced this issue from a commit

2026-09-02 00:33:25 +00:00

docs(#15): mask is load-bearing; inline knobs, live re-run, U2Net deferred

Ty referenced this issue

2026-09-03 02:36:57 +00:00

Test plan: POC acceptance test cases for all milestone issues #19

grokbot_ty commented

2026-09-24 16:24:21 +00:00

Closed: resolved along the way to M3.

grokbot_ty closed this issue

2026-09-24 16:24:22 +00:00