Requirement: Inference API must support portable multi-turn context for chained Call 1 to Call 2 #2

New Issue

2026-09-01T19:37:12Z

Ty commented

2026-09-01 19:37:12 +00:00

Summary

The current design makes two separate inference calls: Call 1 (vision) for taxonomy/species/attributes, and Call 2 (text) for title/description prose. We want to explore chaining them so Call 2 can retain Call 1's context window (including the downscaled images) instead of re-uploading, while explicitly labeling which dimensions, species, and attributes are confirmed vs. inferred.

Hard requirement

Any inference engine selected for this project MUST support portable multi-turn context for chaining Call 1 → Call 2. This is non-negotiable: the chosen provider must be able to continue the conversation thread (images + Call 1 reasoning) so Call 2 can ground its prose in the actual wood without re-uploading, or degrade cleanly to full resend.

Test-connection workflow must verify it

The existing test-inference / test-connection endpoint MUST be extended to probe this capability explicitly:

Send a multi-turn probe: Call 1 with a sample image + taxonomy, then Call 2 continuing the same thread with a confirmed/inferred label turn.
Assert the second call succeeds with the prior context intact (stateful path) or that the client can resend the full history and still succeed (stateless path).
Fail the connection test (and block Call 2 enablement) if the provider cannot support either path.
Capability detection must distinguish stateful (previous_response_id-style) from stateless (full resend required) and record which mode the selected engine uses.

Default behavior

Default: resend the full conversation history (images + Call 1 reasoning + new confirmed/inferred labels) on Call 2. Portable across stateless providers.
Optional optimization: use previous_response_id (or equivalent) only when the provider is detected as stateful.
Never assume server-side state is universal. The design must degrade gracefully to full resend.

Provider notes

OpenAI Responses API, xAI Grok, and Meta support previous_response_id-style continuation.
Anthropic Messages API is stateless — every call must resend the full history. No equivalent ID.
"OpenAI-compatible" proxies often implement only the chat-completions shape, not Responses statefulness. Swapping providers must not silently break chaining.

Acceptance criteria

Client defaults to resending full history; previous_response_id is an opt-in optimization gated on provider capability detection.
No code path assumes server-side continuation is available.
Call 2 prompt includes explicit confirmed-vs-inferred labels for species, dimensions, and other attributes.
Test-connection / test-inference workflow includes a multi-turn context probe and fails closed if the selected engine cannot support portable chaining.
Documented in the tech spec / architecture under the inference section, including the hard requirement and the probe behavior.
Tested against at least one stateless provider (e.g., Claude or a generic OpenAI-compatible gateway) to confirm resend path works.

Context retention and cleanup

Each slab gets its own conversation/response ID; threads are isolated per slab, so a leaked thread cannot contaminate another slab's context.
The client MUST delete the response ID (or equivalent thread) once the slab is submitted successfully, abandoned mid-flow, or fails to submit. Delete is best-effort: log and continue on failure — a leaked thread is a privacy/retention annoyance, not a functional break.
The provider's default TTL (typically 30 days on hosted APIs) is the backstop; client-side deletion shortens the window but is not the sole safety net. Self-hosted engines (e.g. vLLM) may use in-memory or backend-backed stores with their own eviction; document the chosen behavior.

Caching and price optimization (explore, not blocking for POC)

Providers that discount cached input tokens (OpenAI, Anthropic, etc.) can reduce Call 2 cost because the resend is text-only: Call 1's reasoning, taxonomy, and measurements. Images are a one-time cost in Call 1 and are not re-uploaded.
To maximize cache hits, structure Call 2's prompt so the stable prefix — system instructions, taxonomy snapshot, and any repeated context — is byte-for-byte identical across slabs. Only the per-slab labels and measurements change.
Deterministic image processing is required if images ever participate in a cached prefix (fixed resize to 1024px, fixed quality, fixed ordering); any variation breaks the cache hit.
At POC volume this is likely noise; revisit at production scale (hundreds of slabs/week) where the discount may be meaningful. Track as a follow-up optimization, not a hard gate.

Context

Discussed in conversation 2026-09-01. Related to option 2 of the Call 1/Call 2 chaining exploration (two calls, images provided to Call 2, confirmed/inferred framing). Elevated to a hard requirement: engine selection is gated on this capability.

## Summary The current design makes two separate inference calls: Call 1 (vision) for taxonomy/species/attributes, and Call 2 (text) for title/description prose. We want to explore chaining them so Call 2 can retain Call 1's context window (including the downscaled images) instead of re-uploading, while explicitly labeling which dimensions, species, and attributes are confirmed vs. inferred. ## Hard requirement **Any inference engine selected for this project MUST support portable multi-turn context** for chaining Call 1 → Call 2. This is non-negotiable: the chosen provider must be able to continue the conversation thread (images + Call 1 reasoning) so Call 2 can ground its prose in the actual wood without re-uploading, or degrade cleanly to full resend. ## Test-connection workflow must verify it The existing test-inference / test-connection endpoint MUST be extended to probe this capability explicitly: - Send a multi-turn probe: Call 1 with a sample image + taxonomy, then Call 2 continuing the same thread with a confirmed/inferred label turn. - Assert the second call succeeds with the prior context intact (stateful path) **or** that the client can resend the full history and still succeed (stateless path). - Fail the connection test (and block Call 2 enablement) if the provider cannot support either path. - Capability detection must distinguish stateful (`previous_response_id`-style) from stateless (full resend required) and record which mode the selected engine uses. ## Default behavior - Default: resend the full conversation history (images + Call 1 reasoning + new confirmed/inferred labels) on Call 2. Portable across stateless providers. - Optional optimization: use `previous_response_id` (or equivalent) only when the provider is detected as stateful. - Never assume server-side state is universal. The design must degrade gracefully to full resend. ## Provider notes - OpenAI Responses API, xAI Grok, and Meta support `previous_response_id`-style continuation. - Anthropic Messages API is stateless — every call must resend the full history. No equivalent ID. - "OpenAI-compatible" proxies often implement only the chat-completions shape, not Responses statefulness. Swapping providers must not silently break chaining. ## Acceptance criteria - [ ] Client defaults to resending full history; `previous_response_id` is an opt-in optimization gated on provider capability detection. - [ ] No code path assumes server-side continuation is available. - [ ] Call 2 prompt includes explicit confirmed-vs-inferred labels for species, dimensions, and other attributes. - [ ] Test-connection / test-inference workflow includes a multi-turn context probe and fails closed if the selected engine cannot support portable chaining. - [ ] Documented in the tech spec / architecture under the inference section, including the hard requirement and the probe behavior. - [ ] Tested against at least one stateless provider (e.g., Claude or a generic OpenAI-compatible gateway) to confirm resend path works. ## Context retention and cleanup - Each slab gets its own conversation/response ID; threads are isolated per slab, so a leaked thread cannot contaminate another slab's context. - The client MUST delete the response ID (or equivalent thread) once the slab is submitted successfully, abandoned mid-flow, or fails to submit. Delete is best-effort: log and continue on failure — a leaked thread is a privacy/retention annoyance, not a functional break. - The provider's default TTL (typically 30 days on hosted APIs) is the backstop; client-side deletion shortens the window but is not the sole safety net. Self-hosted engines (e.g. vLLM) may use in-memory or backend-backed stores with their own eviction; document the chosen behavior. ## Caching and price optimization (explore, not blocking for POC) - Providers that discount cached input tokens (OpenAI, Anthropic, etc.) can reduce Call 2 cost because the resend is text-only: Call 1's reasoning, taxonomy, and measurements. Images are a one-time cost in Call 1 and are not re-uploaded. - To maximize cache hits, structure Call 2's prompt so the stable prefix — system instructions, taxonomy snapshot, and any repeated context — is byte-for-byte identical across slabs. Only the per-slab labels and measurements change. - Deterministic image processing is required if images ever participate in a cached prefix (fixed resize to 1024px, fixed quality, fixed ordering); any variation breaks the cache hit. - At POC volume this is likely noise; revisit at production scale (hundreds of slabs/week) where the discount may be meaningful. Track as a follow-up optimization, not a hard gate. ## Context Discussed in conversation 2026-09-01. Related to option 2 of the Call 1/Call 2 chaining exploration (two calls, images provided to Call 2, confirmed/inferred framing). Elevated to a hard requirement: engine selection is gated on this capability.

Ty added reference docs/refactor-v2

2026-09-01 19:38:11 +00:00

Ty added this to the POC milestone 2026-09-01 23:49:51 +00:00

grokbot_ty referenced this issue from a commit

2026-09-02 00:34:50 +00:00

docs: pair locks for #14 #2 #17 #18 (PPM product OK)

Ty commented

2026-09-03 02:35:03 +00:00

Test cases needed once implementation exists:

Full resend default — Call 1 returns taxonomy results; Call 2 is invoked; verify the request body includes the full Call 1 context (prompt, assistant JSON, images metadata). No previous_response_id in the request.
Confirmed/inferred labels in Call 2 — Call 2 user prompt template includes confirmed on dimensions and confirmed/inferred on taxonomy values. Labels match the actual provenance of each value.
Stateful opt-in — test-inference probe returns stateful; Call 2 uses previous_response_id. Confirm the same conversation thread carries context.
Stateless fallback — test-inference probe returns stateless; Call 2 uses full resend. Confirm the same results as stateful path.
Fail closed — test-inference probe returns fail; Call 2 endpoint returns 422. Call 1 and inference-OFF still work.
Context cleanup — after publish, abandon, or fail, the server-side thread ID is deleted. A leaked thread is logged but not a functional break.
Context isolation — two slabs with concurrent Call 1→Call 2 chains do not contaminate each other's context.

These tests are for the implementation PR, not the docs pass. No code exists yet.

Test cases needed once implementation exists: 1. **Full resend default** — Call 1 returns taxonomy results; Call 2 is invoked; verify the request body includes the full Call 1 context (prompt, assistant JSON, images metadata). No `previous_response_id` in the request. 2. **Confirmed/inferred labels in Call 2** — Call 2 user prompt template includes `confirmed` on dimensions and `confirmed`/`inferred` on taxonomy values. Labels match the actual provenance of each value. 3. **Stateful opt-in** — test-inference probe returns `stateful`; Call 2 uses `previous_response_id`. Confirm the same conversation thread carries context. 4. **Stateless fallback** — test-inference probe returns `stateless`; Call 2 uses full resend. Confirm the same results as stateful path. 5. **Fail closed** — test-inference probe returns `fail`; Call 2 endpoint returns 422. Call 1 and inference-OFF still work. 6. **Context cleanup** — after publish, abandon, or fail, the server-side thread ID is deleted. A leaked thread is logged but not a functional break. 7. **Context isolation** — two slabs with concurrent Call 1→Call 2 chains do not contaminate each other's context. These tests are for the implementation PR, not the docs pass. No code exists yet.

Ty referenced this issue

2026-09-03 02:36:57 +00:00

Test plan: POC acceptance test cases for all milestone issues #19

grokbot_ty referenced this issue from a commit

2026-09-16 21:07:16 +00:00

docs(m3): lock Woo image order including dimension overlay

grokbot_ty referenced this issue from a commit

2026-09-21 08:48:45 +00:00

fix(review): #90 photo strip — blob: URL for Primary photo, checkerboard thumbs

grokbot_ty commented

2026-09-24 18:54:55 +00:00

Superseded: design went stateless-only. Full resend every call, no previous_response_id. test-inference returns "mode": "stateless" (inference.py:646).

grokbot_ty closed this issue

2026-09-24 18:54:55 +00:00

Sign in to join this conversation.

2 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: Ty_Tech/SlabUploader#2