Requirement: Inference API must support portable multi-turn context for chained Call 1 → Call 2 #1

New Issue

2026-09-01T19:36:30Z

Ty commented

2026-09-01 19:36:30 +00:00

Summary

Capture a project requirement around how the two inference calls (vision taxonomy + descriptive text) share context, so the design stays portable across API providers.

Background

Current design makes two separate calls to the inference engine:

Call 1 (vision): sends all inventory photos (downscaled to 1024px) + taxonomy snapshot → structured species/attribute guesses.
Call 2 (text): generates title/description from Call 1's structured results + deterministic measurements + brand/geo.

We are exploring enriching Call 2 by also giving it the vision images, with explicit labels marking which dimensions/species/attributes are confirmed vs inferred. This would let Call 2 ground its prose in the actual grain, figure, spalting, and color rather than only labels.

The portability issue

If Call 2 continues the same thread as Call 1 (retaining the images and Call 1's reasoning in the context window), behavior depends on the provider:

Stateful providers (OpenAI Responses API, xAI Grok, Meta): support previous_response_id — server retains the full thread incl. images; client sends only the new turn.
Stateless providers (Anthropic Messages API, generic OpenAI-compatible gateways): no equivalent ID; every call must resend the entire history.

The proxy is specified as "OpenAI-compatible," but that usually means the chat-completions shape, not Responses-style statefulness. Swapping to Claude or a generic gateway would silently break chaining.

Proposed requirement

The inference client must default to resending the full conversation history (images + Call 1 turns + confirmed/inferred labels) on Call 2. This is the portable path.
previous_response_id (or equivalent) may be used only as an optional optimization, and only when the configured provider is known to support it.
Provider capability for stateful continuation should be detected or configured explicitly — never assumed from "OpenAI-compatible."
Document the chosen behavior in the tech spec / architecture, and add a test or note that the suite still passes when the provider is forced into the stateless/resend path.

Acceptance criteria

Tech spec states the default-resend rule and the optional-optimization exception.
Architecture notes which providers are stateful vs stateless for this use.
No code path assumes previous_response_id is universally available.
End-to-end suite passes with inference off and with a stateless provider configuration.

Notes

Token cost of resending downscaled images twice is modest but should be acknowledged.
This is independent of whether Call 2 ultimately receives raw images or richer structured visual observations from Call 1; the context-portability rule applies either way.

## Summary Capture a project requirement around how the two inference calls (vision taxonomy + descriptive text) share context, so the design stays portable across API providers. ## Background Current design makes two separate calls to the inference engine: - **Call 1 (vision):** sends all inventory photos (downscaled to 1024px) + taxonomy snapshot → structured species/attribute guesses. - **Call 2 (text):** generates title/description from Call 1's structured results + deterministic measurements + brand/geo. We are exploring enriching Call 2 by also giving it the vision images, with explicit labels marking which dimensions/species/attributes are **confirmed** vs **inferred**. This would let Call 2 ground its prose in the actual grain, figure, spalting, and color rather than only labels. ## The portability issue If Call 2 continues the same thread as Call 1 (retaining the images and Call 1's reasoning in the context window), behavior depends on the provider: - **Stateful providers** (OpenAI Responses API, xAI Grok, Meta): support `previous_response_id` — server retains the full thread incl. images; client sends only the new turn. - **Stateless providers** (Anthropic Messages API, generic OpenAI-compatible gateways): no equivalent ID; every call must resend the entire history. The proxy is specified as "OpenAI-compatible," but that usually means the chat-completions *shape*, not Responses-style statefulness. Swapping to Claude or a generic gateway would silently break chaining. ## Proposed requirement 1. The inference client **must default to resending the full conversation history** (images + Call 1 turns + confirmed/inferred labels) on Call 2. This is the portable path. 2. `previous_response_id` (or equivalent) may be used **only as an optional optimization**, and only when the configured provider is known to support it. 3. Provider capability for stateful continuation should be detected or configured explicitly — never assumed from "OpenAI-compatible." 4. Document the chosen behavior in the tech spec / architecture, and add a test or note that the suite still passes when the provider is forced into the stateless/resend path. ## Acceptance criteria - [ ] Tech spec states the default-resend rule and the optional-optimization exception. - [ ] Architecture notes which providers are stateful vs stateless for this use. - [ ] No code path assumes `previous_response_id` is universally available. - [ ] End-to-end suite passes with inference off *and* with a stateless provider configuration. ## Notes - Token cost of resending downscaled images twice is modest but should be acknowledged. - This is independent of whether Call 2 ultimately receives raw images or richer structured visual observations from Call 1; the context-portability rule applies either way.

Ty closed this issue

2026-09-01 19:38:11 +00:00

grokbot_ty referenced this issue from a commit

2026-09-16 21:07:16 +00:00

docs(m3): lock Woo image order including dimension overlay

grokbot_ty referenced this issue

2026-09-21 05:04:28 +00:00

Prompt changes for call #1 #86

grokbot_ty referenced this issue from a commit

2026-09-21 08:48:45 +00:00

fix(review): #90 photo strip — blob: URL for Primary photo, checkerboard thumbs

Sign in to join this conversation.