From 6f7b12e27d6323630c2899412db8d01ca8efa341 Mon Sep 17 00:00:00 2001 From: Leonard Date: Thu, 9 Jul 2026 14:10:38 +0000 Subject: [PATCH] =?UTF-8?q?Add=20producer=20brief=20006=20=E2=80=94=20Comp?= =?UTF-8?q?uter-Use=20Agents?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- prompts/producer_brief_006.md | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) create mode 100644 prompts/producer_brief_006.md diff --git a/prompts/producer_brief_006.md b/prompts/producer_brief_006.md new file mode 100644 index 0000000..255f5b2 --- /dev/null +++ b/prompts/producer_brief_006.md @@ -0,0 +1,18 @@ +# Producer Brief — Episode 006: Computer-Use Agents (GUI Agents Go to Production) + +## Topic +Agents that operate a real GUI — clicking, scrolling, typing — rather than calling clean APIs. By 2026 this is a production category: Anthropic's Claude controls full desktops, OpenAI's Operator became ChatGPT agent mode, Google's Gemini steers browsers with Project Mariner. The promise: an agent that can use any software a human can. The reality: reliability is still the wall. + +## Why It Matters +Computer use is the bridge between "AI that talks" and "AI that does the boring clicking." If it gets reliable, it eats a huge class of RPA/QA/ops work. But the benchmarks (OSWorld) and the field reports both say the same thing: agents misidentify UI elements in dense or unfamiliar interfaces, latency is brutal (seconds per action vs milliseconds via API), and "low-risk tasks first" is the honest deployment advice. The show should argue the real ceiling, not the demo reel. + +## Seed Question (goes to Leonard first) +"Computer-use agents are supposed to be the big unlock — an AI that can use any software a human can, clicking through a GUI instead of needing a clean API. By 2026 Claude controls full desktops, OpenAI folded Operator into ChatGPT agent mode, Gemini's doing browser steering. But the field reports all say the same thing: they misread dense interfaces and a single click takes seconds. Is this actually ready, or are we shipping demos and calling it production?" + +## Context Notes (background depth — NOT a script) +- Anthropic computer use: agent loop = request tool action → your app runs it → return result. Acknowledged challenge: scrolling/dragging/zooming still hard; advised to start with low-risk tasks. +- OpenAI Operator launched Jan 2025 preview → folded into ChatGPT as "agent mode" by Jul 2025; standalone deprecated Aug 2025. +- Google Gemini / Project Mariner: browser steering. +- OSWorld-Verified benchmarks (Q2 2026 comparisons) exist for Claude vs OpenAI vs Gemini — latency, cost-per-task, reliability profiles. +- Failure modes: misidentifies UI elements in dense/unfamiliar interfaces; API task = milliseconds, computer-use task = seconds/minutes. +- Builder angle: Hermes has computer-use + browser tools — does Leonard/Charlie's "ship daily" habit survive contact with a flaky GUI agent?