@womp/spark-agent 6.1.1 → 6.3.0

package/CHANGELOG.md

@@ -4,6 +4,59 @@ Notable changes to `@womp/spark-agent`, newest first. Versions follow
 [Semantic Versioning](https://semver.org/). Add an entry with each version bump; describe what
 changed for hosts and users, and how to migrate across breaking changes.
 
+## 6.3.0 — 2026-10-09
+
+### Added
+
+- Goals start from the conditions in an ordinary message. `SparkAgentService.proposeGoal`,
+  `POST /conversations/:id/goal/proposal` and `SparkChatTransport.proposeGoal` return a title and
+  the message's success conditions, each marked `measurable` or left to the user's review. Hosts
+  supply `SparkAgentServiceOptions.proposeGoal`; `createSparkGoalProposer` is the shared one.
+- The goal `set` command accepts `title` and `conditions`. The service attaches them to the
+  initiating message as a `data-goal` part, the evaluator judges each measurable condition after
+  every try, and `SparkGoal.conditions` records `met` and `reason`. A goal with conditions completes
+  only when no measurable condition is unmet. `SparkChatTransport.sendMessages` forwards
+  `body.goal`, so a confirmed goal is sent with its message.
+- `SparkGoal.tries` keeps each check a goal with conditions has had: the try's number, the run that
+  made it, the evaluator's reason and every condition's result, so a host can show a goal's history.
+- The goal `update_budget` command accepts `maxTurns` as well as `creditBudget`, either or both, so
+  a goal that used its tries can be given more and resumed.
+- `GOAL_CONFIRMATION_DEFAULTS` (5 tries, 2,000 credits) and `MAX_GOAL_LENGTH` from
+  `@womp/spark-agent/ui`, for the confirmation a host shows and the objective it sends.
+- The standalone UI reads each request's conditions, asks the user to confirm them with a tries and
+  a credit cap or to try once, and shows the goal's progress. It estimates credits from model cost
+  at a development rate; it does not bill.
+
+### Changed
+
+- A goal's initiating user message can carry a `data-goal` part. Hosts that render or mirror user
+  message parts should show it as the confirmed goal or skip it. A host that gives the model its
+  own copy of the message, as Crow does from the Swan thread, must keep the part on that copy:
+  it is how the worker reads the conditions.
+
+### Removed
+
+- `parseSparkGoalCommand`. A goal starts from the confirmation, not from a typed `/goal`; hosts
+  send the `set`, `pause`, `resume` and `clear` commands from their own controls.
+
+## 6.2.0 — 2026-10-09
+
+### Added
+
+- `intent-bench` CLI: measures which route Spark takes (scene, OpenSCAD, Geometry Nodes, image,
+  image edit, mesh, image-to-mesh, Flow, import, or a reply) for the build-intent QA requests
+  under the current creation rules, preview follow-ups and attached-photo cases, in parallel over
+  fixtures, with gpt-6-luna by default. See the README.
+
+### Changed
+
+- An attached image without a request gets a question about what to make of it, not a generation.
+  Turning an attached image into 3D generates the mesh from it directly, with no new preview.
+- Characters, creatures, people, food, sculptures, specific vehicles and branded items preview
+  first even when simple or stylized, unless the request names a method or has a source image.
+- `flow-graphs` (v5) builds the requested graph with the catalog's nodes when an input, such as a
+  conversation upload, cannot be expressed in source, and tells the user which input to set.
+
 ## 6.1.1 — 2026-10-08
 
 ### Fixed

package/README.md

@@ -138,6 +138,7 @@ Closing or reloading the page only detaches the SSE subscriber. It does not canc
 POST  /agent/conversations/:conversationId/messages
 PATCH /agent/conversations/:conversationId/settings
 POST  /agent/conversations/:conversationId/goal
+POST  /agent/conversations/:conversationId/goal/proposal
 GET   /agent/runs/:runId/stream?after=0
 POST  /agent/runs/:runId/cancel
 ```
@@ -146,6 +147,15 @@ Conversation settings contain `modelId` and `reasoningEffort`; save them with
 `transport.saveConversationSettings(id, settings)`. The goal endpoint accepts `set`, `pause`,
 `resume`, `clear`, and `update_budget` commands.
 
+A goal starts from an ordinary message. `transport.proposeGoal(id, message, settings)` returns a
+title and the success conditions read from the message, each marked `measurable` or left to the
+user's review; it stores and starts nothing. After the user confirms them with a tries cap
+(`maxTurns`) and a credit cap, send the message with `body.goal` set to a `set` command carrying
+`title`, `conditions`, `maxTurns` and `creditBudget`. The service attaches the confirmed conditions
+to that message as a `data-goal` part, evaluates each measurable condition after every try, and
+records the result on `goal.conditions`. Send the message without `body.goal` to try once. Hosts
+enable proposals with `SparkAgentServiceOptions.proposeGoal`, such as `createSparkGoalProposer`.
+
 The SSE response contains ordered persistence envelopes around standard AI SDK chunks:
 
 ```ts
@@ -262,6 +272,26 @@ Progress goes to stderr; one JSON result goes to stdout. Use `yarn --silent agen
 stdout. Exit codes: 0 completed, 1 run/transport/export failure, 2 invalid arguments, 3 requires
 user action, 130 interrupted.
 
+## Measure intent routing
+
+`yarn intent-bench` checks which route Spark takes for the requests in
+`src/intent-bench-cases.ts`: editable scene, OpenSCAD, Geometry Nodes, image, image edit, mesh,
+image-to-mesh, Flow, import, or a reply without acting. It needs no server or renderer. Each run
+gets Swan's skills over fixtures: an empty scene, the case's attachments and earlier turns, and
+Flow open or closed. Photo cases download their image from Womp's media host. The first call that would change the scene, generate, write a graph or
+import is recorded, and the run stops before it takes effect. Material edits do not decide a
+route, since Geometry Nodes prepares its materials in the scene first.
+
+```sh
+yarn intent-bench                                  # every case 3 times on gpt-6-luna, 24 at once
+yarn intent-bench --only photo-only,F1 --repeat 10 --model gpt-6-astra
+yarn intent-bench --list
+```
+
+It prints each case's routes and pass rate and writes `report.json`, with every run's skills,
+calls, deciding call and reply, under `.spark/intent-bench/`. It exits 1 when any run took a route
+the case does not expect. Model calls are paid; the full set costs about $0.10 on gpt-6-luna.
+
 ## Run cost estimates
 
 The chat displays estimated API token cost in USD after every completed model step, from the

package/dist/capabilities/flow.js

@@ -11,12 +11,13 @@ export function createFlowCapability(host, catalog) {
         skills: [
             {
                 name: "flow-graphs",
-                version: "4",
+                version: "5",
                 description: "Read and edit the open Workbench Flow pipeline: image/mesh generation, processing, and viewers.",
                 instructions: `Use womp.flow for the open Workbench pipeline.
 Read the current graph and flow-graphs-reference before writing. Supply the complete edited s-expression document and the latest revision. Preserve unrelated nodes and stable node IDs. Omitting a node deletes it; empty source clears the graph.
 Existing node configuration, edge identity, layout, runtime metadata, and generated outputs are retained for unchanged node identities/types. Scalar inputs represented in source are replaced by your document. The host retains preservedInputs for those same nodes. Non-private preserved inputs can be explicitly replaced in the document; never recreate asset objects or cached values as JSON strings. Private settings remain editor-owned: preserve existing private connections unchanged between retained nodes and do not introduce new private literals or connections.
 If read returns source null, explain the diagnostics. Do not replace an unreadable graph as though it were empty.
+Build the requested graph from the catalog's nodes even when an input cannot be expressed in source, such as a conversation upload or an editor-owned image: leave that input unconnected and tell the user which node input to set in the editor.
 The Flow canvas must be open. If unavailable, tell the user to open Flow or Split in a project, or Nodes in the sidebar, and wait for their next message. Never claim a graph was created after a failed call.
 Writes validate types, preserve the runner lifecycle, and await commit. They edit the graph; they do not report that generation jobs completed. Repair compiler errors in this same conversation with a fresh read. On committed/unknown receipts after failure, inspect the graph before retrying; do not replay blindly.`,
                 declarations: WOMP_FLOW_DECLARATIONS,

package/dist/client.d.ts

@@ -1,7 +1,7 @@
 import type { ChatRequestOptions, ChatTransport, UIMessageChunk } from "ai";
 import type { SparkRunSettings } from "./server/types.js";
-import type { SparkGoal, SparkGoalCommand } from "./goals.js";
-export type { SparkGoal, SparkGoalCommand } from "./goals.js";
+import type { SparkGoal, SparkGoalCommand, SparkGoalProposal } from "./goals.js";
+export type { SparkGoal, SparkGoalCommand, SparkGoalCondition, SparkGoalProposal, SparkGoalTry } from "./goals.js";
 import type { SparkConversation, SparkConversationSummary, SparkRun } from "./server/types.js";
 import type { SparkUIMessage } from "./ui.js";
 export type { SparkConversation, SparkConversationSummary, SparkRun, SparkRunSettings, } from "./server/types.js";
@@ -44,6 +44,8 @@ export declare class SparkChatTransport implements ChatTransport<SparkUIMessage>
         goal?: SparkGoal;
         run?: SparkRun;
     }>;
+    /** The success conditions read from a message, to confirm before sending it as a goal. */
+    proposeGoal(id: string, message: SparkUIMessage, settings: SparkRunSettings, signal?: AbortSignal): Promise<SparkGoalProposal>;
     getGoal(id: string): Promise<SparkGoal | undefined>;
     getRun(id: string): Promise<SparkRun>;
     saveConversationSettings(id: string, settings: SparkRunSettings): Promise<void>;

package/dist/client.js

@@ -51,7 +51,8 @@ export class SparkChatTransport {
             : options.messages.find(({ id }) => id === options.messageId) ?? options.messages.at(-1);
         if (message === undefined)
             throw new Error("Spark Agent requires a message to send.");
-        const settings = this.settingsFrom(options.body);
+        const requestBody = options.body;
+        const settings = this.settingsFrom(requestBody);
         const response = await this.request(`/conversations/${encodeURIComponent(options.chatId)}/messages`, {
             method: "POST",
             body: JSON.stringify({
@@ -59,6 +60,8 @@ export class SparkChatTransport {
                 message,
                 messageId: options.messageId,
                 settings,
+                // A confirmed goal: the message starts it, and the run is its first try.
+                ...(requestBody?.goal === undefined ? {} : { goal: requestBody.goal }),
             }),
             signal: options.abortSignal,
         }, options.headers);
@@ -126,6 +129,15 @@ export class SparkChatTransport {
             throw await responseError(response);
         return response.json();
     }
+    /** The success conditions read from a message, to confirm before sending it as a goal. */
+    async proposeGoal(id, message, settings, signal) {
+        const response = await this.request(`/conversations/${encodeURIComponent(id)}/goal/proposal`, {
+            method: "POST", body: JSON.stringify({ message, settings }), signal,
+        });
+        if (!response.ok)
+            throw await responseError(response);
+        return (await response.json()).proposal;
+    }
     async getGoal(id) {
         const response = await this.request(`/conversations/${encodeURIComponent(id)}/goal`, { method: "GET" });
         if (!response.ok)

package/dist/goals.d.ts

@@ -1,9 +1,48 @@
 import { z } from "zod";
 import type { SparkRunSettings } from "./server/types.js";
 export declare const MAX_GOAL_LENGTH = 4000;
+export declare const MAX_GOAL_CONDITIONS = 12;
+export declare const MAX_GOAL_CONDITION_LENGTH = 200;
+/** One success condition of a goal, taken from what the user asked for. */
+export declare const sparkGoalConditionSchema: z.ZodObject<{
+    text: z.ZodString;
+    measurable: z.ZodBoolean;
+}, z.core.$strip>;
+/** What a goal's evaluation last found for a condition; unset until one has run. */
+export type SparkGoalCondition = z.infer<typeof sparkGoalConditionSchema> & {
+    met?: boolean;
+    reason?: string;
+};
+/** What the check after one try found: its number, the run that made it, and each condition's result. */
+export type SparkGoalTry = {
+    /** One-based, as the goal counts its tries. */
+    turn: number;
+    runId: string;
+    reason: string;
+    /** In the order of the goal's conditions; a condition left to the user has no result. */
+    conditions: Pick<SparkGoalCondition, "met" | "reason">[];
+};
+/** The conditions read from a user message, for the user to confirm before a goal starts. */
+export declare const sparkGoalProposalSchema: z.ZodObject<{
+    title: z.ZodString;
+    conditions: z.ZodArray<z.ZodObject<{
+        text: z.ZodString;
+        measurable: z.ZodBoolean;
+    }, z.core.$strip>>;
+}, z.core.$strip>;
+export type SparkGoalProposal = z.infer<typeof sparkGoalProposalSchema>;
+/** The caps a goal confirmation offers before the user changes them. */
+export declare const GOAL_CONFIRMATION_DEFAULTS: {
+    readonly tries: 5;
+    readonly credits: 2000;
+};
 export type SparkGoal = {
     id: string;
     objective: string;
+    title?: string;
+    conditions?: SparkGoalCondition[];
+    /** The checks made so far, oldest first. A try that ended before its check has none. */
+    tries?: SparkGoalTry[];
     status: "active" | "paused" | "completed" | "blocked" | "budget_limited";
     settings: SparkRunSettings;
     createdAt: string;
@@ -24,12 +63,18 @@ export type SparkGoal = {
 export declare const sparkGoalCommandSchema: z.ZodDiscriminatedUnion<[z.ZodObject<{
     action: z.ZodLiteral<"set">;
     objective: z.ZodString;
+    title: z.ZodOptional<z.ZodString>;
+    conditions: z.ZodOptional<z.ZodArray<z.ZodObject<{
+        text: z.ZodString;
+        measurable: z.ZodBoolean;
+    }, z.core.$strip>>>;
     creditBudget: z.ZodOptional<z.ZodNumber>;
     maxTurns: z.ZodOptional<z.ZodNumber>;
 }, z.core.$strip>, z.ZodObject<{
     action: z.ZodLiteral<"update_budget">;
     goalId: z.ZodString;
-    creditBudget: z.ZodNumber;
+    creditBudget: z.ZodOptional<z.ZodNumber>;
+    maxTurns: z.ZodOptional<z.ZodNumber>;
 }, z.core.$strip>, z.ZodObject<{
     action: z.ZodEnum<{
         status: "status";
@@ -40,5 +85,3 @@ export declare const sparkGoalCommandSchema: z.ZodDiscriminatedUnion<[z.ZodObjec
 }, z.core.$strip>], "action">;
 export type SparkGoalCommand = z.infer<typeof sparkGoalCommandSchema>;
 export declare const DEFAULT_GOAL_CREDITS = 50000;
-/** Only the explicit slash command starts a goal; prose inside an ordinary message does not. */
-export declare function parseSparkGoalCommand(text: string): SparkGoalCommand | undefined;

package/dist/goals.js

@@ -1,34 +1,39 @@
 import { z } from "zod";
 export const MAX_GOAL_LENGTH = 4_000;
+export const MAX_GOAL_CONDITIONS = 12;
+export const MAX_GOAL_CONDITION_LENGTH = 200;
+/** One success condition of a goal, taken from what the user asked for. */
+export const sparkGoalConditionSchema = z.object({
+    text: z.string().trim().min(1).max(MAX_GOAL_CONDITION_LENGTH),
+    /**
+     * Whether the result can be checked objectively from tool results. The goal is
+     * met when every measurable condition is; the rest are left to the user's review.
+     */
+    measurable: z.boolean(),
+});
+/** The conditions read from a user message, for the user to confirm before a goal starts. */
+export const sparkGoalProposalSchema = z.object({
+    title: z.string().trim().min(1).max(80),
+    conditions: z.array(sparkGoalConditionSchema).max(MAX_GOAL_CONDITIONS),
+});
+/** The caps a goal confirmation offers before the user changes them. */
+export const GOAL_CONFIRMATION_DEFAULTS = { tries: 5, credits: 2_000 };
 export const sparkGoalCommandSchema = z.discriminatedUnion("action", [
     z.object({
         action: z.literal("set"),
         objective: z.string().trim().min(1).max(MAX_GOAL_LENGTH),
+        title: sparkGoalProposalSchema.shape.title.optional(),
+        conditions: sparkGoalProposalSchema.shape.conditions.min(1).optional(),
         creditBudget: z.number().int().positive().max(1_000_000_000).optional(),
         maxTurns: z.number().int().min(1).max(100).optional(),
     }),
+    // Raises or lowers a goal's caps; at least one of them.
     z.object({
         action: z.literal("update_budget"),
         goalId: z.string().min(1),
-        creditBudget: z.number().int().positive().max(1_000_000_000),
+        creditBudget: z.number().int().positive().max(1_000_000_000).optional(),
+        maxTurns: z.number().int().min(1).max(100).optional(),
     }),
     z.object({ action: z.enum(["status", "pause", "resume", "clear"]) }),
 ]);
 export const DEFAULT_GOAL_CREDITS = 50_000;
-/** Only the explicit slash command starts a goal; prose inside an ordinary message does not. */
-export function parseSparkGoalCommand(text) {
-    const match = /^\/goal(?:\s+([\s\S]*))?$/i.exec(text.trim());
-    if (!match)
-        return undefined;
-    let objective = (match[1] ?? "").trim();
-    if (["status", "pause", "resume", "clear"].includes(objective.toLowerCase()))
-        return { action: objective.toLowerCase() };
-    objective = objective.replace(/^["Ҭ]([\s\S]*)["Ӭ]$/, "$1").trim();
-    const cap = /(?:[,;]?[\s]+)?cap\s+at\s+(\S+)\s+credits[.!]?$/i.exec(objective);
-    if (cap && !/^(?:\d+|\d{1,3}(?:,\d{3})+|\d{1,3}(?:\.\d{3})+)$/.test(cap[1]))
-        throw new Error("Use a whole number of credits, for example cap at 1000 credits.");
-    const creditBudget = cap ? Number(cap[1].replace(/[.,]/g, "")) : DEFAULT_GOAL_CREDITS;
-    if (cap)
-        objective = objective.slice(0, cap.index).trim().replace(/^["Ҭ]([\s\S]*)["Ӭ]$/, "$1").trim();
-    return sparkGoalCommandSchema.parse({ action: "set", objective, creditBudget });
-}

package/dist/intent-bench-cases.d.ts

@@ -0,0 +1,42 @@
+import type { OpenScadTarget } from "./capabilities.js";
+/** The first action a run commits to, or how it ended without one. */
+export declare const INTENT_ROUTES: readonly ["scene", "openscad", "geometry-nodes", "image", "image-edit", "mesh", "image-to-mesh", "flow", "import", "reply", "undecided", "error"];
+export type IntentRoute = (typeof INTENT_ROUTES)[number];
+/** A photo downloaded from `url`, or a generated solid color. */
+export type IntentImage = {
+    url: string;
+} | {
+    name: string;
+    color: [number, number, number];
+};
+export type IntentTurn = {
+    role: "user";
+    text?: string;
+    images?: IntentImage[];
+}
+/** Earlier assistant text; `generated` images are listed as that turn's generation outputs. */
+ | {
+    role: "assistant";
+    text: string;
+    generated?: IntentImage[];
+};
+export type IntentCase = {
+    id: string;
+    group: "core" | "workspace" | "conversation" | "photo";
+    /** Earlier turns; the last user turn is the request under test. */
+    turns: IntentTurn[];
+    /** Flow is listed whether or not a canvas is open, as in Swan; "unavailable" omits it. */
+    flow?: "open" | "closed" | "unavailable";
+    /** false omits the 3D stage capabilities (scene, materials, geometry, OpenSCAD, imports). */
+    stage?: boolean;
+    openScadParts?: OpenScadTarget[];
+    /** Acceptable routes. More than one only where the shared prompt allows either. */
+    expect: readonly IntentRoute[];
+    note?: string;
+};
+/**
+ * The build-intent QA requests (22 Sep 2026), with expectations updated to the creation
+ * rules restored in 6.0 (image previews for ambiguous and likeness-dependent requests),
+ * plus preview follow-ups and attached-photo cases.
+ */
+export declare const INTENT_CASES: IntentCase[];

package/dist/intent-bench-cases.js

@@ -0,0 +1,149 @@
+/** The first action a run commits to, or how it ended without one. */
+export const INTENT_ROUTES = [
+    "scene",
+    "openscad",
+    "geometry-nodes",
+    "image",
+    "image-edit",
+    "mesh",
+    "image-to-mesh",
+    "flow",
+    "import",
+    "reply",
+    "undecided",
+    "error",
+];
+const user = (text, images) => ({ role: "user", text, images });
+// A takeaway coffee cup on a table, from Womp's media host.
+const cup = { url: "https://media.womp.com/staging-kkp/thumbnails/ma_IMG_3417_1791494968752.webp" };
+const washer = "difference() { cylinder(h = thickness, d = outer_diameter, $fn = 96); translate([0, 0, -1]) cylinder(h = thickness + 2, d = hole_diameter, $fn = 96); }";
+/**
+ * The build-intent QA requests (22 Sep 2026), with expectations updated to the creation
+ * rules restored in 6.0 (image previews for ambiguous and likeness-dependent requests),
+ * plus preview follow-ups and attached-photo cases.
+ */
+export const INTENT_CASES = [
+    { id: "P1", group: "core", turns: [user("Make a washer: 20 mm outer diameter, 8 mm hole, 2 mm thick.")], expect: ["openscad"] },
+    { id: "P2", group: "core", turns: [user("Make a parametric phone stand where I can adjust the width and tilt angle.")], expect: ["openscad"] },
+    { id: "P3", group: "core", turns: [user("Make a storage tray, 120 × 80 × 25 mm.")], expect: ["openscad"] },
+    { id: "G1", group: "core", turns: [user("Make a branching coral with adjustable branch density.")], expect: ["geometry-nodes"] },
+    { id: "G2", group: "core", turns: [user("Make a rippling surface with adjustable wave speed.")], expect: ["geometry-nodes"] },
+    { id: "G3", group: "core", turns: [user("Make an organic vine where I can adjust the curl and leaf spacing.")], expect: ["geometry-nodes"] },
+    {
+        id: "S1", group: "core", turns: [user("A car")], expect: ["image"],
+        note: "Ambiguous creative request: one image preview, then a choice. The 4.3 QA expected scene.",
+    },
+    { id: "S2", group: "core", turns: [user("Make a cup of water in the scene.")], expect: ["scene"] },
+    {
+        id: "S3", group: "core", turns: [user("Make a friendly little mushroom in 3D.")], expect: ["image"],
+        note: "A character depends on likeness, so it previews first. The 4.3 QA expected scene.",
+    },
+    { id: "S4", group: "core", turns: [user("Make a simple wooden stool in 3D.")], expect: ["scene"], note: "Plain functional objects build directly." },
+    { id: "S5", group: "core", turns: [user("Make a 3D corgi.")], expect: ["image"], note: "Creatures preview first." },
+    { id: "S6", group: "core", turns: [user("Make a 3D model of a Ferrari F40.")], expect: ["image"], note: "Specific vehicles preview first." },
+    { id: "S7", group: "core", turns: [user("a cozy cabin in the woods")], expect: ["image"], note: "Ambiguous creative request." },
+    { id: "I1", group: "core", turns: [user("Generate an image of a red toy car on a white background.")], expect: ["image"] },
+    { id: "I2", group: "core", turns: [user("Make a studio product photo of a blue ceramic cup.")], expect: ["image"] },
+    { id: "I3", group: "core", turns: [user("Make a picture of a small fantasy mushroom house.")], expect: ["image"] },
+    { id: "I4", group: "core", turns: [user("Design a logo for a coffee shop called Bean There.")], expect: ["image"] },
+    { id: "M1", group: "core", turns: [user("Generate a red toy car mesh.")], expect: ["mesh"] },
+    { id: "M2", group: "core", turns: [user("Generate a mesh of a blue ceramic water cup.")], expect: ["mesh"] },
+    { id: "M3", group: "core", turns: [user("Generate a fantasy mushroom house mesh.")], expect: ["mesh"] },
+    { id: "F1", group: "core", flow: "open", turns: [user("Make an image of a red toy car using Flow nodes.")], expect: ["flow"] },
+    { id: "F2", group: "core", flow: "open", turns: [user("Make a ceramic cup mesh using Flow.")], expect: ["flow"] },
+    { id: "F3", group: "core", flow: "open", turns: [user("Set up a nodes workflow that makes an image of a mushroom house.")], expect: ["flow"] },
+    {
+        id: "I1-flow-open", group: "workspace", flow: "open",
+        turns: [user("Generate an image of a red toy car on a white background.")], expect: ["image"],
+        note: "An open Flow canvas does not redirect ordinary generation.",
+    },
+    {
+        id: "F1-flow-closed", group: "workspace", flow: "closed",
+        turns: [user("Make an image of a red toy car using Flow nodes.")], expect: ["reply"],
+        note: "Explains that Flow must be open instead of generating inline.",
+    },
+    {
+        id: "S2-no-stage", group: "workspace", stage: false,
+        turns: [user("Make a cup of water in the scene.")], expect: ["reply"],
+        note: "Explains that the 3D stage is unavailable instead of switching route.",
+    },
+    {
+        id: "P1-follow-up", group: "conversation",
+        turns: [
+            user("Make a washer: 20 mm outer diameter, 8 mm hole, 2 mm thick."),
+            { role: "assistant", text: "I made the washer as an adjustable part (object 12): 20 mm outer diameter, 8 mm hole and 2 mm thick." },
+            user("Make the hole 10 mm."),
+        ],
+        openScadParts: [{
+                kind: "part", objectId: 12, parentId: 0, title: "Washer", source: washer,
+                params: { outer_diameter: 20, hole_diameter: 8, thickness: 2 },
+            }],
+        expect: ["openscad"],
+    },
+    {
+        id: "I1-follow-up", group: "conversation",
+        turns: [
+            user("Generate an image of a red toy car on a white background."),
+            { role: "assistant", text: "Here's your red toy car on a white background.", generated: [{ name: "red-toy-car.png", color: [200, 30, 30] }] },
+            user("Make it blue."),
+        ],
+        expect: ["image-edit"],
+    },
+    {
+        id: "ambiguous-yes", group: "conversation",
+        turns: [
+            user("Generate an image of a red toy car on a white background."),
+            {
+                role: "assistant",
+                text: "Here's your red toy car. I can change its colors, or turn it into a sculpted 3D model.",
+                generated: [{ name: "red-toy-car.png", color: [200, 30, 30] }],
+            },
+            user("Yes"),
+        ],
+        expect: ["reply"],
+        note: "Two options were offered, so a bare yes needs clarification.",
+    },
+    {
+        id: "preview-to-mesh", group: "conversation",
+        turns: [
+            user("A car"),
+            {
+                role: "assistant",
+                text: "Here's a quick preview. Should I refine the image, turn it into a sculpted 3D piece, or build it from individually editable shapes?",
+                generated: [{ name: "car-preview.png", color: [40, 90, 200] }],
+            },
+            user("Turn it into a sculpted 3D piece."),
+        ],
+        expect: ["image-to-mesh"],
+        note: "Reuses the preview as the image-to-mesh input.",
+    },
+    {
+        id: "preview-to-shapes", group: "conversation",
+        turns: [
+            user("A car"),
+            {
+                role: "assistant",
+                text: "Here's a quick preview. Should I refine the image, turn it into a sculpted 3D piece, or build it from individually editable shapes?",
+                generated: [{ name: "car-preview.png", color: [40, 90, 200] }],
+            },
+            user("Build it from editable shapes."),
+        ],
+        expect: ["scene"],
+    },
+    { id: "photo-3d", group: "photo", turns: [user("make this 3D", [cup])], expect: ["image-to-mesh"] },
+    {
+        id: "photo-3d-flow-open", group: "photo", flow: "open", turns: [user("make this 3D", [cup])], expect: ["image-to-mesh"],
+        note: "Without an explicit Flow request, an attached photo is standard image-to-mesh.",
+    },
+    { id: "photo-shapes", group: "photo", turns: [user("build this with editable shapes", [cup])], expect: ["scene"] },
+    {
+        id: "photo-only", group: "photo", turns: [user(undefined, [cup])], expect: ["reply"],
+        note: "A photo alone does not say what to make, so ask. dev0 thread 145108 asked to open Flow instead.",
+    },
+    { id: "photo-only-flow-open", group: "photo", flow: "open", turns: [user(undefined, [cup])], expect: ["reply"] },
+    { id: "photo-flow-closed", group: "photo", flow: "closed", turns: [user("turn this into a 3D model using Flow", [cup])], expect: ["reply"] },
+    {
+        id: "photo-flow-open", group: "photo", flow: "open", turns: [user("turn this into a 3D model using Flow", [cup])], expect: ["flow"],
+        note: "Flow has no node for a chat photo; build the graph with what it has and leave the image input to the user.",
+    },
+];

package/dist/intent-bench-cli.d.ts

@@ -0,0 +1,2 @@
+#!/usr/bin/env node
+export {};

package/dist/intent-bench-cli.js

@@ -0,0 +1,122 @@
+#!/usr/bin/env node
+import { existsSync } from "node:fs";
+import { mkdir, writeFile } from "node:fs/promises";
+import path from "node:path";
+import { loadEnvFile } from "node:process";
+import { parseArgs } from "node:util";
+import { INTENT_CASES } from "./intent-bench-cases.js";
+import { intentBenchModel, runIntentBench, summarizeIntentRuns } from "./intent-bench.js";
+import { SPARK_MODELS } from "./models.js";
+function positiveInteger(value, name) {
+    const number = Number(value);
+    if (!Number.isSafeInteger(number) || number < 1)
+        throw new Error(`${name} must be a positive integer.`);
+    return number;
+}
+async function main() {
+    const { values } = parseArgs({ options: {
+            model: { type: "string", default: "gpt-6-luna" }, reasoning: { type: "string", default: "medium" },
+            repeat: { type: "string", default: "3" }, concurrency: { type: "string", default: "24" },
+            only: { type: "string" }, group: { type: "string" }, "max-steps": { type: "string", default: "12" },
+            "timeout-seconds": { type: "string", default: "240" }, output: { type: "string" },
+            list: { type: "boolean" }, help: { type: "boolean" },
+        } });
+    if (values.help) {
+        console.log(`intent-bench [options]
+
+Measures which route Spark takes for ${INTENT_CASES.length} requests: editable scene, OpenSCAD, Geometry Nodes,
+image, image edit, mesh, image-to-mesh, Flow, import, or a reply without acting.
+Runs use Swan's skills over fixtures: no renderer, no generation, no scene changes. The first
+call that would change something is recorded and the run stops there. Makes paid model calls.
+
+--model <id>              Default: gpt-6-luna. Models: ${SPARK_MODELS.map((model) => model.id).join(", ")}
+--reasoning <effort>      Default: medium
+--repeat <n>              Runs per case; default: 3
+--concurrency <n>         Runs at once; default: 24
+--only <ids>              Comma-separated case IDs (--list shows them)
+--group <names>           core, workspace, conversation, photo
+--max-steps <n>           Model steps per run; default: 12
+--timeout-seconds <n>     Per run; default: 240
+--output <directory>      New directory only; default: .spark/intent-bench/<timestamp>
+--list                    List the cases and expected routes`);
+        return;
+    }
+    if (values.list) {
+        for (const testCase of INTENT_CASES) {
+            const prompt = testCase.turns.at(-1);
+            const text = prompt.role === "user" ? `${prompt.images?.length ? "[photo] " : ""}${prompt.text ?? ""}` : "";
+            console.log(`${testCase.id.padEnd(22)} ${testCase.group.padEnd(13)} ${testCase.expect.join("|").padEnd(22)} ${text}`);
+        }
+        return;
+    }
+    const model = SPARK_MODELS.find((item) => item.id === values.model);
+    if (!model)
+        throw new Error("Select a supported model using --model. See --help.");
+    const reasoning = values.reasoning;
+    if (!model.reasoningEfforts.includes(reasoning))
+        throw new Error("Unsupported reasoning effort for this model.");
+    const only = values.only?.split(",").map((id) => id.trim()).filter(Boolean);
+    const groups = values.group?.split(",").map((group) => group.trim()).filter(Boolean);
+    const unknown = only?.filter((id) => !INTENT_CASES.some((testCase) => testCase.id === id)) ?? [];
+    if (unknown.length)
+        throw new Error(`Unknown case IDs: ${unknown.join(", ")}. Use --list.`);
+    const cases = INTENT_CASES.filter((testCase) => (!only || only.includes(testCase.id)) && (!groups || groups.includes(testCase.group)));
+    if (!cases.length)
+        throw new Error("No cases selected.");
+    const repeat = positiveInteger(values.repeat, "--repeat");
+    const concurrency = positiveInteger(values.concurrency, "--concurrency");
+    const maxSteps = positiveInteger(values["max-steps"], "--max-steps");
+    const timeoutMs = positiveInteger(values["timeout-seconds"], "--timeout-seconds") * 1000;
+    if (existsSync(".env"))
+        loadEnvFile();
+    const resolved = intentBenchModel({ modelId: model.id, reasoningEffort: reasoning });
+    const output = path.resolve(values.output ?? `.spark/intent-bench/${new Date().toISOString().replaceAll(":", "-")}`);
+    // Never reuse a result directory or overwrite a previous benchmark.
+    await mkdir(path.dirname(output), { recursive: true });
+    await mkdir(output);
+    const controller = new AbortController();
+    const stop = () => controller.abort(new Error("Benchmark interrupted."));
+    process.once("SIGINT", stop);
+    process.once("SIGTERM", stop);
+    const total = cases.length * repeat;
+    console.error(`${total} runs (${cases.length} cases × ${repeat}) on ${model.id}/${reasoning}, ${Math.min(concurrency, total)} at once. Results: ${output}`);
+    const startedAt = Date.now();
+    let done = 0;
+    const line = (run) => `${run.pass ? "pass" : "FAIL"}  ${run.caseId.padEnd(22)} #${run.repeat}  ${run.route.padEnd(14)} ${run.error ?? run.decision?.call ?? ""}`;
+    try {
+        const runs = await runIntentBench({
+            cases, repeat, concurrency, model: resolved, maxSteps, timeoutMs, signal: controller.signal,
+            onRun: (run) => console.error(`[${String(++done).padStart(String(total).length)}/${total}] ${line(run)}`),
+        });
+        const summary = summarizeIntentRuns(cases, runs);
+        const passed = runs.filter((run) => run.pass).length;
+        const priced = runs.filter((run) => run.costUsd !== null);
+        const costUsd = priced.reduce((sum, run) => sum + run.costUsd, 0);
+        const durationMs = Date.now() - startedAt;
+        await writeFile(path.join(output, "report.json"), `${JSON.stringify({
+            model: model.id, reasoning, repeat, maxSteps, startedAt: new Date(startedAt).toISOString(), durationMs,
+            passed, runs: runs.length, costUsd: priced.length ? costUsd : null, costStatus: priced.length === runs.length ? "estimated" : priced.length ? "partial" : "unavailable",
+            summary, results: runs,
+        }, null, 2)}\n`);
+        console.log(`\n${"case".padEnd(22)} ${"expected".padEnd(22)} pass   routes`);
+        for (const item of summary) {
+            const routes = Object.entries(item.routes).map(([route, count]) => `${route}×${count}`).join(" ");
+            const mark = item.passed === item.runs ? " " : "!";
+            console.log(`${mark}${item.id.padEnd(21)} ${item.expect.join("|").padEnd(22)} ${`${item.passed}/${item.runs}`.padEnd(6)} ${routes}`);
+        }
+        const reliable = summary.filter((item) => item.passed === item.runs).length;
+        const gaps = runs.filter((run) => run.fixtureErrors.length).length;
+        console.log(`\n${passed}/${runs.length} runs on the expected route (${Math.round((100 * passed) / runs.length)}%), ` +
+            `${reliable}/${summary.length} cases on it every time. ${(durationMs / 1000).toFixed(0)} s, ` +
+            `${priced.length ? `$${costUsd.toFixed(3)} estimated` : "cost unavailable"}.` +
+            (gaps ? `\n${gaps} runs hit a read the fixtures cannot answer (fixtureErrors in the report).` : ""));
+        console.log(`Report: ${path.join(output, "report.json")}`);
+        if (passed < runs.length)
+            process.exitCode = 1;
+    }
+    finally {
+        process.removeListener("SIGINT", stop);
+        process.removeListener("SIGTERM", stop);
+    }
+}
+main().catch((error) => { console.error(error instanceof Error ? error.message : String(error)); process.exitCode = 1; });

package/dist/intent-bench-flow-catalog.d.ts

@@ -0,0 +1 @@
+export declare const INTENT_BENCH_FLOW_CATALOG: string;