@lampdotdev/lamp 0.36.2-nightly.41 → 0.36.2-nightly.43

package/README.md

@@ -19,7 +19,7 @@ Mac app, `projectOf`), or any folder with an `index.html`; without one, the fold
 | `lamp init [--update]` | Puts Lamp's game runtime beside a page that's already there, changing none of the page's own files. `--update` writes again the Lamp files the folder has (`lamp.js` too), adding the game's only to a page that's a game already. |
 | `lamp unpack <file.lampapp> [folder]` | Unpacks a Lamp app file into a new folder beside it, named for it. A file carries none of Lamp's own files: a page gets them beside it as it's unpacked (the runtime for a game, lamp.js, the libraries its import map names), and a Mac app gets Package.swift, build.sh and LampSupport.swift from Lamp's starting files as it's first built (`writeMacStarter`), where it has none. One with a widget, a share extension or a 3D scene is left to Lamp's app, which writes the files they're built with. |
 | `lamp export [--to file] [--anyway]` | Makes the app's Lamp app file as Lamp's Share… does (`Sharing.export`, followed by `cli/lampapp.mjs`): its own files as its kind travels, Assets/, APP.md, what it declares, its credits with the app added to the chain, the licenses they name, its icon's design and lamp-app.json, never Lamp's own files, nor what the folder's `.lampignore` names. Refused when a license its credits name is gone or something in it is shaped like a key; `--anyway` sends work its credits give no license for, and lamp keeps that the person agreed (in its state), so it asks again only for work new to the app. A source credited `"live": true` (fetched while the app runs, nothing of it in the app) is never asked about. The shapes of keys, the names a license goes by and the sizes another Lamp opens are in `Resources/share-rules.json`, which Lamp's app reads too (`ShareRules`). |
-| `lamp dev [--open] [--lan] [--port N]` | Runs the page and says how each change went. Each save goes into every open copy of it, with the card Lamp's app puts on an app taking a change (`page.js`, left out of lamp's pictures; `lamp changing "<words>"` says on it what's changing). `--open` opens a window for it; `--lan` lets other devices on the network open it. |
+| `lamp dev [--open] [--lan] [--port N]` | Runs the page and says how each change went. Each save of a file a page has asked for goes into every open copy of it (a file no page has loaded, such as renders or logs a page's tests write, opens no page again), with the card Lamp's app puts on an app taking a change (`page.js`, left out of lamp's pictures; `lamp changing "<words>"` says on it what's changing). `--open` opens a window for it; `--lan` lets other devices on the network open it. |
 | `lamp open` | Shows the page in a window, running it in the background if it isn't running, and returns at once. |
 | `lamp open --in-lamp` | Gives a Mac app to Lamp's app (`cli/outside.mjs`), over the socket the person's own coding agent uses (`OutsideServer`, `OutsideWork`): `open_project` takes the folder on, with Lamp's review the first time, and from then on `look`, `reload` and `dev` call `apply_change`, so a change goes into the open app as Lamp's app puts it there (`LiveReload`) and its outcome is said: live, held, reopened, not_built, crashed. `use` calls `use_app`, in plain sight in the app Lamp's app has open, and `stop` calls `close_app`. A folder Lamp's app no longer has (its refusal says `taken_on: false`) is lamp's own again, said once. `lamp open --here` has lamp run its own copy again. `LAMP_APP` names another Lamp by its bundle id (a Lamp Dev instance). A Mac app `lamp new --mac` makes has the `lamp-app.json` Lamp's app takes a folder on by. |
 | `lamp look [--reopen] [--wait s] [--json]` | How the changes since the last look went, a picture of the page, its screen as text, new errors and the log. `--reopen` opens the page again from its beginning first. |

package/Resources/skills/web-app/SKILL.md

@@ -23,7 +23,10 @@ opening comment for the whole list. What it gives:
   heard of Lamp finds out how. Where it answers `connect: true` (a page on lamp.dev that hasn't reached this computer
   yet), also show a small "Connect AI" button that calls `LampAI.connect(capability)` with the same kind of AI you asked
   `status` about (`LampAI.connect('pictures')` for pictures) and use the answer it gives as `status`'s;
-  never call `connect()` as the page loads, since it makes the browser ask the person.
+  never call `connect()` as the page loads, since it makes the browser ask the person. For JSON the app reads, give
+  `ask` a `schema` (a JSON Schema: every property in `required`, `additionalProperties: false`), which holds the AI to
+  it, rather than only describing the shape. A long `ask` the person can stop takes a `signal` (an `AbortController`'s),
+  and its failure then has `stopped`, which needs no error shown.
 - **LampAI.act**: the person asks the app to do something in their own words, and the app's own reads and writes do
   it; "Doing things for the person" below, with **LampVoice** and **LampAI.hear** for saying it.
 - **LampStore**: what the app saves, as JSON on the device. Once something is saved, only ever add to it: a new field
@@ -102,6 +105,8 @@ whichever AI the person set up for apps' text, so don't tie it to voice: typing
   and take only the first answer, a button, what was heard or what was typed. With none, it's answered by voice or
   typing; never guess choices from the question's wording.
 - Keep what each write changed for Undo, and undo only what's still as the write left it; say which ones it left.
+- Where the app draws something the AI builds (a scene, a layout, a chart), give it a read that returns
+  `LampAI.picture(canvas)` in its result, so it can look at its work and fix it in the same turn.
 - Show each step while it runs (`onStep`), and cancel the turn (`turn.cancel()`) when the person closes it or the page
   is hidden.
 - `about` says what the app is and its rules for asking back: which person, which day and time, which of two things

package/cli/ai.mjs

@@ -56,10 +56,11 @@ export class EngineTrouble extends Error {
  * computer is read or run whatever the prompt asks. Pictures (`images`, base64 with their `media_type`) go with the
  * request itself: to Claude Code in its input, to Codex as attachments, never through a tool that reads files.
  * Lamp's app runs it this way too (`engineAnswer` in calls.mjs, `AppAI.viaEngine`), naming the program (`tool`) and
- * what it needs set (`env`); on its own lamp finds the person's own on the PATH.
+ * what it needs set (`env`); on its own lamp finds the person's own on the PATH. `spent` hears what a call through
+ * OpenRouter cost; one through the person's own sign-in costs nothing through Lamp.
  */
-export function complete({ system, prompt, schema, seconds = 60, which = engine(), tool, env = {}, model = process.env.LAMP_AI_MODEL || "", images = [], signal } = {}) {
-  if (which === "openrouter") return viaOpenRouter({ system, prompt, schema, seconds, model, images, signal });
+export function complete({ system, prompt, schema, seconds = 60, which = engine(), tool, env = {}, model = process.env.LAMP_AI_MODEL || "", images = [], signal, spent } = {}) {
+  if (which === "openrouter") return viaOpenRouter({ system, prompt, schema, seconds, model, images, signal, spent });
   if (which !== "claude" && which !== "codex") return Promise.reject(new EngineTrouble("missing", missing));
   const folder = mkdtempSync(join(tmpdir(), "lamp-ai-"));
   const sayer = system || "You are a helpful assistant inside an app.";
@@ -156,31 +157,68 @@ function codexFailure(event) {
   return nested || words;
 }
 
+// MARK: The OpenRouter key
+
+const KEY_REFUSED = "OpenRouter didn't accept the key in OPENROUTER_API_KEY: it may have been revoked. Set a new one from openrouter.ai/keys.";
+const KEY_EMPTY = "The OpenRouter key in OPENROUTER_API_KEY has no credit left. Add credit at openrouter.ai.";
+let keyAsked = { key: "", at: 0, problem: null };
+
+/**
+ * What's wrong with the key in OPENROUTER_API_KEY, as OpenRouter says (as Lamp's app asks it, `openRouterVerdict`), or
+ * null: a key that's there isn't one that works (revoked, or out of credit), and `status` says so rather than offer
+ * what every call would then refuse. Asked at most every five minutes, and quickly, since a page waits on its status; when
+ * OpenRouter can't be reached in time, the key may be good.
+ */
+export async function keyProblem(fetcher = fetch) {
+  const key = process.env.OPENROUTER_API_KEY;
+  if (!key) return null;
+  if (keyAsked.key === key && Date.now() - keyAsked.at < 5 * 60_000) return keyAsked.problem;
+  let problem = null;
+  try {
+    const answer = await fetcher("https://openrouter.ai/api/v1/key", { headers: { Authorization: `Bearer ${key}` }, signal: AbortSignal.timeout(4000) });
+    if (answer.status === 401 || answer.status === 403) problem = KEY_REFUSED;
+    else if (answer.status === 200) {
+      const left = (await answer.json().catch(() => ({})))?.data?.limit_remaining;
+      if (typeof left === "number" && left <= 0) problem = KEY_EMPTY;
+    } else return null;
+  } catch { return null; }
+  keyAsked = { key, at: Date.now(), problem };
+  return problem;
+}
+
+/** What an OpenRouter failure says to the page: the key's trouble as the setting to fix, else what OpenRouter said. */
+function openRouterWords(error) {
+  if (error?.status === 401 || error?.status === 403) return KEY_REFUSED;
+  if (error?.status === 402) return KEY_EMPTY;
+  const words = error?.reply?.error?.message;
+  return words ? String(words) : String(error?.message || "OpenRouter didn't answer.");
+}
+
 /** An answer from OpenRouter with the person's key (openrouter.mjs, as Lamp's app sends it), its troubles said as an engine's. */
-async function viaOpenRouter({ system, prompt, schema, seconds, model, images, signal }) {
+async function viaOpenRouter({ system, prompt, schema, seconds, model, images, signal, spent }) {
   try {
-    return (await openRouterText({ key: process.env.OPENROUTER_API_KEY, prompt, images, system: system || "You are a helpful assistant inside an app.",
-      model: model || DEFAULT_MODEL, json: !!schema, schema, seconds, signal })).text;
+    const { text, cost } = await openRouterText({ key: process.env.OPENROUTER_API_KEY, prompt, images, system: system || "You are a helpful assistant inside an app.",
+      model: model || DEFAULT_MODEL, json: !!schema, schema, seconds, signal });
+    spent?.(cost);
+    return text;
   } catch (error) {
     if (!(error instanceof OpenRouterTrouble)) throw error;
-    const words = error.reply?.error?.message;
-    throw new EngineTrouble(error.kind === "refused" ? "failed" : error.kind, words ? String(words) : error.message);
+    spent?.(error.cost);
+    throw new EngineTrouble(error.kind === "refused" ? "failed" : error.kind, error.kind === "refused" ? openRouterWords(error) : error.message);
   }
 }
 
 // MARK: Status and asking
 
-/** `/ai/status`: what can answer here. Text, decisions and actions with an engine; pictures as `pictureRoute` says. */
-export function status(page) {
-  const which = engine();
-  const route = which ? named(which) : null;
-  const available = (kind) => route ? { available: true, route, where: "remote" } : { available: false, reason: missing, setup: "lamp" };
-  const drawer = pictureRoute(page);
-  return {
-    text: !!route, decisions: !!route, pictures: !!drawer, act: !!route,
-    details: { text: available("text"), decisions: available("decisions"), act: available("act"),
-      pictures: drawer ? { available: true, route: named(drawer), where: "remote" } : { available: false, reason: noPictures(page), setup: "lamp" } },
-  };
+/** `/ai/status`: what can answer here. Text, decisions and actions with an engine; pictures as `pictureRoute` says.
+ *  What goes through OpenRouter answers only while it takes the key (`keyProblem`). */
+export async function status(page, problem = keyProblem) {
+  const which = engine(), drawer = pictureRoute(page);
+  const broken = which === "openrouter" || drawer === "openrouter" ? await problem() : null;
+  const one = (by, reason) => !by ? { available: false, reason, setup: "lamp" }
+    : by === "openrouter" && broken ? { available: false, reason: broken, setup: "lamp" } : { available: true, route: named(by), where: "remote" };
+  const details = { text: one(which, missing), decisions: one(which, missing), act: one(which, missing), pictures: one(drawer, noPictures(page)) };
+  return { text: details.text.available, decisions: details.decisions.available, pictures: details.pictures.available, act: details.act.available, details };
 }
 
 // MARK: Models (`LampAI.models`, and `model` on `ask` and `act`)
@@ -281,16 +319,35 @@ const GUARD_SECONDS = 600;
 const secondsFor = (body) => (typeof body?.seconds === "number" && body.seconds > 0 ? Math.min(body.seconds, GUARD_SECONDS) : GUARD_SECONDS);
 
 /** `/ai`: a text answer, or JSON when asked for it. */
-export async function ask(body, here = engines()) {
+export async function ask(body, here = engines(), signal) {
   const prompt = typeof body?.prompt === "string" ? body.prompt.trim() : "";
   if (!prompt) return [400, { error: "Missing prompt." }];
+  // A JSON Schema the answer must fit (`LampAI.ask`'s `schema`): every engine holds the model to it.
+  const schema = body.schema && typeof body.schema === "object" && !Array.isArray(body.schema) ? body.schema : undefined;
+  const json = !!body.json || !!schema;
   let system = typeof body.system === "string" ? body.system : "";
-  if (body.json) system += (system ? "\n\n" : "") + "Answer with JSON only, nothing around it.";
+  if (json) system += (system ? "\n\n" : "") + "Answer with JSON only, nothing around it.";
   const { which, model } = routeFor(body.model, here);
-  try { return [200, { text: await complete({ system, prompt, images: body.images, which, model, seconds: secondsFor(body) }) }]; }
-  catch (error) { return [502, { error: error.message }]; }
+  let cost = 0;
+  const spent = (more) => { cost += Number(more) || 0; };
+  // One time for the whole request, its second try included, as Lamp's app gives it (`AppAI.complete`).
+  const deadline = Date.now() + secondsFor(body) * 1000;
+  const once = (asked) => complete({ system, prompt: asked, schema, images: body.images, which, model, seconds: Math.max(1, (deadline - Date.now()) / 1000), signal, spent });
+  try {
+    let text = await once(prompt);
+    // JSON that doesn't read (cut off, or with words around it) is no use to the page: asked once more, saying so.
+    if (json && !readsAsJSON(text)) {
+      if (deadline - Date.now() < 5000) return [502, { error: "The AI's reply wasn't valid JSON, and there was no time left to ask again.", cost }];
+      text = await once(`${prompt}\n\nYour previous reply was not valid JSON (it may have been cut off). Reply with the complete JSON only.`);
+      if (!readsAsJSON(text)) return [502, { error: "The AI's reply wasn't valid JSON, even on a second try. Ask for less at once.", cost }];
+    }
+    return [200, { text: json ? unfenced(text) : text, cost }];
+  } catch (error) { return [502, { error: error.message, cost }]; }
 }
 
+const unfenced = (text) => String(text).trim().replace(/^```(?:json)?\s*|\s*```$/g, "").trim();
+function readsAsJSON(text) { try { JSON.parse(unfenced(text)); return true; } catch { return false; } }
+
 // MARK: Decisions
 
 /** `/ai/decide`: each question answered by a text model, told and read as Lamp's app does it (app-ai.mjs). */
@@ -311,7 +368,11 @@ export async function decide(body) {
 /** `/ai/act`: one step of the page's agent loop, told and checked as Lamp's app does it (app-ai.mjs). */
 export async function act(body, here = engines()) {
   const first = here[0] ?? null;
-  if (body?.probe === true) return [200, { act: !!first, route: first || "", why: first ? "" : missing }];
+  if (body?.probe === true) {
+    // A key OpenRouter refuses can't answer a step, as `/ai/status` says too.
+    const refused = first === "openrouter" ? await keyProblem() : null;
+    return [200, { act: !!first && !refused, route: first || "", why: refused || (first ? "" : missing) }];
+  }
   if (!first) return [501, { error: missing }];
   let r;
   try { r = readStep(JSON.stringify(body ?? null)); } catch (refused) { return [refused.status ?? 400, { error: refused.message }]; }
@@ -321,26 +382,29 @@ export async function act(body, here = engines()) {
   const seconds = 600;
   const deadline = Date.now() + Math.min(r.seconds, seconds) * 1000;
   const tag = "outside-text-" + randomBytes(4).toString("hex");
-  let retry = null;
+  let retry = null, cost = 0;
   for (let attempt = 0; attempt < 2; attempt++) {
     const left = (deadline - Date.now()) / 1000;
-    if (left < 1) return [504, { error: "This step ran out of time." }];
+    if (left < 1) return [504, { error: "This step ran out of time.", cost }];
     let read;
     try {
       if (which === "openrouter") {
         const body = chatBody(r, tag, retry, model);
-        read = readMessage((await openRouterChat({ key: process.env.OPENROUTER_API_KEY, body, seconds: left })).message, r);
+        const answered = await openRouterChat({ key: process.env.OPENROUTER_API_KEY, body, seconds: left });
+        cost += answered.cost;
+        read = readMessage(answered.message, r);
       } else {
-        const { system, prompt } = planPrompt(r, tag, retry);
-        read = readPlan(await complete({ system, prompt, schema: planSchema(r), seconds: left, which, model }), r);
+        const { system, prompt, images } = planPrompt(r, tag, retry);
+        read = readPlan(await complete({ system, prompt, images, schema: planSchema(r), seconds: left, which, model }), r);
       }
     } catch (error) {
-      const words = error?.reply?.error?.message || error.message;
-      return Date.now() >= deadline || error?.kind === "timeout" ? [504, { error: "This step ran out of time." }] : [502, { error: String(words) }];
+      cost += Number(error?.cost) || 0;
+      const words = error instanceof OpenRouterTrouble && error.kind === "refused" ? openRouterWords(error) : error?.reply?.error?.message || error.message;
+      return Date.now() >= deadline || error?.kind === "timeout" ? [504, { error: "This step ran out of time.", cost }] : [502, { error: String(words), cost }];
     }
-    if (read.answer) return [200, { ...read.answer, route: which, seconds }];
+    if (read.answer) return [200, { ...read.answer, route: which, seconds, cost }];
     if (attempt === 0) { retry = read.problem; continue; }
-    return [200, { reply: "I couldn't work out how to do that.", ask: false, route: which, seconds, trouble: read.problem }];
+    return [200, { reply: "I couldn't work out how to do that.", ask: false, route: which, seconds, trouble: read.problem, cost }];
   }
   return [502, { error: "The AI didn't answer." }];
 }
@@ -404,7 +468,7 @@ export async function makeImage(body, page, app = {}) {
     }
   } catch (error) {
     if (error?.kind === "timeout") return [504, { error: "The picture took too long to make." }];
-    return [502, { error: String(error?.reply?.error?.message || error.message) }];
+    return [502, { error: openRouterWords(error) }];
   }
   const type = pictureType(made);
   if (!type) return [502, { error: route === "codex" ? "Codex didn't make the picture. Check it's signed in." : "The image model didn't return a picture." }];
@@ -483,11 +547,11 @@ export function codexImage({ prompt, references = [], aspectRatio, seconds = PIC
 }
 
 /** A page's AI request, by its route, as Lamp's app answers it: [status, JSON]. `app` is what the app chose for itself
- *  (`imageModel`), when that's known. */
-export async function answer(path, body, page, app = {}) {
-  if (path === "/ai/status") return [200, status(page)];
+ *  (`imageModel`), when that's known; `signal` stops an answer the page no longer waits for. */
+export async function answer(path, body, page, app = {}, signal) {
+  if (path === "/ai/status") return [200, await status(page)];
   if (path === "/ai/models") return [200, models(enginesFor(page))];
-  if (path === "/ai") return ask(body, enginesFor(page));
+  if (path === "/ai") return ask(body, enginesFor(page), signal);
   if (path === "/ai/act") return act(body, enginesFor(page));
   if (path === "/ai/decide") return decide(body);
   if (path === "/ai/hear") return body?.probe === true ? [200, { hear: false, why: "Hearing needs Lamp's app." }] : [501, { error: "Hearing needs Lamp's app: open the app in Lamp." }];

package/cli/app-ai.mjs

@@ -17,7 +17,11 @@
 // `MAX_STEPS` is the most steps an app may raise a turn to (`ACT_LIMITS_MAX` in lamp.js, `LampActSession.mostSteps`):
 // room to build something real a few calls at a time, while a turn that goes round in circles still ends. The most
 // writes follows from it, since each step holds at most `MAX_CALLS`.
-export const MAX_TOOLS = 24, MAX_CALLS = 8, MAX_STEPS = 40, MAX_BODY = 2 * 1024 * 1024, MAX_CHOICES = 4, MAX_CHOICE = 24;
+export const MAX_TOOLS = 24, MAX_CALLS = 8, MAX_STEPS = 40, MAX_BODY = 3.5 * 1024 * 1024, MAX_CHOICES = 4, MAX_CHOICE = 24;
+/** The most pictures a step carries from its tools' results (`LampAI.picture`), the newest; lamp.js sends no more, and
+ *  keeps them small enough (`ACT_PICTURE_BYTES`) for a step with its text results to stay under `MAX_BODY`. */
+export const MAX_PICTURES = 6;
+const PICTURE_TYPES = ["image/jpeg", "image/png", "image/webp", "image/gif"];
 /** The longest a step may take, whatever its route, before its call is taken to have stopped answering (`ACT_STEP_SECONDS`
  *  in lamp.js, `AppAct.turnSeconds`). A turn has no time of its own unless the app gives it one. */
 export const MAX_TURN_SECONDS = 600;
@@ -69,7 +73,7 @@ function parseArg(raw, path) {
  * steps, route, seconds, final, model?}`, each tool's `args` as [name, shape] pairs by name. Throws `Refused`.
  */
 export function readStep(text) {
-  if (Buffer.byteLength(text) > MAX_BODY) throw new Refused(413, "A step can carry at most 2 MB.");
+  if (Buffer.byteLength(text) > MAX_BODY) throw new Refused(413, "A step can carry at most 3.5 MB.");
   let body;
   try { body = JSON.parse(text); } catch {}
   if (!body || typeof body !== "object" || Array.isArray(body)) throw new Refused(400, "A step needs a JSON object.");
@@ -95,6 +99,8 @@ export function readStep(text) {
   const rawSteps = Array.isArray(body.steps) ? body.steps.filter((s) => s && typeof s === "object") : [];
   if (rawSteps.length >= MAX_STEPS) throw new Refused(400, `A turn has at most ${MAX_STEPS} steps.`);
   const seen = new Set(), steps = [];
+  // The pictures the results carry, numbered in the order the turn made them; only the newest MAX_PICTURES are shown.
+  const pictures = [];
   for (const raw of rawSteps) {
     const calls = [];
     for (const call of Array.isArray(raw.calls) ? raw.calls.filter((c) => c && typeof c === "object") : []) {
@@ -104,10 +110,30 @@ export function readStep(text) {
       calls.push({ id: call.id, name: call.name, args: call.args && typeof call.args === "object" && !Array.isArray(call.args) ? call.args : {} });
     }
     if (!calls.length || calls.length > MAX_CALLS) throw new Refused(400, `Every earlier step has 1 to ${MAX_CALLS} calls.`);
-    const results = {};
-    for (const result of Array.isArray(raw.results) ? raw.results : []) if (typeof result?.id === "string") results[result.id] = result;
-    steps.push({ calls, results });
+    const results = {}, shown = [];
+    for (const result of Array.isArray(raw.results) ? raw.results : []) {
+      if (typeof result?.id !== "string") continue;
+      const { pictures: given, ...rest } = result;
+      const call = calls.find((c) => c.id === result.id);
+      // A result's pictures go to the model beside it as pictures, never as text: the result names each one in its place.
+      for (const picture of Array.isArray(given) ? given : []) {
+        // `data` null is one the page no longer sends, which the model has seen.
+        if (!call || !PICTURE_TYPES.includes(picture?.media_type) || (picture.data !== null && (typeof picture.data !== "string" || !/^[A-Za-z0-9+/]+=*$/.test(picture.data)))) {
+          throw new Refused(400, "A result's pictures are each a `media_type` (JPEG, PNG, WebP or GIF) and its `data` in base64.");
+        }
+        // Numbered by the page, which names it so in the result ("[picture 2]"); a number that isn't one of its own is replaced.
+        const taken = new Set(pictures.map((p) => p.number));
+        const number = Number.isInteger(picture.number) && picture.number > 0 && !taken.has(picture.number) ? picture.number : Math.max(0, ...taken) + 1;
+        const one = { number, call: call.name, media_type: picture.media_type, data: picture.data };
+        pictures.push(one);
+        shown.push(one);
+      }
+      results[result.id] = rest;
+    }
+    steps.push({ calls, results, pictures: shown });
   }
+  // Older pictures than the newest few are left out: the model has seen them, and the step says they're no longer shown.
+  for (const old of [...pictures].sort((a, b) => a.number - b.number).slice(0, -MAX_PICTURES)) old.data = null;
   const history = (Array.isArray(body.history) ? body.history : []).filter((h) => h && typeof h === "object").slice(-3)
     .map((h) => ({ said: typeof h.said === "string" ? h.said : "", heard: typeof h.heard === "string" ? h.heard : "" }));
   const seconds = typeof body.seconds === "number" ? body.seconds : 60;
@@ -336,6 +362,16 @@ export function chatBody(r, tag, retry, model) {
   for (const step of r.steps) {
     messages.push({ role: "assistant", content: "", tool_calls: step.calls.map((call) => ({ id: call.id, type: "function", function: { name: call.name, arguments: json(call.args) } })) });
     for (const call of step.calls) messages.push({ role: "tool", tool_call_id: call.id, content: fence(tag, json(step.results[call.id] ?? skipped)) });
+    // A tool's result can't hold a picture on every provider, so the step's pictures follow its results as the person's
+    // message, each named as its result names it.
+    if (step.pictures.length) {
+      const content = [{ type: "text", text: picturesSaid(step.pictures) }];
+      for (const picture of step.pictures.filter((p) => p.data)) {
+        content.push({ type: "text", text: `[picture ${picture.number}], from ${picture.call}:` });
+        content.push({ type: "image_url", image_url: { url: `data:${picture.media_type};base64,${picture.data}` } });
+      }
+      messages.push({ role: "user", content });
+    }
   }
   if (retry) messages.push({ role: "user", content: `Your last answer couldn't be used: ${retry} Answer again.` });
   const tools = r.tools.map((tool) => ({ type: "function", function: { name: tool.name,
@@ -381,12 +417,24 @@ export function planPrompt(r, tag, retry) {
   system += "\n\n" + PLAN;
   if (r.final) system += '\nOnly a reply is possible now: answer {"reply": {...}}.';
   else system += "\n\nThe app's tools:\n" + r.tools.map((tool) => `- ${tool.name}: ${tool.reads ? "Looks up, changes nothing. " : "Changes the app's data. "}${tool.about}\n  Arguments: ${json(toolSchema(tool))}`).join("\n");
+  const images = [];
   r.steps.forEach((step, i) => {
     prompt += `\n\nYour answer ${i + 1} called:\n` + json(step.calls.map((c) => ({ name: c.name, args: c.args })));
     prompt += "\nWhat happened, in that order:\n" + fence(tag, json(step.calls.map((c) => ({ ...(step.results[c.id] ?? skipped), name: c.name }))));
+    if (step.pictures.length) prompt += "\n" + picturesSaid(step.pictures, true);
+    for (const picture of step.pictures) if (picture.data) images.push({ media_type: picture.media_type, data: picture.data });
   });
   if (retry) prompt += `\n\nYour last answer couldn't be used: ${retry} Answer again.`;
-  return { system, prompt };
+  return { system, prompt, images };
+}
+
+/** What's said of a step's pictures: which come with the message, in order, and which are no longer shown. */
+function picturesSaid(pictures, attached = false) {
+  const named = (list) => list.map((p) => `[picture ${p.number}] (from ${p.call})`).join(", ");
+  const sent = pictures.filter((p) => p.data), gone = pictures.filter((p) => !p.data);
+  let said = sent.length ? `${attached ? "Attached to this message, in this order" : "The pictures those results name"}: ${named(sent)}. Look at them before you answer.` : "";
+  if (gone.length) said += `${said ? " " : ""}No longer shown, as newer pictures came since: ${named(gone)}.`;
+  return said;
 }
 
 // MARK: Decisions (`LampAI.decide`) answered by a text model

package/cli/dev.mjs

@@ -180,7 +180,9 @@ export async function serve({ folder: given, port: asked, lan = false, quiet = f
       if (now === null) { known.delete(path); removed.push(path); }
       else { known.set(path, now); changed.push(path); }
     }
-    if (changed.length || removed.length) broadcast(changed.sort(), removed.sort());
+    // A file no page has asked for (renders or logs a page's tests write beside it, say) is nothing an open page shows,
+    // so it opens no page again: a page that asks for it later gets it as it is then.
+    if ([...changed, ...removed].some((path) => served.has(path))) broadcast(changed.sort(), removed.sort());
   }
   // Saves are gathered for a moment, so a change of several files goes to the pages as one.
   let pending = new Set(), timer = null;
@@ -691,7 +693,11 @@ export async function serve({ folder: given, port: asked, lan = false, quiet = f
       // The app's own choice of image model, which Lamp's app keeps for it (`AppModels`), read at each request as Lamp's app reads it.
       let chosen = {};
       try { chosen = JSON.parse(readFileSync(join(folder, ".lamp", "models.json"), "utf8")) ?? {}; } catch {}
-      const [status, reply] = await answerAI(path, given, undefined, { imageModel: chosen.image });
+      // A page that stopped waiting (`signal` on `LampAI.ask`) ends the request, and the answer with it.
+      const gone = new AbortController();
+      response.on("close", () => { if (!response.writableFinished) gone.abort(); });
+      const [status, reply] = await answerAI(path, given, undefined, { imageModel: chosen.image }, gone.signal);
+      if (gone.signal.aborted) return;
       return json(200, { status, body: JSON.stringify(reply) });
     }
     if (request.method === "POST" && url.pathname.startsWith("/__lamp/")) {