@columenlabs/lumen-ai 0.1.253 → 0.1.255

package/README.md

@@ -111,7 +111,7 @@ app.all('/api/assistant/*', (c) => routes(c.req.raw));
 - **Who pays.** The person's pick (the shared keys by default) while it has a key for the tier's provider and money left this month under its cap (an admin sets the shared keys' cap per person, `PUT /app`), else the other; with neither the panel says why.
 - **Tools.** Each group is Off, Read, Read + draft or Read + write per person. A tool is a read when its MCP `readOnlyHint` is set. On Read + draft a write only drafts a card, and runs as the person when they press Save, after access is checked again. A group with `readOnly: true` (read tools only) is offered Off or Read alone. Read + write is offered only for a group with `writes: true`: its writes run at once, and with `undo` set each one's line in the chat has Undo, which calls `undo(caller, { tool, args, result })`; the model is told what was undone. Cards drafted one after another show as one stack with Save all. Use `mcpServerTools` (an SDK `McpServer`), `mcpFetchTools` (a server served as a fetch handler, any SDK), `remoteMcpTools` or `plainTools`.
 - **Web.** `fetch` is a plain GET from the Worker (10 s, 2 MB, never the app's own hosts or private addresses). ScrapeQL's own tools join over its MCP API once an admin pastes a Cloudflare Access service token (`PUT /web/scrapeql`). A Worker on ScrapeQL's own Cloudflare account passes its service binding instead, `scrapeql: { url: 'https://scrapeql/mcp', fetch: env.SCRAPEQL.fetch.bind(env.SCRAPEQL) }`: no token is stored, the tools are always on and Settings shows the link with a Test. Each web step carries `web` (`StepWeb`: a search's `query` and `results`, or the page it `read`), so the panel folds a search's results under its line and numbers each page read under the answer as Sources chips; the model marks facts with those numbers ([1]).
-- **Web lookup (a subagent).** `web: { scrapeql, lookup: true }` swaps fetch and ScrapeQL's own tools for one tool, `web__lookup` (`webLookupTools` from `@columenlabs/lumen-ai/subagents`): given a question (and `urls` when the pages are known), it searches with ScrapeQL, has ScrapeQL's `ask` read the best pages (3 by default, `pages` up to 5) and answer in a few sentences each (`answerTokens`, default 300), and returns `{question, sources: [{url, title, answer}], failed}`: about a thousand tokens, never whole pages, so the assistant's context stays small. No model runs in the kit for it; ScrapeQL picks its own cheapest model. Its step carries `web.lookup` and `web.reads`, which the panel folds under a Looked up line and numbers as Sources chips. `fetch.blockHosts` still applies.
+- **Web lookup (a subagent).** `web: { scrapeql, lookup: true }` swaps fetch and ScrapeQL's own tools for one tool, `web__lookup` (`webLookupTools` from `@columenlabs/lumen-ai/subagents`): given a question (and `urls` when the pages are known), it searches with ScrapeQL, has ScrapeQL's `ask` read the best pages (3 by default, `pages` up to 5) and answer in a few sentences each (`answerTokens`, default 300), and returns `{question, sources: [{url, title, answer}], failed}`: about a thousand tokens, never whole pages, so the assistant's context stays small. No model runs in the kit for it; ScrapeQL picks its own cheapest model. Its step carries `web.lookup` and `web.reads`, which the panel folds under a Looked up line and numbers as Sources chips. `fetch.blockHosts` still applies. Until ScrapeQL is reachable (no binding and no Access token saved yet), the web falls back to `fetch` and the lookup rule is left out, so turning lookup on never takes the web away.
 - **Hand off and advice.** The `escalate` tool hands one task to a higher tier (the person sets next, any or only when asked); Anthropic's advisor tool lets a higher tier send a plan, at most two an answer. A hand-off gives up after `HAND_OFF_MS` (4 minutes) and the turn goes on without it.
 - **Chats** run on the agent (`./agent`, `LumenAgent`, below): the app routes `POST /chat` and each chat's reads, cards and Undo to its own agent, and `assistantRoutes` answers `POST /chat` with 410 `moved`. Chats saved by the old turn loop stay readable (`GET /chats`, `GET|DELETE /chats/:id`) until `assistant.purge()` drops them 30 days after their last message.
 - **Screen moves**: while the person's Settings ▸ Screen is Move (the default), the panel sends its page map (`AssistantMessage.screen`: routes, nav pages, Settings sections, cards on screen, rows to open) and `LumenAgent` gives the model a `screen` tool (`./agent` `screen.ts`). Each call's moves (go, settings, chart, open, point) are checked against the map, kept in the chat and streamed as a `screen` event; the panel runs them through `AssistantScreenProvider` (mounted by `LumenShell`) with an Undo chip, and their outcomes go back with the next message (`outcomes`). Nothing is saved by a move.
@@ -254,7 +254,7 @@ Drafts and writes are cards in the chat's own storage (`lumen_cards` in the agen
 
 **The panel.** `LumenAssistant` reads a Think chat with no change. `ask(input, sink)` runs one answer and sends the panel's own events (`AssistantEvent`: the chat, the text, each tool line, cards, hand-off sections, advice, the use) to `sink`. The app's `POST /chat` returns `askResponse((sink) => chat.ask(input, sink), { signal, stop: () => chat.stop() })`, the same server-sent events `assistantRoutes` sends. `view()` gives a stored chat as the panel's items, and `usage()` gives the chat's use so far. A card's id is its tool call's, so the panel's Save, Dismiss and Undo go to `saveCard`, `dismissCard` and `undoCard`. The app keeps the list of chats, and each chat is its own agent, such as a facet of the person's store. `chunkEvents` and `itemsOf` are the mapping, for an app that streams some other way.
 
-**Tool lines.** Each tool call shows as a line with the tool's own `title` (an MCP tool's `title`, a plain tool's `title`), kept in the chat's storage so a chat read back words its lines the same; a tool with no title shows its name. An app that keeps its insides off the screen overrides `plainSteps()` to return `true`: every line reads its tool's title, else a word for what it did (Look something up, Make a change, Draft a change), with no tool id, row count or error text (a failed line says `didn't work`), and a hand-off section reads Take a closer look with no tier or task. The details go to the app's log instead: `stepFailed({ tool, error, ms })` hears each failed call with its whole error (override it to log it with the chat's ids), and `onStep` carries `error` and `caller` on a failed tool or hand-off. `stepTitleOf(name, kind)` gives a line's title for an app's own card answers.
+**Tool lines.** Each tool call shows as a line with the tool's own `title` (an MCP tool's `title`, a plain tool's `title`), kept in the chat's storage so a chat read back words its lines the same; a tool with no title shows its name. An app that keeps its insides off the screen overrides `plainSteps()` to return `true`: every line reads its tool's title, else a word for what it did (Look something up, Make a change, Draft a change), with no tool id, row count or error text (a failed line says `didn't work`), and a hand-off section reads Take a closer look with no tier or task. The details go to the app's log instead: `stepFailed({ tool, error, ms })` hears each failed call with its whole error (override it to log it with the chat's ids), and `onStep` carries `error` and `caller` on a failed tool or hand-off. `stepTitleOf(name, kind)` gives a line's title for an app's own card answers. **Step lines say what a call did** (`stepLine(c)`): override it to word each call as a verb and what it acted on, from the call's own arguments and answer (`c.input`, `c.output`): present tense while it runs (`state: 'running'`, sent again once the arguments are in), past tense with a short result when it answered (`'done'`), and "Couldn't …" when it failed (`'failed'`, which then shows no `didn't work`). Return `undefined` to keep the tool's title. The kit words its own web lookup (Looking up / Looked up the question). A finished line is kept with the chat, so a chat read back reads the same. In a plain chat the tool search that turns more tools on shows no line.
 
 **A write in progress.** A tool line starts (`step_start`) as soon as the model starts the call, while it is still writing the arguments. For a Read + write tool it carries `write: true`, and `useAssistantChat` marks that running step `writing` until its result lands or the answer ends, so an app can show what the tool changes as being changed now (a pending edit is not ready to accept while the model rewrites it).
 

package/dist/agent/agent.d.ts

@@ -17,6 +17,7 @@ import { type ModelMessage, type ToolSet } from 'ai';
 import type { ModelDef } from '../assistant/models.js';
 import { type AgentConnector, type ConnectorAt } from './connectors.js';
 import { type ModelKey } from './models.js';
+import { type StepCall } from './stream.js';
 import { type AiToolsOptions, type OpenedTools, type SourceAt } from './tools.js';
 import { UsageMeter } from './usage.js';
 /** What a plan's extra tools get: the open tools, a meter of their own and the cap. */
@@ -197,6 +198,16 @@ export declare abstract class LumenAgent<Env = unknown> extends Think<Env> {
      * (`false`) unless the app overrides it; the app logs the details from `onStep`.
      */
     protected plainSteps(): boolean;
+    /**
+     * A tool call's line in the person's words: a verb and what it acted on, from the call's own
+     * arguments and answer. `running` is present tense ("Editing Lid"), `done` past tense with a short
+     * result when it helps ("Edited Lid: wall 2 → 3 mm"), `failed` says it did not ("Couldn't edit
+     * Lid"). Return `undefined` to keep the tool's title. A plain chat's line names no tool id, model,
+     * engine or code. The kit words its own web lookup; the app words its tools by overriding this
+     * (and may call `super.stepLine(c)` for the kit's). A finished line is kept with the chat, so a
+     * chat read back shows the same words.
+     */
+    protected stepLine(c: StepCall): string | undefined;
     /**
      * A tool call this answer failed: its name and whole error (a plain line shows neither). Does
      * nothing unless the app overrides it, e.g. to log it with the chat's ids.

package/dist/agent/agent.js

@@ -201,12 +201,45 @@ export class LumenAgent extends Think {
     plainSteps() {
         return false;
     }
+    /**
+     * A tool call's line in the person's words: a verb and what it acted on, from the call's own
+     * arguments and answer. `running` is present tense ("Editing Lid"), `done` past tense with a short
+     * result when it helps ("Edited Lid: wall 2 → 3 mm"), `failed` says it did not ("Couldn't edit
+     * Lid"). Return `undefined` to keep the tool's title. A plain chat's line names no tool id, model,
+     * engine or code. The kit words its own web lookup; the app words its tools by overriding this
+     * (and may call `super.stepLine(c)` for the kit's). A finished line is kept with the chat, so a
+     * chat read back shows the same words.
+     */
+    stepLine(c) {
+        if (c.tool !== 'web__lookup')
+            return undefined;
+        const q = c.input?.question;
+        const question = typeof q === 'string' ? q.trim() : '';
+        if (!question)
+            return undefined;
+        return c.state === 'running'
+            ? `Looking up ${question}`
+            : c.state === 'failed'
+                ? `Couldn't look up ${question}`
+                : `Looked up ${question}`;
+    }
     /** How this chat words its tool lines: plain or not, with the titles of every tool it offered. */
     #names() {
         return {
             plain: this.plainSteps(),
             titleOf: (n) => this.#turn?.tools.info.find((i) => i.name === n)?.title ??
                 this.#get('titles')?.[n],
+            lineOf: (c) => {
+                if (c.state === 'running')
+                    return this.stepLine(c);
+                const kept = this.#get(`line:${c.id}`);
+                if (kept)
+                    return kept;
+                const line = this.stepLine(c);
+                if (line)
+                    this.#put(`line:${c.id}`, line.slice(0, 200));
+                return line;
+            },
         };
     }
     /**

package/dist/agent/stream.d.ts

@@ -17,7 +17,28 @@ export interface StepNames {
     plain?: boolean;
     /** The tool's own title, when it has one. */
     titleOf?: (name: string) => string | undefined;
+    /**
+     * A line that says what this call did and to what (`LumenAgent.stepLine`): present tense while it
+     * runs, past tense with a short result when it answered, "Couldn't …" when it failed. Left out
+     * (or `undefined`), the line reads the tool's title.
+     */
+    lineOf?: (c: StepCall) => string | undefined;
 }
+/** One tool call as its line is worded: its name, arguments, answer and where it stands. */
+export interface StepCall {
+    id: string;
+    /** The tool's name as the model called it (`group__tool`). */
+    tool: string;
+    input?: unknown;
+    /** Its answer, once it answered. */
+    output?: unknown;
+    state: 'running' | 'done' | 'failed';
+}
+/**
+ * Whether a call shows no line at all: in a plain chat, the tool search that turns more tools on is
+ * the assistant finding its own tools, not something it did for the person.
+ */
+export declare function hiddenStep(name: string, names?: StepNames): boolean;
 /** What a plain line says when a call failed: the assistant reads the error and goes on. */
 export declare const PLAIN_FAILED = "didn't work";
 /** A plain hand-off's heading: a closer look, never the tier it went to. */

package/dist/agent/stream.js

@@ -1,10 +1,32 @@
 import { rowCount } from '../assistant/tools.js';
 import { webStep } from '../assistant/web.js';
+import { TOOL_SEARCH } from './search.js';
 /** A tool's name as a line's title: `notes__find_notes` → `find notes`. */
 export function stepTitle(name) {
     const own = name.includes('__') ? name.slice(name.indexOf('__') + 2) : name;
     return own.replace(/_/g, ' ');
 }
+/** A line's text kept to one short line. */
+const oneLine = (t) => {
+    const s = (t ?? '').replace(/\s+/g, ' ').trim();
+    return s ? s.slice(0, 120) : undefined;
+};
+/** The line the app gave for this call, when it gave one; never throws. */
+function lineFor(names, c) {
+    try {
+        return oneLine(names.lineOf?.(c));
+    }
+    catch {
+        return undefined;
+    }
+}
+/**
+ * Whether a call shows no line at all: in a plain chat, the tool search that turns more tools on is
+ * the assistant finding its own tools, not something it did for the person.
+ */
+export function hiddenStep(name, names = {}) {
+    return !!names.plain && name === TOOL_SEARCH;
+}
 /** What a plain line says when a call failed: the assistant reads the error and goes on. */
 export const PLAIN_FAILED = "didn't work";
 /** A plain hand-off's heading: a closer look, never the tier it went to. */
@@ -88,15 +110,27 @@ function settledCall(id, c, out, ms, cards, screens = new Map(), names = {}) {
     const image = stepImage(o?.image);
     const web = webStep(c.name, c.input, out.output, !!error || obj(out.output)?.ok === false);
     const wrote = card?.kind === 'write' || (o && typeof o.change === 'string');
+    // The app's own line says what the call did; a failed one already says it did not ("Couldn't …").
+    const line = lineFor(names, {
+        id,
+        tool: c.name,
+        input: c.input,
+        ...(error
+            ? {}
+            : { output: o && 'result' in o && o.change !== undefined ? o.result : out.output }),
+        state: error ? 'failed' : 'done',
+    });
     const step = {
         kind: 'step',
         id,
         tool: c.name,
-        title: shownTitle(c.name, names, wrote ? 'write' : 'read'),
+        title: line ?? shownTitle(c.name, names, wrote ? 'write' : 'read'),
         ok: !error,
         ms,
         ...(rows !== undefined && !error && !names.plain ? { rows } : {}),
-        ...(error ? { error: names.plain ? PLAIN_FAILED : error.slice(0, 300) } : {}),
+        ...(error && !(line && names.plain)
+            ? { error: names.plain ? PLAIN_FAILED : error.slice(0, 300) }
+            : {}),
         ...(wrote
             ? { write: { undo: true, status: card?.status === 'undone' ? 'undone' : 'done' } }
             : {}),
@@ -129,7 +163,10 @@ export function chunkEvents(o = {}) {
                     advising = true;
                     return [{ event: 'advice', data: { tier: 'advisor', label: o.advisor ?? 'Advice' } }];
                 }
-                if (name === 'escalate' || name === 'screen' || o.kindOf?.(name) === 'draft')
+                if (name === 'escalate' ||
+                    name === 'screen' ||
+                    o.kindOf?.(name) === 'draft' ||
+                    hiddenStep(name, names))
                     return [];
                 // A tool that makes a change at once says so as it starts, so an app can show that what it
                 // changes is being changed now, before the model has finished writing the call.
@@ -150,7 +187,23 @@ export function chunkEvents(o = {}) {
             case 'tool-input-available': {
                 const c = calls.get(id) ?? { name: String(chunk.toolName ?? ''), t0: Date.now() };
                 calls.set(id, { ...c, input: chunk.input });
-                return [];
+                // Its arguments are in: the running line can say what it is doing and to what.
+                if (c.name === 'advisor' ||
+                    c.name === 'escalate' ||
+                    c.name === 'screen' ||
+                    o.kindOf?.(c.name) === 'draft' ||
+                    hiddenStep(c.name, names))
+                    return [];
+                const line = lineFor(names, { id, tool: c.name, input: chunk.input, state: 'running' });
+                if (!line)
+                    return [];
+                const write = o.kindOf?.(c.name) === 'write';
+                return [
+                    {
+                        event: 'step_start',
+                        data: { id, tool: c.name, title: line, ...(write ? { write } : {}) },
+                    },
+                ];
             }
             case 'tool-output-available':
             case 'tool-output-error': {
@@ -164,6 +217,8 @@ export function chunkEvents(o = {}) {
                     return [{ event: 'advice_done', data: error ? { error } : {} }];
                 }
                 const ms = Date.now() - c.t0;
+                if (hiddenStep(c.name, names))
+                    return [];
                 const out = obj(chunk.output);
                 const said = error ?? (out && typeof out.error === 'string' ? out.error : undefined);
                 if (said !== undefined) {
@@ -256,6 +311,8 @@ export function itemsOf(messages, cards, screens = new Map(), steers = {}, names
                 items.push({ kind: 'advice', tier: 'advisor', label: 'Advice' });
                 continue;
             }
+            if (hiddenStep(tool, names))
+                continue;
             const ended = p.state === 'output-available'
                 ? { output: p.output }
                 : p.state === 'output-error'

package/dist/assistant/engine.js

@@ -118,6 +118,7 @@ function openedSummary(t) {
         reads: t.reads.length > 0,
         batch: true,
         web: t.info.some((i) => i.group === 'web'),
+        lookup: t.info.some((i) => i.name === 'web__lookup'),
     };
 }
 export class Assistant {
@@ -827,36 +828,33 @@ export class Assistant {
             for (const source of Array.isArray(s) ? s : [s])
                 sources.push({ group: g.id, access, source, ...(g.writes ? { writes: true } : {}) });
         }
-        const lookup = this.o.web?.lookup;
-        const blockHosts = this.o.web?.fetch ? this.o.web.fetch.blockHosts : undefined;
-        if ((p.web ?? true) && lookup) {
+        if (p.web ?? true) {
             const sq = this.o.web?.scrapeql;
             const token = sq && !sq.fetch ? await this.state.scrapeql() : null;
-            if (sq && (sq.fetch || token))
+            const scrapeql = sq && (sq.fetch || token) ? scrapeqlTools(sq.url, token, sq.fetch ?? this.o.fetch) : null;
+            const lookup = this.o.web?.lookup;
+            if (lookup && scrapeql) {
+                // The one lookup tool. Without ScrapeQL (no binding, no token yet) the web falls back to fetch.
+                const blockHosts = this.o.web?.fetch ? this.o.web.fetch.blockHosts : undefined;
                 sources.push({
                     group: 'web',
                     access: 'read',
-                    source: webLookupTools(scrapeqlTools(sq.url, token, sq.fetch ?? this.o.fetch), {
+                    source: webLookupTools(scrapeql, {
                         ...(blockHosts ? { blockHosts } : {}),
                         ...(typeof lookup === 'object' ? lookup : {}),
                     }),
                 });
-        }
-        else if (p.web ?? true) {
-            if (this.o.web?.fetch !== false)
-                sources.push({
-                    group: 'web',
-                    access: 'read',
-                    source: webFetchTools(this.o.web?.fetch || {}),
-                });
-            const sq = this.o.web?.scrapeql;
-            const token = sq && !sq.fetch ? await this.state.scrapeql() : null;
-            if (sq && (sq.fetch || token))
-                sources.push({
-                    group: 'web',
-                    access: 'read',
-                    source: scrapeqlTools(sq.url, token, sq.fetch ?? this.o.fetch),
-                });
+            }
+            else {
+                if (this.o.web?.fetch !== false)
+                    sources.push({
+                        group: 'web',
+                        access: 'read',
+                        source: webFetchTools(this.o.web?.fetch || {}),
+                    });
+                if (scrapeql)
+                    sources.push({ group: 'web', access: 'read', source: scrapeql });
+            }
         }
         return sources;
     }
@@ -887,7 +885,7 @@ export class Assistant {
             lines.push('No tools are on now: the person can turn them on in Settings ▸ Tools.');
         if (failed)
             lines.push(`${failed} tool source${failed === 1 ? '' : 's'} could not be reached this turn.`);
-        if (fns.web && this.o.web?.lookup)
+        if (fns.lookup)
             lines.push("Web: web__lookup answers one question from the web: give the full question (and urls when you already know the pages); it returns a short answer from each page it read. Look up again with a sharper question rather than asking for whole pages. The person sees each page a lookup read numbered under your answer, in the order returned (the first lookup's sources first, a later lookup's after them): mark a fact with its number, like [1], and never cite a page that is not among the sources.");
         else if (fns.web)
             lines.push('Web: use web__fetch first for APIs and plain pages; the browser tools only for pages that need JavaScript or after a fetch fails. To find pages, search, then read the one to three that answer the question. The person sees each page you read numbered under your answer, in the order you read them (1 for the first page read in this answer): mark a fact you took from a page with its number, like [1], and never cite a page you did not read.');

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@columenlabs/lumen-ai",
-  "version": "0.1.253",
+  "version": "0.1.255",
   "description": "Lumen's assistant and agent parts: a chat assistant's server (Claude tiers, tools over MCP, the web, hand off, spend caps), serving an MCP server from a Worker, read-only tool sets and metered tool calls.",
   "license": "MIT",
   "type": "module",
@@ -56,8 +56,8 @@
     "@ai-sdk/openai": "4.0.108",
     "@anthropic-ai/sdk": "0.133.0",
     "@cloudflare/think": "0.23.0",
-    "@columenlabs/lumen-ops": "0.1.253",
-    "@columenlabs/lumen-ui-types": "0.1.253",
+    "@columenlabs/lumen-ops": "0.1.255",
+    "@columenlabs/lumen-ui-types": "0.1.255",
     "agents": "0.32.0",
     "ai": "7.0.162",
     "just-bash": "3.6.0",