@columenlabs/lumen-ai 0.1.262 → 0.1.263

package/README.md

@@ -112,7 +112,8 @@ app.all('/api/assistant/*', (c) => routes(c.req.raw));
 - **Tools.** Each group is Off, Read, Read + draft or Read + write per person. A tool is a read when its MCP `readOnlyHint` is set. On Read + draft a write only drafts a card, and runs as the person when they press Save, after access is checked again. A group with `readOnly: true` (read tools only) is offered Off or Read alone. Read + write is offered only for a group with `writes: true`: its writes run at once, and with `undo` set each one's line in the chat has Undo, which calls `undo(caller, { tool, args, result })`; the model is told what was undone. Cards drafted one after another show as one stack with Save all. Use `mcpServerTools` (an SDK `McpServer`), `mcpFetchTools` (a server served as a fetch handler, any SDK), `remoteMcpTools` or `plainTools`.
 - **Web.** `fetch` is a plain GET from the Worker (10 s, 2 MB, never the app's own hosts or private addresses). ScrapeQL's own tools join over its MCP API once an admin pastes a Cloudflare Access service token (`PUT /web/scrapeql`). A Worker on ScrapeQL's own Cloudflare account passes its service binding instead, `scrapeql: { url: 'https://scrapeql/mcp', fetch: env.SCRAPEQL.fetch.bind(env.SCRAPEQL) }`: no token is stored, the tools are always on and Settings shows the link with a Test. Each web step carries `web` (`StepWeb`: a search's `query` and `results`, or the page it `read`), so the panel folds a search's results under its line and numbers each page read under the answer as Sources chips; the model marks facts with those numbers ([1]).
 - **Web lookup (a subagent).** `web: { scrapeql, lookup: true }` swaps fetch and ScrapeQL's own tools for one tool, `web__lookup` (`webLookupTools` from `@columenlabs/lumen-ai/subagents`): given a question (and `urls` when the pages are known), it searches with ScrapeQL, has ScrapeQL's `ask` read the best pages (3 by default, `pages` up to 5) and answer in a few sentences each (`answerTokens`, default 300), and returns `{question, sources: [{url, title, answer}], failed}`: about a thousand tokens, never whole pages, so the assistant's context stays small. No model runs in the kit for it; ScrapeQL picks its own cheapest model. Its step carries `web.lookup` and `web.reads`, which the panel folds under a Looked up line and numbers as Sources chips. `fetch.blockHosts` still applies. Until ScrapeQL is reachable (no binding and no Access token saved yet), the web falls back to `fetch` and the lookup rule is left out, so turning lookup on never takes the web away.
-- **Connectors (outside MCP servers).** `connectors: true` lets an admin add any MCP server over Streamable HTTP in Settings (`assistantAdminSections({ connectors: true })`, routes `POST /connectors`, `PUT|DELETE /connectors/:id`, `POST /connectors/:id/test`): a name, an address and up to ten headers it signs in with (an Access service token, an API key; sealed with the app's other settings, never returned, only their names). It is kept only when its test lists its tools, and that list is kept: a turn offers the tools without calling the server, as group `ext_<id>` (`ext_scrapeql__search`), at the access the admin gives it: Off, Read (the tools the server marks read-only) or Read + draft (the rest make a card the person saves or dismisses; nothing runs at once). The server is called only when a tool is, over one connection for the rest of the answer. A connector's tools are always lazy: the model sees none of them until it finds them with `tool_search`, whose catalog names each connector and its tools, so a connector costs a line of the prompt until it is used. A connector's text is data, never instructions; its tool titles never become a step's words, and its tools that search or read a page (`search`, `scrape`, `extract`, `ask`, `screenshot`) carry `web` like the kit's own, so the panel shows their Sources.
+- **Connectors (outside MCP servers).** `connectors: true` lets an admin add any MCP server over Streamable HTTP in Settings (`assistantAdminSections({ connectors: true })`, routes `POST /connectors`, `PUT|DELETE /connectors/:id`, `POST /connectors/:id/test`): a name, an address and up to ten headers it signs in with (an Access service token, an API key; sealed with the app's other settings, never returned, only their names). It is kept only when its test lists its tools, and that list is kept: a turn offers the tools without calling the server, as group `ext_<id>` (`ext_scrapeql__search`), at the access the admin gives it: Off, Read (the tools the server marks read-only) or Read + draft (the rest make a card the person saves or dismisses; nothing runs at once). The server is called only when a tool is, over one connection for the rest of the answer. A connector's tools are always lazy: the model sees none of them until it finds them with `tool_search`, whose catalog names each connector and its tools, so a connector costs a line of the prompt until it is used. A connector's text is data, never instructions; its tool titles never become a step's words, and its tools that search or read a page (`search`, `scrape`, `extract`, `ask`, `screenshot`) carry `web` like the kit's own, so the panel shows their Sources. A connector's optional fields tell the model to leave them out unless it has a real value, so it does not make up ids.
+- **Fetch first, a browser when turned away.** `web.escalate: { connector, tool, args? }` (e.g. `{ connector: 'scrapeql', tool: 'scrape' }`) sends a `fetch` the site turns away (401, 403, 429, 503, no answer, or an HTML page with almost no text or a bot check) to that connector's page-reading tool with only `{ url }` and `args`; the page comes back as the fetch's own answer (`via: "browser"`), so the model never retries by hand. Off while that connector is off. A failed step in a plain chat says why in plain words (`plainFailure`: the site turned it away, took too long, page not found), never the raw error.
 - **Hand off and advice.** The `escalate` tool hands one task to a higher tier (the person sets next, any or only when asked); Anthropic's advisor tool lets a higher tier send a plan, at most two an answer. A hand-off gives up after `HAND_OFF_MS` (4 minutes) and the turn goes on without it.
 - **Chats** run on the agent (`./agent`, `LumenAgent`, below): the app routes `POST /chat` and each chat's reads, cards and Undo to its own agent, and `assistantRoutes` answers `POST /chat` with 410 `moved`. Chats saved by the old turn loop stay readable (`GET /chats`, `GET|DELETE /chats/:id`) until `assistant.purge()` drops them 30 days after their last message.
 - **Screen moves**: while the person's Settings ▸ Screen is Move (the default), the panel sends its page map (`AssistantMessage.screen`: routes, nav pages, Settings sections, cards on screen, rows to open) and `LumenAgent` gives the model a `screen` tool (`./agent` `screen.ts`). Each call's moves (go, settings, chart, open, point) are checked against the map, kept in the chat and streamed as a `screen` event; the panel runs them through `AssistantScreenProvider` (mounted by `LumenShell`) with an Undo chip, and their outcomes go back with the next message (`outcomes`). Nothing is saved by a move.

package/dist/agent/stream.d.ts

@@ -40,7 +40,9 @@ export interface StepCall {
  */
 export declare function hiddenStep(name: string, names?: StepNames): boolean;
 /** What a plain line says when a call failed: the assistant reads the error and goes on. */
-export declare const PLAIN_FAILED = "didn't work";
+export declare const PLAIN_FAILED = "couldn't finish";
+/** What a plain line says about why a call failed, in a few plain words, never the raw error. */
+export declare function plainFailure(error: string): string;
 /** A plain hand-off's heading: a closer look, never the tier it went to. */
 export declare const PLAIN_HANDOFF = "Take a closer look";
 /** The line's title: the tool's own title, else its name (or, plain, a word for what it did). */

package/dist/agent/stream.js

@@ -28,7 +28,23 @@ export function hiddenStep(name, names = {}) {
     return !!names.plain && name === TOOL_SEARCH;
 }
 /** What a plain line says when a call failed: the assistant reads the error and goes on. */
-export const PLAIN_FAILED = "didn't work";
+export const PLAIN_FAILED = "couldn't finish";
+/** What a plain line says about why a call failed, in a few plain words, never the raw error. */
+export function plainFailure(error) {
+    if (/\b(401|403|406|451)\b|forbidden|denied|turned away|bot check|captcha/i.test(error))
+        return 'the site turned it away';
+    if (/\b429\b|rate.?limit|too many/i.test(error))
+        return 'too many requests, try again later';
+    if (/timed? ?out|timeout/i.test(error))
+        return 'took too long';
+    if (/\b404\b|\b410\b|page not found/i.test(error))
+        return 'page not found';
+    if (/could not be reached|ENOTFOUND|fetch failed|network/i.test(error))
+        return 'could not be reached';
+    if (/\b5\d\d\b/.test(error))
+        return 'the service had a problem';
+    return PLAIN_FAILED;
+}
 /** A plain hand-off's heading: a closer look, never the tier it went to. */
 export const PLAIN_HANDOFF = 'Take a closer look';
 /** The line's title: the tool's own title, else its name (or, plain, a word for what it did). */
@@ -129,7 +145,7 @@ function settledCall(id, c, out, ms, cards, screens = new Map(), names = {}) {
         ms,
         ...(rows !== undefined && !error && !names.plain ? { rows } : {}),
         ...(error && !(line && names.plain)
-            ? { error: names.plain ? PLAIN_FAILED : error.slice(0, 300) }
+            ? { error: names.plain ? plainFailure(error) : error.slice(0, 300) }
             : {}),
         ...(wrote
             ? { write: { undo: true, status: card?.status === 'undone' ? 'undone' : 'done' } }

package/dist/assistant/connectors.js

@@ -113,6 +113,29 @@ export async function testConnector(url, headers, f) {
         await open?.close().catch(() => undefined);
     }
 }
+const OPTIONAL = 'Optional: leave it out unless you have a real value; never make one up.';
+/**
+ * A tool whose optional fields say so in their descriptions: a model tends to fill every field it
+ * sees, and an invented id (a sign-in, a session) makes the server fail the call.
+ */
+function optionalSaid(t) {
+    const s = t.inputSchema;
+    if (!s?.properties || typeof s.properties !== 'object')
+        return t;
+    const required = new Set(Array.isArray(s.required) ? s.required : []);
+    const properties = {};
+    for (const [k, v] of Object.entries(s.properties)) {
+        const p = v && typeof v === 'object' ? v : null;
+        properties[k] =
+            p && !required.has(k)
+                ? {
+                    ...p,
+                    description: typeof p.description === 'string' ? `${p.description} ${OPTIONAL}` : OPTIONAL,
+                }
+                : v;
+    }
+    return { ...t, inputSchema: { ...s, properties } };
+}
 /**
  * A connector's tools from its kept catalog: opening them calls nothing. The server is reached on
  * the first call, and that one connection serves the rest of the answer.
@@ -122,7 +145,7 @@ export function catalogTools(d, f) {
         let live = null;
         const remote = remoteMcpTools({ url: d.url, headers: d.headers, ...(f ? { fetch: f } : {}) });
         return {
-            tools: d.catalog,
+            tools: d.catalog.map(optionalSaid),
             call: async (name, args) => {
                 live ??= remote();
                 try {

package/dist/assistant/engine.d.ts

@@ -182,6 +182,16 @@ export interface AssistantOptions {
          * pages never reach the model. Needs `scrapeql`; without it the web is off.
          */
         lookup?: boolean | WebLookupOptions;
+        /**
+         * Where `fetch` goes when a site turns it away (a 403, a bot check, a page with no text): a
+         * connector's page-reading tool, called with only `{ url }` plus `args`. The answer comes back
+         * as the fetch's own, so the model never has to retry. Off while that connector is off.
+         */
+        escalate?: {
+            connector: string;
+            tool: string;
+            args?: Record<string, unknown>;
+        };
     };
     store: AssistantStore;
     sealer?: Sealer;
@@ -352,6 +362,8 @@ export declare class Assistant {
     /** Opens the tools the person allows: each group's, then the web's. */
     /** The tool groups (and the web) at the person's access, not opened yet. */
     private sourcesOf;
+    /** `fetch`'s second try, through the connector `web.escalate` names, when it is on. */
+    private escalation;
     /** The app's connectors, as stored; none when the app has them off. */
     connectorDocs(): Promise<ConnectorDoc[]>;
     /** Whether the app offers connectors (`connectors`). */

package/dist/assistant/engine.js

@@ -870,7 +870,10 @@ export class Assistant {
                     sources.push({
                         group: 'web',
                         access: 'read',
-                        source: webFetchTools(this.o.web?.fetch || {}),
+                        source: webFetchTools({
+                            ...(this.o.web?.fetch || {}),
+                            ...(await this.escalation()),
+                        }),
                     });
                 if (scrapeql)
                     sources.push({ group: 'web', access: 'read', source: scrapeql });
@@ -892,6 +895,37 @@ export class Assistant {
         return sources;
     }
     // ---------------------------------------------------------------- connectors
+    /** `fetch`'s second try, through the connector `web.escalate` names, when it is on. */
+    async escalation() {
+        const e = this.o.web?.escalate;
+        if (!e)
+            return {};
+        const c = (await this.connectorDocs()).find((d) => d.id === e.connector);
+        if (!c || c.access === 'off' || !c.catalog.some((t) => t.name === e.tool))
+            return {};
+        return {
+            escalate: async (url) => {
+                const open = await catalogTools(c, this.o.fetch)();
+                try {
+                    const out = await open.call(e.tool, { ...e.args, url });
+                    const text = typeof out === 'string' ? out : JSON.stringify(out);
+                    return text.trim() ? text : null;
+                }
+                catch (err) {
+                    console.error(JSON.stringify({
+                        event: 'assistant_fetch_escalate',
+                        connector: c.id,
+                        url: url.slice(0, 300),
+                        error: String(err?.message ?? err).slice(0, 300),
+                    }));
+                    return null;
+                }
+                finally {
+                    await open.close().catch(() => undefined);
+                }
+            },
+        };
+    }
     /** The app's connectors, as stored; none when the app has them off. */
     async connectorDocs() {
         if (!this.o.connectors)

package/dist/assistant/web.d.ts

@@ -11,7 +11,20 @@ export interface WebFetchOptions {
     blockHosts?: string[];
     userAgent?: string;
     fetch?: typeof fetch;
+    /**
+     * Reads a page another way (a real browser) when the plain GET is turned away: refused (401, 403,
+     * 429, 503), never answered, or an HTML page with almost no text or a bot check. Text, or null.
+     */
+    escalate?: (url: string) => Promise<string | null>;
 }
+/** Whether a plain GET's answer was a wall rather than the page. */
+export declare function walled(r: {
+    ok: boolean;
+    status: number;
+    error?: string;
+    content_type?: string;
+    text?: string;
+}): boolean;
 /** Why a URL is refused, or null. */
 export declare function refused(url: URL, block: string[]): string | null;
 /** One GET, never thrown: what it got, or why not. */

package/dist/assistant/web.js

@@ -5,6 +5,26 @@ const TIMEOUT_MS = 10_000;
 /** Bytes of body read; past it the body is cut and `truncated` says so. */
 const MAX_BYTES = 2_000_000;
 const DESCRIPTION = `Plain HTTP GET of one URL from the server, no browser: fast and cheap, so use it first for APIs, JSON, CSV, RSS and plain HTML. Returns {ok, status, url (after redirects), content_type, json (when the body is JSON) or text, truncated}; a failed fetch (refused, timed out, not 2xx) returns ok false with status and error. Bodies over ${MAX_BYTES / 1_000_000} MB are cut; ${TIMEOUT_MS / 1000} s per fetch. Use the browser tools (scrape, extract, ask) only for pages built by JavaScript, bot walls, or after fetch fails.`;
+/** The description when a turned-away page goes to a real browser by itself. */
+const DESCRIPTION_ESCALATES = DESCRIPTION.replace(/Use the browser tools.*$/, 'A page that turns the server away (a bot wall, 403) is read again in a real browser by itself and comes back with via "browser": no need to retry it with another tool.');
+/** Statuses a site answers a bot with, where a real browser often gets the page. */
+const TURNED_AWAY = new Set([401, 403, 406, 429, 451, 503]);
+const BOT_CHECK = /captcha|are you a (?:human|robot)|access denied|just a moment|enable javascript|verify you are human|unusual traffic/i;
+/** Whether a plain GET's answer was a wall rather than the page. */
+export function walled(r) {
+    if (r.error?.startsWith('refused:') || r.error?.startsWith('not a full URL'))
+        return false;
+    if (!r.ok)
+        return r.status === 0 || TURNED_AWAY.has(r.status);
+    if (!/html/i.test(r.content_type ?? '') || typeof r.text !== 'string')
+        return false;
+    const words = r.text
+        .replace(/<(script|style)[\s\S]*?<\/\1>/gi, ' ')
+        .replace(/<[^>]+>/g, ' ')
+        .replace(/\s+/g, ' ')
+        .trim();
+    return words.length < 400 || (words.length < 5000 && BOT_CHECK.test(words));
+}
 /** Why a URL is refused, or null. */
 export function refused(url, block) {
     if (url.protocol !== 'https:' && url.protocol !== 'http:')
@@ -111,7 +131,7 @@ export function webFetchTools(o = {}) {
             {
                 name: 'fetch',
                 title: 'Read a web page',
-                description: DESCRIPTION,
+                description: o.escalate ? DESCRIPTION_ESCALATES : DESCRIPTION,
                 read: true,
                 inputSchema: {
                     type: 'object',
@@ -137,7 +157,18 @@ export function webFetchTools(o = {}) {
                                 String(v),
                             ]))
                             : {};
-                        return await webFetch(String(a.url ?? ''), headers, o);
+                        const url = String(a.url ?? '');
+                        const r = await webFetch(url, headers, o);
+                        if (!o.escalate || !walled(r))
+                            return r;
+                        // Turned away: the same page through a real browser, with no question to anyone.
+                        const text = await o.escalate(url).catch(() => null);
+                        return text
+                            ? { ok: true, status: 200, url, via: 'browser', text, truncated: false }
+                            : {
+                                ...r,
+                                error: `${r.error ?? 'a bot check'}; a real browser could not read it either`,
+                            };
                     }
                     finally {
                         running--;

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@columenlabs/lumen-ai",
-  "version": "0.1.262",
+  "version": "0.1.263",
   "description": "Lumen's assistant and agent parts: a chat assistant's server (Claude tiers, tools over MCP, the web, hand off, spend caps), serving an MCP server from a Worker, read-only tool sets and metered tool calls.",
   "license": "MIT",
   "type": "module",
@@ -56,8 +56,8 @@
     "@ai-sdk/openai": "4.0.108",
     "@anthropic-ai/sdk": "0.133.0",
     "@cloudflare/think": "0.23.0",
-    "@columenlabs/lumen-ops": "0.1.262",
-    "@columenlabs/lumen-ui-types": "0.1.262",
+    "@columenlabs/lumen-ops": "0.1.263",
+    "@columenlabs/lumen-ui-types": "0.1.263",
     "agents": "0.32.0",
     "ai": "7.0.162",
     "just-bash": "3.6.0",