@columenlabs/lumen-ai 0.1.186 → 0.1.188

package/README.md

@@ -98,10 +98,10 @@ app.all('/api/assistant/*', (c) => routes(c.req.raw));
 - **Keys.** Each provider has its own keys (`?provider=anthropic|openai` on the key routes). An admin pastes the shared key everyone uses (`PUT /key`); it is tested with one free call before it is kept, then sealed, and no route returns it. A person may add their own (`PUT /my-key`), with one monthly cap on their own keys (`PUT /my-key/cap`).
 - **ChatGPT sign-in.** In place of their own OpenAI key a person may sign in with their ChatGPT plan (`POST /my-signin`, then `POST /my-signin/finish {url}`): the sign-in opens in a new tab and ends on `http://127.0.0.1:1455/auth/callback`, a page that does not load, whose address the person pastes back. The tokens are sealed as their own key and refreshed before they run out; answers on the plan count tokens and no dollars.
 - **Who pays.** The person's pick (the shared keys by default) while it has a key for the tier's provider and money left this month under its cap (an admin sets the shared keys' cap per person, `PUT /app`), else the other; with neither the panel says why.
-- **Tools.** Each group is Off, Read, Read + draft or Read + write per person. A tool is a read when its MCP `readOnlyHint` is set. On Read + draft a write only drafts a card, and runs as the person when they press Save, after access is checked again. A group with `readOnly: true` (read tools only) is offered Off or Read alone. Read + write is offered only for a group with `writes: true`: its writes run at once, and with `undo` set each one's line in the chat has Undo, which calls `undo(caller, { tool, args, result })` (`POST /chats/:id/writes/:step/undo`); the model is told what was undone. Cards drafted one after another show as one stack with Save all. Use `mcpServerTools` (an SDK `McpServer`), `mcpFetchTools` (a server served as a fetch handler, any SDK), `remoteMcpTools` or `plainTools`.
+- **Tools.** Each group is Off, Read, Read + draft or Read + write per person. A tool is a read when its MCP `readOnlyHint` is set. On Read + draft a write only drafts a card, and runs as the person when they press Save, after access is checked again. A group with `readOnly: true` (read tools only) is offered Off or Read alone. Read + write is offered only for a group with `writes: true`: its writes run at once, and with `undo` set each one's line in the chat has Undo, which calls `undo(caller, { tool, args, result })`; the model is told what was undone. Cards drafted one after another show as one stack with Save all. Use `mcpServerTools` (an SDK `McpServer`), `mcpFetchTools` (a server served as a fetch handler, any SDK), `remoteMcpTools` or `plainTools`.
 - **Web.** `fetch` is a plain GET from the Worker (10 s, 2 MB, never the app's own hosts or private addresses). ScrapeQL's own tools join over its MCP API once an admin pastes a Cloudflare Access service token (`PUT /web/scrapeql`). A Worker on ScrapeQL's own Cloudflare account passes its service binding instead, `scrapeql: { url: 'https://scrapeql/mcp', fetch: env.SCRAPEQL.fetch.bind(env.SCRAPEQL) }`: no token is stored, the tools are always on and Settings shows the link with a Test.
 - **Hand off and advice.** The `escalate` tool hands one task to a higher tier (the person sets next, any or only when asked); Anthropic's advisor tool lets a higher tier send a plan, at most two an answer.
-- **Chats** stream as server-sent events (`POST /chat`); closing the request stops the answer and keeps what was said. Past 150k tokens, or on `/compact`, the history sent to the model becomes a summary. Run `assistant.purge()` once a day to drop chats 30 days after their last message.
+- **Chats** run on the agent (`./agent`, `LumenAgent`, below): the app routes `POST /chat` and each chat's reads, cards and Undo to its own agent, and `assistantRoutes` answers `POST /chat` with 410 `moved`. Chats saved by the old turn loop stay readable (`GET /chats`, `GET|DELETE /chats/:id`) until `assistant.purge()` drops them 30 days after their last message.
 - **Background work** an app runs for a person (a score, a summary) is `assistant.complete(caller, { task, system, prompt, images?, maxTokens })`: one answer on the task's tier (`tasks`, so the person picks its model in Settings ▸ Models), no chat and no tools (`images`, base64 with their media type, are read with the prompt: a scanned page, a screenshot), billed to their payer and counted in their spend and allowance like a chat turn. It throws `AssistantError` `not_connected`, `app_cap` or `own_cap` when nobody can pay, so the caller waits and tries later.
 
 ### Commands, `@` and `#`
@@ -220,11 +220,12 @@ ctx.waitUntil(meaningStep({ def: notesFind, sql: d1Sql(env.DB), embed, index: en
 | `LumenAgent` | An abstract Think agent. The app answers `plan(turn)` (model, key, effort, instructions, tool sources, money left) and hears `settle(end)` (outcome, usage parts, context size, time) when the turn ends |
 | `aiModel(model, key, opts?)` | The AI SDK model for a tier on an API key, or on a ChatGPT plan sign-in (`key.plan`, through `planFetch`) |
 | `planFetch(fetch?, name?)` | Rewrites Responses calls to a plan sign-in's rules: tools in one namespace, no output cap, a forced call as the only tool |
+| `planFallback(primary, fallback, onSwitch)` | A plan's model that hands the call, and the rest of the turn, to `fallback` when the plan reaches its usage limit (a 429 or a usage code, before any output) |
 | `turnSettings(model, opts)` | Output cap and effort as the provider takes them |
 | `openAiTools(sources, opts)` | The app's tool groups as AI SDK tools: reads always, drafts and writes by access, results cut at 16k characters |
 | `UsageMeter`, `tokenUsage(u)` | Prices each step's tokens (cache reads and writes, reasoning) into one usage part |
 
-A ChatGPT plan sign-in is free to the app (`priced` with `free`). Nothing here holds a key or a token: the app passes them per turn.
+A ChatGPT plan sign-in is free to the app (`priced` with `free`). When the plan is used up for now, a turn with `fallback` (`agentTurn` sets the app's first Claude on the same payer's Anthropic key) answers on Claude instead of ending in the limit's error, its use its own part. Nothing here holds a key or a token: the app passes them per turn.
 
 An app with an `Assistant` gets the whole plan from it: `assistant.agentTurn(caller, { message, tier?, context?, refs?, time_zone? })` returns `{ plan, settle }` with the person's tier and model on whoever pays, the tool groups at their access, the kit's rules, the bearings and the page before the message, `escalate` to a higher tier (its own usage part, inside the same cap) and Claude's advisor. `settle` adds the spend to the payer's month and reports the turn through `onStep`. It throws `AssistantError` when nobody can pay.
 

package/dist/agent/agent.d.ts

@@ -61,6 +61,15 @@ export interface AgentTurnPlan {
         maxUses: number;
         maxTokens: number;
     };
+    /**
+     * Who answers when the ChatGPT plan `key` signs in with reaches its usage limit: a Claude model
+     * on the payer's own key, for the rest of the turn. Its use is its own part (`label`).
+     */
+    fallback?: {
+        model: ModelDef;
+        key: ModelKey;
+        label: string;
+    };
     /** Dollars the payer has left; at or past it the model gets no more tools. */
     left?: number;
     /** Most model calls in one answer; 8. */

package/dist/agent/agent.js

@@ -16,7 +16,7 @@ import { Think, } from '@cloudflare/think';
 import { createCompactFunction } from 'agents/sessions';
 import { generateText } from 'ai';
 import { CONNECTOR_GROUP, checkConnectorUrl, connectorGroup, connectorId, connectorSource, } from './connectors.js';
-import { aiModel, turnSettings } from './models.js';
+import { aiModel, planFallback, turnSettings } from './models.js';
 import { LAZY_AT, TOOL_SEARCH, lazyTools } from './search.js';
 import { chunkEvents, itemsOf } from './stream.js';
 import { openAiTools, } from './tools.js';
@@ -348,12 +348,15 @@ export class LumenAgent extends Think {
         if (!last)
             throw new Error('No model to compact with until the next answer.');
         const { plan } = last;
-        const meter = new UsageMeter(plan.model, `${plan.label} · compact`, !!plan.key.plan);
+        let meter = new UsageMeter(plan.model, `${plan.label} · compact`, !!plan.key.plan);
         this.#owed.push(meter);
         const r = await generateText({
-            model: aiModel(plan.model, plan.key, {
-                ...(plan.fetch ? { fetch: plan.fetch } : {}),
-                name: this.constructor.name,
+            model: this.#model(plan, () => {
+                const fb = plan.fallback;
+                if (!fb)
+                    return;
+                meter = new UsageMeter(fb.model, `${fb.label} · compact`, false);
+                this.#owed.push(meter);
             }),
             prompt,
             ...turnSettings(plan.model, { effort: 'default', out: 4000, plan: !!plan.key.plan }),
@@ -361,6 +364,14 @@ export class LumenAgent extends Think {
         meter.add(r.usage);
         return r.text;
     }
+    /** The turn's model; on a ChatGPT plan with a fallback, Claude once the plan's limit is reached. */
+    #model(plan, onSwitch) {
+        const o = { ...(plan.fetch ? { fetch: plan.fetch } : {}), name: this.constructor.name };
+        const model = aiModel(plan.model, plan.key, o);
+        if (!plan.key.plan || !plan.fallback)
+            return model;
+        return planFallback(model, aiModel(plan.fallback.model, plan.fallback.key, o), onSwitch);
+    }
     /** Never read: `beforeTurn` gives every turn its own model. */
     getModel() {
         return '@cf/none';
@@ -442,10 +453,15 @@ export class LumenAgent extends Think {
             this.#turn.system = system;
             this.#turn.lazy = lazy;
         }
+        const turn = this.#turn;
         return {
-            model: aiModel(plan.model, plan.key, {
-                ...(plan.fetch ? { fetch: plan.fetch } : {}),
-                name: this.constructor.name,
+            model: this.#model(plan, () => {
+                const fb = plan.fallback;
+                if (!fb)
+                    return;
+                // Claude answers the rest of the turn on the payer's key: its use is its own part.
+                turn.main = new UsageMeter(fb.model, fb.label, false);
+                turn.meters.push(turn.main);
             }),
             instructions: system,
             messages: withLead(ctx.messages, `${this.#news()}${plan.lead ?? ''}`),

package/dist/agent/models.d.ts

@@ -33,3 +33,12 @@ export declare function turnSettings(model: ModelDef, o: {
     maxOutputTokens?: number;
     providerOptions?: Record<string, Record<string, JSONValue>>;
 };
+/** A ChatGPT plan used up for now: a 429 or a usage code, thrown or sent before any output. */
+export declare function planLimited(err: unknown): boolean;
+/**
+ * `primary` until its ChatGPT plan reaches its usage limit, then `fallback` for that call and
+ * every later one on this model (the rest of the turn). The limit shows either as a thrown error
+ * or as an error part before any output; once output has started nothing is swapped. `onSwitch`
+ * runs once, at the swap.
+ */
+export declare function planFallback(primary: LanguageModel, fallback: LanguageModel, onSwitch: () => void): LanguageModel;

package/dist/agent/models.js

@@ -101,3 +101,101 @@ export function turnSettings(model, o) {
         ? { ...out, providerOptions: { openai } }
         : out;
 }
+/** A ChatGPT plan used up for now: a 429 or a usage code, thrown or sent before any output. */
+export function planLimited(err) {
+    const e = err;
+    if (!e)
+        return false;
+    if (e.statusCode === 429 || e.status === 429)
+        return true;
+    const text = `${e.code ?? ''} ${e.message ?? ''} ${e.responseBody ?? ''} ${typeof err === 'string' ? err : ''}`;
+    return /usage_limit|usage limit|usage_unavailable|insufficient_quota/i.test(text);
+}
+/** Parts that come before an answer starts; an error among them means nothing was said yet. */
+const PREAMBLE = new Set(['stream-start', 'response-metadata']);
+/**
+ * `primary` until its ChatGPT plan reaches its usage limit, then `fallback` for that call and
+ * every later one on this model (the rest of the turn). The limit shows either as a thrown error
+ * or as an error part before any output; once output has started nothing is swapped. `onSwitch`
+ * runs once, at the swap.
+ */
+export function planFallback(primary, fallback, onSwitch) {
+    const p = primary;
+    const f = fallback;
+    let on = p;
+    const swap = () => {
+        if (on === f)
+            return;
+        on = f;
+        onSwitch();
+    };
+    const model = {
+        specificationVersion: p.specificationVersion,
+        provider: p.provider,
+        modelId: p.modelId,
+        get supportedUrls() {
+            return on.supportedUrls;
+        },
+        async doGenerate(o) {
+            if (on === f)
+                return f.doGenerate(o);
+            try {
+                return await p.doGenerate(o);
+            }
+            catch (err) {
+                if (!planLimited(err))
+                    throw err;
+                swap();
+                return f.doGenerate(o);
+            }
+        },
+        async doStream(o) {
+            if (on === f)
+                return f.doStream(o);
+            let r;
+            try {
+                r = await p.doStream(o);
+            }
+            catch (err) {
+                if (!planLimited(err))
+                    throw err;
+                swap();
+                return f.doStream(o);
+            }
+            // Hold the preamble until the first real part shows whether the plan answered.
+            const reader = r.stream.getReader();
+            const held = [];
+            for (;;) {
+                const { done, value } = await reader.read();
+                if (done)
+                    break;
+                if (value.type === 'error' && planLimited(value.error)) {
+                    await reader.cancel().catch(() => undefined);
+                    swap();
+                    return f.doStream(o);
+                }
+                held.push(value);
+                if (!PREAMBLE.has(value.type))
+                    break;
+            }
+            const stream = new ReadableStream({
+                start(c) {
+                    for (const v of held)
+                        c.enqueue(v);
+                },
+                async pull(c) {
+                    const { done, value } = await reader.read();
+                    if (done)
+                        c.close();
+                    else
+                        c.enqueue(value);
+                },
+                cancel(reason) {
+                    return reader.cancel(reason);
+                },
+            });
+            return { ...r, stream };
+        },
+    };
+    return model;
+}

package/dist/assistant/engine.d.ts

@@ -1,4 +1,4 @@
-import type { AllowanceRequest, AllowanceState, AllowanceUnit, AssistantAccess, AssistantChat, AssistantChatRow, AssistantMessage, AssistantOwnWay, AssistantProvider, AssistantStatus, DraftItem, StepItem } from '@columenlabs/lumen-ui-types/assistant';
+import type { AllowanceRequest, AllowanceState, AllowanceUnit, AssistantAccess, AssistantChat, AssistantChatRow, AssistantMessage, AssistantOwnWay, AssistantProvider, AssistantStatus } from '@columenlabs/lumen-ui-types/assistant';
 import type { AgentTurnEnd, AgentTurnPlan } from '../agent/agent.js';
 import { type BearingsExtra, type BearingsNow } from './bearings.js';
 import { type ModelDef, type TaskDef, type TierDef } from './models.js';
@@ -173,7 +173,6 @@ export declare class AssistantError extends Error {
     readonly status: number;
     constructor(message: string, code: string, status?: number);
 }
-type Emit = (event: string, data: unknown) => void;
 /** What a Think turn calls this turn: what one turn asks besides the message itself. */
 export interface AgentTurnInput {
     message: string;
@@ -262,37 +261,17 @@ export declare class Assistant {
     removeChat(caller: Caller, id: string): Promise<void>;
     /** Deletes chats past `keepDays` since their last message; run it once a day. */
     purge(): Promise<number>;
-    private writeChat;
     /** Opens the tools the person allows: each group's, then the web's. */
     /** The tool groups (and the web) at the person's access, not opened yet. */
     private sourcesOf;
-    private openTools;
-    private toolsFor;
-    /** `batch`: any of the reads, up to BATCH_MAX at once, as one step. Null with no reads. */
-    private batchTool;
-    /** One answer as server-sent events. Abort `signal` to stop it. */
-    turnStream(caller: Caller, req: AssistantMessage, signal?: AbortSignal): ReadableStream<Uint8Array>;
-    turn(caller: Caller, req: AssistantMessage, emit: Emit, signal?: AbortSignal): Promise<void>;
     /** What the kit adds to the app's instructions: tools, drafts, hand off, advice, limits. */
     private rules;
     /** The advisor for this answer, or null when off. Never weaker than the model it advises. */
     private advisorFor;
-    private escalateTool;
     /** What the model is asked: an app command's prompt, else the message as typed. */
     private expand;
     /** The tiers the person's message asks for by name ("go deeper" is the next one up). */
     private askedFor;
-    private handOff;
-    /** One tool call: a draft card for a write, else the call itself, timed. */
-    private runTool;
-    /** A batch of reads: each runs as its own call (the same read twice runs once), in parallel, shown as one step. */
-    private runBatch;
-    /** Model calls and tool rounds until the model answers; returns its words. */
-    private loop;
-    /** The loop on OpenAI's Responses API: the same tools, Stop, caps and rounds as Claude's. */
-    private loopOpenAI;
-    /** The loop on Anthropic's Messages API, with the advisor and prompt caching. */
-    private loopClaude;
     /** Swaps the history for a short summary. False with nothing to compact. */
     /**
      * One plain answer from OpenAI, metered on `s`: no tools, no chat. Streamed and read from the raw
@@ -300,7 +279,6 @@ export declare class Assistant {
      * output cap and may send its words only as deltas.
      */
     private openaiOnce;
-    private compact;
     /**
      * One prompt answered on a task's tier as the person: no chat, no tools. It is for background
      * work an app runs for someone (a job's fit score, a summary), billed to their payer and capped
@@ -320,8 +298,6 @@ export declare class Assistant {
         usd: number;
         model: string;
     }>;
-    /** How the person's cards ended since the last answer, once each. */
-    private cardNews;
     /** Starts a person's ChatGPT plan sign-in: the address to open in a new tab. */
     startSignin(caller: Caller, provider: AssistantProvider): Promise<{
         url: string;
@@ -353,10 +329,5 @@ export declare class Assistant {
         detail: string;
         at: number;
     }>;
-    /** Saves (runs, as the person) or dismisses a drafted card. Access is checked again first. */
-    settleDraft(caller: Caller, chatId: string, draftId: string, action: 'save' | 'dismiss'): Promise<DraftItem>;
-    /** Takes back one change a write made at once, through its group's `undo`. */
-    undoWrite(caller: Caller, chatId: string, stepId: string): Promise<StepItem>;
 }
 export declare function createAssistant(o: AssistantOptions): Assistant;
-export {};