genaicode 2.31.0 → 2.33.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (58) hide show
  1. package/README.md +1 -0
  2. package/dist/agents/artifacts.d.ts +16 -6
  3. package/dist/agents/artifacts.js +74 -23
  4. package/dist/agents/artifacts.js.map +1 -1
  5. package/dist/agents/handoff.d.ts +14 -3
  6. package/dist/agents/handoff.js +55 -5
  7. package/dist/agents/handoff.js.map +1 -1
  8. package/dist/agents/session.d.ts +11 -0
  9. package/dist/agents/session.js +22 -5
  10. package/dist/agents/session.js.map +1 -1
  11. package/dist/agents/types.d.ts +2 -2
  12. package/dist/agents.d.ts +1 -1
  13. package/dist/app-agent/http.js +2 -2
  14. package/dist/app-agent/http.js.map +1 -1
  15. package/dist/browser/fake-model.d.ts +43 -0
  16. package/dist/browser/fake-model.js +133 -0
  17. package/dist/browser/fake-model.js.map +1 -0
  18. package/dist/browser/language-model.d.ts +74 -0
  19. package/dist/browser/language-model.js +83 -0
  20. package/dist/browser/language-model.js.map +1 -0
  21. package/dist/browser/nano.d.ts +83 -0
  22. package/dist/browser/nano.js +610 -0
  23. package/dist/browser/nano.js.map +1 -0
  24. package/dist/browser/page-agent.d.ts +74 -0
  25. package/dist/browser/page-agent.js +339 -0
  26. package/dist/browser/page-agent.js.map +1 -0
  27. package/dist/browser/tools.d.ts +36 -0
  28. package/dist/browser/tools.js +190 -0
  29. package/dist/browser/tools.js.map +1 -0
  30. package/dist/browser/webmcp.d.ts +32 -0
  31. package/dist/browser/webmcp.js +150 -0
  32. package/dist/browser/webmcp.js.map +1 -0
  33. package/dist/browser.d.ts +39 -0
  34. package/dist/browser.js +32 -0
  35. package/dist/browser.js.map +1 -0
  36. package/dist/react/agent-chat.css +174 -0
  37. package/dist/react/agent-chat.d.ts +23 -2
  38. package/dist/react/agent-chat.js +114 -13
  39. package/dist/react/agent-chat.js.map +1 -1
  40. package/dist/react/app-agent-chat.d.ts +2 -0
  41. package/dist/react/app-agent-chat.js +1 -1
  42. package/dist/react/app-agent-chat.js.map +1 -1
  43. package/dist/react/attachments.d.ts +41 -0
  44. package/dist/react/attachments.js +162 -0
  45. package/dist/react/attachments.js.map +1 -0
  46. package/dist/react/page-agent-chat.d.ts +38 -0
  47. package/dist/react/page-agent-chat.js +40 -0
  48. package/dist/react/page-agent-chat.js.map +1 -0
  49. package/dist/react.d.ts +7 -2
  50. package/dist/react.js +3 -1
  51. package/dist/react.js.map +1 -1
  52. package/dist/ui/index.js +82 -78
  53. package/dist/ui/web-client.js +187 -12
  54. package/docs/agents.md +26 -4
  55. package/docs/browser.md +200 -0
  56. package/docs/react.md +14 -13
  57. package/docs/semver.md +2 -0
  58. package/package.json +5 -1
package/docs/agents.md CHANGED
@@ -274,8 +274,9 @@ const session = createAgentSession({ agent: claude(), cwd, artifacts });
274
274
  app.get('/artifacts/:id', (req, res) => {
275
275
  const found = artifacts.store.read(req.params.id);
276
276
  if (!found) return res.status(404).end();
277
- const { headers, body } = artifactResponse(found.artifact, found.body);
278
- res.writeHead(200, headers).end(body);
277
+ // The Range header lets audio and video seek.
278
+ const { status, headers, body } = artifactResponse(found.artifact, found.body, { range: req.headers.range });
279
+ res.writeHead(status, headers).end(body);
279
280
  });
280
281
  ```
281
282
 
@@ -285,10 +286,29 @@ app.get('/artifacts/:id', (req, res) => {
285
286
  simply do not get the tool. An app with its own MCP server can mount `store.tool()` there
286
287
  and pass `{ store }` alone; `createAppAgent` does exactly that (see [app-agent.md](app-agent.md)).
287
288
  - Each publish adds an `artifact` entry (`{ id, title, kind, mimeType, bytes, version, path? }`)
288
- to the running turn. The content is copied into memory at that moment (5 MB each, the newest
289
- 30 by default); publishing the same `name` again replaces it and keeps the `id`.
289
+ to the running turn. The content is copied into memory at that moment (5 MB each, 50 MB for
290
+ audio and video, the newest 30 and 200 MB in all by default: `maxBytes`, `maxMediaBytes`,
291
+ `maxArtifacts`, `maxTotalBytes`); publishing the same `name` again replaces it and keeps the `id`.
290
292
  - `<AgentChat artifactUrl={(a) => `/artifacts/${a.id}?v=${a.version}`}>` shows a card with a
291
293
  preview; the terminal chat shows a line and `/web` opens the browser.
294
+ - **Media**: files ending in png, jpg, gif, webp, avif or svg show as a picture (screenshots),
295
+ mp3, wav, ogg, opus, m4a, aac or flac in an audio player, mp4, m4v, webm, mov or ogv in a
296
+ video player. Media is published from a file (`path`), not as `content`. Your page's own
297
+ Content-Security-Policy has to allow `img-src`, `media-src` and `frame-src` for the address
298
+ you serve artifacts at.
299
+ - **Pages that answer back**: a published page can ask the user something (a questionnaire, a
300
+ choice between mockups) and hand the answer to the agent by calling
301
+ `parent.postMessage({ genaicode: 'submit', data }, '*')`, e.g. from a button. `<AgentChat>`
302
+ takes that only from the frame of that card, shows the user the JSON that would be sent and
303
+ waits for **Send to agent**; then `onArtifactSubmit(artifact, data)` runs, by default
304
+ `onSend(artifactSubmission(artifact, data))`, so the agent reads it as the user's next
305
+ message. Pass `onArtifactSubmit={null}` to turn it off. Nothing is sent without that click,
306
+ and a page opened in a tab of its own has nobody to send to.
307
+ - **Other agents**: the store belongs to the session, not the agent. After `setAgent()` or a
308
+ failover the cards still open, the next agent gets the same tool, and its handoff lists what
309
+ was published (title, `name`, kind, version, source file) so it can replace an artifact under
310
+ the same `name`. Pages and documents that were published as `content` are written into the
311
+ handoff transcript, since they exist nowhere else.
292
312
 
293
313
  Artifacts are agent-written content, so they are handled as untrusted:
294
314
 
@@ -303,6 +323,8 @@ Artifacts are agent-written content, so they are handled as untrusted:
303
323
  cookies, storage or DOM of your app, also when its URL is opened directly) and loads
304
324
  nothing from the network, so no CDN scripts either. `<AgentChat>` adds the same sandbox on
305
325
  the frame. SVG is shown as an image, or with scripts off when opened directly.
326
+ - **Answers from a page** are data from agent-written code: they reach the agent only after
327
+ the user saw and confirmed them, at most 20 KB of JSON at a time.
306
328
  - **What is left**: a page can still navigate its own frame to another site and carry data in
307
329
  that URL; browsers offer no switch for that. The page only knows what the agent put in it,
308
330
  but treat a published page like a link the agent sent you.
@@ -0,0 +1,200 @@
1
+ # An agent in the page (`genaicode/browser`)
2
+
3
+ [← README](../README.md)
4
+
5
+ `genaicode/browser` runs an agent **in the web page itself**, on the browser's built-in model
6
+ (Gemini Nano through Chrome's [Prompt API](https://developer.chrome.com/docs/ai/prompt-api)),
7
+ and offers the page's tools to outside agents over [WebMCP](https://webmachinelearning.github.io/webmcp/).
8
+ No server, no API key, no cost per token, and nothing the user types leaves their device.
9
+ Nothing in it imports Node.
10
+
11
+ It is a small model. It is good at "show me X", "what does this page say about Y", "switch
12
+ that on": one or two tool calls and a short answer. It does not write code; for that, put a
13
+ coding agent behind your app with [`genaicode/app-agent`](app-agent.md).
14
+
15
+ ```ts
16
+ import { createPageAgent, exposeWebMcp, type PageTool } from 'genaicode/browser';
17
+
18
+ const tools: PageTool[] = [
19
+ {
20
+ name: 'list_products',
21
+ description: 'The products in the shop, with id, name and price.',
22
+ run: () => products,
23
+ },
24
+ {
25
+ name: 'add_to_cart',
26
+ description: 'Put a product in the cart.',
27
+ inputSchema: { type: 'object', properties: { id: { type: 'string' } }, required: ['id'] },
28
+ write: true, // the user approves it first
29
+ summary: ({ id }) => `Add ${nameOf(id)} to your cart`,
30
+ run: ({ id }) => cart.add(String(id)),
31
+ },
32
+ ];
33
+
34
+ export const assistant = createPageAgent({
35
+ name: 'helper',
36
+ instructions: 'You help shoppers on Example Shop. Prices are in euro.',
37
+ tools,
38
+ });
39
+
40
+ exposeWebMcp(tools); // the same tools for the browser's own agent and extensions
41
+ ```
42
+
43
+ ```tsx
44
+ import { PageAgentChat } from 'genaicode/react';
45
+ import 'genaicode/react/styles.css';
46
+
47
+ <PageAgentChat agent={assistant} emptyState={<p>Ask me about the shop.</p>} />;
48
+ ```
49
+
50
+ `<PageAgentChat>` is [`<AgentChat>`](react.md) wired to the page agent, with what an
51
+ on-device model needs on top: a note where the browser has no model (`unavailable`), a
52
+ "Download the model" button with progress, Approve/Deny for `write` tools, a chip that shows
53
+ how full the context window is, and "New". Without React, read `assistant.get()` and
54
+ `assistant.subscribe()` and call `send`, `stop`, `reset`, `approve` and `prepare` yourself.
55
+
56
+ ## Tools
57
+
58
+ A `PageTool` is a function with a `name`, a `description` and a JSON Schema (`inputSchema`,
59
+ an object schema). `run(args, signal)` returns text, or any value, which is sent as JSON.
60
+
61
+ - `write: true` marks a tool that changes something. It runs only after the user approved the
62
+ call; `summary(args)` is the line they read. An in-page agent asks in the chat, an outside
63
+ agent goes through `exposeWebMcp`'s `confirm`.
64
+ - `validate(args)` may throw to refuse a call before anyone is asked. Required arguments,
65
+ basic types and `enum` values are checked for you, and the problem goes back to the model.
66
+ - Keep the list short (a handful) and the outputs small: the model sees at most
67
+ `maxToolOutput` characters (default 2,000) of each result. `enum` arguments help a small
68
+ model the most.
69
+
70
+ A `PageTool` has the shape of `AppTool` from `genaicode/app-agent`, so one definition can serve
71
+ a coding agent on your server and the model in the page.
72
+
73
+ ## How the model calls tools
74
+
75
+ The Prompt API has no tool calling, so `nano()` breaks every step into choices the browser
76
+ can constrain with a JSON Schema (`responseConstraint`):
77
+
78
+ 1. Which action: one of the tool names, or `reply`.
79
+ 2. The chosen tool's arguments, constrained by its `inputSchema`.
80
+ 3. The tool runs; its result goes back and the model chooses again, up to `maxSteps` (default 6).
81
+ 4. On `reply`, the answer streams as text.
82
+
83
+ The same exact call is never run twice in a task, bad arguments come back as an error the
84
+ model can fix, and a model that never picks `reply` is made to answer when the steps run out.
85
+
86
+ ## The driver: `nano()`
87
+
88
+ `createPageAgent` is a conversation on top of `nano()`, which is a `CodingAgent` like every
89
+ driver in [`genaicode/agents`](agents.md): the same `run(task)`, events and result.
90
+
91
+ ```ts
92
+ import { nano } from 'genaicode/browser';
93
+
94
+ const agent = nano({ instructions, tools });
95
+ if ((await agent.availability()) !== 'unavailable') {
96
+ const run = agent.run({ prompt: 'Which plans include SSO?', cwd: '' });
97
+ for await (const event of run) if (event.type === 'text-delta') render(event.text);
98
+ const { text, sessionId } = await run.result; // pass sessionId as task.resume to go on
99
+ }
100
+ ```
101
+
102
+ - `availability()`: `available`, `downloadable`, `downloading` or `unavailable`.
103
+ - `prepare()` loads the model, downloading it first when needed. A download needs a user
104
+ gesture, so call it from a click; `onDownload(progress)` reports 0 to 1.
105
+ - Chrome keeps the first `responseConstraint` a session was given: a later prompt with another
106
+ schema fails with `UnknownError` or answers in the first one's shape. So `nano` asks every
107
+ constrained step on a copy of the conversation (`session.clone()`), tells it the task so far,
108
+ and lets the copy go; the conversation itself only hears the request for the reply.
109
+ - A schema-constrained call can still fail with an unexplained `UnknownError`. `nano` then asks
110
+ the model the same thing in words and checks the answer itself, and a reply that fails before
111
+ its first word is tried once more. An error that remains names the step: choosing the action,
112
+ filling in the arguments or writing the reply.
113
+ - `onModelCall({ step, ms, constrained, error })` reports every call to the model, for logs and
114
+ for finding the slow step.
115
+ - `task.onApproval` answers `write` tools (without it they are declined);
116
+ `task.permissions.approval` `'auto-approve'` and `'deny'` work as in every driver.
117
+ `permissions.sandbox: 'read-only'` refuses every `write` tool; a page has no other sandbox,
118
+ so `'workspace-write'`, `'unrestricted'` and `'yolo'` fail the task.
119
+ - `task.resume` continues a conversation in the same model session; `forget(sessionId)` frees it.
120
+ When the context window fills up, the browser drops the oldest messages and keeps the
121
+ instructions. `context(sessionId)` says how full it is.
122
+ - Usage is what the task added to the conversation (`usage.totalTokens`), and
123
+ `task.budget.maxTokens` stops a task that goes over it. There is no cost to cap.
124
+ - `task.cwd`, `env` and `mcpServers` mean nothing in a page. Pass `cwd: ''`.
125
+
126
+ `createPageAgent({ agent })` takes any other `CodingAgent` that runs in a browser instead.
127
+
128
+ ## Files, images and voice
129
+
130
+ ```ts
131
+ const assistant = createPageAgent({ tools, nano: { inputs: ['image', 'audio'] } });
132
+ assistant.send('What is on this screenshot?', [file]);
133
+ ```
134
+
135
+ ```tsx
136
+ <PageAgentChat agent={assistant} attachments />
137
+ ```
138
+
139
+ `send(text, files)` takes `File`s or `Blob`s with a prompt, and `nano().run({ prompt, attachments })`
140
+ does the same one level down.
141
+
142
+ - A text file (by MIME type or extension: Markdown, JSON, CSV, source code) is read into the
143
+ prompt, up to `maxAttachmentText` characters (default 4,000). The context window is small, so a
144
+ long file is cut and the model is told so.
145
+ - An image or audio file goes to the model as it is, if `nano: { inputs }` asked for that kind
146
+ and the browser's model takes it. `assistant.get().model.inputs` (and `nano().inputs()`) lists
147
+ what it takes; elsewhere such a file fails the turn with a message that says why. Audio needs
148
+ a GPU in Chrome.
149
+ - Anything else (a PDF, an archive) is refused.
150
+
151
+ With `attachments`, `<PageAgentChat>` adds an attach button, takes pasted and dropped files, and
152
+ shows a microphone where the model takes audio: it records a voice message that is sent as an
153
+ audio file, so speech stays on the device too. The chat offers only the kinds the model takes.
154
+ `<AgentChat attachments={{ accept, voice, max }}>` has the same composer for any host:
155
+ `onSend(text, files)` gets the files, and a turn's `attachments` are drawn under its prompt.
156
+
157
+ ## WebMCP
158
+
159
+ `exposeWebMcp(tools, { confirm, onCall })` registers the tools on `document.modelContext`
160
+ (or `navigator.modelContext` in the first implementations), so an agent outside the page,
161
+ such as the browser's own assistant, calls your functions instead of guessing at the DOM.
162
+ Read tools are marked `readOnlyHint`, `write` tools `consequentialHint`, and a `write` tool
163
+ runs only after `confirm(tool, args)` resolves true (default: the browser's `confirm()`
164
+ dialog). It returns a function that takes the tools back, and does nothing where WebMCP is
165
+ missing, so it is safe to call everywhere.
166
+
167
+ `webMcpTools()` goes the other way: it lists what the page already registered over WebMCP
168
+ (by any script, or by annotated forms) as `PageTool`s for the in-page agent. It returns an
169
+ empty list where the browser cannot list tools.
170
+
171
+ ## Where it runs
172
+
173
+ | | |
174
+ | --------------- | ------------------------------------------------------------------------------------------ |
175
+ | Prompt API | Chrome 148+ on desktop (Windows, macOS, Linux, Chromebook Plus). Not on Android or iOS. |
176
+ | Hardware | About 22 GB of free disk, and 16 GB of RAM or a GPU with more than 4 GB of VRAM. |
177
+ | Languages | English, Japanese, Spanish, German, French (`nano({ language })`, default `en`). |
178
+ | First use | The browser downloads the model once (a few GB); after that it works offline. |
179
+ | WebMCP | A draft: Chrome origin trial (149 to 156), Edge behind a flag. Register a trial token. |
180
+ | Everything else | `availability()` is `unavailable`: show the page without the assistant, or use a fallback. |
181
+
182
+ Check `availability()` (or `<PageAgentChat>`'s `unavailable` slot) and treat the assistant as
183
+ an extra. A common setup loads it only where `'LanguageModel' in self`.
184
+
185
+ ## Tests
186
+
187
+ `fakeLanguageModel(script)` stands in for the browser's model: replies come from `script` in
188
+ order and every prompt is recorded.
189
+
190
+ ```ts
191
+ import { createPageAgent, fakeLanguageModel } from 'genaicode/browser';
192
+
193
+ const model = fakeLanguageModel(['{"action":"list_products"}', '{"action":"reply"}', 'Two products.']);
194
+ const assistant = createPageAgent({ tools, nano: { languageModel: model.factory } });
195
+ ```
196
+
197
+ ## Stability
198
+
199
+ Experimental, like the Prompt API and WebMCP themselves: names here may change in a minor
200
+ release.
package/docs/react.md CHANGED
@@ -71,19 +71,20 @@ HTML; links open in a new tab; GFM tables, strikethrough and task lists), collap
71
71
  own approval requests, queued prompts and a working indicator, and an input where Enter
72
72
  sends, Shift+Enter adds a line and Esc stops. Props:
73
73
 
74
- | Prop | |
75
- | ------------------------------------------------------------------------------ | ----------------------------------------------------------------------------------------------- |
76
- | `state` | `SessionState`, or null before there is a session |
77
- | `onSend(text)`, `onStop()` | Enter or Send; Stop or Esc. No `onStop`, no Stop button |
78
- | `onApprove(id, decision)` | answers `state.approvals`; without it they show as waiting |
79
- | `renderPrompt(text, turn?)` | how a sent prompt shows (return null to hide it) |
80
- | `renderAfterTurn(turn, i)`, `renderFooter()` | your content after a turn, or at the end |
81
- | `notices` | `{ id, afterTurn, text, tone? }[]`: your own lines between turns |
82
- | `artifactUrl(artifact)`, `renderArtifact(artifact, url)` | where your server serves a published artifact; your own card ([artifacts](agents.md#artifacts)) |
83
- | `foldSteps` | fold each turn's tools and edits into one "N steps" line |
84
- | `header`, `headerExtras` | replace the header (null for none), or add to its right end |
85
- | `emptyState`, `placeholder`, `hint` | text before the first turn, in and under the input |
86
- | `formatToolName(name)`, `disabled`, `autoFocus`, `theme`, `className`, `style` | |
74
+ | Prop | |
75
+ | ------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------ |
76
+ | `state` | `SessionState`, or null before there is a session |
77
+ | `onSend(text)`, `onStop()` | Enter or Send; Stop or Esc. No `onStop`, no Stop button |
78
+ | `onApprove(id, decision)` | answers `state.approvals`; without it they show as waiting |
79
+ | `renderPrompt(text, turn?)` | how a sent prompt shows (return null to hide it) |
80
+ | `renderAfterTurn(turn, i)`, `renderFooter()` | your content after a turn, or at the end |
81
+ | `notices` | `{ id, afterTurn, text, tone? }[]`: your own lines between turns |
82
+ | `artifactUrl(artifact)`, `renderArtifact(artifact, url)` | where your server serves a published artifact; your own card ([artifacts](agents.md#artifacts)) |
83
+ | `onArtifactSubmit(artifact, data)` | a page the agent published sent data and the user confirmed it; default sends it as a message; `null` turns it off |
84
+ | `foldSteps` | fold each turn's tools and edits into one "N steps" line |
85
+ | `header`, `headerExtras` | replace the header (null for none), or add to its right end |
86
+ | `emptyState`, `placeholder`, `hint` | text before the first turn, in and under the input |
87
+ | `formatToolName(name)`, `disabled`, `autoFocus`, `theme`, `className`, `style` | |
87
88
 
88
89
  Theme it with CSS custom properties on the component or any ancestor: `--gc-bg`,
89
90
  `--gc-panel`, `--gc-border`, `--gc-text`, `--gc-muted`, `--gc-accent`, `--gc-accent-text`,
package/docs/semver.md CHANGED
@@ -23,6 +23,8 @@ These are covered by minor/patch compatibility within 2.x:
23
23
  minor release until this section lists it as stable. See [agents.md](./agents.md).
24
24
  - `genaicode/react` (`<AgentChat>` props, its `.gc-*` class names and `--gc-*` custom
25
25
  properties) and `genaicode/ui` (`startEmbeddedWeb`), on the same terms.
26
+ - `genaicode/browser` (`nano`, `createPageAgent`, `exposeWebMcp`) and `<PageAgentChat>`, on the
27
+ same terms: the Prompt API and WebMCP under them are still moving.
26
28
 
27
29
  ## Additive changes (minor)
28
30
 
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "genaicode",
3
- "version": "2.31.0",
3
+ "version": "2.33.0",
4
4
  "description": "Drive the coding-agent CLIs you already have (Claude Code, Codex, Copilot, Cursor, Gemini, opencode...) behind one API and one CLI, plus a thin provider-neutral LLM layer.",
5
5
  "author": "Grzegorz Tańczyk",
6
6
  "repository": {
@@ -56,6 +56,10 @@
56
56
  "types": "./dist/app-agent.d.ts",
57
57
  "import": "./dist/app-agent.js"
58
58
  },
59
+ "./browser": {
60
+ "types": "./dist/browser.d.ts",
61
+ "import": "./dist/browser.js"
62
+ },
59
63
  "./vite": {
60
64
  "types": "./dist/vite.d.ts",
61
65
  "import": "./dist/vite.js"