libfx 0.0.13-dev.1431.g1df91f0b4ebf → 0.0.13-dev.1470.g5ec2519d98b9
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +239 -1056
- package/chatgpt-sign-in.js +141 -0
- package/chatgpt.js +68 -0
- package/custom-provider.js +123 -0
- package/durable.js +1 -2
- package/fx-core.wasm +0 -0
- package/fx-harness.js +4 -0
- package/fx-sdk.js +49 -3
- package/fx-term.wasm +0 -0
- package/grok-sign-in.js +145 -0
- package/grok.js +108 -0
- package/libfx.darwin-arm64.node +0 -0
- package/libfx.darwin-x64.node +0 -0
- package/libfx.linux-arm64.node +0 -0
- package/libfx.linux-x64.node +0 -0
- package/node.cjs +519 -45
- package/node.js +10 -2
- package/oauth.js +64 -0
- package/package.json +9 -1
- package/subscription.js +251 -0
- package/types/browser.d.ts +8 -0
- package/types/chatgpt.d.ts +58 -0
- package/types/grok.d.ts +61 -0
- package/types/libfx.d.cts +83 -4
- package/types/node.d.cts +8 -0
- package/types/wasm.d.ts +8 -0
package/README.md
CHANGED
|
@@ -1,1220 +1,403 @@
|
|
|
1
1
|
# libfx
|
|
2
2
|
|
|
3
|
-
|
|
4
|
-
|
|
5
|
-
|
|
6
|
-
it. The same code runs on your machine and on Vercel.
|
|
3
|
+
libfx is the agent from [fx](https://github.com/vercel-labs/fx), the open source coding agent from Vercel Labs, as a JavaScript library. You give it a model, instructions and your own tools, and it runs the conversation: streaming, tool calls, retries, steering and images. It's the same Zig engine that runs fx in your terminal, built as a native addon and as WebAssembly.
|
|
4
|
+
|
|
5
|
+
Sessions are durable. libfx records each step of a turn as it happens, so when a server crashes, a function hits its time limit or you redeploy in the middle of a reply, the next process picks the turn up where it stopped. On Vercel, that takes one route and one line of `vercel.json`.
|
|
7
6
|
|
|
8
7
|
```sh
|
|
9
8
|
npm install libfx
|
|
10
9
|
```
|
|
11
10
|
|
|
12
|
-
Node.js uses the native addon when available and falls back to WebAssembly.
|
|
13
|
-
Browsers use WebAssembly with JSPI. The default package has no runtime
|
|
14
|
-
dependencies and performs no MCP connection, skill scan, process spawn, or
|
|
15
|
-
filesystem read when imported.
|
|
16
|
-
|
|
17
|
-
libfx ships TypeScript declarations for every entry point. Under
|
|
18
|
-
`moduleResolution: "bundler"`, TypeScript reads the browser declarations for
|
|
19
|
-
`libfx` unless `customConditions` includes `"node"`.
|
|
20
|
-
|
|
21
|
-
## Agent
|
|
22
|
-
|
|
23
11
|
```js
|
|
24
12
|
import { createFxAgent } from "libfx";
|
|
25
13
|
|
|
26
14
|
const agent = createFxAgent({
|
|
27
|
-
model: "
|
|
15
|
+
model: "anthropic/claude-haiku-5.5",
|
|
28
16
|
instructions: "Answer in one paragraph.",
|
|
29
17
|
});
|
|
30
18
|
|
|
31
|
-
const turn = agent.session().prompt("
|
|
19
|
+
const turn = agent.session().prompt("What is an agent loop?");
|
|
32
20
|
|
|
33
21
|
for await (const event of turn) {
|
|
34
22
|
if (event.type === "text_delta") process.stdout.write(event.delta);
|
|
35
23
|
}
|
|
36
24
|
|
|
37
|
-
console.log(await turn.result); // { messageId, stopReason, usage }
|
|
38
25
|
await agent.close();
|
|
39
26
|
```
|
|
40
27
|
|
|
41
|
-
|
|
42
|
-
`apiKey` is optional. Without it, libfx uses `AI_GATEWAY_API_KEY`, then on
|
|
43
|
-
Vercel the deployment's OIDC token. `model` defaults to fx's built-in model.
|
|
44
|
-
Tools, MCP clients, and skills are agent options, described in
|
|
45
|
-
[JavaScript tools and instructions](#javascript-tools-and-instructions) and the
|
|
46
|
-
sections after it.
|
|
28
|
+
Model calls go through [AI Gateway](https://vercel.com/docs/ai-gateway), so any model it lists works. Set `AI_GATEWAY_API_KEY` on your machine. On Vercel, libfx uses the deployment's OIDC token and there's no key to manage.
|
|
47
29
|
|
|
48
|
-
|
|
49
|
-
such as an effort level the model does not support, fails that session's
|
|
50
|
-
first turn rather than the `createFxAgent()` call. The turn's `result` has the
|
|
51
|
-
stop reason `error` and the reason in `error`, and `onEvent` receives
|
|
52
|
-
`session.error`.
|
|
30
|
+
Importing libfx doesn't touch the network, read files or start processes, and `createFxAgent()` returns right away. Nothing loads until a session runs its first turn.
|
|
53
31
|
|
|
54
|
-
|
|
55
|
-
creates an agent with the same options can run the same sessions, so create
|
|
56
|
-
one agent for your server and open a session per conversation.
|
|
57
|
-
`agent.close()` waits for the turns this process is running, then lets their
|
|
58
|
-
sessions go.
|
|
32
|
+
The full documentation is at [fx.sh/docs/lib](https://fx.sh/docs/lib).
|
|
59
33
|
|
|
60
|
-
|
|
34
|
+
## Features
|
|
61
35
|
|
|
62
|
-
|
|
63
|
-
|
|
64
|
-
|
|
65
|
-
|
|
36
|
+
- [Streams](https://fx.sh/docs/lib/prompts) text, reasoning and tool calls as the agent works
|
|
37
|
+
- Runs your JavaScript functions as [tools](https://fx.sh/docs/lib/tools), along with [MCP](https://fx.sh/docs/lib/mcp) clients and [skills](https://fx.sh/docs/lib/skills)
|
|
38
|
+
- Keeps [sessions](https://fx.sh/docs/lib/durability) going through crashes, deploys and function timeouts
|
|
39
|
+
- Reconnects to a running turn from any server, starting at the last event you saw
|
|
40
|
+
- [Steers](https://fx.sh/docs/lib/steering) a turn while it runs
|
|
41
|
+
- Takes [images](https://fx.sh/docs/lib/images) in prompts and tool results
|
|
42
|
+
- Gives each conversation its own model, instructions and context
|
|
43
|
+
- Runs on AI Gateway, on a user's own [ChatGPT](#chatgpt-plans) or [Grok](#grok-plans) plan, or on [any OpenAI-compatible server](#custom-providers)
|
|
44
|
+
- Uses a native addon on macOS and Linux, and WebAssembly everywhere else, including browsers
|
|
45
|
+
- Puts the [fx terminal](https://fx.sh/docs/lib/webassembly#embed-the-terminal) in a web page
|
|
46
|
+
- Ships TypeScript types for every entry point, with no runtime dependencies
|
|
47
|
+
|
|
48
|
+
## Sessions
|
|
49
|
+
|
|
50
|
+
An agent is configuration. Create one per server and open a session per conversation:
|
|
66
51
|
|
|
67
52
|
```js
|
|
68
53
|
const turn = agent.session().prompt("Plan the migration.");
|
|
69
54
|
const { sessionId } = await turn.accepted;
|
|
70
55
|
|
|
71
|
-
// Later,
|
|
72
|
-
|
|
56
|
+
// Later, from any process that creates the same agent:
|
|
57
|
+
agent.session(sessionId).prompt("Start with the schema.");
|
|
73
58
|
```
|
|
74
59
|
|
|
75
|
-
|
|
76
|
-
request with the same `messageId` reaches the same session, get one from
|
|
77
|
-
`await agent.newSessionId()` and open it with `agent.session(id)`.
|
|
78
|
-
|
|
79
|
-
On `memory()`, a session id is 1 to 128 letters, digits, `.`, `_`, or `-`. A
|
|
80
|
-
World durability, `local()`, `vercel()`, or `world()`, keeps each session as a
|
|
81
|
-
World run, so the id is a run id: `wrun_` followed by 10 to 64 letters or
|
|
82
|
-
digits. Take it from `agent.newSessionId()` or `turn.accepted`. Vercel's World
|
|
83
|
-
only creates a run whose id was minted in the last 24 hours, so send a new
|
|
84
|
-
id's first prompt within a day. `agent.prompt()` and `agent.checkpoint()` use
|
|
85
|
-
a session the agent opens for itself, for code that holds one conversation.
|
|
86
|
-
|
|
87
|
-
One process runs a session's turns at a time, one turn after another. A
|
|
88
|
-
second process that receives a prompt for a busy session queues it behind the
|
|
89
|
-
running turn. Pass `context` to hand JSON to the session's tools:
|
|
90
|
-
`agent.session(id, { context: { userId } })` gives every tool call
|
|
91
|
-
`context` beside its `executionId` and `sessionId`. Each turn runs with the context of the
|
|
92
|
-
`session()` call whose prompt or steer started it, and a call without
|
|
93
|
-
`context` gives its turns none.
|
|
94
|
-
|
|
95
|
-
A conversation can also choose its own model and instructions.
|
|
96
|
-
`agent.session(id, { model, instructions })` runs the turns it starts with
|
|
97
|
-
that model, a model ID or a model object such as `{ id, effort, fast }`, in
|
|
98
|
-
place of the agent's whole model choice, and with those instructions in place
|
|
99
|
-
of the agent's. They follow the rule `context` follows: a turn keeps the
|
|
100
|
-
settings of the `session()` call that started it, also when another process
|
|
101
|
-
continues it, and a call without them runs with the agent's. Pass a
|
|
102
|
-
conversation's settings on each call; a change applies from its next turn.
|
|
103
|
-
When a session's next turn runs under another model than the one its last
|
|
104
|
-
checkpoint was saved under, the session continues and `onEvent` receives a
|
|
105
|
-
`checkpoint.mismatch` event.
|
|
106
|
-
|
|
107
|
-
### Turns
|
|
108
|
-
|
|
109
|
-
`session.prompt(input)` queues `input` as the session's next turn and returns
|
|
110
|
-
a view of that turn. Prompt input is a string or an array of content blocks,
|
|
111
|
-
as described in [prompt input](#prompt-input), except that image data must be a
|
|
112
|
-
base64 string, because the prompt is stored as JSON before it runs. A turn
|
|
113
|
-
view has:
|
|
114
|
-
|
|
115
|
-
- `messageId`: the turn's id, which libfx chooses unless you pass one.
|
|
116
|
-
- `accepted`: resolves to `{ messageId, sessionId }` once the prompt is stored,
|
|
117
|
-
or at once for a prompt whose `signal` had already aborted, which is never
|
|
118
|
-
stored.
|
|
119
|
-
- `result`: resolves to `{ messageId, stopReason, usage }` when the turn ends.
|
|
120
|
-
- Async iteration over the turn's events, as
|
|
121
|
-
[model and tool events](#prompt-input) describes.
|
|
122
|
-
- `readable`: the same turn as an NDJSON `ReadableStream` that a route can
|
|
123
|
-
return as its response body. Each line is an event with a `cursor`.
|
|
124
|
-
|
|
125
|
-
Dropping a turn view cancels nothing: the turn runs to its end whether or not
|
|
126
|
-
anyone reads it. Call `session.cancel()` to stop it. A prompt's `signal`
|
|
127
|
-
cancels that prompt's turn when it aborts, whether the turn is running or still
|
|
128
|
-
waiting behind another, and never a later turn. A prompt whose `signal` had
|
|
129
|
-
already aborted is not stored: its `result` resolves with the stop reason
|
|
130
|
-
`cancelled`.
|
|
131
|
-
|
|
132
|
-
Give a prompt a `messageId` when the same request can reach your server more
|
|
133
|
-
than once, such as a client retry. A second `prompt()` with an id the session
|
|
134
|
-
already accepted runs nothing: while that turn runs, the view follows it, and
|
|
135
|
-
after it ends, `result` resolves to its outcome with `repeated: true`. When
|
|
136
|
-
that turn failed and its end never reached the stream, `result` resolves with
|
|
137
|
-
the failure from the session's log, without `repeated` if the failure could be
|
|
138
|
-
this request's own run. When the process that ran the turn stopped before
|
|
139
|
-
writing how it ended, `result` resolves with the stop reason `unknown` after
|
|
140
|
-
30 seconds. An id follows the session id rule.
|
|
141
|
-
|
|
142
|
-
`session.stream(cursor)` returns every event the session has produced from
|
|
143
|
-
`cursor` on, as NDJSON, and stays open for the events after them. Any process
|
|
144
|
-
can serve it, so a browser that loses its connection reconnects with the last
|
|
145
|
-
`cursor` it saw and continues where it stopped. Beside the turn events, the
|
|
146
|
-
stream marks each turn with `turn_start` and `turn_end`, a continued turn with
|
|
147
|
-
`turn_resume`, and a turn that stopped for a function deadline with
|
|
148
|
-
`turn_yield`. `turn_start` and `turn_resume` carry the `sessionId`, so a client
|
|
149
|
-
that started a new session learns its id from the first line of
|
|
150
|
-
`turn.readable`.
|
|
151
|
-
|
|
152
|
-
Each line carries the `epoch` of the lease its writer held, and epochs only
|
|
153
|
-
rise over a session's life. A worker that was replaced while it ran can still land lines late, because the World accepts
|
|
154
|
-
them. A worker that takes a turn over writes its `turn_resume` line before it
|
|
155
|
-
goes on, and every reader hides a line whose epoch is lower than one before
|
|
156
|
-
it, so the replaced worker's later lines never show and every reader, on any
|
|
157
|
-
server and after any refresh, shows the same lines. A replaced worker's line
|
|
158
|
-
that lands in the moment between its successor's claim and that `turn_resume`
|
|
159
|
-
line still shows; only a World that refuses stale stream writes closes that
|
|
160
|
-
gap.
|
|
161
|
-
|
|
162
|
-
`session.steer(text)` adds guidance to the running turn at its next model
|
|
163
|
-
request, or to the next turn when none is running. `session.resume()`
|
|
164
|
-
continues a turn a stopped process left open and returns its view, whose
|
|
165
|
-
`result` has the stop reason `idle` when there was none. A prompt also
|
|
166
|
-
continues an open turn before it runs, so most hosts never call `resume()`.
|
|
167
|
-
|
|
168
|
-
### Durability
|
|
169
|
-
|
|
170
|
-
The `durability` option chooses where sessions live. Without it, libfx picks
|
|
171
|
-
one from the environment:
|
|
172
|
-
|
|
173
|
-
| Environment | Durability | Sessions live in |
|
|
174
|
-
| --- | --- | --- |
|
|
175
|
-
| Vercel, where `VERCEL` is set | `vercel()` | Vercel's World, the store and queue behind Vercel Workflow |
|
|
176
|
-
| Node.js elsewhere | `local()` | files in `FX_SESSIONS_DIR`, or `$TMPDIR/libfx/sessions` |
|
|
177
|
-
| Browsers | `memory()` | this page's memory |
|
|
60
|
+
A session runs one turn at a time. A prompt that arrives while a turn is running waits behind it, even when it reaches a different server. A turn runs to the end whether or not anyone reads it; stop it with `session.cancel()` or the prompt's `signal`. `session.steer(text)` adds guidance to the running turn at its next model request.
|
|
178
61
|
|
|
179
|
-
|
|
180
|
-
import { createFxAgent, memory } from "libfx";
|
|
181
|
-
import { local } from "libfx/durable-local";
|
|
62
|
+
If a client might send the same request twice, give the prompt a `messageId`. A prompt with an id the session already accepted runs nothing and follows the original turn.
|
|
182
63
|
|
|
183
|
-
|
|
184
|
-
|
|
64
|
+
Each conversation can bring its own settings. They apply to the turns that call starts:
|
|
65
|
+
|
|
66
|
+
```js
|
|
67
|
+
agent.session(sessionId, {
|
|
68
|
+
model: { id: "anthropic/claude-opus-5.5", effort: "high" },
|
|
69
|
+
instructions: "You review pull requests.",
|
|
70
|
+
context: { userId }, // handed to every tool call
|
|
71
|
+
});
|
|
185
72
|
```
|
|
186
73
|
|
|
187
|
-
|
|
188
|
-
them, and libfx loads one only when a session first needs it, so your app
|
|
189
|
-
installs no other package and an app with no sessions loads neither.
|
|
74
|
+
### Streaming to a browser
|
|
190
75
|
|
|
191
|
-
|
|
192
|
-
World, pass it to `world()` from `libfx/durable-world`:
|
|
76
|
+
`turn.readable` is the turn as newline-delimited JSON, ready to return from a route. Every line has a `cursor`. `session.stream(cursor)` replays everything from that point and stays open, so a client that loses its connection can reconnect to any server and carry on.
|
|
193
77
|
|
|
194
78
|
```js
|
|
195
|
-
|
|
196
|
-
import {
|
|
197
|
-
import { createWorld } from "@workflow/world-postgres";
|
|
79
|
+
// app/api/chat/route.js
|
|
80
|
+
import { agent } from "@/lib/agent";
|
|
198
81
|
|
|
199
|
-
|
|
200
|
-
const
|
|
201
|
-
|
|
82
|
+
export async function POST(request) {
|
|
83
|
+
const sessionId = new URL(request.url).searchParams.get("sessionId") ?? undefined;
|
|
84
|
+
// Check that the caller may use this session.
|
|
85
|
+
const turn = agent.session(sessionId).prompt(await request.text());
|
|
86
|
+
return new Response(turn.readable);
|
|
87
|
+
}
|
|
202
88
|
|
|
203
|
-
|
|
89
|
+
export async function GET(request) {
|
|
90
|
+
const params = new URL(request.url).searchParams;
|
|
91
|
+
const sessionId = params.get("sessionId");
|
|
92
|
+
if (!sessionId) return new Response("sessionId is required", { status: 400 });
|
|
93
|
+
// Same check as POST.
|
|
94
|
+
const stream = agent.session(sessionId).stream(Number(params.get("cursor") ?? 0));
|
|
95
|
+
return new Response(stream);
|
|
96
|
+
}
|
|
204
97
|
```
|
|
205
98
|
|
|
206
|
-
|
|
207
|
-
|
|
208
|
-
|
|
209
|
-
deliveries in turn, as servers sharing a queue do, and a delivery that arrives
|
|
210
|
-
while no agent is listening fails, so the queue delivers it again. Pass a
|
|
211
|
-
function that returns a World instead, and libfx creates, starts, and closes
|
|
212
|
-
it with the agent. `local()` and `vercel()` are `world()` with the World each
|
|
213
|
-
one bundles.
|
|
99
|
+
## Durability
|
|
100
|
+
|
|
101
|
+
libfx stores a prompt before it reports the prompt as accepted, and it stores each step of a turn while the turn runs. If the process running a turn dies, the next process that gets work for that session continues the turn from its last step. The model is told the turn was interrupted, so it can check what happened before going on.
|
|
214
102
|
|
|
215
|
-
|
|
103
|
+
Where sessions live depends on where your code runs, and libfx picks for you:
|
|
216
104
|
|
|
217
|
-
|
|
|
105
|
+
| Where | Durability | Sessions live in |
|
|
218
106
|
| --- | --- | --- |
|
|
219
|
-
|
|
|
220
|
-
|
|
|
221
|
-
|
|
|
222
|
-
|
|
223
|
-
|
|
224
|
-
| `gatewayKey()` | none | The AI Gateway credential when the agent has no `apiKey`. |
|
|
225
|
-
| `name` | `"world"` | The durability's name in errors. |
|
|
226
|
-
|
|
227
|
-
libfx uses the World's `events.create` and `events.list`, its streams, its
|
|
228
|
-
queue, and `createQueueHandler`. It reads an event's position in its run from
|
|
229
|
-
the event's id, which must be lowercase letters, an underscore, and that
|
|
230
|
-
position as 26 digits, as in Workflow's Worlds; a session whose World returns
|
|
231
|
-
another id fails. It sends every message on the topic
|
|
232
|
-
`__libfx_wkf_workflow_session` and handles the prefix
|
|
233
|
-
`__libfx_wkf_workflow_`, which is what a Workflow World's `namespace: "libfx"`
|
|
234
|
-
gives, so a World serving the app's own workflows on the default namespace
|
|
235
|
-
needs a second instance for libfx. A World with `registerHandler` delivers to
|
|
236
|
-
the agent in its process. Any other World delivers over HTTP, so mount
|
|
237
|
-
`agent.wakeHandler()` where it delivers, as [Next.js and
|
|
238
|
-
Vercel](#nextjs-and-vercel) shows.
|
|
239
|
-
|
|
240
|
-
libfx records each step of a turn as it happens. A prompt is stored before
|
|
241
|
-
the caller hears it was accepted, and a tool call with effects is stored
|
|
242
|
-
before it runs, so a crash never loses an accepted prompt and never repeats a
|
|
243
|
-
tool call silently. When a session has no turn in progress, the process that
|
|
244
|
-
takes it sends the next turn's first model request while it writes its claim
|
|
245
|
-
on the session, and it writes, shows, and runs nothing for that turn until
|
|
246
|
-
the claim lands. If another process's claim lands first, the late claim
|
|
247
|
-
counts for nothing and its model request is cancelled. Other records are
|
|
248
|
-
written while the model works, without holding up the turn. A session also
|
|
249
|
-
saves a checkpoint at the end of every turn and wherever a turn yields, so the
|
|
250
|
-
process that continues it reads the checkpoint and the few records after it,
|
|
251
|
-
not the whole history.
|
|
252
|
-
|
|
253
|
-
When the process running a turn stops, the next process to receive work for
|
|
254
|
-
the session continues the turn from its last record:
|
|
255
|
-
|
|
256
|
-
- On `local()`, a session held by a process that died is free at once.
|
|
257
|
-
- On `vercel()`, where libfx cannot tell whether a function still runs, a
|
|
258
|
-
worker renews a 15 second lease while it runs. When its function dies, its
|
|
259
|
-
session frees 15 seconds after the last renewal, and the next delivery for
|
|
260
|
-
it continues the turn. When
|
|
261
|
-
the deadline is near, libfx stops before the next model request and saves
|
|
262
|
-
the turn, and the next invocation continues it. A model request or tool
|
|
263
|
-
call still running shortly before the deadline is cut off: `onEvent`
|
|
264
|
-
receives `session.deadline`, and the next invocation continues the turn,
|
|
265
|
-
running a cut-off idempotent call again and telling the model about any
|
|
266
|
-
other. The event's `cutoffs` counts the times in a row the deadline cut
|
|
267
|
-
off the same step. After 3, the next invocation does not run that step
|
|
268
|
-
again: a call is answered as possibly run, so the model decides what to do
|
|
269
|
-
next, and a model request cancels the turn. So a process never runs on
|
|
270
|
-
after the deadline its hold ends at. A
|
|
271
|
-
process that freezes, or whose renewals stall for a lease, and wakes after
|
|
272
|
-
another took its session over stops at its next write, a renewal included,
|
|
273
|
-
and `onEvent` receives `session.fenced`. Until then it
|
|
274
|
-
can finish a model request or a call to an idempotent tool; it never starts
|
|
275
|
-
a tool with effects, because that waits for its record to be stored. A
|
|
276
|
-
process that started a new turn on an idle session and was taken over
|
|
277
|
-
while still claiming the session runs nothing for that turn and reads the
|
|
278
|
-
session's log again instead, so the prompt it brought is not lost.
|
|
279
|
-
- When a write fails for any other reason, or the engine running a turn stops,
|
|
280
|
-
`onEvent` receives `session.error`, and the same queue message runs the
|
|
281
|
-
session again with a new engine. A turn whose engine stops 3 times is
|
|
282
|
-
cancelled, and the prompts behind it run; one that never got started ends
|
|
283
|
-
with an error instead. When the engine stops twice more, even to cancel the
|
|
284
|
-
turn, the session can run no more turns: the turn and every prompt sent to
|
|
285
|
-
the session end with an error. A turn whose engine stops after storing its
|
|
286
|
-
end resolves with the stop reason `unknown`.
|
|
287
|
-
- When an engine cannot open, the turn waiting on it ends with an error, and
|
|
288
|
-
each later engine that cannot open ends the next waiting prompt the same
|
|
289
|
-
way. A started turn that ended this way stays open in the session, and
|
|
290
|
-
`session.resume()` answers `idle` for it. The next prompt's engine first
|
|
291
|
-
continues it, or cancels it when it was cancelled or its engine kept
|
|
292
|
-
stopping, and then runs that prompt.
|
|
293
|
-
- On `memory()`, sessions end with the process.
|
|
294
|
-
|
|
295
|
-
The model is told when its turn was interrupted, so it can check what
|
|
296
|
-
happened before going on. A turn that stopped at a model request for a
|
|
297
|
-
deadline continues without that notice; one whose call was cut off gets it.
|
|
298
|
-
|
|
299
|
-
A durability holds conversation history only. Every process supplies the
|
|
300
|
-
model, credentials, instructions, tools, MCP clients, and skills when it
|
|
301
|
-
creates the agent. Every checkpoint records the libfx version, tool set, and
|
|
302
|
-
model that saved it. When a saved session, or a checkpoint passed as
|
|
303
|
-
`checkpoint`, was saved under different ones than the agent now has, it still
|
|
304
|
-
loads, and `onEvent` receives a `checkpoint.mismatch` event naming what
|
|
305
|
-
changed.
|
|
306
|
-
|
|
307
|
-
`session.checkpoint()` returns the session's history as opaque bytes, read
|
|
308
|
-
without changing the session. Pass them as `checkpoint` to start a new agent's
|
|
309
|
-
own session from them, for example to move a conversation between
|
|
310
|
-
durabilities.
|
|
311
|
-
|
|
312
|
-
### Tools with effects
|
|
313
|
-
|
|
314
|
-
When a process stops while a tool call runs, libfx cannot know whether the
|
|
315
|
-
call finished. Mark a tool `idempotent: true` when running it twice is safe,
|
|
316
|
-
such as a lookup:
|
|
107
|
+
| Vercel | `vercel()` | Vercel's [Workflow World](https://workflow-sdk.dev/worlds) |
|
|
108
|
+
| Node.js | `local()` | files in `$TMPDIR/libfx/sessions`, or `FX_SESSIONS_DIR` |
|
|
109
|
+
| Browsers | `memory()` | the page's memory |
|
|
110
|
+
|
|
111
|
+
To choose yourself, pass `durability`:
|
|
317
112
|
|
|
318
113
|
```js
|
|
319
|
-
|
|
320
|
-
|
|
321
|
-
{ name: "refund_order", description: "Refunds an order", inputSchema, async execute(input, { executionId }) { /* ... */ } },
|
|
322
|
-
];
|
|
323
|
-
```
|
|
114
|
+
import { createFxAgent, memory } from "libfx";
|
|
115
|
+
import { local } from "libfx/durable-local";
|
|
324
116
|
|
|
325
|
-
|
|
326
|
-
|
|
327
|
-
|
|
328
|
-
so it can check the call's effects before calling it again or ask the user.
|
|
117
|
+
createFxAgent({ durability: local({ dir: ".fx/sessions" }) });
|
|
118
|
+
createFxAgent({ durability: memory() }); // nothing survives the process
|
|
119
|
+
```
|
|
329
120
|
|
|
330
|
-
|
|
331
|
-
crash. Pass it to the service the tool calls, as an idempotency key, so a call
|
|
332
|
-
that runs again changes nothing twice. Calls to idempotent tools also start
|
|
333
|
-
without waiting for their record to be stored.
|
|
121
|
+
To keep sessions in your own database, create a World such as Workflow's Postgres World and pass it to `world()` from `libfx/durable-world`. The [durability guide](https://fx.sh/docs/lib/durability) shows how.
|
|
334
122
|
|
|
335
|
-
|
|
123
|
+
With `local()` and `vercel()`, a session id is a Workflow run id. Take it from `turn.accepted` or `await agent.newSessionId()` instead of making one up.
|
|
336
124
|
|
|
337
|
-
|
|
125
|
+
A session stores only the conversation. Every process passes its own model, credentials, tools, MCP clients and skills when it creates the agent.
|
|
338
126
|
|
|
339
|
-
|
|
340
|
-
const agent = createFxAgent({
|
|
341
|
-
model: { id: "anthropic/claude-opus-5.5-fast", effort: "low", fast: true },
|
|
342
|
-
});
|
|
343
|
-
```
|
|
127
|
+
### On Vercel
|
|
344
128
|
|
|
345
|
-
|
|
346
|
-
`createFxTerminal()`. A string `model` remains supported as shorthand.
|
|
347
|
-
Top-level `effort` and `fast` are deprecated but remain supported with a
|
|
348
|
-
string model or no model; they cannot be mixed with a model object. New code
|
|
349
|
-
should use the model object.
|
|
350
|
-
|
|
351
|
-
`model.effort` sets the reasoning effort for models that advertise effort
|
|
352
|
-
levels. It uses the same vocabulary as the fx CLI's `--effort` flag:
|
|
353
|
-
`"default"` leaves the choice to the model; named levels such as `"low"`,
|
|
354
|
-
`"medium"`, `"high"`, or `"xhigh"` request a specific level. A named level is
|
|
355
|
-
validated at creation; an unsupported level rejects with an Error carrying
|
|
356
|
-
`code: "LIBFX_MODEL_UNSUPPORTED_EFFORT"`, `model`, and
|
|
357
|
-
`capability: "effort"`. Its message names the supported set when available.
|
|
358
|
-
Omitting effort or using `"default"` leaves the model default in place.
|
|
359
|
-
|
|
360
|
-
`model.fast` enables the fast lane for models that advertise one, matching the
|
|
361
|
-
fx CLI's `--fast` flag. A model without a fast path rejects at creation with
|
|
362
|
-
`code: "LIBFX_MODEL_UNSUPPORTED_FAST"`, `model`, and `capability: "fast"`.
|
|
363
|
-
Omitting fast or setting it to `false` leaves the model default in place.
|
|
364
|
-
|
|
365
|
-
`model.ultrafast` requests Ultra mode. It is off by default and maps to
|
|
366
|
-
`openai.serviceTier: "ultrafast"` through the Vercel AI Gateway for models
|
|
367
|
-
whose metadata advertises Ultra eligibility. It uses the higher-cost service tier.
|
|
368
|
-
Set it to `true` only after the host has selected an eligible model; an
|
|
369
|
-
unsupported request rejects with `code: "LIBFX_MODEL_UNSUPPORTED_ULTRAFAST"`,
|
|
370
|
-
`model`, and `capability: "ultrafast"`. Set it to `false` to explicitly disable
|
|
371
|
-
an inherited request. Ultra and Fast are mutually exclusive, so `ultrafast:
|
|
372
|
-
true` disables Fast for the agent. Gateway metadata currently marks Astra
|
|
373
|
-
eligible; libfx does not select Ultra automatically.
|
|
374
|
-
|
|
375
|
-
The new codes replace `LIBFX_UNSUPPORTED_EFFORT` and `LIBFX_UNSUPPORTED_FAST`
|
|
376
|
-
for both nested and legacy top-level settings. Callers that check the old codes
|
|
377
|
-
must update their error handling.
|
|
378
|
-
|
|
379
|
-
The host selects the model. Agent creation does not fetch the Gateway model
|
|
380
|
-
catalog unless effort requests a named level, fast is enabled, or ultrafast is enabled. Prompting
|
|
381
|
-
can resolve model capabilities and context capacity through the supplied
|
|
382
|
-
`fetch`; fx caches that metadata for the agent.
|
|
383
|
-
|
|
384
|
-
`onEvent` receives runtime diagnostics separately from model output. Transport
|
|
385
|
-
events report request start, response status and elapsed time, safe Gateway
|
|
386
|
-
request metadata, failures, and throttled `transport.activity` liveness while a
|
|
387
|
-
response body is streaming. Activity events include the attempt, current chunk
|
|
388
|
-
bytes, and cumulative response bytes. They call the host directly at most once
|
|
389
|
-
per 250 ms rather than entering the normalized turn queue, so an unread turn
|
|
390
|
-
cannot accumulate heartbeat events. Credentials and raw headers are never
|
|
391
|
-
included.
|
|
392
|
-
|
|
393
|
-
libfx makes at most one automatic retry after a retryable transport failure and
|
|
394
|
-
only before model output or tool effects escape. Cancellation prevents a retry.
|
|
395
|
-
|
|
396
|
-
## A single engine
|
|
397
|
-
|
|
398
|
-
`createFxEngine()` is the kernel under `createFxAgent()`: one conversation,
|
|
399
|
-
held in the memory of the process that runs it, with no queue and no session
|
|
400
|
-
store. Use it to run turns inside one request and keep nothing, or to keep a
|
|
401
|
-
conversation in storage of your own through [persistence](#persistence). It
|
|
402
|
-
takes the same model, tool, MCP, and skill options, `apiKey` is required, and
|
|
403
|
-
it resolves once its backend has loaded. In this section, `agent` is an engine:
|
|
129
|
+
Add a route for libfx's queue deliveries:
|
|
404
130
|
|
|
405
131
|
```js
|
|
406
|
-
|
|
132
|
+
// app/api/libfx/route.js
|
|
133
|
+
import { agent } from "@/lib/agent";
|
|
407
134
|
|
|
408
|
-
const
|
|
135
|
+
export const POST = agent.wakeHandler();
|
|
409
136
|
```
|
|
410
137
|
|
|
411
|
-
|
|
138
|
+
Then subscribe it to libfx's topic in `vercel.json`:
|
|
412
139
|
|
|
413
|
-
|
|
414
|
-
|
|
415
|
-
|
|
416
|
-
|
|
417
|
-
|
|
418
|
-
|
|
419
|
-
|
|
420
|
-
|
|
421
|
-
|
|
422
|
-
`resume()`. A session has `steer()`, `cancel()`, and `resume()`.
|
|
423
|
-
- `persistence` and `sessionId` throw, because the durability holds the
|
|
424
|
-
session.
|
|
140
|
+
```json
|
|
141
|
+
{
|
|
142
|
+
"functions": {
|
|
143
|
+
"app/api/libfx/route.js": {
|
|
144
|
+
"experimentalTriggers": [{ "type": "queue/v2beta", "topic": "__libfx_wkf_workflow_session" }]
|
|
145
|
+
}
|
|
146
|
+
}
|
|
147
|
+
}
|
|
148
|
+
```
|
|
425
149
|
|
|
426
|
-
|
|
427
|
-
are marked `writes: true`, in `createFxEngine()` as well. WebAssembly runs
|
|
428
|
-
them one at a time.
|
|
150
|
+
That's all. `prompt()` sends each turn to Vercel Queues, and this route runs it. When a turn gets close to the function's time limit, libfx saves it and the next invocation continues it. When a function dies mid-turn, its session frees about 15 seconds after it last checked in, and the turn carries on in a new invocation. `vercel({ reserveMs, leaseMs })` from `libfx/durable-vercel` changes both timings.
|
|
429
151
|
|
|
430
|
-
###
|
|
152
|
+
### Tools with side effects
|
|
431
153
|
|
|
432
|
-
|
|
433
|
-
blocks. It returns an async iterable of normalized events:
|
|
154
|
+
When a process dies during a tool call, libfx can't know whether the call finished. Mark a tool `idempotent: true` if running it twice is harmless, like a lookup, and it runs again when the turn continues. Any other tool never runs twice on its own: the model gets an error saying the call may have partly run, and decides whether to check, try again or ask the user.
|
|
434
155
|
|
|
435
|
-
|
|
436
|
-
- `reasoning_delta` when supplied by the provider
|
|
437
|
-
- `tool_start` with `id`, `name`, and the tool's `input` object. Inputs over
|
|
438
|
-
64 KiB of JSON arrive as an `inputPreview` string prefix plus
|
|
439
|
-
`inputTruncated: true` instead; the tool still receives complete arguments.
|
|
440
|
-
- `tool_end`
|
|
441
|
-
- `user_message` with `text` when accepted mid-turn steering enters the turn
|
|
156
|
+
Each call also gets an `executionId` that stays the same when the call runs again. Pass it to the service your tool calls as an idempotency key, and a repeated call won't change anything twice.
|
|
442
157
|
|
|
443
|
-
|
|
444
|
-
tool results are lossless and backpressured: a slow reader pauses production
|
|
445
|
-
instead of growing an unlimited event queue. Awaiting only `turn.result` can wait for an unread stream to drain.
|
|
446
|
-
If you only need the result, explicitly discard events:
|
|
158
|
+
## Tools
|
|
447
159
|
|
|
448
160
|
```js
|
|
449
|
-
const
|
|
450
|
-
|
|
451
|
-
|
|
161
|
+
const agent = createFxAgent({
|
|
162
|
+
tools: [{
|
|
163
|
+
name: "get_order",
|
|
164
|
+
description: "Look up an order by id.",
|
|
165
|
+
inputSchema: { type: "object", properties: { id: { type: "string" } }, required: ["id"] },
|
|
166
|
+
idempotent: true,
|
|
167
|
+
async execute({ id }, { signal, executionId, context }) {
|
|
168
|
+
return orders.get(id, { signal });
|
|
169
|
+
},
|
|
170
|
+
}],
|
|
171
|
+
});
|
|
452
172
|
```
|
|
453
173
|
|
|
454
|
-
|
|
455
|
-
`turn.cancel()` and `agent.close()` also release blocked output. Embedded agents
|
|
456
|
-
have no implicit model-step cap, so hosts should cancel turns that exceed their
|
|
457
|
-
own budgets. The CLI's `max_agent_steps` setting does not apply to
|
|
458
|
-
`createFxEngine()`. Transport or message-decoding failures reject the result
|
|
459
|
-
instead of returning success with missing text.
|
|
460
|
-
|
|
461
|
-
Native transport buffers at most 8 MiB of output bytes. Unread SDK events apply
|
|
462
|
-
backpressure at 1 MiB of encoded messages or 256 events. One message can exceed
|
|
463
|
-
that threshold when the queue is empty; an individual encoded ACP message is
|
|
464
|
-
limited to 64 MiB on both backends. These are transport bounds, not a total
|
|
465
|
-
answer-size limit or a bound on retained conversation history.
|
|
466
|
-
|
|
467
|
-
Image blocks accept a `Blob` or `File` with a non-empty `type`, raw bytes
|
|
468
|
-
(`Uint8Array`, `Buffer`, `ArrayBuffer`, or another typed array) with an
|
|
469
|
-
explicit `mimeType`, or canonical base64 (no line wrapping) with an explicit
|
|
470
|
-
`mimeType`. The payload must be PNG, JPEG, GIF, or WebP. For a Blob, the MIME
|
|
471
|
-
type is inferred from `Blob.type`; an empty type or a conflicting explicit
|
|
472
|
-
`mimeType` rejects the input:
|
|
174
|
+
`execute` gets a `signal` that aborts when the turn is cancelled, the call's `executionId`, and the session's `context`. Return any JSON value, or `{ type: "libfx.tool-result", text, images }` to return images.
|
|
473
175
|
|
|
474
|
-
|
|
475
|
-
const turn = agent.prompt([
|
|
476
|
-
{ type: "text", text: "What does this screenshot show?" },
|
|
477
|
-
{ type: "image", data: file, sourceRef: "uploads:screenshot-1" }, // File or Blob, with file.type
|
|
478
|
-
// Or: { type: "image", data: pngBytes, mimeType: "image/png" }
|
|
479
|
-
// Or: { type: "image", data: base64Png, mimeType: "image/png" }
|
|
480
|
-
]);
|
|
481
|
-
```
|
|
176
|
+
On the native backend, the calls from one model response run at the same time. Mark a tool `writes: true` to keep it from overlapping with others. Add `{ name: "web_search", providerExecuted: true }` for AI Gateway's web search, which runs at the provider instead of in your code.
|
|
482
177
|
|
|
483
|
-
|
|
484
|
-
UTF-8 string of at most 512 bytes, without ASCII control characters (0–31 or
|
|
485
|
-
DEL). With a reference, you may omit `data` entirely:
|
|
178
|
+
### MCP and skills
|
|
486
179
|
|
|
487
180
|
```js
|
|
488
|
-
|
|
489
|
-
|
|
490
|
-
|
|
491
|
-
```
|
|
181
|
+
import { createMcpAdapter } from "libfx/mcp";
|
|
182
|
+
import { createSkillsAdapter } from "libfx/skills";
|
|
183
|
+
import { loadSkillFile } from "libfx/skills/node";
|
|
492
184
|
|
|
493
|
-
|
|
494
|
-
|
|
495
|
-
encoded as base64 only for the model request. A prompt may contain up to
|
|
496
|
-
8 images, including reference-only blocks. Each image may contain up to
|
|
497
|
-
3.75 MiB (3,932,160 bytes) of raw data, with at most 6 MiB of raw image data
|
|
498
|
-
per prompt. Once encoded for the model request, those limits are 5 MiB per
|
|
499
|
-
image and 8 MiB per prompt. The prompt's text, encoded images, and metadata
|
|
500
|
-
must fit within the same 8 MiB frame budget.
|
|
501
|
-
|
|
502
|
-
Without `resizeImage`, the SDK checks Blob size before reading it and the actual
|
|
503
|
-
byte count after reading it. If referenced image data would exceed a per-image,
|
|
504
|
-
aggregate image, or combined frame limit, the SDK sends its MIME type and
|
|
505
|
-
reference without image data. Referenced Blobs known to exceed those limits
|
|
506
|
-
remain unread and host-owned. A reference does not increase the limits;
|
|
507
|
-
unreferenced oversized input rejects with typed `RangeError`s.
|
|
508
|
-
|
|
509
|
-
Base64 that is not canonical throws a `TypeError` from `prompt()`. Raw bytes
|
|
510
|
-
are copied before `prompt()` returns, so the caller can reuse its buffer. Blob
|
|
511
|
-
reads are asynchronous: `prompt()` returns a turn, and read failures reject
|
|
512
|
-
`turn.result`. Cancelling or closing during a Blob read settles the turn without
|
|
513
|
-
sending its prompt. Without `resizeImage`, size errors for unreferenced base64
|
|
514
|
-
and raw-byte input throw synchronously from `prompt()`.
|
|
515
|
-
|
|
516
|
-
To downscale or convert prompt images before they are sent, pass the optional
|
|
517
|
-
`resizeImage` hook when creating the agent. It receives `{ bytes, mimeType }`,
|
|
518
|
-
where `bytes` is a `Uint8Array`, and returns `{ bytes, mimeType }` directly or
|
|
519
|
-
as a promise. Returned `bytes` may be any typed array or `ArrayBuffer`.
|
|
520
|
-
For a Node.js host that already uses `sharp`:
|
|
185
|
+
const mcp = await createMcpAdapter(client, { prefix: "github_" }); // your MCP SDK client
|
|
186
|
+
const skills = createSkillsAdapter([await loadSkillFile("./skills/review/SKILL.md")]);
|
|
521
187
|
|
|
522
|
-
|
|
523
|
-
|
|
524
|
-
|
|
525
|
-
const agent = await createFxEngine({
|
|
526
|
-
apiKey,
|
|
527
|
-
model,
|
|
528
|
-
async resizeImage({ bytes }) {
|
|
529
|
-
const png = await sharp(bytes).resize({ width: 1568, withoutEnlargement: true }).png().toBuffer();
|
|
530
|
-
return { bytes: png, mimeType: "image/png" };
|
|
531
|
-
},
|
|
188
|
+
const agent = createFxAgent({
|
|
189
|
+
tools: [...mcp.tools, ...skills.tools],
|
|
190
|
+
instructions: [mcp.instructions, skills.instructions].join("\n\n"),
|
|
532
191
|
});
|
|
533
|
-
```
|
|
534
192
|
|
|
535
|
-
|
|
536
|
-
|
|
537
|
-
|
|
538
|
-
apply to its output rather than its input; image count and frame metadata
|
|
539
|
-
limits still apply. Image prompts with data are prepared asynchronously like
|
|
540
|
-
a Blob prompt, and a failure inside the hook rejects `turn.result`. This
|
|
541
|
-
opt-in preprocessing reads supplied Blob data even when its original size
|
|
542
|
-
exceeds the limits. Reference-only blocks have no bytes and do not call the
|
|
543
|
-
hook or fetch the original. The hook is host-provided, not a built-in image
|
|
544
|
-
converter or a libfx runtime dependency.
|
|
545
|
-
|
|
546
|
-
The kernel sniffs the final bytes and compares them with the claimed MIME type
|
|
547
|
-
for every input form that carries data; a mismatch fails the turn with
|
|
548
|
-
`Invalid image prompt block`. Eligible images reach the model unchanged unless
|
|
549
|
-
your hook changes them. Images are routed only to models that advertise image
|
|
550
|
-
input; for any other model the turn fails with
|
|
551
|
-
`Image prompts are unavailable for the selected model` and no image bytes
|
|
552
|
-
leave the process. Images outside the request's pixel or encoded-size limits,
|
|
553
|
-
and reference-only originals, are withheld with model-visible recovery feedback
|
|
554
|
-
that includes their `sourceRef` when supplied.
|
|
555
|
-
|
|
556
|
-
A source reference is metadata, not an access grant or an automatic fetch.
|
|
557
|
-
Your host owns the original's lifetime, reference resolution, and authorization.
|
|
558
|
-
Expose preparation or retrieval through your existing tools' `execute()`
|
|
559
|
-
callbacks if the agent needs a smaller copy. libfx has no built-in image resizer,
|
|
560
|
-
shell, global converter, or source store, and a reference grants no native
|
|
561
|
-
filesystem authority. If a tool is absent or fails, the original remains
|
|
562
|
-
withheld rather than being sent anyway.
|
|
563
|
-
|
|
564
|
-
For example, a tool can delegate to your app's authorized image store:
|
|
565
|
-
|
|
566
|
-
```js
|
|
567
|
-
const prepareImage = {
|
|
568
|
-
name: "prepare_image",
|
|
569
|
-
description: "Return a new smaller copy of a host-owned image. Use the request limit for maxSide.",
|
|
570
|
-
inputSchema: {
|
|
571
|
-
type: "object",
|
|
572
|
-
properties: { sourceRef: { type: "string" }, maxSide: { type: "integer" } },
|
|
573
|
-
required: ["sourceRef", "maxSide"],
|
|
574
|
-
},
|
|
575
|
-
async execute({ sourceRef, maxSide }, { signal }) {
|
|
576
|
-
const copy = await imageStore.prepareCopy(sourceRef, { maxSide, signal });
|
|
577
|
-
return {
|
|
578
|
-
type: "libfx.tool-result",
|
|
579
|
-
text: "Prepared a new copy; original unchanged.",
|
|
580
|
-
images: [{ type: "image", data: copy.base64, mimeType: copy.mimeType, sourceRef }],
|
|
581
|
-
};
|
|
582
|
-
},
|
|
583
|
-
};
|
|
584
|
-
// Supply prepareImage in createFxEngine({ tools: [prepareImage], ... }).
|
|
193
|
+
// When you're done:
|
|
194
|
+
await agent.close();
|
|
195
|
+
await mcp.close();
|
|
585
196
|
```
|
|
586
197
|
|
|
587
|
-
|
|
588
|
-
validate the requested dimensions, and stop conversion when `signal` aborts.
|
|
589
|
-
The returned copy is checked again by the kernel before model submission.
|
|
590
|
-
Typed host-tool images still use base64 `data`; `resizeImage` prepares prompt
|
|
591
|
-
images only.
|
|
592
|
-
|
|
593
|
-
Version 2 checkpoints retain prompt images as raw image blobs and preserve
|
|
594
|
-
source refs within the existing 4 MiB checkpoint bound on both backends. A
|
|
595
|
-
checkpoint does not store a source file or host-owned original for a
|
|
596
|
-
reference-only block. On restoration, resupply your tools and restore the
|
|
597
|
-
sources those refs identify.
|
|
598
|
-
|
|
599
|
-
Only one top-level prompt may run at a time. While it runs,
|
|
600
|
-
`await turn.steer(text)` appends guidance at the next safe model boundary
|
|
601
|
-
without discarding the in-flight response or completed tool work. Steering also
|
|
602
|
-
accepts an array of text blocks; image and resource steering blocks are rejected.
|
|
603
|
-
Each message is limited to 64 KiB, with at most 64 queued messages and 1 MiB of
|
|
604
|
-
queued steering text. Accepted steering appears as a `user_message` event before
|
|
605
|
-
the model's continued output. For a Blob prompt or one prepared by
|
|
606
|
-
`resizeImage`, steering during that preparation waits for the prompt to be
|
|
607
|
-
sent; cancelling before then rejects the pending steering. Calling `steer()` after the turn settles rejects with
|
|
608
|
-
`no prompt is running`.
|
|
198
|
+
libfx doesn't own the MCP connection. You create the client, handle its auth and close it after the agent.
|
|
609
199
|
|
|
610
|
-
|
|
611
|
-
const turn = agent.prompt("Build the feature.");
|
|
612
|
-
for await (const event of turn) {
|
|
613
|
-
if (event.type === "tool_end" && event.name === "read_file") {
|
|
614
|
-
await turn.steer("Keep the public API backward compatible.");
|
|
615
|
-
}
|
|
616
|
-
}
|
|
617
|
-
```
|
|
618
|
-
|
|
619
|
-
`steer()` resolves to `{ id }`, and the returned promise carries the same `id`
|
|
620
|
-
at once. Until the model sees a steer, `turn.withdraw(id)` takes it back and
|
|
621
|
-
resolves to `withdrawn`; after that it resolves to `already_placed`. With
|
|
622
|
-
[persistence](#persistence), a steer resolves only once its acceptance is stored, a model request
|
|
623
|
-
carries it only after that, and a withdrawal resolves once it is stored, so a
|
|
624
|
-
crash never loses a steer the call reported as accepted: `agent.resume()`
|
|
625
|
-
delivers one the model had not seen yet. The web core accepts a steer when it
|
|
626
|
-
takes it at the next boundary, so there `steer()` resolves then, and a steer
|
|
627
|
-
still queued when the turn ends rejects.
|
|
628
|
-
|
|
629
|
-
`agent.followUp(input)` queues text to run as its own turn once the current
|
|
630
|
-
turn ends, or at once when none is running, instead of failing with
|
|
631
|
-
`a prompt is already in progress`. It returns a promise for that turn; the
|
|
632
|
-
promise carries the follow-up's `id` and `accepted`, which resolves `{ id }`
|
|
633
|
-
once it is stored. With persistence, a follow-up survives a crash. After a
|
|
634
|
-
restore, the follow-ups the session held have no caller, so each waits for
|
|
635
|
-
`agent.resume()`: every call continues the open turn first, then starts the
|
|
636
|
-
next held follow-up, and returns `null` once none is left. Follow-ups this
|
|
637
|
-
agent queues run on their own and do not wait behind held ones. Call `resume()`
|
|
638
|
-
until it returns `null` whenever a session opens, before this agent queues
|
|
639
|
-
follow-ups of its own: one queued during a resumed turn starts when that turn
|
|
640
|
-
ends, and `resume()` throws while it runs. A follow-up whose turn ends before
|
|
641
|
-
libfx records its first `turn_progress` is withdrawn, and one that ends after
|
|
642
|
-
it is recorded like any other turn, so a follow-up never runs twice. `close()`
|
|
643
|
-
rejects follow-ups that have not started; the session's records still hold them.
|
|
644
|
-
|
|
645
|
-
Cancelling a steered turn drops any guidance that has not reached a safe
|
|
646
|
-
boundary and releases its queue. Applied guidance is part of the same history
|
|
647
|
-
turn, so an idle `checkpoint()` includes the full steered conversation.
|
|
648
|
-
|
|
649
|
-
`checkpoint()` returns opaque, bounded, versioned bytes. Concurrent calls run
|
|
650
|
-
one at a time, and a call still waiting for an earlier one fails the same way a
|
|
651
|
-
direct call would if a prompt starts or the agent closes first. A newer libfx
|
|
652
|
-
restores checkpoints from older versions, but an older libfx cannot restore one
|
|
653
|
-
written by a newer version. Restore them only when creating a fresh agent:
|
|
200
|
+
## Models
|
|
654
201
|
|
|
655
202
|
```js
|
|
656
|
-
|
|
203
|
+
createFxAgent({ model: { id: "anthropic/claude-opus-5.5", effort: "low", fast: true } });
|
|
657
204
|
```
|
|
658
205
|
|
|
659
|
-
|
|
660
|
-
or a history change. The next prompt can run normally.
|
|
206
|
+
`effort` and `fast` mean the same as the fx CLI's `--effort` and `--fast` flags. `ultrafast: true` asks for the faster, more expensive service tier on models that offer it. A setting the model doesn't support fails with an error whose `code` names it, such as `LIBFX_MODEL_UNSUPPORTED_EFFORT`.
|
|
661
207
|
|
|
662
|
-
|
|
663
|
-
durable storage and must resupply models, credentials, instructions, tools,
|
|
664
|
-
MCP clients, and skill records. Reasoning effort, Fast mode, and Ultra mode are
|
|
665
|
-
agent-creation options and are not stored in a checkpoint: recreate the agent
|
|
666
|
-
with new `model.effort`, `model.fast`, or `model.ultrafast` values to change
|
|
667
|
-
them, the same path as switching models.
|
|
208
|
+
`listModels({ apiKey })` returns the Gateway's language models. Behind a proxy that only forwards chat requests, pass the catalog yourself as `modelCatalog`, and the agent won't fetch it.
|
|
668
209
|
|
|
669
|
-
|
|
210
|
+
`onEvent` receives diagnostics apart from the model's output: request timing, and session events such as `session.deadline` and `checkpoint.mismatch`. Credentials never appear in them.
|
|
670
211
|
|
|
671
|
-
|
|
672
|
-
new agent can continue it after the process running it stops. libfx writes
|
|
673
|
-
opaque records and checkpoints to the store and reads them back. The store
|
|
674
|
-
never parses them, and libfx never sees how they are stored.
|
|
212
|
+
### ChatGPT plans
|
|
675
213
|
|
|
676
|
-
|
|
677
|
-
import { createFxEngine, createMemoryPersistence } from "libfx";
|
|
214
|
+
An agent can run on a user's ChatGPT plan through Codex instead of AI Gateway. Set `provider: "codex"` and hand libfx the user's ChatGPT credentials:
|
|
678
215
|
|
|
679
|
-
|
|
680
|
-
const agent =
|
|
216
|
+
```js
|
|
217
|
+
const agent = createFxAgent({
|
|
218
|
+
provider: "codex",
|
|
219
|
+
model: "gpt-6.1-sol",
|
|
220
|
+
credentials: ({ provider, sessionId }) => tokens.load(provider, sessionId),
|
|
221
|
+
onCredentials: (credentials, { provider, sessionId }) => tokens.save(provider, sessionId, credentials),
|
|
222
|
+
});
|
|
681
223
|
```
|
|
682
224
|
|
|
683
|
-
|
|
225
|
+
libfx calls `credentials()` before Codex requests, in whichever process runs the turn, and adds the credentials to those requests itself, so they never reach the engine or a saved session. It refreshes them shortly before they expire and once after a rejected request, then passes the new ones to `onCredentials` for you to save. Codex doesn't accept requests from web pages, so these turns run on your server.
|
|
684
226
|
|
|
685
|
-
|
|
686
|
-
interface Persistence {
|
|
687
|
-
load(): Promise<{
|
|
688
|
-
checkpoint?: { data: Uint8Array; through: string };
|
|
689
|
-
journal?: Iterable<JournalRecord> | AsyncIterable<JournalRecord>;
|
|
690
|
-
}>;
|
|
691
|
-
append(input: { expected: string | null; idempotencyKey: string; data: Uint8Array }): Promise<{ cursor: string }>;
|
|
692
|
-
saveCheckpoint?(input: { through: string; data: Uint8Array }): Promise<void>;
|
|
693
|
-
}
|
|
694
|
-
|
|
695
|
-
interface JournalRecord {
|
|
696
|
-
cursor: string;
|
|
697
|
-
data: Uint8Array;
|
|
698
|
-
}
|
|
699
|
-
```
|
|
700
|
-
|
|
701
|
-
`append()` stores one record and resolves to its cursor, a string the store
|
|
702
|
-
chooses that places the record after the ones before it. The session's records
|
|
703
|
-
are its journal. libfx sends records one at a time, in order, and `expected`
|
|
704
|
-
is the cursor of the last record the agent knows of, or `null` for a new
|
|
705
|
-
session. Reject the append with an `FxFencedError` when `expected` is not the
|
|
706
|
-
cursor of the last stored record, so that a second agent writing the same
|
|
707
|
-
session stops the first instead of interleaving with it. `idempotencyKey` is
|
|
708
|
-
unique to each write, so a store that retries a write can return the stored
|
|
709
|
-
record's cursor instead of storing it twice.
|
|
710
|
-
|
|
711
|
-
`createFxEngine()` calls `load()` once. Resolve to the latest checkpoint, if any,
|
|
712
|
-
and the records stored after it, oldest first, each with its cursor. Resolve to
|
|
713
|
-
`{}` for a session with nothing stored. libfx skips any record the checkpoint
|
|
714
|
-
already covers.
|
|
715
|
-
|
|
716
|
-
libfx waits for the store only where a crash could otherwise lose work or
|
|
717
|
-
repeat it: the first record of each turn, which holds the prompt, is stored
|
|
718
|
-
before the model sees it, and a response's tool calls are stored before any of
|
|
719
|
-
them starts. Other appends are not waited for, so a turn's `result` can settle
|
|
720
|
-
before its last records land, and `agent.close()` waits for every append. If
|
|
721
|
-
`append` rejects, libfx stops the current turn at once: it starts no further
|
|
722
|
-
model request or tool call and writes nothing more. The turn fails with an
|
|
723
|
-
error whose `code` is `FX_JOURNAL_APPEND_FAILED` and whose `cause` is your
|
|
724
|
-
error, and the agent refuses later prompts. When no turn was left to report
|
|
725
|
-
the failure, `close()` rejects with it. A model request the turn started while
|
|
726
|
-
the failed append was in flight can still complete.
|
|
727
|
-
|
|
728
|
-
When the store has `saveCheckpoint()`, libfx saves a checkpoint after a turn
|
|
729
|
-
ends or yields once the records stored since the last checkpoint reach
|
|
730
|
-
`checkpointAfterBytes` (default 1 MiB). A checkpoint where a turn yielded
|
|
731
|
-
holds the open turn, and the next load continues it. `through` is the cursor of the last
|
|
732
|
-
record the checkpoint covers, so `load()` can return that checkpoint and only
|
|
733
|
-
the records after it, and the cost of opening a session stays bounded as its
|
|
734
|
-
history grows. libfx takes a checkpoint only while no turn is running. A saved
|
|
735
|
-
checkpoint emits a `checkpoint.save` event. A failed save emits
|
|
736
|
-
`checkpoint.error` and changes nothing: the records still restore the session,
|
|
737
|
-
and the next turn's end tries again. Deleting the records a checkpoint covers
|
|
738
|
-
is up to the store.
|
|
739
|
-
|
|
740
|
-
The records a load reads, after libfx drops the older progress of an open
|
|
741
|
-
turn, must fit in 4 MiB of JSON, a checkpoint in 4 MiB, and the history in
|
|
742
|
-
1,024 turns. Otherwise `createFxEngine()` rejects with an error whose `code` is
|
|
743
|
-
`FX_JOURNAL_TOO_LARGE`. libfx refuses records that repeat, skip, or do not
|
|
744
|
-
parse, and `createFxEngine()` rejects with the reason and the `code`
|
|
745
|
-
`FX_JOURNAL_INVALID`. For records written by a newer libfx, it rejects with an
|
|
746
|
-
`FxJournalVersionError`, whose `code` is `FX_JOURNAL_VERSION`. Each libfx
|
|
747
|
-
release resumes the sessions and checkpoints that the release before it saved,
|
|
748
|
-
so upgrade the processes that read a session before the ones that write it.
|
|
749
|
-
|
|
750
|
-
AI Gateway keys session affinity and prompt caching to the session's id. libfx
|
|
751
|
-
picks a new id for each agent unless you pass `sessionId`, so give a restored
|
|
752
|
-
session the id it had before. An id is 1 to 255 letters, digits, `.`, `_`, or
|
|
753
|
-
`-`. `agent.sessionId` is the id the agent uses.
|
|
754
|
-
|
|
755
|
-
If the last process stopped during a turn, `agent.resume()` continues that
|
|
756
|
-
turn with no new input and returns it, like `prompt()`. The model is told
|
|
757
|
-
"Resuming from unexpected session interruption.". A tool call that was
|
|
758
|
-
running does not run again: it comes back answered with an error saying it may
|
|
759
|
-
have partly run, and the model decides whether to check its effects, call it
|
|
760
|
-
again, or ask the user. A turn that fails or is cancelled ends in the journal
|
|
761
|
-
as it does in the agent, so only a stopped process leaves one to resume.
|
|
762
|
-
`resume()` returns `null` when the session holds no open turn and no held
|
|
763
|
-
follow-up, so a host can call it until it does every time it opens a session:
|
|
227
|
+
Sessions in one process share each refresh. If your app runs on several servers, two of them can notice the same expiring credentials at once, and the provider accepts only the first refresh. Pass `refreshLock` so they take turns:
|
|
764
228
|
|
|
765
229
|
```js
|
|
766
|
-
|
|
767
|
-
for (let turn = agent.resume(); turn; turn = agent.resume()) {
|
|
768
|
-
for await (const event of turn) console.log(event);
|
|
769
|
-
await turn.result;
|
|
770
|
-
}
|
|
230
|
+
refreshLock: ({ provider, sessionId }, refresh) => locks.withLock(`${provider}:${accountOf(sessionId)}`, refresh),
|
|
771
231
|
```
|
|
772
232
|
|
|
773
|
-
|
|
774
|
-
process before the call did anything, pass `resume({ onAmbiguous })`. libfx
|
|
775
|
-
calls `onAmbiguous({ executionId, name, input })` once for each call the turn
|
|
776
|
-
left running, and each call it returns `"rerun"` for runs again under the same
|
|
777
|
-
`executionId` before the turn continues. Every other call keeps the answer that it may
|
|
778
|
-
have partly run.
|
|
779
|
-
|
|
780
|
-
Calling `prompt()` instead ends the open turn as interrupted and starts a new
|
|
781
|
-
one.
|
|
782
|
-
|
|
783
|
-
Give a prompt a `turnId` when the same request can reach libfx more than once,
|
|
784
|
-
such as from a queue that delivers it again after a crash. When the open turn
|
|
785
|
-
has that id, `prompt(input, { turnId })` continues it as `resume()` does, and
|
|
786
|
-
takes `onAmbiguous` the same way. When
|
|
787
|
-
the session's last turn ended with that id, it returns a turn that has already
|
|
788
|
-
ended, with the stop reason `end_turn`, without calling the model. Any other id
|
|
789
|
-
starts a new turn. A turn id is 1 to 128 letters, digits, `.`, `_`, or `-`.
|
|
790
|
-
Without one, libfx names the turn. `turn.id` is the turn's id.
|
|
791
|
-
|
|
792
|
-
To move a running session to another process, call
|
|
793
|
-
`turn.cancel({ reason: "handoff" })`. The turn stops at once, like any
|
|
794
|
-
cancellation, but libfx stores nothing more, so the journal keeps the turn
|
|
795
|
-
open and the agent that opens the session next continues it with `resume()`.
|
|
796
|
-
The handed-off agent then refuses `prompt()`, `resume()`, and `followUp()`;
|
|
797
|
-
close it. A handoff needs persistence.
|
|
798
|
-
|
|
799
|
-
Pass `persistence` instead of a `checkpoint` option; the two cannot be
|
|
800
|
-
combined. Like a checkpoint, persistence holds conversation history only:
|
|
801
|
-
resupply models, credentials, instructions, tools, MCP clients, and skill
|
|
802
|
-
records when you create the agent. A resumed turn continues under the tools the
|
|
803
|
-
new agent has. `createMemoryPersistence()` keeps the records and the latest
|
|
804
|
-
checkpoint in memory as `persistence.records` and `persistence.checkpoint`,
|
|
805
|
-
which suits tests and hosts that copy them to their own storage. It fences an
|
|
806
|
-
append whose `expected` cursor is not its last record's, and returns the stored
|
|
807
|
-
cursor for a repeated `idempotencyKey`. `FxFencedError` is exported by `libfx`;
|
|
808
|
-
its `code` is `FX_FENCED`, which a store of your own can use for the same case.
|
|
809
|
-
`FxJournalVersionError` is exported by `libfx` too.
|
|
233
|
+
libfx calls it with a function to run while you hold a lock on that user's account, for example in Redis. Inside it, libfx asks `credentials()` again and uses what another server already saved. This works for Grok too.
|
|
810
234
|
|
|
811
|
-
|
|
812
|
-
|
|
813
|
-
Model discovery is explicit and does not create an Agent or load native or Wasm
|
|
814
|
-
artifacts:
|
|
235
|
+
To sign a user in, use `libfx/chatgpt`:
|
|
815
236
|
|
|
816
237
|
```js
|
|
817
|
-
import {
|
|
238
|
+
import { startSignIn, finishSignIn } from "libfx/chatgpt";
|
|
818
239
|
|
|
819
|
-
|
|
820
|
-
|
|
821
|
-
|
|
240
|
+
// One request: send the user to signIn.url and keep signIn on the server.
|
|
241
|
+
const signIn = await startSignIn();
|
|
242
|
+
|
|
243
|
+
// A later request: the user pastes the address OpenAI opened.
|
|
244
|
+
const credentials = await finishSignIn(signIn, pastedAddress);
|
|
822
245
|
```
|
|
823
246
|
|
|
824
|
-
`
|
|
825
|
-
language-model IDs. It accepts the same optional `fetch` override as the Agent
|
|
826
|
-
API.
|
|
827
|
-
|
|
828
|
-
An agent reads the AI Gateway model catalog to learn what its model supports,
|
|
829
|
-
such as image input, reasoning effort, and output limits, and those details
|
|
830
|
-
shape every request. It fetches the catalog through your `fetch`. If that
|
|
831
|
-
request fails, for example behind a proxy that only forwards chat requests,
|
|
832
|
-
the agent cannot confirm the model's capabilities and refuses image prompts.
|
|
833
|
-
Pass `modelCatalog` to supply the catalog instead: the entries from
|
|
834
|
-
`https://ai-gateway.vercel.sh/coding-agent/v1/models`, either the response's
|
|
835
|
-
`{ data }` or the array, or only the entries for the models you use. The agent
|
|
836
|
-
then makes no catalog request, so every process that receives the same
|
|
837
|
-
entries builds the same requests:
|
|
247
|
+
After the user signs in, OpenAI opens a page at `localhost:1455` that doesn't load. That's expected. Ask the user to copy the page's whole address and paste it back. If their account or ChatGPT workspace allows device code sign-in, `startDeviceSignIn()` and `pollDeviceSignIn()` show a short code to enter on OpenAI's site instead. Each step returns plain data, so it can finish in another request or function.
|
|
838
248
|
|
|
839
|
-
|
|
840
|
-
const agent = createFxAgent({ apiKey, model, modelCatalog });
|
|
841
|
-
```
|
|
249
|
+
The sign-in uses the same OpenAI client as the Codex CLI, and the user's ChatGPT plan and OpenAI's terms apply to the turns.
|
|
842
250
|
|
|
843
|
-
|
|
251
|
+
### Grok plans
|
|
252
|
+
|
|
253
|
+
An agent can also run on a user's Grok plan. Set `provider: "grok"` and pass the same `credentials` callback, which libfx calls with `provider: "grok"`:
|
|
844
254
|
|
|
845
255
|
```js
|
|
846
256
|
const agent = createFxAgent({
|
|
847
|
-
|
|
848
|
-
model,
|
|
849
|
-
|
|
850
|
-
|
|
851
|
-
name: "lookup",
|
|
852
|
-
description: "Look up a value.",
|
|
853
|
-
inputSchema: {
|
|
854
|
-
type: "object",
|
|
855
|
-
properties: { key: { type: "string" } },
|
|
856
|
-
required: ["key"],
|
|
857
|
-
},
|
|
858
|
-
async execute(input, { signal }) {
|
|
859
|
-
return database.get(input.key, { signal });
|
|
860
|
-
},
|
|
861
|
-
}],
|
|
257
|
+
provider: "grok",
|
|
258
|
+
model: "grok-4.5",
|
|
259
|
+
credentials: ({ provider, sessionId }) => tokens.load(provider, sessionId),
|
|
260
|
+
onCredentials: (credentials, { provider, sessionId }) => tokens.save(provider, sessionId, credentials),
|
|
862
261
|
});
|
|
863
262
|
```
|
|
864
263
|
|
|
865
|
-
|
|
866
|
-
|
|
264
|
+
libfx handles Grok credentials the same way as Codex's: it adds them to Grok requests itself, refreshes them, and checks that a refreshed token still belongs to the same xAI account. Grok doesn't accept requests from web pages either, so these turns run on your server.
|
|
265
|
+
|
|
266
|
+
To sign a user in, use `libfx/grok`. Device code is the simplest:
|
|
867
267
|
|
|
868
268
|
```js
|
|
869
|
-
|
|
870
|
-
apiKey,
|
|
871
|
-
tools: [{ name: "web_search", providerExecuted: true }],
|
|
872
|
-
});
|
|
873
|
-
```
|
|
269
|
+
import { startDeviceSignIn, completeDeviceSignIn } from "libfx/grok";
|
|
874
270
|
|
|
875
|
-
|
|
876
|
-
|
|
877
|
-
names reject agent creation. Provider-executed tools do not
|
|
878
|
-
call host code or require a separate provider key; their `tool_start` and
|
|
879
|
-
`tool_end` events, results, and checkpoint history use the same turn contract.
|
|
880
|
-
Their built-in permission policy is enforced when the request is projected, as
|
|
881
|
-
there is no local call-time effect to approve.
|
|
882
|
-
|
|
883
|
-
`tools` may also be an object keyed by tool name, such as
|
|
884
|
-
`tools: { lookup, save }`, where each value is a descriptor without `name`.
|
|
885
|
-
A descriptor that does include `name` must match its key.
|
|
886
|
-
|
|
887
|
-
When one model response calls several tools, the native backend runs them at
|
|
888
|
-
the same time and returns their results to the model in the order it called
|
|
889
|
-
them. Mark a tool `writes: true` when its calls must not overlap others: it
|
|
890
|
-
starts after every earlier call in the response finishes, and later calls
|
|
891
|
-
wait for it. WebAssembly runs calls one at a time. Every call still goes
|
|
892
|
-
through its own permission check before any of them starts.
|
|
893
|
-
|
|
894
|
-
`execute` receives `{ signal, executionId, sessionId }`, plus `context` when
|
|
895
|
-
the turn has one. `executionId` is the model's id for the call and stays the same when
|
|
896
|
-
the call runs again after a crash, so a tool with an external effect can pass
|
|
897
|
-
it on as an idempotency key. [Tools with effects](#tools-with-effects)
|
|
898
|
-
describes which calls run again.
|
|
899
|
-
When a tool rejects with an empty message, the model receives a non-empty error
|
|
900
|
-
so the provider does not refuse the conversation.
|
|
901
|
-
|
|
902
|
-
For ordinary tools, the JavaScript host is the authority for effects. The same
|
|
903
|
-
descriptors, schemas, cancellation, results, and events are used by N-API and
|
|
904
|
-
WebAssembly.
|
|
905
|
-
Cancelling a prompt aborts its tools' signals and stops waiting for their
|
|
906
|
-
callbacks. Late results and rejections are ignored. Tools remain responsible
|
|
907
|
-
for stopping their own work when their signal is aborted.
|
|
908
|
-
|
|
909
|
-
A host tool may use any name, including the kernel's builtin names such as
|
|
910
|
-
`write_file` and `edit_file`: the kernel routes by the registered executor, so
|
|
911
|
-
a host-defined `write_file` calls the host's `execute()` rather than the
|
|
912
|
-
builtin file mutation.
|
|
913
|
-
|
|
914
|
-
Ordinary objects returned by tools are JSON text. To return rich image content,
|
|
915
|
-
use the typed result:
|
|
271
|
+
// Show device.userCode and a link to device.verificationUrl.
|
|
272
|
+
const device = await startDeviceSignIn();
|
|
916
273
|
|
|
917
|
-
|
|
918
|
-
|
|
919
|
-
type: "libfx.tool-result",
|
|
920
|
-
text: "Original is available through the host image tools.",
|
|
921
|
-
images: [
|
|
922
|
-
{ type: "image", mimeType: "image/png", sourceRef: "uploads:screenshot-1" },
|
|
923
|
-
// A prepared copy may also include data: base64Png.
|
|
924
|
-
],
|
|
925
|
-
};
|
|
274
|
+
// Wait until the user enters the code. pollDeviceSignIn() checks once, for a later request.
|
|
275
|
+
const credentials = await completeDeviceSignIn(device);
|
|
926
276
|
```
|
|
927
277
|
|
|
928
|
-
|
|
929
|
-
`data`. They do not accept raw bytes or Blobs and do not run `resizeImage`.
|
|
930
|
-
References use the same validation, ownership, recovery feedback, and checkpoint
|
|
931
|
-
rules as prompt images. A result may contain up to 8 images, each with up to
|
|
932
|
-
5 MiB of base64-encoded data. Its text, encoded images, and metadata must fit
|
|
933
|
-
within the 8 MiB result/frame bound. Referenced data is omitted if those bounds
|
|
934
|
-
would overflow; an unreferenced oversized result remains a tool error.
|
|
935
|
-
|
|
936
|
-
Instructions are limited to 64 KiB of UTF-8 text, including text assembled by
|
|
937
|
-
the MCP and skills adapters. They are the complete host-owned system context:
|
|
938
|
-
libfx adds no hidden base prompt, and omitting `instructions` sends no system
|
|
939
|
-
message.
|
|
940
|
-
|
|
941
|
-
## MCP
|
|
942
|
-
|
|
943
|
-
`libfx/mcp` accepts a host-owned MCP client. Transport, authentication,
|
|
944
|
-
elicitation, and cleanup remain outside the kernel. The client uses the MCP
|
|
945
|
-
TypeScript SDK v1 signature: `callTool(params, resultSchema?, options?)`, with
|
|
946
|
-
cancellation passed in `options`. Tool text and structured data
|
|
947
|
-
reach the model together. PNG, JPEG, GIF, and WebP tool images reach models that
|
|
948
|
-
advertise image input support; other models receive an explicit omission notice.
|
|
949
|
-
Images are retained in checkpoints within the existing checkpoint size limit.
|
|
950
|
-
Each image may contain up to 5 MiB of base64 data, with at most eight images and
|
|
951
|
-
an 8 MiB result frame. Ordinary host tool objects remain JSON text. Resource and
|
|
952
|
-
prompt options supply text instructions; non-text context has an omission notice.
|
|
953
|
-
Tool catalogs are paginated up to the existing 64-tool bound. Tool names are
|
|
954
|
-
normalized for model APIs, with collisions kept distinct and original names used
|
|
955
|
-
for calls to the MCP client. Each tool description and JSON schema may contain up
|
|
956
|
-
to 64 KiB, within the control message's 8 MiB limit.
|
|
278
|
+
`startSignIn()` and `finishSignIn()` work like ChatGPT's: after the user signs in, xAI opens a page at `127.0.0.1` that doesn't load, and the user pastes its address back. The sign-in uses the same xAI client as the fx CLI, and the user's Grok plan and xAI's terms apply to the turns.
|
|
957
279
|
|
|
958
|
-
|
|
959
|
-
import { createMcpAdapter } from "libfx/mcp";
|
|
280
|
+
### Custom providers
|
|
960
281
|
|
|
961
|
-
|
|
962
|
-
prefix: "github_",
|
|
963
|
-
resources: ["repo://instructions"],
|
|
964
|
-
prompts: ["review"],
|
|
965
|
-
});
|
|
282
|
+
An agent can also run on any server that speaks the OpenAI Chat Completions API, such as a local model server or a router. Pass the provider as an object and name the model it serves:
|
|
966
283
|
|
|
284
|
+
```js
|
|
967
285
|
const agent = createFxAgent({
|
|
968
|
-
|
|
969
|
-
model,
|
|
970
|
-
tools: mcp.tools,
|
|
971
|
-
instructions: mcp.instructions,
|
|
286
|
+
provider: { name: "local", baseUrl: "http://localhost:11434/v1" },
|
|
287
|
+
model: "qwen3:8b",
|
|
972
288
|
});
|
|
973
|
-
|
|
974
|
-
// ...
|
|
975
|
-
await agent.close();
|
|
976
|
-
await mcp.close();
|
|
977
289
|
```
|
|
978
290
|
|
|
979
|
-
|
|
980
|
-
|
|
981
|
-
Use `libfx/skills` for already-loaded records or `libfx/skills/node` to load a
|
|
982
|
-
`SKILL.md` explicitly in Node or Bun.
|
|
291
|
+
Requests go to `{baseUrl}/chat/completions`, which must use HTTPS, or HTTP on `localhost`. A server that needs a key gets it from the same `credentials` callback, which libfx calls with the provider's `name`:
|
|
983
292
|
|
|
984
293
|
```js
|
|
985
|
-
|
|
986
|
-
|
|
987
|
-
|
|
988
|
-
|
|
989
|
-
|
|
990
|
-
|
|
294
|
+
const agent = createFxAgent({
|
|
295
|
+
provider: {
|
|
296
|
+
name: "router",
|
|
297
|
+
baseUrl: "https://openrouter.ai/api/v1",
|
|
298
|
+
models: { "qwen/qwen3-coder": { contextWindow: 128000, supportsToolUse: true } },
|
|
299
|
+
},
|
|
300
|
+
model: "qwen/qwen3-coder",
|
|
301
|
+
credentials: async ({ provider, sessionId }) => ({ apiKey: await keys.load(provider, sessionId) }),
|
|
302
|
+
});
|
|
991
303
|
```
|
|
992
304
|
|
|
993
|
-
|
|
305
|
+
libfx sends the key as a bearer token to that chat URL only, and the key never reaches the engine or a saved session. A key isn't refreshed, so a rejected one ends the turn with the server's error. `models` tells libfx what it can't ask the server: each model's context window and output limit, and whether it takes tools and images. Tool calls work when the model and server support them. In a browser, the server must accept cross-origin requests, and the page can read the key.
|
|
994
306
|
|
|
995
|
-
|
|
996
|
-
createFxAgent({ apiKey, backend: "auto" }); // native, then Wasm fallback
|
|
997
|
-
createFxAgent({ apiKey, backend: "native" }); // require N-API
|
|
998
|
-
createFxAgent({ apiKey, backend: "wasm" }); // require Wasm + JSPI
|
|
999
|
-
```
|
|
307
|
+
Saved sessions follow the agent's provider: point an agent at another base URL and its sessions continue there, history included.
|
|
1000
308
|
|
|
1001
|
-
|
|
1002
|
-
package chooses its generated CommonJS entry automatically and keeps asset
|
|
1003
|
-
paths relative to the installed package.
|
|
309
|
+
## One conversation, no sessions
|
|
1004
310
|
|
|
1005
|
-
|
|
1006
|
-
Agent or terminal:
|
|
311
|
+
`createFxEngine()` is the engine under the agent: one conversation in the memory of the process that runs it, with no queue and no session store. Use it for a single request, or to keep the conversation in storage of your own.
|
|
1007
312
|
|
|
1008
313
|
```js
|
|
1009
|
-
import {
|
|
1010
|
-
|
|
1011
|
-
const info = await getBackendInfo({ surface: "agent", backend: "auto" });
|
|
1012
|
-
// {
|
|
1013
|
-
// surface: "agent",
|
|
1014
|
-
// backend: "native" | "wasm-jspi" | "unavailable",
|
|
1015
|
-
// attempts: [{ backend, available, reason }]
|
|
1016
|
-
// }
|
|
1017
|
-
```
|
|
1018
|
-
|
|
1019
|
-
`surface` is `agent` by default and may also be `terminal`. `backend` has the
|
|
1020
|
-
same `auto`, `native`, and `wasm` selection as the factories. `nativeAddon` and
|
|
1021
|
-
`wasm` select explicit assets with the same meanings as their factory options.
|
|
1022
|
-
The probe loads and validates the native module or compiles the selected Wasm
|
|
1023
|
-
asset, then stops. It does not create a core, open a runtime socket, read
|
|
1024
|
-
credentials, start a model request, or write session state. A remote Wasm
|
|
1025
|
-
source can perform its normal asset fetch.
|
|
1026
|
-
|
|
1027
|
-
Expected environmental failures resolve as structured attempts. Invalid
|
|
1028
|
-
options reject with `TypeError`. The stable reason codes are:
|
|
314
|
+
import { createFxEngine } from "libfx";
|
|
1029
315
|
|
|
1030
|
-
|
|
1031
|
-
| --- | --- |
|
|
1032
|
-
| `LIBFX_UNSUPPORTED_PLATFORM` | No packaged native addon supports the current platform and architecture. |
|
|
1033
|
-
| `LIBFX_NATIVE_ARTIFACT_MISSING` | The selected native addon file is absent. |
|
|
1034
|
-
| `LIBFX_NATIVE_LOAD_FAILED` | Node could not load the selected native addon. |
|
|
1035
|
-
| `LIBFX_NATIVE_API_MISMATCH` | The addon API version is incompatible. |
|
|
1036
|
-
| `LIBFX_NATIVE_SURFACE_MISSING` | The addon does not implement the selected Agent or terminal surface. |
|
|
1037
|
-
| `LIBFX_NATIVE_DISABLED` | `nativeAddon: false` disabled native loading. |
|
|
1038
|
-
| `LIBFX_JSPI_UNAVAILABLE` | The current JavaScript runtime does not provide JSPI. |
|
|
1039
|
-
| `LIBFX_WASM_LOAD_FAILED` | The selected Wasm asset could not be loaded or compiled. |
|
|
1040
|
-
|
|
1041
|
-
The optional `causeCode` field retains a Node error code such as `ENOENT` or
|
|
1042
|
-
`ERR_DLOPEN_FAILED` when one exists. Probe success establishes backend loading
|
|
1043
|
-
only; it does not validate credentials, a future Agent initialization, or a
|
|
1044
|
-
model request.
|
|
1045
|
-
|
|
1046
|
-
Within one loaded SDK module, libfx compiles each stable Wasm source once and
|
|
1047
|
-
creates a separate WebAssembly instance for every Agent. Agent memory, history,
|
|
1048
|
-
tools, cancellation, and shutdown remain isolated. Workers and separate
|
|
1049
|
-
processes maintain their own module caches, as do the ESM and CommonJS entries.
|
|
1050
|
-
|
|
1051
|
-
Node factories and `getBackendInfo()` also accept a `wasm` Promise resolving
|
|
1052
|
-
to an HTTP(S) URL string, a `Response`, Wasm bytes, or a compiled
|
|
1053
|
-
`WebAssembly.Module`. Asset resolver failures propagate from factories and
|
|
1054
|
-
appear as `LIBFX_WASM_LOAD_FAILED` in diagnostics.
|
|
1055
|
-
|
|
1056
|
-
Node.js 20+ is supported. Browser WebAssembly requires a JSPI-capable browser.
|
|
1057
|
-
The Linux x64 and arm64 native addons require glibc 2.34 or newer. Native
|
|
1058
|
-
agents do not require JSPI or experimental Node flags.
|
|
1059
|
-
Some Node versions require `--experimental-wasm-jspi`.
|
|
1060
|
-
Bun 1.4.2 is the tested recommendation for Bun's WebAssembly backend.
|
|
1061
|
-
Bun 1.3.14 can crash when a hot WebAssembly loop resumes through JSPI during
|
|
1062
|
-
JIT tier-up.
|
|
1063
|
-
|
|
1064
|
-
### Next.js and Vercel
|
|
1065
|
-
|
|
1066
|
-
Create agents in a server route using the Node.js runtime. Import `libfx`
|
|
1067
|
-
normally; the package includes its native assets and exposes both ESM and
|
|
1068
|
-
CommonJS Node entrypoints. Native agents support Next.js 15 with webpack and
|
|
1069
|
-
Next.js 16 with webpack or Turbopack, without `serverExternalPackages` or manual
|
|
1070
|
-
native-file inclusion. Webpack's emitted assets are resolved relative to the
|
|
1071
|
-
server bundle, including standalone builds with a custom `distDir` or `assetPrefix`.
|
|
1072
|
-
|
|
1073
|
-
This native setup does not require JSPI. Explicit WebAssembly use still needs
|
|
1074
|
-
JSPI and available Wasm assets; Next.js's standalone tracer excludes `.wasm`
|
|
1075
|
-
files, so a standalone Wasm host must supply those assets separately.
|
|
1076
|
-
|
|
1077
|
-
Create one agent for the server and open a session per request. The route
|
|
1078
|
-
returns the turn's stream and never runs the agent itself:
|
|
316
|
+
const engine = await createFxEngine({ apiKey: process.env.AI_GATEWAY_API_KEY, model: "anthropic/claude-haiku-5.5" });
|
|
1079
317
|
|
|
1080
|
-
|
|
1081
|
-
|
|
1082
|
-
|
|
318
|
+
const turn = engine.prompt("Summarize this file.");
|
|
319
|
+
for await (const event of turn) {
|
|
320
|
+
// ...
|
|
321
|
+
}
|
|
1083
322
|
|
|
1084
|
-
|
|
323
|
+
const checkpoint = await engine.checkpoint(); // opaque bytes
|
|
324
|
+
const restored = await createFxEngine({ apiKey, model, checkpoint });
|
|
1085
325
|
```
|
|
1086
326
|
|
|
1087
|
-
|
|
1088
|
-
// app/api/chat/route.js
|
|
1089
|
-
import { agent } from "@/lib/agent";
|
|
327
|
+
Pass a `persistence` store instead, and the engine records the conversation as it runs, so a new engine can `resume()` a turn the last process left open. A store needs two methods and can add a third. The bytes are opaque, so it never has to understand them:
|
|
1090
328
|
|
|
1091
|
-
|
|
1092
|
-
|
|
1093
|
-
|
|
1094
|
-
|
|
1095
|
-
|
|
329
|
+
```ts
|
|
330
|
+
interface Persistence {
|
|
331
|
+
load(): Promise<{
|
|
332
|
+
checkpoint?: { data: Uint8Array; through: string };
|
|
333
|
+
journal?: Iterable<JournalRecord> | AsyncIterable<JournalRecord>;
|
|
334
|
+
}>;
|
|
335
|
+
append(input: { expected: string | null; idempotencyKey: string; data: Uint8Array }): Promise<{ cursor: string }>;
|
|
336
|
+
saveCheckpoint?(input: { through: string; data: Uint8Array }): Promise<void>;
|
|
1096
337
|
}
|
|
1097
338
|
|
|
1098
|
-
|
|
1099
|
-
|
|
1100
|
-
|
|
1101
|
-
const sessionId = params.get("sessionId");
|
|
1102
|
-
if (!sessionId) return new Response("sessionId is required", { status: 400 });
|
|
1103
|
-
// The same check as POST.
|
|
1104
|
-
const stream = agent.session(sessionId).stream(Number(params.get("cursor") ?? 0));
|
|
1105
|
-
return new Response(stream);
|
|
339
|
+
interface JournalRecord {
|
|
340
|
+
cursor: string;
|
|
341
|
+
data: Uint8Array;
|
|
1106
342
|
}
|
|
1107
343
|
```
|
|
1108
344
|
|
|
1109
|
-
|
|
1110
|
-
session with `local()`. On Vercel, `prompt()` sends the prompt to Vercel Queues, and a separate
|
|
1111
|
-
invocation runs the turn while the `POST` response streams it. If the route's
|
|
1112
|
-
response ends first, the turn keeps running, and the client reconnects with
|
|
1113
|
-
the last `cursor` it read.
|
|
345
|
+
Reject an `append()` whose `expected` isn't the last stored cursor with `FxFencedError`, so two processes can't write the same conversation at once. `createMemoryPersistence()` is a reference store for tests. [Save and restore](https://fx.sh/docs/lib/checkpoints) has the details.
|
|
1114
346
|
|
|
1115
|
-
|
|
1116
|
-
the agent's delivery route and subscribe it to libfx's queue topic in
|
|
1117
|
-
`vercel.json`:
|
|
347
|
+
### Upgrading from 0.0.13
|
|
1118
348
|
|
|
1119
|
-
|
|
1120
|
-
// app/api/libfx/route.js
|
|
1121
|
-
import { agent } from "@/lib/agent";
|
|
1122
|
-
|
|
1123
|
-
export const POST = agent.wakeHandler();
|
|
1124
|
-
```
|
|
349
|
+
In libfx 0.0.13 and earlier, `createFxAgent()` created this engine. It now creates the durable agent, so rename those calls to `createFxEngine()`. Old code keeps running against the agent, with a few differences: breaking out of a turn's events no longer cancels it, turns have no `cancel()` or `steer()` (the session does), and `persistence` and `sessionId` throw because the durability holds the session.
|
|
1125
350
|
|
|
1126
|
-
|
|
1127
|
-
{
|
|
1128
|
-
"functions": {
|
|
1129
|
-
"app/api/libfx/route.js": {
|
|
1130
|
-
"experimentalTriggers": [{ "type": "queue/v2beta", "topic": "__libfx_wkf_workflow_session" }]
|
|
1131
|
-
}
|
|
1132
|
-
}
|
|
1133
|
-
}
|
|
1134
|
-
```
|
|
351
|
+
## Runtimes
|
|
1135
352
|
|
|
1136
|
-
|
|
1137
|
-
|
|
1138
|
-
|
|
1139
|
-
|
|
1140
|
-
|
|
1141
|
-
|
|
1142
|
-
`leaseMs` after its last renewal, 15 seconds unless you pass
|
|
1143
|
-
`vercel({ leaseMs })`. libfx authenticates to the AI Gateway with the deployment's OIDC token
|
|
1144
|
-
unless you pass `apiKey` or set `AI_GATEWAY_API_KEY`.
|
|
353
|
+
| Runtime | Backend |
|
|
354
|
+
| --- | --- |
|
|
355
|
+
| Node.js 20+ on macOS or Linux, x64 or arm64 | Native addon. Linux needs glibc 2.34 or newer. |
|
|
356
|
+
| Node.js elsewhere | WebAssembly with JSPI. Some Node versions need `--experimental-wasm-jspi`. |
|
|
357
|
+
| Browsers | WebAssembly with JSPI, from `libfx/browser` |
|
|
358
|
+
| Bun | The same as Node.js. For WebAssembly, use Bun 1.4.2 or newer. |
|
|
1145
359
|
|
|
1146
|
-
|
|
1147
|
-
route. JavaScript tools and MCP clients remain host-owned and must be supplied
|
|
1148
|
-
when creating an agent, including after checkpoint restoration. The native
|
|
1149
|
-
backend does not enable the CLI's built-in shell or filesystem tools.
|
|
360
|
+
The default `backend: "auto"` tries native first. Pass `"native"` or `"wasm"` to require one, and call `getBackendInfo()` to see what would load without creating anything.
|
|
1150
361
|
|
|
1151
|
-
|
|
362
|
+
Next.js works in Node.js route handlers, with webpack or Turbopack, and doesn't need `serverExternalPackages`. CommonJS apps can `require("libfx")`. With TypeScript's `moduleResolution: "bundler"`, add `"node"` to `customConditions` to get the Node.js types.
|
|
1152
363
|
|
|
1153
|
-
|
|
1154
|
-
connect it to xterm.js with `xtermAdapter()`:
|
|
364
|
+
## The fx terminal in a browser
|
|
1155
365
|
|
|
1156
366
|
```js
|
|
1157
367
|
import { createFxTerminal, xtermAdapter } from "libfx/browser";
|
|
1158
368
|
|
|
1159
|
-
const
|
|
1160
|
-
terminal: xtermAdapter(term),
|
|
1161
|
-
env: { AI_GATEWAY_API_KEY:
|
|
369
|
+
const terminal = await createFxTerminal({
|
|
370
|
+
terminal: xtermAdapter(term), // an xterm.js Terminal
|
|
371
|
+
env: { AI_GATEWAY_API_KEY: shortLivedKey },
|
|
1162
372
|
});
|
|
1163
373
|
|
|
1164
|
-
await
|
|
374
|
+
await terminal.interactive;
|
|
1165
375
|
```
|
|
1166
376
|
|
|
1167
|
-
|
|
1168
|
-
capabilities when its first Fast or Ultra turn starts, without adding catalog
|
|
1169
|
-
requests to startup. After the catalog loads, `/fast` toggles Fast for later
|
|
1170
|
-
turns. Models without Fast support use their normal routing.
|
|
1171
|
-
|
|
1172
|
-
The xterm adapter preserves browser-style composer editing for Shift+Enter,
|
|
1173
|
-
Command+A, Command+C, Command+X, Command+Z, and Command+Shift+Z. Shift+Enter
|
|
1174
|
-
inserts a newline without submitting. A click inside the visible composer moves
|
|
1175
|
-
its caret; pointer drags remain xterm terminal-output selections.
|
|
1176
|
-
When xterm already has an output selection, Command+C copies that selection
|
|
1177
|
-
instead of the composer selection.
|
|
1178
|
-
|
|
1179
|
-
The terminal runtime exposes `interactive`, `exited`, `write`, `resize`, and
|
|
1180
|
-
`abort`. Terminal session, config, OAuth, prompt-history, clipboard, URL, and
|
|
1181
|
-
workspace stores remain terminal-only host integrations. Clipboard copy writes
|
|
1182
|
-
through the host `clipboard.writeText(text)` adapter and defaults to
|
|
1183
|
-
`navigator.clipboard`.
|
|
1184
|
-
|
|
1185
|
-
A terminal with a `workspace` adapter loads AGENTS.md project instructions from
|
|
1186
|
-
it the way the CLI reads them from disk: `<home>/.fx/AGENTS.md`,
|
|
1187
|
-
`<root>/AGENTS.md`, and, when `root` is below `home`, the `AGENTS.md` files in
|
|
1188
|
-
the directories between them. Provide them through an optional `readFile`
|
|
1189
|
-
method:
|
|
1190
|
-
|
|
1191
|
-
```js
|
|
1192
|
-
const workspace = {
|
|
1193
|
-
info: { version: 1, root: "/workspace", cwd: "/workspace", home: "/home/visitor", gitAvailable: false, ephemeral: true },
|
|
1194
|
-
permission: "allow-sandboxed",
|
|
1195
|
-
exec({ command, cwd, signal, timeoutMs, outputLimitBytes }) { /* run one command */ },
|
|
1196
|
-
async readFile({ path, signal }) {
|
|
1197
|
-
// Return a string, UTF-8 bytes (a Uint8Array or ArrayBuffer), or null when the file does not exist.
|
|
1198
|
-
},
|
|
1199
|
-
};
|
|
1200
|
-
```
|
|
377
|
+
This is the fx CLI's interface compiled to WebAssembly. Your page supplies storage for its sessions, settings and sign-in, and can give it a workspace that runs commands. See [WebAssembly](https://fx.sh/docs/lib/webassembly).
|
|
1201
378
|
|
|
1202
|
-
|
|
1203
|
-
`home`. Each read has a 10-second deadline, the interrupt key aborts it through
|
|
1204
|
-
`signal`, and the CLI's project instruction size limits apply. Without
|
|
1205
|
-
`readFile`, fx tells the model that the workspace's project instructions were
|
|
1206
|
-
omitted and records the omission in the full transcript.
|
|
379
|
+
## Entry points
|
|
1207
380
|
|
|
1208
|
-
|
|
1209
|
-
|
|
1210
|
-
|
|
1211
|
-
|
|
1212
|
-
|
|
1213
|
-
|
|
381
|
+
| Import | What's in it |
|
|
382
|
+
| --- | --- |
|
|
383
|
+
| `libfx` | `createFxAgent`, `createFxEngine`, `memory`, `listModels`, `getBackendInfo`, `createMemoryPersistence`, `FxFencedError` |
|
|
384
|
+
| `libfx/node` | The same, for Node.js explicitly |
|
|
385
|
+
| `libfx/browser` | The browser build, plus `createFxTerminal` and `xtermAdapter` |
|
|
386
|
+
| `libfx/wasm` | The WebAssembly SDK without packaged asset paths, so you pass `wasm` yourself |
|
|
387
|
+
| `libfx/durable-local`, `libfx/durable-vercel`, `libfx/durable-world` | `local()`, `vercel()` and `world()` |
|
|
388
|
+
| `libfx/chatgpt` | `startSignIn`, `finishSignIn`, `startDeviceSignIn`, `pollDeviceSignIn`, `completeDeviceSignIn`, `refreshCredentials` |
|
|
389
|
+
| `libfx/grok` | `startSignIn`, `finishSignIn`, `startDeviceSignIn`, `pollDeviceSignIn`, `completeDeviceSignIn`, `refreshCredentials` |
|
|
390
|
+
| `libfx/mcp` | `createMcpAdapter` |
|
|
391
|
+
| `libfx/skills`, `libfx/skills/node` | `createSkillsAdapter`, and `loadSkillFile` for Node.js and Bun |
|
|
1214
392
|
|
|
1215
393
|
## Security
|
|
1216
394
|
|
|
1217
|
-
|
|
1218
|
-
|
|
1219
|
-
|
|
1220
|
-
|
|
395
|
+
Your tools, MCP clients and skill loaders have whatever access your code gives them. libfx validates and orders their calls but grants nothing itself, and the native backend doesn't turn on fx's built-in shell or file tools. Don't put long-lived keys in browser code, and treat the `nativeAddon` and `gatewayChatUrl` options as trusted configuration. Keep ChatGPT credentials, and the `signIn` object `startSignIn()` returns, on the server.
|
|
396
|
+
|
|
397
|
+
## Contributing
|
|
398
|
+
|
|
399
|
+
libfx is developed in the [fx repository](https://github.com/vercel-labs/fx), under `sdk/`. Bug reports and pull requests are welcome. See [CONTRIBUTING.md](https://github.com/vercel-labs/fx/blob/main/CONTRIBUTING.md) to get started.
|
|
400
|
+
|
|
401
|
+
## License
|
|
402
|
+
|
|
403
|
+
[Apache-2.0](https://github.com/vercel-labs/fx/blob/main/LICENSE)
|