libfx 0.0.13 → 0.0.14-dev.1534.g6afa7c33dac5
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +409 -486
- package/browser.js +18 -4
- package/chatgpt-sign-in.js +141 -0
- package/chatgpt.js +68 -0
- package/custom-provider.js +123 -0
- package/durable/local.mjs +190 -0
- package/durable/vercel.mjs +268 -0
- package/durable/world.mjs +1 -0
- package/durable.js +2591 -0
- package/fetch-cleanup.js +12 -0
- package/fx-core.wasm +0 -0
- package/fx-harness.js +220 -0
- package/fx-sdk.js +1226 -115
- package/fx-term.wasm +0 -0
- package/grok-sign-in.js +145 -0
- package/grok.js +108 -0
- package/libfx.darwin-arm64.node +0 -0
- package/libfx.darwin-x64.node +0 -0
- package/libfx.linux-arm64.node +0 -0
- package/libfx.linux-x64.node +0 -0
- package/mcp.js +3 -0
- package/node.cjs +4159 -212
- package/node.js +193 -47
- package/oauth.js +64 -0
- package/package.json +66 -13
- package/subscription.js +251 -0
- package/types/browser.d.ts +139 -0
- package/types/chatgpt.d.ts +58 -0
- package/types/durable-local.d.ts +18 -0
- package/types/durable-vercel.d.ts +22 -0
- package/types/durable-world.d.ts +89 -0
- package/types/grok.d.ts +61 -0
- package/types/libfx.d.cts +935 -0
- package/types/mcp.d.ts +41 -0
- package/types/node.d.cts +158 -0
- package/types/node.d.ts +3 -0
- package/types/skills-node.d.ts +25 -0
- package/types/skills.d.ts +28 -0
- package/types/wasm.d.ts +99 -0
package/README.md
CHANGED
|
@@ -1,611 +1,534 @@
|
|
|
1
1
|
# libfx
|
|
2
2
|
|
|
3
|
-
|
|
4
|
-
|
|
5
|
-
|
|
3
|
+
libfx is the agent from [fx](https://github.com/vercel-labs/fx), the open source coding agent from Vercel Labs, as a JavaScript library. You give it a model, instructions and your own tools, and it runs the conversation: the model loop, streaming, tool calls, retries, steering and images. It's the same Zig engine that runs fx in a terminal, built as a Node.js addon and as WebAssembly.
|
|
4
|
+
|
|
5
|
+
The hard part of running an agent on a server is that the server goes away. A function reaches its time limit in the middle of a reply, a deploy replaces the process, or it crashes during a tool call. libfx stores every prompt before it accepts it and every step of a turn as the turn runs, so the next process that picks up the session continues the turn where it stopped. On Vercel, that takes one route and one line of `vercel.json`.
|
|
6
6
|
|
|
7
7
|
```sh
|
|
8
8
|
npm install libfx
|
|
9
9
|
```
|
|
10
10
|
|
|
11
|
-
Node.js uses the native addon when available and falls back to WebAssembly.
|
|
12
|
-
Browsers use WebAssembly with JSPI. The default package has no runtime
|
|
13
|
-
dependencies and performs no MCP connection, skill scan, process spawn, or
|
|
14
|
-
filesystem read when imported.
|
|
15
|
-
|
|
16
|
-
## Agent
|
|
17
|
-
|
|
18
11
|
```js
|
|
19
12
|
import { createFxAgent } from "libfx";
|
|
20
13
|
|
|
21
|
-
const agent =
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
onEvent(event) {
|
|
25
|
-
if (event.type === "transport.response") console.log(event.elapsedMs);
|
|
26
|
-
},
|
|
14
|
+
const agent = createFxAgent({
|
|
15
|
+
model: "anthropic/claude-haiku-5.5",
|
|
16
|
+
instructions: "Answer in one paragraph.",
|
|
27
17
|
});
|
|
28
18
|
|
|
29
|
-
const turn = agent.prompt("
|
|
19
|
+
const turn = agent.session().prompt("What is an agent loop?");
|
|
30
20
|
|
|
31
21
|
for await (const event of turn) {
|
|
32
22
|
if (event.type === "text_delta") process.stdout.write(event.delta);
|
|
33
23
|
}
|
|
34
24
|
|
|
35
|
-
console.log(await turn.result); // { stopReason, usage }
|
|
36
|
-
const checkpoint = await agent.checkpoint();
|
|
37
25
|
await agent.close();
|
|
38
26
|
```
|
|
39
27
|
|
|
40
|
-
`
|
|
41
|
-
|
|
42
|
-
`
|
|
43
|
-
|
|
28
|
+
Set `AI_GATEWAY_API_KEY` and run it with Node.js 20 or newer. Model calls go through [AI Gateway](https://vercel.com/docs/ai-gateway), so any model it lists works, and on Vercel libfx uses the deployment's OIDC token instead of a key. libfx can also run on a user's own ChatGPT or Grok plan, or on another server through the OpenAI Chat Completions API. See [Providers](#providers).
|
|
29
|
+
|
|
30
|
+
Importing libfx and calling `createFxAgent()` touch no network, files or processes. The engine loads when a session runs its first turn.
|
|
31
|
+
|
|
32
|
+
The full documentation is at [fx.sh/docs/lib](https://fx.sh/docs/lib).
|
|
33
|
+
|
|
34
|
+
## How it fits together
|
|
35
|
+
|
|
36
|
+
| Object | What it is | How long it lives |
|
|
37
|
+
| --- | --- | --- |
|
|
38
|
+
| Agent | Configuration: model, instructions, tools, provider and where sessions live | As long as the server process |
|
|
39
|
+
| Session | One conversation | As long as its store keeps it |
|
|
40
|
+
| Turn | One prompt and everything the agent does to answer it | Until it ends with a stop reason |
|
|
41
|
+
|
|
42
|
+
An agent is configuration, and its conversations live in its sessions. Create each agent once per process with the same options on every server, and open a session for each conversation. Any process with that agent can run, continue or stream any of its sessions. An app can have [several agents](#several-agents), each with its own name.
|
|
43
|
+
|
|
44
|
+
Underneath the agent is `createFxEngine()`, a single conversation with no store, for when you don't want sessions. See [Without sessions](#without-sessions).
|
|
45
|
+
|
|
46
|
+
## Sessions
|
|
44
47
|
|
|
45
48
|
```js
|
|
46
|
-
const
|
|
47
|
-
|
|
48
|
-
|
|
49
|
-
|
|
49
|
+
const turn = agent.session().prompt("Plan the migration.");
|
|
50
|
+
const { sessionId } = await turn.accepted; // the prompt is stored
|
|
51
|
+
|
|
52
|
+
// Later, in any process that creates the same agent:
|
|
53
|
+
agent.session(sessionId).prompt("Start with the schema.");
|
|
50
54
|
```
|
|
51
55
|
|
|
52
|
-
A
|
|
53
|
-
`
|
|
54
|
-
|
|
55
|
-
|
|
56
|
-
|
|
57
|
-
|
|
58
|
-
`"default"` leaves the choice to the model; named levels such as `"low"`,
|
|
59
|
-
`"medium"`, `"high"`, or `"xhigh"` request a specific level. A named level is
|
|
60
|
-
validated at creation; an unsupported level rejects with an Error carrying
|
|
61
|
-
`code: "LIBFX_MODEL_UNSUPPORTED_EFFORT"`, `model`, and
|
|
62
|
-
`capability: "effort"`. Its message names the supported set when available.
|
|
63
|
-
Omitting effort or using `"default"` leaves the model default in place.
|
|
64
|
-
|
|
65
|
-
`model.fast` enables the fast lane for models that advertise one, matching the
|
|
66
|
-
fx CLI's `--fast` flag. A model without a fast path rejects at creation with
|
|
67
|
-
`code: "LIBFX_MODEL_UNSUPPORTED_FAST"`, `model`, and `capability: "fast"`.
|
|
68
|
-
Omitting fast or setting it to `false` leaves the model default in place.
|
|
69
|
-
|
|
70
|
-
`model.ultrafast` requests Ultra mode. It is off by default and maps to
|
|
71
|
-
`openai.serviceTier: "ultrafast"` through the Vercel AI Gateway for models
|
|
72
|
-
whose metadata advertises Ultra eligibility. It uses the higher-cost service tier.
|
|
73
|
-
Set it to `true` only after the host has selected an eligible model; an
|
|
74
|
-
unsupported request rejects with `code: "LIBFX_MODEL_UNSUPPORTED_ULTRAFAST"`,
|
|
75
|
-
`model`, and `capability: "ultrafast"`. Set it to `false` to explicitly disable
|
|
76
|
-
an inherited request. Ultra and Fast are mutually exclusive, so `ultrafast:
|
|
77
|
-
true` disables Fast for the agent. Gateway metadata currently marks Astra
|
|
78
|
-
eligible; libfx does not select Ultra automatically.
|
|
79
|
-
|
|
80
|
-
The new codes replace `LIBFX_UNSUPPORTED_EFFORT` and `LIBFX_UNSUPPORTED_FAST`
|
|
81
|
-
for both nested and legacy top-level settings. Callers that check the old codes
|
|
82
|
-
must update their error handling.
|
|
83
|
-
|
|
84
|
-
The host selects the model. Agent creation does not fetch the Gateway model
|
|
85
|
-
catalog unless effort requests a named level, fast is enabled, or ultrafast is enabled. Prompting
|
|
86
|
-
can resolve model capabilities and context capacity through the supplied
|
|
87
|
-
`fetch`; fx caches that metadata for the agent.
|
|
88
|
-
|
|
89
|
-
`onEvent` receives runtime diagnostics separately from model output. Transport
|
|
90
|
-
events report request start, response status and elapsed time, safe Gateway
|
|
91
|
-
request metadata, failures, and throttled `transport.activity` liveness while a
|
|
92
|
-
response body is streaming. Activity events include the attempt, current chunk
|
|
93
|
-
bytes, and cumulative response bytes. They call the host directly at most once
|
|
94
|
-
per 250 ms rather than entering the normalized turn queue, so an unread turn
|
|
95
|
-
cannot accumulate heartbeat events. Credentials and raw headers are never
|
|
96
|
-
included.
|
|
97
|
-
|
|
98
|
-
libfx makes at most one automatic retry after a retryable transport failure and
|
|
99
|
-
only before model output or tool effects escape. Cancellation prevents a retry.
|
|
100
|
-
|
|
101
|
-
`prompt(input, { signal? })` accepts a string or text, image, and resource
|
|
102
|
-
blocks. It returns an async iterable of normalized events:
|
|
103
|
-
|
|
104
|
-
- `text_delta`
|
|
105
|
-
- `reasoning_delta` when supplied by the provider
|
|
106
|
-
- `tool_start` with `id`, `name`, and the tool's `input` object. Inputs over
|
|
107
|
-
64 KiB of JSON arrive as an `inputPreview` string prefix plus
|
|
108
|
-
`inputTruncated: true` instead; the tool still receives complete arguments.
|
|
109
|
-
- `tool_end`
|
|
110
|
-
- `user_message` with `text` when accepted mid-turn steering enters the turn
|
|
111
|
-
|
|
112
|
-
Consume the turn while it runs, then await `turn.result`. Streamed text and
|
|
113
|
-
tool results are lossless and backpressured: a slow reader pauses production
|
|
114
|
-
instead of growing an unlimited event queue. Awaiting only `turn.result` can wait for an unread stream to drain.
|
|
115
|
-
If you only need the result, explicitly discard events:
|
|
56
|
+
- **One turn at a time.** A prompt sent while a turn runs waits behind it, even when it reaches a different server.
|
|
57
|
+
- **Turns finish on their own.** A turn runs to the end whether or not anyone reads its events, and a turn nobody reads holds no connection open: its view starts reading only once you use its events, `readable` or `result`. Stop it with `session.cancel()`, or with the `signal` you passed to `prompt()`.
|
|
58
|
+
- **Steering.** `session.steer(input)` steers the running turn the way the fx CLI does. While the model streams a reply, the steer cuts it short and the turn continues with it. While a tool runs, the steer waits for the tool. A steer sent while a turn waits to start joins that turn, and when no turn runs or waits, the steer becomes the next turn. A steer with images can't join a turn, so the running turn ends at its next safe point and the steer runs next.
|
|
59
|
+
- **Retries are safe.** Give a prompt or a steer a `messageId`, and a repeated request with the same id runs nothing new. A repeated prompt follows the original turn, and its result has `repeated: true`. One id names one input, so a steer that reuses a prompt's id is rejected with `FX_MESSAGE_ID_IN_USE`. On a queue that keeps messages past the process, such as `vercel()`'s, the sender doesn't read the session first, so the worker drops that steer and emits a `session.error` with the same code. A steer's `user_message` event carries its id as `id` where it joined a turn.
|
|
60
|
+
|
|
61
|
+
Each conversation can bring its own settings, which apply to the turns that session object starts:
|
|
116
62
|
|
|
117
63
|
```js
|
|
118
|
-
|
|
119
|
-
|
|
120
|
-
|
|
64
|
+
agent.session(sessionId, {
|
|
65
|
+
model: { id: "anthropic/claude-opus-5.5", effort: "high" },
|
|
66
|
+
instructions: "You review pull requests.",
|
|
67
|
+
context: { userId }, // handed to every tool call
|
|
68
|
+
});
|
|
121
69
|
```
|
|
122
70
|
|
|
123
|
-
A turn
|
|
124
|
-
`turn.cancel()` and `agent.close()` also release blocked output. Embedded agents
|
|
125
|
-
have no implicit model-step cap, so hosts should cancel turns that exceed their
|
|
126
|
-
own budgets. The CLI's `max_agent_steps` setting does not apply to
|
|
127
|
-
`createFxAgent()`. Transport or message-decoding failures reject the result
|
|
128
|
-
instead of returning success with missing text.
|
|
129
|
-
|
|
130
|
-
Native transport buffers at most 8 MiB of output bytes. Unread SDK events apply
|
|
131
|
-
backpressure at 1 MiB of encoded messages or 256 events. One message can exceed
|
|
132
|
-
that threshold when the queue is empty; an individual encoded ACP message is
|
|
133
|
-
limited to 64 MiB on both backends. These are transport bounds, not a total
|
|
134
|
-
answer-size limit or a bound on retained conversation history.
|
|
135
|
-
|
|
136
|
-
Image blocks accept a `Blob` or `File` with a non-empty `type`, raw bytes
|
|
137
|
-
(`Uint8Array`, `Buffer`, `ArrayBuffer`, or another typed array) with an
|
|
138
|
-
explicit `mimeType`, or canonical base64 (no line wrapping) with an explicit
|
|
139
|
-
`mimeType`. The payload must be PNG, JPEG, GIF, or WebP. For a Blob, the MIME
|
|
140
|
-
type is inferred from `Blob.type`; an empty type or a conflicting explicit
|
|
141
|
-
`mimeType` rejects the input:
|
|
71
|
+
A turn's result says how it ended:
|
|
142
72
|
|
|
143
73
|
```js
|
|
144
|
-
const
|
|
145
|
-
{ type: "text", text: "What does this screenshot show?" },
|
|
146
|
-
{ type: "image", data: file, sourceRef: "uploads:screenshot-1" }, // File or Blob, with file.type
|
|
147
|
-
// Or: { type: "image", data: pngBytes, mimeType: "image/png" }
|
|
148
|
-
// Or: { type: "image", data: base64Png, mimeType: "image/png" }
|
|
149
|
-
]);
|
|
74
|
+
const { stopReason, usage, error } = await turn.result;
|
|
150
75
|
```
|
|
151
76
|
|
|
152
|
-
|
|
153
|
-
|
|
154
|
-
|
|
77
|
+
`stopReason` is `end_turn`, `max_output_tokens`, `max_model_turns`, `refused` or `cancelled`. It's `error` when the turn failed, with the reason in `error`, and `unknown` when the process running it stopped before recording how it ended.
|
|
78
|
+
|
|
79
|
+
### Streaming to a browser
|
|
80
|
+
|
|
81
|
+
`turn.readable` is the turn as newline-delimited JSON, ready to return from a route. `session.stream(cursor)` returns every line after `cursor` and stays open for new ones, so a client that loses its connection reconnects to any server and carries on.
|
|
155
82
|
|
|
156
83
|
```js
|
|
157
|
-
|
|
158
|
-
|
|
159
|
-
|
|
84
|
+
// app/api/chat/route.js
|
|
85
|
+
import { agent } from "@/lib/agent";
|
|
86
|
+
|
|
87
|
+
export async function POST(request) {
|
|
88
|
+
const sessionId = new URL(request.url).searchParams.get("sessionId") ?? undefined;
|
|
89
|
+
// Check that the caller may use this session.
|
|
90
|
+
const turn = agent.session(sessionId).prompt(await request.text());
|
|
91
|
+
return new Response(turn.readable);
|
|
92
|
+
}
|
|
93
|
+
|
|
94
|
+
export async function GET(request) {
|
|
95
|
+
const params = new URL(request.url).searchParams;
|
|
96
|
+
const sessionId = params.get("sessionId");
|
|
97
|
+
if (!sessionId) return new Response("sessionId is required", { status: 400 });
|
|
98
|
+
// Same check as POST.
|
|
99
|
+
return new Response(agent.session(sessionId).stream(Number(params.get("cursor") ?? 0)));
|
|
100
|
+
}
|
|
160
101
|
```
|
|
161
102
|
|
|
162
|
-
|
|
163
|
-
|
|
164
|
-
|
|
165
|
-
|
|
166
|
-
|
|
167
|
-
|
|
168
|
-
|
|
169
|
-
|
|
170
|
-
|
|
171
|
-
|
|
172
|
-
|
|
173
|
-
|
|
174
|
-
|
|
175
|
-
|
|
176
|
-
|
|
177
|
-
|
|
178
|
-
Base64 that is not canonical throws a `TypeError` from `prompt()`. Raw bytes
|
|
179
|
-
are copied before `prompt()` returns, so the caller can reuse its buffer. Blob
|
|
180
|
-
reads are asynchronous: `prompt()` returns a turn, and read failures reject
|
|
181
|
-
`turn.result`. Cancelling or closing during a Blob read settles the turn without
|
|
182
|
-
sending its prompt. Without `resizeImage`, size errors for unreferenced base64
|
|
183
|
-
and raw-byte input throw synchronously from `prompt()`.
|
|
184
|
-
|
|
185
|
-
To downscale or convert prompt images before they are sent, pass the optional
|
|
186
|
-
`resizeImage` hook when creating the agent. It receives `{ bytes, mimeType }`,
|
|
187
|
-
where `bytes` is a `Uint8Array`, and returns `{ bytes, mimeType }` directly or
|
|
188
|
-
as a promise. Returned `bytes` may be any typed array or `ArrayBuffer`.
|
|
189
|
-
For a Node.js host that already uses `sharp`:
|
|
103
|
+
Each line is one JSON object with a `type`: the model's events, such as `text_delta` and `tool_start`, and the turn's own, `turn_start` (which carries the `sessionId`), `turn_end`, `turn_resume` and `turn_yield`. Lines from `session.stream()` carry a `cursor`. Store the last one you read and pass it back to resume after it. The `FxStreamLine` type lists every field.
|
|
104
|
+
|
|
105
|
+
## Durability
|
|
106
|
+
|
|
107
|
+
libfx stores a prompt before `turn.accepted` resolves, and stores each step of a turn while the turn runs. If the process running a turn dies, the next process that gets work for the session continues the turn from its last stored step. The model is told the turn was interrupted, so it can check what happened before going on.
|
|
108
|
+
|
|
109
|
+
Where sessions live depends on where your code runs. libfx picks for you:
|
|
110
|
+
|
|
111
|
+
| Where | Durability | Sessions live in |
|
|
112
|
+
| --- | --- | --- |
|
|
113
|
+
| Vercel | `vercel()` | Vercel's [Workflow World](https://workflow-sdk.dev/worlds), the store and queue behind Vercel Workflow |
|
|
114
|
+
| Node.js | `local()` | Files in `FX_SESSIONS_DIR`, or in `libfx/sessions` under the system's temporary directory |
|
|
115
|
+
| Browsers | `memory()` | The page's memory |
|
|
116
|
+
|
|
117
|
+
To choose yourself, pass `durability`:
|
|
190
118
|
|
|
191
119
|
```js
|
|
192
|
-
import
|
|
193
|
-
|
|
194
|
-
|
|
195
|
-
|
|
196
|
-
|
|
197
|
-
async resizeImage({ bytes }) {
|
|
198
|
-
const png = await sharp(bytes).resize({ width: 1568, withoutEnlargement: true }).png().toBuffer();
|
|
199
|
-
return { bytes: png, mimeType: "image/png" };
|
|
200
|
-
},
|
|
201
|
-
});
|
|
120
|
+
import { createFxAgent, memory } from "libfx";
|
|
121
|
+
import { local } from "libfx/durable-local";
|
|
122
|
+
|
|
123
|
+
createFxAgent({ durability: local({ dir: ".fx/sessions" }) });
|
|
124
|
+
createFxAgent({ durability: memory() }); // nothing outlives the process
|
|
202
125
|
```
|
|
203
126
|
|
|
204
|
-
|
|
205
|
-
|
|
206
|
-
|
|
207
|
-
|
|
208
|
-
|
|
209
|
-
|
|
210
|
-
|
|
211
|
-
exceeds the limits. Reference-only blocks have no bytes and do not call the
|
|
212
|
-
hook or fetch the original. The hook is host-provided, not a built-in image
|
|
213
|
-
converter or a libfx runtime dependency.
|
|
214
|
-
|
|
215
|
-
The kernel sniffs the final bytes and compares them with the claimed MIME type
|
|
216
|
-
for every input form that carries data; a mismatch fails the turn with
|
|
217
|
-
`Invalid image prompt block`. Eligible images reach the model unchanged unless
|
|
218
|
-
your hook changes them. Images are routed only to models that advertise image
|
|
219
|
-
input; for any other model the turn fails with
|
|
220
|
-
`Image prompts are unavailable for the selected model` and no image bytes
|
|
221
|
-
leave the process. Images outside the request's pixel or encoded-size limits,
|
|
222
|
-
and reference-only originals, are withheld with model-visible recovery feedback
|
|
223
|
-
that includes their `sourceRef` when supplied.
|
|
224
|
-
|
|
225
|
-
A source reference is metadata, not an access grant or an automatic fetch.
|
|
226
|
-
Your host owns the original's lifetime, reference resolution, and authorization.
|
|
227
|
-
Expose preparation or retrieval through your existing tools' `execute()`
|
|
228
|
-
callbacks if the agent needs a smaller copy. libfx has no built-in image resizer,
|
|
229
|
-
shell, global converter, or source store, and a reference grants no native
|
|
230
|
-
filesystem authority. If a tool is absent or fails, the original remains
|
|
231
|
-
withheld rather than being sent anyway.
|
|
232
|
-
|
|
233
|
-
For example, a tool can delegate to your app's authorized image store:
|
|
127
|
+
On `local()`, `vercel()` and `world()`, a session id is a Workflow run id, `wrun_` followed by letters and digits. Take it from `turn.accepted` or `await agent.newSessionId()` instead of making one up. On `memory()`, any 1 to 128 letters, digits, `.`, `_` or `-` work.
|
|
128
|
+
|
|
129
|
+
A session stores the conversation and nothing else. Every process passes its own model, credentials, tools, MCP clients and skills when it creates the agent. If a session's saved checkpoint came from a different libfx version, tool set or model, the session still loads, and `onEvent` gets a `checkpoint.mismatch` event naming what changed.
|
|
130
|
+
|
|
131
|
+
### Several agents
|
|
132
|
+
|
|
133
|
+
An app can have more than one durable agent, each with its own model, instructions and tools. Give each one a name:
|
|
234
134
|
|
|
235
135
|
```js
|
|
236
|
-
const
|
|
237
|
-
|
|
238
|
-
description: "Return a new smaller copy of a host-owned image. Use the request limit for maxSide.",
|
|
239
|
-
inputSchema: {
|
|
240
|
-
type: "object",
|
|
241
|
-
properties: { sourceRef: { type: "string" }, maxSide: { type: "integer" } },
|
|
242
|
-
required: ["sourceRef", "maxSide"],
|
|
243
|
-
},
|
|
244
|
-
async execute({ sourceRef, maxSide }, { signal }) {
|
|
245
|
-
const copy = await imageStore.prepareCopy(sourceRef, { maxSide, signal });
|
|
246
|
-
return {
|
|
247
|
-
type: "libfx.tool-result",
|
|
248
|
-
text: "Prepared a new copy; original unchanged.",
|
|
249
|
-
images: [{ type: "image", data: copy.base64, mimeType: copy.mimeType, sourceRef }],
|
|
250
|
-
};
|
|
251
|
-
},
|
|
252
|
-
};
|
|
253
|
-
// Supply prepareImage in createFxAgent({ tools: [prepareImage], ... }).
|
|
136
|
+
export const support = createFxAgent({ name: "support", instructions: "Answer billing questions." });
|
|
137
|
+
export const coding = createFxAgent({ name: "coding", model: "anthropic/claude-opus-5.5", tools: codingTools });
|
|
254
138
|
```
|
|
255
139
|
|
|
256
|
-
`
|
|
257
|
-
|
|
258
|
-
|
|
259
|
-
|
|
260
|
-
|
|
261
|
-
|
|
262
|
-
|
|
263
|
-
|
|
264
|
-
|
|
265
|
-
|
|
266
|
-
sources those refs identify.
|
|
267
|
-
|
|
268
|
-
Only one top-level prompt may run at a time. While it runs,
|
|
269
|
-
`await turn.steer(text)` appends guidance at the next safe model boundary
|
|
270
|
-
without discarding the in-flight response or completed tool work. Steering also
|
|
271
|
-
accepts an array of text blocks; image and resource steering blocks are rejected.
|
|
272
|
-
Each message is limited to 64 KiB, with at most 64 queued messages and 1 MiB of
|
|
273
|
-
queued steering text. Accepted steering appears as a `user_message` event before
|
|
274
|
-
the model's continued output. For a Blob prompt or one prepared by
|
|
275
|
-
`resizeImage`, steering during that preparation waits for the prompt to be
|
|
276
|
-
sent; cancelling before then rejects the pending steering. Calling `steer()` after the turn settles rejects with
|
|
277
|
-
`no prompt is running`.
|
|
140
|
+
A session belongs to the agent that created it, and its turns run only on an agent with that name. A prompt sent through another agent, such as `coding.session(supportSessionId).prompt()`, still runs on `support`, and the sending agent's `onEvent` gets a `session.forwarded` event. An agent created without a name is named `default`.
|
|
141
|
+
|
|
142
|
+
- Agents with the same name are copies of one agent, such as one on each server, and take turns running its sessions.
|
|
143
|
+
- Keep names stable. A session keeps the name it was created with, so renaming an agent leaves its sessions with no agent to run them.
|
|
144
|
+
- A process that gets a turn for a name it has no agent for refuses it with an `FX_NO_AGENT` error naming the agent, and the queue delivers it again.
|
|
145
|
+
- Agents share sessions through their durability. Give agents that share sessions the same durability, or none: agents created without one share the default.
|
|
146
|
+
|
|
147
|
+
### On Vercel
|
|
148
|
+
|
|
149
|
+
Add a route for libfx's queue deliveries:
|
|
278
150
|
|
|
279
151
|
```js
|
|
280
|
-
|
|
281
|
-
|
|
282
|
-
|
|
283
|
-
|
|
152
|
+
// app/api/libfx/route.js
|
|
153
|
+
import { wakeHandler } from "libfx";
|
|
154
|
+
import "@/lib/agents"; // creates the app's agents, so this function has them
|
|
155
|
+
|
|
156
|
+
export const POST = wakeHandler();
|
|
157
|
+
```
|
|
158
|
+
|
|
159
|
+
The route delivers each turn to the agent it belongs to, so one route serves every agent the module it imports creates. With one agent, `agent.wakeHandler()` does the same.
|
|
160
|
+
|
|
161
|
+
Then subscribe it to libfx's topic in `vercel.json`:
|
|
162
|
+
|
|
163
|
+
```json
|
|
164
|
+
{
|
|
165
|
+
"functions": {
|
|
166
|
+
"app/api/libfx/route.js": {
|
|
167
|
+
"experimentalTriggers": [{ "type": "queue/v2beta", "topic": "__libfx_wkf_workflow_session", "retryAfterSeconds": 5 }]
|
|
168
|
+
}
|
|
284
169
|
}
|
|
285
170
|
}
|
|
286
171
|
```
|
|
287
172
|
|
|
288
|
-
|
|
289
|
-
|
|
290
|
-
turn,
|
|
291
|
-
|
|
292
|
-
|
|
293
|
-
|
|
294
|
-
|
|
295
|
-
|
|
173
|
+
That's all. When no other invocation holds the session, `prompt()` runs the turn in the request that called it, so the first model request goes out without a queue hop, and the function stays alive for the turn after the route responds. Vercel Queues keeps the prompt until after that request's time limit, and if the request dies mid-turn, that delivery continues the turn. Other turns go through Vercel Queues to this route:
|
|
174
|
+
|
|
175
|
+
- When a turn gets within 30 seconds of the function's time limit, libfx stops it, and the next invocation continues it. `vercel({ reserveMs })` from `libfx/durable-vercel` changes that margin.
|
|
176
|
+
- A function holds its session for as long as Vercel Queues keeps the turn's message for it, so nothing is written while a turn runs to show the function is alive. If the function's process exits mid-turn, Vercel runs the same delivery again about a second later, and that invocation continues the turn.
|
|
177
|
+
- A function that stops without its process exiting holds the message until the queue's hold on it runs out, up to five minutes. A prompt sent to the session meanwhile continues the turn at that function's time limit, if that comes sooner. `vercel({ leaseMs: 15_000 })` bounds that case: the function renews a 15-second lease on its session every 5 seconds, and the session frees about 15 seconds after the last renewal. With it, every prompt goes through the queue instead of running in the request.
|
|
178
|
+
- A function that only stalled and wakes up after that is refused at its next write, so two processes never write one session.
|
|
179
|
+
|
|
180
|
+
### Tools that change things
|
|
181
|
+
|
|
182
|
+
When a process dies during a tool call, libfx can't know whether the call finished. So each tool says whether it's safe to run twice:
|
|
296
183
|
|
|
297
184
|
```js
|
|
298
|
-
const
|
|
185
|
+
const byId = { type: "object", properties: { id: { type: "string" } }, required: ["id"] };
|
|
186
|
+
|
|
187
|
+
const tools = [
|
|
188
|
+
{
|
|
189
|
+
name: "get_order",
|
|
190
|
+
description: "Look up an order.",
|
|
191
|
+
inputSchema: byId,
|
|
192
|
+
idempotent: true, // a lookup is harmless to repeat
|
|
193
|
+
execute: ({ id }) => orders.get(id),
|
|
194
|
+
},
|
|
195
|
+
{
|
|
196
|
+
name: "refund_order",
|
|
197
|
+
description: "Refund an order.",
|
|
198
|
+
inputSchema: byId,
|
|
199
|
+
idempotent: true, // keyed on executionId, so running it again refunds once
|
|
200
|
+
writes: true,
|
|
201
|
+
execute: ({ id }, { executionId }) => payments.refund(id, { idempotencyKey: executionId }),
|
|
202
|
+
},
|
|
203
|
+
];
|
|
299
204
|
```
|
|
300
205
|
|
|
301
|
-
|
|
302
|
-
or a history change. The next prompt can run normally.
|
|
206
|
+
When the turn continues, a call to an `idempotent: true` tool runs again. A call to any other tool never runs twice on its own: the model gets an error saying the call may have partly run, and decides whether to check, try again or ask the user. The turn's events end the call with that error too, as a `tool_end` with `isError: true`. libfx refuses a tool that doesn't set `idempotent`, so each one makes that choice. A call that finished before the process died keeps its result, whichever kind of tool it is: libfx stores each result as the call finishes.
|
|
303
207
|
|
|
304
|
-
|
|
305
|
-
durable storage and must resupply models, credentials, instructions, tools,
|
|
306
|
-
MCP clients, and skill records. Reasoning effort, Fast mode, and Ultra mode are
|
|
307
|
-
agent-creation options and are not stored in a checkpoint: recreate the agent
|
|
308
|
-
with new `model.effort`, `model.fast`, or `model.ultrafast` values to change
|
|
309
|
-
them, the same path as switching models.
|
|
208
|
+
Each call's `executionId` stays the same when it runs again. Pass it to the service your tool calls as an idempotency key, and a service that honors idempotency keys won't refund twice. That makes the tool safe to mark `idempotent: true` with `writes: true`.
|
|
310
209
|
|
|
311
|
-
|
|
210
|
+
A tool marked `idempotent: true` without `writes: true` starts as soon as the model asks for it, while libfx stores the call. If the process dies before the call is stored, the model asks again and the call gets a new `executionId`, which is harmless for a lookup. Every other tool starts only once its call is stored, so its `executionId` never changes.
|
|
312
211
|
|
|
313
|
-
|
|
314
|
-
|
|
212
|
+
### Your own store
|
|
213
|
+
|
|
214
|
+
`world()` from `libfx/durable-world` keeps sessions in a [Workflow World](https://workflow-sdk.dev/worlds) you create, such as Workflow's Postgres World:
|
|
315
215
|
|
|
316
216
|
```js
|
|
317
|
-
import {
|
|
217
|
+
import { createWorld } from "@workflow/world-postgres";
|
|
218
|
+
import { createFxAgent } from "libfx";
|
|
219
|
+
import { world } from "libfx/durable-world";
|
|
318
220
|
|
|
319
|
-
const
|
|
320
|
-
|
|
221
|
+
const sessions = createWorld({
|
|
222
|
+
connectionString: process.env.DATABASE_URL,
|
|
223
|
+
namespace: "libfx",
|
|
224
|
+
jobPrefix: "libfx_",
|
|
225
|
+
});
|
|
226
|
+
await sessions.start();
|
|
227
|
+
|
|
228
|
+
export const agent = createFxAgent({
|
|
229
|
+
durability: world(sessions, { name: "postgres", queueDurable: true }),
|
|
321
230
|
});
|
|
322
231
|
```
|
|
323
232
|
|
|
324
|
-
`
|
|
325
|
-
|
|
326
|
-
|
|
233
|
+
A World that delivers queue messages over HTTP needs `agent.wakeHandler()`, or `wakeHandler()` for several agents, at the address it posts to. Workflow's Postgres World 5.0.2 posts every message to `/.well-known/workflow/v1/flow` on your app. `world()` takes options for what libfx can't learn from the World itself, such as how long a lease lasts and whether the queue keeps a message across crashes. The `WorldOptions` type documents each one.
|
|
234
|
+
|
|
235
|
+
On a World with Workflow's in-band writer fence, libfx writes each claim and record so that the World itself refuses a write from a worker another has replaced, and nothing of it is stored. libfx finds the fence in the World's capabilities, so there's nothing to configure. On a World without it, libfx detects the replaced worker after its write instead, and the session's log ignores what that worker wrote.
|
|
327
236
|
|
|
328
|
-
##
|
|
237
|
+
## Tools
|
|
329
238
|
|
|
330
239
|
```js
|
|
331
|
-
const agent =
|
|
332
|
-
apiKey,
|
|
333
|
-
model,
|
|
334
|
-
instructions: "Keep answers concise.",
|
|
240
|
+
const agent = createFxAgent({
|
|
335
241
|
tools: [{
|
|
336
|
-
name: "
|
|
337
|
-
description: "Look up
|
|
338
|
-
inputSchema: {
|
|
339
|
-
|
|
340
|
-
|
|
341
|
-
|
|
342
|
-
},
|
|
343
|
-
async execute(input, { signal }) {
|
|
344
|
-
return database.get(input.key, { signal });
|
|
242
|
+
name: "get_order",
|
|
243
|
+
description: "Look up an order by id.",
|
|
244
|
+
inputSchema: { type: "object", properties: { id: { type: "string" } }, required: ["id"] },
|
|
245
|
+
idempotent: true,
|
|
246
|
+
async execute({ id }, { signal }) {
|
|
247
|
+
return orders.get(id, { signal });
|
|
345
248
|
},
|
|
346
249
|
}],
|
|
347
250
|
});
|
|
348
251
|
```
|
|
349
252
|
|
|
350
|
-
|
|
351
|
-
|
|
253
|
+
`execute` receives the model's input and a context:
|
|
254
|
+
|
|
255
|
+
| Field | What it is |
|
|
256
|
+
| --- | --- |
|
|
257
|
+
| `signal` | Aborts when the turn is cancelled |
|
|
258
|
+
| `executionId` | The call's id, the same if the call runs again after a crash |
|
|
259
|
+
| `sessionId` | The session the call belongs to |
|
|
260
|
+
| `context` | The JSON passed as `context` to `agent.session()` |
|
|
261
|
+
|
|
262
|
+
Return a string to send it as is, or any other value to send it as JSON. A thrown error reaches the model as a failed tool result. To return images, return `{ type: "libfx.tool-result", text, images }`, with base64 image data.
|
|
263
|
+
|
|
264
|
+
Validate the input and authorize the action inside `execute`. libfx orders the calls but grants nothing itself.
|
|
265
|
+
|
|
266
|
+
On the native backend, calls to idempotent tools from one model response run at the same time, up to 16 at once. A tool that isn't idempotent runs alone, and so does one marked `writes: true`. Mark a tool `writes: false` to let it overlap with others anyway. On WebAssembly, calls run one after another.
|
|
267
|
+
|
|
268
|
+
`tools` can also be an object keyed by tool name. Add `{ name: "web_search", providerExecuted: true }` for AI Gateway's web search, which runs at the provider instead of in your code.
|
|
269
|
+
|
|
270
|
+
### MCP and skills
|
|
352
271
|
|
|
353
272
|
```js
|
|
354
|
-
|
|
355
|
-
|
|
356
|
-
|
|
273
|
+
import { createMcpAdapter } from "libfx/mcp";
|
|
274
|
+
import { createSkillsAdapter } from "libfx/skills";
|
|
275
|
+
import { loadSkillFile } from "libfx/skills/node";
|
|
276
|
+
|
|
277
|
+
const mcp = await createMcpAdapter(client, { prefix: "github_" }); // your MCP SDK client
|
|
278
|
+
const skills = createSkillsAdapter([await loadSkillFile("./skills/review/SKILL.md")]);
|
|
279
|
+
|
|
280
|
+
const agent = createFxAgent({
|
|
281
|
+
tools: [...mcp.tools, ...skills.tools],
|
|
282
|
+
instructions: [mcp.instructions, skills.instructions].join("\n\n"),
|
|
357
283
|
});
|
|
284
|
+
|
|
285
|
+
// When you're done:
|
|
286
|
+
await agent.close();
|
|
287
|
+
await mcp.close();
|
|
358
288
|
```
|
|
359
289
|
|
|
360
|
-
|
|
361
|
-
|
|
362
|
-
|
|
363
|
-
|
|
364
|
-
|
|
365
|
-
Their built-in permission policy is enforced when the request is projected, as
|
|
366
|
-
there is no local call-time effect to approve.
|
|
367
|
-
|
|
368
|
-
For ordinary tools, the JavaScript host is the authority for effects. The same
|
|
369
|
-
descriptors, schemas, cancellation, results, and events are used by N-API and
|
|
370
|
-
WebAssembly.
|
|
371
|
-
Cancelling a prompt aborts its tools' signals and stops waiting for their
|
|
372
|
-
callbacks. Late results and rejections are ignored. Tools remain responsible
|
|
373
|
-
for stopping their own work when their signal is aborted.
|
|
374
|
-
|
|
375
|
-
A host tool may use any name, including the kernel's builtin names such as
|
|
376
|
-
`write_file` and `edit_file`: the kernel routes by the registered executor, so
|
|
377
|
-
a host-defined `write_file` calls the host's `execute()` rather than the
|
|
378
|
-
builtin file mutation.
|
|
379
|
-
|
|
380
|
-
Ordinary objects returned by tools are JSON text. To return rich image content,
|
|
381
|
-
use the typed result:
|
|
290
|
+
libfx doesn't own the MCP connection. You create the client, handle its auth and close it after the agent. `loadSkillFile()` reads one `SKILL.md` and nothing it references.
|
|
291
|
+
|
|
292
|
+
Every MCP tool comes with `idempotent: false`, since a server's hints can't be trusted. Set it to `true` on a tool you know is safe to run twice. A skill's tools set `idempotent` themselves.
|
|
293
|
+
|
|
294
|
+
## Models
|
|
382
295
|
|
|
383
296
|
```js
|
|
384
|
-
|
|
385
|
-
type: "libfx.tool-result",
|
|
386
|
-
text: "Original is available through the host image tools.",
|
|
387
|
-
images: [
|
|
388
|
-
{ type: "image", mimeType: "image/png", sourceRef: "uploads:screenshot-1" },
|
|
389
|
-
// A prepared copy may also include data: base64Png.
|
|
390
|
-
],
|
|
391
|
-
};
|
|
297
|
+
createFxAgent({ model: { id: "anthropic/claude-opus-5.5", effort: "high", fast: true } });
|
|
392
298
|
```
|
|
393
299
|
|
|
394
|
-
|
|
395
|
-
`data`. They do not accept raw bytes or Blobs and do not run `resizeImage`.
|
|
396
|
-
References use the same validation, ownership, recovery feedback, and checkpoint
|
|
397
|
-
rules as prompt images. A result may contain up to 8 images, each with up to
|
|
398
|
-
5 MiB of base64-encoded data. Its text, encoded images, and metadata must fit
|
|
399
|
-
within the 8 MiB result/frame bound. Referenced data is omitted if those bounds
|
|
400
|
-
would overflow; an unreferenced oversized result remains a tool error.
|
|
401
|
-
|
|
402
|
-
Instructions are limited to 64 KiB of UTF-8 text, including text assembled by
|
|
403
|
-
the MCP and skills adapters. They are the complete host-owned system context:
|
|
404
|
-
libfx adds no hidden base prompt, and omitting `instructions` sends no system
|
|
405
|
-
message.
|
|
406
|
-
|
|
407
|
-
## MCP
|
|
408
|
-
|
|
409
|
-
`libfx/mcp` accepts a host-owned MCP client. Transport, authentication,
|
|
410
|
-
elicitation, and cleanup remain outside the kernel. The client uses the MCP
|
|
411
|
-
TypeScript SDK v1 signature: `callTool(params, resultSchema?, options?)`, with
|
|
412
|
-
cancellation passed in `options`. Tool text and structured data
|
|
413
|
-
reach the model together. PNG, JPEG, GIF, and WebP tool images reach models that
|
|
414
|
-
advertise image input support; other models receive an explicit omission notice.
|
|
415
|
-
Images are retained in checkpoints within the existing checkpoint size limit.
|
|
416
|
-
Each image may contain up to 5 MiB of base64 data, with at most eight images and
|
|
417
|
-
an 8 MiB result frame. Ordinary host tool objects remain JSON text. Resource and
|
|
418
|
-
prompt options supply text instructions; non-text context has an omission notice.
|
|
419
|
-
Tool catalogs are paginated up to the existing 64-tool bound. Tool names are
|
|
420
|
-
normalized for model APIs, with collisions kept distinct and original names used
|
|
421
|
-
for calls to the MCP client. Each tool description and JSON schema may contain up
|
|
422
|
-
to 64 KiB, within the control message's 8 MiB limit.
|
|
300
|
+
`effort` and `fast` mean the same as the fx CLI's `--effort` and `--fast` flags. `ultrafast: true` asks for the faster, more expensive service tier on models that offer it. A setting the model doesn't support fails with an error whose `code` names it, such as `LIBFX_MODEL_UNSUPPORTED_EFFORT`.
|
|
423
301
|
|
|
424
|
-
|
|
425
|
-
import { createMcpAdapter } from "libfx/mcp";
|
|
302
|
+
`listModels({ apiKey })` returns the ids of AI Gateway's language models. Behind a proxy that forwards only chat requests, pass the catalog yourself as `modelCatalog`, and the agent won't fetch it.
|
|
426
303
|
|
|
427
|
-
|
|
428
|
-
|
|
429
|
-
|
|
430
|
-
prompts: ["review"],
|
|
431
|
-
});
|
|
304
|
+
## Providers
|
|
305
|
+
|
|
306
|
+
### AI Gateway
|
|
432
307
|
|
|
433
|
-
|
|
434
|
-
|
|
435
|
-
|
|
436
|
-
|
|
437
|
-
|
|
308
|
+
AI Gateway is the default. libfx uses the first credential it finds: the `apiKey` option, then `AI_GATEWAY_API_KEY`, then on Vercel the deployment's OIDC token.
|
|
309
|
+
|
|
310
|
+
### ChatGPT plans
|
|
311
|
+
|
|
312
|
+
An agent can run on a user's ChatGPT plan through Codex. Set `provider: "codex"` and give libfx the user's credentials:
|
|
313
|
+
|
|
314
|
+
```js
|
|
315
|
+
const agent = createFxAgent({
|
|
316
|
+
provider: "codex",
|
|
317
|
+
model: "gpt-6.1-sol",
|
|
318
|
+
credentials: ({ provider, sessionId }) => tokens.load(provider, sessionId),
|
|
319
|
+
onCredentials: (credentials, { provider, sessionId }) => tokens.save(provider, sessionId, credentials),
|
|
438
320
|
});
|
|
321
|
+
```
|
|
439
322
|
|
|
440
|
-
|
|
441
|
-
|
|
442
|
-
|
|
323
|
+
libfx calls `credentials()` in whichever process runs the turn. It refreshes the credentials shortly before they expire and once after a rejected request, then passes the new ones to `onCredentials` for you to save. Codex doesn't accept requests from web pages, so these turns run on your server.
|
|
324
|
+
|
|
325
|
+
To sign a user in, use `libfx/chatgpt`:
|
|
326
|
+
|
|
327
|
+
```js
|
|
328
|
+
import { startSignIn, finishSignIn } from "libfx/chatgpt";
|
|
329
|
+
|
|
330
|
+
// One request: send the user to signIn.url and keep signIn on the server.
|
|
331
|
+
const signIn = await startSignIn();
|
|
332
|
+
|
|
333
|
+
// A later request: the user pastes back the address OpenAI opened.
|
|
334
|
+
const credentials = await finishSignIn(signIn, pastedAddress);
|
|
443
335
|
```
|
|
444
336
|
|
|
445
|
-
|
|
337
|
+
After the user signs in, OpenAI opens a page at `localhost:1455` that doesn't load. That's expected: ask the user to copy the page's whole address and paste it back. If their account or ChatGPT workspace allows device code sign-in, `startDeviceSignIn()` returns a short code to enter on OpenAI's site instead, and `pollDeviceSignIn()` or `completeDeviceSignIn()` waits for it. Each step returns plain data, so a later request or function can finish it.
|
|
338
|
+
|
|
339
|
+
The sign-in uses the same OpenAI client as the Codex CLI, and the user's ChatGPT plan and OpenAI's terms apply to the turns.
|
|
340
|
+
|
|
341
|
+
### Grok plans
|
|
342
|
+
|
|
343
|
+
`provider: "grok"` runs on a user's Grok plan, with the same `credentials` and `onCredentials` callbacks, called with `provider: "grok"`. libfx also checks that refreshed credentials still belong to the same xAI account. These turns run on your server too.
|
|
446
344
|
|
|
447
|
-
|
|
448
|
-
`SKILL.md` explicitly in Node or Bun.
|
|
345
|
+
To sign a user in, use `libfx/grok`. Device code is the simplest:
|
|
449
346
|
|
|
450
347
|
```js
|
|
451
|
-
import {
|
|
452
|
-
|
|
348
|
+
import { startDeviceSignIn, completeDeviceSignIn } from "libfx/grok";
|
|
349
|
+
|
|
350
|
+
// Show device.userCode and a link to device.verificationUrl.
|
|
351
|
+
const device = await startDeviceSignIn();
|
|
453
352
|
|
|
454
|
-
|
|
455
|
-
const
|
|
456
|
-
const agent = await createFxAgent({ apiKey, model, ...skills });
|
|
353
|
+
// Waits until the user enters the code. pollDeviceSignIn() checks once, for a later request.
|
|
354
|
+
const credentials = await completeDeviceSignIn(device);
|
|
457
355
|
```
|
|
458
356
|
|
|
459
|
-
|
|
357
|
+
`startSignIn()` and `finishSignIn()` work as for ChatGPT, except that xAI opens a page at `127.0.0.1`. The sign-in uses the same xAI client as the fx CLI, and the user's Grok plan and xAI's terms apply to the turns.
|
|
358
|
+
|
|
359
|
+
### Several servers, one account
|
|
360
|
+
|
|
361
|
+
The sessions in one process share each refresh. Across servers, two can notice the same expiring credentials at once, and the provider accepts only the first refresh. Pass `refreshLock` so they take turns:
|
|
460
362
|
|
|
461
363
|
```js
|
|
462
|
-
|
|
463
|
-
await createFxAgent({ apiKey, backend: "native" }); // require N-API
|
|
464
|
-
await createFxAgent({ apiKey, backend: "wasm" }); // require Wasm + JSPI
|
|
364
|
+
refreshLock: ({ provider, sessionId }, refresh) => locks.withLock(`${provider}:${accountOf(sessionId)}`, refresh),
|
|
465
365
|
```
|
|
466
366
|
|
|
467
|
-
|
|
468
|
-
package chooses its generated CommonJS entry automatically and keeps asset
|
|
469
|
-
paths relative to the installed package.
|
|
367
|
+
libfx calls it with a function to run while you hold a lock on that user's account, for example in Redis. Inside it, libfx calls `credentials()` again and uses what another server already saved.
|
|
470
368
|
|
|
471
|
-
|
|
472
|
-
|
|
369
|
+
### Custom providers
|
|
370
|
+
|
|
371
|
+
An agent can run on a server that speaks the OpenAI Chat Completions API, such as a local model server or a router:
|
|
473
372
|
|
|
474
373
|
```js
|
|
475
|
-
|
|
476
|
-
|
|
477
|
-
|
|
478
|
-
|
|
479
|
-
|
|
480
|
-
|
|
481
|
-
|
|
482
|
-
|
|
374
|
+
const router = {
|
|
375
|
+
name: "router",
|
|
376
|
+
baseUrl: "https://openrouter.ai/api/v1",
|
|
377
|
+
models: { "qwen/qwen3-coder": { contextWindow: 128000, supportsToolUse: true } },
|
|
378
|
+
};
|
|
379
|
+
|
|
380
|
+
const agent = createFxAgent({
|
|
381
|
+
provider: router,
|
|
382
|
+
model: "qwen/qwen3-coder",
|
|
383
|
+
credentials: async ({ provider, sessionId }) => ({ apiKey: await keys.load(provider, sessionId) }),
|
|
384
|
+
});
|
|
483
385
|
```
|
|
484
386
|
|
|
485
|
-
`
|
|
486
|
-
same `auto`, `native`, and `wasm` selection as the factories. `nativeAddon` and
|
|
487
|
-
`wasm` select explicit assets with the same meanings as their factory options.
|
|
488
|
-
The probe loads and validates the native module or compiles the selected Wasm
|
|
489
|
-
asset, then stops. It does not create a core, open a runtime socket, read
|
|
490
|
-
credentials, start a model request, or write session state. A remote Wasm
|
|
491
|
-
source can perform its normal asset fetch.
|
|
387
|
+
Requests go to `{baseUrl}/chat/completions`, which must use HTTPS, or HTTP on `localhost`. Leave out `credentials` for a server that needs no key. `models` tells libfx what it can't ask the server: each model's context window and output limit, and whether it takes tools and images. A key isn't refreshed, so a rejected one ends the turn with the server's error. In a browser, the server must accept cross-origin requests, and the page can read the key.
|
|
492
388
|
|
|
493
|
-
|
|
494
|
-
options reject with `TypeError`. The stable reason codes are:
|
|
389
|
+
Saved sessions follow the agent's provider: point an agent at another base URL and its sessions continue there, history included.
|
|
495
390
|
|
|
496
|
-
|
|
497
|
-
|
|
498
|
-
|
|
499
|
-
|
|
500
|
-
|
|
501
|
-
| `LIBFX_NATIVE_API_MISMATCH` | The addon API version is incompatible. |
|
|
502
|
-
| `LIBFX_NATIVE_SURFACE_MISSING` | The addon does not implement the selected Agent or terminal surface. |
|
|
503
|
-
| `LIBFX_NATIVE_DISABLED` | `nativeAddon: false` disabled native loading. |
|
|
504
|
-
| `LIBFX_JSPI_UNAVAILABLE` | The current JavaScript runtime does not provide JSPI. |
|
|
505
|
-
| `LIBFX_WASM_LOAD_FAILED` | The selected Wasm asset could not be loaded or compiled. |
|
|
506
|
-
|
|
507
|
-
The optional `causeCode` field retains a Node error code such as `ENOENT` or
|
|
508
|
-
`ERR_DLOPEN_FAILED` when one exists. Probe success establishes backend loading
|
|
509
|
-
only; it does not validate credentials, a future Agent initialization, or a
|
|
510
|
-
model request.
|
|
511
|
-
|
|
512
|
-
Within one loaded SDK module, libfx compiles each stable Wasm source once and
|
|
513
|
-
creates a separate WebAssembly instance for every Agent. Agent memory, history,
|
|
514
|
-
tools, cancellation, and shutdown remain isolated. Workers and separate
|
|
515
|
-
processes maintain their own module caches, as do the ESM and CommonJS entries.
|
|
516
|
-
|
|
517
|
-
Node factories and `getBackendInfo()` also accept a `wasm` Promise resolving
|
|
518
|
-
to an HTTP(S) URL string, a `Response`, Wasm bytes, or a compiled
|
|
519
|
-
`WebAssembly.Module`. Asset resolver failures propagate from factories and
|
|
520
|
-
appear as `LIBFX_WASM_LOAD_FAILED` in diagnostics.
|
|
521
|
-
|
|
522
|
-
Node.js 20+ is supported. Browser WebAssembly requires a JSPI-capable browser.
|
|
523
|
-
The Linux x64 and arm64 native addons require glibc 2.34 or newer. Native
|
|
524
|
-
agents do not require JSPI or experimental Node flags.
|
|
525
|
-
Some Node versions require `--experimental-wasm-jspi`.
|
|
526
|
-
Bun 1.4.2 is the tested recommendation for Bun's WebAssembly backend.
|
|
527
|
-
Bun 1.3.14 can crash when a hot WebAssembly loop resumes through JSPI during
|
|
528
|
-
JIT tier-up.
|
|
529
|
-
|
|
530
|
-
### Next.js and Vercel
|
|
531
|
-
|
|
532
|
-
Create agents in a server route using the Node.js runtime. Import `libfx`
|
|
533
|
-
normally; the package includes its native assets and exposes both ESM and
|
|
534
|
-
CommonJS Node entrypoints. Native agents support Next.js 15 with webpack and
|
|
535
|
-
Next.js 16 with webpack or Turbopack, without `serverExternalPackages` or manual
|
|
536
|
-
native-file inclusion. Webpack's emitted assets are resolved relative to the
|
|
537
|
-
server bundle, including standalone builds with a custom `distDir` or `assetPrefix`.
|
|
538
|
-
|
|
539
|
-
This native setup does not require JSPI. Explicit WebAssembly use still needs
|
|
540
|
-
JSPI and available Wasm assets; Next.js's standalone tracer excludes `.wasm`
|
|
541
|
-
files, so a standalone Wasm host must supply those assets separately.
|
|
391
|
+
### Where credentials go
|
|
392
|
+
|
|
393
|
+
libfx adds ChatGPT, Grok and custom-provider credentials to each request in JavaScript, just before it's sent, and sends them only to that provider. They never reach the engine. No credential is written to a saved session or passed to `onEvent`, and if a server quotes a key or token back in an error, libfx masks it before the error reaches the turn.
|
|
394
|
+
|
|
395
|
+
## Images
|
|
542
396
|
|
|
543
397
|
```js
|
|
544
|
-
|
|
398
|
+
agent.session().prompt([
|
|
399
|
+
{ type: "text", text: "What's wrong with this chart?" },
|
|
400
|
+
{ type: "image", mimeType: "image/png", data: base64Png },
|
|
401
|
+
]);
|
|
402
|
+
```
|
|
545
403
|
|
|
546
|
-
|
|
404
|
+
A session stores each prompt as JSON before it runs, so a session prompt takes image data as a base64 string. `createFxEngine()` also takes bytes, a `Blob` or a `File`. Each prompt takes up to 8 PNG, JPEG, GIF or WebP images, at most 3.75 MiB each and 6 MiB together. Pass `resizeImage` to downscale or convert prompt images before they're sent.
|
|
547
405
|
|
|
548
|
-
|
|
549
|
-
|
|
550
|
-
|
|
551
|
-
|
|
552
|
-
|
|
553
|
-
|
|
554
|
-
|
|
555
|
-
|
|
556
|
-
|
|
557
|
-
|
|
558
|
-
|
|
559
|
-
|
|
560
|
-
|
|
561
|
-
|
|
406
|
+
## Diagnostics
|
|
407
|
+
|
|
408
|
+
`onEvent` receives diagnostics apart from the model's output: request timing and retries, and session events such as `session.deadline` when a turn stops for a function's time limit. The `FxAgentEvent` type lists them all.
|
|
409
|
+
|
|
410
|
+
## Without sessions
|
|
411
|
+
|
|
412
|
+
`createFxEngine()` is the engine under the agent: one conversation in the memory of one process, with no queue and no session store. Use it for a single request, a subagent inside a tool call, or a conversation you store yourself.
|
|
413
|
+
|
|
414
|
+
```js
|
|
415
|
+
import { createFxEngine } from "libfx";
|
|
416
|
+
|
|
417
|
+
const engine = await createFxEngine({ apiKey: process.env.AI_GATEWAY_API_KEY, model: "anthropic/claude-haiku-5.5" });
|
|
418
|
+
|
|
419
|
+
const turn = engine.prompt("Summarize this file.");
|
|
420
|
+
for await (const event of turn) {
|
|
421
|
+
if (event.type === "text_delta") process.stdout.write(event.delta);
|
|
562
422
|
}
|
|
423
|
+
|
|
424
|
+
const checkpoint = await engine.checkpoint(); // opaque bytes, up to 4 MiB
|
|
425
|
+
await engine.close();
|
|
426
|
+
|
|
427
|
+
const restored = await createFxEngine({ apiKey: process.env.AI_GATEWAY_API_KEY, model: "anthropic/claude-haiku-5.5", checkpoint });
|
|
563
428
|
```
|
|
564
429
|
|
|
565
|
-
|
|
566
|
-
|
|
567
|
-
|
|
568
|
-
|
|
430
|
+
An engine differs from a session:
|
|
431
|
+
|
|
432
|
+
- On AI Gateway, it needs an `apiKey`.
|
|
433
|
+
- One prompt runs at a time, and breaking out of a turn's events cancels the turn.
|
|
434
|
+
- The turn itself has `cancel()` and `steer()`. Like a session's steer, it cuts a streaming reply short and waits for a running tool, and it takes text only.
|
|
435
|
+
|
|
436
|
+
To have the engine record the conversation as it runs, pass a `persistence` store instead of a checkpoint. A new engine on the same store can then `resume()` a turn the last process left open. A store needs two methods and can add a third, and the bytes are opaque, so it never has to understand them:
|
|
437
|
+
|
|
438
|
+
```ts
|
|
439
|
+
interface Persistence {
|
|
440
|
+
load(): Promise<{
|
|
441
|
+
checkpoint?: { data: Uint8Array; through: string };
|
|
442
|
+
journal?: Iterable<JournalRecord> | AsyncIterable<JournalRecord>;
|
|
443
|
+
head?: string | null;
|
|
444
|
+
}>;
|
|
445
|
+
append(input: { expected: string | null; idempotencyKey: string; data: Uint8Array; barrier: boolean; marks?: object[] }): Promise<{ cursor: string }>;
|
|
446
|
+
saveCheckpoint?(input: { through: string; data: Uint8Array }): Promise<void>;
|
|
447
|
+
marks?: boolean;
|
|
448
|
+
}
|
|
449
|
+
|
|
450
|
+
interface JournalRecord {
|
|
451
|
+
cursor: string;
|
|
452
|
+
data: Uint8Array;
|
|
453
|
+
}
|
|
454
|
+
```
|
|
455
|
+
|
|
456
|
+
Reject an `append()` whose `expected` isn't the last stored cursor with `FxFencedError`, so two processes can't write the same conversation. If your store keeps entries of its own after the last record, return the cursor the next `append()` should name as `head`. When a resumed turn left a tool call running, `resume({ onAmbiguous })` lets you return `"rerun"` to run it again under the same `executionId`.
|
|
457
|
+
|
|
458
|
+
The engine sends one `append()` at a time, and everything that happens while one is in flight goes in the next record. `barrier` is `true` when the engine waits for the record before it goes on, such as before a tool call with effects. A store may acknowledge a record with `barrier: false` before it is durable, as long as it keeps records in order and acknowledges a barrier only once it and every record before it are durable. Set `marks: true` to receive each record's `marks`, which say where turns start (`{ start: turnId }`), yield and end, and which inputs a turn took (`{ accepted: id }`), so a store can track turns without reading records.
|
|
569
459
|
|
|
570
|
-
|
|
460
|
+
`prompt()` and `resume()` take `yieldAt`, a time in milliseconds since the epoch: the turn stops before its next model request after that time, with the stop reason `yielded`, and the next `resume()` continues it without telling the model it was interrupted.
|
|
571
461
|
|
|
572
|
-
`
|
|
573
|
-
|
|
462
|
+
A journal or `checkpoint` from a newer libfx fails with `FxJournalVersionError`. `createMemoryPersistence()` is a reference store for tests.
|
|
463
|
+
|
|
464
|
+
## Runtimes
|
|
465
|
+
|
|
466
|
+
| Runtime | Backend |
|
|
467
|
+
| --- | --- |
|
|
468
|
+
| Node.js 20+ on macOS or Linux, x64 or arm64 | Native addon. Linux needs glibc 2.34 or newer. |
|
|
469
|
+
| Node.js elsewhere | WebAssembly with JSPI. Some Node versions need `--experimental-wasm-jspi`. |
|
|
470
|
+
| Browsers | WebAssembly with JSPI, from `libfx/browser` |
|
|
471
|
+
| Bun | The same as Node.js. For WebAssembly, use Bun 1.4.2 or newer. |
|
|
472
|
+
|
|
473
|
+
The default `backend: "auto"` tries the native addon first. Pass `"native"` or `"wasm"` to require one, and call `getBackendInfo()` to see what would load without creating anything.
|
|
474
|
+
|
|
475
|
+
Next.js works in Node.js route handlers, with webpack or Turbopack, and doesn't need `serverExternalPackages`. CommonJS apps can `require("libfx")`. With TypeScript's `moduleResolution: "bundler"`, add `"node"` to `customConditions` to get the Node.js types.
|
|
476
|
+
|
|
477
|
+
## The fx terminal in a browser
|
|
574
478
|
|
|
575
479
|
```js
|
|
576
480
|
import { createFxTerminal, xtermAdapter } from "libfx/browser";
|
|
577
481
|
|
|
578
|
-
const
|
|
579
|
-
terminal: xtermAdapter(term),
|
|
580
|
-
env: { AI_GATEWAY_API_KEY:
|
|
482
|
+
const terminal = await createFxTerminal({
|
|
483
|
+
terminal: xtermAdapter(term), // an xterm.js Terminal
|
|
484
|
+
env: { AI_GATEWAY_API_KEY: shortLivedKey },
|
|
581
485
|
});
|
|
582
486
|
|
|
583
|
-
await
|
|
487
|
+
await terminal.interactive;
|
|
584
488
|
```
|
|
585
489
|
|
|
586
|
-
|
|
587
|
-
|
|
588
|
-
|
|
589
|
-
|
|
590
|
-
|
|
591
|
-
|
|
592
|
-
|
|
593
|
-
The
|
|
594
|
-
`
|
|
595
|
-
|
|
596
|
-
|
|
597
|
-
`
|
|
598
|
-
|
|
599
|
-
|
|
600
|
-
|
|
601
|
-
|
|
602
|
-
|
|
603
|
-
|
|
604
|
-
|
|
490
|
+
This is the fx CLI's interface compiled to WebAssembly. Your page supplies storage for its sessions, settings and sign-in, and can give it a workspace that runs commands. See [Terminal](https://fx.sh/docs/lib/terminal).
|
|
491
|
+
|
|
492
|
+
## Entry points
|
|
493
|
+
|
|
494
|
+
| Import | What's in it |
|
|
495
|
+
| --- | --- |
|
|
496
|
+
| `libfx` | `createFxAgent`, `createFxEngine`, `createFxTerminal`, `memory`, `listModels`, `createMemoryPersistence`, `FxFencedError`, `FxJournalVersionError`, `supportsJspi`, `xtermAdapter`, `encodeXtermKeyEvent`, and on Node.js `getBackendInfo` and `wakeHandler` |
|
|
497
|
+
| `libfx/node`, `libfx/browser` | The same, for Node.js or browsers explicitly |
|
|
498
|
+
| `libfx/wasm` | The WebAssembly SDK without packaged asset paths, so you pass `wasm` yourself |
|
|
499
|
+
| `libfx/durable-local`, `libfx/durable-vercel`, `libfx/durable-world` | `local()`, `vercel()` and `world()` |
|
|
500
|
+
| `libfx/chatgpt`, `libfx/grok` | Sign-in for ChatGPT and Grok plans, and `refreshCredentials` |
|
|
501
|
+
| `libfx/mcp` | `createMcpAdapter` |
|
|
502
|
+
| `libfx/skills`, `libfx/skills/node` | `createSkillsAdapter`, and `loadSkillFile` for Node.js and Bun |
|
|
503
|
+
|
|
504
|
+
Every entry point ships TypeScript types, and the package has no runtime dependencies.
|
|
505
|
+
|
|
506
|
+
## For coding agents
|
|
507
|
+
|
|
508
|
+
- **Docs as Markdown:** [fx.sh/llms.txt](https://fx.sh/llms.txt) indexes every page, and each page has a `.md` version, such as `https://fx.sh/docs/lib/durability.md`. The declarations in `types/` are the exact API.
|
|
509
|
+
- **Agents at module scope:** create each agent once at module scope, not per request, and open `agent.session(id)` for each conversation.
|
|
510
|
+
- **Session ids:** take them from `turn.accepted` or `agent.newSessionId()`. On every durability except `memory()`, an id that isn't a Workflow run id throws.
|
|
511
|
+
- **Side effects:** every tool sets `idempotent`. Set it to `true` only on tools that are safe to run twice, and pass `executionId` to services that take an idempotency key.
|
|
512
|
+
- **Deployments:** on Vercel, the `wakeHandler()` route and its `vercel.json` trigger are both required, or prompts are stored and never run. The route must import the module that creates the agents.
|
|
513
|
+
- **Several agents:** give each durable agent its own `name`, and never rename one that has sessions.
|
|
514
|
+
- **Credentials:** keep keys and ChatGPT or Grok credentials on the server, and the object `startSignIn()` returns too.
|
|
515
|
+
- **Checking a setup:** `getBackendInfo()` reports which backend would load without creating anything or making requests.
|
|
516
|
+
- **Changing libfx:** the source is in the [fx repository](https://github.com/vercel-labs/fx) under `sdk/`. Its `AGENTS.md` files name the tests each change needs.
|
|
517
|
+
|
|
518
|
+
## Upgrading from 0.0.13
|
|
519
|
+
|
|
520
|
+
In libfx 0.0.13 and earlier, `createFxAgent()` created the engine. It now creates the durable agent, so rename those calls to `createFxEngine()`. Old code runs against the agent once its tools set `idempotent`, with a few differences: breaking out of a turn's events no longer cancels it, turns have no `cancel()` or `steer()` (the session does), and `persistence` and `sessionId` throw because the durability holds the session.
|
|
521
|
+
|
|
522
|
+
`libfxApiVersion` is now 3. Every tool given to `createFxAgent()` must set `idempotent`, and a tool's `writes` now defaults to the opposite of `idempotent`, in the engine too, so a tool that isn't marked idempotent runs alone. A journal written by this libfx can't be loaded by an older one, which fails with `FxJournalVersionError`.
|
|
605
523
|
|
|
606
524
|
## Security
|
|
607
525
|
|
|
608
|
-
|
|
609
|
-
|
|
610
|
-
|
|
611
|
-
|
|
526
|
+
Your tools, MCP clients and skill loaders have whatever access your code gives them. libfx orders their calls but grants nothing itself, and the native backend doesn't turn on fx's built-in shell or file tools. Don't put long-lived keys in browser code, and treat the `nativeAddon` and `gatewayChatUrl` options as trusted configuration.
|
|
527
|
+
|
|
528
|
+
## Contributing
|
|
529
|
+
|
|
530
|
+
libfx is developed in the [fx repository](https://github.com/vercel-labs/fx), under `sdk/`. Bug reports and pull requests are welcome. See [CONTRIBUTING.md](https://github.com/vercel-labs/fx/blob/main/CONTRIBUTING.md) to get started.
|
|
531
|
+
|
|
532
|
+
## License
|
|
533
|
+
|
|
534
|
+
[Apache-2.0](https://github.com/vercel-labs/fx/blob/main/LICENSE)
|