ai-sdk-harness-sessions 0.2.0 → 0.3.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -1,5 +1,27 @@
1
1
  # ai-sdk-harness-sessions
2
2
 
3
+ ## 0.3.0
4
+
5
+ ### Minor Changes
6
+
7
+ - 02b3ad7: `onTurn`, a new option of `createSessionManager()`, is called once each turn is over and the session written, with a `TurnReport`: the turn's tokens, cache reads and writes included, the model, tokens and tool calls of each step, and the raw events of its harness — Claude Code's `rate_limit_event`, for instance. An application can account for its sessions, and read the rate limits of a subscription. `SessionStreamResult` may give `steps` and `fullStream`, as a `StreamTextResult` does; an agent whose results give neither reports its tokens only.
8
+
9
+ ## 0.2.1
10
+
11
+ ### Patch Changes
12
+
13
+ - d1448b3: Closing or deleting an `interrupted` session now discards its sandbox, as it does for a `suspended` one. A session the process left live when it stopped without suspending it kept its sandbox forever: with `sandboxPerSession()`, a session created again under the same id then failed, its sandbox's name already taken.
14
+ - 116b98c: A resume that fails no longer closes the session's sandbox lease: it hands it back with `suspend()`, as a suspension does, so the sandbox and the files the next try resumes from are kept. With `sandboxPerSession()`, a failed resume destroyed the session's sandbox while the session stayed `suspended`, waiting to resume from it.
15
+ - a1197a2: Several managers can share one store: the instances of an application behind a load balancer, scaled out, or a new version deployed beside the old one. A store that implements three more methods — `claimSession`, `renewLease` and `releaseSession`, the lease of each manager kept on the database's clock — lets each manager run the sessions it holds:
16
+
17
+ - a starting manager no longer marks `interrupted` the live sessions of the others, only those of a manager whose lease expired;
18
+ - a request for a session another manager runs fails with `SessionOwnedElsewhereError` (a `SessionConflictError`), which names that manager (`owner`, the new `managerId` option) so that the application can route the request to it;
19
+ - a suspended session is resumed by whichever manager receives its next message;
20
+ - once the lease of a manager that crashed has expired, the others mark its live sessions `interrupted`.
21
+
22
+ `leaseTtlMs` sets the lease (30 seconds by default). A store without these methods works as before, with one manager.
23
+ - 32e5121: The message sent after a turn cut short is answered. Claude Code goes on with a turn for a moment after it is told to stop, and the harness session handed what it still said to the next turn: that turn failed with "Received text-delta for missing text part", or ended at once without its answer. A turn cut short by `interrupt()`, `turnTimeoutMs` or the caller's `abortSignal` now starts its harness session again, in the same sandbox and the same conversation, before the next turn.
24
+
3
25
  ## 0.2.0
4
26
 
5
27
  ### Minor Changes
package/README.md CHANGED
@@ -22,7 +22,8 @@ import { createSessionManager, sharedSandbox } from 'ai-sdk-harness-sessions';
22
22
  const sessions = createSessionManager({
23
23
  // The agent of a session, from what you keep with it.
24
24
  agent: ({ metadata }) => agents[metadata.agent],
25
- // One sandbox for every session, each with a port of its own for its bridge.
25
+ // One sandbox for every session, each with a port of its own for its bridge. `openSandbox` is
26
+ // yours: it opens a sandbox of ai-sdk-sandbox-sbx, ai-sdk-sandbox-cloud-run… (see "Sandboxes").
26
27
  sandboxes: sharedSandbox({
27
28
  open: openSandbox,
28
29
  ports: Array.from({ length: 4 }, (_, i) => 4001 + i),
@@ -46,6 +47,7 @@ return turn.toUIMessageStreamResponse();
46
47
  | Every sandbox slot is taken | Suspends the session idle the longest to make room |
47
48
  | Your server stops (`shutdown()`) | Cuts the turns under way short, and suspends every session: the next start resumes them on their next message |
48
49
  | Your server crashes | Marks the sessions it held `interrupted` at the next start: their conversation can still be read |
50
+ | Your server runs on several instances | With a store that gives leases, each instance runs the sessions it holds, and takes over those of one that crashed |
49
51
  | You want the conversation again | `get(id)` gives it as `useChat` shows it, `list()` every session, `subscribe()` every change, for a list to follow |
50
52
 
51
53
  ## Installation
@@ -134,6 +136,8 @@ export async function POST(request: Request) {
134
136
 
135
137
  The turn is read to its end on your server as well: a user who closes the tab does not leave the session hanging, and the session is written either way. `abortSignal: request.signal` lets the stop button of `useChat` cut the turn short; leave it out for turns that go on without the client. `interrupt(id)` cuts a turn short from anywhere.
136
138
 
139
+ A turn cut short — by `interrupt(id)`, `turnTimeoutMs` or the caller's `abortSignal` — starts its harness session again, in the same sandbox, before the session takes another message. A runtime goes on with a turn for a moment after it is told to stop, and its harness session would hand what it still says to the next turn: that turn would end at once, its answer lost ([vercel/ai#23507](https://github.com/vercel/ai/issues/23507)). The conversation goes on where it stopped; starting again takes a few seconds. A harness session that cannot start again leaves the session `suspended`, resumed by the next message.
140
+
137
141
  ### Showing the conversations again
138
142
 
139
143
  ```ts
@@ -183,16 +187,16 @@ stateDiagram-v2
183
187
  busy --> interrupted: the server crashed
184
188
  ```
185
189
 
186
- | Status | What it means |
187
- | ---------------- | ------------------------------------------------------------------------------------------------------------------ |
188
- | `preparing` | Its sandbox and harness session are being opened, or brought back from `suspended`. A message sent meanwhile waits |
189
- | `idle` | Ready for a message |
190
- | `busy` | A turn is under way. Another message is refused until it ends |
191
- | `awaiting-input` | The turn paused on a tool call that waits for you: an approval, or the result of a tool your application runs |
192
- | `suspended` | Stopped, its state kept in the store and its sandbox freed. The next message, or `continue()`, brings it back |
193
- | `closed` | Ended for good. Its conversation can still be read |
194
- | `failed` | A new session that could not start: a sandbox that did not open, say. The next message tries again |
195
- | `interrupted` | Cannot be brought back: the server stopped without suspending it. Its conversation can still be read |
190
+ | Status | What it means |
191
+ | ---------------- | ----------------------------------------------------------------------------------------------------------------------------------------------- |
192
+ | `preparing` | Its sandbox and harness session are being opened, or brought back from `suspended`. A message sent meanwhile waits |
193
+ | `idle` | Ready for a message |
194
+ | `busy` | A turn is under way. Another message is refused until it ends |
195
+ | `awaiting-input` | The turn paused on a tool call that waits for you: an approval, or the result of a tool your application runs |
196
+ | `suspended` | Stopped, its state kept in the store and its sandbox freed. The next message, or `continue()`, brings it back |
197
+ | `closed` | Ended for good. Its conversation can still be read |
198
+ | `failed` | A new session that could not start: a sandbox that did not open, say. The next message tries again |
199
+ | `interrupted` | Cannot be brought back: the server stopped without suspending it. Its conversation can still be read; `close()` or `delete()` frees its sandbox |
196
200
 
197
201
  A session that fails to resume stays `suspended`, with the reason in its `error`: the next message tries again from the same state.
198
202
 
@@ -261,7 +265,7 @@ Where each session runs is up to `sandboxes`:
261
265
  | `sandboxPerSession({ create, resume, maxSessions })` | A sandbox of its own, stopped when the session is suspended, destroyed when it is closed | Sessions that must not see each other: one per user or per repository |
262
266
  | Your own `SessionSandboxes` | What `acquire(sessionId, { resume })` returns | A pool, a sandbox per user shared by their sessions… |
263
267
 
264
- `sharedSandbox()` takes sandboxes that give views of themselves, `fork({ ports })`: those of `ai-sdk-sandbox-sbx` and `ai-sdk-sandbox-cloud-run`. A suspended session hands its port back and keeps its files in the sandbox. With `stopWhenUnused`, the sandbox itself is stopped once no session holds it, and opened again with the next one: on Cloud Run, a stopped sandbox costs its snapshot's storage alone.
268
+ `sharedSandbox()` takes sandboxes that give views of themselves, `fork({ ports })`: those of `ai-sdk-sandbox-sbx`, `ai-sdk-sandbox-cloud-run` and `ai-sdk-sandbox-runtime`. A suspended session hands its port back and keeps its files in the sandbox. With `stopWhenUnused`, the sandbox itself is stopped once no session holds it, and opened again with the next one: on Cloud Run, a stopped sandbox costs its snapshot's storage alone.
265
269
 
266
270
  ```ts
267
271
  import { createSbxNetworkSandboxSession, resumeSbxNetworkSandboxSession } from 'ai-sdk-sandbox-sbx';
@@ -276,6 +280,48 @@ const sandboxes = sandboxPerSession({
276
280
  });
277
281
  ```
278
282
 
283
+ ### Opening the sandbox
284
+
285
+ The package imports no sandbox package: the `open()` of `sharedSandbox()`, and the `create()` and `resume()` of `sandboxPerSession()`, are functions you write with the sandbox package of your choice. Throughout this documentation, `openSandbox` is the function of [The manager](#the-manager): it finds the sandbox again when it exists, and creates it otherwise. To run the sessions somewhere else, change that function and nothing else:
286
+
287
+ | Sandbox package | Its functions | `sharedSandbox()` | `sandboxPerSession()` |
288
+ | ---------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------- | ---------------------- | --------------------- |
289
+ | [`ai-sdk-sandbox-sbx`](https://ai-sdk-harness.pages.dev/docs/packages/sandbox-sbx) | `createSbxNetworkSandboxSession()`, `resumeSbxNetworkSandboxSession()` | Yes | Yes |
290
+ | [`ai-sdk-sandbox-cloud-run`](https://ai-sdk-harness.pages.dev/docs/packages/sandbox-cloud-run) | `createCloudRunNetworkSandboxSession()`, `resumeCloudRunNetworkSandboxSession()` | Yes | Yes |
291
+ | [`ai-sdk-sandbox-runtime`](https://ai-sdk-harness.pages.dev/docs/packages/sandbox-runtime) | `createSrtNetworkSandboxSession()`, `resumeSrtNetworkSandboxSession()` | Yes | Yes |
292
+ | [`ai-sdk-sandbox-microsandbox`](https://ai-sdk-harness.pages.dev/docs/packages/sandbox-microsandbox) | `createMicrosandboxNetworkSandboxSession()`, `resumeMicrosandboxNetworkSandboxSession()` | No: it has no `fork()` | Yes |
293
+
294
+ The same `openSandbox` on Cloud Run, through the sandbox service you deployed:
295
+
296
+ ```ts
297
+ import {
298
+ CloudRunSandboxNotFoundError,
299
+ createCloudRunNetworkSandboxSession,
300
+ resumeCloudRunNetworkSandboxSession,
301
+ } from 'ai-sdk-sandbox-cloud-run';
302
+
303
+ async function openSandbox() {
304
+ const settings = {
305
+ url: process.env.CLOUD_RUN_SANDBOX_URL!,
306
+ sandboxId: 'my-app',
307
+ ports: [4000],
308
+ // The bridges install their dependencies from npm.
309
+ allowedHosts: ['registry.npmjs.org'],
310
+ };
311
+ try {
312
+ const sandbox = await resumeCloudRunNetworkSandboxSession(settings);
313
+ await sandbox.killAllProcesses();
314
+ return sandbox;
315
+ } catch (error) {
316
+ if (!(error instanceof CloudRunSandboxNotFoundError)) throw error;
317
+ }
318
+ const template = await createHarnessSandboxTemplate({ harnesses: [harness] });
319
+ return createCloudRunNetworkSandboxSession({ ...settings, template });
320
+ }
321
+ ```
322
+
323
+ The [Next.js example](https://github.com/fpasquet/ai-sdk-harness/tree/main/examples/next-chat) picks one of the four with an environment variable, `EXAMPLE_SANDBOX`: see its `lib/sandbox.ts`.
324
+
279
325
  ### How many sessions run at once
280
326
 
281
327
  Claude Code and Codex run behind a bridge: a process the harness starts in the sandbox for each session, listening on a port of the sandbox that your server reaches over a WebSocket (published on `127.0.0.1` by `sbx`, tunnelled by the Cloud Run service). The harness takes the first port of the sandbox session it is given, so two sessions handed the same sandbox would both listen on it. `sharedSandbox()` gives each session a view of the sandbox with a port of its own: the same microVM, the same installed tools, a working directory of its own, and a port of `ports` for its bridge.
@@ -323,7 +369,78 @@ const store: SessionStore<Metadata> = {
323
369
 
324
370
  A resume state holds secrets: the token of the harness bridge, the placeholders of the credentials. Keep it apart from what you show, as you would a session token. The [Next.js example](https://github.com/fpasquet/ai-sdk-harness/tree/main/examples/next-chat) keeps its sessions in JSON files (`lib/session-store.ts`).
325
371
 
326
- Keep one manager per store: at its start, a manager takes the sessions the store holds as live for those of a process that crashed, and marks them `interrupted`.
372
+ Keep one manager per store, unless the store gives leases (below): at its start, a manager takes the sessions the store holds as live for those of a process that crashed, and marks them `interrupted`.
373
+
374
+ ### What a turn cost
375
+
376
+ `onTurn` is called once each turn is over and the session written, with a `TurnReport`: the turn's tokens, cache reads and writes included, each of its steps — the model that answered, its tokens and the tools it called, with their input — and the raw events of its harness, in order. Keep them to account for your sessions, or to read the rate limits of a subscription, which Claude Code reports as `rate_limit_event`:
377
+
378
+ ```ts
379
+ const sessions = createSessionManager<Metadata>({
380
+ // …
381
+ onTurn: async ({ sessionId, usage, steps, rawEvents }) => {
382
+ await db.turns.insert({
383
+ sessionId,
384
+ inputTokens: usage.inputTokens,
385
+ cacheReadTokens: usage.inputTokenDetails.cacheReadTokens,
386
+ outputTokens: usage.outputTokens,
387
+ models: [...new Set(steps.map(({ modelId }) => modelId))],
388
+ });
389
+ const limits = rawEvents.filter(
390
+ (event) => (event as { type?: string }).type === 'rate_limit_event',
391
+ );
392
+ if (limits.length > 0) await db.rateLimits.insert(sessionId, limits.at(-1));
393
+ },
394
+ });
395
+ ```
396
+
397
+ A `rate_limit_event` of Claude Code (signed in with a subscription) gives every window in `rate_limit_info.unifiedWindows`, such as `five_hour` and `seven_day`, each with its `utilization` (a fraction) and its `resetsAt` (in seconds). The harness names its tools in lower case: `bash`, not `Bash`.
398
+
399
+ The steps and the raw events come from the agent's result, `steps` and `fullStream`: a `HarnessAgent` gives both; an agent of your own that gives neither reports its tokens only. What `onTurn` throws goes to `onError`; the turn is settled either way.
400
+
401
+ ### Several instances
402
+
403
+ An application that runs on several instances — behind a load balancer, scaled out, or a new version deployed beside the old one — shares one store between their managers. The store tells them apart with leases: three more methods, on the same database, and the managers share it.
404
+
405
+ - Each manager holds the sessions it runs, and renews its own lease three times within `leaseTtlMs` (30 seconds by default).
406
+ - A request for a session another manager runs fails with `SessionOwnedElsewhereError`, which names it (`owner`, its `managerId`): send the request there, or answer 409 and let the client try again. Reading a session, `get()` and `list()`, works from any instance.
407
+ - A suspended session is held by nobody: the next message resumes it on whichever instance receives it.
408
+ - A manager that crashed stops renewing its lease. Once it expired, the others mark its live sessions `interrupted`, within a lease or so, and so does any manager that starts.
409
+
410
+ ```ts
411
+ const store: SessionStore<Metadata> = {
412
+ // …the five methods above, and:
413
+ claimSession: async (id, owner) => {
414
+ // One statement, so that two managers never both win: take the session unless a manager
415
+ // whose lease has not expired holds it.
416
+ const row = await db.query(
417
+ `INSERT INTO session_owners (session_id, owner) VALUES ($1, $2)
418
+ ON CONFLICT (session_id) DO UPDATE SET owner = $2
419
+ WHERE session_owners.owner = $2
420
+ OR NOT EXISTS (SELECT 1 FROM manager_leases l
421
+ WHERE l.owner = session_owners.owner AND l.expires_at > now())
422
+ RETURNING (SELECT owner FROM session_owners WHERE session_id = $1) AS previous_owner`,
423
+ [id, owner],
424
+ );
425
+ if (row) return { claimed: true, previousOwner: row.previous_owner };
426
+ const { owner: holder } = await db.query(
427
+ 'SELECT owner FROM session_owners WHERE session_id = $1',
428
+ [id],
429
+ );
430
+ return { claimed: false, owner: holder };
431
+ },
432
+ renewLease: (owner, ttlMs) =>
433
+ db.query(
434
+ `INSERT INTO manager_leases (owner, expires_at) VALUES ($1, now() + $2 * interval '1 ms')
435
+ ON CONFLICT (owner) DO UPDATE SET expires_at = excluded.expires_at`,
436
+ [owner, ttlMs],
437
+ ),
438
+ releaseSession: (id, owner) =>
439
+ db.query('DELETE FROM session_owners WHERE session_id = $1 AND owner = $2', [id, owner]),
440
+ };
441
+ ```
442
+
443
+ The lease's expiry is read on the database's clock, the one clock all instances share. `delete()` forgets who holds the session too. Give each manager a `managerId` you can route by — the instance's address, a pod name — to send a request to the instance that runs the session; never two running managers the same.
327
444
 
328
445
  ## Reference
329
446
 
@@ -333,7 +450,9 @@ Keep one manager per store: at its start, a manager takes the sessions the store
333
450
  | ----------------------- | ------------------------ | --------------------------------------------------------------------------------------------------- |
334
451
  | `agent` | required | The agent of a session, from `{ id, metadata }`: called at each start and resume |
335
452
  | `sandboxes` | required | `sharedSandbox()`, `sandboxPerSession()` or your own `SessionSandboxes` |
336
- | `store` | in memory | Where sessions are kept: a `SessionStore` |
453
+ | `store` | in memory | Where sessions are kept: a `SessionStore`, shared by several managers when it gives leases |
454
+ | `managerId` | `crypto.randomUUID()` | This manager among those sharing the store, named by `SessionOwnedElsewhereError` |
455
+ | `leaseTtlMs` | 30 seconds | How long this manager holds its sessions without renewing its lease, with a store that gives leases |
337
456
  | `idleTimeoutMs` | never | Suspend a session idle, or awaiting input, for this long |
338
457
  | `closeSuspendedAfterMs` | never | Close a session suspended for this long |
339
458
  | `turnTimeoutMs` | never | Cut a turn short after this long |
@@ -341,6 +460,7 @@ Keep one manager per store: at its start, a manager takes the sessions the store
341
460
  | `approve` | none | Answers an approval before anyone sees it: `{ approved, reason }`, or `undefined` to ask a person |
342
461
  | `generateId` | `crypto.randomUUID` | The ids of new sessions and of the messages of their turns |
343
462
  | `errorMessage` | `getHarnessErrorMessage` | What a client is told of an error in a turn; the session's `error` keeps the full message |
463
+ | `onTurn` | none | Called once each turn is over, with its `TurnReport`: tokens in detail, steps, raw harness events |
344
464
  | `onError` | `console.error` | Called with what fails in the background: a start, a suspension, a write to the store |
345
465
 
346
466
  | Method | What it does |
@@ -365,11 +485,12 @@ Each has an `isInstance()` that recognizes it whichever copy of the package thre
365
485
 
366
486
  - `SessionNotFoundError` (404): no session has this id. Carries `sessionId`.
367
487
  - `SessionConflictError` (409): the session cannot do it in its status — a message while `busy` or `awaiting-input`, `continue()` with nothing to answer, anything once `closed` or `interrupted`. Carries `sessionId` and `status`.
488
+ - `SessionOwnedElsewhereError` (409, or a redirect): another manager sharing the store runs the session. A `SessionConflictError` as well; carries `owner`, that manager's `managerId`.
368
489
  - `SessionCapacityError` (503): every sandbox slot is taken by a session that is not idle. Carries `capacity`.
369
490
 
370
491
  ## Limitations
371
492
 
372
- - **One process per store.** The live sessions are held in the memory of one process: run the manager on one instance, or give each instance its own sessions.
493
+ - **One process per store, unless it gives leases.** Without `claimSession`, `renewLease` and `releaseSession`, the live sessions are held in the memory of one process: run the manager on one instance. With them, a turn under way is still read from the instance that runs it: a client whose request reaches another one gets `SessionOwnedElsewhereError`. And a manager stalled for longer than its lease — a process frozen, not crashed — may find its sessions taken over by another one when it wakes up.
373
494
  - **A turn does not survive a crash.** A session the process held when it crashed is `interrupted`; a clean `shutdown()` suspends it instead. Streams that reconnect after a page reload are not supported yet: a reload cuts the turn its request carried when it passed `abortSignal: request.signal`.
374
495
  - **Approvals after a resume, with Claude Code.** A resumed Claude Code session (`@ai-sdk/harness-claude-code` up to 1.0.152) sends "Continue." in place of the answers to the approvals it asks afterwards: the tool call is rejected, and asked again ([vercel/ai#22549](https://github.com/vercel/ai/issues/22549)). Approvals work in a session until it is suspended; a session that resumed asks again and again. A turn that was waiting for an approval when its session was suspended resumes without it: `continue()` tells the agent the answers in a message.
375
496
  - **No approvals of Codex's own tools.** `@ai-sdk/harness-codex` (up to 1.0.150) runs Codex with approvals turned off and refuses any `permissionMode` but `'allow-all'`, though Codex itself can ask before a command or a patch ([vercel/ai#22550](https://github.com/vercel/ai/issues/22550)). Your own tools can still ask, through `toolApproval`.
@@ -3,8 +3,12 @@ import type { HarnessAgentResumeSessionState, HarnessAgentSession } from '@ai-sd
3
3
  import type { StreamTextResult, ToolApprovalResponse, ToolResultPart, ToolSet } from 'ai';
4
4
  /** A turn's result, whatever the agent's runtime context and output. */
5
5
  type AnyStreamTextResult = StreamTextResult<ToolSet, any, any>;
6
- /** What the manager reads of a turn: a `StreamTextResult` is one. */
7
- export type SessionStreamResult = Pick<AnyStreamTextResult, 'totalUsage' | 'toUIMessageStream'>;
6
+ /**
7
+ * What the manager reads of a turn: a `StreamTextResult` is one. Its `steps` and `fullStream`, when
8
+ * it has them, give `onTurn` the model, the tokens and the tools of each step, and the raw events of
9
+ * the harness.
10
+ */
11
+ export type SessionStreamResult = Partial<Pick<AnyStreamTextResult, 'fullStream' | 'steps'>> & Pick<AnyStreamTextResult, 'totalUsage' | 'toUIMessageStream'>;
8
12
  /**
9
13
  * What the manager needs of an agent: a `HarnessAgent` is one, whatever its harness, tools and
10
14
  * settings.
@@ -1,7 +1,7 @@
1
1
  import type { ToolApprovalResponse, ToolResultPart, UIMessage, UIMessageChunk } from 'ai';
2
2
  import type { SessionAgent } from './agent.js';
3
3
  import type { SessionSandboxes } from './sandboxes.js';
4
- import type { ApprovalVerdict, PendingApproval, SessionRecord, SessionSummary } from './session.js';
4
+ import type { ApprovalVerdict, PendingApproval, SessionRecord, SessionSummary, TurnReport } from './session.js';
5
5
  import type { SessionStore } from './store.js';
6
6
  export interface SessionManagerOptions<METADATA> {
7
7
  /**
@@ -15,8 +15,25 @@ export interface SessionManagerOptions<METADATA> {
15
15
  }) => Promise<SessionAgent> | SessionAgent;
16
16
  /** How sessions get a sandbox: `sharedSandbox()`, `sandboxPerSession()`, or your own. */
17
17
  sandboxes: SessionSandboxes;
18
- /** Where sessions are kept. Default: in memory, `createMemorySessionStore()`. */
18
+ /**
19
+ * Where sessions are kept. Default: in memory, `createMemorySessionStore()`. A store that gives
20
+ * leases (`claimSession`, `renewLease`, `releaseSession`) is shared by several managers: the
21
+ * instances of an application.
22
+ */
19
23
  store?: SessionStore<METADATA>;
24
+ /**
25
+ * This manager, among those sharing the store: what `SessionOwnedElsewhereError.owner` names,
26
+ * to send a request to the instance that runs the session. Default: `crypto.randomUUID()`. Give
27
+ * an id of your own — the instance's address — to route by it; never two running managers the
28
+ * same.
29
+ */
30
+ managerId?: string;
31
+ /**
32
+ * How long this manager's hold on its sessions lasts without renewal, in milliseconds, with a
33
+ * store that gives leases: renewed three times within it, and what another manager waits, after
34
+ * a crash, before taking its sessions over. Default: 30 seconds.
35
+ */
36
+ leaseTtlMs?: number;
20
37
  /**
21
38
  * Suspend a session idle, or awaiting input, for this long, in milliseconds: its harness session
22
39
  * stops, its state is kept and its sandbox freed. The next message brings it back. Default:
@@ -51,6 +68,15 @@ export interface SessionManagerOptions<METADATA> {
51
68
  * `@ai-sdk/harness`. The session's `error` keeps the full message.
52
69
  */
53
70
  errorMessage?: (error: unknown) => string;
71
+ /**
72
+ * Called once each turn is over and the session written, with what the turn cost: its tokens,
73
+ * cache reads and writes included, the model, tokens and tool calls of each step, and the raw
74
+ * events of its harness — what an application needs to account for its sessions, or to read the
75
+ * rate limits of a subscription. What it throws goes to `onError`; the turn is settled either way.
76
+ */
77
+ onTurn?: (report: TurnReport, context: {
78
+ session: SessionSummary<METADATA>;
79
+ }) => PromiseLike<void> | void;
54
80
  /**
55
81
  * Called with what fails in the background: a start, a suspension, a write to the store. Default:
56
82
  * `console.error`.
@@ -105,6 +131,8 @@ export type SessionEvent<METADATA = unknown> = {
105
131
  * message to the last, across suspensions and restarts. See {@link createSessionManager}.
106
132
  */
107
133
  export interface SessionManager<METADATA = unknown> {
134
+ /** This manager, among those sharing the store. */
135
+ readonly managerId: string;
108
136
  /**
109
137
  * Opens a session and returns at once, `preparing`: its sandbox and its harness session are
110
138
  * opened in the background — minutes the very first time, when the harness is installed in the
@@ -1,4 +1,4 @@
1
- import type { UIMessage } from 'ai';
1
+ import type { LanguageModelUsage, UIMessage } from 'ai';
2
2
  /**
3
3
  * Where a session stands in its lifecycle.
4
4
  *
@@ -29,6 +29,36 @@ export interface SessionUsage {
29
29
  outputTokens: number;
30
30
  totalTokens: number;
31
31
  }
32
+ /** A step of a turn: the model that answered, the tokens it used, and the tools it called. */
33
+ export interface TurnStepReport {
34
+ /** The model, as the harness names it: `claude-sonnet-4-5`, for instance. */
35
+ modelId: string | undefined;
36
+ usage: LanguageModelUsage;
37
+ /** The tools it called, with their input: the skill it loaded, the subagent it started… */
38
+ toolCalls: {
39
+ toolCallId: string;
40
+ toolName: string;
41
+ input: unknown;
42
+ }[];
43
+ }
44
+ /**
45
+ * A turn once it is over, for accounting: its tokens, cache reads and writes included, each of its
46
+ * steps, and the raw events its harness sent — rate limits, for instance, which Claude Code reports
47
+ * as `rate_limit_event`.
48
+ */
49
+ export interface TurnReport {
50
+ sessionId: string;
51
+ startedAt: Date;
52
+ endedAt: Date;
53
+ /** What went wrong, when the turn failed or was cut short. */
54
+ failure: string | undefined;
55
+ /** The tokens of the whole turn, over all its streams. */
56
+ usage: LanguageModelUsage;
57
+ /** Its steps, in order; empty when the agent's results do not give them. */
58
+ steps: TurnStepReport[];
59
+ /** The raw events of the harness, in order; empty when the agent's results do not stream them. */
60
+ rawEvents: unknown[];
61
+ }
32
62
  /** An answer to an approval request, as `ToolApprovalResponse` carries it. */
33
63
  export interface ApprovalVerdict {
34
64
  approved: boolean;
@@ -21,6 +21,37 @@ export interface SessionStore<METADATA = unknown> {
21
21
  list(): Promise<SessionSummary<METADATA>[]>;
22
22
  /** What a suspended session needs to resume. */
23
23
  getResumeState(id: string): Promise<HarnessAgentResumeSessionState | undefined>;
24
- /** Forgets the session and its resume state. */
24
+ /** Forgets the session and its resume state, and who holds it. */
25
25
  delete(id: string): Promise<void>;
26
+ /**
27
+ * Optional, with `renewLease` and `releaseSession`: lets several managers share the store — the
28
+ * instances of an application behind a load balancer, or a new version deployed beside the old
29
+ * one. A manager holds each session it runs, and keeps its lease alive while it runs: another
30
+ * manager leaves its sessions alone, and takes them over once its lease expired.
31
+ *
32
+ * Makes `owner` hold the session `id`, in one atomic step (a compare-and-set on a database row),
33
+ * unless another manager whose lease has not expired holds it. The session need not be saved
34
+ * yet: `create()` takes it before writing it.
35
+ */
36
+ claimSession?(id: string, owner: string): Promise<SessionClaim>;
37
+ /**
38
+ * Keeps the lease of the manager `owner` alive for `ttlMs` milliseconds from now, by the store's
39
+ * clock: what decides whether the sessions it holds may be taken over.
40
+ */
41
+ renewLease?(owner: string, ttlMs: number): Promise<void>;
42
+ /** Lets go of the session `id`, when `owner` holds it. */
43
+ releaseSession?(id: string, owner: string): Promise<void>;
26
44
  }
45
+ /** What `claimSession` made of a claim. */
46
+ export type SessionClaim = {
47
+ claimed: false;
48
+ /** The manager that holds the session, its lease alive. */
49
+ owner: string;
50
+ } | {
51
+ claimed: true;
52
+ /**
53
+ * Who held the session before: `null` when nobody did, the claiming manager itself when it
54
+ * held it already, or a manager whose lease expired.
55
+ */
56
+ previousOwner: null | string;
57
+ };
@@ -0,0 +1,19 @@
1
+ import type { SessionStatus } from '../definitions/session.js';
2
+ import { SessionConflictError } from './session-conflict-error.js';
3
+ /** What `isInstance` looks for, shared by every copy of this package. */
4
+ declare const MARKER: unique symbol;
5
+ /**
6
+ * Another manager sharing the store runs the session: send the request to it — the instance
7
+ * `owner` names —, or answer 409 and let the client try again. A `SessionConflictError` as well.
8
+ */
9
+ export declare class SessionOwnedElsewhereError extends SessionConflictError {
10
+ /** The manager that runs the session: its `managerId`. */
11
+ readonly owner: string;
12
+ private readonly [MARKER];
13
+ constructor(sessionId: string, status: SessionStatus,
14
+ /** The manager that runs the session: its `managerId`. */
15
+ owner: string);
16
+ /** Whether `error` is one, whichever copy of the package threw it. */
17
+ static isInstance(error: unknown): error is SessionOwnedElsewhereError;
18
+ }
19
+ export {};
@@ -0,0 +1,22 @@
1
+ import { SessionConflictError } from './session-conflict-error.js';
2
+ /** What `isInstance` looks for, shared by every copy of this package. */
3
+ const MARKER = Symbol.for('ai-sdk-harness-sessions.SessionOwnedElsewhereError');
4
+ /**
5
+ * Another manager sharing the store runs the session: send the request to it — the instance
6
+ * `owner` names —, or answer 409 and let the client try again. A `SessionConflictError` as well.
7
+ */
8
+ export class SessionOwnedElsewhereError extends SessionConflictError {
9
+ owner;
10
+ [MARKER] = true;
11
+ constructor(sessionId, status,
12
+ /** The manager that runs the session: its `managerId`. */
13
+ owner) {
14
+ super(sessionId, status, `The session ${sessionId} is run by another manager, ${owner}.`);
15
+ this.owner = owner;
16
+ this.name = 'SessionOwnedElsewhereError';
17
+ }
18
+ /** Whether `error` is one, whichever copy of the package threw it. */
19
+ static isInstance(error) {
20
+ return typeof error === 'object' && error !== null && MARKER in error;
21
+ }
22
+ }
package/dist/index.d.ts CHANGED
@@ -2,11 +2,12 @@ export type { SessionAgent, SessionStreamResult } from './definitions/agent.js';
2
2
  export type { ContinueOptions, SendOptions, SessionEvent, SessionManager, SessionManagerOptions, SessionTurn, } from './definitions/manager.js';
3
3
  export type { SessionSandboxes, SessionSandboxLease } from './definitions/sandboxes.js';
4
4
  export { LIVE_STATUSES, SESSION_STATUSES } from './definitions/session.js';
5
- export type { PendingApproval, PendingInput, PendingToolCall, SessionRecord, SessionStatus, SessionSummary, SessionUsage, } from './definitions/session.js';
6
- export type { SessionStore } from './definitions/store.js';
5
+ export type { PendingApproval, PendingInput, PendingToolCall, SessionRecord, SessionStatus, SessionSummary, SessionUsage, TurnReport, TurnStepReport, } from './definitions/session.js';
6
+ export type { SessionClaim, SessionStore } from './definitions/store.js';
7
7
  export { SessionCapacityError } from './errors/session-capacity-error.js';
8
8
  export { SessionConflictError } from './errors/session-conflict-error.js';
9
9
  export { SessionNotFoundError } from './errors/session-not-found-error.js';
10
+ export { SessionOwnedElsewhereError } from './errors/session-owned-elsewhere-error.js';
10
11
  export { createSessionManager } from './manager/session-manager.js';
11
12
  export { sandboxPerSession } from './sandboxes/sandbox-per-session.js';
12
13
  export type { SandboxPerSessionOptions } from './sandboxes/sandbox-per-session.js';
package/dist/index.js CHANGED
@@ -2,6 +2,7 @@ export { LIVE_STATUSES, SESSION_STATUSES } from './definitions/session.js';
2
2
  export { SessionCapacityError } from './errors/session-capacity-error.js';
3
3
  export { SessionConflictError } from './errors/session-conflict-error.js';
4
4
  export { SessionNotFoundError } from './errors/session-not-found-error.js';
5
+ export { SessionOwnedElsewhereError } from './errors/session-owned-elsewhere-error.js';
5
6
  export { createSessionManager } from './manager/session-manager.js';
6
7
  export { sandboxPerSession } from './sandboxes/sandbox-per-session.js';
7
8
  export { sharedSandbox } from './sandboxes/shared-sandbox.js';
@@ -0,0 +1,38 @@
1
+ import type { SessionClaim, SessionStore } from '../definitions/store.js';
2
+ type Report = (action: string, sessionId?: string) => (error: unknown) => void;
3
+ /** The methods of a store that shares its sessions between managers. */
4
+ type LeasingStore = Pick<SessionStore<unknown>, 'claimSession' | 'releaseSession' | 'renewLease'>;
5
+ export interface SessionLeasesOptions {
6
+ store: LeasingStore;
7
+ /** This manager. */
8
+ owner: string;
9
+ ttlMs: number;
10
+ report: Report;
11
+ }
12
+ /**
13
+ * The sessions a manager holds in a store it shares with other managers, and its lease that
14
+ * keeps them: renewed in the background, checked by the store when another manager claims one. A
15
+ * store without leases has one manager: every claim succeeds, and nothing is renewed.
16
+ */
17
+ export declare class SessionLeases {
18
+ readonly owner: string;
19
+ /** Whether the store shares its sessions between managers. */
20
+ readonly shared: boolean;
21
+ private readonly store;
22
+ private readonly ttlMs;
23
+ private readonly report;
24
+ private renewal;
25
+ constructor({ store, owner, ttlMs, report }: SessionLeasesOptions);
26
+ /**
27
+ * Takes the lease, then keeps it alive until `stop()`; `onLeaseRenewed` runs once per lease,
28
+ * after a renewal.
29
+ */
30
+ start(onLeaseRenewed: () => void): Promise<void>;
31
+ stop(): void;
32
+ /** Makes this manager hold the session, unless another one does. */
33
+ claim(id: string): Promise<SessionClaim>;
34
+ /** Lets go of the session; a failure is reported, the lease expiring lets go of it anyway. */
35
+ release(id: string): Promise<void>;
36
+ private renew;
37
+ }
38
+ export {};
@@ -0,0 +1,70 @@
1
+ /** How often a manager renews its lease: three times within its life. */
2
+ const RENEWALS_PER_LEASE = 3;
3
+ /** Whether the store gives its lease methods: all three, or none. */
4
+ function sharesSessions(store) {
5
+ const given = [
6
+ typeof store.claimSession,
7
+ typeof store.renewLease,
8
+ typeof store.releaseSession,
9
+ ].filter((type) => type === 'function').length;
10
+ if (given !== 0 && given !== 3) {
11
+ throw new TypeError('A session store shares its sessions with claimSession, renewLease and releaseSession, all three.');
12
+ }
13
+ return given === 3;
14
+ }
15
+ /**
16
+ * The sessions a manager holds in a store it shares with other managers, and its lease that
17
+ * keeps them: renewed in the background, checked by the store when another manager claims one. A
18
+ * store without leases has one manager: every claim succeeds, and nothing is renewed.
19
+ */
20
+ export class SessionLeases {
21
+ owner;
22
+ /** Whether the store shares its sessions between managers. */
23
+ shared;
24
+ store;
25
+ ttlMs;
26
+ report;
27
+ renewal;
28
+ constructor({ store, owner, ttlMs, report }) {
29
+ this.shared = sharesSessions(store);
30
+ this.store = store;
31
+ this.owner = owner;
32
+ this.ttlMs = ttlMs;
33
+ this.report = report;
34
+ }
35
+ /**
36
+ * Takes the lease, then keeps it alive until `stop()`; `onLeaseRenewed` runs once per lease,
37
+ * after a renewal.
38
+ */
39
+ async start(onLeaseRenewed) {
40
+ if (!this.shared)
41
+ return;
42
+ await this.renew();
43
+ let renewals = 0;
44
+ this.renewal = setInterval(() => {
45
+ void this.renew().then(() => {
46
+ renewals += 1;
47
+ if (renewals % RENEWALS_PER_LEASE === 0)
48
+ onLeaseRenewed();
49
+ }, this.report('lease'));
50
+ }, this.ttlMs / RENEWALS_PER_LEASE);
51
+ this.renewal.unref();
52
+ }
53
+ stop() {
54
+ clearInterval(this.renewal);
55
+ }
56
+ /** Makes this manager hold the session, unless another one does. */
57
+ claim(id) {
58
+ const claimed = this.shared ? this.store.claimSession?.(id, this.owner) : undefined;
59
+ return claimed ?? Promise.resolve({ claimed: true, previousOwner: this.owner });
60
+ }
61
+ /** Lets go of the session; a failure is reported, the lease expiring lets go of it anyway. */
62
+ async release(id) {
63
+ if (!this.shared)
64
+ return;
65
+ await this.store.releaseSession?.(id, this.owner)?.catch(this.report('release', id));
66
+ }
67
+ async renew() {
68
+ await this.store.renewLease?.(this.owner, this.ttlMs);
69
+ }
70
+ }
@@ -12,7 +12,9 @@ import type { SessionManager, SessionManagerOptions } from '../definitions/manag
12
12
  * a session the process left live without it is `interrupted` at the next start.
13
13
  * 6. `close()` ends a session for good.
14
14
  *
15
- * Every change is written to the store and sent to the subscribers. Keep one manager per store:
16
- * a second one would take the sessions of the first for those of a crashed process.
15
+ * Every change is written to the store and sent to the subscribers. Several managers share a store
16
+ * that gives them leases (`claimSession`, `renewLease`, `releaseSession`): each runs the sessions it
17
+ * holds. Without them, keep one manager per store: a second one would take the sessions of the
18
+ * first for those of a crashed process.
17
19
  */
18
20
  export declare function createSessionManager<METADATA = unknown>(options: SessionManagerOptions<METADATA>): SessionManager<METADATA>;
@@ -4,12 +4,14 @@ import { randomUUID } from 'node:crypto';
4
4
  import { SessionCapacityError } from '../errors/session-capacity-error.js';
5
5
  import { SessionConflictError } from '../errors/session-conflict-error.js';
6
6
  import { SessionNotFoundError } from '../errors/session-not-found-error.js';
7
+ import { SessionOwnedElsewhereError } from '../errors/session-owned-elsewhere-error.js';
7
8
  import { createMemorySessionStore } from '../store/memory-session-store.js';
8
9
  import { decidedChunks, decidedResponses, signatureOf, verdictOf, } from '../turns/approval-answers.js';
9
10
  import { describeResponses, pendingInputOf, responsesFrom, withResponses, } from '../turns/pending-input.js';
10
11
  import { streamTurn } from '../turns/turn-stream.js';
11
12
  import { isLive, isReady, liveSession, messageOf, newRecord, summaryOf, textOf, } from './live-session.js';
12
13
  import { SessionJournal } from './session-journal.js';
14
+ import { SessionLeases } from './session-leases.js';
13
15
  const INTERRUPTED_BY_CRASH = 'The application stopped without suspending the session.';
14
16
  const CUT_BY_SHUTDOWN = 'The turn was cut short: the application stopped. Send the message again.';
15
17
  const FAILED_BY_SHUTDOWN = 'The application stopped before the session was ready.';
@@ -17,6 +19,8 @@ const NOT_STOPPED = 'The harness session could not be stopped cleanly.';
17
19
  const NOTHING_TO_RESUME = 'Nothing was kept to resume the session from.';
18
20
  /** How long a shutdown waits for a turn it cut short to wind down. */
19
21
  const WIND_DOWN_MS = 10_000;
22
+ /** How long a manager's lease on its sessions lasts, unless renewed. */
23
+ const LEASE_TTL_MS = 30_000;
20
24
  /** How many approvals `approve` answers in one turn, at most, before leaving them to a person. */
21
25
  const MAX_ANSWERED_PER_TURN = 200;
22
26
  /**
@@ -63,6 +67,17 @@ function withoutUnfinishedTurn(state) {
63
67
  const defaultReport = (error, { action, sessionId }) =>
64
68
  // eslint-disable-next-line no-console -- the default, until the application passes onError
65
69
  console.error(`ai-sdk-harness-sessions: ${action}${sessionId ? ` ${sessionId}` : ''}:`, error);
70
+ function settingsOf(options) {
71
+ return {
72
+ idleTimeoutMs: options.idleTimeoutMs ?? 0,
73
+ closeSuspendedAfterMs: options.closeSuspendedAfterMs ?? 0,
74
+ turnTimeoutMs: options.turnTimeoutMs ?? 0,
75
+ suspendIdleWhenFull: options.suspendIdleWhenFull ?? true,
76
+ generateId: options.generateId ?? randomUUID,
77
+ errorMessage: options.errorMessage ?? getHarnessErrorMessage,
78
+ onError: options.onError ?? defaultReport,
79
+ };
80
+ }
66
81
  /**
67
82
  * Manages the sessions of an application with its coding agents, `HarnessAgent`s of any harness:
68
83
  *
@@ -76,17 +91,23 @@ console.error(`ai-sdk-harness-sessions: ${action}${sessionId ? ` ${sessionId}` :
76
91
  * a session the process left live without it is `interrupted` at the next start.
77
92
  * 6. `close()` ends a session for good.
78
93
  *
79
- * Every change is written to the store and sent to the subscribers. Keep one manager per store:
80
- * a second one would take the sessions of the first for those of a crashed process.
94
+ * Every change is written to the store and sent to the subscribers. Several managers share a store
95
+ * that gives them leases (`claimSession`, `renewLease`, `releaseSession`): each runs the sessions it
96
+ * holds. Without them, keep one manager per store: a second one would take the sessions of the
97
+ * first for those of a crashed process.
81
98
  */
82
99
  export function createSessionManager(options) {
83
100
  return new HarnessSessionManager(options);
84
101
  }
85
102
  class HarnessSessionManager {
86
103
  options;
104
+ get managerId() {
105
+ return this.leases.owner;
106
+ }
87
107
  store;
88
108
  settings;
89
109
  journal;
110
+ leases;
90
111
  live = new Map();
91
112
  /** Starts run one at a time: the first one installs the harness in the sandbox. */
92
113
  starting = Promise.resolve();
@@ -96,17 +117,18 @@ class HarnessSessionManager {
96
117
  constructor(options) {
97
118
  this.options = options;
98
119
  this.store = options.store ?? createMemorySessionStore();
99
- this.settings = {
100
- idleTimeoutMs: options.idleTimeoutMs ?? 0,
101
- closeSuspendedAfterMs: options.closeSuspendedAfterMs ?? 0,
102
- turnTimeoutMs: options.turnTimeoutMs ?? 0,
103
- suspendIdleWhenFull: options.suspendIdleWhenFull ?? true,
104
- generateId: options.generateId ?? randomUUID,
105
- errorMessage: options.errorMessage ?? getHarnessErrorMessage,
106
- onError: options.onError ?? defaultReport,
107
- };
120
+ this.settings = settingsOf(options);
108
121
  this.journal = new SessionJournal(this.store, this.report);
109
- this.recovered = this.recover().catch(this.report('recover'));
122
+ this.leases = new SessionLeases({
123
+ store: this.store,
124
+ owner: options.managerId ?? randomUUID(),
125
+ ttlMs: options.leaseTtlMs ?? LEASE_TTL_MS,
126
+ report: this.report,
127
+ });
128
+ this.recovered = this.leases
129
+ .start(() => void this.recoverOrphans().catch(this.report('recover')))
130
+ .then(() => this.recover())
131
+ .catch(this.report('recover'));
110
132
  const { closeSuspendedAfterMs } = this.settings;
111
133
  if (closeSuspendedAfterMs > 0) {
112
134
  this.sweeper = setInterval(() => void this.sweep().catch(this.report('sweep')), Math.min(closeSuspendedAfterMs, 60_000));
@@ -118,7 +140,13 @@ class HarnessSessionManager {
118
140
  const existing = this.live.get(id)?.record ?? (await this.store.get(id));
119
141
  if (existing !== undefined)
120
142
  throw this.conflict(existing, `A session ${id} already exists.`);
121
- await this.makeRoom();
143
+ const claim = await this.leases.claim(id);
144
+ if (!claim.claimed)
145
+ throw new SessionOwnedElsewhereError(id, 'preparing', claim.owner);
146
+ await this.makeRoom().catch(async (error) => {
147
+ await this.leases.release(id);
148
+ throw error;
149
+ });
122
150
  const session = liveSession(newRecord(id, metadata));
123
151
  this.live.set(id, session);
124
152
  await this.transition(session, 'preparing');
@@ -190,23 +218,26 @@ class HarnessSessionManager {
190
218
  const live = this.live.get(id);
191
219
  if (live !== undefined)
192
220
  return this.changeMetadata(live.record, change(live.record.metadata));
193
- const kept = await this.store.get(id);
194
- if (kept === undefined)
195
- throw new SessionNotFoundError(id);
196
- const resumeState = kept.status === 'suspended' ? await this.store.getResumeState(id) : undefined;
197
- const changed = change(kept.metadata);
198
- // Brought back while the store was read: the live session is the one to change.
199
- const racing = this.live.get(id);
200
- if (racing !== undefined)
201
- return this.changeMetadata(racing.record, changed);
202
- return this.changeMetadata(kept, changed, resumeState);
221
+ const { record: kept, release } = await this.hold(id);
222
+ try {
223
+ const resumeState = kept.status === 'suspended' ? await this.store.getResumeState(id) : undefined;
224
+ const changed = change(kept.metadata);
225
+ // Brought back while the store was read: the live session is the one to change.
226
+ const racing = this.live.get(id);
227
+ if (racing !== undefined)
228
+ return await this.changeMetadata(racing.record, changed);
229
+ return await this.changeMetadata(kept, changed, resumeState);
230
+ }
231
+ finally {
232
+ await release();
233
+ }
203
234
  };
204
235
  interrupt = async (id) => {
205
236
  await this.recovered;
206
237
  const session = this.live.get(id);
207
- if (session === undefined && (await this.store.get(id)) === undefined) {
208
- throw new SessionNotFoundError(id);
209
- }
238
+ // Run by another manager, it throws: the request is for that one.
239
+ if (session === undefined)
240
+ await (await this.hold(id)).release();
210
241
  const turn = session?.turn;
211
242
  if (turn === undefined)
212
243
  return;
@@ -217,9 +248,8 @@ class HarnessSessionManager {
217
248
  await this.ready();
218
249
  const session = this.live.get(id);
219
250
  if (session === undefined) {
220
- const kept = await this.store.get(id);
221
- if (kept === undefined)
222
- throw new SessionNotFoundError(id);
251
+ const { record: kept, release } = await this.hold(id);
252
+ await release();
223
253
  if (kept.status === 'suspended')
224
254
  return summaryOf(kept);
225
255
  throw this.conflict(kept);
@@ -241,10 +271,13 @@ class HarnessSessionManager {
241
271
  await session.ending;
242
272
  return summaryOf(session.record);
243
273
  }
244
- const kept = await this.store.get(id);
245
- if (kept === undefined)
246
- throw new SessionNotFoundError(id);
247
- return this.closeKept(kept);
274
+ const { record: kept, release } = await this.hold(id);
275
+ try {
276
+ return await this.closeKept(kept);
277
+ }
278
+ finally {
279
+ await release();
280
+ }
248
281
  };
249
282
  delete = async (id) => {
250
283
  await this.close(id);
@@ -256,10 +289,23 @@ class HarnessSessionManager {
256
289
  return;
257
290
  this.stopped = true;
258
291
  clearInterval(this.sweeper);
292
+ this.leases.stop();
259
293
  await Promise.all([...this.live.values()].map((session) => this.shutdownSession(session)));
260
294
  await this.options.sandboxes.shutdown?.().catch(this.report('shutdown'));
261
295
  await this.journal.flush();
262
296
  };
297
+ /** Hands a turn that is over to `onTurn`: what it throws goes to `onError`, the turn stays settled. */
298
+ async reportTurn(session, turn) {
299
+ if (this.options.onTurn === undefined)
300
+ return;
301
+ const report = { sessionId: session.record.id, ...turn, endedAt: new Date() };
302
+ try {
303
+ await this.options.onTurn(report, { session: summaryOf(session.record) });
304
+ }
305
+ catch (error) {
306
+ this.report('report turn', session.record.id)(error);
307
+ }
308
+ }
263
309
  report = (action, sessionId) => (error) => this.settings.onError(error, { action, ...(sessionId !== undefined && { sessionId }) });
264
310
  async ready() {
265
311
  if (this.stopped)
@@ -267,19 +313,69 @@ class HarnessSessionManager {
267
313
  await this.recovered;
268
314
  }
269
315
  /**
270
- * A session still live in the store was live when the application last stopped without
271
- * suspending it: it cannot be brought back.
316
+ * At the start: a session still live in the store, that no other manager runs, was live when
317
+ * its manager stopped without suspending it — this one, before, or one that crashed: it cannot
318
+ * be brought back.
272
319
  */
273
320
  async recover() {
274
321
  for (const { id, status } of await this.store.list()) {
275
322
  if (!isLive(status))
276
323
  continue;
324
+ const claim = await this.leases.claim(id);
325
+ if (!claim.claimed)
326
+ continue;
277
327
  const record = await this.store.get(id);
278
- if (record === undefined)
328
+ if (record === undefined || !isLive(record.status)) {
329
+ await this.leases.release(id);
279
330
  continue;
280
- await this.transition(liveSession(record), 'interrupted', { error: INTERRUPTED_BY_CRASH });
331
+ }
332
+ await this.interruptOrphan(record);
281
333
  }
282
334
  }
335
+ /**
336
+ * Then, once per lease: the sessions of the managers sharing the store whose lease expired —
337
+ * they crashed — are interrupted.
338
+ */
339
+ async recoverOrphans() {
340
+ if (this.stopped)
341
+ return;
342
+ for (const { id, status } of await this.store.list()) {
343
+ if (!isLive(status) || this.live.has(id))
344
+ continue;
345
+ await this.hold(id).then(({ release }) => release(), () => undefined);
346
+ }
347
+ }
348
+ /**
349
+ * The session `id` as the store keeps it, held by this manager until `release()`: what to read
350
+ * and change of a session no manager runs. Throws `SessionOwnedElsewhereError` when another
351
+ * manager runs it. A session the store keeps live although no manager runs it any more — its
352
+ * manager crashed — is `interrupted` first.
353
+ */
354
+ async hold(id) {
355
+ if ((await this.store.get(id)) === undefined)
356
+ throw new SessionNotFoundError(id);
357
+ const claim = await this.leases.claim(id);
358
+ // Read again once held: what the manager that held it last wrote.
359
+ const record = await this.store.get(id);
360
+ if (!claim.claimed) {
361
+ throw new SessionOwnedElsewhereError(id, record?.status ?? 'preparing', claim.owner);
362
+ }
363
+ const ours = claim.previousOwner === this.leases.owner;
364
+ const release = () => (ours ? Promise.resolve() : this.leases.release(id));
365
+ if (record === undefined) {
366
+ await release();
367
+ throw new SessionNotFoundError(id);
368
+ }
369
+ if (!ours && isLive(record.status))
370
+ return { record: await this.interruptOrphan(record), release };
371
+ return { record, release };
372
+ }
373
+ /** Interrupts a session left live by a manager that stopped without suspending it. */
374
+ async interruptOrphan(record) {
375
+ const session = liveSession(record);
376
+ await this.transition(session, 'interrupted', { error: INTERRUPTED_BY_CRASH });
377
+ return session.record;
378
+ }
283
379
  /** The one place a session changes status: it is written, unless told not to, and sent out. */
284
380
  transition(session, status, { error, persist = true, resumeState } = {}) {
285
381
  const { record } = session;
@@ -290,7 +386,9 @@ class HarnessSessionManager {
290
386
  else
291
387
  record.error = error;
292
388
  this.journal.updated(record);
293
- return persist ? this.journal.write(record, resumeState) : Promise.resolve();
389
+ const written = persist ? this.journal.write(record, resumeState) : Promise.resolve();
390
+ // Once it is written: the manager that claims it next reads where it stopped.
391
+ return isLive(status) ? written : written.then(() => this.leases.release(record.id));
294
392
  }
295
393
  /** Writes new metadata, without counting it as activity: `updatedAt` stays. */
296
394
  async changeMetadata(record, metadata, resumeState) {
@@ -377,8 +475,9 @@ class HarnessSessionManager {
377
475
  this.report(resumeFrom === undefined ? 'start' : 'resume', id)(error);
378
476
  if (this.live.get(id) === session)
379
477
  this.live.delete(id);
380
- await this.teardown(session);
381
- // A resume that failed keeps what it resumes from: the next message tries again.
478
+ // A resume that failed keeps what it resumes from — its state, its sandbox and its files —:
479
+ // the next message tries again.
480
+ await (resumeFrom === undefined ? this.teardown(session) : this.release(session));
382
481
  await (resumeFrom === undefined
383
482
  ? this.transition(session, 'failed', { error: messageOf(error) })
384
483
  : this.transition(session, 'suspended', {
@@ -395,6 +494,13 @@ class HarnessSessionManager {
395
494
  await handle?.destroy().catch(this.report('destroy', record.id));
396
495
  await lease?.close().catch(this.report('close', record.id));
397
496
  }
497
+ /** Hands the sandbox of a session that never started back, keeping its files for a resume. */
498
+ async release(session) {
499
+ const { lease, record } = session;
500
+ session.lease = undefined;
501
+ clearTimeout(session.idleTimer);
502
+ await lease?.suspend().catch(this.report('suspend', record.id));
503
+ }
398
504
  /**
399
505
  * Suspends the session: its harness session stops, keeping what it needs to resume, and its
400
506
  * sandbox is freed.
@@ -423,12 +529,16 @@ class HarnessSessionManager {
423
529
  async openSession(id) {
424
530
  let session = this.live.get(id);
425
531
  if (session === undefined) {
426
- const kept = await this.store.get(id);
427
- if (kept === undefined)
428
- throw new SessionNotFoundError(id);
429
- if (kept.status !== 'suspended' && kept.status !== 'failed')
430
- throw this.conflict(kept);
431
- session = await this.revive(kept);
532
+ const { record: kept, release } = await this.hold(id);
533
+ try {
534
+ if (kept.status !== 'suspended' && kept.status !== 'failed')
535
+ throw this.conflict(kept);
536
+ session = await this.revive(kept);
537
+ }
538
+ catch (error) {
539
+ await release();
540
+ throw error;
541
+ }
432
542
  }
433
543
  await session.ready.catch(() => undefined);
434
544
  if (this.live.get(id) !== session || !isReady(session))
@@ -481,9 +591,14 @@ class HarnessSessionManager {
481
591
  : undefined;
482
592
  let finish;
483
593
  session.turn = { abort, done: new Promise((resolve) => (finish = resolve)) };
594
+ const startedAt = new Date();
484
595
  await this.transition(session, 'busy');
485
596
  const settle = async (failure) => {
486
597
  clearTimeout(timer);
598
+ // Cut short by an interrupt, a timeout or the caller: before the turn is over, so that a
599
+ // close waits for it, and finds the harness session to destroy.
600
+ if (abort.signal.aborted || callerSignal?.aborted === true)
601
+ await this.renew(session, handle);
487
602
  await this.settleTurn(session, handle, failure);
488
603
  finish();
489
604
  };
@@ -502,7 +617,7 @@ class HarnessSessionManager {
502
617
  generateMessageId: generateId,
503
618
  errorMessage,
504
619
  next: (messages) => this.answerPause(session, messages, answered),
505
- onSettled: async ({ messages, failure, usage }) => {
620
+ onSettled: async ({ messages, failure, usage, report }) => {
506
621
  if (messages !== undefined)
507
622
  record.messages = messages;
508
623
  record.turns += 1;
@@ -510,6 +625,7 @@ class HarnessSessionManager {
510
625
  record.usage.outputTokens += usage.outputTokens ?? 0;
511
626
  record.usage.totalTokens += usage.totalTokens ?? 0;
512
627
  await settle(failure);
628
+ await this.reportTurn(session, { startedAt, failure, ...report });
513
629
  },
514
630
  });
515
631
  // A turn cut short by a close or a shutdown is over once they wrote the session.
@@ -583,7 +699,7 @@ class HarnessSessionManager {
583
699
  // Closed, suspended or shut down meanwhile: whoever did it writes the session.
584
700
  if (this.live.get(record.id) !== session || record.status !== 'busy')
585
701
  return;
586
- const unfinished = handle.hasUnfinishedTurn();
702
+ const unfinished = session.lostTurn === true || handle.hasUnfinishedTurn();
587
703
  if (unfinished)
588
704
  record.pendingInput = session.paused ?? pendingInputOf(record.messages);
589
705
  else
@@ -594,10 +710,58 @@ class HarnessSessionManager {
594
710
  });
595
711
  this.scheduleIdle(session);
596
712
  }
713
+ /**
714
+ * Starts the harness session of a turn cut short again, in the same sandbox, from where it
715
+ * stopped. The runtime goes on with the turn for a while after it is told to stop, and a harness
716
+ * session sends what it still says to the next turn: a stray `finish` ends that turn before it
717
+ * starts, and its answer is lost. A session that cannot start again is `suspended`, resumed by
718
+ * the next message, or `interrupted` when its harness session could not be stopped.
719
+ */
720
+ async renew(session, handle) {
721
+ const { id, status } = session.record;
722
+ // Closed, suspended or shut down meanwhile: whoever did it stops the harness session.
723
+ if (this.live.get(id) !== session || status !== 'busy')
724
+ return;
725
+ const { agent, lease } = session;
726
+ // Started again without the turn it paused in: `continue()` tells the agent the answers.
727
+ session.lostTurn = handle.hasUnfinishedTurn();
728
+ session.handle = undefined;
729
+ const stopped = await handle.stop().catch((error) => {
730
+ this.report('stop', id)(error);
731
+ return undefined;
732
+ });
733
+ if (stopped === undefined || agent === undefined || lease === undefined) {
734
+ if (this.live.get(id) === session)
735
+ this.live.delete(id);
736
+ await this.release(session);
737
+ await this.transition(session, 'interrupted', { error: NOT_STOPPED });
738
+ return;
739
+ }
740
+ const resumeFrom = withoutUnfinishedTurn(stopped);
741
+ try {
742
+ session.handle = await agent.createSession({
743
+ sessionId: id,
744
+ sandboxSession: lease.sandboxSession,
745
+ resumeFrom,
746
+ });
747
+ }
748
+ catch (error) {
749
+ this.report('resume', id)(error);
750
+ if (this.live.get(id) === session)
751
+ this.live.delete(id);
752
+ await this.release(session);
753
+ await this.transition(session, 'suspended', {
754
+ error: messageOf(error),
755
+ resumeState: resumeFrom,
756
+ });
757
+ }
758
+ }
597
759
  async closeKept(kept) {
598
760
  if (kept.status === 'closed')
599
761
  return summaryOf(kept);
600
- if (kept.status === 'suspended') {
762
+ // A suspended session keeps its sandbox for its resume, an interrupted one because nothing freed it: the
763
+ // process stopped while the session held it, or its harness session could not be stopped.
764
+ if (kept.status === 'suspended' || kept.status === 'interrupted') {
601
765
  await this.options.sandboxes.discard?.(kept.id).catch(this.report('discard', kept.id));
602
766
  }
603
767
  const session = liveSession(kept);
@@ -611,9 +775,13 @@ class HarnessSessionManager {
611
775
  for (const { id, status, updatedAt } of await this.store.list()) {
612
776
  if (status !== 'suspended' || updatedAt >= cutoff || this.live.has(id))
613
777
  continue;
614
- const kept = await this.store.get(id);
615
- if (kept?.status === 'suspended' && !this.live.has(id))
778
+ const held = await this.hold(id).catch(() => undefined);
779
+ if (held === undefined)
780
+ continue;
781
+ const { record: kept, release } = held;
782
+ if (kept.status === 'suspended' && !this.live.has(id))
616
783
  await this.closeKept(kept);
784
+ await release();
617
785
  }
618
786
  }
619
787
  /** Closes a live session, cutting its turn short. */
@@ -5,8 +5,8 @@ export type SandboxView = HarnessV1NetworkSandboxSession & {
5
5
  release(): PromiseLike<void>;
6
6
  };
7
7
  /**
8
- * A sandbox that gives views of itself with ports of their own: `ai-sdk-sandbox-sbx` and
9
- * `ai-sdk-sandbox-cloud-run` sessions are.
8
+ * A sandbox that gives views of itself with ports of their own: `ai-sdk-sandbox-sbx`,
9
+ * `ai-sdk-sandbox-cloud-run` and `ai-sdk-sandbox-runtime` sessions are.
10
10
  */
11
11
  export type ForkableSandbox = HarnessV1NetworkSandboxSession & {
12
12
  fork(options: {
@@ -1,5 +1,6 @@
1
1
  import type { LanguageModelUsage, UIMessage, UIMessageChunk } from 'ai';
2
2
  import type { SessionStreamResult } from '../definitions/agent.js';
3
+ import type { TurnStepReport } from '../definitions/session.js';
3
4
  /** The tokens a turn used, added up over its streams. */
4
5
  export type TurnUsage = Pick<LanguageModelUsage, 'inputTokens' | 'outputTokens' | 'totalTokens'>;
5
6
  /** How a turn ended. */
@@ -9,6 +10,12 @@ export interface TurnOutcome {
9
10
  /** What went wrong, when the turn failed or was cut short. */
10
11
  failure: string | undefined;
11
12
  usage: TurnUsage;
13
+ /** The turn's tokens in detail, its steps and the raw events of its harness, for `onTurn`. */
14
+ report: {
15
+ usage: LanguageModelUsage;
16
+ steps: TurnStepReport[];
17
+ rawEvents: unknown[];
18
+ };
12
19
  }
13
20
  /**
14
21
  * What a turn does once one of its streams ends: answers given in between, sent to the client and
@@ -40,6 +40,68 @@ async function readSegment(result, originalMessages, { first, send, options, })
40
40
  }
41
41
  return outcome;
42
42
  }
43
+ /** Tokens of nothing yet: what the turn's tokens are added to. */
44
+ function emptyUsage() {
45
+ return {
46
+ inputTokens: 0,
47
+ inputTokenDetails: { noCacheTokens: 0, cacheReadTokens: 0, cacheWriteTokens: 0 },
48
+ outputTokens: 0,
49
+ outputTokenDetails: { textTokens: 0, reasoningTokens: 0 },
50
+ totalTokens: 0,
51
+ };
52
+ }
53
+ const add = (a, b) => a === undefined && b === undefined ? undefined : (a ?? 0) + (b ?? 0);
54
+ /** `total` plus `spent`, detail by detail: a count neither of them knows stays unknown. */
55
+ function addDetails(total, spent) {
56
+ if (spent === undefined)
57
+ return total;
58
+ return {
59
+ inputTokens: add(total.inputTokens, spent.inputTokens),
60
+ inputTokenDetails: {
61
+ noCacheTokens: add(total.inputTokenDetails.noCacheTokens, spent.inputTokenDetails?.noCacheTokens),
62
+ cacheReadTokens: add(total.inputTokenDetails.cacheReadTokens, spent.inputTokenDetails?.cacheReadTokens),
63
+ cacheWriteTokens: add(total.inputTokenDetails.cacheWriteTokens, spent.inputTokenDetails?.cacheWriteTokens),
64
+ },
65
+ outputTokens: add(total.outputTokens, spent.outputTokens),
66
+ outputTokenDetails: {
67
+ textTokens: add(total.outputTokenDetails.textTokens, spent.outputTokenDetails?.textTokens),
68
+ reasoningTokens: add(total.outputTokenDetails.reasoningTokens, spent.outputTokenDetails?.reasoningTokens),
69
+ },
70
+ totalTokens: add(total.totalTokens, spent.totalTokens),
71
+ };
72
+ }
73
+ /** The steps of a stream of the turn: their model, tokens and tool calls. */
74
+ async function stepsOf(result) {
75
+ const steps = await Promise.resolve(result.steps).catch(() => undefined);
76
+ return (steps ?? []).map((step) => ({
77
+ modelId: step.response?.modelId,
78
+ usage: step.usage,
79
+ toolCalls: (step.toolCalls ?? []).map(({ toolCallId, toolName, input }) => ({
80
+ toolCallId,
81
+ toolName,
82
+ input: input,
83
+ })),
84
+ }));
85
+ }
86
+ /**
87
+ * The raw events a stream of the turn carries, read from its own copy of the stream, beside the
88
+ * client's: what the harness sends that no other part says.
89
+ */
90
+ async function rawEventsOf(result) {
91
+ const events = [];
92
+ if (result.fullStream === undefined)
93
+ return events;
94
+ try {
95
+ for await (const part of result.fullStream) {
96
+ if (part.type === 'raw')
97
+ events.push(part.rawValue);
98
+ }
99
+ }
100
+ catch {
101
+ // The turn's failure is told by the client's stream: the events read so far are kept.
102
+ }
103
+ return events;
104
+ }
43
105
  /** Adds the tokens a stream of the turn used, once it is over. */
44
106
  async function addUsage(usage, result) {
45
107
  const spent = await Promise.resolve(result.totalUsage).catch(() => undefined);
@@ -52,6 +114,9 @@ class TurnRun {
52
114
  options;
53
115
  send;
54
116
  usage = { inputTokens: 0, outputTokens: 0, totalTokens: 0 };
117
+ details = emptyUsage();
118
+ steps = [];
119
+ rawEvents = [];
55
120
  messages;
56
121
  failure;
57
122
  finish;
@@ -63,12 +128,19 @@ class TurnRun {
63
128
  let result = first;
64
129
  let originalMessages = this.options.originalMessages;
65
130
  for (let index = 0; result !== undefined; index += 1) {
131
+ const raw = rawEventsOf(result);
66
132
  const outcome = await readSegment(result, originalMessages, {
67
133
  first: index === 0,
68
134
  send: this.send,
69
135
  options: this.options,
70
136
  });
71
137
  await addUsage(this.usage, result);
138
+ const spent = await Promise.resolve(result.totalUsage).catch(() => undefined);
139
+ this.details = addDetails(this.details, spent);
140
+ const steps = await stepsOf(result);
141
+ const events = await raw;
142
+ this.steps.push(...steps);
143
+ this.rawEvents.push(...events);
72
144
  this.failure ??= outcome.failure;
73
145
  this.finish = outcome.finish;
74
146
  this.messages = outcome.messages ?? this.messages;
@@ -79,7 +151,12 @@ class TurnRun {
79
151
  }
80
152
  if (this.finish !== undefined)
81
153
  this.send(this.finish);
82
- return { messages: this.messages, failure: this.failure, usage: this.usage };
154
+ return {
155
+ messages: this.messages,
156
+ failure: this.failure,
157
+ usage: this.usage,
158
+ report: { usage: this.details, steps: this.steps, rawEvents: this.rawEvents },
159
+ };
83
160
  }
84
161
  /**
85
162
  * What follows a stream that ended well, its approval requests sent first, with the answers
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "ai-sdk-harness-sessions",
3
- "version": "0.2.0",
3
+ "version": "0.3.0",
4
4
  "description": "Sessions for AI SDK harness agents: the lifecycle of conversations with Claude Code, Codex and friends — one turn at a time, suspended when idle, resumed on the next message and after a restart, several of them sharing one sandbox.",
5
5
  "keywords": [
6
6
  "ai",
@@ -47,20 +47,20 @@
47
47
  "ai": "^7.0.0"
48
48
  },
49
49
  "devDependencies": {
50
- "@ai-sdk/harness": "^1.0.138",
51
- "@ai-sdk/harness-claude-code": "^1.0.142",
52
- "@ai-sdk/harness-codex": "^1.0.140",
50
+ "@ai-sdk/harness": "^1.0.147",
51
+ "@ai-sdk/harness-claude-code": "^1.0.151",
52
+ "@ai-sdk/harness-codex": "^1.0.149",
53
53
  "@types/node": "^24.13.5",
54
54
  "@vitest/coverage-v8": "^5.0.3",
55
- "ai": "^7.0.127",
55
+ "ai": "^7.0.136",
56
56
  "typescript": "^6.0.3",
57
57
  "vitest": "^5.0.3",
58
58
  "@repo/eslint-config": "0.0.0",
59
59
  "@repo/prettier-config": "0.0.0",
60
+ "@repo/build": "0.0.0",
60
61
  "@repo/typescript-config": "0.0.0",
61
62
  "@repo/vitest-config": "0.0.0",
62
- "ai-sdk-sandbox-sbx": "0.1.1",
63
- "@repo/build": "0.0.0"
63
+ "ai-sdk-sandbox-sbx": "0.1.3"
64
64
  },
65
65
  "author": {
66
66
  "name": "Fabien Pasquet",