@agentshouse/kit 0.1.0-alpha.30 → 0.1.0-alpha.32

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -12,7 +12,7 @@ The Skill set Kit installs is Matt Pocock's
12
12
  On Linux `amd64` or `arm64`, run the published, versioned bootstrap once:
13
13
 
14
14
  ```sh
15
- bash <(curl -fsSL https://github.com/agentshouse/kit/releases/download/v0.1.0-alpha.30/connect-linux.sh)
15
+ bash <(curl -fsSL https://github.com/agentshouse/kit/releases/download/v0.1.0-alpha.32/connect-linux.sh)
16
16
  ```
17
17
 
18
18
  The bootstrap first establishes Docker Engine from what it observes on the
@@ -584,13 +584,14 @@ Google Sources. After that it acquires only what House names. A
584
584
  subject alone. `runs` takes a resumption or else a waiting Run, readying the
585
585
  agent CLIs, a requested setup and conversation files first. `commands` hands
586
586
  the supervised Session its stops, interrupts and answers and takes a
587
- resumption the same way, without acquiring a Run. An `entries` frame, a change
588
- in a Room the lease admits, delivers the working copies. A pass that took a Run
589
- acquires again once that Run ends, and a pass that found nothing waits for the
590
- next signal. A Run outcome House did not take is reported again every 30
591
- seconds until House answers. The Kit has no idle acquisition cycle, no command
592
- poll and no Session heartbeat: a Session is observed when its process starts
593
- and when its supervisor sees it stop.
587
+ resumption the same way, without acquiring a Run. An `entries` frame carries
588
+ the Change entries of a Room the lease admits, which the resident applies to
589
+ that Room's [working copy](#working-copies) in the order the frames arrive. A
590
+ pass that took a Run acquires again once that Run ends, and a pass that found
591
+ nothing waits for the next signal. A Run outcome House neither took nor finally
592
+ refused is reported again every 30 seconds. The Kit has no idle acquisition
593
+ cycle, no command poll and no Session heartbeat: a Session is observed when its
594
+ process starts and when its supervisor sees it stop.
594
595
 
595
596
  A socket that closes or never opens reconnects after a delay drawn between half
596
597
  and all of one second, doubled for each consecutive failure up to 30 seconds.
@@ -968,7 +969,7 @@ Within the same live Kit and supervised WorkerSession, a lost response or
968
969
  infrastructure failure resends that request with the same identifier. A local
969
970
  socket reconnect delivers the retained outcome to the process.
970
971
  A second deliberate call gets a new identifier; read-only calls carry none.
971
- House refusals pass through to the CLI without automatic resend. [R193](https://github.com/agentshouse/core/blob/main/docs/concept/implementation/08-routine-requests-human-notifications-and-workers.md#r193)
972
+ House refusals pass through to the CLI without automatic resend. [R193](https://github.com/agentshouse/core/blob/main/docs/concept/implementation/14-agents-profiles-sessions-and-runs.md#r193)
972
973
  owns recovery authority, lifetime, and refusal semantics.
973
974
 
974
975
  The launcher speaks only what the adapter contract fixes for this build. Every
@@ -1070,7 +1071,7 @@ Between turns, Kit retains the opaque provider session identifier and the Run's
1070
1071
  Worker, working directory, and route references in an owner-only local record.
1071
1072
  The record carries no transcript or Session credential and grants no authority:
1072
1073
  House must authorize each successor WorkerSession. A reported terminal state and
1073
- owner stops remove the record. [R199](https://github.com/agentshouse/core/blob/main/docs/concept/implementation/08-routine-requests-human-notifications-and-workers.md#r199)
1074
+ owner stops remove the record. [R199](https://github.com/agentshouse/core/blob/main/docs/concept/implementation/14-agents-profiles-sessions-and-runs.md#r199)
1074
1075
  owns conversation continuity.
1075
1076
 
1076
1077
  Launcher exit is the terminal process observation. The Kit closes the local
@@ -1097,7 +1098,11 @@ Provider continuation remains available reports no terminal state at all and
1097
1098
  leaves the Run active for an owner-originated resumption.
1098
1099
 
1099
1100
  The Kit decides that terminal state once, records it beside its stable operation
1100
- identity before it reports it, and reports it from that record. A Run with a
1101
+ identity before it reports it, and reports it from that record. It decides and
1102
+ records it as soon as the stop is recorded and the final text sealed, before it
1103
+ reports the stopped Session and its continuation facts: a refusal of either of
1104
+ those does not change the terminal state, so the Kit reports that recorded state
1105
+ whatever House answered to them. A Run with a
1101
1106
  supervised session records it in that session's durable local binding; a Run
1102
1107
  whose terminal state was decided without one — a resume command this Kit rejects
1103
1108
  because it retains no Provider continuation for the Run, or an acquired Run it
@@ -1109,11 +1114,23 @@ House answers the committed settlement verbatim and the Agent is never executed
1109
1114
  again. A Run House has already settled answers `worker_run_settled`, which the
1110
1115
  Kit accepts as settled and drops the record for.
1111
1116
 
1112
- Every other refusal leaves that record exactly as it is. The Run stays bound to
1113
- this installation, `kit worker run` names the refusal House answered with and
1114
- exits non-zero, and the same decided body is reported again before anything is
1115
- acquired. Nothing is dropped locally that House did not accept, and a refused
1116
- report is never presented to the owner as a delivered one.
1117
+ The Kit keeps that record until House answers it. The record stays exactly as it
1118
+ is while House is unavailable, never answers or answers in a form the Kit cannot
1119
+ read, declares its refusal retryable, answers 429, 502 or 503 without declaring
1120
+ otherwise, refuses the Kit contract or the Kit's lease, including a lease
1121
+ renewal the report waited on, refuses the operation identity as issued ahead of
1122
+ its clock, or answers `conversation_file_send_pending` because a conversation
1123
+ file the Run sent is still being delivered. A report whose operation identity
1124
+ House no longer holds, answered `operation_expired` or `position_expired`, keeps
1125
+ the same body under a new operation identity. Either way the Run stays bound to
1126
+ this installation, `kit worker run` names the refusal House answered with, if
1127
+ any, and exits non-zero, and the same decided body is reported again before
1128
+ anything is acquired. Every other refusal is House's final answer for that
1129
+ report, such as `worker_run_not_found` or `worker_run_not_acquired`: the Kit
1130
+ writes `kit: run <run> outcome refused: <code>` to standard error once, drops
1131
+ the record, and acquires as if the report had been taken. For the Run it
1132
+ supervised in that pass, `kit worker run` also names that refusal and exits
1133
+ non-zero. A refused report is never presented to the owner as a delivered one.
1117
1134
 
1118
1135
  While a WorkerSession is supervised the Kit keeps one durable local binding
1119
1136
  naming this installation, the WorkerRun, the WorkerSession, the exact tmux
@@ -1131,21 +1148,21 @@ instead of treating the session as lost.
1131
1148
 
1132
1149
  A restarted `kit worker run` reconciles the records it kept before it acquires
1133
1150
  anything new. It reports every retained settlement note first, because those
1134
- Runs have no session left to reconcile, and drops each note House accepts or
1135
- already settled. It next consumes the durable commands House holds for this
1136
- Environment, so a requested owner stop ends the exact tmux session before any
1137
- local channel reopens. It then considers only sessions of its own namespace
1138
- whose binding is complete, and never lists unknown sessions, scans host
1139
- processes, searches another namespace, or adopts an arbitrary process. A session
1140
- that is already gone is confirmed gone, reported as one stopped Session with no
1141
- Provider continuation, and settled as a terminal execution failure. A session
1142
- that is still running is restored only after House
1143
- positively replaces the Session authority for the same Run and Session; the Kit
1144
- then reconnects the typed adapter channel, refreshes the started observation,
1145
- and supervises that session to settlement instead of acquiring another Run. An
1146
- explicit House refusal destroys the stale tmux session and exposes no channel.
1147
- House that cannot be reached leaves the binding and the session exactly as they
1148
- are, invents no stopped, active, or authorized state, and acquires nothing.
1151
+ Runs have no session left to reconcile, and drops each note House accepts,
1152
+ already settled or finally refused. It next consumes the durable commands House
1153
+ holds for this Environment, so a requested owner stop ends the exact tmux
1154
+ session before any local channel reopens. It then considers only sessions of its
1155
+ own namespace whose binding is complete, and never lists unknown sessions, scans
1156
+ host processes, searches another namespace, or adopts an arbitrary process. A
1157
+ session that is already gone is confirmed gone, reported as one stopped Session
1158
+ with no Provider continuation, and settled as a terminal execution failure. A
1159
+ session that is still running is restored only after House positively replaces
1160
+ the Session authority for the same Run and Session; the Kit then reconnects the
1161
+ typed adapter channel, refreshes the started observation, and supervises that
1162
+ session to settlement instead of acquiring another Run. An explicit House
1163
+ refusal destroys the stale tmux session and exposes no channel. House that
1164
+ cannot be reached leaves the binding and the session exactly as they are,
1165
+ invents no stopped, active, or authorized state, and acquires nothing.
1149
1166
 
1150
1167
  A restore of House's database starts a new log epoch. Until this Environment
1151
1168
  has reported in it, House refuses every act that touches a Run, acquisition and
@@ -1373,9 +1390,17 @@ state. The House revision of every file content Kit received, keyed by path,
1373
1390
  Git blob and the House state that received it, stays in Kit home under
1374
1391
  `state/working-copies/`, outside the repository. A local commit is a local tree, never a House revision.
1375
1392
 
1376
- The resident Kit keeps every active copy current. Whenever its control stream
1377
- carries a Room change, and whenever it reads the saved choice, one
1378
- `POST /kit/door/discover` tells it where each
1393
+ The resident Kit keeps every active copy current. An `entries` frame on its
1394
+ control stream names a copied Room, the Room's position and log epoch, and the
1395
+ Change entries that brought it there: each changed path with its revision and
1396
+ text, or its removal. The resident applies the entries under the copy's own
1397
+ homes directly, without reading House, when the frame follows on from the copy:
1398
+ the same socket already carried an earlier frame for that Room, the copy's
1399
+ recorded position has reached that earlier frame's position, the log epoch is
1400
+ the copy's, and every changed path carries its revision and text. A frame the
1401
+ copy already holds changes nothing. Any other frame, and every reading of the
1402
+ saved choice, reads House instead, with the frames that arrive during one
1403
+ reading served by the next: one `POST /kit/door/discover` tells it where each
1379
1404
  copied Room stands, and for a Room that moved on it reads the changes after the
1380
1405
  copy's recorded position through `POST /kit/door/enumerate`. A changed, new or
1381
1406
  removed authored file becomes a commit on `refs/house/received`, its House
@@ -1548,7 +1573,12 @@ House alone validates the resulting Library: a batch it refuses answers with
1548
1573
  House's own file path, field, rule, and version. A refusal is reported with the
1549
1574
  current material House returned and the local directory is left for the caller.
1550
1575
  An acceptance updates the recorded position, its log epoch, and the affected
1551
- files from the returned final content.
1576
+ files from the returned final content. It also reads House's own provenance
1577
+ for those files by enumerating from the position the batch started at, and
1578
+ writes each served sidecar whose revision is the accepted one, so a submit,
1579
+ answered or recovered after a lost answer, leaves each file's `_provenance`
1580
+ sidecar at the revision House recorded. Kit never composes that metadata
1581
+ itself.
1552
1582
 
1553
1583
  Pending submissions are written before transfer. A lost response, reconnect, or
1554
1584
  lease renewal recovers that exact request under the same identifier. A domain
@@ -1561,6 +1591,10 @@ the Kit drops it like any other refusal, `rebootstrap` rebuilds the directory,
1561
1591
  and a change still wanted is a new deliberate submit against the rebuilt bases.
1562
1592
  Once House has answered any request for a replica `position_expired`, the
1563
1593
  replica refuses `submit` as `position_expired` until `rebootstrap` rebuilds it.
1594
+ A pending submission House answers `operation_expired` is uncertain, so the
1595
+ Kit first pulls confirmed state. If House already holds the pending change,
1596
+ the Kit drops the pending copy and sends nothing more; only if it does not
1597
+ does the Kit send that change again under a new identifier.
1564
1598
  `rebootstrap` drops a submission still pending against the directory it
1565
1599
  replaces.
1566
1600
 
@@ -2099,6 +2133,19 @@ the previous control lease before starting the next isolated command. Evidence
2099
2133
  is written to `tmp/delivery-evidence/migrator-pair.json` with producer revisions
2100
2134
  and the installed package digest.
2101
2135
 
2136
+ Explicit sync has its own paired proof:
2137
+
2138
+ ```sh
2139
+ HOUSE_CORE_ROOT=<core checkout> HOUSE_AUTH_ROOT=<auth checkout> \
2140
+ KIT_PAIR_SPECS=/kit/test/pair/sync.vitest.ts bash test/pair/run.sh
2141
+ ```
2142
+
2143
+ It proves that an accepted `submit`, answered or recovered after a lost answer,
2144
+ leaves each file's `_provenance` sidecar equal to the provenance House serves,
2145
+ and that a pending submission House answers `operation_expired` is pulled first,
2146
+ then sent again under a new identifier only if it never landed. Evidence is
2147
+ written to `tmp/delivery-evidence/sync-pair.json`.
2148
+
2102
2149
  The retained-original consumers have their own paired proofs:
2103
2150
 
2104
2151
  ```sh
package/bin/kit.js CHANGED
@@ -478,8 +478,10 @@ async function runWorkerRun(args, directory) {
478
478
  `Run ${run.run} stays bound to this installation until House accepts that report` +
479
479
  `${run.refused === null ? "" : ` (${run.refused.code})`}.`,
480
480
  );
481
+ } else if (run.refused !== null) {
482
+ write(`House refused that report (${run.refused.code}); this installation no longer holds Run ${run.run}.`);
481
483
  }
482
- if (!run.settled || reported?.outcome === "failed") process.exitCode = 1;
484
+ if (!run.settled || run.refused !== null || reported?.outcome === "failed") process.exitCode = 1;
483
485
  }
484
486
 
485
487
  function syncCancellation() {
package/bootstrap.json CHANGED
@@ -1,12 +1,12 @@
1
1
  {
2
2
  "schemaVersion": 1,
3
- "version": "0.1.0-alpha.30",
3
+ "version": "0.1.0-alpha.32",
4
4
  "hosts": {
5
5
  "linux": {
6
- "command": "bash <(curl -fsSL https://github.com/agentshouse/kit/releases/download/v0.1.0-alpha.30/connect-linux.sh)",
6
+ "command": "bash <(curl -fsSL https://github.com/agentshouse/kit/releases/download/v0.1.0-alpha.32/connect-linux.sh)",
7
7
  "asset": {
8
8
  "name": "connect-linux.sh",
9
- "url": "https://github.com/agentshouse/kit/releases/download/v0.1.0-alpha.30/connect-linux.sh"
9
+ "url": "https://github.com/agentshouse/kit/releases/download/v0.1.0-alpha.32/connect-linux.sh"
10
10
  },
11
11
  "workspace": {
12
12
  "argument": "--workspace",
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@agentshouse/kit",
3
- "version": "0.1.0-alpha.30",
3
+ "version": "0.1.0-alpha.32",
4
4
  "license": "MIT",
5
5
  "repository": {
6
6
  "type": "git",
@@ -40,6 +40,7 @@ async function held(session, signal, heard, dependencies) {
40
40
  perMessageDeflate: false,
41
41
  });
42
42
  let finished = false;
43
+ const carried = new Map();
43
44
  const finish = (reason = null, refusal = null) => {
44
45
  if (finished) return;
45
46
  finished = true;
@@ -63,7 +64,8 @@ async function held(session, signal, heard, dependencies) {
63
64
  if (frame?.type === "work_available" && WORK_SUBJECTS.includes(frame.subject)) {
64
65
  heard.available(frame.subject);
65
66
  } else if (frame?.type === "entries") {
66
- heard.changed();
67
+ heard.changed(frame, carried.get(frame.authority) ?? null);
68
+ carried.set(frame.authority, frame.position);
67
69
  } else if (VIEW_FRAMES.includes(frame?.type)) {
68
70
  heard.viewed?.(frame);
69
71
  }
package/src/sync.js CHANGED
@@ -13,6 +13,7 @@ import {
13
13
  assertWritableChanges,
14
14
  authorityFor,
15
15
  deriveChanges,
16
+ housePath,
16
17
  OPERATIONS,
17
18
  relativeTo,
18
19
  resultingFiles,
@@ -248,6 +249,12 @@ function enumeratedRecord(path, content, change) {
248
249
  };
249
250
  }
250
251
 
252
+ async function writeSidecar(options, authority, recorded, path, change, signal) {
253
+ const sidecar = sidecarPath(path);
254
+ await writeReplicaFile(locate(options.replica, authority.path, sidecar), change.provenance, signal);
255
+ recorded.set(sidecar, enumeratedRecord(sidecar, change.provenance, change));
256
+ }
257
+
251
258
  async function applyEnumeration(options, state, enumeration, signal) {
252
259
  if (!Array.isArray(enumeration?.authorities)) throw new KitRefusal("kit_sync_rejected", "enumerate");
253
260
  for (const view of enumeration.authorities) {
@@ -271,13 +278,7 @@ async function applyEnumeration(options, state, enumeration, signal) {
271
278
  recorded.set(change.path, enumeratedRecord(change.path, change.content, change));
272
279
  }
273
280
  if (typeof change.provenance === "string") {
274
- const sidecar = sidecarPath(change.path);
275
- await writeReplicaFile(
276
- locate(options.replica, authority.path, sidecar),
277
- change.provenance,
278
- signal,
279
- );
280
- recorded.set(sidecar, enumeratedRecord(sidecar, change.provenance, change));
281
+ await writeSidecar(options, authority, recorded, change.path, change, signal);
281
282
  }
282
283
  }
283
284
  authority.files = [...recorded.values()];
@@ -309,12 +310,40 @@ function uncertainTransport(error) {
309
310
  );
310
311
  }
311
312
 
312
- async function acceptBatch(options, state, outcome, signal, pending = null) {
313
+ async function servedProvenance(session, options, state, authority, signal) {
314
+ const enumeration = await replicaDoor(
315
+ session,
316
+ options,
317
+ state,
318
+ KIT_DOOR_ENUMERATE_PATH,
319
+ {
320
+ prefixes: state.prefixes.filter((prefix) => authorityFor(state, prefix) === authority),
321
+ positions: [
322
+ { authority: authority.authority, position: authority.position, log_epoch: authority.log_epoch },
323
+ ],
324
+ },
325
+ signal,
326
+ );
327
+ const view = Array.isArray(enumeration?.authorities)
328
+ ? enumeration.authorities.find((entry) => entry?.authority === authority.authority)
329
+ : undefined;
330
+ if (!Array.isArray(view?.changes)) throw new KitRefusal("kit_sync_rejected", "enumerate");
331
+ return new Map(
332
+ view.changes
333
+ .filter((change) => typeof change?.provenance === "string")
334
+ .map((change) => [change.path, change]),
335
+ );
336
+ }
337
+
338
+ async function acceptBatch(session, options, state, outcome, signal, pending = null) {
313
339
  const authority = state.authorities.find((entry) => entry.authority === outcome.authority);
314
340
  if (authority === undefined) throw new KitRefusal("kit_sync_rejected", outcome.authority);
315
341
  const reached = changePosition(outcome, "batch");
316
342
  const recorded = new Map(authority.files.map((file) => [file.path, file]));
317
343
  const predicted = predictedReplica(authority, pending);
344
+ const provenance = (outcome.contents ?? []).some((file) => typeof file?.content === "string")
345
+ ? await servedProvenance(session, options, state, authority, signal)
346
+ : new Map();
318
347
  for (const file of removalsFirst(outcome.contents ?? [], (file) => file?.content === null)) {
319
348
  const relative = relativeTo(authority.path, file.path);
320
349
  const destination = locate(options.replica, authority.path, relative);
@@ -340,6 +369,10 @@ async function acceptBatch(options, state, outcome, signal, pending = null) {
340
369
  sha256: "",
341
370
  content: accepted,
342
371
  });
372
+ const served = provenance.get(relative);
373
+ if (served !== undefined && served.revision === file.revision) {
374
+ await writeSidecar(options, authority, recorded, relative, served, signal);
375
+ }
343
376
  }
344
377
  authority.files = [...recorded.values()];
345
378
  Object.assign(authority, reached);
@@ -351,16 +384,65 @@ function sameBody(left, right) {
351
384
  return JSON.stringify(left) === JSON.stringify(right);
352
385
  }
353
386
 
387
+ function contentMap(authority) {
388
+ return Object.fromEntries(authority.files.map((file) => [file.path, file.content]));
389
+ }
390
+
391
+ async function pullConfirmed(session, options, state, signal) {
392
+ const enumeration = await replicaDoor(
393
+ session,
394
+ options,
395
+ state,
396
+ KIT_DOOR_ENUMERATE_PATH,
397
+ { prefixes: state.prefixes, positions: positionsOf(state) },
398
+ signal,
399
+ );
400
+ return applyEnumeration(options, state, enumeration, signal);
401
+ }
402
+
403
+ async function settleExpired(session, options, state, pending, signal) {
404
+ const authority = authorityFor(state, pending.body.changes[0].path);
405
+ const predicted = predictedReplica(authority, pending);
406
+ const before = contentMap(authority);
407
+ await pullConfirmed(session, options, state, signal);
408
+ const confirmed = contentMap(authority);
409
+ const touched = [...new Set([...Object.keys(before), ...Object.keys(predicted)])]
410
+ .filter((path) => before[path] !== predicted[path])
411
+ .sort();
412
+ const landed = touched.length > 0 && touched.every((path) => confirmed[path] === predicted[path]);
413
+ await clearPending(options.directory, state.directory);
414
+ if (!landed) {
415
+ return { pending, accepted: await sendBatch(session, options, state, pending.body.changes, signal) };
416
+ }
417
+ const recorded = new Map(authority.files.map((file) => [file.path, file]));
418
+ return {
419
+ pending,
420
+ accepted: {
421
+ authority: authority.authority,
422
+ position: authority.position,
423
+ log_epoch: authority.log_epoch,
424
+ contents: touched.map((path) => ({
425
+ path: housePath(authority.path, path),
426
+ content: recorded.get(path)?.content ?? null,
427
+ revision: recorded.get(path)?.revision ?? null,
428
+ })),
429
+ },
430
+ };
431
+ }
432
+
354
433
  async function recoverPending(session, options, state, signal) {
355
434
  const pending = await readPending(options.directory, state.directory);
356
435
  if (pending === null) return null;
357
436
  try {
358
437
  const outcome = await replicaDoor(session, options, state, pending.path, pending.body, signal);
359
- const accepted = await acceptBatch(options, state, outcome, signal, pending);
438
+ const accepted = await acceptBatch(session, options, state, outcome, signal, pending);
360
439
  await clearPending(options.directory, state.directory);
361
440
  return { pending, accepted };
362
441
  } catch (error) {
363
442
  if (uncertainTransport(error)) throw error;
443
+ if (error.code === "operation_expired") {
444
+ return settleExpired(session, options, state, pending, signal);
445
+ }
364
446
  await clearPending(options.directory, state.directory);
365
447
  throw error;
366
448
  }
@@ -396,7 +478,7 @@ async function sendBatch(session, options, state, changes, signal) {
396
478
  );
397
479
  try {
398
480
  const outcome = await replicaDoor(session, options, state, KIT_DOOR_BATCH_PATH, body, signal);
399
- last = await acceptBatch(options, state, outcome, signal);
481
+ last = await acceptBatch(session, options, state, outcome, signal);
400
482
  await clearPending(options.directory, state.directory);
401
483
  } catch (error) {
402
484
  if (uncertainTransport(error)) throw error;
@@ -441,16 +523,7 @@ export async function bootstrap(options, dependencies = {}) {
441
523
 
442
524
  export async function pull(options, dependencies = {}) {
443
525
  return withSession(options, dependencies, async (session, signal) => {
444
- const state = await loadState(options);
445
- const enumeration = await replicaDoor(
446
- session,
447
- options,
448
- state,
449
- KIT_DOOR_ENUMERATE_PATH,
450
- { prefixes: state.prefixes, positions: positionsOf(state) },
451
- signal,
452
- );
453
- const next = await applyEnumeration(options, state, enumeration, signal);
526
+ const next = await pullConfirmed(session, options, await loadState(options), signal);
454
527
  return { positions: positionsOf(next) };
455
528
  });
456
529
  }
package/src/tasks.js CHANGED
@@ -24,6 +24,23 @@ export function coalescedTask(signal, fail, work) {
24
24
  return { trigger, idle: () => task === null, settled: () => task };
25
25
  }
26
26
 
27
+ export function orderedTask(signal, fail, work) {
28
+ let task = null;
29
+ const push = (...args) => {
30
+ const next = (task ?? Promise.resolve())
31
+ .then(() => (signal.aborted ? undefined : work(...args)))
32
+ .catch((error) => {
33
+ if (signal.aborted) return;
34
+ fail(error);
35
+ });
36
+ task = next;
37
+ void next.then(() => {
38
+ if (task === next) task = null;
39
+ });
40
+ };
41
+ return { push, idle: () => task === null, settled: () => task };
42
+ }
43
+
27
44
  export function residentWake() {
28
45
  let pending = false;
29
46
  let release = null;
package/src/worker.js CHANGED
@@ -22,12 +22,13 @@ import { inheritedEnvironment } from "./command.js";
22
22
  import { sourceEntries } from "./google/entries.js";
23
23
  import { watchKitWork } from "./kit-work-stream.js";
24
24
  import { finalPlan } from "./plan.js";
25
- import { coalescedTask, residentWake } from "./tasks.js";
25
+ import { coalescedTask, orderedTask, residentWake } from "./tasks.js";
26
26
  import {
27
27
  applyWorkingCopies,
28
28
  deliverWorkingCopies,
29
29
  ownerCaller,
30
30
  pushWorkingCopy,
31
+ receiveWorkingCopyEntries,
31
32
  recordIndexChange,
32
33
  uploadedPaths,
33
34
  workingCopyPaths,
@@ -48,7 +49,7 @@ import {
48
49
  WORKING_COPY_PUSH_METHOD,
49
50
  WORKING_COPY_UPLOAD_COMMAND,
50
51
  } from "./contract.js";
51
- import { EpochReportRequired, formatCommandError, houseFailed, KitRefusal } from "./refusal.js";
52
+ import { EpochReportRequired, formatCommandError, houseAnswered, houseFailed, KitRefusal } from "./refusal.js";
52
53
  import {
53
54
  acceptAssistantText,
54
55
  acquireCommands,
@@ -227,7 +228,21 @@ async function completeOwnerStop(context, bound, requestedAt) {
227
228
  return { destroyed, confirmed: true, failure };
228
229
  }
229
230
 
230
- async function settleReported(context, bound) {
231
+ const REPORT_LATER = ["conversation_file_send_pending", "environment_control_lease_rejected", "operation_id_invalid"];
232
+
233
+ const REPORT_REISSUED = ["operation_expired", "position_expired"];
234
+
235
+ function reportLater(refusal) {
236
+ const retryable = refusal.payload?.error?.retryable;
237
+ return (
238
+ !houseAnswered(refusal) ||
239
+ retryable === true ||
240
+ REPORT_LATER.includes(refusal.code) ||
241
+ ([429, 502, 503].includes(refusal.status) && retryable === undefined)
242
+ );
243
+ }
244
+
245
+ async function settleReported(context, bound, retain) {
231
246
  try {
232
247
  await reportRunOutcome(context.session, bound.run, bound.reported, bound.plan);
233
248
  return { settled: true, failure: null };
@@ -235,14 +250,22 @@ async function settleReported(context, bound) {
235
250
  if (undecided(context, refusal)) return { settled: false, failure: null };
236
251
  if (!(refusal instanceof KitRefusal)) throw refusal;
237
252
  if (refusal.code === "worker_run_settled") return { settled: true, failure: null };
238
- return { settled: false, failure: { code: refusal.code, detail: refusal.message } };
253
+ const failure = { code: refusal.code, detail: refusal.message };
254
+ if (REPORT_REISSUED.includes(refusal.code)) {
255
+ bound.reported = { ...bound.reported, operation: operationId() };
256
+ await retain(bound);
257
+ return { settled: false, failure };
258
+ }
259
+ if (reportLater(refusal)) return { settled: false, failure };
260
+ reported(`run ${bound.run} outcome refused: ${refusal.code}`);
261
+ return { settled: true, failure };
239
262
  }
240
263
  }
241
264
 
242
265
  async function reportTerminalState(context, bound, report) {
243
266
  const settling = { ...bound, reported: { operation: operationId(), ...report } };
244
267
  await writeBinding(context.directory, settling);
245
- const settled = await settleReported(context, settling);
268
+ const settled = await settleReported(context, settling, (held) => writeBinding(context.directory, held));
246
269
  if (settled.settled) await removeBinding(context.directory, settling.tmux);
247
270
  return { ...settled, reported: settling.reported };
248
271
  }
@@ -482,22 +505,35 @@ function stoppedReport(bound) {
482
505
 
483
506
  async function settleStopped(context, bound) {
484
507
  const { stopped } = bound;
485
- const unresolved = (refused) => ({ failure: stopped.failure, refused, settled: false });
508
+ const unresolved = (refused) => ({ failure: stopped.failure, reported: bound.reported, refused, settled: false });
486
509
  if (bound.sealed !== null) {
487
510
  const refused = await dispatchSeal(context, bound);
488
511
  if (refused !== null) return unresolved(refused);
489
512
  }
490
- let failure = stopped.failure;
491
- try {
492
- await observeSessionStop(context.session, bound.session, stopped.textAccepted, bound.plan);
493
- } catch (refusal) {
494
- if (undecided(context, refusal)) return unresolved(null);
495
- if (!(refusal instanceof KitRefusal) || refusal.code !== "conversation_final_text_pending") throw refusal;
496
- failure ??= { code: refusal.code, detail: refusal.message };
497
- await observeSessionStop(context.session, bound.session, false, bound.plan);
498
- }
499
513
  const report = stoppedReport(bound);
514
+ if (report !== null && bound.reported === null) {
515
+ bound.reported = { operation: operationId(), ...report };
516
+ await writeBinding(context.directory, bound);
517
+ }
500
518
  if (report !== null) await removeContinuation(context.directory, bound.run);
519
+ let failure = stopped.failure;
520
+ const observe = async (textAccepted) => {
521
+ try {
522
+ await observeSessionStop(context.session, bound.session, textAccepted, bound.plan);
523
+ return null;
524
+ } catch (refusal) {
525
+ if (undecided(context, refusal)) return undefined;
526
+ if (!(refusal instanceof KitRefusal)) throw refusal;
527
+ return refusal;
528
+ }
529
+ };
530
+ let observed = await observe(stopped.textAccepted);
531
+ if (observed?.code === "conversation_final_text_pending") {
532
+ failure ??= { code: observed.code, detail: observed.message };
533
+ observed = await observe(false);
534
+ }
535
+ if (observed === undefined) return unresolved(null);
536
+ if (observed !== null && report === null) throw observed;
501
537
  try {
502
538
  await reportContinuation(context.session, bound.session, {
503
539
  providerContinuation: report === null,
@@ -505,17 +541,21 @@ async function settleStopped(context, bound) {
505
541
  });
506
542
  } catch (refusal) {
507
543
  if (undecided(context, refusal)) return unresolved(null);
508
- if (!(refusal instanceof KitRefusal) || refusal.code !== "worker_run_settled") throw refusal;
509
- await removeContinuation(context.directory, bound.run);
510
- await removeBinding(context.directory, bound.tmux);
511
- return { failure, refused: null, settled: true };
544
+ if (!(refusal instanceof KitRefusal)) throw refusal;
545
+ if (refusal.code === "worker_run_settled") {
546
+ await removeContinuation(context.directory, bound.run);
547
+ await removeBinding(context.directory, bound.tmux);
548
+ return { failure, refused: null, settled: true };
549
+ }
550
+ if (report === null) throw refusal;
512
551
  }
513
552
  if (report === null) {
514
553
  await removeBinding(context.directory, bound.tmux);
515
554
  return { failure, refused: null, settled: true };
516
555
  }
517
- const settled = await reportTerminalState(context, bound, report);
518
- return { failure, reported: settled.reported, refused: settled.failure, settled: settled.settled };
556
+ const settled = await settleReported(context, bound, (held) => writeBinding(context.directory, held));
557
+ if (settled.settled) await removeBinding(context.directory, bound.tmux);
558
+ return { failure, reported: bound.reported, refused: settled.failure, settled: settled.settled };
519
559
  }
520
560
 
521
561
  function assignmentRoute(context, assignment) {
@@ -601,7 +641,7 @@ async function settleRefused(context, command, failure) {
601
641
  reported: { operation: operationId(), ...failedReport(failure) },
602
642
  };
603
643
  await writeSettlement(context.directory, settling);
604
- const settled = await settleReported(context, settling);
644
+ const settled = await settleReported(context, settling, (held) => writeSettlement(context.directory, held));
605
645
  if (settled.settled) await removeSettlement(context.directory, settling.run);
606
646
  return { run: settling.run, session: settling.session, tmux: null, assistant: [],
607
647
  failure, reported: settling.reported, refused: settled.failure,
@@ -697,7 +737,7 @@ async function reconcile(context) {
697
737
  const stopped = [];
698
738
  const unresolved = [];
699
739
  for (const settlement of await readSettlements(context.directory)) {
700
- const settled = await settleReported(context, settlement);
740
+ const settled = await settleReported(context, settlement, (held) => writeSettlement(context.directory, held));
701
741
  if (settled.settled) await removeSettlement(context.directory, settlement.run);
702
742
  else unresolved.push(unresolvedRun(settlement, settled.failure));
703
743
  }
@@ -706,17 +746,17 @@ async function reconcile(context) {
706
746
  const commands = (await pendingCommands(context))?.stops ?? [];
707
747
  const carried = [];
708
748
  for (const binding of bindings) {
709
- if (binding.reported !== null) {
710
- const settled = await settleReported(context, binding);
711
- if (settled.settled) await removeBinding(context.directory, binding.tmux);
712
- else unresolved.push(unresolvedRun(binding, settled.failure));
713
- continue;
714
- }
715
749
  if (binding.stopped !== null) {
716
750
  const settled = await settleStopped(context, binding);
717
751
  if (!settled.settled) unresolved.push(unresolvedRun(binding, settled.refused));
718
752
  continue;
719
753
  }
754
+ if (binding.reported !== null) {
755
+ const settled = await settleReported(context, binding, (held) => writeBinding(context.directory, held));
756
+ if (settled.settled) await removeBinding(context.directory, binding.tmux);
757
+ else unresolved.push(unresolvedRun(binding, settled.failure));
758
+ continue;
759
+ }
720
760
  const command = commands.find((pending) => pending.run === binding.run) ?? null;
721
761
  const requestedAt = binding.ownerStop ?? command?.requestedAt ?? null;
722
762
  if (requestedAt === null) {
@@ -958,7 +998,7 @@ async function reportedWorkingCopies(act, context, signal) {
958
998
  return outcome;
959
999
  }
960
1000
 
961
- function residentUpdate(context, wake) {
1001
+ export function residentUpdate(context, wake) {
962
1002
  let requested = existsSync(updateDrain(context.directory));
963
1003
  let ready = null;
964
1004
  const waiting = [];
@@ -967,16 +1007,27 @@ function residentUpdate(context, wake) {
967
1007
  return requested;
968
1008
  },
969
1009
  async request(serving) {
1010
+ let heard = null;
1011
+ let hear;
1012
+ const answered = new Promise((resolve) => {
1013
+ hear = (answer) => {
1014
+ heard = answer;
1015
+ resolve(answer);
1016
+ };
1017
+ });
1018
+ waiting.push(hear);
970
1019
  if (!requested) {
971
1020
  requested = true;
972
1021
  await writeText(updateDrain(context.directory), "");
973
1022
  }
974
1023
  wake.notify();
975
1024
  const sessions = await heldSessions(context);
1025
+ if (heard !== null) return heard;
1026
+ if (ready === null && !serving && sessions.length === 0) return answered;
1027
+ waiting.splice(waiting.indexOf(hear), 1);
976
1028
  if (ready !== null) return ready;
977
1029
  if (serving) return { ready: false, sessions, serving };
978
- if (sessions.length > 0) return { ready: false, sessions };
979
- return new Promise((resolve) => waiting.push(resolve));
1030
+ return { ready: false, sessions };
980
1031
  },
981
1032
  answer(answer) {
982
1033
  for (const resolve of waiting.splice(0)) resolve(answer);
@@ -1094,12 +1145,12 @@ export async function runResident(options, dependencies = {}) {
1094
1145
  context.startLogins(needsLogin);
1095
1146
  });
1096
1147
  const unavailable = (error) => reported(`working copies unavailable: ${error?.message ?? error}`);
1097
- const changed = () => {
1148
+ const redeliver = () => {
1098
1149
  if (!update.requested) deliveries.trigger();
1099
1150
  };
1100
1151
  const behind = (outcome) => {
1101
1152
  if (!outcome.behind) return;
1102
- controlSleep(interval, signal).then(changed, () => {});
1153
+ controlSleep(interval, signal).then(redeliver, () => {});
1103
1154
  };
1104
1155
  const copies = coalescedTask(signal, unavailable, async () =>
1105
1156
  behind(await reportedWorkingCopies(applyWorkingCopies, context, signal)),
@@ -1107,6 +1158,12 @@ export async function runResident(options, dependencies = {}) {
1107
1158
  const deliveries = coalescedTask(signal, unavailable, async () =>
1108
1159
  behind(await reportedWorkingCopies(deliverWorkingCopies, context, signal)),
1109
1160
  );
1161
+ const entries = orderedTask(signal, unavailable, async (frame, since) => {
1162
+ if (!(await receiveWorkingCopyEntries(context, frame, since, signal))) redeliver();
1163
+ });
1164
+ const changed = (frame, since) => {
1165
+ if (!update.requested) entries.push(frame, since);
1166
+ };
1110
1167
  sources = sourceEntries({
1111
1168
  session: context.session,
1112
1169
  directory: context.directory,
@@ -1199,6 +1256,7 @@ export async function runResident(options, dependencies = {}) {
1199
1256
  agentRefresh.idle() &&
1200
1257
  copies.idle() &&
1201
1258
  deliveries.idle() &&
1259
+ entries.idle() &&
1202
1260
  sources.idle())
1203
1261
  ) {
1204
1262
  return;
@@ -1212,6 +1270,7 @@ export async function runResident(options, dependencies = {}) {
1212
1270
  await agentRefresh.settled();
1213
1271
  await copies.settled();
1214
1272
  await deliveries.settled();
1273
+ await entries.settled();
1215
1274
  await sources.settled();
1216
1275
  for (const running of logins.values()) running.controller.abort();
1217
1276
  await Promise.all([...logins.values()].map((running) => running.task));
@@ -561,6 +561,55 @@ export async function deliverWorkingCopies(context, signal) {
561
561
  : deliveredOutcome(delivered.refused, delivered.behind);
562
562
  }
563
563
 
564
+ function streamChange(entry) {
565
+ if (entry?.kind === "removed") return { path: entry.path, removed: true };
566
+ return { path: entry?.path, revision: entry?.revision, content: entry?.text, provenance: entry?.provenance };
567
+ }
568
+
569
+ function followsOn(copy, frame, since, changes) {
570
+ const { received } = copy;
571
+ return (
572
+ since !== null &&
573
+ received.log_epoch === frame.log_epoch &&
574
+ BigInt(received.position) >= BigInt(since) &&
575
+ !unreadableChanges(changes)
576
+ );
577
+ }
578
+
579
+ async function carriedEntries(context, path, frame, since, signal) {
580
+ const copy = await readCopy(path);
581
+ if (!copy?.active) return true;
582
+ const { received } = copy;
583
+ if (received.log_epoch === frame.log_epoch && BigInt(frame.position) <= BigInt(received.position)) return true;
584
+ const homes = copy.scope === "private" ? WORKING_COPY_PRIVATE_HOMES : WORKING_COPY_ROOM_HOMES;
585
+ const changes = frame.entries
586
+ .map(streamChange)
587
+ .filter((change) => typeof change.path !== "string" || homes.some((home) => covers(change.path, home)));
588
+ if (!followsOn(copy, frame, since, changes)) return false;
589
+ const view = { changes, position: frame.position, log_epoch: frame.log_epoch };
590
+ const authority = { operations: { write: received.declared.write }, protected_paths: received.declared.protected };
591
+ await writeCopy(context.directory, await incorporated(copy, view, authority, signal), signal);
592
+ return true;
593
+ }
594
+
595
+ export async function receiveWorkingCopyEntries(context, frame, since, signal) {
596
+ if (
597
+ typeof frame.authority !== "string" ||
598
+ typeof frame.log_epoch !== "string" ||
599
+ !/^[0-9]+$/.test(frame.position) ||
600
+ !Array.isArray(frame.entries) ||
601
+ (since !== null && !/^[0-9]+$/.test(since))
602
+ ) {
603
+ return false;
604
+ }
605
+ const copy = (await heldCopies(context.directory)).find(
606
+ (held) => held.active && held.room_ref === frame.authority,
607
+ );
608
+ if (copy === undefined) return true;
609
+ const path = copyPath(context.directory, copy);
610
+ return exclusively(path, () => carriedEntries(context, path, frame, since, signal));
611
+ }
612
+
564
613
  export async function applyWorkingCopies(context, signal) {
565
614
  const selected = (await savedSelection(context.session, signal)).filter(
566
615
  (scope) => scope.kind === "private" || scope.handle !== null,
@@ -913,6 +962,14 @@ export function uploadedPaths(changes) {
913
962
  return [...pushedPaths(changes)].sort();
914
963
  }
915
964
 
965
+ function unreadableChanges(changes) {
966
+ return changes.some(
967
+ (change) =>
968
+ typeof change?.path !== "string" ||
969
+ (change.removed !== true && (typeof change.content !== "string" || typeof change.revision !== "string")),
970
+ );
971
+ }
972
+
916
973
  async function enumerated(context, copy, since, signal) {
917
974
  const answer = await door(
918
975
  context.session,
@@ -925,11 +982,7 @@ async function enumerated(context, copy, since, signal) {
925
982
  !Array.isArray(view?.changes) ||
926
983
  typeof view.position !== "string" ||
927
984
  typeof view.log_epoch !== "string" ||
928
- view.changes.some(
929
- (change) =>
930
- typeof change?.path !== "string" ||
931
- (change.removed !== true && (typeof change.content !== "string" || typeof change.revision !== "string")),
932
- )
985
+ unreadableChanges(view.changes)
933
986
  ) {
934
987
  throw rejected("enumerate");
935
988
  }