@agentshouse/kit 0.1.0-alpha.31 → 0.1.0-alpha.32

package/README.md

@@ -12,7 +12,7 @@ The Skill set Kit installs is Matt Pocock's
 On Linux `amd64` or `arm64`, run the published, versioned bootstrap once:
 
 ```sh
-bash <(curl -fsSL https://github.com/agentshouse/kit/releases/download/v0.1.0-alpha.31/connect-linux.sh)
+bash <(curl -fsSL https://github.com/agentshouse/kit/releases/download/v0.1.0-alpha.32/connect-linux.sh)
 ```
 
 The bootstrap first establishes Docker Engine from what it observes on the
@@ -584,13 +584,14 @@ Google Sources. After that it acquires only what House names. A
 subject alone. `runs` takes a resumption or else a waiting Run, readying the
 agent CLIs, a requested setup and conversation files first. `commands` hands
 the supervised Session its stops, interrupts and answers and takes a
-resumption the same way, without acquiring a Run. An `entries` frame, a change
-in a Room the lease admits, delivers the working copies. A pass that took a Run
-acquires again once that Run ends, and a pass that found nothing waits for the
-next signal. A Run outcome House did not take is reported again every 30
-seconds until House answers. The Kit has no idle acquisition cycle, no command
-poll and no Session heartbeat: a Session is observed when its process starts
-and when its supervisor sees it stop.
+resumption the same way, without acquiring a Run. An `entries` frame carries
+the Change entries of a Room the lease admits, which the resident applies to
+that Room's [working copy](#working-copies) in the order the frames arrive. A
+pass that took a Run acquires again once that Run ends, and a pass that found
+nothing waits for the next signal. A Run outcome House neither took nor finally
+refused is reported again every 30 seconds. The Kit has no idle acquisition
+cycle, no command poll and no Session heartbeat: a Session is observed when its
+process starts and when its supervisor sees it stop.
 
 A socket that closes or never opens reconnects after a delay drawn between half
 and all of one second, doubled for each consecutive failure up to 30 seconds.
@@ -1097,7 +1098,11 @@ Provider continuation remains available reports no terminal state at all and
 leaves the Run active for an owner-originated resumption.
 
 The Kit decides that terminal state once, records it beside its stable operation
-identity before it reports it, and reports it from that record. A Run with a
+identity before it reports it, and reports it from that record. It decides and
+records it as soon as the stop is recorded and the final text sealed, before it
+reports the stopped Session and its continuation facts: a refusal of either of
+those does not change the terminal state, so the Kit reports that recorded state
+whatever House answered to them. A Run with a
 supervised session records it in that session's durable local binding; a Run
 whose terminal state was decided without one — a resume command this Kit rejects
 because it retains no Provider continuation for the Run, or an acquired Run it
@@ -1109,11 +1114,23 @@ House answers the committed settlement verbatim and the Agent is never executed
 again. A Run House has already settled answers `worker_run_settled`, which the
 Kit accepts as settled and drops the record for.
 
-Every other refusal leaves that record exactly as it is. The Run stays bound to
-this installation, `kit worker run` names the refusal House answered with and
-exits non-zero, and the same decided body is reported again before anything is
-acquired. Nothing is dropped locally that House did not accept, and a refused
-report is never presented to the owner as a delivered one.
+The Kit keeps that record until House answers it. The record stays exactly as it
+is while House is unavailable, never answers or answers in a form the Kit cannot
+read, declares its refusal retryable, answers 429, 502 or 503 without declaring
+otherwise, refuses the Kit contract or the Kit's lease, including a lease
+renewal the report waited on, refuses the operation identity as issued ahead of
+its clock, or answers `conversation_file_send_pending` because a conversation
+file the Run sent is still being delivered. A report whose operation identity
+House no longer holds, answered `operation_expired` or `position_expired`, keeps
+the same body under a new operation identity. Either way the Run stays bound to
+this installation, `kit worker run` names the refusal House answered with, if
+any, and exits non-zero, and the same decided body is reported again before
+anything is acquired. Every other refusal is House's final answer for that
+report, such as `worker_run_not_found` or `worker_run_not_acquired`: the Kit
+writes `kit: run <run> outcome refused: <code>` to standard error once, drops
+the record, and acquires as if the report had been taken. For the Run it
+supervised in that pass, `kit worker run` also names that refusal and exits
+non-zero. A refused report is never presented to the owner as a delivered one.
 
 While a WorkerSession is supervised the Kit keeps one durable local binding
 naming this installation, the WorkerRun, the WorkerSession, the exact tmux
@@ -1131,21 +1148,21 @@ instead of treating the session as lost.
 
 A restarted `kit worker run` reconciles the records it kept before it acquires
 anything new. It reports every retained settlement note first, because those
-Runs have no session left to reconcile, and drops each note House accepts or
-already settled. It next consumes the durable commands House holds for this
-Environment, so a requested owner stop ends the exact tmux session before any
-local channel reopens. It then considers only sessions of its own namespace
-whose binding is complete, and never lists unknown sessions, scans host
-processes, searches another namespace, or adopts an arbitrary process. A session
-that is already gone is confirmed gone, reported as one stopped Session with no
-Provider continuation, and settled as a terminal execution failure. A session
-that is still running is restored only after House
-positively replaces the Session authority for the same Run and Session; the Kit
-then reconnects the typed adapter channel, refreshes the started observation,
-and supervises that session to settlement instead of acquiring another Run. An
-explicit House refusal destroys the stale tmux session and exposes no channel.
-House that cannot be reached leaves the binding and the session exactly as they
-are, invents no stopped, active, or authorized state, and acquires nothing.
+Runs have no session left to reconcile, and drops each note House accepts,
+already settled or finally refused. It next consumes the durable commands House
+holds for this Environment, so a requested owner stop ends the exact tmux
+session before any local channel reopens. It then considers only sessions of its
+own namespace whose binding is complete, and never lists unknown sessions, scans
+host processes, searches another namespace, or adopts an arbitrary process. A
+session that is already gone is confirmed gone, reported as one stopped Session
+with no Provider continuation, and settled as a terminal execution failure. A
+session that is still running is restored only after House positively replaces
+the Session authority for the same Run and Session; the Kit then reconnects the
+typed adapter channel, refreshes the started observation, and supervises that
+session to settlement instead of acquiring another Run. An explicit House
+refusal destroys the stale tmux session and exposes no channel. House that
+cannot be reached leaves the binding and the session exactly as they are,
+invents no stopped, active, or authorized state, and acquires nothing.
 
 A restore of House's database starts a new log epoch. Until this Environment
 has reported in it, House refuses every act that touches a Run, acquisition and
@@ -1373,9 +1390,17 @@ state. The House revision of every file content Kit received, keyed by path,
 Git blob and the House state that received it, stays in Kit home under
 `state/working-copies/`, outside the repository. A local commit is a local tree, never a House revision.
 
-The resident Kit keeps every active copy current. Whenever its control stream
-carries a Room change, and whenever it reads the saved choice, one
-`POST /kit/door/discover` tells it where each
+The resident Kit keeps every active copy current. An `entries` frame on its
+control stream names a copied Room, the Room's position and log epoch, and the
+Change entries that brought it there: each changed path with its revision and
+text, or its removal. The resident applies the entries under the copy's own
+homes directly, without reading House, when the frame follows on from the copy:
+the same socket already carried an earlier frame for that Room, the copy's
+recorded position has reached that earlier frame's position, the log epoch is
+the copy's, and every changed path carries its revision and text. A frame the
+copy already holds changes nothing. Any other frame, and every reading of the
+saved choice, reads House instead, with the frames that arrive during one
+reading served by the next: one `POST /kit/door/discover` tells it where each
 copied Room stands, and for a Room that moved on it reads the changes after the
 copy's recorded position through `POST /kit/door/enumerate`. A changed, new or
 removed authored file becomes a commit on `refs/house/received`, its House

package/bin/kit.js

@@ -478,8 +478,10 @@ async function runWorkerRun(args, directory) {
       `Run ${run.run} stays bound to this installation until House accepts that report` +
         `${run.refused === null ? "" : ` (${run.refused.code})`}.`,
     );
+  } else if (run.refused !== null) {
+    write(`House refused that report (${run.refused.code}); this installation no longer holds Run ${run.run}.`);
   }
-  if (!run.settled || reported?.outcome === "failed") process.exitCode = 1;
+  if (!run.settled || run.refused !== null || reported?.outcome === "failed") process.exitCode = 1;
 }
 
 function syncCancellation() {

package/bootstrap.json

@@ -1,12 +1,12 @@
 {
   "schemaVersion": 1,
-  "version": "0.1.0-alpha.31",
+  "version": "0.1.0-alpha.32",
   "hosts": {
     "linux": {
-      "command": "bash <(curl -fsSL https://github.com/agentshouse/kit/releases/download/v0.1.0-alpha.31/connect-linux.sh)",
+      "command": "bash <(curl -fsSL https://github.com/agentshouse/kit/releases/download/v0.1.0-alpha.32/connect-linux.sh)",
       "asset": {
         "name": "connect-linux.sh",
-        "url": "https://github.com/agentshouse/kit/releases/download/v0.1.0-alpha.31/connect-linux.sh"
+        "url": "https://github.com/agentshouse/kit/releases/download/v0.1.0-alpha.32/connect-linux.sh"
       },
       "workspace": {
         "argument": "--workspace",

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@agentshouse/kit",
-  "version": "0.1.0-alpha.31",
+  "version": "0.1.0-alpha.32",
   "license": "MIT",
   "repository": {
     "type": "git",

package/src/kit-work-stream.js

@@ -40,6 +40,7 @@ async function held(session, signal, heard, dependencies) {
       perMessageDeflate: false,
     });
     let finished = false;
+    const carried = new Map();
     const finish = (reason = null, refusal = null) => {
       if (finished) return;
       finished = true;
@@ -63,7 +64,8 @@ async function held(session, signal, heard, dependencies) {
       if (frame?.type === "work_available" && WORK_SUBJECTS.includes(frame.subject)) {
         heard.available(frame.subject);
       } else if (frame?.type === "entries") {
-        heard.changed();
+        heard.changed(frame, carried.get(frame.authority) ?? null);
+        carried.set(frame.authority, frame.position);
       } else if (VIEW_FRAMES.includes(frame?.type)) {
         heard.viewed?.(frame);
       }

package/src/tasks.js

@@ -24,6 +24,23 @@ export function coalescedTask(signal, fail, work) {
   return { trigger, idle: () => task === null, settled: () => task };
 }
 
+export function orderedTask(signal, fail, work) {
+  let task = null;
+  const push = (...args) => {
+    const next = (task ?? Promise.resolve())
+      .then(() => (signal.aborted ? undefined : work(...args)))
+      .catch((error) => {
+        if (signal.aborted) return;
+        fail(error);
+      });
+    task = next;
+    void next.then(() => {
+      if (task === next) task = null;
+    });
+  };
+  return { push, idle: () => task === null, settled: () => task };
+}
+
 export function residentWake() {
   let pending = false;
   let release = null;

package/src/worker.js

@@ -22,12 +22,13 @@ import { inheritedEnvironment } from "./command.js";
 import { sourceEntries } from "./google/entries.js";
 import { watchKitWork } from "./kit-work-stream.js";
 import { finalPlan } from "./plan.js";
-import { coalescedTask, residentWake } from "./tasks.js";
+import { coalescedTask, orderedTask, residentWake } from "./tasks.js";
 import {
   applyWorkingCopies,
   deliverWorkingCopies,
   ownerCaller,
   pushWorkingCopy,
+  receiveWorkingCopyEntries,
   recordIndexChange,
   uploadedPaths,
   workingCopyPaths,
@@ -48,7 +49,7 @@ import {
   WORKING_COPY_PUSH_METHOD,
   WORKING_COPY_UPLOAD_COMMAND,
 } from "./contract.js";
-import { EpochReportRequired, formatCommandError, houseFailed, KitRefusal } from "./refusal.js";
+import { EpochReportRequired, formatCommandError, houseAnswered, houseFailed, KitRefusal } from "./refusal.js";
 import {
   acceptAssistantText,
   acquireCommands,
@@ -227,7 +228,21 @@ async function completeOwnerStop(context, bound, requestedAt) {
   return { destroyed, confirmed: true, failure };
 }
 
-async function settleReported(context, bound) {
+const REPORT_LATER = ["conversation_file_send_pending", "environment_control_lease_rejected", "operation_id_invalid"];
+
+const REPORT_REISSUED = ["operation_expired", "position_expired"];
+
+function reportLater(refusal) {
+  const retryable = refusal.payload?.error?.retryable;
+  return (
+    !houseAnswered(refusal) ||
+    retryable === true ||
+    REPORT_LATER.includes(refusal.code) ||
+    ([429, 502, 503].includes(refusal.status) && retryable === undefined)
+  );
+}
+
+async function settleReported(context, bound, retain) {
   try {
     await reportRunOutcome(context.session, bound.run, bound.reported, bound.plan);
     return { settled: true, failure: null };
@@ -235,14 +250,22 @@ async function settleReported(context, bound) {
     if (undecided(context, refusal)) return { settled: false, failure: null };
     if (!(refusal instanceof KitRefusal)) throw refusal;
     if (refusal.code === "worker_run_settled") return { settled: true, failure: null };
-    return { settled: false, failure: { code: refusal.code, detail: refusal.message } };
+    const failure = { code: refusal.code, detail: refusal.message };
+    if (REPORT_REISSUED.includes(refusal.code)) {
+      bound.reported = { ...bound.reported, operation: operationId() };
+      await retain(bound);
+      return { settled: false, failure };
+    }
+    if (reportLater(refusal)) return { settled: false, failure };
+    reported(`run ${bound.run} outcome refused: ${refusal.code}`);
+    return { settled: true, failure };
   }
 }
 
 async function reportTerminalState(context, bound, report) {
   const settling = { ...bound, reported: { operation: operationId(), ...report } };
   await writeBinding(context.directory, settling);
-  const settled = await settleReported(context, settling);
+  const settled = await settleReported(context, settling, (held) => writeBinding(context.directory, held));
   if (settled.settled) await removeBinding(context.directory, settling.tmux);
   return { ...settled, reported: settling.reported };
 }
@@ -482,22 +505,35 @@ function stoppedReport(bound) {
 
 async function settleStopped(context, bound) {
   const { stopped } = bound;
-  const unresolved = (refused) => ({ failure: stopped.failure, refused, settled: false });
+  const unresolved = (refused) => ({ failure: stopped.failure, reported: bound.reported, refused, settled: false });
   if (bound.sealed !== null) {
     const refused = await dispatchSeal(context, bound);
     if (refused !== null) return unresolved(refused);
   }
-  let failure = stopped.failure;
-  try {
-    await observeSessionStop(context.session, bound.session, stopped.textAccepted, bound.plan);
-  } catch (refusal) {
-    if (undecided(context, refusal)) return unresolved(null);
-    if (!(refusal instanceof KitRefusal) || refusal.code !== "conversation_final_text_pending") throw refusal;
-    failure ??= { code: refusal.code, detail: refusal.message };
-    await observeSessionStop(context.session, bound.session, false, bound.plan);
-  }
   const report = stoppedReport(bound);
+  if (report !== null && bound.reported === null) {
+    bound.reported = { operation: operationId(), ...report };
+    await writeBinding(context.directory, bound);
+  }
   if (report !== null) await removeContinuation(context.directory, bound.run);
+  let failure = stopped.failure;
+  const observe = async (textAccepted) => {
+    try {
+      await observeSessionStop(context.session, bound.session, textAccepted, bound.plan);
+      return null;
+    } catch (refusal) {
+      if (undecided(context, refusal)) return undefined;
+      if (!(refusal instanceof KitRefusal)) throw refusal;
+      return refusal;
+    }
+  };
+  let observed = await observe(stopped.textAccepted);
+  if (observed?.code === "conversation_final_text_pending") {
+    failure ??= { code: observed.code, detail: observed.message };
+    observed = await observe(false);
+  }
+  if (observed === undefined) return unresolved(null);
+  if (observed !== null && report === null) throw observed;
   try {
     await reportContinuation(context.session, bound.session, {
       providerContinuation: report === null,
@@ -505,17 +541,21 @@ async function settleStopped(context, bound) {
     });
   } catch (refusal) {
     if (undecided(context, refusal)) return unresolved(null);
-    if (!(refusal instanceof KitRefusal) || refusal.code !== "worker_run_settled") throw refusal;
-    await removeContinuation(context.directory, bound.run);
-    await removeBinding(context.directory, bound.tmux);
-    return { failure, refused: null, settled: true };
+    if (!(refusal instanceof KitRefusal)) throw refusal;
+    if (refusal.code === "worker_run_settled") {
+      await removeContinuation(context.directory, bound.run);
+      await removeBinding(context.directory, bound.tmux);
+      return { failure, refused: null, settled: true };
+    }
+    if (report === null) throw refusal;
   }
   if (report === null) {
     await removeBinding(context.directory, bound.tmux);
     return { failure, refused: null, settled: true };
   }
-  const settled = await reportTerminalState(context, bound, report);
-  return { failure, reported: settled.reported, refused: settled.failure, settled: settled.settled };
+  const settled = await settleReported(context, bound, (held) => writeBinding(context.directory, held));
+  if (settled.settled) await removeBinding(context.directory, bound.tmux);
+  return { failure, reported: bound.reported, refused: settled.failure, settled: settled.settled };
 }
 
 function assignmentRoute(context, assignment) {
@@ -601,7 +641,7 @@ async function settleRefused(context, command, failure) {
     reported: { operation: operationId(), ...failedReport(failure) },
   };
   await writeSettlement(context.directory, settling);
-  const settled = await settleReported(context, settling);
+  const settled = await settleReported(context, settling, (held) => writeSettlement(context.directory, held));
   if (settled.settled) await removeSettlement(context.directory, settling.run);
   return { run: settling.run, session: settling.session, tmux: null, assistant: [],
     failure, reported: settling.reported, refused: settled.failure,
@@ -697,7 +737,7 @@ async function reconcile(context) {
   const stopped = [];
   const unresolved = [];
   for (const settlement of await readSettlements(context.directory)) {
-    const settled = await settleReported(context, settlement);
+    const settled = await settleReported(context, settlement, (held) => writeSettlement(context.directory, held));
     if (settled.settled) await removeSettlement(context.directory, settlement.run);
     else unresolved.push(unresolvedRun(settlement, settled.failure));
   }
@@ -706,17 +746,17 @@ async function reconcile(context) {
   const commands = (await pendingCommands(context))?.stops ?? [];
   const carried = [];
   for (const binding of bindings) {
-    if (binding.reported !== null) {
-      const settled = await settleReported(context, binding);
-      if (settled.settled) await removeBinding(context.directory, binding.tmux);
-      else unresolved.push(unresolvedRun(binding, settled.failure));
-      continue;
-    }
     if (binding.stopped !== null) {
       const settled = await settleStopped(context, binding);
       if (!settled.settled) unresolved.push(unresolvedRun(binding, settled.refused));
       continue;
     }
+    if (binding.reported !== null) {
+      const settled = await settleReported(context, binding, (held) => writeBinding(context.directory, held));
+      if (settled.settled) await removeBinding(context.directory, binding.tmux);
+      else unresolved.push(unresolvedRun(binding, settled.failure));
+      continue;
+    }
     const command = commands.find((pending) => pending.run === binding.run) ?? null;
     const requestedAt = binding.ownerStop ?? command?.requestedAt ?? null;
     if (requestedAt === null) {
@@ -1105,12 +1145,12 @@ export async function runResident(options, dependencies = {}) {
       context.startLogins(needsLogin);
     });
     const unavailable = (error) => reported(`working copies unavailable: ${error?.message ?? error}`);
-    const changed = () => {
+    const redeliver = () => {
       if (!update.requested) deliveries.trigger();
     };
     const behind = (outcome) => {
       if (!outcome.behind) return;
-      controlSleep(interval, signal).then(changed, () => {});
+      controlSleep(interval, signal).then(redeliver, () => {});
     };
     const copies = coalescedTask(signal, unavailable, async () =>
       behind(await reportedWorkingCopies(applyWorkingCopies, context, signal)),
@@ -1118,6 +1158,12 @@ export async function runResident(options, dependencies = {}) {
     const deliveries = coalescedTask(signal, unavailable, async () =>
       behind(await reportedWorkingCopies(deliverWorkingCopies, context, signal)),
     );
+    const entries = orderedTask(signal, unavailable, async (frame, since) => {
+      if (!(await receiveWorkingCopyEntries(context, frame, since, signal))) redeliver();
+    });
+    const changed = (frame, since) => {
+      if (!update.requested) entries.push(frame, since);
+    };
     sources = sourceEntries({
       session: context.session,
       directory: context.directory,
@@ -1210,6 +1256,7 @@ export async function runResident(options, dependencies = {}) {
               agentRefresh.idle() &&
               copies.idle() &&
               deliveries.idle() &&
+              entries.idle() &&
               sources.idle())
           ) {
             return;
@@ -1223,6 +1270,7 @@ export async function runResident(options, dependencies = {}) {
       await agentRefresh.settled();
       await copies.settled();
       await deliveries.settled();
+      await entries.settled();
       await sources.settled();
       for (const running of logins.values()) running.controller.abort();
       await Promise.all([...logins.values()].map((running) => running.task));

package/src/working-copy.js

@@ -561,6 +561,55 @@ export async function deliverWorkingCopies(context, signal) {
     : deliveredOutcome(delivered.refused, delivered.behind);
 }
 
+function streamChange(entry) {
+  if (entry?.kind === "removed") return { path: entry.path, removed: true };
+  return { path: entry?.path, revision: entry?.revision, content: entry?.text, provenance: entry?.provenance };
+}
+
+function followsOn(copy, frame, since, changes) {
+  const { received } = copy;
+  return (
+    since !== null &&
+    received.log_epoch === frame.log_epoch &&
+    BigInt(received.position) >= BigInt(since) &&
+    !unreadableChanges(changes)
+  );
+}
+
+async function carriedEntries(context, path, frame, since, signal) {
+  const copy = await readCopy(path);
+  if (!copy?.active) return true;
+  const { received } = copy;
+  if (received.log_epoch === frame.log_epoch && BigInt(frame.position) <= BigInt(received.position)) return true;
+  const homes = copy.scope === "private" ? WORKING_COPY_PRIVATE_HOMES : WORKING_COPY_ROOM_HOMES;
+  const changes = frame.entries
+    .map(streamChange)
+    .filter((change) => typeof change.path !== "string" || homes.some((home) => covers(change.path, home)));
+  if (!followsOn(copy, frame, since, changes)) return false;
+  const view = { changes, position: frame.position, log_epoch: frame.log_epoch };
+  const authority = { operations: { write: received.declared.write }, protected_paths: received.declared.protected };
+  await writeCopy(context.directory, await incorporated(copy, view, authority, signal), signal);
+  return true;
+}
+
+export async function receiveWorkingCopyEntries(context, frame, since, signal) {
+  if (
+    typeof frame.authority !== "string" ||
+    typeof frame.log_epoch !== "string" ||
+    !/^[0-9]+$/.test(frame.position) ||
+    !Array.isArray(frame.entries) ||
+    (since !== null && !/^[0-9]+$/.test(since))
+  ) {
+    return false;
+  }
+  const copy = (await heldCopies(context.directory)).find(
+    (held) => held.active && held.room_ref === frame.authority,
+  );
+  if (copy === undefined) return true;
+  const path = copyPath(context.directory, copy);
+  return exclusively(path, () => carriedEntries(context, path, frame, since, signal));
+}
+
 export async function applyWorkingCopies(context, signal) {
   const selected = (await savedSelection(context.session, signal)).filter(
     (scope) => scope.kind === "private" || scope.handle !== null,
@@ -913,6 +962,14 @@ export function uploadedPaths(changes) {
   return [...pushedPaths(changes)].sort();
 }
 
+function unreadableChanges(changes) {
+  return changes.some(
+    (change) =>
+      typeof change?.path !== "string" ||
+      (change.removed !== true && (typeof change.content !== "string" || typeof change.revision !== "string")),
+  );
+}
+
 async function enumerated(context, copy, since, signal) {
   const answer = await door(
     context.session,
@@ -925,11 +982,7 @@ async function enumerated(context, copy, since, signal) {
     !Array.isArray(view?.changes) ||
     typeof view.position !== "string" ||
     typeof view.log_epoch !== "string" ||
-    view.changes.some(
-      (change) =>
-        typeof change?.path !== "string" ||
-        (change.removed !== true && (typeof change.content !== "string" || typeof change.revision !== "string")),
-    )
+    unreadableChanges(view.changes)
   ) {
     throw rejected("enumerate");
   }