@agentshouse/kit 0.1.0-alpha.33 → 0.1.0-alpha.35

package/README.md

@@ -12,7 +12,7 @@ The Skill set Kit installs is Matt Pocock's
 On Linux `amd64` or `arm64`, run the published, versioned bootstrap once:
 
 ```sh
-bash <(curl -fsSL https://github.com/agentshouse/kit/releases/download/v0.1.0-alpha.33/connect-linux.sh)
+bash <(curl -fsSL https://github.com/agentshouse/kit/releases/download/v0.1.0-alpha.35/connect-linux.sh)
 ```
 
 The bootstrap first establishes Docker Engine from what it observes on the
@@ -445,7 +445,8 @@ service to a signed-in account asks only for that service's consent, and each
 service keeps its own grant. An address that answers another consent, or a code
 Google refuses to exchange, stores nothing; a declined consent or a ceremony
 that outlives its ten-minute bound cancels the sign-in, and the Kit tries again
-at the next reconciliation.
+at the next reconciliation. A capture that finds the grant revoked removes it,
+reports the entry `signed_out` and opens a new sign-in at once.
 
 The resident captures each signed-in entry every 15 minutes, with no command.
 An occurrence never overlaps its entry's previous one, and one missed while the
@@ -888,9 +889,10 @@ the same for every adapter kind: the line
 `The agents.house house CLI is available: run house --help.`, the base
 instructions of the Execution attempt plan House acquired, and the current text
 of the owner's How-we-work document `/private/library/how-we-work.md`, which the
-launcher reads with the `shell` tool over the Session's own bridge, following
-House's cursor to the whole text. An absent document, one the Session's ceiling
-cannot read, and empty base instructions add nothing. Any other failed read
+launcher reads with the `shell` tool over the Session's own bridge, running
+the continuation each cut reply's `output_cut` line prints until a reply comes
+back uncut. A document that is absent, or that the Session's ceiling cannot
+read, at the first read adds nothing, and so do empty base instructions. Any other failed read
 refuses the attempt as `kit_how_we_work_unreadable` with the bridge's cause
 before a prompt is sent. A session reloaded with `session/load` already holds
 the block and gets none. It forwards only adapter-classified User-visible assistant
@@ -1576,8 +1578,9 @@ kit sync rebootstrap ./replica
 ```
 
 `bootstrap` writes `ROOM.md`, Library text, House-derived projections, and
-Capture text with its provenance sidecar, and records one position per selected
-authority from the bundle, with the log epoch House issued it in, together with
+Capture text, with the provenance House serves on a file's manifest entry as
+its `_provenance` sidecar, and records one position per selected authority from
+the bundle, with the log epoch House issued it in, together with
 the positive operation scopes House declared for it: `read`, `sync`, `write`,
 and `delete`, each a list of paths relative to the authority under which that
 operation is admitted. Managed replication needs `read` plus `sync` on a Room;
@@ -1586,11 +1589,11 @@ with `operation_denied`. Generated `_provenance` files delivered beside authored
 Library paths are House metadata, not authored Library documents. `pull` presents
 each recorded position with its log epoch, applies each enumerated path's whole
 content or removal, and advances those positions. A removal, pulled or accepted
-from `submit`, also removes each directory it leaves empty up to the authority,
-so a later file can take that path; a directory still holding a file, such as a
-local draft, stays. A position older than House's
-declared retention, or one of a log epoch before House's database was restored,
-is reported as `position_expired`; only `rebootstrap` rebuilds the directory.
+from `submit`, also removes the file's `_provenance` sidecar and each directory
+it leaves empty up to the authority, so a later file can take that path; a
+directory still holding a file, such as a local draft, stays. A position older
+than House's declared retention, or one of a log epoch before House's database
+was restored, is reported as `position_expired`; only `rebootstrap` rebuilds the directory.
 
 `submit` sends field, section, and preamble operations with that value's prior
 content as base, and whole-file, rename, and removal operations with the file
@@ -1803,7 +1806,12 @@ another, beside the live drafts described below:
   the answer is absent or unreadable. File items are pulled back from House,
   refused as `file_too_large` above the 10 MiB image ceiling, uploaded as
   `sendPhoto` with the caption House composed around the link to the retained
-  original, and reported the same way; a pull that fails reports
+  original, and reported the same way. A caption past Telegram's
+  1,024-character caption limit keeps its head under the image, cut at its last
+  line break that fits or else at the limit, and the rest follows in the topic
+  as ordinary messages; the upload is reported with the image's answer when
+  every part was delivered and otherwise with the answer of the first part that
+  was not. A pull that fails reports
   `transfer_failed` and an ambiguous upload answer reports `delivery_unknown`.
   Only an image ever becomes a file item, because House offers every other file
   as a `conversation_file_link` message this queue delivers like any other. The
@@ -1814,6 +1822,10 @@ another, beside the live drafts described below:
   between blocks and inside a block only when that block alone exceeds them,
   and reported once, `delivered` with the first message's id when every part
   was delivered and otherwise with the answer of the first part that was not.
+  A new plain-text message past Telegram's 4,096-character message limit, such
+  as a `conversation_file_link` text, is sent and reported the same way, cut
+  at the last line break that fits or else at the limit, so the first message
+  keeps its reply and keyboard and a link on its own line stays whole.
   Each answer also names the `threads` the bot's topics carry, a
   `conversation_id` with its `chat_id` and `topic_id`, which is where live
   drafts go.
@@ -2177,6 +2189,17 @@ and that a pending submission House answers `operation_expired` is pulled first,
 then sent again under a new identifier only if it never landed. Evidence is
 written to `tmp/delivery-evidence/sync-pair.json`.
 
+The How-we-work reader has its own paired proof:
+
+```sh
+HOUSE_CORE_ROOT=<core checkout> HOUSE_AUTH_ROOT=<auth checkout> \
+  KIT_PAIR_SPECS=/kit/test/pair/how-we-work.vitest.ts bash test/pair/run.sh
+```
+
+It writes a document longer than two of House's cut shell replies and proves
+the installed reader returns its whole text by running each printed
+continuation. Evidence is written to `tmp/delivery-evidence/how-we-work-pair.json`.
+
 The retained-original consumers have their own paired proofs:
 
 ```sh

package/bootstrap.json

@@ -1,12 +1,12 @@
 {
   "schemaVersion": 1,
-  "version": "0.1.0-alpha.33",
+  "version": "0.1.0-alpha.35",
   "hosts": {
     "linux": {
-      "command": "bash <(curl -fsSL https://github.com/agentshouse/kit/releases/download/v0.1.0-alpha.33/connect-linux.sh)",
+      "command": "bash <(curl -fsSL https://github.com/agentshouse/kit/releases/download/v0.1.0-alpha.35/connect-linux.sh)",
       "asset": {
         "name": "connect-linux.sh",
-        "url": "https://github.com/agentshouse/kit/releases/download/v0.1.0-alpha.33/connect-linux.sh"
+        "url": "https://github.com/agentshouse/kit/releases/download/v0.1.0-alpha.35/connect-linux.sh"
       },
       "workspace": {
         "argument": "--workspace",

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@agentshouse/kit",
-  "version": "0.1.0-alpha.33",
+  "version": "0.1.0-alpha.35",
   "license": "MIT",
   "repository": {
     "type": "git",

package/src/adapter-mcp.js

@@ -38,54 +38,70 @@ export async function commandHouseInstructions(bridge) {
   return frame.result.instructions;
 }
 
+const CUT_NOTICE = "shell: output_cut ";
+const CONTINUATION = "; continue with: ";
+const BYTE_CONTINUATION = /\| tail -c \+[1-9][0-9]*$/u;
+
 function shellAnswer(text) {
   const lines = text.split("\n");
   if (!lines[0].startsWith(`protocol: render=${SHELL_RENDER} `)) return null;
   const exit = Number(lines[2].slice("exit: ".length));
+  const truncation = lines[3].slice("truncation: ".length);
   let at = lines[5]?.startsWith("authority: ") ? 6 : 5;
-  if (exit === 0) return { exit, stderr: [], stdout: lines.slice(at).join("\n"), cursor: null };
+  if (exit === 0 && truncation === "none") {
+    return { exit, truncation, stderr: [], stdout: lines.slice(at).join("\n"), continuation: null };
+  }
   const stderr = [];
-  while (lines[at]?.startsWith("stderr: ")) stderr.push(lines[at++].slice("stderr: ".length));
-  const stdout = lines.slice(at);
-  const cursor = stdout.at(-1)?.startsWith("cursor: ") ? stdout.pop().slice("cursor: ".length) : null;
-  return { exit, stderr, stdout: stdout.join("\n"), cursor };
+  let continuation = null;
+  while (continuation === null && lines[at]?.startsWith("stderr: ")) {
+    const line = lines[at++].slice("stderr: ".length);
+    stderr.push(line);
+    const separator = line.indexOf(CONTINUATION);
+    if (line.startsWith(CUT_NOTICE) && separator !== -1) {
+      continuation = line.slice(separator + CONTINUATION.length);
+    }
+  }
+  return { exit, truncation, stderr, stdout: lines.slice(at).join("\n"), continuation };
 }
 
-async function howWeWorkPage(bridge, cursor) {
+async function howWeWorkReply(bridge, command, first) {
   const frame = await bridgeCall(bridge, {
     method: "tools/call",
-    params: {
-      name: "shell",
-      arguments: { command: `cat ${HOW_WE_WORK_PATH}`, ...(cursor === null ? {} : { cursor }) },
-    },
+    params: { name: "shell", arguments: { command } },
   });
   if (frame.error !== undefined) {
     throw new KitRefusal("kit_how_we_work_unreadable", frame.error.message ?? null);
   }
   const text = answerText(frame.result);
   if (frame.result?.isError === true) {
-    if (text.split(":")[0] === "room_not_found") return null;
+    if (first && text.split(":")[0] === "room_not_found") return null;
     throw new KitRefusal("kit_how_we_work_unreadable", text);
   }
-  const page = shellAnswer(text);
-  if (page === null) throw new KitRefusal("kit_how_we_work_unreadable", text);
-  if (page.cursor === null && page.exit !== 0) {
-    if (page.stderr.length === 1 && page.stderr[0].split(" ")[1] === "path_not_found") return null;
-    throw new KitRefusal("kit_how_we_work_unreadable", page.stderr.join("\n"));
+  const reply = shellAnswer(text);
+  if (reply === null) throw new KitRefusal("kit_how_we_work_unreadable", text);
+  if (reply.exit !== 0) {
+    if (first && reply.stderr.length === 1 && reply.stderr[0].split(" ")[1] === "path_not_found") return null;
+    throw new KitRefusal("kit_how_we_work_unreadable", reply.stderr.join("\n"));
+  }
+  if (reply.truncation !== "none" && reply.continuation === null) {
+    throw new KitRefusal("kit_how_we_work_unreadable", reply.stderr.join("\n"));
   }
-  return page;
+  return reply;
 }
 
 export async function commandHouseHowWeWork(bridge) {
-  const pages = [];
-  let cursor = null;
-  do {
-    const page = await howWeWorkPage(bridge, cursor);
-    if (page === null) return "";
-    pages.push(page.stdout);
-    cursor = page.cursor;
-  } while (cursor !== null);
-  return pages.join("\n");
+  let document = "";
+  let command = `cat ${HOW_WE_WORK_PATH}`;
+  let first = true;
+  while (command !== null) {
+    const reply = await howWeWorkReply(bridge, command, first);
+    if (reply === null) return "";
+    document += reply.stdout;
+    command = reply.continuation;
+    if (command !== null && !BYTE_CONTINUATION.test(command)) document += "\n";
+    first = false;
+  }
+  return document;
 }
 
 export async function commandHouseCreateHowWeWork(bridge) {

package/src/contract.js

@@ -61,6 +61,8 @@ export const TELEGRAM_DOWNLOAD_CEILING_BYTES = 20 * 1024 * 1024;
 export const TELEGRAM_BOT_API_TIMEOUT_MS = 30_000;
 export const TELEGRAM_GATEWAY_POLL_MS = 2_000;
 export const TELEGRAM_RICH_MESSAGE_CHARACTERS_MAX = 32_768;
+export const TELEGRAM_MESSAGE_CHARACTERS_MAX = 4_096;
+export const TELEGRAM_CAPTION_CHARACTERS_MAX = 1_024;
 export const TELEGRAM_RICH_MESSAGE_BLOCKS_MAX = 500;
 export const TELEGRAM_CONTROL_KEYBOARD_COMMANDS = ["/stop", "/kill"];
 export const TELEGRAM_MEDIA_KINDS = [

package/src/google/entries.js

@@ -213,7 +213,10 @@ export function sourceEntries(context) {
       const task = signIn(occasion(record), entry);
       signingIn = { source: entry.source, task };
       try {
-        if (await task) record.last = null;
+        if (await task) {
+          record.last = null;
+          attempted.delete(entry.source);
+        }
       } catch (error) {
         if (!record.stop.signal.aborted) throw error;
       } finally {

package/src/sync.js

@@ -131,23 +131,31 @@ function scopeContents(replica, scope) {
   return scope.contents;
 }
 
+function servedFile(scope, path) {
+  return scope.manifest.files.find((file) => file?.path === path);
+}
+
 async function writeScope(replica, scope, signal) {
-  const files = [];
+  const recorded = new Map();
   for (const entry of scopeContents(replica, scope)) {
-    const meta = scope.manifest.files.find((file) => file.path === entry.path);
+    const meta = servedFile(scope, entry.path) ?? {};
     await writeReplicaFile(locate(replica, scope.manifest.path, entry.path), entry.content, signal);
-    files.push({
-      path: entry.path,
-      revision: typeof meta?.revision === "string" ? meta.revision : "",
-      sha256: typeof meta?.sha256 === "string" ? meta.sha256 : "",
-      content: entry.content,
-    });
+    recorded.set(entry.path, enumeratedRecord(entry.path, entry.content, meta));
+    if (typeof meta.provenance === "string") {
+      await writeSidecar({ replica }, scope.manifest, recorded, entry.path, meta, signal);
+    }
   }
-  return files;
+  return [...recorded.values()];
 }
 
 async function replaceScope(replica, previous, scope, signal) {
-  const kept = new Set(scopeContents(replica, scope).map((entry) => entry.path));
+  const kept = new Set(
+    scopeContents(replica, scope).flatMap((entry) =>
+      typeof servedFile(scope, entry.path)?.provenance === "string"
+        ? [entry.path, sidecarPath(entry.path)]
+        : [entry.path],
+    ),
+  );
   const root = locate(replica, scope.manifest.path);
   for (const path of await listFiles(root)) {
     if (!kept.has(path)) await removeReplicaFile(replica, scope.manifest.path, path);
@@ -255,6 +263,12 @@ async function writeSidecar(options, authority, recorded, path, change, signal)
   recorded.set(sidecar, enumeratedRecord(sidecar, change.provenance, change));
 }
 
+async function removeSidecar(options, authority, recorded, path) {
+  const sidecar = sidecarPath(path);
+  await removeReplicaFile(options.replica, authority.path, sidecar);
+  recorded.delete(sidecar);
+}
+
 async function applyEnumeration(options, state, enumeration, signal) {
   if (!Array.isArray(enumeration?.authorities)) throw new KitRefusal("kit_sync_rejected", "enumerate");
   for (const view of enumeration.authorities) {
@@ -267,6 +281,7 @@ async function applyEnumeration(options, state, enumeration, signal) {
       if (change.removed === true) {
         await removeReplicaFile(options.replica, authority.path, change.path);
         recorded.delete(change.path);
+        await removeSidecar(options, authority, recorded, change.path);
         continue;
       }
       if (typeof change.content === "string") {
@@ -360,6 +375,7 @@ async function acceptBatch(session, options, state, outcome, signal, pending = n
     if (accepted === null) {
       if (!preserve) await removeReplicaFile(options.replica, authority.path, relative);
       recorded.delete(relative);
+      await removeSidecar(options, authority, recorded, relative);
       continue;
     }
     if (!preserve) await writeReplicaFile(destination, accepted, signal);

package/src/telegram-view.js

@@ -25,13 +25,16 @@ function richText(value, key = null) {
   return [];
 }
 
-function slices(text) {
+export function slices(text, limit = TELEGRAM_RICH_MESSAGE_CHARACTERS_MAX) {
   const pieces = [];
   let start = 0;
   while (start < text.length) {
-    let end = Math.min(start + TELEGRAM_RICH_MESSAGE_CHARACTERS_MAX, text.length);
+    let end = Math.min(start + limit, text.length);
+    const window = text.slice(start, end);
+    const line = window.lastIndexOf("\n");
     const last = text.charCodeAt(end - 1);
-    if (end < text.length && last >= 0xd800 && last <= 0xdbff) end -= 1;
+    if (end < text.length && line > 0 && /\S/.test(window.slice(0, line))) end = start + line + 1;
+    else if (end < text.length && last >= 0xd800 && last <= 0xdbff) end -= 1;
     pieces.push(text.slice(start, end));
     start = end;
   }

package/src/telegram.js

@@ -10,6 +10,7 @@ import {
   PERSONAL_BOT_OFFSET_RELATIVE_PATH,
   PERSONAL_BOT_TOKEN_RELATIVE_PATH,
   TELEGRAM_BOT_API_TIMEOUT_MS,
+  TELEGRAM_CAPTION_CHARACTERS_MAX,
   TELEGRAM_CONTROL_KEYBOARD_COMMANDS,
   TELEGRAM_DELIVERY_BATCH_MAX,
   TELEGRAM_DELIVERY_KINDS,
@@ -18,6 +19,7 @@ import {
   TELEGRAM_IMAGE_UPLOAD_CEILING_BYTES,
   TELEGRAM_INGRESS_BATCH_MAX,
   TELEGRAM_MEDIA_KINDS,
+  TELEGRAM_MESSAGE_CHARACTERS_MAX,
   TELEGRAM_UNSUPPORTED_MEDIA_FIELDS,
   TELEGRAM_UPDATE_OUTCOMES,
   telegramApiUrl,
@@ -27,7 +29,7 @@ import { heldControl, houseRequest, maintainControl, openSession } from "./house
 import { watchKitWork } from "./kit-work-stream.js";
 import { KitRefusal } from "./refusal.js";
 import { writeText } from "./state.js";
-import { richPieces, turnViews } from "./telegram-view.js";
+import { richPieces, slices, turnViews } from "./telegram-view.js";
 import { kitStreamWriter } from "./turn-frames.js";
 
 const TOKEN_PROMPT = "Paste the BotFather token for this Environment: ";
@@ -285,8 +287,21 @@ export function deliveryCall(delivery) {
   return { method: "sendMessage", body: sendBody(delivery) };
 }
 
+function topicMessage(chat, topic, text) {
+  const body = { chat_id: Number(chat), text };
+  if (topic !== null) body.message_thread_id = Number(topic);
+  return { method: "sendMessage", body };
+}
+
 function deliveryCalls(delivery) {
   const call = deliveryCall(delivery);
+  if (call.method === "sendMessage" && call.body.text.length > TELEGRAM_MESSAGE_CHARACTERS_MAX) {
+    const [first, ...following] = slices(call.body.text, TELEGRAM_MESSAGE_CHARACTERS_MAX);
+    return [
+      { method: call.method, body: { ...call.body, text: first } },
+      ...following.map((text) => topicMessage(delivery.chat_id, delivery.topic_id, text)),
+    ];
+  }
   if (call.method !== "sendRichMessage") return [call];
   return richPieces(call.body.rich_message).map((rich_message) => ({
     method: call.method,
@@ -791,6 +806,27 @@ async function sendQueuedMessage(gateway, token, delivery) {
   return false;
 }
 
+function captionPieces(caption) {
+  if (caption === null || caption.length <= TELEGRAM_CAPTION_CHARACTERS_MAX) return [caption];
+  const [head] = slices(caption, TELEGRAM_CAPTION_CHARACTERS_MAX);
+  return [head, ...slices(caption.slice(head.length), TELEGRAM_MESSAGE_CHARACTERS_MAX)];
+}
+
+async function sentOverflow(gateway, token, upload, overflow) {
+  for (const text of overflow) {
+    const call = topicMessage(upload.chat_id, upload.topic_id, text);
+    let answered;
+    try {
+      answered = await telegramPost(gateway, telegramApiUrl(token, call.method), call.body);
+    } catch (error) {
+      if (cancelled(error)) throw error;
+      return { settled: uploadOutcome(null), answered: null };
+    }
+    if (!botApiAccepted(answered)) return { settled: uploadOutcome(answered), answered };
+  }
+  return null;
+}
+
 async function sendQueuedUpload(gateway, token, upload) {
   let bytes;
   try {
@@ -810,12 +846,13 @@ async function sendQueuedUpload(gateway, token, upload) {
     });
     return false;
   }
+  const [caption, ...overflow] = captionPieces(upload.caption);
   let answered;
   try {
     answered = await telegramUpload(
       gateway,
       telegramApiUrl(token, "sendPhoto"),
-      uploadFields(upload),
+      uploadFields({ ...upload, caption }),
       { field: "photo", name: upload.file_name, bytes },
     );
   } catch (error) {
@@ -826,8 +863,11 @@ async function sendQueuedUpload(gateway, token, upload) {
     });
     return false;
   }
-  await reportUpload(gateway, upload, uploadOutcome(answered));
-  const retryAfter = telegramRetryAfter(answered);
+  const photographed = uploadOutcome(answered);
+  const unsent =
+    photographed.outcome === "delivered" ? await sentOverflow(gateway, token, upload, overflow) : null;
+  await reportUpload(gateway, upload, unsent?.settled ?? photographed);
+  const retryAfter = telegramRetryAfter(unsent === null ? answered : unsent.answered);
   if (retryAfter === null) return false;
   gateway.gate.hold(retryAfter);
   return true;