@inetafrica/open-claudia 3.5.0 → 3.5.4

package/CHANGELOG.md

@@ -1,5 +1,35 @@
 # Changelog
 
+## 3.5.4
+
+Parallel conversations in Kazee Chat threads, with the fixes from an independent review. Canary first, on the owner's own bot. (3.5.2 and 3.5.3 were tagged but never published: 3.5.2 failed its release test, 3.5.3 was cancelled for this build.)
+
+- **A Kazee thread is its own conversation with the bot.** "Reply in thread" on any message starts a conversation that runs alongside the main chat and the other threads: its own session, transcript, queue, typing indicator and live preview, and the bot answers inside the thread. Up to 4 conversations run at once per person (`OC_MAX_PARALLEL_TURNS`) and 8 across the bot (`OC_MAX_PARALLEL_TURNS_GLOBAL`). Past that a message waits, says so, and starts when a slot frees. Inside one conversation turns still run one at a time.
+- **A new thread starts "hybrid":** the message it hangs off, the replies so far, and the last 10 main-chat messages before it, not the whole history. `OC_THREAD_SEED=fresh` leaves the main messages out; `OC_THREAD_SEED=fork` (Claude) resumes the main conversation into the thread instead.
+- **`/stop` stops only the conversation it is sent in, and drops the messages queued in it before the /stop,** naming them (`Cancelled. Dropped 2 queued: "reboot pe2", "and check the…"`). A conversation that is only waiting for a free slot drops its waiting messages the same way. `/stop all` does both for every conversation. A message sent after the /stop still runs. `/status` lists the conversations running now.
+- **Everything a thread turn starts comes back to that thread:** wake-ups, watches and crons, files, photos and voice notes, approvals. A button pressed on a message the bot posted in a thread runs in that thread.
+- **Groups:** a thread reply engages the bot when it is @mentioned, when the bot posted the thread's first message, or once the bot has replied in that thread. The main-chat catch-up leaves thread-only replies out.
+- **A refused thread never loses a reply:** if Kazee Chat refuses the thread (for example its first message was deleted), the reply goes to the main chat, quoting that message.
+- **Fixes from the review:** no typing indicator for a message that waits for a slot (it could stay on for good); a /stop during a compaction no longer eats the next message; in groups, a thread the bot replied in keeps engaging it even when the reply's id came back empty; a button pressed after a restart still runs in its thread; a job made in a forked thread never resumes the main conversation (`OC_THREAD_SEED=fork`); a job also waits for the conversation it replies into; a stuck run's late exit never frees a conversation another run holds; a turn can only post into its own thread; a Kazee send is never repeated because of bookkeeping; recall keeps a quiet conversation's own recent turns while other threads are busy.
+- **Fix:** compaction read the wrong conversation's recent turns (the legacy "Workspace" transcript) when it summarised.
+- **Changed:** background runs that have no chat now run beside chat conversations instead of queueing behind them, and count toward the caps.
+- **Note:** this release also carries 3.5.0 and 3.5.1 (permission phase 0). The fleet's containers still need the bubblewrap fix for staff turns before any roll; this canary is for one bot on macOS, where the environment scrub works.
+
+## 3.5.1
+
+Hardening after an adversarial review of 3.5.0, before any of it reaches the fleet. Canary first.
+
+- **A CLI believes only its own bot about who it acts for.** `whoami` and the staff `tool-gate` answer are asked only of a loopback that the CLI process itself or one of its ancestors is listening on, and never of one started inside a staff turn. An address in `OC_SEND_URL` is no longer enough: in 3.5.0 a staff turn could run its own server that answered "owner", and the keyring and the other owner-only verbs believed it. Linux reads `/proc`; macOS asks `/bin/ps` and `/usr/sbin/lsof`. Anything that cannot be checked is unattributed, which is never the owner.
+- **A staff member's watch check gets only the shell essentials.** It runs with an allowlist (PATH, HOME, user, locale, terminal, temp dirs, the config dir), its run stamp and `OC_GUARDED_RUN=1`. 3.5.0 dropped credentials by name pattern, so ones like `DATABASE_URL` or `MONGO_URI` still reached staff checks.
+- **The owner's watch checks get the bot's environment again,** so an owner watch that reads a credential from env works as it did before 3.5.0.
+- **A run with an incomplete chat origin is not the owner.** An origin that names a person or room but not the transport and room it came from counts as a non-owner. A run with no origin at all is still bot-internal work.
+- **The staff shell screen is wider, and it is not a boundary.** A staff turn's shell command may not name process environments or argument blocks, the keyring, the bot's config dir, per-process `/proc` paths, or credential stores. Correction to 3.5.0: the shell does not reliably refuse credential reads. A staff turn that has a shell can still read what the bot's OS user can read. Phase 2 takes the shell away from staff turns.
+- **The owner's routes outside a chat turn** (terminal, ssh, system cron), where 3.5.0 refuses owner-only work:
+  - the keyring: `/keyring set`, `/keyring get` and `/keyring remove` in the owner's chat;
+  - tool, pack, lessons, entity and ideas edits, and `open-claudia agent`: ask in the owner's chat;
+  - a write or destructive tool run: run it from a chat turn, or re-run it with the one-shot `--approval <id>` from a card the owner approved.
+- **Tests:** the provider parity test names the suite that timed out and allows 180 s per suite. A loaded CI runner overran 90 s on canary-v3.5.0's first build.
+
 ## 3.5.0
 
 Security fixes for the permission system: phase 0 of the redesign in the owner's 25 Sep review. The owner's own chat turns work as before. Staff and external turns lose the ways around the checks. First shipped as a canary on two bots.
@@ -18,7 +48,7 @@ Security fixes for the permission system: phase 0 of the redesign in the owner's
   - entity, pack, lessons and ideas writes;
   - every keyring verb, including `list`;
   - `open-claudia agent`.
-  From a plain terminal, these now need an owner approval token.
+  From a plain terminal these are refused; 3.5.1 lists the owner's routes.
 - **Follow-ups set in a staff turn run at that person's level, in that room.** This covers wakeups, crons and watches. The job's creator is the attested run. Before, it came from fields the run's own shell could forge, with a fallback to the owner. Watch checks run without the bot's secrets, as their creator.
 - **People notes and tasks are owner-only through the bot's commands.**
 - **History stays in the caller's room.** For a non-owner, `recent`, `transcript-search` and `transcript-window` return only the caller's room. `recent-fetch`'s channel filter, which matched nothing, is fixed.
@@ -29,7 +59,7 @@ Security fixes for the permission system: phase 0 of the redesign in the owner's
     - SSH, cloud and kube credentials;
     - other processes' environments.
   - Paths are resolved through symlinks.
-  - The shell refuses the obvious equivalents.
+  - The shell screens a few common equivalents. This is not a boundary (corrected in 3.5.1).
   - A permission hook that cannot run now refuses on a staff turn instead of letting the tool through.
 - **Destructive is never a standing permission.**
   - "Always" on a destructive card is refused.

package/bin/cli.js

@@ -56,18 +56,26 @@ function sendViaLoopback(kind, restArgs) {
   const stat = fs.statSync(filePath);
   const params = new URLSearchParams({ channelId, adapter: adapterId, fileName });
   if (process.env.OC_CHANNEL_USER_ID) params.set("userId", process.env.OC_CHANNEL_USER_ID);
+  // A thread turn's file goes back into its thread.
+  if (process.env.OC_THREAD_ID) params.set("threadId", process.env.OC_THREAD_ID);
   if (caption) params.set("caption", caption);
+  const headers = {
+    Authorization: `Bearer ${sendToken}`,
+    "Content-Type": "application/octet-stream",
+    "Content-Length": stat.size,
+  };
+  // The run's attest lets the bot check the thread against the live run.
+  if (process.env.OC_RUN_ID && process.env.OC_RUN_ATTEST) {
+    headers["X-OC-Run-Id"] = process.env.OC_RUN_ID;
+    headers["X-OC-Run-Attest"] = process.env.OC_RUN_ATTEST;
+  }
 
   const req = http.request({
     method: "POST",
     hostname: u.hostname,
     port: u.port,
     path: `/${kind}?${params.toString()}`,
-    headers: {
-      Authorization: `Bearer ${sendToken}`,
-      "Content-Type": "application/octet-stream",
-      "Content-Length": stat.size,
-    },
+    headers,
   }, (res) => {
     let body = "";
     res.on("data", (c) => { body += c; });

package/bin/loopback-client.js

@@ -7,6 +7,7 @@ const fs = require("fs");
 const path = require("path");
 const http = require("http");
 const configDir = require(path.join(__dirname, "..", "config-dir"));
+const botEndpoint = require(path.join(__dirname, "..", "core", "bot-endpoint"));
 
 function readLoopbackInfo() {
   try {
@@ -40,6 +41,9 @@ function context() {
     sendUrl,
     sendToken,
     channelId: process.env.OC_CHANNEL_ID,
+    // The chat thread the turn is in (set by the runner for thread turns):
+    // files, voice and jobs from this turn go back into that thread.
+    threadId: process.env.OC_THREAD_ID || null,
     adapterId: process.env.OC_CHANNEL_ADAPTER,
     userId: process.env.OC_CHANNEL_USER_ID || null,
     canonicalUserId: process.env.OC_CANONICAL_USER_ID || null,
@@ -71,15 +75,26 @@ function requireContext() {
   return ctx;
 }
 
-function postJson(kind, body) {
+// `verified: true` is for answers this process acts on (tool-gate): the
+// request only goes to an endpoint served by this process's own bot (see
+// core/bot-endpoint), never to whatever OC_SEND_URL names.
+function postJson(kind, body, { verified = false } = {}) {
   const ctx = requireContext();
-  const u = new URL(ctx.sendUrl);
+  let u;
+  if (verified) {
+    const check = botEndpoint.verifyEndpointCached(ctx.sendUrl);
+    if (!check.ok) return Promise.reject(new Error(`the bot's address could not be verified: ${check.reason}`));
+    u = new URL(check.url);
+  } else {
+    u = new URL(ctx.sendUrl);
+  }
   const requestedProject = body && body.project ? String(body.project) : ctx.project;
   const sameProject = !ctx.project || !requestedProject || requestedProject === ctx.project;
   const payload = JSON.stringify({
     ...body,
     ...(kind === "call-say" ? { callSpeechToken: process.env.OC_CALL_SPEECH_TOKEN || "" } : {}),
     channelId: ctx.channelId,
+    threadId: ctx.threadId,
     adapter: ctx.adapterId,
     userId: ctx.userId,
     canonicalUserId: ctx.canonicalUserId,
@@ -134,14 +149,17 @@ function postJson(kind, body) {
 // Who this process acts for, as the RUNNING BOT decides it. Needs no chat
 // context: a process with no live run behind it (stripped env, detached from
 // a finished turn, started outside the bot) comes back unattributed, and so
-// does every failure to ask. Unattributed is never the owner.
+// does every failure to ask. Unattributed is never the owner. The question
+// only goes to an endpoint served by this process's own bot — a server the
+// env points at is not believed (core/bot-endpoint).
 const UNATTRIBUTED = Object.freeze({ attributed: false, owner: false, guarded: true, kind: "unattributed" });
 
 function whoami({ timeoutMs = 5000 } = {}) {
   const ctx = context();
   if (!ctx.runId || !ctx.runAttest || !ctx.sendUrl || !ctx.sendToken) return Promise.resolve(UNATTRIBUTED);
-  let u;
-  try { u = new URL(ctx.sendUrl); } catch (_) { return Promise.resolve(UNATTRIBUTED); }
+  const check = botEndpoint.verifyEndpointCached(ctx.sendUrl);
+  if (!check.ok) return Promise.resolve(Object.freeze({ ...UNATTRIBUTED, reason: check.reason }));
+  const u = new URL(check.url);
   const payload = JSON.stringify({ runId: ctx.runId, originRunId: ctx.runId, runAttest: ctx.runAttest });
   return new Promise((resolve) => {
     const req = http.request({

package/bin/owner-gate.js

@@ -12,7 +12,8 @@ function hasRunAttest(env = process.env) {
 }
 
 function refusal(what, caller) {
-  const why = !(caller && caller.attributed) ? "this process is not attributed to an owner chat turn"
+  const why = !(caller && caller.attributed)
+    ? `this process is not attributed to an owner chat turn${caller && caller.reason ? ` (${caller.reason})` : ""}`
     : caller.name ? `this turn acts for ${caller.name}, not the owner`
       : "this turn does not act for the owner";
   return `⛔ Owner only: ${what} — ${why}.`;

package/bin/schedule.js

@@ -85,6 +85,11 @@ function jobProgress(j) {
   return state ? `${since}${state}` : "";
 }
 
+// A job created in a chat thread fires back into that thread.
+function threadMarker(j) {
+  return j.threadId ? `  (thread ${String(j.threadId).slice(-6)})` : "";
+}
+
 function fmtJob(j) {
   const when = j.kind === "wakeup"
     ? `fires ${new Date(j.fireAt).toISOString()}`
@@ -92,7 +97,7 @@ function fmtJob(j) {
   const proj = j.project ? ` [${j.project}]` : "";
   const src = j.source ? ` (${j.source})` : "";
   const progress = jobProgress(j);
-  return `${j.id}  ${j.kind}  ${j.label || "(no label)"}${proj}${src}  ${when}${progress ? `  [${progress}]` : ""}`;
+  return `${j.id}  ${j.kind}  ${j.label || "(no label)"}${proj}${src}${threadMarker(j)}  ${when}${progress ? `  [${progress}]` : ""}`;
 }
 
 function fmtWatch(w, now = Date.now()) {
@@ -104,7 +109,7 @@ function fmtWatch(w, now = Date.now()) {
         ? "  EXPIRED (fires on next poll)"
         : `  expires ${new Date(Number(w.expiresAt)).toISOString()}`)
       : "  no expiry (pre-timeout watch)";
-  return `${w.id}  ${mode}  polls "${w.schedule}"${expiry}  check: ${w.check}  → ${w.label || "(no label)"}`;
+  return `${w.id}  ${mode}${threadMarker(w)}  polls "${w.schedule}"${expiry}  check: ${w.check}  → ${w.label || "(no label)"}`;
 }
 
 async function runScheduleWakeup(args) {

package/bin/tool.js

@@ -438,7 +438,7 @@ async function run(args) {
         const caller = await client.whoami();
         if (!caller.attributed) {
           if (!approvalId) {
-            console.error(`⛔ ${t.name} (${gate.tier}) refused — not attributed to a live chat turn; write actions need a chat turn or an owner approval token.`);
+            console.error(`⛔ ${t.name} (${gate.tier}) refused — not attributed to a live chat turn; write actions need a chat turn or an owner approval token.${caller.reason ? ` (${caller.reason})` : ""}`);
             process.exitCode = 3; return;
           }
         } else if (!caller.owner && !approvalId) {
@@ -448,7 +448,7 @@ async function run(args) {
             if (!ctx.sendUrl || !ctx.sendToken || !ctx.channelId || !ctx.adapterId) throw new Error("no chat context to ask the bot from");
             decision = await client.postJson("tool-gate", {
               tool: t.name, verb: tools.invokedVerb(toolArgs), tier: gate.tier, command: commandLine,
-            });
+            }, { verified: true });
           } catch (e) {
             decision = { decision: "blocked", reason: `the bot could not vet this action (${e.message})` };
           }

package/bot.js

@@ -131,9 +131,13 @@ async function gracefulShutdown(signal) {
       console.error("memory shutdown drain failed:", error?.message);
       return { drained: false };
     });
+  // Every running conversation (main timeline and each thread) has its own
+  // process on its own lane.
   for (const state of userStates.values()) {
-    if (state.runningProcess) {
-      try { killProcessTree(state.runningProcess.pid, "SIGTERM"); } catch (e) {}
+    if (!(state.lanes instanceof Map)) continue;
+    for (const lane of state.lanes.values()) {
+      if (!lane.runningProcess) continue;
+      try { killProcessTree(lane.runningProcess.pid, "SIGTERM"); } catch (e) {}
     }
   }
   await Promise.all([sideCleanup, memoryCleanup]);