@inetafrica/open-claudia 3.5.14 → 3.5.17

package/CHANGELOG.md

@@ -1,5 +1,63 @@
 # Changelog
 
+## 3.5.17
+
+One combined release (Sumeet, 7 Oct 09:21 EAT): Release 1 below, the 7 Oct security fixes, and the dashboard and provider fixes found on 6 Oct. What people will notice: Codex bots start answering in groups when a message concerns them (their checks failed before), a file over 200 MB sent to a bot is refused, and a /dashboard link opens a Continue page first.
+
+Release 1, "reliable and lean" (Sumeet, 7 Oct: token diet approved 06:12 EAT, task-cleanup plan sections 2, 5 and 6 at 09:45, resuming interrupted work at 10:20). Fewer tokens re-read on every step, scheduled work that actually runs, a dream that reaches the owner, and work that survives a restart. The compaction changes are a canary: off until conversations are listed.
+
+- **Each conversation sees its own tasks.** A new task records the conversation it was made in (the thread, or the main timeline; a plan's subtasks follow the plan). The first turn of a session no longer gets the whole channel's task tree (90-123k characters on the owner's Kazee chat, where every thread shares one list): it gets this conversation's open tasks in full (at most 12, 6,000 characters), one line about the rest of the chat (count and the 5 most recent titles) and the pointer to `open-claudia task list`, at most 7,500 characters in all. Older tasks with no conversation count as "elsewhere". `task list` is unchanged.
+- **Team list and slash commands once per session.** They rode every turn (about 4.5k characters); now they come with the first turn of a session in each conversation (again after a restart, a new conversation or a compaction). Who is speaking and the formatting notes still come every turn.
+- **Token diet canary for compaction.** `TOKEN_DIET_CONVERSATIONS` (conversation keys as the usage ledger writes them, or `*`; empty = off) makes listed conversations compact at `TOKEN_DIET_COMPACT_TOKENS` (200k) instead of 380k, and compact before a turn that resumes them after the prompt cache expired (idle over `TOKEN_DIET_COLD_MINUTES`, 60) when their context is over `TOKEN_DIET_COLD_MIN_TOKENS` (120k); the turn then runs on the fresh session. `/compactwindow` still wins. Every compaction now leaves a usage-ledger record (`kind: "compaction"`, `compactReason`: threshold, cold-resume or manual, the context it replaced and what it cost), so before and after can be measured per conversation; the cost report and the dashboard keep counting turns only.
+- **The dream reports every task list.** The cold-task report read only the old Telegram list; it now covers every task list on the bot (task commands are owner-only, so they are all the owner's), at most 8 lines, each naming its list.
+- **Task archive, with a reason.** `open-claudia task archive <id> --reason "done: <proof>" | "replaced: <by what>" | "dropped: <who decided>" | "handed: <to whom>"` moves a task and its subtasks out of the list without losing them; `task restore <id>` brings them back with the same ids; `task archived [--limit N] [--search <text>]` lists them. Nothing archives by age, and `task done` is unchanged.
+- **Late crons still run.** node-cron skipped any run that started more than a second late, and the bot is often 2-3 seconds late at the top of a minute: the 04:00 dream ran on 7 of 14 nights and the 07:30 daily cron missed 6 and 7 Oct. Crons now tolerate `OC_CRON_LATE_TOLERANCE_MS` (10 min) and the dream `DREAM_LATE_TOLERANCE_MS` (3 h).
+- **The dream catches up.** If it has not run for more than 26 hours, it runs once now (checked at boot and hourly), never beside a running dream and at most once per 26 hours.
+- **The dream summary reaches the owner's main channel.** `DREAM_SUMMARY_TO` (`<adapter>:<channelId>`) if set, else the owner's primary channel from the people list, else Telegram as before. A Kazee-only owner never got it.
+- **A failing recurring job keeps its schedule.** One occurrence that failed five times used to disable a cron for good. Now that occurrence is recorded and skipped and the job runs again at its next time; one-shot wake-ups still give up.
+- **The owner hears when a job dies.** A wake-up or watch that gives up, or a cron occurrence that failed for good, posts one short message in its own conversation (or to the owner if that channel is gone): which job, the error code, and how to re-arm it.
+- **"No response requested." is not an answer.** A wake-up, cron or watch whose whole reply is that placeholder (Claude Code writes it into a session cut short by a restart) is re-run once, in the same session, with "Act on the prompt above now. Your first output must be a tool call or a real reply."
+- **New lessons reach running conversations.** A lesson learned after a conversation started is added to its next turn, once, under "New standing rules since this conversation started:".
+- **Interrupted work picks up after a restart** (Sumeet, 7 Oct: "a way for interrupted threads and work to get resumed automatically"). When a chat turn starts, the bot notes it in a small journal file and removes the note when the turn ends, is stopped or fails. After a restart the bot reads what is left: a turn cut off less than 2 hours ago (`RESUME_MAX_AGE_MIN`) continues in the same conversation with "Picking up where I was cut off (restart at HH:MM)." and an instruction to check what was already done and never repeat a send, push, deploy or write (also when it was the conversation's first turn, or its session is lost: the work may be half done, so it is never replayed as the plain message); an older one is offered back with Continue / Drop buttons for a day; one that was already cut off twice is not run again and the owner is told. Messages that were waiting behind it run afterwards, in order. `RESUME_INTERRUPTED=off` turns this off; safe mode never resumes anything. Scheduled jobs keep their own retry and are never run twice.
+- **/stop no longer loses waiting messages.** Each message it drops comes back with a "Run it" button for a day.
+- **Planned restarts wait for running work.** On a restart, upgrade or rollout the bot stops starting new turns (a message sent then runs right after the restart) and gives running turns time to finish: up to `DRAIN_TIMEOUT_S` (60 s) on /restart, /upgrade and /downgrade, `DRAIN_SIGTERM_S` (6 s) on a rollout or launchd stop, whose sender kills the bot soon after (bot pods allow 15 s); the bot saves its state before it waits. Anything still running picks up after the restart.
+- **A pack is not re-sent for a metadata change.** A memory pack already in the session is injected again only when what it shows changed, not when its "applied on" stamp or parent bookkeeping moves.
+- **Security: a sender's file name stays inside the files folder.** Kazee and Telegram saved a document under the sender's own file name, so a name like `../auth.json` could overwrite the owner list. One helper (`core/safe-filename.js`) keeps the last path segment, drops control and odd characters, refuses hidden or empty names, caps the length and checks the result stays in the folder. A document the guard holds for the owner keeps its file for the approval.
+- **Security: Kazee media downloads are fenced and capped at 200 MB.** The adapter fetched any link a message carried, with no size limit. Now `KAZEE_MEDIA_HOSTS`, when set, is the exact list of hosts allowed; when unset, any public host is allowed but private, loopback and link-local addresses are refused (checked on every lookup answer, so the address cannot change in between); Kazee's own host is always allowed. Files stream to a `.part` file and stop at 200 MB (Kazee Chat's own limit) or 60 s idle; a failed or oversized download is deleted.
+- **Security: the dashboard escapes everything it shows.** Pending users' names, cron labels, session titles, logs, the soul text, config values and usage rows went into the page as HTML, so a Telegram name or cron label carrying markup ran as script in the owner's dashboard.
+- **Security: a bad `/otp/` link answers 400.** A malformed link threw inside the request handler and messaged the owner each time, no sign-in needed. Bad links get a short 400 page; any handler failure answers 500 and is logged.
+- **Fix: /dashboard links survive Kazee Chat's link previews.** Kazee Chat fetches every link in a message to build a preview, which used up the one-time sign-in link before anyone tapped it ("Link expired or already used", 6 Oct). The link now opens a Continue page; only pressing it (a POST) uses the link.
+- **Fix: the Users tab works.** Approve, deny and remove on the bot's dashboard and Agent Space's Users tab answered 404, because the sign-in routes above them took every `/api/auth/*` path.
+- **Fix: a conversation whose saved history is gone starts fresh.** Claude Code answers "No conversation found with session ID" when the session file is missing, and every turn of that conversation failed until someone sent /new (chat-spaces-bot, 6 Oct). The stale session is retired and the turn runs fresh.
+- **Fix: helper agents' notes stay out of the chat.** Text streamed by a helper agent (the Agent/Task tool) was posted as the bot's reply (a security audit's working notes reached the owner, 6 Oct); it now only counts toward spend.
+- **Fix: Codex bots' background checks pass OpenAI's strict schema rules.** Every utility schema broke strict mode (400 invalid_json_schema), so on Codex bots passive group replies, pack review, dedup, the dream, recall filtering and bot hand-off checks all failed. They work now, which is why Codex bots start answering in groups.
+- **Fix: reassurances are never standing rules.** The owner-rule detector took "Never mind, I found it." as a never-directive and saved it, and offered "Don't worry so much about that." as a rule. Sentences opening with don't worry, do not worry or never mind no longer form a rule.
+- **node-cron ^4.6.0.** 4.2.1 has no option to tolerate a late start, which the late-cron fix relies on (its fallback for old versions stays).
+- **Tests:** `test-task-injection.js`, `test-speaker-extras.js`, `test-token-diet.js`, `test-dream-tasks.js`, `test-task-archive.js`, `test-cron-tolerance.js`, `test-dream-schedule.js`, `test-job-failures.js`, `test-scheduled-placeholder.js`, `test-lessons-since.js`, `test-pack-reinject.js`, `test-turn-journal.js` (new); `test-thread-lanes.js`, `test-loopback-owner-gate.js` and `test-provider-scheduler.js` updated. The fake agent CLI gains a `[[placeholder]]` prompt marker. Security and fix tests: `test-safe-filename.js`, `test-media-filename-paths.js`, `test-router-document-guard.js`, `test-web-dashboard-escape.js`, `test-web-bad-requests.js`, `test-kazee-download-limit.js`, `test-web-auth-routes.js`, `test-web-otp-link-preview.js`, `test-codex-output-schema.js` (new); `test-kazee-context-media.js`, `test-provider-refusal-recovery.js`, `test-provider-events.js`, `test-provider-dream.js`, `test-provider-pack-review.js`, `test-guardrail.js` extended.
+
+## 3.5.16
+
+Watch a bot's browser live (Sumeet, 6 Oct: "can that show the browser like a see my computer view?", then "yeh go ahead"). Also carries 3.5.14 and 3.5.15, which never reached the bots: the 3.5.15 build kept failing on a timing-sensitive test, fixed here. Canary on the sales and tickets bots, then the fleet.
+
+- **Live browser view.** The bot's web server lists its running browser sessions (`GET /api/browser/sessions`: name, page address, last change) and streams one session's screen and page address as server-sent events (`GET /api/browser/stream?session=<name>&fps=<n>`), the same transport as the chat stream, behind the usual sign-in (Agent Space's bearer token or a dashboard session). Agent Space shows it in a new Browser tab on each bot's page; the bot's own dashboard has a Browser tab too.
+- **Watch-only.** Nothing flows from a viewer to the browser, so watching can never click or type. Page addresses are cut to origin + path (no query string, fragment or credentials), the stream port stays inside the bot, and frames are capped at 10 a second and skipped, never queued, for a slow viewer.
+- **agent-browser pinned to 0.38.2** in the image: the view reads its session files in `~/.agent-browser` (`<session>.stream`, `.pid`, `.target`) and its screen-stream protocol. On Linux a session counts only while its pid is still an agent-browser process, because the files outlive a pod restart on /data.
+- **Fix: a failed Codex tool-hook check is tried again.** One probe that failed (a timeout on a busy node) blocked every Codex run until the bot restarted (smart-hellen, 5 Oct); only a passing check is remembered now.
+- **Test fix:** test-thread-lanes.js's "still running" checks hold the busy run until the test releases it (`[[hold:<name>]]` in the fake agent) instead of trusting a fixed delay; the third 3.5.15 build failed there on a loaded runner.
+- **Tests:** `test-browser-view.js`, `test-web-browser-view.js` (new); `test-provider-tool-hooks.js` covers the Codex re-check.
+
+## 3.5.15
+
+Bots can hand work to each other in a Kazee group, and decide for themselves when to stop (Sumeet, 5 Oct: "there is times we want bots to talk to one another", then "4 is arbitrary ... we want them to be able to decide when to stop"). Canary first, on the sales and tickets bots.
+
+- **A bot asks another bot on purpose.** A group turn now names the other bots in the room with the exact tag for each (`@[Name](id)`; chat-central turns that markup into a mention, for bots' posts too). A bot tags one only when it needs something specific from it to finish what a person asked, and says what and for whom. Only the final answer can carry a live tag: a stream preview or a notice writes any bot tag as a plain name.
+- **A quick check before every bot tag.** The small utility model reads the room and the draft and keeps the tag only when it asks for something new that moves the work forward; thanks, confirmations, repeats and polishing go out as plain names and wake nobody. A failed check means no tag.
+- **The answer wakes only the bot that asked.** Another bot's reply to a message in which this bot tagged it wakes this bot, so it can use the answer; any other reply from a bot, a shared thread, "Every message" and "When they can help" still never do, and a bot cannot snooze another. Silence ends an exchange.
+- **A safety net, not a fixed stop.** Instead of a hard stop after 4 bot messages, the bots go on until `OC_BOT_CHAIN_LIMIT` (now 10) bot messages in a row since a person wrote, or until one repeats an ask. Then the woken bot posts one check-in: ⏸, the person tagged, two lines on where things stand, "carry on?". Every bot waits for a person after that; a reply to the check-in starts things again. A person who writes "stop" ("that's enough", "hold on", ...) stops the bots until a person writes again.
+- **Permissions don't change.** A bot that asks is a non-owner speaker like anyone else: guarded turns, the owner's OK for changes, grants.
+- **Fix: "@bot be quiet" works from the Kazee apps.** The "be quiet" / "you can talk again" commands of passive groups only recognised a plain "@Name"; the apps write a tag as `@[Name](id)`, so the command sent from the app was never recognised. Tags in that form are now taken out first.
+- **Tests:** `test-kazee-bot-handoffs.js` (new); `test-kazee-bot-loops.js` rewritten for the new rules; `test-passive-replies.js` covers the apps' tag form.
+
 ## 3.5.14
 
 Decisions stop sinking and going stale (Sumeet, 5 Oct: "they tend to fall quite far back in the chat ... after a while they might be invalid"). Bot side of "decisions inline"; works with today's chat-central, and the "waiting on you" strip comes with the server and app changes. Canary first, on the sales bot.

package/Dockerfile

@@ -15,7 +15,9 @@ RUN npm install -g @openai/codex
 # replaces Playwright MCP). Installed at build time as root so the non-root runtime user
 # doesn't hit EACCES on `npm install -g`. Chrome is pinned to the apt chromium binary so it
 # lives outside the /data VOLUME (build-time writes to /data are discarded at runtime).
-RUN npm install -g agent-browser
+# Pinned: the live Browser view (core/browser-view.js) reads its session files and its
+# screen-stream protocol, which 0.38.x has (built-in stream per session, .stream port files).
+RUN npm install -g agent-browser@0.38.2
 ENV AGENT_BROWSER_EXECUTABLE_PATH=/usr/bin/chromium
 
 # Create app directory

package/README.md

@@ -439,6 +439,7 @@ All stored in `~/.open-claudia/`:
 | `TELEGRAM_CHAT_ID` | Yes* | Comma-separated authorized chat IDs |
 | `CHANNELS` | No | Channels to start: `telegram`, `kazee`, or `telegram,kazee` (default `telegram`) |
 | `KAZEE_URL` / `KAZEE_BOT_TOKEN` / `KAZEE_BOT_USER_ID` / `KAZEE_OWNER_USER_ID` | Kazee only | Kazee Chat connection |
+| `KAZEE_MEDIA_HOSTS` | No | Comma-separated hosts (or URLs) Kazee serves file links from. When set, files are fetched only from these and the `KAZEE_URL` host. When unset, any public host works but private, loopback and link-local addresses are refused. Files stop at 200 MB either way |
 | `WORKSPACE` | Yes | Path to your projects directory |
 | `CLAUDE_PATH` / `CODEX_PATH` | No | Optional paths to provider CLIs (auto-detected if in PATH) |
 | `DEFAULT_PROVIDER` | No | Foreground default (`claude` or `codex`); blank deterministically chooses the first compatible provider |

package/bin/cli.js

@@ -648,7 +648,7 @@ Background work (only inside an active bot-spawned task):
   open-claudia watch-list                          List watches on this channel + master switch state
   open-claudia watch-remove <id>                   Remove a watch
   open-claudia watch-on | watch-off                Global master switch for all watches (owner)
-  open-claudia task add|list|start|done|remove     Per-channel todo (survives restarts)
+  open-claudia task add|list|start|done|remove|archive|restore  Per-channel todo (survives restarts)
   open-claudia agent "<prompt>" [--role X] [--provider active|claude|codex]  Spawn a provider-isolated sub-agent
 
 Cluster control (self-management of this bot's own deployment via the broker):

package/bin/task.js

@@ -17,6 +17,10 @@ Per-channel todo list with optional plan/subtask hierarchy.
                                          # a plan won't complete while any subtask is still open
   open-claudia task remove <id>          # removing a plan removes its subtasks
   open-claudia task clear-completed      # prune any leftover completed entries
+  open-claudia task archive <id> --reason "<how it closed>"  # move a plan/task (and its subtasks) out of the list
+        reason (required): "done: <proof>" | "replaced: <by what>" | "dropped: <who decided>" | "handed: <to whom>"
+  open-claudia task restore <id>         # bring an archived task back, same id
+  open-claudia task archived [--limit N] [--search <text>]  # archived tasks, most recent first
 
 Use plans for any work with 3+ distinct steps. Mark subtasks in_progress
 as you start them and done as you finish, so a resumed turn or a new
@@ -166,6 +170,50 @@ async function runClearCompleted() {
   } catch (e) { console.error(e.message); process.exit(1); }
 }
 
+async function runArchive(args) {
+  const reason = takeFlag(args, "reason");
+  const id = args[0];
+  if (!id || !reason) {
+    console.error('Usage: task archive <id> --reason "done: <proof>" | "replaced: <by what>" | "dropped: <who decided>" | "handed: <to whom>"');
+    process.exit(2);
+  }
+  try {
+    const res = await postJson("task-archive", { id, reason });
+    if (!res.ok) { console.error(`Not found: ${id}`); process.exit(1); }
+    const extra = res.alsoArchived ? ` (and ${res.alsoArchived} subtask${res.alsoArchived === 1 ? "" : "s"})` : "";
+    console.log(`Archived ${id}${extra}. Restore with: open-claudia task restore ${id}`);
+    process.exit(0);
+  } catch (e) { console.error(e.message); process.exit(1); }
+}
+
+async function runRestore(args) {
+  const id = args[0];
+  if (!id) { console.error("Usage: task restore <id>"); process.exit(2); }
+  try {
+    const res = await postJson("task-restore", { id });
+    if (!res.ok) { console.error(`Not in the archive: ${id}`); process.exit(1); }
+    const extra = res.alsoRestored ? ` (and ${res.alsoRestored} subtask${res.alsoRestored === 1 ? "" : "s"})` : "";
+    console.log(`Restored ${id}${extra}.`);
+    process.exit(0);
+  } catch (e) { console.error(e.message); process.exit(1); }
+}
+
+async function runArchived(args) {
+  const limit = takeFlag(args, "limit");
+  const search = takeFlag(args, "search");
+  try {
+    const res = await postJson("task-archived", { ...(limit ? { limit: Number(limit) } : {}), ...(search ? { search } : {}) });
+    const list = res.tasks || [];
+    if (list.length === 0) { console.log("No archived tasks."); process.exit(0); }
+    for (const t of list) {
+      const when = t.archivedAt ? new Date(t.archivedAt).toISOString().slice(0, 10) : "?";
+      console.log(`${statusMark(t)} ${t.content}    (${t.id}) · archived ${when} · ${t.closedReason || "?"}: ${t.reason || ""}`);
+      (t.children || []).forEach((c) => console.log(`    ${statusMark(c)} ${c.content}    (${c.id})`));
+    }
+    process.exit(0);
+  } catch (e) { console.error(e.message); process.exit(1); }
+}
+
 async function run(args) {
   const sub = args[0];
   const rest = args.slice(1);
@@ -180,6 +228,9 @@ async function run(args) {
     case "pending": return runUpdate(rest, "pending");
     case "remove": case "rm": return runRemove(rest);
     case "clear-completed": case "clear": return runClearCompleted();
+    case "archive": return runArchive(rest);
+    case "restore": return runRestore(rest);
+    case "archived": return runArchived(rest);
     default:
       console.error(`Unknown task subcommand: ${sub}`);
       console.log(HELP);

package/bot.js

@@ -123,6 +123,16 @@ function persist() {
 async function gracefulShutdown(signal) {
   console.log(`Received ${signal}, shutting down gracefully...`);
   try { stopScheduler(); } catch (e) {}
+  // Save first: whoever sent SIGTERM kills the process soon after (bot pods
+  // allow 15 s, launchd 20 s), and the drain below must not cost the save.
+  persist();
+  // Planned restart (rollout, launchd stop): no new turns; running ones get
+  // DRAIN_SIGTERM_S (6) to finish, the rest stay journaled and resume after
+  // boot (core/turn-journal).
+  try {
+    const journal = require("./core/turn-journal");
+    await journal.drain({ timeoutMs: journal.sigtermDrainTimeoutMs() });
+  } catch (e) { require("./core/turn-journal").markShuttingDown(); }
   try { loopback.stop(); } catch (e) {}
   const sideCleanup = sideChatCoordinator.cancelAll().catch(() => 0);
   const memoryCleanup = require("./core/pack-review")
@@ -362,6 +372,15 @@ async function safeModeBoot() {
     }
   }
 
+  // Turns cut short by the last restart resume (or are offered back), and
+  // messages that were queued behind them are restored in order.
+  try {
+    const resumed = await require("./core/turn-journal").resumeInterrupted(adapters);
+    if (resumed.resumed.length || resumed.asked.length || resumed.refused.length || resumed.restored) {
+      console.log(`Interrupted turns: ${resumed.resumed.length} resumed, ${resumed.asked.length} offered, ${resumed.refused.length} not resumed, ${resumed.restored} queued restored`);
+    }
+  } catch (e) { console.error("interrupted-turn resume failed:", e.message); }
+
   // Kazee Cards: snoozed owner decisions are asked again when due.
   try { require("./core/cards").startDecisionPump(registry); } catch (e) { console.error("Decision pump start failed:", e.message); }