@bastani/atomic 0.9.44-alpha.5 → 0.9.44-alpha.6

package/CHANGELOG.md

@@ -2,6 +2,21 @@
 
 ## [Unreleased]
 
+## [0.9.44-alpha.6] - 2026-10-25
+
+Large workflow fan-outs stay fast for the whole run and recover cleanly from a crash, even with hundreds of pooled child workflows messaging each other.
+
+- **Journal writes that keep up:** transactions queued behind a commit share it, delivery receipts are batched, and pending-message changes for a run are written once per batch, so Intercom sends no longer time out behind journal traffic ([#3942](https://github.com/bastani-inc/atomic/pull/3942), [#3943](https://github.com/bastani-inc/atomic/pull/3943), [#3947](https://github.com/bastani-inc/atomic/pull/3947))
+- **Costs that stay flat as a run grows:** pool commits, delivery transitions, route announcements and receipt bookkeeping now cost what changed rather than the run's whole history, and each sticky delivery receipt is written once instead of repeatedly ([#3944](https://github.com/bastani-inc/atomic/pull/3944), [#3945](https://github.com/bastani-inc/atomic/pull/3945), [#3948](https://github.com/bastani-inc/atomic/pull/3948), [#3951](https://github.com/bastani-inc/atomic/pull/3951), [#3952](https://github.com/bastani-inc/atomic/pull/3952))
+- **Fast, reliable recovery:** `/workflow status` and resume load large journals in seconds, resume no longer rebuilds every child's history, and resumed pooled children no longer fail with `Pool checkpoint identity conflict` or a spurious `Storage unavailable` ([#3950](https://github.com/bastani-inc/atomic/pull/3950), [#3951](https://github.com/bastani-inc/atomic/pull/3951), [#3954](https://github.com/bastani-inc/atomic/pull/3954), [#3955](https://github.com/bastani-inc/atomic/pull/3955))
+- **Bounded delivery states:** route announcements, `intercom list`, `/workflow status` and inspect show at most 5 sticky delivery states per row plus totals ([#3951](https://github.com/bastani-inc/atomic/pull/3951))
+- **Paused stages resume their work:** a background completion that arrives while a paused stage resumes now starts its next turn ([#3953](https://github.com/bastani-inc/atomic/pull/3953))
+
+### Fixed
+
+- Under heavy write load a local SQLite store (`node-sqlite`, including the workflow journal) commits transactions in groups: transactions requested while another is committing share one `BEGIN IMMEDIATE … COMMIT`, each in its own savepoint and in request order. A transaction that fails rolls back only its own writes, and every caller still settles only after the shared commit. In a 480-child pooled workflow the pool coordinator was committing about 60 transcript transactions a second, each taking the write lock and syncing on its own, which starved the workflow host's writes until Intercom sends timed out and worker leases expired.
+- A background task completion that arrives just as a paused workflow stage is resumed now starts the stage's next turn. Pausing a stage cancels its background shells, and their completion notices are written to the session before the stage acts on them. If the stage was resumed during that write, the notice was queued after the resume had already released the stage's held messages, so the stage sat idle with the notice unanswered until something else started a turn.
+
 ## [0.9.44-alpha.5] - 2026-10-25
 
 Large workflow fan-outs keep their Intercom routes and journal writes responsive under load.

package/dist/builtin/intercom/CHANGELOG.md

@@ -4,6 +4,12 @@ All notable changes to the `pi-intercom` extension will be documented in this fi
 
 ## [Unreleased]
 
+## [0.9.44-alpha.6] - 2026-10-25
+
+### Changed
+
+- `intercom list` lists at most 5 sticky delivery states per workflow row: states that still need action first, then the most recent deliveries. A final line reports how many were omitted and the total in each state, and the row details carry the same totals as `deliveryStateCounts` and `omittedDeliveryStates`. In a 640-child workflow, each list result had stored about 3 MB of delivery states in the session transcript. A test with 8,000 queued messages in 163 rows shrinks the list result from about 4.6 MB to about 0.2 MB. The broker applies the same bound to rows announced by older workflow hosts.
+
 ## [0.9.44-alpha.5] - 2026-10-25
 
 ### Fixed

package/dist/builtin/intercom/broker/broker.ts

@@ -15,6 +15,7 @@ import {
   writeMessageIfOpen,
   writeMessageWithOutcome,
 } from "./socket-writes.js";
+import { boundWorkflowPossibleStageRow } from "../workflow-delivery-states.js";
 import { isSocketAnswering } from "./socket-liveness.js";
 import { acquireSocketReplacementLock } from "./socket-replacement-lock.js";
 import { startBrokerWatchdog } from "./watchdog.js";
@@ -303,6 +304,14 @@ function isWorkflowPossibleStageAnnouncements(
 				Number.isInteger(announcement.queuedCount) &&
 				announcement.queuedCount >= 0 &&
 				(announcement.deliveryOnly === undefined || announcement.deliveryOnly === true) &&
+				(announcement.omittedDeliveryStates === undefined ||
+					(Number.isInteger(announcement.omittedDeliveryStates) && announcement.omittedDeliveryStates >= 0)) &&
+				(announcement.deliveryStateCounts === undefined ||
+					(typeof announcement.deliveryStateCounts === "object" &&
+						announcement.deliveryStateCounts !== null &&
+						Object.values(announcement.deliveryStateCounts).every(
+							(count) => Number.isInteger(count) && count >= 0,
+						))) &&
 				(announcement.deliveryStates === undefined || (Array.isArray(announcement.deliveryStates) && announcement.deliveryStates.every((delivery) =>
 					delivery !== null && typeof delivery === "object" &&
 					typeof delivery.messageId === "string" && typeof delivery.runId === "string" &&
@@ -724,6 +733,8 @@ class IntercomBroker {
 					target: row.target,
 					queuedCount: row.queuedCount,
 					...(row.deliveryStates === undefined ? {} : { deliveryStates: row.deliveryStates }),
+					...(row.deliveryStateCounts === undefined ? {} : { deliveryStateCounts: row.deliveryStateCounts }),
+					...(row.omittedDeliveryStates === undefined ? {} : { omittedDeliveryStates: row.omittedDeliveryStates }),
 					...(row.deliveryOnly === undefined ? {} : { deliveryOnly: row.deliveryOnly }),
 					group: roster.group,
 				});
@@ -1913,9 +1924,10 @@ class IntercomBroker {
         // D7 (slice 4): presence replaces, absence keeps — a terminal root publishes `[]`
         // so the broker drops every future row, while nested runs' re-announcements (which
         // never carry the field) preserve the root's rows.
+        // Rows from hosts that announce every delivery state are bounded like current hosts' rows.
         const possibleStages =
           clientMessage.possibleStages !== undefined
-            ? clientMessage.possibleStages
+            ? clientMessage.possibleStages.map((row) => boundWorkflowPossibleStageRow(row))
             : this.workflowRosters.get(clientMessage.runId)?.possibleStages;
 		// Optional path metadata is advisory for old hosts; omission retains a
 		// previously announced edge during two-session route replay.

package/dist/builtin/intercom/index.bundle.mjs

@@ -4082,7 +4082,15 @@ function formatWorkflowStageRow(stage) {
 }
 function formatWorkflowFutureStageRow(stage) {
   return `- \`${stage.target}\` [${stage.deliveryOnly === true ? "delivery" : "future"}] ${stage.queuedCount} queued message${stage.queuedCount === 1 ? "" : "s"}${(stage.deliveryStates ?? []).map((delivery) => `
-  delivery ${delivery.messageId} stage ${delivery.runId}/${delivery.stageId}${delivery.sessionId === undefined ? "" : ` session ${delivery.sessionId}`} ${delivery.state} — ${delivery.reason}`).join("")}`;
+  delivery ${delivery.messageId} stage ${delivery.runId}/${delivery.stageId}${delivery.sessionId === undefined ? "" : ` session ${delivery.sessionId}`} ${delivery.state} — ${delivery.reason}`).join("")}${formatDeliveryStateSummary(stage)}`;
+}
+function formatDeliveryStateSummary(stage) {
+  const omitted = stage.omittedDeliveryStates ?? 0;
+  if (omitted === 0 || stage.deliveryStateCounts === undefined)
+    return "";
+  const counts = Object.entries(stage.deliveryStateCounts).map(([state, count]) => `${count} ${state}`).join(", ");
+  return `
+  … ${omitted} more delivery state${omitted === 1 ? "" : "s"} omitted (all: ${counts})`;
 }
 function registerIntercomTool(pi, deps) {
   const retryIdentities = new RetryIdentityReservations({ maxEntries: deps.retryIdentityMaxEntries });

package/dist/builtin/intercom/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@bastani/intercom",
-  "version": "0.9.44-alpha.5",
+  "version": "0.9.44-alpha.6",
   "private": true,
   "description": "Atomic extension providing a private coordination channel between parent and child agent sessions. Fork of: https://github.com/nicobailon/pi-intercom",
   "contributors": [

package/dist/builtin/intercom/types.ts

@@ -52,7 +52,12 @@ export interface WorkflowPossibleStageAnnouncement {
 	readonly target: string;
 	/** Current number of queued sticky entries matching this target. */
 	readonly queuedCount: number;
+	/** At most a fixed number of exemplar states: unresolved states first, then the latest delivered. */
 	readonly deliveryStates?: readonly WorkflowStickyDeliveryState[];
+	/** Delivery states per state, over every state of this row, including omitted ones. */
+	readonly deliveryStateCounts?: Readonly<Record<string, number>>;
+	/** Delivery states left out of `deliveryStates`. */
+	readonly omittedDeliveryStates?: number;
 	readonly deliveryOnly?: true;
 }
 
@@ -63,6 +68,8 @@ export interface WorkflowFutureStageRosterEntry {
 	readonly queuedCount: number;
 	readonly group: string;
 	readonly deliveryStates?: readonly WorkflowStickyDeliveryState[];
+	readonly deliveryStateCounts?: Readonly<Record<string, number>>;
+	readonly omittedDeliveryStates?: number;
 	readonly deliveryOnly?: true;
 }
 

package/dist/builtin/intercom/workflow-delivery-states.ts

@@ -0,0 +1,48 @@
+import type { WorkflowPossibleStageAnnouncement, WorkflowStickyDeliveryState } from "./types.js";
+
+/** Most delivery states one possible-stage row carries; the rest are counted, not listed. */
+export const WORKFLOW_DELIVERY_STATE_EXEMPLAR_LIMIT = 5;
+
+/** States that still need action, in the order exemplars are chosen; `delivered` follows, most recent first. */
+const UNRESOLVED_STATES = [
+	"queued",
+	"receipt-unverified",
+	"session-replaced",
+	"delivery-unavailable",
+	"root-unresolved",
+	"stage-terminal",
+];
+
+/**
+ * Bound a possible-stage row's delivery states as workflow hosts do, for rows from hosts that announce
+ * every state. Rows already within the limit pass through unchanged.
+ */
+export function boundWorkflowPossibleStageRow(
+	row: WorkflowPossibleStageAnnouncement,
+	limit = WORKFLOW_DELIVERY_STATE_EXEMPLAR_LIMIT,
+): WorkflowPossibleStageAnnouncement {
+	const states = row.deliveryStates;
+	if (states === undefined || states.length <= limit) return row;
+	const exemplars: WorkflowStickyDeliveryState[] = [];
+	for (const state of UNRESOLVED_STATES) {
+		for (const delivery of states) {
+			if (exemplars.length === limit) break;
+			if (delivery.state === state) exemplars.push(delivery);
+		}
+	}
+	for (let index = states.length - 1; index >= 0 && exemplars.length < limit; index--) {
+		if (states[index]!.state === "delivered") exemplars.push(states[index]!);
+	}
+	for (const delivery of states) {
+		if (exemplars.length === limit) break;
+		if (delivery.state !== "delivered" && !UNRESOLVED_STATES.includes(delivery.state)) exemplars.push(delivery);
+	}
+	const counts = new Map<string, number>();
+	for (const delivery of states) counts.set(delivery.state, (counts.get(delivery.state) ?? 0) + 1);
+	return {
+		...row,
+		deliveryStates: exemplars,
+		deliveryStateCounts: row.deliveryStateCounts ?? Object.fromEntries(counts),
+		omittedDeliveryStates: (row.omittedDeliveryStates ?? 0) + states.length - exemplars.length,
+	};
+}

package/dist/builtin/mcp/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@bastani/mcp",
-  "version": "0.9.44-alpha.5",
+  "version": "0.9.44-alpha.6",
   "private": true,
   "description": "Atomic's pi-compatible MCP integration for server connections, OAuth, tools, resources, and terminal management.",
   "contributors": [

package/dist/builtin/subagents/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@bastani/subagents",
-  "version": "0.9.44-alpha.5",
+  "version": "0.9.44-alpha.6",
   "private": true,
   "description": "Atomic extension for delegating tasks to subagents with parallel execution. Fork of: https://github.com/nicobailon/pi-subagents",
   "contributors": [

package/dist/builtin/web-access/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@bastani/web-access",
-  "version": "0.9.44-alpha.5",
+  "version": "0.9.44-alpha.6",
   "private": true,
   "description": "Atomic extension for web search, URL fetching, GitHub repo cloning, PDF/video extraction. Fork of: https://github.com/nicobailon/pi-web-access",
   "contributors": [

package/dist/builtin/workflows/CHANGELOG.md

@@ -6,6 +6,26 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
 
 ## [Unreleased]
 
+## [0.9.44-alpha.6] - 2026-10-25
+
+### Fixed
+
+- After a pooled root was killed and resumed, some child workflows no longer fail with `Extension startup failed: …intercom…: Storage unavailable: custom at custom journal`. While a resumed root and its workers were busy, a worker's lease could run out a moment before the pool extended it. When the root then recorded that a queued Intercom message had reached the worker's stage, it refused the write as if the attempt had been replaced and reported the refusal as unavailable storage, so the stage failed at startup. In a 640-child run this failed 21 children. The pool already keeps a live worker's child when its lease runs out (#3888), and the root now does the same when recording deliveries. Only a reassigned attempt or a new root epoch is refused, with `Stale work lease` or the session reason, and the replaced attempt's child is replayed. A failure of a custom journal now also names the cause's error class and code, though not its message, which may contain credentials.
+- A long-running workflow fan-out whose stages keep sending Intercom messages to stage patterns (for example `workflow:<rootRunId>/child-*/**`) no longer slows down with every message it has sent. Pattern messages stay queued until the root run ends, and each delivery receipt made the workflow host re-check every queued message and every recorded delivery against every run. In a 480-child load test this saturated the host, Intercom sends timed out after 10 s, and about half of them were never delivered. A delivery now re-checks only the messages it changed and the messages delivered to stages that have since ended: with 3,000 queued messages, recording one delivery takes about 1.2 ms instead of 6.7 ms. Delivery results are unchanged.
+- An Intercom send to a workflow stage no longer times out when a wave of stages starts at the same time. Every pending-message change for a run used to wait for the previous one's journal write, so a send that arrived while about 160 starting stages recorded their deliveries waited behind about 160 writes. Changes that arrive while a write is in progress are now applied together and saved in one write. In a test with a 40 ms write, a send issued in the middle of a 160-stage burst returned after 2 writes (about 90 ms) instead of 81 (about 3.3 s).
+- Large workflow fan-outs no longer saturate the local worker pool's coordinator. Every pool commit, including each batch of session transcript entries, was checked for JSON-serializability with a recursive schema whose `$ref` evaluation cost far more than the write itself, so with 160 workers the coordinator spent about 90% of its time validating and stages stopped starting. Plain JSON values now pass a direct structural check; anything unusual still goes through the schema, with the same errors.
+- Intercom sends to stage patterns no longer time out while a large workflow fan-out records delivery receipts. The workflow host wrote each receipt in its own journal transaction, one after another, and a pattern message waited behind every receipt queued before it. When another process also held the journal's write lock, that wait exceeded the 10-second send acknowledgement, and in a 480-child load test delivery fell to 0–2%. Receipts and settlements queued back to back for a run are now written together in one transaction, in the same order and with the same durability. In a test that queues a pattern message halfway through 2,000 receipts from 160 callers while a second process holds the write lock, the message persisted in under 30 ms instead of more than 4 seconds.
+- Resuming a large workflow with many completed child workflows no longer takes minutes before any work restarts. Resume rebuilt every nested run from the whole checkpoint journal once for each cached child, and each rebuild itself rescanned the journal for every child boundary. Resume now rebuilds the nested runs once, reuses them while replay records only its own bookkeeping, and rebuilds each in a single pass. Resuming a pooled root with 640 completed children went from 156 s to 1.3 s (64 children: 271 ms to 37 ms), and resumed runs reach the same state as before.
+- The local worker pool coordinator no longer slows down as a long pooled run grows. Each checkpoint a worker committed was validated against every checkpoint the root had ever recorded, and each commit re-added every usage report, so after about 15 minutes of a 640-child run the coordinator sat near full CPU and Intercom sends began to fail. Validation now reads only the checkpoints recorded since the previous commit, and the root's usage total is kept up to date as usage is written. Validation results and totals are unchanged. With 5,000 recorded checkpoints, a commit takes about 1.2 ms instead of 16 ms, and reading the usage total for 40,000 usage rows takes 0.02 ms instead of 8 ms.
+- Late in a long workflow fan-out, Intercom sends no longer time out while the host announces workflow routes or records delivery receipts. Each root route announcement recomputed every queued pattern message's delivery state on every matching stage, and every recorded receipt re-read the delivery count of every queued message. Both grow for the whole run (about 7,700 queued messages after 21 minutes in a 640-child load test), so one announcement could block the host for seconds. Announcements and receipts now recompute only what changed. With 8,000 queued messages, an announcement after one delivery makes 2 receipt checks instead of 16,000, and recording a receipt reads 1 delivery count instead of 8,000. Delivery counts are unchanged.
+- `/workflow status` from a new session, and resuming a workflow, no longer stall on a journal that holds many delivery receipts. Loading the journal re-read the delivery count of every queued message for every receipt. After a 640-child run with about 800,000 receipts, status had not rendered after 3 minutes. In a test with 200,000 receipts over 8,000 queued messages, opening and loading the run now takes about 3.4 s instead of 168 s.
+- Each delivery of a pattern (sticky) Intercom message to a workflow stage is now written to the run's journal once. When a delivered stage ended, the workflow host archived that delivery's receipt together with every earlier receipt still kept for the message. Those earlier receipts stayed kept, so they were written again each time a later stage ended. In pooled load-test journals about 93% of delivery receipts were exact copies, with one receipt written up to 423 times, and each copy took a slot in the run's journal writes. Receipts that are already archived are no longer written again. Delivery counts are unchanged: the copies never raised the recorded count.
+- A pooled child workflow resumed more than once no longer fails with `Pool checkpoint identity conflict`. Replaying a completed stage records a small replay-metadata checkpoint under an id derived from the stage, stamped with the time of that replay. The first resume recorded it. When a later resume, or a replacement worker, replayed the stage again, the pool coordinator compared the new record byte for byte with the stored one, saw a different time and failed the child. In a 640-child load test, 31 children failed this way after one SIGKILL and resume. A re-recorded checkpoint that differs only in its record time now keeps the first write. A checkpoint with any other difference is still rejected.
+
+### Changed
+
+- Route announcements carry at most 5 sticky delivery states per possible-stage row, with per-state totals and the number omitted. Each state leaves out its target, which the row already names. Workflow status and run inspection list at most 5 delivery states per run in the same way, with the same omitted-states line. Previously each announcement, `intercom list` result and status result carried every delivery state. With 8,000 queued messages across 163 rows, an announcement shrinks from about 4.6 MB to about 0.19 MB.
+
 ## [0.9.44-alpha.5] - 2026-10-25
 
 ### Fixed

package/dist/builtin/workflows/builtin/{chunk-zjecnjxx.js → chunk-6901nsd6.js}

@@ -39,7 +39,7 @@ import {
   reviewerFailureText,
   summarizeReviewConvergence,
   reverify_consolidated_batch
-} from "./chunk-k8d7ehjx.js";
+} from "./chunk-jvna28mq.js";
 import {
   fold_usage
 } from "./chunk-tgt0s5e5.js";