pi-task-dispatcher 3.0.2 → 3.1.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -10,7 +10,7 @@ Each delegation runs in an isolated, in-memory Pi RPC session. After every cohes
10
10
  - **Enforced supervision:** a private parent-child IPC gate calls `ctx.abort()` after persisted tool results; pausing does not depend on the worker following a prompt.
11
11
  - **Explicit control:** continue, revise, recalculate, compact, checkpoint, cancel, or take over.
12
12
  - **Resumable context:** continuation uses the same in-memory worker session and completed tool results.
13
- - **Least privilege:** every start selects an exact isolated-worker tool allowlist; delegation tools and unrelated discovered extensions are unavailable to the worker. Pi's built-in `codemode` extension loads only when explicitly selected.
13
+ - **Least privilege:** every start selects an exact isolated-worker tool allowlist; delegation tools and unrelated discovered extensions are unavailable to the worker. When explicitly selected, a runtime-temporary extension loads Pi's public native `codemode` factory in `only` mode; it is never injected otherwise.
14
14
  - **Model agnostic:** every role inherits Pi's active model and thinking level unless an explicit role override is configured.
15
15
  - **Formula-owned budgets:** the manager supplies semantic workload units, while the dispatcher calculates turn and time ceilings from workload, role, and delegated-model limits.
16
16
  - **Context insight:** boundary reports include worker context usage, compactions, turns, elapsed time, action counts, and recommendations.
@@ -19,11 +19,11 @@ Each delegation runs in an isolated, in-memory Pi RPC session. After every cohes
19
19
 
20
20
  ## Requirements
21
21
 
22
- - Pi 1.0.4 or later
22
+ - Pi 1.1.0 or later
23
23
  - Node.js 22.19.0 or later
24
24
  - One active, authenticated Pi model; optional role-specific models must also be available and authenticated
25
25
 
26
- The supervision gate requires the parent Pi process and child Pi CLI to run under Node.js with an inherited IPC channel. The dispatcher fails closed when that boundary cannot be established; it does not fall back to prompt-only supervision. Worker processes disable ordinary extension discovery and load the generated supervision gate plus Pi's built-in `codemode` extension only when that tool was explicitly selected.
26
+ The supervision gate requires the parent Pi process and child Pi CLI to run under Node.js with an inherited IPC channel. The dispatcher fails closed when that boundary cannot be established; it does not fall back to prompt-only supervision. Worker processes disable ordinary extension discovery and load the generated supervision gate plus a generated native codemode-only extension only when that tool was explicitly selected.
27
27
 
28
28
  ## Installation
29
29
 
@@ -67,6 +67,8 @@ Every role tool requires:
67
67
  - `tools`: the smallest exact allowlist selected from Pi's built-in tools and active for the main agent;
68
68
  - `workload`: semantic estimates used by the deterministic budget formula.
69
69
 
70
+ When `codemode` is active and delegated work materially benefits from JavaScript orchestration, batched/parallel/filtering of selected tools, or classifier/image models, include it in the delegated allowlist rather than reserving it for the manager. Also include each direct tool its script must call; otherwise omit `codemode`.
71
+
70
72
  ```json
71
73
  {
72
74
  "task": "Inspect the parser cache, correct its invalidation, and run the focused tests.",
@@ -89,7 +91,7 @@ A workload unit is intentionally semantic:
89
91
 
90
92
  At least one investigation, change, or verification unit is required.
91
93
 
92
- Isolated workers accept `read`, `bash`, `powershell`, `edit`, `write`, `grep`, `find`, `ls`, and `codemode` when the selected tools are also active in the main Pi session. `codemode` is loaded as an explicit Pi built-in while ordinary extension discovery remains disabled. Its scripts can call only the worker's callable tools and non-LLM model catalog; unrelated manager extensions and delegation tools remain unavailable.
94
+ Isolated workers accept `read`, `bash`, `powershell`, `edit`, `write`, `grep`, `find`, `ls`, and `codemode` when the selected tools are also active in the main Pi session. If `codemode` is selected, the child generates a temporary extension using Pi's public `createCodemodeExtension({ mode: "only" })` API while ordinary extension discovery remains disabled. This makes `codemode` model-visible without exposing selected direct tools to the worker model; its scripts can still call those selected callable tools and the non-LLM model catalog. Unrelated manager extensions and delegation tools remain unavailable.
93
95
 
94
96
  The current formula derives planned work turns from those units, applies a role factor and a bounded scale based on the selected delegated model's own context window, reserves two additional turns for synthesis or recovery, then derives active execution time from calculated turns plus expected long-running time. Administrative ceilings always win. Formula inputs and results are returned in every state report so they can be reviewed and recalculated. Time spent paused for manager deliberation does not consume the active execution budget.
95
97
 
@@ -146,9 +148,10 @@ A paused result contains:
146
148
  - active elapsed time and turns against calculated budgets;
147
149
  - context tokens, context window, percentage, and compaction counts;
148
150
  - a recommended next decision;
149
- - bounded worker text, if any.
151
+ - bounded model-facing worker text, if any;
152
+ - a complete sanitized live presentation ledger in structured result details, including streamed assistant text and child-tool lifecycle/output placeholders without image base64.
150
153
 
151
- The TUI renders the same structured state as a compact block. Expanding the tool result reveals bounded worker text. The complete action ledger and worker text remain in result details for state reconstruction rather than being discarded by display truncation.
154
+ The TUI renders the same structured state as a compact block plus the latest five visual rows of the presentation ledger. When older rows are omitted, Pi's native tool-expansion key hint appears; expanding the tool result reveals the complete captured presentation text, not the 5,000-character model-facing worker-text preview. The complete action ledger, presentation ledger, and worker text remain in result details for state reconstruction rather than being discarded by display truncation.
152
155
 
153
156
  After the first worker state, one unindented, dim footer status remains below Pi's primary report and updates across worker lifecycle states. It retains the newest state observed for every invoked role, with the most recently updated role first and each entry ordered as identity, context, model, and thinking—for example, `engineer-2 · 10.3%/32k • model-name • medium | mechanic-1 · 8.1%/128k • smaller-model • low`. Each percentage and window comes from that delegation's own fresh worker session; a continuation accumulates context in the same job, while a new delegation starts a new context. Current or latest action details appear in the delegation or manager block rather than the footer.
154
157
 
@@ -208,7 +211,7 @@ Version 3 intentionally ignores the version 2 fields `maxTurns`, `timeoutSeconds
208
211
  - Session shutdown or extension reload disposes the child and runtime-controlled temporary files.
209
212
  - Worker process failures reject pending RPC operations and produce a failed state.
210
213
  - Worker role instructions and the supervision gate exist only in a runtime-controlled temporary directory.
211
- - The selected built-in `codemode` extension may create runtime-temporary image outputs; the designer must persist requested outputs explicitly before completion.
214
+ - The selected native codemode-only extension may create runtime-temporary image outputs; the designer must persist requested outputs explicitly before completion.
212
215
  - The worker uses `--no-session`; no worker session file is persisted.
213
216
 
214
217
  Process separation is not a security sandbox. Worker tools execute with the Pi host's operating-system permissions. Tool availability does not establish task scope or authorization.
@@ -220,7 +223,7 @@ npm run check
220
223
  npm run test:gate
221
224
  ```
222
225
 
223
- `check` runs static release invariants and deterministic fake-RPC lifecycle and renderer scenarios. The fake child verifies all role registrations, pause/resume, output-limit recovery, usage deltas, nested codemode usage accounting, media metadata without base64 retention, persistent per-role footer state, full action-ledger retention, intercepted prompts, missing-boundary failure, fail-closed timeouts, pre-aborted operations, partial argument rendering, stable repeated rendering, and manager-state transitions without a model request.
226
+ `check` runs static release invariants and deterministic fake-RPC lifecycle and renderer scenarios. The fake child verifies all role registrations, pause/resume, output-limit recovery, usage deltas, native codemode-only loading and nested direct-tool usage accounting, media metadata without base64 retention, streaming assistant/tool replacement semantics, five-visual-row previews with native expansion hints, complete expanded presentation beyond the 5,000-character model preview, persistent per-role footer state, full action-ledger retention, intercepted prompts, missing-boundary failure, fail-closed timeouts, pre-aborted operations, partial argument rendering, stable repeated rendering, and manager-state transitions without a model request.
224
227
 
225
- `test:gate` performs a small live model integration check with the active parent model exported by Pi through `PI_PROVIDER`, `PI_MODEL`, and `PI_REASONING_LEVEL`, falling back to Pi's configured default when those values are unavailable; it contains no provider or model assumption. Optional `PI_TASK_DISPATCHER_GATE_MODEL` and `PI_TASK_DISPATCHER_GATE_THINKING` overrides are available for explicit test matrices. The probe verifies selective built-in `codemode` loading, a private pause after its completed tool batch, and subsequent completion in the same in-memory child. It requires configured credentials and therefore is not a hermetic unit test.
228
+ `test:gate` performs a small live model integration check with the active parent model exported by Pi through `PI_PROVIDER`, `PI_MODEL`, and `PI_REASONING_LEVEL`, falling back to Pi's configured default when those values are unavailable; it contains no provider or model assumption. Optional `PI_TASK_DISPATCHER_GATE_MODEL` and `PI_TASK_DISPATCHER_GATE_THINKING` overrides are available for explicit test matrices. The probe verifies generated native codemode-only loading, a codemode script calling an explicitly selected direct `read` tool, a private pause after that completed batch, and subsequent completion in the same in-memory child. It requires configured credentials and therefore is not a hermetic unit test.
226
229
 
@@ -10,8 +10,10 @@ import {
10
10
  type ExtensionAPI,
11
11
  type ExtensionToolContext,
12
12
  getAgentDir,
13
+ keyHint,
14
+ truncateToVisualLines,
13
15
  } from "@earendil-works/pi-coding-agent";
14
- import { Text } from "@earendil-works/pi-tui";
16
+ import { stripTerminalSequences, Text, truncateToWidth, type Component } from "@earendil-works/pi-tui";
15
17
  import { Type } from "typebox";
16
18
 
17
19
  type Role = "mechanic" | "assistant" | "engineer" | "designer";
@@ -99,6 +101,17 @@ interface WorkerAction {
99
101
  mediaOutputs?: WorkerMediaOutput[];
100
102
  }
101
103
 
104
+ /** Keep one replaceable presentation record for each assistant response or child tool lifecycle. */
105
+ interface WorkerPresentationEntry {
106
+ sequence: number;
107
+ id: string;
108
+ kind: "assistant" | "tool";
109
+ label: string;
110
+ status: "streaming" | "running" | "completed" | "failed";
111
+ text: string;
112
+ parentToolCallId?: string;
113
+ }
114
+
102
115
  interface ContextMetrics {
103
116
  tokens?: number;
104
117
  contextWindow: number;
@@ -135,6 +148,8 @@ interface WorkerSnapshot {
135
148
  recentActions: WorkerAction[];
136
149
  actionLedger: WorkerAction[];
137
150
  workerText: string;
151
+ /** Complete sanitized delegate activity retained only in structured tool details for UI expansion. */
152
+ presentationText: string;
138
153
  workerTextDisplay: string;
139
154
  workerTextTruncated: boolean;
140
155
  diagnostic?: string;
@@ -142,6 +157,21 @@ interface WorkerSnapshot {
142
157
  usage: UsageTotals;
143
158
  }
144
159
 
160
+ /** Retain only RPC content fields needed for text presentation and image metadata extraction. */
161
+ interface RpcContent {
162
+ type?: string;
163
+ text?: string;
164
+ data?: string;
165
+ mimeType?: string;
166
+ }
167
+
168
+ /** Model the result fields shared by partial and authoritative tool RPC events. */
169
+ interface RpcToolResult {
170
+ content?: RpcContent[];
171
+ usage?: EventUsage;
172
+ estimatedTokensAfter?: number;
173
+ }
174
+
145
175
  interface ParsedEvent {
146
176
  id?: string;
147
177
  type?: string;
@@ -154,14 +184,11 @@ interface ParsedEvent {
154
184
  aborted?: boolean;
155
185
  errorMessage?: string;
156
186
  willRetry?: boolean;
157
- result?: {
158
- content?: Array<{ type?: string; text?: string; data?: string; mimeType?: string }>;
159
- usage?: EventUsage;
160
- estimatedTokensAfter?: number;
161
- };
187
+ result?: RpcToolResult;
188
+ partialResult?: RpcToolResult;
162
189
  message?: {
163
190
  role?: string;
164
- content?: Array<{ type?: string; text?: string }>;
191
+ content?: RpcContent[];
165
192
  usage?: EventUsage;
166
193
  stopReason?: string;
167
194
  errorMessage?: string;
@@ -178,6 +205,7 @@ interface ParsedEvent {
178
205
  parentToolCallId?: string;
179
206
  args?: unknown;
180
207
  isError?: boolean;
208
+ durationMs?: number;
181
209
  }
182
210
 
183
211
  interface GateBoundaryMessage {
@@ -195,6 +223,8 @@ const CONFIG_PATH = TEST_MODE && process.env.PI_TASK_DISPATCHER_CONFIG_PATH
195
223
  const WORKER_ENV = "PI_TASK_DISPATCHER_WORKER";
196
224
  const WIDGET_ID = "pi-task-dispatcher-worker";
197
225
  const MODEL_TEXT_LIMIT = 5_000;
226
+ const PRESENTATION_PREVIEW_ROWS = 5;
227
+ const PRESENTATION_UPDATE_THROTTLE_MS = 100;
198
228
  const STDERR_LIMIT = 16_000;
199
229
  const THINKING_LEVELS = new Set<ThinkingLevel>(["off", "minimal", "low", "medium", "high", "xhigh", "max"]);
200
230
  const ROLES: Role[] = ["mechanic", "assistant", "engineer", "designer"];
@@ -253,9 +283,12 @@ const WorkloadSchema = Type.Object({
253
283
  expectedLongRunningSeconds: Type.Integer({ minimum: 0, maximum: 3_600 }),
254
284
  });
255
285
 
286
+ /** Tell managers when a codemode delegation grant is warranted without granting it automatically. */
287
+ const CODEMODE_SELECTION_GUIDANCE = "When codemode is active and delegated work materially benefits from JavaScript orchestration, batched/parallel/filtering of selected tools, or classifier/image models, include codemode in the delegated allowlist rather than reserving it for the manager; also include each direct tool the codemode script must call; otherwise omit it.";
288
+
256
289
  const DelegateParams = Type.Object({
257
290
  task: Type.String({ minLength: 1, maxLength: 12_000 }),
258
- tools: Type.Array(Type.String({ minLength: 1 }), { maxItems: 32 }),
291
+ tools: Type.Array(Type.String({ minLength: 1 }), { maxItems: 32, description: CODEMODE_SELECTION_GUIDANCE }),
259
292
  workload: WorkloadSchema,
260
293
  });
261
294
 
@@ -345,10 +378,17 @@ function headTail(text: string, limit = MODEL_TEXT_LIMIT): { text: string; trunc
345
378
  };
346
379
  }
347
380
 
381
+ /** Remove terminal controls from child-provided text before it enters a structured snapshot or renderer. */
382
+ function sanitizePresentationText(value: unknown): string {
383
+ if (typeof value !== "string") return "";
384
+ return stripTerminalSequences(value)
385
+ .replace(/\r\n?/g, "\n")
386
+ .replace(/[\u0000-\u0008\u000B\u000C\u000E-\u001F\u007F]/g, "");
387
+ }
388
+
348
389
  /** Keep one-line task and activity clues compact without introducing multiline omission markers. */
349
390
  function compactLine(value: unknown, limit = 180): string {
350
- if (typeof value !== "string") return "Preparing delegation…";
351
- const normalized = value.replaceAll(/\s+/g, " ").trim();
391
+ const normalized = sanitizePresentationText(value).replaceAll(/\s+/g, " ").trim();
352
392
  if (!normalized) return "Preparing delegation…";
353
393
  return normalized.length <= limit ? normalized : `${normalized.slice(0, Math.max(1, limit - 1))}…`;
354
394
  }
@@ -449,21 +489,53 @@ function getDirectPiInvocation(args: string[]): { command: string; args: string[
449
489
  /** Render a bounded activity label without embedding bulky command content. */
450
490
  function formatActivity(name: string, args: unknown): string {
451
491
  if (name === "bash" && isRecord(args) && typeof args.command === "string") {
452
- return `$ ${headTail(args.command, 140).text.replaceAll("\n", " ")}`;
492
+ return `$ ${headTail(sanitizePresentationText(args.command), 140).text.replaceAll("\n", " ")}`;
453
493
  }
454
- if (isRecord(args) && typeof args.path === "string") return `${name} ${args.path}`;
455
- return name;
494
+ if (isRecord(args) && typeof args.path === "string") return `${name} ${sanitizePresentationText(args.path)}`;
495
+ return sanitizePresentationText(name) || "tool";
456
496
  }
457
497
 
458
498
  /** Extract a text-only assistant report from one finalized RPC message. */
459
499
  function assistantText(event: ParsedEvent): string {
460
500
  return (event.message?.content ?? [])
461
- .filter((part) => part.type === "text" && part.text)
462
- .map((part) => part.text as string)
501
+ .filter((part) => part.type === "text" && typeof part.text === "string")
502
+ .map((part) => sanitizePresentationText(part.text))
503
+ .filter(Boolean)
463
504
  .join("\n")
464
505
  .trim();
465
506
  }
466
507
 
508
+ /** Extract text and image placeholders without retaining image base64 in state or rendered output. */
509
+ function presentationResult(result: RpcToolResult | undefined): { text: string; mediaOutputs: WorkerMediaOutput[] } {
510
+ const text: string[] = [];
511
+ const mediaOutputs: WorkerMediaOutput[] = [];
512
+ // Inspect content block types without persisting image payloads or redundant partial snapshots.
513
+ for (const part of result?.content ?? []) {
514
+ if (part.type === "text" && typeof part.text === "string") {
515
+ const value = sanitizePresentationText(part.text);
516
+ if (value) text.push(value);
517
+ continue;
518
+ }
519
+ if (part.type !== "image") continue;
520
+ const mimeType = sanitizePresentationText(part.mimeType) || "image";
521
+ const bytes = typeof part.data === "string" ? Buffer.from(part.data, "base64").length : 0;
522
+ if (typeof part.data === "string") mediaOutputs.push({ mimeType, bytes });
523
+ text.push(bytes > 0 ? `[image ${mimeType} · ${bytes} bytes]` : `[image ${mimeType}]`);
524
+ }
525
+ return { text: text.join("\n"), mediaOutputs };
526
+ }
527
+
528
+ /** Build the native codemode extension in only mode so selected direct tools stay script-callable but model-hidden. */
529
+ function codemodeOnlySource(): string {
530
+ return `/** Load Pi's public codemode factory without loading discovered extensions. */
531
+ import { createCodemodeExtension, type ExtensionAPI } from "@earendil-works/pi-coding-agent";
532
+ const codemodeOnly = createCodemodeExtension({ mode: "only" });
533
+ export default function codemodeOnlyExtension(pi: ExtensionAPI): void {
534
+ codemodeOnly(pi);
535
+ }
536
+ `;
537
+ }
538
+
467
539
  /** Build the worker-only gate extension that stops the agent loop after persisted tool results. */
468
540
  function gateSource(): string {
469
541
  return `/** Pause this worker after each persisted tool batch so only the parent dispatcher can resume it. */
@@ -520,6 +592,11 @@ class WorkerJob {
520
592
  private requestSequence = 0;
521
593
  private actionSequence = 0;
522
594
  private completionSequence = 0;
595
+ private presentationSequence = 0;
596
+ private readonly presentationEntries: WorkerPresentationEntry[] = [];
597
+ private readonly presentationById = new Map<string, WorkerPresentationEntry>();
598
+ private readonly assistantTextBlocks = new Map<number, string>();
599
+ private activeAssistantPresentation: WorkerPresentationEntry | undefined;
523
600
  private activeActions = new Map<string, WorkerAction>();
524
601
  private pendingRequests = new Map<string, { resolve: (event: ParsedEvent) => void; reject: (error: Error) => void }>();
525
602
  private settleWaiter: { resolve: () => void; reject: (error: Error) => void } | undefined;
@@ -530,6 +607,7 @@ class WorkerJob {
530
607
  private activeElapsedMs = 0;
531
608
  private activeStartedAt: number | undefined;
532
609
  private progressCallback: ((snapshot: WorkerSnapshot) => void) | undefined;
610
+ private progressUpdate: ReturnType<typeof setTimeout> | undefined;
533
611
  private pausedLease: ReturnType<typeof setTimeout> | undefined;
534
612
  private closePromise: Promise<void> | undefined;
535
613
 
@@ -566,8 +644,12 @@ class WorkerJob {
566
644
  this.temporaryDirectory = await fs.promises.mkdtemp(path.join(os.tmpdir(), "pi-task-dispatcher-"));
567
645
  const rolePromptPath = path.join(this.temporaryDirectory, `${this.role}.md`);
568
646
  const gatePath = path.join(this.temporaryDirectory, "supervision-gate.ts");
647
+ const codemodeOnlyPath = path.join(this.temporaryDirectory, "codemode-only.ts");
569
648
  await fs.promises.writeFile(rolePromptPath, ROLE_PROMPTS[this.role], { encoding: "utf8", mode: 0o600 });
570
649
  await fs.promises.writeFile(gatePath, gateSource(), { encoding: "utf8", mode: 0o600 });
650
+ if (this.tools.includes("codemode")) {
651
+ await fs.promises.writeFile(codemodeOnlyPath, codemodeOnlySource(), { encoding: "utf8", mode: 0o600 });
652
+ }
571
653
 
572
654
  const args = [
573
655
  "--mode", "rpc",
@@ -577,7 +659,7 @@ class WorkerJob {
577
659
  "--thinking", this.thinking,
578
660
  ...(this.tools.length > 0 ? ["--tools", this.tools.join(",")] : ["--no-tools"]),
579
661
  "--append-system-prompt", rolePromptPath,
580
- ...(this.tools.includes("codemode") ? ["--extension", "builtin:codemode"] : []),
662
+ ...(this.tools.includes("codemode") ? ["--extension", codemodeOnlyPath] : []),
581
663
  "--extension", gatePath,
582
664
  ];
583
665
  const invocation = getDirectPiInvocation(args);
@@ -681,6 +763,7 @@ class WorkerJob {
681
763
  /** Return a stable state projection without exposing process handles or raw RPC records. */
682
764
  snapshot(): WorkerSnapshot {
683
765
  const display = headTail(this.workerText);
766
+ const presentationText = this.presentationText();
684
767
  const completed = this.actions.filter((action) => action.status === "completed").length;
685
768
  const failed = this.actions.filter((action) => action.status === "failed").length;
686
769
  const currentAction = [...this.actions].reverse().find((action) => action.status === "running");
@@ -714,6 +797,7 @@ class WorkerJob {
714
797
  recentActions,
715
798
  actionLedger: this.actions.map((action) => ({ ...action, mediaOutputs: action.mediaOutputs?.map((output) => ({ ...output })) })),
716
799
  workerText: this.workerText,
800
+ presentationText,
717
801
  workerTextDisplay: display.text,
718
802
  workerTextTruncated: display.truncated,
719
803
  diagnostic: this.diagnostic,
@@ -726,6 +810,8 @@ class WorkerJob {
726
810
  async dispose(): Promise<void> {
727
811
  if (this.closePromise) return await this.closePromise;
728
812
  this.clearPausedLease();
813
+ if (this.progressUpdate) clearTimeout(this.progressUpdate);
814
+ this.progressUpdate = undefined;
729
815
  this.closePromise = (async () => {
730
816
  const child = this.child;
731
817
  if (child && child.exitCode === null && child.signalCode === null) {
@@ -789,7 +875,7 @@ class WorkerJob {
789
875
  });
790
876
  }
791
877
 
792
- /** Reduce one complete JSONL record into requests, activity, usage, and settlement state. */
878
+ /** Reduce one complete JSONL record into requests, activity, usage, presentation, and settlement state. */
793
879
  private processLine(line: string): void {
794
880
  if (!line.trim()) return;
795
881
  let event: ParsedEvent;
@@ -806,7 +892,20 @@ class WorkerJob {
806
892
  }
807
893
  return;
808
894
  }
895
+ if (event.type === "message_start" && event.message?.role === "assistant") {
896
+ this.beginAssistantPresentation();
897
+ this.emitProgress();
898
+ return;
899
+ }
900
+ if (event.type === "message_update" && event.assistantMessageEvent) {
901
+ this.updateAssistantPresentation(event.assistantMessageEvent);
902
+ this.scheduleProgressUpdate();
903
+ return;
904
+ }
809
905
  if (event.type === "message_end" && event.message?.role === "assistant") {
906
+ const text = assistantText(event);
907
+ this.completeAssistantPresentation(text);
908
+ if (text) this.workerText = text;
810
909
  const stopReason = event.message.stopReason;
811
910
  const gateAbort =
812
911
  (stopReason === "aborted" || stopReason === "error") &&
@@ -815,10 +914,9 @@ class WorkerJob {
815
914
  this.turns++;
816
915
  this.lastStopReason = stopReason;
817
916
  addUsage(this.usage, event.message.usage ?? event.usage);
818
- const text = assistantText(event);
819
- if (text) this.workerText = text;
820
- if (event.message.errorMessage) this.diagnostic = event.message.errorMessage;
917
+ if (event.message.errorMessage) this.diagnostic = sanitizePresentationText(event.message.errorMessage);
821
918
  }
919
+ this.emitProgress();
822
920
  return;
823
921
  }
824
922
  if (event.type === "tool_execution_start" && event.toolCallId && event.toolName) {
@@ -834,23 +932,36 @@ class WorkerJob {
834
932
  };
835
933
  this.actions.push(action);
836
934
  this.activeActions.set(action.toolCallId, action);
935
+ this.ensureToolPresentation(event.toolCallId, event.toolName, event.args, event.parentToolCallId);
837
936
  this.emitProgress();
838
937
  return;
839
938
  }
939
+ if (event.type === "tool_execution_update" && event.toolCallId && event.toolName) {
940
+ const entry = this.ensureToolPresentation(event.toolCallId, event.toolName, event.args, event.parentToolCallId);
941
+ entry.status = "running";
942
+ entry.text = presentationResult(event.partialResult).text;
943
+ this.scheduleProgressUpdate();
944
+ return;
945
+ }
840
946
  if (event.type === "tool_execution_end" && event.toolCallId) {
947
+ const result = presentationResult(event.result);
841
948
  const action = this.activeActions.get(event.toolCallId);
949
+ const entry = this.ensureToolPresentation(
950
+ event.toolCallId,
951
+ event.toolName ?? action?.toolName ?? "tool",
952
+ event.args,
953
+ event.parentToolCallId ?? action?.parentToolCallId,
954
+ );
955
+ entry.status = event.isError ? "failed" : "completed";
956
+ entry.text = result.text;
842
957
  if (action) {
843
958
  action.status = event.isError ? "failed" : "completed";
844
959
  action.endedAt = Date.now();
845
960
  action.completionSequence = ++this.completionSequence;
846
- const mediaOutputs = (event.result?.content ?? []).flatMap((part) =>
847
- part.type === "image" && typeof part.mimeType === "string" && typeof part.data === "string"
848
- ? [{ mimeType: part.mimeType, bytes: Buffer.from(part.data, "base64").length }]
849
- : []
850
- );
851
- if (mediaOutputs.length > 0) {
852
- action.mediaOutputs = mediaOutputs;
853
- action.label += ` · ${mediaOutputs.length} image${mediaOutputs.length === 1 ? "" : "s"}`;
961
+ if (result.mediaOutputs.length > 0) {
962
+ action.mediaOutputs = result.mediaOutputs;
963
+ action.label += ` · ${result.mediaOutputs.length} image${result.mediaOutputs.length === 1 ? "" : "s"}`;
964
+ entry.label = action.label;
854
965
  }
855
966
  this.activeActions.delete(event.toolCallId);
856
967
  }
@@ -873,6 +984,96 @@ class WorkerJob {
873
984
  if (event.type === "agent_settled") this.settleWaiter?.resolve();
874
985
  }
875
986
 
987
+ /** Start a replaceable assistant entry so streamed deltas never duplicate retained presentation text. */
988
+ private beginAssistantPresentation(): WorkerPresentationEntry {
989
+ if (this.activeAssistantPresentation) return this.activeAssistantPresentation;
990
+ this.assistantTextBlocks.clear();
991
+ const entry: WorkerPresentationEntry = {
992
+ sequence: ++this.presentationSequence,
993
+ id: `assistant-${this.presentationSequence}`,
994
+ kind: "assistant",
995
+ label: "assistant",
996
+ status: "streaming",
997
+ text: "",
998
+ };
999
+ this.presentationEntries.push(entry);
1000
+ this.presentationById.set(entry.id, entry);
1001
+ this.activeAssistantPresentation = entry;
1002
+ return entry;
1003
+ }
1004
+
1005
+ /** Apply JSON/RPC assistant text deltas and completed-block replacements by content index. */
1006
+ private updateAssistantPresentation(update: NonNullable<ParsedEvent["assistantMessageEvent"]>): void {
1007
+ if (!new Set(["text_start", "text_delta", "text_end"]).has(update.type ?? "")) return;
1008
+ const entry = this.beginAssistantPresentation();
1009
+ const index = typeof update.contentIndex === "number" ? update.contentIndex : 0;
1010
+ if (update.type === "text_start") this.assistantTextBlocks.set(index, "");
1011
+ else if (update.type === "text_delta") {
1012
+ this.assistantTextBlocks.set(index, `${this.assistantTextBlocks.get(index) ?? ""}${sanitizePresentationText(update.delta)}`);
1013
+ } else {
1014
+ // JSON/RPC text_end content is authoritative for this streamed block.
1015
+ this.assistantTextBlocks.set(index, sanitizePresentationText(update.content));
1016
+ }
1017
+ entry.text = [...this.assistantTextBlocks.entries()]
1018
+ .sort(([left], [right]) => left - right)
1019
+ .map(([, value]) => value)
1020
+ .filter(Boolean)
1021
+ .join("\n")
1022
+ .trim();
1023
+ }
1024
+
1025
+ /** Replace the live assistant entry with the authoritative final assistant message. */
1026
+ private completeAssistantPresentation(text: string): void {
1027
+ const entry = this.activeAssistantPresentation ?? (text ? this.beginAssistantPresentation() : undefined);
1028
+ if (!entry) return;
1029
+ entry.text = text;
1030
+ entry.status = "completed";
1031
+ this.assistantTextBlocks.clear();
1032
+ this.activeAssistantPresentation = undefined;
1033
+ }
1034
+
1035
+ /** Reuse one ordered presentation record per tool call while preserving child-call relationships. */
1036
+ private ensureToolPresentation(toolCallId: string, toolName: string, args: unknown, parentToolCallId?: string): WorkerPresentationEntry {
1037
+ const id = `tool-${toolCallId}`;
1038
+ const existing = this.presentationById.get(id);
1039
+ if (existing) return existing;
1040
+ const action = this.activeActions.get(toolCallId);
1041
+ const entry: WorkerPresentationEntry = {
1042
+ sequence: ++this.presentationSequence,
1043
+ id,
1044
+ kind: "tool",
1045
+ label: action?.label ?? formatActivity(toolName, args),
1046
+ status: "running",
1047
+ text: "",
1048
+ parentToolCallId,
1049
+ };
1050
+ this.presentationEntries.push(entry);
1051
+ this.presentationById.set(id, entry);
1052
+ return entry;
1053
+ }
1054
+
1055
+ /** Format retained presentation entries from their latest state without retaining repeated partial snapshots. */
1056
+ private presentationText(): string {
1057
+ return this.presentationEntries
1058
+ .map((entry) => {
1059
+ if (entry.kind === "assistant") {
1060
+ if (!entry.text) return "";
1061
+ return `${entry.status === "streaming" ? "assistant streaming" : "assistant"}:\n${entry.text}`;
1062
+ }
1063
+ const kind = entry.parentToolCallId ? "child tool" : "tool";
1064
+ const lines = [`${kind} start: ${entry.label}`];
1065
+ if (entry.status === "running") {
1066
+ if (entry.text) lines.push(`${kind} update: ${entry.label}`, entry.text);
1067
+ } else {
1068
+ lines.push(`${kind} ${entry.status}: ${entry.label}`);
1069
+ if (entry.text) lines.push(entry.text);
1070
+ }
1071
+ return lines.join("\n");
1072
+ })
1073
+ .filter(Boolean)
1074
+ .join("\n\n");
1075
+ }
1076
+
876
1077
  /** Execute one or more supervised action batches while respecting calculated ceilings. */
877
1078
  private async runStages(
878
1079
  initialMessage: string,
@@ -1082,8 +1283,19 @@ class WorkerJob {
1082
1283
  if (directory) await fs.promises.rm(directory, { recursive: true, force: true });
1083
1284
  }
1084
1285
 
1286
+ /** Coalesce token and partial-result bursts before rebuilding and serializing the complete snapshot. */
1287
+ private scheduleProgressUpdate(): void {
1288
+ if (this.progressUpdate) return;
1289
+ this.progressUpdate = setTimeout(() => {
1290
+ this.progressUpdate = undefined;
1291
+ this.emitProgress();
1292
+ }, PRESENTATION_UPDATE_THROTTLE_MS);
1293
+ }
1294
+
1085
1295
  /** Refresh the partial tool block with elapsed time and the latest active command. */
1086
1296
  private emitProgress(): void {
1297
+ if (this.progressUpdate) clearTimeout(this.progressUpdate);
1298
+ this.progressUpdate = undefined;
1087
1299
  try {
1088
1300
  this.progressCallback?.(this.snapshot());
1089
1301
  } catch {
@@ -1172,6 +1384,15 @@ function formatElapsed(durationMs: number): string {
1172
1384
  return seconds < 60 ? `${seconds}s` : `${Math.floor(seconds / 60)}m ${seconds % 60}s`;
1173
1385
  }
1174
1386
 
1387
+ /** Use Pi's configured expansion binding when interactive UI state exists, with a harmless noninteractive render fallback. */
1388
+ function expansionHint(theme: any): string {
1389
+ try {
1390
+ return keyHint("app.tools.expand", "to expand");
1391
+ } catch {
1392
+ return theme.fg("muted", "to expand");
1393
+ }
1394
+ }
1395
+
1175
1396
  /** Format one bounded state report for both model consumption and non-TUI modes. */
1176
1397
  function formatSnapshot(snapshot: WorkerSnapshot): string {
1177
1398
  const context = snapshot.context.percent === undefined
@@ -1233,29 +1454,144 @@ function renderArgumentSummary(value: unknown): { task: string; workload: string
1233
1454
  return { task: compactLine(value.task), workload: `${tools} tools · ${investigation}/${change}/${verification} I/C/V units` };
1234
1455
  }
1235
1456
 
1236
- /** Create a wrapping component whose text can reflect shared renderer state after execution updates. */
1237
- function dynamicText(build: () => string): { render(width: number): string[]; invalidate(): void } {
1238
- const text = new Text("", 0, 0);
1239
- let currentText: string | undefined;
1240
- return {
1241
- render(width) {
1242
- let nextText: string;
1243
- try {
1244
- nextText = build();
1245
- } catch {
1246
- nextText = "Preparing delegation…";
1247
- }
1248
- // Text.setText() invalidates the TUI, so call it only when observable content changed.
1249
- if (nextText !== currentText) {
1250
- currentText = nextText;
1251
- text.setText(nextText);
1457
+ /** Reuse dynamic text components while changing their source only outside the TUI render cycle. */
1458
+ class DynamicTextComponent {
1459
+ private readonly text = new Text("", 0, 0);
1460
+ private currentText: string | undefined;
1461
+ private build: () => string;
1462
+
1463
+ constructor(build: () => string) {
1464
+ this.build = build;
1465
+ }
1466
+
1467
+ setBuild(build: () => string): void {
1468
+ this.build = build;
1469
+ }
1470
+
1471
+ render(width: number): string[] {
1472
+ let nextText: string;
1473
+ try {
1474
+ nextText = this.build();
1475
+ } catch {
1476
+ nextText = "Preparing delegation…";
1477
+ }
1478
+ // Text.setText() invalidates the TUI, so call it only when observable content changed.
1479
+ if (nextText !== this.currentText) {
1480
+ this.currentText = nextText;
1481
+ this.text.setText(nextText);
1482
+ }
1483
+ return this.text.render(width);
1484
+ }
1485
+
1486
+ invalidate(): void {
1487
+ this.text.invalidate();
1488
+ }
1489
+ }
1490
+
1491
+ /** Reuse the call-renderer component from Pi's context instead of allocating one for every update. */
1492
+ function dynamicText(context: any, build: () => string, stateKey = "callComponent"): Component {
1493
+ const state = rendererState(context);
1494
+ const previous = context.lastComponent;
1495
+ const component = previous instanceof DynamicTextComponent
1496
+ ? previous
1497
+ : state[stateKey] instanceof DynamicTextComponent
1498
+ ? state[stateKey] as DynamicTextComponent
1499
+ : new DynamicTextComponent(build);
1500
+ component.setBuild(build);
1501
+ state[stateKey] = component;
1502
+ return component;
1503
+ }
1504
+
1505
+ /** Render the full presentation ledger with Pi's native visual-row truncation and expansion hint. */
1506
+ class DelegateResultComponent {
1507
+ private snapshot: WorkerSnapshot | undefined;
1508
+ private expanded = false;
1509
+ private isPartial = false;
1510
+ private standalone = false;
1511
+ private theme: any = undefined;
1512
+ private signature = "";
1513
+ private cachedWidth: number | undefined;
1514
+ private cachedLines: string[] | undefined;
1515
+
1516
+ update(snapshot: WorkerSnapshot, expanded: boolean, isPartial: boolean, theme: any, standalone: boolean): void {
1517
+ const presentation = snapshot.presentationText ?? "";
1518
+ const signature = [
1519
+ snapshot.jobId,
1520
+ snapshot.status,
1521
+ snapshot.taskDisplay,
1522
+ snapshot.phase,
1523
+ snapshot.sequence,
1524
+ snapshot.turns,
1525
+ snapshot.durationMs,
1526
+ snapshot.currentActivity ?? "",
1527
+ snapshot.actionCounts.completed,
1528
+ snapshot.actionCounts.failed,
1529
+ snapshot.recommendation,
1530
+ presentation,
1531
+ expanded,
1532
+ isPartial,
1533
+ standalone,
1534
+ ].join("\u0000");
1535
+ if (this.signature === signature && this.theme === theme) return;
1536
+ this.snapshot = snapshot;
1537
+ this.expanded = expanded;
1538
+ this.isPartial = isPartial;
1539
+ this.standalone = standalone;
1540
+ this.theme = theme;
1541
+ this.signature = signature;
1542
+ this.cachedWidth = undefined;
1543
+ this.cachedLines = undefined;
1544
+ }
1545
+
1546
+ render(width: number): string[] {
1547
+ if (this.cachedWidth === width && this.cachedLines) return this.cachedLines;
1548
+ const snapshot = this.snapshot;
1549
+ if (!snapshot) return [];
1550
+ const color = snapshot.status === "paused" ? "warning" : snapshot.status === "completed" ? "success" : isTerminal(snapshot.status) && snapshot.status !== "completed" ? "error" : "accent";
1551
+ const latestActivity = snapshot.currentActivity ?? snapshot.recentActions.at(-1)?.label;
1552
+ const activityLabel = snapshot.currentActivity ? "Now" : "Last";
1553
+ const header = [
1554
+ this.standalone ? this.theme.fg(color, this.theme.bold(`${snapshot.jobId}: ${snapshot.status}`)) : "",
1555
+ this.standalone ? this.theme.fg("muted", `Task: ${snapshot.taskDisplay}`) : "",
1556
+ latestActivity ? this.theme.fg(snapshot.currentActivity ? "warning" : "muted", `${activityLabel}: ${latestActivity}`) : "",
1557
+ this.theme.fg("dim", `${snapshot.phase} · boundary ${snapshot.sequence} · turn ${snapshot.turns}/${snapshot.budget.turns}`),
1558
+ this.theme.fg("muted", `${snapshot.actionCounts.completed} actions complete · ${snapshot.actionCounts.failed} failed · ${formatContext(snapshot)}`),
1559
+ this.theme.fg("accent", snapshot.recommendation),
1560
+ ].filter(Boolean).join("\n");
1561
+ const lines = new Text(header, 0, 0).render(width);
1562
+ const presentation = snapshot.presentationText ?? "";
1563
+ if (presentation) {
1564
+ // Reapply styling per line because Pi resets terminal attributes at every rendered row.
1565
+ const styledPresentation = presentation
1566
+ .split("\n")
1567
+ .map((line) => this.theme.fg("dim", line))
1568
+ .join("\n");
1569
+ if (this.expanded) {
1570
+ lines.push("", ...new Text(styledPresentation, 0, 0).render(width));
1571
+ } else {
1572
+ const preview = truncateToVisualLines(styledPresentation, PRESENTATION_PREVIEW_ROWS, width, 0, "end");
1573
+ if (preview.visualLines.length > 0) {
1574
+ lines.push("");
1575
+ if (preview.skippedCount > 0) {
1576
+ const hint = this.theme.fg("muted", `… ${preview.skippedCount} earlier visual rows omitted (`)
1577
+ + expansionHint(this.theme)
1578
+ + this.theme.fg("muted", ")");
1579
+ lines.push(truncateToWidth(hint, width, "..."));
1580
+ }
1581
+ lines.push(...preview.visualLines);
1582
+ }
1252
1583
  }
1253
- return text.render(width);
1254
- },
1255
- invalidate() {
1256
- text.invalidate();
1257
- },
1258
- };
1584
+ }
1585
+ lines.push(this.theme.fg("dim", `${this.isPartial ? "Elapsed" : "Took"}: ${formatElapsed(snapshot.durationMs)}`));
1586
+ this.cachedWidth = width;
1587
+ this.cachedLines = lines;
1588
+ return lines;
1589
+ }
1590
+
1591
+ invalidate(): void {
1592
+ this.cachedWidth = undefined;
1593
+ this.cachedLines = undefined;
1594
+ }
1259
1595
  }
1260
1596
 
1261
1597
  /** Render only nonredundant result details; the shared call header owns identity, task, and status. */
@@ -1266,25 +1602,20 @@ function renderWorkerResult(
1266
1602
  theme: any,
1267
1603
  context: any,
1268
1604
  standalone = false,
1269
- ): Text {
1605
+ ): Component {
1270
1606
  const snapshot = result.details as WorkerSnapshot | undefined;
1271
- if (!snapshot) return new Text(theme.fg(isPartial ? "warning" : "muted", result.content[0]?.text ?? "Delegation state unavailable"), 0, 0);
1272
- rendererState(context).snapshot = snapshot;
1273
- const color = snapshot.status === "paused" ? "warning" : snapshot.status === "completed" ? "success" : isTerminal(snapshot.status) && snapshot.status !== "completed" ? "error" : "accent";
1274
- const elapsedLabel = isPartial ? "Elapsed" : "Took";
1275
- const latestActivity = snapshot.currentActivity ?? snapshot.recentActions.at(-1)?.label;
1276
- const activityLabel = snapshot.currentActivity ? "Now" : "Last";
1277
- const lines = [
1278
- standalone ? theme.fg(color, theme.bold(`${snapshot.jobId}: ${snapshot.status}`)) : "",
1279
- standalone ? theme.fg("muted", `Task: ${snapshot.taskDisplay}`) : "",
1280
- latestActivity ? theme.fg(snapshot.currentActivity ? "warning" : "muted", `${activityLabel}: ${latestActivity}`) : "",
1281
- theme.fg("dim", `${snapshot.phase} · boundary ${snapshot.sequence} · turn ${snapshot.turns}/${snapshot.budget.turns}`),
1282
- theme.fg("muted", `${snapshot.actionCounts.completed} actions complete · ${snapshot.actionCounts.failed} failed · ${formatContext(snapshot)}`),
1283
- theme.fg("accent", snapshot.recommendation),
1284
- expanded && snapshot.workerTextDisplay ? `\n${snapshot.workerTextDisplay}` : "",
1285
- theme.fg("dim", `${elapsedLabel}: ${formatElapsed(snapshot.durationMs)}`),
1286
- ].filter(Boolean);
1287
- return new Text(lines.join("\n"), 0, 0);
1607
+ if (!snapshot) return dynamicText(context, () => theme.fg(isPartial ? "warning" : "muted", result.content[0]?.text ?? "Delegation state unavailable"), "resultTextComponent");
1608
+ const state = rendererState(context);
1609
+ state.snapshot = snapshot;
1610
+ const previous = context.lastComponent;
1611
+ const component = previous instanceof DelegateResultComponent
1612
+ ? previous
1613
+ : state.resultComponent instanceof DelegateResultComponent
1614
+ ? state.resultComponent as DelegateResultComponent
1615
+ : new DelegateResultComponent();
1616
+ component.update(snapshot, expanded, isPartial, theme, standalone);
1617
+ state.resultComponent = component;
1618
+ return component;
1288
1619
  }
1289
1620
 
1290
1621
  /** Package a snapshot as bounded model content plus complete machine-readable details. */
@@ -1316,6 +1647,7 @@ function registerDelegate(pi: ExtensionAPI, role: Role): void {
1316
1647
  ...(role === "assistant"
1317
1648
  ? ["When no other worker role applies, use delegate_assistant for the user-requested work instead of performing it directly. Retain delegation decisions, worker supervision, result integration, and the final response."]
1318
1649
  : []),
1650
+ CODEMODE_SELECTION_GUIDANCE,
1319
1651
  "Supply semantic workload units rather than raw turn or timeout values.",
1320
1652
  "Inspect every paused state and choose continue, revise, recalculate, compact, checkpoint, cancel, or take over.",
1321
1653
  "Worker reports remain evidence to review, not proof of correctness.",
@@ -1326,7 +1658,7 @@ function registerDelegate(pi: ExtensionAPI, role: Role): void {
1326
1658
  annotations: { readOnlyHint: false, destructiveHint: true, idempotentHint: false, openWorldHint: true },
1327
1659
  renderCall(args, theme, context) {
1328
1660
  const expectedId = `${role}-${nextJobIndex}`;
1329
- return dynamicText(() => {
1661
+ return dynamicText(context, () => {
1330
1662
  const snapshot = rendererState(context).snapshot as WorkerSnapshot | undefined;
1331
1663
  const current = renderArgumentSummary(context.args ?? args);
1332
1664
  const id = snapshot?.jobId ?? expectedId;
@@ -1388,7 +1720,7 @@ function registerControl(pi: ExtensionAPI): void {
1388
1720
  executionMode: "sequential",
1389
1721
  annotations: { readOnlyHint: false, destructiveHint: true, idempotentHint: false, openWorldHint: true },
1390
1722
  renderCall(args, theme, context) {
1391
- return dynamicText(() => {
1723
+ return dynamicText(context, () => {
1392
1724
  const snapshot = rendererState(context).snapshot as WorkerSnapshot | undefined;
1393
1725
  const current = isRecord(context.args) ? context.args : isRecord(args) ? args : {};
1394
1726
  const jobId = typeof current.jobId === "string" ? current.jobId : "delegation";
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "pi-task-dispatcher",
3
- "version": "3.0.2",
3
+ "version": "3.1.2",
4
4
  "description": "Run configurable Pi worker models as supervised, observable, resumable jobs.",
5
5
  "type": "module",
6
6
  "keywords": [
@@ -26,7 +26,7 @@
26
26
  "scripts": {
27
27
  "test:gate": "node tests/gate-probe.mjs",
28
28
  "test:fake-rpc": "node tests/fake-rpc-check.mjs",
29
- "check": "node tests/static-check.mjs && node tests/fake-rpc-check.mjs"
29
+ "check": "node tests/static-check.mjs && node tests/strict-jsonl-check.mjs && node tests/fake-rpc-check.mjs"
30
30
  },
31
31
  "peerDependencies": {
32
32
  "@earendil-works/pi-coding-agent": "*",
@@ -70,6 +70,14 @@ try {
70
70
  const control = tools.get("delegate_control");
71
71
  assert(start && designerStart && control, "Expected supervised tools to register.");
72
72
  assert(tools.has("delegate_mechanical") && tools.has("delegate_assistant"), "Expected every worker role to register.");
73
+ // Validate the shared least-privilege codemode guidance through every registered delegation surface.
74
+ const schemaGuidance = tools.get("delegate_mechanical").parameters.properties.tools.description;
75
+ assert.match(schemaGuidance, /codemode.*allowlist.*direct tool.*otherwise omit/i, "Schema guidance no longer describes the codemode delegation rule.");
76
+ for (const name of ["delegate_mechanical", "delegate_assistant", "delegate_engineering", "delegate_designer"]) {
77
+ const delegate = tools.get(name);
78
+ assert.equal(delegate.parameters.properties.tools.description, schemaGuidance, `${name} does not expose the shared codemode guidance in its tools schema.`);
79
+ assert(delegate.promptGuidelines.includes(schemaGuidance), `${name} does not expose the schema codemode guidance in its manager prompt metadata.`);
80
+ }
73
81
  const theme = { fg(_color, text) { return text; }, bold(text) { return text; } };
74
82
  let rendererInvalidations = 0;
75
83
  const renderContext = { args: {}, state: {}, invalidate() { rendererInvalidations++; }, cwd: projectRoot };
@@ -94,7 +102,10 @@ try {
94
102
  assert.match(partialRender, /Elapsed:/);
95
103
  assert.equal(statusUpdates.at(-1)?.text, "engineer-1 · 11.0%/1k • active-model • medium");
96
104
  const resultComponent = start.renderResult(result, { expanded: false, isPartial: false }, theme, renderContext);
97
- const firstRender = resultComponent.render(100).join("\n");
105
+ renderContext.lastComponent = resultComponent;
106
+ const reusedResultComponent = start.renderResult(result, { expanded: false, isPartial: false }, theme, renderContext);
107
+ assert.equal(reusedResultComponent, resultComponent, "The result renderer did not reuse Pi's last component.");
108
+ const firstRender = reusedResultComponent.render(100).join("\n");
98
109
  assert.match(firstRender, /Took:/);
99
110
  assert.doesNotMatch(firstRender, /Elapsed:/);
100
111
  for (let index = 0; index < 1_000; index++) {
@@ -124,7 +135,7 @@ try {
124
135
  assert.match(managerCall.render(100).join("\n"), /manager: engineer-1 → completed/);
125
136
  assert.doesNotMatch(managerCall.render(100).join("\n"), /continue\/completed/);
126
137
 
127
- // The designer must load only the selected built-in codemode extension and account nested usage once.
138
+ // The designer must load only the selected native codemode-only extension and account nested usage once.
128
139
  await writeFile(configPath, JSON.stringify({ designer: { model: "media-provider/media-model", thinking: "high" } }), "utf8");
129
140
  process.env.PI_TASK_DISPATCHER_FAKE_SCENARIO = "codemode";
130
141
  result = await designerStart.execute("start-designer", { task: "fake image job", tools: ["read", "codemode"], workload }, undefined, undefined, context);
@@ -158,6 +169,57 @@ try {
158
169
  assert.match(statusUpdates.at(-1)?.text ?? "", /designer-2.*\|.*engineer-1/);
159
170
  await writeFile(configPath, "{}\n", "utf8");
160
171
 
172
+ // Streaming RPC updates must replace their partial text, preserve full UI details, and keep model output bounded.
173
+ process.env.PI_TASK_DISPATCHER_FAKE_SCENARIO = "presentation";
174
+ const presentationUpdates = [];
175
+ result = await start.execute(
176
+ "start-presentation",
177
+ { task: "fake presentation job", tools: ["read"], workload },
178
+ undefined,
179
+ (update) => presentationUpdates.push(update.details?.presentationText ?? ""),
180
+ context,
181
+ );
182
+ assert(presentationUpdates.some((text) => text.includes("stream delta that must be replaced")), "Live assistant stream text was not presented.");
183
+ assert(presentationUpdates.some((text) => text.includes("partial read result that must be replaced")), "Live tool partial text was not presented.");
184
+ assert.equal(result.isError, false);
185
+ assert.equal(result.details.status, "paused");
186
+ assert.match(result.details.presentationText, /authoritative first assistant message/);
187
+ assert.match(result.details.presentationText, /tool start: read presentation\.txt/);
188
+ assert.match(result.details.presentationText, /tool completed: read presentation\.txt/);
189
+ assert.match(result.details.presentationText, /final read result six red/);
190
+ assert.doesNotMatch(result.details.presentationText, /stream delta that must be replaced|stream block that must be replaced|partial read result that must be replaced/);
191
+ assert.doesNotMatch(result.details.presentationText, /\u001b/, "Presentation text must not retain terminal control sequences.");
192
+ const presentationRenderContext = { args: {}, state: {}, invalidate() { rendererInvalidations++; }, cwd: projectRoot };
193
+ const collapsedComponent = start.renderResult(result, { expanded: false, isPartial: false }, theme, presentationRenderContext);
194
+ presentationRenderContext.lastComponent = collapsedComponent;
195
+ const collapsed = collapsedComponent.render(100).join("\n");
196
+ assert.match(collapsed, /earlier visual rows omitted/);
197
+ assert.match(collapsed, /to expand/);
198
+ assert.match(collapsed, /final read result six red/);
199
+ assert.doesNotMatch(collapsed, /authoritative first assistant message/, "Collapsed output must retain only the latest visual rows.");
200
+
201
+ result = await control.execute(
202
+ "continue-presentation",
203
+ { jobId: result.details.jobId, action: "continue", actionBatches: 1 },
204
+ undefined,
205
+ undefined,
206
+ context,
207
+ );
208
+ assert.equal(result.isError, false);
209
+ assert.equal(result.details.status, "completed");
210
+ assert(result.details.workerTextDisplay.length <= 5_000, "Model-facing worker text was not bounded.");
211
+ assert(result.details.presentationText.length > 5_000, "Structured presentation text was unexpectedly bounded.");
212
+ assert.match(result.details.presentationText, /MIDDLE-ONLY-IN-EXPANDED-PRESENTATION/);
213
+ assert.doesNotMatch(result.content[0].text, /MIDDLE-ONLY-IN-EXPANDED-PRESENTATION/, "Model-facing state exposed the omitted middle of a long report.");
214
+ const expandedComponent = control.renderResult(result, { expanded: true, isPartial: false }, theme, presentationRenderContext);
215
+ presentationRenderContext.lastComponent = expandedComponent;
216
+ const expanded = expandedComponent.render(100).join("\n");
217
+ assert.match(expanded, /MIDDLE-ONLY-IN-EXPANDED-PRESENTATION/);
218
+ assert.match(expanded, /latest-presentation-marker/);
219
+ const reusedExpandedComponent = control.renderResult(result, { expanded: true, isPartial: false }, theme, presentationRenderContext);
220
+ assert.equal(reusedExpandedComponent, expandedComponent, "Expanded renderer state was not reused.");
221
+ assert.equal(reusedExpandedComponent.render(100).join("\n"), expanded, "Repeated expanded rendering changed output.");
222
+
161
223
  // A tool-free output-limit stop must preserve the session so the manager can recover the report.
162
224
  process.env.PI_TASK_DISPATCHER_FAKE_SCENARIO = "length";
163
225
  result = await start.execute("start-length", { task: "fake truncated job", tools: ["read"], workload }, undefined, undefined, context);
@@ -1,7 +1,7 @@
1
1
  /** Deterministic JSONL/IPC worker used only by the dispatcher's lifecycle tests. */
2
2
 
3
- import { appendFile } from "node:fs/promises";
4
- import readline from "node:readline";
3
+ import { appendFile, readFile } from "node:fs/promises";
4
+ import { readStrictJsonl } from "./strict-jsonl.mjs";
5
5
 
6
6
  const scenario = process.env.PI_TASK_DISPATCHER_FAKE_SCENARIO ?? "normal";
7
7
  const markerPath = process.env.PI_TASK_DISPATCHER_FAKE_PROMPT_MARKER;
@@ -22,7 +22,8 @@ function usage(input, output) {
22
22
  };
23
23
  }
24
24
 
25
- readline.createInterface({ input: process.stdin }).on("line", async (line) => {
25
+ // Process strict LF-framed RPC commands in order, including asynchronous scenario handlers.
26
+ await readStrictJsonl(process.stdin, async (line) => {
26
27
  if (!line.trim()) return;
27
28
  const command = JSON.parse(line);
28
29
  if (command.type === "set_auto_compaction") {
@@ -54,17 +55,91 @@ readline.createInterface({ input: process.stdin }).on("line", async (line) => {
54
55
  return;
55
56
  }
56
57
 
57
- if (scenario === "codemode" && promptCount === 1) {
58
+ if (promptCount === 1) {
58
59
  const toolsIndex = process.argv.indexOf("--tools");
59
60
  const selectedTools = toolsIndex >= 0 ? process.argv[toolsIndex + 1]?.split(",") ?? [] : [];
60
61
  const extensionValues = process.argv.flatMap((argument, index) => argument === "--extension" ? [process.argv[index + 1]] : []);
61
- if (!selectedTools.includes("codemode") || !extensionValues.includes("builtin:codemode")) {
62
- send({ type: "response", id: command.id, command: command.type, success: false, error: "codemode was not isolated and loaded" });
62
+ if (scenario === "normal" && extensionValues.some((value) => value?.endsWith("codemode-only.ts"))) {
63
+ send({ type: "response", id: command.id, command: command.type, success: false, error: "codemode extension was injected without explicit selection" });
63
64
  return;
64
65
  }
66
+ if (scenario === "codemode") {
67
+ const nativeCodemodePath = extensionValues.find((value) => value?.endsWith("codemode-only.ts"));
68
+ let nativeCodemodeSource = "";
69
+ try {
70
+ if (nativeCodemodePath) nativeCodemodeSource = await readFile(nativeCodemodePath, "utf8");
71
+ } catch {
72
+ // The failed source check below reports a deterministic fake-RPC error.
73
+ }
74
+ if (
75
+ !selectedTools.includes("codemode")
76
+ || extensionValues.includes("builtin:codemode")
77
+ || !nativeCodemodePath
78
+ || !nativeCodemodeSource.includes('createCodemodeExtension({ mode: "only" })')
79
+ || !nativeCodemodeSource.includes('from "@earendil-works/pi-coding-agent"')
80
+ ) {
81
+ send({ type: "response", id: command.id, command: command.type, success: false, error: "native codemode-only extension was not isolated and loaded" });
82
+ return;
83
+ }
84
+ }
65
85
  }
66
86
 
67
87
  send({ type: "response", id: command.id, command: command.type, success: true, data: { disposition: "started" } });
88
+ if (scenario === "presentation") {
89
+ if (promptCount === 1) {
90
+ send({ type: "message_start", message: { role: "assistant" } });
91
+ send({ type: "message_update", assistantMessageEvent: { type: "text_start", contentIndex: 0 } });
92
+ send({ type: "message_update", assistantMessageEvent: { type: "text_delta", contentIndex: 0, delta: "stream delta that must be replaced" } });
93
+ // Keep the fake stream active long enough to exercise the dispatcher's coalesced live update.
94
+ await new Promise((resolve) => setTimeout(resolve, 150));
95
+ send({ type: "message_update", assistantMessageEvent: { type: "text_end", contentIndex: 0, content: "stream block that must be replaced" } });
96
+ send({
97
+ type: "message_end",
98
+ message: { role: "assistant", stopReason: "toolUse", content: [{ type: "text", text: "authoritative first assistant message" }], usage: usage(10, 2) },
99
+ });
100
+ send({ type: "tool_execution_start", toolCallId: "fake-presentation-read", toolName: "read", args: { path: "presentation.txt" } });
101
+ send({
102
+ type: "tool_execution_update",
103
+ toolCallId: "fake-presentation-read",
104
+ toolName: "read",
105
+ args: { path: "presentation.txt" },
106
+ partialResult: { content: [{ type: "text", text: "partial read result that must be replaced" }] },
107
+ });
108
+ // Keep the partial tool result observable before the authoritative result replaces it.
109
+ await new Promise((resolve) => setTimeout(resolve, 150));
110
+ send({
111
+ type: "tool_execution_end",
112
+ toolCallId: "fake-presentation-read",
113
+ toolName: "read",
114
+ isError: false,
115
+ result: {
116
+ content: [{ type: "text", text: "final read result one\nfinal read result two\nfinal read result three\nfinal read result four\nfinal read result five\nfinal read result six \u001b[31mred\u001b[0m" }],
117
+ usage: usage(3, 1),
118
+ },
119
+ });
120
+ process.send?.({ type: "task_dispatcher_boundary", turnIndex: 0, toolResults: 1 });
121
+ send({ type: "message_end", message: { role: "assistant", stopReason: "error", errorMessage: "This operation was aborted", content: [] } });
122
+ send({ type: "agent_settled" });
123
+ return;
124
+ }
125
+ const longReport = [
126
+ "presentation opening",
127
+ "A".repeat(2_600),
128
+ "MIDDLE-ONLY-IN-EXPANDED-PRESENTATION",
129
+ "B".repeat(4_000),
130
+ "latest-presentation-marker",
131
+ ].join("\n");
132
+ send({ type: "message_start", message: { role: "assistant" } });
133
+ send({ type: "message_update", assistantMessageEvent: { type: "text_start", contentIndex: 0 } });
134
+ send({ type: "message_update", assistantMessageEvent: { type: "text_delta", contentIndex: 0, delta: "partial final assistant text" } });
135
+ send({ type: "message_update", assistantMessageEvent: { type: "text_end", contentIndex: 0, content: "replaced final assistant text" } });
136
+ send({
137
+ type: "message_end",
138
+ message: { role: "assistant", stopReason: "stop", content: [{ type: "text", text: longReport }], usage: usage(5, 3) },
139
+ });
140
+ send({ type: "agent_settled" });
141
+ return;
142
+ }
68
143
  if (scenario === "length" && promptCount === 1) {
69
144
  send({
70
145
  type: "message_end",
@@ -6,7 +6,7 @@ import { spawn } from "node:child_process";
6
6
  import { mkdtemp, readFile, rm, writeFile } from "node:fs/promises";
7
7
  import { homedir, tmpdir } from "node:os";
8
8
  import path from "node:path";
9
- import readline from "node:readline";
9
+ import { readStrictJsonl } from "./strict-jsonl.mjs";
10
10
 
11
11
  // Resolve the managed Pi release without embedding a machine-specific installation path.
12
12
  async function resolvePiCli() {
@@ -22,7 +22,10 @@ async function resolvePiCli() {
22
22
  const cliPath = await resolvePiCli();
23
23
  const temporaryDirectory = await mkdtemp(path.join(tmpdir(), "pi-task-dispatcher-gate-probe-"));
24
24
  const gatePath = path.join(temporaryDirectory, "gate.ts");
25
+ const codemodeOnlyPath = path.join(temporaryDirectory, "codemode-only.ts");
26
+ const readProbePath = path.join(temporaryDirectory, "codemode-read-probe.txt");
25
27
  let child;
28
+ let stdoutRecords;
26
29
 
27
30
  try {
28
31
  // Abort the active worker run only after its complete tool batch has been persisted.
@@ -43,6 +46,18 @@ export default function (pi: ExtensionAPI): void {
43
46
  `,
44
47
  "utf8",
45
48
  );
49
+ // Use Pi's public factory so only codemode is model-visible while selected direct tools remain script-callable.
50
+ await writeFile(
51
+ codemodeOnlyPath,
52
+ `import { createCodemodeExtension, type ExtensionAPI } from "@earendil-works/pi-coding-agent";
53
+ const codemodeOnly = createCodemodeExtension({ mode: "only" });
54
+ export default function codemodeOnlyExtension(pi: ExtensionAPI): void {
55
+ codemodeOnly(pi);
56
+ }
57
+ `,
58
+ "utf8",
59
+ );
60
+ await writeFile(readProbePath, "native-codemode-only-read-ready\n", "utf8");
46
61
 
47
62
  // Prefer explicit test overrides, then the active parent Pi model exported in PI_* variables.
48
63
  const activeModel = process.env.PI_TASK_DISPATCHER_GATE_MODEL
@@ -55,9 +70,9 @@ export default function (pi: ExtensionAPI): void {
55
70
  "--no-session",
56
71
  "--no-extensions",
57
72
  "--tools",
58
- "codemode",
73
+ "codemode,read",
59
74
  "--extension",
60
- "builtin:codemode",
75
+ codemodeOnlyPath,
61
76
  "--extension",
62
77
  gatePath,
63
78
  ];
@@ -82,26 +97,32 @@ export default function (pi: ExtensionAPI): void {
82
97
  settledResolve = resolve;
83
98
  });
84
99
 
85
- // Collect public RPC events independently from the private boundary signal.
86
- readline.createInterface({ input: child.stdout }).on("line", (line) => {
100
+ // Collect public LF-framed RPC events independently from the private boundary signal.
101
+ stdoutRecords = readStrictJsonl(child.stdout, (line) => {
87
102
  if (!line.trim()) return;
88
103
  const event = JSON.parse(line);
89
104
  events.push(event);
90
105
  if (event.type === "agent_settled") settledResolve();
91
106
  });
107
+ // Surface malformed or unterminated child output during either gated wait without unhandled rejections.
108
+ const stdoutFramingFailure = new Promise((_, reject) => {
109
+ void stdoutRecords.catch(reject);
110
+ });
111
+ void stdoutFramingFailure.catch(() => undefined);
92
112
  child.stderr.on("data", (chunk) => {
93
113
  stderr += chunk.toString();
94
114
  });
95
115
  child.on("message", (message) => boundaries.push(message));
96
116
 
97
- // Run one codemode batch and require the gate to settle before a productive follow-up turn.
117
+ // Run one codemode batch that invokes the selected direct read tool from the script, then require the gate to settle.
98
118
  child.stdin.write(`${JSON.stringify({
99
119
  id: "probe-1",
100
120
  type: "prompt",
101
- message: "Call codemode exactly once with the raw JavaScript source: text('pi-task-dispatcher-codemode-ready')\nDo not use any other tool and do not add other script statements.",
121
+ message: `Call codemode exactly once. In its JavaScript source, call await tools.read({ path: ${JSON.stringify(readProbePath)} }); and then call text("pi-task-dispatcher-codemode-ready"). Do not call tools directly and do not make any other tool calls.`,
102
122
  })}\n`);
103
123
  await Promise.race([
104
124
  settled,
125
+ stdoutFramingFailure,
105
126
  new Promise((_, reject) => setTimeout(() => reject(new Error("First gated run did not settle.")), 120_000)),
106
127
  ]);
107
128
 
@@ -112,7 +133,13 @@ export default function (pi: ExtensionAPI): void {
112
133
  throw new Error("The first run did not execute the expected tool-use turn.");
113
134
  }
114
135
  if (!events.some((event) => event.type === "tool_execution_end" && event.toolName === "codemode" && !event.isError)) {
115
- throw new Error("The isolated worker did not complete the selected built-in codemode tool.");
136
+ throw new Error("The isolated worker did not complete the selected native codemode tool.");
137
+ }
138
+ if (!events.some((event) => event.type === "tool_execution_end" && event.toolName === "read" && event.parentToolCallId)) {
139
+ throw new Error("The codemode script could not call the explicitly selected direct read tool.");
140
+ }
141
+ if (events.some((event) => event.type === "tool_execution_start" && event.toolName === "read" && !event.parentToolCallId)) {
142
+ throw new Error("The worker called read directly instead of routing it through codemode-only mode.");
116
143
  }
117
144
 
118
145
  // Continue in the same in-memory RPC session and require a normal final answer without another tool batch.
@@ -123,10 +150,11 @@ export default function (pi: ExtensionAPI): void {
123
150
  child.stdin.write(`${JSON.stringify({
124
151
  id: "probe-2",
125
152
  type: "prompt",
126
- message: "Continue from the completed codemode result. Do not call tools. Report only the exact marker produced by the script.",
153
+ message: "Continue from the completed native codemode result. Do not call tools. Report only the exact marker produced by the script.",
127
154
  })}\n`);
128
155
  await Promise.race([
129
156
  settled,
157
+ stdoutFramingFailure,
130
158
  new Promise((_, reject) => setTimeout(() => reject(new Error("Continuation run did not settle.")), 120_000)),
131
159
  ]);
132
160
 
@@ -139,17 +167,22 @@ export default function (pi: ExtensionAPI): void {
139
167
  .trim();
140
168
  if (!finalText.includes("pi-task-dispatcher-codemode-ready")) throw new Error(`Unexpected continuation output: ${finalText}`);
141
169
 
142
- console.log(JSON.stringify({ firstBoundaryCount, firstAssistantMessages: firstAssistantMessages.length, finalText }, null, 2));
170
+ console.log(JSON.stringify({ firstBoundaryCount, firstAssistantMessages: firstAssistantMessages.length, finalText, nativeCodemodeOnly: true }, null, 2));
143
171
  if (stderr.trim()) console.error(stderr.trim());
144
172
  } finally {
145
- // Close the reusable child and remove only the runtime-controlled probe directory.
146
- if (child && child.exitCode === null && child.signalCode === null) {
147
- child.stdin?.end();
148
- await Promise.race([
149
- new Promise((resolve) => child.once("close", resolve)),
150
- new Promise((resolve) => setTimeout(resolve, 5_000)),
151
- ]);
152
- if (child.exitCode === null && child.signalCode === null) child.kill("SIGKILL");
173
+ try {
174
+ // Close the reusable child before validating that its final stdout tail was LF-framed.
175
+ if (child && child.exitCode === null && child.signalCode === null) {
176
+ child.stdin?.end();
177
+ await Promise.race([
178
+ new Promise((resolve) => child.once("close", resolve)),
179
+ new Promise((resolve) => setTimeout(resolve, 5_000)),
180
+ ]);
181
+ if (child.exitCode === null && child.signalCode === null) child.kill("SIGKILL");
182
+ }
183
+ if (stdoutRecords) await stdoutRecords;
184
+ } finally {
185
+ // Remove only the runtime-controlled probe directory, even when stdout framing failed.
186
+ await rm(temporaryDirectory, { recursive: true, force: true });
153
187
  }
154
- await rm(temporaryDirectory, { recursive: true, force: true });
155
188
  }
@@ -5,8 +5,11 @@
5
5
  import { readFile } from "node:fs/promises";
6
6
 
7
7
  const packageJson = JSON.parse(await readFile(new URL("../package.json", import.meta.url), "utf8"));
8
- const source = await readFile(new URL("../extensions/task-dispatcher.ts", import.meta.url), "utf8");
9
- const gateProbe = await readFile(new URL("./gate-probe.mjs", import.meta.url), "utf8");
8
+ const [source, gateProbe, fakeRpcWorker] = await Promise.all([
9
+ readFile(new URL("../extensions/task-dispatcher.ts", import.meta.url), "utf8"),
10
+ readFile(new URL("./gate-probe.mjs", import.meta.url), "utf8"),
11
+ readFile(new URL("./fake-rpc-worker.mjs", import.meta.url), "utf8"),
12
+ ]);
10
13
 
11
14
  // Keep task budgets formula-owned and prevent accidental restoration of the v2 role limits.
12
15
  for (const forbidden of ["checkpointGraceSeconds", "timeoutSeconds", "maxTurns"]) {
@@ -27,7 +30,9 @@ for (const required of [
27
30
  "delegate_designer",
28
31
  "delegate_control",
29
32
  "delegate_status",
30
- "builtin:codemode",
33
+ "codemode-only.ts",
34
+ "createCodemodeExtension",
35
+ 'mode: "only"',
31
36
  "task_dispatcher_boundary",
32
37
  "without an enforced IPC boundary",
33
38
  "--no-extensions",
@@ -35,9 +40,25 @@ for (const required of [
35
40
  "set_auto_compaction",
36
41
  "get_session_stats",
37
42
  "renderResult",
43
+ "truncateToVisualLines",
44
+ "truncateToWidth",
45
+ 'keyHint("app.tools.expand", "to expand")',
46
+ "stripTerminalSequences",
47
+ "message_update",
48
+ "tool_execution_update",
49
+ "PRESENTATION_UPDATE_THROTTLE_MS",
50
+ "context.lastComponent",
38
51
  ]) {
39
52
  if (!source.includes(required)) throw new Error(`Missing supervised-protocol marker: ${required}`);
40
53
  }
41
54
 
42
- if (!String(packageJson.version).startsWith("3.")) throw new Error(`Expected a v3 package version, found ${packageJson.version}`);
55
+ if (packageJson.version !== "3.1.1") throw new Error(`Expected release version 3.1.1, found ${packageJson.version}`);
56
+ // Keep test-side RPC framing independent from readline's broader line-separator behavior.
57
+ for (const [name, reader] of [["fake RPC worker", fakeRpcWorker], ["gate probe", gateProbe]]) {
58
+ if (reader.includes("node:readline")) throw new Error(`${name} still imports node:readline.`);
59
+ }
60
+ if (source.includes("builtin:codemode")) throw new Error("The dispatcher must not load codemode through a built-in extension selector.");
61
+ if (!gateProbe.includes("createCodemodeExtension") || !gateProbe.includes('mode: "only"')) {
62
+ throw new Error("The live gate probe does not exercise the native codemode-only extension.");
63
+ }
43
64
  console.log(`Static supervised-protocol checks passed for ${packageJson.name}@${packageJson.version}.`);
@@ -0,0 +1,48 @@
1
+ /**
2
+ * Verify the strict UTF-8 JSONL framing contract used by RPC-oriented test readers.
3
+ */
4
+
5
+ import assert from "node:assert/strict";
6
+ import { Readable } from "node:stream";
7
+ import { readStrictJsonl } from "./strict-jsonl.mjs";
8
+
9
+ const encoder = new TextEncoder();
10
+
11
+ /** Collect records from deliberately controlled byte chunks. */
12
+ async function collectRecords(chunks) {
13
+ const records = [];
14
+ await readStrictJsonl(Readable.from(chunks), async (record) => {
15
+ records.push(record);
16
+ });
17
+ return records;
18
+ }
19
+
20
+ // Keep raw JSON string separators as payload instead of treating them as record boundaries.
21
+ const rawSeparatorValue = `before${String.fromCodePoint(0x2028)}middle${String.fromCodePoint(0x2029)}after`;
22
+ const rawSeparatorRecord = `{"value":"${rawSeparatorValue}"}`;
23
+ const separatorRecords = await collectRecords([encoder.encode(`${rawSeparatorRecord}\n`)]);
24
+ assert.deepEqual(separatorRecords, [rawSeparatorRecord]);
25
+ assert.equal(JSON.parse(separatorRecords[0]).value, rawSeparatorValue);
26
+ assert.doesNotMatch(rawSeparatorRecord, /\\u2028|\\u2029/);
27
+
28
+ // Split a four-byte character mid-sequence to require streaming UTF-8 decoding.
29
+ const multibyteRecord = JSON.stringify({ value: "split 🧪 UTF-8" });
30
+ const multibyteBytes = encoder.encode(`${multibyteRecord}\n`);
31
+ const multibyteStart = multibyteBytes.indexOf(0xf0);
32
+ assert.notEqual(multibyteStart, -1, "Expected the test record to contain a four-byte UTF-8 character.");
33
+ assert.deepEqual(
34
+ await collectRecords([multibyteBytes.subarray(0, multibyteStart + 1), multibyteBytes.subarray(multibyteStart + 1)]),
35
+ [multibyteRecord],
36
+ );
37
+
38
+ // Accept CRLF transport framing while returning the JSON record without its framing CR.
39
+ const crlfRecord = JSON.stringify({ framing: "crlf" });
40
+ assert.deepEqual(await collectRecords([encoder.encode(`${crlfRecord}\r\n`)]), [crlfRecord]);
41
+
42
+ // Reject a final record that has bytes but no LF terminator.
43
+ await assert.rejects(
44
+ collectRecords([encoder.encode('{"unfinished":true}')]),
45
+ /unterminated tail/,
46
+ );
47
+
48
+ console.log("Strict JSONL framing checks passed.");
@@ -0,0 +1,32 @@
1
+ /**
2
+ * Decode UTF-8 JSONL test streams with LF as the only record delimiter.
3
+ */
4
+
5
+ /** Deliver complete LF-framed records while preserving raw Unicode JSON string content. */
6
+ export async function readStrictJsonl(input, onRecord) {
7
+ const decoder = new TextDecoder("utf-8", { fatal: true });
8
+ let tail = "";
9
+
10
+ /** Emit buffered records and remove at most one CR from each LF terminator. */
11
+ const emitRecords = async () => {
12
+ let lineFeed = tail.indexOf("\n");
13
+ // Drain complete LF-framed records before retaining an incomplete tail.
14
+ while (lineFeed !== -1) {
15
+ const record = tail.slice(0, lineFeed);
16
+ tail = tail.slice(lineFeed + 1);
17
+ await onRecord(record.endsWith("\r") ? record.slice(0, -1) : record);
18
+ lineFeed = tail.indexOf("\n");
19
+ }
20
+ };
21
+
22
+ // Decode chunks incrementally so an incomplete multibyte character cannot become replacement text.
23
+ for await (const chunk of input) {
24
+ tail += decoder.decode(chunk, { stream: true });
25
+ await emitRecords();
26
+ }
27
+ // Flush pending decoder bytes before deciding whether the final record was properly framed.
28
+ tail += decoder.decode();
29
+ await emitRecords();
30
+ // Reject a nonempty EOF tail instead of accepting an ambiguous partial JSON record.
31
+ if (tail !== "") throw new Error("Strict JSONL input ended with an unterminated tail.");
32
+ }