pi-supernova 0.7.1 → 0.8.1

package/README.md

@@ -12,34 +12,56 @@ Ordinary JavaScript control flow remains available; the guest command bindings
 are only `read`, `edit`, `write`, and `bash`. Supernova supplies retrieval,
 transactional file operations, batching, bounded results and the grouped nova UI.
 
-## Unreleased
-
-- **`parallel: true` on `programs`:** independent entries run at once (up to 8),
-  keep result order, and do not stop siblings on failure. Sequential is still
-  the default.
-- **JSON `.length`:** `read({path, json:".items.length"})` returns the array
-  length without dumping the array.
-- Prefer `edit` for a file you already read; `write` still replaces the file.
-
-## What is new in 0.7.0
-
-- **Faster batches:** guest workers pipeline their successor, so sequential
-  programs run about twice as fast with no token cost.
-- **Hardened guest:** `process.kill` is sealed out; patch hunks report
-  relocation and ambiguous hunks fail instead of misapplying.
-- **Leaner receipts:** multi-edit output caps at 32 matches with exact totals,
-  write receipts go workspace-relative, evidence/outline reads go compact.
-
-## What is new in 0.6.0
-
-- **Shared batch input:** supply top-level `data` once; each program gets an
-  independent copy unless it supplies its own replacement data.
-- **Conflict protection:** byte snapshots survive partial reads and body-cache
-  eviction; receipt generation cannot silently rebase a pending write.
-- **Read fidelity:** staged declarations remain discoverable in large/new files,
-  line windows preserve source endings, and `complete` always means the whole file.
-- **Explicit failures:** incompatible read modes, budget-limited matches, captured
-  overrides and conflicting new-file aliases no longer silently change outcomes.
+## What is new in 0.8.0
+
+- **Shared program source:** top-level `code` or `file` supplies a batch default;
+  entries may override it. A shared program is sent once instead of in every entry,
+  and defaults count once against the 48,000-character admission cap.
+- **Explicit object defaults:** `mergeData:true` shallowly overlays per-entry data
+  onto common data (entry keys win; nested objects are replaced). Whole-input
+  replacement remains the default.
+- **Checkpoint failures throw:** a failed `edit(async () => {...})` rolls back and
+  rethrows its original cause; catch explicitly when rejecting a candidate is
+  intentional. Ignored failures no longer report success.
+- **Accurate failure cards:** the nova card reads the host's error flag, shows the
+  original cause and `committed`/`rolledBack` totals, marks writes whose
+  persistence cannot be attributed as attempted, and labels pure JavaScript runs
+  instead of "complete".
+- **Bounded, explicit reads:** errors state both limits (`160 lines / 8192
+  characters`) with copyable recovery (`offset`, `about`, `complete:true`, and
+  `Promise.allSettled` for optional siblings). Markdown edits skip code-reference
+  searches; exact-symbol evidence excludes generic matches.
+- **Fail-closed images:** unsupported formats (for example BMP) fail before model
+  delivery with PNG-conversion guidance, and sets over 16 images / 20 MiB report
+  aggregate sizes instead of silently omitting attachments. Pending changes roll back.
+- **Shell follows the program clock:** `bash()` inherits the program's `timeoutMs`;
+  explicit per-command limits still win.
+
+### Tokens: 0.7.1 to 0.8.0 (`js-tiktoken`, `o200k_base` / `cl100k_base`)
+
+| Metric | 0.7.1 | 0.8.0 | Change |
+|---|---:|---:|---:|
+| Standing definition per request | 596 / 588 | 631 / 626 | +35 / +38 |
+| Frozen 6-call mixed workload, total traffic | 9,596 / 9,458 | 9,841 / 9,724 | +2.6% / +2.8% |
+| 16-program job with shared source + data (32 files) | 17,771 / 17,595 | 3,587 / 3,533 | -79.8% / -79.9% |
+| 8 programs sharing a 48-path input | 16,309 / 14,649 | 5,499 / 5,197 | -66.3% / -64.5% |
+
+Rows 3-4 deliver identical complete outputs and files; only argument placement
+changes. Row 2 repeats no inputs, so it pays the +35-token guidance and nothing
+else. Method, gates and limits: [TOKEN_COSTS.md](https://github.com/AdityaVG13/pi-stack/blob/main/packages/pi-supernova/docs/TOKEN_COSTS.md).
+
+### Speed: engine micro-benchmarks (Apple M5 Max, Node 26.7)
+
+| Benchmark | Before | After | Change |
+|---|---:|---:|---:|
+| Package 200-row report (median, 10k iterations) | 0.1665 ms | 0.1314 ms | -21% |
+| Package nested source object (median) | 0.0366 ms | 0.0240 ms | -35% |
+| Idle worker exit | 267 ms | 17 ms | -94% |
+| 8-file read wave p50 / p95 (300 samples) | 1.90 / 2.74 ms | 1.77 / 2.53 ms | -7% / -8% |
+| Cold unbatched p95 vs coalesced warm p95 (8 reads) | 13.85 ms | 2.39 ms | -83% |
+
+Identical output hashes before and after. Local engine benchmarks, not end-to-end
+agent latency or provider time.
 
 See the [changelog](https://github.com/AdityaVG13/pi-stack/blob/main/packages/pi-supernova/docs/CHANGELOG.md)
 and [token measurements](https://github.com/AdityaVG13/pi-stack/blob/main/packages/pi-supernova/docs/TOKEN_COSTS.md).
@@ -162,6 +184,13 @@ lines. Structural warnings and source windows are not substitutes for tests.
 
 ### Safe read-modify-write
 
+Path-only `read(path)` requires at most **160 lines and 8192 UTF-16 characters**.
+A short document can exceed the character limit. For larger files use
+`read(path,{offset:1,limit:80})` (one-based lines), `read(path,{about:"keywords"})`,
+or `read(path,{complete:true})`. The default raw-text read budget is **31,744
+characters**, derived from the configured call/return budgets, not an unlimited
+full-file buffer. Large JSONL needs a bounded parser through `bash({command,args})`.
+
 Plain reads are bounded views, not guaranteed full-file buffers. Use
 `read({path:"file.txt",complete:true})` when code needs the complete file; it
 throws rather than handing back partial text. Prefer `edit` for large-file
@@ -252,12 +281,21 @@ for short one-off operations. See [token measurements](https://github.com/Aditya
 }
 ~~~
 
-Use programs instead of top-level code/file. Supply 1--32 entries, each with
-code OR file and optional data. Top-level data supplies an optional default for
-each entry; explicit entry data replaces it entirely, including null, false, 0
-and empty strings. Every guest receives its own copy, not a shared mutable heap.
-The JSON-encoded array (or `{programs,data}` when defaults are supplied) must fit
-maxCodeChars. Common input counts once; result representations and output limits are unchanged.
+Supply 1--32 `programs` entries. Top-level `code` OR `file` supplies an optional
+source default; an entry's own `code` OR `file` replaces it. Without a shared
+source, every entry still requires its own. A shared file is reread when each
+entry runs, so earlier commits can create or update it.
+
+Top-level `data` supplies an optional input default. By default, explicit entry
+`data` replaces it entirely, including null, false, 0 and empty strings. With
+**`mergeData:true`**, both the default and every explicit entry input must be
+objects: own entry fields override default fields in a **shallow** merge. Nested
+objects are replaced, not recursively merged. Each guest receives its own copy;
+mutations cannot leak between guests or back into caller-owned inputs.
+
+The JSON-encoded array plus any shared code/file/data must fit `maxCodeChars`.
+Common source/input counts once, before expansion. Every entry is validated
+before any program runs. Result representations and output limits are unchanged.
 Entries run sequentially in fresh guests and commit separately. A successful
 entry can create the file executed by a later entry. No implicit retries,
 reordering, shared heap or nested batches are introduced.
@@ -278,6 +316,26 @@ This avoids repeating literal arguments, without a compression codec or result e
 Mutating `data` in one guest cannot affect the next. An entry with `data:null`
 receives null, not the shared object; there is no implicit object merge.
 
+For the same program with varying inputs, no temporary script or input file is
+needed. For example, audit both files with a common term:
+
+```json
+{
+  "code": "const text=await read({path:data.path,complete:true}); return {path:data.path,found:text.includes(data.term),text};",
+  "data": {"term":"TODO"},
+  "mergeData": true,
+  "programs": [
+    {"data":{"path":"src/a.js"}},
+    {"data":{"path":"src/b.js"}}
+  ]
+}
+```
+
+The complete source is still returned twice, once for each requested file; shared
+arguments do not authorize result deduplication or context rewriting. Existing
+programs that depend on whole-input replacement keep that behavior unless the
+caller explicitly requests `mergeData:true`.
+
 The batch stops on the first failed entry, cancellation/deadline, or exhausted
 output/log/image budget. Earlier successful commits remain; only the active
 program's uncommitted writes roll back. Admission errors throw before any program.
@@ -290,7 +348,7 @@ Set `parallel: true` with `programs` to run independent entries concurrently
 (up to 8 at once). Each still gets a fresh guest and its own commit; results stay
 in submission order. A failed entry does not stop siblings. Two entries writing
 the same file race: the losing commit reports a conflict. Sequential remains the
-default. `parallel` is invalid on a lone `code` or `file` call.
+default. `parallel` and `mergeData` are invalid on a lone `code` or `file` call.
 
 The outer deadline, host-call budget, log allowance, text budget and image limits
 are shared across the batch. Individual read budgets are not reduced. Every
@@ -306,7 +364,10 @@ settings, hide observations, or infer a plan on the agent's behalf.
 ### Large inputs and report outputs
 
 The default program limit is 48,000 UTF-16 code units (configurable via
-`maxCodeChars` and exposed in the tool schema). Split larger documents into
+`maxCodeChars` and exposed in the tool schema). The same cap applies separately
+to serialized JSON `data`, including quote/newline escaping and object keys.
+Oversized input fails before commands run and reports its actual serialized size.
+Split larger documents into
 separate invocations: first `write(path, firstChunk)`, then
 `write({path,content:nextChunk,append:true})`. Append uses the complete internal
 file buffer, never a bounded model-facing read; it retains conflict checks and
@@ -316,7 +377,9 @@ file and publish it only when complete. External write overrides reject append.
 
 Supernova is a bounded foreground executor, not a durable background-job manager.
 For long archive scans, use resumable chunks or a host background-job tool and write
-progress records under `.work`. Set the inner `bash` timeout shorter than the
+progress records under `.work`. Shell commands inherit the current program
+`timeoutMs` unless they specify their own; increasing the outer deadline no longer
+leaves a hidden 60-second shell cap. Set the inner `bash` timeout shorter than the
 outer program timeout (for example 10 seconds inside a 20-second program) to retain
 bounded shell diagnostics. A hard guest deadline cannot guarantee pending shell
 output delivery; progress files survive shell execution but staged VFS writes may
@@ -344,12 +407,21 @@ Inputs are capped at 16 MiB, including staged files. JSON reads require regular
 files and reject named pipes without waiting for a writer. The entire input must
 be valid JSON before any selection. Each selector is budgeted before allocating
 the next slice; sparse selector/path/edit arrays are rejected. Selected JSON must
-fit the ordinary read budget or the
-read throws; it is never returned as malformed/truncated JSON. Oversized unwindowed
+fit the ordinary read budget. Oversized selections return a routing object
+`{status:"too_large",path,keys}` (or `length` for an array), not the requested
+array/object: check `status` before calling `.map` or `.filter`, then select
+narrower fields or slices. The input-size cap still throws before projection;
+JSON is never returned malformed or silently truncated. Oversized unwindowed
 plain .json reads also fail with a projection hint. Explicit offset/limit or
 resolve:true still allow raw inspection, but line windows are not JSON documents.
 Do not combine json with complete, line windows, or source views. External read
 overrides reject JSON projection rather than silently ignoring the option.
+Uncaught read errors abort the program, including `return {a:await read(...),
+b:await read(...)}`; earlier successful values are not an implicit partial return.
+For optional sources, explicitly return `await Promise.allSettled(paths.map(path =>
+read(path)))`. This keeps successful text and per-path errors without weakening
+rollback for uncaught failures.
+
 Other read options, even false-valued flags, do not bypass a captured external
 read executor; its policy, transforms and failures remain authoritative.
 
@@ -386,9 +458,11 @@ recovery backups before retrying. Import-based mutations and shell side effects
 are outside the VFS counters; this is not a filesystem audit.
 
 `edit(async () => {...})` creates a nested filesystem checkpoint. It returns
-`{ok:true,committed:true,value}` on success or `{ok:false,committed:false,error}` on
-failure. Shell commands, overlapping/nested checkpoints, and concurrent commands
-outside the active callback are rejected. Await the checkpoint before proceeding.
+`{ok:true,committed:true,value}` on success. On failure it rolls back and rethrows
+the cause, so an ignored failed checkpoint cannot report program success. Use
+`try { await edit(async () => {...}); } catch (error) {...}` for deliberate recovery.
+Shell commands, overlapping/nested checkpoints, and concurrent commands outside
+the active callback are rejected. Await the checkpoint before proceeding.
 
 ## Context, caching and failure fidelity
 
@@ -396,9 +470,15 @@ outside the active callback are rejected. Await the checkpoint before proceeding
   is not proof the model still retains an earlier result after compaction.
 - Oversized text reads provide an exact next-line offset. A single line too large
   for the budget fails explicitly instead of pretending it was read completely.
+- Model attachments support PNG, JPEG, GIF and WebP. BMP and other unsupported
+  MIME types fail before attachment or commit, rather than causing a provider
+  HTTP 400 on the next request. Convert those sources to PNG first; Supernova
+  does not silently convert, resize or modify the original image.
 - Returned images remain image content blocks, including in arrays/objects. Images
   not returned by the program stay out of model output. Returned images are limited
-  to 16 attachments / 20 MiB; resize or return fewer when necessary.
+  to 16 attachments / 20 MiB. Overflow fails the program with the aggregate count
+  and byte size, returns no images, and rolls back pending writes. Resize or return
+  fewer images when necessary.
 - Dense multiline string arrays can render as verbatim source blocks instead of
   escaped string literals. Each block gives its array index and exact UTF-16 length;
   strings and result types are unchanged. This is output framing, not source
@@ -455,7 +535,10 @@ There is no dependency on or automatic routing to any consumer package.
 
 CodeMode executes trusted JavaScript in a terminable worker, **not a security
 sandbox**. The four adapters constrain writes/edits to the workspace and allow
-explicit external reads. JavaScript imports and shell commands still have process
+explicit external reads. Absolute temporary-directory paths outside the workspace
+are not write destinations: use a workspace path such as `.work/verification.log`,
+or a separately authorized external command. Errors identify the rejected path
+and workspace, including symlink escapes. JavaScript imports and shell commands still have process
 privileges. Do not run untrusted programs as though these adapters isolate them.
 
 Pi preflights the outer `supernova` call. Internal primitives do not emit ordinary

package/docs/CHANGELOG.md

@@ -2,6 +2,93 @@
 
 ## Unreleased
 
+## [0.8.1] - 2026-09-19
+
+### Fixed
+
+- Reads, windows, edits and appends reject **non-UTF-8** files with the path and
+  a conversion hint instead of returning U+FFFD, which an edit or append could
+  have written back as corruption. Prefix windows still drop one partial
+  character at the byte cut; diff/receipt snapshots stay tolerant so an explicit
+  `replace:true` still overwrites any file.
+- Unknown options now fail loudly on every command: `read` (for example a
+  foreign `{start,end}` window, which used to return the whole file) and
+  `bash`/`write`/`edit` (`env`, `maxOutputChars`, `mode`, `all` were dropped
+  silently). Each error names the option and the supported set.
+- Filesystem failures name the path or command: write and edit no longer say
+  "read path is a directory"; `ENOTDIR`/`EACCES`/`EPERM`/`EROFS`/`ENOSPC` no
+  longer leak raw codes or the `.supernova-<uuid>.new` temporary; `bash cwd`
+  must be a directory (`spawn ENOTDIR` is gone).
+- `readWindow` no longer leaks an unhandled rejection while `finally` awaits
+  `file.close()`; failing window reads settle cleanly.
+- Edit target misses report the closest matching line with its exact bytes
+  instead of only the file head; multi-edit failures name the entry
+  (`edit 2 of 3`); `edit(path,{oldText,newText}|{edits}|{patch})` dispatches.
+- Syntax errors quote the offending source line and column with a caret;
+  `file:` programs name the file, and invalid UTF-8 names the file too.
+- JSON projection reports the parse position with the offending line when V8
+  provides one, and ignores one leading BOM (also in write checks).
+- Timeout and batch-deadline messages report elapsed time against the limit, and
+  a deadline-killed program is no longer reported as a plain program failure.
+
+### Internals
+
+- 259 package tests, actual Pi + OMP host smoke, isolated Spark run, a
+  536-program stress pass, and both frozen token gates.
+
+## [0.8.0] - 2026-09-19
+
+### Added
+
+- Program batches accept a top-level `code` OR `file` as a source default, so a
+  shared program is sent once instead of in every entry. Entries may still
+  override it, defaults count once against the 48,000-character admission cap,
+  and every entry keeps its own fresh guest and its own commit.
+- `mergeData: true` opts into a shallow object overlay of the top-level `data`
+  object and each entry's own object data (entry keys win; nested objects are
+  replaced, not merged). Whole-input replacement stays the default.
+
+### Changed
+
+- Standing reference now states the read limits and their recovery forms
+  (path-only `160 lines / 8192 characters`, `offset`/`about`/`complete:true`,
+  one-based windows, the complete-read budget) and the optional-read contract
+  (`Promise.allSettled` keeps successful siblings): +35/+38 definition tokens per
+  request versus 0.7.1, paid back many times over on repeated-source batches.
+- `bash()` inherits the program's `timeoutMs` instead of an independent 60s
+  default; explicit per-command limits still win.
+- Missing-file errors point at directory/source-question recovery and
+  `Promise.allSettled`.
+- Result packaging reuses decoded result branches and skips escaped rendering
+  when the complete raw framing is provably shorter; settled programs no longer
+  arm a 250 ms drain timer.
+
+### Fixed
+
+- Failed `edit(async () => {...})` checkpoints roll back and rethrow the original
+  cause instead of reporting success; an explicit `catch` still recovers.
+- Pi failure cards render as failures: the renderer reads the host's error flag
+  from render context, shows the original cause, prints `committed`/`rolledBack`
+  totals, marks writes whose persistence cannot be attributed as attempted, and
+  labels pure JavaScript execution instead of "complete".
+- Unsupported image formats (for example BMP) fail before model delivery or
+  commit with PNG-conversion guidance instead of attaching unusable data.
+- Oversized image sets report aggregate sizes instead of silently dropping
+  attachments; pending writes roll back.
+- Rust lifetimes and loop labels no longer produce false edit/write warnings;
+  genuinely broken strings, brackets and character literals still warn.
+- Markdown/MDX/RST/TXT edits skip declaration-reference searches for fenced code
+  and capital labels; usage evidence ignores declarations, generic calls and
+  prefix-only matches.
+- Missing argv data identifies the offending argument index; oversized `data`
+  reports its serialized size and a lossless chunked-write recovery.
+
+### Internals
+
+- 243 package tests, actual Pi and OMP host smoke, isolated Spark run, a
+  536-program stress pass, and both frozen token gates. `docs/TOKEN_COSTS.md`
+  records the traffic baselines, their limits and the measured 0.7.1 comparisons.
+
 ## [0.7.1] - 2026-09-17
 
 ### Fixed

package/docs/TOKEN_COSTS.md

@@ -21,6 +21,154 @@ The benchmark uses js-tiktoken 1.0.21, pinned as a development dependency. It ru
 real Supernova programs in temporary workspaces without provider calls or downloads.
 The token regression gate also runs in the normal test suite.
 
+## Unreleased explicit batch reuse (2026-09-19)
+
+**79.82% / 79.92% less traffic on the shared-source/object-input workload, not a
+claim of 80% savings on every task.** Source and object defaults remove repeated
+arguments without compression, citations, result elision, context rewriting,
+reduced limits or changes to reasoning settings. Every program still executes in
+a fresh guest with its own commit. No model decision point is removed.
+
+The missing capability was combining common executable/input values with distinct
+per-entry inputs without repeating them or first saving helper files. Batches now
+accept top-level `code` OR `file` as a source default. `mergeData:true` explicitly
+opts into a shallow object overlay; the legacy whole-input replacement remains
+the default. Entries can override the source, and nested data objects are replaced,
+not recursively merged. Nothing is inferred or automatically deduplicated.
+
+### Frozen eligible workload
+
+The pre-feature working tree, including the earlier unreleased papercut fixes,
+ran 16 independent package-doc scaffolds. Each program wrote a LICENSE and README,
+reread both exact staged contents, and returned its complete receipts. The control
+repeated the same executable and literal license/introduction with a distinct
+directory per entry. Its encoded program array was **32,353 characters**, within
+the existing 48,000-character admission cap. This is not an inadmissible baseline.
+
+The candidate supplies the same complete code and common data once, plus those
+same per-entry directories. Both arms use **one tool invocation plus the final
+model handoff**, the same 16-program schedule, and all 32 final files. The baseline
+was captured before the implementation; neither program source nor data was padded
+or changed to meet the gate.
+
+| Tokenizer | Before | After | Reduction | Argument tokens before / after | Unchanged result tokens |
+| --- | ---: | ---: | ---: | ---: | ---: |
+| o200k_base | 17,771 | 3,587 | **79.82%** | 7,685 / 643 | 1,039 |
+| cl100k_base | 17,595 | 3,533 | **79.92%** | 7,619 / 637 | 1,007 |
+
+Accounting is `2*definition + 2*arguments + complete result`. It includes both
+requests' standing definitions, generated arguments and their replay. The complete
+normalized output equals the pre-feature snapshot byte-for-byte, and the contents
+and complete inventory of all 32 files are checked. Only run counters and elapsed
+times are normalized, as in the existing benchmark. No hidden helper files appear.
+
+The new gate requires **at least 78% on both tokenizers**, not an argument-only
+percentage. Its fixture is
+[`batch-reuse-baseline.json`](https://github.com/AdityaVG13/pi-stack/blob/main/packages/pi-supernova/tests/efficiency/batch-reuse-baseline.json),
+with SHA-256 `589960c417006630872c16fae9ae5be4f5cdbdddbf86dda9fd9377651352055a`.
+
+This workload benefits from substantial repeated source and input. Unique-input,
+large-result, or already-file-backed jobs need not see that gain. Saving a reusable
+program/input file was already an alternative when extra workspace artifacts and
+setup were acceptable; this is not a claim of 80% over that different workflow.
+
+### Existing workload stays separate
+
+The original six-call schedule, its argument hash, all 13 logical operations,
+failures, decision boundaries and complete outputs remain unchanged. It already
+uses saved programs and does not exercise the new source defaults. Clearer shorter
+standing guidance offsets the new option's schema cost:
+
+| Tokenizer | At start of this pass | Now | Additional reduction | Versus original non-batched baseline |
+| --- | ---: | ---: | ---: | ---: |
+| o200k_base | 10,191 | 9,841 | **3.43%** | 65.02% |
+| cl100k_base | 10,067 | 9,724 | **3.41%** | 65.07% |
+
+The definition measures 631/626 tokens, versus 681/675 at the start. The original
+traffic gate was strengthened from 40% to 65%, without changing its baseline. An
+initial shorter guidance draft failed the existing recovery/surface checks; the
+copyable offset recovery and explicit selector/view guidance were restored rather
+than removing those requirements. There is no claim of 70--80% on this workload.
+
+### Verification and boundaries
+
+New source/overlay regressions and the reuse gate failed before implementation.
+The strengthened existing traffic gate also failed first. Current verification:
+239 package tests, both tokenizers' gates, lint, actual Pi argument validation and
+TUI smoke, actual network-denied OMP execution, and 71 targeted tests on an isolated
+Spark copy under Node v24.16.0. The actual-host probes exercise shared source,
+object overlays and independent guest copies.
+
+The existing stress runner passed 33,024 independent reads, 20 commits/140 conflicts,
+cancellation, complete-source fidelity and program batches. On Apple M5 Max,
+Node v26.7.0, the 300-sample, eight-file check measured pristine-warm median/p95
+1.645/2.390 ms versus unbatched-cold p95 13.848 ms; its existing acceptance check
+passed. This is a runtime smoke measurement, **not a new end-to-end speedup claim**.
+
+Complete-output and file equality prove the tested contracts, not unchanged
+live-model quality. There was no provider/model A/B evaluation, no reasoning-budget
+change, and no compression feature enabled. Deployment, session restart and
+publication are separate; installed OMP packages were not changed by this pass.
+
+## Unreleased optimization pass (2026-09-19)
+
+Measured against the working tree immediately before this pass, **including the
+unreleased papercut fixes**, not against a published release or the older baselines
+below. Machine: Apple M5 Max, 18 CPUs, 48 GiB RAM, macOS arm64, Node v26.7.0.
+
+| Measurement | Before | After | Reduction |
+| --- | ---: | ---: | ---: |
+| Definition tokens, o200k_base | 699 | 658 | 5.87% |
+| Definition tokens, cl100k_base | 691 | 651 | 5.79% |
+| Fixed six-call traffic, o200k_base | 10,317 | 10,030 | 2.78% |
+| Fixed six-call traffic, cl100k_base | 10,179 | 9,899 | 2.75% |
+| 200-row report packaging, median | 0.1665 ms | 0.1314 ms | 21.09% |
+| Nested source packaging, median | 0.0366 ms | 0.0240 ms | 34.53% |
+
+A subsequent Cortex papercut follow-up adds explicit data-limit, workspace-write,
+and inherited-timeout guidance. That later definition measures 674/667 tokens
+(o200k_base/cl100k_base), with fixed-workload totals of 10,142/10,011. The table
+above records the optimization checkpoint, before that additional guidance.
+
+The Mac/Spark recheck adds optional-read recovery and removes the unsupported BMP
+attachment advertisement. The resulting definition is 681/675 tokens, with fixed
+traffic totals of 10,191/10,067. Both frozen-workload gates still pass. Verification
+now includes 235 Mac package tests, 51 focused tests on an isolated Spark source
+copy (Node v24.16.0), and actual Pi/OMP smoke on Mac. Installed OMP packages were
+not updated by either follow-up.
+
+Token savings come only from shorter standing guidance. Arguments, decision
+boundaries and complete logical result text remain unchanged. No compression,
+elision, lower output limits, or history rewriting was introduced or enabled.
+
+Packaging now copies only changed branches and skips an escaped-source rendering
+when the existing complete raw rendering is provably shorter. The benchmark runs
+10,000 measured iterations per payload after warmup. Complete packaged-output
+SHA-256 hashes match before/after, including typed values, emitted text, images,
+logs and truncation flags. Local acceptance: identical hashes and at least 10%
+lower median packaging time for both fixtures; this is not a timing-sensitive CI
+gate or an end-to-end agent speed claim.
+
+Successful runs no longer leave a 250 ms drain timer alive; early completion of
+pending calls clears its fallback timer. A cold child-process probe exited at
+267 ms before versus 17 ms after, while result delivery itself remained about
+17 ms in both. Cancellation and the 250 ms bound on stuck calls remain tested.
+The 300-sample, eight-file benchmark measured prewarmed median/p95 of
+1.904/2.743 ms before and 1.765/2.530 ms after. These filesystem timings are noisy;
+no generalized I/O speedup is claimed, and prewarming/model latency is excluded.
+
+Reproduce the local latency and stress checks:
+
+~~~sh
+SUPERNOVA_MEASURE_SAMPLES=300 npm run measure --prefix packages/pi-supernova
+node packages/pi-supernova/tests/efficiency/stress.mjs
+~~~
+
+Verification: 229 package tests, both tokenizers' existing frozen-workload gates,
+lint, actual Pi loader/TUI and OMP execution smoke tests, plus the stress runner
+(33,024 independent reads, write contention, cancellation and source fidelity).
+No version bump, installation update, commit or publication is part of this pass.
+
 ## Workload
 
 The baseline is a frozen, non-batched implementation snapshot with program-file
@@ -72,12 +220,12 @@ text contributes to totals through later history, not as a second charge.
 
 Observed for 0.6.0 on 2026-09-15, on an Apple M5 Max running macOS and Node v26.7.0.
 
-| Tokenizer | Non-batched baseline | Batched baseline (d444eb7) | Current | Further reduction | Total reduction |
+| Tokenizer | Non-batched baseline | Batched baseline (d444eb7) | 0.6.0 | Further reduction | Total reduction |
 | --- | ---: | ---: | ---: | ---: | ---: |
 | o200k_base | 28,130 | 18,535 | 9,843 | **46.90%** | **65.01%** |
 | cl100k_base | 27,841 | 18,310 | 9,726 | **46.88%** | **65.07%** |
 
-The gate requires at least 40% reduction on **each tokenizer for the complete
+The gate now requires at least 65% reduction on **each tokenizer for the complete
 workload**, not for every scenario individually. Token counts and reductions are
 computed from the recorded baseline and fresh execution results, not constants
 returned by the runtime. A second gate requires another 19% against the measured
@@ -86,7 +234,7 @@ as the original workload: removing a decision boundary cannot satisfy this gate.
 
 ### Definition and result accounting
 
-The current serialized definition is 602 tokens with o200k_base and 595 with
+The 0.6.0 serialized definition was 602 tokens with o200k_base and 595 with
 cl100k_base, versus 908 and 901 in the frozen non-batched baseline. It retains
 command signatures, complete-read and JSON limits, array-read failure rules,
 transaction boundaries, batch defaults and edit/view guidance on every request.

package/index.js

@@ -212,6 +212,7 @@ export function registerCodeMode(pi) {
 
   function rejectLoneParallel(params) {
     if (params?.parallel !== undefined) throw new Error("parallel applies to the programs array; no commands ran");
+    if (params?.mergeData !== undefined) throw new Error("mergeData applies to the programs array; no commands ran");
   }
 
   function bindRunSignal(signal) {
@@ -224,8 +225,8 @@ export function registerCodeMode(pi) {
     return { runController, abortRun };
   }
 
-  function openRunBridge(ctx, runCwd, budget, runController) {
-    const runBridge = bridge.fork({ getCwd: () => runCwd, budget });
+  function openRunBridge(ctx, runCwd, budget, runController, timeoutMs) {
+    const runBridge = bridge.fork({ getCwd: () => runCwd, budget, timeoutMs });
     runBridge.bindCallContext(ctx, runController.signal);
     runBridge.resetCallBudget();
 
@@ -332,6 +333,7 @@ export function registerCodeMode(pi) {
         data: Type.Optional(Type.Unknown()),
       }, {additionalProperties:false}), {minItems:1,maxItems:32})),
       parallel: Type.Optional(Type.Boolean()),
+      mergeData: Type.Optional(Type.Boolean()),
     }),
     // One self-owned result frame is shared by Pi and OMP; renderCall stays empty
     // so separate call/result slots cannot duplicate the lifecycle card.
@@ -345,7 +347,7 @@ export function registerCodeMode(pi) {
       cancelWarmTimer();
       const runCwd = isString(ctx?.cwd) && ctx.cwd ? ctx.cwd : cwd;
       const { runController, abortRun } = bindRunSignal(signal);
-      const runBridge = openRunBridge(ctx, runCwd, budget, runController);
+      const runBridge = openRunBridge(ctx, runCwd, budget, runController, params?.timeoutMs);
       const call = ++programSeq;
       runBridge.ledger.beginProgram(call);
       const emitProgress = progressEmitter(onUpdate);

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "pi-supernova",
-  "version": "0.7.1",
+  "version": "0.8.1",
   "description": "CodeMode for Pi and OMP: read, edit, write and bash, with transactional files, source views and shared-input program batches.",
   "type": "module",
   "author": "AdityaVG13",

package/src/adapters/bash.js

@@ -1,3 +1,4 @@
+import * as fs from "node:fs/promises";
 import { isString } from "../shared/decode.js";
 import { unwrapIfFullyQuoted } from "../fs/text-ops.js";
 import { sourceForReferences } from "../fs/source-window.js";
@@ -35,6 +36,12 @@ export function createBash(ctx) {
       const { literal, command, argv } = parseBash(params);
       const targetCwd = params?.cwd ? await resolveWorkspacePath(cwd, params.cwd, "bash cwd", true) : cwd;
 
+      if (params?.cwd !== undefined) {
+        const st = await fs.stat(targetCwd).catch(() => null);
+
+        if (!st?.isDirectory()) throw new Error("bash cwd is not a directory: " + params.cwd);
+      }
+
       const transactionBarrier = await vfs.prepareExternalMutation("bash");
       let res;
 
@@ -43,7 +50,7 @@ export function createBash(ctx) {
           cwd: targetCwd,
           env: hooks.commandEnv(),
           commandLabel: literal ? command : undefined,
-          timeoutMs: params?.timeoutMs,
+          timeoutMs: params?.timeoutMs === undefined ? config.timeoutMs : params.timeoutMs,
           signal,
           maxOutputChars: config.maxCallResultChars,
         });

package/src/adapters/edit.js

@@ -131,6 +131,7 @@ export function createEdit(ctx) {
   }
 
   async function changedDeclarationRefs(cwd, target, original, updated, diff, signal) {
+    if (/\.(md|mdx|rst|txt)$/i.test(target)) return "";
     const names = collectChangedNames(target, original, updated, diff);
 
     if (names.size === 0) return "";
@@ -143,7 +144,7 @@ export function createEdit(ctx) {
     } catch (error) {
       signal?.throwIfAborted();
 
-      return "references unavailable: " + error.message;
+      return "references unavailable: " + String(error.message).slice(0, 512);
     }
   }
 

package/src/adapters/errors.js

@@ -25,7 +25,7 @@ export function imageTooLarge(rel, size) {
 }
 
 export function missingFile(targetPath) {
-  const error = new Error("no such file: " + targetPath + " (locate it with read using a directory path or source question)");
+  const error = new Error("no such file: " + targetPath + " (locate it with read using a directory path or source question; use Promise.allSettled for optional reads to retain successful siblings)");
   error.code = "ENOENT";
   return error;
 }