pi-supernova 0.10.1 → 0.11.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (81) hide show
  1. package/README.md +376 -221
  2. package/docs/CHANGELOG.md +107 -5
  3. package/docs/TOKEN_COSTS.md +149 -324
  4. package/index.js +40 -11
  5. package/package.json +4 -3
  6. package/src/adapters/bash.js +40 -10
  7. package/src/adapters/edit.js +77 -23
  8. package/src/adapters/errors.js +4 -0
  9. package/src/adapters/index.js +1 -0
  10. package/src/adapters/list.js +6 -3
  11. package/src/adapters/read-focus.js +6 -0
  12. package/src/adapters/read-image.js +17 -9
  13. package/src/adapters/read-indexed.js +72 -0
  14. package/src/adapters/read-text.js +19 -2
  15. package/src/adapters/read.js +57 -13
  16. package/src/adapters/refs.js +17 -7
  17. package/src/adapters/write.js +14 -21
  18. package/src/bridge/host-bridge.js +152 -39
  19. package/src/bridge/invoke.js +3 -1
  20. package/src/bridge/tool-registry.js +132 -16
  21. package/src/context/evidence-graph.js +1 -0
  22. package/src/context/evidence-rank.js +1 -0
  23. package/src/context/evidence.js +4 -2
  24. package/src/context/fuzzy.js +31 -8
  25. package/src/context/outline.js +33 -9
  26. package/src/context/query.js +7 -2
  27. package/src/context/repo-index.js +8 -3
  28. package/src/context/search-files.js +20 -0
  29. package/src/context/search.js +25 -10
  30. package/src/context/snap-search.js +28 -6
  31. package/src/context/snap.js +8 -3
  32. package/src/context/source-entry.js +64 -28
  33. package/src/context/spans.js +1 -0
  34. package/src/contract/bash.js +16 -7
  35. package/src/contract/edit.js +26 -7
  36. package/src/contract/program.js +2 -1
  37. package/src/contract/read.js +46 -46
  38. package/src/fs/background.js +10 -2
  39. package/src/fs/check.js +844 -46
  40. package/src/fs/commit.js +89 -15
  41. package/src/fs/diff.js +12 -9
  42. package/src/fs/directory.js +14 -4
  43. package/src/fs/file-io.js +28 -7
  44. package/src/fs/glob.js +93 -11
  45. package/src/fs/json-read.js +3 -2
  46. package/src/fs/json-size.js +2 -1
  47. package/src/fs/lines.js +49 -2
  48. package/src/fs/patch.js +26 -10
  49. package/src/fs/process-tree.js +19 -1
  50. package/src/fs/read-window.js +36 -20
  51. package/src/fs/source-window.js +5 -10
  52. package/src/fs/text-ops.js +39 -19
  53. package/src/fs/vfs.js +296 -41
  54. package/src/fs/workspace.js +24 -6
  55. package/src/output/bottleneck.js +50 -25
  56. package/src/output/final.js +20 -0
  57. package/src/output/format.js +97 -36
  58. package/src/output/outcome.js +25 -9
  59. package/src/runtime/batch-input.js +17 -3
  60. package/src/runtime/guest-api.js +32 -7
  61. package/src/runtime/guest-worker.js +22 -4
  62. package/src/runtime/parallel.js +9 -49
  63. package/src/runtime/program-batch.js +53 -28
  64. package/src/runtime/program-file.js +2 -1
  65. package/src/runtime/program.js +94 -41
  66. package/src/runtime/reference.js +10 -9
  67. package/src/runtime/runtime.js +86 -17
  68. package/src/runtime/worker-pool.js +10 -1
  69. package/src/shared/decode.js +9 -8
  70. package/src/shared/image-worker.js +7 -0
  71. package/src/shared/image.js +58 -15
  72. package/src/shared/png.js +18 -0
  73. package/src/shared/result.js +30 -9
  74. package/src/shared/syntax-context.js +7 -1
  75. package/src/shared/utf8.js +11 -13
  76. package/src/ui/host-render.js +4 -0
  77. package/src/ui/omp-frame.js +2 -0
  78. package/src/ui/progress.js +2 -1
  79. package/src/ui/render-measure.js +11 -0
  80. package/src/ui/render.js +92 -31
  81. package/src/ui/trace.js +8 -5
package/README.md CHANGED
@@ -11,169 +11,26 @@ The model submits JavaScript, not four separately advertised native tools.
11
11
  Ordinary JavaScript control flow remains available; the guest command bindings
12
12
  are only `read`, `edit`, `write`, and `bash`. Supernova supplies retrieval,
13
13
  transactional file operations, batching, bounded results and the grouped nova UI.
14
-
15
- ## What is new in 0.10.1
16
-
17
- - JSON reads that still send a leftover `selector` key (the standing schema's
18
- `json:true|selector` union, misread as a second option) fold into `json`.
19
- `json:true` plus `selector:".field"` or `selector:"field"` is `.field`.
20
- Two real projections (`json:".a"` and `selector:".b"`) still fail.
21
-
22
- ## What is new in 0.10.0
23
-
24
- - Background bash sessions support launch, polling, input and stop with bounded
25
- transcripts, deadlines, ownership and process-tree cleanup. macOS/Linux offer
26
- interactive PTYs; Windows uses pipes and explicitly rejects PTYs.
27
- - Literal argv stays native across platforms. Session changes, file commits,
28
- read provenance and error diagnostics retain their safety boundaries during
29
- asynchronous work. See [verification](#verification) for the cross-platform
30
- Node/Bun matrix and its limits.
31
-
32
- ## What is new in 0.9.0
33
-
34
- - **Text clipping does not stop batches:** sequential and parallel programs keep
35
- running when the combined display text exceeds its allowance. Results disclose
36
- truncation; execution, deadline, host-call, log and image limits remain enforced.
37
- - **Decoded image validation:** PNG/JPEG/GIF/WebP reads and returned attachments
38
- require matching formats, canonical base64 and decodable pixels, including all
39
- GIF/WebP frames. PNG preflight checks chunk boundaries and CRCs too. Failures
40
- occur before shell boundaries or final commit/delivery, with no image attached.
41
- - **Isolated image work:** Sharp 0.35.4 decodes at most 32 million pixels across
42
- all frames, one image at a time, in a Node/Bun subprocess with a 5-second kill
43
- deadline. Text-only work never loads the decoder. A bounded 16-entry digest cache
44
- avoids decoding unchanged bytes again; neither image data nor file paths are cached.
45
- Encoded limits remain 16 attachments / 20 MiB. Sharp's platform-specific optional
46
- dependencies must be installed; decoder failure never falls back to unchecked data.
47
- Local decoding does not guarantee acceptance under every provider's image policy.
48
-
49
- - **Data is not a display preview:** ordinary text reads return complete data up to
50
- 64 MiB; JSON selections remain actual values and directories remain arrays.
51
- The former 160-line / 8192-character and 31,744-character restrictions no longer
52
- constrain computation inside a program. Only returned/logged text is displayed.
53
- - **Large batches stay bounded:** small reads share one reply; larger items are
54
- delivered with acknowledgements within eight I/O slots, not retained as one
55
- giant host-side batch. Explicit arrays and coalesced reads use the same path.
56
- - **Output work stays in the worker:** bounded formatting avoids expanding large
57
- values before clipping. Model-visible source previews retain exact ranges and
58
- continuation; a clipped preview cannot be passed back as a complete edit view.
59
-
60
- - **Cancellation no longer crashes the host:** bounded reads and CAS signing use
61
- abort-checked file handles instead of aborting streams. A failed read can cancel
62
- sibling reads without an uncaught `AbortError` terminating OMP.
63
- - **Memory is charged to the guest:** worker-local heap and external buffers
64
- replace process-wide RSS accounting. Bun enforcement remains best-effort.
65
- - **Focused internals:** read adapters, file I/O, transactions, worker lifecycle,
66
- tool ownership, source ranking and rendering have separate modules. All read
67
- modes, batching, checkpoints and rollback behavior remain supported.
68
-
69
- **0.9.0 release-candidate baseline:** 315/315 package tests passed on Node and
70
- Bun, plus actual Pi/OMP and clean tarball installation checks. See the newer
71
- [cross-platform verification results and coverage limits](#verification) below.
72
-
73
- ### Concurrent file operations
74
-
75
- Batching and explicit parallel programs remain supported. Within one program,
76
- `Promise.all` now overlaps native edits and writes to different files, up to eight
77
- operations at once:
78
-
79
- ```js
80
- await Promise.all([
81
- edit("src/a.js", "oldA", "newA"),
82
- edit("src/b.js", "oldB", "newB"),
83
- ]);
84
- return await bash("npm test");
85
- ```
86
-
87
- Same-file operations retain submission order. Reads before/after mutations,
88
- `bash`, edit checkpoints, and overridden mutating tools remain ordering barriers.
89
- Shell calls inside one program stay sequential; use `programs` with
90
- `parallel:true` for explicitly independent shell workflows or separate JS workers.
91
- Edits still stage until program success (or a shell boundary); transactional disk
92
- commits retain their conflict checks. `await edit(...)` one after another is still
93
- sequential, and multiple replacements in one file remain one edit operation.
94
-
95
- ## What is new in 0.8.2
96
-
97
- This patch release fixes shell failure handling, cancellation and parallel-batch
98
- limits, and makes one-call batching guidance explicit.
99
-
100
- - **Shell quoting stays intact:** quoted executable paths are no longer unwrapped.
101
- Shell syntax errors suggest literal `bash({command,args})` with `data` for
102
- embedded scripts, or a quoted heredoc. Commands are not rewritten or retried.
103
- - **Validation before commit:** invalid timeouts and null-byte
104
- arguments are rejected before the shell boundary flushes staged files.
105
- - **Useful failure output:** long command labels are bounded so the original
106
- stderr is not crowded out by a repeated script.
107
- - **Timeouts retain diagnostics:** the outer program deadline stops the worker
108
- and gives pending host calls a bounded drain to retain shell output. Explicit
109
- cancellation is reported separately from timeout. The outer `timeoutMs` covers
110
- every wait and command, including `sleep`.
111
- - **Parallel budgets fail honestly:** exceeding the shared log or image
112
- allowance marks the batch failed and stops queued entries. Already-running
113
- entries settle; their results and completed commits remain. Aggregate logs stay
114
- capped rather than multiplying the allowance per guest. In 0.8.2 this also
115
- applied to output text; 0.9.0 makes display-text clipping nonfatal.
116
- - **Batch known work in one call:** combine independent reads/checks with
117
- `Promise.all`, then sequence edits and verification in the same program. Use
118
- another invocation when returned evidence is needed for the next decision.
119
-
120
- Verified on **macOS / Node 26.7**: 267 package tests (384 repository tests),
121
- 2,328 stress invocations, actual Pi/OMP host checks, lint and both token-budget
122
- checks. This is not a claim of exhaustive platform or formal mutation testing.
123
-
124
- ## 0.8.0 features and measurements
125
-
126
- - **Shared program source:** top-level `code` or `file` supplies a batch default;
127
- entries may override it. A shared program is sent once instead of in every entry,
128
- and defaults count once against the 48,000-character admission cap.
129
- - **Explicit object defaults:** `mergeData:true` shallowly overlays per-entry data
130
- onto common data (entry keys win; nested objects are replaced). Whole-input
131
- replacement remains the default.
132
- - **Checkpoint failures throw:** a failed `edit(async () => {...})` rolls back and
133
- rethrows its original cause; catch explicitly when rejecting a candidate is
134
- intentional. Ignored failures no longer report success.
135
- - **Accurate failure cards:** the nova card reads the host's error flag, shows the
136
- original cause and `committed`/`rolledBack` totals, marks writes whose
137
- persistence cannot be attributed as attempted, and labels pure JavaScript runs
138
- instead of "complete".
139
- - **Historical read limits (superseded in 0.9.0):** errors stated both limits (`160 lines / 8192
140
- characters`) with copyable recovery (`offset`, `about`, `complete:true`, and
141
- `Promise.allSettled` for optional siblings). Markdown edits skip code-reference
142
- searches; exact-symbol evidence excludes generic matches.
143
- - **Fail-closed images:** unsupported formats (for example BMP) fail before model
144
- delivery with PNG-conversion guidance, and sets over 16 images / 20 MiB report
145
- aggregate sizes instead of silently omitting attachments. Pending changes roll back.
146
- - **Shell follows the program clock:** `bash()` inherits the program's `timeoutMs`;
147
- explicit per-command limits still win.
148
-
149
- ### Tokens: 0.7.1 to 0.8.0 (`js-tiktoken`, `o200k_base` / `cl100k_base`)
150
-
151
- | Metric | 0.7.1 | 0.8.0 | Change |
152
- |---|---:|---:|---:|
153
- | Standing definition per request | 596 / 588 | 631 / 626 | +35 / +38 |
154
- | Frozen 6-call mixed workload, total traffic | 9,596 / 9,458 | 9,841 / 9,724 | +2.6% / +2.8% |
155
- | 16-program job with shared source + data (32 files) | 17,771 / 17,595 | 3,587 / 3,533 | -79.8% / -79.9% |
156
- | 8 programs sharing a 48-path input | 16,309 / 14,649 | 5,499 / 5,197 | -66.3% / -64.5% |
157
-
158
- Rows 3-4 deliver identical complete outputs and files; only argument placement
159
- changes. Row 2 repeats no inputs, so it pays the +35-token guidance and nothing
160
- else. Method, gates and limits: [TOKEN_COSTS.md](https://github.com/AdityaVG13/pi-stack/blob/main/packages/pi-supernova/docs/TOKEN_COSTS.md).
161
-
162
- ### Speed: engine micro-benchmarks (Apple M5 Max, Node 26.7)
163
-
164
- | Benchmark | Before | After | Change |
165
- |---|---:|---:|---:|
166
- | Package 200-row report (median, 10k iterations) | 0.1665 ms | 0.1314 ms | -21% |
167
- | Package nested source object (median) | 0.0366 ms | 0.0240 ms | -35% |
168
- | Idle worker exit | 267 ms | 17 ms | -94% |
169
- | 8-file read wave p50 / p95 (300 samples) | 1.90 / 2.74 ms | 1.77 / 2.53 ms | -7% / -8% |
170
- | Cold unbatched p95 vs coalesced warm p95 (8 reads) | 13.85 ms | 2.39 ms | -83% |
171
-
172
- Identical output hashes before and after. Local engine benchmarks, not end-to-end
173
- agent latency or provider time.
174
-
175
- See the [changelog](https://github.com/AdityaVG13/pi-stack/blob/main/packages/pi-supernova/docs/CHANGELOG.md)
176
- and [token measurements](https://github.com/AdityaVG13/pi-stack/blob/main/packages/pi-supernova/docs/TOKEN_COSTS.md).
14
+ There is no `supernova` object inside a program: use `await bash(...)`, not
15
+ `const { bash } = supernova`. Do not shadow the four command names with variables.
16
+
17
+ ## Contents
18
+
19
+ 1. [Install and update](#install-and-update)
20
+ 2. [Four guest commands](#four-guest-commands)
21
+ 3. [Tokens: native tool calls vs Supernova](#tokens-native-tool-calls-vs-supernova)
22
+ 4. [TUI presentation](#tui-presentation)
23
+ 5. [Background terminal sessions](#background-terminal-sessions)
24
+ 6. [JSON reports and targeted text audits](#json-reports-and-targeted-text-audits)
25
+ 7. [Execution and automatic batching](#execution-and-automatic-batching)
26
+ 8. [Context, caching and failure fidelity](#context-caching-and-failure-fidelity)
27
+ 9. [Optional workspace change notifications](#optional-workspace-change-notifications)
28
+ 10. [Security and host boundary](#security-and-host-boundary)
29
+ 11. [Development and evidence](#development-and-evidence)
30
+ 12. [Research and prior art](#research-and-prior-art)
31
+ 13. [Release notes](#release-notes)
32
+ 14. [Limits](#limits)
33
+ 15. [License](#license)
177
34
 
178
35
  ## Install and update
179
36
 
@@ -190,8 +47,16 @@ Local checkout installs are for development, not distribution:
190
47
  pi install /path/to/pi-stack/packages/pi-supernova
191
48
  ```
192
49
 
193
- 0.9.0 adds Sharp for image validation. Keep its platform-specific optional
194
- dependencies enabled. For a local checkout, refresh dependencies before starting
50
+ Image reads and returned images are validated with Sharp: at most 32 million
51
+ decoded pixels across all frames, one image at a time, in a subprocess with a
52
+ 5-second kill deadline; text-only work never loads it. Exact-content cache hits
53
+ do not wait for unrelated decoders. Cancelled queued validations withdraw without
54
+ releasing the active decoder's memory slot. Decoder errors retain that slot and
55
+ watchdog until the child closes; repeated errors keep the original diagnostic.
56
+ Whole image reads and whole text
57
+ windows sign their loaded bytes rather than rereading the file; partial windows
58
+ still sign the complete file. Keep Sharp's
59
+ platform-specific optional dependencies enabled. For a local checkout, refresh dependencies before starting
195
60
  the host:
196
61
 
197
62
  ```bash
@@ -203,11 +68,11 @@ between operating systems or CPU architectures. Published-package installation
203
68
  resolves these dependencies through the host's package manager.
204
69
 
205
70
  Git pushes do not update npm installations. Publish the new npm version first,
206
- then reinstall it in your host. To pin **0.9.0** once it is published:
71
+ then reinstall it in your host. To pin **0.11.0** after it is published:
207
72
 
208
73
  ```bash
209
- pi install npm:pi-supernova@0.9.0
210
- omp install npm:pi-supernova@0.9.0
74
+ pi install npm:pi-supernova@0.11.0
75
+ omp install npm:pi-supernova@0.11.0
211
76
  ```
212
77
 
213
78
  In Pi, `pi list` shows the configured package sources. A local path uses that
@@ -242,10 +107,15 @@ settings. The runtime does not silently rewrite your tool policy.
242
107
  | `edit` | `edit(view,text)` CAS-replaces that span; `edit(view,old,new)` is unique inside it |
243
108
  | `edit` | `edit({path,patch})`; unified patch application |
244
109
  | `edit` | `edit(async () => {...})`; filesystem-only checkpoint, described below |
245
- | `write` | `write(path,text)`, `write({path,content})`; atomic replacement |
110
+ | `write` | `write(path,text,options?)`, `write({path,content})`; atomic replacement |
246
111
  | `bash` | `bash(command,{cwd,timeoutMs})`, `bash({command,timeoutMs})`; bounded output, nonzero exits throw |
247
112
  | `bash` | `bash({command,args:[...]})`; literal executable argv, without shell expansion of argument strings |
248
113
 
114
+ Patch hunks must advance past the preceding hunk's output region, including its
115
+ context lines. Neither nominal matches nor relocation may consume earlier output;
116
+ adjacent hunks at the boundary remain valid. A rejected patch leaves its target
117
+ unchanged, and an uncaught rejection rolls back the current staged writes.
118
+
249
119
  `bash` also accepts `timeout` in seconds for familiar object arguments. `timeoutMs`
250
120
  is milliseconds and takes precedence. The owned adapter launches executable argv
251
121
  directly on every supported OS, including Windows; use the string form for shell
@@ -257,6 +127,10 @@ Windows limits process command lines to 32K characters; use a workspace script f
257
127
  for larger payloads. Argument payloads are not repeated in owned direct-execution errors;
258
128
  stdout/stderr, exit status and source context remain. Session environment variables are taken
259
129
  from the current execution context, not inherited from a different parent session.
130
+ Pending owned writes are committed before foreground or background launch. A `cwd`
131
+ created by those writes is accepted, including in-workspace symlink spellings.
132
+ Unrelated missing directories, file paths, and escaping paths still fail before
133
+ pending files commit; bash remains forbidden inside an edit checkpoint.
260
134
 
261
135
  Shell strings are executed unchanged, including quoted executable paths. For inline
262
136
  Python/Node scripts, prefer literal argv with `data` instead of nested shell quotes:
@@ -303,7 +177,20 @@ not as an absent match. Do not combine incompatible modes such as `outline:true`
303
177
  and `evidence:true`.
304
178
 
305
179
  Ordinary reads stay self-contained. Outlines and graph evidence remain explicit
306
- options, not mandatory stages of source resolution. Ordinary calls also get:
180
+ options, not mandatory stages of source resolution. An exact declaration-name
181
+ `about` query expands matching declarations ahead of enclosing keyword-heavy
182
+ scopes; this also preserves identifiers such as `get` that are prose stop words.
183
+ Other questions retain relevance ranking, and folded declarations keep line-based
184
+ recovery guidance. If the view exceeds its budget, expanded bodies appear before
185
+ the folded declaration map with their original source line numbers; clipped
186
+ navigation is explicitly disclosed. Views that fit retain source order.
187
+ Focused outlines collect
188
+ lexical caller hints for all expanded declarations in one workspace scan. The
189
+ per-name order and counts are preserved, staged callers stay visible, and the
190
+ matching-row snapshot lives only for that outline. Existing index cache rules
191
+ still apply; these hints are not semantic caller resolution. Focused text shaping
192
+ for files without declarations skips the unrelated caller catalog entirely.
193
+ Ordinary calls also get:
307
194
 
308
195
  - Bounded fuzzy filename hints when a bare source name has no literal match. Existing
309
196
  frecency/directory ranking orders hints; fuzzy matches never select a path. This
@@ -321,12 +208,52 @@ options, not mandatory stages of source resolution. Ordinary calls also get:
321
208
  Distant edit regions have separate windows and continuation pointers for omitted
322
209
  lines. Structural warnings and source windows are not substitutes for tests.
323
210
 
211
+ ### Optional indexed source discovery
212
+
213
+ ```js
214
+ const view = await read({query:"formatLabel", path:"src/labels.js", indexed:true});
215
+ if (view.status !== "found") return view;
216
+ await edit(view, "label.trim()", "label.trimStart()");
217
+ ```
218
+
219
+ This opt-in route uses a callable `isearch` tool (pi-indexer protocol 6), then
220
+ performs Supernova's own byte read and returns an ordinary source view. Indexer
221
+ text/fingerprints are locators, not read receipts or edit tokens. Returned paths
222
+ must stay inside both the workspace and requested scope, including symlink targets.
223
+ Canonical absolute locator paths for an aliased workspace are mapped back to its
224
+ logical scoped path; this does not admit external paths or escaping symlinks.
225
+ Multiple or incompletely selected candidates remain ambiguous. Staged writes use
226
+ the existing overlay-aware resolver instead, so uncommitted source stays visible.
227
+ Unavailable/disabled/excluded providers fail explicitly. The host's callable set
228
+ is authoritative: on Pi v0.99, inactive `codemode`/`deferred` tools may remain
229
+ callable, while hidden, model-only and inactive direct tools cannot be invoked.
230
+ Normal call budgets and traces include the internal isearch operation.
231
+
232
+ Pi v0.99 delegation uses the current tool invocation's `ctx.tools` and
233
+ `ctx.executeTool()`. Indexer and host overrides pass through host validation,
234
+ permissions, cancellation and nested tool hooks; a denied call never falls back
235
+ to a captured raw executor. No legacy event-bus delegation runs on that path.
236
+ Older Pi hosts can use the synchronous, version-1
237
+ `pi-indexer:readonly-provider` handshake, bound to the active definition's schema
238
+ identity and session. Exactly one provider is accepted; withdrawal, replacement,
239
+ session changes and shutdown revoke access. That older read-only API does not
240
+ replay nested Pi hooks. OMP keeps its native registry/approval route.
241
+
242
+ Both packages work independently. Ordinary source reads retain their existing
243
+ resolver. Explicit windows, about, evidence, outline and JSON cannot combine with
244
+ indexed:true. Named edits already read source internally; no new blanket prior-read
245
+ authorization rule is claimed. View-span comparison and commit CAS are unchanged.
246
+
324
247
  ### Safe read-modify-write
325
248
 
326
249
  A plain file read returns its complete text up to the **64 MiB UTF-8 I/O limit**,
327
250
  independently of model-facing character budgets. Explicit `offset`/`limit` reads
328
- return exact line windows, including long lines and LF/CRLF endings, or fail at
329
- the same byte ceiling. Staged and on-disk data obey the same ceiling.
251
+ return exact line windows, including long lines, BOM characters and LF/CRLF
252
+ endings, or fail at the same byte ceiling. Malformed UTF-8 fails rather than
253
+ discarding bytes. Only JSON parsing consumes a leading BOM. Staged and on-disk
254
+ data obey the same ceiling. Line-statistics scans adapt to dense newline regions
255
+ after collecting the bounded preview, then return to native searches for sparse
256
+ regions; counts, EOF handling and preview bytes remain unchanged.
330
257
  `complete:true` additionally rejects a window that omits part of the file.
331
258
  For larger files/JSONL, use a bounded parser through `bash({command,args})`.
332
259
 
@@ -334,6 +261,10 @@ Compute on the full value and return only what the model needs. A direct large
334
261
  return is an explicitly truncated display, not a complete artifact to parse or
335
262
  write back. `resolve:true` source previews retain path/range/continuation metadata;
336
263
  clipped previews are not editable. Prefer `edit` for replacements.
264
+ Plain read text is not an editable view: pass the filename to `edit`, or use
265
+ `read(path,{resolve:true})`. Overlong edit/patch paths fail with bounded recovery
266
+ guidance instead of echoing a source body mistaken for a filename. Near-match
267
+ previews show exact nearby bytes without guessing why a replacement failed.
337
268
  A write after reading that path still requires `edit` or explicit `replace:true`.
338
269
  Writes reject Supernova truncation markers, including legacy host-result markers.
339
270
  For intentionally writing literal marker documentation only, opt in with
@@ -341,9 +272,18 @@ For intentionally writing literal marker documentation only, opt in with
341
272
  full dataflow tracking or a security sandbox.
342
273
 
343
274
  Read/modify/write conflict checks retain a signature of the actual disk bytes,
344
- including for partial and large-file reads. A fresh explicit text read refreshes
275
+ including for empty windows, partial and large-file reads. A fresh explicit text read refreshes
345
276
  that observation; internal receipt reads and body-cache eviction do not. Commits
346
- reject changed content and conflicting symlink aliases, including new file paths.
277
+ reject changed content; explicit checkpoints also reject conflicting symlink aliases,
278
+ including new file paths.
279
+ Overlays, read observations and commits share canonical filesystem identity:
280
+ reading through another symlink spelling sees staged bytes, including in directory
281
+ listings and source queries. A retargeted observed or staged path conflicts rather
282
+ than redirecting a write. Internal edit/patch reads do not refresh that observation.
283
+ Logical paths remain in receipts. Identity lookups are shared only within one
284
+ native operation; the next operation and commit resolve them afresh. Missing paths
285
+ canonicalize their existing ancestors; case/Unicode collisions between new names
286
+ are checked by the filesystem at publication, not guessed by case folding.
347
287
  These checks do not provide a cross-process lock or make shell/import mutations
348
288
  transactional. Extensionless filenames also support `complete:true`, for example
349
289
  `read({path:"LICENSE",complete:true})`.
@@ -406,7 +346,9 @@ an arbitrary JavaScript module or a Python/shell script.
406
346
 
407
347
  Program files must be regular UTF-8 files inside the workspace, including symlink
408
348
  targets. Invalid encoding, oversized input and syntax errors fail before commands;
409
- no truncated prefix is executed. Review untrusted source before running it.
349
+ no truncated prefix is executed. A file changed during loading is rejected before
350
+ compilation; diagnostic source windows omit unstable snapshots as well. Review
351
+ untrusted source before running it.
410
352
  Use ordinary `edit` to revise saved programs. This is explicit source reuse, not
411
353
  conversation compression: prior calls and read results remain intact. Creation
412
354
  costs an additional call unless combined with other work, so prefer inline code
@@ -480,30 +422,42 @@ arguments do not authorize result deduplication or context rewriting. Existing
480
422
  programs that depend on whole-input replacement keep that behavior unless the
481
423
  caller explicitly requests `mergeData:true`.
482
424
 
483
- The batch stops on the first failed entry, cancellation/deadline, or exhausted
484
- log/image budget. Display-text clipping does not stop execution or mark a
485
- successful batch failed. Earlier successful commits remain; only the active
486
- program's uncommitted writes roll back. Admission errors throw before any program.
487
- Execution failures return a **typed stop report**, rather than throwing away prior
488
- results/images: isError and details.ok identify failure, details.programs contains
489
- every attempted result, and details.attempted/total identifies unstarted work.
490
- Single code/file invocations retain their existing throwing behavior.
425
+ Independent batch entries continue after ordinary failures, including in sequential
426
+ mode. Cancellation/deadline, exhausted shared call/image budgets, or an uncertain
427
+ filesystem outcome stop remaining work. Display-text and console-log clipping do not stop execution or mark
428
+ a successful batch failed; clipped logs set `details.logTruncated` and carry a
429
+ bounded warning. Completed edit/write steps remain saved; only an active explicit
430
+ checkpoint's uncommitted writes roll back. Admission errors throw before any program.
431
+ Execution failures return a **typed partial/stop report**, rather than throwing away prior
432
+ results/images. A `tool_result` hook also projects the error flag into Pi, which
433
+ otherwise treats a resolved tool call as successful. `isError` and `details.ok`
434
+ identify failure, `details.programs` contains
435
+ every attempted result with its zero-based `details.programIndex`.
436
+ `details.failedPrograms` and `details.notRunPrograms` identify exactly what needs
437
+ attention; `details.attempted/total` gives the counts. Single code/file invocations
438
+ still throw on uncaught errors, but saved edits are listed in the error and
439
+ `details.savedPaths`. Do not repeat them. Arbitrary dependent JavaScript cannot
440
+ be resumed after an uncaught exception; split independent work into `programs`
441
+ or use `Promise.allSettled` for independent commands.
491
442
 
492
443
  Set `parallel: true` with `programs` to run independent entries concurrently
493
444
  (up to 8 at once). Each still gets a fresh guest and its own commit; results stay
494
445
  in submission order. A failed entry does not stop siblings. Two entries writing
495
446
  the same file race: the losing commit reports a conflict. Sequential remains the
496
447
  default. `parallel` and `mergeData` are invalid on a lone `code` or `file` call.
497
- Log/image budget overflow marks the batch failed and stops queued entries;
498
- already-running entries settle and their completed commits remain. Text overflow
499
- only clips the displayed result and sets `details.returnTruncated`; queued entries
500
- still run. Parallel execution does not multiply the aggregate allowances.
501
-
502
- The outer deadline, host-call budget, log allowance, text budget and image limits
503
- are shared across the batch. Individual read budgets are not reduced. Every
504
- attempted program's text is assembled in length-delimited blocks, then the combined
505
- display is clipped if necessary. Images retain program/image labels. Return
506
- summaries or use focused windows when you need every result to fit on display.
448
+ Image budget overflow marks the batch failed and stops queued entries;
449
+ already-running entries settle and their completed commits remain. Text/log
450
+ overflow only clips delivery and sets `details.returnTruncated` or
451
+ `details.logTruncated`; queued entries still run. Parallel execution does not multiply the aggregate allowances.
452
+
453
+ The outer deadline, host-call budget, batch log allowance, text budget and image
454
+ limits are shared across the batch. Batch `details.logs` stays capped at the log
455
+ allowance; individual receipts retain their bounded per-program logs. Individual read budgets are not reduced. Every
456
+ attempted program's text is assembled in length-delimited blocks. Oversized batches
457
+ divide the available text budget among per-program previews, so a large success
458
+ does not hide a later failure. Extremely small budgets may also clip the framing.
459
+ Images retain program/image labels. Return summaries or use focused windows when
460
+ you need complete results on display.
507
461
 
508
462
  Batch only continuations already chosen by the agent, such as edit then known
509
463
  verification, or create then run known audits. Keep a separate call whenever new
@@ -518,15 +472,25 @@ to serialized JSON `data`, including quote/newline escaping and object keys.
518
472
  Oversized input fails before commands run and reports its actual serialized size.
519
473
  Split larger documents into
520
474
  separate invocations: first `write(path, firstChunk)`, then
521
- `write({path,content:nextChunk,append:true})`. Append uses the complete internal
475
+ `write({path,content:nextChunk,append:true})`. Positional `write(path,text,options)`
476
+ also accepts `append`, `replace`, and `allowReadArtifacts` boolean flags. Unknown
477
+ options and mixed object/positional signatures reject; options never override the
478
+ path or content arguments. Append uses the complete internal
522
479
  file buffer, never a bounded model-facing read; it retains conflict checks and
523
- per-program rollback. Missing files are created. Multiple invocations are not
480
+ explicit-checkpoint rollback. Missing files are created. Multiple invocations are not
524
481
  one atomic transaction: for an all-or-nothing publication, assemble a new staging
525
482
  file and publish it only when complete. External write overrides reject append.
526
483
 
527
484
  Supernova supports bounded foreground execution and session-owned background terminals
528
485
  (see below), not durable jobs across host restarts. For long archive scans, use a
529
486
  background terminal or resumable chunks and write progress records under `.work`.
487
+ Owned foreground `bash` returns complete combined stdout/stderr to the program
488
+ within a 2,097,152 UTF-16-code-unit capture ceiling; model/trace output remains a
489
+ bounded preview. Exceeding the capture ceiling throws instead of returning
490
+ corrupted partial JSON. Redirect larger output to a file and use
491
+ `read({path,complete:true})` or JSON projection. External bash overrides remain
492
+ authoritative and may have their own output limits.
493
+
530
494
  Foreground shell commands inherit the current program `timeoutMs` unless they
531
495
  specify their own; increasing the outer deadline no longer
532
496
  leaves a hidden 60-second shell cap. Set the inner `bash` timeout shorter than the
@@ -535,15 +499,77 @@ deadline covers **all** waits and commands, including `sleep`; a shell's own `ti
535
499
  command does not extend it. On a deadline or cancellation, the worker stops and
536
500
  pending host calls get a bounded 250ms drain to retain owned-shell diagnostics and
537
501
  finalize process termination. Non-cooperating host executors may still outlive that
538
- drain. Cancellation is reported separately from timeout; neither triggers a retry.
539
- Progress files survive shell execution but staged VFS writes may roll back.
502
+ drain. The same deadline covers the final workspace commit; cancelled queued
503
+ commits withdraw without writing later or releasing another transaction's lock.
504
+ Read-only results do not wait for that lock. In-progress filesystem I/O and safe
505
+ rollback must settle before reporting completion, so they can extend elapsed time.
506
+ Cancellation or session invalidation during publication triggers recovery, including
507
+ on the final file. After a completed flush, cancellation keeps the committed files
508
+ but prevents the pending external command from starting; those versions are not
509
+ reported as rolled back.
510
+ Cancellation is reported separately from timeout; neither triggers a retry.
511
+ Progress files and completed edit/write steps survive later failures; active explicit
512
+ checkpoint writes may roll back.
513
+ Termination is a hard stop: guest `catch`/`finally` cleanup is not guaranteed to run.
540
514
  On macOS/Linux, foreground and background commands share process-group cleanup:
541
515
  normal leader exit, cancellation and timeout retire ordinary descendants before
542
516
  reporting completion. A shell's `command &` does not create a durable job; launch
543
517
  the long-running command itself with `background:true` instead.
544
518
 
545
- Large returned objects are bounded previews, not retained artifacts. Select fields
546
- and array windows before returning, rather than parsing a truncated preview.
519
+ Filesystem paths preserve literal leading/trailing filename whitespace; malformed
520
+ Unicode paths are rejected instead of aliasing replacement-character filenames.
521
+ Owned text writes, appends, edits and patches require well-formed Unicode. Unpaired
522
+ UTF-16 surrogates are rejected rather than silently encoded as replacement characters.
523
+ Valid surrogate pairs, BOMs, combining characters and line endings are preserved.
524
+ Use an explicit byte-writing command for non-UTF-8 file content.
525
+
526
+ Large returned objects are bounded previews, not retained artifacts. Oversized
527
+ named/record batches divide the display budget among sibling entries instead of
528
+ letting the first file hide the rest. Clipped strings disclose their original
529
+ UTF-16 length; collections that cannot fit disclose omitted item counts. Error
530
+ messages escape lone surrogate code units before display budgeting, preserving
531
+ valid Unicode and newlines. This
532
+ does not shorten values inside the guest or create resumable artifact handles.
533
+ Select fields and array windows before returning, rather than parsing a truncated
534
+ preview; read explicit file windows when you need complete source.
535
+
536
+ ## Tokens: native tool calls vs Supernova
537
+
538
+ 0.11.0 keeps complete results: committed native edits and writes stay, independent `programs` continue after ordinary errors, and `indexed:true` opens owned source.
539
+ Source and result text are not compressed; the remaining lever is the standing definition (645 / 633 tokens on o200k_base / cl100k_base).
540
+ Fixed 13-program replay (`js-tiktoken` 1.0.21): 28,130 -> 9,819 tokens on o200k_base (65% less than unbatched). Shared source and data, 16 programs, one model call: 17,771 -> 3,487 (80%).
541
+ Eight programs sharing a 48-path input: 16,309 -> 5,527 (66%). Same six-call schedule versus `d444eb7`: 18,535 -> 9,819 (47%).
542
+ Local 8-file worker, Apple M5 Max, Node 22.23.3: unbatched cold p95 19.7 ms (8 bridge calls); coalesced pristine-warm p95 3.1 ms (1 call).
543
+ Method, both encodings, and limits: [TOKEN_COSTS.md](https://github.com/AdityaVG13/pi-stack/blob/main/packages/pi-supernova/docs/TOKEN_COSTS.md).
544
+
545
+ ## TUI presentation
546
+
547
+ Pi and OMP use the compact Nova result card. The default view stays anchored to
548
+ its latest eight calls during execution and after completion, rather than jumping
549
+ back to the earliest operations. Completion is explicitly labelled.
550
+
551
+ - Successful collapsed cards show call rows, not returned source or JSON bodies;
552
+ expand to inspect the result. JavaScript-only runs still show a short result.
553
+ Latest-change previews use five diff rows; failures keep at most eight
554
+ diagnostic rows with expansion hints. Failed or rolled-back changes are
555
+ labelled as attempts, not successful persistence.
556
+ - Expansion shows up to 24 recent calls, up to 24 diff rows per operation, logs,
557
+ and a larger result preview. It is not an unlimited transcript or file artifact.
558
+ - Partial updates deliver the latest state on a coalesced 32 ms cadence (about
559
+ 31 updates/second at most), not one repaint per operation. Completion cancels
560
+ queued updates; snapshot frames do not change after delivery.
561
+ - Each displayed value has a 32,000-character UI text budget before layout.
562
+ Long strings retain a labelled head/tail preview; clipping does not shorten
563
+ the machine-facing value. Select fields or read explicit source windows when
564
+ complete content is needed. Expansion does not remove the UI text budget.
565
+ - Printable ASCII wraps in bulk; Unicode uses grapheme-aware wrapping. Each card
566
+ retains one bounded, uncolored result layout across repaints and expansion.
567
+ Content or width changes replace it; theme invalidation still repaints all rows.
568
+ Terminal-width safety remains part of the renderer contract.
569
+
570
+ These are package-owned renderers using the host TUI, not modifications to Pi or
571
+ a second terminal renderer. Headless host checks establish layout and repaint
572
+ contracts, not live-terminal frame-rate, subjective smoothness or model speed.
547
573
 
548
574
  ## Background terminal sessions
549
575
 
@@ -595,7 +621,10 @@ return await bash({action:"list"});
595
621
  - `cursor` is an absolute UTF-16 output offset. Pass the previous cursor for
596
622
  incremental output, or omit it to replay retained output. stdout/stderr share
597
623
  a 65,536-character tail. `truncated:true` and `outputStart` disclose discarded
598
- history; polling does not consume it. `waitMs` is 0--30,000 (default 0).
624
+ history; polling does not consume it. A retained boundary or manually supplied
625
+ cursor inside a UTF-16 pair advances to the next complete code point, with
626
+ `outputStart` and `truncated:true` disclosing the skipped partial character.
627
+ `waitMs` is 0--30,000 (default 0).
599
628
  - Input is literal, at most 16,384 characters per call. Include `\n` for Enter
600
629
  and `\u0003` for Ctrl-C in a PTY. Input is not echoed into tool traces, but
601
630
  the child/terminal may echo it into output. Do not send secrets casually.
@@ -636,7 +665,6 @@ return await bash({action:"list"});
636
665
  override is rejected rather than silently ignoring background options or
637
666
  bypassing the override's permissions.
638
667
 
639
-
640
668
  ## JSON reports and targeted text audits
641
669
 
642
670
  For JSON, select fields inside the read adapter, **before** output budgeting:
@@ -647,6 +675,9 @@ return {verdict, values};
647
675
  ```
648
676
 
649
677
  Selectors support "." (root), .field, .nested[0], .items[0:10], and .["quoted.key"].
678
+ Hyphenated names such as `.credentials.openai-codex` are literal own-property
679
+ keys, not subtraction. Other punctuation and leading hyphens use quoted keys;
680
+ selectors still never evaluate code or traverse prototypes.
650
681
  Use json:true for the complete parsed value. Put the selector in `json`
651
682
  (`json:".field"`), not a second `selector` key; a leftover `selector` folds
652
683
  when `json` is absent, `true`, or `"."`. Selectors are not full jq: pipes,
@@ -662,7 +693,9 @@ separate **64 MiB estimated storage limit per read**, including aggregate
662
693
  multi-selector expansion. Within it, arrays/objects/scalars retain their actual
663
694
  values, even when larger than the display. No routing object is substituted:
664
695
  `.map` and `.filter` work on the selected array. Multi-selector values remain
665
- independently mutable. The input/storage safety limits throw with guidance.
696
+ independently mutable. Storage accounting charges scalar strings in the same
697
+ pass rather than retaining a second string worklist; container and alias
698
+ accounting, aggregate limits and failure guidance are unchanged.
666
699
  Plain .json reads are raw text, including malformed JSON; parsing is requested
667
700
  only by `json`. Explicit line windows are not necessarily JSON documents.
668
701
  Do not combine json with complete, line windows, or source views. External read
@@ -680,7 +713,7 @@ Uncaught read errors abort the program, including `return {a:await read(...),
680
713
  b:await read(...)}`; earlier successful values are not an implicit partial return.
681
714
  For optional sources, explicitly return `await Promise.allSettled(paths.map(path =>
682
715
  read(path)))`. This keeps successful text and per-path errors without weakening
683
- rollback for uncaught failures.
716
+ explicit checkpoint rollback. Completed edits outside a checkpoint remain saved.
684
717
 
685
718
  Other read options, even false-valued flags, do not bypass a captured external
686
719
  read executor; its policy, transforms and failures remain authoritative.
@@ -690,8 +723,21 @@ explicit offset/limit. Use complete reads for computation within the 64 MiB
690
723
  ceiling, and return a summary. JSON over 16 MiB needs a streaming parser via bash. Arbitrary returned objects still have bounded previews, not implicit
691
724
  continuation handles.
692
725
 
726
+ Successful subprocess cleanup wakes on pipe closure rather than waiting for the
727
+ fallback poll. Completion still requires both closed output pipes and retired
728
+ owned POSIX process groups; cancellation, escalation and cleanup deadlines are
729
+ unchanged. This applies to source-search subprocesses and background terminals.
730
+
693
731
  ## Execution and automatic batching
694
732
 
733
+ ```js
734
+ await Promise.all([
735
+ edit("src/a.js", "oldA", "newA"),
736
+ edit("src/b.js", "oldB", "newB"),
737
+ ]);
738
+ return await bash("npm test");
739
+ ```
740
+
695
741
  Put already-known independent reads and checks in **one** Supernova program using
696
742
  `Promise.all` (or `Promise.allSettled` when failures should remain independent).
697
743
  Sequence edits and their known verification in that same program. Start another
@@ -723,18 +769,30 @@ and non-yielding code can prevent sampling until it reaches a command or returns
723
769
  The outer deadline still terminates non-yielding workers. This is not a hard
724
770
  process-memory or security boundary.
725
771
 
726
- File changes are staged until program success. A throw before an external-mutation
727
- barrier rolls them back. Shell execution flushes preceding changes; external shell
728
- side effects cannot be rolled back. Stale commits fail explicitly rather than
729
- silently overwriting successful concurrent changes. This is not a cross-process
730
- filesystem lock. Outcomes explicitly report committed/rolledBack **file versions**
772
+ Each successful `edit` or `write` command publishes its changes before returning.
773
+ A later failed command, returned-image error, or uncaught exception does not undo
774
+ those saved steps. A single multi-replacement `edit({path,edits:[...]})` is still one
775
+ atomic step: its replacements use original coordinates and must all validate.
776
+ Use separate commands or independent `programs` for best-effort work; use an explicit
777
+ checkpoint when the changes must succeed together. Shell execution flushes any
778
+ pending changes; external shell side effects cannot be rolled back. Stale commits fail explicitly rather than
779
+ silently overwriting successful concurrent changes. New files use atomic
780
+ no-clobber publication, so filesystem-equivalent names and destinations created
781
+ during staging conflict rather than replace existing data. This requires hard-link
782
+ support on the workspace filesystem; unsupported creation fails without an unsafe
783
+ rename/copy fallback. Existing files recheck their observed version after staging
784
+ and before atomic replacement. These checks are not a cross-process filesystem
785
+ lock. Outcomes explicitly report
786
+ committed/rolledBack **file versions**
731
787
  (counted per flush/checkpoint, not unique paths) and external-call attempts. A
732
- successful inner checkpoint merges into the program, not necessarily onto disk.
733
- Pending commits or failed recovery are reported as uncertain: inspect disk and
734
- recovery backups before retrying. Import-based mutations and shell side effects
788
+ successful explicit checkpoint publishes the group together before returning.
789
+ Recovery checks destinations against the bytes this transaction published before
790
+ restoring or removing them. Detected intervening changes remain intact, and any
791
+ original backup is retained. Pending commits or failed recovery are reported as
792
+ uncertain: inspect disk and recovery backups before retrying. Import-based mutations and shell side effects
735
793
  are outside the VFS counters; this is not a filesystem audit.
736
794
 
737
- `edit(async () => {...})` creates a nested filesystem checkpoint. It returns
795
+ `edit(async () => {...})` creates an explicit atomic filesystem checkpoint. It returns
738
796
  `{ok:true,committed:true,value}` on success. On failure it rolls back and rethrows
739
797
  the cause, so an ignored failed checkpoint cannot report program success. Use
740
798
  `try { await edit(async () => {...}); } catch (error) {...}` for deliberate recovery.
@@ -743,15 +801,17 @@ checkpoints, and concurrent commands outside the active callback are rejected.
743
801
  Await the checkpoint before running shell checks. A checkpoint cannot roll back
744
802
  external shell effects. For a temporary mutation test, save the original text,
745
803
  edit and run the check outside a checkpoint, then explicitly restore it. Catch
746
- the check failure, restore, and let that program succeed so the restoration
747
- commits before reporting the failure. A `finally` restoration followed by an
748
- uncaught error is only staged and rolls back too. Keep a backup: cancellation
804
+ the check failure and restore before reporting it. A completed `finally` restoration
805
+ outside a checkpoint stays saved even if the program subsequently throws. Keep a backup: cancellation
749
806
  or a worker deadline can prevent cleanup from running.
750
807
 
751
808
  ## Context, caching and failure fidelity
752
809
 
753
810
  - Plain reads are not replaced with earlier-context references. A local cache hit
754
811
  is not proof the model still retains an earlier result after compaction.
812
+ - Concurrent windows into the same observed file version share only an in-flight
813
+ conflict-signature hash. Completed hashes are not a read cache; every window
814
+ still checks its own version, and writes retain compare-and-swap validation.
755
815
  - Text and JSON computation is independent of display limits. Source previews
756
816
  provide exact continuation; raw displayed text can be clipped and must not be
757
817
  parsed as a complete file. I/O/storage limits still fail explicitly.
@@ -802,7 +862,7 @@ provides an advisory `workspace:changed` event for independent cache/index consu
802
862
  `cwd` identifies the calling workspace. `paths` contains absolute file paths after
803
863
  a successful disk flush; paths may contain filesystem aliases. The frozen event
804
864
  and path array contain no source text. Checkpoint merges, restored rollbacks, and
805
- read-only programs emit nothing. A shell boundary can flush paths before a later
865
+ read-only programs emit no change event. A shell boundary can flush paths before a later
806
866
  program failure, so notifications are not conditional on overall tool success.
807
867
 
808
868
  `paths: null` means the changed paths are unknown: shell or delegated mutation
@@ -816,6 +876,29 @@ Observer failures cannot roll back writes. This is an optional extension convent
816
876
  not a built-in host standard, durable event log, or cross-process filesystem watcher.
817
877
  There is no dependency on or automatic routing to any consumer package.
818
878
 
879
+ ### Optional file-read receipts
880
+
881
+ A successful program may emit `workspace:read` with the same frozen
882
+ `{version:1,cwd,paths}` envelope, containing at most 256 distinct absolute paths.
883
+ These are file-open hints from native on-disk text reads, not source versions or
884
+ proof that the model saw the text. A program can summarize a read and still
885
+ qualify. Receipts cover ordinary, ranged and resolved text reads, including
886
+ inline/streamed batches; paths may contain filesystem aliases.
887
+
888
+ Provenance is carried by a private host-side marker, never inferred from source
889
+ content, result-shaped JSON or the truncated tool trace. Staged overlay reads,
890
+ empty windows, failed reads, JSON projections, directories, images and
891
+ structural/focused-only outputs do not qualify. Failed programs publish no read
892
+ receipt; caught read failures do not discard successfully delivered siblings.
893
+ Checkpoint merges do not publish before outer program success. Successful earlier
894
+ programs in a later-failing batch may already have published, like disk commits.
895
+
896
+ `pi.events.on("workspace:read", handler)` is optional, synchronous and best-effort.
897
+ Subscriber errors do not fail the read/program, and there is no import or
898
+ requirement on an indexer, cache or memory service. A consumer should deduplicate
899
+ these hints and apply its own repository/inclusion rules; this is neither a
900
+ complete I/O audit nor an authenticated or durable cross-process event stream.
901
+
819
902
  ## Security and host boundary
820
903
 
821
904
  CodeMode executes trusted JavaScript in a terminable worker, **not a security
@@ -826,14 +909,30 @@ or a separately authorized external command. Errors identify the rejected path
826
909
  and workspace, including symlink escapes. JavaScript imports and shell commands still have process
827
910
  privileges. Do not run untrusted programs as though these adapters isolate them.
828
911
 
829
- Pi preflights the outer `supernova` call. Internal primitives do not emit ordinary
830
- native `tool_call` events, so third-party guards that only recognize top-level
831
- `edit` or `bash` need CodeMode-aware handling. Configured exclusions and supported
912
+ Pi preflights the outer `supernova` call. It registers with `model-only` exposure
913
+ on hosts supporting that contract, preventing recursive invocation by native
914
+ codemode. The guest surface is still exactly `read`, `write`, `edit`, and `bash`;
915
+ this does not expose arbitrary registered tools as guest functions.
916
+
917
+ Owned filesystem primitives keep their transactional adapters and do not emit
918
+ ordinary native `tool_call` events, so third-party guards that only recognize
919
+ top-level `edit` or `bash` need CodeMode-aware handling. On Pi v0.99, delegated
920
+ host overrides and indexed discovery emit real nested hooks with the host's
921
+ parent IDs. Their usage is aggregated by Pi, not added again by Supernova. Configured exclusions and supported
832
922
  host-session execution safeguards remain enforced. Guards inspecting code/file
833
923
  inputs must also understand the programs array; its entries do not emit separate
834
924
  top-level tool_call events. Actual-host smoke checks are
835
925
  not a claim that every third-party permission extension has been validated.
836
926
 
927
+ Delegated tools with `outputSchema` can return `structuredContent`: successful
928
+ values reach the existing command as data, independently of UI text/details.
929
+ Oversized structured values become explicitly clipped JSON display strings under
930
+ `maxCallResultChars`, never silently reshaped objects or complete-file claims.
931
+ Host errors still throw through the guest command, even when the nested outcome
932
+ stores its error flag outside the result. Redaction hooks can remove structured
933
+ content; Supernova then uses the redacted text rather than stale data. Use explicit
934
+ `read({path:...})` for remote filenames that could otherwise look like symbols.
935
+
837
936
  ## Development and evidence
838
937
 
839
938
  Implementation is grouped under `src/`; all replacement tests are under `tests/`.
@@ -969,6 +1068,9 @@ not hard real-time guarantees.
969
1068
  It excludes model latency, provider tokens and prewarm time; it is not a universal
970
1069
  comparison against every CodeMode implementation.
971
1070
 
1071
+ Supernova retains its own execution engine and supported public Pi/OMP host
1072
+ APIs. A rebase onto native Pi codemode is not part of this release.
1073
+
972
1074
  See [the changelog](https://github.com/AdityaVG13/pi-stack/blob/main/packages/pi-supernova/docs/CHANGELOG.md) for changes and compatibility notes.
973
1075
 
974
1076
  ## Research and prior art
@@ -980,12 +1082,65 @@ an additional model call are not silently invoked by the tools.
980
1082
 
981
1083
  | Work | What we use it for | Where |
982
1084
  |------|--------------------|-------|
983
- | **Zero-Mem: Zero-Token Memory Operations for LLM Agents**, Xiao, Zhu, Zhang, Chen, Hong, Zhuang, Zhang, Chen, Ouyang, Ren, Huang (arXiv:2607.29377) | Evidence selection: entity–context graph with co-occurrence weights (eq. 3–4), turn/window/episode hierarchy as line/span/file (eq. 5, 11), query profile and relational/local routing (eq. 6–7), lexical entity alignment and one propagation step (eq. 8–9), personalized PageRank over spans (eq. 10), per-view normalisation and ρ-weighted fusion (eq. 12–13), closure with bridges and neighbours (eq. 14), deterministic calibration (eq. 15). Top-K = 5 follows the paper's Top-5 ≈ Top-10 finding. | `src/context/evidence.js` |
1085
+ | **Zero-Mem: Zero-Token Memory Operations for LLM Agents**, Xiao, Zhu, Zhang, Chen, Hong, Zhuang, Zhang, Chen, Ouyang, Ren, Huang (arXiv:2607.29377) | Evidence selection: entity-context graph with co-occurrence weights (eq. 3-4), turn/window/episode hierarchy as line/span/file (eq. 5, 11), query profile and relational/local routing (eq. 6-7), lexical entity alignment and one propagation step (eq. 8-9), personalized PageRank over spans (eq. 10), per-view normalisation and ρ-weighted fusion (eq. 12-13), closure with bridges and neighbours (eq. 14), deterministic calibration (eq. 15). Top-K = 5 follows the paper's Top-5 ≈ Top-10 finding. | `src/context/evidence.js` |
984
1086
  | **Agent Zero Memory: Provenance-Aware Long-Term Memory for LLM Agents**, Zhu, Wu (arXiv:2608.29606) | Every returned unit carries provenance (path, line range, verbatim text); the L0→L1→L2 read discipline (`read(query)` → `read(path, {about})` → `read(path, offset, limit)`); the citation-lock idea that a model should only cite what it actually opened. | `src/context/evidence.js`, `src/context/outline.js`, tool guidance |
985
1087
  | **Harness-of-Harness: Multi-Day Autonomous Software Development with Continual Improvement**, Yan, Su, et al. (arXiv:2609.01481) | Progressive disclosure (index first, detail on demand) and carrying evidence forward instead of reconstructing it from code. | outline / result shaping |
986
1088
  | **Act More, Decide Less: Skill-Guided Adaptive Action Chunking for Long-Horizon LLM Agents**, Yang, Jin, Zhao, et al. (arXiv:2609.02042) | Framing: one supernova program is an action chunk (one model decision, many primitive actions, stop at the first failing one). | runtime design |
987
- | **fff**, Dmitriy Kovalenko, MIT, <https://github.com/dmtrKovalenko/fff> | File search. We reimplemented fff's ranking in plain JavaScript after reading its Rust sources (`crates/fff-core/src/score.rs`, `dbs/frecency.rs`, `path_utils.rs`); the formulas and constants are fff's, the code is ours, and nothing runs out of process. Bounded fuzzy filename hints now run automatically for unmatched bare source names, using in-memory frecency and directory distance without extra filesystem probes. The full internal search implementation also retains typo-tolerant fuzzy path matching with boundary/consecutive/case bonuses; smart-case; exact-filename +40% and filename +20% bonuses; frecency boost `base·f/100` with fff's AI-mode decay (3-day half-life, 7-day window) and modification-recency steps (30s/5m/15m/1h/4h); git-modified +15%; directory-distance penalty from the current file (−1 per hop, floor −20); definition-first result hinting; fuzzy fallback on zero literal matches; weak-match cutoff; watcher-driven index refresh. Git/mtime boosts and full indexed grep are not mandatory stages of ordinary reads. Not ported: fff's SIMD/frizbee matcher (ours is an fzf-style greedy match with backward tightening), LMDB persistence (frecency is per session), and the MCP/Neovim surfaces. | `src/context/fuzzy.js`, `src/context/repo-index.js`, `src/bridge/host-bridge.js` |
1089
+ | **fff**, Dmitriy Kovalenko, MIT, <https://github.com/dmtrKovalenko/fff> | File search. We reimplemented fff's ranking in plain JavaScript after reading its Rust sources (`crates/fff-core/src/score.rs`, `dbs/frecency.rs`, `path_utils.rs`); the formulas and constants are fff's, the code is ours, and nothing runs out of process. Bounded fuzzy filename hints now run automatically for unmatched bare source names, using in-memory frecency and directory distance without extra filesystem probes. The full internal search implementation also retains typo-tolerant fuzzy path matching with boundary/consecutive/case bonuses; smart-case; exact-filename +40% and filename +20% bonuses; frecency boost `base·f/100` with fff's AI-mode decay (3-day half-life, 7-day window) and modification-recency steps (30s/5m/15m/1h/4h); git-modified +15%; directory-distance penalty from the current file (-1 per hop, floor -20); definition-first result hinting; fuzzy fallback on zero literal matches; weak-match cutoff; watcher-driven index refresh. Git/mtime boosts and full indexed grep are not mandatory stages of ordinary reads. Not ported: fff's SIMD/frizbee matcher (ours is an fzf-style greedy match with backward tightening), LMDB persistence (frecency is per session), and the MCP/Neovim surfaces. | `src/context/fuzzy.js`, `src/context/repo-index.js`, `src/bridge/host-bridge.js` |
1090
+
1091
+ ## Release notes
1092
+
1093
+ - Pi 0.99+ nested execution uses the current host invocation and its permissions;
1094
+ optional `indexed:true` source lookup opens owned, editable source bytes.
1095
+ - Final commit deadlines, concurrent filesystem reads and atomic publication
1096
+ keep transaction outcomes explicit under cancellation and contention.
1097
+ - Compact Nova cards retain successful call rows without source/JSON clutter;
1098
+ Unicode layout reuse and dense-line scans reduce measured local costs.
1099
+ - Command capture, write options, JSON selectors and edit diagnostics fix
1100
+ session-derived failures without weakening exact-match or workspace guards.
1101
+ - Image decoders retain ownership until close; repeated errors, watchdogs and
1102
+ failed retries remain fail-closed. Background output preserves code points.
1103
+ - Overlay keys, conflict signatures and commit destinations share canonical
1104
+ filesystem identity. Logical paths stay in receipts; identity reuse is
1105
+ call-local; commits resolve destinations afresh.
1106
+ - Successful native `edit`/`write` steps stay saved. Later read errors, failed
1107
+ shells, invalid images or uncaught JavaScript do not undo them. Independent
1108
+ `programs` continue after ordinary errors. `edit(async () => { ... })` and a
1109
+ single `edits[]` replacement set remain atomic. Failed results expose
1110
+ `details.savedPaths` and batch indices so only unsaved work is retried.
1111
+
1112
+ ### Previous release: 0.10.2
1113
+
1114
+ - Results print the `mutations:` line only when it informs: files committed or
1115
+ rolled back, an uncertain filesystem outcome, or shell side effects left behind
1116
+ by a failed program. It had appeared on most results, usually all zeros.
1117
+ - Clearer guest errors: naming a variable after a command (`const read = await
1118
+ read(...)`) says so, `supernova(...)` inside a program says to call the
1119
+ commands directly, and an empty `oldText` explains how to insert.
1120
+
1121
+ Earlier releases: [changelog](https://github.com/AdityaVG13/pi-stack/blob/main/packages/pi-supernova/docs/CHANGELOG.md).
1122
+
1123
+ ## Limits
1124
+
1125
+ ### Reviewed concurrency and ranking boundaries
1126
+
1127
+ Owned reads reject a view if an in-process Supernova commit overlapped its read path or search scope; disk text, window and image observations settle pending commits before publishing or validating their baseline. Query/evidence views use the same read-revision check. A rejected view must be retried. This is conservative within a directory/search scope, not a repeatable-read or multi-file snapshot promise. Read cancellation does not release another transaction's commit ownership. Staged overlay reads remain speculative by design.
1128
+
1129
+ Markdown `about` reads use bounded keyword windows over prose and code blocks, not declaration outlines. Unmatched terms report no matching text.
1130
+
1131
+ Existing destinations with multiple hard links are refused rather than silently splitting their aliases. Other processes, older Supernova instances, delegated tools and shell commands do not participate in the in-process protocol. In particular, existing-file version checks still have a check-to-rename race against an uncooperative external writer, including during recovery. Portable Node provides no atomic exchange/compare-and-swap primitive here; **external-writer losslessness is not established**. Failed recovery continues to retain backups and report uncertain state.
1132
+
1133
+ Frecency capacity eviction is now LRU. Fuzzy matching uses a length-preserving lowercase projection so expanding characters cannot shift source offsets or filename bonuses; it preserves contextual lowercase but is not full Unicode normalization. Evidence ranking remains heuristic: min-max fusion is outlier-sensitive and fixed-iteration floating-point PageRank is approximate, not a certified converged ranking.
1134
+
1135
+ ### Read-only reference outlines
1136
+
1137
+ `read({path, outline:true})` uses the same read-only filesystem path rules as
1138
+ plain reads, including absolute external references and symlinks to them.
1139
+ The outline remains bounded to a 2 MiB source read. This does not grant external
1140
+ write or edit access; those operations still require workspace admission, and
1141
+ a denied mutation rolls back pending workspace writes before a barrier.
988
1142
 
989
1143
  ## License
990
1144
 
991
1145
  MIT. fff is © Dmitriy Kovalenko and contributors, also MIT.
1146
+