pi-supernova 0.10.1 → 0.11.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (81) hide show
  1. package/README.md +376 -221
  2. package/docs/CHANGELOG.md +107 -5
  3. package/docs/TOKEN_COSTS.md +149 -324
  4. package/index.js +40 -11
  5. package/package.json +4 -3
  6. package/src/adapters/bash.js +40 -10
  7. package/src/adapters/edit.js +77 -23
  8. package/src/adapters/errors.js +4 -0
  9. package/src/adapters/index.js +1 -0
  10. package/src/adapters/list.js +6 -3
  11. package/src/adapters/read-focus.js +6 -0
  12. package/src/adapters/read-image.js +17 -9
  13. package/src/adapters/read-indexed.js +72 -0
  14. package/src/adapters/read-text.js +19 -2
  15. package/src/adapters/read.js +57 -13
  16. package/src/adapters/refs.js +17 -7
  17. package/src/adapters/write.js +14 -21
  18. package/src/bridge/host-bridge.js +152 -39
  19. package/src/bridge/invoke.js +3 -1
  20. package/src/bridge/tool-registry.js +132 -16
  21. package/src/context/evidence-graph.js +1 -0
  22. package/src/context/evidence-rank.js +1 -0
  23. package/src/context/evidence.js +4 -2
  24. package/src/context/fuzzy.js +31 -8
  25. package/src/context/outline.js +33 -9
  26. package/src/context/query.js +7 -2
  27. package/src/context/repo-index.js +8 -3
  28. package/src/context/search-files.js +20 -0
  29. package/src/context/search.js +25 -10
  30. package/src/context/snap-search.js +28 -6
  31. package/src/context/snap.js +8 -3
  32. package/src/context/source-entry.js +64 -28
  33. package/src/context/spans.js +1 -0
  34. package/src/contract/bash.js +16 -7
  35. package/src/contract/edit.js +26 -7
  36. package/src/contract/program.js +2 -1
  37. package/src/contract/read.js +46 -46
  38. package/src/fs/background.js +10 -2
  39. package/src/fs/check.js +844 -46
  40. package/src/fs/commit.js +89 -15
  41. package/src/fs/diff.js +12 -9
  42. package/src/fs/directory.js +14 -4
  43. package/src/fs/file-io.js +28 -7
  44. package/src/fs/glob.js +93 -11
  45. package/src/fs/json-read.js +3 -2
  46. package/src/fs/json-size.js +2 -1
  47. package/src/fs/lines.js +49 -2
  48. package/src/fs/patch.js +26 -10
  49. package/src/fs/process-tree.js +19 -1
  50. package/src/fs/read-window.js +36 -20
  51. package/src/fs/source-window.js +5 -10
  52. package/src/fs/text-ops.js +39 -19
  53. package/src/fs/vfs.js +296 -41
  54. package/src/fs/workspace.js +24 -6
  55. package/src/output/bottleneck.js +50 -25
  56. package/src/output/final.js +20 -0
  57. package/src/output/format.js +97 -36
  58. package/src/output/outcome.js +25 -9
  59. package/src/runtime/batch-input.js +17 -3
  60. package/src/runtime/guest-api.js +32 -7
  61. package/src/runtime/guest-worker.js +22 -4
  62. package/src/runtime/parallel.js +9 -49
  63. package/src/runtime/program-batch.js +53 -28
  64. package/src/runtime/program-file.js +2 -1
  65. package/src/runtime/program.js +94 -41
  66. package/src/runtime/reference.js +10 -9
  67. package/src/runtime/runtime.js +86 -17
  68. package/src/runtime/worker-pool.js +10 -1
  69. package/src/shared/decode.js +9 -8
  70. package/src/shared/image-worker.js +7 -0
  71. package/src/shared/image.js +58 -15
  72. package/src/shared/png.js +18 -0
  73. package/src/shared/result.js +30 -9
  74. package/src/shared/syntax-context.js +7 -1
  75. package/src/shared/utf8.js +11 -13
  76. package/src/ui/host-render.js +4 -0
  77. package/src/ui/omp-frame.js +2 -0
  78. package/src/ui/progress.js +2 -1
  79. package/src/ui/render-measure.js +11 -0
  80. package/src/ui/render.js +92 -31
  81. package/src/ui/trace.js +8 -5
@@ -1,390 +1,215 @@
1
- # Token usage and benchmarks
1
+ # Token usage and local speed (0.11.0)
2
2
 
3
- Supernova reduces repeated tool traffic through explicit program reuse, batching
4
- and shared batch input defaults.
5
- It can also reduce escaping in nested multiline results with lossless text framing.
6
- These mechanisms do not summarize results, rewrite conversation history, or change
7
- reasoning settings. Savings depend on the workload.
3
+ Supernova 0.11.0 cuts repeated tool traffic by batching already-known
4
+ continuations, sharing program source and batch input, and keeping a shorter
5
+ standing definition. Source text and program results stay complete. Native
6
+ `edit` / `write` steps stay committed; independent `programs` continue after
7
+ ordinary errors; `indexed:true` opens owned source. These mechanisms do not
8
+ summarize results, rewrite conversation history, or change reasoning settings.
9
+ Savings depend on the workload.
8
10
 
9
- See the [API guide](https://github.com/AdityaVG13/pi-stack/blob/main/packages/pi-supernova/README.md) for program-file and batch usage.
11
+ See the [API guide](https://github.com/AdityaVG13/pi-stack/blob/main/packages/pi-supernova/README.md)
12
+ for program-file and batch usage.
10
13
 
11
14
  ## Reproduce
12
15
 
13
16
  From a repository checkout with development dependencies installed:
14
17
 
15
- ~~~sh
18
+ ```sh
16
19
  npm run test:tokens --prefix packages/pi-supernova
17
- npm test --prefix packages/pi-supernova
18
- ~~~
20
+ npm run measure --prefix packages/pi-supernova
21
+ ```
19
22
 
20
- The benchmark uses js-tiktoken 1.0.21, pinned as a development dependency. It runs
21
- real Supernova programs in temporary workspaces without provider calls or downloads.
22
- The token regression gate also runs in the normal test suite.
23
+ Recorded 2026-10-04 on Apple M5 Max, darwin, Node v22.23.3, js-tiktoken 1.0.21
24
+ (dev dependency). Real programs in temporary workspaces; no provider calls or
25
+ downloads. The token regression gate also runs in the normal test suite.
23
26
 
24
- ## 0.9.0 data-path tuning (2026-09-20)
27
+ ## Model tool traffic
25
28
 
26
- Compared with the preserved pre-tuning 0.9.0 package on Apple M5 Max / Node
27
- 26.7.0. No provider calls, changed reasoning settings or regenerated token
28
- snapshots. This improves working-data fidelity and bounded execution; it is not
29
- an across-the-board latency reduction.
29
+ Traffic counts model requests: serialized definition + generated arguments +
30
+ all prior tool arguments/results replayed, including the final answer handoff.
31
+ New result text is counted when consumed, not charged twice. Includes setup;
32
+ the full startup reference is retained, with no separate discovery call.
33
+ Hand-authored text-only workload and batch schedule; only counters, timing, and
34
+ temporary write receipts are normalized. Excludes provider envelopes, unrelated
35
+ conversation, reasoning tokens, and cache/billing. No model-quality A/B claim.
30
36
 
31
- | Measurement | Before | After |
32
- | --- | ---: | ---: |
33
- | 800-entry directory, median / p95 (60 samples) | 13.179 / 23.356 ms | 8.966 / 12.795 ms |
34
- | Eight coalesced warm reads, median / p95 (80 alternating pairs) | 1.679 / 2.616 ms | 1.788 / 2.596 ms |
35
- | Eight unbatched cold reads, median / p95 (same pairs) | 12.719 / 15.057 ms | 13.136 / 15.224 ms |
36
- | Eight coalesced cold reads, median / p95 (same pairs) | 13.292 / 20.052 ms | 13.818 / 18.546 ms |
37
- | 200-row report formatting, median (10,000 iterations) | 0.1510 ms | 0.1595 ms |
38
- | Nested source formatting, median (10,000 iterations) | 0.02523 ms | 0.02480 ms |
39
-
40
- Directory results have identical full-listing hashes/counts: the median fell
41
- 32%, p95 45%. Small-read medians instead increased about 3--7%; the new accounting
42
- adds about 0.009 ms to this small report formatter. Earlier non-interleaved samples
43
- suggested a warm-read gain, so the table uses alternating measurements rather than
44
- claiming a noisy speedup. Prewarm and provider time are excluded. Output hashes
45
- for both formatting fixtures match exactly; these are local samples, not latency
46
- bounds. Directory acceptance required identical data and lower median time;
47
- read-wave acceptance required identical complete values and unchanged 8-to-1
48
- coalescing, with regressions reported rather than hidden.
49
-
50
- The frozen six-call traffic is now 9,771 / 9,647 tokens (o200k_base / cl100k_base),
51
- versus 9,813 / 9,689 at the pre-tuning checkpoint. The difference is shorter
52
- standing guidance, not missing observations. Shared-source/object-input traffic
53
- is 3,567 / 3,511; every output and all 32 files still match the frozen snapshot.
54
-
55
- Focused regressions verify the formerly failing full-text/JSON/directory cases,
56
- 64 two-MiB reads in a Node host capped at 128 MiB, blocked-delivery cancellation
57
- and follow-up use, bounded billion-character string expansions, sparse returns,
58
- and complete/partial source-view edit rules. Bun runs the same tests, but its
59
- heap flags and worker sampling do not provide Node's hard heap enforcement.
60
- Installed copies were not modified; publication and restart remain separate.
61
-
62
- ## Unreleased explicit batch reuse (2026-09-19)
63
-
64
- **79.82% / 79.92% less traffic on the shared-source/object-input workload, not a
65
- claim of 80% savings on every task.** Source and object defaults remove repeated
66
- arguments without compression, citations, result elision, context rewriting,
67
- reduced limits or changes to reasoning settings. Every program still executes in
68
- a fresh guest with its own commit. No model decision point is removed.
69
-
70
- The missing capability was combining common executable/input values with distinct
71
- per-entry inputs without repeating them or first saving helper files. Batches now
72
- accept top-level `code` OR `file` as a source default. `mergeData:true` explicitly
73
- opts into a shallow object overlay; the legacy whole-input replacement remains
74
- the default. Entries can override the source, and nested data objects are replaced,
75
- not recursively merged. Nothing is inferred or automatically deduplicated.
76
-
77
- ### Frozen eligible workload
78
-
79
- The pre-feature working tree, including the earlier unreleased papercut fixes,
80
- ran 16 independent package-doc scaffolds. Each program wrote a LICENSE and README,
81
- reread both exact staged contents, and returned its complete receipts. The control
82
- repeated the same executable and literal license/introduction with a distinct
83
- directory per entry. Its encoded program array was **32,353 characters**, within
84
- the existing 48,000-character admission cap. This is not an inadmissible baseline.
85
-
86
- The candidate supplies the same complete code and common data once, plus those
87
- same per-entry directories. Both arms use **one tool invocation plus the final
88
- model handoff**, the same 16-program schedule, and all 32 final files. The baseline
89
- was captured before the implementation; neither program source nor data was padded
90
- or changed to meet the gate.
91
-
92
- | Tokenizer | Before | After | Reduction | Argument tokens before / after | Unchanged result tokens |
93
- | --- | ---: | ---: | ---: | ---: | ---: |
94
- | o200k_base | 17,771 | 3,587 | **79.82%** | 7,685 / 643 | 1,039 |
95
- | cl100k_base | 17,595 | 3,533 | **79.92%** | 7,619 / 637 | 1,007 |
96
-
97
- Accounting is `2*definition + 2*arguments + complete result`. It includes both
98
- requests' standing definitions, generated arguments and their replay. The complete
99
- normalized output equals the pre-feature snapshot byte-for-byte, and the contents
100
- and complete inventory of all 32 files are checked. Only run counters and elapsed
101
- times are normalized, as in the existing benchmark. No hidden helper files appear.
102
-
103
- The new gate requires **at least 78% on both tokenizers**, not an argument-only
104
- percentage. Its fixture is
105
- [`batch-reuse-baseline.json`](https://github.com/AdityaVG13/pi-stack/blob/main/packages/pi-supernova/tests/efficiency/batch-reuse-baseline.json),
106
- with SHA-256 `589960c417006630872c16fae9ae5be4f5cdbdddbf86dda9fd9377651352055a`.
107
-
108
- This workload benefits from substantial repeated source and input. Unique-input,
109
- large-result, or already-file-backed jobs need not see that gain. Saving a reusable
110
- program/input file was already an alternative when extra workspace artifacts and
111
- setup were acceptable; this is not a claim of 80% over that different workflow.
112
-
113
- ### Existing workload stays separate
114
-
115
- The original six-call schedule, its argument hash, all 13 logical operations,
116
- failures, decision boundaries and complete outputs remain unchanged. It already
117
- uses saved programs and does not exercise the new source defaults. Clearer shorter
118
- standing guidance offsets the new option's schema cost:
119
-
120
- | Tokenizer | At start of this pass | Now | Additional reduction | Versus original non-batched baseline |
121
- | --- | ---: | ---: | ---: | ---: |
122
- | o200k_base | 10,191 | 9,841 | **3.43%** | 65.02% |
123
- | cl100k_base | 10,067 | 9,724 | **3.41%** | 65.07% |
124
-
125
- The definition measures 631/626 tokens, versus 681/675 at the start. The original
126
- traffic gate was strengthened from 40% to 65%, without changing its baseline. An
127
- initial shorter guidance draft failed the existing recovery/surface checks; the
128
- copyable offset recovery and explicit selector/view guidance were restored rather
129
- than removing those requirements. There is no claim of 70--80% on this workload.
130
-
131
- ### Verification and boundaries
132
-
133
- New source/overlay regressions and the reuse gate failed before implementation.
134
- The strengthened existing traffic gate also failed first. Current verification:
135
- 239 package tests, both tokenizers' gates, lint, actual Pi argument validation and
136
- TUI smoke, actual network-denied OMP execution, and 71 targeted tests on an isolated
137
- Spark copy under Node v24.16.0. The actual-host probes exercise shared source,
138
- object overlays and independent guest copies.
139
-
140
- The existing stress runner passed 33,024 independent reads, 20 commits/140 conflicts,
141
- cancellation, complete-source fidelity and program batches. On Apple M5 Max,
142
- Node v26.7.0, the 300-sample, eight-file check measured pristine-warm median/p95
143
- 1.645/2.390 ms versus unbatched-cold p95 13.848 ms; its existing acceptance check
144
- passed. This is a runtime smoke measurement, **not a new end-to-end speedup claim**.
145
-
146
- Complete-output and file equality prove the tested contracts, not unchanged
147
- live-model quality. There was no provider/model A/B evaluation, no reasoning-budget
148
- change, and no compression feature enabled. Deployment, session restart and
149
- publication are separate; installed OMP packages were not changed by this pass.
150
-
151
- ## Unreleased optimization pass (2026-09-19)
152
-
153
- Measured against the working tree immediately before this pass, **including the
154
- unreleased papercut fixes**, not against a published release or the older baselines
155
- below. Machine: Apple M5 Max, 18 CPUs, 48 GiB RAM, macOS arm64, Node v26.7.0.
156
-
157
- | Measurement | Before | After | Reduction |
158
- | --- | ---: | ---: | ---: |
159
- | Definition tokens, o200k_base | 699 | 658 | 5.87% |
160
- | Definition tokens, cl100k_base | 691 | 651 | 5.79% |
161
- | Fixed six-call traffic, o200k_base | 10,317 | 10,030 | 2.78% |
162
- | Fixed six-call traffic, cl100k_base | 10,179 | 9,899 | 2.75% |
163
- | 200-row report packaging, median | 0.1665 ms | 0.1314 ms | 21.09% |
164
- | Nested source packaging, median | 0.0366 ms | 0.0240 ms | 34.53% |
165
-
166
- A subsequent Cortex papercut follow-up adds explicit data-limit, workspace-write,
167
- and inherited-timeout guidance. That later definition measures 674/667 tokens
168
- (o200k_base/cl100k_base), with fixed-workload totals of 10,142/10,011. The table
169
- above records the optimization checkpoint, before that additional guidance.
170
-
171
- The Mac/Spark recheck adds optional-read recovery and removes the unsupported BMP
172
- attachment advertisement. The resulting definition is 681/675 tokens, with fixed
173
- traffic totals of 10,191/10,067. Both frozen-workload gates still pass. Verification
174
- now includes 235 Mac package tests, 51 focused tests on an isolated Spark source
175
- copy (Node v24.16.0), and actual Pi/OMP smoke on Mac. Installed OMP packages were
176
- not updated by either follow-up.
177
-
178
- Token savings come only from shorter standing guidance. Arguments, decision
179
- boundaries and complete logical result text remain unchanged. No compression,
180
- elision, lower output limits, or history rewriting was introduced or enabled.
181
-
182
- Packaging now copies only changed branches and skips an escaped-source rendering
183
- when the existing complete raw rendering is provably shorter. The benchmark runs
184
- 10,000 measured iterations per payload after warmup. Complete packaged-output
185
- SHA-256 hashes match before/after, including typed values, emitted text, images,
186
- logs and truncation flags. Local acceptance: identical hashes and at least 10%
187
- lower median packaging time for both fixtures; this is not a timing-sensitive CI
188
- gate or an end-to-end agent speed claim.
189
-
190
- Successful runs no longer leave a 250 ms drain timer alive; early completion of
191
- pending calls clears its fallback timer. A cold child-process probe exited at
192
- 267 ms before versus 17 ms after, while result delivery itself remained about
193
- 17 ms in both. Cancellation and the 250 ms bound on stuck calls remain tested.
194
- The 300-sample, eight-file benchmark measured prewarmed median/p95 of
195
- 1.904/2.743 ms before and 1.765/2.530 ms after. These filesystem timings are noisy;
196
- no generalized I/O speedup is claimed, and prewarming/model latency is excluded.
197
-
198
- Reproduce the local latency and stress checks:
199
-
200
- ~~~sh
201
- SUPERNOVA_MEASURE_SAMPLES=300 npm run measure --prefix packages/pi-supernova
202
- node packages/pi-supernova/tests/efficiency/stress.mjs
203
- ~~~
204
-
205
- Verification: 229 package tests, both tokenizers' existing frozen-workload gates,
206
- lint, actual Pi loader/TUI and OMP execution smoke tests, plus the stress runner
207
- (33,024 independent reads, write contention, cancellation and source fidelity).
208
- No version bump, installation update, commit or publication is part of this pass.
209
-
210
- ## Workload
211
-
212
- The baseline is a frozen, non-batched implementation snapshot with program-file
213
- reuse and nested source framing already enabled. It is not a comparison against
214
- a separately published release.
215
-
216
- | Scenario | Baseline calls | Batched calls |
37
+ The mixed-workload baseline is a frozen non-batched snapshot of the same
38
+ programs, not a separate product. All rows below deliver identical complete
39
+ outputs and files; only call and argument placement change.
40
+
41
+ Formally, with definition cost D, generated argument cost A_i, result cost R_i,
42
+ and prior tool-history cost H_i = sum(A_j + R_j) for j < i:
43
+
44
+ ```text
45
+ Total = sum(D + A_i + H_i, i = 1..N) + D + H_(N+1)
46
+ ```
47
+
48
+ ### Mixed 13-program replay
49
+
50
+ Fixed inspect / reproduce / repair / verify, JSON report update, and five-term
51
+ audit. Full tool-history replay.
52
+
53
+ | Scenario | Unbatched calls | 0.11.0 calls |
217
54
  | --- | ---: | ---: |
218
55
  | Inspect source, reproduce a failure, repair and verify | 4 | 3 |
219
56
  | Inspect, update and verify a JSON report | 3 | 2 |
220
57
  | Create an audit program and run it for five search terms | 6 | 1 |
221
58
  | **Total** | **13** | **6** |
222
59
 
223
- Both executions perform the same 13 logical programs. The batch schedule is
224
- hand-authored: it groups known edit-then-verify and create-then-audit continuations,
225
- while keeping decision points after inspection and the failing check separate.
226
- No model chooses the schedule during this test. The runtime does not infer plans
227
- or contain benchmark-specific paths, search terms, or expected results.
228
-
229
- This workload deliberately includes repeated audits that benefit from batching.
230
- It is a reproducible regression fixture, not a representative sample of all agent
231
- tasks or a guarantee of savings when a model decides how to use the tool.
60
+ | Tokenizer | Unbatched | Supernova 0.11.0 | Reduction |
61
+ | --- | ---: | ---: | ---: |
62
+ | o200k_base | 28,130 | 9,819 | 65.09% |
63
+ | cl100k_base | 27,841 | 9,657 | 65.31% |
232
64
 
233
- ## Accounting
65
+ Same six-call schedule versus frozen revision `d444eb7`: 18,535 -> 9,819
66
+ (47.02%) on o200k_base and 18,310 -> 9,657 (47.26%) on cl100k_base. The gate
67
+ requires at least another 19% on unchanged programs and results, and at most
68
+ 35% of unbatched traffic.
234
69
 
235
- For each model request, the benchmark counts:
70
+ By group, o200k_base:
236
71
 
237
- - The serialized Supernova definition: description, parameters and prompt guidance.
238
- - All preceding tool arguments and complete result text in the request history.
239
- - Newly generated tool arguments, serialized as JSON.
72
+ | Group | Unbatched calls | Unbatched tokens | 0.11.0 calls | 0.11.0 tokens |
73
+ | --- | ---: | ---: | ---: | ---: |
74
+ | repair | 4 | 4,242 | 3 | 2,245 |
75
+ | report | 3 | 4,231 | 2 | 2,253 |
76
+ | reuse | 6 | 15,807 | 1 | 1,643 |
77
+ | final handoff | 0 | 3,850 | 0 | 3,678 |
240
78
 
241
- It also counts the definition and full tool history sent for the final answer
242
- request. A newly produced tool result is counted when the next model request
243
- consumes it, not twice. Program creation is included. The complete startup
244
- reference is retained, so this workload needs no separate discovery call.
79
+ By group, cl100k_base:
245
80
 
246
- Formally, with definition cost D, generated argument cost A_i, result cost R_i,
247
- and prior tool-history cost H_i = sum(A_j + R_j) for j < i:
81
+ | Group | Unbatched calls | Unbatched tokens | 0.11.0 calls | 0.11.0 tokens |
82
+ | --- | ---: | ---: | ---: | ---: |
83
+ | repair | 4 | 4,211 | 3 | 2,206 |
84
+ | report | 3 | 4,197 | 2 | 2,217 |
85
+ | reuse | 6 | 15,632 | 1 | 1,615 |
86
+ | final handoff | 0 | 3,801 | 0 | 3,619 |
248
87
 
249
- ~~~text
250
- Total = sum(D + A_i + H_i, i = 1..N) + D + H_(N+1)
251
- ~~~
88
+ Standing tool definition per request: 908 -> 645 tokens (o200k_base) and
89
+ 901 -> 633 (cl100k_base). That serialized definition is resent on every model
90
+ request. Source and result text are not compressed.
252
91
 
253
- This assumes full tool-history replay. The JSON report includes per-scenario
254
- breakdowns. Its results field records newly produced text for inspection; that
255
- text contributes to totals through later history, not as a second charge.
92
+ ### Shared source and data
256
93
 
257
- ## Measured results
94
+ 16 independent programs, 32 final files, one model call. Complete outputs match.
95
+ Accounting is `2*definition + 2*arguments + complete result`.
258
96
 
259
- Observed for 0.6.0 on 2026-09-15, on an Apple M5 Max running macOS and Node v26.7.0.
97
+ | Tokenizer | Repeated arguments | Shared defaults | Reduction |
98
+ | --- | ---: | ---: | ---: |
99
+ | o200k_base | 17,771 | 3,487 | 80.38% |
100
+ | cl100k_base | 17,595 | 3,435 | 80.48% |
260
101
 
261
- | Tokenizer | Non-batched baseline | Batched baseline (d444eb7) | 0.6.0 | Further reduction | Total reduction |
262
- | --- | ---: | ---: | ---: | ---: | ---: |
263
- | o200k_base | 28,130 | 18,535 | 9,843 | **46.90%** | **65.01%** |
264
- | cl100k_base | 27,841 | 18,310 | 9,726 | **46.88%** | **65.07%** |
102
+ Eight programs sharing a 48-path input, complete outputs equal:
265
103
 
266
- The gate now requires at least 65% reduction on **each tokenizer for the complete
267
- workload**, not for every scenario individually. Token counts and reductions are
268
- computed from the recorded baseline and fresh execution results, not constants
269
- returned by the runtime. A second gate requires another 19% against the measured
270
- batched baseline from commit d444eb7. Its six-call argument hash is pinned as well
271
- as the original workload: removing a decision boundary cannot satisfy this gate.
104
+ | Tokenizer | Repeated arguments | Shared defaults | Reduction |
105
+ | --- | ---: | ---: | ---: |
106
+ | o200k_base | 16,309 | 5,527 | 66.11% |
107
+ | cl100k_base | 14,649 | 5,211 | 64.43% |
272
108
 
273
- ### Definition and result accounting
109
+ These are not a claim of 80% savings on every task. Shared defaults require
110
+ equal complete result text in both arms after only run-metadata normalization.
274
111
 
275
- The 0.6.0 serialized definition was 602 tokens with o200k_base and 595 with
276
- cl100k_base, versus 908 and 901 in the frozen non-batched baseline. It retains
277
- command signatures, complete-read and JSON limits, array-read failure rules,
278
- transaction boundaries, batch defaults and edit/view guidance on every request.
112
+ ### File-program reuse (arguments only)
279
113
 
280
- For the fixed six-call schedule, the accounting can also be written as:
114
+ Five later invocations of a saved program versus five inline copies, plus one
115
+ setup write. Break-even is two executions on both encodings.
281
116
 
282
- ~~~text
283
- Total = (N+1)*D + sum((N-i+2)*A_i + (N-i+1)*R_i, i = 1..N)
284
- ~~~
117
+ | Tokenizer | Five inline | Five file + setup | Saved argument tokens |
118
+ | --- | ---: | ---: | ---: |
119
+ | o200k_base | 580 | 249 | 331 |
120
+ | cl100k_base | 570 | 242 | 328 |
285
121
 
286
- The definition is counted seven times, including the final handoff. Batch result
287
- framing and every attempted program's text are counted too. One-off calls should
288
- not be assumed to benefit from batching.
122
+ ### Component formatting
289
123
 
290
- The report also includes separate source-framing and argument-reuse comparisons.
291
- Those component measurements are not total-session savings, and the reported
292
- argument-only break-even excludes other request costs. Text framing is selected
293
- by character length, not a runtime tokenizer; it need not reduce tokens for every
294
- input or encoding.
124
+ Separate from the traffic gate. Lossless text framing on nested multiline
125
+ source; compact scalars and short line lists are unchanged.
295
126
 
296
- ## Shared batch input: a separate 0.6.0 measurement
127
+ | Fixture | o200k before | o200k after | o200k saved | cl100k before | cl100k after | cl100k saved |
128
+ | --- | ---: | ---: | ---: | ---: | ---: | ---: |
129
+ | jsonSource | 1,697 | 1,488 | 12.32% | 1,680 | 1,468 | 12.62% |
130
+ | runtimeSource | 4,406 | 3,862 | 12.35% | 4,379 | 3,834 | 12.45% |
131
+ | nestedSources | 6,115 | 5,348 | 12.54% | 6,071 | 5,300 | 12.70% |
132
+ | compactScalars | 17 | 17 | 0% | 16 | 16 | 0% |
133
+ | shortLines | 17 | 17 | 0% | 17 | 17 | 0% |
297
134
 
298
- Eight independent audit programs use the same list of 48 source paths. The before
299
- arm repeats the literal `data` in every entry; the after arm supplies it once at
300
- the top level. Both execute the same programs and return the same complete source
301
- strings and typed results. The before request fits the existing admission cap;
302
- this is not a comparison against a hypothetical request that could never run.
135
+ ## Local worker speed
303
136
 
304
- | Tokenizer | Repeated-input traffic | Shared-input traffic | Reduction | Arguments before / after | Unchanged result tokens |
305
- | --- | ---: | ---: | ---: | ---: | ---: |
306
- | o200k_base | 16,309 | 5,441 | **66.64%** | 6,461 / 1,043 | 2,151 |
307
- | cl100k_base | 14,649 | 5,135 | **64.95%** | 5,684 / 945 | 2,055 |
137
+ `npm run measure --prefix packages/pi-supernova`. 200 measured waves, 8 files
138
+ per wave. Identical full per-file results; coalesced arms use 1 bridge call
139
+ instead of 8. Pristine-warm p95 is below unbatched cold. Local filesystem and
140
+ worker only: excludes prewarm time, model latency, and provider tokens.
141
+ Character counts are not token counts. Max is worst observed, not a real-time
142
+ guarantee.
308
143
 
309
- Each arm has one tool invocation followed by the final answer request:
144
+ | Arm | p50 ms | p95 ms | Bridge calls |
145
+ | --- | ---: | ---: | ---: |
146
+ | unbatched CodeMode cold | 17.69 | 19.75 | 8 |
147
+ | coalesced CodeMode cold | 20.09 | 26.56 | 1 |
148
+ | coalesced CodeMode pristine-warm | 2.31 | 3.14 | 1 |
310
149
 
311
- ~~~text
312
- Total = 2*D + 2*A + R
313
- ~~~
150
+ Coalesced cold is not faster than unbatched cold on this machine. The cold cost
151
+ is worker spawn; warm reuse is the speed win.
314
152
 
315
- Arguments are charged when generated and when replayed; the complete result is
316
- charged on handoff. The standing guidance measures 602 definition tokens per
317
- request with o200k_base (595 with cl100k_base), and that cost is included in
318
- the after totals.
319
- There is no source compression, result elision, hidden output or lost decision
320
- boundary. Programs receive fresh data copies, not a shared mutable heap.
153
+ Packaging, 10,000 iterations:
321
154
 
322
- The gate requires at least 70% less argument traffic and 60% less replay-inclusive
323
- traffic in each encoding, plus equality of the complete normalized output. This
324
- workload deliberately exercises repeated input; it is not an average task-cost
325
- estimate. The original 13-program/six-call benchmark remains separate and intact.
155
+ | Fixture | p50 ms | p95 ms | chars |
156
+ | --- | ---: | ---: | ---: |
157
+ | 200-row report | 0.163 | 0.171 | 10,474 |
158
+ | nested source | 0.037 | 0.040 | 8,046 |
326
159
 
327
160
  ## Benchmark integrity
328
161
 
329
- The [baseline fixture](https://github.com/AdityaVG13/pi-stack/blob/main/packages/pi-supernova/tests/efficiency/token-baseline.json) contains the source
330
- inputs, definition, workload hash, arguments and complete outputs. The
331
- [runner](https://github.com/AdityaVG13/pi-stack/blob/main/packages/pi-supernova/tests/efficiency/workflow.mjs) executes the registered tool with real
332
- workers and filesystem operations. It checks that:
162
+ The [baseline fixture](https://github.com/AdityaVG13/pi-stack/blob/main/packages/pi-supernova/tests/efficiency/token-baseline.json)
163
+ contains the source inputs, definition, workload hash, arguments and complete
164
+ outputs. The
165
+ [runner](https://github.com/AdityaVG13/pi-stack/blob/main/packages/pi-supernova/tests/efficiency/workflow.mjs)
166
+ executes the registered tool with real workers and filesystem operations.
333
167
 
334
168
  - The workload hash matches the frozen baseline, and the six-call argument hash
335
169
  matches the prior batched execution.
336
- - Every original argument and expected failure matches; complete logical results
337
- match the explicit current newline contract described below.
170
+ - Every original argument and expected failure matches.
338
171
  - Batched output contains every original result, with no unaccounted outer text.
339
172
  - No result is truncated, and final repaired source and JSON contents match.
340
173
 
341
- Only run IDs, elapsed times and temporary workspace prefixes in write receipts
342
- are normalized. Batch framing lengths are adjusted to match that normalized text.
343
- The workload, batch schedule, recorded comparison totals and acceptance thresholds
344
- are fixed test inputs, not production execution rules. The baseline is not regenerated
345
- by the benchmark and is checked against its SHA-256 before execution.
346
-
347
- Contract v2 preserves the terminating newline of one selected source line. The
348
- runner derives that single expected-output correction from the frozen input, not
349
- from candidate output. The historical fixture and its traffic counts stay
350
- untouched; programs, arguments, failures and decision boundaries are unchanged.
351
- The shared-input comparison separately requires equal complete result text in
352
- both arms, after only run-metadata normalization.
353
-
354
- Contract v3 reports write receipts relative to the workspace (`wrote rel/path`,
355
- matching the long-standing `edited <rel>` form) instead of absolute paths. The
356
- runner strips the frozen `/workspace/` prefix from baseline outputs; the
357
- historical fixture and its traffic counts stay untouched. Temporary workspace
358
- prefixes no longer appear in live receipts, so that normalization only applies
359
- to the frozen baseline side.
360
-
361
- The README and these docs ship in the npm tarball. Benchmarks and test fixtures
362
- remain in the GitHub checkout, so their links above use GitHub URLs.
174
+ Only run IDs, elapsed times, and temporary workspace prefixes in write receipts
175
+ are normalized. The workload, batch schedule, recorded comparison totals, and
176
+ acceptance thresholds are fixed test inputs, not production execution rules.
177
+ The baseline is not regenerated by the benchmark.
363
178
 
364
179
  Recorded baseline SHA-256:
365
180
 
366
- ~~~text
181
+ ```text
367
182
  96964f990f481ac05afaefdd02001bd15f61835a8381349a61e38c06209d7508
368
- ~~~
183
+ ```
184
+
185
+ Shared-source reuse fixture SHA-256:
186
+
187
+ ```text
188
+ 589960c417006630872c16fae9ae5be4f5cdbdddbf86dda9fd9377651352055a
189
+ ```
190
+
191
+ Citation elision (`seenWindow`) is experimental, disabled by default, and is
192
+ not the source of the savings above.
193
+
194
+ The README and these docs ship in the npm tarball. Benchmarks and test fixtures
195
+ remain in the GitHub checkout, so their links above use GitHub URLs.
369
196
 
370
197
  ## Limitations
371
198
 
372
199
  - These are local estimates of model tool-token traffic, not provider billing.
373
200
  Provider-specific envelopes, unrelated user/system messages, reasoning tokens,
374
- image token costs and cache discounts are excluded.
201
+ image token costs, and cache discounts are excluded.
375
202
  - There is no live-model A/B quality evaluation. Preserving observations and
376
203
  reasoning settings does not establish unchanged end-to-end task quality.
377
- - Batching is appropriate only for already-chosen continuations. Actions requiring
378
- a new model decision must remain separate calls.
379
- - The report also has an experimental citation-elision arm. It is disabled by
380
- default and is not the source of the non-compressive savings reported here.
381
- - Existing read, output, log, image and execution limits still apply. The
204
+ - Batching is appropriate only for already-chosen continuations. Actions that
205
+ need a new model decision must remain separate calls.
206
+ - Existing read, output, log, image, and execution limits still apply. The
382
207
  benchmark does not obtain savings by lowering them or hiding truncation.
383
208
 
384
209
  ## Design references
385
210
 
386
- These sources informed the implementation choices; the measurements above compare
387
- Supernova implementations only, not the performance of these packages.
211
+ These sources informed the implementation choices; the measurements above
212
+ compare Supernova implementations only, not the performance of these packages.
388
213
 
389
214
  | Reference | Mechanisms studied |
390
215
  | --- | --- |
@@ -396,6 +221,6 @@ Supernova implementations only, not the performance of these packages.
396
221
  | [pi-cache-optimizer 2.8.2](https://github.com/jiangge/pi-cache-optimizer) | Cache-prefix and prompt handling |
397
222
 
398
223
  Unpinned repository links refer to inspected source, not independently verified
399
- package versions. Supernova retains fresh guests, explicit transaction boundaries
400
- and its general command surface rather than adopting persistent notebook state,
401
- restricted CLI catalogs or provider/history rewriting.
224
+ package versions. Supernova retains fresh guests, explicit transaction
225
+ boundaries, and its general command surface rather than adopting persistent
226
+ notebook state, restricted CLI catalogs, or provider/history rewriting.