mlx-bun 0.0.13 → 0.3.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (427) hide show
  1. package/README.md +71 -578
  2. package/bin/mlx-bun.mjs +1 -1
  3. package/docs/README.md +91 -0
  4. package/docs/{design → archive}/hlg-sampling.md +3 -3
  5. package/docs/{design → archive/investigations}/audio-input-plan.md +4 -4
  6. package/docs/{design → archive/investigations}/colibri-glm52-port.md +1 -1
  7. package/docs/{design → archive/investigations}/diffusion-gemma-port.md +3 -3
  8. package/docs/{design → archive/investigations}/faithful-l1-consolidation.md +4 -4
  9. package/docs/{investigations → archive/investigations}/hlg-sampling-investigation.md +1 -1
  10. package/docs/{design → archive/investigations}/minicpm5-decode-megakernel.md +11 -11
  11. package/docs/{design → archive/investigations}/optimization_plan.md +1 -1
  12. package/docs/archive/investigations/pareto-specialized-runtime-findings.md +92 -0
  13. package/docs/{design → archive/investigations}/pre-colibri-stabilization.md +5 -5
  14. package/docs/{design → archive/investigations}/the-dreaming-handoff.md +2 -2
  15. package/docs/{planning → archive/planning}/chat-ui-vision.md +1 -1
  16. package/docs/{planning → archive/planning}/chunk-task-roadmap.md +1 -1
  17. package/docs/{planning → archive/planning}/curve-sampler-research-plan.md +4 -4
  18. package/docs/{planning → archive/planning}/memory-docs-and-dag-plan.md +1 -1
  19. package/docs/{planning → archive/planning}/release-notes-v0.0.9.md +1 -1
  20. package/docs/archive/planning/release-notes-v0.2.0.md +49 -0
  21. package/docs/{planning → archive/planning}/website-readme-pass-plan.md +1 -1
  22. package/docs/design/batching.md +569 -0
  23. package/docs/design/decode-speed-program.md +3178 -169
  24. package/docs/design/dreaming-nightly-pipeline.md +675 -282
  25. package/docs/design/generic-model-support.md +579 -334
  26. package/docs/design/kv-cache.md +680 -0
  27. package/docs/design/orpo-training.md +673 -1239
  28. package/docs/design/speculative-decoding.md +1627 -0
  29. package/docs/design/turboquant.md +2102 -0
  30. package/docs/design/unified-engine-frontier-plan.md +1831 -727
  31. package/docs/design/web-chat-redesign.md +666 -1083
  32. package/docs/planning/PRODUCT_ROADMAP.md +34 -1
  33. package/docs/planning/release-notes-v0.3.0.md +35 -0
  34. package/docs/planning/video-series.md +421 -0
  35. package/docs/reference/benchmarks.md +1162 -0
  36. package/docs/reference/cli.md +395 -149
  37. package/docs/reference/distribution.md +321 -181
  38. package/docs/reference/environment.md +348 -0
  39. package/docs/reference/glossary.md +39 -0
  40. package/docs/reference/library-api.md +580 -142
  41. package/docs/reference/memory.md +227 -64
  42. package/docs/reference/models.md +137 -6
  43. package/docs/reference/server-api.md +938 -422
  44. package/docs/reference/server-config.md +652 -320
  45. package/docs/reference/training.md +393 -254
  46. package/docs/reference/troubleshooting.md +166 -0
  47. package/package.json +10 -5
  48. package/src/admission-plan.ts +18 -0
  49. package/src/anthropic.ts +92 -3
  50. package/src/audio/decode.ts +2 -2
  51. package/src/audio/features.ts +2 -2
  52. package/src/backends/mlx/autoregressive.ts +108 -0
  53. package/src/backends/mlx/batch-group.ts +1646 -0
  54. package/src/backends/mlx/diffusion.ts +39 -0
  55. package/src/backends/mlx/draft-target.ts +47 -0
  56. package/src/backends/mlx/evaluation.ts +105 -0
  57. package/src/backends/mlx/fill-append.ts +45 -0
  58. package/src/backends/mlx/gateway-binding.ts +71 -0
  59. package/src/backends/mlx/graph.ts +38 -0
  60. package/src/backends/mlx/memory-client.ts +189 -0
  61. package/src/backends/mlx/memory-guard.ts +40 -0
  62. package/src/backends/mlx/methods.ts +90 -0
  63. package/src/backends/mlx/model-prompt.ts +164 -0
  64. package/src/backends/mlx/model-serving.ts +79 -0
  65. package/src/backends/mlx/prefill.ts +33 -0
  66. package/src/backends/mlx/request-policy.ts +20 -0
  67. package/src/backends/mlx/rollback.ts +51 -0
  68. package/src/backends/mlx/serial-executor.ts +273 -0
  69. package/src/backends/mlx/speculative.ts +114 -0
  70. package/src/backends/mlx/state-views.ts +26 -0
  71. package/src/backends/mlx/text-engine.ts +73 -0
  72. package/src/cli.ts +189 -61
  73. package/src/client.ts +66 -0
  74. package/src/config.ts +21 -4
  75. package/src/contracts/completion.ts +16 -0
  76. package/src/contracts/execution.ts +49 -0
  77. package/src/contracts/generation.ts +103 -0
  78. package/src/contracts/host.ts +7 -0
  79. package/src/contracts/jobs.ts +66 -0
  80. package/src/contracts/pi-provider.ts +30 -0
  81. package/src/contracts/pi.ts +259 -0
  82. package/src/contracts/resources.ts +18 -0
  83. package/src/contracts/scheduling.ts +27 -0
  84. package/src/contracts/tsconfig.json +14 -0
  85. package/src/diffusion/diffusion-generate.ts +218 -153
  86. package/src/embed.ts +1 -1
  87. package/src/engine/admission.ts +59 -0
  88. package/src/engine/cancellation.ts +38 -0
  89. package/src/engine/engine.ts +56 -0
  90. package/src/engine/execution-coordinator.ts +70 -0
  91. package/src/engine/execution-plan.ts +51 -0
  92. package/src/engine/resources.ts +42 -0
  93. package/src/engine/scheduler.ts +67 -0
  94. package/src/engine/session.ts +247 -0
  95. package/src/eval/README.md +44 -1
  96. package/src/eval/kl.ts +263 -23
  97. package/src/eval/reference-logits.ts +207 -0
  98. package/src/eval/runner.ts +10 -79
  99. package/src/eval/tasks/bfcl.ts +2 -1
  100. package/src/eval/tasks/gsm8k.ts +2 -1
  101. package/src/eval/tasks/hashhop.ts +2 -1
  102. package/src/eval/tasks/humaneval.ts +5 -4
  103. package/src/eval/tasks/ifeval.ts +2 -1
  104. package/src/eval/tasks/mmlu.ts +2 -1
  105. package/src/expert-io.ts +3 -1
  106. package/src/expert-offload.ts +3 -1
  107. package/src/fill/echo-index.ts +188 -0
  108. package/src/fill/fill-session.ts +452 -0
  109. package/src/fill/proposal.ts +68 -0
  110. package/src/fill/schema-rows.ts +506 -0
  111. package/src/fill/tool-boundary.ts +103 -0
  112. package/src/fit.ts +24 -43
  113. package/src/generate.ts +740 -337
  114. package/src/grammar.ts +14 -8
  115. package/src/harness-pi.ts +15 -34
  116. package/src/hub-rest.ts +4 -4
  117. package/src/index.ts +27 -0
  118. package/src/inference/denoising.ts +17 -0
  119. package/src/inference/graph.ts +33 -0
  120. package/src/inference/prefill.ts +39 -0
  121. package/src/inference/rollback.ts +7 -0
  122. package/src/jobs/db.ts +4 -0
  123. package/src/jobs/index.ts +2 -0
  124. package/src/jobs/job-entry.ts +15 -14
  125. package/src/jobs/runner.ts +98 -13
  126. package/src/jobs/task-client.ts +19 -0
  127. package/src/jobs/types.ts +2 -58
  128. package/src/kv-scheme.ts +199 -0
  129. package/src/kv-store.ts +459 -234
  130. package/src/lab/README.md +18 -0
  131. package/src/{assets → lab/curve}/curve-designer.html +2 -2
  132. package/src/{curve-sampler.ts → lab/curve/curve-sampler.ts} +2 -2
  133. package/src/{expert-trace.ts → lab/expert-trace/expert-trace.ts} +7 -5
  134. package/src/{model → lab/paged-kv}/paged-kv.ts +9 -6
  135. package/src/library.ts +40 -0
  136. package/src/lora.ts +18 -0
  137. package/src/media-fetch.ts +20 -4
  138. package/src/memory/article.ts +1 -1
  139. package/src/memory/chunk.ts +1 -1
  140. package/src/memory/model.ts +32 -170
  141. package/src/memory/prompts.ts +1 -1
  142. package/src/memory/rest.ts +1 -1
  143. package/src/memory/schedule.ts +1 -1
  144. package/src/memory/stages.ts +4 -1
  145. package/src/memory/vault.ts +3 -4
  146. package/src/mlx/array.ts +9 -10
  147. package/src/mlx/compile.ts +12 -2
  148. package/src/mlx/ffi.ts +66 -22
  149. package/src/mlx/materialize.ts +23 -0
  150. package/src/mlx/metal-kernel.ts +10 -11
  151. package/src/mlx/ops.ts +64 -19
  152. package/src/mlx/quantized-rows.ts +36 -0
  153. package/src/mlx/token-bitmask.ts +52 -0
  154. package/src/mlx/turboquant-kv-decode.ts +82 -0
  155. package/src/mlx/turboquant-kv-inverse.ts +122 -0
  156. package/src/mlx/turboquant-ops.ts +33 -3
  157. package/src/mlx/turboquant-tables.ts +2 -2
  158. package/src/model/batched-mask.ts +9 -3
  159. package/src/model/batched-quant.ts +2 -2
  160. package/src/model/batched-rotating-quant.ts +45 -99
  161. package/src/model/batched-rotating-state.ts +73 -0
  162. package/src/model/batched-rotating.ts +57 -106
  163. package/src/model/batched-row-storage.ts +127 -0
  164. package/src/model/compiled-decode.ts +4 -3
  165. package/src/model/diffusion-gemma.ts +8 -8
  166. package/src/model/factory.ts +87 -49
  167. package/src/model/fingerprint.ts +1 -1
  168. package/src/model/flash-attention.ts +3 -3
  169. package/src/model/gemma4-base.ts +187 -17
  170. package/src/model/gemma4.ts +6 -6
  171. package/src/model/generated/gemma4-12b.ts +52 -148
  172. package/src/model/generated/gemma4-26b.ts +34 -94
  173. package/src/model/generated/gemma4-e4b.ts +28 -76
  174. package/src/model/glm52-cache.ts +19 -9
  175. package/src/model/glm52-dsa.ts +13 -9
  176. package/src/model/glm52-memory.ts +28 -1
  177. package/src/model/implementation.ts +69 -0
  178. package/src/model/minicpm5.ts +7 -8
  179. package/src/model/profile.ts +443 -0
  180. package/src/model/qwen-append.ts +12 -0
  181. package/src/model/qwen-conv.ts +52 -0
  182. package/src/model/qwen3-delta.ts +98 -2
  183. package/src/model/qwen3-mrope.ts +252 -0
  184. package/src/model/qwen3_5-checkpoint.ts +164 -0
  185. package/src/model/qwen3_5.ts +336 -60
  186. package/src/model/support.ts +1 -1
  187. package/src/model/trellis-balanced-scatter.ts +51 -0
  188. package/src/model/trellis-linear.ts +672 -0
  189. package/src/model/trellis-shared-m.ts +151 -0
  190. package/src/model/trellis-shared-scatter.ts +144 -0
  191. package/src/model/trellis-splitk-prefill.ts +158 -0
  192. package/src/model/trellis-tiled-prefill.ts +136 -0
  193. package/src/model/trellis-vector-expand.ts +62 -0
  194. package/src/model/trellis-wide-prefill.ts +94 -0
  195. package/src/model/universal/dense.ts +1 -1
  196. package/src/model/universal/modules.ts +8 -1
  197. package/src/native/frame_extract.swift +140 -0
  198. package/src/native-pack.ts +15 -7
  199. package/src/pi-provider.ts +6 -64
  200. package/src/pi-web.ts +14 -320
  201. package/src/prompt-cache.ts +97 -54
  202. package/src/quantize/allocator.ts +1 -1
  203. package/src/quantize/atomic-output.ts +38 -0
  204. package/src/quantize/calibration.ts +2 -1
  205. package/src/quantize/config-writer.ts +16 -3
  206. package/src/quantize/index.ts +28 -1
  207. package/src/quantize/job.ts +11 -0
  208. package/src/quantize/quantizer.ts +265 -100
  209. package/src/quantize/rotate.ts +603 -0
  210. package/src/quantize/safetensors-writer.ts +79 -1
  211. package/src/quantize/sensitivity.ts +41 -59
  212. package/src/quantize/trellis.ts +586 -0
  213. package/src/quantize/weight-transform.ts +292 -0
  214. package/src/responses.ts +130 -3
  215. package/src/runtime-config.ts +81 -0
  216. package/src/sampler.ts +276 -4
  217. package/src/serve/admin-routes.ts +184 -0
  218. package/src/serve/aux-routes.ts +132 -0
  219. package/src/serve/batch-scheduler.ts +8 -1558
  220. package/src/serve/chat-request.ts +391 -0
  221. package/src/serve/chat-stage.ts +195 -0
  222. package/src/serve/checkpoint-identity.ts +64 -0
  223. package/src/serve/completion-executor.ts +354 -0
  224. package/src/serve/completion-sink.ts +157 -0
  225. package/src/serve/discovery-routes.ts +204 -0
  226. package/src/serve/generation-gateway.ts +211 -243
  227. package/src/serve/http.ts +159 -0
  228. package/src/serve/inference-request.ts +79 -0
  229. package/src/serve/isolate.ts +297 -105
  230. package/src/serve/kv-budget.ts +14 -0
  231. package/src/serve/lab-routes.ts +341 -0
  232. package/src/serve/lane-registry.ts +2 -1
  233. package/src/serve/model-admin-routes.ts +155 -0
  234. package/src/serve/model-binding.ts +43 -0
  235. package/src/serve/model-host.ts +512 -0
  236. package/src/serve/openai-wire.ts +103 -0
  237. package/src/serve/parent-application.ts +56 -0
  238. package/src/serve/pipeline.ts +33 -0
  239. package/src/serve/preparation.ts +33 -0
  240. package/src/serve/prompt-response-trace.ts +147 -0
  241. package/src/serve/request-plan.ts +162 -0
  242. package/src/serve/request-prep.ts +346 -0
  243. package/src/serve/response-lease.ts +24 -0
  244. package/src/serve/responses-client.ts +69 -0
  245. package/src/serve/session-completion-engine.ts +110 -0
  246. package/src/serve/session-search.ts +1 -1
  247. package/src/serve/static-routes.ts +101 -0
  248. package/src/serve/text-completion-stage.ts +104 -0
  249. package/src/serve/token-streams.ts +519 -0
  250. package/src/server.ts +523 -3551
  251. package/src/spec/assistant-source.ts +10 -44
  252. package/src/spec/deepspec-source.ts +6 -6
  253. package/src/spec/dflash-source.ts +9 -8
  254. package/src/spec/dspark/ab-stats.ts +3 -3
  255. package/src/spec/dspark/calibration.ts +2 -2
  256. package/src/spec/dspark/deepspec-module.ts +2 -2
  257. package/src/spec/dspark/generate-dflash.ts +2 -2
  258. package/src/spec/dspark/generate.ts +2 -2
  259. package/src/spec/dspark/loader.ts +1 -1
  260. package/src/spec/dspark/module-dflash.ts +5 -4
  261. package/src/spec/dspark/module.ts +3 -3
  262. package/src/spec/dspark/quantize-drafter.ts +1 -1
  263. package/src/spec/generate.ts +1 -1
  264. package/src/spec/glm52-mtp-source.ts +9 -1
  265. package/src/spec/ngram-source.ts +1 -1
  266. package/src/spec/prefix-state.ts +45 -0
  267. package/src/spec/qwen-mtp-source.ts +528 -0
  268. package/src/spec/serve-loop.ts +179 -213
  269. package/src/spec/source.ts +42 -10
  270. package/src/spec/two-model.ts +1 -1
  271. package/src/ssd-cache.ts +95 -24
  272. package/src/ssd-durability.ts +207 -0
  273. package/src/tool-call.ts +51 -14
  274. package/src/train/flash-cce.ts +14 -13
  275. package/src/train/forward.ts +38 -1
  276. package/src/train/loss.ts +3 -2
  277. package/src/train/prefix-shared.ts +5 -1
  278. package/src/train/segmented.ts +9 -6
  279. package/src/train/trainer.ts +9 -8
  280. package/src/vision/preprocess.ts +1 -1
  281. package/src/vision/prompt.ts +48 -2
  282. package/src/vision/qwen3vl-preprocess.ts +409 -0
  283. package/src/vision/qwen3vl-prompt.ts +196 -0
  284. package/src/vision/qwen3vl-tower.ts +555 -0
  285. package/src/vision/video-frames.ts +148 -0
  286. package/src/web/app.html +1 -1
  287. package/src/web/app.js +15 -6
  288. package/src/web/src/adapters-panel.ts +3 -3
  289. package/src/web/src/assistant.ts +2 -36
  290. package/src/web/src/chat.ts +16 -9
  291. package/src/web/src/composer.ts +4 -4
  292. package/src/web/src/hub.ts +1 -1
  293. package/src/web/src/markdown.ts +0 -0
  294. package/src/web/src/memory-panel.ts +1 -1
  295. package/src/web/src/model-picker.ts +1 -1
  296. package/src/web/src/palette.ts +1 -1
  297. package/src/web/src/protocol.ts +3 -39
  298. package/src/web/src/sessions.ts +1 -1
  299. package/src/web/src/ui-catalog.ts +4 -18
  300. package/src/web-assets.ts +47 -0
  301. package/src/weights.ts +134 -33
  302. package/docs/archive/mlx-bun-lab-report.html +0 -275
  303. package/docs/archive/wwdc-mlx-bun.html +0 -848
  304. package/docs/design/adapters-end-to-end.md +0 -205
  305. package/docs/design/batching-perf-path.md +0 -255
  306. package/docs/design/batching-v2-plan.md +0 -529
  307. package/docs/design/bucketing-stage.md +0 -197
  308. package/docs/design/compat-cli-surface-design.md +0 -230
  309. package/docs/design/docs-reorg-plan.md +0 -165
  310. package/docs/design/dspark-serving-program.md +0 -371
  311. package/docs/design/dspark-speculative-decoding.md +0 -268
  312. package/docs/design/grammar-spec-batching-integration.md +0 -301
  313. package/docs/design/memory-inference-path.md +0 -422
  314. package/docs/design/memory-synthesis.md +0 -287
  315. package/docs/design/memory-system.md +0 -237
  316. package/docs/design/mlx-lm-tool-parity-plan.md +0 -545
  317. package/docs/design/omlx-adoption-map.md +0 -98
  318. package/docs/design/orpo-dynamic-lambda.md +0 -220
  319. package/docs/design/orpo-future-enhancements.md +0 -340
  320. package/docs/design/paged-kv-cache.md +0 -214
  321. package/docs/design/parallel-slots.md +0 -567
  322. package/docs/design/parity-tier-dag.md +0 -161
  323. package/docs/design/repo-cleanup-plan.md +0 -189
  324. package/docs/design/runtime-isolation.md +0 -263
  325. package/docs/design/segmented-backward-training.md +0 -543
  326. package/docs/design/spec-decode-larger-targets.md +0 -229
  327. package/docs/design/ssd-kv-cold-tier.md +0 -288
  328. package/docs/design/structured-output.md +0 -484
  329. package/docs/design/tauri-desktop-app.md +0 -127
  330. package/docs/design/the-dreaming-master-plan.md +0 -445
  331. package/docs/design/turboquant-kv.md +0 -235
  332. package/docs/design/w4a16-compute-precision-spike.md +0 -232
  333. package/docs/design/web-chat-beat-matrix.md +0 -315
  334. package/docs/design/write-pipeline-entity.md +0 -468
  335. package/docs/investigations/curve-runs/bisector-route.json +0 -98
  336. package/docs/investigations/curve-runs/bisector-trace.json +0 -1393
  337. package/docs/investigations/curve-runs/curve-terrain.json +0 -1179
  338. package/docs/investigations/curve-runs/curve-terrain.svg +0 -42
  339. package/docs/investigations/hlg-ab.html +0 -34
  340. package/docs/investigations/hlg-explorer.html +0 -238
  341. package/docs/investigations/hlg-figs/cliff.svg +0 -1
  342. package/docs/investigations/hlg-figs/cross.svg +0 -1
  343. package/docs/investigations/hlg-figs/edges.svg +0 -1
  344. package/docs/investigations/hlg-figs/k20-grids.svg +0 -1
  345. package/docs/investigations/hlg-figs/k20-peaks.svg +0 -1
  346. package/docs/investigations/hlg-figs/lw-curve.svg +0 -1
  347. package/docs/investigations/hlg-figs/terrain.svg +0 -1
  348. package/docs/investigations/hlg-report.html +0 -262
  349. package/docs/investigations/hlg-runs/hlg-ab.json +0 -149
  350. package/docs/investigations/hlg-runs/hlg-blind-prompts.md +0 -87
  351. package/docs/investigations/hlg-runs/hlg-blind.json +0 -107
  352. package/docs/investigations/hlg-runs/hlg-cliff.json +0 -197
  353. package/docs/investigations/hlg-runs/hlg-cross.json +0 -505
  354. package/docs/investigations/hlg-runs/hlg-edge.json +0 -388
  355. package/docs/investigations/hlg-runs/hlg-eval1.json +0 -25
  356. package/docs/investigations/hlg-runs/hlg-frontier.json +0 -86
  357. package/docs/investigations/hlg-runs/hlg-grid.json +0 -242
  358. package/docs/investigations/hlg-runs/hlg-grid2.json +0 -343
  359. package/docs/investigations/hlg-runs/hlg-lw-coarse.json +0 -110
  360. package/docs/investigations/hlg-runs/hlg-map.json +0 -383
  361. package/docs/investigations/hlg-runs/run0-original.log +0 -159
  362. package/docs/investigations/hlg-runs/runA-recommended.log +0 -131
  363. package/docs/investigations/hlg-runs/runAA-cliff-cpm5.log +0 -26
  364. package/docs/investigations/hlg-runs/runAB-grid2-e4b.log +0 -18
  365. package/docs/investigations/hlg-runs/runAC-peak-e4b-partial.log +0 -25
  366. package/docs/investigations/hlg-runs/runAD-ab-samples.log +0 -8
  367. package/docs/investigations/hlg-runs/runAE-edge-e4b.log +0 -51
  368. package/docs/investigations/hlg-runs/runAF-logit-calibration.log +0 -10
  369. package/docs/investigations/hlg-runs/runAG-eval-config2.log +0 -16
  370. package/docs/investigations/hlg-runs/runB-gain-variants.log +0 -192
  371. package/docs/investigations/hlg-runs/runC-pivots.log +0 -162
  372. package/docs/investigations/hlg-runs/runD-oetf.log +0 -132
  373. package/docs/investigations/hlg-runs/runE-eotf.log +0 -162
  374. package/docs/investigations/hlg-runs/runF-shaper-working.log +0 -86
  375. package/docs/investigations/hlg-runs/runG-shaper-sweep.log +0 -101
  376. package/docs/investigations/hlg-runs/runH-wsweep.log +0 -176
  377. package/docs/investigations/hlg-runs/runI-asweep.log +0 -176
  378. package/docs/investigations/hlg-runs/runJ-knobmap.log +0 -49
  379. package/docs/investigations/hlg-runs/runK-frontier.log +0 -19
  380. package/docs/investigations/hlg-runs/runL-widemap.log +0 -65
  381. package/docs/investigations/hlg-runs/runM-frontier-emb.log +0 -15
  382. package/docs/investigations/hlg-runs/runN-verify.log +0 -116
  383. package/docs/investigations/hlg-runs/runO-frontier-seed5000.log +0 -15
  384. package/docs/investigations/hlg-runs/runP-grid-AxSm.log +0 -15
  385. package/docs/investigations/hlg-runs/runQ-grid-tgW.log +0 -15
  386. package/docs/investigations/hlg-runs/runR-blind-transcript.log +0 -79
  387. package/docs/investigations/hlg-runs/runS-cross-coherence.log +0 -33
  388. package/docs/investigations/hlg-runs/runT-cross-cpm5.log +0 -33
  389. package/docs/investigations/hlg-runs/runU-cross-12b.log +0 -33
  390. package/docs/investigations/hlg-runs/runV-lw-e4b.log +0 -22
  391. package/docs/investigations/hlg-runs/runW-lw-12b.log +0 -22
  392. package/docs/investigations/hlg-runs/runX-lw-cpm5.log +0 -22
  393. package/docs/investigations/hlg-runs/runY-cliff-e4b.log +0 -26
  394. package/docs/investigations/hlg-runs/runZ-cliff-12b.log +0 -26
  395. package/docs/planning/IDEAS.md +0 -69
  396. package/docs/reference/embedding.md +0 -197
  397. package/docs/reference/features-matrix.md +0 -100
  398. package/docs/reference/orpo-quickstart.md +0 -135
  399. package/src/flags.ts +0 -13
  400. /package/docs/{investigations → archive/investigations}/chunk-finetune-experiment.md +0 -0
  401. /package/docs/{investigations → archive/investigations}/colibri-oracle-pin.md +0 -0
  402. /package/docs/{investigations → archive/investigations}/curve-bisector-routing.md +0 -0
  403. /package/docs/{investigations → archive/investigations}/decode-roofline-lookagain.md +0 -0
  404. /package/docs/{investigations → archive/investigations}/dspark-handoff.md +0 -0
  405. /package/docs/{investigations → archive/investigations}/expert-offload-single-user-moe.md +0 -0
  406. /package/docs/{investigations → archive/investigations}/indexshare-performance-spike.md +0 -0
  407. /package/docs/{investigations → archive/investigations}/kernel-perf-review-2026-07.md +0 -0
  408. /package/docs/{investigations → archive/investigations}/lab-build-journal.md +0 -0
  409. /package/docs/{investigations → archive/investigations}/openwiki-evaluation.md +0 -0
  410. /package/docs/{investigations → archive/investigations}/orpo-base-uf-experiment-and-directions.md +0 -0
  411. /package/docs/{investigations → archive/investigations}/orpo-flash-cce-pin-leak.md +0 -0
  412. /package/docs/{investigations → archive/investigations}/orpo-uf-testing-handoff.md +0 -0
  413. /package/docs/{investigations → archive/investigations}/pi-builtin-investigation.md +0 -0
  414. /package/docs/{investigations → archive/investigations}/segmented-backward-handoff.md +0 -0
  415. /package/docs/{investigations → archive/investigations}/starter-model-port-handoff.md +0 -0
  416. /package/docs/{investigations → archive/investigations}/steel-flash-cce-handoff.md +0 -0
  417. /package/docs/{investigations → archive/investigations}/trainer-validation-experiment.md +0 -0
  418. /package/docs/{planning → archive/planning}/chat-ui-vision-memory-first.md +0 -0
  419. /package/docs/{planning → archive/planning}/journal.md +0 -0
  420. /package/docs/{planning → archive/planning}/pi-integration-review.md +0 -0
  421. /package/docs/{planning → archive/planning}/release-notes-v0.0.10.md +0 -0
  422. /package/docs/{planning → archive/planning}/release-notes-v0.0.11.md +0 -0
  423. /package/docs/{planning → archive/planning}/release-notes-v0.0.12.md +0 -0
  424. /package/docs/{planning → archive/planning}/release-notes-v0.0.13.md +0 -0
  425. /package/docs/{planning → archive/planning}/web-ui-pass-plan.md +0 -0
  426. /package/{docs/investigations → src/serve/assets}/curve-terrain.html +0 -0
  427. /package/{docs/dag → src/serve/assets}/training-inference-map.html +0 -0
@@ -1,1112 +1,695 @@
1
- # Web chat redesign: the definitive plan
2
-
3
- Status: proposed · Owner: web/chat surface · Scope: `src/web/app.html`, `src/pi-web.ts`, `src/server.ts` (chat routes), `src/memory/*`, `src/lora.ts` · Horizon: one quarter, phased
1
+ ---
2
+ status: active
3
+ axis: USING
4
+ canonical-for: web-chat
5
+ plan-anchor: none
6
+ last-verified: 2026-08-23
7
+ ---
4
8
 
5
- This document merges three independent design visions, ten market-research
6
- briefs, and a line-level audit of the current UI and backend into one
7
- execution plan. It supersedes `docs/planning/chat-ui-vision.md` and
8
- `docs/planning/chat-ui-vision-memory-first.md` as the working plan (keep
9
- those two as historical inputs; this is the doc that gets executed against).
10
- It also absorbs and re-prioritizes every item in `docs/planning/web-ui-pass-plan.md`
11
- — those bug-fixes are folded into Phase 0/1 below by number.
9
+ # Web chat — the built-in chat surface (canonical design)
10
+
11
+ Scope: `src/web/` (app.html + `src/web/src/*.ts` built into `app.js`),
12
+ `src/pi-web.ts` (the WebSocket bridge), the thin REST wrappers the UI calls
13
+ (`src/memory/rest.ts`, `src/hub-rest.ts`, `src/serve/session-search.ts`,
14
+ `src/serve/model-admin-routes.ts`), `src/tool-approvals.ts`.
15
+
16
+ This is the one canonical doc for the web chat. It absorbs the former
17
+ beat matrix (`docs/design/web-chat-redesign.md`, kept as Appendix A), the web half
18
+ of `docs/design/web-chat-redesign.md` (§3.6 here), and the desktop-shell idea (one
19
+ History line). Status/changelog prose belongs in PLAN.md; this file holds
20
+ the design, the as-built contract, and the open items.
21
+
22
+ **Standing decision (2026-06-14): the chat UI is a Pi embed, not a
23
+ hand-rolled chat.** The browser talks to an in-process
24
+ `@earendil-works/pi-coding-agent` `AgentSession` over `/ws/chat`
25
+ (`src/pi-web.ts`). Pi owns the agent loop, tool execution, session files
26
+ (new/list/open/fork/delete, `SessionManager.forkFrom`), the tree
27
+ navigation behind edit/regenerate (`AgentSession.navigateTree`),
28
+ compaction, thinking levels, and the steering/follow-up queue. Nothing in
29
+ this design rebuilds those; the web work is (a) frames that expose them,
30
+ (b) UI that makes them legible, (c) the few capabilities Pi has no
31
+ concept of (memory vault, LoRA adapters, serving lanes, hardware fit,
32
+ the app-aware assistant).
12
33
 
13
34
  ---
14
35
 
15
- ## 1. Executive summary
16
-
17
- **The thesis:** mlx-bun's web chat should not try to be a better ChatGPT. It
18
- should be the only local chat surface where the AI visibly, provably gets
19
- better at knowing *you*, where switching to a domain-specific brain is a
20
- first-class routing decision instead of a filename in a dropdown, and where
21
- "local and private" is a proof you can see on screen, not a claim in a
22
- README. Every serious competitor surveyed — Ollama, LM Studio, Open WebUI,
23
- Odysseus, Jan, Msty, macMLX — either has no personalization story at all
24
- (the MLX-native peers, Ollama, LM Studio) or bolts memory on as a flat
25
- prompt-stuffed fact list (Open WebUI, AnythingLLM, ChatGPT, Claude). None of
26
- them combine automatic personalization **with** weight-level adaptation
27
- (LoRA hot-swap) **with** native MLX serving speed **with** a legible,
28
- auditable tool-trust model. That combination is mlx-bun's only defensible
29
- wedge, and today the UI hides it: the backend (`src/pi-web.ts`, `src/memory/tools.ts`,
30
- `src/lora.ts`, `src/server.ts`) already implements most of it; the 2737-line
31
- `src/web/app.html` shows almost none of it.
32
-
33
- **The strategic bet (revised 2026-07-06):** best-in-class is a **superset,
34
- not a wedge**. The goal is the single best UI for macOS local AI, period —
35
- so the qualification round is conceding *nothing*: for every capability a
36
- user judges a local chat app by, we match or beat the best current
37
- implementation, or we make a written skip decision with a substrate reason.
38
- The acceptance test: nobody ever says "I miss how Ollama / LM Studio /
39
- Open WebUI / optiq had X." The axis-by-axis requirement table lives in
40
- [web-chat-beat-matrix.md](web-chat-beat-matrix.md) (12 axes, 116 capability
41
- rows, each MATCH / BEAT / SKIP-with-reason) — it is the coverage checklist
42
- every phase below is planned against. *(An earlier draft of this section
43
- framed the bet as "don't chase Open WebUI's breadth or LM Studio's polish";
44
- that conflated where the moat is with what to build, and is reversed.)*
45
-
46
- **On top of that conceded-nothing surface**, we stack the three things no
47
- competitor's architecture allows them to show: (1) memory with a vault you
48
- can open, search, and watch self-heal after a correction — today that is
49
- git-tracked articles + nightly synthesis (real, shipped); the "graduates
50
- into a trained memory LoRA" rung is roadmap, not current capability (the
51
- adapter runtime exists, the memory fine-tunes don't yet — the UI must never
52
- claim it before it ships); (2) LoRA routing as a legible per-surface
53
- decision, not a Python-API-only mechanism (optiq built the engine, never
54
- the UI; LM Studio's own bug tracker confirms adapter hot-swap is
55
- unsupported industry-wide); (3) serving speed and tool trust as ambient,
56
- always-on proof — tok/s, TTFT, which lane is active (serial / spec-decode /
57
- batched), and exactly what a tool call touched on disk — because we control
58
- the whole stack from Metal kernel to composer pixel and nobody else
59
- surveyed does. The differentiators are the reason to *switch*; they are
60
- never an excuse for a missing table stake.
61
-
62
- **The pragmatic constraint:** almost none of this requires new backend
63
- capability. Session fork, tool approval, adapter-scoped KV, the memory tool
64
- suite, structured output, the sampler extension set — all of it exists
65
- server-side today with zero or thin UI. The fastest, highest-leverage path
66
- is wiring existing capability to new UI, not a rewrite. Where a rewrite *is*
67
- warranted (the streaming-render architecture, message-tree branching,
68
- virtualized long threads), the plan says so explicitly and scopes it as its
69
- own phase.
36
+ ## 1. Thesis and principles
37
+
38
+ **Thesis.** mlx-bun's web chat should not try to be a better ChatGPT. It
39
+ is the local chat surface where the assistant visibly, provably gets
40
+ better at knowing *you* (a git-tracked memory vault you can open and
41
+ diff), where switching to a domain-specific brain is a routing decision
42
+ (LoRA adapters, stackable), and where "local, private, fast" is shown on
43
+ screen (lane badge, tok/s, TTFT, provenance chips) instead of asserted.
44
+
45
+ **Doctrine (revised 2026-07-06): superset, not wedge.** Concede nothing a
46
+ daily LM Studio / Open WebUI / optiq-Lab user would miss — every
47
+ capability in Appendix A is MATCH, BEAT, or a written SKIP with a
48
+ substrate reason — and stack the differentiators on top. The
49
+ differentiators are the reason to switch, never an excuse for a missing
50
+ table stake.
51
+
52
+ **Principles.**
53
+
54
+ 1. **Wire before you invent.** The server already had most of the
55
+ capability (sampler set, structured output, adapters, memory tools,
56
+ fork, approval gate); the highest-leverage work was wiring, and the
57
+ remaining open items are still mostly wiring.
58
+ 2. **Show, don't assert.** Privacy, speed, memory each get a UI proof.
59
+ 3. **Progressive disclosure, one flag.** Chat is the whole app for a
60
+ first-time user; Quantize / Fine-tune / Build Dataset / Status /
61
+ Curves / Routes sit behind one persisted Developer toggle.
62
+ 4. **Quiet retrieval, legible provenance.** Memory tools fire silently;
63
+ every consulted article gets a small expandable chip.
64
+ 5. **Speed is ambient.** tok/s, TTFT, context-fill, and the serving lane
65
+ live next to the composer, server-reported, never client-inferred.
66
+ 6. **Local-only is an honest simplification.** Single user, own machine:
67
+ "runs on your machine, with your files" plus a per-call approval gate,
68
+ not multi-tenant sandbox theater.
69
+ 7. **Keyboard-first, mouse-complete.** Every primary action has a key
70
+ path and a visible one.
71
+ 8. **Never hijack.** No spotlight, panel, or approval ever traps focus or
72
+ blocks the screen; the user is never in a state they have to escape.
73
+ 9. **The bar for done is visual polish** at 375 / 768 / 1024 px and
74
+ desktop, 200 % zoom, `prefers-reduced-motion` honored, no layout shift
75
+ while streaming. Visual QA at phase boundaries has the standing of
76
+ tests.
70
77
 
71
78
  ---
72
79
 
73
- ## 2. Where we are
74
-
75
- ### 2.1 What's genuinely good today
76
-
77
- Read from `src/web/app.html` and `src/pi-web.ts` directly, not assumed:
78
-
79
- - **A real, working streaming agent bridge.** `PiWebSession` in `src/pi-web.ts`
80
- is a legitimately sophisticated per-connection bridge over WebSocket:
81
- token/thinking deltas, tool-call cards with live per-call approval gating
82
- (`installApprovalGate`, `pi-web.ts:748-774`, 120s auto-deny via
83
- `APPROVAL_TIMEOUT_MS`), a follow-up/steering queue distinguishing live
84
- mid-stream steering from queued follow-ups (`queue_update` event,
85
- `pi-web.ts:390-397`), disk-persisted sessions with new/list/open/fork/delete
86
- (`SessionManager.forkFrom`, `pi-web.ts:704-714`), per-turn LoRA injection
87
- (`injectAdapter`, `pi-web.ts:412-418`), and per-turn sampling overrides
88
- (`injectSampling`) — all through a pure, unit-tested event-mapping layer
89
- (`mapEventToFrames`).
90
- - **A distinctive, cohesive visual language.** A real design-token system
91
- (`:root` vars, `app.html:17-33`), gradient hero type, hairline cards,
92
- ambient blooms — reads as considered, not a generic Bootstrap clone, across
93
- all 7 tabs consistently.
94
- - **A hand-rolled markdown renderer that is actually careful.** `mdToHtml`/`mdInline`
95
- (`app.html:1113-1238`) escape-then-restore code spans, gate link/autolink
96
- targets through `mdSafeUrl` against `javascript:` URLs, and support GFM
97
- tables/task-lists/fenced code with a working copy button — a rare area
98
- where the hand-rolled approach is already competitive with a library.
99
- - **Real session depth.** Fork/list/delete round-trip through pi's own
100
- `SessionManager` and render in the sidebar with per-row actions
101
- (`app.html:1735-1736`).
102
- - **Live perf instrumentation already computed.** Context-window meter
103
- (`app.html:1696-1703`) and post-turn tok/s + TTFT (`app.html:1667-1677`)
104
- exist — just under-designed as small dim text instead of a first-class
105
- signal.
106
-
107
- ### 2.2 The most damaging gaps
108
-
109
- Ranked by severity, from the three-part current-state audit:
110
-
111
- 1. **(Critical) No personal-memory surface exists at all.** The memory tool
112
- suite (`memory_resolve/category/read/section/links/infobox/list/status/search`
113
- plus `reference_*`, `src/memory/tools.ts:60-76`) is silently wired into
114
- the web chat's tool allowlist whenever a vault exists
115
- (`webChatToolAllowlist(memoryEnabled)`, `pi-web.ts:78-83`) and a
116
- `memoryHint` is appended to the system prompt — but nothing downstream
117
- renders memory tool calls differently from a generic tool card, there is
118
- no vault browser, no status indicator, no way to see, search, correct, or
119
- trust what the model just read about you. This is the single largest gap
120
- between the product's stated identity ("local-to-you", MEMORY.md) and
121
- what ships.
122
- 2. **(High) LoRA routing is one flat, undescribed dropdown.** `<select id="chat-adapter">`
123
- (`app.html:615-618`) lists raw ids with rank (`${a.id}${a.rank ? " · r"+a.rank : ""}`,
124
- unescaped — a live XSS-shaped bug per finding #15 in `web-ui-pass-plan.md`).
125
- No description of specialty, no three-state (available/loaded/selected)
126
- visibility even though `/v1/adapters/available` already returns
127
- `compatible: boolean`, no multi-adapter stacking exposure even though
128
- `resolveSpec` in `src/lora.ts` already supports `"a+b"` composition.
129
- 3. **(High) The tool-approval UI is fully built and architecturally dead
130
- in web chat.** The approve/deny card with diff preview exists
131
- (`app.html:1586-1610`), but the browser session's tool allowlist is
132
- hardcoded to `WELCOME_TOOLS = ["read", "web_search"]` plus memory tools —
133
- all in `READ_ONLY_TOOLS` — so `bash`/`edit`/`write` (the only
134
- `GATED_TOOLS`) never reach the browser surface at all
135
- (`buildPiAgentSurface({ webTools: true, codingTools: false })`,
136
- `pi-web.ts:543`). The approval dialog only fires in pi's separate
137
- terminal/coding-agent embed.
138
- 4. **(High) No message-level actions.** No copy on assistant text (only
139
- inside code fences), no edit-and-resend, no regenerate, no branch-from-a-
140
- specific-message (only whole-session fork from the sidebar).
141
- 5. **(High) Mobile chat loses the sidebar with no replacement.**
142
- `@media (max-width:760px){#chat-sidebar{display:none}}` (`app.html:345`)
143
- — no drawer, no hamburger. New chat and all session management become
144
- unreachable on a phone viewport.
145
- 6. **(High) No in-chat model switching.** `#nav-model` is a read-only label
146
- with no click handler at all (`app.html:527`).
147
- 7. **(Medium) Streaming re-parses the entire message on every token.**
148
- `appendDelta` (`app.html:1522-1531`) does `a.text += delta; a.textNode.innerHTML
149
- = mdToHtml(a.text)` — full markdown re-parse + innerHTML replace per
150
- token. This is O(n²) over a response and becomes the visible bottleneck
151
- on exactly the fast, spec-decode-accelerated generations the product
152
- exists to showcase.
153
- 8. **(Medium) Sampling popover exposes 3 of ~11 server-supported
154
- parameters**, and the "recommended" defaults are hardcoded to one
155
- model's shape (`SAMP_REC`, `app.html:1824-1825`, MiniCPM5-tuned),
156
- silently wrong for every other served model (`web-ui-pass-plan.md` #14).
157
- 9. **(Medium) No syntax highlighting, no LaTeX, no diagram rendering.**
158
- Code fences are plain-escaped monochrome `<pre><code>` (`mdCodeBlock`,
159
- `app.html:1139-1143`).
160
- 10. **(Medium/Low) Dark-only theming, ~11 total `aria-*` attributes across
161
- 2737 lines, no documented keyboard shortcuts beyond Enter/Shift+Enter,
162
- no chat search, generic non-personalized hero/empty-state.**
163
-
164
- Plus the full tactical list already catalogued in `docs/planning/web-ui-pass-plan.md`
165
- (queue-bar array-truthiness bug, library staleness, `[object Object]` adapter
166
- error, dead `status-page.html`, Routes-tab 404 in the compiled binary, stale
167
- vision copy, etc.) — folded into Phase 0/1 below by number, not repeated in
168
- full here.
169
-
170
- ### 2.3 Free wins — backend capability the UI hides today
171
-
172
- These require **zero new backend work**, only wiring:
173
-
174
- | Capability | Where it lives | Current UI exposure |
175
- |---|---|---|
176
- | Full sampler extension set: `min_p`, `xtc_probability`/`xtc_threshold`, repetition/presence/frequency penalties + context windows, `logit_bias` | `src/server.ts` request schema (~`:403-414`, `min_p`/XTC doc-comments; line refs drift with the active dspark branch — grep `min_p` to relocate) | 3 of ~11 fields (temp/top_p/top_k) |
177
- | `logprobs`/`top_logprobs` | `src/server.ts:425-431, 982-1044` | none |
178
- | Structured output: `response_format` json_schema/json_object, `guided_grammar`/`guided_regex`/`guided_choice` via xgrammar | `src/server.ts:458-471, 1610-1625` | none |
179
- | Memory tool suite + `memoryHint` | `src/memory/tools.ts`, `pi-web.ts:78-83` | none — fires silently |
180
- | `vaultStatus()` (article count, git state, mtimes, schedule state) | `src/memory/vault.ts:1081` | none |
181
- | `memory_links` (wikilink graph, inbound/outbound) | `src/memory/tools.ts` | none |
182
- | Adapter three-state model: available (disk) / loaded (mounted) / selected (per-request); `compatible` flag; `"a+b"` stacking | `src/lora.ts`, `GET /v1/adapters/available`, `resolveSpec` | one flat single-select dropdown |
183
- | Tool-call approval gate with diff/arg preview | `pi-web.ts:748-774`, `app.html:1586-1610` | dead code in web chat (allowlist excludes gated tools) |
184
- | Fork-from-session (`SessionManager.forkFrom`) | `pi-web.ts:704-714` | sidebar-level only, not per-message |
185
- | `Responses` API `previous_response_id` resumption, Anthropic `cache_control`/`tool_choice` forcing | `src/responses.ts`, `src/anthropic.ts` | API-only, no chat UI concept |
186
- | Prompt-cache-hit telemetry (`cache_read_input_tokens`, `cached_tokens`) | server usage payloads | only the aggregate context meter |
187
- | `GET /fit` hardware-fit prediction (decode tok/s, safe context per SKU) | `src/fit.ts` | none in chat (Status tab only, indirectly) |
188
- | HLG tone-curve sampling (`hlg` request block) | `src/server.ts:449-457, 483-504`, standalone Curve Designer page | disconnected from main chat composer |
189
-
190
- **Two near-free wins that DO need thin server additions** (verified 2026-07-06
191
- — easy to mistake for pure wiring, budget them explicitly):
192
-
193
- - **Lane badge**: the serving lanes (serial / spec-decode / batched) exist
194
- internally (`GenerationGateway` routing, `src/server.ts:1162-1168`) but
195
- **no field on any usage payload or WS frame reports which lane served a
196
- turn**. The perf strip's tok/s / TTFT / context-fill are client-computed
197
- already; the lane badge needs a small per-turn `lane` field added
198
- server-side (usage payload + the pi-web `turn_end`/usage frame) first.
199
- - **Memory panel data**: `memory_list`/`memory_search`/`memory_links` are
200
- **agent tools** (model-callable, `src/memory/tools.ts`) and `vaultStatus()`
201
- is a library function — none have HTTP routes. The vault browser (§5.5)
202
- needs thin REST wrappers (e.g. `GET /api/memory/status|list|search|article`
203
- + a git log/diff read) added in Phase 2.
80
+ ## 2. Architecture as built
81
+
82
+ ### 2.1 The bridge: `src/pi-web.ts`
83
+
84
+ One `PiWebSession` per WebSocket connection wraps one Pi `AgentSession`.
85
+ Pi events map to frames through the pure, unit-tested `mapEventToFrames`
86
+ (`tests/pi-web.test.ts`). The frame types are exported **type-only** and
87
+ imported by the frontend (`src/web/src/chat.ts`), so contract drift
88
+ between browser and server is a compile error, not a runtime bug.
89
+
90
+ Client → server (`ClientMessage`):
91
+
92
+ | Frame | Purpose |
93
+ |---|---|
94
+ | `prompt` (text, images?) | send a turn; images ride as attachments |
95
+ | `abort` | stop the turn (`session.abort()`; verified to abort in-flight tool subprocesses) |
96
+ | `approval` (callId, decision, editedArgs?, alwaysAllow?) | answer a gated tool call; edited args are substituted in place; `alwaysAllow` persists to `src/tool-approvals.ts` |
97
+ | `set_thinking` | reasoning channel on/off (Pi thinking level medium ↔ off) |
98
+ | `set_coding_tools` | opt-in "let it touch files" (default OFF); takes effect on the *next* session because Pi bakes the tool list at session creation |
99
+ | `set_adapter` (id \| null) | active LoRA adapter for subsequent turns (`"a+b"` stacks) |
100
+ | `set_sampling` (scope session \| next_turn, temperature … seed) | full sampler set; `next_turn` is a one-shot override composed over the session-level one |
101
+ | `set_system_prompt` | per-session user prompt layered over the built-in surface prompt (applied per turn via `before_agent_start`) |
102
+ | `new_session` / `list_sessions` / `open_session` / `fork_session` / `delete_session` | Pi session lifecycle |
103
+ | `regenerate` / `edit_resend` / `switch_sibling` | message actions; all are sibling branches in the same session file via `navigateTree` |
104
+ | `context` (AppUiContext) | app-aware assistant: route/view/step + `uiSnapshot`, pushed on route and wizard-step change |
105
+
106
+ Server → client (`ServerMessage`):
107
+
108
+ | Frame | Purpose |
109
+ |---|---|
110
+ | `ready` (model, vision, audio, thinking, genDefaults) | capability negotiation; drives image attach, thinking toggle, sampling defaults |
111
+ | `turn_start` / `text_delta` / `thinking_delta` / `turn_end` (lane?) | streaming; `lane` is `"serial" \| "serial+spec" \| "batched"` from `src/serve/lane-registry.ts`, keyed by the assistant message's `responseId`, absent when unknown — never guessed |
112
+ | `tool_start` / `tool_update` / `tool_end` / `tool_approval_request` | tool cards and the approval card |
113
+ | `queue_update` (steering[], followUp[]) | live steering vs queued follow-ups |
114
+ | `history` / `sessions` / `siblings` | transcript replay, sidebar, `< i/n >` toggle data |
115
+ | `context` (tokens, contextWindow, percent) | context-window meter (same name as the client frame, opposite direction) |
116
+ | `coding_tools` (active, pending) | honest current-vs-pending state of the opt-in |
117
+ | `tool_approvals` (alwaysAllow[]) | durable always-allow set, on `ready` and after change |
118
+ | `ui_navigate` / `ui_spotlight` | app-aware assistant side channel, validated server-side |
119
+ | `error` | surfaced in the thread |
120
+
121
+ Injection hooks (all Pi extension hooks, no custom provider plumbing):
122
+ `before_provider_request` injects `adapter` (`injectAdapter`) and sampling
123
+ (`injectSampling` / `composeSampling`); `before_agent_start` injects the
124
+ user system prompt and the ambient app-context line; `tool_call` /
125
+ `tool_result` carry the approval gate and loop hygiene.
126
+
127
+ ### 2.2 The frontend: `src/web/`
128
+
129
+ Vanilla TypeScript, no framework. `scripts/build-web.ts` bundles
130
+ `src/web/src/main.ts` into `src/web/app.js`, which `src/server.ts` serves
131
+ at `/assets/app.js`; `app.html` holds the markup and CSS. Runtime stays
132
+ self-contained (no CDN); the only vendored asset is highlight.js
133
+ (`src/web/vendor/`). The bundler errors if a value import ever pulls
134
+ server code into the browser bundle (type-only imports from `pi-web.ts`
135
+ are the intended seam).
136
+
137
+ | Module | Owns |
138
+ |---|---|
139
+ | `shell.ts` | DOM helpers, toast, focus trap, theme (auto/dark/light), shortcut sheet (Cmd/Ctrl+/), mobile drawer, hash router, connection pill + model identity poll, Developer toggle (`DEV_TABS`), `/dag` probe for the Routes tab |
140
+ | `chat.ts` | the WebSocket, thread render, streaming, tool cards, approval card, queue bar, Sources panel, memory-chip dispatch, app-context push |
141
+ | `composer.ts` | attachments, `#` mention picker, sampling popover (full set + Advanced + scope), system prompt + presets v1 (localStorage), adapter quick-select, perf strip (tok/s, TTFT, context, lane) |
142
+ | `sessions.ts` | sidebar list/search/fork/delete/export, full-text "in messages" search, sibling toggle, per-message actions (copy / regenerate / edit) |
143
+ | `markdown.ts` | `mdToHtml`, block-memoized `renderBlocksIncremental`, highlight pass, `[n]` citation linkify, Canvas toggle for html/svg fences |
144
+ | `memory-panel.ts` | Memory panel, provenance chips, personalized hero chips + first-run consent card |
145
+ | `adapters-panel.ts` | adapter routing table (three states, RAM cost, stacking) |
146
+ | `model-picker.ts` | `#nav-model` popover over `/library` with per-row `/fit` verdict |
147
+ | `hub.ts` | Model Hub panel (`/api/hub/*`), download progress via `/downloads` |
148
+ | `rag.ts` | client-side BM25 over attached-file text |
149
+ | `palette.ts` | command palette (Cmd/Ctrl+K), built via createElement |
150
+ | `assistant.ts` + `ui-catalog.ts` | `captureUiSnapshot`, spotlight resolution/overlay, route + spotlight catalog |
151
+ | `quantize.ts` / `finetune.ts` / `dataset.ts` / `status.ts` | the Developer tabs (job SSE via `api.ts` `jobStream`) |
152
+ | `protocol.ts` / `api.ts` | frontend-only shapes (job stream) and the JSON/SSE helpers |
153
+
154
+ `sw.js` + `manifest.webmanifest`: shell-only cache-first service worker
155
+ for installability and instant shell — explicitly not offline chat.
156
+
157
+ ### 2.3 Tool surface and trust model
158
+
159
+ `webChatToolAllowlist(memoryEnabled, codingTools)` in `pi-web.ts`:
160
+
161
+ - Always: `read`, `web_search` (`WELCOME_TOOLS`) and the three app-aware
162
+ tools (`get_current_app_context`, `navigate_app`, `spotlight_ui`).
163
+ - With a vault: the memory and reference read tools (`src/memory/tools.ts`).
164
+ - Opt-in only: `grep`, `find`, `ls`, and the gated `bash` / `edit` /
165
+ `write`.
166
+
167
+ Everything except the gated three is `READ_ONLY_TOOLS` (auto-allowed).
168
+ Gated calls open the approval card: argument preview, editable arguments,
169
+ allow / deny / always-allow, auto-deny after `APPROVAL_TIMEOUT_MS`.
170
+ Always-allow is durable and keyed by tool name (`src/tool-approvals.ts`).
171
+ Loop hygiene (`LOOP_HYGIENE`): consecutive-identical-call blocking (with
172
+ the prior result inlined, since Pi's `tool_call` hook can only block with
173
+ an error result), a failed-retry budget nudge, and a turn cap with
174
+ force-finish. Format-aware tool-call repair lives in `src/tool-call.ts`
175
+ (`RepairKind`), tagged on the parsed call.
176
+
177
+ ### 2.4 REST the UI depends on
178
+
179
+ `GET /v1/adapters` (mounted; `ram_bytes`), `GET /v1/adapters/available`
180
+ (on disk; `compatible`, `mounted`), `POST` / `DELETE /v1/adapters`;
181
+ `GET /fit`, `/library`, `/downloads`, `/stats`, `/v1/models`;
182
+ `/api/memory/{status,list,search,article,links,history,diff}` +
183
+ `POST /api/memory/init` (`src/memory/rest.ts`); `/api/hub/{local,search}`,
184
+ `POST /api/hub/download`, `POST /api/hub/serve` (`src/hub-rest.ts`);
185
+ `/api/sessions/{search,export}`; `/api/gc/{plan,execute}`;
186
+ `/api/jobs/:id/stream`. User-facing contract: `docs/reference/server-api.md`.
187
+
188
+ ### 2.5 Regression harness
189
+
190
+ `tests/web-app.test.ts` (happy-dom) covers the pure client functions:
191
+ streamed-then-finalized output must equal one-shot `mdToHtml` (the
192
+ renderer's own parity oracle), Canvas fence detection and view-state
193
+ survival, `renderQueue` length semantics, `api()` envelope unwrapping,
194
+ `esc()` discipline on every interpolation site, adapter/model/hub row
195
+ renderers, mention picker, BM25, Sources panel, snapshot capture and
196
+ spotlight resolution, catalog validation. `tests/pi-web.test.ts` covers
197
+ the frame mapping, sampling composition, loop hygiene, and app-aware
198
+ tools. Frontend modules type-check in the same `tsc` pass as the server.
204
199
 
205
200
  ---
206
201
 
207
- ## 3. Competitive landscape
208
-
209
- | Product | Best-in-class at | Structural ceiling vs. mlx-bun |
210
- |---|---|---|
211
- | **mlx-lm** (the oracle) | Correctness/API reference; continuous batching; byte-capped LRU prompt cache; tokenizer-declared capability flags | Zero UI of any kind — its own docs point users elsewhere. No memory, no adapter routing, no structured output, no vision at the server layer. We already win by existing. |
212
- | **mlx-optiq** | Per-layer sensitivity-guided mixed-precision (weights + KV); Model Arena (blind A/B); Hub (browse+load+chat in one screen); multi-adapter hot-swap via `AdapterActivation` (Python API) | Proved the adapter hot-swap *mechanism* (~50MB/adapter, 10 resident ≈ 6.1GB) but never shipped it above the Python API — no UI, no CLI flag. No memory/personalization concept at all. |
213
- | **Ollama** | Zero-friction install/run fusion; `ollama launch <tool>` (redirect Claude Code/Codex at itself); `:cloud` suffix hybrid; visible `ollama ps` resource table | No persistent memory, no LoRA routing, thin first-party GUI (borrows its "best chat UI" reputation from third-party clients). |
214
- | **Open WebUI** | Broadest feature surface (RAG, Channels, RBAC, Functions/Pipelines, MCP, Arena+Elo); native tool calling preserves KV cache | Backend-agnostic by design → zero access to token-level internals (no spec-decode control, no adapter routing, no KV quant visibility). Memory is prompt-injection bounded by context window, not weight-level. Documented long-thread freeze bug (#23067). Enterprise-flavored (RBAC/Channels) — wrong shape for a single-user product. |
215
- | **LM Studio** | Hardware-fit traffic-light on downloads; User/Power/Developer progressive disclosure; Presets (shareable, model-bound config); transparent inline-vs-RAG; Split View | Closed source. Zero memory/personalization layer — presets are static and manual. Zero LoRA hot-swap UI concept. Electron-heavier than a native shell. |
216
- | **Odysseus** | Cookbook hardware-aware model recommender; self-evolving Skills (teacher-escalation authoring); persistent Memory (Observer/Reflector, dedup, nightly audit) | Not MLX-native (targets vLLM/llama.cpp, MLX via Ollama's separate backend). Publicly bad security posture (SSRF/auth-bypass CVEs, unsandboxed shell). No adapter/LoRA concept — customization is prompt/Skills-markdown only, not weight-level. "Vibe-coded" reputation. |
217
- | **AnythingLLM** | Automatic memory extraction (Observer/Reflector, live "3/20" counters, workspace-vs-global scope) — the single feature reportedly driving installs | Memory is workspace-siloed prompt-stuffing, explicitly sent verbatim to whatever provider is configured (no on-device-only guarantee). No LoRA/adapter personalization at all. |
218
- | **Jan** | Inline non-blocking MCP approval cards + global "Allow All"; router mode (single process, on-demand load/unload); MTP/spec-decode toggle in consumer settings (validates our own dspark serial-routing) | Generic single-thread chat core once you strip the MCP/assistant chrome; no memory, no adapter routing. |
219
- | **Msty** | Split Chat (parallel synced multi-model panes); Persona Studio + Crew Conversations + Shadow Persona | Closed-source-leaning; feature-sprawl (Turnstiles/Forge Mode naming). No real adapter-weight mechanism — personas are prompt bundles. |
220
- | **macMLX / mlx-serve** | Genuinely MLX-native, Python-free, same technical identity claim as us (OpenAI+Anthropic API, tiered SSD KV cache, Benchmark tab, quick-launcher hotkey) | Shallow personalization: no cross-session memory, no LoRA routing, basic-to-absent persona support. Proves "MLX-native + fast API" alone is **not** a moat — we already have peers there. |
221
- | **ChatGPT / Claude.ai** | Artifacts/Canvas; highlight-to-scope-edit; edit-message=branch w/ linear toggle; two-layer transparent memory (explicit facts + background synthesis); Projects (scoped knowledge+memory) | Memory mechanism is a black box even though output is legible — synthesis runs on someone else's infrastructure, unauditable. No tree view (only linear branch toggle) — third-party extensions exist to fill this gap. No LoRA-equivalent — Projects route RAG context, not model weights. Voice mode is a hard, cloud-scale-economics problem. |
222
-
223
- **What we'd lose vs. mlx-lm:** nothing — mlx-lm has no UI, no session
224
- persistence, no adapter routing, no structured output at the server layer,
225
- no vision at the server layer. Every "loss" is actually a gap we already
226
- close by existing. The one thing to protect explicitly is **bit-exact
227
- parity as a mode**: `--batch N` / flag-free paths must keep matching
228
- mlx-lm's outputs exactly, because "drop-in replacement, losing nothing" is
229
- a standing product promise (see MEMORY.md), not just a chat-UI concern.
230
-
231
- **How we beat Open WebUI / LM Studio (revised):** by conceding nothing and
232
- then stacking what they can't copy. Concretely, per the
233
- [beat matrix](web-chat-beat-matrix.md): we MATCH every table stake a user
234
- would miss — system prompts/presets, document chat with citations, in-app
235
- model browse/download (optiq's Hub flow, plus a real `/fit` verdict per row
236
- that optiq lacks), sandboxed HTML canvas, per-message sampling, session
237
- auto-titling, export, shortcuts, themes, mobile — and we skip only where the
238
- engine lacks the substrate (image generation, voice), in writing. The two
239
- exceptions we deliberately don't chase: Open WebUI's multi-tenant enterprise
240
- plumbing (Channels, RBAC) — wrong shape for a single-user local product, a
241
- scope decision rather than a gap — and cloud-scale voice. THEN the wedge:
242
- we're the only surface whose memory is **weight-level-capable** (git-tracked
243
- vault + nightly synthesis today, memory-LoRA rung on the roadmap — not a
244
- context-window fact list with a hard cap) and whose adapter routing is
245
- **real** (LM Studio has zero adapter concept; Open WebUI is
246
- backend-agnostic and structurally cannot see adapters at all). Both are
247
- already true at the engine level — the redesign's job is to make them
248
- visible, legible, and trustworthy, which neither competitor's architecture
249
- permits even if they wanted to.
250
-
251
- *(Caveat: the Odysseus row above rests on the research agent's public-web
252
- identification — Cookbook recommender / Skills / Observer-Reflector memory /
253
- SSRF CVEs. If that isn't the Odysseus meant, that row needs a re-run;
254
- nothing else in this doc depends on it.)*
202
+ ## 3. Surfaces
203
+
204
+ ### 3.1 Onboarding / hero
205
+
206
+ Server-driven chips instead of static marketing copy: one drawn from real
207
+ vault content when `/api/memory/status` reports articles, one offering an
208
+ available adapter, and a first-run, skippable memory-consent card that
209
+ creates the vault via `POST /api/memory/init` — consent happens in the
210
+ same first minute as the first chat, not as a CLI step discovered later.
211
+ The system prompt (`buildWebChatSystemPrompt`) is deliberately short: a
212
+ long welcome prompt made the ~1B starter model greet instead of answer.
213
+
214
+ ### 3.2 Composer and message actions
215
+
216
+ - Sampling popover: temperature / top_p / top_k plus an Advanced
217
+ disclosure with `min_p`, XTC, repetition / presence / frequency
218
+ penalties with context size, and seed; defaults from `ready.genDefaults`
219
+ (per-model, not a hardcoded shape); scope control session vs next turn.
220
+ - System prompt field + presets v1 (name + prompt + sampling; localStorage
221
+ only, no wire shape).
222
+ - Adapter quick-select (`#chat-adapter`) with a gear to the routing table
223
+ (§3.6). A swap note says so: a new adapter starts a fresh KV segment.
224
+ - `#` mention picker: attached files and vault articles as one retrieval
225
+ gesture.
226
+ - Perf strip: tok/s, TTFT, context-fill, lane badge (from `turn_end.lane`).
227
+ - Per assistant turn: copy, regenerate; per user turn: edit-and-resend
228
+ with a `< i/n >` sibling toggle. All three are branches of the same Pi
229
+ session (no new file), so history stays in one place. A full tree
230
+ *view* is deliberately not built (open, §5).
231
+
232
+ ### 3.3 Streaming render
233
+
234
+ `renderBlocksIncremental` splits the message at block boundaries, leaves
235
+ settled blocks alone, and re-parses only the live tail, throttled to
236
+ animation frames; `finishStreaming` runs one full `mdToHtml` pass so the
237
+ final DOM is byte-identical to a one-shot render (tested). Syntax
238
+ highlighting (vendored highlight.js) applies to completed blocks only.
239
+ Steering and follow-up queues render from `queue_update` with length
240
+ checks (the array-truthiness bug is fixed and regression-tested).
241
+
242
+ ### 3.4 Tool cards and approval
243
+
244
+ Generic tool calls keep the running / ok / fail card. Memory and reference
245
+ tool calls render as a **provenance chip** (citation style, expandable,
246
+ opens the article in the Memory panel) — the "assistant looked something
247
+ up about you" moment reads differently from "ran a shell command". The
248
+ approval card (§2.3) is inline and non-blocking. The settings copy says
249
+ that toggling coding tools applies to the next new chat (`coding_tools`
250
+ frame), never implying immediacy Pi can't deliver.
251
+
252
+ ### 3.5 Memory panel
253
+
254
+ Reachable from the chat sidebar, not a nav tab (memory is chat-adjacent,
255
+ it is the product, not a developer tool). Two screens: list (status strip
256
+ from `vaultStatus()`, search, personal articles and a `Reference/`
257
+ sub-list) and article (rendered content, inbound/outbound wikilinks line,
258
+ a History toggle backed by `/api/memory/history` and `/api/memory/diff` —
259
+ the "watch it self-heal after a correction" moment made literal, which
260
+ cloud memory UX structurally cannot offer). The agent-tool surface stays
261
+ read-only; only nightly synthesis writes (`docs/design/dreaming-nightly-pipeline.md`,
262
+ `docs/design/dreaming-nightly-pipeline.md`). The plan's wikilink *graph* view is not built —
263
+ the panel shows a links line (open, §5).
264
+
265
+ ### 3.6 Model and adapter switching
266
+
267
+ **Model picker.** `#nav-model` opens a popover over `/library`, each row
268
+ with the `/fit` verdict computed for this Mac (predicted decode tok/s,
269
+ safe context). There is **no in-process model swap**: the runtime is one
270
+ model per process (like mlx-lm), `POST /api/hub/serve` answers
271
+ `restart_required` with the exact `mlx-bun serve` command, and the picker
272
+ offers that command to copy. The live-swap mechanism that exists
273
+ (`ModelPool` spawn-overlap) lives only on the opt-in `--isolate` proxy,
274
+ which 501s `/ws/chat` — see `docs/reference/server-config.md`. Model Hub (`hub.ts`)
275
+ adds browse / HF search / download with progress; it never auto-downloads
276
+ or auto-serves.
277
+
278
+ **Adapters — the three-state model** (folded from
279
+ `docs/design/web-chat-redesign.md`; the design keeps the states separate and lets
280
+ the user move an adapter between them explicitly):
281
+
282
+ 1. **Available** — on disk, compatible or not, costs nothing.
283
+ `GET /v1/adapters/available` scans `~/.cache/mlx-bun-finetunes/` and
284
+ `~/.cache/mlx-bun/adapters/` (`listAvailableAdapters`, `src/lora.ts`);
285
+ `id` = directory basename; `rank` / `scale` / `base_model` read from
286
+ `optiq_lora_config.json` or `adapter_config.json`, with the base repo id
287
+ recovered from the stored snapshot path.
288
+ 2. **Loaded / mounted** — `POST /v1/adapters {id, path}` into the running
289
+ server; costs memory (`ram_bytes` on `GET /v1/adapters`);
290
+ `DELETE /v1/adapters/:id` unmounts.
291
+ 3. **Selected for this request** — the `adapter` field: `"none"` (default)
292
+ | `"<id>"` | `"a+b"` (stacked, `resolveSpec`). Unknown id throws —
293
+ loud, never a silent base fallback.
294
+
295
+ The routing table (`adapters-panel.ts`, behind the composer gear) shows
296
+ every on-disk adapter with badges for all three states, base-model / rank
297
+ / size / RAM meta, mount / unmount / select / clear actions, and "stack"
298
+ tick-boxes that compose two mounted adapters as `"a+b"`. Selecting an
299
+ unmounted adapter mounts it first, then sends `set_adapter`; the
300
+ `before_provider_request` hook injects the id. Invariants: default none;
301
+ an adapter must be mounted before it can be selected; the quick-select and
302
+ the table are two views of one selection.
303
+
304
+ Compatibility as built: `compatible = base_model == null ||
305
+ bareName(base_model) === bareName(servedModelId)` — a bare-name match on
306
+ the base model, not the repo-id match the original note described.
307
+
308
+ The **CLI twin** is the Pi extension
309
+ `scripts/packaging/pi-extensions/mlx-bun-adapter.ts` (`/adapter`,
310
+ `/adapter <id>`, `/adapter off`; server via `MLX_BUN_URL`; same
311
+ `before_provider_request` mechanism, in-memory selection). It belongs to
312
+ the CLI reference (`docs/reference/cli.md`, alongside `serve --adapter`),
313
+ not to this doc.
314
+
315
+ ### 3.7 Sessions
316
+
317
+ Sidebar with title filter, a full-text "in messages" fallback
318
+ (`/api/sessions/search`, confined to the session dir), fork / delete /
319
+ export (Markdown / JSON via `/api/sessions/export`). Titles derive from
320
+ the session name or first user message (`toSessionListItems`). On narrow
321
+ viewports the sidebar becomes a focus-trapped slide-over drawer.
322
+
323
+ ### 3.8 Information architecture and settings
324
+
325
+ Chat is the product; everything else is a developer tool. One persisted
326
+ Developer toggle (`shell.ts`) reveals Quantize / Fine-tune / Build Dataset
327
+ / Status / Curves / Routes. Curves is a separate full page (`/curves`,
328
+ labelled as such); Routes is a hash route whose tab visibility is gated by
329
+ a `/dag` probe so the compiled binary never shows a broken iframe. Deep
330
+ links keep working regardless of the toggle. Status's RAM/KV "Memory"
331
+ card keeps meaning system memory; the vault is always "Memory panel".
332
+ Settings group as Chat (sampling, thinking, coding-tools opt-in,
333
+ always-allow list), Memory (status, consent), Developer (the toggle).
334
+
335
+ ### 3.9 Chat-with-files (RAG v1)
336
+
337
+ Attached text files below an inline threshold go into the prompt verbatim;
338
+ above it, a pure client-side BM25 retriever (`rag.ts`, no embeddings, no
339
+ network) selects top chunks, injects them as a numbered context block, and
340
+ the reply's `[n]` markers linkify into a collapsed Sources panel.
341
+ Citations survive regenerate / edit; they are not persisted into history
342
+ replay (needs a `HistoryItem` field — open). A vector upgrade over
343
+ `src/embed.ts` is explicitly later.
344
+
345
+ ### 3.10 Code and Canvas
346
+
347
+ Fenced code gets copy + highlighting. `html` / `svg` fences get a
348
+ Preview | Source toggle: a lazily created `sandbox="allow-scripts"`
349
+ `srcdoc` iframe, never `allow-same-origin`, height-clamped, view state
350
+ preserved across re-render and streaming finish. The persistent Artifacts
351
+ side panel is not built (open, §5).
352
+
353
+ ### 3.11 The app-aware assistant
354
+
355
+ The agent can see and act on the app it lives in — no screenshots, no
356
+ vision model in the loop. Product theory: guided tours fail on timing
357
+ (help before intent), chatbot help fails on grounding (intent without
358
+ state); this is the only shape with both, and the acceptance test is a
359
+ feeling — someone is there with you.
360
+
361
+ - **See:** on route / wizard-step change the browser pushes a `context`
362
+ frame (route, view, step, `uiSnapshot` of visible interactive elements,
363
+ capped ~120, agent chrome excluded). `get_current_app_context` returns
364
+ it; a compact ambient line (route / view / step) is auto-prepended to
365
+ every turn so the model never answers blind.
366
+ - **Navigate:** `navigate_app` validated server-side against the route
367
+ catalog → `ui_navigate` → the hash router. Reversible, no approval.
368
+ - **Point:** `spotlight_ui` resolves ref → selector → label → catalog
369
+ target against the last snapshot and shows a hand-rolled overlay:
370
+ `pointer-events: none` (the control stays clickable through it), no
371
+ focus trap, auto-dismiss in seconds, any input dismisses, reduced-motion
372
+ honored. Model-invented selectors are a miss, never a throw.
373
+
374
+ Scope guard: these tools exist only on the pi-web bridge (never the
375
+ OpenAI / Anthropic API surface); the snapshot never includes cross-origin
376
+ content; approval-gated `ui_act` (fill this form, start this job) is open
377
+ work and must render through the §3.4 approval card.
378
+
379
+ ### 3.12 Speed and trust as ambient proof
380
+
381
+ The perf strip's lane badge is server-reported per turn (risk: a stale or
382
+ wrong badge is a P0, because the differentiator rests on it being
383
+ trustworthy). Context-fill comes from Pi's `getContextUsage`. What is still
384
+ missing from the trust surface is listed in §5.
255
385
 
256
386
  ---
257
387
 
258
- ## 4. Design principles
259
-
260
- 1. **Concede nothing; differentiate on top.** The beat matrix
261
- ([web-chat-beat-matrix.md](web-chat-beat-matrix.md)) is the coverage
262
- contract: every capability row is MATCHed, BEATen, or SKIPped in writing
263
- with a substrate reason — never silently absent. Within that, the
264
- memory/LoRA flywheel gets the design attention (it's the reason to
265
- switch), but "it's just table stakes" is never a reason a table stake
266
- ships worse than the best competitor's version. The test for every
267
- release: would a daily LM Studio / Open WebUI / optiq-Lab user miss
268
- anything? If yes, that gap has a name and a written decision.
269
- 2. **Show, don't assert.** Privacy, speed, and memory are all *provable*
270
- locally (no network egress, real tok/s, an inspectable git-tracked vault).
271
- Every one of these claims gets a UI element that proves it, not prose
272
- that states it. This mirrors the CLAUDE.md ground rule "every perf claim
273
- gets a number" — extend it to every trust claim getting a UI proof.
274
- 3. **Progressive disclosure, one flag, not two apps.** LM Studio's
275
- User/Power-User/Developer toggle is the reference: one codebase, one
276
- settings switch, not a fork. Quantize/Fine-tune/Build-Dataset/Curves/Routes
277
- collapse behind a single "Developer" toggle so Chat is the whole app for
278
- a first-time user.
279
- 4. **Quiet retrieval, legible provenance.** Memory tools fire silently by
280
- design (never a context dump) but every consulted source gets a small,
281
- dismissable, expandable citation — the AnythingLLM/ChatGPT memory-panel
282
- legibility bar, applied to a strictly deeper (weight-level) mechanism.
283
- 5. **Speed is ambient, not a stat you have to look up.** tok/s, TTFT,
284
- context-fill, and which serving lane is active (serial / spec-decode /
285
- batched ×N) are always visible near the composer, the way LM Studio 0.4
286
- made context-fullness and processing state ambient chrome.
287
- 6. **Wire before you invent.** Prefer surfacing an existing backend
288
- capability over designing a new one. Section 2.3's free-wins table is the
289
- backlog triage order, not a suggestion.
290
- 7. **Local-only is an honest simplification, not a hedge.** Single-user,
291
- local-machine execution means tool access, file writes, and shell
292
- commands can be presented plainly ("runs on your machine, with your
293
- files") instead of building sandbox theater Open WebUI/Odysseus need for
294
- multi-tenant deployments we don't have.
295
- 8. **Keyboard-first for the power user, mouse-complete for everyone else.**
296
- Every primary action (new chat, send, stop, regenerate, switch
297
- model/adapter, command palette) has a keyboard path, but nothing is
298
- keyboard-only-discoverable — mirror ChatGPT's documented shortcut sheet
299
- (`Cmd/Ctrl+/`) as the baseline, not a stretch goal.
300
- 9. **The bar for done is perfect visual polish.** No overlapping text at
301
- any viewport width, no animation jitter, no layout shift on stream, no
302
- flash of unstyled/mispositioned anything. A feature that works but
303
- looks unfinished is not done — it ships when it feels like a polished,
304
- professional, excellent experience. Concretely: every animation is
305
- compositor-friendly (transform/opacity only), interruptible, and
306
- respects `prefers-reduced-motion`; every overlay/popover is tested at
307
- 375px, 768px, 1024px (iPad — where the 2026-07-06 nav-clip /
308
- placeholder-wrap / pill-overflow jank actually surfaced), and desktop
309
- widths and at 200% zoom; streaming text never
310
- causes scroll jank or reflow of completed content (§5.3's memoization
311
- is a polish requirement, not just a perf fix). Visual QA at phase
312
- boundaries is part of the definition of done, same standing as tests.
388
+ ## 4. Tech-stack decisions
389
+
390
+ - **Vanilla, no SPA framework.** The framework-native ideas (block
391
+ memoization, windowing, focus traps) port as functions; a migration is a
392
+ multi-month distraction at this file's complexity. Recorded so a future
393
+ session doesn't relitigate it without new evidence.
394
+ - **Typed module split over a single inline script.** The split's payoff
395
+ is the typed WS/API contract, not navigability: the Phase 0 bug class
396
+ (field-name drift, array truthiness, response-shape mismatch) became
397
+ build errors.
398
+ - **Vendored, no-CDN assets only.** highlight.js is vendored; KaTeX was
399
+ planned and is not vendored (open).
400
+ - **PWA = installability + instant shell**, never offline chat.
401
+ - **Theming:** auto / dark / light via `data-theme`, follows
402
+ `prefers-color-scheme`; animations transform/opacity only and gated by
403
+ `prefers-reduced-motion`.
404
+ - **A11y:** `role="log" aria-live="polite"` on the thread, focus traps with
405
+ Escape-returns-focus on every overlay, a shortcut sheet with ChatGPT's
406
+ vocabulary where sensible.
407
+ - **Hard rule for every overlay:** the spotlight is not a modal and must
408
+ never be wired into the modal machinery (`closeTopOverlay` / `trapFocus`).
313
409
 
314
410
  ---
315
411
 
316
- ## 5. The redesigned chat experience
317
-
318
- ### 5.1 Onboarding / empty state
319
-
320
- **New vs. today:** the hero ("Local. Private. Yours.", `app.html:564-570`)
321
- currently shows 4 static marketing chips regardless of state. Replace with
322
- server-driven chips: if a vault exists, one chip is drawn from real content
323
- ("Ask about {last-synthesized-article}"); if adapters are available, one
324
- offers to try one; if this is a first run, a one-line skippable memory
325
- consent card appears inline ("Let mlx-bun learn from this conversation? Two
326
- clicks, reversible.") — because `mlx-bun memory init` today is CLI-only and
327
- invisible to anyone who never opens a terminal. This directly serves the
328
- "install to chat under a minute" north star: personalization consent should
329
- happen in the same 60 seconds as first chat, not as a separate discovered
330
- CLI step.
331
-
332
- **Backend used:** `vaultStatus()` (`src/memory/vault.ts:1081`), `GET
333
- /v1/adapters/available`, existing hero markup as scaffold.
334
-
335
- ### 5.2 The composer and message actions
336
-
337
- **Composer additions (extend, don't rebuild):**
338
- - **LoRA/persona chip** replacing the bare `<select id="chat-adapter">`:
339
- active adapter's name + one-line specialty description (falls back to
340
- "general-purpose" until metadata exists), available/incompatible adapters
341
- grayed by the `compatible` flag `/v1/adapters/available` already returns.
342
- A note appears on swap ("starts a fresh KV segment") since that's true.
343
- - **Sampling popover** grows from 3 sliders to the full server-supported set
344
- behind an "Advanced" disclosure: `min_p`, XTC (`xtc_probability`/`xtc_threshold`),
345
- the three penalty families with context windows, `logit_bias`, seed.
346
- Defaults resolve per-model from `generation_config.json` (fixing the
347
- hardcoded MiniCPM5-shaped `SAMP_REC`, `web-ui-pass-plan.md` #14) instead of
348
- one hardcoded shape.
349
- - **Perf strip** under the composer, always visible: tok/s, TTFT,
350
- context-fill %, and a **lane badge** (`serial · spec-decode active` /
351
- `batched ×N` / `parity mode`) — replacing the small dim tok/s text. This is
352
- the one thing no competitor's UI has, because none of them control the
353
- serving lane the way we do.
354
- - **Unified `#`/`@` retrieval mention**: typing `#` opens a picker across
355
- *both* attached files and vault articles as one retrieval gesture (Open
356
- WebUI's `#`-mention pattern, extended so "attach a file" and "search my
357
- memory" are the same interaction, not two parallel systems).
358
-
359
- **Message actions (new — currently only code-block copy exists):**
360
- - Per assistant turn: copy full message, regenerate, "branch from here"
361
- (reuses `SessionManager.forkFrom`, currently only exposed at the
362
- whole-session level).
363
- - Editing a user message creates a lightweight sibling with a `< 1/2 >`
364
- toggle above it — the ChatGPT/Claude convention users already have muscle
365
- memory for. **Do not** build a full tree-view graph in this phase (real
366
- effort neither incumbent bothered to ship); a tree view is a Phase 5
367
- stretch differentiator (see §6, and note both cloud incumbents ship only
368
- the linear toggle — a from-scratch tree view is a genuine, buildable gap
369
- we can claim later, not a launch requirement).
370
-
371
- ### 5.3 Streaming + steering
372
-
373
- Fix the O(n²) re-render (`appendDelta`, `app.html:1522-1531`) by splitting
374
- into completed-block + live-tail-block: parse paragraph/fence boundaries,
375
- skip re-parsing completed blocks, only reprocess the tail (Streamdown/t3.chat's
376
- block-memoization pattern, ported as an idea into the existing vanilla
377
- renderer — no framework needed). Ship self-hosted syntax highlighting (Shiki
378
- or a lighter alternative, applied only to completed blocks) and KaTeX for
379
- math in the same pass, since code fences currently render as monochrome
380
- plain text. This is not optional polish: on a fast local model (the whole
381
- product pitch), the current renderer makes the *frontend* the visible
382
- bottleneck.
383
-
384
- Steering and the follow-up queue already work server-side
385
- (`queue_update` carries distinct `steering`/`followUp` arrays,
386
- `pi-web.ts:390-397`) but render broken today (truthy-tested instead of
387
- length-tested arrays, `web-ui-pass-plan.md` #2) — fix as part of this pass
388
- since it's touching the same render path.
389
-
390
- ### 5.4 Tool-call & approval display
391
-
392
- Keep the existing running/ok/fail visual language
393
- (`app.html:1545-1584`). Two concrete changes:
394
- 1. **Wire the web chat's tool allowlist to include gated tools** (`bash`/`edit`/`write`)
395
- behind an explicit, off-by-default "let it touch files" toggle in
396
- settings (`buildPiAgentSurface({ codingTools: true })` on that path) —
397
- so the already-built approval-card UI (`app.html:1586-1610`) stops being
398
- dead code. Extend the card to show editable arguments before
399
- approve/deny (LM Studio's pattern, beyond mlx-bun's current binary
400
- allow/deny) and persist "always allow this tool" to a real config file
401
- keyed by tool identity (fixing the fragile-persistence failure mode
402
- documented against Claude Desktop/Cowork in the research).
403
- 2. **Give memory tool calls a distinct card style** — not the wrench icon,
404
- something closer to a citation chip — so "the assistant looked something
405
- up about you" reads differently from "the assistant ran a shell
406
- command." This is the single highest-leverage rendering change in the
407
- whole plan given §2.2 finding #1.
408
-
409
- ### 5.5 Memory surface (view / edit / forget, LoRA indicator)
410
-
411
- A real **Memory panel**, reachable from the chat sidebar (not a 7th peer tab
412
- to Quantize/Fine-tune — memory is chat-adjacent, one click away, because
413
- it's the product, not a dev tool). Contents:
414
- - **Vault browser + search**, backed by `memory_list`/`memory_search`,
415
- with a `Reference/` sub-view for mlx-bun's own docs (clearly separated
416
- from personal articles) via `reference_list`/`reference_search`.
417
- - **Per-article view with a git-log/diff toggle** — the "watch it self-heal"
418
- moment made literal: after a correction, a user can see the exact article
419
- diff from the next nightly synthesis. This is the sharpest wedge against
420
- ChatGPT/Claude's memory UX, which shows *what* is remembered but never
421
- *how* — because their synthesis runs on someone else's infrastructure.
422
- Ours runs locally and the mechanism itself is inspectable.
423
- - **Status strip**: `vaultStatus()` already returns article count, git
424
- state, last-synthesis time — render it plainly.
425
- - **Wikilink graph view** using `memory_links` (inbound/outbound per
426
- article) — cheap given the tool already resolves this data; a genuine
427
- "knowledge graph of what the model knows about you" view nobody else in
428
- the survey has because nobody else has a git-tracked personal wiki.
429
- - **Per-turn provenance chip** on any assistant message that consulted
430
- memory tools (from §5.4) — links into this same panel.
431
-
432
- ### 5.6 Model & adapter switching
433
-
434
- Make the nav model label (`#nav-model`, currently a dead `<span>` with no
435
- click handler) clickable to a lightweight picker: what's loaded, hardware-fit
436
- signal (green/yellow/red, LM Studio's pattern) computed against this
437
- specific Mac's measured bandwidth via `GET /fit` — not a generic heuristic,
438
- since we already have per-SKU fit prediction in `src/fit.ts`. Model swap
439
- should preserve session history and re-prime cleanly (every competitor
440
- surveyed is documented as bad at this — LM Studio has an open bug where
441
- switching models clears the selector and forces manual reselection; a
442
- genuinely smooth swap is a believable, demonstrable win).
443
-
444
- Adapter selection: promote from flat dropdown (§5.2) to a routing table in
445
- settings showing all three states (available on disk / loaded+mounted /
446
- selected for this request) with RAM cost visible per loaded adapter (optiq's
447
- own numbers: ~50MB/adapter, 10 resident ≈ 6.1GB — a concrete budget to design
448
- the UI affordance against), and expose `"a+b"` stacking since `resolveSpec`
449
- already supports it server-side.
450
-
451
- ### 5.7 Sessions / projects
452
-
453
- Add a search box above the session list (trivial, local-only, full-text —
454
- no reason to skip it given Claude's own title-only search is a widely-cited
455
- annoyance and we have no server-cost constraint). On mobile, replace the
456
- currently-vanishing sidebar (`app.html:345`) with a slide-over drawer
457
- triggered by a hamburger — nothing fancier needed for v1.
458
-
459
- A lightweight **persona/bundle** concept (system prompt + adapter selection
460
- + tool allowlist + memory scope, saved and nameable) is a Phase 5 item (see
461
- §6) — not required for the core redesign, but the composer/settings data
462
- model should be shaped so a "save this as a named bundle" action is a
463
- thin layer on top, not a re-architecture.
464
-
465
- ### 5.8 Settings
466
-
467
- Collapse into three groups: **Chat** (sampling, thinking toggle, tool
468
- allowlist/gating), **Memory** (vault path, consent, synthesis schedule
469
- status — read from `memory_status`), **Developer** (the progressive-disclosure
470
- gate for Quantize/Fine-tune/Build-Dataset/Curves/Routes — see §8).
471
-
472
- ### 5.9 Structured output
473
-
474
- Server already supports `response_format` json_schema/json_object and
475
- `guided_grammar`/`guided_regex`/`guided_choice` via xgrammar
476
- (`src/server.ts:458-471`) with zero UI anywhere. Ship a lightweight
477
- schema/grammar picker in the Advanced sampling disclosure (§5.2) — even a
478
- raw JSON-schema textarea with live validation is pure whitespace relative to
479
- every competitor (mlx-lm has no `response_format` at all; most local UIs
480
- don't expose it even when their server supports it).
481
-
482
- ### 5.10 Vision / multimodal
483
-
484
- Vision is already wired end-to-end correctly (capability negotiation via
485
- the `ready` frame's `vision` flag, `pi-provider.ts:93-100`). The only fix
486
- needed here is the stale copy bug (`web-ui-pass-plan.md` #6: "switch to the
487
- 12B for vision" is wrong now that e2b/e4b/26B/31B are all vision-capable via
488
- `visionCapable(m)`). No new UI work required for v1 beyond that correction
489
- and folding image/doc attach into the unified `#` retrieval gesture (§5.2).
490
-
491
- ### 5.11 Artifacts / code
492
-
493
- Fenced-code rendering gets a copy button (already exists) plus syntax
494
- highlighting (§5.3). **Canvas v1 — fenced HTML rendered in a sandboxed
495
- iframe with a view-source toggle — is a Phase 3 item** (optiq Lab ships
496
- exactly this today; "I miss optiq's Canvas" is a disallowed outcome). The
497
- full Artifacts/Canvas *side-panel* (persistent, versioned,
498
- highlight-to-scope-edit) stays a Phase 5 stretch — both cloud incumbents'
499
- own experience (OpenAI partially walked Canvas back in May 2026 for the
500
- common case) argues for opt-in-by-heuristic (auto-open above ~10 lines of
501
- code / high fence density), not a mandatory pane.
412
+ ## 5. Open work
413
+
414
+ Nothing below has a PLAN.md phase; it is the backlog this doc owns.
415
+ Ordered by leverage.
416
+
417
+ **Residuals from the shipped phases (verified against src 2026-08-23):**
418
+
419
+ - KaTeX / math rendering — planned in the render pass, not vendored.
420
+ - Structured-output picker (JSON schema / grammar textarea with live
421
+ validation) — server supports `response_format` and
422
+ `guided_grammar|regex|choice`; no composer UI.
423
+ - Wikilink graph view — `/api/memory/links` exists; the panel renders a
424
+ links line only.
425
+ - Session-preserving live model swap — blocked on a reload seam at the
426
+ engine boundary (`docs/reference/server-config.md`); until then the Hub / picker
427
+ hand back the restart command.
428
+ - RAG citations persisted into history replay (`HistoryItem` field).
429
+ - Long-thread windowing (anchored-to-bottom DOM recycling) — not built;
430
+ months-long sessions are more likely here than in a stateless chatbot.
431
+ - Adapter metadata: the discovery rows are indistinguishable when several
432
+ adapters share base / rank / scale; the trainer should stamp task / data
433
+ / date (and a stable repo id) into the adapter config at save time.
434
+ - Overnight training from the web UI ("start before bed, wake up to an
435
+ adapter"): the fine-tune wizard streams a job (`/api/jobs/:id/stream`),
436
+ but a reconnectable monitor with completion → "available" hand-off is
437
+ not designed. `--force-adapter` (serve pinned to one adapter for an
438
+ unattended pipeline) is a CLI item, tracked in the CLI reference.
439
+
440
+ **Trust and speed differentiators (former Phase 4):**
441
+
442
+ - "Temporary chat" pill — provably never written to disk.
443
+ - Per-session disk-touch tally (files read / written) as a trust log
444
+ distinct from tool cards; per-session "zero cloud calls" indicator.
445
+ - Prompt-cache-hit telemetry per turn ("reused N cached tokens").
446
+ - logprobs-derived confidence overlay (server already returns
447
+ `logprobs` / `top_logprobs`).
448
+ - HLG tone-curve controls in the composer (today only on the Curves page).
449
+ - App-aware assistant maturity: a persistent minimized affordance on every
450
+ view (Developer tabs included) that minimizes on engagement and
451
+ re-expands only on invocation; process-state enrichment of the snapshot
452
+ (job progress, field completion) beyond route / step.
453
+
454
+ **Stretch (former Phase 5; pick one or two at a time):**
455
+
456
+ - Full conversation tree view over the sibling groups Pi already stores.
457
+ - Persona bundles (system prompt + adapter + tool allowlist + memory scope)
458
+ — presets v1's data model is shaped so this is a thin layer.
459
+ - Compare / split view: one prompt against base vs adapter, or two quant
460
+ tiers, side by side with per-pane tok/s (optiq Arena's mechanics, plus
461
+ the comparisons Arena cannot express).
462
+ - Approval-gated `ui_act` tools.
463
+ - Artifacts side panel, opt-in by heuristic; Mermaid; vector RAG upgrade;
464
+ MCP client; Skills-style procedural memory (all Appendix A rows with a
465
+ written state).
466
+ - Memory that graduates into a trained memory LoRA — roadmap; the UI must
467
+ never claim it before it ships.
468
+
469
+ **Risks that stay live:** the superset doctrine ballooning scope (keep
470
+ items independently shippable; re-audit Appendix A at each boundary, not
471
+ mid-stream); the lane badge going stale in a mode transition (server-
472
+ driven only); always-allow persistence regressing to the fragility seen in
473
+ other desktop agents (versioned config keyed by tool identity, migration
474
+ test); the memory diff view unsettling a user (user-initiated inspection,
475
+ pause / reset one click away).
502
476
 
503
477
  ---
504
478
 
505
- ## 6. Differentiator features (deep treatment)
506
-
507
- These five are the headline bets — the features that make this unmistakably
508
- mlx-bun and structurally hard for any surveyed competitor to copy without
509
- rebuilding their own serving stack or personalization architecture.
510
-
511
- ### 6.1 Memory as a visible, weight-grounded flywheel
512
-
513
- Not a fact list (AnythingLLM, ChatGPT, Claude) — a git-tracked vault where
514
- chat-time reads are strictly read-only (`src/memory/tools.ts`) and only
515
- nightly synthesis writes (`docs/design/memory-system.md`). The UI's job:
516
- (a) a provenance chip on any turn that consulted the vault, (b) a real
517
- Memory panel (§5.5) with a git-diff toggle so a user can *watch* an article
518
- evolve after a correction — literally the "self-healing through ordinary
519
- use" story (`docs/design/the-dreaming-master-plan.md`) made visible instead
520
- of asserted, (c) eventually, a visible marker distinguishing "this fact
521
- lives only as raw context" vs. "this has been baked into your memory LoRA" —
522
- a distinction no competitor's architecture can even express, since none of
523
- them synthesize memory into trained weights.
524
-
525
- ### 6.2 LoRA-per-context routing, made legible
526
-
527
- optiq proved the resident-multi-adapter mechanism works (~50MB/adapter,
528
- 6.1GB for 10 resident vs. 56GB for 10 full model copies) but shipped it
529
- Python-API-only. LM Studio's own bug tracker confirms adapter hot-swap is
530
- unsupported. mlx-bun already has `/v1/adapters`, adapter-namespaced KV, and
531
- per-request injection working end to end (`src/lora.ts`,
532
- `docs/design/adapters-end-to-end.md`). The UI work is turning this into a
533
- routing table: which adapter is active per-surface, why (matched from
534
- surface / explicitly selected), its RAM cost, and — the genuine escalation —
535
- support for stacking (`"a+b"`) directly in the picker, something the backend
536
- already does and no UI anywhere exposes.
537
-
538
- ### 6.3 On-device privacy, proven not asserted
539
-
540
- A one-click "Temporary chat" pill (matching ChatGPT/Claude's composer-header
541
- convention) that — unlike cloud temporary chat, which OpenAI's own
542
- disclosures say is still retained 30 days for "safety" — can be made
543
- **provably** never written to disk at all, because we control the whole
544
- stack. Pair this with a per-session "zero cloud calls" indicator and a
545
- running tally of what the agent touched on disk (files read/written this
546
- session) as a trust surface distinct from generic tool-call cards.
547
-
548
- ### 6.4 Speed surfaced as a felt experience, not a hidden stat
549
-
550
- The perf strip (§5.2) — tok/s, TTFT, context-fill, lane badge — is the
551
- direct answer to "serving speed is the user metric" (MEMORY.md). No
552
- surveyed competitor (including the MLX-native peers macMLX/mlx-serve, which
553
- do have a Benchmark tab) integrates lane-awareness (serial vs. spec-decode
554
- vs. batched) into the *live chat surface* itself — they show a benchmark
555
- number in a separate tab, we can show which lane is running the actual
556
- conversation right now. This directly explains latency variance instead of
557
- letting it read as jank, and ties concretely into the active `dspark`
558
- spec-decode branch.
559
-
560
- ### 6.5 Local tool transparency as a trust surface, not a permission dialog
561
-
562
- Every competitor's tool-approval UX is either a blocking modal (old Jan,
563
- Claude Desktop) or a binary allow/deny with no argument preview (most).
564
- mlx-bun already has the approval round-trip built (`pi-web.ts:748-774`); the
565
- differentiator is finishing it properly: inline non-blocking cards (Jan
566
- v0.8.0's pattern) with editable arguments before execution (LM Studio's
567
- pattern) and durable, tool-identity-keyed "always allow" persistence (fixing
568
- the failure mode publicly documented against Claude Desktop). Combined with
569
- §6.3's disk-touch tally, this reframes "the agent can run bash" from a scary
570
- permission dialog into a legible, reviewable log — the natural trust story
571
- for a single-user local product that structurally doesn't need Open WebUI's
572
- sandbox theater.
573
-
574
- ### 6.6 The app-aware assistant (added 2026-07-06)
575
-
576
- The chat agent can see and act on the app it lives in. Verified against the
577
- full beat matrix: **no product in the surveyed field has this** (matrix
578
- Axis 12). Reference implementation proven in-house on the same pi SDK:
579
- `~/Code/PortfolioManager`.
580
-
581
- **Why this matters (the product theory, not the feature):** both existing
582
- in-app help paradigms fail on the same axis from opposite ends. Guided
583
- tours fail on *timing* — they front-load help before intent exists, so
584
- users skip them and the knowledge is gone by the time they care. Chatbot
585
- help fails on *grounding* — it arrives at the moment of intent but blind:
586
- no idea what screen you're on, what step you're in, what you just tried; a
587
- search bar with a face, and users feel it immediately. The app-aware
588
- assistant is the only shape with both: help at the moment of intent,
589
- grounded in current state. Instead of screenshotting and circling a thing
590
- for an AI, you ask — and it highlights, navigates, takes you there. **The
591
- acceptance test is a feeling: like someone is there with you.** Not a
592
- tour, not a search bar — presence.
593
-
594
- **Design consequences the feeling forces** (a naive port misses these):
595
-
596
- - **Ambient across every view, not confined to the Chat tab.** A guide you
597
- must leave the Quantize tab to talk to is blind again by construction.
598
- Ship a lightweight persistent assistant affordance on every view —
599
- Developer tabs included — with the Chat tab as the deep surface
600
- (PortfolioManager's floating-panel pattern proves it on our SDK).
601
- **And the panel itself is never intrusive:** the moment the user engages
602
- the page (clicks Quantize, focuses a form field), the panel minimizes
603
- with a smooth animation to a small persistent affordance — you know it's
604
- there, it isn't blocking you. It re-expands on invocation, never on its
605
- own initiative.
606
- - **The snapshot carries *process state*, not just elements.** "Knows what
607
- part of the process you're in" means the context push includes wizard
608
- step (Quantize step 2 of 4), running job progress, which fields are
609
- filled — beyond PortfolioManager's visible-element list.
610
- - **Never answer blind.** Tool-pull alone (the agent *choosing* to call
611
- `get_current_app_context`) sometimes won't fire, and one ungrounded
612
- answer breaks the "with you" illusion. Auto-attach a compact ambient
613
- context line (route + view + step) to every turn; the deep snapshot
614
- stays on-demand.
615
- - **Choreography:** the spotlight lands *while* the explanation streams
616
- ("it's right here —" and the highlight appears), not as a report
617
- afterwards.
618
- - **Never hijack (hard constraint).** The worst tour UX is blocking the
619
- screen when the user wants to DO something. The spotlight is an overlay,
620
- never a modal: it traps no focus, blocks no clicks (the highlighted
621
- control stays clickable *through* it), auto-dismisses in seconds, and
622
- any user input dismisses it instantly. Navigation only happens when the
623
- user asked to be taken somewhere — never uninvited mid-task. The agent
624
- is a hand pointing, not a hand on the wheel; there is never a state the
625
- user has to escape from. (Same rule already governs approval cards —
626
- inline and non-blocking, §6.5.)
627
-
628
- Mechanism — three tools plus one client loop, no screenshots, no vision
629
- model in the loop:
630
-
631
- 1. **See**: on route change the web app pushes a structured `uiSnapshot`
632
- (visible interactive elements + labeled regions as
633
- `{ref, label, role, selector}`, ~120-element cap, agent chrome
634
- excluded) as a `context` WS frame; a `get_current_app_context` tool
635
- returns it. The screenshot→upload→"what am I looking at" round-trip
636
- disappears because the app narrates itself, locally.
637
- 2. **Navigate**: `navigate_app` validated against a curated route catalog
638
- (`data-spotlight` anchors + route map, PortfolioManager's
639
- `ui-catalog.json` pattern) → `ui_navigate` frame → the browser routes.
640
- "Take me to Quantize" literally clicks the tab. Reversible, no approval
641
- needed.
642
- 3. **Point**: `spotlight_ui` locates a control by ref/label/catalog-id,
643
- optionally navigates first, and highlights it with a hint popover
644
- (hand-rolled backdrop-cutout overlay in our tokens, ~80 lines —
645
- no vendored tour library needed), auto-dismissing.
646
-
647
- Why it's ours to win: (a) our web chat IS a pi agent over WS — the port is
648
- mechanical (`pi-web.ts` tools + two frames + a snapshot builder in the
649
- app); (b) it resolves the Developer-toggle discoverability tension §8
650
- creates — hidden capability becomes conversationally discoverable ("can
651
- this make a smaller model?" → reveal, navigate, spotlight); (c) the
652
- escalation nobody else can ship: approval-gated `ui_act` tools (fill this
653
- form, start this job) rendered through the §6.5 approval-card trust
654
- surface — guided *actions* with visible allow/deny, where PortfolioManager
655
- deliberately stops at navigate+point and generic computer-use agents act
656
- with no in-app trust model at all; (d) with the memory vault, "take me to
657
- that adapter I trained last week" actually resolves.
658
-
659
- Scope guard: the snapshot never includes cross-origin content; the three
660
- tools live on the pi-web bridge only (never the OpenAI/Anthropic API
661
- surface); `ui_act` is Phase 5 and requires the Phase-2 approval surface
662
- first.
479
+ ## History
480
+
481
+ - 2026-06-12 — minimal `/chat` page (SSE against `/v1/chat/completions`).
482
+ - 2026-06-14 — chat replaced with the Pi embed over `/ws/chat`
483
+ (`src/pi-web.ts`), unified hash-routed SPA with the Developer tabs.
484
+ - 2026-06-16 — adapter discovery endpoint, web selector, and the Pi CLI
485
+ extension (`docs/design/web-chat-redesign.md` written; its web half now lives in
486
+ §3.6, the CLI half belongs to the CLI reference). The Tauri desktop-shell
487
+ idea (`docs/design/web-chat-redesign.md`: separate repo, sidecar `mlx-bun serve`,
488
+ WKWebView on `/#/chat`) was written in this period; it is tracked in
489
+ neither PLAN.md nor STATUS.md and is not a plan — the only desktop-specific
490
+ item kept here is that a shell must wrap the same page, no bespoke UI.
491
+ - 2026-07-06 — this plan merged three design visions and the market survey;
492
+ Phase 0 (UI-only bug pass) and Phase 1 (block-memoized render,
493
+ highlighting, full sampler popover, perf strip + lane, message actions,
494
+ drawer, theme, shortcuts, a11y) landed. Beat matrix written (Appendix A).
495
+ - 2026-07-07 — Phase 2 (typed module split, memory panel + REST, adapter
496
+ routing table, system prompt + presets, approval gate wired, Developer
497
+ IA) and Phase 3 (files RAG, Model Hub, Canvas, per-turn sampling scope,
498
+ self-healing tool calls + loop hygiene, app-aware assistant v1, command
499
+ palette, full-text search, export, PWA) landed.
500
+ - 2026-08-23 — consolidated into this canonical doc; verified against src.
663
501
 
664
502
  ---
665
503
 
666
- ## 7. Tech-stack & architecture decision
667
-
668
- **Recommendation: stay vanilla-single-file, but stop hand-rolling
669
- everything.** Do not migrate to React/Svelte/a SPA framework. Reasoning:
670
-
671
- - The research is explicit that every framework-based pattern found
672
- (Streamdown's block memoization, react-virtuoso's windowing,
673
- Radix/Base-UI's accessible primitives) assumes a component-diffing
674
- runtime — but the *ideas* (memoize completed blocks, window a long list,
675
- trap focus in a popover) port cleanly into vanilla JS without importing
676
- the library. The current file already proves this pattern works (the
677
- hand-rolled markdown renderer is competitive).
678
- - A framework migration is a multi-month distraction that trades a
679
- self-contained, no-build, no-CDN served single file (the current
680
- `app.html` header comment's own stated constraint) for a build pipeline,
681
- for a benefit (component reuse, cleaner state management) that matters
682
- more at team scale than at this file's actual complexity once the
683
- concrete gaps (memoized rendering, syntax highlighting, math, a11y) are
684
- closed directly.
685
- - The three visions converged independently on the same conclusion ("no
686
- React needed, just a smarter diff in the existing vanilla renderer") —
687
- treat that convergence as signal, not coincidence.
688
-
689
- **What does change:**
690
- - **Split `app.html` into modules that build into the single served file**,
691
- not a runtime framework. At 2737 lines and growing (memory panel, adapter
692
- routing table, perf strip, message actions are all net-new surface), a
693
- single file becomes a real maintenance cost even without a framework.
694
- Introduce a minimal build step (`bun build` with inline output, still
695
- zero external CDN dependency at runtime) that concatenates
696
- `src/web/{shell,chat,memory,adapters,quantize,finetune,dataset,status}.{html,css,ts}`
697
- into the one file the server ships. This preserves every runtime
698
- constraint (self-contained, no CSP-breaking externals) while making the
699
- source tree navigable.
700
- **The split's primary payoff is the typed contract, not navigability**
701
- (amended 2026-07-06): once the inline script is TypeScript, it imports
702
- `ServerMessage`/`ClientMessage` from `src/pi-web.ts` and the job-stream /
703
- API response shapes from their source modules — and the single largest
704
- empirical bug class disappears at compile time. Look at what Phase 0
705
- actually fixed: `e.lr` vs `e.learning_rate`, array-truthiness on a WS
706
- frame, a response-shape mismatch, a dead string-enum — every one is
707
- contract drift between untyped inline JS and typed TS, and every one
708
- becomes a build error after the split. **Scheduled: the split is Phase 2's
709
- first work item** — before the memory panel and routing table add major
710
- new surface to the monolith, not after.
711
- - **Frontend regression harness** (amended 2026-07-06 — this repo
712
- oracle-gates everything else; the UI can't be the one untested surface):
713
- (a) formalize the inline-script syntax check (extract `<script>`,
714
- `new Function()` it) as a test/hygiene-gate step instead of an ad-hoc
715
- command; (b) a `tests/web-app.test.ts` DOM-level unit file (happy-dom or
716
- equivalent) covering the pure client functions — `splitBlocks`/streaming
717
- memoization (streamed-then-finalized output must equal one-shot
718
- `mdToHtml` on a fixture corpus: the renderer's own parity oracle),
719
- `renderQueue`, `api()` error unwrapping, esc() discipline on
720
- interpolation sites; (c) post-split, type-checking the frontend modules
721
- in the same `tsc --noEmit` pass as the server. Playwright-class
722
- end-to-end stays out of scope until there's a model-free server stub to
723
- drive it against.
724
- - **Inline, self-hosted Shiki (or a lighter alternative) and KaTeX** as
725
- vendored, no-CDN assets bundled at build time — a real dependency-weight
726
- tradeoff against the "no build, no CDN" constraint, but justified because
727
- monochrome code and raw-LaTeX-backslash output are concrete, embarrassing
728
- gaps (§2.2 #9) with no vanilla-hand-rolled alternative that doesn't
729
- reinvent a TextMate-grammar highlighter from scratch.
730
- - **Hand-roll block-memoized streaming and message-list windowing** rather
731
- than adopt react-virtuoso — port the *idea* (completed-block skip-reparse,
732
- windowed DOM recycling anchored to bottom scroll) as vanilla functions.
733
-
734
- **Tauri desktop compatibility:** the Tauri shell today wraps "the same page,
735
- no bespoke UI" (`docs/design/tauri-desktop-app.md:50-51`). Keep that
736
- contract — every feature in this plan ships to both surfaces for free by
737
- construction, since nothing here depends on a browser-only API. The one
738
- Tauri-specific opportunity (deferred to Phase 5, not required): a global
739
- hotkey + floating companion window (mirroring ChatGPT macOS's Option+Space
740
- and mlx-serve's Ctrl-Space quick-launcher) — a concrete way for the desktop
741
- wrap to earn native-app status instead of being a browser tab in a frame,
742
- made cheap by the vision pipeline already existing server-side.
743
-
744
- **Theming:** add `prefers-color-scheme` handling and a light theme (dark-only
745
- today, hardcoded `--bg:#000`). Respect `prefers-reduced-motion` on the
746
- ambient bloom/shimmer animations.
747
-
748
- **Responsive/mobile/PWA:** mobile drawer (§5.7); add a web app manifest +
749
- minimal cache-first service worker for the app shell (installable
750
- "add to home screen," zero-cost given the app is already a single served
751
- page) — explicitly scoped as *shell-instant-load + installability*, not
752
- meaningful offline chat (the app is useless without the local inference
753
- server running, so deep offline-caching strategy is not worth over-investing
754
- in). Test standalone-mode viewport height explicitly since browser chrome
755
- disappearing changes available height in ways normal responsive testing
756
- misses.
757
-
758
- **A11y:** add `role="log" aria-live="polite" aria-atomic="false"` on the
759
- message thread, throttled to paragraph/sentence boundaries (not per-token —
760
- combining focus-move and live-region announcement in the same update is the
761
- documented anti-pattern per WCAG ARIA23), verified against actual VoiceOver
762
- testing rather than one blog post's guidance. Full keyboard tab order
763
- through composer/sidebar/popovers, visible focus rings, Escape-to-close on
764
- every popover with focus returned to trigger. Ship a `Cmd/Ctrl+/` shortcut
765
- sheet matching ChatGPT's vocabulary where sensible (`Cmd/Ctrl+Shift+O` new
766
- chat, `Shift+Esc` focus composer, `Cmd/Ctrl+Shift+C` copy last response) —
767
- zero cost to originality, closes an embarrassing baseline gap, and "copy
768
- last response" reuses the existing copy-button code.
769
-
770
- **Long-thread performance:** windowed rendering (anchored-to-bottom DOM
771
- recycling) once a session exceeds ~150-200 rendered turns — the
772
- memory-flywheel/long-session north star means power users will accumulate
773
- very long sessions over months, more likely here than in a typical
774
- stateless chatbot, so this failure mode (documented against Open WebUI as
775
- issue #23067) is a higher-probability event for us than for competitors.
504
+ ## Appendix A — beat matrix (condensed)
776
505
 
777
- ---
506
+ Every capability a user judges a local chat app by, judged against the best
507
+ current implementation in the surveyed field (mlx-lm, mlx-optiq, Ollama,
508
+ Open WebUI, LM Studio, Odysseus, Jan, Msty, GPT4All, AnythingLLM,
509
+ Enchanted, macMLX/mlx-serve, ChatGPT, Claude.ai). **MATCH** = parity is
510
+ enough; **BEAT** = structurally better because of something only we have;
511
+ **SKIP** = the engine lacks the substrate, stated so silence is never an
512
+ oversight. State column verified against src 2026-08-23. (Caveat: the
513
+ Odysseus rows rest on a public-web identification of that product; no
514
+ verdict flips on them.)
778
515
 
779
- ## 8. Information architecture
780
-
781
- **Chat is the product. Everything else is a developer tool.** Collapse the
782
- current 7 hash-routed tabs (Chat / Quantize / Fine-tune / Build Dataset /
783
- Status / Curves / Routes) behind a single top-level split:
784
-
785
- - **Chat** (default, always visible) — the entire app for a first-time or
786
- casual user. Contains, reachable via sidebar/settings without leaving the
787
- view: Memory panel, adapter routing, sampling/structured-output advanced
788
- settings, model switcher.
789
- - **Developer** (one nav toggle, LM Studio's User/Power-User/Developer
790
- pattern collapsed to a binary since we don't need three tiers yet) —
791
- reveals Quantize, Fine-tune, Build Dataset, Status, Curves, Routes as a
792
- sub-nav. Persisted as a local preference (don't re-ask every session).
793
-
794
- This is not hiding capability — it's matching disclosure to audience. A
795
- user who never intends to quantize a model should never see that tab; a
796
- developer flips one switch once and gets everything, including the existing
797
- tabs completely unchanged in their internal design. The app-aware assistant
798
- (§6.6) is the escape hatch that makes the collapse safe: hidden capability
799
- stays conversationally discoverable — "can this make a smaller model?" and
800
- the agent reveals the Developer tabs, navigates, and spotlights the control,
801
- so progressive disclosure never becomes buried capability (risk #4's
802
- strongest mitigation). Status's RAM/KV "Memory"
803
- card keeps its current meaning (system memory) — the personal-memory vault
804
- gets its own distinctly-named panel (§5.5) precisely to avoid the homonym
805
- confusion the audit flagged.
806
-
807
- Routes (currently an iframe of a DAG, 404s in the compiled binary per
808
- `web-ui-pass-plan.md` #17) and Curves (a separate full-page route, never
809
- highlights in nav per #19) both move under Developer and get their standing
810
- bugs fixed as part of the IA move (§9 Phase 1).
516
+ **Axis 1 — Chat fundamentals**
811
517
 
812
- ---
518
+ | Capability | Verdict | State |
519
+ |---|---|---|
520
+ | Streaming with a prominent Stop | MATCH | shipped |
521
+ | Stop aborts in-flight tool work, not just tokens | MATCH | shipped (verified via Pi abort) |
522
+ | Disk persistence, no account | MATCH | shipped (Pi sessions) |
523
+ | Regenerate | MATCH | shipped |
524
+ | Edit-and-resend as a sibling branch, `< i/n >` | MATCH | shipped |
525
+ | Full conversation tree view | BEAT | open |
526
+ | Live steering + follow-up queue while streaming | BEAT | shipped |
527
+ | Keyboard reach for stop / regenerate / new chat | MATCH | shipped |
528
+ | Full-message copy | MATCH | shipped |
529
+ | Delete a single message | MATCH | open |
530
+ | Compaction seam marked in the thread | MATCH | open |
531
+
532
+ **Axis 2 — Rendering**
533
+
534
+ | Capability | Verdict | State |
535
+ |---|---|---|
536
+ | GFM markdown (hand-rolled, competitive) | MATCH | shipped |
537
+ | Syntax highlighting, vendored | MATCH | shipped |
538
+ | LaTeX / KaTeX | MATCH | open |
539
+ | Mermaid | MATCH (low priority) | open |
540
+ | Block-memoized streaming (no O(n²) re-parse) | BEAT | shipped |
541
+ | Sandboxed HTML / SVG preview with view-source | MATCH | shipped (Canvas v1) |
542
+ | Persistent Artifacts side panel | MATCH (opt-in) | open |
543
+ | logprobs confidence overlay | BEAT | open (server side exists) |
544
+ | Structured-output / grammar picker | BEAT | open (server side exists) |
545
+
546
+ **Axis 3 — Model discovery and management**
547
+
548
+ | Capability | Verdict | State |
549
+ |---|---|---|
550
+ | Hardware-fit verdict per model, measured for this Mac | BEAT | shipped (picker + Hub) |
551
+ | Browse / search / download / load in one panel | BEAT | shipped, except live load (restart command) |
552
+ | Capability badges (vision, quant tier) | MATCH | shipped (Hub chips) |
553
+ | In-chat model switch without restart | MATCH | open (no in-process swap seam) |
554
+ | Swap preserves session history | BEAT | open (same blocker) |
555
+ | Resident multi-model pool | MATCH | open |
556
+ | Running-models resource table with adapter / KV tier / draft columns | BEAT | open |
557
+ | Per-layer quantization sensitivity visualization | BEAT | open |
558
+ | `mlx-bun launch <tool>` one-command coding-tool wiring | BEAT | open (CLI) |
559
+
560
+ **Axis 4 — Prompts, presets, sampling**
561
+
562
+ | Capability | Verdict | State |
563
+ |---|---|---|
564
+ | Full sampler set in the UI | MATCH | shipped (`logit_bias` not exposed) |
565
+ | Per-model correct defaults | MATCH | shipped (`ready.genDefaults`) |
566
+ | Per-message sampling scope | MATCH | shipped (`next_turn`) |
567
+ | Saved presets (prompt + sampling) | MATCH | shipped v1 (local only) |
568
+ | Slash-command prompt library with variables | MATCH | open |
569
+ | Model-string variant shorthand (`:no-think`) | MATCH | open |
570
+ | Persona bundles bound to a real adapter swap | BEAT | open |
571
+ | Explicit "warm this session" | MATCH | open |
572
+
573
+ **Axis 5 — Documents and RAG**
574
+
575
+ | Capability | Verdict | State |
576
+ |---|---|---|
577
+ | Zero-config attach-and-answer | MATCH | shipped |
578
+ | Transparent inline-vs-retrieval decision | MATCH | shipped (threshold in `rag.ts`) |
579
+ | `[n]` citations + Sources panel | MATCH | shipped |
580
+ | Unified `#` mention over files + vault | MATCH | shipped |
581
+ | Per-document retrieval-mode toggle | MATCH | open |
582
+ | Hybrid BM25 + vector + rerank | SKIP (for now) | open, deferred tier |
583
+ | Semantic search over the user's own chat history | BEAT | partial (full-text body search shipped; semantic open) |
813
584
 
814
- ## 9. Phased roadmap
815
-
816
- Each phase is independently shippable and includes the relevant tactical
817
- fixes from `docs/planning/web-ui-pass-plan.md` by number. Ordered by
818
- effort/reward, starting with free wins, ending with differentiator
819
- features. The [beat matrix](web-chat-beat-matrix.md) is the coverage
820
- checklist across all phases: at any release point, every matrix row is
821
- either shipped, in a scheduled phase below, or a written SKIP — a MATCH row
822
- with none of those three states is a planning bug.
823
-
824
- ### Phase 0 — Bug fixes, one UI-only PR (no server changes, ~2-3 days)
825
-
826
- Ship first, blocks nothing, touches no backend:
827
-
828
- - `web-ui-pass-plan.md` #1: fine-tune live stats field-name mismatch
829
- (`e.lr`/`e.tps` → `e.learning_rate`/`e.tokens_per_sec`).
830
- - #2: queue-bar array-truthiness bug (`.length` guards, `app.html:1683-1684`).
831
- - #4: `[object Object]` adapter-mount error (unwrap `data.error.message` in
832
- `api()`, `app.html:1080`).
833
- - #5: quantize inspect result parsing (`support: boolean`, not string enum).
834
- - #6: stale "switch to 12B for vision" copy → model-agnostic.
835
- - #15 (partial): escape `a.id`/`a.path` interpolation in the adapter
836
- dropdown (`app.html:1897-1898`) — this is a live unescaped-HTML injection
837
- point, fix immediately regardless of the larger adapter-UI rework in
838
- Phase 2.
839
- - #16: delete dead `src/status-page.html` + `scripts/status-page-stub.ts`.
840
- - #18: CLI banner copy fix (`src/cli.ts:64`).
841
-
842
- ### Phase 1 — Free wins: wire existing backend, fix the render bottleneck (1-2 weeks)
843
-
844
- The highest-leverage phase — almost all wiring of existing capability +
845
- one architectural render fix. Two deliberate, thin server additions ride
846
- along (scoped in §2.3's caveat): the per-turn `lane` field for the perf
847
- strip, and the gc plan/execute endpoints (#9):
848
-
849
- - **Streaming render fix** (§5.3): block-memoized append, replacing
850
- `appendDelta`'s full re-parse. This unblocks every other visual change in
851
- this phase since they all touch the same render path.
852
- - Syntax highlighting (vendored Shiki or equivalent) + KaTeX, applied to
853
- completed blocks only.
854
- - Sampling popover: full parameter set (`min_p`, XTC, penalty families +
855
- context windows, `logit_bias`, seed) behind an Advanced disclosure;
856
- per-model `generation_config.json`-sourced defaults (fixes
857
- `web-ui-pass-plan.md` #14 and #8). Wire end-to-end: popover → `set_sampling`
858
- WS frame → `SamplingOverrides` → `injectSampling` (all three currently
859
- hardcode 3 fields, `pi-web.ts:171-176`).
860
- - Perf strip: tok/s, TTFT, context-fill, lane badge (serial/spec-decode/batched)
861
- under the composer. Server side: add the per-turn `lane` field to the
862
- usage payload + pi-web frame (see §2.3 caveat) — the badge must be
863
- server-driven, never client-inferred (risk #5).
864
- - Message actions: copy full assistant message, regenerate, edit-message-as-branch
865
- with `< 1/2 >` toggle (reuses `SessionManager.forkFrom`).
866
- - Adapter dropdown → chip with specialty description + `compatible` graying
867
- (still single-select in this phase; full routing table is Phase 2).
868
- - `web-ui-pass-plan.md` #3 (library staleness), #7 (`sft_scope` control),
869
- #9 (gc affordance — needs the new `/api/gc/plan`/`/api/gc/execute`
870
- endpoints), #11-13 (the queued `listCanonical`/`visionCapable`/`supportTier`
871
- wiring, land as its own small PR once server-side lands), #20 (library
872
- "too big" row fix), #21 documented as a known limitation.
873
- - Mobile sidebar → slide-over drawer.
874
- - Session search box.
875
- - `prefers-color-scheme` + light theme + `prefers-reduced-motion` guards.
876
- - `Cmd/Ctrl+/` shortcut sheet + ChatGPT-vocabulary bindings.
877
- - `role="log" aria-live="polite"` on the thread, focus-trap/Escape on
878
- popovers.
879
-
880
- ### Phase 2 — Module split, memory surface + adapter routing table (2-3 weeks)
881
-
882
- The two headline differentiators, now that rendering is fixed and settings
883
- have room — preceded by the structural fix that keeps the growing surface
884
- honest:
885
-
886
- - **Module split FIRST** (§7, amended): split `app.html` into
887
- build-concatenated `src/web/*.ts` modules (one served file unchanged at
888
- runtime) so the frontend imports `ServerMessage`/`ClientMessage` from
889
- `src/pi-web.ts` and job/API shapes from their source modules — the
890
- Phase-0 bug class (contract drift) becomes a compile error before the
891
- memory panel and routing table add major new surface. Ship the frontend
892
- regression harness (§7) in the same pass: formalized script-syntax gate,
893
- `tests/web-app.test.ts` over the pure client functions (streaming
894
- memoization vs. one-shot `mdToHtml` parity fixtures, `renderQueue`,
895
- `api()` unwrapping), frontend modules in the same `tsc --noEmit` pass.
896
- - **Per-chat system prompt** (beat matrix Axis 4 — table stakes in every
897
- competitor, currently impossible: the web chat's prompt is hardcoded in
898
- `buildPiAgentSurface`/`buildWebChatSystemPrompt`): an editable
899
- system-prompt field per session, layered over the built-in surface prompt,
900
- carried on the WS session; **presets v1** (named system-prompt + sampling
901
- bundle, saved locally) shaped so Phase 5's persona bundles (adapter +
902
- tools + memory scope) are a thin extension.
903
- - **Session auto-titling** (Axis 11): v1 derives from the first user
904
- message; model-generated titles optional and never queued ahead of real
905
- turns.
906
- - **Vault-init endpoint** (§5.1 prerequisite): the first-run consent card
907
- creates the vault from the web UI — needs a thin
908
- `POST /api/memory/init` wrapping what `mlx-bun memory init` does in the
909
- CLI (today it's CLI-only and invisible to non-terminal users).
910
- - Memory REST wrappers (prerequisite, server-side — see §2.3 caveat):
911
- `GET /api/memory/status|list|search|article` + a git log/diff read,
912
- thin delegating wrappers over `src/memory/vault.ts` / `tools.ts`
913
- internals. The agent-tool surface stays read-only and unchanged.
914
- - Memory panel (§5.5): vault browser, search, per-article git-diff view,
915
- status strip, `Reference/` sub-view, wikilink graph via `memory_links`.
916
- - Memory provenance chips on assistant turns that consulted memory tools
917
- (distinct card style from generic tool calls, §5.4).
918
- - Personalized hero/empty-state chips driven by `vaultStatus()` +
919
- `/v1/adapters/available` (§5.1); inline memory-consent card for first run.
920
- - Adapter routing table: three-state visibility (available/loaded/selected),
921
- RAM cost per loaded adapter, `"a+b"` stacking support in the picker.
922
- - Model switcher: clickable `#nav-model`, hardware-fit badge via `GET /fit`,
923
- session-preserving swap.
924
- - Tool-approval gate properly wired: `codingTools: true` opt-in toggle in
925
- settings, editable arguments before approve/deny, durable tool-identity-keyed
926
- "always allow" persistence.
927
- - Structured-output picker (JSON-schema textarea + live validation) in the
928
- Advanced disclosure.
929
- - Unified `#` retrieval mention (files + vault as one picker).
930
- - Information architecture move: collapse Quantize/Fine-tune/Build
931
- Dataset/Status/Curves/Routes behind the Developer toggle (§8); fix
932
- `web-ui-pass-plan.md` #17 (Routes 404 in compiled binary) and #19 (Curves
933
- nav highlight) as part of this move.
934
-
935
- ### Phase 3 — Concede nothing: the table-stakes sweep (2-3 weeks)
936
-
937
- (Added 2026-07-06 under the superset doctrine — the beat-matrix MATCH rows
938
- a daily LM Studio / Open WebUI / optiq-Lab user would immediately miss.
939
- Each item names its matrix axis.)
940
-
941
- - **Chat-with-files RAG v1** (Axis 5): attach-and-answer over attached
942
- documents with `[n]` citation markers + a sources panel — optiq Lab's
943
- BM25 dependency-free bar is the v1 target; we have `src/embed.ts` for a
944
- vector upgrade later, but don't block v1 on it. Extend the unified `#`
945
- mention (Phase 2) so files and vault articles are one retrieval gesture.
946
- - **Model Hub** (Axis 3): in-app browse (published quants / HF search /
947
- local) → download with progress → one-click load-to-server + new-chat,
948
- session-preserving hot-swap — optiq's Hub flow, PLUS the `/fit`
949
- hardware-verdict column per row that optiq lacks. "Install to chat under
950
- a minute" stops dead-ending at the CLI.
951
- - **Canvas v1** (Axis 2): fenced-HTML blocks rendered in a sandboxed
952
- iframe with a view-source toggle (optiq's shipped pattern; the full
953
- Artifacts side-panel stays Phase 5).
954
- - **Per-message sampling scope** (Axis 4): apply-to-next-turn-only option
955
- in the sampling popover (optiq's per-message temp/max-tokens/thinking
956
- granularity), on top of the session-level overrides from Phase 1.
957
- - **Self-healing tool-call loop** (Axis 7): investigate what the pi SDK
958
- already retries; add format-aware malformed-call repair (we own the
959
- per-model parsers in `src/tool-call.ts`), consecutive-duplicate dedup, a
960
- retry budget, and a loop cap with force-finish — small local models
961
- malform calls more than cloud frontiers, so this matters more here than
962
- it does for optiq.
963
- - **Stop kills tool subprocesses** (Axis 1): verify/wire abort through the
964
- pi tool layer so Stop ends running bash/web-fetch work, not just the
965
- token stream.
966
- - **Chat export** (Axis 10): Markdown/plain-text per session; portable
967
- JSON export/import (sessions are already local JSON on disk).
968
- - **Command palette** (Axis 10, `Cmd/Ctrl+K`): new chat / session search /
969
- model + adapter switch / thinking toggle / theme.
970
- - **Full-text session search** (Axis 10 BEAT): body search, not
971
- title-only — no cloud indexing-cost excuse applies locally.
972
- - **PWA installability** (Axis 10): manifest + shell service worker
973
- (installability + instant shell, explicitly not offline chat).
974
- - **App-aware assistant v1** (§6.6, matrix Axis 12 — promoted from Phase 4
975
- 2026-07-06: best wow-per-effort in this phase): `uiSnapshot` context push
976
- on route change + a compact ambient context line (route/view/step)
977
- auto-attached to every turn, `get_current_app_context` + `navigate_app` +
978
- `spotlight_ui` tools on the pi-web bridge, curated route/anchor catalog,
979
- hand-rolled spotlight overlay choreographed with the streaming
980
- explanation. Pattern proven in `~/Code/PortfolioManager` on the same pi
981
- SDK. The ambient every-view floating panel and process-state snapshot
982
- enrichment mature in Phase 4; approval-gated `ui_act` stays Phase 5.
983
-
984
- ### Phase 4 — Trust & speed differentiators (1-2 weeks)
585
+ **Axis 6 — Memory and personalization**
985
586
 
986
- - "Temporary chat" pill — provably never written to disk.
987
- - Per-session disk-touch tally (files read/written this session) as a
988
- trust log distinct from tool-call cards.
989
- - Prompt-cache-hit telemetry surfaced per turn ("served from cache").
990
- - logprobs-derived confidence visualization (optional, behind Advanced) —
991
- wire the existing `logprobs`/`top_logprobs` server support into an
992
- opt-in per-turn confidence overlay.
993
- - HLG tone-curve controls folded into the main chat composer (currently
994
- disconnected, only reachable via the standalone Curve Designer page).
995
- - **App-aware assistant: ambient panel maturity** (§6.6): the floating
996
- assistant affordance on every view (Developer tabs included) with the
997
- Chat tab as the deep surface, and process-state enrichment of the
998
- snapshot (wizard step, job progress, field completion) — building on
999
- Phase 3's v1.
1000
-
1001
- ### Phase 5 — Stretch differentiators (ongoing, pick 1-2 per quarter after Phase 4)
1002
-
1003
- The next tier of unmistakable bets:
1004
-
1005
- - Full conversation tree view (not just linear `< 1/2 >` toggle) — a
1006
- genuine gap neither ChatGPT nor Claude has shipped natively.
1007
- - Persona/bundle system: named (system prompt + adapter + tool allowlist +
1008
- memory scope) bundles, switchable in one click, exportable/importable.
1009
- - Compare/split view: same prompt against two adapters or two quant tiers
1010
- simultaneously (Msty's Split Chat pattern, repointed at LoRA A/B testing
1011
- instead of cross-vendor model shopping).
1012
- - Overnight/background training loop with a persistent monitor and
1013
- completion notification ("wake up to a new adapter") — closes
1014
- `docs/design/adapters-end-to-end.md` §D, currently PENDING.
1015
- - Tauri-specific: global hotkey + floating companion window with instant
1016
- screenshot capture (vision pipeline already exists).
1017
- - Opt-in-by-heuristic Artifacts/Canvas side-panel for long code/HTML blocks
1018
- with highlight-to-scope-edit (Canvas v1 iframe ships in Phase 3).
1019
- - MCP client support (beat matrix Axis 7 — table stakes for 2026 agentic
1020
- chat per Open WebUI/LM Studio/Jan; moderate-large build behind the
1021
- existing tool plumbing, scheduled here rather than silently absent).
1022
- - Skills-style self-evolving procedural memory (Odysseus's pattern; large
1023
- separate system, matrix SKIP-for-v1 — revisit once the memory panel has
1024
- usage data).
1025
- - Mermaid diagram rendering (matrix Axis 2 — add if the code-heavy audience
1026
- demands it; completed-block hook makes it cheap to slot in).
1027
- - Vector upgrade to chat-with-files RAG (BM25 v1 → `src/embed.ts` hybrid).
1028
- - **Approval-gated `ui_act` tools** (§6.6 escalation): the app-aware
1029
- assistant graduates from navigate+point to fill-this-form /
1030
- start-this-job, every action rendered through the approval-card gate
1031
- with argument preview — guided actions with a visible trust surface,
1032
- which neither PortfolioManager (stops at navigate+point) nor generic
1033
- computer-use agents (act with no in-app trust model) ship. Requires
1034
- Phase 2's approval surface + Phase 4's v1.
587
+ | Capability | Verdict | State |
588
+ |---|---|---|
589
+ | Visible, searchable memory (vault browser) | BEAT | shipped |
590
+ | Per-turn provenance chip | BEAT | shipped |
591
+ | Git history / diff of what synthesis changed | BEAT | shipped |
592
+ | Wikilink graph view | BEAT | open (links line only) |
593
+ | Scope control (session / surface / global) | MATCH | open |
594
+ | Pause vs reset as distinct actions | MATCH | open |
595
+ | Memory graduating into trained weights, marked in UI | BEAT | roadmap — never claim early |
596
+ | First-run consent in the hero | BEAT | shipped |
597
+ | Synthesis schedule status visible | MATCH | shipped (status strip) |
598
+ | Self-evolving procedural "Skills" | SKIP (v1) | open, deferred |
599
+
600
+ **Axis 7 — Tools and agents**
601
+
602
+ | Capability | Verdict | State |
603
+ |---|---|---|
604
+ | Tool calls as distinct cards | MATCH | shipped |
605
+ | Inline non-blocking approval cards | MATCH | shipped |
606
+ | Editable arguments before approval | MATCH | shipped |
607
+ | Durable tool-identity-keyed always-allow | BEAT | shipped (`src/tool-approvals.ts`) |
608
+ | Gated tools reachable in web chat (opt-in) | MATCH | shipped |
609
+ | Sandboxed Python tool | MATCH | open |
610
+ | Self-healing malformed tool calls | MATCH | shipped (`src/tool-call.ts`) |
611
+ | Loop hygiene (dedup, retry budget, cap, force-finish) | MATCH | shipped |
612
+ | Argument / diff preview before approval | BEAT | shipped (args); file diff open |
613
+ | Per-session disk-touch tally | BEAT | open |
614
+ | Curated default tool pack on first run | MATCH | partial (read, web_search, memory, app-aware) |
615
+ | MCP client | MATCH | open |
616
+ | Web search + URL fetch without keys | MATCH | `web_search` shipped; `web_fetch` not in the web allowlist |
617
+ | Approval-fatigue-aware defaults (read-only auto-allowed) | MATCH | shipped |
618
+ | `tool_choice` forcing in the chat UI | SKIP | API only by design |
619
+ | Visual agent-flow builder | SKIP | not planned |
620
+ | Deep-research trace panel | MATCH | open |
621
+
622
+ **Axis 8 — Multimodal**
623
+
624
+ | Capability | Verdict | State |
625
+ |---|---|---|
626
+ | Image attach + inference | MATCH | shipped (gated by `ready.vision`) |
627
+ | No text-only tax from the vision path | MATCH | verify |
628
+ | Image generation | SKIP | no substrate |
629
+ | Speech-to-speech voice | SKIP | no substrate |
630
+ | Native OS dictation / TTS | SKIP | no substrate |
631
+ | Document attach (PDF / DOCX) with inline-vs-RAG | MATCH | text files shipped; binary formats open |
632
+ | Global hotkey + floating companion window with screenshot | BEAT | open (needs a desktop shell) |
1035
633
 
1036
- ---
634
+ **Axis 9 — Performance and transparency**
1037
635
 
1038
- ## 10. Success metrics & risks
1039
-
1040
- ### Success metrics
1041
-
1042
- - **Memory visibility:** % of sessions where a memory-consulting turn shows
1043
- a provenance chip (should approach 100% of turns where memory tools
1044
- actually fired — a wiring correctness metric, not an engagement metric).
1045
- - **Adapter routing usage:** adapter-swap events per week per active user,
1046
- post-Phase-2 vs. pre-redesign baseline (currently near-zero given the
1047
- dropdown's poor legibility).
1048
- - **Render performance:** p95 time-to-paint for a 2000-token streamed
1049
- response, measured on the M4 Pro reference machine per the `benchmark.sh`
1050
- discipline — before/after the block-memoization fix, quotable only from a
1051
- preflight-gated run.
1052
- - **Tool-approval engagement:** % of gated-tool calls that get an explicit
1053
- approve/deny vs. auto-deny timeout (a proxy for whether the trust surface
1054
- is legible enough to act on, not just present).
1055
- - **Mobile session continuity:** % of mobile sessions where a user
1056
- successfully switches sessions via the new drawer (currently impossible).
1057
- - **A11y baseline:** VoiceOver walkthrough of a full send→stream→tool-call→
1058
- approve cycle completes with no unannounced state changes.
1059
-
1060
- ### Top risks & mitigations
1061
-
1062
- 1. **Scope creep into a framework rewrite.** Mitigation: §7's explicit
1063
- recommendation against a SPA framework migration, with the reasoning
1064
- documented so a future session doesn't relitigate it without new
1065
- evidence.
1066
- 2. **Memory panel becomes a context-dump UX that violates "quiet retrieval"
1067
- (Design Principle 4).** Mitigation: provenance chips are collapsed by
1068
- default, expand-on-click only; no chat-thread content is duplicated into
1069
- the panel automatically.
1070
- 3. **Adapter routing table adds setup friction before first chat**, cutting
1071
- against "install to chat under a minute." Mitigation: the routing table
1072
- is progressive disclosure behind settings; the composer chip default
1073
- ("general-purpose", no adapter) requires zero configuration.
1074
- 4. **The IA collapse (Developer toggle) hides a capability an existing user
1075
- relies on daily** (e.g. someone who lives in Quantize). Mitigation: the
1076
- toggle persists as a local preference and is one click, not a
1077
- re-onboarding flow; existing deep-links (`#/quantize` etc.) keep working
1078
- unchanged.
1079
- 5. **Perf-strip lane badge becomes stale or wrong** during a mode
1080
- transition (e.g. spec-decode falling back to serial mid-session).
1081
- Mitigation: badge is server-driven per-turn from the same instrumentation
1082
- already used for tok/s, not client-inferred; treat any mismatch as a P0
1083
- bug since the entire differentiator rests on it being trustworthy.
1084
- 6. **The tool-approval "always allow" persistence reintroduces the exact
1085
- fragility documented against Claude Desktop** (resets on update). Mitigation:
1086
- store approvals in a versioned, file-backed config keyed by stable tool
1087
- identity from day one, with an explicit migration test in CI across a
1088
- simulated version bump.
1089
- 7. **Memory git-diff view surfaces something the user finds unsettling**
1090
- (seeing exactly how much was inferred about them). Mitigation: frame the
1091
- diff view as user-initiated inspection, not a push notification; pair
1092
- with the existing pause/reset actions (already a good ChatGPT/Claude
1093
- pattern) so the "off switch" is never more than one click from the
1094
- panel that shows the most.
1095
- 8. **(Added 2026-07-06) The superset doctrine balloons scope until nothing
1096
- ships.** Matching 116 matrix rows is a quarter-plus of work; the
1097
- original wedge-only framing failed by conceding too much, and the
1098
- correction can fail by attempting everything at once. Mitigation: the
1099
- phases stay independently shippable and strictly ordered
1100
- (fundamentals → structure → table-stakes sweep → trust/speed →
1101
- stretch); within Phase 3 the sweep is ordered by "what would a daily
1102
- competitor user miss first"; the matrix's shipped/scheduled/SKIP states
1103
- are re-audited at each phase boundary rather than renegotiated
1104
- mid-phase.
1105
- 9. **(Added 2026-07-06) The untested, untyped frontend erodes as it
1106
- grows.** Phase 0's bug class (contract drift between inline JS and
1107
- typed TS) recurs with every new surface until the module split lands.
1108
- Mitigation: the split + frontend regression harness are Phase 2's
1109
- FIRST work item (§7), before the memory panel and routing table add
1110
- the next thousand lines; until then, every new WS frame or API shape
1111
- consumed by `app.html` gets a line in `tests/pi-web.test.ts` proving
1112
- the producing side's contract.
636
+ | Capability | Verdict | State |
637
+ |---|---|---|
638
+ | Live tok/s + TTFT | MATCH | shipped |
639
+ | Context-window meter | MATCH | shipped |
640
+ | Serving-lane badge in the live chat | BEAT | shipped |
641
+ | Prompt-cache-hit telemetry per turn | BEAT | open |
642
+ | In-app benchmark tab | MATCH | open (harness exists: `scripts/bench-serve.ts`) |
643
+ | Compare / split view (base vs adapter, parity vs fast path) | BEAT | open |
644
+ | Concurrency / batch-slot indicator | MATCH | open |
645
+ | `/fit` surfaced beyond Status | BEAT | shipped (picker, Hub) |
646
+ | Auto-sized context to available memory | BEAT | open (UI); admission clamps server-side |
647
+ | Provable zero-cloud-calls indicator | BEAT | open |
648
+
649
+ **Axis 10 — Polish**
650
+
651
+ | Capability | Verdict | State |
652
+ |---|---|---|
653
+ | Dark + light, `prefers-color-scheme` | MATCH | shipped |
654
+ | `prefers-reduced-motion` | MATCH | shipped |
655
+ | Shortcut sheet (Cmd/Ctrl+/) | MATCH | shipped |
656
+ | Command palette (Cmd/Ctrl+K) | MATCH | shipped |
657
+ | Responsive to 375 px with a drawer | MATCH | shipped |
658
+ | PWA installability | MATCH | shipped |
659
+ | `aria-live` thread, focus traps, Escape | MATCH | shipped |
660
+ | Developer progressive disclosure | MATCH | shipped |
661
+ | Server-driven personalized onboarding chips | BEAT | shipped |
662
+ | Chat export (Markdown / JSON) | MATCH | shipped |
663
+ | Shareable links | SKIP | scope decision (single-user, local) |
664
+ | Full-text search across message bodies | BEAT | shipped |
665
+ | Session pins / folders | MATCH | open |
666
+ | Clean quit / no zombie process in a desktop shell | MATCH | n/a until a shell exists |
667
+
668
+ **Axis 11 — Sessions**
669
+
670
+ | Capability | Verdict | State |
671
+ |---|---|---|
672
+ | New / list / open / fork / delete | MATCH | shipped |
673
+ | Session search box | MATCH | shipped |
674
+ | Per-message branch-from-here | MATCH | shipped (edit / regenerate siblings) |
675
+ | Projects (scoped knowledge + memory + adapter) | BEAT | open (persona bundles) |
676
+ | Temporary / incognito chat, provably not on disk | BEAT | open |
677
+ | Multi-adapter compare | BEAT | open |
678
+ | Portable JSON export / import | MATCH | export shipped; import open |
679
+
680
+ **Axis 12 — App-aware assistant**
681
+
682
+ | Capability | Verdict | State |
683
+ |---|---|---|
684
+ | Agent sees the live UI (structured snapshot) | BEAT | shipped |
685
+ | Agent navigates the app | BEAT | shipped |
686
+ | Agent spotlights a control | BEAT | shipped |
687
+ | Approval-gated agent UI actions (`ui_act`) | BEAT (stretch) | open |
688
+
689
+ **The beats** (ranked by visible wow × structural defensibility): lane
690
+ badge in live chat · memory git-diff view · per-turn provenance chip ·
691
+ legible, stackable adapter routing · base-vs-adapter compare view · model
692
+ swap that preserves the session · prompt-cache-hit telemetry · provably
693
+ local temporary chat · diff preview before approval · full-text body
694
+ search · the app-aware assistant. The first four and the last two are
695
+ shipped; the rest are §5.