mlx-bun 0.2.0 → 0.4.0

package/docs/README.md

@@ -0,0 +1,94 @@
+# docs/ map (generated — do not edit; `bun scripts/check-hygiene.ts --write-docs-map`)
+
+## docs/archive/
+
+Frozen history — read-only, never extended in place; recover raw data via git history.
+
+- [hlg-sampling.md](../docs/archive/hlg-sampling.md) — HLG sampling — a piecewise tone curve on the logits
+- [investigations/audio-input-plan.md](../docs/archive/investigations/audio-input-plan.md) — Audio input — design & implementation plan (gemma-4 audio tower)
+- [investigations/chunk-finetune-experiment.md](../docs/archive/investigations/chunk-finetune-experiment.md) — Fine-tuning experiment: chunking task (2026-06-16)
+- [investigations/colibri-glm52-port.md](../docs/archive/investigations/colibri-glm52-port.md) — GLM-5.2 on a 32 GB Mac — native MLX port of the Colibri hierarchy
+- [investigations/colibri-oracle-pin.md](../docs/archive/investigations/colibri-oracle-pin.md) — Colibri oracle pin for the native GLM-5.2 port
+- [investigations/curve-bisector-routing.md](../docs/archive/investigations/curve-bisector-routing.md) — Curve sampler: bisectors, routing, and the falsification
+- [investigations/decode-roofline-lookagain.md](../docs/archive/investigations/decode-roofline-lookagain.md) — Decode perf "look again" — adversarial re-exam of the M=1 floor claim (2026-07-01)
+- [investigations/diffusion-gemma-port.md](../docs/archive/investigations/diffusion-gemma-port.md) — DiffusionGemma port — plan & parity strategy
+- [investigations/dspark-handoff.md](../docs/archive/investigations/dspark-handoff.md) — DSpark — handoff
+- [investigations/expert-offload-single-user-moe.md](../docs/archive/investigations/expert-offload-single-user-moe.md) — Expert offload for single-user local MoE — findings + design sketch
+- [investigations/faithful-l1-consolidation.md](../docs/archive/investigations/faithful-l1-consolidation.md) — Faithful → L1: consolidating the "match mlx-lm" path
+- [investigations/hlg-sampling-investigation.md](../docs/archive/investigations/hlg-sampling-investigation.md) — Investigation: HLG sampling — a piecewise tone curve on the logits
+- [investigations/indexshare-performance-spike.md](../docs/archive/investigations/indexshare-performance-spike.md) — IndexShare / IndexCache performance spike
+- [investigations/kernel-perf-review-2026-07.md](../docs/archive/investigations/kernel-perf-review-2026-07.md) — Kernel/perf deep review — 2026-07-01 (condensed, tracked record)
+- [investigations/lab-build-journal.md](../docs/archive/investigations/lab-build-journal.md) — Build journal — mlx-bun Lab (web UI + native quantize/train/dataset + pi web chat)
+- [investigations/minicpm5-decode-megakernel.md](../docs/archive/investigations/minicpm5-decode-megakernel.md) — MiniCPM5 Decode Megakernel — end-to-end build plan (handoff)
+- [investigations/openjarvis-fit.md](../docs/archive/investigations/openjarvis-fit.md) — OpenJarvis research and fit with mlx-bun
+- [investigations/openwiki-evaluation.md](../docs/archive/investigations/openwiki-evaluation.md) — OpenWiki — evaluation & fit analysis
+- [investigations/optimization_plan.md](../docs/archive/investigations/optimization_plan.md) — mlx-bun: gemma4.ts decompose & optimize — action plan
+- [investigations/orpo-base-uf-experiment-and-directions.md](../docs/archive/investigations/orpo-base-uf-experiment-and-directions.md) — ORPO from-base on UltraFeedback (mixed-5bpw) — results + research directions
+- [investigations/orpo-flash-cce-pin-leak.md](../docs/archive/investigations/orpo-flash-cce-pin-leak.md) — flash-CCE host-buffer pin leak — post-mortem + L3 verification approach
+- [investigations/orpo-uf-testing-handoff.md](../docs/archive/investigations/orpo-uf-testing-handoff.md) — ORPO-on-UF — results + testing-phase handoff
+- [investigations/pareto-specialized-runtime-findings.md](../docs/archive/investigations/pareto-specialized-runtime-findings.md) — Pareto-specialized runtime findings
+- [investigations/pi-builtin-investigation.md](../docs/archive/investigations/pi-builtin-investigation.md) — Investigation: pi as a "built-in" terminal for mlx-bun
+- [investigations/pre-colibri-stabilization.md](../docs/archive/investigations/pre-colibri-stabilization.md) — Pre-Colibri stabilization program
+- [investigations/qwen38-27b-campaign-log.md](../docs/archive/investigations/qwen38-27b-campaign-log.md) — Qwen3.8-27B optimization campaign — execution log (2026-09-04 → 2026-09-08)
+- [investigations/segmented-backward-handoff.md](../docs/archive/investigations/segmented-backward-handoff.md) — Handoff — segmented backward training (2026-06-16)
+- [investigations/starter-model-port-handoff.md](../docs/archive/investigations/starter-model-port-handoff.md) — Handoff: the sub-GB starter model port (publishing gate 1)
+- [investigations/steel-flash-cce-handoff.md](../docs/archive/investigations/steel-flash-cce-handoff.md) — Steel flash-CCE — handoff
+- [investigations/the-dreaming-handoff.md](../docs/archive/investigations/the-dreaming-handoff.md) — THE DREAMING — HANDOFF (2026-06-28)
+- [investigations/trainer-validation-experiment.md](../docs/archive/investigations/trainer-validation-experiment.md) — Trainer validation experiment
+- [planning/chat-ui-vision-memory-first.md](../docs/archive/planning/chat-ui-vision-memory-first.md) — Chat UI Vision: The Model That Knows You
+- [planning/chat-ui-vision.md](../docs/archive/planning/chat-ui-vision.md) — The mlx-bun chat surface: a vision
+- [planning/chunk-task-roadmap.md](../docs/archive/planning/chunk-task-roadmap.md) — Chunk-task quality roadmap (2026-06-17)
+- [planning/curve-sampler-research-plan.md](../docs/archive/planning/curve-sampler-research-plan.md) — Curve sampler — preregistered research plan (distinctness track)
+- [planning/journal.md](../docs/archive/planning/journal.md) — Engineering Journal
+- [planning/memory-docs-and-dag-plan.md](../docs/archive/planning/memory-docs-and-dag-plan.md) — Memory docs organization + DAG-view refresh — review & plan
+- [planning/pi-integration-review.md](../docs/archive/planning/pi-integration-review.md) — pi integration review — internal embeds + `harness pi` (2026-07-01)
+- [planning/release-notes-v0.0.10.md](../docs/archive/planning/release-notes-v0.0.10.md) — mlx-bun v0.0.10 — batching parity, SSD KV cache, sharper serving
+- [planning/release-notes-v0.0.11.md](../docs/archive/planning/release-notes-v0.0.11.md) — v0.0.11 release notes
+- [planning/release-notes-v0.0.12.md](../docs/archive/planning/release-notes-v0.0.12.md) — v0.0.12 release notes
+- [planning/release-notes-v0.0.13.md](../docs/archive/planning/release-notes-v0.0.13.md) — v0.0.13 release notes
+- [planning/release-notes-v0.0.9.md](../docs/archive/planning/release-notes-v0.0.9.md) — v0.0.9 release notes (DRAFT — Josh edits + publishes)
+- [planning/release-notes-v0.2.0.md](../docs/archive/planning/release-notes-v0.2.0.md) — mlx-bun v0.2.0
+- [planning/release-notes-v0.3.0.md](../docs/archive/planning/release-notes-v0.3.0.md) — mlx-bun v0.3.0
+- [planning/web-ui-pass-plan.md](../docs/archive/planning/web-ui-pass-plan.md) — Web UI pass — ranked fix list (2026-07-01 audit)
+- [planning/website-readme-pass-plan.md](../docs/archive/planning/website-readme-pass-plan.md) — Website + README pass — audit findings and fix plan (2026-07-01)
+
+## docs/design/
+
+Active engineering design docs — one per topic; status lives in front matter + PLAN.md.
+
+- [batching.md](../docs/design/batching.md) — Batching — continuous scheduling for `--batch N`
+- [decode-speed-program.md](../docs/design/decode-speed-program.md) — Inference performance program
+- [dreaming-nightly-pipeline.md](../docs/design/dreaming-nightly-pipeline.md) — The Dreaming — memory write path + synthesis (canonical design)
+- [generic-model-support.md](../docs/design/generic-model-support.md) — Model support — run anything mlx-lm runs, and the ports we target
+- [kv-cache.md](../docs/design/kv-cache.md) — KV cache — residency and layout
+- [orpo-training.md](../docs/design/orpo-training.md) — Training — design (ORPO, flash-CCE head, prefix sharing, segmented backward)
+- [speculative-decoding.md](../docs/design/speculative-decoding.md) — Speculative decoding — one verifier, every draft source
+- [turboquant.md](../docs/design/turboquant.md) — TurboQuant — rotation-based quantization (KV leg landed, weights leg open)
+- [unified-engine-frontier-plan.md](../docs/design/unified-engine-frontier-plan.md) — Engine architecture — four layers, per-scheme oracles, one request path
+- [web-chat-redesign.md](../docs/design/web-chat-redesign.md) — Web chat — the built-in chat surface (canonical design)
+
+## docs/planning/
+
+Living product/vision docs (USING side).
+
+- [PRODUCT_ROADMAP.md](../docs/planning/PRODUCT_ROADMAP.md) — PRODUCT ROADMAP
+- [ResearchTopics.md](../docs/planning/ResearchTopics.md) — Research Topics
+- [release-notes-v0.4.0.md](../docs/planning/release-notes-v0.4.0.md) — mlx-bun v0.4.0
+- [video-series.md](../docs/planning/video-series.md) — mlx-bun video series
+
+## docs/reference/
+
+User-facing reference — the ONLY home for flags, routes, models, numbers, and environment facts.
+
+- [benchmarks.md](../docs/reference/benchmarks.md) — mlx-bun benchmark results (curated)
+- [cli.md](../docs/reference/cli.md) — CLI reference
+- [distribution.md](../docs/reference/distribution.md) — Distributing mlx-bun — build, sign, notarize, publish
+- [environment.md](../docs/reference/environment.md) — Reference environment and platform facts
+- [glossary.md](../docs/reference/glossary.md) — mlx-bun serving
+- [library-api.md](../docs/reference/library-api.md) — Library API
+- [memory.md](../docs/reference/memory.md) — Memory — the personal wiki
+- [models.md](../docs/reference/models.md) — Models: supported roster + management (get / scan / ls / gc)
+- [server-api.md](../docs/reference/server-api.md) — Server API
+- [server-config.md](../docs/reference/server-config.md) — Server configuration
+- [training.md](../docs/reference/training.md) — Training / fine-tuning reference
+- [troubleshooting.md](../docs/reference/troubleshooting.md) — Troubleshooting

package/docs/{design → archive/investigations}/audio-input-plan.md

@@ -210,7 +210,7 @@ All under `optiq/vlm/_mlxvlm/models/gemma4/` in the oracle venv:
 ### 3.4 Parity strategy (tree doctrine)
 
 Oracle = optiq internal gemma4 model in the pinned venv, driven by
-`scripts/regen-audio-goldens.py` (modeled on their calibration loader;
+`regen-audio-goldens.py (deleted; git history)` (modeled on their calibration loader;
 remember `enable_thinking` pinning for any templated prompt). Tiers follow
 the e4b vision precedent exactly:
 
@@ -238,7 +238,7 @@ Expectation (to be *measured*, not asserted): the tower is tiny —
 ≤750 mel frames → ~≤188 post-subsample positions × 12 layers × hidden
 1024 bf16 — so added TTFT should be dominated by CPU feature extraction
 plus one extra graph eval; sidecar RSS delta is the 752 bf16 tensors.
-Every number that gets quoted comes from preflight-gated `benchmark.sh`
+Every number that gets quoted comes from preflight-gated `scripts/bench-serve.ts all`
 cells (tower ms, TTFT delta vs text-only, peak RSS delta) and lands in
 `benchmarks/RESULTS.md`.
 
@@ -247,7 +247,7 @@ cells (tower ms, TTFT delta vs text-only, peak RSS delta) and lands in
 **Phase A0 — groundwork**
 - [ ] Bind `mlx_conv2d` (full header signature; CPU/GPU sanity test vs
       oracle conv output)
-- [ ] `scripts/regen-audio-goldens.py` + fixtures; dump mel golden,
+- [ ] `regen-audio-goldens.py (deleted; git history)` + fixtures; dump mel golden,
       tower-output golden, spliced ids, greedy ids
 - [ ] Resolve open questions §3.3 (write the answers into this doc)
 - Exit: goldens on disk; masking/zeroing semantics documented.
@@ -332,7 +332,7 @@ cells (tower ms, TTFT delta vs text-only, peak RSS delta) and lands in
       docs-map hygiene green. ✅ 2026-07-07
 
 **Phase A5 — bench + coverage**
-- [ ] benchmark.sh cells → RESULTS.md (tower ms, TTFT delta, RSS delta)
+- [ ] scripts/bench-serve.ts all cells → RESULTS.md (tower ms, TTFT delta, RSS delta)
 - [ ] 12B audio cell: rebuild its sidecar via optiq `build_vision_sidecar`
       (selective download), own goldens — per-model doctrine, every
       (model, quant) cell gets its own validation or an explicit deferral

package/docs/{design → archive/investigations}/diffusion-gemma-port.md

@@ -120,7 +120,7 @@ tied 4-bit head → fp32 softcap** — is **BIT-EXACT vs the optiq golden**: arg
 maxDiff 0.0, relRMSE 0.0, meanKL 0.0 (`tests/diffusion-parity.test.ts`,
 `MLX_BUN_TEST_DIFFUSION=1`). Per-stage sub-gates (enc_hidden / dec_hidden / presoftcap /
 per-layer / layer-0 attn·dense·MoE) all 0.0 (`scripts/experiments/diffusion-{stage,layer,l0}-diag.ts`,
-golden dumps in `scripts/gen-diffusion-golden.py`).
+golden dumps in `scripts/oracle/gen-diffusion-golden.py`).
 - **The one bug found + fixed (per-model parity gotcha):** the Router pre-projection
   norm must be the literal two-step `rms_norm(x, None, eps) * scale * hidden**-0.5`,
   **NOT** the gemma4-style fold `rms_norm(x, scale*hidden**-0.5)`. The fold changes the
@@ -152,7 +152,7 @@ reuse, linear temp schedule, the un-mask loop, BOTH samplers (confidence-thresho
 OptiQ public default, entropy-bound = engine default), self-conditioning feedback,
 stability/EOS, canvas block loop. **Token-for-token parity vs the optiq engine at
 temp 0 on a fixed seed for BOTH samplers** (`tests/diffusion-gen-parity.test.ts`,
-`MLX_BUN_TEST_DIFFUSION=1`; golden `scripts/gen-diffusion-gen-golden.py`):
+`MLX_BUN_TEST_DIFFUSION=1`; golden `scripts/oracle/gen-diffusion-gen-golden.py`):
 confidence 17 tok / 7 steps, entropy 15 tok / 48 steps — both `.tokens`/`.steps`/
 `.finishReason` identical.
 - **RNG parity (the crux) SOLVED:** bound `mlx_random_randint` (+`mlx_random_seed`,
@@ -241,7 +241,7 @@ output matches optiq `generate`; HTTP test green; no regression to AR models.
 
 ### D4 — Performance (L3, optional, measured) `[ ]`
 `confidence-threshold` is already the fast path (optiq: ~58 tok/s code / ~9
-prose). Measure mlx-bun tok/s with `benchmark.sh`; fuse kernels only if a number
+prose). Measure mlx-bun tok/s with `scripts/bench-serve.ts all`; fuse kernels only if a number
 says so. KV holds only the prompt (no per-step growth → no KV-quant needed).
 **Exit:** quotable tok/s within range of optiq on this machine; any fused kernel
 gated by KL + the 6-task quality suite (no bit-exact ancestor for novel fusions).

package/docs/{design → archive/investigations}/faithful-l1-consolidation.md

@@ -275,13 +275,13 @@ kernels, selected (or not) by the individual flags.
 ### Phase 4 — Docs + benchmark ✅ LANDED 2026-07-04 (numbers Josh-gated)
 
 Docs updated: `features-matrix.md` (new `--compiled-activations` + `--fused-gelu`
-rows), `server-config.md` + `cli.md` (Phase 2). Reworked `scripts/bench-faithful-matrix.ts`
-+ `benchmark.sh` comparison 0 for the new model — since faithful is now the default,
+rows), `server-config.md` + `cli.md` (Phase 2). Reworked `scripts/bench-levers.ts faithful-matrix`
++ `scripts/bench-serve.ts all` comparison 0 for the new model — since faithful is now the default,
 the matrix measures what **removing** each faithful kernel costs and what the L3
 custom kernels add on top (`− compiled-decode`, `− compiled activations`,
 `+ custom fused-gelu`), all vs mlx-lm; tsc 0. **The actual numbers are Josh-gated**
-(clean/rebooted machine — `./benchmark.sh --redo`, or
-`bun scripts/bench-faithful-matrix.ts`); loaded-machine absolutes are noise. Promote
+(clean/rebooted machine — `bun scripts/bench-serve.ts all --redo`, or
+`bun scripts/bench-levers.ts faithful-matrix`); loaded-machine absolutes are noise. Promote
 quotable rows to `benchmarks/RESULTS.md` after the clean run.
 
 ## Status: consolidation complete

package/docs/{design → archive/investigations}/minicpm5-decode-megakernel.md

@@ -108,7 +108,7 @@ a 1-token parity probe before trusting it (Phase 2 gate catches mismatch).
 **Validation goldens (ALREADY EXIST, no regen needed):**
 - L1: `goldens/minicpm5-parity.json` (prompt_ids `[0,608,4894,304,6918,357]`,
   greedy_ids[100], vocab 130560) + `goldens/minicpm5-logits-step{0..99}.bin`
-  (f32 last-position logits). Produced by `scripts/regen-minicpm5-goldens.ts`.
+  (f32 last-position logits). Produced by `scripts/regen.ts minicpm5`.
 - L2: `goldens/minicpm5-kv-parity.json` + `…-kv-logits-step*.bin`.
 
 **Reference forward to copy (the source of truth for correctness):**
@@ -132,7 +132,7 @@ All default-OFF, `bun test tests/minicpm5-*.test.ts` green. From this session:
   @ M=8–256, ~1 ULP correct.** This is a working prototype of one megakernel STAGE
   and the dequant/GEMV patterns to copy. (Note: it currently corrupts when CHAINED
   in the lazy graph — that's the whole reason for the megakernel; the kernel math
-  itself is correct, proven by `scripts/experiments/fused-mlp-check.ts`.)
+  itself is correct, proven by `fused-mlp-check.ts (deleted; git history)`.)
 - `src/model/steel-linear-kernel.ts` — `steelLinear`: plain quantized linear as our
   dispatch (q/k/v/o/down), ~1 ULP vs `ops.quantizedMatmul`. The projection GEMV
   pattern.
@@ -232,7 +232,7 @@ megapack-check.ts`.) This catches layout bugs before any kernel.
 
 ### Phase 2 — Single-threadgroup correct megakernel (G=1)
 **Files:** `src/model/megakernel-kernel.ts` (the `MetalKernel` + dispatch),
-`scripts/gen-minicpm5-megakernel.ts` (emits the per-layer-bits-baked SOURCE into
+`gen-minicpm5-megakernel.ts (deleted; git history)` (emits the per-layer-bits-baked SOURCE into
 `src/model/generated/minicpm5-megakernel.ts`).
 **Design:** ONE threadgroup (`G=1`, `TGN=256` threads). `hidden[H]` in threadgroup
 memory (6 KB). Loop layers; each stage uses `threadgroup_barrier`. GEMVs: the 256
@@ -246,13 +246,13 @@ down). KV read/write from `KBUF`/`VBUF`. Final: finalNorm + lmHead GEMV → logi
 **Transcribe** `src/model/minicpm5.ts` op-for-op; match dtypes: accumulate in f32,
 round to bf16 at each op boundary to track the reference (`(T)` casts, as `fusedMlp`
 does). RoPE: match `ops.rope` exactly (verify with a single-layer probe).
-**Gate (THE correctness gate):** `scripts/experiments/megakernel-teacherforced.ts`
+**Gate (THE correctness gate):** `megakernel-teacherforced.ts (deleted 2026-08-23; git history)`
 (copy `swiglu-teacherforced.ts`): drive 100 decode steps via the megakernel,
 teacher-forced on `goldens/minicpm5-parity.json`, KV cache in `KBUF`/`VBUF`. Require
 **≥98/100 argmax agreement, deterministic across 3 runs, no NaN.** If an early step
 mismatches, bisect by stage: add a debug output for `hidden` after layer L (env
 `MLX_BUN_MEGA_DUMP_LAYER=L`) and compare to the reference model's per-layer hidden
-(capture via the wrapper trick in `scripts/experiments/swiglu-layerN.ts`).
+(capture via the wrapper trick in `swiglu-layerN.ts (deleted; git history)`).
 
 ### Phase 3 — Persistent multi-threadgroup (G>1) for SPEED
 **Do:** generalize Phase 2 to `G` threadgroups. Move `hidden`,`xn`,`hn`,`q/k/v/attn/
@@ -332,7 +332,7 @@ Swap `KBUF/VBUF` for the per-layer quantized-KV format. **Facts gathered:**
   q = min(round((w-bias)/scale), n_bins)     // dequant: scale*q+bias
   ```
 **Implementation steps:**
-1. ✅ **DONE — quantize formula validated** (`scripts/experiments/kv-quant-formula-check.ts`):
+1. ✅ **DONE — quantize formula validated** (`kv-quant-formula-check.ts (deleted 2026-08-23; git history)`):
    the affine_quantize transcription matches `ops.quantize` to bf16 ULP (scale/bias diff
    1.8e-3 @4-bit, 6e-5 @8-bit; dequant residual = expected bf16-storage rounding). The
    riskiest piece is de-risked. Compute scale/bias in f32, store bf16 (as mlx does).
@@ -411,7 +411,7 @@ Guard: bf16 KV (L1) only, no adapters, no vision. Add `--megakernel` CLI flag.
 of the old `tests/perf-kernel-oracle.test.ts` — deleted 2026-07-05 with its kernel,
 unified-engine-frontier-plan.md §6; the envelope-gate methodology stands on its
 own); paired decode tok/s A/B logged. Update
-`STATUS.md` + `benchmarks/RESULTS.md` (preflight-gated `benchmark.sh` for the
+`STATUS.md` + `benchmarks/RESULTS.md` (preflight-gated `scripts/bench-serve.ts all` for the
 quotable number — `[[dirty-machine-numbers-are-garbage]]`).
 
 ---
@@ -462,12 +462,12 @@ so its 1-ULP/op drift is within contract — bit-exact is a focused REBUILD, not
 
 ```
 src/model/megakernel-pack.ts            # Phase 1: weight repack + layout table
-scripts/gen-minicpm5-megakernel.ts      # Phase 2: emit per-layer-baked kernel SOURCE
+gen-minicpm5-megakernel.ts (deleted; git history)      # Phase 2: emit per-layer-baked kernel SOURCE
 src/model/generated/minicpm5-megakernel.ts   # (generated) the kernel SOURCE + bits
 src/model/megakernel-kernel.ts          # Phase 2/3: MetalKernel + decodeMegakernel()
-scripts/experiments/megapack-check.ts        # Phase 1 gate
-scripts/experiments/megakernel-teacherforced.ts  # Phase 2/3 correctness gate
-scripts/experiments/megakernel-perf.ts       # Phase 3/5 paired tok/s A/B
+megapack-check.ts (deleted 2026-08-23; git history)        # Phase 1 gate
+megakernel-teacherforced.ts (deleted 2026-08-23; git history)  # Phase 2/3 correctness gate
+megakernel-perf.ts (deleted 2026-08-23; git history)       # Phase 3/5 paired tok/s A/B
 tests/minicpm5-megakernel.test.ts       # Phase 5 CI gate (env-gated)
 ```
 Modify: `src/model/minicpm5.ts` (add `decodeStep` megakernel branch),

package/docs/archive/investigations/openjarvis-fit.md

@@ -0,0 +1,198 @@
+# OpenJarvis research and fit with mlx-bun
+
+Inspected September 8, 2026. OpenJarvis source revision
+[`e25e77d36ee487aa456978e7a7f19a7f6d712260`](https://github.com/open-jarvis/OpenJarvis/tree/e25e77d36ee487aa456978e7a7f19a7f6d712260).
+mlx-bun revision `673b43f39b88d8f6438e625aaaceca00851d0fbf`.
+This is a source/documentation review, not a reproduced benchmark or a verified
+live integration. No servers, model downloads, inference, or training ran.
+
+OpenJarvis is useful both as a possible consumer of mlx-bun and as prior work
+for the local adaptation research. Its most useful architectural idea is a
+versioned configuration that can be evaluated and changed across model,
+agent, tools, memory, and runtime settings. mlx-bun already supplies several
+of those components. The remaining opportunity is to connect them through
+replayable task evidence and controlled updates.
+
+## What the diagram means for this project
+
+The [architecture documentation][architecture] separates five configurable
+components. Interfaces and hardware surround them. The table maps those
+roles to documented mlx-bun capabilities; it does not claim identical scope.
+
+| OpenJarvis component | Existing mlx-bun contribution | Work above or around the engine |
+|---|---|---|
+| Intelligence | Supported model graphs, model discovery, quantization and adapters | Task-aware model/adapter selection, with measured loading and switching costs |
+| Engine | Native MLX inference, HTTP serving, TypeScript library, caching and batching | An OpenJarvis compatibility check and an adapter for richer runtime measurements |
+| Agents | Built-in Pi and integration with an external Pi | Compare task loops, including bounded RLM, outside numerical inference code |
+| Tools and memory | Pi tools, embeddings, personal Markdown wiki and synthesis | Common tool contracts, evidence provenance and replayable memory snapshots |
+| Learning | LoRA SFT/DPO/ORPO training and adapter serving | Verified data collection, candidate evaluation, retention tests and promotion |
+| User interfaces | CLI and browser application | Messaging/desktop integrations only when a concrete workload needs them |
+
+Local sources: [CLI](../../reference/cli.md),
+[library API](../../reference/library-api.md),
+[server API](../../reference/server-api.md),
+[memory](../../reference/memory.md), [training](../../reference/training.md).
+The Dreaming is memory synthesis, not weight training. Its current
+[`runPipeline`](../../../src/memory/pipeline.ts) assumes conversations are
+already ingested; closing that ingestion gap remains existing work.
+Serving a model does not imply its architecture supports every training path.
+
+## RLM implementation and its limits
+
+OpenJarvis includes a concrete [`RLMAgent`][rlm] with a persistent Python
+[`RLMRepl`][repl]. It installs external context in a variable, executes
+model-written code, exposes `llm_query` and `llm_batch`, and can return an
+answer through `FINAL`, `FINAL_VAR`, or the `answer` dictionary.
+
+This is a useful reference for the bounded external-context experiment in
+the existing `harness_research/RESEARCH_PROGRAM.md`. It belongs in the task
+execution loop. A separate optimizer could improve its prompt, decomposition
+rules or tool configuration over time.
+
+The inspected implementation has several details an experiment must account for:
+
+- `llm_batch` runs subqueries sequentially. Its name does not imply engine
+  batching or parallel execution.
+- Subqueries call `engine.generate` directly and can resolve one round of
+  returned tool calls. They do not create child REPL agents. This path should
+  not be described as arbitrary-depth recursive RLM execution.
+- `AgentResult.metadata` accumulates root-call usage. `_make_sub_query`
+  returns text and does not add leaf or follow-up usage to that accumulator.
+  Measure every backend request independently for whole-task accounting.
+- Root turns and per-call output tokens have limits, but these methods do not
+  impose one shared subcall/token/deadline budget. A single REPL program can
+  request many leaves. Enforce shared admission limits in a request broker.
+- The REPL checks prohibited substrings and then invokes in-process Python
+  `exec`. Output truncation happens after execution. These mechanisms are not
+  process isolation or a hard execution/memory deadline. The project has
+  separate sandbox facilities; a benchmark must explicitly select isolation
+  and keep authoritative evaluation data outside the candidate environment.
+- `_resolve_context` reads `AgentContext.metadata["context"]` or memory
+  results, while the normal user input still enters the root prompt.
+  Put the large corpus in the external-context field, not the user question.
+  Existing conversation messages are also retained by
+  [`BaseAgent._build_messages`][base-agent].
+
+The appropriate comparison remains retrieval versus an external-context
+REPL without subcalls versus bounded depth-1 RLM, on dense document
+aggregation. Small lookup tasks establish connectivity but give weak
+evidence about RLM's value. On a serial Mac GPU, extra model calls can erase
+the benefit of smaller contexts; measure complete tasks.
+
+## Self-improvement is several separate implementations
+
+| Mechanism | What the inspected source establishes |
+|---|---|
+| Learned routing | [`LearnedRouterPolicy`][routing] maps coarse query classes to models using trace outcomes and feedback. This is distinct from training language-model weights. |
+| Configuration search | [`SpecSearchOrchestrator`][search] connects diagnosis, typed edit planning, execution, evaluation and recording. Runtime dependencies must be supplied. |
+| Candidate gate | [`BenchmarkGate`][gate] compares overall score and per-cluster regressions using a versioned benchmark and repeatable subsample seed. |
+| Weight training | [`SFTTrainer`][sft] delegates LoRA training to a separate trainer. [`GRPOTrainer`][grpo] contains a Torch/Transformers training loop with policy and reference models. Neither was executed in this review. |
+| Weight updates inside spec search | [`LoraStubApplier`][lora-stub] rejects `LORA_FINETUNE` and raises `NotImplementedError` if applied. Separate trainers do not establish an integrated spec-search training cycle. |
+
+The [spec-search guide][search-guide] describes a teacher diagnosing local
+student failures and proposing changes. Its quickstart uses fake
+teacher/student/judge components. It illustrates the orchestration, not
+measured improvement by a deployed learning system.
+
+There is documentation drift worth preserving in a reproduction. The guide's
+gate predicate requires improvement on the targeted failure cluster. The
+inspected `BenchmarkGate.evaluate` instead requires improvement in overall
+score and rejects excessive per-cluster regressions; it receives no target
+cluster argument. Its constructor defaults to a 0.05 regression allowance,
+while the guide's configuration uses 0.01. `from_config` forwards the supplied
+value. Pin the actual configuration and predicate instead of assuming that
+the prose, constructor defaults, and configured execution are interchangeable.
+
+The [learning architecture page][learning] also describes older router names
+and a GRPO router stub. Current learned routing is consolidated in
+`learning/routing/learned_router.py`, while the separate intelligence GRPO
+trainer has implementation code. A blanket claim that "GRPO is a stub"
+would therefore be misleading.
+
+The [paper][paper] is close prior work for the research program: it studies
+typed specifications and teacher-guided edits across several components,
+with local inference after search. Its reported experiments span multiple
+models and hardware classes, including Macs and larger workstations.
+They do not establish the same gains on our M1 Max or M4 Pro, or prove that
+the checked-out code reproduces every reported experiment. Its acknowledged
+limitations include judge bias, statistical precision and search convergence.
+We should reproduce a bounded comparison before adopting performance claims.
+
+## Integration path
+
+The current [engine registry][engines] already registers `MLXEngine` through
+the shared [`_OpenAICompatibleEngine`][http-engine]. That class accepts a
+host URL, probes `/v1/models`, and calls `/v1/chat/completions`. The generic
+`OpenAICompatEngine` offers the same interface for explicit endpoints.
+This makes using mlx-bun as the inference backend a plausible small
+integration, without importing OpenJarvis into the inference engine.
+
+For a future smoke test, construct the generic engine with mlx-bun's origin
+URL, without `/v1`, and choose the exact model ID from `list_models()`.
+Check plain completion, tool-call round trips, streaming completion/error
+handling, cancellation, reasoning output, model identity and usage against
+raw HTTP responses. This review did not perform that test.
+
+Two adapter behaviors need special attention:
+
+- Synchronous generation retries a tool-bearing HTTP 400 without `tools`
+  and `tool_choice`. A successful retry does not establish tool compatibility.
+- It takes the maximum of reported prompt tokens and an estimate, and labels
+  reported prompt tokens as `prompt_tokens_evaluated`. Those fields cannot
+  establish actual prefill work or cache reuse. Retain raw server measurements.
+
+Use HTTP for the first comparison. A direct TypeScript implementation can
+later use mlx-bun's library if measured requirements justify it.
+
+## Recommended direction
+
+Keep mlx-bun responsible for inference correctness, resource management,
+training execution and adapter lifecycle. Keep experimental task loops and
+the process that changes them in `harness_research` initially. Use the same
+backend for Pi, a controlled loop and OpenJarvis comparisons.
+
+The next useful steps extend the existing research program:
+
+1. Complete the already proposed repeated-failure-feedback experiment in
+   `harness_research/NEXT_EXPERIMENT.md`. OpenJarvis strengthens the case for
+   including both a prompt-only comparison and executable loop changes.
+2. Define an immutable run identity covering model artifact, quantization,
+   adapter, agent code, prompts, tools, memory snapshot, runtime settings,
+   task revision and evaluation revision. Record all root/leaf/teacher calls.
+3. Add an OpenJarvis baseline through HTTP after the compatibility checks.
+   Preserve its full configuration and raw request trace.
+4. Run the separate RLM ablation with one shared request budget, isolated
+   execution and serial GPU admission. Include forced finalization in costs.
+5. Introduce typed candidate edits, frozen evaluation and rollback. Start with
+   bounded prompt/tool/loop changes before searching many dimensions together.
+6. Connect verified examples to existing LoRA training only after the fixed
+   task loop is measurable. Compare frozen, controller-only, adapter-only and
+   combined adaptation, including retention and collection/training/eval cost.
+
+Personal memory, task scratch state, training data and learned adapters need
+separate identities. A useful memory article is not automatically a training
+example, and a plausible answer is not automatically a verified success.
+Background synthesis, judging and training all consume the same Mac budget.
+The existing serialized scheduling proposal remains appropriate for the first
+study; this research supplies no evidence for concurrent training and serving.
+
+The opportunity is a local system that improves verified task outcomes while
+remaining usable on a personal Mac. OpenJarvis provides a close comparison
+and concrete implementation ideas. Our differentiator must be demonstrated
+through that complete operating loop, including its costs and failures.
+
+[architecture]: https://github.com/open-jarvis/OpenJarvis/blob/e25e77d36ee487aa456978e7a7f19a7f6d712260/docs/architecture/overview.md
+[learning]: https://open-jarvis.github.io/OpenJarvis/architecture/learning/
+[paper]: https://arxiv.org/html/2605.17172v1
+[rlm]: https://github.com/open-jarvis/OpenJarvis/blob/e25e77d36ee487aa456978e7a7f19a7f6d712260/src/openjarvis/agents/rlm.py
+[repl]: https://github.com/open-jarvis/OpenJarvis/blob/e25e77d36ee487aa456978e7a7f19a7f6d712260/src/openjarvis/agents/rlm_repl.py
+[base-agent]: https://github.com/open-jarvis/OpenJarvis/blob/e25e77d36ee487aa456978e7a7f19a7f6d712260/src/openjarvis/agents/_stubs.py
+[routing]: https://github.com/open-jarvis/OpenJarvis/blob/e25e77d36ee487aa456978e7a7f19a7f6d712260/src/openjarvis/learning/routing/learned_router.py
+[search]: https://github.com/open-jarvis/OpenJarvis/blob/e25e77d36ee487aa456978e7a7f19a7f6d712260/src/openjarvis/learning/spec_search/orchestrator.py
+[gate]: https://github.com/open-jarvis/OpenJarvis/blob/e25e77d36ee487aa456978e7a7f19a7f6d712260/src/openjarvis/learning/spec_search/gate/benchmark_gate.py
+[sft]: https://github.com/open-jarvis/OpenJarvis/blob/e25e77d36ee487aa456978e7a7f19a7f6d712260/src/openjarvis/learning/intelligence/sft_trainer.py
+[grpo]: https://github.com/open-jarvis/OpenJarvis/blob/e25e77d36ee487aa456978e7a7f19a7f6d712260/src/openjarvis/learning/intelligence/grpo_trainer.py
+[lora-stub]: https://github.com/open-jarvis/OpenJarvis/blob/e25e77d36ee487aa456978e7a7f19a7f6d712260/src/openjarvis/learning/spec_search/execute/appliers/lora_stub.py
+[search-guide]: https://github.com/open-jarvis/OpenJarvis/blob/e25e77d36ee487aa456978e7a7f19a7f6d712260/docs/user-guide/llm-guided-spec-search.md
+[engines]: https://github.com/open-jarvis/OpenJarvis/blob/e25e77d36ee487aa456978e7a7f19a7f6d712260/src/openjarvis/engine/openai_compat_engines.py
+[http-engine]: https://github.com/open-jarvis/OpenJarvis/blob/e25e77d36ee487aa456978e7a7f19a7f6d712260/src/openjarvis/engine/_openai_compat.py

package/docs/{design → archive/investigations}/optimization_plan.md

@@ -39,7 +39,7 @@ chasing host overhead that compile already removed.
 
 **Steps:**
 
-1. **Characterize before touching anything.** Run `scripts/decode-split.ts`
+1. **Characterize before touching anything.** Run `decode-split.ts (deleted; git history)`
    (the per-step wall-time splitter that already exists) on all three models at
    @600 and @8k. Record GPU-blocked time vs JS-graph-build time per step. This
    is the control measurement; everything in Phase A is judged against it. The

package/docs/{design → archive/investigations}/pre-colibri-stabilization.md

@@ -96,7 +96,7 @@ change that state.
 | LEAK-03 | P2 | fixed | `src/mlx/{autograd,checkpoint,custom-vjp}.ts` constructors | Failure out-parameter slots are initialized with live handles that may not be freed when native constructor calls fail. | Failure-injection tests for all three constructors prove slots, closures, and callbacks return to baseline with no double free. |
 | HYG-01 | P2 | fixed | `src/train/job.ts`, empty Gemma block | A config read and stale comment claim load-bearing environment setup, but the branch is empty. | Remove the read/block/comment; job parsing and training setup tests stay green. |
 | HYG-02 | P2 | fixed | `tsconfig.json` exclusions | Shipped web sources and tests that run in the gate are excluded from the zero-error typecheck contract. | Remove the exclusions or replace them with an explicit checked project reference; repository and web typechecks are green with no hidden files. |
-| HYG-03 | P2 | fixed | `scripts/test.sh` test glob | `scripts/bench-serving-load.test.ts` is model-free but outside the gate. | Gate enumerates and runs it exactly once; a sentinel/failing-case check proves it cannot silently fall out again. |
+| HYG-03 | P2 | fixed | `scripts/test.sh` test glob | `tests/bench-serving-load.test.ts` is model-free but outside the gate. | Gate enumerates and runs it exactly once; a sentinel/failing-case check proves it cannot silently fall out again. |
 | SEC-01 | P0 | fixed | `src/web-tools.ts`, `web_fetch` | Model-selected URLs bypass the existing public-destination policy, follow redirects automatically, and buffer the full body. This permits SSRF into loopback/private services and unbounded response buffering. | Reuse one shared fetch-policy implementation: reject private literals and DNS resolutions, revalidate every redirect, combine caller abort with timeout, and stream to a byte cap. Tests cover loopback, private DNS, public-to-private redirect, redirect limit, chunked over-cap, and normal public text. |
 | SEC-02 | P1 | fixed | `src/download.ts`, `blobId` | An unvalidated blob ID reaches `join()` and symlink creation, unlike the adjacent validated digest. It requires a hostile HF-compatible endpoint. | Reject any non-lowercase-40-hex SHA-1 blob ID before path construction; traversal and malformed-ID tests fail closed. |
 | DEPLOY-01 | D | deferred | `src/server.ts`, WebSocket upgrade | No WebSocket Origin enforcement. | Deferred while bind is loopback-only. Reopen before any supported non-loopback bind, tunnel, reverse proxy, or browser origin; then add an allowlist and cross-origin rejection tests. |
@@ -127,7 +127,7 @@ change that state.
   transform constructors return slots, closures, and callbacks to zero under
   repeated failure injection. The root typecheck now includes DOM/web sources
   and the formerly excluded web tests. `scripts/test.sh` enumerates
-  `scripts/bench-serving-load.test.ts` exactly once and rejects gate drift.
+  `tests/bench-serving-load.test.ts` exactly once and rejects gate drift.
 - `bunx tsc --noEmit`, `bun scripts/check-hygiene.ts`, `bash -n
   scripts/test.sh`, and `git diff --check` are green after Wave 3. The full
   post-Wave-3 repository gate and both performance measurements remain open.
@@ -140,7 +140,7 @@ change that state.
   Standard PEFT `use_rslora` now mounts and saves at exactly
   `alpha / sqrt(rank)`. Custom-VJP also frees the first callback if allocation
   of the second callback throws.
-- **PERF-01:** `scripts/bench-stream-decoder.ts` measured the original
+- **PERF-01:** `bench-stream-decoder.ts (deleted 2026-08-23; git history)` measured the original
   full-history decoder against the bounded exact-suffix implementation with
   five paired repetitions. MiniCPM5 medians were 142.62 → 27.40 ms at 512
   tokens (5.20×), 576.01 → 56.45 ms at 1,024 (10.20×), and 2,284.57 →
@@ -148,7 +148,7 @@ change that state.
   113.62–116.06 ms). Every per-push chunk and final byte stream matched at
   64–2,048 tokens; the e4b 1,024-token check was 88.62 → 10.25 ms. Raw,
   gitignored artifacts are under `reports/perf-01-stream-decoder-*.json`.
-- **PERF-02:** `scripts/bench-logprobs-readback.ts` confirmed the reported
+- **PERF-02:** `bench-logprobs-readback.ts (deleted 2026-08-23; git history)` confirmed the reported
   barrier on a clean M1 Max 32 GB with Bun 1.3.14 and
   Qwen2.5-0.5B-Instruct-4bit (256 tokens, two warmups, five measured rounds).
   At clean base `00e597e`, off/logprobs/top-5 medians were
@@ -302,7 +302,7 @@ after any overlapping merge.
 | Adapters/training | `tests/{lora,train-*,ifeval}.test.ts`; delayed writes, PEFT + mlx-lm layouts, unequal padding, optimizer LR scales, frozen scorer corpus |
 | Grammar/Responses store | `tests/{grammar,grammar-jump,responses,server}.test.ts`; degrade forms, top-logprob alignment, fake-clock reordered LRU |
 | MLX/resource ownership | focused diffusion/weights/autograd/checkpoint/custom-VJP failure loops with handle/allocation baselines |
-| Repository gate | `bun scripts/check-hygiene.ts`, `bunx tsc --noEmit`, `bash scripts/test.sh`; prove `scripts/bench-serving-load.test.ts` is included |
+| Repository gate | `bun scripts/check-hygiene.ts`, `bunx tsc --noEmit`, `bash scripts/test.sh`; prove `tests/bench-serving-load.test.ts` is included |
 | Performance | deterministic raw artifact, paired A/B, exact output parity, quiet-machine preflight before any curated claim |
 
 ## 8. Exit gate