mlx-bun 0.4.0 → 1.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +1453 -88
- package/THIRD_PARTY_NOTICES.md +882 -0
- package/bin/mlx-bun.convert.mjs +4 -0
- package/bin/mlx-bun.fuse.mjs +4 -0
- package/bin/mlx-bun.generate.mjs +4 -0
- package/bin/mlx-bun.lora.mjs +4 -0
- package/bin/mlx-bun.mjs +13 -21
- package/bin/mlx-bun.server.mjs +4 -0
- package/bin/mlx-bun.upload.mjs +4 -0
- package/dist/web/app.js +9584 -0
- package/package.json +50 -53
- package/src/cli/ane-install.ts +76 -0
- package/src/cli/args.ts +179 -0
- package/src/cli/engine-entry.ts +68 -0
- package/src/cli/inference.ts +214 -0
- package/src/cli/job-entry.ts +97 -0
- package/src/cli/library-host.ts +117 -0
- package/src/cli/main.ts +57 -0
- package/src/cli/memory-engine.ts +163 -0
- package/src/cli/memory.ts +36 -0
- package/src/cli/mlx-lm-aliases.ts +309 -0
- package/src/cli/model-choice.ts +33 -0
- package/src/cli/model-selection.ts +84 -0
- package/src/cli/module-host.ts +11 -0
- package/src/cli/module-verbs.ts +50 -0
- package/src/cli/numerical-policy.ts +12 -0
- package/src/cli/perplexity.ts +89 -0
- package/src/cli/serve-host.ts +423 -0
- package/src/cli/serve-isolated.ts +217 -0
- package/src/cli/serve-options.ts +106 -0
- package/src/cli/serve-state.ts +205 -0
- package/src/cli/serve.ts +500 -0
- package/src/cli/served-model-host.ts +102 -0
- package/src/cli/server-entry.ts +148 -0
- package/src/cli/serving-unit.ts +177 -0
- package/src/{tui.ts → cli/terminal.ts} +13 -8
- package/src/cli/worker-adapters.ts +34 -0
- package/src/cli/worker-entry.ts +272 -0
- package/src/cli/worker-unit.ts +179 -0
- package/src/engine/adapter-operation.ts +26 -0
- package/src/engine/cache-services.ts +114 -0
- package/src/engine/completion.ts +39 -0
- package/src/engine/generation-gateway.ts +358 -0
- package/src/engine/index.ts +59 -0
- package/src/engine/kv-budget.ts +120 -0
- package/src/engine/media-preparation.ts +232 -0
- package/src/engine/model-binding.ts +24 -0
- package/src/engine/model-host.ts +513 -0
- package/src/engine/model-serving.ts +89 -0
- package/src/{serve → engine}/preparation.ts +10 -8
- package/src/{serve → engine}/session-completion-engine.ts +6 -6
- package/src/engine/telemetry.ts +109 -0
- package/src/hub/downloads.ts +116 -0
- package/src/jobs/db.ts +16 -7
- package/src/jobs/events.ts +29 -0
- package/src/jobs/executable.ts +5 -0
- package/src/jobs/host.ts +51 -0
- package/src/jobs/in-process.ts +55 -0
- package/src/{contracts/jobs.ts → jobs/protocol.ts} +7 -9
- package/src/jobs/runner.ts +150 -231
- package/src/jobs/service.ts +75 -0
- package/src/jobs/sse.ts +59 -23
- package/src/jobs/worker-memory.ts +46 -0
- package/src/jobs/worker-process.ts +163 -0
- package/src/jobs/worker-supervisor.ts +258 -0
- package/src/modules.ts +58 -0
- package/src/publishing/catalog-hub.ts +18 -0
- package/src/publishing/credentials.ts +31 -0
- package/src/publishing/upload.ts +37 -0
- package/src/residency/leased-adapters.ts +21 -0
- package/src/residency/local-records.ts +16 -0
- package/src/residency/model-residency.ts +405 -0
- package/src/residency/record-index.ts +28 -0
- package/src/residency/resident-estimate.ts +50 -0
- package/src/{anthropic.ts → server/anthropic.ts} +3 -81
- package/src/server/cache-routes.ts +36 -0
- package/src/{serve → server}/chat-request.ts +12 -10
- package/src/{serve → server}/chat-stage.ts +27 -22
- package/src/{client.ts → server/client.ts} +28 -2
- package/src/{serve → server}/completion-executor.ts +42 -42
- package/src/server/decision-body.ts +55 -0
- package/src/server/decision-routes.ts +134 -0
- package/src/server/decision-wire.ts +228 -0
- package/src/{serve → server}/discovery-routes.ts +42 -83
- package/src/server/generated-token-history.ts +57 -0
- package/src/{serve → server}/http.ts +25 -13
- package/src/{serve → server}/inference-request.ts +5 -5
- package/src/server/job-routes.ts +20 -0
- package/src/server/media-prompt.ts +63 -0
- package/src/server/memory-completion-client.ts +46 -0
- package/src/server/model-routes.ts +192 -0
- package/src/{serve → server}/openai-wire.ts +3 -2
- package/src/{serve → server}/pipeline.ts +1 -1
- package/src/server/prompt-builder.ts +13 -0
- package/src/server/prompt-contracts.ts +17 -0
- package/src/server/proxy-routes.ts +187 -0
- package/src/server/publishing-routes.ts +41 -0
- package/src/{serve → server}/request-plan.ts +8 -8
- package/src/{serve → server}/request-prep.ts +82 -68
- package/src/{serve → server}/responses-client.ts +30 -13
- package/src/{responses.ts → server/responses.ts} +12 -93
- package/src/server/routes.ts +205 -0
- package/src/server/start.ts +72 -0
- package/src/server/status-routes.ts +108 -0
- package/src/{serve → server}/text-completion-stage.ts +25 -4
- package/src/server/text-prompt.ts +12 -0
- package/src/{serve → server}/token-streams.ts +36 -38
- package/src/server/worker-routes.ts +311 -0
- package/src/storage/paths.ts +61 -0
- package/src/web/assets.ts +37 -0
- package/src/web/{src → browser}/api.ts +5 -23
- package/src/web/browser/chat-host.ts +17 -0
- package/src/web/browser/installed-panels.ts +6 -0
- package/src/web/browser/main.ts +55 -0
- package/src/web/browser/markdown.ts +279 -0
- package/src/web/browser/memory-panel.ts +9 -0
- package/src/web/browser/palette.ts +64 -0
- package/src/web/browser/protocol.ts +13 -0
- package/src/web/browser/shell.ts +467 -0
- package/src/web/{src → browser}/status.ts +32 -36
- package/src/web/build.ts +51 -0
- package/src/web/public/app.html +1054 -0
- package/src/web/{sw.js → public/sw.js} +4 -3
- package/src/web/{vendor → public/vendor}/hljs-theme.css +2 -2
- package/src/web/{vendor → public/vendor}/hljs.js +2 -2
- package/THIRD_PARTY_LICENSES.md +0 -51
- package/docs/README.md +0 -94
- package/docs/archive/hlg-sampling.md +0 -350
- package/docs/archive/investigations/audio-input-plan.md +0 -356
- package/docs/archive/investigations/chunk-finetune-experiment.md +0 -84
- package/docs/archive/investigations/colibri-glm52-port.md +0 -1486
- package/docs/archive/investigations/colibri-oracle-pin.md +0 -1284
- package/docs/archive/investigations/curve-bisector-routing.md +0 -123
- package/docs/archive/investigations/decode-roofline-lookagain.md +0 -348
- package/docs/archive/investigations/diffusion-gemma-port.md +0 -602
- package/docs/archive/investigations/dspark-handoff.md +0 -224
- package/docs/archive/investigations/expert-offload-single-user-moe.md +0 -292
- package/docs/archive/investigations/faithful-l1-consolidation.md +0 -310
- package/docs/archive/investigations/hlg-sampling-investigation.md +0 -970
- package/docs/archive/investigations/indexshare-performance-spike.md +0 -357
- package/docs/archive/investigations/kernel-perf-review-2026-07.md +0 -171
- package/docs/archive/investigations/lab-build-journal.md +0 -157
- package/docs/archive/investigations/minicpm5-decode-megakernel.md +0 -489
- package/docs/archive/investigations/openjarvis-fit.md +0 -198
- package/docs/archive/investigations/openwiki-evaluation.md +0 -249
- package/docs/archive/investigations/optimization_plan.md +0 -346
- package/docs/archive/investigations/orpo-base-uf-experiment-and-directions.md +0 -120
- package/docs/archive/investigations/orpo-flash-cce-pin-leak.md +0 -167
- package/docs/archive/investigations/orpo-uf-testing-handoff.md +0 -126
- package/docs/archive/investigations/pareto-specialized-runtime-findings.md +0 -92
- package/docs/archive/investigations/pi-builtin-investigation.md +0 -308
- package/docs/archive/investigations/pre-colibri-stabilization.md +0 -329
- package/docs/archive/investigations/qwen38-27b-campaign-log.md +0 -1092
- package/docs/archive/investigations/segmented-backward-handoff.md +0 -148
- package/docs/archive/investigations/starter-model-port-handoff.md +0 -342
- package/docs/archive/investigations/steel-flash-cce-handoff.md +0 -489
- package/docs/archive/investigations/the-dreaming-handoff.md +0 -141
- package/docs/archive/investigations/trainer-validation-experiment.md +0 -173
- package/docs/archive/planning/chat-ui-vision-memory-first.md +0 -199
- package/docs/archive/planning/chat-ui-vision.md +0 -50
- package/docs/archive/planning/chunk-task-roadmap.md +0 -77
- package/docs/archive/planning/curve-sampler-research-plan.md +0 -331
- package/docs/archive/planning/journal.md +0 -316
- package/docs/archive/planning/memory-docs-and-dag-plan.md +0 -194
- package/docs/archive/planning/pi-integration-review.md +0 -178
- package/docs/archive/planning/release-notes-v0.0.10.md +0 -57
- package/docs/archive/planning/release-notes-v0.0.11.md +0 -239
- package/docs/archive/planning/release-notes-v0.0.12.md +0 -80
- package/docs/archive/planning/release-notes-v0.0.13.md +0 -77
- package/docs/archive/planning/release-notes-v0.0.9.md +0 -122
- package/docs/archive/planning/release-notes-v0.2.0.md +0 -49
- package/docs/archive/planning/release-notes-v0.3.0.md +0 -35
- package/docs/archive/planning/web-ui-pass-plan.md +0 -213
- package/docs/archive/planning/website-readme-pass-plan.md +0 -224
- package/docs/design/batching.md +0 -1841
- package/docs/design/decode-speed-program.md +0 -2349
- package/docs/design/dreaming-nightly-pipeline.md +0 -684
- package/docs/design/generic-model-support.md +0 -705
- package/docs/design/kv-cache.md +0 -824
- package/docs/design/orpo-training.md +0 -680
- package/docs/design/speculative-decoding.md +0 -1990
- package/docs/design/turboquant.md +0 -2212
- package/docs/design/unified-engine-frontier-plan.md +0 -1880
- package/docs/design/web-chat-redesign.md +0 -695
- package/docs/planning/PRODUCT_ROADMAP.md +0 -594
- package/docs/planning/ResearchTopics.md +0 -119
- package/docs/planning/release-notes-v0.4.0.md +0 -45
- package/docs/planning/video-series.md +0 -421
- package/docs/reference/benchmarks.md +0 -3613
- package/docs/reference/cli.md +0 -585
- package/docs/reference/distribution.md +0 -355
- package/docs/reference/environment.md +0 -398
- package/docs/reference/glossary.md +0 -39
- package/docs/reference/library-api.md +0 -674
- package/docs/reference/memory.md +0 -260
- package/docs/reference/models.md +0 -334
- package/docs/reference/server-api.md +0 -1775
- package/docs/reference/server-config.md +0 -846
- package/docs/reference/training.md +0 -462
- package/docs/reference/troubleshooting.md +0 -158
- package/src/admission-plan.ts +0 -18
- package/src/audio/conformer.ts +0 -647
- package/src/audio/decode.ts +0 -133
- package/src/audio/features.ts +0 -202
- package/src/audio/transcode.ts +0 -67
- package/src/backends/mlx/assistant-target.ts +0 -46
- package/src/backends/mlx/autoregressive-draft-rows.ts +0 -120
- package/src/backends/mlx/autoregressive.ts +0 -108
- package/src/backends/mlx/batch-group.ts +0 -1790
- package/src/backends/mlx/cache-attachment.ts +0 -67
- package/src/backends/mlx/cache-layout.ts +0 -31
- package/src/backends/mlx/checkpoint-state.ts +0 -53
- package/src/backends/mlx/continuation-persistence.ts +0 -106
- package/src/backends/mlx/continuation-request.ts +0 -43
- package/src/backends/mlx/continuation.ts +0 -77
- package/src/backends/mlx/diffusion.ts +0 -39
- package/src/backends/mlx/draft-target.ts +0 -32
- package/src/backends/mlx/evaluation.ts +0 -105
- package/src/backends/mlx/fill-append.ts +0 -45
- package/src/backends/mlx/full-kv-row-append.ts +0 -32
- package/src/backends/mlx/full-kv-row-donor.ts +0 -54
- package/src/backends/mlx/full-transitioning-kv-rows.ts +0 -65
- package/src/backends/mlx/gateway-binding.ts +0 -130
- package/src/backends/mlx/graph.ts +0 -38
- package/src/backends/mlx/kv-codec.ts +0 -9
- package/src/backends/mlx/kv-maintenance.ts +0 -133
- package/src/backends/mlx/kv-tensor-rows.ts +0 -212
- package/src/backends/mlx/memory-client.ts +0 -189
- package/src/backends/mlx/methods.ts +0 -90
- package/src/backends/mlx/model-prompt.ts +0 -164
- package/src/backends/mlx/model-serving.ts +0 -79
- package/src/backends/mlx/prefill-cohort.ts +0 -89
- package/src/backends/mlx/prefill-rows.ts +0 -200
- package/src/backends/mlx/prefill.ts +0 -33
- package/src/backends/mlx/projected-context-rows.ts +0 -72
- package/src/backends/mlx/request-policy.ts +0 -20
- package/src/backends/mlx/request-state-policy.ts +0 -41
- package/src/backends/mlx/rollback.ts +0 -89
- package/src/backends/mlx/row-sampling.ts +0 -49
- package/src/backends/mlx/serial-executor.ts +0 -242
- package/src/backends/mlx/speculative-group.ts +0 -359
- package/src/backends/mlx/speculative-round.ts +0 -161
- package/src/backends/mlx/speculative.ts +0 -114
- package/src/backends/mlx/state-rows.ts +0 -104
- package/src/backends/mlx/state-views.ts +0 -46
- package/src/backends/mlx/target-layout-capability.ts +0 -35
- package/src/backends/mlx/text-engine.ts +0 -73
- package/src/backends/mlx/transitioning-kv-rows.ts +0 -165
- package/src/chat-template.ts +0 -382
- package/src/cli.ts +0 -3168
- package/src/config.ts +0 -399
- package/src/contracts/completion.ts +0 -16
- package/src/contracts/execution.ts +0 -60
- package/src/contracts/generation.ts +0 -103
- package/src/contracts/host.ts +0 -7
- package/src/contracts/pi-provider.ts +0 -30
- package/src/contracts/pi.ts +0 -259
- package/src/contracts/prefix-cache.ts +0 -23
- package/src/contracts/resources.ts +0 -25
- package/src/contracts/sampling.ts +0 -23
- package/src/contracts/scheduling.ts +0 -45
- package/src/contracts/tsconfig.json +0 -14
- package/src/dataset/generators.ts +0 -1087
- package/src/dataset/index.ts +0 -12
- package/src/dataset/job.ts +0 -81
- package/src/dataset/llm.ts +0 -71
- package/src/dataset/registry.ts +0 -528
- package/src/diffusion/diffusion-generate.ts +0 -507
- package/src/download.ts +0 -505
- package/src/embed.ts +0 -77
- package/src/engine/admission.ts +0 -59
- package/src/engine/cancellation.ts +0 -38
- package/src/engine/engine.ts +0 -56
- package/src/engine/execution-coordinator.ts +0 -70
- package/src/engine/execution-plan.ts +0 -53
- package/src/engine/resources.ts +0 -53
- package/src/engine/scheduler.ts +0 -72
- package/src/engine/session.ts +0 -247
- package/src/eval/README.md +0 -130
- package/src/eval/ifeval.ts +0 -81
- package/src/eval/kl-prompts.ts +0 -29
- package/src/eval/kl.ts +0 -590
- package/src/eval/perplexity.ts +0 -164
- package/src/eval/quality-db.ts +0 -92
- package/src/eval/reference-logits.ts +0 -207
- package/src/eval/runner.ts +0 -124
- package/src/eval/score.ts +0 -49
- package/src/eval/tasks/bfcl.ts +0 -535
- package/src/eval/tasks/gsm8k.ts +0 -98
- package/src/eval/tasks/hashhop.ts +0 -139
- package/src/eval/tasks/humaneval.ts +0 -275
- package/src/eval/tasks/ifeval.ts +0 -525
- package/src/eval/tasks/mmlu.ts +0 -157
- package/src/eval/ultrafeedback.ts +0 -60
- package/src/evaldb.ts +0 -199
- package/src/expert-io.ts +0 -393
- package/src/expert-offload-build.ts +0 -77
- package/src/expert-offload.ts +0 -76
- package/src/expert-residency.ts +0 -1134
- package/src/expert-usage.ts +0 -415
- package/src/fill/echo-index.ts +0 -188
- package/src/fill/fill-session.ts +0 -452
- package/src/fill/proposal.ts +0 -68
- package/src/fill/schema-rows.ts +0 -506
- package/src/fill/tool-boundary.ts +0 -103
- package/src/fit.ts +0 -249
- package/src/generate.ts +0 -1346
- package/src/grammar.ts +0 -591
- package/src/harness-pi.ts +0 -184
- package/src/hf-push.ts +0 -628
- package/src/hub-rest.ts +0 -242
- package/src/index.ts +0 -63
- package/src/inference/denoising.ts +0 -17
- package/src/inference/draft-acceptance.ts +0 -41
- package/src/inference/draft-output.ts +0 -49
- package/src/inference/graph.ts +0 -33
- package/src/inference/prefill.ts +0 -39
- package/src/inference/rollback.ts +0 -7
- package/src/jobs/index.ts +0 -31
- package/src/jobs/job-entry.ts +0 -107
- package/src/jobs/task-client.ts +0 -19
- package/src/jobs/types.ts +0 -2
- package/src/kv-scheme.ts +0 -257
- package/src/kv-store.ts +0 -1099
- package/src/lab/README.md +0 -18
- package/src/lab/curve/curve-designer.html +0 -303
- package/src/lab/curve/curve-sampler.ts +0 -122
- package/src/lab/expert-trace/expert-trace.ts +0 -69
- package/src/lab/paged-kv/paged-kv-rows.ts +0 -83
- package/src/lab/paged-kv/paged-kv.ts +0 -270
- package/src/library.ts +0 -40
- package/src/lora.ts +0 -408
- package/src/md-modules.d.ts +0 -49
- package/src/media-fetch.ts +0 -442
- package/src/memory/article.ts +0 -558
- package/src/memory/chunk-validation.ts +0 -150
- package/src/memory/chunk.ts +0 -318
- package/src/memory/cluster.ts +0 -351
- package/src/memory/coverage.ts +0 -134
- package/src/memory/crosslink.ts +0 -0
- package/src/memory/db.ts +0 -253
- package/src/memory/entity.ts +0 -203
- package/src/memory/gate.ts +0 -187
- package/src/memory/history.ts +0 -152
- package/src/memory/ingest.ts +0 -335
- package/src/memory/model.ts +0 -183
- package/src/memory/normalize.ts +0 -671
- package/src/memory/parse.ts +0 -22
- package/src/memory/pipeline.ts +0 -275
- package/src/memory/prompts.ts +0 -65
- package/src/memory/query.ts +0 -334
- package/src/memory/reconcile.ts +0 -734
- package/src/memory/redirect.ts +0 -108
- package/src/memory/reindex.ts +0 -325
- package/src/memory/resolve.ts +0 -307
- package/src/memory/rest.ts +0 -245
- package/src/memory/route.ts +0 -272
- package/src/memory/schedule.ts +0 -165
- package/src/memory/stages.ts +0 -777
- package/src/memory/synthesize.ts +0 -1638
- package/src/memory/tools.ts +0 -749
- package/src/memory/vault.ts +0 -1194
- package/src/memory/wikify.ts +0 -1064
- package/src/mlx/array.ts +0 -388
- package/src/mlx/autograd.ts +0 -304
- package/src/mlx/checkpoint.ts +0 -149
- package/src/mlx/compile.ts +0 -168
- package/src/mlx/custom-vjp.ts +0 -188
- package/src/mlx/ffi.ts +0 -512
- package/src/mlx/materialize.ts +0 -23
- package/src/mlx/metal-kernel.ts +0 -159
- package/src/mlx/ops.ts +0 -1021
- package/src/mlx/quantized-rows.ts +0 -36
- package/src/mlx/token-bitmask.ts +0 -52
- package/src/mlx/turboquant-kv-decode.ts +0 -82
- package/src/mlx/turboquant-kv-inverse.ts +0 -122
- package/src/mlx/turboquant-ops.ts +0 -861
- package/src/mlx/turboquant-tables.ts +0 -237
- package/src/mmap.ts +0 -90
- package/src/model/artifact-identity.ts +0 -73
- package/src/model/batched-kv.ts +0 -235
- package/src/model/batched-mask.ts +0 -279
- package/src/model/batched-quant.ts +0 -212
- package/src/model/batched-quantized-kv.ts +0 -144
- package/src/model/batched-rotating-quant.ts +0 -353
- package/src/model/batched-rotating-state.ts +0 -149
- package/src/model/batched-rotating.ts +0 -338
- package/src/model/batched-row-storage.ts +0 -179
- package/src/model/batched-ssm.ts +0 -52
- package/src/model/batched-turboquant-kv.ts +0 -93
- package/src/model/compiled-decode.ts +0 -712
- package/src/model/decoded-kv-donor.ts +0 -12
- package/src/model/delayed-quantized-kv.ts +0 -72
- package/src/model/delayed-rotating-quantized-kv.ts +0 -93
- package/src/model/delayed-turboquant-kv.ts +0 -55
- package/src/model/diffusion-gemma.ts +0 -1026
- package/src/model/factory.ts +0 -189
- package/src/model/fingerprint.ts +0 -33
- package/src/model/flash-attention.ts +0 -694
- package/src/model/full-prefill-padding.ts +0 -56
- package/src/model/full-prefill-row.ts +0 -105
- package/src/model/gemma4-base.ts +0 -2351
- package/src/model/gemma4.ts +0 -1315
- package/src/model/generated/gemma4-12b.ts +0 -429
- package/src/model/generated/gemma4-26b.ts +0 -448
- package/src/model/generated/gemma4-e4b.ts +0 -717
- package/src/model/generated/index.ts +0 -18
- package/src/model/glm52-atlas.ts +0 -511
- package/src/model/glm52-cache.ts +0 -792
- package/src/model/glm52-config.ts +0 -262
- package/src/model/glm52-container.ts +0 -305
- package/src/model/glm52-coupling.ts +0 -388
- package/src/model/glm52-dsa.ts +0 -606
- package/src/model/glm52-expert-layout.ts +0 -253
- package/src/model/glm52-layout.ts +0 -152
- package/src/model/glm52-memory.ts +0 -500
- package/src/model/glm52-mla.ts +0 -792
- package/src/model/glm52-moe.ts +0 -223
- package/src/model/glm52-pilot.ts +0 -537
- package/src/model/glm52-reference.ts +0 -445
- package/src/model/glm52-residency.ts +0 -557
- package/src/model/glm52-resident-weights.ts +0 -135
- package/src/model/glm52-streamed-experts.ts +0 -453
- package/src/model/glm52-streamed-metal.ts +0 -610
- package/src/model/glm52-weights.ts +0 -503
- package/src/model/glm52.ts +0 -1132
- package/src/model/implementation.ts +0 -69
- package/src/model/kv-attention-view.ts +0 -67
- package/src/model/minicpm5.ts +0 -376
- package/src/model/profile.ts +0 -443
- package/src/model/qwen-append.ts +0 -12
- package/src/model/qwen-conv.ts +0 -52
- package/src/model/qwen3-delta-state.ts +0 -74
- package/src/model/qwen3-delta.ts +0 -451
- package/src/model/qwen3-moe.ts +0 -404
- package/src/model/qwen3-mrope.ts +0 -252
- package/src/model/qwen3.ts +0 -277
- package/src/model/qwen3_5-checkpoint.ts +0 -164
- package/src/model/qwen3_5.ts +0 -819
- package/src/model/rotating-kv-donor.ts +0 -35
- package/src/model/rotating-kv-layout.ts +0 -262
- package/src/model/rotating-row-transaction.ts +0 -35
- package/src/model/speculative-rotating-kv.ts +0 -101
- package/src/model/ssm-prefill-padding.ts +0 -39
- package/src/model/support.ts +0 -102
- package/src/model/trellis-balanced-scatter.ts +0 -51
- package/src/model/trellis-linear.ts +0 -672
- package/src/model/trellis-shared-m.ts +0 -151
- package/src/model/trellis-shared-scatter.ts +0 -144
- package/src/model/trellis-splitk-prefill.ts +0 -158
- package/src/model/trellis-tiled-prefill.ts +0 -136
- package/src/model/trellis-vector-expand.ts +0 -62
- package/src/model/trellis-wide-prefill.ts +0 -94
- package/src/model/turboquant-codec.ts +0 -83
- package/src/model/universal/archs.ts +0 -332
- package/src/model/universal/dense.ts +0 -612
- package/src/model/universal/modules.ts +0 -170
- package/src/model/universal/rope.ts +0 -381
- package/src/native/expert_io.c +0 -523
- package/src/native/frame_extract.swift +0 -140
- package/src/native-pack.ts +0 -162
- package/src/pi-provider.ts +0 -53
- package/src/pi-session.ts +0 -81
- package/src/pi-terminal.ts +0 -274
- package/src/pi-web.ts +0 -1932
- package/src/preflight.ts +0 -158
- package/src/prompt-cache.ts +0 -460
- package/src/quantize/allocator.ts +0 -627
- package/src/quantize/atomic-output.ts +0 -38
- package/src/quantize/calibration.ts +0 -181
- package/src/quantize/config-writer.ts +0 -145
- package/src/quantize/index.ts +0 -84
- package/src/quantize/job.ts +0 -162
- package/src/quantize/quantizer.ts +0 -666
- package/src/quantize/rotate.ts +0 -603
- package/src/quantize/safetensors-writer.ts +0 -249
- package/src/quantize/sensitivity.ts +0 -216
- package/src/quantize/trellis.ts +0 -586
- package/src/quantize/weight-transform.ts +0 -292
- package/src/registry.ts +0 -602
- package/src/runtime-config.ts +0 -81
- package/src/safetensors.ts +0 -158
- package/src/sampler.ts +0 -1011
- package/src/serve/admin-routes.ts +0 -184
- package/src/serve/assets/curve-terrain.html +0 -105
- package/src/serve/assets/training-inference-map.html +0 -575
- package/src/serve/aux-routes.ts +0 -132
- package/src/serve/batch-scheduler.ts +0 -8
- package/src/serve/checkpoint-identity.ts +0 -64
- package/src/serve/generation-gateway.ts +0 -496
- package/src/serve/isolate.ts +0 -609
- package/src/serve/kv-budget.ts +0 -14
- package/src/serve/lab-routes.ts +0 -341
- package/src/serve/lane-registry.ts +0 -45
- package/src/serve/model-admin-routes.ts +0 -155
- package/src/serve/model-binding.ts +0 -43
- package/src/serve/model-host.ts +0 -511
- package/src/serve/parent-application.ts +0 -56
- package/src/serve/prompt-response-trace.ts +0 -147
- package/src/serve/response-lease.ts +0 -24
- package/src/serve/session-search.ts +0 -326
- package/src/serve/static-routes.ts +0 -101
- package/src/server.ts +0 -1314
- package/src/spec/assistant-rows.ts +0 -92
- package/src/spec/assistant-source.ts +0 -126
- package/src/spec/cache-identity.ts +0 -22
- package/src/spec/deepspec-rows.ts +0 -29
- package/src/spec/deepspec-source.ts +0 -145
- package/src/spec/dflash-source.ts +0 -149
- package/src/spec/drafter.ts +0 -304
- package/src/spec/dspark/ab-stats.ts +0 -136
- package/src/spec/dspark/calibration.ts +0 -112
- package/src/spec/dspark/data-dflash.ts +0 -156
- package/src/spec/dspark/data.ts +0 -201
- package/src/spec/dspark/deepspec-module.ts +0 -776
- package/src/spec/dspark/generate-dflash.ts +0 -245
- package/src/spec/dspark/generate.ts +0 -290
- package/src/spec/dspark/loader.ts +0 -23
- package/src/spec/dspark/loss.ts +0 -213
- package/src/spec/dspark/module-dflash.ts +0 -560
- package/src/spec/dspark/module.ts +0 -540
- package/src/spec/dspark/quantize-drafter.ts +0 -91
- package/src/spec/dspark/sample.ts +0 -112
- package/src/spec/generate.ts +0 -276
- package/src/spec/glm52-mtp-graph.ts +0 -29
- package/src/spec/glm52-mtp-rows.ts +0 -132
- package/src/spec/glm52-mtp-source.ts +0 -317
- package/src/spec/glm52-mtp-state.ts +0 -33
- package/src/spec/ngram-source.ts +0 -204
- package/src/spec/projected-draft-rows.ts +0 -73
- package/src/spec/qwen-mtp-module.ts +0 -171
- package/src/spec/qwen-mtp-rows.ts +0 -219
- package/src/spec/qwen-mtp-source.ts +0 -383
- package/src/spec/qwen-mtp-state.ts +0 -30
- package/src/spec/serve-loop.ts +0 -585
- package/src/spec/source.ts +0 -215
- package/src/spec/two-model-rows.ts +0 -61
- package/src/spec/two-model.ts +0 -113
- package/src/ssd-cache.ts +0 -372
- package/src/ssd-durability.ts +0 -211
- package/src/tokenizer.ts +0 -335
- package/src/tool-approvals.ts +0 -131
- package/src/tool-call.ts +0 -695
- package/src/train/dataset.ts +0 -484
- package/src/train/diffusion-lora.ts +0 -216
- package/src/train/export.ts +0 -32
- package/src/train/flash-cce.ts +0 -1234
- package/src/train/forward.ts +0 -266
- package/src/train/fuse.ts +0 -243
- package/src/train/index.ts +0 -46
- package/src/train/job.ts +0 -205
- package/src/train/lora-params.ts +0 -254
- package/src/train/loss.ts +0 -1591
- package/src/train/merge.ts +0 -195
- package/src/train/optimizer.ts +0 -183
- package/src/train/prefix-shared.ts +0 -490
- package/src/train/rank.ts +0 -137
- package/src/train/segmented.ts +0 -1828
- package/src/train/steel-qmm-header.ts +0 -1461
- package/src/train/trainer.ts +0 -1221
- package/src/train/watch.ts +0 -542
- package/src/vision/diffusion-vision.ts +0 -620
- package/src/vision/embedder.ts +0 -168
- package/src/vision/preprocess.ts +0 -247
- package/src/vision/prompt.ts +0 -373
- package/src/vision/qwen3vl-preprocess.ts +0 -409
- package/src/vision/qwen3vl-prompt.ts +0 -196
- package/src/vision/qwen3vl-tower.ts +0 -555
- package/src/vision/siglip.ts +0 -599
- package/src/vision/video-frames.ts +0 -148
- package/src/web/app.html +0 -2143
- package/src/web/app.js +0 -5703
- package/src/web/skills/memory/SKILL.md +0 -120
- package/src/web/skills/web-research/SKILL.md +0 -46
- package/src/web/skills.ts +0 -97
- package/src/web/src/adapters-panel.ts +0 -288
- package/src/web/src/assistant.ts +0 -451
- package/src/web/src/chat.ts +0 -1057
- package/src/web/src/composer.ts +0 -959
- package/src/web/src/dataset.ts +0 -132
- package/src/web/src/finetune.ts +0 -267
- package/src/web/src/hub.ts +0 -428
- package/src/web/src/main.ts +0 -98
- package/src/web/src/markdown.ts +0 -0
- package/src/web/src/memory-panel.ts +0 -572
- package/src/web/src/model-picker.ts +0 -159
- package/src/web/src/palette.ts +0 -346
- package/src/web/src/protocol.ts +0 -26
- package/src/web/src/quantize.ts +0 -167
- package/src/web/src/rag.ts +0 -274
- package/src/web/src/sessions.ts +0 -469
- package/src/web/src/shell.ts +0 -804
- package/src/web/src/ui-catalog.ts +0 -78
- package/src/web/vendor/README.md +0 -61
- package/src/web-assets.ts +0 -47
- package/src/web-tools.ts +0 -478
- package/src/weights.ts +0 -255
- /package/src/{serve → server}/completion-sink.ts +0 -0
- /package/src/web/{icon.svg → public/icon.svg} +0 -0
- /package/src/web/{manifest.webmanifest → public/manifest.webmanifest} +0 -0
- /package/src/web/{vendor → public/vendor}/hljs-LICENSE +0 -0
|
@@ -1,213 +0,0 @@
|
|
|
1
|
-
# Web UI pass — ranked fix list (2026-07-01 audit)
|
|
2
|
-
|
|
3
|
-
Read-only audit of every web surface against the current server contract:
|
|
4
|
-
`src/web/app.html` (unified SPA: chat / quantize / finetune / dataset /
|
|
5
|
-
status+library / routes), `src/assets/curve-designer.html` (/curves),
|
|
6
|
-
`src/pi-web.ts` (WS chat bridge), `src/server.ts` routes, plus the dead
|
|
7
|
-
`src/status-page.html`. Method: diffed every fetch/WS consumer in the pages
|
|
8
|
-
against the actual handler shapes in `server.ts`, `src/jobs/types.ts`,
|
|
9
|
-
`src/train/{job,trainer}.ts`, `src/quantize/job.ts`, `src/download.ts`,
|
|
10
|
-
`src/registry.ts`, `src/model/support.ts`.
|
|
11
|
-
|
|
12
|
-
**Live click-through was skipped deliberately**: GPU-loading test suites
|
|
13
|
-
(`bun test tests/server.test.ts …`, loads MiniCPM5) were running for the whole
|
|
14
|
-
audit window, and starting a second model server would have contended with
|
|
15
|
-
them. Every finding below is source-verified on both sides of the contract.
|
|
16
|
-
|
|
17
|
-
Port sweep: no stale `8090` or hardcoded URLs remain in any html/js.
|
|
18
|
-
`curve-designer.html:92` falls back to `http://localhost:8080` (matches the
|
|
19
|
-
new default) and uses `location.origin` when served at `/curves`. ✓
|
|
20
|
-
|
|
21
|
-
---
|
|
22
|
-
|
|
23
|
-
## WRONG (bugs — fix first)
|
|
24
|
-
|
|
25
|
-
1. **Fine-tune live stats never show LR or tok/s — metric field-name mismatch.**
|
|
26
|
-
`src/web/app.html:2317-2318` reads `e.lr` / `e.tps`, but the trainer emits
|
|
27
|
-
`learning_rate` / `tokens_per_sec` (`src/train/trainer.ts:559-560`, same at
|
|
28
|
-
:724-727 and :966-969; contract comment in `src/jobs/types.ts:23`). The
|
|
29
|
-
"learning rate" and "tok/s" chart stats stay `—` for the entire run.
|
|
30
|
-
Also update the protocol doc-comment at `app.html:1089` (`lr?`, `tps?`).
|
|
31
|
-
**Fix:** read `e.learning_rate` and `e.tokens_per_sec`.
|
|
32
|
-
|
|
33
|
-
2. **Chat queue bar renders arrays as strings and empty tags.**
|
|
34
|
-
`pi-web.ts:183` sends `queue_update` with `steering: readonly string[]` and
|
|
35
|
-
`followUp: readonly string[]` (arrays). `app.html:1683-1684` does
|
|
36
|
-
`if (q.steering) … esc(String(q.steering))` — an empty array is truthy, so
|
|
37
|
-
every `queue_update` renders empty `steering:`/`queued:` pills, and multiple
|
|
38
|
-
queued messages render comma-joined.
|
|
39
|
-
**Fix:** guard with `.length` and render `q.followUp.map(...)` (or
|
|
40
|
-
`q.steering[q.steering.length-1]`).
|
|
41
|
-
|
|
42
|
-
3. **Library table loads exactly once per app lifetime — new quants never appear.**
|
|
43
|
-
`app.html:2603` gates `loadLibrary()`/`loadFit()` behind `fitLoaded`, which
|
|
44
|
-
is never reset. The server explicitly invalidates its 30 s `/library` cache
|
|
45
|
-
when a quantize job completes so "the next poll rescans and shows it"
|
|
46
|
-
(`src/server.ts:2510-2513`) — but the UI never re-polls. A model quantized,
|
|
47
|
-
downloaded, or gc'd mid-session never shows until a full page reload.
|
|
48
|
-
**Fix:** refresh the library on its own interval (e.g. every 15 s inside
|
|
49
|
-
`tick()`), and force one refresh when a quantize job reaches `done`.
|
|
50
|
-
|
|
51
|
-
4. **Adapter mount failure toasts "[object Object]".**
|
|
52
|
-
`POST /v1/adapters` errors use the OpenAI envelope `{error:{message}}`
|
|
53
|
-
(`src/server.ts:1600`), and the `api()` helper (`app.html:1080`) keeps
|
|
54
|
-
`data.error` as that object; `onSelectAdapter` (`app.html:1908-1909`) then
|
|
55
|
-
string-concatenates it. Any mount failure reads
|
|
56
|
-
`adapter: [object Object]`.
|
|
57
|
-
**Fix:** in `api()`, unwrap
|
|
58
|
-
`data.error && typeof data.error === "object" ? data.error.message : …`.
|
|
59
|
-
|
|
60
|
-
5. **Quantize inspect result parsing targets a response shape the server never sends.**
|
|
61
|
-
`inspectModel` returns `support: boolean` (`src/quantize/job.ts:101`), but
|
|
62
|
-
`app.html:2135-2140` checks `d.support === "supported" | "ok" | "untested"`
|
|
63
|
-
(all dead branches) and, when `support` is false, prints
|
|
64
|
-
`esc(d.support || "unknown architecture")` → always "unknown architecture"
|
|
65
|
-
even when `arch` was resolved. `size_gb` is also rendered raw (unrounded
|
|
66
|
-
float, and `0` when a direct path bypasses the registry).
|
|
67
|
-
**Fix:** treat `support` as boolean; show `arch` + "not quantizable" on
|
|
68
|
-
false; `(+d.size_gb).toFixed(2)` and hide when 0.
|
|
69
|
-
|
|
70
|
-
6. **Stale vision claim in the chat attach path.**
|
|
71
|
-
`app.html:1848`: "This model can't see images — switch to the **12B** for
|
|
72
|
-
vision." SigLIP landed: e2b/e4b/26B/31B are vision-capable too
|
|
73
|
-
(`src/registry.ts:213-219` `visionCapable`), so the 12B-only advice is
|
|
74
|
-
wrong.
|
|
75
|
-
**Fix:** model-agnostic copy: "This model can't see images — serve a
|
|
76
|
-
vision-capable model (Gemma vision builds show a *vision* tag in the
|
|
77
|
-
Library)."
|
|
78
|
-
|
|
79
|
-
---
|
|
80
|
-
|
|
81
|
-
## MISSING (current capabilities the UI doesn't expose)
|
|
82
|
-
|
|
83
|
-
7. **`sft_scope` absent from the fine-tune form.**
|
|
84
|
-
The finetune job accepts `sft_scope: "full" | "response"` and validates it
|
|
85
|
-
at submit (`src/train/job.ts:54,74-76`); default is now `"full"`
|
|
86
|
-
(paper/TRL-faithful, commit d32fe32). The ORPO panel
|
|
87
|
-
(`app.html:791-800`) has λ + schedule but no scope control, and
|
|
88
|
-
`collectHP()` (`app.html:2276-2289`) never sends it.
|
|
89
|
-
**Fix:** add a segmented control (full · response) to `#f-orpo-extra` with
|
|
90
|
-
a hint ("full = paper/TRL chosen-NLL over the whole sequence; response =
|
|
91
|
-
pre-2026-07 response-only"), include `sft_scope` in `collectHP()` when
|
|
92
|
-
method === "orpo".
|
|
93
|
-
|
|
94
|
-
8. **`min_p` / `xtc_probability` / `xtc_threshold` / `logprobs` not in any request-builder UI.**
|
|
95
|
-
Server supports them per-request (`src/server.ts:280-304`, wired at
|
|
96
|
-
:1138-1141). The chat sampling popover (`app.html:592-613`) only has
|
|
97
|
-
temperature/top_p/top_k — and the whole chain is width-limited: the
|
|
98
|
-
`set_sampling` WS frame (`pi-web.ts:160`), `SamplingOverrides`
|
|
99
|
-
(`pi-web.ts:408-412`), and `injectSampling` (`pi-web.ts:420-440`) all only
|
|
100
|
-
carry those three fields.
|
|
101
|
-
**Fix (scoped):** add `min_p` (slider 0–0.5) end-to-end: popover row →
|
|
102
|
-
`set_sampling` frame → `SamplingOverrides.min_p` → `injectSampling`. XTC and
|
|
103
|
-
logprobs are power-user/API features; document rather than add sliders.
|
|
104
|
-
|
|
105
|
-
9. **No gc/reclaim story in Library or Downloads.**
|
|
106
|
-
`mlx-bun gc` exists (CLI `src/cli.ts:1100`; planner
|
|
107
|
-
`src/registry.ts:375-455` `planGc`/`executeGc`) and the download path
|
|
108
|
-
prints "gc reclaims ~X GB" (`cli.ts:1015-1018`) — but the web Library shows
|
|
109
|
-
superseded revisions' disk cost with no affordance. **Needs a new server
|
|
110
|
-
endpoint first** (e.g. `GET /api/gc/plan`, `POST /api/gc/execute {yes}`),
|
|
111
|
-
then a Library footer line: "N superseded snapshots · reclaim X GB —
|
|
112
|
-
[Clean up]". (Separate from the listCanonical wiring below, but pairs
|
|
113
|
-
naturally with it: dedupe hides the stale revisions, gc deletes them.)
|
|
114
|
-
|
|
115
|
-
10. **Other accepted-but-hidden finetune/quantize knobs** (lower priority):
|
|
116
|
-
`grad_accumulation_steps`, `warm_start_adapter`, adapter naming (submit
|
|
117
|
-
always writes `adapter-<timestamp>`, `src/server.ts:2529-2530`) on the
|
|
118
|
-
finetune form; `reference`, `calibration_mix`, `n_calibration`
|
|
119
|
-
(`src/server.ts:2460-2461`) on the mixed-bpw quantize step. Add an
|
|
120
|
-
"Advanced" disclosure rather than more top-level fields.
|
|
121
|
-
|
|
122
|
-
---
|
|
123
|
-
|
|
124
|
-
## BLOCKED on the queued server.ts listCanonical / visionCapable / tier wiring
|
|
125
|
-
|
|
126
|
-
The functions exist and are tested (`Registry.listCanonical`
|
|
127
|
-
`src/registry.ts:181`, `visionCapable` :217, `supportTier`
|
|
128
|
-
`src/model/support.ts:53`) but the server still serves the old view. UI
|
|
129
|
-
consumption points to touch when the wiring lands:
|
|
130
|
-
|
|
131
|
-
11. **Revision duplicates in /library and /v1/models.**
|
|
132
|
-
`src/server.ts:1263` (`/library`) and :1491 (`/v1/models`) iterate
|
|
133
|
-
`reg.list()` — one row per cached snapshot revision. Both should use
|
|
134
|
-
`reg.listCanonical()`. UI needs no change; the Library table and every
|
|
135
|
-
/v1/models-driven picker (external pi, CLI) currently show dupes.
|
|
136
|
-
|
|
137
|
-
12. **`vision` flag understates capability.**
|
|
138
|
-
`src/server.ts:1278` sends `vision: m.hasVisionSidecar`, which misses the
|
|
139
|
-
12B's encoder-free `gemma4_unified_vision` (config-declared, no sidecar
|
|
140
|
-
file). Should be `visionCapable(m)`. UI consumption at `app.html:2663`
|
|
141
|
-
then needs no change.
|
|
142
|
-
|
|
143
|
-
13. **`supported` is a boolean; tiers are invisible.**
|
|
144
|
-
`src/server.ts:1264` sends `supported: isSupportedModelRecord(...)`.
|
|
145
|
-
Generic Tier-0 models (universal-dense descriptors) now count as
|
|
146
|
-
supported — correct — but the Library can't distinguish "targeted
|
|
147
|
-
(L2/L3 paths)" from "generic (L1 monolith only)". When the server sends
|
|
148
|
-
`tier: "targeted" | "generic" | null` (from `supportTier`), update:
|
|
149
|
-
- `app.html:2659` status cell: keep `unsupported (model_type)` for null;
|
|
150
|
-
add a dim "generic" badge for Tier-0 rows.
|
|
151
|
-
- Keep `supported` as a boolean alongside `tier` for one release so the
|
|
152
|
-
UI change can land independently.
|
|
153
|
-
|
|
154
|
-
---
|
|
155
|
-
|
|
156
|
-
## CONFUSING / polish
|
|
157
|
-
|
|
158
|
-
14. **Sampling popover "recommended" values are hardcoded to MiniCPM5.**
|
|
159
|
-
`app.html:1824-1825` (`SAMP_REC {topP:.95, topK:0}`, `recTemp()` 0.9/0.7)
|
|
160
|
-
claims to "mirror server toOptions defaultTemp", but the server resolves
|
|
161
|
-
defaults from each model's `generation_config.json`
|
|
162
|
-
(`src/server.ts:1120-1132`). For any other served model the dimmed "auto"
|
|
163
|
-
readouts lie. Fix: surface `genDefaults` (e.g. on the `ready` frame or
|
|
164
|
-
`/v1/models` extras) and seed the popover from it.
|
|
165
|
-
|
|
166
|
-
15. **Adapter dropdown goes stale after training.**
|
|
167
|
-
`refreshAdapters()` runs only at chat init and on `ready` frames
|
|
168
|
-
(`app.html:1789-1792, 2037`). An adapter trained in the Fine-tune tab
|
|
169
|
-
doesn't appear until reconnect/reload. Refresh on finetune `done` and on
|
|
170
|
-
chat `enter()`. Also escape the interpolation at `app.html:1897-1898`
|
|
171
|
-
(`a.id`/`a.path` go into HTML unescaped).
|
|
172
|
-
|
|
173
|
-
16. **Dead file: `src/status-page.html` (497 lines) + `scripts/status-page-stub.ts`.**
|
|
174
|
-
Nothing imports it since the SPA absorbed status (only
|
|
175
|
-
`scripts/status-page-stub.ts` references it). Delete both, or move the
|
|
176
|
-
stub note into the SPA docs.
|
|
177
|
-
|
|
178
|
-
17. **Routes tab breaks in the compiled binary.**
|
|
179
|
-
`#/routes` iframes `/dag`, which `readFileSync`s
|
|
180
|
-
`../docs/dag/training-inference-map.html` (`src/server.ts:1231`) — absent
|
|
181
|
-
outside the repo checkout, so the tab shows a bare 404 string. Either
|
|
182
|
-
embed the artifact like app.html, or hide the tab when `/dag` 404s.
|
|
183
|
-
|
|
184
|
-
18. **CLI banner vs landing route.** `src/cli.ts:64` says "status page at
|
|
185
|
-
http://localhost:8080/", but `/` lands on `#/chat`. Say "web app" or point
|
|
186
|
-
at `/#/status`.
|
|
187
|
-
|
|
188
|
-
19. **Curves nav tab never highlights.** It's a full-page `href="/curves"`
|
|
189
|
-
(`app.html:520`), outside the hash router, so the active-tab logic
|
|
190
|
-
(`app.html:1390`) can't mark it. Cosmetic; acceptable, but a
|
|
191
|
-
`aria-current` on the curve page's back-link side would be tidier.
|
|
192
|
-
|
|
193
|
-
20. **Library "too big" rows hide the useful number.** `app.html:2665-2666`
|
|
194
|
-
shows max context/decode only when `a.fits` (fit solved at 8192); a model
|
|
195
|
-
that fits at a smaller context shows `—`. Show `max_safe_context`
|
|
196
|
-
whenever it's > 0.
|
|
197
|
-
|
|
198
|
-
21. **Downloads visibility is per-process.** `/downloads` reads a
|
|
199
|
-
process-local tracker (`src/download.ts:276-279`) — a `mlx-bun get` in
|
|
200
|
-
another terminal is invisible to the nav pill/status card. Note in the UI
|
|
201
|
-
copy or leave; fixing needs a shared store.
|
|
202
|
-
|
|
203
|
-
---
|
|
204
|
-
|
|
205
|
-
## Suggested landing order
|
|
206
|
-
|
|
207
|
-
1. Bugs #1–#6 (one UI-only PR; no server changes).
|
|
208
|
-
2. #7 sft_scope + #15 adapter refresh (small, independent).
|
|
209
|
-
3. The queued server wiring (#11–#13) as its own PR: `listCanonical` +
|
|
210
|
-
`visionCapable` + `tier` in `/library` and `/v1/models`, then the
|
|
211
|
-
one-line UI badge.
|
|
212
|
-
4. #8 min_p end-to-end, #9 gc endpoint + Library affordance.
|
|
213
|
-
5. Cleanup: #16 dead files, #17 routes tab, #14 genDefaults surfacing.
|
|
@@ -1,224 +0,0 @@
|
|
|
1
|
-
# Website + README pass — audit findings and fix plan (2026-07-01)
|
|
2
|
-
|
|
3
|
-
Scope: the `website/` Astro subproject (mlx-bun.dev), `README.md`, and the
|
|
4
|
-
`docs/reference/` sources that feed both. Audited read-only against current
|
|
5
|
-
main **and against the published `v0.0.8` tag** (what installers actually
|
|
6
|
-
get). Every claim below was verified in source (`git show v0.0.8:…` for
|
|
7
|
-
release attribution), not doc-vs-doc.
|
|
8
|
-
|
|
9
|
-
## 0. The one decision that gates everything
|
|
10
|
-
|
|
11
|
-
**The site and README already document v0.0.9 behavior.** The published
|
|
12
|
-
v0.0.8 binary serves on **port 8090, bound to all interfaces**; main (and
|
|
13
|
-
every doc: README, quickstart, http-api, the synced server-config page)
|
|
14
|
-
says **8080 + loopback**. A fresh `brew install` user today follows the
|
|
15
|
-
quickstart and every curl example fails. The five auto-synced reference
|
|
16
|
-
pages likewise already describe `--max-tokens`, logprobs, `--sft-scope`,
|
|
17
|
-
and `/v1/completions` — all post-tag.
|
|
18
|
-
|
|
19
|
-
Two coherent options:
|
|
20
|
-
|
|
21
|
-
- **(A) Recommended: cut v0.0.9, then deploy the site.** All docs become
|
|
22
|
-
true simultaneously; nothing below needs version annotations. The
|
|
23
|
-
release-notes draft already exists (`docs/archive/planning/release-notes-v0.0.9.md`).
|
|
24
|
-
- **(B)** If v0.0.9 is not imminent: hold site deploys, and treat every
|
|
25
|
-
item marked *v0.0.9* below as blocked. (Reverting docs to 8090 is churn
|
|
26
|
-
for nothing — don't.)
|
|
27
|
-
|
|
28
|
-
The rest of this plan assumes (A); each item is still marked
|
|
29
|
-
**[safe-now]** (true for v0.0.8 users too) or **[v0.0.9]** (true only
|
|
30
|
-
after release).
|
|
31
|
-
|
|
32
|
-
## 1. Mechanism: how site content is produced
|
|
33
|
-
|
|
34
|
-
- **Generated (cannot drift):** 5 pages built from `docs/reference/*.md`
|
|
35
|
-
by `website/scripts/sync-reference-docs.mjs`, run on every
|
|
36
|
-
dev/build via the `astro:config:setup` hook in `astro.config.mjs`:
|
|
37
|
-
`server-api.md`, `server-config.md`, `training.md` → `reference/`;
|
|
38
|
-
`library-api.md`, `embedding.md` → `guides/`. Generated copies are
|
|
39
|
-
gitignored. This works well — verified accurate against main.
|
|
40
|
-
- **Hand-written (tracked, CAN silently drift, and did):** `index.mdx`,
|
|
41
|
-
`getting-started/{introduction,installation,quickstart,models}`,
|
|
42
|
-
`guides/{http-api,troubleshooting}`, `reference/cli.md`, `about/*` (4).
|
|
43
|
-
The three that drifted are exactly the ones duplicating CLI/model facts:
|
|
44
|
-
`reference/cli.md`, `getting-started/models.md`, `guides/http-api.md`.
|
|
45
|
-
- **Not synced at all:** `docs/reference/{memory,models,orpo-quickstart,distribution}.md`.
|
|
46
|
-
The first three are user-facing and site-worthy; distribution.md is
|
|
47
|
-
maintainer-ops, keep repo-only.
|
|
48
|
-
|
|
49
|
-
### Mechanism bugs
|
|
50
|
-
|
|
51
|
-
1. **Deploy trigger misses the source docs.** `.github/workflows/deploy-site.yml`
|
|
52
|
-
triggers only on `paths: website/**`. Edits to `docs/reference/*.md`
|
|
53
|
-
alone never redeploy — the "cannot drift" pages drift *in production*
|
|
54
|
-
until the next unrelated website commit. **Fix [safe-now]:** add
|
|
55
|
-
`docs/reference/**` to the paths filter.
|
|
56
|
-
2. **"Edit page" links on the 5 generated pages 404.** Starlight's global
|
|
57
|
-
`editLink.baseUrl` points at `website/src/content/docs/…`, but those
|
|
58
|
-
files are gitignored, so GitHub 404s. **Fix [safe-now]:** have the sync
|
|
59
|
-
script inject per-page `editUrl:` frontmatter pointing at the real
|
|
60
|
-
source (`docs/reference/<file>.md`).
|
|
61
|
-
3. **Local dev/build needs Node ≥ 22.12** (Astro 6). The machine default
|
|
62
|
-
is nvm Node 18 → `bun run build` fails locally; CI uses Node 22 and is
|
|
63
|
-
fine. **Fix [safe-now]:** add `"engines": { "node": ">=22.12.0" }` to
|
|
64
|
-
`website/package.json` and a one-liner in `website/README.md`
|
|
65
|
-
(`nvm use 22` or `bunx --bun astro build`).
|
|
66
|
-
|
|
67
|
-
### Build health (measured)
|
|
68
|
-
|
|
69
|
-
- Build: **green** — 18 pages, pagefind index, sitemap (built with Node 22 semantics via `bunx --bun astro build`).
|
|
70
|
-
- Internal links: **all 17 unique internal hrefs resolve** to built routes.
|
|
71
|
-
- GitHub `blob/main` links (22 unique): **all targets exist** in the repo.
|
|
72
|
-
- Assets: `public/{CNAME,install.sh,favicon.svg}` present.
|
|
73
|
-
- The only broken-link class is the editLink bug above.
|
|
74
|
-
|
|
75
|
-
## 2. WRONG (fix first — actively misleads)
|
|
76
|
-
|
|
77
|
-
| # | Where | Claim | Truth | Gate |
|
|
78
|
-
|---|---|---|---|---|
|
|
79
|
-
| W1 | Site quickstart/installation/index-hero, README (lines 96/108/208/235/286), synced server-config | server on `localhost:8080`, loopback | **v0.0.8 = port 8090, all interfaces**; 8080+loopback is post-tag (8ee691d) | **[v0.0.9]** — resolves itself under option (A) |
|
|
80
|
-
| W2 | `website/…/reference/cli.md:29` | `--no-kv-quant` / `--kv-bits <n>` | Never existed. Real flag: `--kv-quant config\|off\|4\|8` | [safe-now] |
|
|
81
|
-
| W3 | `website/…/reference/cli.md:30` | `--adapter id=dir` | Syntax is `--adapter <dir>` | [safe-now] |
|
|
82
|
-
| W4 | `README.md:304` | "Published to npm as `mlx-bun` (current: 0.0.4)" | 0.0.8 | [safe-now] |
|
|
83
|
-
| W5 | `getting-started/models.md:19` + `introduction.md:33` | "Currently MiniCPM5 plus the Gemma-4 quants… Qwen 3.x is next on the roadmap" | **Qwen3.5-4B is supported in v0.0.8** (support.ts at tag) — and `about/comparison.md:39` already says so; the site contradicts itself | [safe-now] |
|
|
84
|
-
| W6 | `README.md:39`, `installation.mdx:22` | pin example `MLX_BUN_VERSION=v0.0.4` | Use `v0.0.8` (or drop the pinned example) | [safe-now] |
|
|
85
|
-
|
|
86
|
-
## 3. STALE (true once, now behind)
|
|
87
|
-
|
|
88
|
-
- S1 `getting-started/models.md` table: add the Qwen3.5-4B row (copy the
|
|
89
|
-
README's row: 3.1 GB, 8 GB, thinking + tool calling). [safe-now]
|
|
90
|
-
- S2 `guides/http-api.md:85-89` endpoint list: `/stats`, `/fit`,
|
|
91
|
-
`/library`, `/downloads` existed at v0.0.8 and are undocumented
|
|
92
|
-
[safe-now]; `/v1/completions`, `/v1/embeddings`, `/health`,
|
|
93
|
-
logprobs/top_logprobs are **[v0.0.9]**.
|
|
94
|
-
- S3 `about/benchmarks.md:6`: "mlx-optiq 0.2.1" — oracle venv now runs
|
|
95
|
-
0.2.4; also "clean-machine re-measure is pending" (line 35) — refresh or
|
|
96
|
-
date-stamp when the re-measure lands. [safe-now, low priority]
|
|
97
|
-
- S4 README "Status" section: reflect the v0.0.9 wave (CI gate now exists,
|
|
98
|
-
Tier-0 generics, mlx-lm tool parity) — **[v0.0.9]**.
|
|
99
|
-
|
|
100
|
-
## 4. MISSING (goals with no page)
|
|
101
|
-
|
|
102
|
-
- M1 **Memory system — goal 5 — has ZERO website presence.** The
|
|
103
|
-
README sells it; the site never mentions it. `docs/reference/memory.md`
|
|
104
|
-
is accurate and site-worthy. **Fix:** add to the sync MAP →
|
|
105
|
-
`guides/memory.md` ("Personal memory") + a home-page feature card + a
|
|
106
|
-
sidebar entry. Read path (init/status/open/search/agent tools) shipped
|
|
107
|
-
in v0.0.8 [safe-now]; `memory synthesize` as a real pipeline is
|
|
108
|
-
**[v0.0.9]** (f7dad8f is post-tag) — under option (A) publish the whole
|
|
109
|
-
page at once.
|
|
110
|
-
- M2 **CLI reference covers 7 of ~21 verbs.** Missing and in v0.0.8:
|
|
111
|
-
`gen/generate`, `bench`, `train`, `train-watch`, `memory`, `pi`, `help`
|
|
112
|
-
[safe-now]. Missing and post-tag: `embed`, `gc`, `fuse`, `convert`,
|
|
113
|
-
`upload`, `perplexity`, `setup` alias **[v0.0.9]**. **Structural fix:**
|
|
114
|
-
stop hand-writing it — create `docs/reference/cli.md` as the tracked
|
|
115
|
-
source (or generate from `printHelp`) and add it to the sync MAP, so the
|
|
116
|
-
CLI page can never drift again.
|
|
117
|
-
- M3 **Model management** (`get`/`scan`/`ls`/`gc`, cache layout, gc safety
|
|
118
|
-
rails): `docs/reference/models.md` is accurate + site-worthy → sync to
|
|
119
|
-
`guides/model-management.md`. gc verb is **[v0.0.9]**; the rest safe-now.
|
|
120
|
-
- M4 **Drop-in mlx-lm replacement — goal 1 — has no page.** It's v0.0.9's
|
|
121
|
-
headline (mlx_lm.server endpoint/flag/sampler/logprobs parity, same
|
|
122
|
-
port/host defaults, `fuse`/`convert`/`perplexity`/`upload` tool parity).
|
|
123
|
-
**Fix [v0.0.9]:** new `guides/drop-in-mlx-lm.md` — "stop
|
|
124
|
-
`mlx_lm.server`, start `mlx-bun serve`, same curl works" with the
|
|
125
|
-
parity table and the deliberately-not-ported list.
|
|
126
|
-
- M5 **Tier-0 generic models** (UniversalDense, 11 archs, L1-gated,
|
|
127
|
-
bit-exact goldens for llama/qwen2/gemma2): reframes the scope story from
|
|
128
|
-
"a few models" to "runs what mlx-lm runs; *optimizes* the targets".
|
|
129
|
-
Rewrite `models.md` around the targeted-vs-generic tier split (`ls`
|
|
130
|
-
already labels it). **[v0.0.9]**.
|
|
131
|
-
- M6 **The lab — goal 6 — is invisible.** ORPO training (orpo-quickstart
|
|
132
|
-
is accurate + site-worthy), DSpark speculative decoding, the curve
|
|
133
|
-
designer (`/curves` UI ships in v0.0.8!), diffusion-gemma, the
|
|
134
|
-
train-watch dashboard. **Fix:** new `about/lab.md` (or top-level "The
|
|
135
|
-
lab") presenting these as the experimental playground with GitHub links
|
|
136
|
-
into docs/design + docs/investigations; sync orpo-quickstart →
|
|
137
|
-
`guides/fine-tuning-quickstart.md`. Mostly [safe-now] (frame as
|
|
138
|
-
experimental); `--sft-scope` mention is [v0.0.9].
|
|
139
|
-
- M7 **README link coverage:** memory paragraphs (137–145) should link
|
|
140
|
-
`docs/reference/memory.md` + `docs/design/memory-synthesis.md`; the
|
|
141
|
-
model table (155) should link `docs/reference/models.md`; nothing links
|
|
142
|
-
to **mlx-bun.dev** anywhere except install.sh — add a "Docs:
|
|
143
|
-
https://mlx-bun.dev" line at the top. [safe-now]
|
|
144
|
-
|
|
145
|
-
## 5. Structure / information architecture
|
|
146
|
-
|
|
147
|
-
### Website — proposed sidebar (new items marked)
|
|
148
|
-
|
|
149
|
-
```
|
|
150
|
-
Getting started
|
|
151
|
-
Introduction · Installation · Quickstart · Choosing a model (rewrite: tiers, M5)
|
|
152
|
-
Guides
|
|
153
|
-
The HTTP API
|
|
154
|
-
Drop-in for mlx-lm ← NEW (M4, v0.0.9)
|
|
155
|
-
Personal memory ← NEW (M1, synced from docs/reference/memory.md)
|
|
156
|
-
Model management ← NEW (M3, synced from docs/reference/models.md)
|
|
157
|
-
Using the library · Embedding in a Mac app
|
|
158
|
-
Fine-tuning quickstart ← NEW (M6, synced from orpo-quickstart.md)
|
|
159
|
-
Troubleshooting
|
|
160
|
-
Reference
|
|
161
|
-
CLI (→ synced, M2) · Server API · Server configuration · Training & fine-tuning
|
|
162
|
-
About
|
|
163
|
-
Why mlx-bun · How it compares · Benchmarks · Correctness
|
|
164
|
-
The lab ← NEW (M6)
|
|
165
|
-
```
|
|
166
|
-
|
|
167
|
-
### Six-goals legibility scorecard (today → after)
|
|
168
|
-
|
|
169
|
-
| Goal | Today | After |
|
|
170
|
-
|---|---|---|
|
|
171
|
-
| 1 Drop-in mlx-lm replacement | implicit ("OpenAI-compatible") | M4 page + home card names mlx_lm.server explicitly |
|
|
172
|
-
| 2 JS library / Mac apps | ✅ library + embedding guides | unchanged (already good) |
|
|
173
|
-
| 3 Memory/quality/speed | ✅ benchmarks + correctness + fit | S3 refresh |
|
|
174
|
-
| 4 One command <60s, 4 paths | ✅ home hero + installation | W1 port fix is the blocker |
|
|
175
|
-
| 5 Memory showcase | ❌ absent | M1 guide + home feature card |
|
|
176
|
-
| 6 Playground | ❌ absent | M6 lab page + home card |
|
|
177
|
-
|
|
178
|
-
- **Home page:** the 6 feature cards cover goals 1–4 twice and 5–6 never.
|
|
179
|
-
Swap/add two cards: "Personal memory" (goal 5) and "The lab" (goal 6);
|
|
180
|
-
make the Protocols card say "drop-in for mlx_lm.server" (goal 1).
|
|
181
|
-
- **Reading order** is otherwise sound: hero → install → quickstart →
|
|
182
|
-
guides works; introduction.md and why.md duplicate two paragraphs
|
|
183
|
-
verbatim — acceptable, but keep them in sync or have intro link out.
|
|
184
|
-
|
|
185
|
-
### README
|
|
186
|
-
|
|
187
|
-
- The 60-second promise is buried at line 92 inside the *fourth* install
|
|
188
|
-
path. Move it into the opening: "one command, chatting in under a
|
|
189
|
-
minute" as the first bold claim, install one-liner immediately after —
|
|
190
|
-
matching the website hero (they should tell the same story; today the
|
|
191
|
-
README leads with the product-bet essay).
|
|
192
|
-
- Add the six goals as an explicit bulleted frame (the "What mlx-bun is"
|
|
193
|
-
section at line 120 is close — reorder it to map 1:1 and give the memory
|
|
194
|
-
system and the lab their own H2s with doc links).
|
|
195
|
-
- Add the mlx-bun.dev docs link at the top (M7).
|
|
196
|
-
- Keep README ⊆ website: README states, website explains — every README
|
|
197
|
-
section should end with a link to the corresponding site page once M1–M6
|
|
198
|
-
land.
|
|
199
|
-
|
|
200
|
-
## 6. Execution order
|
|
201
|
-
|
|
202
|
-
1. **[safe-now, no release needed]** W2–W6, S1–S3, M7, mechanism fixes
|
|
203
|
-
(deploy paths filter, editUrl injection, engines field). Note: do NOT
|
|
204
|
-
deploy the site after these until step 2, because the synced pages
|
|
205
|
-
already carry v0.0.9 content (W1).
|
|
206
|
-
2. **Josh: cut v0.0.9** (release-notes draft exists).
|
|
207
|
-
3. **[with v0.0.9]** W1 becomes true; land M1–M6 + S4 + README
|
|
208
|
-
restructure; deploy once. New synced pages = add 4 entries to the
|
|
209
|
-
sync-script MAP (cli, memory, models→model-management,
|
|
210
|
-
orpo-quickstart→fine-tuning-quickstart) + sidebar entries + link
|
|
211
|
-
rewrites for the new routes.
|
|
212
|
-
|
|
213
|
-
## Appendix: release-attribution ground truth (verified via `git show v0.0.8:`)
|
|
214
|
-
|
|
215
|
-
- **In v0.0.8:** verbs get/scan/ls/fit/serve/gen/bench/evals/pi/harness/
|
|
216
|
-
train/train-watch/memory(read-path); Qwen3.5 + DiffusionGemma support;
|
|
217
|
-
`/curves`, `/dag`, `/stats`, `/fit`, `/library`, `/downloads`, chat UI;
|
|
218
|
-
port **8090**, host **all interfaces**.
|
|
219
|
-
- **Post-tag (v0.0.9):** port 8080 + loopback, `--temp`/`--max-tokens`,
|
|
220
|
-
`/v1/completions`, `/health`, `/v1/embeddings`, logprobs/top_logprobs,
|
|
221
|
-
sampler parity (min_p/XTC/logit_bias/penalties), verbs
|
|
222
|
-
embed/gc/fuse/convert/perplexity/upload, `--sft-scope`, Tier-0
|
|
223
|
-
UniversalDense (11 archs, L1 goldens), memory synthesize (real
|
|
224
|
-
pipeline), registry dedupe, CI gate, `--l2` tier-contract fix.
|