mlx-bun 0.5.0 → 1.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +1449 -109
- package/THIRD_PARTY_NOTICES.md +882 -0
- package/bin/mlx-bun.convert.mjs +4 -0
- package/bin/mlx-bun.fuse.mjs +4 -0
- package/bin/mlx-bun.generate.mjs +4 -0
- package/bin/mlx-bun.lora.mjs +4 -0
- package/bin/mlx-bun.mjs +13 -21
- package/bin/mlx-bun.server.mjs +4 -0
- package/bin/mlx-bun.upload.mjs +4 -0
- package/dist/web/app.js +9584 -0
- package/package.json +50 -53
- package/src/cli/ane-install.ts +76 -0
- package/src/cli/args.ts +179 -0
- package/src/cli/engine-entry.ts +68 -0
- package/src/cli/inference.ts +214 -0
- package/src/cli/job-entry.ts +97 -0
- package/src/cli/library-host.ts +117 -0
- package/src/cli/main.ts +57 -0
- package/src/cli/memory-engine.ts +163 -0
- package/src/cli/memory.ts +36 -0
- package/src/cli/mlx-lm-aliases.ts +309 -0
- package/src/cli/model-choice.ts +33 -0
- package/src/cli/model-selection.ts +84 -0
- package/src/cli/module-host.ts +11 -0
- package/src/cli/module-verbs.ts +50 -0
- package/src/cli/numerical-policy.ts +12 -0
- package/src/cli/perplexity.ts +89 -0
- package/src/cli/serve-host.ts +423 -0
- package/src/cli/serve-isolated.ts +217 -0
- package/src/cli/serve-options.ts +106 -0
- package/src/cli/serve-state.ts +205 -0
- package/src/cli/serve.ts +500 -0
- package/src/cli/served-model-host.ts +102 -0
- package/src/cli/server-entry.ts +148 -0
- package/src/cli/serving-unit.ts +177 -0
- package/src/{tui.ts → cli/terminal.ts} +13 -8
- package/src/cli/worker-adapters.ts +34 -0
- package/src/cli/worker-entry.ts +272 -0
- package/src/cli/worker-unit.ts +179 -0
- package/src/engine/adapter-operation.ts +26 -0
- package/src/engine/cache-services.ts +114 -0
- package/src/engine/completion.ts +39 -0
- package/src/engine/generation-gateway.ts +358 -0
- package/src/engine/index.ts +59 -0
- package/src/engine/kv-budget.ts +120 -0
- package/src/engine/media-preparation.ts +232 -0
- package/src/engine/model-binding.ts +24 -0
- package/src/engine/model-host.ts +513 -0
- package/src/engine/model-serving.ts +89 -0
- package/src/{serve → engine}/preparation.ts +7 -5
- package/src/{serve → engine}/session-completion-engine.ts +6 -6
- package/src/engine/telemetry.ts +109 -0
- package/src/hub/downloads.ts +116 -0
- package/src/jobs/db.ts +16 -7
- package/src/jobs/events.ts +29 -0
- package/src/jobs/executable.ts +5 -0
- package/src/jobs/host.ts +51 -0
- package/src/jobs/in-process.ts +55 -0
- package/src/{contracts/jobs.ts → jobs/protocol.ts} +7 -9
- package/src/jobs/runner.ts +150 -231
- package/src/jobs/service.ts +75 -0
- package/src/jobs/sse.ts +59 -23
- package/src/jobs/worker-memory.ts +46 -0
- package/src/jobs/worker-process.ts +163 -0
- package/src/jobs/worker-supervisor.ts +258 -0
- package/src/modules.ts +58 -0
- package/src/publishing/catalog-hub.ts +18 -0
- package/src/publishing/credentials.ts +31 -0
- package/src/publishing/upload.ts +37 -0
- package/src/residency/leased-adapters.ts +21 -0
- package/src/residency/local-records.ts +16 -0
- package/src/residency/model-residency.ts +405 -0
- package/src/residency/record-index.ts +28 -0
- package/src/residency/resident-estimate.ts +50 -0
- package/src/{anthropic.ts → server/anthropic.ts} +3 -81
- package/src/server/cache-routes.ts +36 -0
- package/src/{serve → server}/chat-request.ts +10 -10
- package/src/{serve → server}/chat-stage.ts +14 -14
- package/src/{client.ts → server/client.ts} +28 -2
- package/src/{serve → server}/completion-executor.ts +33 -41
- package/src/server/decision-body.ts +55 -0
- package/src/server/decision-routes.ts +134 -0
- package/src/server/decision-wire.ts +228 -0
- package/src/{serve → server}/discovery-routes.ts +25 -82
- package/src/{serve → server}/http.ts +25 -13
- package/src/{serve → server}/inference-request.ts +5 -5
- package/src/server/job-routes.ts +20 -0
- package/src/server/media-prompt.ts +63 -0
- package/src/server/memory-completion-client.ts +46 -0
- package/src/server/model-routes.ts +192 -0
- package/src/{serve → server}/openai-wire.ts +3 -2
- package/src/{serve → server}/pipeline.ts +1 -1
- package/src/server/prompt-builder.ts +13 -0
- package/src/server/prompt-contracts.ts +17 -0
- package/src/server/proxy-routes.ts +187 -0
- package/src/server/publishing-routes.ts +41 -0
- package/src/{serve → server}/request-plan.ts +6 -8
- package/src/{serve → server}/request-prep.ts +47 -50
- package/src/{serve → server}/responses-client.ts +30 -13
- package/src/{responses.ts → server/responses.ts} +12 -93
- package/src/server/routes.ts +205 -0
- package/src/server/start.ts +72 -0
- package/src/server/status-routes.ts +108 -0
- package/src/{serve → server}/text-completion-stage.ts +25 -4
- package/src/server/text-prompt.ts +12 -0
- package/src/{serve → server}/token-streams.ts +36 -38
- package/src/server/worker-routes.ts +311 -0
- package/src/storage/paths.ts +61 -0
- package/src/web/assets.ts +37 -0
- package/src/web/{src → browser}/api.ts +5 -23
- package/src/web/browser/chat-host.ts +17 -0
- package/src/web/browser/installed-panels.ts +6 -0
- package/src/web/browser/main.ts +55 -0
- package/src/web/browser/markdown.ts +279 -0
- package/src/web/browser/memory-panel.ts +9 -0
- package/src/web/browser/palette.ts +64 -0
- package/src/web/browser/protocol.ts +13 -0
- package/src/web/browser/shell.ts +467 -0
- package/src/web/{src → browser}/status.ts +32 -36
- package/src/web/build.ts +51 -0
- package/src/web/public/app.html +1054 -0
- package/src/web/{sw.js → public/sw.js} +4 -3
- package/src/web/{vendor → public/vendor}/hljs-theme.css +2 -2
- package/src/web/{vendor → public/vendor}/hljs.js +2 -2
- package/THIRD_PARTY_LICENSES.md +0 -51
- package/docs/README.md +0 -107
- package/docs/archive/hlg-sampling.md +0 -350
- package/docs/archive/investigations/audio-input-plan.md +0 -356
- package/docs/archive/investigations/benchmark-ledger-through-v0.4.0.md +0 -3617
- package/docs/archive/investigations/chunk-finetune-experiment.md +0 -84
- package/docs/archive/investigations/colibri-glm52-port.md +0 -1486
- package/docs/archive/investigations/colibri-oracle-pin.md +0 -1284
- package/docs/archive/investigations/curve-bisector-routing.md +0 -123
- package/docs/archive/investigations/decode-roofline-lookagain.md +0 -348
- package/docs/archive/investigations/diffusion-gemma-port.md +0 -602
- package/docs/archive/investigations/dspark-handoff.md +0 -224
- package/docs/archive/investigations/expert-offload-single-user-moe.md +0 -292
- package/docs/archive/investigations/faithful-l1-consolidation.md +0 -310
- package/docs/archive/investigations/hlg-sampling-investigation.md +0 -970
- package/docs/archive/investigations/indexshare-performance-spike.md +0 -357
- package/docs/archive/investigations/kernel-perf-review-2026-07.md +0 -171
- package/docs/archive/investigations/lab-build-journal.md +0 -157
- package/docs/archive/investigations/minicpm5-decode-megakernel.md +0 -489
- package/docs/archive/investigations/openjarvis-fit.md +0 -198
- package/docs/archive/investigations/openwiki-evaluation.md +0 -249
- package/docs/archive/investigations/optimization_plan.md +0 -346
- package/docs/archive/investigations/orpo-base-uf-experiment-and-directions.md +0 -120
- package/docs/archive/investigations/orpo-flash-cce-pin-leak.md +0 -167
- package/docs/archive/investigations/orpo-uf-testing-handoff.md +0 -126
- package/docs/archive/investigations/pareto-specialized-runtime-findings.md +0 -92
- package/docs/archive/investigations/pi-builtin-investigation.md +0 -308
- package/docs/archive/investigations/pre-colibri-stabilization.md +0 -329
- package/docs/archive/investigations/qwen38-27b-campaign-log.md +0 -1092
- package/docs/archive/investigations/segmented-backward-handoff.md +0 -148
- package/docs/archive/investigations/starter-model-port-handoff.md +0 -342
- package/docs/archive/investigations/steel-flash-cce-handoff.md +0 -489
- package/docs/archive/investigations/the-dreaming-handoff.md +0 -141
- package/docs/archive/investigations/trainer-validation-experiment.md +0 -173
- package/docs/archive/planning/chat-ui-vision-memory-first.md +0 -199
- package/docs/archive/planning/chat-ui-vision.md +0 -50
- package/docs/archive/planning/chunk-task-roadmap.md +0 -77
- package/docs/archive/planning/curve-sampler-research-plan.md +0 -331
- package/docs/archive/planning/journal.md +0 -316
- package/docs/archive/planning/memory-docs-and-dag-plan.md +0 -194
- package/docs/archive/planning/pi-integration-review.md +0 -178
- package/docs/archive/planning/release-notes-v0.0.10.md +0 -57
- package/docs/archive/planning/release-notes-v0.0.11.md +0 -239
- package/docs/archive/planning/release-notes-v0.0.12.md +0 -80
- package/docs/archive/planning/release-notes-v0.0.13.md +0 -77
- package/docs/archive/planning/release-notes-v0.0.9.md +0 -122
- package/docs/archive/planning/release-notes-v0.2.0.md +0 -49
- package/docs/archive/planning/release-notes-v0.3.0.md +0 -35
- package/docs/archive/planning/release-notes-v0.4.0.md +0 -45
- package/docs/archive/planning/release-notes-v0.5.0.md +0 -37
- package/docs/archive/planning/web-ui-pass-plan.md +0 -213
- package/docs/archive/planning/website-readme-pass-plan.md +0 -224
- package/docs/design/batching.md +0 -2054
- package/docs/design/decode-speed-program.md +0 -2404
- package/docs/design/dreaming-nightly-pipeline.md +0 -684
- package/docs/design/generic-model-support.md +0 -821
- package/docs/design/kv-cache.md +0 -993
- package/docs/design/orpo-training.md +0 -680
- package/docs/design/speculative-decoding.md +0 -2129
- package/docs/design/turboquant.md +0 -2225
- package/docs/design/unified-engine-frontier-plan.md +0 -1987
- package/docs/design/web-chat-redesign.md +0 -695
- package/docs/planning/PRODUCT_ROADMAP.md +0 -593
- package/docs/planning/ResearchTopics.md +0 -119
- package/docs/planning/public-readiness-handoff.md +0 -244
- package/docs/planning/public-readiness.md +0 -149
- package/docs/planning/video-series.md +0 -421
- package/docs/reference/benchmarks.md +0 -3587
- package/docs/reference/cli.md +0 -646
- package/docs/reference/distribution.md +0 -380
- package/docs/reference/environment.md +0 -494
- package/docs/reference/glossary.md +0 -39
- package/docs/reference/library-api.md +0 -688
- package/docs/reference/memory.md +0 -260
- package/docs/reference/models.md +0 -346
- package/docs/reference/server-api.md +0 -1945
- package/docs/reference/server-config.md +0 -973
- package/docs/reference/training.md +0 -462
- package/docs/reference/troubleshooting.md +0 -158
- package/src/admission-plan.ts +0 -18
- package/src/audio/audiotoolbox.ts +0 -112
- package/src/audio/conformer.ts +0 -655
- package/src/audio/decode.ts +0 -133
- package/src/audio/features.ts +0 -202
- package/src/audio/mic-capture.ts +0 -98
- package/src/audio/silero-vad.ts +0 -379
- package/src/audio/transcode.ts +0 -67
- package/src/audio/whisper-decode.ts +0 -774
- package/src/audio/whisper-fast.ts +0 -732
- package/src/audio/whisper-format.ts +0 -73
- package/src/audio/whisper-mel.ts +0 -185
- package/src/audio/whisper-timing.ts +0 -274
- package/src/audio/whisper-tokenizer.ts +0 -239
- package/src/audio/whisper-transcribe.ts +0 -410
- package/src/backends/mlx/assistant-target.ts +0 -46
- package/src/backends/mlx/autoregressive-draft-rows.ts +0 -120
- package/src/backends/mlx/autoregressive.ts +0 -129
- package/src/backends/mlx/batch-group.ts +0 -1874
- package/src/backends/mlx/cache-attachment.ts +0 -67
- package/src/backends/mlx/cache-layout.ts +0 -31
- package/src/backends/mlx/checkpoint-state.ts +0 -53
- package/src/backends/mlx/continuation-persistence.ts +0 -107
- package/src/backends/mlx/continuation-request.ts +0 -43
- package/src/backends/mlx/continuation.ts +0 -77
- package/src/backends/mlx/diffusion.ts +0 -39
- package/src/backends/mlx/draft-target.ts +0 -32
- package/src/backends/mlx/encoder-cache.ts +0 -25
- package/src/backends/mlx/evaluation.ts +0 -105
- package/src/backends/mlx/fill-append.ts +0 -58
- package/src/backends/mlx/fill-group.ts +0 -497
- package/src/backends/mlx/full-kv-row-append.ts +0 -32
- package/src/backends/mlx/full-kv-row-donor.ts +0 -54
- package/src/backends/mlx/full-transitioning-kv-rows.ts +0 -65
- package/src/backends/mlx/gateway-binding.ts +0 -174
- package/src/backends/mlx/graph.ts +0 -38
- package/src/backends/mlx/kv-codec.ts +0 -9
- package/src/backends/mlx/kv-maintenance.ts +0 -161
- package/src/backends/mlx/kv-tensor-rows.ts +0 -212
- package/src/backends/mlx/memory-client.ts +0 -189
- package/src/backends/mlx/methods.ts +0 -90
- package/src/backends/mlx/mixed-iteration.ts +0 -62
- package/src/backends/mlx/model-prompt.ts +0 -178
- package/src/backends/mlx/model-serving.ts +0 -80
- package/src/backends/mlx/paged-cache-codec.ts +0 -27
- package/src/backends/mlx/prefill-cohort.ts +0 -100
- package/src/backends/mlx/prefill-policy.ts +0 -26
- package/src/backends/mlx/prefill-rows.ts +0 -244
- package/src/backends/mlx/prefill.ts +0 -33
- package/src/backends/mlx/projected-context-rows.ts +0 -72
- package/src/backends/mlx/prompt-input.ts +0 -29
- package/src/backends/mlx/qwen-prompt-input.ts +0 -40
- package/src/backends/mlx/request-policy.ts +0 -20
- package/src/backends/mlx/request-state-policy.ts +0 -54
- package/src/backends/mlx/rollback.ts +0 -92
- package/src/backends/mlx/row-sampling.ts +0 -49
- package/src/backends/mlx/serial-executor.ts +0 -251
- package/src/backends/mlx/speculative-group.ts +0 -438
- package/src/backends/mlx/speculative-round.ts +0 -161
- package/src/backends/mlx/speculative.ts +0 -131
- package/src/backends/mlx/state-rows.ts +0 -104
- package/src/backends/mlx/state-views.ts +0 -46
- package/src/backends/mlx/target-layout-capability.ts +0 -35
- package/src/backends/mlx/text-engine.ts +0 -73
- package/src/backends/mlx/transitioning-kv-rows.ts +0 -173
- package/src/chat-template.ts +0 -382
- package/src/cli.ts +0 -3490
- package/src/config.ts +0 -399
- package/src/contracts/completion.ts +0 -16
- package/src/contracts/execution.ts +0 -70
- package/src/contracts/generation.ts +0 -103
- package/src/contracts/host.ts +0 -7
- package/src/contracts/object-cache.ts +0 -8
- package/src/contracts/pi-provider.ts +0 -30
- package/src/contracts/pi.ts +0 -261
- package/src/contracts/prefix-cache.ts +0 -33
- package/src/contracts/resources.ts +0 -25
- package/src/contracts/sampling.ts +0 -23
- package/src/contracts/scheduling.ts +0 -54
- package/src/contracts/tsconfig.json +0 -14
- package/src/dataset/generators.ts +0 -1087
- package/src/dataset/index.ts +0 -12
- package/src/dataset/job.ts +0 -81
- package/src/dataset/llm.ts +0 -71
- package/src/dataset/registry.ts +0 -528
- package/src/diffusion/diffusion-generate.ts +0 -507
- package/src/download.ts +0 -505
- package/src/embed.ts +0 -77
- package/src/engine/admission.ts +0 -59
- package/src/engine/cancellation.ts +0 -38
- package/src/engine/engine.ts +0 -56
- package/src/engine/execution-coordinator.ts +0 -70
- package/src/engine/execution-plan.ts +0 -64
- package/src/engine/execution-tasks.ts +0 -44
- package/src/engine/namespaced-cache.ts +0 -13
- package/src/engine/resources.ts +0 -53
- package/src/engine/scheduler.ts +0 -92
- package/src/engine/session.ts +0 -247
- package/src/eval/README.md +0 -130
- package/src/eval/ifeval.ts +0 -81
- package/src/eval/kl-prompts.ts +0 -29
- package/src/eval/kl.ts +0 -590
- package/src/eval/perplexity.ts +0 -164
- package/src/eval/quality-db.ts +0 -92
- package/src/eval/reference-logits.ts +0 -207
- package/src/eval/runner.ts +0 -124
- package/src/eval/score.ts +0 -49
- package/src/eval/tasks/bfcl.ts +0 -535
- package/src/eval/tasks/gsm8k.ts +0 -98
- package/src/eval/tasks/hashhop.ts +0 -139
- package/src/eval/tasks/humaneval.ts +0 -275
- package/src/eval/tasks/ifeval.ts +0 -525
- package/src/eval/tasks/mmlu.ts +0 -157
- package/src/eval/ultrafeedback.ts +0 -60
- package/src/evaldb.ts +0 -199
- package/src/expert-io.ts +0 -393
- package/src/expert-offload-build.ts +0 -77
- package/src/expert-offload.ts +0 -76
- package/src/expert-residency.ts +0 -1134
- package/src/expert-usage.ts +0 -415
- package/src/fill/echo-index.ts +0 -188
- package/src/fill/fill-session.ts +0 -463
- package/src/fill/proposal.ts +0 -68
- package/src/fill/schema-rows.ts +0 -506
- package/src/fill/tool-boundary.ts +0 -103
- package/src/fit.ts +0 -249
- package/src/generate.ts +0 -1352
- package/src/grammar.ts +0 -604
- package/src/harness-pi.ts +0 -184
- package/src/hf-push.ts +0 -628
- package/src/hub-rest.ts +0 -242
- package/src/index.ts +0 -63
- package/src/inference/denoising.ts +0 -17
- package/src/inference/draft-acceptance.ts +0 -41
- package/src/inference/draft-output.ts +0 -49
- package/src/inference/graph.ts +0 -33
- package/src/inference/prefill.ts +0 -45
- package/src/inference/rollback.ts +0 -7
- package/src/jobs/index.ts +0 -31
- package/src/jobs/job-entry.ts +0 -107
- package/src/jobs/task-client.ts +0 -19
- package/src/jobs/types.ts +0 -2
- package/src/kv-scheme.ts +0 -279
- package/src/kv-store.ts +0 -1135
- package/src/lab/README.md +0 -22
- package/src/lab/curve/curve-designer.html +0 -303
- package/src/lab/curve/curve-sampler.ts +0 -122
- package/src/lab/expert-trace/expert-trace.ts +0 -69
- package/src/lab/paged-kv/paged-attention.ts +0 -117
- package/src/lab/paged-kv/paged-kv-rows.ts +0 -111
- package/src/lab/paged-kv/paged-kv.ts +0 -336
- package/src/library.ts +0 -40
- package/src/lora.ts +0 -408
- package/src/md-modules.d.ts +0 -54
- package/src/media-fetch.ts +0 -442
- package/src/memory/article.ts +0 -558
- package/src/memory/chunk-validation.ts +0 -150
- package/src/memory/chunk.ts +0 -318
- package/src/memory/cluster.ts +0 -351
- package/src/memory/coverage.ts +0 -134
- package/src/memory/crosslink.ts +0 -0
- package/src/memory/db.ts +0 -253
- package/src/memory/entity.ts +0 -203
- package/src/memory/gate.ts +0 -187
- package/src/memory/history.ts +0 -152
- package/src/memory/ingest.ts +0 -335
- package/src/memory/model.ts +0 -183
- package/src/memory/normalize.ts +0 -671
- package/src/memory/parse.ts +0 -22
- package/src/memory/pipeline.ts +0 -275
- package/src/memory/prompts.ts +0 -65
- package/src/memory/query.ts +0 -334
- package/src/memory/reconcile.ts +0 -734
- package/src/memory/redirect.ts +0 -108
- package/src/memory/reindex.ts +0 -325
- package/src/memory/resolve.ts +0 -307
- package/src/memory/rest.ts +0 -245
- package/src/memory/route.ts +0 -272
- package/src/memory/schedule.ts +0 -165
- package/src/memory/stages.ts +0 -777
- package/src/memory/synthesize.ts +0 -1638
- package/src/memory/tools.ts +0 -749
- package/src/memory/vault.ts +0 -1194
- package/src/memory/wikify.ts +0 -1064
- package/src/mlx/array.ts +0 -425
- package/src/mlx/autograd.ts +0 -304
- package/src/mlx/checkpoint.ts +0 -149
- package/src/mlx/compile.ts +0 -168
- package/src/mlx/custom-vjp.ts +0 -188
- package/src/mlx/ffi.ts +0 -531
- package/src/mlx/materialize.ts +0 -23
- package/src/mlx/metal-kernel.ts +0 -159
- package/src/mlx/normalized-argmax.ts +0 -116
- package/src/mlx/ops.ts +0 -1108
- package/src/mlx/quantized-rows.ts +0 -36
- package/src/mlx/token-bitmask.ts +0 -52
- package/src/mlx/turboquant-kv-decode.ts +0 -82
- package/src/mlx/turboquant-kv-inverse.ts +0 -122
- package/src/mlx/turboquant-ops.ts +0 -861
- package/src/mlx/turboquant-tables.ts +0 -237
- package/src/mmap.ts +0 -90
- package/src/model/artifact-identity.ts +0 -73
- package/src/model/batched-kv.ts +0 -235
- package/src/model/batched-mask.ts +0 -279
- package/src/model/batched-quant.ts +0 -212
- package/src/model/batched-quantized-kv.ts +0 -144
- package/src/model/batched-rotating-quant.ts +0 -353
- package/src/model/batched-rotating-state.ts +0 -149
- package/src/model/batched-rotating.ts +0 -338
- package/src/model/batched-row-storage.ts +0 -179
- package/src/model/batched-ssm.ts +0 -52
- package/src/model/batched-turboquant-kv.ts +0 -94
- package/src/model/compiled-decode.ts +0 -712
- package/src/model/decoded-kv-donor.ts +0 -12
- package/src/model/delayed-quantized-kv.ts +0 -72
- package/src/model/delayed-rotating-quantized-kv.ts +0 -93
- package/src/model/delayed-turboquant-kv.ts +0 -57
- package/src/model/diffusion-gemma.ts +0 -1026
- package/src/model/factory.ts +0 -215
- package/src/model/fingerprint.ts +0 -45
- package/src/model/flash-attention.ts +0 -694
- package/src/model/full-prefill-padding.ts +0 -56
- package/src/model/full-prefill-row.ts +0 -105
- package/src/model/gemma4-base.ts +0 -2370
- package/src/model/gemma4.ts +0 -1395
- package/src/model/generated/gemma4-12b.ts +0 -429
- package/src/model/generated/gemma4-26b.ts +0 -448
- package/src/model/generated/gemma4-e4b.ts +0 -717
- package/src/model/generated/index.ts +0 -18
- package/src/model/glm52-atlas.ts +0 -511
- package/src/model/glm52-cache.ts +0 -792
- package/src/model/glm52-config.ts +0 -262
- package/src/model/glm52-container.ts +0 -305
- package/src/model/glm52-coupling.ts +0 -388
- package/src/model/glm52-dsa.ts +0 -606
- package/src/model/glm52-expert-layout.ts +0 -253
- package/src/model/glm52-layout.ts +0 -152
- package/src/model/glm52-memory.ts +0 -500
- package/src/model/glm52-mla.ts +0 -792
- package/src/model/glm52-moe.ts +0 -223
- package/src/model/glm52-pilot.ts +0 -537
- package/src/model/glm52-reference.ts +0 -445
- package/src/model/glm52-residency.ts +0 -557
- package/src/model/glm52-resident-weights.ts +0 -135
- package/src/model/glm52-streamed-experts.ts +0 -453
- package/src/model/glm52-streamed-metal.ts +0 -610
- package/src/model/glm52-weights.ts +0 -503
- package/src/model/glm52.ts +0 -1132
- package/src/model/implementation.ts +0 -69
- package/src/model/kv-attention-view.ts +0 -70
- package/src/model/minicpm5.ts +0 -376
- package/src/model/profile.ts +0 -468
- package/src/model/quantized-append-attention.ts +0 -31
- package/src/model/qwen-append.ts +0 -12
- package/src/model/qwen-conv.ts +0 -52
- package/src/model/qwen3-delta-state.ts +0 -74
- package/src/model/qwen3-delta.ts +0 -451
- package/src/model/qwen3-moe.ts +0 -404
- package/src/model/qwen3-mrope.ts +0 -250
- package/src/model/qwen3.ts +0 -277
- package/src/model/qwen3_5-checkpoint.ts +0 -164
- package/src/model/qwen3_5.ts +0 -881
- package/src/model/rotating-kv-donor.ts +0 -35
- package/src/model/rotating-kv-layout.ts +0 -262
- package/src/model/rotating-row-transaction.ts +0 -35
- package/src/model/speculative-rotating-kv.ts +0 -101
- package/src/model/ssm-prefill-padding.ts +0 -39
- package/src/model/support.ts +0 -110
- package/src/model/token-groups.ts +0 -66
- package/src/model/trellis-balanced-scatter.ts +0 -51
- package/src/model/trellis-linear.ts +0 -672
- package/src/model/trellis-shared-m.ts +0 -151
- package/src/model/trellis-shared-scatter.ts +0 -144
- package/src/model/trellis-splitk-prefill.ts +0 -158
- package/src/model/trellis-tiled-prefill.ts +0 -136
- package/src/model/trellis-vector-expand.ts +0 -62
- package/src/model/trellis-wide-prefill.ts +0 -94
- package/src/model/turboquant-codec.ts +0 -89
- package/src/model/universal/archs.ts +0 -332
- package/src/model/universal/dense.ts +0 -612
- package/src/model/universal/modules.ts +0 -170
- package/src/model/universal/rope.ts +0 -381
- package/src/model/whisper.ts +0 -435
- package/src/native/expert_io.c +0 -523
- package/src/native/frame_extract.swift +0 -140
- package/src/native/mic_capture.swift +0 -92
- package/src/native-pack.ts +0 -163
- package/src/pi-provider.ts +0 -53
- package/src/pi-session.ts +0 -81
- package/src/pi-terminal.ts +0 -274
- package/src/pi-web.ts +0 -1940
- package/src/preflight.ts +0 -158
- package/src/prompt-cache.ts +0 -682
- package/src/quantize/allocator.ts +0 -627
- package/src/quantize/atomic-output.ts +0 -38
- package/src/quantize/calibration.ts +0 -181
- package/src/quantize/config-writer.ts +0 -145
- package/src/quantize/index.ts +0 -84
- package/src/quantize/job.ts +0 -162
- package/src/quantize/quantizer.ts +0 -666
- package/src/quantize/rotate.ts +0 -603
- package/src/quantize/safetensors-writer.ts +0 -249
- package/src/quantize/sensitivity.ts +0 -216
- package/src/quantize/trellis.ts +0 -586
- package/src/quantize/weight-transform.ts +0 -292
- package/src/registry.ts +0 -602
- package/src/runtime-config.ts +0 -77
- package/src/safetensors.ts +0 -165
- package/src/sampler.ts +0 -1012
- package/src/serve/admin-routes.ts +0 -184
- package/src/serve/assets/curve-terrain.html +0 -105
- package/src/serve/assets/training-inference-map.html +0 -575
- package/src/serve/audio-routes.ts +0 -323
- package/src/serve/aux-routes.ts +0 -132
- package/src/serve/batch-scheduler.ts +0 -8
- package/src/serve/checkpoint-identity.ts +0 -64
- package/src/serve/generation-gateway.ts +0 -537
- package/src/serve/isolate.ts +0 -609
- package/src/serve/kv-budget.ts +0 -14
- package/src/serve/lab-routes.ts +0 -341
- package/src/serve/lane-registry.ts +0 -45
- package/src/serve/model-admin-routes.ts +0 -155
- package/src/serve/model-binding.ts +0 -49
- package/src/serve/model-host.ts +0 -511
- package/src/serve/parent-application.ts +0 -56
- package/src/serve/prompt-response-trace.ts +0 -161
- package/src/serve/response-lease.ts +0 -24
- package/src/serve/session-search.ts +0 -326
- package/src/serve/static-routes.ts +0 -101
- package/src/serve/transcription-server.ts +0 -64
- package/src/serve/transcription-service.ts +0 -502
- package/src/server.ts +0 -1282
- package/src/spec/assistant-rows.ts +0 -93
- package/src/spec/assistant-source.ts +0 -126
- package/src/spec/cache-identity.ts +0 -22
- package/src/spec/deepspec-rows.ts +0 -29
- package/src/spec/deepspec-source.ts +0 -145
- package/src/spec/dflash-source.ts +0 -149
- package/src/spec/drafter.ts +0 -304
- package/src/spec/dspark/ab-stats.ts +0 -136
- package/src/spec/dspark/calibration.ts +0 -112
- package/src/spec/dspark/data-dflash.ts +0 -156
- package/src/spec/dspark/data.ts +0 -201
- package/src/spec/dspark/deepspec-module.ts +0 -776
- package/src/spec/dspark/generate-dflash.ts +0 -245
- package/src/spec/dspark/generate.ts +0 -290
- package/src/spec/dspark/loader.ts +0 -23
- package/src/spec/dspark/loss.ts +0 -213
- package/src/spec/dspark/module-dflash.ts +0 -560
- package/src/spec/dspark/module.ts +0 -540
- package/src/spec/dspark/quantize-drafter.ts +0 -91
- package/src/spec/dspark/sample.ts +0 -112
- package/src/spec/generate.ts +0 -276
- package/src/spec/glm52-mtp-graph.ts +0 -29
- package/src/spec/glm52-mtp-rows.ts +0 -132
- package/src/spec/glm52-mtp-source.ts +0 -318
- package/src/spec/glm52-mtp-state.ts +0 -33
- package/src/spec/ngram-source.ts +0 -226
- package/src/spec/projected-draft-rows.ts +0 -73
- package/src/spec/qwen-mtp-module.ts +0 -171
- package/src/spec/qwen-mtp-rows.ts +0 -230
- package/src/spec/qwen-mtp-source.ts +0 -386
- package/src/spec/qwen-mtp-state.ts +0 -30
- package/src/spec/serve-loop.ts +0 -587
- package/src/spec/source.ts +0 -228
- package/src/spec/two-model-rows.ts +0 -62
- package/src/spec/two-model.ts +0 -113
- package/src/ssd-cache.ts +0 -431
- package/src/ssd-durability.ts +0 -209
- package/src/storage/host-buffer.ts +0 -24
- package/src/storage/kv-writer.ts +0 -83
- package/src/storage/kv-writer.worker.js +0 -264
- package/src/storage/retention-policy.ts +0 -37
- package/src/tiered-prompt-cache.ts +0 -45
- package/src/tokenizer.ts +0 -335
- package/src/tool-approvals.ts +0 -131
- package/src/tool-call.ts +0 -695
- package/src/train/dataset.ts +0 -484
- package/src/train/diffusion-lora.ts +0 -216
- package/src/train/export.ts +0 -32
- package/src/train/flash-cce.ts +0 -1234
- package/src/train/forward.ts +0 -266
- package/src/train/fuse.ts +0 -243
- package/src/train/index.ts +0 -46
- package/src/train/job.ts +0 -205
- package/src/train/lora-params.ts +0 -254
- package/src/train/loss.ts +0 -1591
- package/src/train/merge.ts +0 -195
- package/src/train/optimizer.ts +0 -183
- package/src/train/prefix-shared.ts +0 -490
- package/src/train/rank.ts +0 -137
- package/src/train/segmented.ts +0 -1828
- package/src/train/steel-qmm-header.ts +0 -1461
- package/src/train/trainer.ts +0 -1221
- package/src/train/watch.ts +0 -542
- package/src/vision/diffusion-vision.ts +0 -620
- package/src/vision/embedder.ts +0 -176
- package/src/vision/preprocess.ts +0 -247
- package/src/vision/prompt.ts +0 -409
- package/src/vision/qwen3vl-preprocess.ts +0 -409
- package/src/vision/qwen3vl-prompt.ts +0 -233
- package/src/vision/qwen3vl-tower.ts +0 -566
- package/src/vision/siglip.ts +0 -607
- package/src/vision/video-frames.ts +0 -148
- package/src/web/app.html +0 -2155
- package/src/web/app.js +0 -5863
- package/src/web/skills/memory/SKILL.md +0 -120
- package/src/web/skills/web-research/SKILL.md +0 -46
- package/src/web/skills.ts +0 -97
- package/src/web/src/adapters-panel.ts +0 -288
- package/src/web/src/assistant.ts +0 -451
- package/src/web/src/chat.ts +0 -1060
- package/src/web/src/composer.ts +0 -962
- package/src/web/src/dataset.ts +0 -132
- package/src/web/src/finetune.ts +0 -267
- package/src/web/src/hub.ts +0 -428
- package/src/web/src/main.ts +0 -98
- package/src/web/src/markdown.ts +0 -0
- package/src/web/src/memory-panel.ts +0 -572
- package/src/web/src/model-picker.ts +0 -159
- package/src/web/src/palette.ts +0 -346
- package/src/web/src/protocol.ts +0 -26
- package/src/web/src/quantize.ts +0 -167
- package/src/web/src/rag.ts +0 -274
- package/src/web/src/sessions.ts +0 -469
- package/src/web/src/shell.ts +0 -804
- package/src/web/src/ui-catalog.ts +0 -78
- package/src/web/src/voice.ts +0 -141
- package/src/web/vendor/README.md +0 -61
- package/src/web-assets.ts +0 -47
- package/src/web-tools.ts +0 -478
- package/src/weights.ts +0 -255
- /package/src/{serve → server}/completion-sink.ts +0 -0
- /package/src/{serve → server}/generated-token-history.ts +0 -0
- /package/src/web/{icon.svg → public/icon.svg} +0 -0
- /package/src/web/{manifest.webmanifest → public/manifest.webmanifest} +0 -0
- /package/src/web/{vendor → public/vendor}/hljs-LICENSE +0 -0
|
@@ -1,7 +1,8 @@
|
|
|
1
1
|
// mlx-bun service worker — SHELL ONLY, not offline chat.
|
|
2
2
|
//
|
|
3
|
-
// Scope (
|
|
4
|
-
//
|
|
3
|
+
// Scope (`02d723a:docs/design/web-chat-redesign.md` Appendix A,
|
|
4
|
+
// beat-matrix Axis 10 "PWA installability"): installability + instant shell
|
|
5
|
+
// load. The app is useless without the
|
|
5
6
|
// local inference server actually running (every chat turn is a live
|
|
6
7
|
// WebSocket to this same origin's model process), so caching API/WS
|
|
7
8
|
// traffic would be actively misleading — a "successful" cached response
|
|
@@ -18,7 +19,7 @@
|
|
|
18
19
|
//
|
|
19
20
|
// Versioned by CACHE_NAME so a deploy invalidates old shells; bump the
|
|
20
21
|
// suffix here if the shell file set changes.
|
|
21
|
-
const CACHE_NAME = "mlxbun-shell-
|
|
22
|
+
const CACHE_NAME = "mlxbun-shell-monorepo-v1";
|
|
22
23
|
const SHELL_FILES = [
|
|
23
24
|
"/",
|
|
24
25
|
"/assets/app.js",
|
|
@@ -1,5 +1,5 @@
|
|
|
1
1
|
/* Vendored highlight.js theme, hand-derived from mlx-bun's own design
|
|
2
|
-
* tokens (see the :root / [data-theme="light"] blocks in src/web/app.html)
|
|
2
|
+
* tokens (see the :root / [data-theme="light"] blocks in apps/mlx-bun/src/web/public/app.html)
|
|
3
3
|
* rather than one of highlight.js's stock themes — keeps code-block colors
|
|
4
4
|
* on the same palette as the rest of the app in both themes. Code blocks
|
|
5
5
|
* deliberately keep a dark chrome in BOTH themes (see app.html's
|
|
@@ -9,7 +9,7 @@
|
|
|
9
9
|
*
|
|
10
10
|
* Applies to `.bubble pre code.hljs` (see the highlightIn() hook in
|
|
11
11
|
* app.html) — every `<span class="hljs-*">` produced by the vendored
|
|
12
|
-
* hljs.js bundle (src/web/vendor/hljs.js, BSD-3-Clause, see its header).
|
|
12
|
+
* hljs.js bundle (apps/mlx-bun/src/web/public/vendor/hljs.js, BSD-3-Clause, see its header).
|
|
13
13
|
*/
|
|
14
14
|
|
|
15
15
|
.bubble pre code.hljs{ display:block; }
|
|
@@ -16,10 +16,10 @@
|
|
|
16
16
|
* permission. THIS SOFTWARE IS PROVIDED "AS IS" WITHOUT WARRANTY OF ANY
|
|
17
17
|
* KIND. See the upstream LICENSE file for the full text.
|
|
18
18
|
*
|
|
19
|
-
* Rebuild: see src/web/vendor/README.md (or the header of this bundling
|
|
19
|
+
* Rebuild: see `02d723a:src/web/vendor/README.md` (or the header of this bundling
|
|
20
20
|
* step in the mlx-bun repo) — `bun add highlight.js`, register the chosen
|
|
21
21
|
* languages onto `highlight.js/lib/core`, `bun build --minify`.
|
|
22
|
-
* NOT loaded from a CDN; served locally by src/
|
|
22
|
+
* NOT loaded from a CDN; served locally by apps/mlx-bun/src/web/assets.ts.
|
|
23
23
|
*/
|
|
24
24
|
var O3=Object.create;var{getPrototypeOf:A3,defineProperty:c1,getOwnPropertyNames:p3}=Object;var u3=Object.prototype.hasOwnProperty;function j3(z){return this[z]}var N3,R3,_3=(z,Z,q)=>{var V=z!=null&&typeof z==="object";if(V){var y=Z?N3??=new WeakMap:R3??=new WeakMap,k=y.get(z);if(k)return k}q=z!=null?O3(A3(z)):{};let Q=Z||!z||!z.__esModule?c1(q,"default",{value:z,enumerable:!0}):q;for(let J of p3(z))if(!u3.call(Q,J))c1(Q,J,{get:j3.bind(z,J),enumerable:!0});if(V)y.set(z,Q);return Q};var c3=(z,Z)=>()=>(Z||z((Z={exports:{}}).exports,Z),Z.exports);var v3=c3((E5,$3)=>{function h1(z){if(z instanceof Map)z.clear=z.delete=z.set=function(){throw Error("map is read-only")};else if(z instanceof Set)z.add=z.clear=z.delete=function(){throw Error("set is read-only")};return Object.freeze(z),Object.getOwnPropertyNames(z).forEach((Z)=>{let q=z[Z],V=typeof q;if((V==="object"||V==="function")&&!Object.isFrozen(q))h1(q)}),z}class w1{constructor(z){if(z.data===void 0)z.data={};this.data=z.data,this.isMatchIgnored=!1}ignoreMatch(){this.isMatchIgnored=!0}}function d1(z){return z.replace(/&/g,"&").replace(/</g,"<").replace(/>/g,">").replace(/"/g,""").replace(/'/g,"'")}function t(z,...Z){let q=Object.create(null);for(let V in z)q[V]=z[V];return Z.forEach(function(V){for(let y in V)q[y]=V[y]}),q}var S3="</span>",S1=(z)=>{return!!z.scope},T3=(z,{prefix:Z})=>{if(z.startsWith("language:"))return z.replace("language:","language-");if(z.includes(".")){let q=z.split(".");return[`${Z}${q.shift()}`,...q.map((V,y)=>`${V}${"_".repeat(y+1)}`)].join(" ")}return`${Z}${z}`};class r1{constructor(z,Z){this.buffer="",this.classPrefix=Z.classPrefix,z.walk(this)}addText(z){this.buffer+=d1(z)}openNode(z){if(!S1(z))return;let Z=T3(z.scope,{prefix:this.classPrefix});this.span(Z)}closeNode(z){if(!S1(z))return;this.buffer+=S3}value(){return this.buffer}span(z){this.buffer+=`<span class="${z}">`}}var T1=(z={})=>{let Z={children:[]};return Object.assign(Z,z),Z};class b1{constructor(){this.rootNode=T1(),this.stack=[this.rootNode]}get top(){return this.stack[this.stack.length-1]}get root(){return this.rootNode}add(z){this.top.children.push(z)}openNode(z){let Z=T1({scope:z});this.add(Z),this.stack.push(Z)}closeNode(){if(this.stack.length>1)return this.stack.pop();return}closeAllNodes(){while(this.closeNode());}toJSON(){return JSON.stringify(this.rootNode,null,4)}walk(z){return this.constructor._walk(z,this.rootNode)}static _walk(z,Z){if(typeof Z==="string")z.addText(Z);else if(Z.children)z.openNode(Z),Z.children.forEach((q)=>this._walk(z,q)),z.closeNode(Z);return z}static _collapse(z){if(typeof z==="string")return;if(!z.children)return;if(z.children.every((Z)=>typeof Z==="string"))z.children=[z.children.join("")];else z.children.forEach((Z)=>{b1._collapse(Z)})}}class m1 extends b1{constructor(z){super();this.options=z}addText(z){if(z==="")return;this.add(z)}startScope(z){this.openNode(z)}endScope(){this.closeNode()}__addSublanguage(z,Z){let q=z.root;if(Z)q.scope=`language:${Z}`;this.add(q)}toHTML(){return new r1(this,this.options).value()}finalize(){return this.closeAllNodes(),!0}}function $1(z){if(!z)return null;if(typeof z==="string")return z;return z.source}function t1(z){return s("(?=",z,")")}function i3(z){return s("(?:",z,")*")}function E3(z){return s("(?:",z,")?")}function s(...z){return z.map((q)=>$1(q)).join("")}function g3(z){let Z=z[z.length-1];if(typeof Z==="object"&&Z.constructor===Object)return z.splice(z.length-1,1),Z;else return{}}function f1(...z){return"("+(g3(z).capture?"":"?:")+z.map((V)=>$1(V)).join("|")+")"}function l1(z){return new RegExp(z.toString()+"|").exec("").length-1}function n3(z,Z){let q=z&&z.exec(Z);return q&&q.index===0}var h3=/\[(?:[^\\\]]|\\.)*\]|\(\??|\\([1-9][0-9]*)|\\./;function U1(z,{joinWith:Z}){let q=0;return z.map((V)=>{q+=1;let y=q,k=$1(V),Q="";while(k.length>0){let J=h3.exec(k);if(!J){Q+=k;break}if(Q+=k.substring(0,J.index),k=k.substring(J.index+J[0].length),J[0][0]==="\\"&&J[1])Q+="\\"+String(Number(J[1])+y);else if(Q+=J[0],J[0]==="(")q++}return Q}).map((V)=>`(${V})`).join(Z)}var d3=/\b\B/,a1="[a-zA-Z]\\w*",B1="[a-zA-Z_]\\w*",s1="\\b\\d+(\\.\\d+)?",o1="(-?)(\\b0[xX][a-fA-F0-9]+|(\\b\\d+(\\.\\d*)?|\\.\\d+)([eE][-+]?\\d+)?)",e1="\\b(0b[01]+)",r3="!|!=|!==|%|%=|&|&&|&=|\\*|\\*=|\\+|\\+=|,|-|-=|/=|/|:|;|<<|<<=|<=|<|===|==|=|>>>=|>>=|>=|>>>|>>|>|\\?|\\[|\\{|\\(|\\^|\\^=|\\||\\|=|\\|\\||~",m3=(z={})=>{let Z=/^#![ ]*\//;if(z.binary)z.begin=s(Z,/.*\b/,z.binary,/\b.*/);return t({scope:"meta",begin:Z,end:/$/,relevance:0,"on:begin":(q,V)=>{if(q.index!==0)V.ignoreMatch()}},z)},v1={begin:"\\\\[\\s\\S]",relevance:0},t3={scope:"string",begin:"'",end:"'",illegal:"\\n",contains:[v1]},l3={scope:"string",begin:'"',end:'"',illegal:"\\n",contains:[v1]},a3={begin:/\b(a|an|the|are|I'm|isn't|don't|doesn't|won't|but|just|should|pretty|simply|enough|gonna|going|wtf|so|such|will|you|your|they|like|more)\b/},y1=function(z,Z,q={}){let V=t({scope:"comment",begin:z,end:Z,contains:[]},q);V.contains.push({scope:"doctag",begin:"[ ]*(?=(TODO|FIXME|NOTE|BUG|OPTIMIZE|HACK|XXX):)",end:/(TODO|FIXME|NOTE|BUG|OPTIMIZE|HACK|XXX):/,excludeBegin:!0,relevance:0});let y=f1("I","a","is","so","us","to","at","if","in","it","on",/[A-Za-z]+['](d|ve|re|ll|t|s|n)/,/[A-Za-z]+[-][a-z]+/,/[A-Za-z][a-z]{2,}/);return V.contains.push({begin:s(/[ ]+/,"(",y,/[.]?[:]?([.][ ]|[ ])/,"){3}")}),V},s3=y1("//","$"),o3=y1("/\\*","\\*/"),e3=y1("#","$"),z5={scope:"number",begin:s1,relevance:0},Z5={scope:"number",begin:o1,relevance:0},q5={scope:"number",begin:e1,relevance:0},J5={scope:"regexp",begin:/\/(?=[^/\n]*\/)/,end:/\/[gimuy]*/,contains:[v1,{begin:/\[/,end:/\]/,relevance:0,contains:[v1]}]},Q5={scope:"title",begin:a1,relevance:0},$5={scope:"title",begin:B1,relevance:0},v5={begin:"\\.\\s*"+B1,relevance:0},V5=function(z){return Object.assign(z,{"on:begin":(Z,q)=>{q.data._beginMatch=Z[1]},"on:end":(Z,q)=>{if(q.data._beginMatch!==Z[1])q.ignoreMatch()}})},W1=Object.freeze({__proto__:null,APOS_STRING_MODE:t3,BACKSLASH_ESCAPE:v1,BINARY_NUMBER_MODE:q5,BINARY_NUMBER_RE:e1,COMMENT:y1,C_BLOCK_COMMENT_MODE:o3,C_LINE_COMMENT_MODE:s3,C_NUMBER_MODE:Z5,C_NUMBER_RE:o1,END_SAME_AS_BEGIN:V5,HASH_COMMENT_MODE:e3,IDENT_RE:a1,MATCH_NOTHING_RE:d3,METHOD_GUARD:v5,NUMBER_MODE:z5,NUMBER_RE:s1,PHRASAL_WORDS_MODE:a3,QUOTE_STRING_MODE:l3,REGEXP_MODE:J5,RE_STARTERS_RE:r3,SHEBANG:m3,TITLE_MODE:Q5,UNDERSCORE_IDENT_RE:B1,UNDERSCORE_TITLE_MODE:$5});function X5(z,Z){if(z.input[z.index-1]===".")Z.ignoreMatch()}function F5(z,Z){if(z.className!==void 0)z.scope=z.className,delete z.className}function W5(z,Z){if(!Z)return;if(!z.beginKeywords)return;if(z.begin="\\b("+z.beginKeywords.split(" ").join("|")+")(?!\\.)(?=\\b|\\s)",z.__beforeBegin=X5,z.keywords=z.keywords||z.beginKeywords,delete z.beginKeywords,z.relevance===void 0)z.relevance=0}function Y5(z,Z){if(!Array.isArray(z.illegal))return;z.illegal=f1(...z.illegal)}function y5(z,Z){if(!z.match)return;if(z.begin||z.end)throw Error("begin & end are not supported with match");z.begin=z.match,delete z.match}function k5(z,Z){if(z.relevance===void 0)z.relevance=1}var H5=(z,Z)=>{if(!z.beforeMatch)return;if(z.starts)throw Error("beforeMatch cannot be used with starts");let q=Object.assign({},z);Object.keys(z).forEach((V)=>{delete z[V]}),z.keywords=q.keywords,z.begin=s(q.beforeMatch,t1(q.begin)),z.starts={relevance:0,contains:[Object.assign(q,{endsParent:!0})]},z.relevance=0,delete q.beforeMatch},K5=["of","and","for","in","not","or","if","then","parent","list","value"],G5="keyword";function z3(z,Z,q=G5){let V=Object.create(null);if(typeof z==="string")y(q,z.split(" "));else if(Array.isArray(z))y(q,z);else Object.keys(z).forEach(function(k){Object.assign(V,z3(z[k],Z,k))});return V;function y(k,Q){if(Z)Q=Q.map((J)=>J.toLowerCase());Q.forEach(function(J){let X=J.split("|");V[X[0]]=[k,w5(X[0],X[1])]})}}function w5(z,Z){if(Z)return Number(Z);return b5(z)?0:1}function b5(z){return K5.includes(z.toLowerCase())}var i1={},a=(z)=>{console.error(z)},E1=(z,...Z)=>{console.log(`WARN: ${z}`,...Z)},e=(z,Z)=>{if(i1[`${z}/${Z}`])return;console.log(`Deprecated as of ${z}. ${Z}`),i1[`${z}/${Z}`]=!0},Y1=Error();function Z3(z,Z,{key:q}){let V=0,y=z[q],k={},Q={};for(let J=1;J<=Z.length;J++)Q[J+V]=y[J],k[J+V]=!0,V+=l1(Z[J-1]);z[q]=Q,z[q]._emit=k,z[q]._multi=!0}function f5(z){if(!Array.isArray(z.begin))return;if(z.skip||z.excludeBegin||z.returnBegin)throw a("skip, excludeBegin, returnBegin not compatible with beginScope: {}"),Y1;if(typeof z.beginScope!=="object"||z.beginScope===null)throw a("beginScope must be object"),Y1;Z3(z,z.begin,{key:"beginScope"}),z.begin=U1(z.begin,{joinWith:""})}function U5(z){if(!Array.isArray(z.end))return;if(z.skip||z.excludeEnd||z.returnEnd)throw a("skip, excludeEnd, returnEnd not compatible with endScope: {}"),Y1;if(typeof z.endScope!=="object"||z.endScope===null)throw a("endScope must be object"),Y1;Z3(z,z.end,{key:"endScope"}),z.end=U1(z.end,{joinWith:""})}function B5(z){if(z.scope&&typeof z.scope==="object"&&z.scope!==null)z.beginScope=z.scope,delete z.scope}function x5(z){if(B5(z),typeof z.beginScope==="string")z.beginScope={_wrap:z.beginScope};if(typeof z.endScope==="string")z.endScope={_wrap:z.endScope};f5(z),U5(z)}function P5(z){function Z(Q,J){return new RegExp($1(Q),"m"+(z.case_insensitive?"i":"")+(z.unicodeRegex?"u":"")+(J?"g":""))}class q{constructor(){this.matchIndexes={},this.regexes=[],this.matchAt=1,this.position=0}addRule(Q,J){J.position=this.position++,this.matchIndexes[this.matchAt]=J,this.regexes.push([J,Q]),this.matchAt+=l1(Q)+1}compile(){if(this.regexes.length===0)this.exec=()=>null;let Q=this.regexes.map((J)=>J[1]);this.matcherRe=Z(U1(Q,{joinWith:"|"}),!0),this.lastIndex=0}exec(Q){this.matcherRe.lastIndex=this.lastIndex;let J=this.matcherRe.exec(Q);if(!J)return null;let X=J.findIndex((K,U)=>U>0&&K!==void 0),H=this.matchIndexes[X];return J.splice(0,X),Object.assign(J,H)}}class V{constructor(){this.rules=[],this.multiRegexes=[],this.count=0,this.lastIndex=0,this.regexIndex=0}getMatcher(Q){if(this.multiRegexes[Q])return this.multiRegexes[Q];let J=new q;return this.rules.slice(Q).forEach(([X,H])=>J.addRule(X,H)),J.compile(),this.multiRegexes[Q]=J,J}resumingScanAtSamePosition(){return this.regexIndex!==0}considerAll(){this.regexIndex=0}addRule(Q,J){if(this.rules.push([Q,J]),J.type==="begin")this.count++}exec(Q){let J=this.getMatcher(this.regexIndex);J.lastIndex=this.lastIndex;let X=J.exec(Q);if(this.resumingScanAtSamePosition())if(X&&X.index===this.lastIndex);else{let H=this.getMatcher(0);H.lastIndex=this.lastIndex+1,X=H.exec(Q)}if(X){if(this.regexIndex+=X.position+1,this.regexIndex===this.count)this.considerAll()}return X}}function y(Q){let J=new V;if(Q.contains.forEach((X)=>J.addRule(X.begin,{rule:X,type:"begin"})),Q.terminatorEnd)J.addRule(Q.terminatorEnd,{type:"end"});if(Q.illegal)J.addRule(Q.illegal,{type:"illegal"});return J}function k(Q,J){let X=Q;if(Q.isCompiled)return X;[F5,y5,x5,H5].forEach((K)=>K(Q,J)),z.compilerExtensions.forEach((K)=>K(Q,J)),Q.__beforeBegin=null,[W5,Y5,k5].forEach((K)=>K(Q,J)),Q.isCompiled=!0;let H=null;if(typeof Q.keywords==="object"&&Q.keywords.$pattern)Q.keywords=Object.assign({},Q.keywords),H=Q.keywords.$pattern,delete Q.keywords.$pattern;if(H=H||/\w+/,Q.keywords)Q.keywords=z3(Q.keywords,z.case_insensitive);if(X.keywordPatternRe=Z(H,!0),J){if(!Q.begin)Q.begin=/\B|\b/;if(X.beginRe=Z(X.begin),!Q.end&&!Q.endsWithParent)Q.end=/\B|\b/;if(Q.end)X.endRe=Z(X.end);if(X.terminatorEnd=$1(X.end)||"",Q.endsWithParent&&J.terminatorEnd)X.terminatorEnd+=(Q.end?"|":"")+J.terminatorEnd}if(Q.illegal)X.illegalRe=Z(Q.illegal);if(!Q.contains)Q.contains=[];if(Q.contains=[].concat(...Q.contains.map(function(K){return D5(K==="self"?Q:K)})),Q.contains.forEach(function(K){k(K,X)}),Q.starts)k(Q.starts,J);return X.matcher=y(X),X}if(!z.compilerExtensions)z.compilerExtensions=[];if(z.contains&&z.contains.includes("self"))throw Error("ERR: contains `self` is not supported at the top-level of a language. See documentation.");return z.classNameAliases=t(z.classNameAliases||{}),k(z)}function q3(z){if(!z)return!1;return z.endsWithParent||q3(z.starts)}function D5(z){if(z.variants&&!z.cachedVariants)z.cachedVariants=z.variants.map(function(Z){return t(z,{variants:null},Z)});if(z.cachedVariants)return z.cachedVariants;if(q3(z))return t(z,{starts:z.starts?t(z.starts):null});if(Object.isFrozen(z))return t(z);return z}var C5="11.11.1";class J3 extends Error{constructor(z,Z){super(z);this.name="HTMLInjectionError",this.html=Z}}var G1=d1,g1=t,n1=Symbol("nomatch"),L5=7,Q3=function(z){let Z=Object.create(null),q=Object.create(null),V=[],y=!0,k="Could not find the language '{}', did you forget to load/include a language module?",Q={disableAutodetect:!0,name:"Plain text",contains:[]},J={ignoreUnescapedHTML:!1,throwUnescapedHTML:!1,noHighlightRe:/^(no-?highlight)$/i,languageDetectRe:/\blang(?:uage)?-([\w-]+)\b/i,classPrefix:"hljs-",cssSelector:"pre code",languages:null,__emitter:m1};function X($){return J.noHighlightRe.test($)}function H($){let W=$.className+" ";W+=$.parentNode?$.parentNode.className:"";let F=J.languageDetectRe.exec(W);if(F){let w=R(F[1]);if(!w)E1(k.replace("{}",F[1])),E1("Falling back to no-highlight mode for this block.",$);return w?F[1]:"no-highlight"}return W.split(/\s+/).find((w)=>X(w)||R(w))}function K($,W,F){let w="",B="";if(typeof W==="object")w=$,F=W.ignoreIllegals,B=W.language;else e("10.7.0","highlight(lang, code, ...args) has been deprecated."),e("10.7.0",`Please use highlight(code, options) instead.
|
|
25
25
|
https://github.com/highlightjs/highlight.js/issues/2277`),B=$,w=W;if(F===void 0)F=!0;let j={code:w,language:B};m("before:highlight",j);let c=j.result?j.result:U(j.language,j.code,F);return c.code=j.code,m("after:highlight",c),c}function U($,W,F,w){let B=Object.create(null);function j(v,Y){return v.keywords[Y]}function c(){if(!G.keywords){_.addText(L);return}let v=0;G.keywordPatternRe.lastIndex=0;let Y=G.keywordPatternRe.exec(L),b="";while(Y){b+=L.substring(v,Y.index);let C=h.case_insensitive?Y[0].toLowerCase():Y[0],S=j(G,C);if(S){let[r,M3]=S;if(_.addText(b),b="",B[C]=(B[C]||0)+1,B[C]<=L5)F1+=M3;if(r.startsWith("_"))b+=Y[0];else{let I3=h.classNameAliases[r]||r;n(Y[0],I3)}}else b+=Y[0];v=G.keywordPatternRe.lastIndex,Y=G.keywordPatternRe.exec(L)}b+=L.substring(v),_.addText(b)}function g(){if(L==="")return;let v=null;if(typeof G.subLanguage==="string"){if(!Z[G.subLanguage]){_.addText(L);return}v=U(G.subLanguage,L,!0,_1[G.subLanguage]),_1[G.subLanguage]=v._top}else v=f(L,G.subLanguage.length?G.subLanguage:null);if(G.relevance>0)F1+=v.relevance;_.__addSublanguage(v._emitter,v.language)}function E(){if(G.subLanguage!=null)g();else c();L=""}function n(v,Y){if(v==="")return;_.startScope(Y),_.addText(v),_.endScope()}function u1(v,Y){let b=1,C=Y.length-1;while(b<=C){if(!v._emit[b]){b++;continue}let S=h.classNameAliases[v[b]]||v[b],r=Y[b];if(S)n(r,S);else L=r,c(),L="";b++}}function j1(v,Y){if(v.scope&&typeof v.scope==="string")_.openNode(h.classNameAliases[v.scope]||v.scope);if(v.beginScope){if(v.beginScope._wrap)n(L,h.classNameAliases[v.beginScope._wrap]||v.beginScope._wrap),L="";else if(v.beginScope._multi)u1(v.beginScope,Y),L=""}return G=Object.create(v,{parent:{value:G}}),G}function N1(v,Y,b){let C=n3(v.endRe,b);if(C){if(v["on:end"]){let S=new w1(v);if(v["on:end"](Y,S),S.isMatchIgnored)C=!1}if(C){while(v.endsParent&&v.parent)v=v.parent;return v}}if(v.endsWithParent)return N1(v.parent,Y,b)}function x3(v){if(G.matcher.regexIndex===0)return L+=v[0],1;else return K1=!0,0}function P3(v){let Y=v[0],b=v.rule,C=new w1(b),S=[b.__beforeBegin,b["on:begin"]];for(let r of S){if(!r)continue;if(r(v,C),C.isMatchIgnored)return x3(Y)}if(b.skip)L+=Y;else{if(b.excludeBegin)L+=Y;if(E(),!b.returnBegin&&!b.excludeBegin)L=Y}return j1(b,v),b.returnBegin?0:Y.length}function D3(v){let Y=v[0],b=W.substring(v.index),C=N1(G,v,b);if(!C)return n1;let S=G;if(G.endScope&&G.endScope._wrap)E(),n(Y,G.endScope._wrap);else if(G.endScope&&G.endScope._multi)E(),u1(G.endScope,v);else if(S.skip)L+=Y;else{if(!(S.returnEnd||S.excludeEnd))L+=Y;if(E(),S.excludeEnd)L=Y}do{if(G.scope)_.closeNode();if(!G.skip&&!G.subLanguage)F1+=G.relevance;G=G.parent}while(G!==C.parent);if(C.starts)j1(C.starts,v);return S.returnEnd?0:Y.length}function C3(){let v=[];for(let Y=G;Y!==h;Y=Y.parent)if(Y.scope)v.unshift(Y.scope);v.forEach((Y)=>_.openNode(Y))}let X1={};function R1(v,Y){let b=Y&&Y[0];if(L+=v,b==null)return E(),0;if(X1.type==="begin"&&Y.type==="end"&&X1.index===Y.index&&b===""){if(L+=W.slice(Y.index,Y.index+1),!y){let C=Error(`0 width match regex (${$})`);throw C.languageName=$,C.badRule=X1.rule,C}return 1}if(X1=Y,Y.type==="begin")return P3(Y);else if(Y.type==="illegal"&&!F){let C=Error('Illegal lexeme "'+b+'" for mode "'+(G.scope||"<unnamed>")+'"');throw C.mode=G,C}else if(Y.type==="end"){let C=D3(Y);if(C!==n1)return C}if(Y.type==="illegal"&&b==="")return L+=`
|
package/THIRD_PARTY_LICENSES.md
DELETED
|
@@ -1,51 +0,0 @@
|
|
|
1
|
-
# Third-party licenses
|
|
2
|
-
|
|
3
|
-
mlx-bun is MIT-licensed. It builds on, links against, and ports code
|
|
4
|
-
from the following projects.
|
|
5
|
-
|
|
6
|
-
## Linked libraries
|
|
7
|
-
|
|
8
|
-
| Project | License | Use |
|
|
9
|
-
|---|---|---|
|
|
10
|
-
| [MLX](https://github.com/ml-explore/mlx) (Apple) | MIT | All GPU compute — Metal kernels, lazy eval engine (`libmlx`) |
|
|
11
|
-
| [mlx-c](https://github.com/ml-explore/mlx-c) (Apple) | MIT | C API we bind via `bun:ffi` (`libmlxc.dylib`, installed via Homebrew) |
|
|
12
|
-
|
|
13
|
-
## Ported code
|
|
14
|
-
|
|
15
|
-
These files are TypeScript ports of Python sources; the port preserves
|
|
16
|
-
the upstream algorithm and is a derivative work of the original.
|
|
17
|
-
|
|
18
|
-
| Source | License | Ported into |
|
|
19
|
-
|---|---|---|
|
|
20
|
-
| [mlx-lm](https://github.com/ml-explore/mlx-lm) (Apple, MIT) — `models/gemma4_text.py`, `models/cache.py`, `models/base.py`, `models/rope_utils.py`, `sample_utils.py`, `tool_parsers/gemma4.py` | MIT | `src/model/gemma4.ts`, `src/sampler.ts`, `src/tool-call.ts` |
|
|
21
|
-
| [mlx-vlm](https://github.com/Blaizzy/mlx-vlm) (BSD-3-Clause) — Gemma-4 unified `VisionEmbedder`, `MultimodalEmbedder`, image preprocessing (as vendored in mlx-optiq) | BSD-3-Clause | `src/vision/embedder.ts`, `src/vision/preprocess.ts` |
|
|
22
|
-
| mlx-optiq (MIT) — vision sidecar wiring, fused quantized SDPA orchestration | MIT | `src/vision/`, quantized-KV paths in `src/model/gemma4.ts` |
|
|
23
|
-
| [Pillow](https://github.com/python-pillow/Pillow) (MIT-CMU / HPND) — `ImagingResample` bicubic convolution algorithm | HPND | `resizeBicubic` in `src/vision/preprocess.ts` (algorithm port) |
|
|
24
|
-
| [Colibri](https://github.com/JustVugg/colibri) (pinned development oracle `44e489b196c9b7876b3d37a0570ebf1c6f90f54c`) — direct-container layout, expert-residency/I/O policy, GLM-5.2 execution and Metal-kernel reference | Apache-2.0 | Native GLM-5.2 implementation under `src/model/glm52-*`, `src/expert-*`, and `src/native/expert_io.c`; rewritten for Bun + MLX, with direct Colibri retained only as an external oracle |
|
|
25
|
-
|
|
26
|
-
## Runtime npm dependencies
|
|
27
|
-
|
|
28
|
-
| Package | License | Use |
|
|
29
|
-
|---|---|---|
|
|
30
|
-
| `@huggingface/tokenizers` | Apache-2.0 | Pure-JS tokenizer (tokenizer.json) |
|
|
31
|
-
| `@huggingface/jinja` | MIT | Chat-template rendering |
|
|
32
|
-
| `fast-png` | MIT | PNG decode (raw pixels for the vision path) |
|
|
33
|
-
| `@mlc-ai/web-xgrammar` | Apache-2.0 | Grammar-constrained decoding (WASM build of [xgrammar](https://github.com/mlc-ai/xgrammar) by the MLC-ai team). `response_format` / `guided_grammar` / `guided_regex` / `guided_choice` compile to a GrammarMatcher whose per-step token bitmask masks invalid logits (`src/grammar.ts`). Idea ported from oMLX (`api/grammar.py`), same engine, WASM-packaged. |
|
|
34
|
-
|
|
35
|
-
Run `bun pm licenses` (or inspect `node_modules/*/LICENSE`) for the full
|
|
36
|
-
text of each npm dependency's license, including transitive ones.
|
|
37
|
-
|
|
38
|
-
## Models
|
|
39
|
-
|
|
40
|
-
Model weights (e.g. `mlx-community/gemma-4-12B-it-OptiQ-4bit`) are not
|
|
41
|
-
part of this repository and are governed by their own licenses and use
|
|
42
|
-
policies (e.g. Google's Gemma Terms of Use). Check the model card on
|
|
43
|
-
Hugging Face before redistribution.
|
|
44
|
-
|
|
45
|
-
The validated GLM-5.2 direct-container artifact,
|
|
46
|
-
[`mateogrgic/GLM-5.2-colibri-int4-with-int8-mtp`](https://huggingface.co/mateogrgic/GLM-5.2-colibri-int4-with-int8-mtp),
|
|
47
|
-
declares MIT and identifies itself as a Colibri conversion/derivative of
|
|
48
|
-
[`zai-org/GLM-5.2-FP8`](https://huggingface.co/zai-org/GLM-5.2-FP8), with
|
|
49
|
-
lineage through `jlnsrk/GLM-5.2-colibri-int4` and added int8 MTP weights. The
|
|
50
|
-
artifact is downloaded separately into the Hugging Face cache and is not part
|
|
51
|
-
of the mlx-bun distribution.
|
package/docs/README.md
DELETED
|
@@ -1,107 +0,0 @@
|
|
|
1
|
-
# Documentation
|
|
2
|
-
|
|
3
|
-
Generated index. Update with `bun scripts/check-hygiene.ts --write-docs-map`.
|
|
4
|
-
|
|
5
|
-
## Start here
|
|
6
|
-
|
|
7
|
-
- Run mlx-bun: [quickstart](../README.md#quickstart), [models](reference/models.md), [CLI](reference/cli.md), [troubleshooting](reference/troubleshooting.md).
|
|
8
|
-
- Build an application: [TypeScript library](reference/library-api.md), [HTTP API](reference/server-api.md), [server configuration](reference/server-config.md).
|
|
9
|
-
- Assess the results: [benchmarks and correctness evidence](reference/benchmarks.md).
|
|
10
|
-
- Contribute: [contribution guide](../CONTRIBUTING.md), [engine architecture](design/unified-engine-frontier-plan.md), [current work](../PLAN.md).
|
|
11
|
-
|
|
12
|
-
## docs/reference/
|
|
13
|
-
|
|
14
|
-
User-facing reference — the ONLY home for flags, routes, models, numbers, and environment facts.
|
|
15
|
-
|
|
16
|
-
- [benchmarks.md](../docs/reference/benchmarks.md) — mlx-bun benchmark results
|
|
17
|
-
- [cli.md](../docs/reference/cli.md) — CLI reference
|
|
18
|
-
- [distribution.md](../docs/reference/distribution.md) — Distributing mlx-bun — build, sign, notarize, publish
|
|
19
|
-
- [environment.md](../docs/reference/environment.md) — Reference environment and platform facts
|
|
20
|
-
- [glossary.md](../docs/reference/glossary.md) — mlx-bun serving
|
|
21
|
-
- [library-api.md](../docs/reference/library-api.md) — Library API
|
|
22
|
-
- [memory.md](../docs/reference/memory.md) — Memory — the personal wiki
|
|
23
|
-
- [models.md](../docs/reference/models.md) — Models: supported roster + management (get / scan / ls / gc)
|
|
24
|
-
- [server-api.md](../docs/reference/server-api.md) — Server API
|
|
25
|
-
- [server-config.md](../docs/reference/server-config.md) — Server configuration
|
|
26
|
-
- [training.md](../docs/reference/training.md) — Training / fine-tuning reference
|
|
27
|
-
- [troubleshooting.md](../docs/reference/troubleshooting.md) — Troubleshooting
|
|
28
|
-
|
|
29
|
-
## docs/design/
|
|
30
|
-
|
|
31
|
-
Active engineering design docs — one per topic; status lives in front matter + PLAN.md.
|
|
32
|
-
|
|
33
|
-
- [batching.md](../docs/design/batching.md) — Batching — continuous scheduling for `--batch N`
|
|
34
|
-
- [decode-speed-program.md](../docs/design/decode-speed-program.md) — Inference performance program
|
|
35
|
-
- [dreaming-nightly-pipeline.md](../docs/design/dreaming-nightly-pipeline.md) — The Dreaming — memory write path + synthesis (canonical design)
|
|
36
|
-
- [generic-model-support.md](../docs/design/generic-model-support.md) — Model support — run anything mlx-lm runs, and the ports we target
|
|
37
|
-
- [kv-cache.md](../docs/design/kv-cache.md) — KV cache — residency and layout
|
|
38
|
-
- [orpo-training.md](../docs/design/orpo-training.md) — Training — design (ORPO, flash-CCE head, prefix sharing, segmented backward)
|
|
39
|
-
- [speculative-decoding.md](../docs/design/speculative-decoding.md) — Speculative decoding — one verifier, every draft source
|
|
40
|
-
- [turboquant.md](../docs/design/turboquant.md) — TurboQuant — rotation-based quantization (KV leg landed, weights leg open)
|
|
41
|
-
- [unified-engine-frontier-plan.md](../docs/design/unified-engine-frontier-plan.md) — Engine architecture — four layers, per-scheme oracles, one request path
|
|
42
|
-
- [web-chat-redesign.md](../docs/design/web-chat-redesign.md) — Web chat — the built-in chat surface (canonical design)
|
|
43
|
-
|
|
44
|
-
## docs/planning/
|
|
45
|
-
|
|
46
|
-
Living product/vision docs (USING side).
|
|
47
|
-
|
|
48
|
-
- [PRODUCT_ROADMAP.md](../docs/planning/PRODUCT_ROADMAP.md) — PRODUCT ROADMAP
|
|
49
|
-
- [ResearchTopics.md](../docs/planning/ResearchTopics.md) — Research Topics
|
|
50
|
-
- [public-readiness-handoff.md](../docs/planning/public-readiness-handoff.md) — Public readiness — handoff (2026-09-11)
|
|
51
|
-
- [public-readiness.md](../docs/planning/public-readiness.md) — Preparing mlx-bun for the MLX community
|
|
52
|
-
- [video-series.md](../docs/planning/video-series.md) — mlx-bun video series
|
|
53
|
-
|
|
54
|
-
## docs/archive/
|
|
55
|
-
|
|
56
|
-
Frozen history — read-only, never extended in place; recover raw data via git history.
|
|
57
|
-
|
|
58
|
-
- [hlg-sampling.md](../docs/archive/hlg-sampling.md) — HLG sampling — a piecewise tone curve on the logits
|
|
59
|
-
- [investigations/audio-input-plan.md](../docs/archive/investigations/audio-input-plan.md) — Audio input — design & implementation plan (gemma-4 audio tower)
|
|
60
|
-
- [investigations/benchmark-ledger-through-v0.4.0.md](../docs/archive/investigations/benchmark-ledger-through-v0.4.0.md) — Benchmark ledger through v0.4.0
|
|
61
|
-
- [investigations/chunk-finetune-experiment.md](../docs/archive/investigations/chunk-finetune-experiment.md) — Fine-tuning experiment: chunking task (2026-06-16)
|
|
62
|
-
- [investigations/colibri-glm52-port.md](../docs/archive/investigations/colibri-glm52-port.md) — GLM-5.2 on a 32 GB Mac — native MLX port of the Colibri hierarchy
|
|
63
|
-
- [investigations/colibri-oracle-pin.md](../docs/archive/investigations/colibri-oracle-pin.md) — Colibri oracle pin for the native GLM-5.2 port
|
|
64
|
-
- [investigations/curve-bisector-routing.md](../docs/archive/investigations/curve-bisector-routing.md) — Curve sampler: bisectors, routing, and the falsification
|
|
65
|
-
- [investigations/decode-roofline-lookagain.md](../docs/archive/investigations/decode-roofline-lookagain.md) — Decode perf "look again" — adversarial re-exam of the M=1 floor claim (2026-07-01)
|
|
66
|
-
- [investigations/diffusion-gemma-port.md](../docs/archive/investigations/diffusion-gemma-port.md) — DiffusionGemma port — plan & parity strategy
|
|
67
|
-
- [investigations/dspark-handoff.md](../docs/archive/investigations/dspark-handoff.md) — DSpark — handoff
|
|
68
|
-
- [investigations/expert-offload-single-user-moe.md](../docs/archive/investigations/expert-offload-single-user-moe.md) — Expert offload for single-user local MoE — findings + design sketch
|
|
69
|
-
- [investigations/faithful-l1-consolidation.md](../docs/archive/investigations/faithful-l1-consolidation.md) — Faithful → L1: consolidating the "match mlx-lm" path
|
|
70
|
-
- [investigations/hlg-sampling-investigation.md](../docs/archive/investigations/hlg-sampling-investigation.md) — Investigation: HLG sampling — a piecewise tone curve on the logits
|
|
71
|
-
- [investigations/indexshare-performance-spike.md](../docs/archive/investigations/indexshare-performance-spike.md) — IndexShare / IndexCache performance spike
|
|
72
|
-
- [investigations/kernel-perf-review-2026-07.md](../docs/archive/investigations/kernel-perf-review-2026-07.md) — Kernel/perf deep review — 2026-07-01 (condensed, tracked record)
|
|
73
|
-
- [investigations/lab-build-journal.md](../docs/archive/investigations/lab-build-journal.md) — Build journal — mlx-bun Lab (web UI + native quantize/train/dataset + pi web chat)
|
|
74
|
-
- [investigations/minicpm5-decode-megakernel.md](../docs/archive/investigations/minicpm5-decode-megakernel.md) — MiniCPM5 Decode Megakernel — end-to-end build plan (handoff)
|
|
75
|
-
- [investigations/openjarvis-fit.md](../docs/archive/investigations/openjarvis-fit.md) — OpenJarvis research and fit with mlx-bun
|
|
76
|
-
- [investigations/openwiki-evaluation.md](../docs/archive/investigations/openwiki-evaluation.md) — OpenWiki — evaluation & fit analysis
|
|
77
|
-
- [investigations/optimization_plan.md](../docs/archive/investigations/optimization_plan.md) — mlx-bun: gemma4.ts decompose & optimize — action plan
|
|
78
|
-
- [investigations/orpo-base-uf-experiment-and-directions.md](../docs/archive/investigations/orpo-base-uf-experiment-and-directions.md) — ORPO from-base on UltraFeedback (mixed-5bpw) — results + research directions
|
|
79
|
-
- [investigations/orpo-flash-cce-pin-leak.md](../docs/archive/investigations/orpo-flash-cce-pin-leak.md) — flash-CCE host-buffer pin leak — post-mortem + L3 verification approach
|
|
80
|
-
- [investigations/orpo-uf-testing-handoff.md](../docs/archive/investigations/orpo-uf-testing-handoff.md) — ORPO-on-UF — results + testing-phase handoff
|
|
81
|
-
- [investigations/pareto-specialized-runtime-findings.md](../docs/archive/investigations/pareto-specialized-runtime-findings.md) — Pareto-specialized runtime findings
|
|
82
|
-
- [investigations/pi-builtin-investigation.md](../docs/archive/investigations/pi-builtin-investigation.md) — Investigation: pi as a "built-in" terminal for mlx-bun
|
|
83
|
-
- [investigations/pre-colibri-stabilization.md](../docs/archive/investigations/pre-colibri-stabilization.md) — Pre-Colibri stabilization program
|
|
84
|
-
- [investigations/qwen38-27b-campaign-log.md](../docs/archive/investigations/qwen38-27b-campaign-log.md) — Qwen3.8-27B optimization campaign — execution log (2026-09-04 → 2026-09-08)
|
|
85
|
-
- [investigations/segmented-backward-handoff.md](../docs/archive/investigations/segmented-backward-handoff.md) — Handoff — segmented backward training (2026-06-16)
|
|
86
|
-
- [investigations/starter-model-port-handoff.md](../docs/archive/investigations/starter-model-port-handoff.md) — Handoff: the sub-GB starter model port (publishing gate 1)
|
|
87
|
-
- [investigations/steel-flash-cce-handoff.md](../docs/archive/investigations/steel-flash-cce-handoff.md) — Steel flash-CCE — handoff
|
|
88
|
-
- [investigations/the-dreaming-handoff.md](../docs/archive/investigations/the-dreaming-handoff.md) — THE DREAMING — HANDOFF (2026-06-28)
|
|
89
|
-
- [investigations/trainer-validation-experiment.md](../docs/archive/investigations/trainer-validation-experiment.md) — Trainer validation experiment
|
|
90
|
-
- [planning/chat-ui-vision-memory-first.md](../docs/archive/planning/chat-ui-vision-memory-first.md) — Chat UI Vision: The Model That Knows You
|
|
91
|
-
- [planning/chat-ui-vision.md](../docs/archive/planning/chat-ui-vision.md) — The mlx-bun chat surface: a vision
|
|
92
|
-
- [planning/chunk-task-roadmap.md](../docs/archive/planning/chunk-task-roadmap.md) — Chunk-task quality roadmap (2026-06-17)
|
|
93
|
-
- [planning/curve-sampler-research-plan.md](../docs/archive/planning/curve-sampler-research-plan.md) — Curve sampler — preregistered research plan (distinctness track)
|
|
94
|
-
- [planning/journal.md](../docs/archive/planning/journal.md) — Engineering Journal
|
|
95
|
-
- [planning/memory-docs-and-dag-plan.md](../docs/archive/planning/memory-docs-and-dag-plan.md) — Memory docs organization + DAG-view refresh — review & plan
|
|
96
|
-
- [planning/pi-integration-review.md](../docs/archive/planning/pi-integration-review.md) — pi integration review — internal embeds + `harness pi` (2026-07-01)
|
|
97
|
-
- [planning/release-notes-v0.0.10.md](../docs/archive/planning/release-notes-v0.0.10.md) — mlx-bun v0.0.10 — batching parity, SSD KV cache, sharper serving
|
|
98
|
-
- [planning/release-notes-v0.0.11.md](../docs/archive/planning/release-notes-v0.0.11.md) — v0.0.11 release notes
|
|
99
|
-
- [planning/release-notes-v0.0.12.md](../docs/archive/planning/release-notes-v0.0.12.md) — v0.0.12 release notes
|
|
100
|
-
- [planning/release-notes-v0.0.13.md](../docs/archive/planning/release-notes-v0.0.13.md) — v0.0.13 release notes
|
|
101
|
-
- [planning/release-notes-v0.0.9.md](../docs/archive/planning/release-notes-v0.0.9.md) — v0.0.9 release notes (DRAFT — Josh edits + publishes)
|
|
102
|
-
- [planning/release-notes-v0.2.0.md](../docs/archive/planning/release-notes-v0.2.0.md) — mlx-bun v0.2.0
|
|
103
|
-
- [planning/release-notes-v0.3.0.md](../docs/archive/planning/release-notes-v0.3.0.md) — mlx-bun v0.3.0
|
|
104
|
-
- [planning/release-notes-v0.4.0.md](../docs/archive/planning/release-notes-v0.4.0.md) — mlx-bun v0.4.0
|
|
105
|
-
- [planning/release-notes-v0.5.0.md](../docs/archive/planning/release-notes-v0.5.0.md) — mlx-bun v0.5.0
|
|
106
|
-
- [planning/web-ui-pass-plan.md](../docs/archive/planning/web-ui-pass-plan.md) — Web UI pass — ranked fix list (2026-07-01 audit)
|
|
107
|
-
- [planning/website-readme-pass-plan.md](../docs/archive/planning/website-readme-pass-plan.md) — Website + README pass — audit findings and fix plan (2026-07-01)
|
|
@@ -1,350 +0,0 @@
|
|
|
1
|
-
# HLG sampling — a piecewise tone curve on the logits
|
|
2
|
-
|
|
3
|
-
Status: **implemented, found neutral, superseded by Curve Designer** — `applyHlg` shipped in `src/sampler.ts`; investigation found HLG a wash vs plain temperature; the Curve Designer (`src/curve-sampler.ts`) is its v2 replacement. See [docs/archive/investigations/hlg-sampling-investigation.md](investigations/hlg-sampling-investigation.md) for findings. This doc remains as the design rationale.
|
|
4
|
-
Owner: sampling layer (`src/sampler.ts`)
|
|
5
|
-
Default: **off** (`--hlg-sampling off` = today's temperature sampler, untouched)
|
|
6
|
-
|
|
7
|
-
> This is the canonical **design/rationale** for HLG sampling. Live
|
|
8
|
-
> **status/tracking** (checkboxes, findings) lives in PLAN.md under
|
|
9
|
-
> "Phase 19 — HLG sampling". The user-facing flag/param reference lives in
|
|
10
|
-
> [docs/reference/server-config.md](../reference/server-config.md) and
|
|
11
|
-
> [docs/reference/server-api.md](../reference/server-api.md) once shipped.
|
|
12
|
-
|
|
13
|
-
## Outcome / Retrospective
|
|
14
|
-
|
|
15
|
-
HLG was implemented through **Piece 2** (pure curve + neutrality tests + serial wiring, `applyHlg` live in `src/sampler.ts`). Evaluation found it **neutral vs plain temperature** — no measurable gain in diversity or quality at matched output entropy. As a result, HLG is **superseded by the Curve Designer** (`src/curve-sampler.ts`), which generalizes the same region-shaping idea with an editable control-point interface and cleaner UX. Full empirical findings are in [docs/archive/investigations/hlg-sampling-investigation.md](investigations/hlg-sampling-investigation.md).
|
|
16
|
-
|
|
17
|
-
The sections below (Goal through Correctness) remain as design rationale. Pieces 3–6 of Phasing and the Open Questions were never executed; see the [Phasing](#phasing-the-piece-n-increments) and [Open Questions](#open-questions--risks) sections for historical context.
|
|
18
|
-
|
|
19
|
-
## Goal
|
|
20
|
-
|
|
21
|
-
Add a sampling transform — gated entirely behind `--hlg-sampling on` and a
|
|
22
|
-
per-request `hlg` object — that replaces temperature's single global slope
|
|
23
|
-
with a **pivoted, region-aware contrast curve** on the per-token
|
|
24
|
-
log-probabilities.
|
|
25
|
-
|
|
26
|
-
The move is a **cross-domain transplant**: take a mathematical structure that is
|
|
27
|
-
standard in one field — a piecewise nonlinear transfer function — and apply it
|
|
28
|
-
in one that has never used it. HLG (Hybrid Log-Gamma) is the structure we lift:
|
|
29
|
-
where a plain gamma is one power curve, HLG is **piecewise**, shaping the low,
|
|
30
|
-
middle, and high range of a signal under separate constraints. A logit
|
|
31
|
-
distribution is another such signal. The hypothesis is that shaping its three
|
|
32
|
-
regions independently buys something a single global temperature cannot — and
|
|
33
|
-
that claim is settled by measurement in *this* domain, not by the structure's
|
|
34
|
-
track record in its original one.
|
|
35
|
-
|
|
36
|
-
This is a post-inference, **model-agnostic** transform — it operates on the
|
|
37
|
-
logits vector after the forward pass, so it applies identically to every model
|
|
38
|
-
(MiniCPM5 + the three Gemma-4 quants) and every lane (serial + `--batch N`)
|
|
39
|
-
with **zero per-model code**. It is the first sampling feature in the project
|
|
40
|
-
that has **no oracle ancestor** (neither mlx-lm nor optiq does this), so it is
|
|
41
|
-
gated by KL + quality + diversity, not parity — with one exception that makes
|
|
42
|
-
it safe to ship: see [Correctness](#correctness--evaluation).
|
|
43
|
-
|
|
44
|
-
## Why temperature isn't enough (the thesis)
|
|
45
|
-
|
|
46
|
-
Temperature scales every logit by the same factor: `logit / T`. In tone-curve
|
|
47
|
-
terms it is a **single straight line through the origin** — one global slope.
|
|
48
|
-
Lower `T` steepens it everywhere (sharper, more confident); higher `T` flattens
|
|
49
|
-
it everywhere (more diverse).
|
|
50
|
-
|
|
51
|
-
Its limitation is a **coupling**: the only way temperature loosens the model's
|
|
52
|
-
grip on its top token is by raising `T`, which *simultaneously* lifts the tail.
|
|
53
|
-
You cannot, with one slope, reduce top-token dominance **and** keep a soft floor
|
|
54
|
-
under the tail. Raise the temperature to get novelty and you also import
|
|
55
|
-
low-probability noise; lower it to suppress noise and you re-collapse onto the
|
|
56
|
-
single most likely token. One number cannot move two regions independently.
|
|
57
|
-
|
|
58
|
-
The fix is structural, not a better choice of `T`: make the slope **depend on
|
|
59
|
-
the region of the distribution**. This is precisely what nonlinear transfer
|
|
60
|
-
functions do in signal and color encoding — a single piecewise curve shapes the
|
|
61
|
-
low, middle, and high range under separate constraints. The cross-domain idea
|
|
62
|
-
here is to transplant that apparatus onto a logit distribution, which is just
|
|
63
|
-
another value distribution. Mapped onto the three regions of a token
|
|
64
|
-
distribution, the curve's response is:
|
|
65
|
-
|
|
66
|
-
- **High-probability region** (the model's most confident tokens) →
|
|
67
|
-
**shoulder**: roll the slope off so the top values compress toward their
|
|
68
|
-
neighbours. The model stops being *so fixed* on its favourite answer.
|
|
69
|
-
- **Mid-probability region** (where novelty and interest live) → **gain > 1**:
|
|
70
|
-
steepen the slope. The local-contrast boost separates the
|
|
71
|
-
plausible-and-interesting from the merely-mediocre.
|
|
72
|
-
- **Low-probability tail** → **toe**: gently reduce the slope toward a soft
|
|
73
|
-
floor. Suppress the tail *smoothly* — unlike top-p/top-k, which hard-clip it
|
|
74
|
-
to `-inf`, and unlike high temperature, which inflates it.
|
|
75
|
-
|
|
76
|
-
The payoff is the **decoupling** temperature cannot achieve: roll off the high
|
|
77
|
-
end *and* hold a soft floor under the tail at the same time — a region of
|
|
78
|
-
sampling behaviour no single-knob method reaches. Whether that decoupling is
|
|
79
|
-
*useful* is an empirical question, settled in [Correctness](#correctness--evaluation),
|
|
80
|
-
not by the pedigree of the curve.
|
|
81
|
-
|
|
82
|
-
## The transfer function
|
|
83
|
-
|
|
84
|
-
We operate in **log-probability space**, the natural "stops" axis for a
|
|
85
|
-
probability distribution (the sampler already computes `toLogprobs`,
|
|
86
|
-
[src/sampler.ts:127](../../src/sampler.ts)). Let `ℓ_i` be each token's log-prob
|
|
87
|
-
and `μ` the pivot (middle grey — see [Pivot](#the-pivot--middle-grey)). Define
|
|
88
|
-
the centered coordinate
|
|
89
|
-
|
|
90
|
-
```
|
|
91
|
-
z_i = ℓ_i − μ
|
|
92
|
-
```
|
|
93
|
-
|
|
94
|
-
and remap it with a **C¹ log-toe / linear-mid / log-shoulder** curve. The
|
|
95
|
-
piecewise form is deliberate — it honours HLG's own breakpoints (HLG literally
|
|
96
|
-
switches from a gamma segment to a log segment at E = 1/12):
|
|
97
|
-
|
|
98
|
-
```
|
|
99
|
-
⎧ −m·w − m·β_t·ln(1 + (−z − w)/β_t) z < −w (toe / shadows)
|
|
100
|
-
g(z) = ⎨ m·z |z| ≤ w (gain / mids)
|
|
101
|
-
⎩ m·w + m·β_h·ln(1 + ( z − w)/β_h) z > w (shoulder / highlights)
|
|
102
|
-
```
|
|
103
|
-
|
|
104
|
-
The remapped logits are `ℓ'_i = g(z_i)`. The `+μ` constant shift cancels under
|
|
105
|
-
softmax, so we drop it and feed `g(z_i)` straight to the existing
|
|
106
|
-
`randomCategorical` draw.
|
|
107
|
-
|
|
108
|
-
Properties that matter:
|
|
109
|
-
|
|
110
|
-
- **Slope** is `m` at the pivot and decays as `m·β / (β + d)` a distance `d`
|
|
111
|
-
past each knot — continuous at the knots (C¹), monotone, and always positive.
|
|
112
|
-
- **Monotone ⇒ ranking-preserving.** `g` never reorders tokens: it is a pure
|
|
113
|
-
contrast/tone operation, not a re-ranker. Pairwise orderings and the argmax
|
|
114
|
-
are invariant. (Tested — see [Correctness](#correctness--evaluation).)
|
|
115
|
-
- **`-inf` safe.** A token masked by top-p/top-k (`ℓ_i = -inf`) lands in the toe
|
|
116
|
-
branch and maps to `-inf` — masking survives the remap, no NaN.
|
|
117
|
-
|
|
118
|
-
### Knobs → what each controls
|
|
119
|
-
|
|
120
|
-
| Param | Controls | Effect |
|
|
121
|
-
| --- | --- | --- |
|
|
122
|
-
| `pivot` μ | center of the curve | where the contrast boost is anchored in the distribution (see below) |
|
|
123
|
-
| `gain` m | mid-region slope | `m > 1` sharpens mid-probability tokens; folds temperature (`m = 1/T`) |
|
|
124
|
-
| `width` w | extent of the mid region | how many nats around the pivot count as "mids" |
|
|
125
|
-
| `shoulder` β_h | high-prob rolloff | small = compresses the top tokens fast → top-token diversity |
|
|
126
|
-
| `toe` β_t | low-prob rolloff | small = quick gentle fade of the tail; large = keep more tail |
|
|
127
|
-
|
|
128
|
-
All knobs are in **nats** (the log-prob unit), so they are scale-free across
|
|
129
|
-
models and contexts.
|
|
130
|
-
|
|
131
|
-
### The degeneracies are the safety anchors
|
|
132
|
-
|
|
133
|
-
The curve is a **strict generalization of temperature**, and that is what makes
|
|
134
|
-
a no-oracle path safe to ship — the default and neutral paths reduce *exactly*
|
|
135
|
-
to today's sampler:
|
|
136
|
-
|
|
137
|
-
- `m = 1`, rolloff off (`w → ∞`, or `β → ∞`) ⟹ `g(z) = z` ⟹ **bit-exact the
|
|
138
|
-
current sampler at T = 1**.
|
|
139
|
-
- rolloff off (`w → ∞`), any `m` ⟹ `g(z) = m·z` ⟹ **bit-exact temperature
|
|
140
|
-
`T = 1/m`** (the existing `mulScalar(cur, 1/temperature)` step).
|
|
141
|
-
|
|
142
|
-
So there is a configuration in which HLG is provably identical to temperature,
|
|
143
|
-
down to the bit. That is the parity-equivalent gate (below).
|
|
144
|
-
|
|
145
|
-
### Greedy is untouched
|
|
146
|
-
|
|
147
|
-
Under greedy decoding (`temperature === 0`) the sampler returns `argmax`
|
|
148
|
-
([src/sampler.ts:78](../../src/sampler.ts)). Because `g` is monotone it cannot
|
|
149
|
-
change the argmax, so **HLG is a no-op at temperature 0 by construction** and is
|
|
150
|
-
only engaged for `temperature > 0`. The greedy parity gate is therefore
|
|
151
|
-
trivially preserved.
|
|
152
|
-
|
|
153
|
-
## The pivot
|
|
154
|
-
|
|
155
|
-
Choosing the pivot is the genuinely interesting part: the curve's center has to
|
|
156
|
-
mean something across distributions as different as a near-deterministic next
|
|
157
|
-
token and a wide-open one. Three modes, all cheap, exposed as `pivot_mode`
|
|
158
|
-
(default **top-anchored**). The other two ship as documented opt-in modes —
|
|
159
|
-
losing experiments become flags, never deletions.
|
|
160
|
-
|
|
161
|
-
The pivot is computed from the **full, normalized log-prob vector before any
|
|
162
|
-
top-p/top-k masking**, so masked `-inf` entries never contaminate it.
|
|
163
|
-
|
|
164
|
-
1. **`top` — top-anchored (DEFAULT).** `μ = ℓ_max − c`. The curve centers `c`
|
|
165
|
-
nats below the most-probable token. Predictable, scale-free, one `max`
|
|
166
|
-
reduction. `c` is the meaningful knob; default calibration-pending (Piece 5
|
|
167
|
-
sweep).
|
|
168
|
-
2. **`entropy` — entropy-anchored.** `μ = Σ p_i ℓ_i = −H`, the distribution's
|
|
169
|
-
center of mass in tonal space. Auto-exposure: confident contexts (low
|
|
170
|
-
entropy) pull the pivot up near the top; flat contexts push it down. One
|
|
171
|
-
dot-product, information-theoretically natural, less predictable.
|
|
172
|
-
3. **`median` — mass-median.** `μ` = the log-prob at the 50% cumulative-mass
|
|
173
|
-
boundary (literal auto-levels / histogram median). Most adaptive; reuses the
|
|
174
|
-
sort top-p already performs.
|
|
175
|
-
|
|
176
|
-
`width` may likewise be a fixed nat value or, in a later increment, auto-derived
|
|
177
|
-
from the log-prob spread (an "auto-contrast" mode). Default fixed for now.
|
|
178
|
-
|
|
179
|
-
## Temperature semantics
|
|
180
|
-
|
|
181
|
-
When `hlg` is active, the request's `temperature` **folds into the mid gain**:
|
|
182
|
-
`m = 1 / T`. Temperature *is* the mid-tone contrast, and HLG adds the toe and
|
|
183
|
-
shoulder on top of it. This avoids double-applying a slope and preserves the
|
|
184
|
-
intuition that "temperature" still means "how punchy the mids are." With
|
|
185
|
-
rolloff off, `m = 1/T` and `g(z) = z/T` — exactly temperature, as above.
|
|
186
|
-
|
|
187
|
-
## Where it lands in the code
|
|
188
|
-
|
|
189
|
-
Sampling is fully model-agnostic and shared by every model and lane
|
|
190
|
-
([src/sampler.ts](../../src/sampler.ts), post-inference), so this is a single
|
|
191
|
-
transform with no per-model branches.
|
|
192
|
-
|
|
193
|
-
- **The transform** — a new `applyHlg(lp, params)` in
|
|
194
|
-
[src/sampler.ts](../../src/sampler.ts). HLG is a **replacement sampler**:
|
|
195
|
-
`makeSampler` branches `if (hlg.enabled)` → the curve is the *whole*
|
|
196
|
-
post-logits step (its toe does the tail control), `else` → exactly what we
|
|
197
|
-
used to do (top-p, top-k, then `mulScalar(cur, 1/temperature)`). The two are
|
|
198
|
-
mutually exclusive — when HLG is on, top-p/top-k are **not** applied, because
|
|
199
|
-
the smooth toe is their replacement, not a layer on top of them. Branchless:
|
|
200
|
-
compute the toe / mid / shoulder expressions and select on `z` vs `±w` with
|
|
201
|
-
`where`. Follows the file's strict `owned[]` → `dispose()` discipline (every
|
|
202
|
-
intermediate freed; only the chosen token id crosses to JS).
|
|
203
|
-
- **Sampler options** — extend `SamplerOptions`
|
|
204
|
-
([src/sampler.ts:11](../../src/sampler.ts)) with an optional `hlg` sub-object;
|
|
205
|
-
`makeSampler` builds the HLG step when present, else the existing path
|
|
206
|
-
verbatim.
|
|
207
|
-
|
|
208
|
-
Three sampler **construction** sites pass the params through; the
|
|
209
|
-
degeneracy guarantees keep every existing parity gate green:
|
|
210
|
-
|
|
211
|
-
- **Serial** — `makeSampler(options)` in
|
|
212
|
-
[src/generate.ts:218](../../src/generate.ts).
|
|
213
|
-
- **Batched per-row** — `makeSampler(options)` in
|
|
214
|
-
[src/serve/generation-gateway.ts:127](../../src/serve/generation-gateway.ts).
|
|
215
|
-
Each row already owns its sampler closure, so `hlg` params batch per-request
|
|
216
|
-
for free; the draw site in
|
|
217
|
-
[src/serve/batch-scheduler.ts:276](../../src/serve/batch-scheduler.ts) is
|
|
218
|
-
unchanged beyond construction.
|
|
219
|
-
|
|
220
|
-
### MLX op feasibility
|
|
221
|
-
|
|
222
|
-
All elementwise: `exp`, `sub`, `mul`/`mulScalar`, `where`, plus `log` / `log1p`.
|
|
223
|
-
The pivot needs one `max` (top), one dot-product (entropy), or the existing sort
|
|
224
|
-
(median). Cost is O(V) per token — the same order as the existing top-p step,
|
|
225
|
-
negligible against the forward pass, but measured (Piece 5). If `mlx_log1p`
|
|
226
|
-
isn't already bound, bind it from the header first — read the full signature to
|
|
227
|
-
avoid the trailing-stream-arg footgun (CLAUDE.md).
|
|
228
|
-
|
|
229
|
-
## Flag, config, and API surface
|
|
230
|
-
|
|
231
|
-
Conventions per [docs/reference/server-config.md](../reference/server-config.md)
|
|
232
|
-
and `serverRuntimeFlags()` ([src/cli.ts](../../src/cli.ts)).
|
|
233
|
-
|
|
234
|
-
- **CLI**: `--hlg-sampling on|off` (default **off**), plus `--hlg-width`,
|
|
235
|
-
`--hlg-shoulder`, `--hlg-toe`, and the top-anchor offset `--hlg-pivot-offset`
|
|
236
|
-
(the `c` above). There is **no `--hlg-gain`**: the mid gain folds from
|
|
237
|
-
`--temperature` (`m = 1/T`), so temperature stays the one contrast knob and a
|
|
238
|
-
second one would be redundant. `--hlg-pivot top|entropy|median` arrives with
|
|
239
|
-
the alternate pivot modes in Piece 4 (top-anchored is the only mode until
|
|
240
|
-
then). Parsed with the existing `onOff` / `numFlag` helpers; validated (exit
|
|
241
|
-
on bad value); defaults onto `ServerOptions`.
|
|
242
|
-
- **ServerOptions** ([src/server.ts](../../src/server.ts)): an `hlg?` config
|
|
243
|
-
object, resolved in `toOptions` with the standard precedence
|
|
244
|
-
(request > CLI default > model config > built-in), exactly like the existing
|
|
245
|
-
sampling defaults.
|
|
246
|
-
- **Per-request**: an `hlg` object on `ChatRequest` (unknown fields are already
|
|
247
|
-
tolerated), so it works over OpenAI `/v1/chat/completions`, Anthropic
|
|
248
|
-
`/v1/messages`, and `/v1/responses` with no protocol-specific plumbing.
|
|
249
|
-
Example:
|
|
250
|
-
```jsonc
|
|
251
|
-
{
|
|
252
|
-
"temperature": 0.7, // becomes the mid gain m = 1/T
|
|
253
|
-
"hlg": {
|
|
254
|
-
"enabled": true,
|
|
255
|
-
"pivot": "top", "pivot_offset": 6.0,
|
|
256
|
-
"width": 4.0, "shoulder": 4.0, "toe": 6.0
|
|
257
|
-
}
|
|
258
|
-
}
|
|
259
|
-
```
|
|
260
|
-
|
|
261
|
-
## Correctness & evaluation
|
|
262
|
-
|
|
263
|
-
This is a novel path with **no oracle ancestor**, so it is gated by KL +
|
|
264
|
-
quality + diversity rather than parity — except for the degeneracy gates, which
|
|
265
|
-
*are* parity gates and must be bit-exact.
|
|
266
|
-
|
|
267
|
-
1. **Neutrality gate (bit-exact)** — `tests/hlg-sampling-parity.test.ts`:
|
|
268
|
-
- flag-off ⟹ the sampler code path is byte-identical to today's (`makeSampler`
|
|
269
|
-
unchanged when `hlg` absent);
|
|
270
|
-
- `hlg` in *identity config* (`gain = 1`, rolloff off) reduces to the current
|
|
271
|
-
sampler with `maxDiff === 0`;
|
|
272
|
-
- `hlg` rolloff-off, `gain = 1/T` reduces to temperature `T` bit-exact.
|
|
273
|
-
Run across **all four models** and **both lanes** (serial + batched) — the
|
|
274
|
-
param plumbing differs per lane, the math does not.
|
|
275
|
-
2. **Monotonicity / ranking preservation** — property test over random logits:
|
|
276
|
-
`g` never reorders tokens; argmax invariant; every pairwise ordering held.
|
|
277
|
-
3. **KL characterization** — reuse [src/eval/kl.ts](../../src/eval/kl.ts)
|
|
278
|
-
`evaluateKlSelfFlag` (env lever off vs on) to *characterize* (not pass/fail)
|
|
279
|
-
how far each knob moves the distribution from temperature-only. Output: a
|
|
280
|
-
KL-vs-knob curve per model, recorded for the design log. The distribution is
|
|
281
|
-
*supposed* to move; the point is to map the knob space, not to bound drift.
|
|
282
|
-
4. **Capability guardrail** — `bun scripts/eval.ts capability --candidate e4b`
|
|
283
|
-
(and 12B, the reference target) with HLG on vs off. The six tasks
|
|
284
|
-
(GSM8K / MMLU / IFEval / BFCL / HumanEval / HashHop) are accuracy-shaped, so
|
|
285
|
-
they measure the **cost**: does shaping the distribution break reasoning or
|
|
286
|
-
tool-calling? That is the guardrail HLG must not regress materially.
|
|
287
|
-
5. **Diversity lens (the benefit)** — the six tasks won't show the upside, since
|
|
288
|
-
the whole point is mid-tone novelty. Add a small open-ended eval
|
|
289
|
-
(`src/eval/tasks/diversity.ts`): distinct-n / self-BLEU / cross-sample
|
|
290
|
-
entropy on a creative-prompt set, **HLG vs temperature matched to the same
|
|
291
|
-
output entropy**. The thesis is confirmed iff, at equal entropy, HLG yields
|
|
292
|
-
higher diversity without the tail-garbage temperature would import.
|
|
293
|
-
6. **Perf A/B** — a `bench-compat-vs-perf`-style env-lever A/B plus a
|
|
294
|
-
`./benchmark.sh` row: decode tok/s + TTFT, on vs off, to show ~0 decode
|
|
295
|
-
regression. Lands in [benchmarks/RESULTS.md](../../benchmarks/RESULTS.md) §3.
|
|
296
|
-
|
|
297
|
-
## Phasing (the "piece N" increments)
|
|
298
|
-
|
|
299
|
-
> **Historical / superseded.** Pieces 1–2 shipped. Pieces 3–6 were never executed — HLG was evaluated after Piece 2 and found neutral; the feature was superseded by the Curve Designer before Piece 3 began.
|
|
300
|
-
|
|
301
|
-
1. **Pure curve + neutrality tests** — `applyHlg(lp, params)` in `sampler.ts`,
|
|
302
|
-
top-anchored pivot only; degeneracy + monotonicity tests green across all
|
|
303
|
-
four models. (No flag wired yet — pure, fully-tested function.)
|
|
304
|
-
2. **Serial wiring** — CLI flags + `ServerOptions` + `toOptions`; live on the
|
|
305
|
-
serial server; per-request `hlg`.
|
|
306
|
-
3. **Batched wiring** — gateway/scheduler sampler construction; batched
|
|
307
|
-
neutrality test.
|
|
308
|
-
4. **Pivot modes 2 & 3** — entropy + median, behind `--hlg-pivot`.
|
|
309
|
-
5. **Eval + benchmark** — KL characterization, capability guardrail, diversity
|
|
310
|
-
lens, perf A/B; set shipped defaults from the sweep; RESULTS.md row.
|
|
311
|
-
6. **Docs** — finalize this doc, the investigation write-up, PLAN/STATUS,
|
|
312
|
-
server-config / server-api / README, and a memory note.
|
|
313
|
-
|
|
314
|
-
## Open questions / risks
|
|
315
|
-
|
|
316
|
-
> **Historical / superseded.** These questions were open at the time of design. HLG evaluation concluded with a neutral verdict before they were answered; the Curve Designer supersedes this path. Questions are preserved as design context.
|
|
317
|
-
|
|
318
|
-
- **Default knob values** are calibration-pending — set empirically by the
|
|
319
|
-
Piece 5 sweep, not guessed. Initial exploration points: `c ≈ 6`, `w ≈ 4`,
|
|
320
|
-
`β_h ≈ 4`, `β_t ≈ 6` nats; confirm or replace with measured values.
|
|
321
|
-
- **Pivot for very low-entropy distributions** (one token ~1.0): the curve has
|
|
322
|
-
almost nothing to shape; verify it degrades gracefully to "near-greedy" rather
|
|
323
|
-
than amplifying numerical noise in the tail.
|
|
324
|
-
- **HLG replaces top-p/top-k** (resolved, not a rail). When HLG is on it *is*
|
|
325
|
-
the sampler — no top-p/top-k underneath; the toe is the smooth cousin of
|
|
326
|
-
min-p and does the tail control. Open question is whether the soft toe
|
|
327
|
-
suppresses the tail *enough* on a 262k vocab to match the hard cutoffs, or
|
|
328
|
-
whether garbage leaks — exactly what the `hlg-compare` harness probes.
|
|
329
|
-
- **Does the benefit survive the capability tasks?** If HLG measurably helps
|
|
330
|
-
diversity but dents GSM8K/BFCL, it ships default-off with a documented
|
|
331
|
-
use-case (creative/open-ended), not as a global default — consistent with the
|
|
332
|
-
project's default-off-flags ethos.
|
|
333
|
-
|
|
334
|
-
## Provenance — and why the source domain doesn't justify it
|
|
335
|
-
|
|
336
|
-
This is a **cross-domain transplant**. Nonlinear transfer functions are a mature
|
|
337
|
-
apparatus in signal and color encoding for reshaping a value distribution under
|
|
338
|
-
region-dependent constraints; a logit distribution is just another value
|
|
339
|
-
distribution, so the apparatus carries over. HLG (Hybrid Log-Gamma, ARIB
|
|
340
|
-
STD-B67 / ITU-R BT.2100) is the specific structure we lift — a gamma segment for
|
|
341
|
-
the lower range and a logarithmic segment for the upper range, joined at a
|
|
342
|
-
breakpoint — because its piecewise shape maps cleanly onto "boost the mids, roll
|
|
343
|
-
off the extremes."
|
|
344
|
-
|
|
345
|
-
The provenance is where the *structure* comes from; it is **not** the argument
|
|
346
|
-
for it. That a transfer function is standard practice in another domain says
|
|
347
|
-
nothing about whether it helps token sampling — that is decided only by
|
|
348
|
-
measurement (the KL characterization, the capability guardrail, and the
|
|
349
|
-
diversity lens in [Correctness](#correctness--evaluation)). The full empirical
|
|
350
|
-
write-up is in [docs/archive/investigations/hlg-sampling-investigation.md](investigations/hlg-sampling-investigation.md).
|