Project-Chronos 0.2.1__tar.gz → 0.3.1a0__tar.gz

project_chronos-0.3.1a0/CHANGELOG.md

@@ -0,0 +1,150 @@
+# Changelog
+
+## [Unreleased]
+
+## [0.3.1a0] — 2026-11-07 (beta)
+
+Core half of 0.3.1: cluster-locality training on by default, inference that
+adapts its expert paging to how the checkpoint was trained, and lazy experts in
+the bundle. The rewritten Web UI follows in 0.3.1a1.
+
+### Changed
+- Cluster locality is on by default for new training: `train_chronos*.py --lambda_cluster_locality` defaults to 0.3, the planner and presets set λ = 0.3 for MoE models with more than 2k experts, and the window follows the data (`recommended_window`: power of two nearest the median SFT record, clamped to [16, seq_len]; 32 without record lengths). The `ChronosConfig` class default stays 0.0, so a stored recipe without the field keeps its original objective on resume, and checkpoints written before the field existed load as λ = 0. The λ = 0.3 default comes from the synthetic sweep below (from-scratch training); its effect on real data and on fine-tuning has not been measured.
+- Lazy-expert inference picks its page layout from the checkpoint (`paging="auto"`): weights trained with λ > 0 are paged one training cluster per SSD read, other weights one expert per read. `paging="cluster"|"expert"` forces a layout. The layout is part of the expert-cache directory name (`…_mlx_c2`, `…_c2`), so a cache written for one layout is never reused for the other. Per-expert cache names are unchanged.
+
+### Added
+- `chronos/lazy_experts.py`: one lazy-expert implementation for the Web UI, bundle and server — `LazyOptions` (paging, residency, decay, decode/verify waits, RAM ratio; validated), `page_layout`, `checkpoint_cache_dir`, the expert budget, and `build_mlx_lazy_engine` (trunk only in memory, cache built or reused).
+- Torch offload writes cluster layouts as `cluster_<c>.ctsr` files with a v2 manifest (`ClusterStorage.write_clusters_from_states`); `setup_from_state_dict/_reader(clusters=…)`.
+- MLX `generate(decode_wait_s=0.012, verify_wait_s=0.02, repetition_penalty=1.0)`. The penalty uses the CTRL rule and is rejected together with `speculative=True`, which needs penalty 1.0 to stay exact.
+- `chronos bundle translate|eval --lazy-experts` with `--paging`, `--residency`, `--decay`, `--ram-ratio`, `--decode-wait`, `--verify-wait`. The cache lives outside the bundle (`$CHRONOS_EXPERT_CACHE`, default `~/.cache/chronos/expert_cache`); `eval` also gains `--backend` and `--dtype`.
+- `chronos bundle finetune --[no-]cluster-locality --lambda-cluster --cluster-window --clusters` (on by default; dense models and models with fewer than two clusters stay off). The spec records where each value came from (`extra.cluster_locality_source`); the window comes from the median SFT record length of `--data`.
+- `chronos serve --paging auto|cluster|expert`; `/api/ps` metadata reports the resolved layout.
+- Web UI Inference tab: paging, planner decay and the two prefetch waits, with four-language labels.
+
+- Next-use residency planning for lazy MLX experts (`chronos/residency_planner.py`, `generate(residency="planned")`). The planner records the page-access trace of every forward, keyed by (layer, loop iteration). The previous forward of the same kind, and in speculative decoding the queue of draft traces since the last verify, then predicts when each page is needed next. The store evicts the page needed latest (`MLXExpertStore.set_eviction_ranker`). The running dispatch's pending experts are never evicted. Pages needed early are prefetched only into spare slots. Greedy output is unchanged. The Web UI Inference tab has a switch for it, on by default (MLX lazy only); the stats table shows the eviction policy.
+- Cluster-locality training constraint `lambda_cluster_locality`, with `cluster_locality_clusters` (0 = ⌈E/2k⌉) and `cluster_locality_window`. The loss is 1 − I(window; cluster)/log C over every MoE dispatch, recurrent iterations included. It keeps a stream inside few expert clusters and spreads different streams over different clusters. The partition (`chronos/expert_clusters.py`) is the one lazy inference pages by. Torch and MLX trainers and all stage scripts take it.
+- `tools/bench_residency.py`: LRU window vs planned residency, plain and speculative, on a real checkpoint.
+- `tools/bench_cluster_locality.py`: a tiny-model training A/B of the constraint, reporting held-out CE, cluster entropies, topic purity and lazy-decode cluster reads.
+
+### Measured (reports/residency_20261107/)
+- 50M s13996 checkpoint, 16 FLORES+ dev prompts, greedy, CPU, 12 hot / 12 RAM pages.
+  - Plain decoding: planned residency needs 4438 synchronous SSD loads instead of 6876 (−35%). Throughput is 22.47 vs 23.17 tok/s.
+  - Speculative decoding: 6258 vs 9297 loads (−33%), and 18.96 vs 16.03 tok/s.
+  - All outputs are identical to plain greedy (16/16).
+  - The checkpoint was trained without the constraint and routes near-uniformly over all 36 experts within a stream.
+- Tiny model on synthetic 4-topic streams, 300 steps, λ=0.3 vs 0.
+  - Window cluster entropy falls from 0.49 to 0.005; topic→cluster purity rises from 0.81 to 0.999.
+  - Marginal cluster entropy rises from 0.71 to 0.99, and expert-usage entropy is unchanged (0.87).
+  - Held-out CE is 1.712 vs 1.703.
+  - Cluster-paged lazy decoding needs 0.023 vs 1.48 cluster reads per token.
+
+### Fixed
+- MLX lazy store: an expert evicted by the prefetch thread between an SSD read and promotion raised "could not be loaded on demand". Reads now return the page they loaded.
+
+## [0.3.0] — 2026-11-06
+
+v3 kernel: recurrent-depth MoE. Quality results for the 50M en→zh demo are not
+included yet; they will be added after the 72-hour run and its ablations.
+
+### Added
+- Web UI Inference tab: self-speculative switch, draft depth and γ. All four runner paths use it (Torch / MLX, full DRAM / lazy experts). The stats table shows acceptance, tokens per verify, and verify / draft forwards. Models without a shared KV stream (`per_iteration`, depth 1) decode plainly and say so.
+- `ChronosInferenceEngine.generate_speculative`: self-speculative decoding with lazy experts. Every draft and verify forward runs under the current availability masks, and the lookahead prediction is prefetched before it. `chronos.runtime.speculative.speculative_supported(cfg)` tells callers whether a model can draft.
+- `tools/bench_speculative.py`: plain vs speculative decoding on the real runtimes. Reports wall time, tokens/s, acceptance, identical-output count and SSD loads, with the environment and checkpoint hash.
+- Recurrent-depth execution: prelude → shared recurrent groups looped `recurrent_depth` times with a contractive update → coda.
+- `recurrent_attention_mode`: `shared_first` (default; attention every iteration, K/V from iteration 0), `per_iteration` (ablation), `first_only` (v2 semantics).
+- Tensor-only ACT early exit, exact and batch-invariant; per-token decode exit without KV gaps.
+- Recurrent-depth self-speculative decoding (`chronos/runtime/speculative.py`): greedy-exact, rejection sampling, cache rollback.
+- MLX self-speculative decode loop (`chronos/mlx/speculative.py`), Torch parity: same invariant, rejection sampling with `mx.random` keys, cache truncation for MLA, SWA and int8 entries. `ChronosMLXInferenceEngine.generate(speculative=True, draft_depth=, gamma=)` runs it with the lazy-expert masks; experts each draft routes to are unioned with the lookahead prediction and prefetched before the verify forward. `SlidingWindowAttentionMLX.cache_slack` keeps `window + γ_max + 1` entries for rollback.
+- `chronos bundle build|pack|check|probe|translate|eval|prepare-data|finetune|serve`: a self-contained model bundle (weights + sidecar, tokenizer, FLORES+ dev/devtest, vendored minimind, wheel, source recipe, recommended fine-tune tiers, launcher, setup script, EN/ZH README, NOTICE, SHA-256 file table). `bundle translate` takes `--backend torch|mlx` and `--speculative`.
+- `CHRONOS_MINIMIND_DIR` points Chronos at a local minimind copy (bundles set it to `third_party/minimind`), so no git clone or network access is needed.
+- Stage scripts take `--tokenizer_path`; `load_stage_tokenizer` loads the checkpoint's own tokenizer and refuses one that does not fit `vocab_size`.
+- `chronos longrun` spec fields `init_checkpoint` (start from existing weights, checked against `model_config`) and `export_prefix`.
+- Sorted grouped MoE dispatch: `mx.gather_mm` on MLX (requires mlx >= 0.32.4), `argsort` + `index_add_` on Torch; per-expert reference paths kept for parity tests.
+- Single-dtype precision chain for CPU / CUDA / MPS / MLX with FP32 only in RMSNorm statistics, SDPA softmax, RoPE tables, router softmax, LM-head logits and optimizer masters; bit-exact MLX↔Torch bf16 bridge.
+- `chronos serve`: Ollama/OpenAI protocol server on the native runtime (lazy experts, speculative decoding); optional `/monitor/` mount.
+- `chronos tokenizer`: self-built byte-fallback BPE with language-balanced merges and HF-compatible `tokenizer.json`.
+- `chronos corpus download|build|shards`: pinned downloads, cleaning, global dedup, eval-set leak removal, uint16 shards.
+- `chronos longrun` (alias `run`): time-budgeted resumable MLX training with NaN rollback, validation and translation probes, and run telemetry (`status.json`, `runtime.json`, `parameters.jsonl`).
+- `chronos monitor`: read-only training monitor page + JSON API (loopback by default; key login, session cookie, rate limiting off loopback); Training Monitor tab in the Web UI.
+- `chronos.model.factory.fit_config`: parameter-budget configuration search.
+- Optional extras `serve` and `translate` with pinned versions.
+- Inference and memory options, all off by default: `kv_cache_dtype="int8"` (MLA `c_kv` and SWA k/v, 32-element groups), `mla_decode="absorbed"|"auto"` with optional `mla_cache_rope`, `router_scope="shared"`, the `draft_exit_kl_coef` / `draft_exit_depth` training term, and hard expert budgets `max_resident_experts` / `ram_budget_mb` with prediction-protected LRU and per-virtual-layer prefetch. Torch and MLX both implement them.
+- Adaptive training planner (`chronos.model.planner`, Config tab → "Auto plan"). Inputs: target size, MoE on/off, experts, and either active experts or active parameters, plus a memory-saving level and datasets. Outputs: a v3 architecture, scaling-law LR/batch, a memory fit, and independent pretrain / SFT / DPO / ORPO / GRPO / distill values. Dataset profiling supplies vocabulary, sequence length, token count and tokenizer path.
+- Per-stage overrides (`config["stage_overrides"]`); the Train tab merges the selected stage's values.
+- Five v3 presets generated by the planner (`ui/presets_v3.json`, `python -m ui.presets`) replace the old presets; old names are aliases.
+- Tokenizer and Serve tabs in the Web UI; the Train tab honours `tokenizer_path`.
+
+### Changed
+- Routing defaults to top-2 plus a shared expert (top-1 with `norm_topk_prob=True` gave the gate no CE gradient).
+- `--dtype auto` resolves to fp32 on MPS (native bf16 measured slower there).
+- `lookahead_source` now comes from the prelude output.
+- Repetition penalty divides positive logits and multiplies negative ones.
+- `gradio` is constrained to `>=5.0,<6`; `tokenizers` is a declared dependency.
+- `setup.py` is a shim; metadata lives only in `pyproject.toml`.
+- `mla_decode` defaults to `expanded`. With `auto`, a bf16 prefill took the expanded path and the following decode took the absorbed path. Prefill+decode then differed from the full forward pass by 0.06 in max logit (measured). `absorbed` and `auto` are opt-in.
+- Minimum torch is 2.6: transformers 5 disables its PyTorch backend below that version, which broke imports.
+- CI installs torch 2.10.0 (CPU) and pins the rest in `.github/ci-constraints.txt`, tested on Python 3.10 and 3.11. The dead token-based publish job was removed; releases publish through `pypi.yml` (trusted publishing).
+- The Web UI no longer assumes a task.
+  - The monitor (page and Gradio tab) discovers evaluation metrics from whatever numeric keys a run logs on validation / final events. Keys are grouped by metric name (`probe_en_zh_bleu`, `qa_accuracy`, `pass1` …) and charted per family, with an improvement direction for known metrics.
+  - Counts and durations are listed as context and not charted.
+  - Samples show input / reference / output for any schema (`src`/`prompt`/`question`, `ref`/`answer`, `hyp`/`response`/`prediction`). Other fields appear as tags.
+  - Offline inspection reports are flattened to every numeric leaf.
+- Data tab: scans a file, directory or glob of jsonl / json / parquet / csv / tsv / txt. Reports record shapes (chat, preference, instruction, prompt–response, parallel pair, plain text), field presence and types, scripts, duplicates and per-record character lengths over all text fields.
+
+### Fixed
+- The Inference tab loaded the minimind tokenizer for every checkpoint. It now uses the checkpoint's own tokenizer: the config `tokenizer_path`, the bundle's `tokenizer/` folder, or the sidecar path. Minimind is the last resort.
+- MLX inference ran in training mode when the model was converted from a freshly built torch model (the Inference tab did this). MoE then masked experts at random (`fallback_mask_prob`), so repeated greedy runs disagreed. `ChronosMLXInferenceEngine` now puts its model in eval mode.
+- The Torch lazy engine failed with "Inference tensors cannot be saved for backward" when a plain request followed a speculative one, because experts paged in under `torch.inference_mode` were reused. The decode paths now use `torch.no_grad`.
+- The v2 every-iteration attention switch rewrote a single KV slot per iteration (prefill+decode vs full forward gap 0.94).
+- CLI registered `serve`, `tokenizer` and `corpus` twice.
+- Monitor CPU time on macOS: `rusage` times are Mach ticks, now scaled by `mach_timebase_info`.
+- `apply_inference_overrides` validated overrides against a config that used class-default KV heads (4). Every serve/inference override on models with other head counts (e.g. the 50M demo, 6/2) raised.
+- The UI estimator ignored `recurrent_groups` and `recurrent_attention_mode`; its counts now match `count_parameters`.
+- Planner/dataset profiling counted GRPO prompt files (empty assistant turn) as SFT.
+- Stage scripts loaded the minimind tokenizer (6,400 entries) for checkpoints trained with another vocabulary.
+- A forward with `recurrent_depth` below `min_recurrent_depth` (a speculative draft) gave the last state zero ACT weight on MLX, so the coda saw zeros (max logit gap to Torch 16.6 on the 50M checkpoint, measured; draft acceptance 2 % instead of 45 %). Torch special-cased depth 1 but had the same fault for 2 ≤ depth < `min_recurrent_depth`. The minimum now clamps to the executed depth; training depths are unaffected.
+- MLX ≥ 0.32 binds a lazy `mx.load` to the loading thread's stream; experts prefetched by the background thread raised "There is no Stream(cpu, N) in current thread" when promoted on the decode thread. Pages are now evaluated in the thread that loads them.
+
+### Known gaps
+- Self-speculative decoding is exact but **not faster** for the 50M demo, on CPU, GPU or lazy experts. All figures are measured on the pretrain step-13,996 checkpoint, greedy, batch 1, FLORES+ dev en→zh, draft depth 1, γ 4. The 72-hour run was paused (SIGSTOP) for the GPU runs. Reports are in `reports/speculative/`.
+
+  | Runtime | Plain tok/s | Speculative tok/s | Ratio | Acceptance | Identical |
+  |---|---|---|---|---|---|
+  | MLX GPU bf16, all experts in DRAM | 112.0 | 107.4 | 0.96× | 42 % | 22/24 |
+  | MLX GPU bf16, lazy experts (12 of 216 pages) | 43.7 | 32.0 | 0.73× | 42 % | 22/24 |
+  | Torch MPS fp32, all experts | 32.1 | 25.9 | 0.81× | 40 % | 12/12 |
+  | Torch MPS fp32, lazy experts, sync misses | 17.0 | 14.7 | 0.86× | 40 % | 12/12 |
+
+  Draft depth 2 raised acceptance to 55–59 % but did not change the result (0.72–0.92×).
+
+  At this size one full-depth step costs about as much as a batched verify pass, so drafting at depth 1 of 4 does not save enough.
+
+  With lazy experts, drafts and verify passes route to different experts. Speculative decoding then read 1.4–1.6× more expert pages from SSD (12,092 vs 8,372 for MLX).
+
+  bf16 on the GPU flips near-tie tokens (the 2/24 and 3/24 mismatches). In GPU fp32 one sentence differed, at a top-1/top-2 logit margin of 0.0006. CPU fp32 matches exactly. The switch therefore stays off by default.
+- At a 1280 px window, Gradio moves the Auto-Tune and IO Monitor tabs into its "…" overflow menu.
+- Planner peak-memory figures are estimates; for the 50M recipe the estimate (16.4 GB) is about twice the measured MLX peak (8.72 GB).
+- Gradio 5.35 does not render the LinePlot legend; colour keys are written into plot titles instead.
+
+## [0.1.0] — 2026-04-20
+
+### Added
+- `ChronosConfig`: extends `MiniMindConfig` with lookahead, shared experts, λ1/λ2, hybrid attention params
+- `LookaheadRouter`: lightweight dense predictor inserted after block 0, outputs [B, S, K+1, E] routing probs
+- `TemporalLocalityLoss`: L_total = L_CE + λ1·L_balance + λ2·Σ‖E_t − E_{t-1}‖²
+- `ChronosMOEFeedForward`: shared experts always in VRAM + soft gating fallback on cache miss
+- `MLAAttention`: Multi-head Latent Attention — KV cache stores latent vectors (8-16x smaller)
+- `SlidingWindowAttention`: KV cache capped at `window_size` tokens, O(1) memory per decode step
+- Hybrid attention: even layers → MLA, odd layers → SlidingWindow
+- `ExpertStore`: three-tier VRAM/RAM/SSD storage with dynamic pinned memory safety limit
+- `AsyncPrefetcher`: background thread prefetches SSD→RAM driven by LookaheadRouter predictions
+- `CacheManager`: unified cache interface for inference engine
+- `ChronosInferenceEngine`: end-to-end decode loop with async prefetch + soft gating
+- `cluster_layout.py`: co-occurrence matrix + Louvain/greedy clustering for sequential SSD layout
+- `ChronosAutoTuner`: extends Optuna TPE search with λ1/λ2/lookahead_steps dimensions
+- `chronos eval`: Phase 1 validation — t+1/t+2 lookahead accuracy + LRU cache hit rate
+- `chronos benchmark`: Phase 3 — PPL, TPS, KV cache memory, layer-wise analysis
+- `chronos export`: expert cluster layout generation
+- 8-test smoke suite (`tests/test_smoke.py`)
+- GitHub Actions CI (Python 3.10/3.11, lint + tests)
+- PyPI packaging (`pyproject.toml`, `MANIFEST.in`)

project_chronos-0.3.1a0/MANIFEST.in

@@ -0,0 +1,13 @@
+include README.md README_zh.md LICENSE THIRD_PARTY_NOTICES.md CHANGELOG.md requirements.txt
+recursive-include chronos/monitor/static *.html *.js *.css
+include ui/event_contract.js
+global-exclude __pycache__ *.py[cod] .DS_Store
+prune runs
+prune out
+prune expert_cache
+prune expert_cache_mlx
+prune results
+prune reports
+prune .archive
+prune Translation?Training?Data
+prune chronos/mlx/training/recipes

{project_chronos-0.2.1/Project_Chronos.egg-info → project_chronos-0.3.1a0}/PKG-INFO

@@ -1,37 +1,50 @@
 Metadata-Version: 2.4
 Name: Project_Chronos
-Version: 0.2.1
-Summary: On-device low-latency lookahead dual-layer MoE inference — Lookahead Routing + Async Prefetch + Hybrid Attention
+Version: 0.3.1a0
+Summary: Storage-aware recurrent-depth MoE: lookahead expert prefetch, SSD-lazy experts, hybrid MLA/SWA attention, self-speculative decoding
 License-Expression: Apache-2.0
-Keywords: llm,moe,inference,lookahead-routing,hybrid-attention,edge-ai
+Project-URL: Homepage, https://github.com/FonaTech/Project_Chronos
+Project-URL: Issues, https://github.com/FonaTech/Project_Chronos/issues
+Keywords: llm,moe,inference,lookahead-routing,hybrid-attention,recurrent-depth,speculative-decoding,mlx,edge-ai
 Classifier: Development Status :: 3 - Alpha
+Classifier: Operating System :: OS Independent
 Classifier: Intended Audience :: Science/Research
 Classifier: Programming Language :: Python :: 3.10
 Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
 Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
 Requires-Python: >=3.10
 Description-Content-Type: text/markdown
 License-File: LICENSE
-Requires-Dist: torch>=2.4.0
+Requires-Dist: torch>=2.6.0
 Requires-Dist: transformers>=4.40.0
 Requires-Dist: datasets>=2.18.0
 Requires-Dist: safetensors>=0.4.0
 Requires-Dist: optuna>=3.6.0
 Requires-Dist: numpy>=1.24.0
 Requires-Dist: psutil>=5.9.0
+Requires-Dist: tokenizers>=0.19.0
 Requires-Dist: tqdm>=4.66.0
-Requires-Dist: gradio>=4.0.0
+Requires-Dist: gradio<6,>=5.0
 Provides-Extra: cluster
 Requires-Dist: python-louvain>=0.16; extra == "cluster"
 Requires-Dist: networkx>=3.0; extra == "cluster"
 Provides-Extra: mlx
-Requires-Dist: mlx>=0.12.0; extra == "mlx"
+Requires-Dist: mlx>=0.32.4; extra == "mlx"
+Provides-Extra: serve
+Requires-Dist: fastapi==0.115.9; extra == "serve"
+Requires-Dist: uvicorn==0.34.3; extra == "serve"
+Provides-Extra: translate
+Requires-Dist: sacrebleu==2.6.0; extra == "translate"
+Requires-Dist: sacremoses==0.2.0; extra == "translate"
 Provides-Extra: dev
 Requires-Dist: pytest>=7.0; extra == "dev"
 Requires-Dist: black>=24.0; extra == "dev"
 Requires-Dist: ruff>=0.4.0; extra == "dev"
+Requires-Dist: httpx>=0.27; extra == "dev"
+Requires-Dist: fastapi==0.115.9; extra == "dev"
+Requires-Dist: uvicorn==0.34.3; extra == "dev"
 Dynamic: license-file
-Dynamic: requires-python
 
 # Project Chronos (Experimental)
 
@@ -95,6 +108,81 @@ Under a simulated 30 ms SSD delay, the new path keeps **35 ms+ per token** of pi
 
 ---
 
+## v3 kernel: recurrent-depth MoE (0.3.0)
+
+v3 reuses a few physical layers many times. A prelude runs once, each shared recurrent group loops `recurrent_depth` times with a contractive update `h ← h + dt·(block(h + e_loop) − h)`, and a coda runs once. The 50M demo has 6 physical layers that execute as 18 virtual layers (prelude 1, 4 recurrent layers × depth 4, coda 1).
+
+```mermaid
+flowchart LR
+    X["tokens"] --> P["Prelude<br/>runs once"]
+    P --> G["Recurrent group<br/>MLA / SWA + top-2 MoE"]
+    G -->|"loop d = 1..D<br/>contractive step, ACT halt"| G
+    G --> C["Coda<br/>runs once"]
+    C --> H["LM head"]
+    P -. "lookahead source" .-> LR["LookaheadRouter<br/>expert prefetch"]
+```
+
+**What changed from v2**
+
+| | v2 | v3 |
+|---|---|---|
+| Attention inside the loop | Iteration 0 only (`first_only`) | Every iteration |
+| KV cache per recurrent layer | 1 slot; the every-iteration attention switch rewrote it each iteration (measured prefill+decode vs full-forward gap 0.94) | `shared_first`: Q recomputed each iteration, K/V from iteration 0, so KV slots = physical layers |
+| Routing | top-1 (gate received no CE gradient) | top-2 + 1 shared expert |
+| MoE dispatch | Python loop over experts with `.item()` syncs | Sorted grouped dispatch: `gather_mm` on MLX, `argsort` + `index_add_` on Torch |
+| Early exit | Host sync every step | Tensor-only ACT, exact and batch-invariant; decode can exit per token without KV gaps |
+| Decoding | Plain autoregressive | Recurrent-depth self-speculative decoding |
+
+`per_iteration` (one KV slot per virtual layer) is kept as an ablation. Old v2 checkpoints load as `first_only`; their logits match a stored golden tensor within 1e-5.
+
+**Self-speculative decoding (no extra parameters).** The same model drafts γ tokens at a shallow depth and verifies them with one full-depth batched forward pass. Greedy output is token-for-token identical to full-depth greedy. Sampling uses rejection sampling, which preserves the full-depth distribution. After a rejection, the cache is truncated: MLA slices `c_kv`, and the sliding-window cache keeps `window + γ_max` entries. Torch (`chronos/runtime/speculative.py`) and MLX (`chronos/mlx/speculative.py`, `ChronosMLXInferenceEngine.generate(speculative=True)`) share the algorithm. Tests cover greedy identity, the rejection distribution and rollback against a fresh prefill on both backends. The Web UI Inference tab has a switch for it, with draft depth and γ.
+
+For the 50M demo it is exact but not faster on any runtime we measured (pretrain step-13,996 checkpoint, greedy, batch 1, FLORES+ dev en→zh, draft depth 1, γ 4; reports in `reports/speculative/`):
+
+| Runtime | Plain tok/s | Speculative tok/s | Ratio | Acceptance |
+|---|---|---|---|---|
+| MLX GPU bf16, all experts in DRAM | 112.0 | 107.4 | 0.96× | 42 % |
+| MLX GPU bf16, lazy experts (12 of 216 pages resident) | 43.7 | 32.0 | 0.73× | 42 % |
+| Torch MPS fp32, all experts | 32.1 | 25.9 | 0.81× | 40 % |
+| Torch MPS fp32, lazy experts | 17.0 | 14.7 | 0.86× | 40 % |
+
+At this size a full-depth step costs about as much as a batched verify pass. With lazy experts, drafts and verify passes also route to different experts, so speculative decoding read 1.4–1.6× more expert pages from SSD. Greedy outputs were identical in CPU fp32 and Torch MPS fp32. In GPU bf16, near-tie tokens flipped in 2 of 24 sentences. The switch therefore stays off by default.
+
+**Precision chain.** One compute dtype runs end to end. FP32 is used only where it pays: RMSNorm statistics, SDPA softmax, RoPE tables, router softmax, LM-head logits and optimizer master weights. The MLX↔Torch bf16 bridge is bit-exact. `auto` picks bf16 on MLX GPU and CUDA, and fp32 on CPU and MPS (MPS bf16 measured slower; see below).
+
+**Tooling added in v3**
+
+| Command | What it does |
+|---|---|
+| `chronos tokenizer` | Self-built byte-fallback BPE with script-aware pre-tokenization and language-balanced merges; writes an HF-compatible `tokenizer.json` + chat template |
+| `chronos corpus download / build / shards` | Pinned downloads, cleaning, global dedup, eval-set leak removal, uint16 shards |
+| `chronos longrun --spec recipe.json` (alias `run`) | Time-budgeted MLX run: warmup + cosine per phase wall-clock budget, NaN rollback, bit-exact resume, validation and translation probes |
+| `chronos serve` | Ollama/OpenAI protocol server on the native runtime (see below) |
+| `chronos monitor` | Read-only training monitor page + JSON API (see below) |
+
+`chronos.model.factory.fit_config` searches for a configuration that matches total and active parameter targets.
+
+**Status of the 50M en→zh demo (in progress).**
+- Model: 49.69M total / 12.09M active parameters, 36 experts, MLA latent 128, SWA window 128, 16k self-built vocabulary.
+- Run: a 72-hour MLX bf16 run (pretrain, then SFT) started on 2026-11-05.
+- Interim measurement: valid CE 2.477 at pretrain step 13,996 of about 24.7k planned (it was 3.012 at step 1,656). That checkpoint is packaged as `chronos-enzh-50m-pretrain-s13996` (see Model bundle).
+- BLEU/chrF on FLORES+ and WMT22, and the ablations (attention mode, depth, lazy-expert budget, speculative γ), will be published after the run finishes. No quality numbers are claimed before then.
+
+**Inference and memory options (all off by default).** Each option is a config field and a `chronos serve` / Inference-tab override. Tests compare each one with its reference path on Torch and MLX.
+
+| Field | Values (default first) | Effect |
+|---|---|---|
+| `kv_cache_dtype` | `auto`, `int8` | `int8` stores MLA `c_kv` and SWA k/v per token in groups of 32 with a scale; rollback and speculative decoding work on it |
+| `mla_decode` | `expanded`, `absorbed`, `auto` | `absorbed` scores against `c_kv` without expanding K/V. `auto` absorbs only short cached chunks; it is opt-in because a bf16 prefill (expanded) followed by absorbed decode drifted from the full forward pass (max logit gap 0.06, measured) |
+| `mla_cache_rope` | `false`, `true` | Also caches the roped key, trading memory for decode speed |
+| `router_scope` | `per_iteration`, `shared` | `shared` reuses iteration 0's experts for the rest of the loop, so fewer experts are resident per token |
+| `draft_exit_kl_coef`, `draft_exit_depth` | `0`, `1` | Training term: KL from the shallow exit to the detached full-depth logits, which raises speculative acceptance |
+| `max_resident_experts`, `ram_budget_mb` | unset | Hard caps on the fast tier and the RAM tier. LRU eviction never evicts experts the lookahead router predicts. Prefetch covers every loop iteration of a virtual layer |
+
+Web UI: the Tokenizer tab (extract → train → evaluate) and the Serve tab (start/stop `chronos serve`; the API key goes through the environment, never argv).
+
+---
+
 ## Three-tier storage (M1: cluster-aware safetensors)
 
 ```mermaid
@@ -224,7 +312,7 @@ flowchart LR
 
 Training dtype and resource policy:
 
-- `--dtype auto` is the default. MPS/MLX resolve to BF16-first for training stability, CUDA/XPU resolve to FP16, and CPU resolves to FP32 unless `--dtype float16` or `--dtype bfloat16` is set explicitly.
+- `--dtype auto` is the default. One compute dtype covers weights, activations, KV cache and expert pages; FP32 is kept only inside fused kernels (SDPA softmax, RMSNorm statistics), for RoPE tables, router softmax, loss logits and the optimizer's FP32 masters/moments. `auto` resolves to BF16 on MLX GPU, BF16 on CUDA (FP16 if unsupported, with loss scaling), and FP32 on CPU and MPS. On MPS, native BF16 measured slower than FP32 (Chronos 50M on an M5 Max with torch 2.10: 0.749 vs 0.619 s per training step, 49.1 vs 37.3 ms per decoded token). Pass `--dtype bfloat16` explicitly to trade speed for half the weight and KV memory.
 - CPU training configures PyTorch to use physical cores by default. Override with `--cpu_threads` or `--cpu_budget_percent`.
 - On macOS, MPS/MLX training forces DataLoader workers to `0` by default to avoid Metal command-buffer crashes from multiprocessing. CPU/CUDA still use worker processes; advanced users can override the guard with `CHRONOS_ALLOW_METAL_DATALOADER_WORKERS=1`.
 - Native MLX training pushes UI logs, scalar readouts, and chart points every `log_interval` steps, and Web UI Stop is checked at each batch boundary.
@@ -235,7 +323,7 @@ The six-stage MLX trainer mirrors the PyTorch loss stack in `chronos.mlx.*`:
 masked CE, DPO/ORPO/GRPO/distillation losses, load balance, temporal locality,
 lookahead soft-target supervision, lookahead top-k hit loss, and router KL
 anchor. Numerically sensitive pieces run in FP32 even when model weights use
-BF16/FP16, which is why `auto` prefers BF16 on MLX/MPS: FP16 has too little
+BF16/FP16, which is why `auto` prefers BF16 over FP16 on MLX GPU: FP16 has too little
 exponent range for router softmax, CE, and Adam moments on small unstable
 training runs.
 
@@ -406,6 +494,13 @@ pip install -e ".[dev]"
 pip install "Project_Chronos[mlx]"
 ```
 
+**Optional extras** (versions pinned):
+
+```bash
+pip install "Project_Chronos[serve]"      # chronos serve / chronos monitor (fastapi, uvicorn)
+pip install "Project_Chronos[translate]"  # corpus detokenization + BLEU/chrF probes (sacremoses, sacrebleu)
+```
+
 **vLLM serving (optional, Linux + CUDA only):**
 
 ```bash
@@ -413,16 +508,17 @@ pip install vllm
 ```
 
 > **minimind dependency**: Project Chronos uses [minimind](https://github.com/jingyaogong/minimind) as its MoE kernel.
-> If it is not found locally, Chronos clones it automatically into `~/.cache/chronos/minimind-master/` on first import.
+> If it is not found locally, Chronos clones it automatically into `~/.cache/chronos/minimind-master/` on first import (this needs `git` and network access).
+> Set `CHRONOS_MINIMIND_DIR=/path/to/minimind` to use a local copy instead; model bundles ship one under `third_party/minimind` and their launcher sets the variable.
 > minimind is licensed under **Apache-2.0**. See [THIRD_PARTY_NOTICES.md](THIRD_PARTY_NOTICES.md) for attribution details.
 
-**Requirements**: Python 3.10+, PyTorch 2.4+
+**Requirements**: Python 3.10+, PyTorch 2.6+
 
 ---
 
 ## Quick start
 
-### Web UI (M6: 8 tabs, 4 languages)
+### Web UI (12 tabs, 4 languages)
 
 ```bash
 chronos-ui
@@ -432,17 +528,127 @@ python chronos_app.py
 
 Tabs included:
 
-- `Config` with a live parameter / memory estimator merged in from the old Designer
-- `Train` with its own `data_path`
+- `Config`: v3 presets, the auto planner (below) and a live parameter / memory estimator
+- `Data` for dataset scanning and conversion
+- `Tokenizer`: extract text from datasets, train the Chronos BPE, evaluate and save
+- `Train` with its own `data_path`; applies the selected stage's overrides
+- `Training Monitor` (read-only view of `runs/`; see below)
 - `6-Stage Pipeline` with per-stage dataset paths
 - `Inference`
 - `Export` for FP16/Q8_0 safetensors and GGUF deployment artifacts
+- `Serve`: start and stop `chronos serve` and view its log
 - `Benchmark` with Markdown table + comparison plots
 - `Auto-Tune` with persistent logs and one-click `Apply Best -> Config`
 - `IO Monitor`
 
+At a 1280 px wide window, Gradio moves the last tabs (Auto-Tune, IO Monitor) into its "…" overflow menu.
+
+### Presets and the auto planner
+
+The Config tab ships five v3 presets. Each one is generated by the planner and stored in `ui/presets_v3.json`; regenerate the file with `python -m ui.presets`. Parameter counts are measured with a meta-device model; peak memory is an estimate.
+
+| Preset | Total / active params (measured) | Shape | Memory level | Est. training peak |
+|---|---|---|---|---|
+| Chronos-v3 Tiny (smoke / CI) | 1.51M / 1.29M | E4 top-2, vocab 6400, seq 128, depth 2 | none | 2.13 GB |
+| Chronos-v3 50M (en→zh recipe) | 49.69M / 12.09M | h384, L6, E36 top-2, vocab 16384, seq 512 | none | 16.4 GB (the live run measures 8.72 GB peak) |
+| Chronos-v3 200M (32 GB) — default | 200.72M / 28.20M | h576, E32 top-2 | balanced | 10.14 GB |
+| Chronos-v3 500M (32 GB) | 499.53M / 61.59M | E48 top-3 | aggressive | 9.26 GB |
+| Chronos-v3 1B (64 GB) | 1002.17M / 99.74M | E64 top-4, seq 2048 | aggressive | 18.92 GB |
+
+The old preset names (Recommended-CN, Tiny/Small/Medium/Large) still load; they map to the closest v3 tier.
+
+**Auto planner** (Config tab → "Auto plan"). Inputs:
+- target total parameters, and whether to use MoE;
+- the expert count, plus either the number of active experts or an active-parameter target;
+- a memory-saving level: `none`, `balanced`, `aggressive` or `extreme`;
+- optional dataset paths, RAM, token budget, vocabulary size and sequence length.
+
+`chronos.model.planner.plan_training` then does the following:
+1. **Profiles the datasets.** It reads the shard manifest or samples the files, classifies records as pretrain, SFT, preference or RL, and reads the tokenizer report. The vocabulary size, sequence length (p95 / cap), token count and tokenizer path come from the data when available.
+2. **Searches the architecture.** It looks for the hidden size, layer count, expert width and GQA heads whose meta-device count is closest to the target.
+3. **Sets LR and batch size** from the DeepSeek LLM compute-optimal fits (η = 0.3118·C^−0.125, B = 0.2920·C^0.3271), capped for wide models.
+4. **Fits memory.** It picks micro-batch, accumulation, int8 KV, router scope, activation checkpointing, optimizer paging and an expert RAM budget for the memory level.
+5. **Writes per-stage values.** Pretrain, SFT, DPO, ORPO, GRPO and distill each get their own learning rate, batch, epochs, save interval and stage-specific terms (β, temperature, α).
+
+The stage table is editable; the Train tab merges the selected stage's row over the shared values, and the pretrain row stays in sync with the shared widgets. "Save plan" writes the config, stage values and report to JSON. Every number in the report is labelled measured or estimated.
+
 Built-in i18n: `zh-Hans`, `zh-Hant`, `en`, `ja`
 
+### Training monitor (console tab + standalone page/API)
+
+A `chronos longrun` run directory holds `run_spec.json`, `metrics.jsonl`,
+`status.json` (heartbeat), `runtime.json` (precision policy, device, versions,
+code hash), `parameters.jsonl` (per-tensor weight/grad/update statistics and
+per-virtual-layer routing), `samples.jsonl`, `inspections/` and checkpoints.
+The console's **Training Monitor** tab and a standalone page both read these
+files and never touch the training device. Runs started before telemetry
+existed still show progress; their status is derived from `metrics.jsonl` and
+the process table.
+
+```bash
+chronos monitor --runs-dir runs                    # http://127.0.0.1:7870/
+chronos serve --models-dir exports --monitor-runs-dir runs   # same page at /monitor/
+```
+
+JSON API (read-only): `GET /api/state?run=&history=1`, `/api/metrics?run=&phase=&since=`,
+`/api/parameters`, `/api/tensors`, `/api/tensor?name=`, `/api/log?bytes=`, `/api/runs`.
+
+Security: the default bind is 127.0.0.1 with no authentication. Any other
+address requires `--allow-remote` and `--api-key` (or `CHRONOS_MONITOR_KEY`).
+Browsers sign in once and get an HttpOnly, SameSite=Strict session cookie;
+scripts send `Authorization: Bearer <key>`. Failed logins are rate-limited.
+Off loopback, add `--ssl-certfile/--ssl-keyfile` or a TLS reverse proxy,
+otherwise the key crosses the network in clear text.
+
+### Ollama / OpenAI protocol server
+
+`chronos serve` runs the native Chronos runtime behind the Ollama and OpenAI HTTP APIs. The kernel is not converted to another architecture, so recurrent depth, SSD-lazy experts with lookahead prefetch, compressed KV and self-speculative decoding stay active.
+
+```bash
+pip install "Project_Chronos[serve]"
+chronos serve --models-dir ./exports --port 11435        # 11434 would collide with a running Ollama
+OLLAMA_HOST=127.0.0.1:11435 ollama list
+OLLAMA_HOST=127.0.0.1:11435 ollama run <model-name> "Hello"
+```
+
+- Endpoints: `/api/version`, `/api/tags`, `/api/show`, `/api/ps`, `/api/pull`, `/api/generate` and `/api/chat` (streaming NDJSON), `/v1/models`, `/v1/chat/completions` (SSE).
+- Models are `<name>.pth` + `<name>.config.json` pairs found under `--models-dir`.
+- Chronos options go in the request `options`: `chronos_speculative`, `chronos_draft_depth`, `chronos_gamma`, `chronos_miss_policy`.
+- Stock `ollama` / llama.cpp cannot execute Chronos GGUF files (MLA+SWA hybrid layers and the contractive recurrence have no stock equivalent); use this server instead.
+- Security: binds 127.0.0.1 without authentication by default. Other addresses require `--allow-remote` and `--api-key`; clients then send `Authorization: Bearer <key>`.
+
+### Model bundle (ready-to-run pretrained model)
+
+`chronos bundle build` packages one exported checkpoint of a `chronos longrun` run into a folder that works right after unpacking. Building only reads the run directory.
+
+```bash
+chronos bundle build --run-dir runs/enzh_50m_v1 --which best --wheel dist/project_chronos-0.3.0-py3-none-any.whl --pack
+```
+
+| Bundle path | Content |
+|---|---|
+| `model/` | weights + `.config.json` sidecar, tokenizer (`model/tokenizer/`) |
+| `eval/` | FLORES+ dev/devtest (CC BY-SA 4.0) for `bundle eval` / `probe` |
+| `data/sft_valid/` | the run's SFT validation shards |
+| `third_party/minimind/` | vendored minimind (Apache-2.0), so no git clone is needed |
+| `wheels/`, `bin/setup.sh`, `bin/chronos` | offline install into `.venv`, launcher that sets `CHRONOS_BUNDLE` / `CHRONOS_MINIMIND_DIR` |
+| `recipes/source_run_spec.json`, `recommended.json` | the recipe that produced the weights; fine-tune tiers `smoke` / `quick` / `standard` / `full` |
+| `bundle.json` | provenance (step, tokens, valid CE), measured quality probe, SHA-256 of every file |
+
+On the recipient side:
+
+```bash
+bin/setup.sh                                              # venv + wheel + `bundle check`
+bin/chronos bundle translate "The weather is very nice today."
+bin/chronos bundle translate --backend mlx --speculative --file examples/translate_en.txt
+bin/chronos bundle eval --split devtest -n 200
+bin/chronos bundle prepare-data my_pairs.jsonl --out work/data      # FLORES+ is removed from your data
+bin/chronos bundle finetune --data work/data --run-dir work/ft --tier quick --mode sft
+bin/chronos bundle serve -- --port 11435
+```
+
+`--speculative` needs `--repetition-penalty 1.0` (its default), because the penalty changes the target distribution at every position. `finetune` writes a `chronos longrun` spec with `init_checkpoint` set to the bundle weights and refuses a run directory that already holds a run. Bundles built from data with unknown licences (WMT19, OPUS-100) say in `NOTICE.md` that the weights are for research only.
+
 ### Deployment export
 
 ```bash
@@ -575,7 +781,10 @@ Project_Chronos/
 │   ├── deps.py                    # Auto-download minimind if missing
 │   ├── __init__.py                # AutoConfig / AutoModelForCausalLM registration
 │   ├── model/
-│   │   ├── config.py              # ChronosConfig
+│   │   ├── config.py              # ChronosConfig (v3 fields, v2 migration)
+│   │   ├── recurrent.py           # Loop/group embeddings, contractive step, tensor-only ACT
+│   │   ├── precision_ops.py       # RMSNorm/SDPA/RoPE with one compute dtype
+│   │   ├── factory.py             # Parameter-budget config search
 │   │   ├── hybrid_attention.py    # MLAAttention + SlidingWindowAttention
 │   │   ├── lookahead_router.py    # Per-token lookahead predictor
 │   │   ├── moe_chronos.py         # ChronosMOEFeedForward + shared experts + soft gating
@@ -594,7 +803,11 @@ Project_Chronos/
 │   │   └── prefill_scheduler.py   # Prefill-time expert preloader
 │   ├── mlx/
 │   │   ├── attention.py / moe.py / model.py / expert_store.py / inference.py
+│   │   ├── moe_kernels.py         # Sorted gather_mm dispatch (mlx >= 0.32.4)
+│   │   ├── speculative.py         # MLX self-speculative decoding (Torch parity)
+│   │   └── training/              # longrun.py, telemetry.py, recipes/
 │   ├── runtime/
+│   │   ├── speculative.py         # Recurrent-depth self-speculative decoding
 │   │   ├── cache_manager.py       # prefetch_for_next_step / ensure_resident
 │   │   ├── inference_engine.py    # End-to-end inference engine
 │   │   └── metrics.py             # MetricsBus for the IO Monitor
@@ -613,14 +826,19 @@ Project_Chronos/
 │   │   ├── io_profiler.py         # Lookahead validation
 │   │   └── benchmark.py           # End-to-end benchmarking
 │   ├── data/
-│   │   └── flexible_dataset.py    # Flexible JSONL dataset loader
+│   │   ├── flexible_dataset.py    # Flexible JSONL dataset loader
+│   │   └── corpus/                # download / clean / dedup / leak filter / shards
+│   ├── tokenizer/                 # Chronos BPE builder (extract, normalize, train, evaluate)
+│   ├── monitor/                   # Read-only run monitor: snapshot, API server, static page
 │   ├── backend/
 │   │   ├── __init__.py            # BackendDispatcher (cpu/mps/cuda/mlx)
 │   │   ├── dispatcher.py          # Capability probing + priority logic
 │   │   └── ext/opencl.py          # Third-party OpenCL extension hook
 │   ├── _backend_legacy.py         # Backward-compatible older APIs
+│   ├── bundle/                    # chronos bundle: build/check/translate/finetune
 │   ├── serving/
-│   │   ├── __init__.py
+│   │   ├── ollama_server.py       # chronos serve: Ollama/OpenAI protocol layer
+│   │   ├── runtime.py             # Model discovery, lazy experts, speculative generation
 │   │   └── vllm_adapter.py        # Optional vLLM registration
 │   └── cli.py                     # Unified CLI
 ├── ui/                            # Gradio Web UI (zh-Hans / zh-Hant / en / ja)
@@ -628,7 +846,9 @@ Project_Chronos/
 │   ├── estimator.py               # Live parameter / memory estimator
 │   └── tabs/
 │       ├── config_tab.py          # Config + Designer merged together
+│       ├── data_tab.py
 │       ├── train_tab.py           # Owns data_path
+│       ├── monitor_tab.py         # Training Monitor (shares chronos.monitor)
 │       ├── pipeline_tab.py        # Per-stage datasets across all 6 stages
 │       ├── inference_tab.py
 │       ├── benchmark_tab.py       # Markdown table + gr.BarPlot
@@ -646,8 +866,9 @@ Project_Chronos/
 │   ├── compare_minimind_chronos_v2.py
 │   └── compare_minimind_chronos_v3.py
 ├── tests/
-│   ├── test_smoke.py
-│   ├── test_smoke_cuda.py
+│   ├── test_smoke.py / test_smoke_cuda.py
+│   ├── test_v3_kernel.py / test_speculative.py / test_moe_dispatch.py / test_precision_chain.py
+│   ├── test_tokenizer_builder.py / test_longrun.py / test_monitor.py
 │   └── fixtures/
 ├── docs/
 │   └── vllm_integration.md
@@ -689,7 +910,19 @@ timeline
         : HF IO, vLLM adapter, multi-backend dispatch
         : Stage 6 distillation and pluggable rewards
         : Web UI v2, benchmark plots, IO Monitor
+    v3 (0.3.0)
+        : Recurrent-depth kernel with shared_first KV
+        : top-2 + shared expert, grouped MoE dispatch
+        : Self-speculative decoding (Torch and MLX)
+        : Ollama/OpenAI protocol server
+        : Ready-to-run model bundles (chronos bundle)
+        : Chronos tokenizer, corpus pipeline, long-run trainer
+        : Training monitor page, API and console tab
+        : int8 KV, MLA absorbed decode, shared router scope (opt-in)
+        : Auto planner, v3 presets, tokenizer and serve tabs
     Next
+        : 50M en-zh results and ablations
+        : Speculative decoding: a draft path that pays off on larger models and lazy experts
         : Train IntentClassifier on large activation corpora
         : Benchmark 7B+ checkpoints
         : Inject masks on the vLLM worker path