justfit-llm 2.3.0__tar.gz

justfit_llm-2.3.0/.gitignore

@@ -0,0 +1,10 @@
+__pycache__/
+*.pyc
+.venv/
+venv/
+source/build/
+local-state/
+*.dylib
+build-receipt.json
+*.zip
+.env

justfit_llm-2.3.0/ANE_POLICY.md

@@ -0,0 +1,33 @@
+# ANE vision eligibility
+
+The release policy is `gpu_relief`, with per-node, per-capacity acceptance.
+4K/8K refer to raw patch capacities, not text context lengths. Select the smallest
+ready graph that fits the preserved preprocessed geometry.
+
+| Current native scheduler state | Language ANE FFN state | Eligible phase |
+| --- | --- | --- |
+| Any text prefill, with or without decoding lanes | off | qualified prefill |
+| Any text prefill, with or without decoding lanes | split/stopped | GPU fallback |
+| Decoding, no text prefill | known off/split/stopped | qualified decode |
+| Idle, only queued, unknown status | any | GPU fallback |
+
+Each leaf explicitly sets `phase_qualification: {"prefill": false, "decode": false}`
+until its hardware/service acceptance supports the corresponding phase. These
+booleans express an operator-reviewed deployment contract, not a speed estimate.
+There is no automatic promotion from a standalone encoder benchmark.
+
+A second waiting image or a multi-image request uses the existing native route.
+All external workers retain fixed native memory leases, host-reserve admission,
+bounded output payloads, cached-only graph loading and actual placement checks.
+No request compiles an asset. A transient prewarm admission denial defers the
+prewarm; it never bypasses runtime admission. Invalid assets or hashes fail load.
+
+Eligibility is rechecked after serialization and memory waits. Native ownership
+is authoritative across subsequent phase changes: language FFN jobs drain before
+vision ownership is granted, FFN split remains paused for the lease, and resumes
+on release. Cancellation/failure reaps the worker before releasing its lease.
+
+Evaluate text decode continuity and matched text output/throughput alongside image
+latency, prefill progress, memory and recovery. A slower isolated ANE encoder can
+still reduce interruption of the busy GPU. A single cohort completion difference
+is not proof of accelerator contention or of every lane slowing down.

justfit_llm-2.3.0/CHANGELOG.md

@@ -0,0 +1,50 @@
+# Changelog
+
+## 2.3.0
+
+- Promote the qualified preview to a stable-version release candidate.
+- Distribute the native engine and helpers with Developer ID signatures; verify
+  Apple notarization and exact installed artifacts before publication.
+
+- Support a separately pinned local or HTTPS-manifest native runtime, validating
+  source identity, platform, archive paths and output hashes before installation.
+- Add a relocatable native asset builder; ad-hoc assets are for local qualification
+  and do not claim Developer ID signing, notarization or public hosting.
+
+- Align all three reference node presets with the tested 22,015 MiB native
+  memory budget; document 21,631 MiB as a conservative local alternative.
+
+- Release persistent-state staging memory while retaining immutable buffers
+  through asynchronous writes; preserve direct-to-disk recovery when snapshot
+  memory admission is denied.
+- Isolate serving imports from inherited Python paths and user site-packages.
+- Include a native regression covering state IO ownership, cancellation and
+  memory-constrained disk recovery.
+- Add citation metadata for the JustFit v2 manuscript.
+- `justfit serve` with no arguments creates the workspace, finds the Metal
+  toolchain and builds once, downloads the qualified model revision, selects the
+  preset from chip and memory, and defaults to port 8000. Adds a `justfit` command alias.
+
+- Use neutral source-distribution owner/group metadata.
+- Document installation, configuration and validation for standalone users.
+
+## 2.3 features
+
+- Source-bearing wheel/sdist and explicit workspace, build and serve commands.
+- Native per-request thinking budgets, including speculative boundaries.
+- Conditional 4K/8K ANE vision routing with device-specific phase eligibility.
+- CPU/GPU execution references and language-ANE prefill integration.
+- Cancellation propagation, worker lifecycle and resource-admission checks.
+- Portable ANE export, offline compilation and local profile qualification tools.
+- Managed service disk/log protection and restart port checks.
+- Optional external broker history-preservation and health-probe locking adapters.
+- Management Host/Origin protection, optional bearer key, bounded uploads and
+  updated certificate/image-parser dependencies.
+- Synthetic image fixtures with frozen expected answers.
+
+See [validation](docs/VALIDATION.md) for test procedures and limitations.
+The dated JustFit v2 preprint is linked from the README and citation metadata.
+
+### ANE installation integrity
+
+Revalidate installed graph assets and compiled cache contents against SHA256 before reuse or native prewarm. Missing, changed, extra, linked or special files are refused without overwriting user caches.

justfit_llm-2.3.0/CITATION.cff

@@ -0,0 +1,18 @@
+cff-version: 1.2.0
+message: "If JustFit supports your research, please cite the paper listed in preferred-citation."
+type: software
+title: JustFit
+authors:
+  - family-names: Chen
+    given-names: Yuhua
+preferred-citation:
+  type: unpublished
+  title: "JustFit v2: Long-Context Local LLM Serving with Just-in-Time State Management"
+  authors:
+    - family-names: Chen
+      given-names: Yuhua
+  year: 2026
+  month: 10
+  day: 23
+  url: "https://github.com/YuhuaBillChen/justfit-llm/blob/paper-v2-2026-10-23-r2/paper/2026-10-23/JustFit-v2.pdf"
+  notes: "Preprint, version 2026-10-23 revision 2."

justfit_llm-2.3.0/CONTRIBUTING.md

@@ -0,0 +1,10 @@
+# Contributing
+
+Run the source and service tests in [QUICKSTART.md](QUICKSTART.md). Include the
+node/chip, macOS, model revision, configuration and artifact hashes when reporting
+a reproducible issue. Exclude credentials, private prompts and model weights.
+Runtime changes need focused behavioral tests; accelerator changes additionally
+need matched device evidence. Keep upstream attribution and native build receipts.
+
+Report JustFit issues to https://github.com/YuhuaBillChen/justfit-llm/issues.
+Upstream Splash maintainers are not the support contact for this modified project.

justfit_llm-2.3.0/LICENSE

@@ -0,0 +1,202 @@
+
+                                 Apache License
+                           Version 2.0, January 2004
+                        http://www.apache.org/licenses/
+
+   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+   1. Definitions.
+
+      "License" shall mean the terms and conditions for use, reproduction,
+      and distribution as defined by Sections 1 through 9 of this document.
+
+      "Licensor" shall mean the copyright owner or entity authorized by
+      the copyright owner that is granting the License.
+
+      "Legal Entity" shall mean the union of the acting entity and all
+      other entities that control, are controlled by, or are under common
+      control with that entity. For the purposes of this definition,
+      "control" means (i) the power, direct or indirect, to cause the
+      direction or management of such entity, whether by contract or
+      otherwise, or (ii) ownership of fifty percent (50%) or more of the
+      outstanding shares, or (iii) beneficial ownership of such entity.
+
+      "You" (or "Your") shall mean an individual or Legal Entity
+      exercising permissions granted by this License.
+
+      "Source" form shall mean the preferred form for making modifications,
+      including but not limited to software source code, documentation
+      source, and configuration files.
+
+      "Object" form shall mean any form resulting from mechanical
+      transformation or translation of a Source form, including but
+      not limited to compiled object code, generated documentation,
+      and conversions to other media types.
+
+      "Work" shall mean the work of authorship, whether in Source or
+      Object form, made available under the License, as indicated by a
+      copyright notice that is included in or attached to the work
+      (an example is provided in the Appendix below).
+
+      "Derivative Works" shall mean any work, whether in Source or Object
+      form, that is based on (or derived from) the Work and for which the
+      editorial revisions, annotations, elaborations, or other modifications
+      represent, as a whole, an original work of authorship. For the purposes
+      of this License, Derivative Works shall not include works that remain
+      separable from, or merely link (or bind by name) to the interfaces of,
+      the Work and Derivative Works thereof.
+
+      "Contribution" shall mean any work of authorship, including
+      the original version of the Work and any modifications or additions
+      to that Work or Derivative Works thereof, that is intentionally
+      submitted to Licensor for inclusion in the Work by the copyright owner
+      or by an individual or Legal Entity authorized to submit on behalf of
+      the copyright owner. For the purposes of this definition, "submitted"
+      means any form of electronic, verbal, or written communication sent
+      to the Licensor or its representatives, including but not limited to
+      communication on electronic mailing lists, source code control systems,
+      and issue tracking systems that are managed by, or on behalf of, the
+      Licensor for the purpose of discussing and improving the Work, but
+      excluding communication that is conspicuously marked or otherwise
+      designated in writing by the copyright owner as "Not a Contribution."
+
+      "Contributor" shall mean Licensor and any individual or Legal Entity
+      on behalf of whom a Contribution has been received by Licensor and
+      subsequently incorporated within the Work.
+
+   2. Grant of Copyright License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      copyright license to reproduce, prepare Derivative Works of,
+      publicly display, publicly perform, sublicense, and distribute the
+      Work and such Derivative Works in Source or Object form.
+
+   3. Grant of Patent License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      (except as stated in this section) patent license to make, have made,
+      use, offer to sell, sell, import, and otherwise transfer the Work,
+      where such license applies only to those patent claims licensable
+      by such Contributor that are necessarily infringed by their
+      Contribution(s) alone or by combination of their Contribution(s)
+      with the Work to which such Contribution(s) was submitted. If You
+      institute patent litigation against any entity (including a
+      cross-claim or counterclaim in a lawsuit) alleging that the Work
+      or a Contribution incorporated within the Work constitutes direct
+      or contributory patent infringement, then any patent licenses
+      granted to You under this License for that Work shall terminate
+      as of the date such litigation is filed.
+
+   4. Redistribution. You may reproduce and distribute copies of the
+      Work or Derivative Works thereof in any medium, with or without
+      modifications, and in Source or Object form, provided that You
+      meet the following conditions:
+
+      (a) You must give any other recipients of the Work or
+          Derivative Works a copy of this License; and
+
+      (b) You must cause any modified files to carry prominent notices
+          stating that You changed the files; and
+
+      (c) You must retain, in the Source form of any Derivative Works
+          that You distribute, all copyright, patent, trademark, and
+          attribution notices from the Source form of the Work,
+          excluding those notices that do not pertain to any part of
+          the Derivative Works; and
+
+      (d) If the Work includes a "NOTICE" text file as part of its
+          distribution, then any Derivative Works that You distribute must
+          include a readable copy of the attribution notices contained
+          within such NOTICE file, excluding those notices that do not
+          pertain to any part of the Derivative Works, in at least one
+          of the following places: within a NOTICE text file distributed
+          as part of the Derivative Works; within the Source form or
+          documentation, if provided along with the Derivative Works; or,
+          within a display generated by the Derivative Works, if and
+          wherever such third-party notices normally appear. The contents
+          of the NOTICE file are for informational purposes only and
+          do not modify the License. You may add Your own attribution
+          notices within Derivative Works that You distribute, alongside
+          or as an addendum to the NOTICE text from the Work, provided
+          that such additional attribution notices cannot be construed
+          as modifying the License.
+
+      You may add Your own copyright statement to Your modifications and
+      may provide additional or different license terms and conditions
+      for use, reproduction, or distribution of Your modifications, or
+      for any such Derivative Works as a whole, provided Your use,
+      reproduction, and distribution of the Work otherwise complies with
+      the conditions stated in this License.
+
+   5. Submission of Contributions. Unless You explicitly state otherwise,
+      any Contribution intentionally submitted for inclusion in the Work
+      by You to the Licensor shall be under the terms and conditions of
+      this License, without any additional terms or conditions.
+      Notwithstanding the above, nothing herein shall supersede or modify
+      the terms of any separate license agreement you may have executed
+      with Licensor regarding such Contributions.
+
+   6. Trademarks. This License does not grant permission to use the trade
+      names, trademarks, service marks, or product names of the Licensor,
+      except as required for reasonable and customary use in describing the
+      origin of the Work and reproducing the content of the NOTICE file.
+
+   7. Disclaimer of Warranty. Unless required by applicable law or
+      agreed to in writing, Licensor provides the Work (and each
+      Contributor provides its Contributions) on an "AS IS" BASIS,
+      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+      implied, including, without limitation, any warranties or conditions
+      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+      PARTICULAR PURPOSE. You are solely responsible for determining the
+      appropriateness of using or redistributing the Work and assume any
+      risks associated with Your exercise of permissions under this License.
+
+   8. Limitation of Liability. In no event and under no legal theory,
+      whether in tort (including negligence), contract, or otherwise,
+      unless required by applicable law (such as deliberate and grossly
+      negligent acts) or agreed to in writing, shall any Contributor be
+      liable to You for damages, including any direct, indirect, special,
+      incidental, or consequential damages of any character arising as a
+      result of this License or out of the use or inability to use the
+      Work (including but not limited to damages for loss of goodwill,
+      work stoppage, computer failure or malfunction, or any and all
+      other commercial damages or losses), even if such Contributor
+      has been advised of the possibility of such damages.
+
+   9. Accepting Warranty or Additional Liability. While redistributing
+      the Work or Derivative Works thereof, You may choose to offer,
+      and charge a fee for, acceptance of support, warranty, indemnity,
+      or other liability obligations and/or rights consistent with this
+      License. However, in accepting such obligations, You may act only
+      on Your own behalf and on Your sole responsibility, not on behalf
+      of any other Contributor, and only if You agree to indemnify,
+      defend, and hold each Contributor harmless for any liability
+      incurred by, or claims asserted against, such Contributor by reason
+      of your accepting any such warranty or additional liability.
+
+   END OF TERMS AND CONDITIONS
+
+   APPENDIX: How to apply the Apache License to your work.
+
+      To apply the Apache License to your work, attach the following
+      boilerplate notice, with the fields enclosed by brackets "[]"
+      replaced with your own identifying information. (Don't include
+      the brackets!)  The text should be enclosed in the appropriate
+      comment syntax for the file format. We also recommend that a
+      file or class name and description of purpose be included on the
+      same "printed page" as the copyright notice for easier
+      identification within third-party archives.
+
+   Copyright [yyyy] [name of copyright owner]
+
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.

justfit_llm-2.3.0/MANIFEST.in

@@ -0,0 +1,18 @@
+include CITATION.cff
+include *.md *.json *.py *.toml *.txt .gitignore MANIFEST.in
+recursive-include source *
+recursive-include vision-tools *
+recursive-include service *
+recursive-include tests *
+recursive-include scripts *
+recursive-include config *
+recursive-include docs *
+recursive-include paper *
+recursive-include website *
+recursive-include python *.py
+prune build
+prune dist
+prune source/build
+prune source/.venv
+prune source/venv
+global-exclude __pycache__ *.pyc *.dylib .DS_Store

justfit_llm-2.3.0/NOTICE

@@ -0,0 +1,14 @@
+JustFit 2.3 is derived from Splash 1.3.0.
+Upstream: https://github.com/incoai/splash
+Pinned upstream commit: 0d5a035ea26d1cc566e3c7ee4e08e5e06fa68cf7
+
+This distribution contains modified runtime/frontend code and additional JustFit
+workers, service integration and release tooling. See ORIGIN.json and CHANGELOG.md.
+The upstream Apache-2.0 license, THIRD_PARTY_NOTICES and inline notices are retained.
+Model weights and machine-specific compiled assets are not included.
+
+JustFit product identity: the modified engine is named justfit-engine; its GPU
+libraries are justfit.metallib and justfit-verify16.metallib. The Python commands
+are justfit and justfit-llm. These names do not imply upstream endorsement.
+Published upstream model repository names and packed-weight format identifiers
+are retained for compatibility. Modified source files carry JustFit notices.

justfit_llm-2.3.0/ORIGIN.json

@@ -0,0 +1,14 @@
+{
+  "project": "JustFit",
+  "project_version": "2.3.0",
+  "upstream_project": "Splash",
+  "upstream_version": "1.3.0",
+  "upstream_commit": "0d5a035ea26d1cc566e3c7ee4e08e5e06fa68cf7",
+  "source_identity": "package-manifest.json pins the complete modified source tree; optional integration patches are not the complete upstream-to-JustFit diff.",
+  "attribution": [
+    "LICENSE",
+    "NOTICE",
+    "source/THIRD_PARTY_NOTICES"
+  ],
+  "optional_integrations": "service/broker-integration contains version-specific external adapter patches, not a standalone broker."
+}

justfit_llm-2.3.0/PKG-INFO

@@ -0,0 +1,201 @@
+Metadata-Version: 2.4
+Name: justfit-llm
+Version: 2.3.0
+Summary: Apple silicon inference with coordinated CPU, GPU and ANE work
+License-Expression: Apache-2.0
+Project-URL: Homepage, https://github.com/YuhuaBillChen/justfit-llm
+Project-URL: Repository, https://github.com/YuhuaBillChen/justfit-llm
+Project-URL: Issues, https://github.com/YuhuaBillChen/justfit-llm/issues
+Classifier: Development Status :: 5 - Production/Stable
+Classifier: Operating System :: MacOS :: MacOS X
+Classifier: Programming Language :: Python :: 3
+Requires-Python: <3.15,>=3.12
+Description-Content-Type: text/markdown
+License-File: LICENSE
+License-File: NOTICE
+License-File: source/THIRD_PARTY_NOTICES
+Requires-Dist: annotated-doc==0.0.5
+Requires-Dist: anyio==4.14.2
+Requires-Dist: attrs==26.1.0
+Requires-Dist: certifi==2026.10.19
+Requires-Dist: cffi==2.1.1
+Requires-Dist: click==8.4.2
+Requires-Dist: cryptography==50.0.1
+Requires-Dist: filelock==3.32.3
+Requires-Dist: fsspec==2026.7.0
+Requires-Dist: h11==0.16.0
+Requires-Dist: hf-xet==1.6.0
+Requires-Dist: httpcore==1.0.9
+Requires-Dist: httpx==0.28.1
+Requires-Dist: huggingface_hub==1.28.0
+Requires-Dist: idna==3.19
+Requires-Dist: Jinja2==3.1.6
+Requires-Dist: jsonschema==4.26.0
+Requires-Dist: jsonschema-specifications==2025.9.1
+Requires-Dist: llguidance==1.8.0
+Requires-Dist: markdown-it-py==4.2.0
+Requires-Dist: MarkupSafe==3.0.3
+Requires-Dist: mdurl==0.1.2
+Requires-Dist: numpy==2.5.2
+Requires-Dist: packaging==26.3
+Requires-Dist: Pillow==13.0.0
+Requires-Dist: Pygments==2.21.0
+Requires-Dist: pycparser==3.0
+Requires-Dist: pypdfium2==5.13.0
+Requires-Dist: PyYAML==6.0.3
+Requires-Dist: referencing==0.37.0
+Requires-Dist: regex==2026.7.19
+Requires-Dist: rich==15.0.0
+Requires-Dist: rpds-py==2026.6.3
+Requires-Dist: safetensors==0.8.0
+Requires-Dist: shellingham==1.5.4
+Requires-Dist: tokenizers==0.22.2
+Requires-Dist: tqdm==4.70.0
+Requires-Dist: transformers==5.15.1
+Requires-Dist: typer==0.27.1
+Requires-Dist: typing_extensions==4.16.0
+Requires-Dist: fastapi==0.141.1
+Requires-Dist: uvicorn==0.52.1
+Requires-Dist: annotated-types==0.8.0
+Requires-Dist: pydantic==2.14.1
+Requires-Dist: pydantic_core==2.50.1
+Requires-Dist: starlette==1.9.0
+Requires-Dist: typing-inspection==0.4.4
+Dynamic: license-file
+
+<div align="center">
+  <picture>
+    <source media="(max-width: 600px)" srcset="docs/assets/justfit-banner-mobile.svg">
+    <img src="docs/assets/justfit-banner.svg" alt="JustFit — long-context LLM serving on Apple silicon" width="100%">
+  </picture>
+
+  [![Version](https://img.shields.io/badge/version-2.3.0-087f78)](CHANGELOG.md)
+  [![License: Apache 2.0](https://img.shields.io/badge/License-Apache%202.0-246a96)](LICENSE)
+  [![Apple silicon](https://img.shields.io/badge/platform-Apple%20silicon-526b81)](#supported-setup)
+
+  **[Get started](#get-started) · [Results](#measured-results) · [Documentation](#documentation) · [简体中文](README.zh-CN.md)**
+</div>
+
+JustFit is a local inference system for serving a **27B language-and-vision model on memory-constrained Macs**. It coordinates compressed context, a resident speculative draft, concurrent requests and CPU/GPU/ANE work under one unified-memory budget.
+
+**Stable version: `2.3.0`.** This source tree prepares the stable release. Package-index and release-asset publication are pending; use the reviewed wheel and native runtime supplied with this release candidate. The immutable [RC24 preview](https://test.pypi.org/project/justfit-llm/2.3.0rc24/) remains available separately.
+
+## Why JustFit
+
+- **Keep long histories alongside the draft.** Warm-prefix tests reach a 320K aggregate page capacity across two live requests while retaining the speculative draft. This is a tested B2 bound, not a 320K single-request or cold-prefill guarantee.
+- **Make concurrent work useful.** Bound transient workspaces, share compressed pages and preserve committed state across scheduling and cold restarts.
+- **Put idle accelerators to work.** Use CPU-assisted prefill on M4 Pro, GPU A8 paths on M5 Pro/M6, and qualified ANE work when execution phase and memory allow it. Vision offload aims to leave GPU capacity for text; it does not guarantee lower image latency.
+- **Serve through a familiar API.** OpenAI-compatible chat, images, streaming, structured output and native-enforced thinking budgets.
+
+## Measured results
+
+<a href="docs/assets/system-comparison.png"><img src="docs/assets/system-comparison.png" alt="Four-system comparison on M4 Pro, M5 Pro and M6: tested retained context, 16K prefill and 16K decode rates" width="100%"></a>
+
+**Capacity and speed are separate experiments.** Solid bars show tested B1 bounds; hatching extends them to the B2 total. Both JustFit generations reach a 320K page capacity using their respective warm-prefix protocols; v2 retains its draft. Speed uses eight fixed SPEED-Bench 16K tasks with 512 forced output tokens. Weight formats, decoders and profiles differ. M4 Pro/M5 Pro have physical 24 GiB; M6 has physical 32 GiB, with 8 GiB locked only for the v1/v2 B2 capacity tests. M6 speed runs use physical 32 GiB.
+
+At 16K, v2's median decode rate is **2.0–2.5× the v1 rate** across these configurations. Relative to upstream Splash 1.3.0, median prefill improves **9–46%**, while singleton decode declines **2–9%**. These whole-system results are not isolated mechanism gains or a chip-generation ranking.
+
+[Protocols, exact rates and limitations →](docs/RESULTS.md)
+
+## Get started
+
+### Supported setup
+
+- **Apple silicon and macOS 27.0+.** Reference configurations: M4 Pro, M5 Pro and M6, with at least 24 GiB. Other chip variants are not individually qualified.
+- **Python 3.12 recommended**; the package declares support for Python 3.12–3.14.
+- **Compatible Xcode/CoreAI SDK and Metal compiler toolchain** for this source preview. `justfit serve` finds installed tools; it does not install Xcode or the SDK.
+- Internet access and storage for the pinned model pack: approximately **17.4 GB**, plus dependencies, build outputs and caches. The model repository is publicly accessible without a Hugging Face token at the checked revision.
+
+Installing the Python package on Linux does not enable native inference there. The standalone GPU engine's macOS 26.4 source target is separate from this complete macOS 27/CoreAI setup.
+
+### Install the reviewed release
+
+Create a fresh environment. Install the reviewed wheel downloaded alongside this source tree:
+
+```bash
+python3.12 -m venv .venv
+source .venv/bin/activate
+
+python -m pip install --index-url https://pypi.org/simple/ -r requirements-runtime.txt
+python -m pip install ./justfit_llm-2.3.0-py3-none-any.whl
+justfit verify
+justfit doctor
+justfit serve
+```
+
+First run: create a versioned workspace, build the native runtime, download and verify the pinned model, choose the device preset, then serve on **`http://127.0.0.1:8000`**. Later runs reuse the build and model. Ctrl+C stops the foreground server. GPU vision works without a compiled ANE vision profile.
+
+If the Metal compiler is missing, install the component with `xcodebuild -downloadComponent MetalToolchain` after installing compatible Xcode. [Step-by-step setup and overrides](QUICKSTART.md) cover custom ports, local models and prebuilt-runtime inputs. Use the matching 2.3.0 runtime and published SHA256 to skip local compilation; see the linked instructions.
+
+### Send a request
+
+```bash
+curl http://127.0.0.1:8000/v1/chat/completions \
+  -H 'Content-Type: application/json' \
+  -d '{
+    "model": "incoai/Qwen3.8-27B-Splash",
+    "messages": [{"role": "user", "content": "What is 17 multiplied by 19?"}],
+    "max_tokens": 64,
+    "reasoning_effort": "none"
+  }'
+```
+
+The model ID retains the upstream runtime pack's name; the engine and CLI are JustFit. The default model is pinned to revision `9d27070b71f7142c6b6025f03ac011d70a73cb48`.
+
+## How it works
+
+| Mechanism | Responsibility |
+| --- | --- |
+| **KVExec** | Consume compressed history with bounded reconstruction and rotated-domain attention. |
+| **PhaseSwap** | Control component and workspace lifetimes while the speculative draft stays resident. |
+| **StateTrans** | Preserve committed KV, recurrent and draft state across scheduling and persistence. |
+| **CrossFire** | Coordinate CPU, GPU and ANE work using phase ownership and memory admission. |
+
+### ANE behavior
+
+| Reference preset | Language prefill | ANE vision eligibility |
+| --- | --- | --- |
+| M4 Pro / `m4` | CPU-assisted GPU | Text prefill or decode |
+| M5 Pro / `m5` | GPU A8 | Text prefill or decode |
+| M6 / `m6` | GPU A8 + ANE FFN | Decode with no active text prefill |
+
+Vision eligibility requires qualified 4K/8K assets, a matching chip and OS build, and memory admission. **4K/8K count image patches, not text tokens or screen pixels.** Idle requests, unsupported inputs and unavailable resources use GPU. M6 reserves ANE for language prefill before admitting vision work. [ANE setup and download interface](docs/ANE_DOWNLOADS.md).
+
+## Documentation
+
+| I want to… | Start here |
+| --- | --- |
+| Install, build or choose a port/model | [Quickstart](QUICKSTART.md) · [Python installation](docs/PYPI.md) |
+| Understand the measurements | [Results and protocols](docs/RESULTS.md) · [Preview validation](docs/PREVIEW_VALIDATION.json) |
+| Change device settings | [Configuration](docs/CONFIGURATION.md) · [ANE policy](ANE_POLICY.md) |
+| Prepare ANE assets or a prebuilt runtime | [ANE downloads](docs/ANE_DOWNLOADS.md) · [Offline provisioning](docs/ANE_PROVISIONING.md) · [Runtime distribution](docs/RUNTIME_DISTRIBUTION.md) |
+| Deploy a managed service | [Deployment and rollback](docs/DEPLOYMENT.md) · [Broker integration](service/broker-integration/README.md) |
+| Control reasoning output | [Thinking budget](docs/THINKING_BUDGET.md) |
+| Report a problem or contribute | [Issues](https://github.com/YuhuaBillChen/justfit-llm/issues) · [Contributing](CONTRIBUTING.md) · [Security](SECURITY.md) |
+
+The default endpoint is local. Network deployment needs appropriate authentication and TLS; see [security scope](SECURITY.md). Native runtime sources are in `source/`, ANE tooling in `vision-tools/`, and optional managed-service components in `service/`.
+
+## Paper and citation
+
+**JustFit v2: Long-Context Local LLM Serving with Just-in-Time State Management** — Yuhua Chen, 2026. **[Read the preprint (PDF)](https://github.com/YuhuaBillChen/justfit-llm/blob/paper-v2-2026-10-23-r2/paper/2026-10-23/JustFit-v2.pdf)** · [Paper and citation files](paper/README.md). This dated version remains available as the manuscript evolves; an arXiv link will be added when public.
+
+If JustFit supports your research, please cite:
+
+```bibtex
+@unpublished{chen2026justfitv2,
+  author = {Chen, Yuhua},
+  title = {{JustFit v2}: Long-Context Local LLM Serving with Just-in-Time State Management},
+  year = {2026},
+  month = oct,
+  note = {Preprint, version 2026-10-23 revision 2},
+  url = {https://github.com/YuhuaBillChen/justfit-llm/blob/paper-v2-2026-10-23-r2/paper/2026-10-23/JustFit-v2.pdf}
+}
+```
+
+[Machine-readable citation](CITATION.cff) · [JustFit v1 paper](https://arxiv.org/abs/2609.17475)
+
+## Acknowledgments and license
+
+JustFit's native runtime is derived from **[Splash 1.3.0](https://github.com/incoai/splash)**. Upstream authors, licenses and implementation credits are preserved in [NOTICE](NOTICE) and [third-party notices](source/THIRD_PARTY_NOTICES). Model weights and derived assets retain their own licenses.
+
+Code is available under [Apache 2.0](LICENSE). JustFit maintainers handle this project's support; please direct JustFit issues here rather than to upstream maintainers.