laya-mlx 0.1.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,15 @@
1
+ .venv/
2
+ .upstream/
3
+ .cache/
4
+ models/
5
+ *.safetensors
6
+ __pycache__/
7
+ .pytest_cache/
8
+ .ruff_cache/
9
+ *.egg-info/
10
+ build/
11
+ dist/
12
+ .DS_Store
13
+ .env
14
+ *.log
15
+ artifacts/
laya_mlx-0.1.0/LICENSE ADDED
@@ -0,0 +1,176 @@
1
+ Apache License
2
+ Version 2.0, January 2004
3
+ http://www.apache.org/licenses/
4
+
5
+ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
6
+
7
+ 1. Definitions.
8
+
9
+ "License" shall mean the terms and conditions for use, reproduction,
10
+ and distribution as defined by Sections 1 through 9 of this document.
11
+
12
+ "Licensor" shall mean the copyright owner or entity authorized by
13
+ the copyright owner that is granting the License.
14
+
15
+ "Legal Entity" shall mean the union of the acting entity and all
16
+ other entities that control, are controlled by, or are under common
17
+ control with that entity. For the purposes of this definition,
18
+ "control" means (i) the power, direct or indirect, to cause the
19
+ direction or management of such entity, whether by contract or
20
+ otherwise, or (ii) ownership of fifty percent (50%) or more of the
21
+ outstanding shares, or (iii) beneficial ownership of such entity.
22
+
23
+ "You" (or "Your") shall mean an individual or Legal Entity
24
+ exercising permissions granted by this License.
25
+
26
+ "Source" form shall mean the preferred form for making modifications,
27
+ including but not limited to software source code, documentation
28
+ source, and configuration files.
29
+
30
+ "Object" form shall mean any form resulting from mechanical
31
+ transformation or translation of a Source form, including but
32
+ not limited to compiled object code, generated documentation,
33
+ and conversions to other media types.
34
+
35
+ "Work" shall mean the work of authorship, whether in Source or
36
+ Object form, made available under the License, as indicated by a
37
+ copyright notice that is included in or attached to the work
38
+ (an example is provided in the Appendix below).
39
+
40
+ "Derivative Works" shall mean any work, whether in Source or Object
41
+ form, that is based on (or derived from) the Work and for which the
42
+ editorial revisions, annotations, elaborations, or other modifications
43
+ represent, as a whole, an original work of authorship. For the purposes
44
+ of this License, Derivative Works shall not include works that remain
45
+ separable from, or merely link (or bind by name) to the interfaces of,
46
+ the Work and Derivative Works thereof.
47
+
48
+ "Contribution" shall mean any work of authorship, including
49
+ the original version of the Work and any modifications or additions
50
+ to that Work or Derivative Works thereof, that is intentionally
51
+ submitted to Licensor for inclusion in the Work by the copyright owner
52
+ or by an individual or Legal Entity authorized to submit on behalf of
53
+ the copyright owner. For the purposes of this definition, "submitted"
54
+ means any form of electronic, verbal, or written communication sent
55
+ to the Licensor or its representatives, including but not limited to
56
+ communication on electronic mailing lists, source code control systems,
57
+ and issue tracking systems that are managed by, or on behalf of, the
58
+ Licensor for the purpose of discussing and improving the Work, but
59
+ excluding communication that is conspicuously marked or otherwise
60
+ designated in writing by the copyright owner as "Not a Contribution."
61
+
62
+ "Contributor" shall mean Licensor and any individual or Legal Entity
63
+ on behalf of whom a Contribution has been received by Licensor and
64
+ subsequently incorporated within the Work.
65
+
66
+ 2. Grant of Copyright License. Subject to the terms and conditions of
67
+ this License, each Contributor hereby grants to You a perpetual,
68
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
69
+ copyright license to reproduce, prepare Derivative Works of,
70
+ publicly display, publicly perform, sublicense, and distribute the
71
+ Work and such Derivative Works in Source or Object form.
72
+
73
+ 3. Grant of Patent License. Subject to the terms and conditions of
74
+ this License, each Contributor hereby grants to You a perpetual,
75
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
76
+ (except as stated in this section) patent license to make, have made,
77
+ use, offer to sell, sell, import, and otherwise transfer the Work,
78
+ where such license applies only to those patent claims licensable
79
+ by such Contributor that are necessarily infringed by their
80
+ Contribution(s) alone or by combination of their Contribution(s)
81
+ with the Work to which such Contribution(s) was submitted. If You
82
+ institute patent litigation against any entity (including a
83
+ cross-claim or counterclaim in a lawsuit) alleging that the Work
84
+ or a Contribution incorporated within the Work constitutes direct
85
+ or contributory patent infringement, then any patent licenses
86
+ granted to You under this License for that Work shall terminate
87
+ as of the date such litigation is filed.
88
+
89
+ 4. Redistribution. You may reproduce and distribute copies of the
90
+ Work or Derivative Works thereof in any medium, with or without
91
+ modifications, and in Source or Object form, provided that You
92
+ meet the following conditions:
93
+
94
+ (a) You must give any other recipients of the Work or
95
+ Derivative Works a copy of this License; and
96
+
97
+ (b) You must cause any modified files to carry prominent notices
98
+ stating that You changed the files; and
99
+
100
+ (c) You must retain, in the Source form of any Derivative Works
101
+ that You distribute, all copyright, patent, trademark, and
102
+ attribution notices from the Source form of the Work,
103
+ excluding those notices that do not pertain to any part of
104
+ the Derivative Works; and
105
+
106
+ (d) If the Work includes a "NOTICE" text file as part of its
107
+ distribution, then any Derivative Works that You distribute must
108
+ include a readable copy of the attribution notices contained
109
+ within such NOTICE file, excluding those notices that do not
110
+ pertain to any part of the Derivative Works, in at least one
111
+ of the following places: within a NOTICE text file distributed
112
+ as part of the Derivative Works; within the Source form or
113
+ documentation, if provided along with the Derivative Works; or,
114
+ within a display generated by the Derivative Works, if and
115
+ wherever such third-party notices normally appear. The contents
116
+ of the NOTICE file are for informational purposes only and
117
+ do not modify the License. You may add Your own attribution
118
+ notices within Derivative Works that You distribute, alongside
119
+ or as an addendum to the NOTICE text from the Work, provided
120
+ that such additional attribution notices cannot be construed
121
+ as modifying the License.
122
+
123
+ You may add Your own copyright statement to Your modifications and
124
+ may provide additional or different license terms and conditions
125
+ for use, reproduction, or distribution of Your modifications, or
126
+ for any such Derivative Works as a whole, provided Your use,
127
+ reproduction, and distribution of the Work otherwise complies with
128
+ the conditions stated in this License.
129
+
130
+ 5. Submission of Contributions. Unless You explicitly state otherwise,
131
+ any Contribution intentionally submitted for inclusion in the Work
132
+ by You to the Licensor shall be under the terms and conditions of
133
+ this License, without any additional terms or conditions.
134
+ Notwithstanding the above, nothing herein shall supersede or modify
135
+ the terms of any separate license agreement you may have executed
136
+ with Licensor regarding such Contributions.
137
+
138
+ 6. Trademarks. This License does not grant permission to use the trade
139
+ names, trademarks, service marks, or product names of the Licensor,
140
+ except as required for reasonable and customary use in describing the
141
+ origin of the Work and reproducing the content of the NOTICE file.
142
+
143
+ 7. Disclaimer of Warranty. Unless required by applicable law or
144
+ agreed to in writing, Licensor provides the Work (and each
145
+ Contributor provides its Contributions) on an "AS IS" BASIS,
146
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
147
+ implied, including, without limitation, any warranties or conditions
148
+ of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
149
+ PARTICULAR PURPOSE. You are solely responsible for determining the
150
+ appropriateness of using or redistributing the Work and assume any
151
+ risks associated with Your exercise of permissions under this License.
152
+
153
+ 8. Limitation of Liability. In no event and under no legal theory,
154
+ whether in tort (including negligence), contract, or otherwise,
155
+ unless required by applicable law (such as deliberate and grossly
156
+ negligent acts) or agreed to in writing, shall any Contributor be
157
+ liable to You for damages, including any direct, indirect, special,
158
+ incidental, or consequential damages of any character arising as a
159
+ result of this License or out of the use or inability to use the
160
+ Work (including but not limited to damages for loss of goodwill,
161
+ work stoppage, computer failure or malfunction, or any and all
162
+ other commercial damages or losses), even if such Contributor
163
+ has been advised of the possibility of such damages.
164
+
165
+ 9. Accepting Warranty or Additional Liability. While redistributing
166
+ the Work or Derivative Works thereof, You may choose to offer,
167
+ and charge a fee for, acceptance of support, warranty, indemnity,
168
+ or other liability obligations and/or rights consistent with this
169
+ License. However, in accepting such obligations, You may act only
170
+ on Your own behalf and on Your sole responsibility, not on behalf
171
+ of any other Contributor, and only if You agree to indemnify,
172
+ defend, and hold each Contributor harmless for any liability
173
+ incurred by, or claims asserted against, such Contributor by reason
174
+ of your accepting any such warranty or additional liability.
175
+
176
+ END OF TERMS AND CONDITIONS
laya_mlx-0.1.0/NOTICE ADDED
@@ -0,0 +1,14 @@
1
+ laya-mlx
2
+ Copyright 2026 laya-mlx contributors
3
+
4
+ This product includes software derived from Laya:
5
+ https://github.com/NandhaKishorM/laya
6
+ Copyright Convai Innovations and Laya contributors. Licensed under Apache-2.0.
7
+ Upstream source revision: 6a5819129eb220570792e417e49723d697efd76f
8
+
9
+ The token sequence construction, question rendering, confidence calculation,
10
+ presets, email utilities and language router are adapted from Laya.
11
+ The neural network is reimplemented using Apple's MLX, following Laya's
12
+ DecisionModel and the ModernBERT architecture in Hugging Face Transformers.
13
+ Model weights are downloaded separately from Convai Innovations on Hugging Face;
14
+ they are not included in this repository.
@@ -0,0 +1,287 @@
1
+ Metadata-Version: 2.5
2
+ Name: laya-mlx
3
+ Version: 0.1.0
4
+ Summary: Native MLX inference for Laya typed decision models on Apple silicon
5
+ Project-URL: Repository, https://github.com/mizorewww/laya-mlx
6
+ Project-URL: Upstream, https://github.com/NandhaKishorM/laya
7
+ Project-URL: Documentation, https://github.com/mizorewww/laya-mlx#readme
8
+ Project-URL: Issues, https://github.com/mizorewww/laya-mlx/issues
9
+ Project-URL: Benchmarks, https://github.com/mizorewww/laya-mlx/blob/main/BENCHMARKS.md
10
+ License-Expression: Apache-2.0
11
+ License-File: LICENSE
12
+ License-File: NOTICE
13
+ Keywords: apple-silicon,decision-model,laya,local-ai,mlx,typed-decisions
14
+ Classifier: Development Status :: 4 - Beta
15
+ Classifier: Intended Audience :: Developers
16
+ Classifier: Operating System :: MacOS
17
+ Classifier: Programming Language :: Python :: 3
18
+ Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
19
+ Requires-Python: >=3.11
20
+ Requires-Dist: huggingface-hub<2,>=0.34
21
+ Requires-Dist: mlx<0.33,>=0.32.2; sys_platform == 'darwin' and platform_machine == 'arm64'
22
+ Requires-Dist: numpy>=1.26
23
+ Requires-Dist: tokenizers<1,>=0.21
24
+ Provides-Extra: benchmark
25
+ Requires-Dist: matplotlib>=3.10; extra == 'benchmark'
26
+ Requires-Dist: pyarrow>=18; extra == 'benchmark'
27
+ Provides-Extra: demo
28
+ Requires-Dist: pillow<13,>=12; extra == 'demo'
29
+ Requires-Dist: rich<16,>=15; extra == 'demo'
30
+ Provides-Extra: dev
31
+ Requires-Dist: build>=1; extra == 'dev'
32
+ Requires-Dist: pytest>=8; extra == 'dev'
33
+ Requires-Dist: ruff>=0.12; extra == 'dev'
34
+ Provides-Extra: reference
35
+ Requires-Dist: safetensors>=0.6; extra == 'reference'
36
+ Requires-Dist: torch<3,>=2.14; extra == 'reference'
37
+ Requires-Dist: transformers<6,>=5.17; extra == 'reference'
38
+ Description-Content-Type: text/markdown
39
+
40
+ # Laya-MLX
41
+
42
+ ![Laya MLX playing Snake — actual decisions, original speed](https://raw.githubusercontent.com/mizorewww/laya-mlx/main/docs/assets/snake-demo.gif)
43
+
44
+ **Open-weight typed decisions, running natively on Apple Silicon.**
45
+
46
+ **13.4 ms** median end-to-end for a short English typed decision. **7.4 ms** with the multilingual checkpoint. **0 output tokens.** Local MLX inference, with no PyTorch, Transformers runtime, or cloud API.
47
+
48
+ [中文](https://github.com/mizorewww/laya-mlx/blob/main/README.zh-CN.md) · [Benchmarks](https://github.com/mizorewww/laya-mlx/blob/main/BENCHMARKS.md) · [Snake demo](https://github.com/mizorewww/laya-mlx/blob/main/docs/SNAKE_DEMO.md) · [Hugging Face weights](https://huggingface.co/aac6fef/laya-mlx)
49
+
50
+ The GIF is an original-speed render of a real local Snake run. Every move calls Laya; the visible cycle safety layer can correct unsafe proposals. The latency figures above are the separate **one-question API benchmark**, not the frame time of the three-question Snake loop. [Watch the 30-second MP4](https://github.com/mizorewww/laya-mlx/blob/main/docs/assets/snake-demo.mp4) · [Snake speed and stability](https://github.com/mizorewww/laya-mlx/blob/main/docs/SNAKE_BENCHMARKS.md).
51
+
52
+ ## Quick start
53
+
54
+ ```bash
55
+ pip install laya-mlx
56
+ ```
57
+
58
+ ```python
59
+ import laya_mlx as laya
60
+
61
+ agent = laya.load("aac6fef/laya-mlx")
62
+ result = agent.predict(
63
+ "I was billed twice. Please refund the duplicate.",
64
+ {
65
+ "department": {
66
+ "type": "choice",
67
+ "instructions": "Who should handle this?",
68
+ "criteria": ["billing", "technical", "sales"],
69
+ }
70
+ },
71
+ )
72
+ print(result["answers"]["department"])
73
+ ```
74
+
75
+ Apple Silicon, Python 3.11+, macOS 14+. First load downloads the checkpoint; later inference is fully local. The measured environment is macOS 27.2, Python 3.12.13 and MLX 0.32.2. That MLX release supplies macOS 14, 15 and 26 wheels; the local installer selected the 26 wheel. Older supported macOS versions were not tested on this machine.
76
+
77
+ Run the terminal demo:
78
+
79
+ ```bash
80
+ pip install 'laya-mlx[demo]'
81
+ hf download aac6fef/laya-multilingual-mlx
82
+ laya-snake
83
+ ```
84
+
85
+ Download once before the offline demo. Use a terminal at least 104 × 35 cells. Space pauses, ↑/↓ changes speed, R resets and Q quits. `laya-snake --max-speed` makes a fresh decision for every move without pacing. [Recording, controls and exact metric meanings](https://github.com/mizorewww/laya-mlx/blob/main/docs/SNAKE_DEMO.md).
86
+
87
+ `laya-snake --optimize --max-speed` enables the tested compilation and prefix-reuse path: **75.40 moves/s across 2,400 moves**, zero deaths and 2 visible safety interventions in the paired M3 Max test. This was about **6.5% faster** than its same-run eager control. [Gameplay, performance and correctness evidence](https://github.com/mizorewww/laya-mlx/blob/main/docs/SNAKE_OPTIMIZATION.md).
88
+
89
+ ## Performance on M3 Max
90
+
91
+ | FP16, end-to-end | Laya 421M | Multilingual 322M |
92
+ |---|---:|---:|
93
+ | One short question, P50 | **13.42 ms** | **7.39 ms** |
94
+ | One short question, P95 | **13.92 ms** | **7.79 ms** |
95
+ | 50-question throughput | **146.8 q/s** | **395.0 q/s** |
96
+ | Peak MLX allocation, one short question | **943.6 MiB** | **687.6 MiB** |
97
+
98
+ M3 Max, 40 GPU cores, 128 GiB memory. Timing includes prompt preparation, tokenization, tensors, synchronized inference, calibration and result formatting; model loading is excluded. The 50-question measurement uses `batch_size=64`; the API defaults to 16. Different lengths, question counts and runtime conditions change latency. [Full method and every timing sample](https://github.com/mizorewww/laya-mlx/blob/main/BENCHMARKS.md).
99
+
100
+ **Port fidelity:** all three checkpoints matched the upstream selected answer on **63/63 validation questions in both FP32 and FP16** — 378/378 comparisons. Each configuration also passed 100 repeated finite, deterministic calls with zero measured active-memory growth. This measures fidelity on those fixtures, not accuracy on every possible question. [Probability errors and validation](https://github.com/mizorewww/laya-mlx/blob/main/BENCHMARKS.md#numerical-parity-and-stability).
101
+
102
+ ## Why typed decisions?
103
+
104
+ Software often needs a choice, a rubric score or a probability. Laya answers those constrained questions in a bidirectional forward pass, without token-by-token decoding or generated JSON.
105
+
106
+ ```text
107
+ state + typed question → bidirectional encoder → decision heads → probabilities
108
+ ```
109
+
110
+ - `choice`: probabilities over named options.
111
+ - `score`: probabilities over ordered rubric levels and their expected score.
112
+ - `noul`: P(true) for a proposition.
113
+
114
+ Question rows are batched independently. Their bidirectional encoder representations depend on both state and question; this runtime does not claim to encode the state once and reuse its hidden states across arbitrary questions.
115
+
116
+ The encoder, decision Transformer, scoring head and action head all run in MLX. Tokenization uses Hugging Face's Rust tokenizer. The original pretrained weights, question formatting, calibration and output schema are retained. This is an independent MLX port, not an official Convai Innovations release.
117
+
118
+ ## Supported checkpoints
119
+
120
+ | Model | Encoder | Parameters | Context limit | Purpose |
121
+ |---|---|---:|---:|---|
122
+ | `convaiinnovations/laya` | ModernBERT-large | 421M | 512 | English |
123
+ | `convaiinnovations/laya-multilingual` | mmBERT-base | 322M | 1,024 | Multilingual input |
124
+ | `convaiinnovations/laya-typed-decisions` | ModernBERT-large | 421M | 1,024 | Upstream typed-decisions workflows |
125
+
126
+ Context includes instructions, options and state. All three use the original weights, prompt formatting, temperature calibration, and output schema. This repository provides inference and conversion; RLCD training and fine-tuning remain in the upstream project. It is an independent port, not an official Convai Innovations release.
127
+
128
+ Pre-converted FP16 checkpoints are published on Hugging Face:
129
+
130
+ - [aac6fef/laya-mlx](https://huggingface.co/aac6fef/laya-mlx)
131
+ - [aac6fef/laya-multilingual-mlx](https://huggingface.co/aac6fef/laya-multilingual-mlx)
132
+ - [aac6fef/laya-typed-decisions-mlx](https://huggingface.co/aac6fef/laya-typed-decisions-mlx)
133
+
134
+ Load these directly with `laya.load("aac6fef/laya-mlx")`, or use the original checkpoint IDs above. Each published checkpoint includes its model card, validation results, provenance, license and file checksums. All 36 published files passed strict remote checksum verification; pinned revisions and weight hashes are recorded in [hub-publication.json](https://github.com/mizorewww/laya-mlx/blob/main/benchmarks/results/hub-publication.json).
135
+
136
+ ## Development install
137
+
138
+ ```bash
139
+ gh repo clone mizorewww/laya-mlx
140
+ cd laya-mlx
141
+ uv sync --extra demo
142
+ uv run --extra demo laya-snake
143
+ ```
144
+
145
+ Or install the latest GitHub revision with `pip install 'git+https://github.com/mizorewww/laya-mlx.git'`. Model weights are downloaded separately and are excluded from Git.
146
+
147
+ ## Python API
148
+
149
+ ```python
150
+ import laya_mlx as laya
151
+
152
+ agent = laya.load("aac6fef/laya-mlx", dtype="float16")
153
+ result = agent.predict(
154
+ "I was billed twice. Please refund the duplicate today.",
155
+ {
156
+ "department": {
157
+ "type": "choice",
158
+ "instructions": "Which team should handle this request?",
159
+ "criteria": {
160
+ "billing": "invoices, payments, refunds",
161
+ "technical": "bugs and outages",
162
+ "sales": "new purchases",
163
+ },
164
+ },
165
+ "urgency": {
166
+ "type": "score",
167
+ "instructions": "How urgent is this request?",
168
+ "criteria": ["not urgent", "soon", "critical"],
169
+ },
170
+ "refund": {
171
+ "type": "noul",
172
+ "instructions": "Does the customer ask for money back?",
173
+ },
174
+ },
175
+ )
176
+ print(result["answers"])
177
+ ```
178
+
179
+ `system_one` is an alias for `predict`. States can be text, JSON dictionaries, or conversation lists. `choice` accepts a dictionary or a list of unique labels; `score` returns the expected zero-based rubric level; `noul` returns P(true). Results retain upstream's four-decimal rounding, `action.act_probability`, and token usage fields.
180
+
181
+ The default precision is FP16. Use `dtype="float32"` for closer numerical agreement. Probabilities can differ slightly across precisions even when the selected label agrees; see the measured errors in [BENCHMARKS.md](https://github.com/mizorewww/laya-mlx/blob/main/BENCHMARKS.md). BF16 can be requested but is not part of the published validation matrix.
182
+
183
+ `batch_size=16` caps the number of questions per forward pass; larger requests are processed in chunks. Increase it when memory allows. `device="gpu"` or `device="cpu"` selects a device explicitly; otherwise MLX's default device is used.
184
+
185
+ For repeated workloads, opt into `compile=True`, `pad_to_multiple=16` and `cache_prompts=True` when loading an Agent. The prefix cache is bounded to 128 questions and shares CPU state tokenization, while every question still gets its own encoder computation. Compilation has a first-use cost and shape specialization; padding may make some workloads slower. All three options default to disabled. [Measured Snake ablation and usage](https://github.com/mizorewww/laya-mlx/blob/main/docs/SNAKE_OPTIMIZATION.md).
186
+
187
+ ```python
188
+ agent = laya.load("./models/laya", dtype="float32", batch_size=32)
189
+ # Select one checkpoint inside upstream's bundled repository:
190
+ multi = laya.load("convaiinnovations/laya", subfolder="multilingual")
191
+ # Pin a Hub revision for reproducibility:
192
+ agent = laya.load(
193
+ "convaiinnovations/laya",
194
+ revision="c5d78730f3493e4fe16d61507ef4b78eef7318cf",
195
+ )
196
+ ```
197
+
198
+ Loading validates every parameter name and shape. Unsupported encoders and non-default RoPE scaling fail explicitly. ModernBERT's global/local attention pattern, inclusive sliding-window boundary, distinct local/global RoPE bases, and first-layer normalization behavior are preserved.
199
+
200
+ ## Language routing and presets
201
+
202
+ ```python
203
+ from laya_mlx import Router, triage_questions
204
+
205
+ router = Router(dtype="float16", max_loaded=2)
206
+ result = router.predict({"message": "发票被重复扣款,请退款。"}, triage_questions())
207
+ print(result["routing"]) # multilingual
208
+
209
+ # Choose the specialized checkpoint explicitly:
210
+ result = router.predict(state, questions, task="typed_decisions")
211
+ ```
212
+
213
+ The router, language heuristics, email helpers and application presets are adapted from upstream. `Router(preload=True)` keeps all three checkpoints resident; `attach`, `preload`, `unload`, explicit `lang=`, and explicit `model=` are supported. Typed-decisions workflow detection stays opt-in. The port preserves model limitations: English checkpoints are not substitutes for the multilingual checkpoint, and confidence does not guarantee accuracy.
214
+
215
+ ## Command line
216
+
217
+ ```bash
218
+ uv run laya-mlx predict \
219
+ --model aac6fef/laya-mlx \
220
+ --state-file examples/state.json \
221
+ --questions examples/questions.json
222
+
223
+ uv run laya-mlx predict \
224
+ --model aac6fef/laya-multilingual-mlx \
225
+ --state '发票被重复扣款,请退款。' \
226
+ --questions examples/questions.json
227
+ ```
228
+
229
+ ## Export an MLX checkpoint
230
+
231
+ ```bash
232
+ uv run laya-mlx convert \
233
+ --model convaiinnovations/laya \
234
+ --dtype float16 \
235
+ --output models/laya-mlx-fp16
236
+
237
+ uv run laya-mlx predict \
238
+ --model models/laya-mlx-fp16 \
239
+ --state-file examples/state.json \
240
+ --questions examples/questions.json
241
+ ```
242
+
243
+ The export contains `model.safetensors`, encoder and agent configurations, tokenizer files and `mlx_config.json`. Existing output directories are never overwritten. This is a parameter-name/dtype conversion, not quantization or retraining. The source checkpoints already store FP16 weights; choosing FP32 increases arithmetic precision, not the precision of the source weights.
244
+
245
+ ## Tests and benchmarks
246
+
247
+ ```bash
248
+ uv sync --extra dev --extra reference --extra benchmark --extra demo
249
+ source .venv/bin/activate
250
+ gh repo clone NandhaKishorM/laya .upstream
251
+ git -C .upstream checkout 6a5819129eb220570792e417e49723d697efd76f
252
+ pytest -q
253
+ python -m benchmarks.download
254
+ python -m benchmarks.validate --repeats 100
255
+ python -m benchmarks.run --iterations 50 --warmup 5
256
+ python -m benchmarks.accuracy --per-class 64
257
+ python -m benchmarks.report
258
+ ```
259
+
260
+ Run GPU measurements sequentially. Unit tests use small random models and include direct comparisons with Transformers and the pinned upstream decision head. Real checkpoint validation tests tokenization, logits, calibrated probabilities, repeated outputs and active memory growth. The benchmark runs each backend/checkpoint in a fresh process and stores every timing sample in [benchmarks/results](https://github.com/mizorewww/laya-mlx/blob/main/benchmarks/results). The [full report](https://github.com/mizorewww/laya-mlx/blob/main/BENCHMARKS.md) explains the timing boundaries and precision differences.
261
+
262
+ GitHub Actions runs small-model CPU tests on a macOS arm64 runner. Full checkpoint GPU benchmarks are measured locally and are not part of hosted CI.
263
+
264
+ ## Performance research
265
+
266
+ The performance investigations include both mathematical analysis and independent local experiments:
267
+
268
+ - [Initial performance research](https://github.com/mizorewww/laya-mlx/blob/main/docs/PERFORMANCE_RESEARCH.md): implementation bottlenecks, MLX kernel dispatch, and a controlled experiment plan.
269
+ - [Mathematical investigation of a further 10× speedup](https://github.com/mizorewww/laya-mlx/blob/main/docs/MATH_10X_RESEARCH.md): arithmetic budgets, conditional bandwidth bounds, real weight spectra, exact reuse, and smaller-model designs.
270
+ - [Engineering investigation](https://github.com/mizorewww/laya-mlx/blob/main/docs/ENGINEERING_10X_RESEARCH.md): measured compilation, quantization, final-head selection, custom Metal kernels, and representative matrix multiplications.
271
+
272
+ [experiments/](https://github.com/mizorewww/laya-mlx/blob/main/experiments) contains the research scripts and their raw measurements. The published runtime's performance and validation results are in [BENCHMARKS.md](https://github.com/mizorewww/laya-mlx/blob/main/BENCHMARKS.md); each experimental variant has its own timing and correctness results.
273
+
274
+ The current investigation does not support a further universal 10× speedup with the same checkpoints. Selected cases show approximately 1.03–1.08× paired median speedups; the engineering report gives the uncertainty intervals, quantization fidelity results, and custom Metal kernel measurements.
275
+
276
+ To prepare model cards and verified exports for publication, install the reference extras and run:
277
+
278
+ ```bash
279
+ python -m scripts.prepare_hub --account YOUR_HF_USERNAME
280
+ hf upload YOUR_HF_USERNAME/laya-mlx models/hub/laya-mlx . --exclude '.cache/*'
281
+ ```
282
+
283
+ The preparation script checks every exported tensor against its original FP16 source. Upload the other two prepared folders in the same way, then use `hf cache verify REPO_ID --local-dir EXPORT_PATH` to check the remote files.
284
+
285
+ ## Attribution and license
286
+
287
+ Apache-2.0; see [LICENSE](https://github.com/mizorewww/laya-mlx/blob/main/LICENSE) and [NOTICE](https://github.com/mizorewww/laya-mlx/blob/main/NOTICE). Laya and its pretrained weights are by Convai Innovations and upstream contributors. Prompt construction, output formatting, language routing, email utilities and presets are adapted from [NandhaKishorM/laya](https://github.com/NandhaKishorM/laya) at commit `6a5819129eb220570792e417e49723d697efd76f`. The neural architecture is reimplemented in MLX following Laya and Hugging Face ModernBERT.