laya-mlx 0.1.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- laya_mlx-0.1.0/.gitignore +15 -0
- laya_mlx-0.1.0/LICENSE +176 -0
- laya_mlx-0.1.0/NOTICE +14 -0
- laya_mlx-0.1.0/PKG-INFO +287 -0
- laya_mlx-0.1.0/README.md +248 -0
- laya_mlx-0.1.0/laya_mlx/__init__.py +34 -0
- laya_mlx-0.1.0/laya_mlx/__main__.py +3 -0
- laya_mlx-0.1.0/laya_mlx/agent.py +262 -0
- laya_mlx-0.1.0/laya_mlx/cli.py +55 -0
- laya_mlx-0.1.0/laya_mlx/common.py +119 -0
- laya_mlx-0.1.0/laya_mlx/convert.py +48 -0
- laya_mlx-0.1.0/laya_mlx/email.py +101 -0
- laya_mlx-0.1.0/laya_mlx/lang.py +367 -0
- laya_mlx-0.1.0/laya_mlx/model.py +249 -0
- laya_mlx-0.1.0/laya_mlx/prepared.py +59 -0
- laya_mlx-0.1.0/laya_mlx/presets.py +193 -0
- laya_mlx-0.1.0/laya_mlx/router.py +378 -0
- laya_mlx-0.1.0/laya_mlx/snake/__init__.py +1 -0
- laya_mlx-0.1.0/laya_mlx/snake/__main__.py +3 -0
- laya_mlx-0.1.0/laya_mlx/snake/benchmark.py +293 -0
- laya_mlx-0.1.0/laya_mlx/snake/cli.py +294 -0
- laya_mlx-0.1.0/laya_mlx/snake/game.py +156 -0
- laya_mlx-0.1.0/laya_mlx/snake/policy.py +220 -0
- laya_mlx-0.1.0/laya_mlx/snake/replay.py +256 -0
- laya_mlx-0.1.0/laya_mlx/snake/ui.py +169 -0
- laya_mlx-0.1.0/laya_mlx/tokenizer.py +27 -0
- laya_mlx-0.1.0/pyproject.toml +59 -0
laya_mlx-0.1.0/LICENSE
ADDED
|
@@ -0,0 +1,176 @@
|
|
|
1
|
+
Apache License
|
|
2
|
+
Version 2.0, January 2004
|
|
3
|
+
http://www.apache.org/licenses/
|
|
4
|
+
|
|
5
|
+
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
|
|
6
|
+
|
|
7
|
+
1. Definitions.
|
|
8
|
+
|
|
9
|
+
"License" shall mean the terms and conditions for use, reproduction,
|
|
10
|
+
and distribution as defined by Sections 1 through 9 of this document.
|
|
11
|
+
|
|
12
|
+
"Licensor" shall mean the copyright owner or entity authorized by
|
|
13
|
+
the copyright owner that is granting the License.
|
|
14
|
+
|
|
15
|
+
"Legal Entity" shall mean the union of the acting entity and all
|
|
16
|
+
other entities that control, are controlled by, or are under common
|
|
17
|
+
control with that entity. For the purposes of this definition,
|
|
18
|
+
"control" means (i) the power, direct or indirect, to cause the
|
|
19
|
+
direction or management of such entity, whether by contract or
|
|
20
|
+
otherwise, or (ii) ownership of fifty percent (50%) or more of the
|
|
21
|
+
outstanding shares, or (iii) beneficial ownership of such entity.
|
|
22
|
+
|
|
23
|
+
"You" (or "Your") shall mean an individual or Legal Entity
|
|
24
|
+
exercising permissions granted by this License.
|
|
25
|
+
|
|
26
|
+
"Source" form shall mean the preferred form for making modifications,
|
|
27
|
+
including but not limited to software source code, documentation
|
|
28
|
+
source, and configuration files.
|
|
29
|
+
|
|
30
|
+
"Object" form shall mean any form resulting from mechanical
|
|
31
|
+
transformation or translation of a Source form, including but
|
|
32
|
+
not limited to compiled object code, generated documentation,
|
|
33
|
+
and conversions to other media types.
|
|
34
|
+
|
|
35
|
+
"Work" shall mean the work of authorship, whether in Source or
|
|
36
|
+
Object form, made available under the License, as indicated by a
|
|
37
|
+
copyright notice that is included in or attached to the work
|
|
38
|
+
(an example is provided in the Appendix below).
|
|
39
|
+
|
|
40
|
+
"Derivative Works" shall mean any work, whether in Source or Object
|
|
41
|
+
form, that is based on (or derived from) the Work and for which the
|
|
42
|
+
editorial revisions, annotations, elaborations, or other modifications
|
|
43
|
+
represent, as a whole, an original work of authorship. For the purposes
|
|
44
|
+
of this License, Derivative Works shall not include works that remain
|
|
45
|
+
separable from, or merely link (or bind by name) to the interfaces of,
|
|
46
|
+
the Work and Derivative Works thereof.
|
|
47
|
+
|
|
48
|
+
"Contribution" shall mean any work of authorship, including
|
|
49
|
+
the original version of the Work and any modifications or additions
|
|
50
|
+
to that Work or Derivative Works thereof, that is intentionally
|
|
51
|
+
submitted to Licensor for inclusion in the Work by the copyright owner
|
|
52
|
+
or by an individual or Legal Entity authorized to submit on behalf of
|
|
53
|
+
the copyright owner. For the purposes of this definition, "submitted"
|
|
54
|
+
means any form of electronic, verbal, or written communication sent
|
|
55
|
+
to the Licensor or its representatives, including but not limited to
|
|
56
|
+
communication on electronic mailing lists, source code control systems,
|
|
57
|
+
and issue tracking systems that are managed by, or on behalf of, the
|
|
58
|
+
Licensor for the purpose of discussing and improving the Work, but
|
|
59
|
+
excluding communication that is conspicuously marked or otherwise
|
|
60
|
+
designated in writing by the copyright owner as "Not a Contribution."
|
|
61
|
+
|
|
62
|
+
"Contributor" shall mean Licensor and any individual or Legal Entity
|
|
63
|
+
on behalf of whom a Contribution has been received by Licensor and
|
|
64
|
+
subsequently incorporated within the Work.
|
|
65
|
+
|
|
66
|
+
2. Grant of Copyright License. Subject to the terms and conditions of
|
|
67
|
+
this License, each Contributor hereby grants to You a perpetual,
|
|
68
|
+
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
|
69
|
+
copyright license to reproduce, prepare Derivative Works of,
|
|
70
|
+
publicly display, publicly perform, sublicense, and distribute the
|
|
71
|
+
Work and such Derivative Works in Source or Object form.
|
|
72
|
+
|
|
73
|
+
3. Grant of Patent License. Subject to the terms and conditions of
|
|
74
|
+
this License, each Contributor hereby grants to You a perpetual,
|
|
75
|
+
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
|
76
|
+
(except as stated in this section) patent license to make, have made,
|
|
77
|
+
use, offer to sell, sell, import, and otherwise transfer the Work,
|
|
78
|
+
where such license applies only to those patent claims licensable
|
|
79
|
+
by such Contributor that are necessarily infringed by their
|
|
80
|
+
Contribution(s) alone or by combination of their Contribution(s)
|
|
81
|
+
with the Work to which such Contribution(s) was submitted. If You
|
|
82
|
+
institute patent litigation against any entity (including a
|
|
83
|
+
cross-claim or counterclaim in a lawsuit) alleging that the Work
|
|
84
|
+
or a Contribution incorporated within the Work constitutes direct
|
|
85
|
+
or contributory patent infringement, then any patent licenses
|
|
86
|
+
granted to You under this License for that Work shall terminate
|
|
87
|
+
as of the date such litigation is filed.
|
|
88
|
+
|
|
89
|
+
4. Redistribution. You may reproduce and distribute copies of the
|
|
90
|
+
Work or Derivative Works thereof in any medium, with or without
|
|
91
|
+
modifications, and in Source or Object form, provided that You
|
|
92
|
+
meet the following conditions:
|
|
93
|
+
|
|
94
|
+
(a) You must give any other recipients of the Work or
|
|
95
|
+
Derivative Works a copy of this License; and
|
|
96
|
+
|
|
97
|
+
(b) You must cause any modified files to carry prominent notices
|
|
98
|
+
stating that You changed the files; and
|
|
99
|
+
|
|
100
|
+
(c) You must retain, in the Source form of any Derivative Works
|
|
101
|
+
that You distribute, all copyright, patent, trademark, and
|
|
102
|
+
attribution notices from the Source form of the Work,
|
|
103
|
+
excluding those notices that do not pertain to any part of
|
|
104
|
+
the Derivative Works; and
|
|
105
|
+
|
|
106
|
+
(d) If the Work includes a "NOTICE" text file as part of its
|
|
107
|
+
distribution, then any Derivative Works that You distribute must
|
|
108
|
+
include a readable copy of the attribution notices contained
|
|
109
|
+
within such NOTICE file, excluding those notices that do not
|
|
110
|
+
pertain to any part of the Derivative Works, in at least one
|
|
111
|
+
of the following places: within a NOTICE text file distributed
|
|
112
|
+
as part of the Derivative Works; within the Source form or
|
|
113
|
+
documentation, if provided along with the Derivative Works; or,
|
|
114
|
+
within a display generated by the Derivative Works, if and
|
|
115
|
+
wherever such third-party notices normally appear. The contents
|
|
116
|
+
of the NOTICE file are for informational purposes only and
|
|
117
|
+
do not modify the License. You may add Your own attribution
|
|
118
|
+
notices within Derivative Works that You distribute, alongside
|
|
119
|
+
or as an addendum to the NOTICE text from the Work, provided
|
|
120
|
+
that such additional attribution notices cannot be construed
|
|
121
|
+
as modifying the License.
|
|
122
|
+
|
|
123
|
+
You may add Your own copyright statement to Your modifications and
|
|
124
|
+
may provide additional or different license terms and conditions
|
|
125
|
+
for use, reproduction, or distribution of Your modifications, or
|
|
126
|
+
for any such Derivative Works as a whole, provided Your use,
|
|
127
|
+
reproduction, and distribution of the Work otherwise complies with
|
|
128
|
+
the conditions stated in this License.
|
|
129
|
+
|
|
130
|
+
5. Submission of Contributions. Unless You explicitly state otherwise,
|
|
131
|
+
any Contribution intentionally submitted for inclusion in the Work
|
|
132
|
+
by You to the Licensor shall be under the terms and conditions of
|
|
133
|
+
this License, without any additional terms or conditions.
|
|
134
|
+
Notwithstanding the above, nothing herein shall supersede or modify
|
|
135
|
+
the terms of any separate license agreement you may have executed
|
|
136
|
+
with Licensor regarding such Contributions.
|
|
137
|
+
|
|
138
|
+
6. Trademarks. This License does not grant permission to use the trade
|
|
139
|
+
names, trademarks, service marks, or product names of the Licensor,
|
|
140
|
+
except as required for reasonable and customary use in describing the
|
|
141
|
+
origin of the Work and reproducing the content of the NOTICE file.
|
|
142
|
+
|
|
143
|
+
7. Disclaimer of Warranty. Unless required by applicable law or
|
|
144
|
+
agreed to in writing, Licensor provides the Work (and each
|
|
145
|
+
Contributor provides its Contributions) on an "AS IS" BASIS,
|
|
146
|
+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
|
|
147
|
+
implied, including, without limitation, any warranties or conditions
|
|
148
|
+
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
|
|
149
|
+
PARTICULAR PURPOSE. You are solely responsible for determining the
|
|
150
|
+
appropriateness of using or redistributing the Work and assume any
|
|
151
|
+
risks associated with Your exercise of permissions under this License.
|
|
152
|
+
|
|
153
|
+
8. Limitation of Liability. In no event and under no legal theory,
|
|
154
|
+
whether in tort (including negligence), contract, or otherwise,
|
|
155
|
+
unless required by applicable law (such as deliberate and grossly
|
|
156
|
+
negligent acts) or agreed to in writing, shall any Contributor be
|
|
157
|
+
liable to You for damages, including any direct, indirect, special,
|
|
158
|
+
incidental, or consequential damages of any character arising as a
|
|
159
|
+
result of this License or out of the use or inability to use the
|
|
160
|
+
Work (including but not limited to damages for loss of goodwill,
|
|
161
|
+
work stoppage, computer failure or malfunction, or any and all
|
|
162
|
+
other commercial damages or losses), even if such Contributor
|
|
163
|
+
has been advised of the possibility of such damages.
|
|
164
|
+
|
|
165
|
+
9. Accepting Warranty or Additional Liability. While redistributing
|
|
166
|
+
the Work or Derivative Works thereof, You may choose to offer,
|
|
167
|
+
and charge a fee for, acceptance of support, warranty, indemnity,
|
|
168
|
+
or other liability obligations and/or rights consistent with this
|
|
169
|
+
License. However, in accepting such obligations, You may act only
|
|
170
|
+
on Your own behalf and on Your sole responsibility, not on behalf
|
|
171
|
+
of any other Contributor, and only if You agree to indemnify,
|
|
172
|
+
defend, and hold each Contributor harmless for any liability
|
|
173
|
+
incurred by, or claims asserted against, such Contributor by reason
|
|
174
|
+
of your accepting any such warranty or additional liability.
|
|
175
|
+
|
|
176
|
+
END OF TERMS AND CONDITIONS
|
laya_mlx-0.1.0/NOTICE
ADDED
|
@@ -0,0 +1,14 @@
|
|
|
1
|
+
laya-mlx
|
|
2
|
+
Copyright 2026 laya-mlx contributors
|
|
3
|
+
|
|
4
|
+
This product includes software derived from Laya:
|
|
5
|
+
https://github.com/NandhaKishorM/laya
|
|
6
|
+
Copyright Convai Innovations and Laya contributors. Licensed under Apache-2.0.
|
|
7
|
+
Upstream source revision: 6a5819129eb220570792e417e49723d697efd76f
|
|
8
|
+
|
|
9
|
+
The token sequence construction, question rendering, confidence calculation,
|
|
10
|
+
presets, email utilities and language router are adapted from Laya.
|
|
11
|
+
The neural network is reimplemented using Apple's MLX, following Laya's
|
|
12
|
+
DecisionModel and the ModernBERT architecture in Hugging Face Transformers.
|
|
13
|
+
Model weights are downloaded separately from Convai Innovations on Hugging Face;
|
|
14
|
+
they are not included in this repository.
|
laya_mlx-0.1.0/PKG-INFO
ADDED
|
@@ -0,0 +1,287 @@
|
|
|
1
|
+
Metadata-Version: 2.5
|
|
2
|
+
Name: laya-mlx
|
|
3
|
+
Version: 0.1.0
|
|
4
|
+
Summary: Native MLX inference for Laya typed decision models on Apple silicon
|
|
5
|
+
Project-URL: Repository, https://github.com/mizorewww/laya-mlx
|
|
6
|
+
Project-URL: Upstream, https://github.com/NandhaKishorM/laya
|
|
7
|
+
Project-URL: Documentation, https://github.com/mizorewww/laya-mlx#readme
|
|
8
|
+
Project-URL: Issues, https://github.com/mizorewww/laya-mlx/issues
|
|
9
|
+
Project-URL: Benchmarks, https://github.com/mizorewww/laya-mlx/blob/main/BENCHMARKS.md
|
|
10
|
+
License-Expression: Apache-2.0
|
|
11
|
+
License-File: LICENSE
|
|
12
|
+
License-File: NOTICE
|
|
13
|
+
Keywords: apple-silicon,decision-model,laya,local-ai,mlx,typed-decisions
|
|
14
|
+
Classifier: Development Status :: 4 - Beta
|
|
15
|
+
Classifier: Intended Audience :: Developers
|
|
16
|
+
Classifier: Operating System :: MacOS
|
|
17
|
+
Classifier: Programming Language :: Python :: 3
|
|
18
|
+
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
|
|
19
|
+
Requires-Python: >=3.11
|
|
20
|
+
Requires-Dist: huggingface-hub<2,>=0.34
|
|
21
|
+
Requires-Dist: mlx<0.33,>=0.32.2; sys_platform == 'darwin' and platform_machine == 'arm64'
|
|
22
|
+
Requires-Dist: numpy>=1.26
|
|
23
|
+
Requires-Dist: tokenizers<1,>=0.21
|
|
24
|
+
Provides-Extra: benchmark
|
|
25
|
+
Requires-Dist: matplotlib>=3.10; extra == 'benchmark'
|
|
26
|
+
Requires-Dist: pyarrow>=18; extra == 'benchmark'
|
|
27
|
+
Provides-Extra: demo
|
|
28
|
+
Requires-Dist: pillow<13,>=12; extra == 'demo'
|
|
29
|
+
Requires-Dist: rich<16,>=15; extra == 'demo'
|
|
30
|
+
Provides-Extra: dev
|
|
31
|
+
Requires-Dist: build>=1; extra == 'dev'
|
|
32
|
+
Requires-Dist: pytest>=8; extra == 'dev'
|
|
33
|
+
Requires-Dist: ruff>=0.12; extra == 'dev'
|
|
34
|
+
Provides-Extra: reference
|
|
35
|
+
Requires-Dist: safetensors>=0.6; extra == 'reference'
|
|
36
|
+
Requires-Dist: torch<3,>=2.14; extra == 'reference'
|
|
37
|
+
Requires-Dist: transformers<6,>=5.17; extra == 'reference'
|
|
38
|
+
Description-Content-Type: text/markdown
|
|
39
|
+
|
|
40
|
+
# Laya-MLX
|
|
41
|
+
|
|
42
|
+

|
|
43
|
+
|
|
44
|
+
**Open-weight typed decisions, running natively on Apple Silicon.**
|
|
45
|
+
|
|
46
|
+
**13.4 ms** median end-to-end for a short English typed decision. **7.4 ms** with the multilingual checkpoint. **0 output tokens.** Local MLX inference, with no PyTorch, Transformers runtime, or cloud API.
|
|
47
|
+
|
|
48
|
+
[中文](https://github.com/mizorewww/laya-mlx/blob/main/README.zh-CN.md) · [Benchmarks](https://github.com/mizorewww/laya-mlx/blob/main/BENCHMARKS.md) · [Snake demo](https://github.com/mizorewww/laya-mlx/blob/main/docs/SNAKE_DEMO.md) · [Hugging Face weights](https://huggingface.co/aac6fef/laya-mlx)
|
|
49
|
+
|
|
50
|
+
The GIF is an original-speed render of a real local Snake run. Every move calls Laya; the visible cycle safety layer can correct unsafe proposals. The latency figures above are the separate **one-question API benchmark**, not the frame time of the three-question Snake loop. [Watch the 30-second MP4](https://github.com/mizorewww/laya-mlx/blob/main/docs/assets/snake-demo.mp4) · [Snake speed and stability](https://github.com/mizorewww/laya-mlx/blob/main/docs/SNAKE_BENCHMARKS.md).
|
|
51
|
+
|
|
52
|
+
## Quick start
|
|
53
|
+
|
|
54
|
+
```bash
|
|
55
|
+
pip install laya-mlx
|
|
56
|
+
```
|
|
57
|
+
|
|
58
|
+
```python
|
|
59
|
+
import laya_mlx as laya
|
|
60
|
+
|
|
61
|
+
agent = laya.load("aac6fef/laya-mlx")
|
|
62
|
+
result = agent.predict(
|
|
63
|
+
"I was billed twice. Please refund the duplicate.",
|
|
64
|
+
{
|
|
65
|
+
"department": {
|
|
66
|
+
"type": "choice",
|
|
67
|
+
"instructions": "Who should handle this?",
|
|
68
|
+
"criteria": ["billing", "technical", "sales"],
|
|
69
|
+
}
|
|
70
|
+
},
|
|
71
|
+
)
|
|
72
|
+
print(result["answers"]["department"])
|
|
73
|
+
```
|
|
74
|
+
|
|
75
|
+
Apple Silicon, Python 3.11+, macOS 14+. First load downloads the checkpoint; later inference is fully local. The measured environment is macOS 27.2, Python 3.12.13 and MLX 0.32.2. That MLX release supplies macOS 14, 15 and 26 wheels; the local installer selected the 26 wheel. Older supported macOS versions were not tested on this machine.
|
|
76
|
+
|
|
77
|
+
Run the terminal demo:
|
|
78
|
+
|
|
79
|
+
```bash
|
|
80
|
+
pip install 'laya-mlx[demo]'
|
|
81
|
+
hf download aac6fef/laya-multilingual-mlx
|
|
82
|
+
laya-snake
|
|
83
|
+
```
|
|
84
|
+
|
|
85
|
+
Download once before the offline demo. Use a terminal at least 104 × 35 cells. Space pauses, ↑/↓ changes speed, R resets and Q quits. `laya-snake --max-speed` makes a fresh decision for every move without pacing. [Recording, controls and exact metric meanings](https://github.com/mizorewww/laya-mlx/blob/main/docs/SNAKE_DEMO.md).
|
|
86
|
+
|
|
87
|
+
`laya-snake --optimize --max-speed` enables the tested compilation and prefix-reuse path: **75.40 moves/s across 2,400 moves**, zero deaths and 2 visible safety interventions in the paired M3 Max test. This was about **6.5% faster** than its same-run eager control. [Gameplay, performance and correctness evidence](https://github.com/mizorewww/laya-mlx/blob/main/docs/SNAKE_OPTIMIZATION.md).
|
|
88
|
+
|
|
89
|
+
## Performance on M3 Max
|
|
90
|
+
|
|
91
|
+
| FP16, end-to-end | Laya 421M | Multilingual 322M |
|
|
92
|
+
|---|---:|---:|
|
|
93
|
+
| One short question, P50 | **13.42 ms** | **7.39 ms** |
|
|
94
|
+
| One short question, P95 | **13.92 ms** | **7.79 ms** |
|
|
95
|
+
| 50-question throughput | **146.8 q/s** | **395.0 q/s** |
|
|
96
|
+
| Peak MLX allocation, one short question | **943.6 MiB** | **687.6 MiB** |
|
|
97
|
+
|
|
98
|
+
M3 Max, 40 GPU cores, 128 GiB memory. Timing includes prompt preparation, tokenization, tensors, synchronized inference, calibration and result formatting; model loading is excluded. The 50-question measurement uses `batch_size=64`; the API defaults to 16. Different lengths, question counts and runtime conditions change latency. [Full method and every timing sample](https://github.com/mizorewww/laya-mlx/blob/main/BENCHMARKS.md).
|
|
99
|
+
|
|
100
|
+
**Port fidelity:** all three checkpoints matched the upstream selected answer on **63/63 validation questions in both FP32 and FP16** — 378/378 comparisons. Each configuration also passed 100 repeated finite, deterministic calls with zero measured active-memory growth. This measures fidelity on those fixtures, not accuracy on every possible question. [Probability errors and validation](https://github.com/mizorewww/laya-mlx/blob/main/BENCHMARKS.md#numerical-parity-and-stability).
|
|
101
|
+
|
|
102
|
+
## Why typed decisions?
|
|
103
|
+
|
|
104
|
+
Software often needs a choice, a rubric score or a probability. Laya answers those constrained questions in a bidirectional forward pass, without token-by-token decoding or generated JSON.
|
|
105
|
+
|
|
106
|
+
```text
|
|
107
|
+
state + typed question → bidirectional encoder → decision heads → probabilities
|
|
108
|
+
```
|
|
109
|
+
|
|
110
|
+
- `choice`: probabilities over named options.
|
|
111
|
+
- `score`: probabilities over ordered rubric levels and their expected score.
|
|
112
|
+
- `noul`: P(true) for a proposition.
|
|
113
|
+
|
|
114
|
+
Question rows are batched independently. Their bidirectional encoder representations depend on both state and question; this runtime does not claim to encode the state once and reuse its hidden states across arbitrary questions.
|
|
115
|
+
|
|
116
|
+
The encoder, decision Transformer, scoring head and action head all run in MLX. Tokenization uses Hugging Face's Rust tokenizer. The original pretrained weights, question formatting, calibration and output schema are retained. This is an independent MLX port, not an official Convai Innovations release.
|
|
117
|
+
|
|
118
|
+
## Supported checkpoints
|
|
119
|
+
|
|
120
|
+
| Model | Encoder | Parameters | Context limit | Purpose |
|
|
121
|
+
|---|---|---:|---:|---|
|
|
122
|
+
| `convaiinnovations/laya` | ModernBERT-large | 421M | 512 | English |
|
|
123
|
+
| `convaiinnovations/laya-multilingual` | mmBERT-base | 322M | 1,024 | Multilingual input |
|
|
124
|
+
| `convaiinnovations/laya-typed-decisions` | ModernBERT-large | 421M | 1,024 | Upstream typed-decisions workflows |
|
|
125
|
+
|
|
126
|
+
Context includes instructions, options and state. All three use the original weights, prompt formatting, temperature calibration, and output schema. This repository provides inference and conversion; RLCD training and fine-tuning remain in the upstream project. It is an independent port, not an official Convai Innovations release.
|
|
127
|
+
|
|
128
|
+
Pre-converted FP16 checkpoints are published on Hugging Face:
|
|
129
|
+
|
|
130
|
+
- [aac6fef/laya-mlx](https://huggingface.co/aac6fef/laya-mlx)
|
|
131
|
+
- [aac6fef/laya-multilingual-mlx](https://huggingface.co/aac6fef/laya-multilingual-mlx)
|
|
132
|
+
- [aac6fef/laya-typed-decisions-mlx](https://huggingface.co/aac6fef/laya-typed-decisions-mlx)
|
|
133
|
+
|
|
134
|
+
Load these directly with `laya.load("aac6fef/laya-mlx")`, or use the original checkpoint IDs above. Each published checkpoint includes its model card, validation results, provenance, license and file checksums. All 36 published files passed strict remote checksum verification; pinned revisions and weight hashes are recorded in [hub-publication.json](https://github.com/mizorewww/laya-mlx/blob/main/benchmarks/results/hub-publication.json).
|
|
135
|
+
|
|
136
|
+
## Development install
|
|
137
|
+
|
|
138
|
+
```bash
|
|
139
|
+
gh repo clone mizorewww/laya-mlx
|
|
140
|
+
cd laya-mlx
|
|
141
|
+
uv sync --extra demo
|
|
142
|
+
uv run --extra demo laya-snake
|
|
143
|
+
```
|
|
144
|
+
|
|
145
|
+
Or install the latest GitHub revision with `pip install 'git+https://github.com/mizorewww/laya-mlx.git'`. Model weights are downloaded separately and are excluded from Git.
|
|
146
|
+
|
|
147
|
+
## Python API
|
|
148
|
+
|
|
149
|
+
```python
|
|
150
|
+
import laya_mlx as laya
|
|
151
|
+
|
|
152
|
+
agent = laya.load("aac6fef/laya-mlx", dtype="float16")
|
|
153
|
+
result = agent.predict(
|
|
154
|
+
"I was billed twice. Please refund the duplicate today.",
|
|
155
|
+
{
|
|
156
|
+
"department": {
|
|
157
|
+
"type": "choice",
|
|
158
|
+
"instructions": "Which team should handle this request?",
|
|
159
|
+
"criteria": {
|
|
160
|
+
"billing": "invoices, payments, refunds",
|
|
161
|
+
"technical": "bugs and outages",
|
|
162
|
+
"sales": "new purchases",
|
|
163
|
+
},
|
|
164
|
+
},
|
|
165
|
+
"urgency": {
|
|
166
|
+
"type": "score",
|
|
167
|
+
"instructions": "How urgent is this request?",
|
|
168
|
+
"criteria": ["not urgent", "soon", "critical"],
|
|
169
|
+
},
|
|
170
|
+
"refund": {
|
|
171
|
+
"type": "noul",
|
|
172
|
+
"instructions": "Does the customer ask for money back?",
|
|
173
|
+
},
|
|
174
|
+
},
|
|
175
|
+
)
|
|
176
|
+
print(result["answers"])
|
|
177
|
+
```
|
|
178
|
+
|
|
179
|
+
`system_one` is an alias for `predict`. States can be text, JSON dictionaries, or conversation lists. `choice` accepts a dictionary or a list of unique labels; `score` returns the expected zero-based rubric level; `noul` returns P(true). Results retain upstream's four-decimal rounding, `action.act_probability`, and token usage fields.
|
|
180
|
+
|
|
181
|
+
The default precision is FP16. Use `dtype="float32"` for closer numerical agreement. Probabilities can differ slightly across precisions even when the selected label agrees; see the measured errors in [BENCHMARKS.md](https://github.com/mizorewww/laya-mlx/blob/main/BENCHMARKS.md). BF16 can be requested but is not part of the published validation matrix.
|
|
182
|
+
|
|
183
|
+
`batch_size=16` caps the number of questions per forward pass; larger requests are processed in chunks. Increase it when memory allows. `device="gpu"` or `device="cpu"` selects a device explicitly; otherwise MLX's default device is used.
|
|
184
|
+
|
|
185
|
+
For repeated workloads, opt into `compile=True`, `pad_to_multiple=16` and `cache_prompts=True` when loading an Agent. The prefix cache is bounded to 128 questions and shares CPU state tokenization, while every question still gets its own encoder computation. Compilation has a first-use cost and shape specialization; padding may make some workloads slower. All three options default to disabled. [Measured Snake ablation and usage](https://github.com/mizorewww/laya-mlx/blob/main/docs/SNAKE_OPTIMIZATION.md).
|
|
186
|
+
|
|
187
|
+
```python
|
|
188
|
+
agent = laya.load("./models/laya", dtype="float32", batch_size=32)
|
|
189
|
+
# Select one checkpoint inside upstream's bundled repository:
|
|
190
|
+
multi = laya.load("convaiinnovations/laya", subfolder="multilingual")
|
|
191
|
+
# Pin a Hub revision for reproducibility:
|
|
192
|
+
agent = laya.load(
|
|
193
|
+
"convaiinnovations/laya",
|
|
194
|
+
revision="c5d78730f3493e4fe16d61507ef4b78eef7318cf",
|
|
195
|
+
)
|
|
196
|
+
```
|
|
197
|
+
|
|
198
|
+
Loading validates every parameter name and shape. Unsupported encoders and non-default RoPE scaling fail explicitly. ModernBERT's global/local attention pattern, inclusive sliding-window boundary, distinct local/global RoPE bases, and first-layer normalization behavior are preserved.
|
|
199
|
+
|
|
200
|
+
## Language routing and presets
|
|
201
|
+
|
|
202
|
+
```python
|
|
203
|
+
from laya_mlx import Router, triage_questions
|
|
204
|
+
|
|
205
|
+
router = Router(dtype="float16", max_loaded=2)
|
|
206
|
+
result = router.predict({"message": "发票被重复扣款,请退款。"}, triage_questions())
|
|
207
|
+
print(result["routing"]) # multilingual
|
|
208
|
+
|
|
209
|
+
# Choose the specialized checkpoint explicitly:
|
|
210
|
+
result = router.predict(state, questions, task="typed_decisions")
|
|
211
|
+
```
|
|
212
|
+
|
|
213
|
+
The router, language heuristics, email helpers and application presets are adapted from upstream. `Router(preload=True)` keeps all three checkpoints resident; `attach`, `preload`, `unload`, explicit `lang=`, and explicit `model=` are supported. Typed-decisions workflow detection stays opt-in. The port preserves model limitations: English checkpoints are not substitutes for the multilingual checkpoint, and confidence does not guarantee accuracy.
|
|
214
|
+
|
|
215
|
+
## Command line
|
|
216
|
+
|
|
217
|
+
```bash
|
|
218
|
+
uv run laya-mlx predict \
|
|
219
|
+
--model aac6fef/laya-mlx \
|
|
220
|
+
--state-file examples/state.json \
|
|
221
|
+
--questions examples/questions.json
|
|
222
|
+
|
|
223
|
+
uv run laya-mlx predict \
|
|
224
|
+
--model aac6fef/laya-multilingual-mlx \
|
|
225
|
+
--state '发票被重复扣款,请退款。' \
|
|
226
|
+
--questions examples/questions.json
|
|
227
|
+
```
|
|
228
|
+
|
|
229
|
+
## Export an MLX checkpoint
|
|
230
|
+
|
|
231
|
+
```bash
|
|
232
|
+
uv run laya-mlx convert \
|
|
233
|
+
--model convaiinnovations/laya \
|
|
234
|
+
--dtype float16 \
|
|
235
|
+
--output models/laya-mlx-fp16
|
|
236
|
+
|
|
237
|
+
uv run laya-mlx predict \
|
|
238
|
+
--model models/laya-mlx-fp16 \
|
|
239
|
+
--state-file examples/state.json \
|
|
240
|
+
--questions examples/questions.json
|
|
241
|
+
```
|
|
242
|
+
|
|
243
|
+
The export contains `model.safetensors`, encoder and agent configurations, tokenizer files and `mlx_config.json`. Existing output directories are never overwritten. This is a parameter-name/dtype conversion, not quantization or retraining. The source checkpoints already store FP16 weights; choosing FP32 increases arithmetic precision, not the precision of the source weights.
|
|
244
|
+
|
|
245
|
+
## Tests and benchmarks
|
|
246
|
+
|
|
247
|
+
```bash
|
|
248
|
+
uv sync --extra dev --extra reference --extra benchmark --extra demo
|
|
249
|
+
source .venv/bin/activate
|
|
250
|
+
gh repo clone NandhaKishorM/laya .upstream
|
|
251
|
+
git -C .upstream checkout 6a5819129eb220570792e417e49723d697efd76f
|
|
252
|
+
pytest -q
|
|
253
|
+
python -m benchmarks.download
|
|
254
|
+
python -m benchmarks.validate --repeats 100
|
|
255
|
+
python -m benchmarks.run --iterations 50 --warmup 5
|
|
256
|
+
python -m benchmarks.accuracy --per-class 64
|
|
257
|
+
python -m benchmarks.report
|
|
258
|
+
```
|
|
259
|
+
|
|
260
|
+
Run GPU measurements sequentially. Unit tests use small random models and include direct comparisons with Transformers and the pinned upstream decision head. Real checkpoint validation tests tokenization, logits, calibrated probabilities, repeated outputs and active memory growth. The benchmark runs each backend/checkpoint in a fresh process and stores every timing sample in [benchmarks/results](https://github.com/mizorewww/laya-mlx/blob/main/benchmarks/results). The [full report](https://github.com/mizorewww/laya-mlx/blob/main/BENCHMARKS.md) explains the timing boundaries and precision differences.
|
|
261
|
+
|
|
262
|
+
GitHub Actions runs small-model CPU tests on a macOS arm64 runner. Full checkpoint GPU benchmarks are measured locally and are not part of hosted CI.
|
|
263
|
+
|
|
264
|
+
## Performance research
|
|
265
|
+
|
|
266
|
+
The performance investigations include both mathematical analysis and independent local experiments:
|
|
267
|
+
|
|
268
|
+
- [Initial performance research](https://github.com/mizorewww/laya-mlx/blob/main/docs/PERFORMANCE_RESEARCH.md): implementation bottlenecks, MLX kernel dispatch, and a controlled experiment plan.
|
|
269
|
+
- [Mathematical investigation of a further 10× speedup](https://github.com/mizorewww/laya-mlx/blob/main/docs/MATH_10X_RESEARCH.md): arithmetic budgets, conditional bandwidth bounds, real weight spectra, exact reuse, and smaller-model designs.
|
|
270
|
+
- [Engineering investigation](https://github.com/mizorewww/laya-mlx/blob/main/docs/ENGINEERING_10X_RESEARCH.md): measured compilation, quantization, final-head selection, custom Metal kernels, and representative matrix multiplications.
|
|
271
|
+
|
|
272
|
+
[experiments/](https://github.com/mizorewww/laya-mlx/blob/main/experiments) contains the research scripts and their raw measurements. The published runtime's performance and validation results are in [BENCHMARKS.md](https://github.com/mizorewww/laya-mlx/blob/main/BENCHMARKS.md); each experimental variant has its own timing and correctness results.
|
|
273
|
+
|
|
274
|
+
The current investigation does not support a further universal 10× speedup with the same checkpoints. Selected cases show approximately 1.03–1.08× paired median speedups; the engineering report gives the uncertainty intervals, quantization fidelity results, and custom Metal kernel measurements.
|
|
275
|
+
|
|
276
|
+
To prepare model cards and verified exports for publication, install the reference extras and run:
|
|
277
|
+
|
|
278
|
+
```bash
|
|
279
|
+
python -m scripts.prepare_hub --account YOUR_HF_USERNAME
|
|
280
|
+
hf upload YOUR_HF_USERNAME/laya-mlx models/hub/laya-mlx . --exclude '.cache/*'
|
|
281
|
+
```
|
|
282
|
+
|
|
283
|
+
The preparation script checks every exported tensor against its original FP16 source. Upload the other two prepared folders in the same way, then use `hf cache verify REPO_ID --local-dir EXPORT_PATH` to check the remote files.
|
|
284
|
+
|
|
285
|
+
## Attribution and license
|
|
286
|
+
|
|
287
|
+
Apache-2.0; see [LICENSE](https://github.com/mizorewww/laya-mlx/blob/main/LICENSE) and [NOTICE](https://github.com/mizorewww/laya-mlx/blob/main/NOTICE). Laya and its pretrained weights are by Convai Innovations and upstream contributors. Prompt construction, output formatting, language routing, email utilities and presets are adapted from [NandhaKishorM/laya](https://github.com/NandhaKishorM/laya) at commit `6a5819129eb220570792e417e49723d697efd76f`. The neural architecture is reimplemented in MLX following Laya and Hugging Face ModernBERT.
|