mlx-beam 0.1.0.dev5__tar.gz → 0.1.0.dev6__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (13) hide show
  1. mlx_beam-0.1.0.dev6/PKG-INFO +82 -0
  2. mlx_beam-0.1.0.dev6/README.md +54 -0
  3. {mlx_beam-0.1.0.dev5 → mlx_beam-0.1.0.dev6}/mlx_beam/_version.py +2 -2
  4. {mlx_beam-0.1.0.dev5 → mlx_beam-0.1.0.dev6}/pyproject.toml +2 -2
  5. mlx_beam-0.1.0.dev5/PKG-INFO +0 -73
  6. mlx_beam-0.1.0.dev5/README.md +0 -45
  7. {mlx_beam-0.1.0.dev5 → mlx_beam-0.1.0.dev6}/.gitignore +0 -0
  8. {mlx_beam-0.1.0.dev5 → mlx_beam-0.1.0.dev6}/CHANGELOG.md +0 -0
  9. {mlx_beam-0.1.0.dev5 → mlx_beam-0.1.0.dev6}/LICENSE +0 -0
  10. {mlx_beam-0.1.0.dev5 → mlx_beam-0.1.0.dev6}/NOTICE +0 -0
  11. {mlx_beam-0.1.0.dev5 → mlx_beam-0.1.0.dev6}/mlx_beam/__init__.py +0 -0
  12. {mlx_beam-0.1.0.dev5 → mlx_beam-0.1.0.dev6}/mlx_beam/cli.py +0 -0
  13. {mlx_beam-0.1.0.dev5 → mlx_beam-0.1.0.dev6}/tests/test_cli.py +0 -0
mlx_beam-0.1.0.dev6/PKG-INFO ADDED
@@ -0,0 +1,82 @@
1
+ Metadata-Version: 2.5
2
+ Name: mlx-beam
3
+ Version: 0.1.0.dev6
4
+ Summary: Light and modular inference engine, built on MLX.
5
+ Project-URL: Homepage, https://p4ik.github.io/mlx-beam/
6
+ Project-URL: Repository, https://github.com/p4ik/mlx-beam
7
+ Project-URL: Changelog, https://github.com/p4ik/mlx-beam/blob/main/CHANGELOG.md
8
+ Project-URL: Issues, https://github.com/p4ik/mlx-beam/issues
9
+ Author: the mlx-beam contributors
10
+ License-Expression: Apache-2.0
11
+ License-File: LICENSE
12
+ License-File: NOTICE
13
+ Keywords: apple-silicon,inference,llm,mlx,moe
14
+ Classifier: Development Status :: 2 - Pre-Alpha
15
+ Classifier: Environment :: Console
16
+ Classifier: Intended Audience :: Developers
17
+ Classifier: License :: OSI Approved :: Apache Software License
18
+ Classifier: Operating System :: MacOS
19
+ Classifier: Programming Language :: Python :: 3
20
+ Classifier: Programming Language :: Python :: 3 :: Only
21
+ Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
22
+ Requires-Python: >=3.11
23
+ Requires-Dist: mlx>=0.32; sys_platform == 'darwin'
24
+ Provides-Extra: dev
25
+ Requires-Dist: pre-commit>=4; extra == 'dev'
26
+ Requires-Dist: pytest>=8; extra == 'dev'
27
+ Description-Content-Type: text/markdown
28
+
29
+ # mlx-beam
30
+
31
+ **B.E.A.M. — Batched Engine for Apple Metal.** Light and modular inference engine, built on [MLX](https://github.com/ml-explore/mlx).
32
+
33
+ > **Work in progress.** See the status table below and the [changelog](https://github.com/p4ik/mlx-beam/blob/main/CHANGELOG.md).
34
+
35
+ ## Install
36
+
37
+ ```bash
38
+ uv tool install mlx-beam
39
+ beam doctor
40
+ ```
41
+
42
+ Inside a uv project: `uv add mlx-beam`, then `uv run beam doctor`.
43
+
44
+ `beam doctor` reports the Python, MLX, device and memory it sees (`--json` for scripts) and exits non-zero when MLX is missing or fails to load. It is the only command so far.
45
+
46
+ ## What sets it apart
47
+
48
+ - **Robust prefix cache** — RAM and SSD tiers, checkpoints for hybrid models. Survives model swaps and restarts.
49
+ - **Expert streaming** — Mixture-of-experts models larger than memory. Residency configurable, from minimal RAM to fully resident.
50
+ - **No bloat** — The core is the token path. Vision, audio, conversion, structured output and tool-call repair are optional extras.
51
+ - **Batched MTP** — Multi-token prediction stays on with many requests at once.
52
+ - **Batched vision** — Images go through the same scheduler; no request waits behind a picture.
53
+ - **No stalls** — A short request beside a long prefill answers in seconds.
54
+ - **Mixed-precision KV cache** — Bits per layer, set at conversion.
55
+ - **Thinking budget** — A hard cap on the reasoning trace, per request.
56
+ - **Responses API** — Next to chat completions, stateless.
57
+
58
+ The engine reads standard MLX checkpoints and the B.E.A.M. package layout (`extras/` next to the shards; see the model cards under [huggingface.co/p4ik](https://huggingface.co/p4ik)).
59
+
60
+ ## Why it exists
61
+
62
+ Existing MLX servers either stop at the basics or grow things that have no place in an inference engine: a built-in game, a cloud path that arrives with an update. The ones we ran daily also had bugs where it matters most: prefix cache, batching under load, vision. B.E.A.M. keeps the core to the token path and fixes those paths at the source. Everything else is an extra you choose to install; nothing ever ships in the core that you did not ask for.
63
+
64
+ ## Status
65
+
66
+ | Piece | State |
67
+ |---|---|
68
+ | CLI, packaging, CI | skeleton |
69
+ | Vendored mlx-lm base | planned |
70
+ | Prefix cache with recurrent-state checkpoints | planned |
71
+ | Multi-token prediction in the batch | planned |
72
+ | Expert streaming from SSD | planned |
73
+
74
+ Measured numbers are published as they are measured, with machine, model and date.
75
+
76
+ ## Contributing
77
+
78
+ See [CONTRIBUTING.md](https://github.com/p4ik/mlx-beam/blob/main/CONTRIBUTING.md). Rules for coding agents are in [AGENTS.md](https://github.com/p4ik/mlx-beam/blob/main/AGENTS.md).
79
+
80
+ ## License
81
+
82
+ Apache-2.0. Vendored components keep their own licenses; see [NOTICE](https://github.com/p4ik/mlx-beam/blob/main/NOTICE).
mlx_beam-0.1.0.dev6/README.md ADDED
@@ -0,0 +1,54 @@
1
+ # mlx-beam
2
+
3
+ **B.E.A.M. — Batched Engine for Apple Metal.** Light and modular inference engine, built on [MLX](https://github.com/ml-explore/mlx).
4
+
5
+ > **Work in progress.** See the status table below and the [changelog](https://github.com/p4ik/mlx-beam/blob/main/CHANGELOG.md).
6
+
7
+ ## Install
8
+
9
+ ```bash
10
+ uv tool install mlx-beam
11
+ beam doctor
12
+ ```
13
+
14
+ Inside a uv project: `uv add mlx-beam`, then `uv run beam doctor`.
15
+
16
+ `beam doctor` reports the Python, MLX, device and memory it sees (`--json` for scripts) and exits non-zero when MLX is missing or fails to load. It is the only command so far.
17
+
18
+ ## What sets it apart
19
+
20
+ - **Robust prefix cache** — RAM and SSD tiers, checkpoints for hybrid models. Survives model swaps and restarts.
21
+ - **Expert streaming** — Mixture-of-experts models larger than memory. Residency configurable, from minimal RAM to fully resident.
22
+ - **No bloat** — The core is the token path. Vision, audio, conversion, structured output and tool-call repair are optional extras.
23
+ - **Batched MTP** — Multi-token prediction stays on with many requests at once.
24
+ - **Batched vision** — Images go through the same scheduler; no request waits behind a picture.
25
+ - **No stalls** — A short request beside a long prefill answers in seconds.
26
+ - **Mixed-precision KV cache** — Bits per layer, set at conversion.
27
+ - **Thinking budget** — A hard cap on the reasoning trace, per request.
28
+ - **Responses API** — Next to chat completions, stateless.
29
+
30
+ The engine reads standard MLX checkpoints and the B.E.A.M. package layout (`extras/` next to the shards; see the model cards under [huggingface.co/p4ik](https://huggingface.co/p4ik)).
31
+
32
+ ## Why it exists
33
+
34
+ Existing MLX servers either stop at the basics or grow things that have no place in an inference engine: a built-in game, a cloud path that arrives with an update. The ones we ran daily also had bugs where it matters most: prefix cache, batching under load, vision. B.E.A.M. keeps the core to the token path and fixes those paths at the source. Everything else is an extra you choose to install; nothing ever ships in the core that you did not ask for.
35
+
36
+ ## Status
37
+
38
+ | Piece | State |
39
+ |---|---|
40
+ | CLI, packaging, CI | skeleton |
41
+ | Vendored mlx-lm base | planned |
42
+ | Prefix cache with recurrent-state checkpoints | planned |
43
+ | Multi-token prediction in the batch | planned |
44
+ | Expert streaming from SSD | planned |
45
+
46
+ Measured numbers are published as they are measured, with machine, model and date.
47
+
48
+ ## Contributing
49
+
50
+ See [CONTRIBUTING.md](https://github.com/p4ik/mlx-beam/blob/main/CONTRIBUTING.md). Rules for coding agents are in [AGENTS.md](https://github.com/p4ik/mlx-beam/blob/main/AGENTS.md).
51
+
52
+ ## License
53
+
54
+ Apache-2.0. Vendored components keep their own licenses; see [NOTICE](https://github.com/p4ik/mlx-beam/blob/main/NOTICE).
{mlx_beam-0.1.0.dev5 → mlx_beam-0.1.0.dev6}/mlx_beam/_version.py RENAMED
@@ -18,7 +18,7 @@ version_tuple: tuple[int | str, ...]
18
18
  commit_id: str | None
19
19
  __commit_id__: str | None
20
20
 
21
- __version__ = version = '0.1.0.dev5'
22
- __version_tuple__ = version_tuple = (0, 1, 0, 'dev5')
21
+ __version__ = version = '0.1.0.dev6'
22
+ __version_tuple__ = version_tuple = (0, 1, 0, 'dev6')
23
23
 
24
24
  __commit_id__ = commit_id = None
{mlx_beam-0.1.0.dev5 → mlx_beam-0.1.0.dev6}/pyproject.toml RENAMED
@@ -5,7 +5,7 @@ build-backend = "hatchling.build"
5
5
  [project]
6
6
  name = "mlx-beam"
7
7
  dynamic = ["version"]
8
- description = "B.E.A.M. - Batched Engine for Apple Metal: an MLX inference engine for hybrid and MoE models on Apple silicon."
8
+ description = "Light and modular inference engine, built on MLX."
9
9
  readme = "README.md"
10
10
  license = "Apache-2.0"
11
11
  license-files = ["LICENSE", "NOTICE"]
@@ -33,7 +33,7 @@ dev = ["pytest>=8", "pre-commit>=4"]
33
33
  beam = "mlx_beam.cli:main"
34
34
 
35
35
  [project.urls]
36
- Homepage = "https://github.com/p4ik/mlx-beam"
36
+ Homepage = "https://p4ik.github.io/mlx-beam/"
37
37
  Repository = "https://github.com/p4ik/mlx-beam"
38
38
  Changelog = "https://github.com/p4ik/mlx-beam/blob/main/CHANGELOG.md"
39
39
  Issues = "https://github.com/p4ik/mlx-beam/issues"
mlx_beam-0.1.0.dev5/PKG-INFO DELETED
@@ -1,73 +0,0 @@
1
- Metadata-Version: 2.5
2
- Name: mlx-beam
3
- Version: 0.1.0.dev5
4
- Summary: B.E.A.M. - Batched Engine for Apple Metal: an MLX inference engine for hybrid and MoE models on Apple silicon.
5
- Project-URL: Homepage, https://github.com/p4ik/mlx-beam
6
- Project-URL: Repository, https://github.com/p4ik/mlx-beam
7
- Project-URL: Changelog, https://github.com/p4ik/mlx-beam/blob/main/CHANGELOG.md
8
- Project-URL: Issues, https://github.com/p4ik/mlx-beam/issues
9
- Author: the mlx-beam contributors
10
- License-Expression: Apache-2.0
11
- License-File: LICENSE
12
- License-File: NOTICE
13
- Keywords: apple-silicon,inference,llm,mlx,moe
14
- Classifier: Development Status :: 2 - Pre-Alpha
15
- Classifier: Environment :: Console
16
- Classifier: Intended Audience :: Developers
17
- Classifier: License :: OSI Approved :: Apache Software License
18
- Classifier: Operating System :: MacOS
19
- Classifier: Programming Language :: Python :: 3
20
- Classifier: Programming Language :: Python :: 3 :: Only
21
- Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
22
- Requires-Python: >=3.11
23
- Requires-Dist: mlx>=0.32; sys_platform == 'darwin'
24
- Provides-Extra: dev
25
- Requires-Dist: pre-commit>=4; extra == 'dev'
26
- Requires-Dist: pytest>=8; extra == 'dev'
27
- Description-Content-Type: text/markdown
28
-
29
- # mlx-beam
30
-
31
- **B.E.A.M. — Batched Engine for Apple Metal.** An [MLX](https://github.com/ml-explore/mlx) inference engine for hybrid-attention and mixture-of-experts language models on Apple silicon.
32
-
33
- > **Work in progress.** See the status table below and the [changelog](CHANGELOG.md).
34
-
35
- ## What it is for
36
-
37
- Models like Qwen3.8 (Gated DeltaNet + attention) and Qwen3.8-Flash-Next (125B-A6B MoE with an on-disk n-gram table) do not fit the assumptions of a plain transformer server: their recurrent state cannot be trimmed like a KV cache, their experts do not fit in memory, and speculative decoding under batching needs a rollback that most engines lack. B.E.A.M. is built around those cases:
38
-
39
- - a prefix cache with recurrent-state checkpoints, so a long conversation resumes from disk instead of re-prefilling;
40
- - multi-token prediction inside continuous batching;
41
- - expert streaming from SSD with a resident pool, for models larger than RAM;
42
- - a small, explicit OpenAI-compatible API, with everything else as optional extras.
43
-
44
- The engine reads standard MLX checkpoints and the B.E.A.M. package layout (`extras/` next to the shards; see the model cards under [huggingface.co/p4ik](https://huggingface.co/p4ik)).
45
-
46
- ## Install
47
-
48
- ```bash
49
- uv tool install "mlx-beam @ git+https://github.com/p4ik/mlx-beam"
50
- beam doctor
51
- ```
52
-
53
- Inside an existing uv project: `uv add "mlx-beam @ git+https://github.com/p4ik/mlx-beam"`, then `uv run beam doctor`.
54
-
55
- `beam doctor` prints the Python, MLX, device and memory it sees (`--json` for scripts) and exits non-zero when MLX is missing or fails to load. It is the only command so far.
56
-
57
- ## Status
58
-
59
- | Piece | State |
60
- |---|---|
61
- | CLI, packaging, CI | skeleton |
62
- | vendored mlx-lm base | planned |
63
- | prefix cache with recurrent-state checkpoints | planned |
64
- | MTP in the batch | planned |
65
- | expert streaming | planned |
66
-
67
- ## Contributing
68
-
69
- See [CONTRIBUTING.md](CONTRIBUTING.md). Working rules for people and agents are in [AGENTS.md](AGENTS.md).
70
-
71
- ## License
72
-
73
- Apache-2.0. Vendored components keep their own licenses; see [NOTICE](NOTICE) and `VENDORED.md`.
mlx_beam-0.1.0.dev5/README.md DELETED
@@ -1,45 +0,0 @@
1
- # mlx-beam
2
-
3
- **B.E.A.M. — Batched Engine for Apple Metal.** An [MLX](https://github.com/ml-explore/mlx) inference engine for hybrid-attention and mixture-of-experts language models on Apple silicon.
4
-
5
- > **Work in progress.** See the status table below and the [changelog](CHANGELOG.md).
6
-
7
- ## What it is for
8
-
9
- Models like Qwen3.8 (Gated DeltaNet + attention) and Qwen3.8-Flash-Next (125B-A6B MoE with an on-disk n-gram table) do not fit the assumptions of a plain transformer server: their recurrent state cannot be trimmed like a KV cache, their experts do not fit in memory, and speculative decoding under batching needs a rollback that most engines lack. B.E.A.M. is built around those cases:
10
-
11
- - a prefix cache with recurrent-state checkpoints, so a long conversation resumes from disk instead of re-prefilling;
12
- - multi-token prediction inside continuous batching;
13
- - expert streaming from SSD with a resident pool, for models larger than RAM;
14
- - a small, explicit OpenAI-compatible API, with everything else as optional extras.
15
-
16
- The engine reads standard MLX checkpoints and the B.E.A.M. package layout (`extras/` next to the shards; see the model cards under [huggingface.co/p4ik](https://huggingface.co/p4ik)).
17
-
18
- ## Install
19
-
20
- ```bash
21
- uv tool install "mlx-beam @ git+https://github.com/p4ik/mlx-beam"
22
- beam doctor
23
- ```
24
-
25
- Inside an existing uv project: `uv add "mlx-beam @ git+https://github.com/p4ik/mlx-beam"`, then `uv run beam doctor`.
26
-
27
- `beam doctor` prints the Python, MLX, device and memory it sees (`--json` for scripts) and exits non-zero when MLX is missing or fails to load. It is the only command so far.
28
-
29
- ## Status
30
-
31
- | Piece | State |
32
- |---|---|
33
- | CLI, packaging, CI | skeleton |
34
- | vendored mlx-lm base | planned |
35
- | prefix cache with recurrent-state checkpoints | planned |
36
- | MTP in the batch | planned |
37
- | expert streaming | planned |
38
-
39
- ## Contributing
40
-
41
- See [CONTRIBUTING.md](CONTRIBUTING.md). Working rules for people and agents are in [AGENTS.md](AGENTS.md).
42
-
43
- ## License
44
-
45
- Apache-2.0. Vendored components keep their own licenses; see [NOTICE](NOTICE) and `VENDORED.md`.
{mlx_beam-0.1.0.dev5 → mlx_beam-0.1.0.dev6}/.gitignore RENAMED
File without changes
{mlx_beam-0.1.0.dev5 → mlx_beam-0.1.0.dev6}/LICENSE RENAMED
File without changes
{mlx_beam-0.1.0.dev5 → mlx_beam-0.1.0.dev6}/NOTICE RENAMED
File without changes