vmlx 1.0.0__py3-none-any.whl

vmlx-1.0.0.dist-info/METADATA

@@ -0,0 +1,340 @@
+Metadata-Version: 2.4
+Name: vmlx
+Version: 1.0.0
+Summary: Local AI inference for Apple Silicon — Text, Image, Video & Audio generation on Mac
+Author-email: Jinho Jang <eric@jangq.ai>
+License: Apache-2.0
+Project-URL: Homepage, https://github.com/vmlxllm/vmlx
+Project-URL: Documentation, https://github.com/vmlxllm/vmlx#readme
+Project-URL: Repository, https://github.com/vmlxllm/vmlx
+Keywords: llm,mlx,apple-silicon,vllm,inference,transformers
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Developers
+Classifier: Intended Audience :: Science/Research
+Classifier: License :: OSI Approved :: Apache Software License
+Classifier: Operating System :: MacOS
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+Requires-Dist: mlx>=0.29.0
+Requires-Dist: mlx-lm>=0.30.2
+Requires-Dist: mlx-vlm>=0.1.0
+Requires-Dist: transformers>=4.40.0
+Requires-Dist: tokenizers>=0.19.0
+Requires-Dist: huggingface-hub>=0.23.0
+Requires-Dist: numpy>=1.24.0
+Requires-Dist: pillow>=10.0.0
+Requires-Dist: tqdm>=4.66.0
+Requires-Dist: pyyaml>=6.0
+Requires-Dist: requests>=2.28.0
+Requires-Dist: tabulate>=0.9.0
+Requires-Dist: opencv-python-headless>=4.8.0
+Requires-Dist: psutil>=5.9.0
+Requires-Dist: fastapi>=0.100.0
+Requires-Dist: uvicorn>=0.23.0
+Requires-Dist: mcp>=1.0.0
+Requires-Dist: jsonschema>=4.0.0
+Requires-Dist: mlx-embeddings>=0.0.5
+Provides-Extra: ui
+Requires-Dist: gradio>=4.0.0; extra == "ui"
+Requires-Dist: pytz>=2024.1; extra == "ui"
+Provides-Extra: dev
+Requires-Dist: pytest>=7.0.0; extra == "dev"
+Requires-Dist: pytest-asyncio>=0.21.0; extra == "dev"
+Requires-Dist: black>=23.0.0; extra == "dev"
+Requires-Dist: ruff>=0.1.0; extra == "dev"
+Requires-Dist: mypy>=1.0.0; extra == "dev"
+Provides-Extra: vllm
+Requires-Dist: vllm>=0.4.0; extra == "vllm"
+Provides-Extra: vision
+Requires-Dist: torch>=2.3.0; extra == "vision"
+Requires-Dist: torchvision>=0.18.0; extra == "vision"
+Provides-Extra: audio
+Requires-Dist: mlx-audio>=0.2.9; extra == "audio"
+Requires-Dist: sounddevice>=0.4.0; extra == "audio"
+Requires-Dist: soundfile>=0.12.0; extra == "audio"
+Requires-Dist: scipy>=1.10.0; extra == "audio"
+Requires-Dist: numba>=0.57.0; extra == "audio"
+Requires-Dist: tiktoken>=0.5.0; extra == "audio"
+Requires-Dist: misaki[ja,zh]>=0.5.0; extra == "audio"
+Requires-Dist: spacy>=3.7.0; extra == "audio"
+Requires-Dist: num2words>=0.5.0; extra == "audio"
+Requires-Dist: loguru>=0.7.0; extra == "audio"
+Requires-Dist: phonemizer>=3.2.0; extra == "audio"
+Requires-Dist: ordered_set>=4.1.0; extra == "audio"
+Requires-Dist: cn2an>=0.5.0; extra == "audio"
+Requires-Dist: fugashi>=1.3.0; extra == "audio"
+Requires-Dist: unidic-lite>=1.0.0; extra == "audio"
+Requires-Dist: jieba>=0.42.0; extra == "audio"
+Provides-Extra: jang
+Requires-Dist: jang>=1.0.0; extra == "jang"
+Provides-Extra: image
+Requires-Dist: mflux>=0.16.0; extra == "image"
+
+<p align="center">
+  <picture>
+    <source media="(prefers-color-scheme: dark)" srcset="https://vmlx.net/logos/png/wordmark-dark-600x150.png">
+    <source media="(prefers-color-scheme: light)" srcset="https://vmlx.net/logos/png/wordmark-light-600x150.png">
+    <img alt="vMLX" src="https://vmlx.net/logos/png/wordmark-transparent-600x150.png" width="400">
+  </picture>
+</p>
+
+<p align="center">
+  <strong>Native macOS AI inference — local models, remote endpoints, zero config</strong>
+</p>
+
+<p align="center">
+  <a href="https://vmlx.net">Website</a> · <a href="panel/CHANGELOG.md">Panel Changelog</a> · <a href="CHANGELOG.md">Engine Changelog</a> · <a href="docs/">Documentation</a>
+</p>
+
+---
+
+## What is vMLX?
+
+vMLX is a native macOS application for running AI models on Apple Silicon. It bundles a custom inference engine with a full-featured desktop interface — manage sessions, chat with models, download from HuggingFace, connect to remote APIs, and use agentic tool-calling workflows.
+
+- **Local inference** with GPU acceleration via MLX
+- **Remote endpoints** — connect to any OpenAI-compatible API
+- **HuggingFace downloader** — search, download, and serve models in-app
+- **Built-in tools** — file I/O, shell, search, image reading, ask_user interrupt
+- **MCP integration** — Model Context Protocol tool servers (local sessions)
+
+---
+
+## Key Features
+
+### Inference Engine (v0.2.18)
+
+| Feature | Description |
+|---------|-------------|
+| **Paged KV Cache** | Memory-efficient caching with prefix sharing and block-level reuse |
+| **KV Cache Quantization** | Q4/Q8 quantized cache storage (2–4× memory savings) |
+| **Prefix Cache** | Token-level prefix matching for fast prompt reuse across requests |
+| **Continuous Batching** | Concurrent request handling with slot management |
+| **VLM Caching** | Full KV cache pipeline for vision-language models (Qwen-VL, Gemma 3, etc.) |
+| **Mamba Hybrid Support** | Auto-detects mixed KVCache + MambaCache models (Qwen3.5-VL, Qwen3-Coder-Next, Nemotron) |
+| **Streaming Detokenizer** | Per-request UTF-8 buffering — emoji, CJK, Arabic render correctly |
+| **Request Cancellation** | Stop inference mid-stream via API or connection close |
+| **OpenAI-Compatible API** | Chat Completions + Responses API with full streaming support |
+| **Speculative Decoding** | Draft model acceleration (20-90% speedup, zero quality loss) |
+
+### Desktop App (Panel v1.2.1)
+
+| Feature | Description |
+|---------|-------------|
+| **Multi-session** | Run multiple models simultaneously on different ports |
+| **Remote endpoints** | Connect to OpenAI, Groq, local vLLM, or any compatible API |
+| **HuggingFace browser** | Search, download, and install MLX models with progress tracking |
+| **Agentic tools** | File I/O, shell, search, image reading with auto-continue loops (up to 10 iterations) |
+| **Per-chat settings** | Temperature, Top P/K, Min P, Repeat Penalty, Stop Sequences, Max Tokens |
+| **Reasoning display** | Collapsible thinking sections for Qwen3, DeepSeek-R1, GLM-4.7 |
+| **Tool parsers** | hermes, pythonic, llama3, mistral, minimax, qwen3, nemotron, step3p5, and more |
+| **Auto-detection** | Reads model config JSON for automatic parser and cache type selection |
+| **Persistent history** | SQLite-backed chat history with metrics, tool calls, and reasoning content |
+| **Live metrics** | TTFT, tokens/sec, prompt processing speed, prefix cache hits |
+
+---
+
+## Quick Start
+
+### Desktop App (recommended)
+
+```bash
+# Clone and build
+git clone https://github.com/vmlxllm/vmlx.git
+cd vmlx/panel
+
+# Install dependencies
+npm install
+
+# Development mode
+npm run dev
+
+# Build and install to /Applications
+bash scripts/build-and-install.sh
+```
+
+### Engine Only (CLI)
+
+```bash
+# Install
+uv tool install git+https://github.com/vmlxllm/vmlx.git
+# or
+pip install git+https://github.com/vmlxllm/vmlx.git
+
+# Start server
+vmlx-engine serve mlx-community/Llama-3.2-3B-Instruct-4bit --port 8000
+
+# With continuous batching
+vmlx-engine serve mlx-community/Llama-3.2-3B-Instruct-4bit --port 8000 --continuous-batching
+
+# With API key
+vmlx-engine serve mlx-community/Llama-3.2-3B-Instruct-4bit --port 8000 --api-key your-key
+
+# With speculative decoding (20-90% faster)
+vmlx-engine serve mlx-community/Llama-3.2-3B-Instruct-4bit --port 8000 \
+  --speculative-model mlx-community/Llama-3.2-1B-Instruct-4bit \
+  --num-draft-tokens 3
+```
+
+### Use with OpenAI SDK
+
+```python
+from openai import OpenAI
+
+client = OpenAI(base_url="http://localhost:8000/v1", api_key="not-needed")
+
+response = client.chat.completions.create(
+    model="default",
+    messages=[{"role": "user", "content": "Hello!"}],
+)
+print(response.choices[0].message.content)
+```
+
+---
+
+## API Endpoints
+
+| Endpoint | Description |
+|----------|-------------|
+| `POST /v1/chat/completions` | Chat Completions API (streaming) |
+| `POST /v1/responses` | Responses API (streaming) |
+| `GET /v1/models` | List loaded models |
+| `GET /health` | Server health + model info |
+| `POST /v1/mcp/execute` | Execute MCP tool |
+| `GET /v1/cache/stats` | Prefix cache statistics |
+| `POST /v1/cache/warm` | Pre-warm cache with prompt |
+| `DELETE /v1/cache` | Clear prefix cache |
+| `POST /v1/chat/completions/{id}/cancel` | Cancel inference (save GPU) |
+| `POST /v1/embeddings` | Text embeddings (mlx-embeddings) |
+
+---
+
+## Reasoning Models
+
+Extract thinking process from reasoning-capable models:
+
+```bash
+vmlx-engine serve mlx-community/Qwen3-8B-4bit --reasoning-parser qwen3
+```
+
+| Parser | Models | Format |
+|--------|--------|--------|
+| `qwen3` | Qwen3, QwQ, MiniMax M2/M2.5, StepFun | `<think>` / `</think>` tags |
+| `deepseek_r1` | DeepSeek-R1, Gemma 3, Phi-4 Reasoning, GLM-4.7, GLM-Z1 | Lenient `<think>` (handles missing open tag) |
+| `openai_gptoss` | GLM-4.7 Flash, GPT-OSS | Harmony `<\|channel\|>analysis/final` protocol |
+
+---
+
+## Tool Calling
+
+Built-in agentic tools available in the desktop app:
+
+| Category | Tools |
+|----------|-------|
+| **File** | read_file, write_file, edit_file, patch_file, batch_edit, copy, move, delete, create_directory, list_directory, read_image |
+| **Search** | search_files, find_files, file_info, get_diagnostics, get_tree, diff_files |
+| **Shell** | run_command, spawn_process, get_process_output |
+| **Web** | fetchUrl, brave_search |
+| **Utility** | ask_user (interactive interrupt) |
+
+Plus MCP tool server passthrough for local sessions.
+
+---
+
+## Architecture
+
+```
+┌─────────────────────────────────────────────────────────┐
+│                    vMLX Desktop App                      │
+│              (Electron + React + TypeScript)              │
+└─────────────────────────────────────────────────────────┘
+                           │
+              ┌────────────┴────────────┐
+              ▼                         ▼
+┌──────────────────────┐  ┌──────────────────────┐
+│   Local vmlx-engine     │  │   Remote Endpoints   │
+│   (spawned process)  │  │ (OpenAI, Groq, etc.) │
+└──────────────────────┘  └──────────────────────┘
+              │
+              ▼
+┌─────────────────────────────────────────────────────────┐
+│                      vMLX Engine                             │
+│         (FastAPI + MLX inference + caching)               │
+└─────────────────────────────────────────────────────────┘
+              │
+    ┌─────────┼──────────┬──────────┐
+    ▼         ▼          ▼          ▼
+┌────────┐┌────────┐┌────────┐┌────────────┐
+│ mlx-lm ││mlx-vlm ││mlx-aud ││mlx-embed   │
+│ (LLMs) ││(Vision)││(Audio) ││(Embeddings)│
+└────────┘└────────┘└────────┘└────────────┘
+              │
+              ▼
+┌─────────────────────────────────────────────────────────┐
+│                     Apple MLX                            │
+│             (Metal GPU + Unified Memory)                 │
+└─────────────────────────────────────────────────────────┘
+```
+
+---
+
+## Tech Stack
+
+| Layer | Technology |
+|-------|-----------|
+| Desktop app | Electron 28 + React 18 + TypeScript |
+| Styling | Tailwind CSS |
+| Database | SQLite (WAL mode, better-sqlite3) |
+| Inference engine | vMLX Engine v0.2.18 (Python, FastAPI) |
+| ML framework | Apple MLX (Metal GPU acceleration) |
+| Build | electron-vite + electron-builder |
+| Tests | Vitest (panel: 542 tests), pytest (engine: 1595 tests) |
+| Python | Bundled relocatable Python 3.12 |
+
+---
+
+## Recent Changes
+
+### Panel v1.2.1 / Engine v0.2.18 (2026-03-09)
+- **Tool calling fix**: `enableAutoToolChoice` default changed from `false` to `undefined` (auto-detect) — MCP and built-in tools now work out of the box without manual enable
+- **MCP tool result truncation**: MCP tool results now capped at same limit as built-in tools (50KB default) to prevent context overflow
+- **Command preview parity**: `buildCommandPreview` in SessionSettings now matches actual `buildArgs` logic for auto-tool-choice flags
+- **Old config migration**: Stored sessions with `enableAutoToolChoice: false` auto-migrate to `undefined` on load
+- **2137 total tests**: 1595 engine + 542 panel (12 new regression tests for tool calling and MCP)
+
+### Panel v1.2.0 / Engine v0.2.18 (2026-03-09)
+- **HuggingFace download fix**: Download progress no longer stuck at 0% — tqdm `\r` chunk splitting, ANSI stripping, highest-percent extraction
+- **HF browser NaN/Unknown fix**: Model ages and authors display correctly (uses `createdAt` fallback, extracts author from modelId)
+- **macOS 15 launch fix**: `minimumSystemVersion` corrected from 26.0.0 to 14.0.0 (fixes GitHub #10)
+- **Deep stability audit**: 14 fixes across paged cache block lifecycle, KV dequantize safety, reasoning marker detection, tool fallback, Mistral JSON validation
+- **CancelledError SSE hang**: Engine cancellation now unblocks all waiting SSE consumers
+- **2125 total tests**: 1595 engine + 530 panel with full regression coverage
+
+### Panel v1.1.4 / Engine v0.2.12 (2026-03-07)
+- **tool_choice="none" fix**: Content no longer swallowed when tool markers detected with tools suppressed
+- **suppress_reasoning**: Reasoning leaks plugged in both API paths
+- **First-launch UX**: Auto-creates initial chat, dynamic About page version
+- **1571 engine tests**, **530 panel tests** across 6 vitest suites
+
+See [Panel Changelog](panel/CHANGELOG.md) and [Engine Changelog](CHANGELOG.md) for full history.
+
+---
+
+## Current Version
+
+**Engine v0.2.18** / **Panel v1.2.1** — macOS 26+ (Tahoe) for local inference, macOS 14+ for remote endpoints. Apple Silicon (M1, M2, M3, M4)
+
+## Links
+
+- **Website**: [vmlx.net](https://vmlx.net)
+- **Contact**: admin@vmlx.net
+
+## License
+
+Apache 2.0 — see [LICENSE](LICENSE) for details.

vmlx-1.0.0.dist-info/RECORD

@@ -0,0 +1,99 @@
+vmlx_engine/__init__.py,sha256=fdRdlj9A2xlAgdLmTeJLsrQCwMKK2I8fYQdG3SVNkv8,3786
+vmlx_engine/attention.py,sha256=OnPcOUsK0Kzs9u_K4VS6Xa6Ss8piMsNlHVkxsAB2nlg,7139
+vmlx_engine/benchmark.py,sha256=_GC86qPz76E2NPHLDYInYrvJ_By8JIq65SjXsBk6xS4,55298
+vmlx_engine/block_disk_store.py,sha256=xy1ls9RmxEGIHSS72Zzmw_twFXf32cVaqQ-65ql5ois,29097
+vmlx_engine/cli.py,sha256=Snr_kbaDnVJdGjP-Tq2kZmxt1JJckFsqlxGc9iUuJz0,51056
+vmlx_engine/disk_cache.py,sha256=x4S15n2FjSFNfL67fS21D4Fv-JGS-fcBDxcP53w-97s,17037
+vmlx_engine/embedding.py,sha256=dkO6YjP2O4tLJEeaig4v4wkTqByuDMG3MJuZlSFVmTw,3397
+vmlx_engine/engine_core.py,sha256=bToRsn8ybXPkl1aSz6_A_zTeGwDNUBLGEECY_-xC7Qc,27504
+vmlx_engine/gradio_app.py,sha256=5kEuDf-gkLBHydorh8DCDxLT0qrvQT6K9loGhlnV7gQ,13118
+vmlx_engine/gradio_text_app.py,sha256=_0q5OevibP-KSER82qAE7PwvheekStS0PBSG6BemEAE,5088
+vmlx_engine/image_gen.py,sha256=OSfvJk17sDHgyJDw6ZoApLeRXzwiELnv_P_eK5QDOSk,9232
+vmlx_engine/memory_cache.py,sha256=W7oLaMspIgq-yhLGrFDsu5-KthR3LgmBxtIp4ccYL38,23958
+vmlx_engine/mllm_batch_generator.py,sha256=0JQk-OJMFtCYAZ783NBQgSPxerEmMg3fPoXE3-edJh4,80639
+vmlx_engine/mllm_cache.py,sha256=E2nJx-7L1rMj-uzMsszP4mNfYLYjcfWY0RxdbNrvJSU,16548
+vmlx_engine/mllm_scheduler.py,sha256=SLj006oozcz3UKUMianf_SIIf1mCs4-Sx9k2Jh3TvlU,89599
+vmlx_engine/mlx_platform.py,sha256=CfzG8mIQP7OK9pX_vWRUy7OZLruWsjTnBPxoMF-unCY,10116
+vmlx_engine/model_config_registry.py,sha256=6_qUByzetyf248iONNqGhZd7TMfXm-vu52BegCleKbQ,7856
+vmlx_engine/model_configs.py,sha256=0SQhq_7IqF3fYmWZ5ggQ51dekiw1h_vBVgAGilsk7cM,18640
+vmlx_engine/model_registry.py,sha256=bOgEVbUgRZvl4jA-qnnsNrH3aFvYFwHeeI_8ejKWANw,5827
+vmlx_engine/model_runner.py,sha256=5oakkkNSqtzTap9A1Xw8scEpc0AEnRHOn0FMx9zaG7k,15681
+vmlx_engine/multimodal_processor.py,sha256=qzDcZzszmQ5DxPZ7-RtwdTWt3D0j-QqgCJNd_FI8X1Y,6722
+vmlx_engine/optimizations.py,sha256=xiVkF2qXhe9rEraHMJmLqCDmpZZLVFDwTAa80hQn5UU,4103
+vmlx_engine/output_collector.py,sha256=aNUIf5Lmgeu-jdl40n67m50B8iDHvoxxBwqCKSOAQ84,8075
+vmlx_engine/paged_cache.py,sha256=rQgcFixssVrtUL8ZvcMGrRbPwA9_onrdkbOv25aeNEQ,47190
+vmlx_engine/plugin.py,sha256=78NTg1Pi2D4VamUKlZhUAdhR9q3SNH_rYLDmHZYYkN8,3926
+vmlx_engine/prefix_cache.py,sha256=Yr9zcOfs8hNwMzMXK7vU6aA3csDpfw4c9R543txjBdk,48108
+vmlx_engine/request.py,sha256=eZOzNX6sTj07RVSTxvyUKa2MYARftKPqbNpCY5-sfX8,7550
+vmlx_engine/reranker.py,sha256=TjBCAI8qoRndeuUvrG105CcooU6jooH9FFvXun3dkeE,7301
+vmlx_engine/scheduler.py,sha256=Ot7sj2snRJ-qu_p5E6Cwec3wR5yGyRkjLGl8yf0eTGQ,101206
+vmlx_engine/server.py,sha256=lgL69yBWwrRE84WzomFpp_xKkhi4vMgHRDusMs5w5zc,193534
+vmlx_engine/simple.py,sha256=ibUBsfInVD_r-qQTLr9LATBrw24ZfyVnPJjW-myOUa4,15110
+vmlx_engine/speculative.py,sha256=fcMC8_ZhL3XRGixNDMa4PLbcrT4fiUvzBZvfVCs5rGE,8183
+vmlx_engine/vision_embedding_cache.py,sha256=VK5Gxc75UPTfYxKoVUsRjLLLOyqn2e8QYON1Fz3KKh8,6873
+vmlx_engine/worker.py,sha256=RPVRtSg9nlQIQNu-vuKLG3Kdp7uMh3KlVEjWZ0RnGjg,8627
+vmlx_engine/api/__init__.py,sha256=-qlYDRLM_ehkJzOatKGtAECZyjoaacWTPI4me2Xcm6I,2614
+vmlx_engine/api/anthropic_adapter.py,sha256=_UjQlaP5gERYif69cwra5iRioZuaFJya8KVgSPYLKR8,22239
+vmlx_engine/api/models.py,sha256=jI0IkgdDG3fG3VqV76xikr1p1fpqM0KN9raxag145eA,22697
+vmlx_engine/api/streaming.py,sha256=8Vbix5ulFgZNlN294bXwaimvtSUt-DyVhXYVd7RoqSI,7372
+vmlx_engine/api/tool_calling.py,sha256=nw6XW1RkPlpQL1-fzmmUNXuPgbNacqThb7MVzS_1bk0,23111
+vmlx_engine/api/utils.py,sha256=JHL0in1jJAL2Eols9B0yvEbpvRmWHsUbAaCLTN_3u0g,11611
+vmlx_engine/audio/__init__.py,sha256=veFwcZDlL3E4YE9ewCr1-ehM0EoN7Jo-2HVtuqfXAsw,578
+vmlx_engine/audio/processor.py,sha256=b7rIKNbsiA23O0anzZ0YdLGBEKp3sTDLkNWiIZqqixc,6561
+vmlx_engine/audio/stt.py,sha256=6l4_Crg8KpKmItsIOwkLQ5dTiYge0hdyWYOx3lhLsU0,5076
+vmlx_engine/audio/tts.py,sha256=TrrNFjo4_2_-rirMTUZVpJBfg0oYRAN6AgsX_7TcPa4,8942
+vmlx_engine/commands/__init__.py,sha256=BWEp1IBfddVrcik90AHCtBlVbtsJO3uor6Eo5qGHFAk,74
+vmlx_engine/commands/convert.py,sha256=oGI2JM19CXK2bhmRmOeRFJU4esgkSG-ZY1j-FsNQrZ0,17574
+vmlx_engine/commands/doctor.py,sha256=BgsVGhpQZSuJ54NsaJQqUjJG_vaBd9lcETuWtRDm4eU,10138
+vmlx_engine/commands/info.py,sha256=NyrrLnRoxD9IjSZ90x3kgMshTZ2GNaxZmOcffjKXoJg,671
+vmlx_engine/commands/list.py,sha256=gGuc88GNYbv1J2OJwR2eaOnMISPNSrBuSjFnRh6zvv8,1149
+vmlx_engine/engine/__init__.py,sha256=Jm93jgs2EVplFYqXKF6k1N3wsZ1q0szM1EgIBKeg-hI,771
+vmlx_engine/engine/base.py,sha256=_qW-_7_qtQzdVl6bZFpgSUZqBpHAUK0SEpMsg-DiO7Q,5541
+vmlx_engine/engine/batched.py,sha256=1KV50l2oSBtRiffy87MTGKaroufB1la4DsdiJUDBOKw,32660
+vmlx_engine/engine/simple.py,sha256=YN2Hha1nvTKeVvbWrtPcD2iQRrlNt_Kymkk_VNRNMXk,28797
+vmlx_engine/mcp/__init__.py,sha256=mpX1cOkGxezkMFg2ASdE8pW6Anfo-i1fYp4tlC39a2g,2124
+vmlx_engine/mcp/client.py,sha256=nP62shUP2kf5txLJjuogxoqgLQ0y4JY6mol6c1FbftM,12363
+vmlx_engine/mcp/config.py,sha256=ZXp6w6YBINmZHWf7PNZBZyyCuPk-DVMwSq8XO8E2Zdw,5229
+vmlx_engine/mcp/executor.py,sha256=5Qe-hieGPW_COoX2MDs-Y6h248OAgF6wGHLHlE542h0,16459
+vmlx_engine/mcp/manager.py,sha256=SvHmOHIQT-PWsg0na_Xo6Yyh-FQiw5Z0qHvVnAxMLJQ,8991
+vmlx_engine/mcp/security.py,sha256=-B-PDwix7YUllc8SFku3zuI4hHHFsS8gt2h3Qfl0AQM,23719
+vmlx_engine/mcp/tools.py,sha256=gX6CDAD6scQulaHj_p4KexDjS8upGZmifyVg7_MJtAc,4405
+vmlx_engine/mcp/types.py,sha256=IU6zWy9kyZFIciNPnm3FICmCuq9Lq2xhgF9i9wJ2Nt8,5170
+vmlx_engine/models/__init__.py,sha256=Y0tba7FfCsZqr_OA4FaAW7ndXmom13b0CgdJVoI8kPI,442
+vmlx_engine/models/llm.py,sha256=T-h2yeLkQTpQkak80ZGdeYnRcuWVd7xdHYoTrqPNuSo,12042
+vmlx_engine/models/mllm.py,sha256=JQSeruPgVpYOo6_frsOLtqy1FMd-KCagQWVQuzE0Mj0,70665
+vmlx_engine/reasoning/__init__.py,sha256=IhiUam8mASfxMakZ0xDeQtRtpZ1tyHm4o40hfBYswD0,2773
+vmlx_engine/reasoning/base.py,sha256=Po-W6B5fPxl1MlFnddYLtR6vH2PxDQBJXgp_W51GbLQ,3462
+vmlx_engine/reasoning/deepseek_r1_parser.py,sha256=xhIHgCDYxf1bfwK4ejikx4QRp-iBfgrZrNES93jWFyk,4119
+vmlx_engine/reasoning/gptoss_parser.py,sha256=wvbFYneLzPib8ehxijtZUUNcEWyTGJYUag-hZk9vKnM,14324
+vmlx_engine/reasoning/qwen3_parser.py,sha256=k8CqjArFDZlYl810SCcrA9GzlzYNINm33VxzadLZDgE,2545
+vmlx_engine/reasoning/think_parser.py,sha256=zIce3GaAm1ZGcTqjWx0UZslZ4rIh_UzHk74Dos8XbPg,8888
+vmlx_engine/tool_parsers/__init__.py,sha256=sMy9yR9kOM4EDvqsJ1pJN91dwzf_MSk3t59BTzuGBrA,2799
+vmlx_engine/tool_parsers/abstract_tool_parser.py,sha256=NiwhncFKGx0IpCYrLFt0CFnI-4y66Kgff7JpI_V2Wew,9597
+vmlx_engine/tool_parsers/auto_tool_parser.py,sha256=wdRYsuDw1SiTHzul-PfhRUGy2p9VIt8z_g7Zbp3t8Lo,14144
+vmlx_engine/tool_parsers/deepseek_tool_parser.py,sha256=zm4B8uYEEZtgJ_3zQTVfWeVl0grfjxq9l5pNTHjHydE,5639
+vmlx_engine/tool_parsers/functionary_tool_parser.py,sha256=SXiNBErgr79wQRijeFDhh_wEJ6fXO1kMReL8nC-0Tfg,6494
+vmlx_engine/tool_parsers/glm47_tool_parser.py,sha256=HYjk4auMSAZGhtPHIz4qoW7zvEPxoK8evMI3eIRpUxw,7909
+vmlx_engine/tool_parsers/granite_tool_parser.py,sha256=kDYG2vuCm0xDczy58pyDDQiiZMctP7UfwDSSwAr_5qE,4740
+vmlx_engine/tool_parsers/hermes_tool_parser.py,sha256=USwc1d0FZ8hUestH_NkI1m8586F3zxxilfiX7xhqkTo,9559
+vmlx_engine/tool_parsers/kimi_tool_parser.py,sha256=Qhp7ys5FDSPuEPvsBdJandkhoe-2ET5mamVsp8Ghf4Y,5331
+vmlx_engine/tool_parsers/llama_tool_parser.py,sha256=uU8CilBQbWy8pYMjUeLYxbMbZFGToRITm5Ua_Zq-1eI,4004
+vmlx_engine/tool_parsers/minimax_tool_parser.py,sha256=zFIUtJy6AOws86qE5wGUlnUu0PCwSQ7y05UrW0Vduvc,12276
+vmlx_engine/tool_parsers/mistral_tool_parser.py,sha256=0H2HtR2HXRWm_GEs-e-HHzRmf9SaRMm5vyXfHkzSM-E,9455
+vmlx_engine/tool_parsers/nemotron_tool_parser.py,sha256=YlHsxc_NypHJF_fpnGSxEziIe6Xywf-IDhcO1vfPogE,5364
+vmlx_engine/tool_parsers/qwen_tool_parser.py,sha256=kQldPdxL2bqRMOMACF5OUp2p5GDBgX4rRPYQo_vQIPs,5324
+vmlx_engine/tool_parsers/step3p5_tool_parser.py,sha256=OXy-PjGkR-zSwjOGBd5_fTlA5aeruPfOX3DB62buMRA,7857
+vmlx_engine/tool_parsers/xlam_tool_parser.py,sha256=Xkxeq8R3jZl4Ovc_gXnDWC8sUWztqURthexQur-V8S4,5754
+vmlx_engine/utils/__init__.py,sha256=CzYWXj2cfLI59cCbkpFTiSsHHgnlT6RFfof6ruJLNyY,166
+vmlx_engine/utils/cache_types.py,sha256=NJ-6u2jfyBshbFNIJ0Wfm5FTiUczrom5RhpbpczgMlM,6191
+vmlx_engine/utils/chat_templates.py,sha256=q4kyDOq7NW3xUbTegn-7eXTF_0NBIwS4tVxGW5uqUM0,11528
+vmlx_engine/utils/jang_loader.py,sha256=j6BKtKN05tBiauMpmdyvZKQ08lZBRSuOnJxHE0LrF1A,21898
+vmlx_engine/utils/mamba_cache.py,sha256=Mjju8utGS9pTfK4Os4kd9QsQX4Tzt6qRRi8gYzqa_W8,12567
+vmlx_engine/utils/model_inspector.py,sha256=L600Cw0vm4ZLui_Qxib_zINsGDu5SUCv1GR-26PN5UQ,21151
+vmlx_engine/utils/nemotron_latent_moe.py,sha256=dDY8ENbFhE-vnirEokWWdg7hhqfQ4pNyG6Ud0xbi7AE,8764
+vmlx_engine/utils/tokenizer.py,sha256=emzGrBwscFiP0Ci5ZFgVxlRhJa64hiUHrQvkDpPKQSA,9317
+vmlx-1.0.0.dist-info/METADATA,sha256=7Apcm4VI6NXvmmGMIP8W8hG9aUhQfg6xYUDA7yHQYo8,15010
+vmlx-1.0.0.dist-info/WHEEL,sha256=aeYiig01lYGDzBgS8HxWXOg3uV61G9ijOsup-k9o1sk,91
+vmlx-1.0.0.dist-info/entry_points.txt,sha256=8mdqB1DfpzN621JwfowYRrbiYziJohYxXdpynFujhrQ,279
+vmlx-1.0.0.dist-info/top_level.txt,sha256=lpCt_1sm95ED0QLpzudgm3VGOkaD9q9MZCkC8NDTkak,12
+vmlx-1.0.0.dist-info/RECORD,,

vmlx-1.0.0.dist-info/WHEEL

@@ -0,0 +1,5 @@
+Wheel-Version: 1.0
+Generator: setuptools (82.0.1)
+Root-Is-Purelib: true
+Tag: py3-none-any
+

vmlx-1.0.0.dist-info/entry_points.txt

@@ -0,0 +1,9 @@
+[console_scripts]
+vmlx = vmlx_engine.cli:main
+vmlx-engine = vmlx_engine.cli:main
+vmlx-engine-bench = vmlx_engine.benchmark:main
+vmlx-engine-chat = vmlx_engine.gradio_app:main
+vmlx-serve = vmlx_engine.cli:main
+
+[vllm.platform_plugins]
+mlx = vmlx_engine.plugin:mlx_platform_plugin

vmlx-1.0.0.dist-info/top_level.txt

@@ -0,0 +1 @@
+vmlx_engine

vmlx_engine/__init__.py

@@ -0,0 +1,138 @@
+# SPDX-License-Identifier: Apache-2.0
+"""
+vmlx-engine: Apple Silicon MLX backend for vLLM
+
+This package provides native Apple Silicon GPU acceleration for vLLM
+using Apple's MLX framework, mlx-lm for LLMs, and mlx-vlm for
+vision-language models.
+
+Features:
+- Continuous batching via vLLM-style scheduler
+- OpenAI-compatible API server
+- Support for LLM and multimodal models
+"""
+
+__version__ = "0.2.18"
+
+# All imports are lazy to allow usage on non-Apple Silicon platforms
+# (e.g., CI running on Linux) where mlx_lm is not available.
+
+
+def __getattr__(name):
+    """Lazy load all components to avoid mlx_lm import on non-Apple platforms."""
+    # Request management
+    if name in ("Request", "RequestOutput", "RequestStatus", "SamplingParams"):
+        from vmlx_engine import request
+
+        return getattr(request, name)
+
+    # Scheduler
+    if name in ("Scheduler", "SchedulerConfig", "SchedulerOutput"):
+        from vmlx_engine import scheduler
+
+        return getattr(scheduler, name)
+
+    # Engine
+    if name in ("EngineCore", "AsyncEngineCore", "EngineConfig"):
+        from vmlx_engine import engine_core
+
+        return getattr(engine_core, name)
+
+    # Prefix cache
+    if name in ("PrefixCacheManager", "PrefixCacheStats", "BlockAwarePrefixCache"):
+        from vmlx_engine import prefix_cache
+
+        return getattr(prefix_cache, name)
+
+    # Paged cache
+    if name in ("PagedCacheManager", "CacheBlock", "BlockTable", "CacheStats"):
+        from vmlx_engine import paged_cache
+
+        return getattr(paged_cache, name)
+
+    # MLLM cache (with legacy VLM aliases)
+    if name in (
+        "MLLMCacheManager",
+        "MLLMCacheStats",
+        "VLMCacheManager",
+        "VLMCacheStats",
+    ):
+        from vmlx_engine import mllm_cache
+
+        # Map legacy VLM names to MLLM
+        mllm_name = name.replace("VLM", "MLLM") if name.startswith("VLM") else name
+        return getattr(mllm_cache, mllm_name)
+
+    # Model registry
+    if name in ("get_registry", "ModelOwnershipError"):
+        from vmlx_engine import model_registry
+
+        return getattr(model_registry, name)
+
+    # Model config registry
+    if name in ("get_model_config_registry", "ModelConfigRegistry", "ModelConfig"):
+        from vmlx_engine import model_config_registry
+
+        return getattr(model_config_registry, name)
+
+    # vLLM integration components (require torch)
+    if name == "MLXPlatform":
+        from vmlx_engine.mlx_platform import MLXPlatform
+
+        return MLXPlatform
+    if name == "MLXWorker":
+        from vmlx_engine.worker import MLXWorker
+
+        return MLXWorker
+    if name == "MLXModelRunner":
+        from vmlx_engine.model_runner import MLXModelRunner
+
+        return MLXModelRunner
+    if name == "MLXAttentionBackend":
+        from vmlx_engine.attention import MLXAttentionBackend
+
+        return MLXAttentionBackend
+
+    raise AttributeError(f"module {__name__!r} has no attribute {name!r}")
+
+
+__all__ = [
+    # Core (lazy loaded, require torch)
+    "MLXPlatform",
+    "MLXWorker",
+    "MLXModelRunner",
+    "MLXAttentionBackend",
+    # Request management
+    "Request",
+    "RequestOutput",
+    "RequestStatus",
+    "SamplingParams",
+    # Scheduler
+    "Scheduler",
+    "SchedulerConfig",
+    "SchedulerOutput",
+    # Engine
+    "EngineCore",
+    "AsyncEngineCore",
+    "EngineConfig",
+    # Model registry
+    "get_registry",
+    "ModelOwnershipError",
+    # Prefix cache (LLM)
+    "PrefixCacheManager",
+    "PrefixCacheStats",
+    "BlockAwarePrefixCache",
+    # Paged cache (memory efficiency)
+    "PagedCacheManager",
+    "CacheBlock",
+    "BlockTable",
+    "CacheStats",
+    # MLLM cache (images/videos)
+    "MLLMCacheManager",
+    "MLLMCacheStats",
+    # Legacy aliases
+    "VLMCacheManager",
+    "VLMCacheStats",
+    # Version
+    "__version__",
+]