llamastudio 1.0.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- llamastudio-1.0.0/PKG-INFO +332 -0
- llamastudio-1.0.0/README.md +298 -0
- llamastudio-1.0.0/app/__init__.py +1 -0
- llamastudio-1.0.0/app/chat.py +526 -0
- llamastudio-1.0.0/app/cli.py +560 -0
- llamastudio-1.0.0/app/config.py +165 -0
- llamastudio-1.0.0/app/downloader.py +284 -0
- llamastudio-1.0.0/app/gpu_utils.py +103 -0
- llamastudio-1.0.0/app/launcher.py +192 -0
- llamastudio-1.0.0/app/logger.py +52 -0
- llamastudio-1.0.0/app/main.py +446 -0
- llamastudio-1.0.0/app/model_manager.py +184 -0
- llamastudio-1.0.0/app/server_manager.py +401 -0
- llamastudio-1.0.0/app/templates/index.html +3978 -0
- llamastudio-1.0.0/app/tools.py +236 -0
- llamastudio-1.0.0/llamastudio.egg-info/PKG-INFO +332 -0
- llamastudio-1.0.0/llamastudio.egg-info/SOURCES.txt +28 -0
- llamastudio-1.0.0/llamastudio.egg-info/dependency_links.txt +1 -0
- llamastudio-1.0.0/llamastudio.egg-info/entry_points.txt +3 -0
- llamastudio-1.0.0/llamastudio.egg-info/requires.txt +11 -0
- llamastudio-1.0.0/llamastudio.egg-info/top_level.txt +1 -0
- llamastudio-1.0.0/pyproject.toml +58 -0
- llamastudio-1.0.0/setup.cfg +4 -0
- llamastudio-1.0.0/tests/test_chat.py +69 -0
- llamastudio-1.0.0/tests/test_downloader.py +206 -0
- llamastudio-1.0.0/tests/test_gpu.py +118 -0
- llamastudio-1.0.0/tests/test_local_models.py +129 -0
- llamastudio-1.0.0/tests/test_models.py +92 -0
- llamastudio-1.0.0/tests/test_regex.py +180 -0
- llamastudio-1.0.0/tests/test_sandbox.py +72 -0
|
@@ -0,0 +1,332 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: llamastudio
|
|
3
|
+
Version: 1.0.0
|
|
4
|
+
Summary: FastAPI backend & desktop launcher for running local models via llama.cpp
|
|
5
|
+
Author: LLamaStudio contributors
|
|
6
|
+
Project-URL: Homepage, https://github.com/gnulnx/LlamaStudio
|
|
7
|
+
Project-URL: Repository, https://github.com/gnulnx/LlamaStudio
|
|
8
|
+
Project-URL: Issues, https://github.com/gnulnx/LlamaStudio/issues
|
|
9
|
+
Keywords: llama.cpp,llm,local-ai,fastapi,gguf
|
|
10
|
+
Classifier: Development Status :: 4 - Beta
|
|
11
|
+
Classifier: Environment :: Console
|
|
12
|
+
Classifier: Framework :: FastAPI
|
|
13
|
+
Classifier: Intended Audience :: End Users/Desktop
|
|
14
|
+
Classifier: Operating System :: MacOS
|
|
15
|
+
Classifier: Operating System :: POSIX :: Linux
|
|
16
|
+
Classifier: Programming Language :: Python :: 3
|
|
17
|
+
Classifier: Programming Language :: Python :: 3.10
|
|
18
|
+
Classifier: Programming Language :: Python :: 3.11
|
|
19
|
+
Classifier: Programming Language :: Python :: 3.12
|
|
20
|
+
Classifier: Programming Language :: Python :: 3.13
|
|
21
|
+
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
|
|
22
|
+
Requires-Python: >=3.10
|
|
23
|
+
Description-Content-Type: text/markdown
|
|
24
|
+
Requires-Dist: fastapi>=0.115.0
|
|
25
|
+
Requires-Dist: httpx
|
|
26
|
+
Requires-Dist: jinja2
|
|
27
|
+
Requires-Dist: pydantic-settings
|
|
28
|
+
Requires-Dist: rich
|
|
29
|
+
Requires-Dist: rich-click
|
|
30
|
+
Requires-Dist: uvicorn
|
|
31
|
+
Provides-Extra: dev
|
|
32
|
+
Requires-Dist: pytest; extra == "dev"
|
|
33
|
+
Requires-Dist: ruff; extra == "dev"
|
|
34
|
+
|
|
35
|
+
# ๐ฆ LLamaStudio
|
|
36
|
+
|
|
37
|
+
A desktop chat interface and local server manager for `llama.cpp`, crafted with **FastAPI** + **HTMX** for ultra-lightweight, zero-framework execution.
|
|
38
|
+
|
|
39
|
+
**LLamaStudio** is a self-contained local workspace that manages model lifecycles, features a smart VRAM estimator, scans local folders, and lets you search and download models directly from the Hugging Face Hub.
|
|
40
|
+
|
|
41
|
+
---
|
|
42
|
+
|
|
43
|
+
## ๐ธ Screenshots & Showcase
|
|
44
|
+
|
|
45
|
+
### 1. Main Chat Dashboard
|
|
46
|
+
A Pop!_OS-harmonized dark interface with streaming, collapsible markdown reasoning (thinking) processes, and real-time agentic tool execution logs.
|
|
47
|
+

|
|
48
|
+
|
|
49
|
+
### 2. GGUF Model Browser & Settings
|
|
50
|
+
A dynamic local model explorer that scans your directories and lets you adjust context length, GPU offload layers, CPU threads, flash attention, and KV cache quantizations on the fly.
|
|
51
|
+

|
|
52
|
+
|
|
53
|
+
### 3. Hugging Face Discover Hub
|
|
54
|
+
Browse the entire Hugging Face GGUF catalog. Features a **Smart VRAM Offload Estimator** calibrated to your hardware, and a floating background download progress card with live speed (MB/s), ETA, and cancel controls.
|
|
55
|
+

|
|
56
|
+
|
|
57
|
+
---
|
|
58
|
+
|
|
59
|
+
## โจ Key Features
|
|
60
|
+
|
|
61
|
+
- **โก Zero Node Modules**: Built with HTMX, Tailwind CSS (via CDN), and Vanilla JS. It is incredibly fast, responsive, and has a memory footprint of just a few megabytes.
|
|
62
|
+
- **๐งญ Hugging Face Discover Tab**: Search the public Hugging Face Hub for GGUF models directly inside the app, view readmes, select quantizations, and download files in the background.
|
|
63
|
+
- **๐ Smart VRAM Estimator**: calulated specifically for your hardware (fits fully on **RTX 5090 32GB VRAM**, partial offload warning, or heavy CPU fallback warning).
|
|
64
|
+
- **๐ Automatic Model Scanning**: Scans standard directories (like `~/.lmstudio/models`) automatically on startup or via a one-click rescan button.
|
|
65
|
+
- **๐ช Process Lifecycle Manager**: The underlying `llama-server` process only spins up when you explicitly load a model, releasing all system resources and GPU VRAM instantly when you click "Eject".
|
|
66
|
+
- **๐ง Configurable Workspace Sandboxing**: Supports sandboxed agentic tool use (file read/write, commands, etc.) with real-time logs in the UI. Sandboxing boundaries can be easily broadened or bypassed entirely via a `.env` file or environment variables to grant the model system access.
|
|
67
|
+
- **๐ฅ๏ธ XDG-Compliant Persistence**: Conversations and model settings profiles are stored outside the codebase directory in standard `~/.config/llamastudio/` with automated backward-compatible migrations!
|
|
68
|
+
- **๐ฆ Full Linux & macOS Portability**: Server binaries and model directories are resolved dynamically on startup.
|
|
69
|
+
|
|
70
|
+
---
|
|
71
|
+
|
|
72
|
+
## ๐ ๏ธ Installation & Setup
|
|
73
|
+
|
|
74
|
+
LLamaStudio is compatible with **Linux** and **macOS** out-of-the-box. Choose your OS and python virtual environment preference below.
|
|
75
|
+
|
|
76
|
+
### ๐ง 1. Linux Installation
|
|
77
|
+
|
|
78
|
+
#### Prerequisites
|
|
79
|
+
1. **Python 3.10+** (Recommended: Python 3.13)
|
|
80
|
+
2. **llama.cpp** built from source (or pre-compiled binary):
|
|
81
|
+
- By default, the app dynamically looks for the `llama-server` binary globally on your system PATH or locally inside your home directory at `~/llama.cpp/build/bin/llama-server`.
|
|
82
|
+
|
|
83
|
+
#### Environment Setup
|
|
84
|
+
|
|
85
|
+
##### Option A: Install from PyPI
|
|
86
|
+
```bash
|
|
87
|
+
pip install llamastudio
|
|
88
|
+
```
|
|
89
|
+
|
|
90
|
+
##### Option B: Conda / Miniconda from source
|
|
91
|
+
```bash
|
|
92
|
+
# 1. Clone the repository
|
|
93
|
+
git clone https://github.com/gnulnx/LlamaStudio.git
|
|
94
|
+
cd LlamaStudio
|
|
95
|
+
|
|
96
|
+
# 2. Create and activate a conda environment
|
|
97
|
+
conda create -n llamastudio python=3.13 -y
|
|
98
|
+
conda activate llamastudio
|
|
99
|
+
|
|
100
|
+
# 3. Install LlamaStudio and its dependencies
|
|
101
|
+
pip install -e .
|
|
102
|
+
```
|
|
103
|
+
|
|
104
|
+
##### Option C: Python Virtualenv (`venv`) from source
|
|
105
|
+
```bash
|
|
106
|
+
# 1. Clone the repository
|
|
107
|
+
git clone https://github.com/gnulnx/LlamaStudio.git
|
|
108
|
+
cd LlamaStudio
|
|
109
|
+
|
|
110
|
+
# 2. Create and activate a python venv environment
|
|
111
|
+
python3 -m venv .venv
|
|
112
|
+
source .venv/bin/activate
|
|
113
|
+
|
|
114
|
+
# 3. Install LlamaStudio and its dependencies
|
|
115
|
+
pip install -e .
|
|
116
|
+
```
|
|
117
|
+
|
|
118
|
+
#### ๐ฅ๏ธ Linux Desktop Launcher Integration (Optional)
|
|
119
|
+
To integrate LLamaStudio directly into your Linux Application launcher menu (e.g., GNOME / Pop!_OS):
|
|
120
|
+
```bash
|
|
121
|
+
# 1. Copy the desktop file to your local applications directory
|
|
122
|
+
cp llamastudio.desktop ~/.local/share/applications/
|
|
123
|
+
|
|
124
|
+
# 2. Copy the custom SVG icon to your local icons directory
|
|
125
|
+
mkdir -p ~/.local/share/icons/hicolor/128x128/apps/
|
|
126
|
+
cp llamastudio.svg ~/.local/share/icons/hicolor/128x128/apps/
|
|
127
|
+
|
|
128
|
+
# 3. Update your desktop database and icon cache
|
|
129
|
+
update-desktop-database ~/.local/share/applications/
|
|
130
|
+
gtk-update-icon-cache -f -t ~/.local/share/icons
|
|
131
|
+
```
|
|
132
|
+
*Note: If you are using a virtualenv, edit the executable path inside `~/.local/share/applications/llamastudio.desktop` to point to your specific `.venv/bin/python` interpreter.*
|
|
133
|
+
|
|
134
|
+
---
|
|
135
|
+
|
|
136
|
+
### ๐ 2. macOS Installation
|
|
137
|
+
|
|
138
|
+
#### Prerequisites
|
|
139
|
+
1. **Python 3.10+**
|
|
140
|
+
2. **llama.cpp** installed globally via Homebrew (highly recommended for macOS):
|
|
141
|
+
```bash
|
|
142
|
+
brew install llama.cpp
|
|
143
|
+
```
|
|
144
|
+
*(This automatically places the `llama-server` binary globally on your system PATH, which LLamaStudio will auto-detect immediately!)*
|
|
145
|
+
|
|
146
|
+
#### Environment Setup
|
|
147
|
+
|
|
148
|
+
##### Option A: Install from PyPI
|
|
149
|
+
```bash
|
|
150
|
+
pip install llamastudio
|
|
151
|
+
```
|
|
152
|
+
|
|
153
|
+
##### Option B: Conda / Miniconda from source
|
|
154
|
+
```bash
|
|
155
|
+
# 1. Clone the repository
|
|
156
|
+
git clone https://github.com/gnulnx/LlamaStudio.git
|
|
157
|
+
cd LlamaStudio
|
|
158
|
+
|
|
159
|
+
# 2. Create and activate environment
|
|
160
|
+
conda create -n llamastudio python=3.13 -y
|
|
161
|
+
conda activate llamastudio
|
|
162
|
+
|
|
163
|
+
# 3. Install LlamaStudio and its dependencies
|
|
164
|
+
pip install -e .
|
|
165
|
+
```
|
|
166
|
+
|
|
167
|
+
##### Option C: Python Virtualenv (`venv`) from source
|
|
168
|
+
```bash
|
|
169
|
+
# 1. Clone the repository
|
|
170
|
+
git clone https://github.com/gnulnx/LlamaStudio.git
|
|
171
|
+
cd LlamaStudio
|
|
172
|
+
|
|
173
|
+
# 2. Create and activate venv
|
|
174
|
+
python3 -m venv .venv
|
|
175
|
+
source .venv/bin/activate
|
|
176
|
+
|
|
177
|
+
# 3. Install LlamaStudio and its dependencies
|
|
178
|
+
pip install -e .
|
|
179
|
+
```
|
|
180
|
+
|
|
181
|
+
---
|
|
182
|
+
|
|
183
|
+
### ๐ช 3. Windows Installation
|
|
184
|
+
*Note: Native Windows execution is currently **untested**.*
|
|
185
|
+
However, you can run LLamaStudio on Windows seamlessly via **WSL2** (Windows Subsystem for Linux) by following the standard **Linux Installation** guide above.
|
|
186
|
+
|
|
187
|
+
Pull requests extending native Windows support (e.g., resolving `.exe` binaries) are highly welcome!
|
|
188
|
+
|
|
189
|
+
---
|
|
190
|
+
|
|
191
|
+
## ๐ Running the Application
|
|
192
|
+
|
|
193
|
+
### Option A: Via Unified CLI (`lls` - Recommended)
|
|
194
|
+
You can link and install LlamaStudio's CLI utility locally to control the desktop app and server seamlessly:
|
|
195
|
+
```bash
|
|
196
|
+
# Start the desktop application server and open browser UI
|
|
197
|
+
lls reload
|
|
198
|
+
```
|
|
199
|
+
|
|
200
|
+
### Option B: Via App Launcher Command
|
|
201
|
+
After installing from PyPI or source, run:
|
|
202
|
+
```bash
|
|
203
|
+
llamastudio
|
|
204
|
+
```
|
|
205
|
+
|
|
206
|
+
### Option C: Via Direct Startup Script
|
|
207
|
+
When working from a source checkout, activate your virtual environment and run the startup script:
|
|
208
|
+
```bash
|
|
209
|
+
# Activate your env (Conda)
|
|
210
|
+
conda activate llamastudio
|
|
211
|
+
# OR (Venv)
|
|
212
|
+
source .venv/bin/activate
|
|
213
|
+
|
|
214
|
+
# Start the application
|
|
215
|
+
python start.py
|
|
216
|
+
```
|
|
217
|
+
This will launch the FastAPI backend on `http://127.0.0.1:8765` and automatically open your default browser to the chat dashboard.
|
|
218
|
+
|
|
219
|
+
### Via Application Menu (Linux)
|
|
220
|
+
Search for **LLamaStudio** in your desktop search bar (press Super, type "Llama") and click to launch!
|
|
221
|
+
|
|
222
|
+
---
|
|
223
|
+
|
|
224
|
+
## ๐ ๏ธ Unified Command-Line Interface (`lls`)
|
|
225
|
+
|
|
226
|
+
LlamaStudio features a CLI built using `rich-click` for visual dashboards and operational efficiency.
|
|
227
|
+
|
|
228
|
+
### CLI Subcommands Reference
|
|
229
|
+
|
|
230
|
+
| Command | Usage | Description |
|
|
231
|
+
| :--- | :--- | :--- |
|
|
232
|
+
| `reload` | `lls reload` | Gracefully starts or restarts the desktop FastAPI application backend. |
|
|
233
|
+
| `status` | `lls status` | Visual dashboard of FastAPI backend status, loaded model parameters, and GPU memory (VRAM). |
|
|
234
|
+
| `ls` | `lls ls` | Prints an elegant table of all GGUF models scanned across local directories. |
|
|
235
|
+
| `load` | `lls load [MODEL]` | Boots the server with a GGUF model. If `MODEL` is omitted, prompts you with an interactive menu. |
|
|
236
|
+
| `eject` | `lls eject` | Gracefully unloads the active model to free GPU and CPU RAM. |
|
|
237
|
+
| `oneshot`| `lls oneshot "prompt"` | Streams thinking traces, text, and executes agentic tools directly in your terminal. |
|
|
238
|
+
|
|
239
|
+
For example, to boot a model interactively:
|
|
240
|
+
```bash
|
|
241
|
+
$ lls load
|
|
242
|
+
Available Scanned Models:
|
|
243
|
+
1. Qwen3.6-35B-A3B-UD-Q5_K_M (25.2 GB)
|
|
244
|
+
2. gemma-4-26B-A4B-it-Q8_0 (25.0 GB)
|
|
245
|
+
3. DeepSeek-R1-Distill-Qwen-32B-Q5_K_M (21.7 GB)
|
|
246
|
+
|
|
247
|
+
Select a model number to load: 3
|
|
248
|
+
Loading model 'DeepSeek-R1-Distill-Qwen-32B-Q5_K_M'...
|
|
249
|
+
```
|
|
250
|
+
|
|
251
|
+
---
|
|
252
|
+
|
|
253
|
+
## โ๏ธ Configuration & Customization
|
|
254
|
+
|
|
255
|
+
The application runs fully out-of-the-box with no manual configuration. However, you can customize default settings inside `app/config.py` or by using a local `.env` file:
|
|
256
|
+
- `LLAMA_SERVER_BIN`: The absolute path to your `llama-server` binary (automatically resolved on PATH/home).
|
|
257
|
+
- `MODEL_DIRS`: List of local directories to scan for GGUF model files (defaults to `~/.lmstudio/models`).
|
|
258
|
+
- `APP_PORT`: FastAPI web server port (defaults to `8765`).
|
|
259
|
+
|
|
260
|
+
---
|
|
261
|
+
|
|
262
|
+
## ๐ก๏ธ Workspace Sandboxing & Embodiment
|
|
263
|
+
|
|
264
|
+
By default, LlamaStudio restricts agent tools (like reading, writing, and listing files) to the repository directory to prevent accidental path traversals.
|
|
265
|
+
|
|
266
|
+
If you want to grant your model more control (e.g., to act as a system embodiment or write to external directories), you can customize these boundaries by placing a `.env` file in the root directory:
|
|
267
|
+
|
|
268
|
+
* **Broaden the Sandbox Root** (e.g., allow the agent access to your entire home directory):
|
|
269
|
+
```ini
|
|
270
|
+
LLAMASTUDIO_WORKSPACE_ROOT=/home/username
|
|
271
|
+
```
|
|
272
|
+
* **Completely Disable Sandboxing** (gives the agent full filesystem control based on the running process's permissions):
|
|
273
|
+
```ini
|
|
274
|
+
LLAMASTUDIO_DISABLE_SANDBOX=true
|
|
275
|
+
```
|
|
276
|
+
|
|
277
|
+
Restart or reload the backend (`lls reload`) after modifying the `.env` file to apply changes.
|
|
278
|
+
|
|
279
|
+
---
|
|
280
|
+
|
|
281
|
+
## ๐งช Testing Suite
|
|
282
|
+
|
|
283
|
+
LlamaStudio features both standard unit tests and comprehensive GGUF integration tests.
|
|
284
|
+
|
|
285
|
+
### 1. Standard Unit Tests
|
|
286
|
+
Verify local installation and confirm backend routing, regex parsing, and sandboxing safety behaviors by running our mock-based test suite:
|
|
287
|
+
```bash
|
|
288
|
+
python -m unittest discover tests
|
|
289
|
+
```
|
|
290
|
+
|
|
291
|
+
### 2. GGUF Model Integration Tests
|
|
292
|
+
For local environments containing active GPUs and downloaded models, you can run the full multi-model GGUF tool-calling integration suite to verify real-time execution robustness across various chat templates:
|
|
293
|
+
```bash
|
|
294
|
+
# Run GGUF model integration tests locally
|
|
295
|
+
./tests/test_all.sh
|
|
296
|
+
```
|
|
297
|
+
*(These tests are automatically skipped in standard CI/CD environments and default `pytest` runs using `@pytest.mark.skipif` to keep pipeline checks fast.)*
|
|
298
|
+
|
|
299
|
+
---
|
|
300
|
+
|
|
301
|
+
## ๐๏ธ Project Structure
|
|
302
|
+
|
|
303
|
+
```
|
|
304
|
+
LlamaStudio/
|
|
305
|
+
โโโ start.py # Main entrypoint (starts FastAPI + opens browser)
|
|
306
|
+
โโโ pyproject.toml # Package metadata, CLI entrypoint, and dependencies
|
|
307
|
+
โโโ llamastudio.desktop # GNOME/Linux desktop launcher metadata
|
|
308
|
+
โโโ llamastudio.svg # Custom application vector icon
|
|
309
|
+
โโโ app/
|
|
310
|
+
โ โโโ config.py # Settings & dynamic path configurations
|
|
311
|
+
โ โโโ main.py # FastAPI backend endpoints & routing
|
|
312
|
+
โ โโโ chat.py # Conversations registry, templates & chat streaming
|
|
313
|
+
โ โโโ downloader.py # Async background download manager (chunked writes)
|
|
314
|
+
โ โโโ model_manager.py # Scans local paths and Hugging Face Hub
|
|
315
|
+
โ โโโ server_manager.py # llama-server subprocess process lifecycle controller
|
|
316
|
+
โ โโโ logger.py # Centralized logger
|
|
317
|
+
โ โโโ tools.py # Sandboxed local workspace tools for LLM agent use
|
|
318
|
+
โ โโโ templates/
|
|
319
|
+
โ โโโ index.html # Interactive HTMX frontend interface
|
|
320
|
+
โโโ tests/
|
|
321
|
+
โ โโโ test_downloader.py # Comprehensive mocked test cases (100% passing)
|
|
322
|
+
โโโ imgs/
|
|
323
|
+
โโโ chat_interface.png # Screenshot: Main Chat interface
|
|
324
|
+
โโโ model_settings.png # Screenshot: Model explorer & settings
|
|
325
|
+
โโโ discover_models.png# Screenshot: HF Discover & Downloader panel
|
|
326
|
+
```
|
|
327
|
+
|
|
328
|
+
---
|
|
329
|
+
|
|
330
|
+
## ๐ License
|
|
331
|
+
|
|
332
|
+
LLamaStudio is open-source software licensed under the [MIT License](LICENSE).
|
|
@@ -0,0 +1,298 @@
|
|
|
1
|
+
# ๐ฆ LLamaStudio
|
|
2
|
+
|
|
3
|
+
A desktop chat interface and local server manager for `llama.cpp`, crafted with **FastAPI** + **HTMX** for ultra-lightweight, zero-framework execution.
|
|
4
|
+
|
|
5
|
+
**LLamaStudio** is a self-contained local workspace that manages model lifecycles, features a smart VRAM estimator, scans local folders, and lets you search and download models directly from the Hugging Face Hub.
|
|
6
|
+
|
|
7
|
+
---
|
|
8
|
+
|
|
9
|
+
## ๐ธ Screenshots & Showcase
|
|
10
|
+
|
|
11
|
+
### 1. Main Chat Dashboard
|
|
12
|
+
A Pop!_OS-harmonized dark interface with streaming, collapsible markdown reasoning (thinking) processes, and real-time agentic tool execution logs.
|
|
13
|
+

|
|
14
|
+
|
|
15
|
+
### 2. GGUF Model Browser & Settings
|
|
16
|
+
A dynamic local model explorer that scans your directories and lets you adjust context length, GPU offload layers, CPU threads, flash attention, and KV cache quantizations on the fly.
|
|
17
|
+

|
|
18
|
+
|
|
19
|
+
### 3. Hugging Face Discover Hub
|
|
20
|
+
Browse the entire Hugging Face GGUF catalog. Features a **Smart VRAM Offload Estimator** calibrated to your hardware, and a floating background download progress card with live speed (MB/s), ETA, and cancel controls.
|
|
21
|
+

|
|
22
|
+
|
|
23
|
+
---
|
|
24
|
+
|
|
25
|
+
## โจ Key Features
|
|
26
|
+
|
|
27
|
+
- **โก Zero Node Modules**: Built with HTMX, Tailwind CSS (via CDN), and Vanilla JS. It is incredibly fast, responsive, and has a memory footprint of just a few megabytes.
|
|
28
|
+
- **๐งญ Hugging Face Discover Tab**: Search the public Hugging Face Hub for GGUF models directly inside the app, view readmes, select quantizations, and download files in the background.
|
|
29
|
+
- **๐ Smart VRAM Estimator**: calulated specifically for your hardware (fits fully on **RTX 5090 32GB VRAM**, partial offload warning, or heavy CPU fallback warning).
|
|
30
|
+
- **๐ Automatic Model Scanning**: Scans standard directories (like `~/.lmstudio/models`) automatically on startup or via a one-click rescan button.
|
|
31
|
+
- **๐ช Process Lifecycle Manager**: The underlying `llama-server` process only spins up when you explicitly load a model, releasing all system resources and GPU VRAM instantly when you click "Eject".
|
|
32
|
+
- **๐ง Configurable Workspace Sandboxing**: Supports sandboxed agentic tool use (file read/write, commands, etc.) with real-time logs in the UI. Sandboxing boundaries can be easily broadened or bypassed entirely via a `.env` file or environment variables to grant the model system access.
|
|
33
|
+
- **๐ฅ๏ธ XDG-Compliant Persistence**: Conversations and model settings profiles are stored outside the codebase directory in standard `~/.config/llamastudio/` with automated backward-compatible migrations!
|
|
34
|
+
- **๐ฆ Full Linux & macOS Portability**: Server binaries and model directories are resolved dynamically on startup.
|
|
35
|
+
|
|
36
|
+
---
|
|
37
|
+
|
|
38
|
+
## ๐ ๏ธ Installation & Setup
|
|
39
|
+
|
|
40
|
+
LLamaStudio is compatible with **Linux** and **macOS** out-of-the-box. Choose your OS and python virtual environment preference below.
|
|
41
|
+
|
|
42
|
+
### ๐ง 1. Linux Installation
|
|
43
|
+
|
|
44
|
+
#### Prerequisites
|
|
45
|
+
1. **Python 3.10+** (Recommended: Python 3.13)
|
|
46
|
+
2. **llama.cpp** built from source (or pre-compiled binary):
|
|
47
|
+
- By default, the app dynamically looks for the `llama-server` binary globally on your system PATH or locally inside your home directory at `~/llama.cpp/build/bin/llama-server`.
|
|
48
|
+
|
|
49
|
+
#### Environment Setup
|
|
50
|
+
|
|
51
|
+
##### Option A: Install from PyPI
|
|
52
|
+
```bash
|
|
53
|
+
pip install llamastudio
|
|
54
|
+
```
|
|
55
|
+
|
|
56
|
+
##### Option B: Conda / Miniconda from source
|
|
57
|
+
```bash
|
|
58
|
+
# 1. Clone the repository
|
|
59
|
+
git clone https://github.com/gnulnx/LlamaStudio.git
|
|
60
|
+
cd LlamaStudio
|
|
61
|
+
|
|
62
|
+
# 2. Create and activate a conda environment
|
|
63
|
+
conda create -n llamastudio python=3.13 -y
|
|
64
|
+
conda activate llamastudio
|
|
65
|
+
|
|
66
|
+
# 3. Install LlamaStudio and its dependencies
|
|
67
|
+
pip install -e .
|
|
68
|
+
```
|
|
69
|
+
|
|
70
|
+
##### Option C: Python Virtualenv (`venv`) from source
|
|
71
|
+
```bash
|
|
72
|
+
# 1. Clone the repository
|
|
73
|
+
git clone https://github.com/gnulnx/LlamaStudio.git
|
|
74
|
+
cd LlamaStudio
|
|
75
|
+
|
|
76
|
+
# 2. Create and activate a python venv environment
|
|
77
|
+
python3 -m venv .venv
|
|
78
|
+
source .venv/bin/activate
|
|
79
|
+
|
|
80
|
+
# 3. Install LlamaStudio and its dependencies
|
|
81
|
+
pip install -e .
|
|
82
|
+
```
|
|
83
|
+
|
|
84
|
+
#### ๐ฅ๏ธ Linux Desktop Launcher Integration (Optional)
|
|
85
|
+
To integrate LLamaStudio directly into your Linux Application launcher menu (e.g., GNOME / Pop!_OS):
|
|
86
|
+
```bash
|
|
87
|
+
# 1. Copy the desktop file to your local applications directory
|
|
88
|
+
cp llamastudio.desktop ~/.local/share/applications/
|
|
89
|
+
|
|
90
|
+
# 2. Copy the custom SVG icon to your local icons directory
|
|
91
|
+
mkdir -p ~/.local/share/icons/hicolor/128x128/apps/
|
|
92
|
+
cp llamastudio.svg ~/.local/share/icons/hicolor/128x128/apps/
|
|
93
|
+
|
|
94
|
+
# 3. Update your desktop database and icon cache
|
|
95
|
+
update-desktop-database ~/.local/share/applications/
|
|
96
|
+
gtk-update-icon-cache -f -t ~/.local/share/icons
|
|
97
|
+
```
|
|
98
|
+
*Note: If you are using a virtualenv, edit the executable path inside `~/.local/share/applications/llamastudio.desktop` to point to your specific `.venv/bin/python` interpreter.*
|
|
99
|
+
|
|
100
|
+
---
|
|
101
|
+
|
|
102
|
+
### ๐ 2. macOS Installation
|
|
103
|
+
|
|
104
|
+
#### Prerequisites
|
|
105
|
+
1. **Python 3.10+**
|
|
106
|
+
2. **llama.cpp** installed globally via Homebrew (highly recommended for macOS):
|
|
107
|
+
```bash
|
|
108
|
+
brew install llama.cpp
|
|
109
|
+
```
|
|
110
|
+
*(This automatically places the `llama-server` binary globally on your system PATH, which LLamaStudio will auto-detect immediately!)*
|
|
111
|
+
|
|
112
|
+
#### Environment Setup
|
|
113
|
+
|
|
114
|
+
##### Option A: Install from PyPI
|
|
115
|
+
```bash
|
|
116
|
+
pip install llamastudio
|
|
117
|
+
```
|
|
118
|
+
|
|
119
|
+
##### Option B: Conda / Miniconda from source
|
|
120
|
+
```bash
|
|
121
|
+
# 1. Clone the repository
|
|
122
|
+
git clone https://github.com/gnulnx/LlamaStudio.git
|
|
123
|
+
cd LlamaStudio
|
|
124
|
+
|
|
125
|
+
# 2. Create and activate environment
|
|
126
|
+
conda create -n llamastudio python=3.13 -y
|
|
127
|
+
conda activate llamastudio
|
|
128
|
+
|
|
129
|
+
# 3. Install LlamaStudio and its dependencies
|
|
130
|
+
pip install -e .
|
|
131
|
+
```
|
|
132
|
+
|
|
133
|
+
##### Option C: Python Virtualenv (`venv`) from source
|
|
134
|
+
```bash
|
|
135
|
+
# 1. Clone the repository
|
|
136
|
+
git clone https://github.com/gnulnx/LlamaStudio.git
|
|
137
|
+
cd LlamaStudio
|
|
138
|
+
|
|
139
|
+
# 2. Create and activate venv
|
|
140
|
+
python3 -m venv .venv
|
|
141
|
+
source .venv/bin/activate
|
|
142
|
+
|
|
143
|
+
# 3. Install LlamaStudio and its dependencies
|
|
144
|
+
pip install -e .
|
|
145
|
+
```
|
|
146
|
+
|
|
147
|
+
---
|
|
148
|
+
|
|
149
|
+
### ๐ช 3. Windows Installation
|
|
150
|
+
*Note: Native Windows execution is currently **untested**.*
|
|
151
|
+
However, you can run LLamaStudio on Windows seamlessly via **WSL2** (Windows Subsystem for Linux) by following the standard **Linux Installation** guide above.
|
|
152
|
+
|
|
153
|
+
Pull requests extending native Windows support (e.g., resolving `.exe` binaries) are highly welcome!
|
|
154
|
+
|
|
155
|
+
---
|
|
156
|
+
|
|
157
|
+
## ๐ Running the Application
|
|
158
|
+
|
|
159
|
+
### Option A: Via Unified CLI (`lls` - Recommended)
|
|
160
|
+
You can link and install LlamaStudio's CLI utility locally to control the desktop app and server seamlessly:
|
|
161
|
+
```bash
|
|
162
|
+
# Start the desktop application server and open browser UI
|
|
163
|
+
lls reload
|
|
164
|
+
```
|
|
165
|
+
|
|
166
|
+
### Option B: Via App Launcher Command
|
|
167
|
+
After installing from PyPI or source, run:
|
|
168
|
+
```bash
|
|
169
|
+
llamastudio
|
|
170
|
+
```
|
|
171
|
+
|
|
172
|
+
### Option C: Via Direct Startup Script
|
|
173
|
+
When working from a source checkout, activate your virtual environment and run the startup script:
|
|
174
|
+
```bash
|
|
175
|
+
# Activate your env (Conda)
|
|
176
|
+
conda activate llamastudio
|
|
177
|
+
# OR (Venv)
|
|
178
|
+
source .venv/bin/activate
|
|
179
|
+
|
|
180
|
+
# Start the application
|
|
181
|
+
python start.py
|
|
182
|
+
```
|
|
183
|
+
This will launch the FastAPI backend on `http://127.0.0.1:8765` and automatically open your default browser to the chat dashboard.
|
|
184
|
+
|
|
185
|
+
### Via Application Menu (Linux)
|
|
186
|
+
Search for **LLamaStudio** in your desktop search bar (press Super, type "Llama") and click to launch!
|
|
187
|
+
|
|
188
|
+
---
|
|
189
|
+
|
|
190
|
+
## ๐ ๏ธ Unified Command-Line Interface (`lls`)
|
|
191
|
+
|
|
192
|
+
LlamaStudio features a CLI built using `rich-click` for visual dashboards and operational efficiency.
|
|
193
|
+
|
|
194
|
+
### CLI Subcommands Reference
|
|
195
|
+
|
|
196
|
+
| Command | Usage | Description |
|
|
197
|
+
| :--- | :--- | :--- |
|
|
198
|
+
| `reload` | `lls reload` | Gracefully starts or restarts the desktop FastAPI application backend. |
|
|
199
|
+
| `status` | `lls status` | Visual dashboard of FastAPI backend status, loaded model parameters, and GPU memory (VRAM). |
|
|
200
|
+
| `ls` | `lls ls` | Prints an elegant table of all GGUF models scanned across local directories. |
|
|
201
|
+
| `load` | `lls load [MODEL]` | Boots the server with a GGUF model. If `MODEL` is omitted, prompts you with an interactive menu. |
|
|
202
|
+
| `eject` | `lls eject` | Gracefully unloads the active model to free GPU and CPU RAM. |
|
|
203
|
+
| `oneshot`| `lls oneshot "prompt"` | Streams thinking traces, text, and executes agentic tools directly in your terminal. |
|
|
204
|
+
|
|
205
|
+
For example, to boot a model interactively:
|
|
206
|
+
```bash
|
|
207
|
+
$ lls load
|
|
208
|
+
Available Scanned Models:
|
|
209
|
+
1. Qwen3.6-35B-A3B-UD-Q5_K_M (25.2 GB)
|
|
210
|
+
2. gemma-4-26B-A4B-it-Q8_0 (25.0 GB)
|
|
211
|
+
3. DeepSeek-R1-Distill-Qwen-32B-Q5_K_M (21.7 GB)
|
|
212
|
+
|
|
213
|
+
Select a model number to load: 3
|
|
214
|
+
Loading model 'DeepSeek-R1-Distill-Qwen-32B-Q5_K_M'...
|
|
215
|
+
```
|
|
216
|
+
|
|
217
|
+
---
|
|
218
|
+
|
|
219
|
+
## โ๏ธ Configuration & Customization
|
|
220
|
+
|
|
221
|
+
The application runs fully out-of-the-box with no manual configuration. However, you can customize default settings inside `app/config.py` or by using a local `.env` file:
|
|
222
|
+
- `LLAMA_SERVER_BIN`: The absolute path to your `llama-server` binary (automatically resolved on PATH/home).
|
|
223
|
+
- `MODEL_DIRS`: List of local directories to scan for GGUF model files (defaults to `~/.lmstudio/models`).
|
|
224
|
+
- `APP_PORT`: FastAPI web server port (defaults to `8765`).
|
|
225
|
+
|
|
226
|
+
---
|
|
227
|
+
|
|
228
|
+
## ๐ก๏ธ Workspace Sandboxing & Embodiment
|
|
229
|
+
|
|
230
|
+
By default, LlamaStudio restricts agent tools (like reading, writing, and listing files) to the repository directory to prevent accidental path traversals.
|
|
231
|
+
|
|
232
|
+
If you want to grant your model more control (e.g., to act as a system embodiment or write to external directories), you can customize these boundaries by placing a `.env` file in the root directory:
|
|
233
|
+
|
|
234
|
+
* **Broaden the Sandbox Root** (e.g., allow the agent access to your entire home directory):
|
|
235
|
+
```ini
|
|
236
|
+
LLAMASTUDIO_WORKSPACE_ROOT=/home/username
|
|
237
|
+
```
|
|
238
|
+
* **Completely Disable Sandboxing** (gives the agent full filesystem control based on the running process's permissions):
|
|
239
|
+
```ini
|
|
240
|
+
LLAMASTUDIO_DISABLE_SANDBOX=true
|
|
241
|
+
```
|
|
242
|
+
|
|
243
|
+
Restart or reload the backend (`lls reload`) after modifying the `.env` file to apply changes.
|
|
244
|
+
|
|
245
|
+
---
|
|
246
|
+
|
|
247
|
+
## ๐งช Testing Suite
|
|
248
|
+
|
|
249
|
+
LlamaStudio features both standard unit tests and comprehensive GGUF integration tests.
|
|
250
|
+
|
|
251
|
+
### 1. Standard Unit Tests
|
|
252
|
+
Verify local installation and confirm backend routing, regex parsing, and sandboxing safety behaviors by running our mock-based test suite:
|
|
253
|
+
```bash
|
|
254
|
+
python -m unittest discover tests
|
|
255
|
+
```
|
|
256
|
+
|
|
257
|
+
### 2. GGUF Model Integration Tests
|
|
258
|
+
For local environments containing active GPUs and downloaded models, you can run the full multi-model GGUF tool-calling integration suite to verify real-time execution robustness across various chat templates:
|
|
259
|
+
```bash
|
|
260
|
+
# Run GGUF model integration tests locally
|
|
261
|
+
./tests/test_all.sh
|
|
262
|
+
```
|
|
263
|
+
*(These tests are automatically skipped in standard CI/CD environments and default `pytest` runs using `@pytest.mark.skipif` to keep pipeline checks fast.)*
|
|
264
|
+
|
|
265
|
+
---
|
|
266
|
+
|
|
267
|
+
## ๐๏ธ Project Structure
|
|
268
|
+
|
|
269
|
+
```
|
|
270
|
+
LlamaStudio/
|
|
271
|
+
โโโ start.py # Main entrypoint (starts FastAPI + opens browser)
|
|
272
|
+
โโโ pyproject.toml # Package metadata, CLI entrypoint, and dependencies
|
|
273
|
+
โโโ llamastudio.desktop # GNOME/Linux desktop launcher metadata
|
|
274
|
+
โโโ llamastudio.svg # Custom application vector icon
|
|
275
|
+
โโโ app/
|
|
276
|
+
โ โโโ config.py # Settings & dynamic path configurations
|
|
277
|
+
โ โโโ main.py # FastAPI backend endpoints & routing
|
|
278
|
+
โ โโโ chat.py # Conversations registry, templates & chat streaming
|
|
279
|
+
โ โโโ downloader.py # Async background download manager (chunked writes)
|
|
280
|
+
โ โโโ model_manager.py # Scans local paths and Hugging Face Hub
|
|
281
|
+
โ โโโ server_manager.py # llama-server subprocess process lifecycle controller
|
|
282
|
+
โ โโโ logger.py # Centralized logger
|
|
283
|
+
โ โโโ tools.py # Sandboxed local workspace tools for LLM agent use
|
|
284
|
+
โ โโโ templates/
|
|
285
|
+
โ โโโ index.html # Interactive HTMX frontend interface
|
|
286
|
+
โโโ tests/
|
|
287
|
+
โ โโโ test_downloader.py # Comprehensive mocked test cases (100% passing)
|
|
288
|
+
โโโ imgs/
|
|
289
|
+
โโโ chat_interface.png # Screenshot: Main Chat interface
|
|
290
|
+
โโโ model_settings.png # Screenshot: Model explorer & settings
|
|
291
|
+
โโโ discover_models.png# Screenshot: HF Discover & Downloader panel
|
|
292
|
+
```
|
|
293
|
+
|
|
294
|
+
---
|
|
295
|
+
|
|
296
|
+
## ๐ License
|
|
297
|
+
|
|
298
|
+
LLamaStudio is open-source software licensed under the [MIT License](LICENSE).
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
# LLamaStudio - llama.cpp desktop frontend
|