llamastudio 1.0.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,332 @@
1
+ Metadata-Version: 2.4
2
+ Name: llamastudio
3
+ Version: 1.0.0
4
+ Summary: FastAPI backend & desktop launcher for running local models via llama.cpp
5
+ Author: LLamaStudio contributors
6
+ Project-URL: Homepage, https://github.com/gnulnx/LlamaStudio
7
+ Project-URL: Repository, https://github.com/gnulnx/LlamaStudio
8
+ Project-URL: Issues, https://github.com/gnulnx/LlamaStudio/issues
9
+ Keywords: llama.cpp,llm,local-ai,fastapi,gguf
10
+ Classifier: Development Status :: 4 - Beta
11
+ Classifier: Environment :: Console
12
+ Classifier: Framework :: FastAPI
13
+ Classifier: Intended Audience :: End Users/Desktop
14
+ Classifier: Operating System :: MacOS
15
+ Classifier: Operating System :: POSIX :: Linux
16
+ Classifier: Programming Language :: Python :: 3
17
+ Classifier: Programming Language :: Python :: 3.10
18
+ Classifier: Programming Language :: Python :: 3.11
19
+ Classifier: Programming Language :: Python :: 3.12
20
+ Classifier: Programming Language :: Python :: 3.13
21
+ Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
22
+ Requires-Python: >=3.10
23
+ Description-Content-Type: text/markdown
24
+ Requires-Dist: fastapi>=0.115.0
25
+ Requires-Dist: httpx
26
+ Requires-Dist: jinja2
27
+ Requires-Dist: pydantic-settings
28
+ Requires-Dist: rich
29
+ Requires-Dist: rich-click
30
+ Requires-Dist: uvicorn
31
+ Provides-Extra: dev
32
+ Requires-Dist: pytest; extra == "dev"
33
+ Requires-Dist: ruff; extra == "dev"
34
+
35
+ # ๐Ÿฆ™ LLamaStudio
36
+
37
+ A desktop chat interface and local server manager for `llama.cpp`, crafted with **FastAPI** + **HTMX** for ultra-lightweight, zero-framework execution.
38
+
39
+ **LLamaStudio** is a self-contained local workspace that manages model lifecycles, features a smart VRAM estimator, scans local folders, and lets you search and download models directly from the Hugging Face Hub.
40
+
41
+ ---
42
+
43
+ ## ๐Ÿ“ธ Screenshots & Showcase
44
+
45
+ ### 1. Main Chat Dashboard
46
+ A Pop!_OS-harmonized dark interface with streaming, collapsible markdown reasoning (thinking) processes, and real-time agentic tool execution logs.
47
+ ![Main Chat Dashboard](imgs/chat_interface.png)
48
+
49
+ ### 2. GGUF Model Browser & Settings
50
+ A dynamic local model explorer that scans your directories and lets you adjust context length, GPU offload layers, CPU threads, flash attention, and KV cache quantizations on the fly.
51
+ ![GGUF Model Browser](imgs/model_settings.png)
52
+
53
+ ### 3. Hugging Face Discover Hub
54
+ Browse the entire Hugging Face GGUF catalog. Features a **Smart VRAM Offload Estimator** calibrated to your hardware, and a floating background download progress card with live speed (MB/s), ETA, and cancel controls.
55
+ ![Hugging Face Discover Hub](imgs/discover_models.png)
56
+
57
+ ---
58
+
59
+ ## โœจ Key Features
60
+
61
+ - **โšก Zero Node Modules**: Built with HTMX, Tailwind CSS (via CDN), and Vanilla JS. It is incredibly fast, responsive, and has a memory footprint of just a few megabytes.
62
+ - **๐Ÿงญ Hugging Face Discover Tab**: Search the public Hugging Face Hub for GGUF models directly inside the app, view readmes, select quantizations, and download files in the background.
63
+ - **๐Ÿš€ Smart VRAM Estimator**: calulated specifically for your hardware (fits fully on **RTX 5090 32GB VRAM**, partial offload warning, or heavy CPU fallback warning).
64
+ - **๐Ÿ“‚ Automatic Model Scanning**: Scans standard directories (like `~/.lmstudio/models`) automatically on startup or via a one-click rescan button.
65
+ - **๐Ÿช Process Lifecycle Manager**: The underlying `llama-server` process only spins up when you explicitly load a model, releasing all system resources and GPU VRAM instantly when you click "Eject".
66
+ - **๐Ÿ”ง Configurable Workspace Sandboxing**: Supports sandboxed agentic tool use (file read/write, commands, etc.) with real-time logs in the UI. Sandboxing boundaries can be easily broadened or bypassed entirely via a `.env` file or environment variables to grant the model system access.
67
+ - **๐Ÿ–ฅ๏ธ XDG-Compliant Persistence**: Conversations and model settings profiles are stored outside the codebase directory in standard `~/.config/llamastudio/` with automated backward-compatible migrations!
68
+ - **๐Ÿ“ฆ Full Linux & macOS Portability**: Server binaries and model directories are resolved dynamically on startup.
69
+
70
+ ---
71
+
72
+ ## ๐Ÿ› ๏ธ Installation & Setup
73
+
74
+ LLamaStudio is compatible with **Linux** and **macOS** out-of-the-box. Choose your OS and python virtual environment preference below.
75
+
76
+ ### ๐Ÿง 1. Linux Installation
77
+
78
+ #### Prerequisites
79
+ 1. **Python 3.10+** (Recommended: Python 3.13)
80
+ 2. **llama.cpp** built from source (or pre-compiled binary):
81
+ - By default, the app dynamically looks for the `llama-server` binary globally on your system PATH or locally inside your home directory at `~/llama.cpp/build/bin/llama-server`.
82
+
83
+ #### Environment Setup
84
+
85
+ ##### Option A: Install from PyPI
86
+ ```bash
87
+ pip install llamastudio
88
+ ```
89
+
90
+ ##### Option B: Conda / Miniconda from source
91
+ ```bash
92
+ # 1. Clone the repository
93
+ git clone https://github.com/gnulnx/LlamaStudio.git
94
+ cd LlamaStudio
95
+
96
+ # 2. Create and activate a conda environment
97
+ conda create -n llamastudio python=3.13 -y
98
+ conda activate llamastudio
99
+
100
+ # 3. Install LlamaStudio and its dependencies
101
+ pip install -e .
102
+ ```
103
+
104
+ ##### Option C: Python Virtualenv (`venv`) from source
105
+ ```bash
106
+ # 1. Clone the repository
107
+ git clone https://github.com/gnulnx/LlamaStudio.git
108
+ cd LlamaStudio
109
+
110
+ # 2. Create and activate a python venv environment
111
+ python3 -m venv .venv
112
+ source .venv/bin/activate
113
+
114
+ # 3. Install LlamaStudio and its dependencies
115
+ pip install -e .
116
+ ```
117
+
118
+ #### ๐Ÿ–ฅ๏ธ Linux Desktop Launcher Integration (Optional)
119
+ To integrate LLamaStudio directly into your Linux Application launcher menu (e.g., GNOME / Pop!_OS):
120
+ ```bash
121
+ # 1. Copy the desktop file to your local applications directory
122
+ cp llamastudio.desktop ~/.local/share/applications/
123
+
124
+ # 2. Copy the custom SVG icon to your local icons directory
125
+ mkdir -p ~/.local/share/icons/hicolor/128x128/apps/
126
+ cp llamastudio.svg ~/.local/share/icons/hicolor/128x128/apps/
127
+
128
+ # 3. Update your desktop database and icon cache
129
+ update-desktop-database ~/.local/share/applications/
130
+ gtk-update-icon-cache -f -t ~/.local/share/icons
131
+ ```
132
+ *Note: If you are using a virtualenv, edit the executable path inside `~/.local/share/applications/llamastudio.desktop` to point to your specific `.venv/bin/python` interpreter.*
133
+
134
+ ---
135
+
136
+ ### ๐Ÿ 2. macOS Installation
137
+
138
+ #### Prerequisites
139
+ 1. **Python 3.10+**
140
+ 2. **llama.cpp** installed globally via Homebrew (highly recommended for macOS):
141
+ ```bash
142
+ brew install llama.cpp
143
+ ```
144
+ *(This automatically places the `llama-server` binary globally on your system PATH, which LLamaStudio will auto-detect immediately!)*
145
+
146
+ #### Environment Setup
147
+
148
+ ##### Option A: Install from PyPI
149
+ ```bash
150
+ pip install llamastudio
151
+ ```
152
+
153
+ ##### Option B: Conda / Miniconda from source
154
+ ```bash
155
+ # 1. Clone the repository
156
+ git clone https://github.com/gnulnx/LlamaStudio.git
157
+ cd LlamaStudio
158
+
159
+ # 2. Create and activate environment
160
+ conda create -n llamastudio python=3.13 -y
161
+ conda activate llamastudio
162
+
163
+ # 3. Install LlamaStudio and its dependencies
164
+ pip install -e .
165
+ ```
166
+
167
+ ##### Option C: Python Virtualenv (`venv`) from source
168
+ ```bash
169
+ # 1. Clone the repository
170
+ git clone https://github.com/gnulnx/LlamaStudio.git
171
+ cd LlamaStudio
172
+
173
+ # 2. Create and activate venv
174
+ python3 -m venv .venv
175
+ source .venv/bin/activate
176
+
177
+ # 3. Install LlamaStudio and its dependencies
178
+ pip install -e .
179
+ ```
180
+
181
+ ---
182
+
183
+ ### ๐ŸชŸ 3. Windows Installation
184
+ *Note: Native Windows execution is currently **untested**.*
185
+ However, you can run LLamaStudio on Windows seamlessly via **WSL2** (Windows Subsystem for Linux) by following the standard **Linux Installation** guide above.
186
+
187
+ Pull requests extending native Windows support (e.g., resolving `.exe` binaries) are highly welcome!
188
+
189
+ ---
190
+
191
+ ## ๐Ÿš€ Running the Application
192
+
193
+ ### Option A: Via Unified CLI (`lls` - Recommended)
194
+ You can link and install LlamaStudio's CLI utility locally to control the desktop app and server seamlessly:
195
+ ```bash
196
+ # Start the desktop application server and open browser UI
197
+ lls reload
198
+ ```
199
+
200
+ ### Option B: Via App Launcher Command
201
+ After installing from PyPI or source, run:
202
+ ```bash
203
+ llamastudio
204
+ ```
205
+
206
+ ### Option C: Via Direct Startup Script
207
+ When working from a source checkout, activate your virtual environment and run the startup script:
208
+ ```bash
209
+ # Activate your env (Conda)
210
+ conda activate llamastudio
211
+ # OR (Venv)
212
+ source .venv/bin/activate
213
+
214
+ # Start the application
215
+ python start.py
216
+ ```
217
+ This will launch the FastAPI backend on `http://127.0.0.1:8765` and automatically open your default browser to the chat dashboard.
218
+
219
+ ### Via Application Menu (Linux)
220
+ Search for **LLamaStudio** in your desktop search bar (press Super, type "Llama") and click to launch!
221
+
222
+ ---
223
+
224
+ ## ๐Ÿ› ๏ธ Unified Command-Line Interface (`lls`)
225
+
226
+ LlamaStudio features a CLI built using `rich-click` for visual dashboards and operational efficiency.
227
+
228
+ ### CLI Subcommands Reference
229
+
230
+ | Command | Usage | Description |
231
+ | :--- | :--- | :--- |
232
+ | `reload` | `lls reload` | Gracefully starts or restarts the desktop FastAPI application backend. |
233
+ | `status` | `lls status` | Visual dashboard of FastAPI backend status, loaded model parameters, and GPU memory (VRAM). |
234
+ | `ls` | `lls ls` | Prints an elegant table of all GGUF models scanned across local directories. |
235
+ | `load` | `lls load [MODEL]` | Boots the server with a GGUF model. If `MODEL` is omitted, prompts you with an interactive menu. |
236
+ | `eject` | `lls eject` | Gracefully unloads the active model to free GPU and CPU RAM. |
237
+ | `oneshot`| `lls oneshot "prompt"` | Streams thinking traces, text, and executes agentic tools directly in your terminal. |
238
+
239
+ For example, to boot a model interactively:
240
+ ```bash
241
+ $ lls load
242
+ Available Scanned Models:
243
+ 1. Qwen3.6-35B-A3B-UD-Q5_K_M (25.2 GB)
244
+ 2. gemma-4-26B-A4B-it-Q8_0 (25.0 GB)
245
+ 3. DeepSeek-R1-Distill-Qwen-32B-Q5_K_M (21.7 GB)
246
+
247
+ Select a model number to load: 3
248
+ Loading model 'DeepSeek-R1-Distill-Qwen-32B-Q5_K_M'...
249
+ ```
250
+
251
+ ---
252
+
253
+ ## โš™๏ธ Configuration & Customization
254
+
255
+ The application runs fully out-of-the-box with no manual configuration. However, you can customize default settings inside `app/config.py` or by using a local `.env` file:
256
+ - `LLAMA_SERVER_BIN`: The absolute path to your `llama-server` binary (automatically resolved on PATH/home).
257
+ - `MODEL_DIRS`: List of local directories to scan for GGUF model files (defaults to `~/.lmstudio/models`).
258
+ - `APP_PORT`: FastAPI web server port (defaults to `8765`).
259
+
260
+ ---
261
+
262
+ ## ๐Ÿ›ก๏ธ Workspace Sandboxing & Embodiment
263
+
264
+ By default, LlamaStudio restricts agent tools (like reading, writing, and listing files) to the repository directory to prevent accidental path traversals.
265
+
266
+ If you want to grant your model more control (e.g., to act as a system embodiment or write to external directories), you can customize these boundaries by placing a `.env` file in the root directory:
267
+
268
+ * **Broaden the Sandbox Root** (e.g., allow the agent access to your entire home directory):
269
+ ```ini
270
+ LLAMASTUDIO_WORKSPACE_ROOT=/home/username
271
+ ```
272
+ * **Completely Disable Sandboxing** (gives the agent full filesystem control based on the running process's permissions):
273
+ ```ini
274
+ LLAMASTUDIO_DISABLE_SANDBOX=true
275
+ ```
276
+
277
+ Restart or reload the backend (`lls reload`) after modifying the `.env` file to apply changes.
278
+
279
+ ---
280
+
281
+ ## ๐Ÿงช Testing Suite
282
+
283
+ LlamaStudio features both standard unit tests and comprehensive GGUF integration tests.
284
+
285
+ ### 1. Standard Unit Tests
286
+ Verify local installation and confirm backend routing, regex parsing, and sandboxing safety behaviors by running our mock-based test suite:
287
+ ```bash
288
+ python -m unittest discover tests
289
+ ```
290
+
291
+ ### 2. GGUF Model Integration Tests
292
+ For local environments containing active GPUs and downloaded models, you can run the full multi-model GGUF tool-calling integration suite to verify real-time execution robustness across various chat templates:
293
+ ```bash
294
+ # Run GGUF model integration tests locally
295
+ ./tests/test_all.sh
296
+ ```
297
+ *(These tests are automatically skipped in standard CI/CD environments and default `pytest` runs using `@pytest.mark.skipif` to keep pipeline checks fast.)*
298
+
299
+ ---
300
+
301
+ ## ๐Ÿ—๏ธ Project Structure
302
+
303
+ ```
304
+ LlamaStudio/
305
+ โ”œโ”€โ”€ start.py # Main entrypoint (starts FastAPI + opens browser)
306
+ โ”œโ”€โ”€ pyproject.toml # Package metadata, CLI entrypoint, and dependencies
307
+ โ”œโ”€โ”€ llamastudio.desktop # GNOME/Linux desktop launcher metadata
308
+ โ”œโ”€โ”€ llamastudio.svg # Custom application vector icon
309
+ โ”œโ”€โ”€ app/
310
+ โ”‚ โ”œโ”€โ”€ config.py # Settings & dynamic path configurations
311
+ โ”‚ โ”œโ”€โ”€ main.py # FastAPI backend endpoints & routing
312
+ โ”‚ โ”œโ”€โ”€ chat.py # Conversations registry, templates & chat streaming
313
+ โ”‚ โ”œโ”€โ”€ downloader.py # Async background download manager (chunked writes)
314
+ โ”‚ โ”œโ”€โ”€ model_manager.py # Scans local paths and Hugging Face Hub
315
+ โ”‚ โ”œโ”€โ”€ server_manager.py # llama-server subprocess process lifecycle controller
316
+ โ”‚ โ”œโ”€โ”€ logger.py # Centralized logger
317
+ โ”‚ โ”œโ”€โ”€ tools.py # Sandboxed local workspace tools for LLM agent use
318
+ โ”‚ โ””โ”€โ”€ templates/
319
+ โ”‚ โ””โ”€โ”€ index.html # Interactive HTMX frontend interface
320
+ โ”œโ”€โ”€ tests/
321
+ โ”‚ โ””โ”€โ”€ test_downloader.py # Comprehensive mocked test cases (100% passing)
322
+ โ””โ”€โ”€ imgs/
323
+ โ”œโ”€โ”€ chat_interface.png # Screenshot: Main Chat interface
324
+ โ”œโ”€โ”€ model_settings.png # Screenshot: Model explorer & settings
325
+ โ””โ”€โ”€ discover_models.png# Screenshot: HF Discover & Downloader panel
326
+ ```
327
+
328
+ ---
329
+
330
+ ## ๐Ÿ“„ License
331
+
332
+ LLamaStudio is open-source software licensed under the [MIT License](LICENSE).
@@ -0,0 +1,298 @@
1
+ # ๐Ÿฆ™ LLamaStudio
2
+
3
+ A desktop chat interface and local server manager for `llama.cpp`, crafted with **FastAPI** + **HTMX** for ultra-lightweight, zero-framework execution.
4
+
5
+ **LLamaStudio** is a self-contained local workspace that manages model lifecycles, features a smart VRAM estimator, scans local folders, and lets you search and download models directly from the Hugging Face Hub.
6
+
7
+ ---
8
+
9
+ ## ๐Ÿ“ธ Screenshots & Showcase
10
+
11
+ ### 1. Main Chat Dashboard
12
+ A Pop!_OS-harmonized dark interface with streaming, collapsible markdown reasoning (thinking) processes, and real-time agentic tool execution logs.
13
+ ![Main Chat Dashboard](imgs/chat_interface.png)
14
+
15
+ ### 2. GGUF Model Browser & Settings
16
+ A dynamic local model explorer that scans your directories and lets you adjust context length, GPU offload layers, CPU threads, flash attention, and KV cache quantizations on the fly.
17
+ ![GGUF Model Browser](imgs/model_settings.png)
18
+
19
+ ### 3. Hugging Face Discover Hub
20
+ Browse the entire Hugging Face GGUF catalog. Features a **Smart VRAM Offload Estimator** calibrated to your hardware, and a floating background download progress card with live speed (MB/s), ETA, and cancel controls.
21
+ ![Hugging Face Discover Hub](imgs/discover_models.png)
22
+
23
+ ---
24
+
25
+ ## โœจ Key Features
26
+
27
+ - **โšก Zero Node Modules**: Built with HTMX, Tailwind CSS (via CDN), and Vanilla JS. It is incredibly fast, responsive, and has a memory footprint of just a few megabytes.
28
+ - **๐Ÿงญ Hugging Face Discover Tab**: Search the public Hugging Face Hub for GGUF models directly inside the app, view readmes, select quantizations, and download files in the background.
29
+ - **๐Ÿš€ Smart VRAM Estimator**: calulated specifically for your hardware (fits fully on **RTX 5090 32GB VRAM**, partial offload warning, or heavy CPU fallback warning).
30
+ - **๐Ÿ“‚ Automatic Model Scanning**: Scans standard directories (like `~/.lmstudio/models`) automatically on startup or via a one-click rescan button.
31
+ - **๐Ÿช Process Lifecycle Manager**: The underlying `llama-server` process only spins up when you explicitly load a model, releasing all system resources and GPU VRAM instantly when you click "Eject".
32
+ - **๐Ÿ”ง Configurable Workspace Sandboxing**: Supports sandboxed agentic tool use (file read/write, commands, etc.) with real-time logs in the UI. Sandboxing boundaries can be easily broadened or bypassed entirely via a `.env` file or environment variables to grant the model system access.
33
+ - **๐Ÿ–ฅ๏ธ XDG-Compliant Persistence**: Conversations and model settings profiles are stored outside the codebase directory in standard `~/.config/llamastudio/` with automated backward-compatible migrations!
34
+ - **๐Ÿ“ฆ Full Linux & macOS Portability**: Server binaries and model directories are resolved dynamically on startup.
35
+
36
+ ---
37
+
38
+ ## ๐Ÿ› ๏ธ Installation & Setup
39
+
40
+ LLamaStudio is compatible with **Linux** and **macOS** out-of-the-box. Choose your OS and python virtual environment preference below.
41
+
42
+ ### ๐Ÿง 1. Linux Installation
43
+
44
+ #### Prerequisites
45
+ 1. **Python 3.10+** (Recommended: Python 3.13)
46
+ 2. **llama.cpp** built from source (or pre-compiled binary):
47
+ - By default, the app dynamically looks for the `llama-server` binary globally on your system PATH or locally inside your home directory at `~/llama.cpp/build/bin/llama-server`.
48
+
49
+ #### Environment Setup
50
+
51
+ ##### Option A: Install from PyPI
52
+ ```bash
53
+ pip install llamastudio
54
+ ```
55
+
56
+ ##### Option B: Conda / Miniconda from source
57
+ ```bash
58
+ # 1. Clone the repository
59
+ git clone https://github.com/gnulnx/LlamaStudio.git
60
+ cd LlamaStudio
61
+
62
+ # 2. Create and activate a conda environment
63
+ conda create -n llamastudio python=3.13 -y
64
+ conda activate llamastudio
65
+
66
+ # 3. Install LlamaStudio and its dependencies
67
+ pip install -e .
68
+ ```
69
+
70
+ ##### Option C: Python Virtualenv (`venv`) from source
71
+ ```bash
72
+ # 1. Clone the repository
73
+ git clone https://github.com/gnulnx/LlamaStudio.git
74
+ cd LlamaStudio
75
+
76
+ # 2. Create and activate a python venv environment
77
+ python3 -m venv .venv
78
+ source .venv/bin/activate
79
+
80
+ # 3. Install LlamaStudio and its dependencies
81
+ pip install -e .
82
+ ```
83
+
84
+ #### ๐Ÿ–ฅ๏ธ Linux Desktop Launcher Integration (Optional)
85
+ To integrate LLamaStudio directly into your Linux Application launcher menu (e.g., GNOME / Pop!_OS):
86
+ ```bash
87
+ # 1. Copy the desktop file to your local applications directory
88
+ cp llamastudio.desktop ~/.local/share/applications/
89
+
90
+ # 2. Copy the custom SVG icon to your local icons directory
91
+ mkdir -p ~/.local/share/icons/hicolor/128x128/apps/
92
+ cp llamastudio.svg ~/.local/share/icons/hicolor/128x128/apps/
93
+
94
+ # 3. Update your desktop database and icon cache
95
+ update-desktop-database ~/.local/share/applications/
96
+ gtk-update-icon-cache -f -t ~/.local/share/icons
97
+ ```
98
+ *Note: If you are using a virtualenv, edit the executable path inside `~/.local/share/applications/llamastudio.desktop` to point to your specific `.venv/bin/python` interpreter.*
99
+
100
+ ---
101
+
102
+ ### ๐Ÿ 2. macOS Installation
103
+
104
+ #### Prerequisites
105
+ 1. **Python 3.10+**
106
+ 2. **llama.cpp** installed globally via Homebrew (highly recommended for macOS):
107
+ ```bash
108
+ brew install llama.cpp
109
+ ```
110
+ *(This automatically places the `llama-server` binary globally on your system PATH, which LLamaStudio will auto-detect immediately!)*
111
+
112
+ #### Environment Setup
113
+
114
+ ##### Option A: Install from PyPI
115
+ ```bash
116
+ pip install llamastudio
117
+ ```
118
+
119
+ ##### Option B: Conda / Miniconda from source
120
+ ```bash
121
+ # 1. Clone the repository
122
+ git clone https://github.com/gnulnx/LlamaStudio.git
123
+ cd LlamaStudio
124
+
125
+ # 2. Create and activate environment
126
+ conda create -n llamastudio python=3.13 -y
127
+ conda activate llamastudio
128
+
129
+ # 3. Install LlamaStudio and its dependencies
130
+ pip install -e .
131
+ ```
132
+
133
+ ##### Option C: Python Virtualenv (`venv`) from source
134
+ ```bash
135
+ # 1. Clone the repository
136
+ git clone https://github.com/gnulnx/LlamaStudio.git
137
+ cd LlamaStudio
138
+
139
+ # 2. Create and activate venv
140
+ python3 -m venv .venv
141
+ source .venv/bin/activate
142
+
143
+ # 3. Install LlamaStudio and its dependencies
144
+ pip install -e .
145
+ ```
146
+
147
+ ---
148
+
149
+ ### ๐ŸชŸ 3. Windows Installation
150
+ *Note: Native Windows execution is currently **untested**.*
151
+ However, you can run LLamaStudio on Windows seamlessly via **WSL2** (Windows Subsystem for Linux) by following the standard **Linux Installation** guide above.
152
+
153
+ Pull requests extending native Windows support (e.g., resolving `.exe` binaries) are highly welcome!
154
+
155
+ ---
156
+
157
+ ## ๐Ÿš€ Running the Application
158
+
159
+ ### Option A: Via Unified CLI (`lls` - Recommended)
160
+ You can link and install LlamaStudio's CLI utility locally to control the desktop app and server seamlessly:
161
+ ```bash
162
+ # Start the desktop application server and open browser UI
163
+ lls reload
164
+ ```
165
+
166
+ ### Option B: Via App Launcher Command
167
+ After installing from PyPI or source, run:
168
+ ```bash
169
+ llamastudio
170
+ ```
171
+
172
+ ### Option C: Via Direct Startup Script
173
+ When working from a source checkout, activate your virtual environment and run the startup script:
174
+ ```bash
175
+ # Activate your env (Conda)
176
+ conda activate llamastudio
177
+ # OR (Venv)
178
+ source .venv/bin/activate
179
+
180
+ # Start the application
181
+ python start.py
182
+ ```
183
+ This will launch the FastAPI backend on `http://127.0.0.1:8765` and automatically open your default browser to the chat dashboard.
184
+
185
+ ### Via Application Menu (Linux)
186
+ Search for **LLamaStudio** in your desktop search bar (press Super, type "Llama") and click to launch!
187
+
188
+ ---
189
+
190
+ ## ๐Ÿ› ๏ธ Unified Command-Line Interface (`lls`)
191
+
192
+ LlamaStudio features a CLI built using `rich-click` for visual dashboards and operational efficiency.
193
+
194
+ ### CLI Subcommands Reference
195
+
196
+ | Command | Usage | Description |
197
+ | :--- | :--- | :--- |
198
+ | `reload` | `lls reload` | Gracefully starts or restarts the desktop FastAPI application backend. |
199
+ | `status` | `lls status` | Visual dashboard of FastAPI backend status, loaded model parameters, and GPU memory (VRAM). |
200
+ | `ls` | `lls ls` | Prints an elegant table of all GGUF models scanned across local directories. |
201
+ | `load` | `lls load [MODEL]` | Boots the server with a GGUF model. If `MODEL` is omitted, prompts you with an interactive menu. |
202
+ | `eject` | `lls eject` | Gracefully unloads the active model to free GPU and CPU RAM. |
203
+ | `oneshot`| `lls oneshot "prompt"` | Streams thinking traces, text, and executes agentic tools directly in your terminal. |
204
+
205
+ For example, to boot a model interactively:
206
+ ```bash
207
+ $ lls load
208
+ Available Scanned Models:
209
+ 1. Qwen3.6-35B-A3B-UD-Q5_K_M (25.2 GB)
210
+ 2. gemma-4-26B-A4B-it-Q8_0 (25.0 GB)
211
+ 3. DeepSeek-R1-Distill-Qwen-32B-Q5_K_M (21.7 GB)
212
+
213
+ Select a model number to load: 3
214
+ Loading model 'DeepSeek-R1-Distill-Qwen-32B-Q5_K_M'...
215
+ ```
216
+
217
+ ---
218
+
219
+ ## โš™๏ธ Configuration & Customization
220
+
221
+ The application runs fully out-of-the-box with no manual configuration. However, you can customize default settings inside `app/config.py` or by using a local `.env` file:
222
+ - `LLAMA_SERVER_BIN`: The absolute path to your `llama-server` binary (automatically resolved on PATH/home).
223
+ - `MODEL_DIRS`: List of local directories to scan for GGUF model files (defaults to `~/.lmstudio/models`).
224
+ - `APP_PORT`: FastAPI web server port (defaults to `8765`).
225
+
226
+ ---
227
+
228
+ ## ๐Ÿ›ก๏ธ Workspace Sandboxing & Embodiment
229
+
230
+ By default, LlamaStudio restricts agent tools (like reading, writing, and listing files) to the repository directory to prevent accidental path traversals.
231
+
232
+ If you want to grant your model more control (e.g., to act as a system embodiment or write to external directories), you can customize these boundaries by placing a `.env` file in the root directory:
233
+
234
+ * **Broaden the Sandbox Root** (e.g., allow the agent access to your entire home directory):
235
+ ```ini
236
+ LLAMASTUDIO_WORKSPACE_ROOT=/home/username
237
+ ```
238
+ * **Completely Disable Sandboxing** (gives the agent full filesystem control based on the running process's permissions):
239
+ ```ini
240
+ LLAMASTUDIO_DISABLE_SANDBOX=true
241
+ ```
242
+
243
+ Restart or reload the backend (`lls reload`) after modifying the `.env` file to apply changes.
244
+
245
+ ---
246
+
247
+ ## ๐Ÿงช Testing Suite
248
+
249
+ LlamaStudio features both standard unit tests and comprehensive GGUF integration tests.
250
+
251
+ ### 1. Standard Unit Tests
252
+ Verify local installation and confirm backend routing, regex parsing, and sandboxing safety behaviors by running our mock-based test suite:
253
+ ```bash
254
+ python -m unittest discover tests
255
+ ```
256
+
257
+ ### 2. GGUF Model Integration Tests
258
+ For local environments containing active GPUs and downloaded models, you can run the full multi-model GGUF tool-calling integration suite to verify real-time execution robustness across various chat templates:
259
+ ```bash
260
+ # Run GGUF model integration tests locally
261
+ ./tests/test_all.sh
262
+ ```
263
+ *(These tests are automatically skipped in standard CI/CD environments and default `pytest` runs using `@pytest.mark.skipif` to keep pipeline checks fast.)*
264
+
265
+ ---
266
+
267
+ ## ๐Ÿ—๏ธ Project Structure
268
+
269
+ ```
270
+ LlamaStudio/
271
+ โ”œโ”€โ”€ start.py # Main entrypoint (starts FastAPI + opens browser)
272
+ โ”œโ”€โ”€ pyproject.toml # Package metadata, CLI entrypoint, and dependencies
273
+ โ”œโ”€โ”€ llamastudio.desktop # GNOME/Linux desktop launcher metadata
274
+ โ”œโ”€โ”€ llamastudio.svg # Custom application vector icon
275
+ โ”œโ”€โ”€ app/
276
+ โ”‚ โ”œโ”€โ”€ config.py # Settings & dynamic path configurations
277
+ โ”‚ โ”œโ”€โ”€ main.py # FastAPI backend endpoints & routing
278
+ โ”‚ โ”œโ”€โ”€ chat.py # Conversations registry, templates & chat streaming
279
+ โ”‚ โ”œโ”€โ”€ downloader.py # Async background download manager (chunked writes)
280
+ โ”‚ โ”œโ”€โ”€ model_manager.py # Scans local paths and Hugging Face Hub
281
+ โ”‚ โ”œโ”€โ”€ server_manager.py # llama-server subprocess process lifecycle controller
282
+ โ”‚ โ”œโ”€โ”€ logger.py # Centralized logger
283
+ โ”‚ โ”œโ”€โ”€ tools.py # Sandboxed local workspace tools for LLM agent use
284
+ โ”‚ โ””โ”€โ”€ templates/
285
+ โ”‚ โ””โ”€โ”€ index.html # Interactive HTMX frontend interface
286
+ โ”œโ”€โ”€ tests/
287
+ โ”‚ โ””โ”€โ”€ test_downloader.py # Comprehensive mocked test cases (100% passing)
288
+ โ””โ”€โ”€ imgs/
289
+ โ”œโ”€โ”€ chat_interface.png # Screenshot: Main Chat interface
290
+ โ”œโ”€โ”€ model_settings.png # Screenshot: Model explorer & settings
291
+ โ””โ”€โ”€ discover_models.png# Screenshot: HF Discover & Downloader panel
292
+ ```
293
+
294
+ ---
295
+
296
+ ## ๐Ÿ“„ License
297
+
298
+ LLamaStudio is open-source software licensed under the [MIT License](LICENSE).
@@ -0,0 +1 @@
1
+ # LLamaStudio - llama.cpp desktop frontend