lcode-cli 0.15.0__tar.gz → 0.16.0__tar.gz

{lcode_cli-0.15.0 → lcode_cli-0.16.0}/CHANGELOG.md

@@ -6,6 +6,19 @@ All notable changes to lcode are documented here. The format follows
 
 ## [Unreleased]
 
+## [0.16.0] - 2026-10-05
+
+### Added
+
+- Other model servers: LM Studio, llama.cpp's llama-server, vLLM, MLX's server, or any other
+  server with an OpenAI-compatible API, instead of Ollama. `lcode config set backend lmstudio` (or
+  `llama.cpp`, `vllm`, `mlx`, `openai` with `base_url`); a key can come from `LCODE_API_KEY`. Tool
+  calls, streaming, reasoning (on and off), images and structured answers work as with Ollama.
+  lcode reads the context window from the server, matches the model setting to the models it
+  serves, and moves thinking that a server sends as part of the answer out of it. `lcode doctor`,
+  `lcode models`, `lcode bench` and `lcode index` work with every backend. See
+  [Other model servers](https://nasser1941.github.io/lcode/servers/), with setup for each.
+
 ## [0.15.0] - 2026-10-05
 
 ### Added
@@ -385,7 +398,8 @@ First public release.
 - `AGENTS.md` project instructions and `/init` to generate them.
 - One-line installer for Linux and macOS.
 
-[Unreleased]: https://github.com/nasser1941/lcode/compare/v0.15.0...HEAD
+[Unreleased]: https://github.com/nasser1941/lcode/compare/v0.16.0...HEAD
+[0.16.0]: https://github.com/nasser1941/lcode/compare/v0.15.0...v0.16.0
 [0.15.0]: https://github.com/nasser1941/lcode/compare/v0.14.0...v0.15.0
 [0.14.0]: https://github.com/nasser1941/lcode/compare/v0.13.0...v0.14.0
 [0.13.0]: https://github.com/nasser1941/lcode/compare/v0.12.0...v0.13.0

{lcode_cli-0.15.0 → lcode_cli-0.16.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: lcode-cli
-Version: 0.15.0
+Version: 0.16.0
 Summary: A local-first terminal coding agent powered by open-weight models on your own GPU or Mac, via Ollama.
 Project-URL: Homepage, https://nasser1941.github.io/lcode/
 Project-URL: Documentation, https://nasser1941.github.io/lcode/
@@ -102,7 +102,7 @@ through [Ollama](https://ollama.com), and when it needs current information it c
   session as short notes, saved by the model, by you (`/remember`) or by a quick check at the end
   of a session. Not whole conversations, and never secrets.
 - **Bring your own model.** Eight curated open-weight models, all tested end to end, or any Ollama model
-  with tool calling.
+  with tool calling. Or use LM Studio, llama.cpp, vLLM or MLX instead of Ollama.
 - **MCP servers, ready to go.** Connect Jira and Confluence, GitHub, AWS, Google Drive, Grafana,
   Google Cloud, Sentry, Linear, Notion, Postgres, Kubernetes, Metabase, Encord, Valohai and more with one command
   (`lcode mcp add atlassian`), image generation with ComfyUI, or any other MCP server. Browser sign-in (OAuth) is built in.
@@ -208,8 +208,8 @@ fills up. Details: [How it works](https://nasser1941.github.io/lcode/how-it-work
 
 See the pinned [Roadmap issue](https://github.com/nasser1941/lcode/issues/10). Next up:
 
-- [More local backends: LM Studio, llama.cpp, vLLM and MLX](https://github.com/nasser1941/lcode/issues/47)
 - [Git workflow: /commit, /review, /pr and parallel worktrees](https://github.com/nasser1941/lcode/issues/50)
+- [Automation: JSON output, a Python API and a GitHub Action for self-hosted runners](https://github.com/nasser1941/lcode/issues/48)
 - More models in the catalog, and test reports from your hardware ([report one](https://github.com/nasser1941/lcode/issues/new?template=model_request.yml))
 
 ## Contributing

{lcode_cli-0.15.0 → lcode_cli-0.16.0}/README.md

@@ -66,7 +66,7 @@ through [Ollama](https://ollama.com), and when it needs current information it c
   session as short notes, saved by the model, by you (`/remember`) or by a quick check at the end
   of a session. Not whole conversations, and never secrets.
 - **Bring your own model.** Eight curated open-weight models, all tested end to end, or any Ollama model
-  with tool calling.
+  with tool calling. Or use LM Studio, llama.cpp, vLLM or MLX instead of Ollama.
 - **MCP servers, ready to go.** Connect Jira and Confluence, GitHub, AWS, Google Drive, Grafana,
   Google Cloud, Sentry, Linear, Notion, Postgres, Kubernetes, Metabase, Encord, Valohai and more with one command
   (`lcode mcp add atlassian`), image generation with ComfyUI, or any other MCP server. Browser sign-in (OAuth) is built in.
@@ -172,8 +172,8 @@ fills up. Details: [How it works](https://nasser1941.github.io/lcode/how-it-work
 
 See the pinned [Roadmap issue](https://github.com/nasser1941/lcode/issues/10). Next up:
 
-- [More local backends: LM Studio, llama.cpp, vLLM and MLX](https://github.com/nasser1941/lcode/issues/47)
 - [Git workflow: /commit, /review, /pr and parallel worktrees](https://github.com/nasser1941/lcode/issues/50)
+- [Automation: JSON output, a Python API and a GitHub Action for self-hosted runners](https://github.com/nasser1941/lcode/issues/48)
 - More models in the catalog, and test reports from your hardware ([report one](https://github.com/nasser1941/lcode/issues/new?template=model_request.yml))
 
 ## Contributing

{lcode_cli-0.15.0 → lcode_cli-0.16.0}/docs/bench.md

@@ -12,6 +12,10 @@ lcode bench --json results.json       # also save machine-readable results
 lcode bench --markdown                # also print a table to paste into a test report
 ```
 
+`lcode bench` also runs against [other model servers](servers.md), such as llama-server or LM
+Studio: `LCODE_BACKEND=llama.cpp lcode bench`. The server keeps the model loaded and doesn't report
+its memory use, so those columns stay empty.
+
 ```text
 lcode bench · NVIDIA GeForce RTX 4080 Laptop GPU (12 GB VRAM), 31 GB RAM · Ollama 0.32.15
 
@@ -158,6 +162,7 @@ existing fields keep their meaning until `schema` changes.
   "schema": 1,
   "lcode": "0.3.1",
   "ollama": "0.32.15",
+  "server": "Ollama 0.32.15",
   "date": "2026-09-30T19:09:09+02:00",
   "hardware": {
     "os": "linux", "cpu": "13th Gen Intel(R) Core(TM) i9-13980HX", "ram_gib": 30.96,
@@ -197,6 +202,7 @@ existing fields keep their meaning until `schema` changes.
 
 | Field | Meaning |
 |---|---|
+| `ollama`, `server` | The Ollama version, or `null` with [another model server](servers.md); `server` names the server either way |
 | `runs[].model` | The name you gave (catalog key or Ollama tag); `ollama_model` is the model that ran |
 | `runs[].context`, `num_batch`, `think` | The settings used. lcode lowers the batch size or context if the GPU runs out of memory, and reports what it ended up using |
 | `runs[].memory_gb`, `gpu_percent` | From Ollama's `/api/ps`; `null` if unavailable |

{lcode_cli-0.15.0 → lcode_cli-0.16.0}/docs/configuration.md

@@ -15,11 +15,13 @@ lcode config path                      # print the file location
 
 | Setting | Default | Meaning |
 |---|---|---|
-| `model` | `qwen3.6-35b` | Catalog key (see `lcode models`) or any installed Ollama model tag |
+| `model` | `qwen3.6-35b` | Catalog key (see `lcode models`) or any installed Ollama model tag; with another [backend](servers.md), a model the server serves |
 | `context` | largest that fits | Context window in tokens; accepts `128k`, `1m` |
 | `num_batch` | 512 | Prompt batch size. Larger reads prompts faster but needs more GPU memory |
 | `keep_alive` | `30m` | How long Ollama keeps the model in memory after the last request |
 | `ollama_host` | `http://localhost:11434` | Ollama server URL |
+| `backend` | `ollama` | The model server: `ollama`, `lmstudio`, `llama.cpp`, `vllm`, `mlx` or `openai` (any OpenAI-compatible API); see [Other model servers](servers.md) |
+| `base_url` | the backend's | The server's OpenAI-compatible address, e.g. `http://localhost:1234/v1` |
 | `permission_mode` | `ask` | `ask`, `plan`, `auto-edit` or `yolo` |
 | `think` | `true` | Let the model reason before answering |
 | `web` | `on` | Web search and page fetching: `on`, `ask` (before each search/website) or `off` |
@@ -65,6 +67,9 @@ Environment variables override the file, which is useful for one-off runs and CI
 | `LCODE_NUM_BATCH` | `num_batch` |
 | `LCODE_KEEP_ALIVE` | `keep_alive` |
 | `OLLAMA_HOST` | `ollama_host` (same variable the Ollama CLI uses) |
+| `LCODE_BACKEND` | `backend` |
+| `LCODE_BASE_URL` | `base_url` |
+| `LCODE_API_KEY` or `OPENAI_API_KEY` | API key for an [OpenAI-compatible server](servers.md) that needs one (never stored in the config file) |
 | `LCODE_WEB` | `web` |
 | `LCODE_SANDBOX` | `sandbox` |
 | `LCODE_MEMORY` | `memory` |

{lcode_cli-0.15.0 → lcode_cli-0.16.0}/docs/installation.md

@@ -151,6 +151,11 @@ lcode config set ollama_host http://gpu-server:11434
 Your prompts and the code the model reads are then sent to that server. Ollama has no
 authentication; only expose it on a network you trust (or through an SSH tunnel).
 
+### LM Studio, llama.cpp, vLLM or MLX instead of Ollama
+
+lcode also works with any server that has an OpenAI-compatible API: `lcode config set backend
+lmstudio` (or `llama.cpp`, `vllm`, `mlx`). See [Other model servers](servers.md).
+
 ### Windows
 
 Use [WSL2](https://learn.microsoft.com/windows/wsl/install) with Ubuntu and follow the Ubuntu

{lcode_cli-0.15.0 → lcode_cli-0.16.0}/docs/quickstart.md

@@ -12,7 +12,7 @@ lcode
 
 ```text
 ╭──────────────────────────────────────────────╮
-│ lcode v0.15.0 — local coding agent           │
+│ lcode v0.16.0 — local coding agent           │
 │                                              │
 │ model    lcode-qwen3.6-35b                   │
 │ context  256K tokens                         │

lcode_cli-0.16.0/docs/servers.md

@@ -0,0 +1,136 @@
+# Other model servers
+
+lcode is built around [Ollama](https://ollama.com), but it also works with any server that has an
+OpenAI-compatible API: [LM Studio](https://lmstudio.ai),
+[llama.cpp](https://github.com/ggml-org/llama.cpp)'s `llama-server`, [vLLM](https://docs.vllm.ai)
+and the [MLX](https://github.com/ml-explore/mlx-lm) server. Tool calls, streaming answers, reasoning
+and all of lcode's tools work the same.
+
+```bash
+lcode config set backend lmstudio      # or llama.cpp, vllm, mlx
+lcode doctor                           # checks the server, the model and its context window
+lcode
+```
+
+| `backend` | Server | Default address |
+|---|---|---|
+| `ollama` (default) | Ollama | `ollama_host`, `http://localhost:11434` |
+| `lmstudio` | LM Studio | `http://localhost:1234/v1` |
+| `llama.cpp` | llama-server | `http://localhost:8080/v1` |
+| `vllm` | vLLM | `http://localhost:8000/v1` |
+| `mlx` | `mlx_lm.server` | `http://localhost:8080/v1` |
+| `openai` | Any other OpenAI-compatible API | Set `base_url` |
+
+- **Another address:** `lcode config set base_url http://gpu-box:8000/v1`.
+- **A server that needs a key:** put it in `LCODE_API_KEY` (or `OPENAI_API_KEY`). lcode sends it as a
+  bearer token and never writes it to the config file.
+- **For one run:** `LCODE_BACKEND=llama.cpp lcode`.
+
+Your prompts and the code the model reads go to that server, so use one you run yourself or trust.
+
+## Choosing the model
+
+The `model` setting names a model the server serves. `lcode models` lists them:
+
+```text
+$ lcode models
+Models served by LM Studio at http://localhost:1234/v1
+  google/gemma-4-12b
+  qwen/qwen3.6-35b-a3b  current
+```
+
+Part of the name is enough when only one model matches (`lcode -m gemma`). A server with a single
+model, like llama-server, uses it whatever the setting says. `lcode setup` only installs models
+into Ollama: download models in the server itself.
+
+## The context window
+
+With Ollama, lcode chooses the context window and loads the model with it. Other servers load the
+model themselves, with their own window, so lcode reads its size from the server:
+
+- llama-server: the `-c` option;
+- vLLM: `--max-model-len`;
+- LM Studio: the context length the model was loaded with.
+
+lcode then summarizes the conversation before it outgrows that window. The MLX server and Ollama's
+own OpenAI endpoint don't report a size, so lcode assumes 32K; set the real size with
+`lcode config set context 64k`. A smaller value than the server's is fine (`/ctx 16k`): lcode just
+summarizes sooner.
+
+lcode's instructions and tool definitions take about 6K tokens, so give the model at least 32K.
+
+## Setting up each server
+
+### LM Studio
+
+1. Download a model that supports tool use, such as Qwen3.6 35B A3B or Gemma 4.
+2. Load it with a context length of 32K or more. LM Studio's default of 4K is too small for a
+   coding agent.
+3. Start the server: in the Developer tab, or with `lms server start`.
+4. Point lcode at it:
+
+   ```bash
+   lcode config set backend lmstudio
+   lcode models                           # the ids LM Studio serves
+   lcode config set model qwen/qwen3.6-35b-a3b
+   ```
+
+On a Mac, LM Studio's MLX models are the fastest route. Load the model before you start lcode: a
+model that LM Studio loads on request gets LM Studio's default context length.
+
+### llama.cpp (llama-server)
+
+```bash
+llama-server -m Qwen3.6-35B-A3B-Q4_K_M.gguf --alias qwen3.6-35b --jinja -c 65536 -ngl 99 --n-cpu-moe 20
+lcode config set backend llama.cpp
+```
+
+- `--jinja` is required for tool calls.
+- `-c` sets the context window, which lcode reads.
+- `--alias` gives the model a short name; otherwise its name is the file's path.
+- On a GPU too small for the whole model, `--n-cpu-moe N` (or `--cpu-moe`) keeps a mixture-of-experts
+  model's expert layers in RAM.
+
+With Docker and an NVIDIA GPU:
+
+```bash
+docker run -d --gpus all -p 127.0.0.1:8080:8080 -v ~/models:/models ghcr.io/ggml-org/llama.cpp:server-cuda \
+  -m /models/Qwen3.6-35B-A3B-Q4_K_M.gguf --alias qwen3.6-35b --jinja -c 65536 -ngl 99 --n-cpu-moe 20 \
+  --host 0.0.0.0
+```
+
+### vLLM
+
+```bash
+vllm serve Qwen/Qwen3-32B --max-model-len 32768 \
+  --enable-auto-tool-choice --tool-call-parser hermes --reasoning-parser qwen3
+lcode config set backend vllm
+```
+
+vLLM needs `--enable-auto-tool-choice` and the tool-call parser for the model's family (see vLLM's
+[tool calling](https://docs.vllm.ai/en/latest/features/tool_calling.html) page). A reasoning parser
+keeps the model's thinking out of its answers.
+
+### MLX (Apple silicon)
+
+```bash
+pip install mlx-lm
+mlx_lm.server --model mlx-community/Qwen3-30B-A3B-4bit
+lcode config set backend mlx
+lcode config set context 32k
+```
+
+## What's different from Ollama
+
+| | Ollama | Other servers |
+|---|---|---|
+| Context window | lcode chooses it, and falls back to a smaller one when the GPU runs out of memory | The server's; lcode reads it |
+| Reasoning on and off (`/think`, `--no-think`) | Yes | Yes, for models whose chat template has a switch (Qwen and others) on llama-server and vLLM |
+| Thinking shown separately | Yes | Yes. When a server sends the thinking as part of the answer, lcode moves it out |
+| Images | The model's own vision, or `vision_model` | Set `vision_model` to a model the server serves that can see images |
+| [Code search](search.md#semantic-code-search) | Any installed embedding model | Needs the server to serve an embedding model too, as LM Studio can |
+| Freeing the GPU for MCP tools that need it | Yes | No: the server decides what stays loaded |
+| `lcode bench` memory columns | Measured | Empty: the server doesn't report them |
+
+`lcode bench` works the same way with any backend, so you can compare the same model on two
+servers: `LCODE_BACKEND=llama.cpp lcode bench`.

{lcode_cli-0.15.0 → lcode_cli-0.16.0}/mkdocs.yml

@@ -98,6 +98,7 @@ nav:
       - Code intelligence: lsp.md
       - Repository map and code search: search.md
       - Models & context windows: models.md
+      - Other model servers: servers.md
       - Benchmarking models: bench.md
       - MCP servers: mcp.md
       - Sandbox: sandbox.md

{lcode_cli-0.15.0 → lcode_cli-0.16.0}/pyproject.toml

@@ -83,6 +83,7 @@ ignore = [
 "src/lcode/repomap.py" = ["E501"]
 "src/lcode/codesearch.py" = ["E501"]
 "src/lcode/bench_backend.py" = ["E501"]
+"src/lcode/backends.py" = ["E501"]
 
 [tool.pytest.ini_options]
 testpaths = ["tests"]

{lcode_cli-0.15.0 → lcode_cli-0.16.0}/src/lcode/__init__.py

@@ -1,3 +1,3 @@
 """lcode — a local-first terminal coding agent powered by open-weight models via Ollama."""
 
-__version__ = "0.15.0"
+__version__ = "0.16.0"

{lcode_cli-0.15.0 → lcode_cli-0.16.0}/src/lcode/agent.py

@@ -570,7 +570,8 @@ class Agent:
                 self.console.print(f"[dim]{self.settings.model} does not support reasoning; continuing without.[/]")
                 yield from self.chat(messages, tools, False)
                 return
-            if any(marker in error for marker in GPU_MEMORY_ERRORS):
+            # Only Ollama loads the model with lcode's settings; other servers manage their own memory.
+            if getattr(self.ollama, "kind", "ollama") == "ollama" and any(m in error for m in GPU_MEMORY_ERRORS):
                 batch = self.settings.num_batch
                 if not started and batch and batch > SAFE_NUM_BATCH:
                     # Larger batches read prompts faster but need extra VRAM that isn't always free.