opencode-localhost 0.7.0 → 0.7.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +222 -76
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -1,28 +1,78 @@
|
|
|
1
1
|
# opencode-localhost
|
|
2
2
|
|
|
3
|
-
|
|
4
|
-
|
|
5
|
-
|
|
3
|
+
[](https://www.npmjs.com/package/opencode-localhost)
|
|
4
|
+
[](./LICENSE)
|
|
5
|
+
|
|
6
|
+
Run local models in [opencode](https://opencode.ai).
|
|
7
|
+
|
|
8
|
+
Your GGUF files appear in the model picker like any other provider.
|
|
9
|
+
`llama-server` is started, supervised and stopped for you. A panel keeps GPU,
|
|
10
|
+
VRAM and CPU on screen while you work.
|
|
6
11
|
|
|
7
12
|
```
|
|
8
|
-
HARDWARE
|
|
9
|
-
GPU0 14.
|
|
10
|
-
GPU1 13.
|
|
11
|
-
CPU
|
|
13
|
+
HARDWARE memory compute │ llama.cpp │ lmstudio-community/
|
|
14
|
+
GPU0 14.3/15.9G 87% │ ● 127.0.0.1:9337 │ Qwen3.6-35B-A3B-GGUF
|
|
15
|
+
GPU1 13.7/15.9G 82% │ [stop] │ 245760 ctx · q8_0 KV
|
|
16
|
+
CPU 10.8/31.2G 31% │ │ [change]
|
|
12
17
|
```
|
|
13
18
|
|
|
14
|
-
|
|
19
|
+
It is a plugin. It does not fork or patch opencode.
|
|
20
|
+
|
|
21
|
+
## Contents
|
|
22
|
+
|
|
23
|
+
- [What it does](#what-it-does)
|
|
24
|
+
- [Requirements](#requirements)
|
|
25
|
+
- [Installation](#installation)
|
|
26
|
+
- [First run](#first-run)
|
|
27
|
+
- [Configuration](#configuration)
|
|
28
|
+
- [Daily use](#daily-use)
|
|
29
|
+
- [Troubleshooting](#troubleshooting)
|
|
30
|
+
- [Limitations](#limitations)
|
|
31
|
+
- [Contributing](#contributing)
|
|
32
|
+
|
|
33
|
+
## What it does
|
|
34
|
+
|
|
35
|
+
**Discovers your models.** Scans a directory you choose for `.gguf` files,
|
|
36
|
+
including nested layouts like `<publisher>/<repo>/model.gguf`, and registers
|
|
37
|
+
each one — every quantisation as its own entry.
|
|
38
|
+
|
|
39
|
+
**Runs the server.** Starts `llama-server` on your first message and keeps it
|
|
40
|
+
running. Nothing is spawned at launch; `[start]` and `[stop]` are in the panel
|
|
41
|
+
when you want to decide yourself.
|
|
42
|
+
|
|
43
|
+
**Reports the truth about context.** opencode compacts a conversation against
|
|
44
|
+
the model's advertised window. The context you configure per model is what gets
|
|
45
|
+
advertised, so a model loaded at 245k is not compacted as though it were 32k.
|
|
46
|
+
|
|
47
|
+
**Shows what the hardware is doing.** Per-GPU memory and utilisation, system RAM
|
|
48
|
+
and CPU, the loaded model with its launch flags, and live progress while weights
|
|
49
|
+
stream into VRAM.
|
|
50
|
+
|
|
51
|
+
**Keeps settings in llama.cpp's own format.** Per-model options live in a
|
|
52
|
+
standard llama.cpp preset file, passed to `llama-server` unmodified. Any flag
|
|
53
|
+
llama.cpp accepts works, whether or not this plugin knows about it.
|
|
15
54
|
|
|
16
55
|
## Requirements
|
|
17
56
|
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
-
|
|
57
|
+
| | |
|
|
58
|
+
|---|---|
|
|
59
|
+
| **opencode** | tested against 1.18.5; older versions are untested |
|
|
60
|
+
| **llama-server** | build [llama.cpp](https://github.com/ggml-org/llama.cpp) yourself, or install it (`brew install llama.cpp`, a release archive, your package manager) |
|
|
61
|
+
| **models** | one or more `.gguf` files in a directory |
|
|
62
|
+
| **GPU stats** | optional. NVIDIA only; without it the hardware rows are simply omitted |
|
|
63
|
+
|
|
64
|
+
## Installation
|
|
22
65
|
|
|
23
|
-
|
|
66
|
+
Clone the repository and install its dependencies:
|
|
24
67
|
|
|
25
|
-
|
|
68
|
+
```sh
|
|
69
|
+
git clone https://github.com/dushyant30suthar/opencode-localhost
|
|
70
|
+
cd opencode-localhost
|
|
71
|
+
bun install
|
|
72
|
+
```
|
|
73
|
+
|
|
74
|
+
Then point opencode at the directory. It goes in **two files**, because opencode
|
|
75
|
+
loads server-side and terminal-side plugins through separate hosts:
|
|
26
76
|
|
|
27
77
|
```jsonc
|
|
28
78
|
// ~/.config/opencode/opencode.jsonc
|
|
@@ -34,58 +84,91 @@ opencode loads server-side and TUI plugins separately, so it goes in two files.
|
|
|
34
84
|
{ "plugin": ["/absolute/path/to/opencode-localhost"] }
|
|
35
85
|
```
|
|
36
86
|
|
|
37
|
-
|
|
38
|
-
|
|
39
|
-
|
|
87
|
+
> **Why a path rather than the package name?**
|
|
88
|
+
>
|
|
89
|
+
> The package is published to npm, but installing it by name currently gives
|
|
90
|
+
> you the provider without the panel. opencode resolves an npm plugin against a
|
|
91
|
+
> wrapper `package.json` it generates in `~/.cache/opencode/packages/`, which
|
|
92
|
+
> carries no `exports` field, so the terminal-side entry point is never found
|
|
93
|
+
> and is skipped without an error. The server-side entry survives on a
|
|
94
|
+
> different fallback, which makes the result look half-broken rather than
|
|
95
|
+
> unconfigured.
|
|
96
|
+
>
|
|
97
|
+
> Referencing the directory avoids that resolution path entirely. When opencode
|
|
98
|
+
> resolves npm plugins against the real package, `{ "plugin":
|
|
99
|
+
> ["opencode-localhost"] }` will work with no change here.
|
|
100
|
+
|
|
101
|
+
## First run
|
|
102
|
+
|
|
103
|
+
Start opencode. The panel appears below the prompt and reports what is missing:
|
|
104
|
+
|
|
105
|
+
```
|
|
106
|
+
HARDWARE memory compute │ llama.cpp │ no model loaded
|
|
107
|
+
GPU0 0.4/15.9G 12% │ ✕ not set up │ [change]
|
|
108
|
+
GPU1 0.4/15.9G 1% │ models-dir not set│
|
|
109
|
+
CPU 7.1/31.2G 16% │ │
|
|
40
110
|
```
|
|
41
111
|
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
|
|
45
|
-
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
|
|
112
|
+
Run `/localhost` and set two things:
|
|
113
|
+
|
|
114
|
+
```
|
|
115
|
+
✓ llama.cpp /usr/local/bin/llama-server
|
|
116
|
+
✗ vLLM not installed — pip install vllm
|
|
117
|
+
✗ MLX Apple Silicon only
|
|
118
|
+
✗ OpenVINO not installed — pip install openvino-genai
|
|
119
|
+
✓ Models directory ~/models
|
|
120
|
+
○ Server stopped [start] 127.0.0.1:9337
|
|
121
|
+
```
|
|
122
|
+
|
|
123
|
+
- **llama.cpp** — every `llama-server` found on `$PATH`, plus *Enter a path…*
|
|
124
|
+
for a build that is not on it
|
|
125
|
+
- **Models directory** — where your `.gguf` files live
|
|
126
|
+
|
|
127
|
+
Nothing is guessed. `$PATH` is searched for the binary and nothing else; the
|
|
128
|
+
models directory has no default, because only you know where it is.
|
|
129
|
+
|
|
130
|
+
Once both are set the models appear in the picker within a couple of seconds.
|
|
131
|
+
No restart.
|
|
132
|
+
|
|
133
|
+
## Configuration
|
|
49
134
|
|
|
50
|
-
|
|
51
|
-
missing.
|
|
135
|
+
Two files, both under `~/.config/opencode/providers/llamacpp/`.
|
|
52
136
|
|
|
53
|
-
|
|
137
|
+
### `server.ini` — how the server is run
|
|
54
138
|
|
|
55
|
-
|
|
56
|
-
|
|
139
|
+
Written on first start. Read by this plugin, which builds `llama-server`'s
|
|
140
|
+
command line from it.
|
|
57
141
|
|
|
58
142
|
```ini
|
|
59
143
|
[server]
|
|
60
|
-
bin
|
|
61
|
-
models-dir = ~/models
|
|
62
|
-
host
|
|
63
|
-
port
|
|
144
|
+
bin = /usr/local/bin/llama-server
|
|
145
|
+
models-dir = ~/models
|
|
146
|
+
host = 127.0.0.1
|
|
147
|
+
port = 9337
|
|
64
148
|
models-max = 1
|
|
65
|
-
api-key
|
|
149
|
+
api-key =
|
|
66
150
|
```
|
|
67
151
|
|
|
68
|
-
|
|
69
|
-
|
|
70
|
-
|
|
71
|
-
|
|
72
|
-
|
|
73
|
-
|
|
152
|
+
| Key | Meaning |
|
|
153
|
+
|---|---|
|
|
154
|
+
| `bin` | path to `llama-server`. Empty means search `$PATH` |
|
|
155
|
+
| `models-dir` | directory to scan for `.gguf` files. Required |
|
|
156
|
+
| `host` | `127.0.0.1` keeps the server on this machine |
|
|
157
|
+
| `port` | listen port |
|
|
158
|
+
| `models-max` | how many models may occupy VRAM at once. `1` swaps instead of stacking |
|
|
159
|
+
| `api-key` | when set, `llama-server` enforces bearer auth |
|
|
74
160
|
|
|
75
|
-
|
|
161
|
+
### `models.ini` — how each model is loaded
|
|
76
162
|
|
|
77
|
-
|
|
78
|
-
|
|
79
|
-
llama.cpp preset file, handed to `llama-server --models-preset` verbatim. A
|
|
80
|
-
section is appended when a new `.gguf` appears and **never modified
|
|
81
|
-
afterwards** — it is yours.
|
|
163
|
+
Generated for you the first time your models are scanned. This is a standard
|
|
164
|
+
llama.cpp preset file, handed to `llama-server --models-preset` unmodified.
|
|
82
165
|
|
|
83
|
-
|
|
84
|
-
|
|
166
|
+
A section is appended when a new `.gguf` appears and **is never edited
|
|
167
|
+
afterwards**. It is yours.
|
|
85
168
|
|
|
86
169
|
```ini
|
|
87
170
|
[unsloth/Qwen3.6-35B-A3B-GGUF]
|
|
88
|
-
model = /
|
|
171
|
+
model = /home/you/models/unsloth/Qwen3.6-35B-A3B-GGUF/Qwen3.6-35B-A3B-Q4_K_M.gguf
|
|
89
172
|
ctx-size = 245760
|
|
90
173
|
ubatch-size = 2048
|
|
91
174
|
gpu-layers = 99
|
|
@@ -98,46 +181,109 @@ top-p = 0.95
|
|
|
98
181
|
|
|
99
182
|
Any `llama-server` flag works as a key, without the leading dashes.
|
|
100
183
|
|
|
101
|
-
|
|
184
|
+
The section name is the model's identity. Renaming a section is safe — it keeps
|
|
185
|
+
governing its file and its settings keep being used.
|
|
186
|
+
|
|
187
|
+
Two settings have different lifecycles:
|
|
102
188
|
|
|
103
189
|
| Change | Takes effect |
|
|
104
190
|
|---|---|
|
|
105
|
-
| launch flags — `ctx-size`, `gpu-layers`, `tensor-split`, `cache-type-*` | next model
|
|
191
|
+
| launch flags — `ctx-size`, `gpu-layers`, `tensor-split`, `cache-type-*` | next time the model loads |
|
|
106
192
|
| sampling — `temp`, `top-p`, `top-k`, `min-p` | next message, no reload |
|
|
107
193
|
|
|
108
|
-
Sampling is
|
|
109
|
-
own per-model defaults never override what you set here
|
|
194
|
+
Sampling is attached to each request rather than baked into the server process,
|
|
195
|
+
so opencode's own per-model defaults never override what you set here, and
|
|
196
|
+
changing it does not cost a reload.
|
|
197
|
+
|
|
198
|
+
`ctx-size` is also what opencode compacts against. Set it to what the model
|
|
199
|
+
really loads with, or long conversations will compact far earlier than they
|
|
200
|
+
need to.
|
|
201
|
+
|
|
202
|
+
## Daily use
|
|
203
|
+
|
|
204
|
+
Open the model picker (`/model`, or `[change]` in the panel) and choose a model
|
|
205
|
+
under **Localhost-llama.cpp**.
|
|
206
|
+
|
|
207
|
+
Send a message. The first one is slow — that is the server starting and the
|
|
208
|
+
weights streaming into VRAM. The panel shows it:
|
|
209
|
+
|
|
210
|
+
```
|
|
211
|
+
HARDWARE memory compute │ llama.cpp │ ◐ unsloth/
|
|
212
|
+
GPU0 8.1/15.9G 64% │ ● 127.0.0.1:9337 │ Laguna-S-2.1-GGUF
|
|
213
|
+
GPU1 7.4/15.9G 61% │ [stop] │ ████░░░░░░ 43%
|
|
214
|
+
CPU 11.2/31.2G 38% │ │ text_model
|
|
215
|
+
```
|
|
216
|
+
|
|
217
|
+
Subsequent messages are warm.
|
|
218
|
+
|
|
219
|
+
The same panel appears in the session sidebar, stacked rather than in columns.
|
|
220
|
+
|
|
221
|
+
**Controls**
|
|
222
|
+
|
|
223
|
+
| | |
|
|
224
|
+
|---|---|
|
|
225
|
+
| `[start]` / `[stop]` | run or stop the server |
|
|
226
|
+
| `[change]` | opencode's model picker |
|
|
227
|
+
| `/localhost` | paths and server state |
|
|
228
|
+
| `ctrl+p` → *Local models* | the same actions, from the command palette |
|
|
110
229
|
|
|
111
|
-
|
|
112
|
-
|
|
230
|
+
Both actions are registered commands (`localhost.server.toggle` and opencode's
|
|
231
|
+
`model.list`), so they can be bound to keys in your keybinds config.
|
|
113
232
|
|
|
114
|
-
##
|
|
233
|
+
## Troubleshooting
|
|
115
234
|
|
|
116
|
-
|
|
117
|
-
|
|
118
|
-
|
|
119
|
-
|
|
235
|
+
**The panel is missing entirely.** The terminal-side plugin is not loading.
|
|
236
|
+
Check `tui.jsonc` exists and points at the directory — it is a separate file
|
|
237
|
+
from `opencode.jsonc` and easy to miss. Confirm `bun install` has been run in
|
|
238
|
+
the clone; the panel needs its own dependencies.
|
|
120
239
|
|
|
121
|
-
|
|
240
|
+
**No provider in the model picker.** The engine is not configured. Run
|
|
241
|
+
`/localhost` — a backend with no binary or no models directory does not
|
|
242
|
+
register. If both are set and the picker is still empty, no `.gguf` files were
|
|
243
|
+
found under `models-dir`.
|
|
122
244
|
|
|
123
|
-
|
|
124
|
-
|
|
125
|
-
|
|
126
|
-
|
|
245
|
+
**A model fails to load and retries.** The server log has the reason:
|
|
246
|
+
|
|
247
|
+
```sh
|
|
248
|
+
tail -40 ~/.local/state/opencode/providers/llamacpp/server.log
|
|
249
|
+
```
|
|
250
|
+
|
|
251
|
+
The usual cause is a path in `models.ini` that `llama-server` cannot open.
|
|
252
|
+
Paths must be absolute — `llama-server` does not expand `~`.
|
|
253
|
+
|
|
254
|
+
**Conversations compact too early.** `ctx-size` for that model does not match
|
|
255
|
+
what it actually loads with. The panel shows the value in effect.
|
|
256
|
+
|
|
257
|
+
**Settings changed but nothing happened.** Launch flags apply on the next model
|
|
258
|
+
load, not immediately. Stop and start the server, or switch models and back.
|
|
127
259
|
|
|
128
260
|
## Limitations
|
|
129
261
|
|
|
130
|
-
-
|
|
131
|
-
|
|
132
|
-
|
|
133
|
-
|
|
134
|
-
|
|
135
|
-
|
|
136
|
-
|
|
137
|
-
|
|
138
|
-
|
|
139
|
-
|
|
262
|
+
**No warm-on-select.** opencode does not expose the selected model to plugins,
|
|
263
|
+
so a model begins loading on your first message rather than the moment you pick
|
|
264
|
+
it.
|
|
265
|
+
|
|
266
|
+
**Local only.** The panel reads `nvidia-smi` and the server directly, so it goes
|
|
267
|
+
blank if the terminal runs on a different machine from the model server.
|
|
268
|
+
|
|
269
|
+
**NVIDIA only for GPU statistics.** Everything else works without a GPU; the
|
|
270
|
+
hardware rows are omitted.
|
|
271
|
+
|
|
272
|
+
**llama.cpp only, for now.** The architecture is backend-agnostic and the setup
|
|
273
|
+
screen lists vLLM, MLX and OpenVINO with their install state, but only llama.cpp
|
|
274
|
+
is implemented.
|
|
275
|
+
|
|
276
|
+
## Contributing
|
|
277
|
+
|
|
278
|
+
[`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md) covers how the two halves fit
|
|
279
|
+
together, the constraints opencode's plugin API imposes, and what implementing
|
|
280
|
+
another backend involves.
|
|
281
|
+
|
|
282
|
+
```sh
|
|
283
|
+
bun install
|
|
284
|
+
bunx tsc --noEmit -p tsconfig.json
|
|
285
|
+
```
|
|
140
286
|
|
|
141
287
|
## License
|
|
142
288
|
|
|
143
|
-
MIT
|
|
289
|
+
MIT — see [LICENSE](./LICENSE).
|