opencode-localhost 0.1.0

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 dushyant30suthar
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -0,0 +1,130 @@
+# opencode-localhost
+
+Run local models in [opencode](https://opencode.ai). Your `.gguf` files show up
+in the model picker as an ordinary provider, `llama-server` is started and
+supervised for you, and a panel shows GPU, VRAM and CPU while you work.
+
+```
+ HARDWARE      memory  compute │ PROVIDER       │ MODEL
+ GPU0     14.2/16.0G      87%  │ llama.cpp      │ Qwen3.6-35B-A3B
+ GPU1     13.8/16.0G      82%  │ ● running      │ 245760 ctx · q8_0 KV
+ CPU      18.1/31.0G      24%  │ :9337 web off  │ ngl 99 · 68.2 tok/s
+```
+
+No fork of opencode, no patches. It is a plugin.
+
+## Requirements
+
+- opencode
+- a `llama-server` binary — [build it](https://github.com/ggml-org/llama.cpp) or
+  install it however you like, then make sure it is on `$PATH`
+- some `.gguf` files
+
+## Install
+
+opencode loads server-side and TUI plugins separately, so it goes in two files.
+
+```jsonc
+// ~/.config/opencode/opencode.jsonc
+{ "plugin": ["opencode-localhost"] }
+```
+
+```jsonc
+// ~/.config/opencode/tui.jsonc
+{ "plugin": ["opencode-localhost"] }
+```
+
+Start opencode. The panel appears under the prompt and tells you what is still
+missing.
+
+## Setup
+
+On first run it writes `~/.config/opencode/providers/llamacpp/server.ini`.
+Exactly one thing has no sensible default:
+
+```ini
+[server]
+bin =                              # empty = look on $PATH
+models-dir = ~/models              # ← set this
+host = 127.0.0.1
+port = 9337
+models-max = 1
+api-key =
+```
+
+Set `models-dir` and save. No restart: the panel notices within a couple of
+seconds, scans your models, generates `models.ini`, starts `llama-server`, and
+asks opencode to re-read its config so the provider appears in the picker.
+
+Nothing is guessed. If `llama-server` is not on `$PATH`, set `bin` — the panel
+says so rather than failing silently.
+
+## Per-model settings
+
+`~/.config/opencode/providers/llamacpp/models.ini` is written **for you**, the
+first time your models are scanned — you never create it by hand. It is a
+llama.cpp preset file, handed to `llama-server --models-preset` verbatim. A
+section is appended when a new `.gguf` appears and **never modified
+afterwards** — it is yours.
+
+The section name is the model id, so renaming a section is safe: it keeps
+governing its file, and its settings keep being used.
+
+```ini
+[unsloth/Qwen3.6-35B-A3B-GGUF]
+model        = /path/to/Qwen3.6-35B-A3B-Q4_K_M.gguf
+ctx-size     = 245760
+ubatch-size  = 2048
+gpu-layers   = 99
+flash-attn   = on
+cache-type-k = q8_0
+cache-type-v = q8_0
+temp         = 0.6
+top-p        = 0.95
+```
+
+Any `llama-server` flag works as a key, without the leading dashes.
+
+Two different lifecycles:
+
+| Change | Takes effect |
+|---|---|
+| launch flags — `ctx-size`, `gpu-layers`, `tensor-split`, `cache-type-*` | next model load |
+| sampling — `temp`, `top-p`, `top-k`, `min-p` | next message, no reload |
+
+Sampling is sent per request rather than baked into the server, so opencode's
+own per-model defaults never override what you set here.
+
+`ctx-size` is also what opencode compacts against — set it to what the model
+really loads with, or long conversations will compact far too early.
+
+## Adding another backend
+
+`src/server/llamacpp/` implements `src/server/backend.ts`. vLLM or anything else
+is a sibling folder implementing the same three methods — discover models, start
+the server, say where it is. Its own config file, in its own native format,
+under `~/.config/opencode/providers/<backend>/`.
+
+## Security
+
+`host` defaults to `127.0.0.1`, so the server is reachable only from this
+machine. If you set `0.0.0.0` to use it from elsewhere on your network, set
+`api-key` as well — `llama-server` then enforces bearer auth, and any
+OpenAI-compatible client can connect with it.
+
+## Limitations
+
+- **Provider refresh only happens from the home screen.** opencode's only
+  reload path disposes live instances, so it is not fired mid-session; the
+  panel shows **restart opencode** in that case instead.
+- **No warm-on-select.** opencode's TUI does not expose the selected model to
+  plugins, so a model starts loading on your first message rather than the
+  moment you pick it.
+- **Local only.** The panel reads `nvidia-smi` and the server directly, so it
+  goes blank if the TUI runs on a different machine from opencode.
+- **NVIDIA only** for GPU stats. Everything else works without a GPU; the
+  hardware rows are simply omitted.
+
+## License
+
+MIT

package/package.json

@@ -0,0 +1,57 @@
+{
+  "name": "opencode-localhost",
+  "version": "0.1.0",
+  "description": "Run local models in opencode. llama.cpp as a provider, with live GPU, VRAM and CPU telemetry in the sidebar.",
+  "keywords": [
+    "opencode",
+    "opencode-plugin",
+    "llama.cpp",
+    "llama-server",
+    "local-models",
+    "gguf",
+    "vllm",
+    "gpu",
+    "self-hosted"
+  ],
+  "license": "MIT",
+  "author": "dushyant30suthar",
+  "repository": {
+    "type": "git",
+    "url": "git+https://github.com/dushyant30suthar/opencode-localhost.git"
+  },
+  "homepage": "https://github.com/dushyant30suthar/opencode-localhost#readme",
+  "bugs": "https://github.com/dushyant30suthar/opencode-localhost/issues",
+  "type": "module",
+  "exports": {
+    "./server": "./src/server/index.ts",
+    "./tui": "./src/tui/index.tsx"
+  },
+  "files": [
+    "src",
+    "tsconfig.json",
+    "README.md",
+    "LICENSE"
+  ],
+  "peerDependencies": {
+    "@opentui/solid": ">=0.4.5",
+    "solid-js": ">=1.9.0"
+  },
+  "peerDependenciesMeta": {
+    "@opentui/solid": {
+      "optional": true
+    },
+    "solid-js": {
+      "optional": true
+    }
+  },
+  "devDependencies": {
+    "@opencode-ai/plugin": "^1.18.4",
+    "@opentui/solid": "^0.4.5",
+    "@types/node": "^26.1.1",
+    "solid-js": "^1.9.0",
+    "typescript": "^7.0.2"
+  },
+  "engines": {
+    "node": ">=22"
+  }
+}

package/src/server/backend.ts

@@ -0,0 +1,38 @@
+import type { ProviderStatus } from "../shared/types.ts"
+
+/**
+ * What every backend implements. llama.cpp today; vLLM and OpenVINO are new
+ * folders implementing this, with no change to the plugin around them.
+ *
+ * A backend owns exactly three things: knowing what models exist, getting its
+ * server running, and saying where to reach it. Everything backend-specific —
+ * config file format, launch flags, discovery layout — stays inside.
+ */
+
+export type DiscoveredModel = {
+  /** Stable id shown in opencode's picker and used as the API model id. */
+  id: string
+  name: string
+  /** Real context window this model loads with. opencode compacts against it. */
+  context: number
+  output: number
+  /** Sampling to send per request, so opencode's own defaults never override. */
+  sampling: Record<string, number>
+}
+
+export interface Backend {
+  readonly id: string
+  readonly name: string
+
+  /** Never throws. "unconfigured" is a normal answer on a fresh install. */
+  status(): Promise<ProviderStatus>
+
+  /** Empty when unconfigured or when the directory holds nothing usable. */
+  models(): Promise<DiscoveredModel[]>
+
+  /** Idempotent: starts the server only if it is not already answering. */
+  start(): Promise<ProviderStatus>
+
+  baseURL(): string
+  apiKey(): string | undefined
+}

package/src/server/index.ts

@@ -0,0 +1,103 @@
+import type { Backend, DiscoveredModel } from "./backend.ts"
+import { create as llamacpp } from "./llamacpp/index.ts"
+
+/**
+ * The server half.
+ *
+ * opencode loads plugins before it reads `cfg.provider`, specifically so a
+ * plugin's config() hook can add to it. That is the whole integration: we
+ * discover models on disk, start the backend, and write a provider into the
+ * config opencode is about to read. From there it is an ordinary provider.
+ *
+ * Sampling is handled in chat.params rather than baked into the server launch,
+ * so changing `temp` in models.ini takes effect on the next message instead of
+ * requiring a model reload — and so opencode's own per-model heuristics, which
+ * know nothing about local models, never override it.
+ */
+
+const BACKENDS: Backend[] = [llamacpp()]
+
+/** Cached per process: discovery walks the filesystem and starts a server. */
+const sampling = new Map<string, Record<string, number>>()
+
+function providerEntry(backend: Backend, models: DiscoveredModel[]) {
+  return {
+    name: `${backend.name} (local)`,
+    api: backend.baseURL(),
+    npm: "@ai-sdk/openai-compatible",
+    options: {
+      baseURL: backend.baseURL(),
+      // the SDK requires some value; an unauthenticated server ignores it
+      apiKey: backend.apiKey() ?? "local",
+    },
+    models: Object.fromEntries(
+      models.map((model) => [
+        model.id,
+        {
+          name: model.name,
+          limit: { context: model.context, output: model.output },
+          cost: { input: 0, output: 0 },
+          options: {},
+        },
+      ]),
+    ),
+  }
+}
+
+async function register(input: any) {
+  for (const backend of BACKENDS) {
+    const status = await backend.status().catch(() => undefined)
+    // unconfigured or missing binary: contribute nothing. The panel explains why.
+    if (!status || status.state === "unconfigured" || status.state === "failed") continue
+
+    // Not up yet: start it in the background and register nothing this run.
+    //
+    // Awaiting here would stall opencode's startup for as long as the server
+    // takes to answer — once on a fresh setup, and on *every* launch if the
+    // server is failing to start. opencode reads its config once per instance
+    // and exposes no way to re-read it, so the provider genuinely cannot appear
+    // until the next launch; the panel says so rather than leaving it silent.
+    if (status.state !== "running") {
+      void backend.start().catch(() => {})
+      continue
+    }
+
+    const models = await backend.models().catch(() => [])
+    if (models.length === 0) continue
+
+    for (const model of models) {
+      if (Object.keys(model.sampling).length > 0) {
+        sampling.set(`${backend.id}/${model.id}`, model.sampling)
+      }
+    }
+
+    input.provider = input.provider ?? {}
+    input.provider[backend.id] = providerEntry(backend, models)
+  }
+}
+
+const server = async () => ({
+  config: async (input: any) => {
+    await register(input).catch(() => {})
+  },
+
+  "chat.params": async (input: any, output: any) => {
+    const providerID = input?.model?.providerID
+    if (!BACKENDS.some((backend) => backend.id === providerID)) return
+    const values = sampling.get(`${providerID}/${input?.model?.id}`)
+    if (!values) return
+    // opencode names three of these directly and passes the rest through options
+    if ("temperature" in values) output.temperature = values["temperature"]
+    if ("top_p" in values) output.topP = values["top_p"]
+    if ("top_k" in values) output.topK = values["top_k"]
+    const extra = { ...values }
+    delete extra["temperature"]
+    delete extra["top_p"]
+    delete extra["top_k"]
+    if (Object.keys(extra).length > 0) {
+      output.options = { ...(output.options ?? {}), ...extra }
+    }
+  },
+})
+
+export default { id: "opencode-localhost", server }

package/src/server/llamacpp/discover.ts

@@ -0,0 +1,92 @@
+import fs from "fs/promises"
+import path from "path"
+
+/**
+ * Find .gguf files under the models directory.
+ *
+ * llama-server's own --models-dir scan is one level deep, but the common
+ * layouts nest: LM Studio uses <root>/<publisher>/<repo>/*.gguf and a plain
+ * download folder is flat. We walk a bounded depth so both work without the
+ * user having to know which one they have.
+ */
+
+const MAX_DEPTH = 3
+
+export type LocalModel = {
+  /** Section name: "publisher/repo:Q6_K", or "publisher/repo" when unambiguous. */
+  id: string
+  file: string
+  /** Vision projector beside the weights, if present. */
+  mmproj?: string
+}
+
+type Found = {
+  dir: string
+  /** Path segments below the models root, for naming. */
+  relative: string[]
+  files: string[]
+  mmproj?: string
+}
+
+/** "Qwen3.6-27B-Q6_K.gguf" -> "Q6_K". Multi-shard suffixes stripped first. */
+export function quantOf(filename: string): string | undefined {
+  const stem = filename.replace(/\.gguf$/i, "").replace(/-\d{5}-of-\d{5}$/, "")
+  return stem.match(/[-.]((?:I?Q\d[\w.]*)|F16|F32|BF16|MXFP4[\w]*|UD-[\w.]+)$/i)?.[1]
+}
+
+function nameFor(relative: string[], filename: string, multiple: boolean): string {
+  const base = relative.length > 0 ? relative.join("/").replace(/-GGUF$/i, "") : "models"
+  if (!multiple) return base
+  const quant = quantOf(filename)
+  return `${base}:${quant ?? filename.replace(/\.gguf$/i, "")}`
+}
+
+async function walk(dir: string, relative: string[], out: Found[]): Promise<void> {
+  if (relative.length > MAX_DEPTH) return
+  const entries = await fs.readdir(dir, { withFileTypes: true }).catch(() => [])
+  const files: string[] = []
+  let mmproj: string | undefined
+  for (const entry of entries) {
+    if (entry.isDirectory()) {
+      await walk(path.join(dir, entry.name), [...relative, entry.name], out)
+      continue
+    }
+    if (!entry.isFile() || !entry.name.toLowerCase().endsWith(".gguf")) continue
+    if (/mmproj/i.test(entry.name)) {
+      mmproj = entry.name
+      continue
+    }
+    // multi-shard models: only the first shard is passed to llama-server
+    if (/-\d{5}-of-\d{5}\.gguf$/i.test(entry.name)) {
+      if (entry.name.includes("-00001-of-")) files.push(entry.name)
+      continue
+    }
+    files.push(entry.name)
+  }
+  if (files.length > 0) out.push({ dir, relative, files: files.sort(), mmproj })
+}
+
+export async function scan(root: string): Promise<LocalModel[]> {
+  if (!root) return []
+  const found: Found[] = []
+  await walk(root, [], found)
+
+  const models: LocalModel[] = []
+  const taken = new Set<string>()
+  for (const group of found) {
+    // one entry per quant file, so every variant is separately selectable
+    const multiple = group.files.length > 1
+    for (const file of group.files) {
+      let id = nameFor(group.relative, file, multiple)
+      if (taken.has(id)) id = nameFor(group.relative, file, true)
+      if (taken.has(id)) continue
+      taken.add(id)
+      models.push({
+        id,
+        file: path.join(group.dir, file),
+        mmproj: group.mmproj ? path.join(group.dir, group.mmproj) : undefined,
+      })
+    }
+  }
+  return models.sort((a, b) => a.id.localeCompare(b.id))
+}

package/src/server/llamacpp/index.ts

@@ -0,0 +1,188 @@
+import fs from "fs/promises"
+import path from "path"
+import { spawn } from "child_process"
+import { stateDir, collapseHome } from "../../shared/paths.ts"
+import type { ProviderStatus } from "../../shared/types.ts"
+import type { Backend, DiscoveredModel } from "../backend.ts"
+import * as Server from "./server-ini.ts"
+import * as Models from "./models-ini.ts"
+import { scan } from "./discover.ts"
+
+/**
+ * The llama.cpp backend: find llama-server, start it against models.ini, and
+ * report what it has. Everything llama.cpp-specific lives here and below.
+ */
+
+const STATE = stateDir(Server.BACKEND)
+const PID_FILE = path.join(STATE, "server.pid")
+const LOG_FILE = path.join(STATE, "server.log")
+
+const PROBE_TIMEOUT = 1_500
+const START_TIMEOUT = 20_000
+const POLL_INTERVAL = 400
+
+/** Only $PATH. Guessing at build directories bakes one machine into everyone's. */
+async function onPath(): Promise<string | undefined> {
+  const dirs = (process.env["PATH"] || "").split(path.delimiter).filter(Boolean)
+  for (const dir of dirs) {
+    const candidate = path.join(dir, "llama-server")
+    const stat = await fs.stat(candidate).catch(() => undefined)
+    if (stat?.isFile()) return candidate
+  }
+  return undefined
+}
+
+async function executable(file: string): Promise<boolean> {
+  const stat = await fs.stat(file).catch(() => undefined)
+  return !!stat?.isFile()
+}
+
+/** Reachable when it answers /v1/models; undefined means nothing is listening. */
+async function reachable(baseURL: string, apiKey: string, timeout: number): Promise<boolean> {
+  try {
+    const res = await fetch(`${baseURL}/models`, {
+      signal: AbortSignal.timeout(timeout),
+      headers: apiKey ? { Authorization: `Bearer ${apiKey}` } : {},
+    })
+    return res.ok
+  } catch {
+    return false
+  }
+}
+
+export function create(): Backend {
+  let settings: Server.ServerSettings | undefined
+  let starting: Promise<ProviderStatus> | undefined
+
+  // Re-read every time rather than caching: server.ini is small, and a cached
+  // copy meant editing it did nothing until the process restarted — including
+  // for the panel, which polls this several times a minute.
+  const config = async () => (settings = await Server.load())
+  const baseURL = () => {
+    const host = settings?.host === "0.0.0.0" ? "127.0.0.1" : (settings?.host ?? "127.0.0.1")
+    return `http://${host}:${settings?.port ?? 9337}/v1`
+  }
+
+  async function resolveBin(cfg: Server.ServerSettings): Promise<string | undefined> {
+    if (cfg.bin) return (await executable(cfg.bin)) ? cfg.bin : undefined
+    const found = await onPath()
+    // record it so the file always shows what is actually being used
+    if (found) await Server.update("bin", found)
+    return found
+  }
+
+  async function status(): Promise<ProviderStatus> {
+    const cfg = await config()
+    const bin = await resolveBin(cfg)
+    if (!bin) {
+      return {
+        state: "unconfigured",
+        missing: "binary",
+        message: "llama-server not found",
+        hint: `set bin in ${collapseHome(Server.FILE)}`,
+      }
+    }
+    if (!cfg.modelsDir) {
+      return {
+        state: "unconfigured",
+        missing: "models-dir",
+        message: "models-dir not set",
+        hint: `set it in ${collapseHome(Server.FILE)}`,
+      }
+    }
+    if (await reachable(baseURL(), cfg.apiKey, PROBE_TIMEOUT)) {
+      return { state: "running", endpoint: baseURL() }
+    }
+    return { state: "stopped" }
+  }
+
+  async function models(): Promise<DiscoveredModel[]> {
+    const cfg = await config()
+    if (!cfg.modelsDir) return []
+    const local = await scan(cfg.modelsDir)
+    if (local.length === 0) return []
+    const { names } = await Models.sync(local)
+    const perModel = await Models.settings()
+    return local.map((model) => {
+      // the section name governs, so a renamed section keeps its tuned values
+      const id = names.get(model.file) ?? model.id
+      const entry = perModel[id]
+      const context = entry?.context || 32_768
+      return {
+        id,
+        name: id.split("/").filter(Boolean).at(-1) ?? id,
+        context,
+        // thinking models spend the reasoning budget from the output allowance;
+        // half the window, capped, leaves room for a long plan without truncating
+        output: Math.min(32_768, Math.max(4_096, Math.floor(context / 2))),
+        sampling: entry?.sampling ?? {},
+      }
+    })
+  }
+
+  async function launch(cfg: Server.ServerSettings, bin: string): Promise<ProviderStatus> {
+    const local = await scan(cfg.modelsDir)
+    if (local.length === 0) {
+      return {
+        state: "unconfigured",
+        missing: "models-dir",
+        message: "no .gguf files found",
+        hint: collapseHome(cfg.modelsDir),
+      }
+    }
+    const { preset } = await Models.sync(local)
+    await fs.mkdir(STATE, { recursive: true }).catch(() => {})
+    const log = await fs.open(LOG_FILE, "a").catch(() => undefined)
+    try {
+      const child = spawn(bin, Server.argv(cfg, preset), {
+        detached: true,
+        stdio: ["ignore", log?.fd ?? "ignore", log?.fd ?? "ignore"],
+      })
+      child.on("error", () => {})
+      child.unref()
+      if (child.pid) await fs.writeFile(PID_FILE, `${child.pid}\n`).catch(() => {})
+    } catch (error) {
+      return { state: "failed", message: "could not start", hint: String(error).slice(0, 60) }
+    } finally {
+      await log?.close().catch(() => {})
+    }
+
+    const deadline = Date.now() + START_TIMEOUT
+    while (Date.now() < deadline) {
+      if (await reachable(baseURL(), cfg.apiKey, PROBE_TIMEOUT)) {
+        return { state: "running", endpoint: baseURL() }
+      }
+      await new Promise((resolve) => setTimeout(resolve, POLL_INTERVAL))
+    }
+    return { state: "failed", message: "server did not start", hint: collapseHome(LOG_FILE) }
+  }
+
+  /** Idempotent and single-flight: concurrent callers share one attempt. */
+  async function start(): Promise<ProviderStatus> {
+    if (starting) return starting
+    starting = (async () => {
+      const cfg = await config()
+      const bin = await resolveBin(cfg)
+      if (!bin || !cfg.modelsDir) return status()
+      if (await reachable(baseURL(), cfg.apiKey, PROBE_TIMEOUT)) {
+        return { state: "running", endpoint: baseURL() } as ProviderStatus
+      }
+      return launch(cfg, bin)
+    })()
+    try {
+      return await starting
+    } finally {
+      starting = undefined
+    }
+  }
+
+  return {
+    id: Server.BACKEND,
+    name: "llama.cpp",
+    status,
+    models,
+    start,
+    baseURL,
+    apiKey: () => settings?.apiKey || undefined,
+  }
+}