dsh-local-models 0.2.0 → 0.3.1

package/README-es.md

@@ -11,7 +11,7 @@ Construido sobre el `llama.cpp` upstream sin modificaciones (`llama-server`). Si
 - **Estimación de VRAM en vivo** — pesos + los tipos de caché K/V seleccionados + estado recurrente + cómputo/grafo + overhead frente al total de GPU detectado (nvidia-smi / sysfs de amdgpu, sumados entre todas las GPUs, 16 GB asumidos cuando se desconoce), con filas de entra / margen de seguridad / ctx máximo que entra (ver [Problemas conocidos](./KNOWN_ISSUES.md) para la precisión en la familia Gemma)
 - **Profiles** — guarda configuraciones de lanzamiento con nombre y recárgalas con un clic
 - **Modo router** — sirve todos los perfiles guardados desde un único endpoint compatible con OpenAI (`--models-preset`); los modelos se cargan bajo demanda, uno residente a la vez por defecto. Arrancar el router registra (o vuelve a registrar) automáticamente sus modelos en dsh — sin pulsar Register a mano.
-- **Register in dsh** — escribe el servidor listo como ruta de proveedor `llm-pi-ai` (con modalidad de visión + niveles de razonamiento, y salida máxima anunciada en 131K tokens para que los bloques largos de razonamiento xhigh no se trunquen)
+- **Register in dsh** — escribe el servidor listo como ruta de proveedor `llm-pi-ai` (con modalidad de visión + niveles de razonamiento, y salida máxima anunciada en 32K tokens (limitada a la mitad de la ventana para que la compactación conserve presupuesto de presión; sube maxTokens por request explícitamente para bloques largos de razonamiento xhigh))
 - **Overlay de terminal** — tail en vivo del log de `llama-server` desde la pestaña
 
 ## Requirements

package/README-hi.md

@@ -11,7 +11,7 @@ stock upstream `llama.cpp` (`llama-server`) के विरुद्ध बन
 - **Live VRAM estimate** — weights + चुने गए K/V cache types + recurrent state + compute/graph + overhead, detected GPU total के विरुद्ध (nvidia-smi / amdgpu sysfs, सभी GPUs पर summed, अज्ञात होने पर 16 GB माना गया), fits / safe-margin / max-ctx-that-fits पंक्तियों के साथ (Gemma-family सटीकता के लिए [Known issues](./KNOWN_ISSUES.md) देखें)
 - **Profiles** — नामित launch configurations सहेजें, एक क्लिक में फिर लोड करें
 - **Router mode** — सभी सहेजे गए profiles को एक OpenAI-compatible endpoint (`--models-preset`) से serve करें; models माँग पर लोड होते हैं, default रूप से एक बार में एक ही resident रहता है। router शुरू करने पर dsh में उसके models अपने आप (फिर से) register हो जाते हैं — manual Register दबाने की ज़रूरत नहीं।
-- **Register in dsh** — तैयार server को `llm-pi-ai` provider route के रूप में लिखता है (vision modality + thinking levels शामिल, max output 131K tokens बताया गया ताकि लंबे xhigh thinking blocks कटें नहीं)
+- **Register in dsh** — तैयार server को `llm-pi-ai` provider route के रूप में लिखता है (vision modality + thinking levels शामिल, max output 32K tokens बताया गया (window के आधे तक सीमित ताकि compaction का pressure budget बचा रहे; लंबे xhigh thinking blocks के लिए per-request maxTokens स्पष्ट रूप से बढ़ाएं))
 - **Terminal overlay** — tab से ही `llama-server` log का live tail
 
 ## Requirements

package/README-pt.md

@@ -11,7 +11,7 @@ Construído sobre o `llama.cpp` upstream sem modificações (`llama-server`). Se
 - **Estimativa de VRAM ao vivo** — pesos + os tipos de cache K/V selecionados + estado recorrente + compute/grafo + overhead contra o total detectado de GPU (nvidia-smi / sysfs do amdgpu, somados entre as GPUs, 16 GB presumidos quando desconhecido), com linhas de cabe / margem de segurança / ctx máximo que cabe (veja [Known issues](./KNOWN_ISSUES.md) para a precisão na família Gemma)
 - **Profiles** — salve configurações de inicialização nomeadas e recarregue com um clique
 - **Modo Router** — serve todos os profiles salvos a partir de um único endpoint compatível com OpenAI (`--models-preset`); os modelos carregam sob demanda, um residente por vez por padrão. Iniciar o router registra (ou re-registra) automaticamente seus modelos no dsh — sem precisar clicar em Register manualmente.
-- **Register in dsh** — grava o servidor pronto como uma rota de provedor `llm-pi-ai` (modalidade de visão + thinking levels incluídos, saída máxima anunciada em 131K tokens para que blocos longos de thinking xhigh não sejam truncados)
+- **Register in dsh** — grava o servidor pronto como uma rota de provedor `llm-pi-ai` (modalidade de visão + thinking levels incluídos, saída máxima anunciada em 32K tokens (limitada à metade da janela para que a compactação mantenha budget de pressão; suba o maxTokens por request explicitamente para blocos longos de thinking xhigh))
 - **Overlay de terminal** — tail ao vivo do log do `llama-server` direto da aba
 
 ## Requirements

package/README-zh.md

@@ -11,7 +11,7 @@
 - **实时显存估算** —— 权重 + 所选的 K/V cache 类型 + 循环状态 + 计算/图 + 额外开销，与检测到的 GPU 总显存对比（nvidia-smi / amdgpu sysfs，多卡求和，未知时按 16 GB 计），并给出 fits / safe-margin / max-ctx-that-fits 三行结果（Gemma 系列的准确性见 [Known issues](./KNOWN_ISSUES.md)）
 - **Profiles** —— 保存具名的启动配置，一键重新加载
 - **路由模式** —— 用一个 OpenAI 兼容端点（`--models-preset`）服务所有已保存的 profile；模型按需加载，默认同一时刻只驻留一个。启动路由会自动在 dsh 中（重新）注册它的模型 —— 无需手动点 Register。
-- **Register in dsh** —— 把已就绪的服务器写成一个 `llm-pi-ai` 提供方路由（包含视觉模态 + thinking levels，最大输出声明为 131K tokens，以免较长的 xhigh thinking 块被截断）
+- **Register in dsh** —— 把已就绪的服务器写成一个 `llm-pi-ai` 提供方路由（包含视觉模态 + thinking levels，最大输出声明为 32K tokens（上限为窗口的一半，以便 compact 保留压力预算；较长的 xhigh thinking 块请显式提高单次请求的 maxTokens））
 - **终端浮层** —— 在标签页里实时跟踪 `llama-server` 日志
 
 ## Requirements

package/README.md

@@ -1,5 +1,7 @@
 # dsh-local-models
 
+[![Listed on dsh-plugin.org](https://dsh-plugin.org/badges/listed.svg)](https://dsh-plugin.org/plugins/vmarcelo49/dsh-local-models)
+
 A `dsh` addon that adds a **Local Models** tab to the dsh Web GUI: pick a `.gguf` file, tune context and speculative decoding, watch a live VRAM estimate, and load it through `llama-server` — then register the running server as an LLM provider in dsh with one click.
 
 Built against stock upstream `llama.cpp` (`llama-server`). No fork, no patches, no build step: the client bundle is hand-written `React.createElement` (no JSX toolchain) and the node half is dependency-free.
@@ -11,7 +13,7 @@ Built against stock upstream `llama.cpp` (`llama-server`). No fork, no patches,
 - **Live VRAM estimate** — weights + the selected K/V cache types + recurrent state + compute/graph + overhead against the detected GPU total (nvidia-smi / amdgpu sysfs, summed across GPUs, 16 GB assumed when unknown), with fits / safe-margin / max-ctx-that-fits rows (see [Known issues](./KNOWN_ISSUES.md) for Gemma-family accuracy)
 - **Profiles** — save named launch configurations, reload in one click
 - **Router mode** — serve all saved profiles from one OpenAI-compatible endpoint (`--models-preset`); models load on demand, one resident at a time by default. Starting the router automatically (re-)registers its models in dsh — no manual Register press.
-- **Register in dsh** — writes the ready server as an `llm-pi-ai` provider route (vision modality + thinking levels included, max output advertised at 131K tokens so long xhigh thinking blocks aren't truncated)
+- **Register in dsh** — writes the ready server as an `llm-pi-ai` provider route (vision modality + thinking levels included, max output advertised at 32K tokens (capped at half the window so compaction keeps a pressure budget; raise per-request maxTokens explicitly for long xhigh thinking blocks))
 - **Terminal overlay** — live tail of the `llama-server` log from the tab
 
 ## Requirements
@@ -136,3 +138,5 @@ See [KNOWN_ISSUES.md](./KNOWN_ISSUES.md) — most notably, the VRAM estimate is
 ## License
 
 MIT — see [LICENSE](./LICENSE).
+
+

package/lib/index.js

@@ -84,15 +84,99 @@ const LOG_PATH = join(dataDir(), "llama-server.log");
  * ~15-22 s per image. Set LOCAL_MODELS_MMPROJ_CPU=0 to keep vision on the GPU. */
 const MMPROJ_CPU = process.env.LOCAL_MODELS_MMPROJ_CPU !== "0";
 /**
- * Max output advertised when a server is registered in dsh: 131K tokens,
- * the full daily window. The qwen3.x series is a heavy thinker — at xhigh
- * the thinking block alone regularly exceeds the old 32K default and the
- * reply gets cut off — so registration offers the whole window as max
- * output. Still capped at the launched ctx in the profile builders below:
- * a small-ctx profile must never advertise more output than it has room
- * for.
+ * Max output advertised when a server is registered in dsh: 32K tokens,
+ * matching the llm-pi-ai adapter default. This matters beyond truncation:
+ * the advertised maxTokens becomes the adapter's defaultMaxTokens, which
+ * dsh-compaction-basic uses as its reserved output `O` in `W - O - headroom`.
+ * Advertising the whole window (the old 131K) leaves no message budget at
+ * `W = O` — proactive compaction warns once and never fires.
+ *
+ * NOTE — the 32K cap alone does NOT buy a late threshold under the core
+ * defaults (headroomTokens 65536, thresholdRatio 0.8, retainRatio 0.16; see
+ * compactionBudgetFor() below). For a 131072 window with O = 32768 the
+ * effective threshold is min(0.8 * W, W - O - headroom) = 32768, i.e. the
+ * session starts compacting at ~32-38K pressure tokens (the meter prices
+ * tools + system on top of the surface, so the visible trigger lands a few K
+ * above the raw threshold) — and with xhigh + preserveThinking every turn
+ * adds 10-20K thinking tokens, so compaction fires again on the next step:
+ * the "compaction loop". The honest fix for mid-size windows is a per-model
+ * headroom override in the dsh profile (headroom ~6-16K moves the 131K
+ * threshold to ~82-92K), not a smaller O — shrinking O truncates the xhigh
+ * thinking blocks this cap exists to protect. Heavy xhigh thinking that
+ * needs more must raise per-request maxTokens explicitly (which honestly
+ * moves the compaction threshold earlier instead of silently disabling it).
+ */
+const MAX_OUTPUT_TOKENS = 32768;
+/** Default output share of the window: never offer more than half as max
+ * output, so a message budget always survives (`W - O > 0`). */
+function defaultMaxOutput(contextWindow) {
+	return Math.max(1, Math.min(MAX_OUTPUT_TOKENS, Math.floor(contextWindow / 2)));
+}
+/**
+ * dsh-compaction-basic defaults this plugin's advertisement is priced
+ * against. Duplicated here (not imported) because the core package is not a
+ * dependency of this plugin — keep in sync with upstream's resolveConfig().
  */
-const MAX_OUTPUT_TOKENS = 131072;
+export const COMPACTION_DEFAULTS = {
+	thresholdRatio: 0.8,
+	headroomTokens: 65536,
+	retainRatio: 0.16,
+};
+/**
+ * Price one advertised window the way dsh-compaction-basic's
+ * resolveCompactSpec() does, so the tab/skill can tell the user WHEN
+ * proactive compaction will actually fire — before they hit the loop.
+ *
+ *   messageBudget = W - O            (history available to messages)
+ *   pressure      = messageBudget - headroom
+ *   threshold     = min(floor(W * ratio), pressure)   (fire at/above this)
+ *   retain        = floor(messageBudget * retainRatio) (tail kept per compact)
+ *
+ * `viable` is false when the pressure budget is <= 0: proactive compaction
+ * is then disabled entirely (core warns once and only recovers on overflow).
+ * That is the 96K-window cliff: W = 98304, O = 32768 leaves exactly 0.
+ *
+ * `recommendedHeadroom` is the headroom override that lands the threshold at
+ * ~70% of the window (clamped to [4096, 65536] so a summary + one retry
+ * always fit): the value to put in a `modelPolicies` entry for this route.
+ * Pure; `maxTokens` defaults to what buildProviderProfile would advertise.
+ */
+export function compactionBudgetFor(contextWindow, opts = {}) {
+	const W = Number.isInteger(contextWindow) && contextWindow > 0 ? contextWindow : 8192;
+	const O = Number.isInteger(opts.maxTokens) && opts.maxTokens > 0
+		? opts.maxTokens
+		: defaultMaxOutput(W);
+	const ratio = typeof opts.thresholdRatio === "number" && opts.thresholdRatio > 0 && opts.thresholdRatio <= 1
+		? opts.thresholdRatio
+		: COMPACTION_DEFAULTS.thresholdRatio;
+	const headroom = Number.isInteger(opts.headroomTokens) && opts.headroomTokens >= 0
+		? opts.headroomTokens
+		: COMPACTION_DEFAULTS.headroomTokens;
+	const retainRatio = typeof opts.retainRatio === "number" && opts.retainRatio > 0 && opts.retainRatio < ratio
+		? opts.retainRatio
+		: COMPACTION_DEFAULTS.retainRatio;
+	const messageBudget = W - O;
+	const pressureBudget = messageBudget - headroom;
+	const thresholdTokens = Math.floor(Math.min(W * ratio, pressureBudget));
+	const retainTokens = Math.floor(messageBudget * retainRatio);
+	// Headroom that would put the threshold at ~70% of the window: the
+	// pressure budget must cover 0.7 * W, i.e. headroom <= message - 0.7W.
+	const recommendedHeadroom = Math.max(
+		4096,
+		Math.min(COMPACTION_DEFAULTS.headroomTokens, messageBudget - Math.floor(W * 0.7)),
+	);
+	return {
+		contextWindow: W,
+		maxTokens: O,
+		headroomTokens: headroom,
+		messageBudget,
+		pressureBudget,
+		thresholdTokens,
+		retainTokens,
+		viable: pressureBudget > 0 && retainTokens < thresholdTokens,
+		recommendedHeadroom,
+	};
+}
 /** Fixed-MTP ceiling. The tab offers 0-7 and the API clamps here: upstream
  * accepts any `--spec-draft-n-max` and clamps the effective depth to the
  * model's own nextn depth at load. Depth 3 is the measured 16 GiB sweet spot
@@ -1622,12 +1706,17 @@ export function buildProviderProfile(st, requestedRoute) {
 			id: modelId,
 			name: modelId,
 			contextWindow,
-			// Advertise the full 131K window as max output: the qwen3.x series
-			// is a heavy thinker and at xhigh its thinking block alone blows
-			// past the old 32K ceiling, so long replies get truncated. Still
-			// never more than the window itself — a small-ctx profile must not
-			// offer more output than it has room for.
-			maxTokens: Math.min(MAX_OUTPUT_TOKENS, contextWindow),
+			// Capped at half the window (see MAX_OUTPUT_TOKENS): the qwen3.x
+			// series is a heavy thinker and at xhigh its thinking block alone
+			// blows past small ceilings, so 32K is the default — but never more
+			// than half the window, or compaction's `W - O - headroom` budget
+			// collapses and proactive compaction never fires. NOTE: under the
+			// core defaults (headroom 65536) a 131072 window still thresholds
+			// at ~32K — see compactionBudgetFor(); mid-size windows need a
+			// per-model headroom override, not a smaller O. Need more room
+			// for a monster thinking block? Raise per-request maxTokens
+			// explicitly instead.
+			maxTokens: defaultMaxOutput(contextWindow),
 			// Declaring image input is what makes a hand-declared vision model
 			// usable: without it dsh rejects image attachments ("model does not
 			// declare image input"). mmprojPath is set when a --mmproj file was
@@ -1940,7 +2029,7 @@ export function buildRouterProfile(profiles, modalitiesByModel) {
 			id,
 			name: id,
 			contextWindow: p.ctx ?? 8192,
-			maxTokens: Math.min(MAX_OUTPUT_TOKENS, p.ctx ?? 8192),
+			maxTokens: defaultMaxOutput(p.ctx ?? 8192),
 			input,
 			reasoningEfforts: { off: "none", low: "low", medium: "medium", xhigh: "xhigh" },
 			compat: { supportsReasoningEffort: true },

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "dsh-local-models",
-  "version": "0.2.0",
+  "version": "0.3.1",
   "description": "dsh addon: a Local Models settings tab that starts and stops a llama-server child process living with the dsh host process.",
   "type": "module",
   "main": "lib/index.js",

package/skills/dsh-local-models-ops/SKILL.md

@@ -98,10 +98,37 @@ provider route on the dsh webserver. Single-model mode needs the manual
 **Register in dsh** press; router mode has no Register button — starting
 the router auto-registers the `local-router` route once the server is
 ready (a refresh poll fires it; stopping first cancels a pending one).
-The registered model advertises `maxTokens: min(131072, ctx)` — the full
-131K window for the daily full-window qwen3.x profile (heavy-thinking
-models at xhigh emit thinking blocks past the old 32K default and get
-truncated), and the launched ctx for smaller windows.
+The registered model advertises `maxTokens: min(32768, floor(ctx/2))` — a
+32K cap at half the window. The advertised value becomes the adapter's
+`defaultMaxTokens`, which compaction uses as its reserved output `O` in
+`W - O - headroom`: advertising the whole window (`O = W`) leaves no
+message budget and proactive compaction never fires. Heavy-thinking models
+at xhigh that need more than 32K must raise per-request maxTokens
+explicitly (which honestly moves the compaction threshold earlier).
+Compaction threshold under the core defaults (headroomTokens 65536,
+thresholdRatio 0.8, retainRatio 0.16) is `min(0.8*W, W - O - headroom)` —
+so a 131072 window with O = 32768 thresholds at **32768**, firing at
+~32-38K pressure tokens and looping with xhigh + preserveThinking (every
+turn re-adds 10-20K thinking tokens). A 98304 window is worse: pressure
+exactly 0, proactive compaction disabled entirely. The fix is a per-model
+headroom override in the dsh profile (needs a `dsh web` restart — loader
+normalization happens at boot), NOT a smaller O (that truncates xhigh
+thinking). Compute it with `compactionBudgetFor(ctx).recommendedHeadroom`
+(~6.5K for 131K → ~92K threshold; ~4K floor for 96K → ~61K). Example for
+the Swift 131K route (also add siblings as needed):
+
+```yaml
+- id: compaction-basic
+  name: '@deepseek-ai/dsh-compaction-basic'
+  config:
+    modelPolicies:
+      - provider: local-router
+        model: swift-1-5-qwen3-8-27b
+        headroomTokens: 6554
+```
+
+Shortcut with zero config edits: run the bigger window instead — the
+200K/250K profiles threshold at ~106K/158K under defaults.
 For opencode against the spawned server:
 - the opencode config resolves per directory (project `opencode.json` beats
   nothing; the global `~/.config/opencode/opencode.json(c)` is authoritative