dsh-llama-cpp-sampling-params 0.1.1 → 0.2.0

package/README.md

@@ -43,38 +43,45 @@ Then restart `dsh web` (or refresh the GUI page).
 
 ## Configure models
 
-Set the per-model sampling table in `$DSH_HOME/settings.yaml`. The section applies **live** — edits take effect without a restart.
+The `models` table lives in your dsh settings. **Where it goes depends on your dsh version:**
+
+| dsh | where the table goes |
+|-----|----------------------|
+| **0.2.x** | the profile's `cordis.patch.yml`, as a top-level `sampling-params` entry |
+| **0.1.x** | `$DSH_HOME/settings.yaml`, under a `sampling-params:` section (applies live) |
+
+### DSH 0.2.x
+
+Add a top-level entry to your profile's `cordis.patch.yml` (edit `cordis.patch.yml`, not `cordis.yml`). The `id: sampling-params` is the binding key and must match the plugin's namespace:
 
 ```yaml
-sampling-params:
-  models:
-    # Key = the exact model id (alias) sent on the wire.
-    model-t:   # Think
-      temperature: 0.6
-      top_p: 0.95
-      top_k: 20
-      min_p: 0.05
-      repeat_penalty: 1.0
-      presence_penalty: 0.0
-      frequency_penalty: 0.0
-    model-i:   # Instruct
-      temperature: 0.2
-      top_p: 0.8
-      top_k: 20
-      min_p: 0.05
-      repeat_penalty: 1.0
-      presence_penalty: 0.0
-      frequency_penalty: 0.0
-    model-p:   # Planner
-      temperature: 1.0
-      top_p: 0.95
-      top_k: 20
-      min_p: 0.0
-      repeat_penalty: 1.0
-      presence_penalty: 0.0
-      frequency_penalty: 0.0
+- id: sampling-params
+  name: dsh-llama-cpp-sampling-params
+  config:
+    models:
+      # Key = the exact model id (alias) sent on the wire.
+      model-t:   # Think
+        temperature: 0.6
+        top_p: 0.95
+        top_k: 20
+        min_p: 0.05
+        repeat_penalty: 1.0
+        presence_penalty: 0.0
+        frequency_penalty: 0.0
+      model-i:   # Instruct
+        temperature: 0.2
+        top_p: 0.8
+        top_k: 20
+        min_p: 0.05
+        repeat_penalty: 1.0
+        presence_penalty: 0.0
+        frequency_penalty: 0.0
 ```
 
+### Migrating from 0.1.x to 0.2.x
+
+Upgrading dsh to 0.2.x renames `$DSH_HOME/settings.yaml` to `settings.yaml.imported` (a frozen backup, no longer read). Your `sampling-params.models` table is stranded there. To migrate, copy the `models:` block out of `settings.yaml.imported` and add it as the `sampling-params` entry in your profile's `cordis.patch.yml`, re-indenting the block by **+2 spaces** to nest under `config:`. The one-time auto-import does not re-run, so do this by hand (or via the settings UI).
+
 > Each model is a **complete** sampling set — every field is required, so configure each alias with all seven numbers.
 
 ## Ports
@@ -107,6 +114,7 @@ The `params` object of the returned slot reflects `temperature`, `top_p`, `top_k
 
 ## Compatibility
 
+- Supports dsh **0.1.x and 0.2.x** — the settings read path is version-adaptive (`settings.register` on 0.1.x, the profile-entry config on 0.2.x).
 - Requires dsh with `@deepseek-ai/dsh-settings` and `@deepseek-ai/schemastery` (both ship with dsh).
 - Target must be an OpenAI-compatible gateway that honors these wire fields (llama.cpp `llama-server` does).
 - The wrapper is guarded by a `Symbol.for` flag so hot-reload never double-wraps.

package/README.zh.md

@@ -43,38 +43,45 @@ dsh plugin --profile web add link:C:/path/to/dsh-llama-cpp-sampling-params
 
 ## 配置模型
 
-在 `$DSH_HOME/settings.yaml` 設定每個模型的採樣表。該區段**即時生效**——修改無需重啟。
+`models` 表放在你的 dsh 設定裡。**放哪裡取決於你的 dsh 版本：**
+
+| dsh | 表放哪裡 |
+|-----|---------|
+| **0.2.x** | profile 的 `cordis.patch.yml`，作為頂層 `sampling-params` entry |
+| **0.1.x** | `$DSH_HOME/settings.yaml` 的 `sampling-params:` 區段（即時生效） |
+
+### DSH 0.2.x
+
+在 profile 的 `cordis.patch.yml` 加一個頂層 entry（改 `cordis.patch.yml`，不是 `cordis.yml`）。`id: sampling-params` 是 binding key，必須與插件 namespace 相符：
 
 ```yaml
-sampling-params:
-  models:
-    # 鍵 = 發送到 wire 的精確 model id（別名）。
-    model-t:   # Think
-      temperature: 0.6
-      top_p: 0.95
-      top_k: 20
-      min_p: 0.05
-      repeat_penalty: 1.0
-      presence_penalty: 0.0
-      frequency_penalty: 0.0
-    model-i:   # Instruct
-      temperature: 0.2
-      top_p: 0.8
-      top_k: 20
-      min_p: 0.05
-      repeat_penalty: 1.0
-      presence_penalty: 0.0
-      frequency_penalty: 0.0
-    model-p:   # Planner
-      temperature: 1.0
-      top_p: 0.95
-      top_k: 20
-      min_p: 0.0
-      repeat_penalty: 1.0
-      presence_penalty: 0.0
-      frequency_penalty: 0.0
+- id: sampling-params
+  name: dsh-llama-cpp-sampling-params
+  config:
+    models:
+      # 鍵 = 發送到 wire 的精確 model id（別名）。
+      model-t:   # Think
+        temperature: 0.6
+        top_p: 0.95
+        top_k: 20
+        min_p: 0.05
+        repeat_penalty: 1.0
+        presence_penalty: 0.0
+        frequency_penalty: 0.0
+      model-i:   # Instruct
+        temperature: 0.2
+        top_p: 0.8
+        top_k: 20
+        min_p: 0.05
+        repeat_penalty: 1.0
+        presence_penalty: 0.0
+        frequency_penalty: 0.0
 ```
 
+### 從 0.1.x 遷移到 0.2.x
+
+升級 dsh 到 0.2.x 會把 `$DSH_HOME/settings.yaml` 改名成 `settings.yaml.imported`（凍結備份，不再讀取），你的 `sampling-params.models` 表就擱淺在那裡。要遷移：把 `models:` 區塊從 `settings.yaml.imported` 抽出，加進 profile 的 `cordis.patch.yml` 作為 `sampling-params` entry，並將該區塊**+2 空格**縮排以嵌在 `config:` 下。一次性自動 import 不會重跑，所以要手動搬（或經設定 UI）。
+
 > 每個模型是**完整**採樣組——每個欄位必填，因此為每個別名配置全部七個數值。
 
 ## 連接埠
@@ -107,6 +114,7 @@ curl "http://<host>:<port>/slots?model=<model-id>"
 
 ## 相容性
 
+- 支援 dsh **0.1.x 與 0.2.x**——設定讀取路徑是 version-adaptive（0.1.x 用 `settings.register`、0.2.x 用 profile-entry config）。
 - 需要 dsh 內含 `@deepseek-ai/dsh-settings` 與 `@deepseek-ai/schemastery`（兩者隨 dsh 提供）。
 - 目標必須是遵循這些 wire 欄位的 OpenAI 相容閘道（llama.cpp `llama-server` 符合）。
 - Wrapper 以 `Symbol.for` 標記保護，hot-reload 不會重複 wrap。

package/index.js

@@ -1,140 +1,178 @@
-/**
- * dsh-llama-cpp-sampling-params
- *
- * Injects per-model-alias sampling parameters into every chat-completions
- * request sent to an OpenAI-compatible local gateway (llama.cpp).
- *
- * How models work:
- *   - llama.cpp serves ONE loaded model under several aliases (e.g.
- *     `model-i`, `model-p`, `model-t`) that all point at the same underlying
- *     GGUF — so there is NO extra VRAM cost.
- *   - dsh's `llm-pi-ai` lists each alias as a separate selectable model, so
- *     switching model in the UI picks a sampling role.
- *   - This plugin reads the `model` field of each request (the alias) and
- *     looks it up in the `sampling-params` `models` table. If the alias is
- *     configured, it stamps that model's sampling set onto the wire body.
- *     Unconfigured aliases pass through byte-identical.
- *
- * Why a fetch wrapper: dsh's `GenerateOptions` only carries temperature /
- * maxTokens / stop — it never sends top_p, top_k, min_p, repeat_penalty,
- * presence_penalty, or frequency_penalty. Moreover, agent-loop requests arrive
- * deep-frozen (mutation throws) and the pi-ai adapter does not forward
- * `onPayload` from GenerateOptions. Wrapping `globalThis.fetch` operates at
- * the transport layer, below all of those abstractions, and simply rewrites
- * the already-built JSON body before it leaves the process.
- *
- * Zero conflict: the fields dsh already sends (temperature / maxTokens) are
- * left to dsh unless a model explicitly sets them; every other sampling field
- * is one dsh never sends, so there is no competing source.
- */
-import z from "@deepseek-ai/schemastery";
-
-const name = "sampling-params";
-const inject = ["settings"];
-
-// llama.cpp wire field names (snake_case). NOTE: repetition penalty is
-// `repeat_penalty` on llama.cpp, NOT `repetition_penalty`.
-const WIRE_KEYS = [
-  "temperature",
-  "top_p",
-  "top_k",
-  "min_p",
-  "repeat_penalty",
-  "presence_penalty",
-  "frequency_penalty",
-];
-
-// One model's full sampling set. Each field is required (a model sets every
-// number explicitly); a model that omits a field is rejected by the schema, so
-// configure each alias with a complete set.
-const Model = z.object({
-  temperature: z.number(),
-  top_p: z.number(),
-  top_k: z.number(),
-  min_p: z.number(),
-  repeat_penalty: z.number(),
-  presence_penalty: z.number(),
-  frequency_penalty: z.number(),
-});
-
-// Plugin config: a table keyed by the exact model id (alias) sent on the
-// wire. Each key's value is the sampling set stamped onto requests that name
-// that alias. Example:
-//   models:
-//     model-i: { temperature: 0.2, top_p: 0.8, ... }
-//     model-p: { temperature: 1.0, ... }
-//     model-t: { temperature: 0.6, ... }
-const Config = z.object({
-  models: z.dict(Model).default({}),
-});
-
-// Mark so we never double-wrap globalThis.fetch (hot-reload safe).
-const WRAPPED = Symbol.for("dsh-llama-cpp-sampling-params.fetch-wrapped");
-
-// The OpenAI-compatible chat-completions path we inject into. Only requests
-// whose URL carries this path are touched; everything else passes through
-// byte-identical.
-const CHAT_COMPLETIONS_PATH = "/v1/chat/completions";
-
-function apply(ctx, config = {}) {
-  const log = ctx.logger("sampling-params");
-
-  // Register the settings section live so role values can be edited without
-  // a restart. NOTE: we pass the namespace as a plain string, not via
-  // `settingsNamespace()` from @deepseek-ai/dsh-settings — the export was
-  // added in 0.1.1 and is absent in some older/newer versions, so importing
-  // it directly breaks plugin load on those dsh builds.
-  const scope = ctx.settings.register("sampling-params", Config, {
-    base: config,
-    applies: "live",
-  });
-
-  // Read the sampling set for one exact model id/alias.
-  const readModel = (modelId) => {
-    try {
-      const section = scope.get();
-      return section?.models?.[modelId];
-    } catch {
-      return undefined;
-    }
-  };
-
-  const registeredModels = (() => {
-    try { return Object.keys(scope.get()?.models ?? {}); } catch { return []; }
-  })();
-  log.info(`sampling-params applied (fetch wrapper); models=${registeredModels.length}`);
-
-  if (typeof globalThis.fetch === "function" && !globalThis.fetch[WRAPPED]) {
-    const originalFetch = globalThis.fetch;
-    globalThis.fetch = async function (input, init) {
-      try {
-        const url = typeof input === "string" ? input : input?.url;
-        const bodyText = typeof init?.body === "string" ? init.body : null;
-        if (
-          typeof url === "string" &&
-          url.includes(CHAT_COMPLETIONS_PATH) &&
-          bodyText !== null
-        ) {
-          const body = JSON.parse(bodyText);
-          if (body && typeof body.model === "string") {
-            const model = readModel(body.model);
-            if (model) {
-              for (const key of WIRE_KEYS) {
-                if (typeof model[key] === "number") body[key] = model[key];
-              }
-              init.body = JSON.stringify(body);
-            }
-          }
-        }
-      } catch (error) {
-        // Never break LLM traffic: on any parse/matching error, send the
-        // request through untouched.
-        log.warn(`sampling-params inject skipped: ${error?.message ?? error}`);
-      }
-      return originalFetch.call(this, input, init);
-    };
-    globalThis.fetch[WRAPPED] = true;
-  }
-}
-
-export { apply, name, inject, Config };
+/**
+ * dsh-llama-cpp-sampling-params
+ *
+ * Injects per-model-alias sampling parameters into every chat-completions
+ * request sent to an OpenAI-compatible local gateway (llama.cpp).
+ *
+ * How models work:
+ *   - llama.cpp serves ONE loaded model under several aliases (e.g.
+ *     `model-i`, `model-p`, `model-t`) that all point at the same underlying
+ *     GGUF — so there is NO extra VRAM cost.
+ *   - dsh's `llm-pi-ai` lists each alias as a separate selectable model, so
+ *     switching model in the UI picks a sampling role.
+ *   - This plugin reads the `model` field of each request (the alias) and
+ *     looks it up in the `sampling-params` `models` table. If the alias is
+ *     configured, it stamps that model's sampling set onto the wire body.
+ *     Unconfigured aliases pass through byte-identical.
+ *
+ * Why a fetch wrapper: dsh's `GenerateOptions` only carries temperature /
+ * maxTokens / stop — it never sends top_p, top_k, min_p, repeat_penalty,
+ * presence_penalty, or frequency_penalty. Moreover, agent-loop requests arrive
+ * deep-frozen (mutation throws) and the pi-ai adapter does not forward
+ * `onPayload` from GenerateOptions. Wrapping `globalThis.fetch` operates at
+ * the transport layer, below all of those abstractions, and simply rewrites
+ * the already-built JSON body before it leaves the process.
+ *
+ * Config source across DSH versions: the `models` table is the plugin's
+ * settings. On DSH 0.1.x the plugin registers that section with
+ * `ctx.settings.register(...)` and reads it live via the returned scope. On
+ * 0.2.x the settings service was rebuilt around a profile-entry model
+ * (`describe` / `mutate` / `configure`) and no longer exposes `register`; a
+ * settings edit restarts the plugin fiber and re-runs `apply(ctx, config)`
+ * with the new config, so the plugin caches that config in a module-level
+ * `liveConfig` and reads it there. `apply` detects which path is available at
+ * runtime, so the same code works on both.
+ *
+ * Zero conflict: the fields dsh already sends (temperature / maxTokens) are
+ * left to dsh unless a model explicitly sets them; every other sampling field
+ * is one dsh never sends, so there is no competing source.
+ */
+import z from "@deepseek-ai/schemastery";
+
+const name = "sampling-params";
+const inject = ["settings"];
+
+// llama.cpp wire field names (snake_case). NOTE: repetition penalty is
+// `repeat_penalty` on llama.cpp, NOT `repetition_penalty`.
+const WIRE_KEYS = [
+  "temperature",
+  "top_p",
+  "top_k",
+  "min_p",
+  "repeat_penalty",
+  "presence_penalty",
+  "frequency_penalty",
+];
+
+// One model's full sampling set. Each field is required (a model sets every
+// number explicitly); a model that omits a field is rejected by the schema, so
+// configure each alias with a complete set.
+const Model = z.object({
+  temperature: z.number(),
+  top_p: z.number(),
+  top_k: z.number(),
+  min_p: z.number(),
+  repeat_penalty: z.number(),
+  presence_penalty: z.number(),
+  frequency_penalty: z.number(),
+});
+
+// Plugin config: a table keyed by the exact model id (alias) sent on the
+// wire. Each key's value is the sampling set stamped onto requests that name
+// that alias. Example:
+//   models:
+//     model-i: { temperature: 0.2, top_p: 0.8, ... }
+//     model-p: { temperature: 1.0, ... }
+//     model-t: { temperature: 0.6, ... }
+const Config = z.object({
+  models: z.dict(Model).default({}),
+});
+
+// Mark so we never double-wrap globalThis.fetch (hot-reload safe).
+const WRAPPED = Symbol.for("dsh-llama-cpp-sampling-params.fetch-wrapped");
+
+// The OpenAI-compatible chat-completions path we inject into. Only requests
+// whose URL carries this path are touched; everything else passes through
+// byte-identical.
+const CHAT_COMPLETIONS_PATH = "/v1/chat/completions";
+
+// Module-level live config, refreshed on every apply() call. Under DSH 0.2.x a
+// settings edit restarts the plugin fiber and re-runs apply() with the new
+// config, so this always reflects the latest `models` table. Under 0.1.x the
+// registered settings scope (created inside apply below) is the live source
+// instead and this only holds the initial value.
+let liveConfig = { models: {} };
+
+function apply(ctx, config = {}) {
+  const log = ctx.logger("sampling-params");
+  liveConfig = config;
+
+  // Register a live settings scope on DSH 0.1.x, where `ctx.settings.register`
+  // exists. On 0.2.x that method is gone (the settings service moved to a
+  // profile-entry model: describe / mutate / configure), so scope stays null
+  // and readModel below falls back to the config that 0.2.x re-passes to apply
+  // on every restart. The namespace is passed as a plain string (not via
+  // `settingsNamespace()`) so importing it never breaks plugin load.
+  let scope = null;
+  const settings = ctx.settings;
+  if (settings && typeof settings.register === "function") {
+    try {
+      scope = settings.register("sampling-params", Config, { base: config, applies: "live" });
+    } catch (error) {
+      log.warn(`sampling-params settings.register failed; using config param: ${error?.message ?? error}`);
+      scope = null;
+    }
+  }
+
+  // Read the sampling set for one exact model id/alias. Prefer the live scope
+  // (0.1.x); otherwise read the latest config passed to apply (0.2.x restart).
+  const readModel = (modelId) => {
+    if (scope) {
+      try {
+        const model = scope.get()?.models?.[modelId];
+        if (model) return model;
+      } catch {
+        // fall through to the config-param path
+      }
+    }
+    try {
+      return liveConfig?.models?.[modelId];
+    } catch {
+      return undefined;
+    }
+  };
+
+  const registeredModels = (() => {
+    try {
+      const models = scope ? (scope.get()?.models ?? {}) : (liveConfig?.models ?? {});
+      return Object.keys(models);
+    } catch {
+      return [];
+    }
+  })();
+  log.info(`sampling-params applied (fetch wrapper); models=${registeredModels.length}`);
+
+  if (typeof globalThis.fetch === "function" && !globalThis.fetch[WRAPPED]) {
+    const originalFetch = globalThis.fetch;
+    globalThis.fetch = async function (input, init) {
+      try {
+        const url = typeof input === "string" ? input : input?.url;
+        const bodyText = typeof init?.body === "string" ? init.body : null;
+        if (
+          typeof url === "string" &&
+          url.includes(CHAT_COMPLETIONS_PATH) &&
+          bodyText !== null
+        ) {
+          const body = JSON.parse(bodyText);
+          if (body && typeof body.model === "string") {
+            const model = readModel(body.model);
+            if (model) {
+              for (const key of WIRE_KEYS) {
+                if (typeof model[key] === "number") body[key] = model[key];
+              }
+              init.body = JSON.stringify(body);
+            }
+          }
+        }
+      } catch (error) {
+        // Never break LLM traffic: on any parse/matching error, send the
+        // request through untouched.
+        log.warn(`sampling-params inject skipped: ${error?.message ?? error}`);
+      }
+      return originalFetch.call(this, input, init);
+    };
+    globalThis.fetch[WRAPPED] = true;
+  }
+}
+
+export { apply, name, inject, Config };

package/package.json

@@ -1,47 +1,47 @@
-{
-  "name": "dsh-llama-cpp-sampling-params",
-  "version": "0.1.1",
-  "description": "DSH plugin: inject per-model sampling parameters (top_p, top_k, min_p, repeat_penalty, presence_penalty, frequency_penalty) into every chat-completions request sent to a local llama.cpp / llama-server gateway.",
-  "author": "devhang",
-  "repository": {
-    "type": "git",
-    "url": "git+https://github.com/devhang/dsh-llama-cpp-sampling-params.git"
-  },
-  "homepage": "https://github.com/devhang/dsh-llama-cpp-sampling-params#readme",
-  "bugs": {
-    "url": "https://github.com/devhang/dsh-llama-cpp-sampling-params/issues"
-  },
-  "type": "module",
-  "main": "index.js",
-  "files": [
-    "index.js",
-    "cordis.patch.yml",
-    "README.md",
-    "README.zh.md",
-    "LICENSE"
-  ],
-  "keywords": [
-    "deepseek-harness",
-    "dsh",
-    "dsh-plugin",
-    "llama.cpp",
-    "llama-server",
-    "sampling",
-    "top_p",
-    "min_p",
-    "repeat_penalty"
-  ],
-  "license": "MIT",
-  "dsh": {
-    "bundle": {
-      "patch": "./cordis.patch.yml"
-    }
-  },
-  "peerDependencies": {
-    "@deepseek-ai/cordis": "^4.0.1",
-    "@deepseek-ai/dsh-settings": ">=0.1.1-rc.1 <0.2.0-0"
-  },
-  "dependencies": {
-    "@deepseek-ai/schemastery": "^3.18.2"
-  }
-}
+{
+  "name": "dsh-llama-cpp-sampling-params",
+  "version": "0.2.0",
+  "description": "DSH plugin: inject per-model sampling parameters (top_p, top_k, min_p, repeat_penalty, presence_penalty, frequency_penalty) into every chat-completions request sent to a local llama.cpp / llama-server gateway.",
+  "author": "devhang",
+  "repository": {
+    "type": "git",
+    "url": "git+https://github.com/devhang/dsh-llama-cpp-sampling-params.git"
+  },
+  "homepage": "https://github.com/devhang/dsh-llama-cpp-sampling-params#readme",
+  "bugs": {
+    "url": "https://github.com/devhang/dsh-llama-cpp-sampling-params/issues"
+  },
+  "type": "module",
+  "main": "index.js",
+  "files": [
+    "index.js",
+    "cordis.patch.yml",
+    "README.md",
+    "README.zh.md",
+    "LICENSE"
+  ],
+  "keywords": [
+    "deepseek-harness",
+    "dsh",
+    "dsh-plugin",
+    "llama.cpp",
+    "llama-server",
+    "sampling",
+    "top_p",
+    "min_p",
+    "repeat_penalty"
+  ],
+  "license": "MIT",
+  "dsh": {
+    "bundle": {
+      "patch": "./cordis.patch.yml"
+    }
+  },
+  "peerDependencies": {
+    "@deepseek-ai/cordis": "^4.0.1",
+    "@deepseek-ai/dsh-settings": ">=0.1.1-rc.1 <0.3.0-0"
+  },
+  "dependencies": {
+    "@deepseek-ai/schemastery": "^3.18.2"
+  }
+}