dsh-llama-cpp-sampling-params 0.1.0

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 dsh-llama-cpp-sampling-params contributors
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -0,0 +1,116 @@
+# dsh-llama-cpp-sampling-params
+
+A [DeepSeek Harness](https://github.com/deepseek-ai/deepseek-harness) (`dsh`) plugin that injects **per-model sampling parameters** into every chat-completions request sent to a local **llama.cpp / llama-server** gateway.
+
+## Why
+
+dsh's `LlmCallConfig` only carries `temperature` / `maxTokens` / `stop` — it never sends `top_p`, `top_k`, `min_p`, `repeat_penalty`, `presence_penalty`, or `frequency_penalty`. llama.cpp accepts all of these **per request**. This plugin stamps the matching model's sampling set onto the wire body of chat-completions calls, so you can switch sampling behavior (e.g. a high-temperature creative role vs. a low-temperature coding role) **by switching the model alias** — no model reload, no extra VRAM.
+
+## How it works
+
+The plugin wraps `globalThis.fetch` at the transport layer:
+
+1. Every `fetch()` call in the dsh process flows through the wrapper.
+2. For requests whose URL contains `/v1/chat/completions`, the wrapper parses the JSON body.
+3. It reads `body.model` (the alias), looks it up in the `sampling-params` `models` table.
+4. If found, it stamps the sampling set onto the parsed body and re-serializes `init.body`.
+5. The original fetch sends the modified body.
+
+**Why a fetch wrapper:** dsh's agent-loop requests arrive **deep-frozen** (mutation throws), and the pi-ai adapter does not forward `onPayload` from `GenerateOptions`. Wrapping `fetch` operates below all of those abstractions — it edits the already-built JSON body before it leaves the process, so no dsh internal is touched.
+
+## How roles work
+
+- llama.cpp can serve **one loaded GGUF under several aliases** (e.g. `model-i`, `model-p`, `model-t`), all pointing at the same underlying weights — so switching alias costs **no extra VRAM**.
+- dsh's `llm-pi-ai` lists each alias as a separate selectable model, so switching the model in the UI picks a sampling role.
+- This plugin reads the `model` field of each request (the alias) and looks it up in the `sampling-params` `models` table. **Unconfigured aliases pass through untouched.**
+
+## Zero conflict
+
+- The fields dsh already sends (`temperature` / `maxTokens`) are left to dsh unless a model explicitly sets them.
+- Every other sampling field is one dsh never sends, so there is no competing source — the plugin simply overrides the llama.cpp server default.
+
+## Install
+
+```sh
+# From npm (once published)
+dsh plugin --profile web add dsh-llama-cpp-sampling-params
+
+# From a local directory
+dsh plugin --profile web add link:C:/path/to/dsh-llama-cpp-sampling-params
+```
+
+Then restart `dsh web` (or refresh the GUI page).
+
+## Configure models
+
+Set the per-model sampling table in `$DSH_HOME/settings.yaml`. The section applies **live** — edits take effect without a restart.
+
+```yaml
+sampling-params:
+  models:
+    # Key = the exact model id (alias) sent on the wire.
+    model-t:   # Think
+      temperature: 0.6
+      top_p: 0.95
+      top_k: 20
+      min_p: 0.05
+      repeat_penalty: 1.0
+      presence_penalty: 0.0
+      frequency_penalty: 0.0
+    model-i:   # Instruct
+      temperature: 0.2
+      top_p: 0.8
+      top_k: 20
+      min_p: 0.05
+      repeat_penalty: 1.0
+      presence_penalty: 0.0
+      frequency_penalty: 0.0
+    model-p:   # Planner
+      temperature: 1.0
+      top_p: 0.95
+      top_k: 20
+      min_p: 0.0
+      repeat_penalty: 1.0
+      presence_penalty: 0.0
+      frequency_penalty: 0.0
+```
+
+> Each model is a **complete** sampling set — every field is required, so configure each alias with all seven numbers.
+
+## Ports
+
+This plugin **never hardcodes an LLM port**. It matches on the `/v1/chat/completions` path in the request URL — the host and port come from your dsh provider configuration (e.g. `llm-pi-ai.providers.llama.baseURL`). The examples below use `<host>:<port>` as a placeholder.
+
+## Wire field names
+
+llama.cpp uses snake_case wire names. **Note:** repetition penalty is `repeat_penalty`, **not** `repetition_penalty`.
+
+| Model key | llama.cpp wire field |
+|-----------|----------------------|
+| `temperature` | `temperature` |
+| `top_p` | `top_p` |
+| `top_k` | `top_k` |
+| `min_p` | `min_p` |
+| `repeat_penalty` | `repeat_penalty` |
+| `presence_penalty` | `presence_penalty` |
+| `frequency_penalty` | `frequency_penalty` |
+
+## Verifying the sampling was applied
+
+llama-server's `/slots` endpoint echoes the sampling parameters the server actually adopted. Query it for a model to confirm a request's sampling took effect:
+
+```sh
+curl "http://<host>:<port>/slots?model=<model-id>"
+```
+
+The `params` object of the returned slot reflects `temperature`, `top_p`, `top_k`, `min_p`, `repeat_penalty`, `presence_penalty`, and `frequency_penalty` — a match against the model's configured values confirms the injection reached the server.
+
+## Compatibility
+
+- Requires dsh with `@deepseek-ai/dsh-settings` and `@deepseek-ai/schemastery` (both ship with dsh).
+- Target must be an OpenAI-compatible gateway that honors these wire fields (llama.cpp `llama-server` does).
+- The wrapper is guarded by a `Symbol.for` flag so hot-reload never double-wraps.
+
+## License
+
+MIT

package/README.zh.md

@@ -0,0 +1,116 @@
+# dsh-llama-cpp-sampling-params
+
+一個 [DeepSeek Harness](https://github.com/deepseek-ai/deepseek-harness)（`dsh`）插件，將**每個模型的採樣參數**注入發送到本地 **llama.cpp / llama-server** 閘道的每個 chat-completions 請求。
+
+## 為什麼
+
+dsh 的 `LlmCallConfig` 只帶 `temperature` / `maxTokens` / `stop`——從不發送 `top_p`、`top_k`、`min_p`、`repeat_penalty`、`presence_penalty` 或 `frequency_penalty`。llama.cpp 全部支援這些**按請求**參數。本插件將匹配模型的採樣組蓋印到 chat-completions 呼叫的 wire body 上，讓你**切換模型別名**就能切換採樣行為（例如高溫創意角色 vs. 低溫編碼角色）——無需重新載入模型、不佔額外 VRAM。
+
+## 運作原理
+
+插件在 transport 層 wrap `globalThis.fetch`：
+
+1. dsh process 內每個 `fetch()` 呼叫都經過 wrapper。
+2. URL 包含 `/v1/chat/completions` 的請求，wrapper 解析 JSON body。
+3. 讀取 `body.model`（別名），在 `sampling-params` 的 `models` 表查詢。
+4. 找到就將採樣組蓋印到解析後的 body，重新序列化 `init.body`。
+5. 原始 fetch 發送修改後的 body。
+
+**為什麼用 fetch wrapper：** dsh 的 agent-loop 請求到達時是 **deep-frozen**（mutate 會 throw），且 pi-ai adapter 不會從 `GenerateOptions` 轉發 `onPayload`。Wrap `fetch` 在所有這些抽象之下操作——它直接編輯已組好的 JSON body，不碰任何 dsh 內部。
+
+## 角色如何運作
+
+- llama.cpp 可將**一個已載入的 GGUF 以多個別名**（如 `model-i`、`model-p`、`model-t`）提供，全部指向相同的底層權重——切換別名**不佔額外 VRAM**。
+- dsh 的 `llm-pi-ai` 將每個別名列為可選模型，切換 UI 裡的模型就選定採樣角色。
+- 本插件讀取每個請求的 `model` 欄位（別名），在 `sampling-params` 的 `models` 表查詢。**未配置的別名原樣通過。**
+
+## 零衝突
+
+- dsh 已發送的欄位（`temperature` / `maxTokens`）留給 dsh，除非模型明確設定。
+- 其他採樣欄位是 dsh 從不發送的，因此沒有競爭來源——插件僅覆蓋 llama.cpp server 預設值。
+
+## 安裝
+
+```sh
+# 從 npm（發布後）
+dsh plugin --profile web add dsh-llama-cpp-sampling-params
+
+# 從本地目錄
+dsh plugin --profile web add link:C:/path/to/dsh-llama-cpp-sampling-params
+```
+
+然後重啟 `dsh web`（或刷新 GUI 頁面）。
+
+## 配置模型
+
+在 `$DSH_HOME/settings.yaml` 設定每個模型的採樣表。該區段**即時生效**——修改無需重啟。
+
+```yaml
+sampling-params:
+  models:
+    # 鍵 = 發送到 wire 的精確 model id（別名）。
+    model-t:   # Think
+      temperature: 0.6
+      top_p: 0.95
+      top_k: 20
+      min_p: 0.05
+      repeat_penalty: 1.0
+      presence_penalty: 0.0
+      frequency_penalty: 0.0
+    model-i:   # Instruct
+      temperature: 0.2
+      top_p: 0.8
+      top_k: 20
+      min_p: 0.05
+      repeat_penalty: 1.0
+      presence_penalty: 0.0
+      frequency_penalty: 0.0
+    model-p:   # Planner
+      temperature: 1.0
+      top_p: 0.95
+      top_k: 20
+      min_p: 0.0
+      repeat_penalty: 1.0
+      presence_penalty: 0.0
+      frequency_penalty: 0.0
+```
+
+> 每個模型是**完整**採樣組——每個欄位必填，因此為每個別名配置全部七個數值。
+
+## 連接埠
+
+本插件**從不硬編碼 LLM port**。它匹配請求 URL 中的 `/v1/chat/completions` 路徑——host 和 port 來自你的 dsh provider 配置（如 `llm-pi-ai.providers.llama.baseURL`）。以下範例以 `<host>:<port>` 作為佔位符。
+
+## Wire 欄位名稱
+
+llama.cpp 使用 snake_case wire 名稱。**注意：** 重複懲罰是 `repeat_penalty`，**不是** `repetition_penalty`。
+
+| 模型鍵 | llama.cpp wire 欄位 |
+|--------|---------------------|
+| `temperature` | `temperature` |
+| `top_p` | `top_p` |
+| `top_k` | `top_k` |
+| `min_p` | `min_p` |
+| `repeat_penalty` | `repeat_penalty` |
+| `presence_penalty` | `presence_penalty` |
+| `frequency_penalty` | `frequency_penalty` |
+
+## 驗證採樣已生效
+
+llama-server 的 `/slots` 端點回顯 server 實際採用的採樣參數。查詢模型以確認請求的採樣生效：
+
+```sh
+curl "http://<host>:<port>/slots?model=<model-id>"
+```
+
+返回 slot 的 `params` 物件反映 `temperature`、`top_p`、`top_k`、`min_p`、`repeat_penalty`、`presence_penalty` 和 `frequency_penalty`——與模型配置值相符即確認注入已達 server。
+
+## 相容性
+
+- 需要 dsh 內含 `@deepseek-ai/dsh-settings` 與 `@deepseek-ai/schemastery`（兩者隨 dsh 提供）。
+- 目標必須是遵循這些 wire 欄位的 OpenAI 相容閘道（llama.cpp `llama-server` 符合）。
+- Wrapper 以 `Symbol.for` 標記保護，hot-reload 不會重複 wrap。
+
+## 授權
+
+MIT

package/cordis.patch.yml

@@ -0,0 +1,49 @@
+# dsh-llama-cpp-sampling-params — bundle patch.
+#
+# Registers the plugin on the host. The plugin wraps globalThis.fetch to
+# intercept chat-completions requests and stamp the matching sampling set
+# onto the wire body before it leaves the process.
+# The config block below is the BASE layer: a `sampling-params:` section in
+# $DSH_HOME/settings.yaml overrides it without a restart.
+- insert:
+    - id: sampling-params
+      name: dsh-llama-cpp-sampling-params
+      config:
+        # One entry per model. Key = the exact model id (alias) sent on the
+        # wire (the `model` field in the chat-completions request body).
+        #
+        # The plugin never hardcodes an LLM port — it matches on the
+        # /v1/chat/completions path in the request URL. Host and port come
+        # from dsh's provider baseURL (e.g. llm-pi-ai.providers.llama.baseURL).
+        #
+        # Wire field names (llama.cpp): temperature, top_p, top_k, min_p,
+        # repeat_penalty (NOT repetition_penalty), presence_penalty,
+        # frequency_penalty. Each model is a complete set — all fields required.
+        #
+        # Example (keys are placeholders):
+        #   models:
+        #     model-t:   # Think
+        #       temperature: 0.6
+        #       top_p: 0.95
+        #       top_k: 20
+        #       min_p: 0.05
+        #       repeat_penalty: 1.0
+        #       presence_penalty: 0.0
+        #       frequency_penalty: 0.0
+        #     model-i:   # Instruct
+        #       temperature: 0.2
+        #       top_p: 0.8
+        #       top_k: 20
+        #       min_p: 0.05
+        #       repeat_penalty: 1.0
+        #       presence_penalty: 0.0
+        #       frequency_penalty: 0.0
+        #     model-p:   # Planner
+        #       temperature: 1.0
+        #       top_p: 0.95
+        #       top_k: 20
+        #       min_p: 0.0
+        #       repeat_penalty: 1.0
+        #       presence_penalty: 0.0
+        #       frequency_penalty: 0.0
+        models: {}

package/index.js

@@ -0,0 +1,138 @@
+/**
+ * dsh-llama-cpp-sampling-params
+ *
+ * Injects per-model-alias sampling parameters into every chat-completions
+ * request sent to an OpenAI-compatible local gateway (llama.cpp).
+ *
+ * How models work:
+ *   - llama.cpp serves ONE loaded model under several aliases (e.g.
+ *     `model-i`, `model-p`, `model-t`) that all point at the same underlying
+ *     GGUF — so there is NO extra VRAM cost.
+ *   - dsh's `llm-pi-ai` lists each alias as a separate selectable model, so
+ *     switching model in the UI picks a sampling role.
+ *   - This plugin reads the `model` field of each request (the alias) and
+ *     looks it up in the `sampling-params` `models` table. If the alias is
+ *     configured, it stamps that model's sampling set onto the wire body.
+ *     Unconfigured aliases pass through byte-identical.
+ *
+ * Why a fetch wrapper: dsh's `GenerateOptions` only carries temperature /
+ * maxTokens / stop — it never sends top_p, top_k, min_p, repeat_penalty,
+ * presence_penalty, or frequency_penalty. Moreover, agent-loop requests arrive
+ * deep-frozen (mutation throws) and the pi-ai adapter does not forward
+ * `onPayload` from GenerateOptions. Wrapping `globalThis.fetch` operates at
+ * the transport layer, below all of those abstractions, and simply rewrites
+ * the already-built JSON body before it leaves the process.
+ *
+ * Zero conflict: the fields dsh already sends (temperature / maxTokens) are
+ * left to dsh unless a model explicitly sets them; every other sampling field
+ * is one dsh never sends, so there is no competing source.
+ */
+import z from "@deepseek-ai/schemastery";
+import { settingsNamespace } from "@deepseek-ai/dsh-settings";
+
+const name = "sampling-params";
+const inject = ["settings"];
+
+// llama.cpp wire field names (snake_case). NOTE: repetition penalty is
+// `repeat_penalty` on llama.cpp, NOT `repetition_penalty`.
+const WIRE_KEYS = [
+  "temperature",
+  "top_p",
+  "top_k",
+  "min_p",
+  "repeat_penalty",
+  "presence_penalty",
+  "frequency_penalty",
+];
+
+// One model's full sampling set. Each field is required (a model sets every
+// number explicitly); a model that omits a field is rejected by the schema, so
+// configure each alias with a complete set.
+const Model = z.object({
+  temperature: z.number(),
+  top_p: z.number(),
+  top_k: z.number(),
+  min_p: z.number(),
+  repeat_penalty: z.number(),
+  presence_penalty: z.number(),
+  frequency_penalty: z.number(),
+});
+
+// Plugin config: a table keyed by the exact model id (alias) sent on the
+// wire. Each key's value is the sampling set stamped onto requests that name
+// that alias. Example:
+//   models:
+//     model-i: { temperature: 0.2, top_p: 0.8, ... }
+//     model-p: { temperature: 1.0, ... }
+//     model-t: { temperature: 0.6, ... }
+const Config = z.object({
+  models: z.dict(Model).default({}),
+});
+
+// Mark so we never double-wrap globalThis.fetch (hot-reload safe).
+const WRAPPED = Symbol.for("dsh-llama-cpp-sampling-params.fetch-wrapped");
+
+// The OpenAI-compatible chat-completions path we inject into. Only requests
+// whose URL carries this path are touched; everything else passes through
+// byte-identical.
+const CHAT_COMPLETIONS_PATH = "/v1/chat/completions";
+
+function apply(ctx, config = {}) {
+  const log = ctx.logger("sampling-params");
+
+  // Register the settings section live so role values can be edited without
+  // a restart.
+  const scope = ctx.settings.register(settingsNamespace("sampling-params"), Config, {
+    base: config,
+    applies: "live",
+  });
+
+  // Read the sampling set for one exact model id/alias.
+  const readModel = (modelId) => {
+    try {
+      const section = scope.get();
+      return section?.models?.[modelId];
+    } catch {
+      return undefined;
+    }
+  };
+
+  const registeredModels = (() => {
+    try { return Object.keys(scope.get()?.models ?? {}); } catch { return []; }
+  })();
+  log.info(`sampling-params applied (fetch wrapper); models=${registeredModels.length}`);
+
+  if (typeof globalThis.fetch === "function" && !globalThis.fetch[WRAPPED]) {
+    const originalFetch = globalThis.fetch;
+    globalThis.fetch = async function (input, init) {
+      try {
+        const url = typeof input === "string" ? input : input?.url;
+        const bodyText = typeof init?.body === "string" ? init.body : null;
+        if (
+          typeof url === "string" &&
+          url.includes(CHAT_COMPLETIONS_PATH) &&
+          bodyText !== null
+        ) {
+          const body = JSON.parse(bodyText);
+          if (body && typeof body.model === "string") {
+            const model = readModel(body.model);
+            if (model) {
+              for (const key of WIRE_KEYS) {
+                if (typeof model[key] === "number") body[key] = model[key];
+              }
+              init.body = JSON.stringify(body);
+            }
+          }
+        }
+      } catch (error) {
+        // Never break LLM traffic: on any parse/matching error, send the
+        // request through untouched.
+        log.warn(`sampling-params inject skipped: ${error?.message ?? error}`);
+      }
+      return originalFetch.call(this, input, init);
+    };
+    globalThis.fetch[WRAPPED] = true;
+  }
+}
+
+export { apply, name, inject, Config };

package/package.json

@@ -0,0 +1,47 @@
+{
+  "name": "dsh-llama-cpp-sampling-params",
+  "version": "0.1.0",
+  "description": "DSH plugin: inject per-model sampling parameters (top_p, top_k, min_p, repeat_penalty, presence_penalty, frequency_penalty) into every chat-completions request sent to a local llama.cpp / llama-server gateway.",
+  "author": "devhang",
+  "repository": {
+    "type": "git",
+    "url": "git+https://github.com/devhang/dsh-llama-cpp-sampling-params.git"
+  },
+  "homepage": "https://github.com/devhang/dsh-llama-cpp-sampling-params#readme",
+  "bugs": {
+    "url": "https://github.com/devhang/dsh-llama-cpp-sampling-params/issues"
+  },
+  "type": "module",
+  "main": "index.js",
+  "files": [
+    "index.js",
+    "cordis.patch.yml",
+    "README.md",
+    "README.zh.md",
+    "LICENSE"
+  ],
+  "keywords": [
+    "deepseek-harness",
+    "dsh",
+    "dsh-plugin",
+    "llama.cpp",
+    "llama-server",
+    "sampling",
+    "top_p",
+    "min_p",
+    "repeat_penalty"
+  ],
+  "license": "MIT",
+  "dsh": {
+    "bundle": {
+      "patch": "./cordis.patch.yml"
+    }
+  },
+  "peerDependencies": {
+    "@deepseek-ai/cordis": "^4.0.1",
+    "@deepseek-ai/dsh-settings": ">=0.1.1-rc.1 <0.2.0-0"
+  },
+  "dependencies": {
+    "@deepseek-ai/schemastery": "^3.18.2"
+  }
+}