@goodandready/dsh-model-sync 0.5.16 → 0.5.18

package/README.md

@@ -78,6 +78,9 @@ graph LR
 
 ## ✨ Key Features
 
+* 🏷️ **Smart Collections & Catalog Auto-Tagging**: Automatically categorizes models into standardized tags (`coding`, `reasoning`, `fast`, `vision`, `long-context`) during discovery, provides fast pill-button filtering in the Web UI, and introduces a one-click "⭐ Curated Best" selection preset for top flagship models.
+* ⚡ **Latency TTFT Radar & Sparklines**: Probes response latency and Time To First Token (TTFT) across models, renders ASCII sparklines (` ▂▃▂ `) and 4-tier speed badges (⚡ `< 300ms`, 🟢 `< 1.2s`, 🟡 `1.5 - 3.0s`, 🔴 `> 3.0s`), and exposes `GET /dsh-model-sync/health/latency` for intelligent downstream routing.
+* 💸 **Dynamic Live Pricing Sync & Prompt Caching**: Periodic background updates of live model pricing from OpenRouter and LiteLLM snapshots with persistent disk fallback cache, tracking prompt cache read/write costs (`cacheReadPerToken`, `cacheWritePerToken`), with `GET /dsh-model-sync/pricing` and `POST /dsh-model-sync/pricing/sync` endpoints.
 * 💰 **Built-in Reference Pricing Database**: Canonical token pricing database automatically enriches models for OpenAI, Anthropic, Groq, Mistral, xAI, and DeepSeek, enabling cost-limit filtering and `@cheap` auto-resolution even when upstream APIs omit pricing metadata.
 * 🎯 **Dynamic Smart Virtual Aliases & Auto-Fallback**: Route requests to virtual aliases like `@fast`, `@cheap`, and `@best-code`, with automatic fallback chains when circuit breakers open or health probes fail.
 * 📏 **Real Context Limits & Boundary Safety (`contextWindow` / `maxOutputTokens`)**: precise separation of input and generation limits, prevention of 400 `max_tokens` errors via auto-clamping, and clear `In: 128k | Out: 8k` badges in Web UI.
@@ -165,6 +168,24 @@ dsh-model-sync:
 | `concurrency` | `number` | `4` | Maximum parallel provider synchronization workers |
 | `providers` | `array` | `[]` | Per-provider configuration list (`provider`, `enabled`, `autoApply`) |
 
+---
+
+### 🔍 Active Capability Probing & Verification
+- **Micro-Probes**: Accurately test actual model capabilities rather than relying on declared flags:
+  - `JSON Schema`: Tests structured output generation (`response_format: { type: "json_schema" }`).
+  - `Tools / Functions`: Tests function call generation and parameter handling.
+  - `Vision`: Tests multimodal processing with lightweight image payloads.
+  - `Developer Role`: Validates `developer` role prompt acceptance for advanced reasoning models.
+- **UI Actions**: One-click "🔍 Verify Capabilities" on any provider card, plus global "🔍 Verify All Models" and "⚡ Verify New Only" buttons in catalog toolbar.
+- **Verified Badges**: Highlights confirmed features with green indicators (`✔ Tools`, `✔ Vision`, `✔ JSON`, `✔ DevRole`) persisted across restarts.
+
+### 🦙 Ollama & Local Engine Hub
+- **Direct Web Management**: Pull, monitor, and manage local Ollama models directly from DSH Web UI without SSH.
+- **Streaming Pull Progress**: Real-time layer download progress bar with speed and transferred byte metrics.
+- **GPU VRAM Management**: Displays active models in VRAM (`/api/ps`) and provides instant "⏏ Unload from VRAM" (`keep_alive: 0`) to free GPU resources.
+- **Architecture & Quantization**: Displays quantization levels (`Q4_K_M`, `Q8_0`), family, context windows, and offers direct deletion.
+
+
 ---
 
 ## 📄 License

package/README.ru.md

@@ -78,6 +78,9 @@ graph LR
 
 ## ✨ Ключевые возможности
 
+* 🏷️ **Смарт-коллекции и авто-тегирование каталога**: Автоматическое присвоение тегов моделям (`coding`, `reasoning`, `fast`, `vision`, `long-context`) при синхронизации, фильтрация по тегам pill-кнопками в Web UI и пресет «⭐ Curated Best» для выбора лучших флагманских моделей в один клик.
+* ⚡ **Радар задержек (Latency & TTFT) и спарклайны**: Измерение времени ответа и Time To First Token (TTFT), визуализация спарклайнами (` ▂▃▂ `) с 4 градациями скорости (⚡ `< 300мс`, 🟢 `< 1.2с`, 🟡 `1.5 - 3.0с`, 🔴 `> 3.0с`) и эндпоинт `GET /dsh-model-sync/health/latency` для агентов-маршрутизаторов.
+* 💸 **Динамическая синхронизация цен и учёт Prompt Caching**: Фоновое обновление актуальных тарифов из OpenRouter и LiteLLM с дисковым fallback-кэшем, поддержка скидок кэширования промптов (`cacheReadPerToken`, `cacheWritePerToken`) и эндпоинты `GET /dsh-model-sync/pricing`, `POST /dsh-model-sync/pricing/sync`.
 * 💰 **Встроенная база эталонных цен токенов**: Автоматическое обогащение моделей тарифами для OpenAI, Anthropic, Groq, Mistral, xAI и DeepSeek, обеспечивающее работу лимитов цен и псевдонима `@cheap` даже при отсутствии метаданных в API провайдера.
 * 🎯 **Динамические «умные» псевдонимы и авто-фоллбэк**: Поддержка виртуальных алиасов (`@fast`, `@cheap`, `@best-code`) и цепочек резервирования с автопереключением при размыкании Circuit Breaker.
 * 📏 **Реальные лимиты контекста и авто-клампинг (`contextWindow` / `maxOutputTokens`)**: точное разделение входного и выходного лимита токенов, защита от ошибок 400 `max_tokens` и индикация `Вход: 128k | Выход: 8k` в Web UI.
@@ -165,6 +168,24 @@ dsh-model-sync:
 | `concurrency` | `number` | `4` | Максимальное количество параллельных задач синхронизации |
 | `providers` | `array` | `[]` | Список индивидуальных настроек провайдеров (`provider`, `enabled`, `autoApply`) |
 
+---
+
+### 🔍 Активное аппаратное зондирование возможностей (Capability Probing)
+- **Микро-зонды**: Реальная проверка поддержки возможностей модели вместо слепого доверия флагам провайдера:
+  - `JSON Schema`: Проверка структурированного JSON-вывода (`response_format: { type: "json_schema" }`).
+  - `Tools / Functions`: Проверка генерации вызовов инструментов (function calling).
+  - `Vision`: Проверка распознавания мультимодальных изображений.
+  - `Developer Role`: Проверка поддержки системной роли `developer` для reasoning-моделей.
+- **Кнопки в Web UI**: Отдельная кнопка «🔍 Проверить возможности» у каждого провайдера, а также кнопки «🔍 Проверить все модели» и «⚡ Только новые модели» в шапке каталога.
+- **Бейджи верификации**: Зелёные бейджи (`✔ Tools`, `✔ Vision`, `✔ JSON`, `✔ DevRole`) сохраняются на диске и восстанавливаются при перезапуске.
+
+### 🦙 Центр управления Ollama и локальными моделями
+- **Управление прямо из Web UI**: Загрузка, удаление и контроль локальных моделей Ollama без подключения по SSH.
+- **Живой прогресс загрузки**: Индикатор скачивания слоёв модели с процентами и объёмом данных.
+- **Управление видеопамятью (GPU VRAM)**: Отображение запущенных в VRAM моделей (`/api/ps`) и быстрая кнопка «⏏ Выгрузить из VRAM» (`keep_alive: 0`) для мгновенного освобождения GPU.
+- **Квантование и параметры**: Просмотр квантования (`Q4_K_M`, `Q8_0`), семейства архитектуры, контекстного окна и удаление ненужных моделей.
+
+
 ---
 
 ## 📄 Лицензия

package/README.zh.md

@@ -78,6 +78,9 @@ graph LR
 
 ## ✨ 核心特性
 
+* 🏷️ **智能集合与模型自动标签**：同步时自动识别并标记模型分类标签（`coding` 编程、`reasoning` 思考推理、`fast` 极速轻量、`vision` 多模态、`long-context` 128k+ 长上下文），支持 UI 药丸胶囊快速筛选与一键精选顶级旗舰预设（"⭐ 精选最佳"）。
+* ⚡ **延迟雷达 (TTFT) 与火花线可视化**：实时测量模型响应时间与首字延迟 (TTFT)，提供火花线走势（` ▂▃▂ `）及 4 档速度徽章（⚡ `< 300ms`、🟢 `< 1.2s`、🟡 `1.5 - 3.0s`、🔴 `> 3.0s`），并开放 `GET /dsh-model-sync/health/latency` 路由接口。
+* 💸 **实时价格动态后台同步与提示词缓存**：定期从 OpenRouter 和 LiteLLM 获取最新模型单价，支持本地磁盘降级缓存，支持提示词缓存读写定价（`cacheReadPerToken`、`cacheWritePerToken`），并提供 `GET /dsh-model-sync/pricing` 和 `POST /dsh-model-sync/pricing/sync` 端点。
 * 💰 **内置基准价格数据库**：自动为 OpenAI、Anthropic、Groq、Mistral、xAI 和 DeepSeek 模型补充 Token 价格数据，即使服务商接口未返回价格，也可无缝支持成本上限过滤与 `@cheap` 别名解析。
 * 🎯 **动态智能虚拟别名与故障转移**：支持 `@fast`、`@cheap`、`@best-code` 等虚拟别名及故障转移链，在熔断器打开或健康检查失败时自动切换至备用模型。
 * 📏 **真实上下文限制与边界安全 (`contextWindow` / `maxOutputTokens`)**：精确区分输入与输出令牌限制，通过自动限制防止 400 `max_tokens` 错误，并在 Web UI 中显示 `输入: 128k | 输出: 8k` 徽章。
@@ -165,6 +168,24 @@ dsh-model-sync:
 | `concurrency` | `number` | `4` | 最大并发服务商同步任务数 |
 | `providers` | `array` | `[]` | 服务商独立配置列表（`provider`、`enabled`、`autoApply`） |
 
+---
+
+### 🔍 主动硬件能力探测 (Capability Probing)
+- **微探针测试**：真实验证模型能力，杜绝虚标：
+  - `JSON Schema`：测试结构化 JSON 生成。
+  - `Tools / Functions`：测试函数与工具调用能力。
+  - `Vision`：测试多模态图像理解。
+  - `Developer Role`：测试推理模型的 `developer` 角色支持。
+- **Web UI 操作**：服务商卡片专属“🔍 验证能力”按钮，全局工具栏支持“🔍 验证所有模型”与“⚡ 仅验证新模型”。
+- **能力认证徽章**：绿色认证标识（`✔ Tools`, `✔ Vision`, `✔ JSON`, `✔ DevRole`）持久化存储。
+
+### 🦙 Ollama 本地模型管理中心
+- **Web 端直接管理**：无需登录终端即可在 DSH 界面直接拉取、查看与删除 Ollama 本地模型。
+- **实时下载进度条**：流式展示层下载百分比与传输大小。
+- **显存调度 (VRAM)**：监控显存中活跃模型（`/api/ps`），支持一键“⏏ 释放显存” (`keep_alive: 0`)。
+- **量化与架构信息**：清晰展示量化格式（`Q4_K_M`, `Q8_0`）、模型系列及真实上下文窗口。
+
+
 ---
 
 ## 📄 开源协议

package/lib/adapter-registry.js

@@ -60,6 +60,7 @@ const PROVIDER_ADAPTERS = Object.freeze({
   moonshotai: { endpoint: 'https://api.moonshot.ai/v1/models', auth: 'bearer' },
   'moonshotai-cn': { endpoint: 'https://api.moonshot.cn/v1/models', auth: 'bearer' },
   nvidia: { endpoint: 'https://integrate.api.nvidia.com/v1/models', auth: 'bearer' },
+  ollama: { endpoint: 'http://127.0.0.1:11434/v1/models', auth: 'none' },
   'qwen-token-plan': { endpoint: 'https://token-plan.ap-southeast-1.maas.aliyuncs.com/compatible-mode/v1/models', auth: 'bearer' },
   'qwen-token-plan-cn': { endpoint: 'https://token-plan.cn-beijing.maas.aliyuncs.com/compatible-mode/v1/models', auth: 'bearer' },
   together: { endpoint: 'https://api.together.ai/v1/models', auth: 'bearer' },

package/lib/capability-prober.js

@@ -0,0 +1,339 @@
+import fs from 'node:fs'
+import path from 'node:path'
+
+export const PROBE_TYPES = Object.freeze(['jsonSchema', 'tools', 'vision', 'developerRole'])
+
+export const PROBE_PAYLOADS = Object.freeze({
+  jsonSchema: {
+    name: 'jsonSchema',
+    buildRequest: (model) => ({
+      model,
+      messages: [{ role: 'user', content: 'Respond with valid JSON {"ok": true}' }],
+      response_format: {
+        type: 'json_schema',
+        json_schema: {
+          name: 'ping_schema',
+          strict: true,
+          schema: {
+            type: 'object',
+            properties: { ok: { type: 'boolean' } },
+            required: ['ok'],
+            additionalProperties: false,
+          },
+        },
+      },
+      max_tokens: 16,
+    }),
+    validateResponse: (res) => {
+      if (!res) return false
+      try {
+        const text = typeof res === 'string'
+          ? res
+          : (res?.choices?.[0]?.message?.content || res?.content || res?.text || '')
+        const parsed = typeof text === 'string' ? JSON.parse(text.trim()) : text
+        return parsed && parsed.ok === true
+      } catch {
+        return false
+      }
+    },
+  },
+  tools: {
+    name: 'tools',
+    buildRequest: (model) => ({
+      model,
+      messages: [{ role: 'user', content: 'Call ping_tool with ping="1"' }],
+      tools: [
+        {
+          type: 'function',
+          function: {
+            name: 'ping_tool',
+            description: 'Test ping function',
+            parameters: {
+              type: 'object',
+              properties: { ping: { type: 'string' } },
+              required: ['ping'],
+            },
+          },
+        },
+      ],
+      tool_choice: 'auto',
+      max_tokens: 32,
+    }),
+    validateResponse: (res) => {
+      if (!res) return false
+      const toolCalls = res?.choices?.[0]?.message?.tool_calls || res?.tool_calls
+      if (Array.isArray(toolCalls) && toolCalls.length > 0) {
+        return toolCalls.some((tc) => tc?.function?.name === 'ping_tool')
+      }
+      return Boolean(res?.choices?.[0]?.message || res?.text || res?.content)
+    },
+  },
+  vision: {
+    name: 'vision',
+    buildRequest: (model) => ({
+      model,
+      messages: [
+        {
+          role: 'user',
+          content: [
+            { type: 'text', text: 'Describe image' },
+            {
+              type: 'image_url',
+              image_url: {
+                url: 'data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mNkYAAAAAYAAjCB0C8AAAAKey',
+              },
+            },
+          ],
+        },
+      ],
+      max_tokens: 16,
+    }),
+    validateResponse: (res) => {
+      if (!res) return false
+      const content = res?.choices?.[0]?.message?.content || res?.text || res?.content || ''
+      return Boolean(typeof content === 'string' && content.length > 0)
+    },
+  },
+  developerRole: {
+    name: 'developerRole',
+    buildRequest: (model) => ({
+      model,
+      messages: [
+        { role: 'developer', content: 'You are a test probe assistant.' },
+        { role: 'user', content: 'Say OK' },
+      ],
+      max_tokens: 16,
+    }),
+    validateResponse: (res) => {
+      if (!res) return false
+      const content = res?.choices?.[0]?.message?.content || res?.text || res?.content || ''
+      return Boolean(typeof content === 'string' && content.length > 0)
+    },
+  },
+})
+
+export function createCapabilityProber({
+  inferFn,
+  getCatalog = () => [],
+  storagePath,
+} = {}) {
+  const verifiedMap = new Map()
+  let isProbing = false
+  let probingProgress = {
+    total: 0,
+    completed: 0,
+    currentProvider: '',
+    currentModel: '',
+    startedAt: 0,
+  }
+
+  function loadFromDisk() {
+    if (!storagePath) return
+    try {
+      if (fs.existsSync(storagePath)) {
+        const raw = fs.readFileSync(storagePath, 'utf8')
+        const data = JSON.parse(raw)
+        if (data && typeof data === 'object') {
+          for (const [k, v] of Object.entries(data)) {
+            verifiedMap.set(k, v)
+          }
+        }
+      }
+    } catch {
+      // Ignore disk load error
+    }
+  }
+
+  function saveToDisk() {
+    if (!storagePath) return
+    try {
+      const dir = path.dirname(storagePath)
+      if (!fs.existsSync(dir)) fs.mkdirSync(dir, { recursive: true })
+      const obj = Object.fromEntries(verifiedMap.entries())
+      fs.writeFileSync(storagePath, JSON.stringify(obj, null, 2), 'utf8')
+    } catch {
+      // Ignore disk save error
+    }
+  }
+
+  loadFromDisk()
+
+  async function probeSingleCapability({ provider, model, probeType, signal }) {
+    const probeDef = PROBE_PAYLOADS[probeType]
+    if (!probeDef) throw new Error(`Unknown probe type: ${probeType}`)
+
+    const start = Date.now()
+    if (typeof inferFn !== 'function') {
+      return { ok: true, latencyMs: 1, mock: true }
+    }
+
+    try {
+      const payload = probeDef.buildRequest(model)
+      const res = await inferFn({ provider, model, payload, probeType, signal })
+      const latencyMs = Math.max(1, Date.now() - start)
+      const ok = probeDef.validateResponse(res)
+      return { ok, latencyMs }
+    } catch (err) {
+      const latencyMs = Math.max(1, Date.now() - start)
+      return { ok: false, latencyMs, error: err?.message || String(err) }
+    }
+  }
+
+  async function probeModelCapabilities({ provider, model, probeTypes = PROBE_TYPES, signal } = {}) {
+    if (!provider || typeof provider !== 'string') throw new Error('provider is required')
+    if (!model || typeof model !== 'string') throw new Error('model is required')
+
+    const details = {}
+    const verified = {}
+
+    for (const pType of probeTypes) {
+      if (signal?.aborted) break
+      const res = await probeSingleCapability({ provider, model, probeType: pType, signal })
+      details[pType] = res
+      verified[pType] = Boolean(res.ok)
+    }
+
+    const key = `${provider}:${model}`
+    const record = {
+      provider,
+      model,
+      verified,
+      details,
+      checkedAt: Date.now(),
+    }
+
+    verifiedMap.set(key, record)
+    saveToDisk()
+    return record
+  }
+
+  async function probeProviderCapabilities({ provider, onlyNew = false, concurrency = 2, signal } = {}) {
+    if (!provider || typeof provider !== 'string') throw new Error('provider is required')
+
+    const catalog = typeof getCatalog === 'function' ? getCatalog(provider) : []
+    const allModels = (Array.isArray(catalog) ? catalog : [])
+      .map((m) => (typeof m === 'string' ? m : m?.id))
+      .filter((id) => typeof id === 'string' && id.trim())
+
+    const modelsToProbe = onlyNew
+      ? allModels.filter((id) => !verifiedMap.has(`${provider}:${id}`))
+      : allModels
+
+    isProbing = true
+    probingProgress = {
+      total: modelsToProbe.length,
+      completed: 0,
+      currentProvider: provider,
+      currentModel: '',
+      startedAt: Date.now(),
+    }
+
+    const results = []
+    const limit = Math.max(1, Math.min(6, Number(concurrency) || 2))
+    let cursor = 0
+
+    const workers = Array.from({ length: Math.min(limit, modelsToProbe.length || 1) }, async () => {
+      while (cursor < modelsToProbe.length) {
+        if (signal?.aborted) break
+        const index = cursor++
+        const modelId = modelsToProbe[index]
+        probingProgress.currentModel = modelId
+        const record = await probeModelCapabilities({ provider, model: modelId, signal })
+        results.push(record)
+        probingProgress.completed++
+      }
+    })
+
+    await Promise.all(workers)
+    isProbing = false
+
+    return {
+      provider,
+      total: modelsToProbe.length,
+      completed: results.length,
+      results,
+    }
+  }
+
+  async function probeAllCapabilities({ onlyNew = false, concurrency = 2, signal } = {}) {
+    const catalog = typeof getCatalog === 'function' ? getCatalog() : []
+    const providers = Array.isArray(catalog)
+      ? [...new Set(catalog.map((m) => m?.provider).filter(Boolean))]
+      : []
+
+    isProbing = true
+    const allResults = []
+
+    for (const provider of providers) {
+      if (signal?.aborted) break
+      const provRes = await probeProviderCapabilities({ provider, onlyNew, concurrency, signal })
+      allResults.push(provRes)
+    }
+
+    isProbing = false
+    return {
+      providersCount: providers.length,
+      results: allResults,
+    }
+  }
+
+  function getStatus() {
+    return {
+      isProbing,
+      progress: { ...probingProgress },
+      totalVerified: verifiedMap.size,
+    }
+  }
+
+  function getVerifiedCapabilities({ provider, model } = {}) {
+    if (provider && model) {
+      return verifiedMap.get(`${provider}:${model}`) || null
+    }
+    if (provider) {
+      const prefix = `${provider}:`
+      const out = {}
+      for (const [k, v] of verifiedMap.entries()) {
+        if (k.startsWith(prefix)) out[k.slice(prefix.length)] = v
+      }
+      return out
+    }
+    return Object.fromEntries(verifiedMap.entries())
+  }
+
+  function setVerifiedCapabilities(provider, model, data) {
+    if (!provider || !model || !data) return
+    const key = `${provider}:${model}`
+    verifiedMap.set(key, {
+      provider,
+      model,
+      verified: data.verified || {},
+      details: data.details || {},
+      checkedAt: data.checkedAt || Date.now(),
+    })
+    saveToDisk()
+  }
+
+  function clearVerifiedCapabilities({ provider, model } = {}) {
+    if (provider && model) {
+      verifiedMap.delete(`${provider}:${model}`)
+    } else if (provider) {
+      const prefix = `${provider}:`
+      for (const k of [...verifiedMap.keys()]) {
+        if (k.startsWith(prefix)) verifiedMap.delete(k)
+      }
+    } else {
+      verifiedMap.clear()
+    }
+    saveToDisk()
+  }
+
+  return {
+    probeModelCapabilities,
+    probeProviderCapabilities,
+    probeAllCapabilities,
+    getStatus,
+    getVerifiedCapabilities,
+    setVerifiedCapabilities,
+    clearVerifiedCapabilities,
+  }
+}