@acosmi/sdk-ts 1.1.0 → 1.2.0

package/CHANGELOG.md

@@ -5,6 +5,30 @@ All notable changes to `@acosmi/sdk-ts` will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [1.2.0] — 2026-05-18
+
+### Added
+
+- **`ManagedModel.inputModalities`** — 新增可选字段, 类型 `Array<'text' | 'image'>`, 描述模型可接收的用户输入模态; 'image' 表示模型可直接接收 screenshot/image 输入. listModels / listModelsWithStatus 在写缓存与返回前会归一化上游 snake_case 字段名 `input_modalities` → camelCase `inputModalities` (兼容老网关), camelCase 与 snake_case 同时存在时 camelCase 胜.
+
+- **`ModelCapabilities.supports_desktop_visual_understanding`** — 新增可选字段, 标识模型适合作为桌面截图解析 sidecar (输入 screenshot, 输出结构化 UI 描述, 供非多模态主模型消费). 与 `inputModalities=['image']` 是正交两件事: 前者描述"模型能不能吃图", 后者描述"运营是否把该模型标为桌面 UI 解析专用 sidecar". `zeroModelCapabilities()` 显式置 `false`, 避免 `undefined` 导致调用方误判.
+
+- **`InputModality`** 类型导出 — `'text' | 'image'`.
+
+- **Model catalog helpers (4 个)** — CrabCode desktop automation / computer-use 选模型用, 严格按 SDK 字段, 禁止模型名 substring 推断:
+  - `modelSupportsInputModality(model, modality): boolean`
+  - `modelSupportsImageInput(model): boolean`
+  - `findFirstModelByInputModality(models, modality): ManagedModel | null` — 按 catalog 顺序, 跳过 `isEnabled === false`
+  - `findDesktopVisualUnderstandingModel(models): ManagedModel | null` — 选择规则: isEnabled !== false + capabilities.supports_desktop_visual_understanding === true + inputModalities 含 'image' + isDefault 优先 / 否则 catalog 顺序第一个
+
+### Notes
+
+- 上游 `ManagedModel` 缺失 `inputModalities` 时, SDK 保留 `undefined` 不自动补 `['text']` — 调用方必须保守按 text-only / unknown 处理, 严禁默认假设支持 image.
+- 客户端不应硬编码模型名, 应完全依赖 SDK catalog 能力字段做模型选择.
+- 23 新单测全绿 (8 listModels 归一化 + 15 helpers); 全量 79/79 passed, typecheck/lint/build clean.
+
+---
+
 ## [1.1.0] — 2026-05-06
 
 ### Added

package/README.md

@@ -7,9 +7,9 @@
 ## 状态
 
 - 端口源：[acosmi-sdk-go](https://github.com/acosmi/acosmi-sdk-go) v1.0.0（与 Go SDK 联动稳定测试版）
-- 当前版本：**1.1.0**（新增 SDK-facing Agent Runs 公开 API；详见 [CHANGELOG](./CHANGELOG.md)）
-- 测试：56/56 vitest 全绿，typecheck/lint/build 0 错误；packed-tarball smoke (`npm run test:pack`) 在 prepublishOnly 闸内
-- 包链接：[npm](https://www.npmjs.com/package/@acosmi/sdk-ts/v/1.1.0) · [tarball](https://registry.npmjs.org/@acosmi/sdk-ts/-/sdk-ts-1.1.0.tgz) · [GitHub Release](https://github.com/acosmi/sdk-ts/releases/tag/v1.1.0)
+- 当前版本：**1.2.0**（新增 `ManagedModel.inputModalities` + `ModelCapabilities.supports_desktop_visual_understanding` + 4 catalog helpers，供 CrabCode desktop automation / computer-use 选模型用；详见 [CHANGELOG](./CHANGELOG.md)）
+- 测试：79/79 vitest 全绿，typecheck/lint/build 0 错误；packed-tarball smoke (`npm run test:pack`) 在 prepublishOnly 闸内
+- 包链接：[npm](https://www.npmjs.com/package/@acosmi/sdk-ts/v/1.2.0) · [tarball](https://registry.npmjs.org/@acosmi/sdk-ts/-/sdk-ts-1.2.0.tgz) · [GitHub Release](https://github.com/acosmi/sdk-ts/releases/tag/v1.2.0)
 
 ## 安装
 
@@ -181,7 +181,7 @@ const client = new Client({ serverURL: 'https://acosmi.com', tokenStore: new Fil
 | **Chat**     | `chat`, `chatStream`, `chatStreamWithUsage`                                          |
 | **Agent Runs** | `agentRuns.create`, `agentRuns.stream`, `agentRuns.run`, `agentRuns.cancel`, `agentRuns.get`, `agentRuns.listArtifacts`, `agentRuns.downloadArtifact`, `agentRuns.submitLocalToolResult`, `agentRuns.runWithLocalTools` |
 | **Auth**     | `login`, `logout`, `ensureToken`, `forceRefresh`, `discover`, `authorize`, `exchangeCode`, `refreshToken` |
-| **Models**   | `listModels`, `listModelsWithStatus`, `getModelCapabilities`, `getQuotaSummary`      |
+| **Models**   | `listModels`, `listModelsWithStatus`, `getModelCapabilities`, `getQuotaSummary`, `modelSupportsInputModality`, `modelSupportsImageInput`, `findFirstModelByInputModality`, `findDesktopVisualUnderstandingModel` |
 | **Skills**   | `browseSkills`, `browseSkillsList`, `getSkillDetail`, `resolveSkill`, `installSkill`, `downloadSkill`, `uploadSkill`, `generateSkill`, `optimizeSkill`, `validateSkill` |
 | **Tools**    | `listTools`, `getTool`                                                               |
 | **Wallet**   | `getWalletStats`, `getWalletTransactions`                                            |
@@ -230,6 +230,38 @@ const resp = await client.chat('claude-opus-4-7', {
 
 `AllowedDomains` / `BlockedDomains` 互斥，同时传入抛 `Error`。
 
+### 示例：桌面视觉理解 sidecar 选模型（CrabCode desktop automation / computer-use）
+
+```ts
+import {
+  Client,
+  findDesktopVisualUnderstandingModel,
+  modelSupportsImageInput,
+} from '@acosmi/sdk-ts';
+
+const client = new Client({ serverURL: 'https://acosmi.com' });
+const models = await client.listModels();
+
+// 1) 主模型是否能直接吃截图？
+const primaryCanSeeImages = modelSupportsImageInput(
+  models.find((m) => m.modelId === 'deepseek-v4') ?? null,
+);
+
+// 2) 不能 → 走桌面视觉 sidecar：截图先送 sidecar 解析为结构化 UI 描述，再喂主模型
+const sidecar = findDesktopVisualUnderstandingModel(models);
+if (!sidecar) {
+  throw new Error('No desktop visual understanding model available — 让管理员在网关启用一个 sidecar 模型');
+}
+console.log('sidecar →', sidecar.modelId);
+```
+
+红线：
+
+- `ManagedModel.inputModalities` 用于客户端判断模型可接收的用户输入类型（'text' | 'image'）。
+- `capabilities.supports_desktop_visual_understanding` 用于选择专门解析桌面截图的视觉 sidecar 模型，与 `inputModalities` 是正交两件事（普通视觉模型不一定擅长 UI 解析）。
+- 客户端不应硬编码模型名做能力推断，应完全依赖 SDK catalog 字段。
+- 上游未下发 `inputModalities` 时 SDK 保持 `undefined`，调用方必须保守按 text-only / unknown 处理。
+
 ### 示例：钱包 + 余额 + 流量包购买
 
 ```ts

package/dist/browser/index.mjs

@@ -1013,6 +1013,45 @@ var init_adapters = __esm({
 
 // src/index.ts
 init_types();
+
+// src/model-helpers.ts
+function modelSupportsInputModality(model, modality) {
+  if (!model) return false;
+  const mods = model.inputModalities;
+  if (!Array.isArray(mods)) return false;
+  return mods.includes(modality);
+}
+function modelSupportsImageInput(model) {
+  return modelSupportsInputModality(model, "image");
+}
+function findFirstModelByInputModality(models, modality) {
+  if (!Array.isArray(models)) return null;
+  for (const m of models) {
+    if (!m) continue;
+    if (m.isEnabled === false) continue;
+    if (!modelSupportsInputModality(m, modality)) continue;
+    return m;
+  }
+  return null;
+}
+function findDesktopVisualUnderstandingModel(models) {
+  if (!Array.isArray(models)) return null;
+  const candidates = [];
+  for (const m of models) {
+    if (!m) continue;
+    if (m.isEnabled === false) continue;
+    if (m.capabilities?.supports_desktop_visual_understanding !== true) continue;
+    if (!modelSupportsInputModality(m, "image")) continue;
+    candidates.push(m);
+  }
+  if (candidates.length === 0) return null;
+  for (const m of candidates) {
+    if (m.isDefault === true) return m;
+  }
+  return candidates[0];
+}
+
+// src/index.ts
 init_adapters();
 
 // src/auth.ts
@@ -2588,14 +2627,15 @@ var Client = class _Client {
       null,
       signal
     );
-    this.modelCache = result.data;
+    const normalized = normalizeInputModalities(result.data);
+    this.modelCache = normalized;
     this.modelCacheTimeMs = Date.now();
     let status = "";
     if (headers) {
       const h = headers.get("X-Entitlement-Filter-Status");
       if (h) status = h;
     }
-    return { models: result.data, status };
+    return { models: normalized, status };
   }
   /** 查询当前用户账户级权益总览 (v0.19+) */
   async getQuotaSummary(signal) {
@@ -3228,9 +3268,24 @@ function zeroModelCapabilities() {
     supports_token_efficient: false,
     supports_redact_thinking: false,
     max_input_tokens: 0,
-    max_output_tokens: 0
+    max_output_tokens: 0,
+    supports_desktop_visual_understanding: false
   };
 }
+function normalizeInputModalities(models) {
+  if (!Array.isArray(models)) return models;
+  for (const m of models) {
+    if (!m || typeof m !== "object") continue;
+    if (Array.isArray(m.inputModalities)) continue;
+    const snake = m.input_modalities;
+    if (Array.isArray(snake)) {
+      m.inputModalities = snake.filter(
+        (v) => v === "text" || v === "image"
+      );
+    }
+  }
+  return models;
+}
 function withRequestTimeout(ms, parent) {
   const ctl = new AbortController();
   const timer = setTimeout(() => ctl.abort(), ms);
@@ -4576,6 +4631,6 @@ Client.prototype.getBugReport = async function(bugID, signal) {
   return resp.data;
 };
 
-export { AgentRunStreamError, AgentRunsClient, AnthropicAdapter, BucketClassCommercial, BucketClassGeneric, BusinessError, Client, LocalStorageTokenStore as DefaultBrowserTokenStore, DefaultRetryPolicy, ErrAuthDenied, ErrBrowserOpen, ErrDiscovery, ErrRegistration, ErrSSLProxy, ErrTimeout, ErrTokenExchange, EventAuthURL, EventComplete, EventError, FileTokenStore, FilterStatusAdminBypass, FilterStatusDisabledByFlag, FilterStatusFallbackMissingUser, FilterStatusFallbackNoBuckets, FilterStatusFallbackTkdistError, FilterStatusFallbackTkdistSkew, FilterStatusInternalBypass, FilterStatusOK, FilterStatusUnknown, HTTPError, InMemoryTokenStore, LocalStorageTokenStore, ModelNotFoundError, NetworkError, OpenAIAdapter, OrderTerminalError, ProviderFormat, RateLimitError, ScopeAI, ScopeAccount, ScopeEntitlements, ScopeModels, ScopeModelsChat, ScopeProfile, ScopeSkillStore, ScopeSkills, ScopeTokenPackages, ScopeTools, ScopeToolsExecute, ScopeWallet, ScopeWalletReadonly, ServerToolTypeWebSearch, StreamError, ThinkingHigh, ThinkingHighMinMaxTokens, ThinkingMax, ThinkingMaxFallbackMaxTokens, ThinkingOff, allScopes, anthropicResponseTextContent, anthropicResponseThinkingContent, anthropicResponseToolUseBlocks, apiResponseBusinessError, apiResponseGetMessage, authorize, bucketInfoIsCommercial, bucketRowIsCommercial, buildBetas, commerceScopes, computeBackoff, defaultRetryable, defaultSafeToRetry, discover, effectivePolicy, exchangeCode, extractAnthropicBlockMeta, fileLockDefaults, getAdapter, getAdapterForModel, isSSLError, modelScopes, newFileTokenStore, newThinkingConfig, newTokenSet, newWebSearchTool, parseNotificationEvent, parseSettlement, parseSourcesEvent, refreshToken, register, revokeToken, sanitize_exports as sanitize, skillScopes, tokenSetIsExpired, uniqueMerge };
+export { AgentRunStreamError, AgentRunsClient, AnthropicAdapter, BucketClassCommercial, BucketClassGeneric, BusinessError, Client, LocalStorageTokenStore as DefaultBrowserTokenStore, DefaultRetryPolicy, ErrAuthDenied, ErrBrowserOpen, ErrDiscovery, ErrRegistration, ErrSSLProxy, ErrTimeout, ErrTokenExchange, EventAuthURL, EventComplete, EventError, FileTokenStore, FilterStatusAdminBypass, FilterStatusDisabledByFlag, FilterStatusFallbackMissingUser, FilterStatusFallbackNoBuckets, FilterStatusFallbackTkdistError, FilterStatusFallbackTkdistSkew, FilterStatusInternalBypass, FilterStatusOK, FilterStatusUnknown, HTTPError, InMemoryTokenStore, LocalStorageTokenStore, ModelNotFoundError, NetworkError, OpenAIAdapter, OrderTerminalError, ProviderFormat, RateLimitError, ScopeAI, ScopeAccount, ScopeEntitlements, ScopeModels, ScopeModelsChat, ScopeProfile, ScopeSkillStore, ScopeSkills, ScopeTokenPackages, ScopeTools, ScopeToolsExecute, ScopeWallet, ScopeWalletReadonly, ServerToolTypeWebSearch, StreamError, ThinkingHigh, ThinkingHighMinMaxTokens, ThinkingMax, ThinkingMaxFallbackMaxTokens, ThinkingOff, allScopes, anthropicResponseTextContent, anthropicResponseThinkingContent, anthropicResponseToolUseBlocks, apiResponseBusinessError, apiResponseGetMessage, authorize, bucketInfoIsCommercial, bucketRowIsCommercial, buildBetas, commerceScopes, computeBackoff, defaultRetryable, defaultSafeToRetry, discover, effectivePolicy, exchangeCode, extractAnthropicBlockMeta, fileLockDefaults, findDesktopVisualUnderstandingModel, findFirstModelByInputModality, getAdapter, getAdapterForModel, isSSLError, modelScopes, modelSupportsImageInput, modelSupportsInputModality, newFileTokenStore, newThinkingConfig, newTokenSet, newWebSearchTool, parseNotificationEvent, parseSettlement, parseSourcesEvent, refreshToken, register, revokeToken, sanitize_exports as sanitize, skillScopes, tokenSetIsExpired, uniqueMerge };
 //# sourceMappingURL=index.mjs.map
 //# sourceMappingURL=index.mjs.map