@nvae/llmswitch 0.6.0 → 0.8.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -146,6 +146,216 @@ llms bridge reload claude
146
146
  llms bridge reload codex --profile my-provider
147
147
  ```
148
148
 
149
+ ### 8. 对外提供 AI 网关
150
+
151
+ Bridge 服务的是本机的 Claude Code / Codex / OpenCode。如果要让**第三方客户端**通过一个端口访问你配置的模型,用 gateway:
152
+
153
+ ```bash
154
+ # 1. 准备供应商(可从已有工具配置导入,按上游去重)
155
+ llms gateway provider import
156
+ # 或手动添加(自动探测接口类型与模型列表)
157
+ llms gateway provider add
158
+
159
+ # 2. 创建网关 API Key(明文只显示一次,请立即保存)
160
+ llms gateway key create --name my-app
161
+
162
+ # 3. 启动网关
163
+ llms gateway start
164
+
165
+ # 4. 查看状态与可路由模型
166
+ llms gateway status
167
+ llms gateway models
168
+ ```
169
+
170
+ 默认监听 `127.0.0.1:17900`。第三方客户端直接把它当成 OpenAI 或 Anthropic 端点使用:
171
+
172
+ ```bash
173
+ # OpenAI 格式
174
+ curl http://127.0.0.1:17900/v1/chat/completions \
175
+ -H "Authorization: Bearer llmsk-..." \
176
+ -H "Content-Type: application/json" \
177
+ -d '{"model":"deepseek-chat","messages":[{"role":"user","content":"hi"}]}'
178
+
179
+ # Anthropic 格式(同一个上游,网关自动转换)
180
+ curl http://127.0.0.1:17900/v1/messages \
181
+ -H "x-api-key: llmsk-..." \
182
+ -H "Content-Type: application/json" \
183
+ -d '{"model":"deepseek-chat","max_tokens":1024,"messages":[{"role":"user","content":"hi"}]}'
184
+ ```
185
+
186
+ **任意入口格式 ↔ 任意上游格式**。三种入口(OpenAI Chat、OpenAI Responses、Anthropic Messages)与三种上游格式可自由组合,含流式与工具调用;入口与上游格式相同时原样透传,避免无谓的转换损耗。已知限制:OpenAI Responses 的有状态特性(`previous_response_id`、`store`、后台模式)只在同格式透传时可用,跨格式转换不支持。
187
+
188
+ 上游地址默认拼 `/v1` 前缀;非标准路径的上游可自定义:
189
+
190
+ ```bash
191
+ # 直连 baseUrl(如 Gemini OpenAI 兼容端点 …/v1beta/openai)
192
+ llms gateway provider add --path-prefix ""
193
+ # 自定义前缀
194
+ llms gateway provider edit my-provider --path-prefix v2
195
+ ```
196
+
197
+ | 端点 | 说明 |
198
+ | --- | --- |
199
+ | `GET /v1/models` | 可路由模型列表(按 Key 作用域过滤) |
200
+ | `GET /v1/models/{id}` | 单个模型详情(OpenAI 客户端兼容) |
201
+ | `POST /v1/chat/completions` | OpenAI Chat Completions |
202
+ | `POST /v1/completions` | Legacy Text Completions(`prompt` 自动转为消息) |
203
+ | `POST /v1/messages` | Anthropic Messages |
204
+ | `POST /v1/messages/count_tokens` | Anthropic 上游走原生接口,其他上游本地计算(见下) |
205
+ | `POST /v1/responses` | OpenAI Responses |
206
+ | `POST /v1/embeddings` | 仅 OpenAI 兼容上游 |
207
+ | `GET /health` | 存活探针(无需鉴权,不含任何配置信息) |
208
+
209
+ 每个响应都带 `x-request-id`(客户端可传入以关联日志),并转发到上游,方便全链路排查。
210
+
211
+ **模型路由解析顺序**:
212
+
213
+ 1. 显式别名(`llms gateway route add`)
214
+ 2. 限定写法 `provider/model` 或 `provider:model`
215
+ 3. 裸模型 id(在某个 provider 的模型列表中)
216
+ 4. 未声明模型列表的 provider(作为 passthrough 兜底)
217
+ 5. `config set --default-provider` 指定的兜底供应商
218
+
219
+ 多个供应商提供同一模型时,按 `priority` 升序排列,自动构成 fallback 链:
220
+
221
+ ```bash
222
+ # 别名 + 显式 fallback
223
+ llms gateway route add gpt-4o --provider azure --model gpt-4o-2024-11 --fallback openrouter/openai/gpt-4o
224
+
225
+ # 查看某个模型 id 的实际路由顺序
226
+ llms gateway resolve gpt-4o
227
+ ```
228
+
229
+ **Provider fallback**:上游返回 429/5xx 等可重试状态或连接失败时自动换下一个供应商。响应一旦开始写出(流式首帧之后)便不再切换,避免给客户端拼接两段不一致的输出。
230
+
231
+ ```bash
232
+ llms gateway config set --fallback true --max-attempts 3 --retry-statuses 429,500,502,503,504
233
+ ```
234
+
235
+ **熔断冷却**:连续失败的上游会进入指数退避冷却(5s 起,封顶 2 分钟),冷却期内路由直接跳过它;全部候选都在冷却时仍会照常尝试。`llms gateway status` 会显示当前冷却中的供应商。
236
+
237
+ ```bash
238
+ # 测试供应商连通性(拉模型列表;--call 额外发一次 1-token 补全)
239
+ llms gateway provider test my-provider --call
240
+
241
+ # 从上游重新拉取模型列表
242
+ llms gateway provider refresh-models my-provider
243
+
244
+ # 自定义上游请求头(可重复传)
245
+ llms gateway provider edit my-provider --header "X-Title: my-app"
246
+ ```
247
+
248
+ **API Key 管理**:仅存储哈希,支持作用域、限额、过期、编辑与换发。
249
+
250
+ ```bash
251
+ # 限定供应商、模型、接口格式与速率
252
+ llms gateway key create --name partner \
253
+ --providers deepseek --models deepseek-chat \
254
+ --formats openai-chat --rate-limit 60 --expires-in-days 30 \
255
+ --daily-requests 5000
256
+
257
+ llms gateway key list
258
+ llms gateway key edit <id> --rate-limit 120 # 改限额/作用域/续期
259
+ llms gateway key rotate <id> # 换发明文,旧 Key 立即失效
260
+ llms gateway key revoke <id>
261
+ ```
262
+
263
+ 作用域说明:`--models` 按"客户端请求里的模型写法"匹配——限定别名就只用别名拼写访问,限定 `provider/model` 则两种写法都可;拼写错误在创建时会收到警告。
264
+
265
+ **限流**:计数持久化在 `gateway/rate-limit.json`,重启不丢失,多个网关进程共享同一份计数。响应会带标准限流头,客户端可据此自行退避:
266
+
267
+ ```
268
+ X-RateLimit-Limit: 60
269
+ X-RateLimit-Remaining: 59
270
+ X-RateLimit-Reset: 1787894760
271
+ Retry-After: 43 # 仅 429 时出现
272
+ ```
273
+
274
+ ```bash
275
+ # 默认限额(Key 未单独设置时生效)
276
+ llms gateway config set --rate-limit 120
277
+
278
+ # 查看各 Key 当前窗口用量 / 清空计数
279
+ llms gateway ratelimit show
280
+ llms gateway ratelimit reset
281
+ ```
282
+
283
+ `--rate-limit` 的语义:`-1` 完全不限流(豁免全局默认),`0` 继承全局默认,`> 0` 硬上限;`--daily-requests` 额外提供按 UTC 日的请求配额。
284
+
285
+ 计数文件读写有锁保护;极端争用下拿不到锁时会放行请求而非阻塞流量,宁可限额略松也不卡住线上调用。
286
+
287
+ **Token 计数**:`count_tokens` 优先走上游原生接口(Anthropic 格式上游),拿不到时在本地计算并在 `llm_switch` 字段里说明来源:
288
+
289
+ ```json
290
+ {
291
+ "input_tokens": 1234,
292
+ "llm_switch": {
293
+ "estimated": true,
294
+ "reason": "upstream_not_anthropic",
295
+ "method": "heuristic",
296
+ "breakdown": { "text": 30, "images": 1190, "tools": 0, "overhead": 14 }
297
+ }
298
+ }
299
+ ```
300
+
301
+ 本地计算分两档:
302
+
303
+ - `heuristic`(默认,无额外依赖):按书写系统分别计数(中日韩、拉丁、数字各有不同的字符/token 比),再加上每条消息、每个工具 schema 与请求信封的结构开销。实测对中英日韩散文、JSON 与表情符号的平均绝对误差约 11%,且绝大多数样本偏高而非偏低——用于判断"这个请求装不装得下"时偏保守更安全。标点密集的源码是已知弱项,可能低估约 15%。
304
+ - `tokenizer`(可选,精确):装上 `gpt-tokenizer` 后自动启用,对 OpenAI 系编码是精确值,其他词表下也比启发式更接近。
305
+
306
+ ```bash
307
+ # 需要精确计数时自行安装,llmswitch 不强制依赖它
308
+ npm install -g gpt-tokenizer
309
+
310
+ # 强制使用启发式
311
+ export LLM_SWITCH_DISABLE_TOKENIZER=1
312
+ ```
313
+
314
+ 两档都会额外计入图片开销:从 base64 头部解析 PNG / JPEG / GIF / WebP 的真实像素尺寸,按 `宽 × 高 / 750` 折算;无法判定尺寸时(例如 URL 图片)按保守值计入。
315
+
316
+ **用量统计**:网关按天记录每个(Key / 供应商 / 模型)组合的请求数与 token 用量,持久化在 `gateway/usage.json`(保留 90 天):
317
+
318
+ ```bash
319
+ llms gateway usage --days 7
320
+ llms gateway usage --json
321
+ llms gateway usage reset
322
+ ```
323
+
324
+ **日志**:写入 `~/.config/llm-switch/gateway/gateway.log`,仅记录 Key 的 id,不记录明文或上游密钥;每行含 `req=<request-id>` 可与客户端和上游日志关联。启动时若超过 10MB 自动轮转为 `gateway.log.old`:
325
+
326
+ ```bash
327
+ llms gateway logs # 最后 100 行
328
+ llms gateway logs --lines 500
329
+ llms gateway logs --follow # 持续跟踪
330
+ ```
331
+
332
+ **运行时限额**(超时/并发/请求体大小)通过环境变量调整,与 Bridge 共用:
333
+
334
+ | 环境变量 | 默认 | 说明 |
335
+ | --- | --- | --- |
336
+ | `LLM_SWITCH_MAX_CONCURRENCY` | 16 | 最大并发请求数 |
337
+ | `LLM_SWITCH_MAX_BODY_BYTES` | 16MB | 请求体上限 |
338
+ | `LLM_SWITCH_MAX_RESPONSE_BYTES` | 32MB | 上游响应上限 |
339
+ | `LLM_SWITCH_CONNECT_TIMEOUT_MS` | 30000 | 上游连接超时 |
340
+ | `LLM_SWITCH_IDLE_TIMEOUT_MS` | 90000 | 流式空闲超时 |
341
+ | `LLM_SWITCH_TOTAL_TIMEOUT_MS` | 600000 | 单请求总超时 |
342
+
343
+ `llms gateway config show` 会一并显示当前生效值。
344
+
345
+ **对外暴露的安全要求**:默认只绑回环地址。绑到非回环地址必须显式传 `--allow-remote`,且至少存在一个有效 API Key,否则拒绝启动。
346
+
347
+ ```bash
348
+ llms gateway start --host 0.0.0.0 --allow-remote
349
+ ```
350
+
351
+ 网关只提供明文 HTTP,请放在反向代理(Nginx / Caddy)后面终止 TLS,不要把裸 HTTP 直接暴露到公网。浏览器直连需显式开启 CORS(已允许 `anthropic-beta` 等请求头,并暴露限流与 request-id 响应头):
352
+
353
+ ```bash
354
+ llms gateway config set --cors-origins https://app.example.com
355
+ ```
356
+
357
+ 日志写入 `~/.config/llm-switch/gateway/gateway.log`,仅记录 Key 的 id,不记录明文或上游密钥。
358
+
149
359
  ---
150
360
 
151
361
  ## 常用命令
@@ -161,6 +371,15 @@ llms bridge reload codex --profile my-provider
161
371
  | `llms <tool> model` | 选择模型 |
162
372
  | `llms launch/run <tool> [model]` | 启动工具 |
163
373
  | `llms bridge status` | 查看 Bridge 状态 |
374
+ | `llms gateway start` | 启动对外 AI 网关 |
375
+ | `llms gateway provider import` | 从工具配置导入网关供应商 |
376
+ | `llms gateway provider test <name>` | 测试供应商连通性 |
377
+ | `llms gateway key create` | 创建网关 API Key |
378
+ | `llms gateway key rotate <id>` | 换发 API Key |
379
+ | `llms gateway status` | 查看网关状态 |
380
+ | `llms gateway ratelimit show` | 查看各 Key 限流用量 |
381
+ | `llms gateway usage` | 查看按天聚合的用量统计 |
382
+ | `llms gateway logs` | 查看网关日志 |
164
383
  | `llms path` | 查看数据目录 |
165
384
 
166
385
  `<tool>` 可选 `claude`、`codex`、`opencode`。
@@ -179,6 +398,7 @@ llms launch --help
179
398
  | 数据 | 位置 |
180
399
  | --- | --- |
181
400
  | llmswitch 配置 | `~/.config/llm-switch/` |
401
+ | 网关供应商 / Key / 日志 | `~/.config/llm-switch/gateway/` |
182
402
  | Claude Code | `~/.claude/settings.json` |
183
403
  | Codex | `~/.codex/config.toml` |
184
404
  | OpenCode | `~/.config/opencode/opencode.json` |
@@ -6,6 +6,7 @@ import { atomicWriteFile, backupFile, ensureDir } from "../utils/fs.js";
6
6
  import { applyProxyToEnvRecord, clearProxyEnvKeys } from "../utils/proxy.js";
7
7
  import { getBackupsDir, getOpenCodeAuthPath, getOpenCodeConfigDir, getOpenCodeConfigPath, } from "../utils/paths.js";
8
8
  import { setActiveProfile } from "../store/profiles.js";
9
+ import { ensureBridgeForProfile, profileNeedsBridge, } from "../bridge/manager.js";
9
10
  function providerId(name) {
10
11
  return `llms-${name}`.replace(/[^a-zA-Z0-9_-]/g, "-");
11
12
  }
@@ -30,7 +31,7 @@ export function readOpenCodeAuth(path = getOpenCodeAuthPath()) {
30
31
  return {};
31
32
  return JSON.parse(readFileSync(path, "utf8"));
32
33
  }
33
- export function buildOpenCodeProviderBlock(profile) {
34
+ export function buildOpenCodeProviderBlock(profile, overrides) {
34
35
  const models = {};
35
36
  for (const id of profile.models.list) {
36
37
  models[id] = { name: id };
@@ -39,10 +40,12 @@ export function buildOpenCodeProviderBlock(profile) {
39
40
  models[profile.models.default] = { name: profile.models.default };
40
41
  }
41
42
  const options = {
42
- baseURL: normalizeBaseUrlForFormat(profile.apiFormat, profile.baseUrl),
43
+ baseURL: overrides?.baseURL ||
44
+ normalizeBaseUrlForFormat(profile.apiFormat, profile.baseUrl),
43
45
  };
44
- if (profile.apiKey) {
45
- options.apiKey = profile.apiKey;
46
+ const apiKey = overrides?.apiKey ?? profile.apiKey;
47
+ if (apiKey) {
48
+ options.apiKey = apiKey;
46
49
  }
47
50
  if (profile.headers && Object.keys(profile.headers).length > 0) {
48
51
  options.headers = { ...profile.headers };
@@ -54,19 +57,23 @@ export function buildOpenCodeProviderBlock(profile) {
54
57
  models,
55
58
  };
56
59
  }
57
- export function buildOpenCodeConfig(existing, profile) {
60
+ export function buildOpenCodeConfig(existing, profile, overrides) {
58
61
  assertCompatible("opencode", profile.apiFormat);
59
62
  const id = providerId(profile.name);
60
63
  const providers = {
61
64
  ...(existing.provider || {}),
62
65
  };
63
- providers[id] = buildOpenCodeProviderBlock(profile);
64
- // Optional top-level env for proxy (OpenCode may pass through)
66
+ providers[id] = buildOpenCodeProviderBlock(profile, overrides);
67
+ // Optional top-level env for proxy (OpenCode may pass through). When the
68
+ // profile routes through the bridge, the upstream proxy is applied inside the
69
+ // bridge; skip env-var proxy injection so OpenCode doesn't apply its own.
65
70
  const env = {
66
71
  ...(existing.env || {}),
67
72
  };
68
73
  clearProxyEnvKeys(env);
69
- applyProxyToEnvRecord(env, profile.proxy);
74
+ if (!overrides) {
75
+ applyProxyToEnvRecord(env, profile.proxy);
76
+ }
70
77
  const next = {
71
78
  ...existing,
72
79
  $schema: existing.$schema || "https://opencode.ai/config.json",
@@ -92,18 +99,25 @@ export function buildOpenCodeAuth(existing, profile) {
92
99
  }
93
100
  return next;
94
101
  }
95
- export function applyOpenCodeProfile(profile) {
102
+ export async function applyOpenCodeProfile(profile) {
96
103
  assertCompatible("opencode", profile.apiFormat);
97
104
  ensureDir(getOpenCodeConfigDir());
98
105
  ensureDir(dirname(getOpenCodeAuthPath()));
106
+ let bridgeConnection = null;
107
+ if (profileNeedsBridge(profile)) {
108
+ bridgeConnection = await ensureBridgeForProfile(profile, "opencode");
109
+ }
99
110
  const configPath = getOpenCodeConfigPath();
100
111
  const authPath = getOpenCodeAuthPath();
101
112
  const existing = readOpenCodeConfig(configPath);
102
113
  const backupPath = backupFile(configPath, getBackupsDir("opencode"), "opencode");
103
114
  backupFile(authPath, getBackupsDir("opencode"), "auth");
104
- const nextConfig = buildOpenCodeConfig(existing, profile);
115
+ const nextConfig = buildOpenCodeConfig(existing, profile, bridgeConnection
116
+ ? { baseURL: bridgeConnection.baseUrl, apiKey: bridgeConnection.clientToken }
117
+ : undefined);
105
118
  atomicWriteFile(configPath, JSON.stringify(nextConfig, null, 2) + "\n");
106
- const nextAuth = buildOpenCodeAuth(readOpenCodeAuth(authPath), profile);
119
+ const bridgeApiKey = bridgeConnection?.clientToken || profile.apiKey;
120
+ const nextAuth = buildOpenCodeAuth(readOpenCodeAuth(authPath), { ...profile, apiKey: bridgeApiKey });
107
121
  atomicWriteFile(authPath, JSON.stringify(nextAuth, null, 2) + "\n");
108
122
  setActiveProfile("opencode", profile.name);
109
123
  return {
@@ -111,7 +125,9 @@ export function applyOpenCodeProfile(profile) {
111
125
  profile: profile.name,
112
126
  configPath,
113
127
  backupPath,
114
- restartHint: "请重新启动 OpenCode 会话以使配置与代理生效。",
128
+ restartHint: bridgeConnection
129
+ ? "已通过本地 bridge 启用供应商(上游代理在 bridge 内生效)。请重新启动 OpenCode 会话使配置生效。"
130
+ : "请重新启动 OpenCode 会话以使配置与代理生效。",
115
131
  };
116
132
  }
117
133
  export function deactivateOpenCodeProfile(profileName) {
@@ -0,0 +1,332 @@
1
+ /**
2
+ * Translate Anthropic Messages responses (stream/non-stream) → OpenAI Chat
3
+ * Completions.
4
+ *
5
+ * Reverse direction of `anthropic-translate-response.ts`. Used by the gateway
6
+ * when an Anthropic upstream must be presented in Chat Completions shape (the
7
+ * gateway's hub format).
8
+ */
9
+ function newId(prefix) {
10
+ return `${prefix}-${Date.now().toString(36)}${Math.random().toString(36).slice(2, 10)}`;
11
+ }
12
+ function asRecord(value) {
13
+ if (value && typeof value === "object" && !Array.isArray(value)) {
14
+ return value;
15
+ }
16
+ return null;
17
+ }
18
+ function numberOr(value, fallback = 0) {
19
+ return typeof value === "number" ? value : fallback;
20
+ }
21
+ /** Anthropic stop_reason → Chat Completions finish_reason. */
22
+ export function anthropicStopReasonToFinishReason(reason) {
23
+ switch (reason) {
24
+ case "tool_use":
25
+ return "tool_calls";
26
+ case "max_tokens":
27
+ return "length";
28
+ case "refusal":
29
+ return "content_filter";
30
+ case "end_turn":
31
+ case "stop_sequence":
32
+ return "stop";
33
+ default:
34
+ return reason == null ? null : "stop";
35
+ }
36
+ }
37
+ function mapUsage(usage) {
38
+ if (!usage)
39
+ return undefined;
40
+ const prompt = numberOr(usage.input_tokens);
41
+ const completion = numberOr(usage.output_tokens);
42
+ const out = {
43
+ prompt_tokens: prompt,
44
+ completion_tokens: completion,
45
+ total_tokens: prompt + completion,
46
+ };
47
+ const cacheRead = usage.cache_read_input_tokens;
48
+ const cacheWrite = usage.cache_creation_input_tokens;
49
+ if (typeof cacheRead === "number") {
50
+ out.prompt_tokens_details = { cached_tokens: cacheRead };
51
+ }
52
+ if (typeof cacheWrite === "number") {
53
+ out.cache_creation_input_tokens = cacheWrite;
54
+ }
55
+ return out;
56
+ }
57
+ function stringifyToolInput(input) {
58
+ if (typeof input === "string")
59
+ return input;
60
+ try {
61
+ return JSON.stringify(input ?? {});
62
+ }
63
+ catch {
64
+ return "{}";
65
+ }
66
+ }
67
+ /** Non-streaming Anthropic message → Chat Completions object. */
68
+ export function anthropicMessageToChatCompletion(message, fallbackModel = "") {
69
+ const blocks = Array.isArray(message.content) ? message.content : [];
70
+ const textParts = [];
71
+ const thinkingParts = [];
72
+ const toolCalls = [];
73
+ for (const raw of blocks) {
74
+ const block = asRecord(raw);
75
+ if (!block)
76
+ continue;
77
+ const type = String(block.type || "");
78
+ if (type === "text" && typeof block.text === "string") {
79
+ textParts.push(block.text);
80
+ continue;
81
+ }
82
+ if (type === "thinking" && typeof block.thinking === "string") {
83
+ thinkingParts.push(block.thinking);
84
+ continue;
85
+ }
86
+ if (type === "tool_use") {
87
+ toolCalls.push({
88
+ index: toolCalls.length,
89
+ id: String(block.id || newId("call")),
90
+ type: "function",
91
+ function: {
92
+ name: String(block.name || "tool"),
93
+ arguments: stringifyToolInput(block.input),
94
+ },
95
+ });
96
+ }
97
+ }
98
+ const chatMessage = {
99
+ role: "assistant",
100
+ content: textParts.length ? textParts.join("") : null,
101
+ };
102
+ if (thinkingParts.length) {
103
+ chatMessage.reasoning_content = thinkingParts.join("");
104
+ }
105
+ if (toolCalls.length)
106
+ chatMessage.tool_calls = toolCalls;
107
+ const finishReason = anthropicStopReasonToFinishReason(message.stop_reason) ??
108
+ (toolCalls.length ? "tool_calls" : "stop");
109
+ const out = {
110
+ id: typeof message.id === "string" && message.id
111
+ ? message.id.replace(/^msg_/, "chatcmpl-")
112
+ : newId("chatcmpl"),
113
+ object: "chat.completion",
114
+ created: Math.floor(Date.now() / 1000),
115
+ model: String(message.model || fallbackModel || ""),
116
+ choices: [
117
+ {
118
+ index: 0,
119
+ message: chatMessage,
120
+ finish_reason: finishReason,
121
+ logprobs: null,
122
+ },
123
+ ],
124
+ };
125
+ const usage = mapUsage(asRecord(message.usage));
126
+ if (usage)
127
+ out.usage = usage;
128
+ return out;
129
+ }
130
+ export function createAnthropicToChatStreamState(model, options = {}) {
131
+ return {
132
+ id: newId("chatcmpl"),
133
+ model,
134
+ created: Math.floor(Date.now() / 1000),
135
+ blocks: new Map(),
136
+ nextToolIndex: 0,
137
+ roleEmitted: false,
138
+ finishReason: null,
139
+ inputTokens: 0,
140
+ outputTokens: 0,
141
+ hasUsage: false,
142
+ completed: false,
143
+ includeUsage: options.includeUsage !== false,
144
+ };
145
+ }
146
+ function chunk(state, delta, finishReason = null) {
147
+ return {
148
+ id: state.id,
149
+ object: "chat.completion.chunk",
150
+ created: state.created,
151
+ model: state.model,
152
+ choices: [
153
+ {
154
+ index: 0,
155
+ delta,
156
+ finish_reason: finishReason,
157
+ logprobs: null,
158
+ },
159
+ ],
160
+ };
161
+ }
162
+ function ensureRole(state, out) {
163
+ if (state.roleEmitted)
164
+ return;
165
+ state.roleEmitted = true;
166
+ out.push(chunk(state, { role: "assistant", content: "" }));
167
+ }
168
+ /**
169
+ * Parse one Anthropic SSE line into an event object. Anthropic sends
170
+ * `event: <name>` followed by `data: {...}`; the JSON payload carries `type`,
171
+ * so only data lines are meaningful.
172
+ */
173
+ export function parseAnthropicSseLine(line) {
174
+ const trimmed = line.trim();
175
+ if (!trimmed.startsWith("data:"))
176
+ return null;
177
+ const data = trimmed.slice(5).trim();
178
+ if (!data)
179
+ return null;
180
+ if (data === "[DONE]")
181
+ return "done";
182
+ try {
183
+ return JSON.parse(data);
184
+ }
185
+ catch {
186
+ return null;
187
+ }
188
+ }
189
+ export class AnthropicStreamError extends Error {
190
+ errorType;
191
+ constructor(message, errorType = "api_error") {
192
+ super(message);
193
+ this.errorType = errorType;
194
+ this.name = "AnthropicStreamError";
195
+ }
196
+ }
197
+ /**
198
+ * Convert one Anthropic SSE event into zero or more Chat Completions chunks.
199
+ * Throws `AnthropicStreamError` when the upstream emits an error event.
200
+ */
201
+ export function anthropicEventToChatChunks(event, state) {
202
+ const out = [];
203
+ const type = String(event.type || "");
204
+ switch (type) {
205
+ case "message_start": {
206
+ const message = asRecord(event.message);
207
+ if (message) {
208
+ if (typeof message.model === "string" && message.model) {
209
+ state.model = message.model;
210
+ }
211
+ if (typeof message.id === "string" && message.id) {
212
+ state.id = message.id.replace(/^msg_/, "chatcmpl-");
213
+ }
214
+ const usage = asRecord(message.usage);
215
+ if (usage) {
216
+ state.inputTokens = numberOr(usage.input_tokens, state.inputTokens);
217
+ state.outputTokens = numberOr(usage.output_tokens, state.outputTokens);
218
+ state.hasUsage = true;
219
+ }
220
+ }
221
+ ensureRole(state, out);
222
+ return out;
223
+ }
224
+ case "content_block_start": {
225
+ ensureRole(state, out);
226
+ const index = numberOr(event.index, -1);
227
+ const block = asRecord(event.content_block);
228
+ if (!block || index < 0)
229
+ return out;
230
+ if (String(block.type || "") !== "tool_use")
231
+ return out;
232
+ const toolIndex = state.nextToolIndex++;
233
+ const id = String(block.id || newId("call"));
234
+ const name = String(block.name || "tool");
235
+ state.blocks.set(index, { toolIndex, id, name });
236
+ out.push(chunk(state, {
237
+ tool_calls: [
238
+ {
239
+ index: toolIndex,
240
+ id,
241
+ type: "function",
242
+ function: { name, arguments: "" },
243
+ },
244
+ ],
245
+ }));
246
+ return out;
247
+ }
248
+ case "content_block_delta": {
249
+ ensureRole(state, out);
250
+ const delta = asRecord(event.delta);
251
+ if (!delta)
252
+ return out;
253
+ const deltaType = String(delta.type || "");
254
+ if (deltaType === "text_delta" && typeof delta.text === "string") {
255
+ if (delta.text)
256
+ out.push(chunk(state, { content: delta.text }));
257
+ return out;
258
+ }
259
+ if (deltaType === "thinking_delta" && typeof delta.thinking === "string") {
260
+ if (delta.thinking) {
261
+ out.push(chunk(state, { reasoning_content: delta.thinking }));
262
+ }
263
+ return out;
264
+ }
265
+ if (deltaType === "input_json_delta" &&
266
+ typeof delta.partial_json === "string") {
267
+ const index = numberOr(event.index, -1);
268
+ const entry = state.blocks.get(index);
269
+ if (!entry || !delta.partial_json)
270
+ return out;
271
+ out.push(chunk(state, {
272
+ tool_calls: [
273
+ {
274
+ index: entry.toolIndex,
275
+ function: { arguments: delta.partial_json },
276
+ },
277
+ ],
278
+ }));
279
+ }
280
+ return out;
281
+ }
282
+ case "message_delta": {
283
+ const delta = asRecord(event.delta);
284
+ const finish = anthropicStopReasonToFinishReason(delta?.stop_reason);
285
+ if (finish)
286
+ state.finishReason = finish;
287
+ const usage = asRecord(event.usage);
288
+ if (usage) {
289
+ state.outputTokens = numberOr(usage.output_tokens, state.outputTokens);
290
+ state.inputTokens = numberOr(usage.input_tokens, state.inputTokens);
291
+ state.hasUsage = true;
292
+ }
293
+ return out;
294
+ }
295
+ case "message_stop":
296
+ return finishAnthropicToChatStream(state);
297
+ case "error": {
298
+ const error = asRecord(event.error);
299
+ throw new AnthropicStreamError(String(error?.message || "上游 Anthropic 流返回错误"), String(error?.type || "api_error"));
300
+ }
301
+ default:
302
+ // ping / content_block_stop / unknown events carry no chat payload.
303
+ return out;
304
+ }
305
+ }
306
+ function finishAnthropicToChatStream(state) {
307
+ if (state.completed)
308
+ return [];
309
+ state.completed = true;
310
+ const out = [];
311
+ const finishReason = state.finishReason || (state.blocks.size ? "tool_calls" : "stop");
312
+ out.push(chunk(state, {}, finishReason));
313
+ if (state.includeUsage && state.hasUsage) {
314
+ out.push({
315
+ id: state.id,
316
+ object: "chat.completion.chunk",
317
+ created: state.created,
318
+ model: state.model,
319
+ choices: [],
320
+ usage: {
321
+ prompt_tokens: state.inputTokens,
322
+ completion_tokens: state.outputTokens,
323
+ total_tokens: state.inputTokens + state.outputTokens,
324
+ },
325
+ });
326
+ }
327
+ return out;
328
+ }
329
+ /** Emit the terminal chunks when the upstream stream ends without message_stop. */
330
+ export function forceCompleteAnthropicToChatStream(state) {
331
+ return finishAnthropicToChatStream(state);
332
+ }