@alilis/k-hat 0.2.6 → 0.2.8
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +36 -138
- package/dist/{cli.js → cli/cli.js} +64 -12
- package/dist/{config.js → core/config.js} +3 -0
- package/dist/core/probe.js +15 -0
- package/dist/{key-protector.js → data/key-protector.js} +1 -1
- package/dist/{portable-vault.js → data/portable-vault.js} +2 -2
- package/dist/{store.js → data/store.js} +11 -4
- package/dist/{vault.js → data/vault.js} +1 -1
- package/dist/{admin.js → network/admin.js} +17 -17
- package/dist/{selector.js → network/selector.js} +13 -1
- package/dist/network/server.js +383 -0
- package/dist/service.js +283 -0
- package/dist/{tui-state.js → ui/tui/tui-state.js} +4 -2
- package/dist/{tui.js → ui/tui/tui.js} +7 -5
- package/dist/{web-ui.js → ui/web-ui.js} +42 -11
- package/package.json +6 -5
- package/dist/server.js +0 -238
- /package/dist/{logger.js → core/logger.js} +0 -0
- /package/dist/{router.js → core/router.js} +0 -0
- /package/dist/{types.js → core/types.js} +0 -0
- /package/dist/{dpapi.js → data/dpapi.js} +0 -0
- /package/dist/{tui-client.js → ui/tui/tui-client.js} +0 -0
- /package/dist/{tui-main.js → ui/tui/tui-main.js} +0 -0
- /package/dist/{tui-types.js → ui/tui/tui-types.js} +0 -0
package/README.md
CHANGED
|
@@ -1,82 +1,19 @@
|
|
|
1
|
-
|
|
1
|
+
## 停止后台代理
|
|
2
2
|
|
|
3
|
-
|
|
4
|
-
|
|
5
|
-
把 Codex、OpenCode、ZCode 等 Agent Tool 的 API 端点统一指向 khat,之后更换任何 key 都不必再碰这些工具的配置。当一个 key 触发限流(429)、鉴权失败(401)或余额耗尽(402)时,代理自动切换到同 Provider 的其他 key,你的编码工作流不中断。
|
|
6
|
-
|
|
7
|
-
## 特性
|
|
8
|
-
|
|
9
|
-
- **单端口双协议**:一个 HTTP 入口同时承载 OpenAI 族与 Anthropic 族协议,按路径区分
|
|
10
|
-
- **模型路由**:按请求中的 `model` 字段路由到目标 Provider,支持 `provider/model` 显式消歧
|
|
11
|
-
- **多 key 加权轮询**:一个 Provider 可配多个 key,按权重平滑分配请求
|
|
12
|
-
- **自动切换**:上游返回 429 / 401 / 402 时标记 key 不可用并换下一个 key 重发(仅首字节前)
|
|
13
|
-
- **加密密钥库**:全部机密(key 明文、访问令牌)存放在 `vault.json` 的加密 payload 中,payload 使用 AES-256-GCM 加密,32 字节主密钥由操作系统绑定(Windows DPAPI)保护,配置文件零机密可安全备份
|
|
14
|
-
- **流式透传**:SSE 逐 chunk 透传,不缓冲、不改写响应内容;客户端断开级联取消上游请求
|
|
15
|
-
- **后台守护与自愈**:`khat start` 在后台启动 supervisor,崩溃后指数退避重启;`khat stop` 优雅停止,`--foreground` 用于诊断
|
|
16
|
-
- **可观测性**:零机密 JSONL 请求日志、按 key 的请求/失败/字节/Token 计数器、`khat log` 与 `/_keys/logs`
|
|
17
|
-
- **本地管理页**:`khat ui` 以 60 秒单次票据打开内嵌 UI,查看状态、日志并恢复不可用 key
|
|
18
|
-
- **零运行时依赖**:纯 Node.js/TypeScript,仅用 Node 内置模块
|
|
19
|
-
|
|
20
|
-
## 安装
|
|
21
|
-
|
|
22
|
-
**方式一:从 npm 安装(推荐)**
|
|
23
|
-
|
|
24
|
-
```bash
|
|
25
|
-
npm install -g @alilis/k-hat
|
|
26
|
-
```
|
|
27
|
-
|
|
28
|
-
之后任意目录都能直接使用下文的 `khat <命令>`。卸载用 `npm uninstall -g @alilis/k-hat`。
|
|
29
|
-
|
|
30
|
-
> Windows 下若提示找不到 `khat`,确认 `%APPDATA%\npm` 在 PATH 中(重开终端后生效)。
|
|
31
|
-
|
|
32
|
-
**方式二:从源码运行**
|
|
33
|
-
|
|
34
|
-
```bash
|
|
35
|
-
git clone https://github.com/aweirdocc/k-hat
|
|
36
|
-
cd k-hat
|
|
37
|
-
npm install # 安装依赖的同时会自动编译 TypeScript(prepare 钩子)
|
|
3
|
+
khat stop
|
|
38
4
|
```
|
|
39
5
|
|
|
40
|
-
|
|
41
|
-
|
|
42
|
-
```bash
|
|
43
|
-
npm run khat -- status
|
|
44
|
-
npm run khat -- tui
|
|
45
|
-
```
|
|
6
|
+
然后在你的工具中把 `base_url` 指向 `http://127.0.0.1:8787/v1`(或 `http://127.0.0.1:8787`,视工具而定),认证令牌填 `khat init` 生成的访问令牌。
|
|
46
7
|
|
|
47
|
-
|
|
8
|
+
### 服务化部署(M7)
|
|
48
9
|
|
|
49
10
|
```bash
|
|
50
|
-
#
|
|
51
|
-
khat
|
|
52
|
-
|
|
53
|
-
|
|
54
|
-
# 2. 添加 Provider
|
|
55
|
-
khat provider add deepseek --base-url https://api.deepseek.com --name DeepSeek --protocol openai
|
|
56
|
-
|
|
57
|
-
# 3. 添加 key(省略 --value 时交互式提示输入;--weight 为该 key 的轮询权重,默认 1)
|
|
58
|
-
khat key add deepseek main --weight 3
|
|
59
|
-
|
|
60
|
-
# 4. 添加路由:模型名 -> Provider
|
|
61
|
-
khat route add deepseek-v4-pro deepseek
|
|
62
|
-
|
|
63
|
-
# 5. 后台启动代理(管理变更命令也会在需要时自动启动它)
|
|
64
|
-
khat start
|
|
65
|
-
|
|
66
|
-
# 6. 查看状态
|
|
67
|
-
khat status
|
|
68
|
-
|
|
69
|
-
# 7. 打开本地管理页
|
|
70
|
-
khat ui
|
|
71
|
-
|
|
72
|
-
# 或在终端中打开交互式管理界面
|
|
73
|
-
khat tui
|
|
74
|
-
|
|
75
|
-
# 8. 停止后台代理
|
|
76
|
-
khat stop
|
|
11
|
+
khat service install # 注册开机自启(Windows 计划任务 / macOS launchd / Linux systemd --user)
|
|
12
|
+
khat service uninstall # 取消开机自启
|
|
13
|
+
khat service status # 查看服务注册状态与守护进程运行状态
|
|
77
14
|
```
|
|
78
15
|
|
|
79
|
-
|
|
16
|
+
> **注意**:service 安装时会固定 `KHAT_HOME` 为安装时解析的数据目录;`khat stop` 不会卸载服务,下次登录仍自动启动;Windows 上未采用 NSSM(因用户级 DPAPI vault 无法以 LocalSystem 运行)。
|
|
80
17
|
|
|
81
18
|
## CLI 命令
|
|
82
19
|
|
|
@@ -97,6 +34,28 @@ khat stop
|
|
|
97
34
|
|
|
98
35
|
> Provider、key 和路由的新增、更新、删除与恢复等管理变更通过本地 Admin API 执行;若 daemon 尚未运行,命令会自动启动它。
|
|
99
36
|
|
|
37
|
+
### 模型列表接口
|
|
38
|
+
|
|
39
|
+
代理提供带访问令牌认证的只读模型列表接口:
|
|
40
|
+
|
|
41
|
+
```bash
|
|
42
|
+
TOKEN=$(khat token show)
|
|
43
|
+
curl -s -H "Authorization: Bearer $TOKEN" http://127.0.0.1:8787/models
|
|
44
|
+
```
|
|
45
|
+
|
|
46
|
+
`GET /models` 和兼容 OpenAI 客户端的 `GET /v1/models` 返回当前配置中启用的 `routes`,不会请求上游 Provider,也不会返回 `upstreamModel`:
|
|
47
|
+
|
|
48
|
+
```json
|
|
49
|
+
{
|
|
50
|
+
"object": "list",
|
|
51
|
+
"data": [
|
|
52
|
+
{ "id": "deepseek-chat", "object": "model", "created": 0, "owned_by": "deepseek" }
|
|
53
|
+
]
|
|
54
|
+
}
|
|
55
|
+
```
|
|
56
|
+
|
|
57
|
+
未配置路由时 `data` 为空;禁用路由不会出现在列表中。请求必须携带 `Authorization: Bearer <access-token>`,否则返回 HTTP 401。
|
|
58
|
+
|
|
100
59
|
### 访问令牌
|
|
101
60
|
|
|
102
61
|
| 命令 | 说明 |
|
|
@@ -124,7 +83,7 @@ khat stop
|
|
|
124
83
|
| `khat key list [<provider>]` | 列出 key(默认脱敏,仅显示尾 4 位) |
|
|
125
84
|
| `khat key remove <provider> <id>` | 删除 key |
|
|
126
85
|
| `khat key enable <provider> <id>` | 将被标记不可用的 key 恢复为可用 |
|
|
127
|
-
| `khat key test <provider> <id> [--model <m>]` |
|
|
86
|
+
| `khat key test <provider> <id> [--model <m>]` | 直连配置的上游发送一次最小请求探活;成功后自动恢复 key,不会仅因探测返回 401/402/429 就标记 key 不可用 |
|
|
128
87
|
|
|
129
88
|
### 路由管理
|
|
130
89
|
|
|
@@ -140,7 +99,7 @@ khat stop
|
|
|
140
99
|
| 变量 | 说明 |
|
|
141
100
|
|---|---|
|
|
142
101
|
| `KHAT_HOME` | 数据目录(默认 `~/.khat`) |
|
|
143
|
-
| `KHAT_ACCESS_TOKEN` |
|
|
102
|
+
| `KHAT_ACCESS_TOKEN` | 覆盖密钥库中的访问密钥(主要供测试) |
|
|
144
103
|
|
|
145
104
|
## 常见异常与处理
|
|
146
105
|
|
|
@@ -162,68 +121,7 @@ khat stop
|
|
|
162
121
|
| `daemon started but health check failed` | 后台 daemon 启动异常 | `khat start --foreground` 前台运行观察日志 |
|
|
163
122
|
| `--weight must be a positive integer` | 权重不是正整数 | 传入 ≥1 的整数 |
|
|
164
123
|
| `unknown provider: <x>` / `unknown key: <x/y>` | 引用了不存在的 Provider / Key | 用 `khat provider list` / `khat key list` 核对 |
|
|
165
|
-
| `
|
|
166
|
-
|
|
|
167
|
-
| `
|
|
168
|
-
| `
|
|
169
|
-
|
|
170
|
-
### 代理返回的入口错误
|
|
171
|
-
|
|
172
|
-
| 状态码 | 含义 | 处理 |
|
|
173
|
-
|---|---|---|
|
|
174
|
-
| 400 | 入口路径与 Provider 协议不匹配,或请求体非法 | 检查工具配置的入口路径与 `base_url` |
|
|
175
|
-
| 401 | 访问令牌无效 | 用 `khat token show` 核对工具里配置的令牌 |
|
|
176
|
-
| 404 | 请求的模型没有匹配的路由 | `khat route add <model> <provider>` |
|
|
177
|
-
| 502 | 上游返回错误(如 key 探活失败) | 查看 `khat log` 与 key 状态 |
|
|
178
|
-
|
|
179
|
-
### 使用注意事项
|
|
180
|
-
|
|
181
|
-
- **变更命令会自动拉起 daemon**:`provider / key / route` 等管理命令通过本地 Admin API 执行,若后台服务未运行会自动启动(单写者设计),无需手动 `khat start`。
|
|
182
|
-
- **UI 票据单次有效**:`khat ui` 签发的票据 60 秒有效、只能交换一次会话;管理页保持打开不受影响,重新打开需再次 `khat ui`。
|
|
183
|
-
- **密钥库加密描述**:`vault.json` 是合法 JSON 信封,机密载荷经 AES-256-GCM 加密,32 字节主密钥由 Windows DPAPI 保护;文件本身不是整体二进制密文。
|
|
184
|
-
|
|
185
|
-
## 协议支持
|
|
186
|
-
|
|
187
|
-
| 入口路径 | 协议 | 认证头 | 服务的工具 |
|
|
188
|
-
|---|---|---|---|
|
|
189
|
-
| `/v1/chat/completions` | OpenAI Chat | `Authorization: Bearer` | OpenCode、Cursor(部分) |
|
|
190
|
-
| `/v1/responses` | OpenAI Responses | `Authorization: Bearer` | Codex |
|
|
191
|
-
| `/v1/messages` | Anthropic Messages | `x-api-key` + `anthropic-version` | ZCode |
|
|
192
|
-
|
|
193
|
-
采用**同族直通**策略:OpenAI 入口转发到 OpenAI 上游,Anthropic 入口转发到 Anthropic 上游,只重写认证头与目标 URL,不改写请求体。入口路径与 Provider 协议不匹配时返回 400。
|
|
194
|
-
|
|
195
|
-
## 数据安全
|
|
196
|
-
|
|
197
|
-
- **密钥库加密落盘**:`vault.json` 是 JSON 信封;其中 `payload` 使用 AES-256-GCM 加密,32 字节主密钥由 Windows DPAPI(CurrentUser 作用域)保护,key 明文仅存在于守护进程内存
|
|
198
|
-
- **配置零机密**:`config.json` 只存 `vaultRef` 引用,可放心进 Git/网盘备份
|
|
199
|
-
- **日志零机密**:不记录 key 明文、请求/响应体、认证头;CLI 展示 key 默认脱敏
|
|
200
|
-
- **入口双重防护**:所有请求校验 256-bit 访问令牌;默认只绑定 loopback,不对局域网暴露
|
|
201
|
-
- **上游强制 HTTPS**:证书校验,不提供禁用开关(仅 loopback 主机允许明文 http)
|
|
202
|
-
|
|
203
|
-
## 项目结构
|
|
204
|
-
|
|
205
|
-
```
|
|
206
|
-
src/
|
|
207
|
-
├── server.ts # HTTP 服务与转发器(双协议、认证头重写、超时、重试)
|
|
208
|
-
├── router.ts # 模型名 -> Provider 路由
|
|
209
|
-
├── selector.ts # 平滑加权轮询
|
|
210
|
-
├── store.ts # config/state/vault 存储层 + CLI 变更操作
|
|
211
|
-
├── vault.ts # AES-256-GCM 加密密钥库
|
|
212
|
-
├── config.ts # 配置加载、校验、原子写入
|
|
213
|
-
├── dpapi.ts # Windows DPAPI 主密钥保护
|
|
214
|
-
├── types.ts # 类型定义
|
|
215
|
-
└── cli.ts # CLI 入口
|
|
216
|
-
test/ # 自动化测试(node:test)
|
|
217
|
-
docs/ # 需求、可行性、概要设计、ADR
|
|
218
|
-
```
|
|
219
|
-
|
|
220
|
-
## 开发
|
|
221
|
-
|
|
222
|
-
```bash
|
|
223
|
-
npm test # 构建 + 运行全部测试
|
|
224
|
-
npm run build # 仅编译 TypeScript
|
|
225
|
-
npm start # 启动代理
|
|
226
|
-
npm run khat -- status # 源码运行任意 CLI 子命令
|
|
227
|
-
```
|
|
228
|
-
|
|
229
|
-
当前状态与设计决策见 [CONTEXT.md](./CONTEXT.md) 与 [docs/](./docs/)。
|
|
124
|
+
| `khat service is not installed` | service 注册不存在 | `khat service install` |
|
|
125
|
+
| `could not register the scheduled task` | Windows 计划任务注册失败 | 检查任务调度器权限,运行 `schtasks /Query /TN khat` 确认 |
|
|
126
|
+
| `could not load the launch agent` | macOS launchd 加载失败 | 检查 `~/Library/LaunchAgents/local.khat.plist` 是否被手动删除,运行 `launchctl list local.khat` |
|
|
127
|
+
| `could not enable the service` | Linux systemd 启用失败 | 检查 `~/.config/systemd/user/khat.service` 是否被删除,运行 `systemctl --user is-enabled khat` |
|
|
@@ -6,14 +6,15 @@ import { homedir } from 'node:os';
|
|
|
6
6
|
import { join } from 'node:path';
|
|
7
7
|
import { fileURLToPath } from 'node:url';
|
|
8
8
|
import { createInterface } from 'node:readline/promises';
|
|
9
|
-
import { saveJsonAtomic, providerUrl, defaultConfig } from '
|
|
10
|
-
import { createKhatServer } from '
|
|
11
|
-
import { openStore, maskSecret } from '
|
|
12
|
-
import { Vault, generateAccessToken, ACCESS_TOKEN_REF } from '
|
|
13
|
-
import { createKeyProtector } from '
|
|
14
|
-
import { exportPortable, importPortable } from '
|
|
15
|
-
import { LogWriter } from '
|
|
16
|
-
import { runDoctor as inspectDoctor, formatDoctorSuggestion } from '
|
|
9
|
+
import { saveJsonAtomic, providerUrl, defaultConfig } from '../core/config.js';
|
|
10
|
+
import { createKhatServer } from '../network/server.js';
|
|
11
|
+
import { openStore, maskSecret } from '../data/store.js';
|
|
12
|
+
import { Vault, generateAccessToken, ACCESS_TOKEN_REF } from '../data/vault.js';
|
|
13
|
+
import { createKeyProtector } from '../data/key-protector.js';
|
|
14
|
+
import { exportPortable, importPortable } from '../data/portable-vault.js';
|
|
15
|
+
import { LogWriter } from '../core/logger.js';
|
|
16
|
+
import { runDoctor as inspectDoctor, formatDoctorSuggestion } from '../doctor.js';
|
|
17
|
+
import { createServiceManager } from '../service.js';
|
|
17
18
|
const dataDir = process.env.KHAT_HOME ?? join(homedir(), '.khat');
|
|
18
19
|
const configPath = join(dataDir, 'config.json');
|
|
19
20
|
const statePath = join(dataDir, 'state.json');
|
|
@@ -30,6 +31,9 @@ Setup
|
|
|
30
31
|
start [--foreground] [--port <n>] start the proxy as a background daemon (or foreground with --foreground);
|
|
31
32
|
--port <n> changes the persisted listen port
|
|
32
33
|
stop stop the background daemon
|
|
34
|
+
service install register khat to auto-start at logon (Task Scheduler / launchd / systemd --user)
|
|
35
|
+
service uninstall remove the auto-start registration
|
|
36
|
+
service status show the service registration and daemon state
|
|
33
37
|
status show providers, keys, routes and key health
|
|
34
38
|
doctor detect local Agent Tools and show connection guidance
|
|
35
39
|
log [--tail <n>] show recent masked proxy request logs
|
|
@@ -195,7 +199,14 @@ function keyStatusLine(store, providerId, keyId) {
|
|
|
195
199
|
const state = store.states[`${providerId}/${keyId}`];
|
|
196
200
|
if (!state || state.status === 'available')
|
|
197
201
|
return { status: 'available', note: '' };
|
|
198
|
-
|
|
202
|
+
const lastError = `last error: ${state.lastError?.http ?? '?'} at ${state.lastError?.at ?? '?'}`;
|
|
203
|
+
if (state.status === 'cooldown') {
|
|
204
|
+
const remaining = Date.parse(state.cooldownUntil ?? '') - Date.now();
|
|
205
|
+
if (remaining <= 0)
|
|
206
|
+
return { status: 'available', note: '' };
|
|
207
|
+
return { status: 'cooldown', note: ` cooling down, auto-retry in ${Math.ceil(remaining / 1000)}s (${lastError})` };
|
|
208
|
+
}
|
|
209
|
+
return { status: 'unavailable', note: ` ${lastError}` };
|
|
199
210
|
}
|
|
200
211
|
function printStatus(store) {
|
|
201
212
|
const token = store.vault.get(ACCESS_TOKEN_REF);
|
|
@@ -243,6 +254,28 @@ async function daemonRunning() {
|
|
|
243
254
|
return false;
|
|
244
255
|
}
|
|
245
256
|
}
|
|
257
|
+
async function readDaemonPid() {
|
|
258
|
+
if (!(await daemonRunning()))
|
|
259
|
+
return undefined;
|
|
260
|
+
try {
|
|
261
|
+
return Number.parseInt((await readFile(pidPath, 'utf8')).trim(), 10);
|
|
262
|
+
}
|
|
263
|
+
catch {
|
|
264
|
+
return undefined;
|
|
265
|
+
}
|
|
266
|
+
}
|
|
267
|
+
function printServiceStatus(service, daemonPid) {
|
|
268
|
+
if (!service.installed) {
|
|
269
|
+
console.log('khat service is not installed (run khat service install)');
|
|
270
|
+
return;
|
|
271
|
+
}
|
|
272
|
+
const enabled = service.enabled === null ? 'unknown' : service.enabled ? 'on' : 'off';
|
|
273
|
+
const running = service.running === null ? 'unknown' : service.running ? 'yes' : 'no';
|
|
274
|
+
console.log('khat service');
|
|
275
|
+
console.log(` auto-start: ${enabled}`);
|
|
276
|
+
console.log(` running: ${running}${service.detail ? ` (${service.detail})` : ''}`);
|
|
277
|
+
console.log(` daemon: ${daemonPid !== undefined ? `running (pid ${daemonPid})` : 'not running'}`);
|
|
278
|
+
}
|
|
246
279
|
async function runForeground() {
|
|
247
280
|
const store = await openInitializedStore();
|
|
248
281
|
const secrets = {};
|
|
@@ -368,7 +401,7 @@ async function runTui() {
|
|
|
368
401
|
if (!token)
|
|
369
402
|
throw new Error('no access token found; run khat init');
|
|
370
403
|
await ensureDaemon();
|
|
371
|
-
const { runTui: startTui } = await import('
|
|
404
|
+
const { runTui: startTui } = await import('../ui/tui/tui-main.js');
|
|
372
405
|
await startTui(`http://${store.config.bind}:${store.config.port}`, token);
|
|
373
406
|
}
|
|
374
407
|
async function runUi() {
|
|
@@ -402,7 +435,7 @@ async function runLog(flags) {
|
|
|
402
435
|
return;
|
|
403
436
|
}
|
|
404
437
|
for (const entry of entries) {
|
|
405
|
-
console.log(`${entry.ts}\t${entry.status}\t${entry.provider}/${entry.key.split('/').pop()}\t${entry.model}\t${entry.durationMs}
|
|
438
|
+
console.log(`${entry.ts}\t${entry.status}\t${entry.provider}/${entry.key.split('/').pop()}\t${entry.model ?? '(probe)'}\t${entry.durationMs ?? '—'}\t${entry.tokensIn ?? 0}/${entry.tokensOut ?? 0} tokens`);
|
|
406
439
|
}
|
|
407
440
|
}
|
|
408
441
|
async function runDoctor() {
|
|
@@ -497,6 +530,25 @@ try {
|
|
|
497
530
|
case 'stop':
|
|
498
531
|
await runStop();
|
|
499
532
|
break;
|
|
533
|
+
case 'service': {
|
|
534
|
+
const action = positionals[0];
|
|
535
|
+
const manager = createServiceManager(process.platform, { dataDir });
|
|
536
|
+
if (action === 'install') {
|
|
537
|
+
await openInitializedStore();
|
|
538
|
+
console.log(await manager.install({ startNow: !(await daemonRunning()) }));
|
|
539
|
+
}
|
|
540
|
+
else if (action === 'uninstall') {
|
|
541
|
+
console.log(await manager.uninstall());
|
|
542
|
+
}
|
|
543
|
+
else if (action === 'status') {
|
|
544
|
+
printServiceStatus(await manager.status(), await readDaemonPid());
|
|
545
|
+
}
|
|
546
|
+
else {
|
|
547
|
+
console.error('usage: khat service install | uninstall | status');
|
|
548
|
+
process.exitCode = 1;
|
|
549
|
+
}
|
|
550
|
+
break;
|
|
551
|
+
}
|
|
500
552
|
case 'status':
|
|
501
553
|
printStatus(await openInitializedStore());
|
|
502
554
|
break;
|
|
@@ -677,7 +729,7 @@ try {
|
|
|
677
729
|
console.log(`${providerId}/${keyId} is available`);
|
|
678
730
|
}
|
|
679
731
|
else if ([401, 402, 429].includes(response.status))
|
|
680
|
-
console.log('this error marks a key unavailable during proxying; check the key on the provider side');
|
|
732
|
+
console.log(response.status === 429 ? 'a 429 during proxying puts the key in cooldown; it revives automatically when the cooldown expires' : 'this error marks a key unavailable during proxying; check the key on the provider side');
|
|
681
733
|
}
|
|
682
734
|
else {
|
|
683
735
|
console.error('usage: khat key add | list | update | remove | enable | test');
|
|
@@ -53,6 +53,9 @@ export function validateConfig(config) {
|
|
|
53
53
|
if (!Number.isInteger(config.port) || config.port < 1 || config.port > 65535) {
|
|
54
54
|
throw new Error('Invalid config: port must be an integer between 1 and 65535');
|
|
55
55
|
}
|
|
56
|
+
if (config.revision !== undefined && (!Number.isInteger(config.revision) || config.revision < 0)) {
|
|
57
|
+
throw new Error('Invalid config: revision must be a non-negative integer');
|
|
58
|
+
}
|
|
56
59
|
for (const provider of config.providers) {
|
|
57
60
|
if (!provider.id || !provider.baseUrl || (provider.protocol !== 'openai' && provider.protocol !== 'anthropic') || !Array.isArray(provider.keys))
|
|
58
61
|
throw new Error(`Invalid provider: ${provider.id}`);
|
|
@@ -0,0 +1,15 @@
|
|
|
1
|
+
import { providerUrl } from '../core/config.js';
|
|
2
|
+
const ANTHROPIC_VERSION = '2023-06-01';
|
|
3
|
+
/** Minimal one-token ping shared by the Admin probe endpoint and the background prober. */
|
|
4
|
+
export async function probeKey(provider, model, secret) {
|
|
5
|
+
const path = provider.protocol === 'anthropic' ? '/v1/messages' : '/v1/chat/completions';
|
|
6
|
+
const headers = provider.protocol === 'anthropic'
|
|
7
|
+
? { 'content-type': 'application/json', 'x-api-key': secret, 'anthropic-version': ANTHROPIC_VERSION }
|
|
8
|
+
: { 'content-type': 'application/json', authorization: `Bearer ${secret}` };
|
|
9
|
+
const body = provider.protocol === 'anthropic'
|
|
10
|
+
? { model, max_tokens: 1, messages: [{ role: 'user', content: 'ping' }] }
|
|
11
|
+
: { model, max_tokens: 1, messages: [{ role: 'user', content: 'ping' }], stream: false };
|
|
12
|
+
const response = await fetch(providerUrl(provider.baseUrl, path), { method: 'POST', headers, body: JSON.stringify(body), signal: AbortSignal.timeout(30_000) });
|
|
13
|
+
await response.arrayBuffer();
|
|
14
|
+
return response.status;
|
|
15
|
+
}
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
import { spawn } from 'node:child_process';
|
|
2
2
|
import { createCipheriv, createDecipheriv, randomBytes } from 'node:crypto';
|
|
3
|
-
import { dpapiProtector } from '
|
|
3
|
+
import { dpapiProtector } from '../data/dpapi.js';
|
|
4
4
|
const SERVICE = 'khat-vault';
|
|
5
5
|
const ACCOUNT = 'khat';
|
|
6
6
|
function run(command, args, input = '') {
|
|
@@ -1,8 +1,8 @@
|
|
|
1
1
|
import { createCipheriv, createDecipheriv, pbkdf2Sync, randomBytes } from 'node:crypto';
|
|
2
2
|
import { access, mkdir, readFile, rename, rm } from 'node:fs/promises';
|
|
3
3
|
import { join } from 'node:path';
|
|
4
|
-
import { saveJsonAtomic, validateConfig, defaultConfig } from '
|
|
5
|
-
import { Vault } from '
|
|
4
|
+
import { saveJsonAtomic, validateConfig, defaultConfig } from '../core/config.js';
|
|
5
|
+
import { Vault } from '../data/vault.js';
|
|
6
6
|
const FORMAT = 'khat-vault-export';
|
|
7
7
|
const VERSION = 1;
|
|
8
8
|
const AAD = 'khat-vault-export-v1';
|
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
import { mkdir } from 'node:fs/promises';
|
|
2
2
|
import { join } from 'node:path';
|
|
3
|
-
import { readJsonFile, saveJsonAtomic, loadState, validateConfig, defaultConfig } from '
|
|
4
|
-
import { Vault } from '
|
|
3
|
+
import { readJsonFile, saveJsonAtomic, loadState, validateConfig, defaultConfig } from '../core/config.js';
|
|
4
|
+
import { Vault } from '../data/vault.js';
|
|
5
5
|
/** Ids appear in `provider/model` routing syntax and in vault refs, so they cannot contain `/`. */
|
|
6
6
|
const ID_PATTERN = /^[A-Za-z0-9_-]+$/;
|
|
7
7
|
export function maskSecret(value) {
|
|
@@ -43,7 +43,12 @@ export async function openStore(dir, protector) {
|
|
|
43
43
|
release();
|
|
44
44
|
}
|
|
45
45
|
},
|
|
46
|
-
saveConfig: () =>
|
|
46
|
+
saveConfig: () => {
|
|
47
|
+
// Monotonic marker: lets the config watcher tell our own writes (and stale reads of them)
|
|
48
|
+
// apart from genuine external edits.
|
|
49
|
+
config.revision = Number.isInteger(config.revision) ? config.revision + 1 : 1;
|
|
50
|
+
return saveJsonAtomic(configPath, config);
|
|
51
|
+
},
|
|
47
52
|
saveState: () => saveJsonAtomic(statePath, { keys: store.states, counters: store.counters }),
|
|
48
53
|
recordCounter: (keyRef, delta) => {
|
|
49
54
|
const current = store.counters[keyRef] ?? { requests: 0, failed: 0, bytesOut: 0, tokensIn: 0, tokensOut: 0 };
|
|
@@ -57,7 +62,9 @@ export async function openStore(dir, protector) {
|
|
|
57
62
|
if (!stateTimer)
|
|
58
63
|
stateTimer = setTimeout(() => {
|
|
59
64
|
stateTimer = undefined;
|
|
60
|
-
|
|
65
|
+
store.saveState().catch((error) => {
|
|
66
|
+
console.error(`[khat] failed to persist state: ${error?.message ?? error}`);
|
|
67
|
+
});
|
|
61
68
|
}, 1_000);
|
|
62
69
|
stateTimer.unref();
|
|
63
70
|
}
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
import { randomBytes, createCipheriv, createDecipheriv } from 'node:crypto';
|
|
2
2
|
import { readFile } from 'node:fs/promises';
|
|
3
|
-
import { saveJsonAtomic } from '
|
|
3
|
+
import { saveJsonAtomic } from '../core/config.js';
|
|
4
4
|
const VAULT_VERSION = 1;
|
|
5
5
|
const AAD = 'khat-vault-v1';
|
|
6
6
|
// Releases before the kay->khat rename wrote their payload under this AAD; files saved that way stay readable and are rewritten under the current AAD on their next save().
|
|
@@ -1,11 +1,13 @@
|
|
|
1
|
-
import { addProvider, updateProvider, removeProvider, addKey, updateKey, removeKey, enableKey, setKeyEnabled, addRoute, updateRoute, setRouteEnabled, removeRoute, maskSecret } from '
|
|
2
|
-
import { LogWriter } from '
|
|
1
|
+
import { addProvider, updateProvider, removeProvider, addKey, updateKey, removeKey, enableKey, setKeyEnabled, addRoute, updateRoute, setRouteEnabled, removeRoute, maskSecret } from '../data/store.js';
|
|
2
|
+
import { LogWriter } from '../core/logger.js';
|
|
3
3
|
import { join } from 'node:path';
|
|
4
|
-
import { UiSessions, serveUi } from '
|
|
5
|
-
import { ACCESS_TOKEN_REF, generateAccessToken } from '
|
|
6
|
-
import { providerUrl } from '
|
|
7
|
-
import { findRoute, routeUpstreamModel } from '
|
|
8
|
-
import {
|
|
4
|
+
import { UiSessions, serveUi } from '../ui/web-ui.js';
|
|
5
|
+
import { ACCESS_TOKEN_REF, generateAccessToken } from '../data/vault.js';
|
|
6
|
+
import { providerUrl } from '../core/config.js';
|
|
7
|
+
import { findRoute, routeUpstreamModel } from '../core/router.js';
|
|
8
|
+
import { keyBlocked } from './selector.js';
|
|
9
|
+
import { probeKey } from '../core/probe.js';
|
|
10
|
+
import { readExport, applyImport } from '../data/portable-vault.js';
|
|
9
11
|
// Admin API is the daemon's single-writer surface (ADR-0005). It is reachable
|
|
10
12
|
// only from loopback and behind the proxy access token, so a local non-loopback
|
|
11
13
|
// client or a process without the token cannot mutate config/state/vault.
|
|
@@ -39,7 +41,7 @@ async function listProviderModels(store, providerId) {
|
|
|
39
41
|
const provider = store.config.providers.find((item) => item.id === providerId);
|
|
40
42
|
if (!provider)
|
|
41
43
|
throw new Error(`unknown provider: ${providerId}`);
|
|
42
|
-
const key = provider.keys.find((item) => item.enabled !== false && store.states[`${provider.id}/${item.id}`]
|
|
44
|
+
const key = provider.keys.find((item) => item.enabled !== false && !keyBlocked(store.states[`${provider.id}/${item.id}`]));
|
|
43
45
|
if (!key)
|
|
44
46
|
throw new Error('provider has no available key');
|
|
45
47
|
const secret = store.vault.get(key.vaultRef);
|
|
@@ -67,11 +69,14 @@ function buildStatus(store) {
|
|
|
67
69
|
keys: provider.keys.map((key) => {
|
|
68
70
|
const state = store.states[`${provider.id}/${key.id}`];
|
|
69
71
|
const secret = store.vault.get(key.vaultRef);
|
|
72
|
+
// An expired cooldown is selectable again, so it must surface as available.
|
|
73
|
+
const status = keyBlocked(state) ? state.status : 'available';
|
|
70
74
|
return {
|
|
71
75
|
id: key.id,
|
|
72
76
|
weight: key.weight,
|
|
73
77
|
...(key.enabled === false ? { enabled: false } : {}),
|
|
74
|
-
status
|
|
78
|
+
status,
|
|
79
|
+
...(status === 'cooldown' ? { cooldownUntil: state.cooldownUntil } : {}),
|
|
75
80
|
lastError: state?.lastError,
|
|
76
81
|
secret: secret ? maskSecret(secret) : null,
|
|
77
82
|
counters: store.counters[`${provider.id}/${key.id}`] ?? { requests: 0, failed: 0, bytesOut: 0, tokensIn: 0, tokensOut: 0 }
|
|
@@ -206,15 +211,10 @@ async function route(req, res, store, sessionAuthorized = false) {
|
|
|
206
211
|
: store.config.routes.find((item) => item.provider === provider.id);
|
|
207
212
|
if (!secret || !route || route.provider !== provider.id)
|
|
208
213
|
throw new Error('a key and route model are required for probing');
|
|
209
|
-
const
|
|
210
|
-
|
|
211
|
-
const headers = upstreamHeaders(provider.protocol, secret, true);
|
|
212
|
-
const body = provider.protocol === 'anthropic' ? { model, max_tokens: 1, messages: [{ role: 'user', content: 'ping' }] } : { model, max_tokens: 1, messages: [{ role: 'user', content: 'ping' }], stream: false };
|
|
213
|
-
const response = await fetch(providerUrl(provider.baseUrl, path), { method: 'POST', headers: headers, body: JSON.stringify(body), signal: AbortSignal.timeout(30_000) });
|
|
214
|
-
await response.arrayBuffer();
|
|
215
|
-
if (response.ok)
|
|
214
|
+
const status = await probeKey(provider, routeUpstreamModel(route), secret);
|
|
215
|
+
if (status >= 200 && status < 300)
|
|
216
216
|
await store.mutate(() => enableKey(store, provider.id, key.id));
|
|
217
|
-
return json(res,
|
|
217
|
+
return json(res, status >= 200 && status < 300 ? 200 : 502, { ok: status >= 200 && status < 300, status });
|
|
218
218
|
}
|
|
219
219
|
}
|
|
220
220
|
if (sub[0] === 'routes') {
|
|
@@ -1,7 +1,19 @@
|
|
|
1
|
+
/** True when a key must not be selected: permanently unavailable, or still cooling down after a 429. */
|
|
2
|
+
export function keyBlocked(state, now = Date.now()) {
|
|
3
|
+
if (!state)
|
|
4
|
+
return false;
|
|
5
|
+
if (state.status === 'unavailable')
|
|
6
|
+
return true;
|
|
7
|
+
if (state.status === 'cooldown') {
|
|
8
|
+
const until = Date.parse(state.cooldownUntil ?? '');
|
|
9
|
+
return Number.isNaN(until) || until > now;
|
|
10
|
+
}
|
|
11
|
+
return false;
|
|
12
|
+
}
|
|
1
13
|
export class WeightedSelector {
|
|
2
14
|
current = new Map();
|
|
3
15
|
select(providerId, keys, states) {
|
|
4
|
-
const available = keys.filter((key) => key.enabled !== false && states[`${providerId}/${key.id}`]
|
|
16
|
+
const available = keys.filter((key) => key.enabled !== false && !keyBlocked(states[`${providerId}/${key.id}`]));
|
|
5
17
|
if (!available.length)
|
|
6
18
|
return undefined;
|
|
7
19
|
let total = 0;
|