pi-ccswitch-auto-switch 0.1.7 → 0.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +9 -7
- package/README.zh-CN.md +9 -7
- package/candidates.ts +52 -6
- package/classify.ts +7 -3
- package/health.ts +60 -6
- package/index.ts +51 -17
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -13,7 +13,8 @@ The extension observes real Pi requests, records sanitized health signals, and
|
|
|
13
13
|
- Endpoint circuit breakers for DNS, connection, server, and streaming failures.
|
|
14
14
|
- Endpoint platform isolation: when several models on the same endpoint (same BaseURL/provider) fail within one round, the whole endpoint is isolated so sibling models from the same platform are not tried one by one.
|
|
15
15
|
- Model-only isolation for missing models and incompatible parameters.
|
|
16
|
-
-
|
|
16
|
+
- Learns model-family content-policy constraints from real failures (`glm-4.5` and `GLM-4.6`, for example, are both `glm`) and avoids the entire family during current and future content-policy failover chains. Evidence expires after 30 days.
|
|
17
|
+
- Applies hard health, failure-domain, policy-family, input-modality, and context filters before ranking candidates by provider diversity, model equivalence, capability compatibility, and historical reliability.
|
|
17
18
|
- Independent-provider semantics: a copied vendor (e.g. `my-provider-copy`) shares the BaseURL but is a distinct provider with its own endpoint key, so isolating one never blocks the other.
|
|
18
19
|
- 90-second first-response and 120-second streaming-idle watchdogs.
|
|
19
20
|
- Exponential cooldowns, `Retry-After` support, and one persisted half-open probe lease per provider.
|
|
@@ -54,7 +55,7 @@ CC Switch should be configured normally. This extension deliberately does not wr
|
|
|
54
55
|
| `/ccswitch refresh` | Refresh Pi's model registry and the status display. |
|
|
55
56
|
| `/ccswitch reactivate <provider/model\|all>` | Clear a breaker and let the next real request verify recovery, while preserving history. |
|
|
56
57
|
| `/ccswitch disable <provider/model>` | Manually exclude a model from failover. |
|
|
57
|
-
| `/ccswitch reset <provider/model\|all>` | Delete selected health history after confirmation. |
|
|
58
|
+
| `/ccswitch reset <provider/model\|all>` | Delete selected health history after confirmation; `all` also clears learned content-policy constraints. |
|
|
58
59
|
| `/ccswitch-test` | Inspect candidate discovery without changing models. |
|
|
59
60
|
|
|
60
61
|
Examples:
|
|
@@ -90,15 +91,16 @@ The state machine waits for Pi's native retry cycle to settle before switching.
|
|
|
90
91
|
| --- | --- | --- |
|
|
91
92
|
| `401`, `403`, quota, billing | Provider | 30 minutes |
|
|
92
93
|
| `429` | Provider | `Retry-After` when present; otherwise 5 minutes |
|
|
93
|
-
| DNS, network, `5xx`, interrupted stream | Endpoint | 2 minutes |
|
|
94
|
+
| DNS, network, `408`, `5xx`, interrupted stream | Endpoint | 2 minutes |
|
|
94
95
|
| `404`, invalid model, incompatible parameters | Model | 15 minutes |
|
|
95
|
-
| Content
|
|
96
|
+
| Content-policy or sensitive-content rejection | Current model + learned model-family constraint | 2-minute model cooldown; 30-day family evidence |
|
|
97
|
+
| Context overflow | Current round only | None |
|
|
96
98
|
|
|
97
|
-
Cooldowns grow exponentially within bounded limits. Context-overflow retries only consider models with a larger context window.
|
|
99
|
+
Cooldowns grow exponentially within bounded limits. Learned policy-constrained families are excluded only after a content-policy rejection; they remain eligible during ordinary rate-limit, network, and model-configuration failovers. Context-overflow retries only consider models with a larger context window, and image requests do not move to a model that explicitly supports text only. Unknown errors are isolated to the current model and still fail over normally. A user cancellation is the only aborted turn that does not trigger failover; watchdog cancellations are recorded as timeouts.
|
|
98
100
|
|
|
99
101
|
## Data and privacy
|
|
100
102
|
|
|
101
|
-
Health state is stored in Pi's agent directory as `ccswitch-auto-switch-state.json`. The extension stores counters, timestamps, cooldowns, and redacted/truncated error summaries. It does not access credentials, authorization headers, CC Switch's database, or Pi's `auth.json`.
|
|
103
|
+
Health state is stored in Pi's agent directory as `ccswitch-auto-switch-state.json`. The extension stores counters, timestamps, cooldowns, learned model-family policy constraints, and redacted/truncated error summaries. It does not access credentials, authorization headers, CC Switch's database, or Pi's `auth.json`.
|
|
102
104
|
|
|
103
105
|
## Development
|
|
104
106
|
|
|
@@ -108,7 +110,7 @@ npm run typecheck
|
|
|
108
110
|
npm test
|
|
109
111
|
```
|
|
110
112
|
|
|
111
|
-
Tests use Node's built-in test runner and cover failure
|
|
113
|
+
Tests use Node's built-in test runner and cover the failure matrix, family-level policy avoidance, provider-first selection, input compatibility, cooldowns, state persistence, and Windows-compatible paths.
|
|
112
114
|
|
|
113
115
|
## License
|
|
114
116
|
|
package/README.zh-CN.md
CHANGED
|
@@ -12,7 +12,8 @@
|
|
|
12
12
|
- 对认证、额度、账单和限流实施 Provider 优先熔断。
|
|
13
13
|
- 对 DNS、连接、服务端和流中断实施端点熔断。
|
|
14
14
|
- 模型不存在或参数不兼容时仅隔离该模型。
|
|
15
|
-
-
|
|
15
|
+
- 内容审查失败会学习模型系列约束(例如 `glm-4.5`、`GLM-4.6` 同属 `glm`),本轮及后续内容审查故障转移会避开整个受限系列;证据 30 天后自动过期。
|
|
16
|
+
- 候选模型先经过健康、故障域、审查系列、输入模态和上下文窗口硬过滤,再按跨 Provider、模型等价性、能力兼容和历史稳定性排序。
|
|
16
17
|
- 首响应 90 秒、流式停滞 120 秒看门狗。
|
|
17
18
|
- 指数退避、`Retry-After` 支持和跨进程 half-open 单探针租约。
|
|
18
19
|
- 原子持久化状态、错误脱敏和有上限的日志。
|
|
@@ -52,7 +53,7 @@ pi install npm:pi-ccswitch-auto-switch
|
|
|
52
53
|
| `/ccswitch refresh` | 刷新 Pi 模型注册表和状态栏。 |
|
|
53
54
|
| `/ccswitch reactivate <provider/model\|all>` | 解除熔断,下一次真实请求验证恢复情况;保留历史。 |
|
|
54
55
|
| `/ccswitch disable <provider/model>` | 手动排除模型。 |
|
|
55
|
-
| `/ccswitch reset <provider/model\|all>` |
|
|
56
|
+
| `/ccswitch reset <provider/model\|all>` | 确认后删除相应健康历史;`all` 同时清除已学习的审查约束。 |
|
|
56
57
|
| `/ccswitch-test` | 仅检查候选发现,不切换模型。 |
|
|
57
58
|
|
|
58
59
|
## 状态栏
|
|
@@ -80,15 +81,16 @@ CCS ✓70/131 · ⏳61 · ⛔0 · 🔄3 · provider/model-id
|
|
|
80
81
|
| --- | --- | --- |
|
|
81
82
|
| `401`、`403`、额度或账单问题 | Provider | 30 分钟 |
|
|
82
83
|
| `429` | Provider | 优先使用 `Retry-After`,否则 5 分钟 |
|
|
83
|
-
| DNS、网络、`5xx`、流中断 | 端点 | 2 分钟 |
|
|
84
|
+
| DNS、网络、`408`、`5xx`、流中断 | 端点 | 2 分钟 |
|
|
84
85
|
| `404`、模型不存在、参数不兼容 | 模型 | 15 分钟 |
|
|
85
|
-
|
|
|
86
|
+
| 内容审查/敏感拦截 | 当前模型 + 模型系列约束 | 模型冷却 2 分钟;系列避让证据 30 天 |
|
|
87
|
+
| 上下文溢出 | 仅本轮 | 无 |
|
|
86
88
|
|
|
87
|
-
|
|
89
|
+
冷却时间会指数增长但有上限。发生内容审查时,插件只在审查故障转移链中避开已标记系列,普通限流、网络或模型配置故障仍可选择这些模型。上下文溢出时只会选择上下文窗口更大的模型;带图片的请求不会切到明确仅支持文本的模型。无法归类的异常按单模型故障处理并正常切换。用户主动取消是唯一不会触发故障转移的异常终止;看门狗取消会记录为超时。
|
|
88
90
|
|
|
89
91
|
## 数据与隐私
|
|
90
92
|
|
|
91
|
-
健康状态保存在 Pi agent 目录的 `ccswitch-auto-switch-state.json
|
|
93
|
+
健康状态保存在 Pi agent 目录的 `ccswitch-auto-switch-state.json`。其中只有计数、时间、冷却信息、模型系列审查约束和脱敏/截断的错误摘要;插件不会访问凭据、Authorization 请求头、CC Switch 数据库或 Pi 的 `auth.json`。
|
|
92
94
|
|
|
93
95
|
## 开发
|
|
94
96
|
|
|
@@ -98,7 +100,7 @@ npm run typecheck
|
|
|
98
100
|
npm test
|
|
99
101
|
```
|
|
100
102
|
|
|
101
|
-
测试使用 Node
|
|
103
|
+
测试使用 Node 内置测试运行器,覆盖失败分类矩阵、系列级审查避让、Provider 优先选择、输入兼容、冷却、状态持久化以及 Windows 路径。
|
|
102
104
|
|
|
103
105
|
## 许可证
|
|
104
106
|
|
package/candidates.ts
CHANGED
|
@@ -1,12 +1,16 @@
|
|
|
1
|
-
import type { ModelRef, ScopedModel } from './types.ts'
|
|
1
|
+
import type { FailureClass, ModelRef, ScopedModel } from './types.ts'
|
|
2
2
|
import { endpointKey, modelKey, type HealthState } from './health.ts'
|
|
3
3
|
|
|
4
4
|
export interface CandidateOptions {
|
|
5
5
|
current?: ModelRef
|
|
6
6
|
tried: Set<string>
|
|
7
|
-
failureKind?:
|
|
7
|
+
failureKind?: FailureClass
|
|
8
8
|
health: HealthState
|
|
9
9
|
avoidEndpoints?: Set<string>
|
|
10
|
+
/** 模型系列已被证实存在内容审查约束时,本轮不再选择该系列。 */
|
|
11
|
+
avoidFamilies?: ReadonlySet<string>
|
|
12
|
+
/** 本轮输入真正需要的模态;显式声明不支持的候选会被排除。 */
|
|
13
|
+
requiredInputs?: readonly string[]
|
|
10
14
|
}
|
|
11
15
|
|
|
12
16
|
export interface CandidateSnapshot {
|
|
@@ -76,22 +80,64 @@ export function chooseCandidate(models: ModelRef[], options: CandidateOptions):
|
|
|
76
80
|
const candidates = models.filter(model => {
|
|
77
81
|
if (options.tried.has(modelKey(model)) || blocked(model, options.health)) return false
|
|
78
82
|
if (options.avoidEndpoints?.has(endpointKey(model))) return false
|
|
83
|
+
if (options.avoidFamilies?.has(modelFamily(model))) return false
|
|
84
|
+
if (!supportsRequiredInputs(model, options.requiredInputs)) return false
|
|
79
85
|
if (options.failureKind === 'context_overflow' && current && (model.contextWindow ?? 0) <= (current.contextWindow ?? 0)) return false
|
|
80
86
|
return true
|
|
81
87
|
})
|
|
82
|
-
return candidates.sort((a, b) => score(a, current) - score(b, current))[0]
|
|
88
|
+
return candidates.sort((a, b) => score(a, current, options.health) - score(b, current, options.health))[0]
|
|
83
89
|
}
|
|
84
90
|
|
|
85
|
-
|
|
91
|
+
/**
|
|
92
|
+
* 返回跨 Provider 稳定的模型系列标识。内容审查通常是模型系列的能力约束,而不是某个 API Key
|
|
93
|
+
* 或单一部署的瞬时故障,因此已知系列使用显式规则;未知模型则保守地按版本号前的 id 归组。
|
|
94
|
+
*/
|
|
95
|
+
export function modelFamily(model: Pick<ModelRef, 'id' | 'name'>): string {
|
|
96
|
+
const identity = `${model.id} ${model.name ?? ''}`.toLowerCase()
|
|
97
|
+
const knownFamilies: Array<[string, RegExp]> = [
|
|
98
|
+
['glm', /(?:^|[^a-z0-9])(?:chat)?glm(?:$|[^a-z])/],
|
|
99
|
+
['qwen', /(?:^|[^a-z0-9])qwen(?:$|[^a-z])/],
|
|
100
|
+
['deepseek', /(?:^|[^a-z0-9])deepseek(?:$|[^a-z])/],
|
|
101
|
+
['kimi', /(?:^|[^a-z0-9])(?:kimi|moonshot)(?:$|[^a-z])/],
|
|
102
|
+
['gpt', /(?:^|[^a-z0-9])gpt(?:$|[^a-z])/],
|
|
103
|
+
['claude', /(?:^|[^a-z0-9])claude(?:$|[^a-z])/],
|
|
104
|
+
['gemini', /(?:^|[^a-z0-9])gemini(?:$|[^a-z])/],
|
|
105
|
+
['llama', /(?:^|[^a-z0-9])llama(?:$|[^a-z])/],
|
|
106
|
+
['mistral', /(?:^|[^a-z0-9])mistral(?:$|[^a-z])/],
|
|
107
|
+
['minimax', /(?:^|[^a-z0-9])minimax(?:$|[^a-z])/],
|
|
108
|
+
['ernie', /(?:^|[^a-z0-9])ernie(?:$|[^a-z])/],
|
|
109
|
+
['doubao', /(?:^|[^a-z0-9])doubao(?:$|[^a-z])/],
|
|
110
|
+
['hunyuan', /(?:^|[^a-z0-9])hunyuan(?:$|[^a-z])/],
|
|
111
|
+
]
|
|
112
|
+
for (const [family, pattern] of knownFamilies) if (pattern.test(identity)) return family
|
|
113
|
+
|
|
114
|
+
const leaf = model.id.toLowerCase().split('/').pop() ?? model.id.toLowerCase()
|
|
115
|
+
return leaf
|
|
116
|
+
.replace(/(?:[-_.]v)?\d+(?:[.-]\d+)*(?:[-_.].*)?$/, '')
|
|
117
|
+
.replace(/[-_.:]+$/, '') || leaf
|
|
118
|
+
}
|
|
119
|
+
|
|
120
|
+
function supportsRequiredInputs(candidate: ModelRef, required: readonly string[] | undefined): boolean {
|
|
121
|
+
if (!required?.length || !candidate.input?.length) return true
|
|
122
|
+
return required.every(input => candidate.input!.includes(input))
|
|
123
|
+
}
|
|
124
|
+
|
|
125
|
+
function score(candidate: ModelRef, current: ModelRef | undefined, health: HealthState): number {
|
|
86
126
|
if (!current) return 0
|
|
127
|
+
// 排序优先级:故障域多样性 > 原模型等价性 > 输入/推理/上下文兼容性 > 历史稳定性。
|
|
128
|
+
// 硬约束(冷却、审查系列、输入模态、上下文溢出)已在上方 filter 中处理。
|
|
87
129
|
const sameProvider = candidate.provider === current.provider ? 1000 : 0
|
|
88
130
|
const differentId = candidate.id === current.id ? 0 : 100
|
|
89
131
|
const inputMismatch = sameInputs(candidate, current) ? 0 : 20
|
|
90
132
|
const reasoningMismatch = candidate.reasoning === current.reasoning ? 0 : 10
|
|
91
133
|
const contextPenalty = (candidate.contextWindow ?? 0) < (current.contextWindow ?? 0) ? 5 : 0
|
|
92
|
-
|
|
134
|
+
const records = [health.models[modelKey(candidate)], health.providers[candidate.provider], health.endpoints[endpointKey(candidate)]]
|
|
135
|
+
const historicalPenalty = Math.min(80, records.reduce((sum, record) => sum + Math.log2((record?.totalFailures ?? 0) + 1) * 4, 0))
|
|
136
|
+
return sameProvider + differentId + inputMismatch + reasoningMismatch + contextPenalty + historicalPenalty
|
|
93
137
|
}
|
|
94
138
|
|
|
95
139
|
function sameInputs(a: ModelRef, b: ModelRef): boolean {
|
|
96
|
-
|
|
140
|
+
const left = a.input ?? []
|
|
141
|
+
const right = b.input ?? []
|
|
142
|
+
return left.length === right.length && left.every(input => right.includes(input))
|
|
97
143
|
}
|
package/classify.ts
CHANGED
|
@@ -7,10 +7,11 @@ export interface Classification {
|
|
|
7
7
|
retryAfterMs?: number
|
|
8
8
|
}
|
|
9
9
|
|
|
10
|
-
const content = /content[ _-]?filter|sensitive|guardrail|policy[_ ]?violation|
|
|
10
|
+
const content = /content[ _-]?(?:filter|policy|blocked|moderation)|sensitive|guardrail|policy[_ -]?violation|responsibleai|safety[_ -]?(?:policy|filter|violation)|security[_ -]?policy|moderation[_ -]?(?:blocked|failed)|blocked by.{0,20}(?:policy|safety|moderation)|data[_ -]?inspection[_ -]?failed|内容(?:审查|审核|安全|违规|被拦截)|安全审核|安全(?:策略|风控).{0,12}(?:阻断|拦截|拒绝)|审核(?:未通过|不通过|失败)|敏感(?:内容|词|信息)?|政治敏感|涉政|不合规|违反.{0,12}(?:安全|政策|规定)/i
|
|
11
11
|
const quota = /quota|billing|insufficient.?balance|out of budget|usage.?limit|credit.?balance|monthly.?limit/i
|
|
12
12
|
const context = /context.?window|context.?length|too many tokens|prompt is too long|input is too long|token limit/i
|
|
13
|
-
const transport = /dns|enotfound|eai_again|socket|connection|network|fetch failed|timed? ?out|timeout|stream ended|terminated|websocket/i
|
|
13
|
+
const transport = /dns|enotfound|eai_again|econn(?:reset|refused|aborted)|epipe|socket|connection|network|fetch failed|tls|certificate|timed? ?out|timeout|stream ended|terminated|websocket|und_err/i
|
|
14
|
+
const overloaded = /overload|server busy|service unavailable|temporarily unavailable|capacity|upstream|bad gateway/i
|
|
14
15
|
|
|
15
16
|
export function classifyFailure(observation: FailureObservation): Classification {
|
|
16
17
|
const text = `${observation.rawStopReason ?? ''} ${observation.message ?? ''}`
|
|
@@ -21,12 +22,15 @@ export function classifyFailure(observation: FailureObservation): Classification
|
|
|
21
22
|
// chooseCandidate 会排除后自动切换到其他模型,而不是同一内容原样重试到本轮上限。
|
|
22
23
|
if (content.test(text)) return { kind: 'content_policy', scope: 'model', roundOnly: false }
|
|
23
24
|
if (context.test(text)) return { kind: 'context_overflow', roundOnly: true }
|
|
25
|
+
if (status === 413) return { kind: 'context_overflow', roundOnly: true }
|
|
24
26
|
if (status === 429) return { kind: 'rate_limit', scope: 'provider', roundOnly: false, retryAfterMs: observation.retryAfterMs }
|
|
27
|
+
if (status === 402) return { kind: 'quota', scope: 'provider', roundOnly: false }
|
|
25
28
|
if (status === 401 || status === 403) return { kind: 'auth', scope: 'provider', roundOnly: false }
|
|
26
29
|
if (quota.test(text)) return { kind: 'quota', scope: 'provider', roundOnly: false }
|
|
30
|
+
if (status === 408) return { kind: 'timeout', scope: 'endpoint', roundOnly: false }
|
|
27
31
|
if (status !== undefined && status >= 500) return { kind: 'endpoint', scope: 'endpoint', roundOnly: false }
|
|
32
|
+
if (transport.test(text) || overloaded.test(text)) return { kind: 'endpoint', scope: 'endpoint', roundOnly: false }
|
|
28
33
|
if (status === 400 || status === 404 || status === 422) return { kind: 'model_config', scope: 'model', roundOnly: false }
|
|
29
|
-
if (transport.test(text)) return { kind: 'endpoint', scope: 'endpoint', roundOnly: false }
|
|
30
34
|
return { kind: 'unknown', scope: 'model', roundOnly: false }
|
|
31
35
|
}
|
|
32
36
|
|
package/health.ts
CHANGED
|
@@ -24,12 +24,25 @@ export interface HealthRecord {
|
|
|
24
24
|
leaseUntil?: number
|
|
25
25
|
}
|
|
26
26
|
|
|
27
|
+
export interface ContentPolicyConstraint {
|
|
28
|
+
observations: number
|
|
29
|
+
lastObservedAt: number
|
|
30
|
+
avoidUntil: number
|
|
31
|
+
lastModel: string
|
|
32
|
+
lastError?: string
|
|
33
|
+
}
|
|
34
|
+
|
|
27
35
|
export interface HealthState {
|
|
28
36
|
schemaVersion: 2
|
|
29
37
|
updatedAt: number
|
|
30
38
|
models: Record<string, HealthRecord>
|
|
31
39
|
providers: Record<string, HealthRecord>
|
|
32
40
|
endpoints: Record<string, HealthRecord>
|
|
41
|
+
/**
|
|
42
|
+
* 已从真实失败中学到的模型系列内容审查约束。它不会影响普通故障选模,只在本轮已经
|
|
43
|
+
* 发生 content_policy 失败时用于避开同类系列。
|
|
44
|
+
*/
|
|
45
|
+
contentPolicyFamilies?: Record<string, ContentPolicyConstraint>
|
|
33
46
|
/** 累计成功切换次数(跨 session 持久化,用于衡量扩展有效程度) */
|
|
34
47
|
switches?: number
|
|
35
48
|
/** 最近成功切换日志(有限条,环形保留) */
|
|
@@ -64,7 +77,7 @@ function safeEndpoint(value: string): string {
|
|
|
64
77
|
}
|
|
65
78
|
|
|
66
79
|
function blank(): HealthState {
|
|
67
|
-
return { schemaVersion: 2, updatedAt: Date.now(), models: {}, providers: {}, endpoints: {}, switches: 0, switchLog: [] }
|
|
80
|
+
return { schemaVersion: 2, updatedAt: Date.now(), models: {}, providers: {}, endpoints: {}, contentPolicyFamilies: {}, switches: 0, switchLog: [] }
|
|
68
81
|
}
|
|
69
82
|
|
|
70
83
|
function redact(text: string | undefined): string | undefined {
|
|
@@ -91,6 +104,7 @@ export class HealthStore {
|
|
|
91
104
|
private state: HealthState = blank()
|
|
92
105
|
private dirty = false
|
|
93
106
|
private replaceOnFlush = false
|
|
107
|
+
private resetModels = new Set<string>()
|
|
94
108
|
|
|
95
109
|
constructor(dir = agentDir()) { this.dir = dir }
|
|
96
110
|
get file(): string { return join(this.dir, STATE_FILE) }
|
|
@@ -143,6 +157,24 @@ export class HealthStore {
|
|
|
143
157
|
this.touch()
|
|
144
158
|
}
|
|
145
159
|
|
|
160
|
+
/**
|
|
161
|
+
* 内容审查约束比限流/网络故障稳定得多,但仍设置 30 天证据有效期,避免模型策略升级后
|
|
162
|
+
* 永久污染路由。再次观察到同系列审查会增加证据计数并刷新有效期。
|
|
163
|
+
*/
|
|
164
|
+
recordContentPolicyConstraint(family: string, model: ModelRef, message?: string): void {
|
|
165
|
+
const constraints = this.state.contentPolicyFamilies ??= {}
|
|
166
|
+
const previous = constraints[family]
|
|
167
|
+
const now = Date.now()
|
|
168
|
+
constraints[family] = {
|
|
169
|
+
observations: (previous?.observations ?? 0) + 1,
|
|
170
|
+
lastObservedAt: now,
|
|
171
|
+
avoidUntil: now + 30 * 24 * 60 * 60_000,
|
|
172
|
+
lastModel: modelKey(model),
|
|
173
|
+
lastError: redact(message),
|
|
174
|
+
}
|
|
175
|
+
this.touch()
|
|
176
|
+
}
|
|
177
|
+
|
|
146
178
|
recordSuccess(model: ModelRef): void {
|
|
147
179
|
this.close('model', modelKey(model))
|
|
148
180
|
this.close('provider', model.provider)
|
|
@@ -192,9 +224,14 @@ export class HealthStore {
|
|
|
192
224
|
}
|
|
193
225
|
|
|
194
226
|
reset(target: string | 'all'): void {
|
|
195
|
-
if (target === 'all')
|
|
196
|
-
|
|
197
|
-
|
|
227
|
+
if (target === 'all') {
|
|
228
|
+
this.state = blank()
|
|
229
|
+
this.replaceOnFlush = true
|
|
230
|
+
this.resetModels.clear()
|
|
231
|
+
} else {
|
|
232
|
+
delete this.state.models[target]
|
|
233
|
+
this.resetModels.add(target)
|
|
234
|
+
}
|
|
198
235
|
this.touch()
|
|
199
236
|
}
|
|
200
237
|
|
|
@@ -221,7 +258,8 @@ export class HealthStore {
|
|
|
221
258
|
async flush(): Promise<void> {
|
|
222
259
|
if (!this.dirty) return
|
|
223
260
|
await this.withLock(async () => {
|
|
224
|
-
this.state = mergeState(await this.readDisk(), this.state)
|
|
261
|
+
if (!this.replaceOnFlush) this.state = mergeState(await this.readDisk(), this.state)
|
|
262
|
+
for (const key of this.resetModels) delete this.state.models[key]
|
|
225
263
|
await this.commit()
|
|
226
264
|
}).catch(() => {})
|
|
227
265
|
}
|
|
@@ -259,6 +297,7 @@ export class HealthStore {
|
|
|
259
297
|
await rename(temp, this.file)
|
|
260
298
|
this.dirty = false
|
|
261
299
|
this.replaceOnFlush = false
|
|
300
|
+
this.resetModels.clear()
|
|
262
301
|
}
|
|
263
302
|
private bucket(scope: HealthScope): Record<string, HealthRecord> {
|
|
264
303
|
return scope === 'model' ? this.state.models : scope === 'provider' ? this.state.providers : this.state.endpoints
|
|
@@ -314,5 +353,20 @@ function mergeState(a: HealthState, b: HealthState): HealthState {
|
|
|
314
353
|
.sort((x, y) => y.at - x.at)
|
|
315
354
|
.filter((entry, index, all) => index === 0 || all[index - 1].at !== entry.at || all[index - 1].from !== entry.from || all[index - 1].to !== entry.to)
|
|
316
355
|
.slice(0, 20)
|
|
317
|
-
|
|
356
|
+
const contentPolicyFamilies: Record<string, ContentPolicyConstraint> = {}
|
|
357
|
+
for (const family of new Set([...Object.keys(a.contentPolicyFamilies ?? {}), ...Object.keys(b.contentPolicyFamilies ?? {})])) {
|
|
358
|
+
const left = a.contentPolicyFamilies?.[family]
|
|
359
|
+
const right = b.contentPolicyFamilies?.[family]
|
|
360
|
+
if (!left) contentPolicyFamilies[family] = right!
|
|
361
|
+
else if (!right) contentPolicyFamilies[family] = left
|
|
362
|
+
else {
|
|
363
|
+
const newest = right.lastObservedAt >= left.lastObservedAt ? right : left
|
|
364
|
+
contentPolicyFamilies[family] = {
|
|
365
|
+
...newest,
|
|
366
|
+
observations: Math.max(left.observations, right.observations),
|
|
367
|
+
avoidUntil: Math.max(left.avoidUntil, right.avoidUntil),
|
|
368
|
+
}
|
|
369
|
+
}
|
|
370
|
+
}
|
|
371
|
+
return { schemaVersion: 2, updatedAt: Math.max(a.updatedAt ?? 0, b.updatedAt ?? 0), models: mergeBucket(a.models ?? {}, b.models ?? {}), providers: mergeBucket(a.providers ?? {}, b.providers ?? {}), endpoints: mergeBucket(a.endpoints ?? {}, b.endpoints ?? {}), contentPolicyFamilies, switches, switchLog: log }
|
|
318
372
|
}
|
package/index.ts
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
import type { ExtensionAPI, ExtensionContext, FailureObservation, ModelRef } from './types.ts'
|
|
2
2
|
import { classifyFailure, parseRetryAfter } from './classify.ts'
|
|
3
|
-
import { candidateSnapshot, effectiveCandidates, chooseCandidate, summarizeCandidateHealth } from './candidates.ts'
|
|
4
|
-
import { HealthStore, endpointKey, modelKey } from './health.ts'
|
|
3
|
+
import { candidateSnapshot, effectiveCandidates, chooseCandidate, modelFamily, summarizeCandidateHealth } from './candidates.ts'
|
|
4
|
+
import { HealthStore, endpointKey, modelKey, type HealthState } from './health.ts'
|
|
5
5
|
|
|
6
6
|
const FIRST_RESPONSE_TIMEOUT = 90_000
|
|
7
7
|
const STREAM_IDLE_TIMEOUT = 120_000
|
|
@@ -32,12 +32,26 @@ interface Round {
|
|
|
32
32
|
observation?: FailureObservation
|
|
33
33
|
model?: ModelRef
|
|
34
34
|
endpointFails?: EndpointFailTracker
|
|
35
|
+
/** 本轮内容审查故障转移中必须避开的模型系列(包含持久化学到的约束)。 */
|
|
36
|
+
avoidFamilies?: Set<string>
|
|
35
37
|
/** 本轮内最后一次成功切换的时间;用于刷新 ROUND_LIMIT 窗口,避免供应商内部重试耗时导致误判“达到上限” */
|
|
36
38
|
lastSwitchAt?: number
|
|
37
39
|
}
|
|
38
40
|
|
|
39
41
|
function key(model: ModelRef | undefined): string | undefined { return model && modelKey(model) }
|
|
40
42
|
function canRetry(ctx: ExtensionContext): boolean { return ctx.mode === 'tui' || ctx.mode === 'rpc' }
|
|
43
|
+
function activePolicyFamilies(state: HealthState, now = Date.now()): Set<string> {
|
|
44
|
+
return new Set(Object.entries(state.contentPolicyFamilies ?? {})
|
|
45
|
+
.filter(([, record]) => record.avoidUntil > now)
|
|
46
|
+
.map(([family]) => family))
|
|
47
|
+
}
|
|
48
|
+
|
|
49
|
+
function resolveModel(ctx: ExtensionContext, provider: string, id: string, previous?: ModelRef): ModelRef {
|
|
50
|
+
// 优先使用注册表/scope 中的完整元数据,避免 turn_end 消息只有 provider/id 时丢失 BaseURL、
|
|
51
|
+
// 上下文窗口和输入模态;previous/ctx.model 作为注册表未命中时的后备。
|
|
52
|
+
const refs = [...ctx.scopedModels.map(item => item.model), ...ctx.modelRegistry.getAvailable(), previous, ctx.model]
|
|
53
|
+
return refs.find(model => model?.provider === provider && model.id === id) ?? { provider, id }
|
|
54
|
+
}
|
|
41
55
|
|
|
42
56
|
export default function (pi: ExtensionAPI) {
|
|
43
57
|
const health = new HealthStore()
|
|
@@ -51,10 +65,9 @@ export default function (pi: ExtensionAPI) {
|
|
|
51
65
|
const armWatchdog = (ctx: ExtensionContext, ms: number, roundId: number) => {
|
|
52
66
|
clearWatchdog()
|
|
53
67
|
timer = setTimeout(() => {
|
|
54
|
-
if (!round || round.id !== roundId ||
|
|
55
|
-
|
|
56
|
-
|
|
57
|
-
}
|
|
68
|
+
if (!round || round.id !== roundId || ctx.isIdle() || round.inTool) return
|
|
69
|
+
round.watchdog = true
|
|
70
|
+
ctx.abort()
|
|
58
71
|
}, ms)
|
|
59
72
|
timer.unref?.()
|
|
60
73
|
}
|
|
@@ -90,7 +103,7 @@ export default function (pi: ExtensionAPI) {
|
|
|
90
103
|
'/ccswitch refresh — 刷新 Pi 模型注册表和状态',
|
|
91
104
|
'/ccswitch reactivate <provider/model|all> — 解除熔断,保留历史',
|
|
92
105
|
'/ccswitch disable <provider/model> — 手动禁用模型',
|
|
93
|
-
'/ccswitch reset <provider/model|all> —
|
|
106
|
+
'/ccswitch reset <provider/model|all> — 清除健康历史;all 也会清除已学习的审查约束(需确认)',
|
|
94
107
|
'/ccswitch-test — 自检候选模型,不实际切换',
|
|
95
108
|
].join('\n'), 'info')
|
|
96
109
|
}
|
|
@@ -104,16 +117,19 @@ export default function (pi: ExtensionAPI) {
|
|
|
104
117
|
const candidates = snapshot.models
|
|
105
118
|
const counts = summarizeCandidateHealth(candidates, state)
|
|
106
119
|
const now = Date.now()
|
|
120
|
+
const policyFamilies = activePolicyFamilies(state, now)
|
|
107
121
|
const rows = candidates.slice(0, 10).map(model => {
|
|
108
122
|
const modelRecord = state.models[modelKey(model)]
|
|
109
123
|
const records = [modelRecord, state.providers[model.provider], state.endpoints[endpointKey(model)]]
|
|
110
124
|
const blockedUntil = Math.max(0, ...records.flatMap(record => [record?.cooldownUntil ?? 0, record?.leaseUntil ?? 0]))
|
|
111
|
-
const
|
|
125
|
+
const policySuffix = policyFamilies.has(modelFamily(model)) ? ' · 内容审查约束' : ''
|
|
126
|
+
const baseSuffix = modelRecord?.disabled ? '手动禁用' : blockedUntil > now ? `自动冷却 ${Math.max(1, Math.ceil((blockedUntil - now) / 60_000))}m` : '健康'
|
|
127
|
+
const suffix = `${baseSuffix}${policySuffix}`
|
|
112
128
|
return `${modelKey(model)} ${suffix}`
|
|
113
129
|
})
|
|
114
130
|
if (!ctx.ui.select) {
|
|
115
131
|
const source = snapshot.source === 'scoped' ? `Pi scope ${snapshot.sourceEntries} 条` : `Pi 注册表 ${snapshot.sourceEntries} 条`
|
|
116
|
-
notify(ctx, `CCSwitch:${source} · 唯一模型 ${counts.total} · 健康 ${counts.healthy} · 自动冷却 ${counts.cooling} · 手动禁用 ${counts.disabled} · 本session切换 ${sessionSwitches} · 累计切换 ${state.switches ?? 0}`, 'info')
|
|
132
|
+
notify(ctx, `CCSwitch:${source} · 唯一模型 ${counts.total} · 健康 ${counts.healthy} · 自动冷却 ${counts.cooling} · 手动禁用 ${counts.disabled} · 审查约束系列 ${policyFamilies.size} · 本session切换 ${sessionSwitches} · 累计切换 ${state.switches ?? 0}`, 'info')
|
|
117
133
|
return
|
|
118
134
|
}
|
|
119
135
|
const scopeLabel = snapshot.source === 'scoped' ? `Pi scope:${snapshot.sourceEntries} 条` : `Pi 可用注册表:${snapshot.sourceEntries} 条`
|
|
@@ -123,7 +139,8 @@ export default function (pi: ExtensionAPI) {
|
|
|
123
139
|
return ` ${time} ${entry.from} 🔄 ${entry.to}`
|
|
124
140
|
})
|
|
125
141
|
const switchPanel = recentSwitches.length ? `\n最近切换:\n${recentSwitches.join('\n')}` : ''
|
|
126
|
-
const
|
|
142
|
+
const policySummary = policyFamilies.size ? [...policyFamilies].join(', ') : '无'
|
|
143
|
+
const action = await ctx.ui.select(`CCSwitch 健康面板\n当前:${key(ctx.model) ?? '无'}\n${scopeLabel} · 唯一模型:${counts.total} · 健康:${counts.healthy} · 自动冷却:${counts.cooling} · 手动禁用:${counts.disabled}\n熔断记录:${counts.breakerRecords} · 审查约束系列:${policySummary}\n${switchSummary}${switchPanel}\n${rows.join('\n') || '没有可用模型'}`, ['刷新', '重新激活当前模型', '禁用当前模型', '重置当前模型历史', '关闭'])
|
|
127
144
|
if (action === '刷新') await refresh(ctx)
|
|
128
145
|
if (action === '重新激活当前模型' && ctx.model) { health.reactivate(ctx.model); await health.flush(); status(ctx); notify(ctx, '已重新激活当前模型') }
|
|
129
146
|
if (action === '禁用当前模型' && ctx.model) { health.disable(modelKey(ctx.model), true); await health.flush(); status(ctx); notify(ctx, '已禁用当前模型', 'warning') }
|
|
@@ -161,20 +178,35 @@ export default function (pi: ExtensionAPI) {
|
|
|
161
178
|
failTracker.isolated.add(ep)
|
|
162
179
|
await health.log(`endpoint ${ep} failed ${epFails} times this round, isolating endpoint`)
|
|
163
180
|
}
|
|
181
|
+
if (classification.kind === 'content_policy') {
|
|
182
|
+
health.recordContentPolicyConstraint(modelFamily(round.model), round.model, round.observation.message)
|
|
183
|
+
round.avoidFamilies ??= new Set()
|
|
184
|
+
for (const family of activePolicyFamilies(health.snapshot)) round.avoidFamilies.add(family)
|
|
185
|
+
}
|
|
164
186
|
if (!classification.roundOnly && classification.scope) health.recordFailure(classification.scope, classification.scope === 'model' ? modelKey(round.model) : classification.scope === 'provider' ? round.model.provider : endpointKey(round.model), classification.kind, round.observation.message, classification.retryAfterMs)
|
|
165
187
|
await health.flush()
|
|
166
188
|
await refresh(ctx)
|
|
167
189
|
const candidates = effectiveCandidates(ctx.scopedModels, ctx.modelRegistry.getAvailable())
|
|
168
|
-
|
|
190
|
+
const candidateOptions = () => ({
|
|
191
|
+
current: round!.model,
|
|
192
|
+
tried: round!.tried,
|
|
193
|
+
failureKind: classification.kind,
|
|
194
|
+
health: health.snapshot,
|
|
195
|
+
avoidEndpoints: failTracker.isolated,
|
|
196
|
+
avoidFamilies: round!.avoidFamilies,
|
|
197
|
+
requiredInputs: round!.images?.length ? ['text', 'image'] : ['text'],
|
|
198
|
+
})
|
|
199
|
+
let next = chooseCandidate(candidates, candidateOptions())
|
|
169
200
|
while (next) {
|
|
170
|
-
if (!await health.claimProvider(next)) { round.tried.add(modelKey(next)); next = chooseCandidate(candidates,
|
|
201
|
+
if (!await health.claimProvider(next)) { round.tried.add(modelKey(next)); next = chooseCandidate(candidates, candidateOptions()); continue }
|
|
171
202
|
round.attempts++
|
|
172
203
|
ctx.ui.setWorkingMessage(`模型异常,正在切换到 ${modelKey(next)}…`)
|
|
204
|
+
const previousModel = round.model
|
|
173
205
|
const set = await pi.setModel(next).catch(() => false)
|
|
174
206
|
if (!set) {
|
|
175
207
|
health.recordFailure('model', modelKey(next), 'model_config', 'Pi refused model selection')
|
|
176
208
|
round.tried.add(modelKey(next)); await health.flush()
|
|
177
|
-
next = chooseCandidate(candidates,
|
|
209
|
+
next = chooseCandidate(candidates, candidateOptions())
|
|
178
210
|
continue
|
|
179
211
|
}
|
|
180
212
|
round.model = next
|
|
@@ -183,7 +215,7 @@ export default function (pi: ExtensionAPI) {
|
|
|
183
215
|
round.lastSwitchAt = Date.now()
|
|
184
216
|
// 本次 session 成功切换计数 + 持久化累计/日志(衡量扩展有效程度)
|
|
185
217
|
sessionSwitches += 1
|
|
186
|
-
const fromKey = key(
|
|
218
|
+
const fromKey = key(previousModel ?? ctx.model)
|
|
187
219
|
health.recordSwitch(fromKey ?? '', modelKey(next), classification.kind)
|
|
188
220
|
await health.flush()
|
|
189
221
|
notify(ctx, `CCSwitch:已切换至 ${modelKey(next)}(${round.attempts}/${MAX_ATTEMPTS})`, 'info')
|
|
@@ -213,7 +245,7 @@ export default function (pi: ExtensionAPI) {
|
|
|
213
245
|
if (event.source === 'extension') return { action: 'continue' }
|
|
214
246
|
clearWatchdog()
|
|
215
247
|
lastStatus = {}
|
|
216
|
-
round = { id: (round?.id ?? 0) + 1, phase: 'monitoring', startedAt: Date.now(), text: event.text, images: event.images, tried: new Set(), attempts: 0, hadTool: false, inTool: false, hadOutput: false, watchdog: false, cleanRetry: false, model: ctx.model, endpointFails: undefined }
|
|
248
|
+
round = { id: (round?.id ?? 0) + 1, phase: 'monitoring', startedAt: Date.now(), text: event.text, images: event.images, tried: new Set(), attempts: 0, hadTool: false, inTool: false, hadOutput: false, watchdog: false, cleanRetry: false, model: ctx.model, endpointFails: undefined, avoidFamilies: undefined }
|
|
217
249
|
status(ctx)
|
|
218
250
|
return { action: 'continue' }
|
|
219
251
|
})
|
|
@@ -240,7 +272,7 @@ export default function (pi: ExtensionAPI) {
|
|
|
240
272
|
const message = event.message
|
|
241
273
|
if (message?.role !== 'assistant' || !round) return
|
|
242
274
|
clearWatchdog()
|
|
243
|
-
round.model =
|
|
275
|
+
round.model = resolveModel(ctx, message.provider, message.model, round.model)
|
|
244
276
|
round.hadOutput ||= Boolean(message.content?.length)
|
|
245
277
|
if (message.stopReason === 'error' || message.stopReason === 'aborted') {
|
|
246
278
|
round.phase = 'settled-error'
|
|
@@ -256,6 +288,7 @@ export default function (pi: ExtensionAPI) {
|
|
|
256
288
|
if (!ctx.isIdle()) return
|
|
257
289
|
if (!canRetry(ctx) && round.observation && round.model) {
|
|
258
290
|
const classification = classifyFailure(round.observation)
|
|
291
|
+
if (classification.kind === 'content_policy') health.recordContentPolicyConstraint(modelFamily(round.model), round.model, round.observation.message)
|
|
259
292
|
if (!classification.roundOnly && classification.scope) {
|
|
260
293
|
const domain = classification.scope === 'model' ? modelKey(round.model) : classification.scope === 'provider' ? round.model.provider : endpointKey(round.model)
|
|
261
294
|
health.recordFailure(classification.scope, domain, classification.kind, round.observation.message, classification.retryAfterMs)
|
|
@@ -293,7 +326,8 @@ export default function (pi: ExtensionAPI) {
|
|
|
293
326
|
const snapshot = candidateSnapshot(ctx.scopedModels, ctx.modelRegistry.getAvailable())
|
|
294
327
|
const counts = summarizeCandidateHealth(snapshot.models, health.snapshot)
|
|
295
328
|
const state = health.snapshot
|
|
329
|
+
const policyFamilies = activePolicyFamilies(state)
|
|
296
330
|
const source = snapshot.source === 'scoped' ? `Pi scope ${snapshot.sourceEntries} 条` : `Pi 注册表 ${snapshot.sourceEntries} 条`
|
|
297
|
-
notify(ctx, `CCSwitch 自检:${source} · 唯一模型 ${counts.total} · 健康 ${counts.healthy} · 自动冷却 ${counts.cooling} · 手动禁用 ${counts.disabled} · 熔断记录 ${counts.breakerRecords} · 本session切换 ${sessionSwitches} · 累计切换 ${state.switches ?? 0}`, counts.total ? 'info' : 'warning')
|
|
331
|
+
notify(ctx, `CCSwitch 自检:${source} · 唯一模型 ${counts.total} · 健康 ${counts.healthy} · 自动冷却 ${counts.cooling} · 手动禁用 ${counts.disabled} · 熔断记录 ${counts.breakerRecords} · 审查约束系列 ${policyFamilies.size} · 本session切换 ${sessionSwitches} · 累计切换 ${state.switches ?? 0}`, counts.total ? 'info' : 'warning')
|
|
298
332
|
}})
|
|
299
333
|
}
|