@goodandready/dsh-moa 0.2.73 → 0.2.74

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -186,6 +186,15 @@ Validates markdown code blocks across all candidate outputs locally using JavaSc
186
186
  ### 20. Early Exit upon Semantic Consensus
187
187
  Compares normalized candidate proposals using token-level Sørensen-Dice similarity. When models arrive at identical or near-identical solutions (meeting or exceeding `early_exit_threshold`, default: `0.95`), the expensive Judge synthesis phase is automatically bypassed. The consensus solution is promoted directly to the workspace, saving up to 100% of judge token costs while accelerating pipeline latency. Configured via `early_exit_consensus_enabled: true` and `early_exit_threshold: 0.95`.
188
188
 
189
+ ### 21. Dynamic Consensus Threshold & Multi-Judge Escalation
190
+ Evaluates pairwise candidate solution divergence (Disagreement Metric, 0.0–1.0) using normalized Sørensen-Dice similarity. When candidate consensus is weak (disagreement meets or exceeds `escalation_disagreement_threshold`, default: `0.40`), the pipeline automatically escalates:
191
+ - `auto_upgrade`: Upgrades from a lightweight or fast aggregator to a designated flagship senior judge (`escalation_judge` or first senior fallback judge).
192
+ - `council_majority`: Convenes a 3-judge panel with majority consensus voting.
193
+ Dispatches `moa/judge-escalation` Cordis events and displays escalation decisions in live stream telemetry. Configured via `escalation_policy: 'single' | 'auto_upgrade' | 'council_majority'`, `escalation_disagreement_threshold: 0.40`, and `escalation_judge`.
194
+
195
+ ### 22. Self-Consistency Temperature Sampling for Single Models
196
+ Allows single-model configurations (such as a local Ollama/vLLM instance or a single API key) to leverage Mixture-of-Agents diversity. When enabled on a preset with 1 model slot, the runner expands it into $N$ parallel candidate streams with distinct temperatures (e.g. 0.2, 0.6, 0.9) and complementary personas (`proposer`, `challenger`, `minimalist`), calculates agreement ratio, and performs full MoA consensus synthesis. Dispatches `moa/self-consistency-expand` Cordis events. Configured via `self_consistency_enabled: true`, `self_consistency_samples: 3`, and `self_consistency_temperatures: [0.2, 0.6, 0.9]`.
197
+
189
198
  ### 6. Live Canvas 1-Click Preview (optional)
190
199
  If `@goodandready/dsh-live-canvas` is installed in the same profile, `dsh-moa` pushes the promoted HTML file to the Live Canvas REST contract (`POST /dsh-live-canvas/api/preview`, served by the same harness webServer) and appends a one-click preview link (`/dsh-live-canvas/sandbox/<id>`) to the answer. Without the plugin the step is skipped silently — no errors in the log, no dead links.
191
200
 
package/README.ru.md CHANGED
@@ -186,6 +186,15 @@ graph TD
186
186
  ### 20. Ранний выход по семантическому консенсусу
187
187
  Сравнивает нормализованные решения кандидатов по токенному сходству Сёренсена-Дайса. Если решения моделей совпадают или практически идентичны (достигнут порог `early_exit_threshold`, по умолчанию 0.95), стадия вызова судьи (Judge) пропускается. Согласованный вариант немедленно продвигается в проект, экономя до 100% токенов судейской коллегии и сокращая задержку ответа. Управляется параметрами `early_exit_consensus_enabled: true` и `early_exit_threshold: 0.95`.
188
188
 
189
+ ### 21. Динамический порог консенсуса и эскалация судейства (Judge Escalation)
190
+ Оценивает степень расхождения между решениями моделей-кандидатов (метрика Disagreement Metric, 0.0–1.0) на основе попарного расстояния Сёренсена-Дайса. Если расхождение превышает порог (`escalation_disagreement_threshold`, по умолчанию 0.40), пайплайн выполняет адаптивную эскалацию:
191
+ - `auto_upgrade`: автоматический переход с базового/быстрого судьи на флагманскую старшую модель (`escalation_judge` или первый доступный резервный судья).
192
+ - `council_majority`: созыв коллегии из 3 судей с голосованием большинством голосов.
193
+ Генерирует событие `moa/judge-escalation` и выводит статус эскалации в телеметрию стрима.
194
+
195
+ ### 22. Самосогласованное температурное сэмплирование для одиночных моделей (Self-Consistency)
196
+ Позволяет пользователям с единственной моделью (локальная Ollama, vLLM или единственный API-ключ) получать преимущества ансамблевой валидации MoA. Автоматически разворачивает одиночный слот на $N$ параллельных запусков с различными температурами (`0.2, 0.6, 0.9`) и дополняющими друг друга инженерными персонами (`proposer`, `challenger`, `minimalist`), вычисляет коэффициент согласия (Self-Consistency Agreement) и синтезирует итоговый результат. Генерирует событие `moa/self-consistency-expand`.
197
+
189
198
  ### 6. Live Canvas 1-клик предпросмотр (опционально)
190
199
  Если в профиле установлен `@goodandready/dsh-live-canvas`, `dsh-moa` отправляет промоученный HTML-файл в REST-контракт Live Canvas (`POST /dsh-live-canvas/api/preview`, тот же webServer харнесса) и добавляет к ответу ссылку на предпросмотр (`/dsh-live-canvas/sandbox/<id>`). Без плагина шаг пропускается тихо — без ошибок в журнале и без битых ссылок.
191
200
 
package/README.zh.md CHANGED
@@ -163,6 +163,15 @@ graph TD
163
163
  ### 20. 语义共识提前退出机制 (Early Exit upon Semantic Consensus)
164
164
  基于 Token 级 Sørensen-Dice 相似度算法比对归一化后的候选方案。当多个候选模型给出高度一致的代码方案(达到或超过 `early_exit_threshold` 阈值,默认 0.95)时,自动跳过高耗费的裁判综合评审阶段,直接将共识方案提升至项目,在大幅缩短响应延迟的同时节省多达 100% 的裁判 Token 支出。通过 `early_exit_consensus_enabled: true` 与 `early_exit_threshold: 0.95` 配置。
165
165
 
166
+ ### 21. 动态共识阈值与裁判自适应升级 (Dynamic Consensus & Judge Escalation)
167
+ 基于归一化 Sørensen-Dice 计算候选方案间的分歧度 (Disagreement Metric 0.0–1.0)。当候选分歧超过设定阈值 (`escalation_disagreement_threshold`,默认 0.40) 时,流水线自动触发升级策略:
168
+ - `auto_upgrade`: 自动将普通/快速裁判升级为指定的高阶旗舰裁判 (`escalation_judge` 或首选候补裁判)。
169
+ - `council_majority`: 组建 3 裁判多数席裁决团。
170
+ 触发 `moa/judge-escalation` 事件并在实时流中呈现决策依据。
171
+
172
+ ### 22. 单模型自洽性温度采样 (Self-Consistency Temperature Sampling)
173
+ 专为仅配置单模型(如本地 Ollama/vLLM 或单一 API Key)的场景设计。开启后自动将单槽位扩展为 $N$ 个并行采样(分别采用 0.2、0.6、0.9 等递增温度与 `proposer`、`challenger`、`minimalist` 互补提示词),评估自洽一致性比率并进行全流程 MoA 综合聚合,触发 `moa/self-consistency-expand` 事件。
174
+
166
175
  ### 6. Live Canvas 一键预览(可选)
167
176
  若同一 profile 中安装了 `@goodandready/dsh-live-canvas`,`dsh-moa` 会将提升后的 HTML 文件推送到 Live Canvas 的 REST 契约(`POST /dsh-live-canvas/api/preview`,由同一 harness webServer 提供服务),并在回答中附上一键预览链接(`/dsh-live-canvas/sandbox/<id>`)。未安装该插件时此步骤静默跳过——日志无报错,也不会出现死链接。
168
177
 
package/lib/client.js CHANGED
@@ -213,7 +213,18 @@ window.__ModuleLoader__.load({
213
213
  "workbench.promote_btn": "Promote Harmonized Files to Workspace",
214
214
  "workbench.promoted_ok": "\u2705 Harmonized deliverable promoted to workspace root!",
215
215
  "workbench.pros": "Pros:",
216
- "workbench.cons": "Cons:"
216
+ "workbench.cons": "Cons:",
217
+ "aggregator.self_consistency_title": "\u{1F3B2} Self-Consistency Temperature Sampling (Single-Model Diversity)",
218
+ "aggregator.self_consistency_desc": "Expands a single reference model into multiple parallel samples with varied temperatures and personas, bringing MoA benefits to single-model setups.",
219
+ "aggregator.self_consistency_samples": "Samples:",
220
+ "aggregator.escalation_title": "\u2696\uFE0F Dynamic Consensus Threshold & Judge Escalation",
221
+ "aggregator.escalation_desc": "Detects divergence across candidates. When consensus is weak, automatically upgrades to a senior flagship judge or convenes a 3-judge panel.",
222
+ "aggregator.escalation_policy": "Escalation Policy:",
223
+ "aggregator.escalation_policy_single": "Standard Single Judge (No Escalation)",
224
+ "aggregator.escalation_policy_upgrade": "Auto-Upgrade to Senior Judge on Divergence",
225
+ "aggregator.escalation_policy_council": "Convene 3-Judge Majority Council on Divergence",
226
+ "aggregator.escalation_threshold": "Divergence Threshold:",
227
+ "aggregator.escalation_judge_label": "Senior Judge:"
217
228
  };
218
229
  const zh = {
219
230
  title: "\u667A\u80FD\u4F53\u6DF7\u5408 (MoA)",
@@ -417,7 +428,18 @@ window.__ModuleLoader__.load({
417
428
  "workbench.promote_btn": "\u63D0\u5347\u7C98\u5408\u7ED3\u679C\u81F3\u9879\u76EE\u6839\u76EE\u5F55",
418
429
  "workbench.promoted_ok": "\u2705 \u7C98\u5408\u65B9\u6848\u5DF2\u6210\u529F\u63D0\u5347\u81F3\u9879\u76EE\u5DE5\u4F5C\u533A\uFF01",
419
430
  "workbench.pros": "\u4F18\u52BF\uFF1A",
420
- "workbench.cons": "\u4E0D\u8DB3\uFF1A"
431
+ "workbench.cons": "\u4E0D\u8DB3\uFF1A",
432
+ "aggregator.self_consistency_title": "\u{1F3B2} \u5355\u6A21\u578B\u81EA\u6D3D\u6027\u6E29\u5EA6\u91C7\u6837 (Self-Consistency)",
433
+ "aggregator.self_consistency_desc": "\u5728\u4EC5\u914D\u7F6E\u5355\u6A21\u578B\u65F6\uFF0C\u751F\u6210\u4E0D\u540C\u6E29\u5EA6\u4E0E\u89D2\u8272\u7684\u591A\u4E2A\u5E76\u884C\u5019\u9009\u65B9\u6848\uFF0C\u8BA9\u672C\u5730 Ollama \u6216\u5355 Key \u4E5F\u4EAB\u6709 MoA \u805A\u5408\u4F18\u52BF\u3002",
434
+ "aggregator.self_consistency_samples": "\u91C7\u6837\u6570:",
435
+ "aggregator.escalation_title": "\u2696\uFE0F \u52A8\u6001\u5171\u8BC6\u9608\u503C\u4E0E\u88C1\u5224\u5347\u7EA7 (Judge Escalation)",
436
+ "aggregator.escalation_desc": "\u68C0\u6D4B\u5019\u9009\u65B9\u6848\u95F4\u7684\u5206\u6B67\u5EA6\u3002\u5F53\u5171\u8BC6\u8F83\u5F31\u65F6\uFF0C\u81EA\u52A8\u5347\u7EA7\u81F3\u9AD8\u9636\u65D7\u8230\u88C1\u5224\u6216\u7EC4\u5EFA 3 \u88C1\u5224\u591A\u6570\u5E2D\u3002",
437
+ "aggregator.escalation_policy": "\u5347\u7EA7\u7B56\u7565:",
438
+ "aggregator.escalation_policy_single": "\u6807\u51C6\u5355\u88C1\u5224 (\u4E0D\u5347\u7EA7)",
439
+ "aggregator.escalation_policy_upgrade": "\u5206\u6B67\u65F6\u81EA\u52A8\u5347\u7EA7\u81F3\u9AD8\u9636\u88C1\u5224",
440
+ "aggregator.escalation_policy_council": "\u5206\u6B67\u65F6\u7EC4\u5EFA 3 \u88C1\u5224\u591A\u6570\u51B3\u7B56\u5E2D",
441
+ "aggregator.escalation_threshold": "\u5206\u6B67\u9608\u503C:",
442
+ "aggregator.escalation_judge_label": "\u9AD8\u9636\u88C1\u5224\u6A21\u578B:"
421
443
  };
422
444
  function makeT(dict, fallback) {
423
445
  return function t(key, vars) {
@@ -2427,6 +2449,116 @@ window.__ModuleLoader__.load({
2427
2449
  t("aggregator.semantic_cache_desc") || "Persists intermediate candidate outputs locally by deterministic SHA-256 fingerprint, eliminating duplicate API costs on retry or repeat requests."
2428
2450
  )
2429
2451
  ),
2452
+ /* Self-Consistency Temperature Sampling (#301) */
2453
+ React.createElement(
2454
+ "div",
2455
+ { style: { display: "flex", flexDirection: "column", gap: 6, padding: "8px 12px", background: "var(--dsw-alias-bg-layer-2)", borderRadius: 8, border: "1px solid var(--dsw-alias-border-l2)", marginTop: 8 } },
2456
+ React.createElement(
2457
+ "label",
2458
+ { style: { display: "flex", alignItems: "center", gap: 8, cursor: "pointer", fontSize: 13, fontWeight: 500 } },
2459
+ React.createElement("input", {
2460
+ type: "checkbox",
2461
+ checked: Boolean(currentPreset.self_consistency_enabled),
2462
+ onChange: (e) => updateCurrentPreset((p) => ({ ...p, self_consistency_enabled: e.target.checked })),
2463
+ style: { accentColor: "var(--dsw-alias-state-brand-primary, var(--dsw-alias-label-primary))", cursor: "pointer" }
2464
+ }),
2465
+ t("aggregator.self_consistency_title") || "\u{1F3B2} Self-Consistency Temperature Sampling (Single-Model Diversity)"
2466
+ ),
2467
+ React.createElement(
2468
+ "div",
2469
+ { style: { fontSize: 12, color: "var(--dsw-alias-label-secondary)", marginLeft: 22 } },
2470
+ t("aggregator.self_consistency_desc") || "Expands a single reference model into multiple parallel samples with varied temperatures and personas, bringing MoA benefits to single-model setups."
2471
+ ),
2472
+ currentPreset.self_consistency_enabled && React.createElement(
2473
+ "div",
2474
+ { style: { marginLeft: 22, marginTop: 4, display: "flex", alignItems: "center", gap: 12 } },
2475
+ React.createElement("span", { style: { fontSize: 12, color: "var(--dsw-alias-label-secondary)" } }, (t("aggregator.self_consistency_samples") || "Samples:") + " " + (currentPreset.self_consistency_samples || 3)),
2476
+ React.createElement("input", {
2477
+ type: "range",
2478
+ min: 2,
2479
+ max: 5,
2480
+ value: currentPreset.self_consistency_samples || 3,
2481
+ onChange: (e) => updateCurrentPreset((p) => ({ ...p, self_consistency_samples: Number(e.target.value) })),
2482
+ style: { width: 100, accentColor: "var(--dsw-alias-state-brand-primary, var(--dsw-alias-label-primary))" }
2483
+ })
2484
+ )
2485
+ ),
2486
+ /* Dynamic Consensus Threshold & Multi-Judge Escalation (#300) */
2487
+ React.createElement(
2488
+ "div",
2489
+ { style: { display: "flex", flexDirection: "column", gap: 6, padding: "8px 12px", background: "var(--dsw-alias-bg-layer-2)", borderRadius: 8, border: "1px solid var(--dsw-alias-border-l2)", marginTop: 8 } },
2490
+ React.createElement(
2491
+ "div",
2492
+ { style: { fontSize: 13, fontWeight: 600, color: "var(--dsw-alias-label-primary)" } },
2493
+ t("aggregator.escalation_title") || "\u2696\uFE0F Dynamic Consensus Threshold & Judge Escalation"
2494
+ ),
2495
+ React.createElement(
2496
+ "div",
2497
+ { style: { fontSize: 12, color: "var(--dsw-alias-label-secondary)" } },
2498
+ t("aggregator.escalation_desc") || "Detects divergence across candidates. When consensus is weak, automatically upgrades to a senior flagship judge or convenes a 3-judge panel."
2499
+ ),
2500
+ React.createElement(
2501
+ "div",
2502
+ { style: { display: "flex", alignItems: "center", gap: 8, marginTop: 4 } },
2503
+ React.createElement("span", { style: { fontSize: 12, color: "var(--dsw-alias-label-secondary)" } }, t("aggregator.escalation_policy") || "Escalation Policy:"),
2504
+ React.createElement(
2505
+ "select",
2506
+ {
2507
+ value: currentPreset.escalation_policy || "single",
2508
+ onChange: (e) => updateCurrentPreset((p) => ({ ...p, escalation_policy: e.target.value })),
2509
+ style: {
2510
+ padding: "4px 8px",
2511
+ fontSize: 12,
2512
+ borderRadius: 6,
2513
+ border: "1px solid var(--dsw-alias-border-l2)",
2514
+ background: "var(--dsw-alias-bg-layer-1)",
2515
+ color: "var(--dsw-alias-label-primary)"
2516
+ }
2517
+ },
2518
+ React.createElement("option", { value: "single" }, t("aggregator.escalation_policy_single") || "Standard Single Judge (No Escalation)"),
2519
+ React.createElement("option", { value: "auto_upgrade" }, t("aggregator.escalation_policy_upgrade") || "Auto-Upgrade to Senior Judge on Divergence"),
2520
+ React.createElement("option", { value: "council_majority" }, t("aggregator.escalation_policy_council") || "Convene 3-Judge Majority Council on Divergence")
2521
+ )
2522
+ ),
2523
+ currentPreset.escalation_policy && currentPreset.escalation_policy !== "single" && React.createElement(
2524
+ "div",
2525
+ { style: { display: "flex", flexDirection: "column", gap: 6, marginTop: 4 } },
2526
+ React.createElement(
2527
+ "div",
2528
+ { style: { display: "flex", alignItems: "center", gap: 12 } },
2529
+ React.createElement("span", { style: { fontSize: 12, color: "var(--dsw-alias-label-secondary)" } }, (t("aggregator.escalation_threshold") || "Divergence Threshold:") + " " + Math.round((currentPreset.escalation_disagreement_threshold ?? 0.4) * 100) + "%"),
2530
+ React.createElement("input", {
2531
+ type: "range",
2532
+ min: 10,
2533
+ max: 90,
2534
+ value: Math.round((currentPreset.escalation_disagreement_threshold ?? 0.4) * 100),
2535
+ onChange: (e) => updateCurrentPreset((p) => ({ ...p, escalation_disagreement_threshold: Number(e.target.value) / 100 })),
2536
+ style: { width: 120, accentColor: "var(--dsw-alias-state-brand-primary, var(--dsw-alias-label-primary))" }
2537
+ })
2538
+ ),
2539
+ currentPreset.escalation_policy === "auto_upgrade" && React.createElement(
2540
+ "div",
2541
+ { style: { display: "flex", alignItems: "center", gap: 8 } },
2542
+ React.createElement("span", { style: { fontSize: 12, color: "var(--dsw-alias-label-secondary)" } }, t("aggregator.escalation_judge_label") || "Senior Judge:"),
2543
+ React.createElement(
2544
+ "div",
2545
+ { style: { flex: 1, maxWidth: 300 } },
2546
+ React.createElement(SearchableModelPicker, {
2547
+ provider: currentPreset.escalation_judge?.provider || "",
2548
+ model: currentPreset.escalation_judge?.model || "",
2549
+ onChange: (prov, mod) => {
2550
+ updateCurrentPreset((p) => ({
2551
+ ...p,
2552
+ escalation_judge: { provider: prov, model: mod }
2553
+ }));
2554
+ },
2555
+ availableModels,
2556
+ t
2557
+ })
2558
+ )
2559
+ )
2560
+ )
2561
+ ),
2430
2562
  /* Fallback Judges Chain */
2431
2563
  React.createElement(
2432
2564
  "div",
package/lib/events.js CHANGED
@@ -7,6 +7,8 @@ export const MOA_EVENTS = {
7
7
  CANDIDATE_COMPLETE: 'moa/candidate-complete',
8
8
  SYNTAX_VALIDATION: 'moa/syntax-validation',
9
9
  EARLY_CONSENSUS: 'moa/early-consensus',
10
+ JUDGE_ESCALATION: 'moa/judge-escalation',
11
+ SELF_CONSISTENCY_EXPAND: 'moa/self-consistency-expand',
10
12
  CRITIQUE_COMPLETE: 'moa/critique-complete',
11
13
  JUDGE_VERDICT: 'moa/judge-verdict',
12
14
  CONTEXT_PRUNING: 'moa/context-pruning',
package/lib/moa-schema.js CHANGED
@@ -83,6 +83,12 @@ export const PresetSchema = z.object({
83
83
  context_pruning_diff_lines: z.number().default(5),
84
84
  semantic_cache_enabled: z.boolean().default(false),
85
85
  semantic_cache_ttl_hours: z.number().default(168),
86
+ self_consistency_enabled: z.boolean().default(false),
87
+ self_consistency_samples: z.number().default(3),
88
+ self_consistency_temperatures: z.array(z.number()).default([0.2, 0.6, 0.9]),
89
+ escalation_policy: z.string().default('single'),
90
+ escalation_disagreement_threshold: z.number().default(0.40),
91
+ escalation_judge: ModelSlotSchema.default({ provider: '', model: '' }),
86
92
  })
87
93
 
88
94
  export const Config = z.object({
@@ -112,6 +118,12 @@ export const Config = z.object({
112
118
  context_pruning_diff_lines: z.number().default(5).volatile(),
113
119
  semantic_cache_enabled: z.boolean().default(false).volatile(),
114
120
  semantic_cache_ttl_hours: z.number().default(168).volatile(),
121
+ self_consistency_enabled: z.boolean().default(false).volatile(),
122
+ self_consistency_samples: z.number().default(3).volatile(),
123
+ self_consistency_temperatures: z.array(z.number()).default([0.2, 0.6, 0.9]).volatile(),
124
+ escalation_policy: z.string().default('single').volatile(),
125
+ escalation_disagreement_threshold: z.number().default(0.40).volatile(),
126
+ escalation_judge: ModelSlotSchema.default({ provider: '', model: '' }).volatile(),
115
127
  smart_routing_model: ModelSlotSchema.default({ provider: '', model: '' }).volatile(),
116
128
  prices: z.dict(PriceRow).default({}).volatile(),
117
129
  presets: z.array(PresetSchema).default([
@@ -1,3 +1,5 @@
1
+ import { expandSelfConsistencySlots } from './self-consistency.js'
2
+ import { emitEvent, MOA_EVENTS } from '../events.js'
1
3
  import { balanceCandidateQuotas } from '../moa-quota-balancer.js'
2
4
  import { collectProjectContext, formatProjectContext, isRefinementTask } from '../workspace/scanner.js'
3
5
  import { applyBudgetGuardrails, createBudgetAbortPayload } from '../moa-budget.js'
@@ -27,7 +29,40 @@ export async function stageProjectContext({
27
29
  const refTemp = typeof preset?.reference_temperature === 'number' ? preset.reference_temperature : 0.6
28
30
  const aggTemp = typeof preset?.aggregator_temperature === 'number' ? preset.aggregator_temperature : 0.4
29
31
  const judgeCriteria = preset?.judge_criteria || ''
30
- let isFastMode = referenceModels.length === 1 && !preset?.curator_synthesis
32
+ const selfConsistencyEnabled = Boolean(preset?.self_consistency_enabled)
33
+ const selfConsistencySamples = typeof preset?.self_consistency_samples === 'number' ? preset.self_consistency_samples : 3
34
+ const selfConsistencyTemperatures = Array.isArray(preset?.self_consistency_temperatures)
35
+ ? preset.self_consistency_temperatures
36
+ : [0.2, 0.6, 0.9]
37
+ const escalationPolicy = preset?.escalation_policy || 'single'
38
+ const escalationDisagreementThreshold = typeof preset?.escalation_disagreement_threshold === 'number'
39
+ ? preset.escalation_disagreement_threshold
40
+ : 0.40
41
+ const escalationJudge = (preset?.escalation_judge?.model) ? preset.escalation_judge : null
42
+
43
+ let effectiveReferenceModels = referenceModels
44
+ let selfConsistencyExpanded = false
45
+ if (selfConsistencyEnabled) {
46
+ const scRes = expandSelfConsistencySlots(referenceModels, {
47
+ enabled: true,
48
+ samples: selfConsistencySamples,
49
+ temperatures: selfConsistencyTemperatures,
50
+ })
51
+ if (scRes.isExpanded) {
52
+ effectiveReferenceModels = scRes.references
53
+ selfConsistencyExpanded = true
54
+ emitEvent(ctx, MOA_EVENTS.SELF_CONSISTENCY_EXPAND, {
55
+ originalModel: referenceModels[0]?.model,
56
+ sampleCount: scRes.sampleCount,
57
+ temperatures: selfConsistencyTemperatures.slice(0, scRes.sampleCount),
58
+ })
59
+ if (typeof onProgress === 'function') {
60
+ onProgress(`🎲 *Self-Consistency: Expanded single model into ${scRes.sampleCount} diverse temperature samples (${selfConsistencyTemperatures.slice(0, scRes.sampleCount).join(', ')})*\n`)
61
+ }
62
+ }
63
+ }
64
+
65
+ let isFastMode = effectiveReferenceModels.length === 1 && !preset?.curator_synthesis
31
66
  const isCuratorSynthesis = Boolean(preset?.curator_synthesis)
32
67
  const isStreamAggregator = preset?.stream_aggregator !== false
33
68
  const isQuorumEnabled = Boolean(preset?.quorum_enabled)
@@ -65,7 +100,7 @@ export async function stageProjectContext({
65
100
  }
66
101
 
67
102
  const budgetCheck = applyBudgetGuardrails({
68
- references: referenceModels,
103
+ references: effectiveReferenceModels,
69
104
  enabled: preset?.budget_guard_enabled,
70
105
  maxBudgetUsd: preset?.max_budget_usd,
71
106
  action: preset?.budget_action || 'trim',
@@ -133,6 +168,8 @@ export async function stageProjectContext({
133
168
  localPreflightCheck, earlyExitConsensusEnabled, earlyExitThreshold,
134
169
  contextPruningEnabled, contextPruningDiffLines,
135
170
  semanticCacheEnabled, semanticCacheTtlHours,
171
+ selfConsistencyEnabled, selfConsistencyExpanded, selfConsistencySamples, selfConsistencyTemperatures,
172
+ escalationPolicy, escalationDisagreementThreshold, escalationJudge,
136
173
  }
137
174
  }
138
175
 
@@ -0,0 +1,160 @@
1
+ /**
2
+ * Dynamic Consensus Threshold & Multi-Judge Escalation (#300)
3
+ *
4
+ * Detects divergence among candidate outputs (Disagreement Metric).
5
+ * When candidate consensus is weak or contradictory, automatically escalates:
6
+ * - auto_upgrade: upgrades from lightweight/fast judge to flagship/senior judge
7
+ * - council_majority: convenes a 3-judge panel with majority voting
8
+ */
9
+
10
+ import { calculateSimilarity } from './early-consensus.js'
11
+
12
+ /**
13
+ * Calculates the Disagreement Metric (0.0 to 1.0) across candidate outputs.
14
+ * 0.0 means perfect consensus (identical solutions).
15
+ * 1.0 means total divergence (completely distinct solutions).
16
+ *
17
+ * @param {Array<object>} candidateOutputs
18
+ * @returns {number} Disagreement metric between 0.0 and 1.0
19
+ */
20
+ export function calculateDisagreementMetric(candidateOutputs = []) {
21
+ if (!Array.isArray(candidateOutputs) || candidateOutputs.length < 2) {
22
+ return 0.0
23
+ }
24
+
25
+ const validTexts = candidateOutputs
26
+ .filter((c) => c && typeof c.text === 'string' && c.text.trim())
27
+ .map((c) => c.text)
28
+
29
+ if (validTexts.length < 2) return 0.0
30
+
31
+ let totalSim = 0
32
+ let pairs = 0
33
+
34
+ for (let i = 0; i < validTexts.length; i++) {
35
+ for (let j = i + 1; j < validTexts.length; j++) {
36
+ totalSim += calculateSimilarity(validTexts[i], validTexts[j])
37
+ pairs++
38
+ }
39
+ }
40
+
41
+ if (pairs === 0) return 0.0
42
+ const avgSim = totalSim / pairs
43
+ const disagreement = Math.max(0.0, Math.min(1.0, 1.0 - avgSim))
44
+ return Number(disagreement.toFixed(4))
45
+ }
46
+
47
+ /**
48
+ * Evaluates whether candidate divergence warrants judge escalation.
49
+ *
50
+ * @param {object} params
51
+ * @param {Array<object>} params.candidateOutputs - Array of successful candidate results
52
+ * @param {object} params.primaryJudge - Primary configured judge slot
53
+ * @param {Array<object>} [params.fallbackJudges=[]] - Chain of backup judges
54
+ * @param {object} [params.escalationJudge] - Explicit flagship upgrade judge
55
+ * @param {string} [params.policy='single'] - 'single' | 'auto_upgrade' | 'council_majority'
56
+ * @param {number} [params.threshold=0.40] - Disagreement threshold to trigger escalation
57
+ * @returns {object} Escalation decision and effective judge(s)
58
+ */
59
+ export function evaluateJudgeEscalation({
60
+ candidateOutputs = [],
61
+ primaryJudge = { provider: '', model: '' },
62
+ fallbackJudges = [],
63
+ escalationJudge = null,
64
+ policy = 'single',
65
+ threshold = 0.40,
66
+ } = {}) {
67
+ const normPolicy = String(policy || 'single').toLowerCase()
68
+
69
+ if (normPolicy === 'single' || !Array.isArray(candidateOutputs) || candidateOutputs.length < 2) {
70
+ return {
71
+ escalated: false,
72
+ escalatedType: 'none',
73
+ judge: primaryJudge,
74
+ councilJudges: [primaryJudge],
75
+ disagreement: 0.0,
76
+ threshold,
77
+ reason: 'Standard single judge evaluation',
78
+ }
79
+ }
80
+
81
+ const disagreement = calculateDisagreementMetric(candidateOutputs)
82
+ const isTriggered = disagreement >= threshold
83
+
84
+ if (!isTriggered) {
85
+ return {
86
+ escalated: false,
87
+ escalatedType: 'none',
88
+ judge: primaryJudge,
89
+ councilJudges: [primaryJudge],
90
+ disagreement,
91
+ threshold,
92
+ reason: `Consensus adequate: disagreement ${(disagreement * 100).toFixed(1)}% < threshold ${(threshold * 100).toFixed(1)}%`,
93
+ }
94
+ }
95
+
96
+ if (normPolicy === 'auto_upgrade') {
97
+ // Pick designated escalationJudge, or the first available fallbackJudge
98
+ let targetJudge = null
99
+ if (escalationJudge && escalationJudge.model) {
100
+ targetJudge = escalationJudge
101
+ } else if (Array.isArray(fallbackJudges) && fallbackJudges.length > 0 && fallbackJudges[0]?.model) {
102
+ targetJudge = fallbackJudges[0]
103
+ } else {
104
+ targetJudge = primaryJudge
105
+ }
106
+
107
+ const upgraded = targetJudge.model !== primaryJudge.model || targetJudge.provider !== primaryJudge.provider
108
+ return {
109
+ escalated: upgraded,
110
+ escalatedType: upgraded ? 'auto_upgrade' : 'none',
111
+ judge: targetJudge,
112
+ originalJudge: primaryJudge,
113
+ councilJudges: [targetJudge],
114
+ disagreement,
115
+ threshold,
116
+ reason: upgraded
117
+ ? `High candidate divergence (${(disagreement * 100).toFixed(1)}% >= ${(threshold * 100).toFixed(1)}%): upgraded judge to ${targetJudge.provider ? targetJudge.provider + ':' : ''}${targetJudge.model}`
118
+ : `High divergence detected (${(disagreement * 100).toFixed(1)}%), but no senior fallback judge configured`,
119
+ }
120
+ }
121
+
122
+ if (normPolicy === 'council_majority') {
123
+ const council = [primaryJudge]
124
+ if (escalationJudge && escalationJudge.model && escalationJudge.model !== primaryJudge.model) {
125
+ council.push(escalationJudge)
126
+ }
127
+ if (Array.isArray(fallbackJudges)) {
128
+ for (const fb of fallbackJudges) {
129
+ if (council.length >= 3) break
130
+ if (fb && fb.model && !council.some((c) => c.model === fb.model && c.provider === fb.provider)) {
131
+ council.push(fb)
132
+ }
133
+ }
134
+ }
135
+
136
+ const hasMulti = council.length > 1
137
+ return {
138
+ escalated: hasMulti,
139
+ escalatedType: hasMulti ? 'council_majority' : 'none',
140
+ judge: council[0],
141
+ originalJudge: primaryJudge,
142
+ councilJudges: council,
143
+ disagreement,
144
+ threshold,
145
+ reason: hasMulti
146
+ ? `High candidate divergence (${(disagreement * 100).toFixed(1)}% >= ${(threshold * 100).toFixed(1)}%): assembled ${council.length}-judge council`
147
+ : `High divergence detected, but fewer than 2 judges available for council`,
148
+ }
149
+ }
150
+
151
+ return {
152
+ escalated: false,
153
+ escalatedType: 'none',
154
+ judge: primaryJudge,
155
+ councilJudges: [primaryJudge],
156
+ disagreement,
157
+ threshold,
158
+ reason: 'Standard single judge',
159
+ }
160
+ }
@@ -1,3 +1,5 @@
1
+ import { evaluateJudgeEscalation } from './escalation.js'
2
+ import { calculateSelfConsistencyAgreement } from './self-consistency.js'
1
3
  import { pruneCandidateEnsemble, formatPruningDirective } from './context-pruner.js'
2
4
  import { buildAstCompositeMergeDirective } from '../moa-ast-merge.js'
3
5
  import { runVisualMoaConsensus } from '../moa-visual-consensus.js'
@@ -106,6 +108,13 @@ export async function runMoAPipeline({
106
108
  })
107
109
  const syntaxDirective = syntaxResult.filterApplied ? formatSyntaxDirective(referenceOutputs, { blind: ctxStage.isBlindEvaluation }) : null
108
110
 
111
+ if (ctxStage.selfConsistencyExpanded && successfulRefs.length > 1) {
112
+ const scAgreement = calculateSelfConsistencyAgreement(successfulRefs)
113
+ if (typeof onProgress === 'function') {
114
+ onProgress(`🎯 *Self-Consistency Agreement: ${(scAgreement.agreementRatio * 100).toFixed(0)}% (${scAgreement.isStrongConsensus ? 'Strong consensus' : 'Divergent sampling'}) across ${scAgreement.sampleCount} samples*\n`)
115
+ }
116
+ }
117
+
109
118
  // Stage 3: Clarifying Questions Branch
110
119
  if (ctxStage.needsQuestions) {
111
120
  return await stageClarifyingQuestions({
@@ -190,11 +199,56 @@ export async function runMoAPipeline({
190
199
  return { error: new Error('Turn aborted') }
191
200
  }
192
201
 
202
+ // Pre-Stage 6: Dynamic Consensus Threshold & Multi-Judge Escalation (#300)
203
+ const escalationResult = evaluateJudgeEscalation({
204
+ candidateOutputs: syntaxResult.validCandidates.length > 0 ? syntaxResult.validCandidates : successfulRefs,
205
+ primaryJudge: ctxStage.primaryJudge,
206
+ fallbackJudges: ctxStage.fallbackJudges,
207
+ escalationJudge: ctxStage.escalationJudge,
208
+ policy: ctxStage.escalationPolicy,
209
+ threshold: ctxStage.escalationDisagreementThreshold,
210
+ })
211
+
212
+ let effectivePrimaryJudge = ctxStage.primaryJudge
213
+ let effectiveFallbackJudges = ctxStage.fallbackJudges
214
+ let effectiveJudgesChain = ctxStage.judgesChain
215
+ let effectivePreset = preset
216
+
217
+ if (escalationResult.escalated) {
218
+ emitEvent(ctx, MOA_EVENTS.JUDGE_ESCALATION, {
219
+ runId,
220
+ policy: ctxStage.escalationPolicy,
221
+ disagreement: escalationResult.disagreement,
222
+ threshold: escalationResult.threshold,
223
+ escalatedType: escalationResult.escalatedType,
224
+ originalJudge: escalationResult.originalJudge,
225
+ escalatedJudge: escalationResult.judge,
226
+ councilJudges: escalationResult.councilJudges,
227
+ })
228
+ if (typeof onProgress === 'function') {
229
+ onProgress(`⚖️ *Judge Escalation (${escalationResult.escalatedType}): ${escalationResult.reason}*\n`)
230
+ }
231
+
232
+ if (escalationResult.escalatedType === 'auto_upgrade') {
233
+ effectivePrimaryJudge = escalationResult.judge
234
+ effectiveFallbackJudges = ctxStage.fallbackJudges.filter(
235
+ (j) => !(j.provider === escalationResult.judge.provider && j.model === escalationResult.judge.model)
236
+ )
237
+ effectiveJudgesChain = [effectivePrimaryJudge, ...effectiveFallbackJudges]
238
+ } else if (escalationResult.escalatedType === 'council_majority') {
239
+ effectivePreset = {
240
+ ...preset,
241
+ multi_judge_enabled: true,
242
+ judge_models: escalationResult.councilJudges,
243
+ }
244
+ }
245
+ }
246
+
193
247
  // Stage 6: Multi-Judge Panel
194
248
  const mjRes = await stageMultiJudge({
195
- preset,
249
+ preset: effectivePreset,
196
250
  successfulRefs: syntaxResult.validCandidates.length > 0 ? syntaxResult.validCandidates : successfulRefs,
197
- referenceOutputs, primaryJudge: ctxStage.primaryJudge,
251
+ referenceOutputs, primaryJudge: effectivePrimaryJudge,
198
252
  userPrompt, judgeCriteria: ctxStage.judgeCriteria, prices, maxTokens: ctxStage.maxTokens,
199
253
  callLlm, signal, onProgress, cwd, runId, isRefinement: ctxStage.isRefinement,
200
254
  isFastMode: ctxStage.isFastMode, collectedCtx: ctxStage.collectedCtx, startTime,
@@ -263,9 +317,9 @@ export async function runMoAPipeline({
263
317
  judgeCriteria: ctxStage.judgeCriteria,
264
318
  isCuratorSynthesis: ctxStage.isCuratorSynthesis, isBlindEvaluation: ctxStage.isBlindEvaluation,
265
319
  priorTurnBaseline: ctxStage.priorTurnBaseline, compositeMergeDirective, fileScoringDirective,
266
- multiJudgeResult, preset, prices, maxTokens: ctxStage.maxTokens, isFastMode: ctxStage.isFastMode,
267
- primaryJudge: ctxStage.primaryJudge, fallbackJudges: ctxStage.fallbackJudges,
268
- judgesChain: ctxStage.judgesChain, aggTemp: ctxStage.aggTemp,
320
+ multiJudgeResult, preset: effectivePreset, prices, maxTokens: ctxStage.maxTokens, isFastMode: ctxStage.isFastMode,
321
+ primaryJudge: effectivePrimaryJudge, fallbackJudges: effectiveFallbackJudges,
322
+ judgesChain: effectiveJudgesChain, aggTemp: ctxStage.aggTemp,
269
323
  aggTimeoutSec: ctxStage.aggTimeoutSec, isStreamAggregator: ctxStage.isStreamAggregator,
270
324
  onStreamDelta, signal, callLlm, onProgress, cwd, runId, isRefinement: ctxStage.isRefinement,
271
325
  collectedCtx: ctxStage.collectedCtx, startTime,
@@ -0,0 +1,135 @@
1
+ /**
2
+ * Self-Consistency Temperature Sampling for Single Models (#301)
3
+ *
4
+ * Allows users with a single model slot (e.g., local Ollama/vLLM or a single API key)
5
+ * to benefit from Mixture of Agents ensemble diversity by generating parallel samples
6
+ * at varying temperatures and complementary system personas, followed by consistency voting.
7
+ */
8
+
9
+ import { ROLE_PERSONA_PROMPTS } from '../moa-prompts.js'
10
+
11
+ export const DEFAULT_SELF_CONSISTENCY_TEMPERATURES = Object.freeze([0.2, 0.6, 0.9])
12
+ export const DEFAULT_SELF_CONSISTENCY_PERSONAS = Object.freeze(['proposer', 'challenger', 'minimalist'])
13
+
14
+ /**
15
+ * Expands a single reference model into multiple parallel proposer slots
16
+ * with distinct temperatures and complementary reasoning personas.
17
+ *
18
+ * @param {Array<object>} referenceModels - Original reference model slots
19
+ * @param {object} [options]
20
+ * @param {boolean} [options.enabled=false] - Whether self-consistency expansion is enabled
21
+ * @param {number} [options.samples=3] - Number of candidate samples to generate (2-10)
22
+ * @param {Array<number>} [options.temperatures] - Array of temperatures for samples
23
+ * @param {Array<string>} [options.personas] - Array of persona role names
24
+ * @returns {{ references: Array<object>, isExpanded: boolean, sampleCount: number }}
25
+ */
26
+ export function expandSelfConsistencySlots(referenceModels = [], options = {}) {
27
+ const {
28
+ enabled = false,
29
+ samples = 3,
30
+ temperatures = DEFAULT_SELF_CONSISTENCY_TEMPERATURES,
31
+ personas = DEFAULT_SELF_CONSISTENCY_PERSONAS,
32
+ } = options
33
+
34
+ // Expansion applies only when explicitly enabled or requested
35
+ if (!enabled) {
36
+ return {
37
+ references: Array.isArray(referenceModels) ? referenceModels : [],
38
+ isExpanded: false,
39
+ sampleCount: Array.isArray(referenceModels) ? referenceModels.length : 0,
40
+ }
41
+ }
42
+
43
+ const rawList = Array.isArray(referenceModels) && referenceModels.length > 0
44
+ ? referenceModels
45
+ : [{ provider: '', model: '' }]
46
+
47
+ // Only expand when a single model slot is configured
48
+ if (rawList.length !== 1) {
49
+ return {
50
+ references: rawList,
51
+ isExpanded: false,
52
+ sampleCount: rawList.length,
53
+ }
54
+ }
55
+
56
+ const baseSlot = rawList[0] || {}
57
+ const count = Math.min(10, Math.max(2, Number(samples) || 3))
58
+ const expandedList = []
59
+
60
+ for (let i = 0; i < count; i++) {
61
+ const temp = Array.isArray(temperatures) && typeof temperatures[i] === 'number'
62
+ ? temperatures[i]
63
+ : (DEFAULT_SELF_CONSISTENCY_TEMPERATURES[i % DEFAULT_SELF_CONSISTENCY_TEMPERATURES.length] || 0.6)
64
+
65
+ const role = Array.isArray(personas) && personas[i]
66
+ ? personas[i]
67
+ : (DEFAULT_SELF_CONSISTENCY_PERSONAS[i % DEFAULT_SELF_CONSISTENCY_PERSONAS.length] || 'proposer')
68
+
69
+ expandedList.push({
70
+ ...baseSlot,
71
+ label: baseSlot.label
72
+ ? `${baseSlot.label} [T=${temp}, ${role}]`
73
+ : `Sample ${i + 1} (${baseSlot.model || 'model'}, T=${temp})`,
74
+ temperature: temp,
75
+ role_persona: role,
76
+ sampleIndex: i + 1,
77
+ isSelfConsistencySample: true,
78
+ })
79
+ }
80
+
81
+ return {
82
+ references: expandedList,
83
+ isExpanded: true,
84
+ sampleCount: expandedList.length,
85
+ }
86
+ }
87
+
88
+ /**
89
+ * Calculates majority agreement and self-consistency metrics across candidate outputs.
90
+ *
91
+ * @param {Array<object>} candidateOutputs - Array of successful candidate results
92
+ * @returns {{ agreementRatio: number, isStrongConsensus: boolean, sampleCount: number }}
93
+ */
94
+ export function calculateSelfConsistencyAgreement(candidateOutputs = []) {
95
+ if (!Array.isArray(candidateOutputs) || candidateOutputs.length < 2) {
96
+ return {
97
+ agreementRatio: 1.0,
98
+ isStrongConsensus: true,
99
+ sampleCount: Array.isArray(candidateOutputs) ? candidateOutputs.length : 0,
100
+ }
101
+ }
102
+
103
+ const texts = candidateOutputs
104
+ .filter((c) => c && typeof c.text === 'string' && c.text.trim())
105
+ .map((c) => c.text.trim().toLowerCase().replace(/\s+/g, ' '))
106
+
107
+ if (texts.length < 2) {
108
+ return {
109
+ agreementRatio: 1.0,
110
+ isStrongConsensus: true,
111
+ sampleCount: texts.length,
112
+ }
113
+ }
114
+
115
+ // Count exact and near-match clusters
116
+ let maxClusterSize = 1
117
+ for (let i = 0; i < texts.length; i++) {
118
+ let clusterSize = 1
119
+ for (let j = 0; j < texts.length; j++) {
120
+ if (i === j) continue
121
+ // If texts are very similar (> 85% overlap)
122
+ if (texts[i] === texts[j] || (texts[i].length > 20 && texts[j].includes(texts[i].slice(0, 40)))) {
123
+ clusterSize++
124
+ }
125
+ }
126
+ if (clusterSize > maxClusterSize) maxClusterSize = clusterSize
127
+ }
128
+
129
+ const agreementRatio = Number((maxClusterSize / texts.length).toFixed(2))
130
+ return {
131
+ agreementRatio,
132
+ isStrongConsensus: agreementRatio >= 0.66,
133
+ sampleCount: texts.length,
134
+ }
135
+ }
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@goodandready/dsh-moa",
3
- "version": "0.2.73",
3
+ "version": "0.2.74",
4
4
  "description": "Mixture of Agents (MoA) plugin for DeepSeek Harness with /moa slash command",
5
5
  "type": "module",
6
6
  "main": "lib/index.js",