@goodandready/dsh-moa 0.2.76 → 0.2.78

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -54,6 +54,10 @@ Single-model AI generation often suffers from blind spots, single-perspective bi
54
54
  11. **Interactive Workbench (File Scoring 1–10, User Pick & Mix, Top-Model Glue & Judge Re-Check)**: Judge scores each candidate file (1–10) with pros and cons. Users can mix and match files across proposals in the WebUI. The turn's top-performing reference model harmonizes cross-file imports and contracts, followed by a final judge re-audit before promotion. Companion integration with [`dsh-time-machine`](https://github.com/goodandready/dsh-time-machine) for safe pre-promotion rollback checkpoints.
55
55
  12. **Adversarial White-Hat Security Fuzzing Gate (#302)**: Red Team security audit scanning candidate code for SQLi, XSS, Command Injection, Path Traversal, ReDoS, and Prototype Pollution with blocking `SECURITY_VETO`.
56
56
  13. **Chain-of-Verification (CoVe) Fact-Checking Gate for Code (#314)**: Independent pre-judge verification formulating factual API questions, validating modules and functions against standard library and project manifests to eliminate hallucinated methods.
57
+ 14. **Runtime CPU & Memory Profiler Gate (#315)**: Sandboxed micro-benchmarking of candidate execution duration, ops/sec throughput, and heap usage with hard timeout protection to select algorithmically optimal code.
58
+ 15. **Mutation Testing Gate for Candidate Tests (#316)**: Injects AST-level fault mutations to compute Mutation Score (killed vs survived mutants), weeding out tautological assertions and fake test suites.
59
+ 16. **Hybrid Private Edge MoA & Cloud Judge Sanitizer (#306)**: Preserves confidentiality with local edge proposers on MiniPC while scrubbing absolute paths, internal IPs, and API secrets before external Cloud Judge arbitration.
60
+ 17. **Heterogeneous Speculative Decoding (#317)**: High-throughput local edge draft coupled with single-pass cloud model verification, slashing response latency by 2-3x.
57
61
 
58
62
  ---
59
63
 
package/README.ru.md CHANGED
@@ -54,6 +54,10 @@
54
54
  11. **Интерактивный воркбенч (Оценка файлов 1–10, выбор в UI, склейка лучшей моделью и повторный аудит)**: Судья оценивает каждый сгенерированный файл каждого кандидата (1–10) с указанием ключевых плюсов и минусов. Пользователь комбинирует желаемый набор файлов в WebUI; лучшая модель раунда вызывается для гармонизации импортов и контрактов, после чего судья выполняет финальную валидацию перед промоушеном. Совместимо с [`dsh-time-machine`](https://github.com/goodandready/dsh-time-machine) для безопасных теневых снимков Git.
55
55
  12. **Состязательный White-Hat фильтр безопасности (#302)**: Red Team аудит безопасности перед арбитражем судьи, выявляющий SQLi, XSS, инъекции команд, Path Traversal, ReDoS и Prototype Pollution с блокирующим вердиктом `SECURITY_VETO`.
56
56
  13. **Фильтр верификации фактов и API (CoVe) (#314)**: Автономная цепочка верификации (Chain-of-Verification), формулирующая проверочные вопросы по вызовам методов и пакетам, исключающая галлюцинированные и несуществующие API библиотек.
57
+ 14. **Профайлер производительности CPU и памяти (#315)**: Изолированный бенчмаркинг времени выполнения кандидатов, пропускной способности (ops/sec) и потребления heap-памяти с защитой от зависаний для выбора алгоритмически оптимального кода.
58
+ 15. **Мутационное тестирование тестов кандидатов (#316)**: Внедрение AST-мутаций кода для расчёта Mutation Score (убитые vs выжившие мутанты), устраняющее ложноположительные и тавтологические тесты.
59
+ 16. **Гибридный приватный Edge MoA (#306)**: Локальные генераторы на MiniPC сохраняют секреты и кодовую базу в контуре, маскируя пути, IP и токены перед отправкой судье в облако.
60
+ 17. **Гетерогенное спекулятивное декодирование (#317)**: Быстрый черновик от локальной edge-модели с пакетной валидацией и доработкой флагманом в облаке с ускорением в 2-3 раза.
57
61
 
58
62
  ---
59
63
 
package/README.zh.md CHANGED
@@ -54,6 +54,10 @@
54
54
  11. **交互式工作台 (1-10分文件打分、自由混搭、最优模型粘合与裁判复审)**:裁判对每个候选模型生成的每个文件进行 1–10 分打分并列出优缺点。用户可在 WebUI 中自由勾选组合所需文件;当轮最高分模型被调用以粘合协同跨文件导入与类型接口,随后由裁判进行最终复审。支持配套插件 [`dsh-time-machine`](https://github.com/goodandready/dsh-time-machine) 在文件提升前创建影子 Git 回滚快照。
55
55
  12. **对抗性白帽安全模糊测试关卡 (#302)**:红队安全审计器在裁判前检查 SQLi、XSS、命令注入、路径穿越、ReDoS 和原型污染漏洞,发现严重漏洞时触发 `SECURITY_VETO` 否决。
56
56
  13. **代码事实与 API 验证链关卡 (CoVe) (#314)**:独立的裁判前事实核验,生成针对性 API 问题并对照标准库及项目配置清单,杜绝调用虚构不存在的幻觉方法。
57
+ 14. **运行期 CPU 与内存性能分析关卡 (#315)**:在隔离沙箱中微基准测试候选方案执行延迟、吞吐量(ops/sec)与堆内存占用,带有硬超时保护,指引裁判挑选算法最优代码。
58
+ 15. **候选测试套件变异测试关卡 (#316)**:注入 AST 级别故障变异以计算变异得分(消灭 vs 存活变异体),剔除恒真废测试与虚假测试套件。
59
+ 16. **混合隐私边缘 MoA 与云端裁判脱敏 (#306)**:本地 MiniPC 边缘模型在内网读取代码生成方案,在提交给外部云端裁判前自动脱敏绝对路径、内网 IP 与 API 密钥。
60
+ 17. **异构投机解码加速 (#317)**:本地小模型快速生成投机代码草稿块,云端旗舰模型单次批处理验证与完善,首包延迟降低 2-3 倍。
57
61
 
58
62
  ---
59
63
 
package/lib/client.js CHANGED
@@ -232,7 +232,15 @@ window.__ModuleLoader__.load({
232
232
  "aggregator.security_gate_title": "\u{1F6E1}\uFE0F Adversarial White-Hat Security Fuzzing Gate",
233
233
  "aggregator.security_gate_desc": "Red-team security auditor checks candidate proposals for SQLi, XSS, ReDoS, Command Injection and Path Traversal with blocking SECURITY_VETO.",
234
234
  "aggregator.cove_gate_title": "\u{1F50D} Chain-of-Verification (CoVe) Fact-Checking Gate",
235
- "aggregator.cove_gate_desc": "Formulates API existence & method signature verification questions, disqualifying candidates that use hallucinated methods."
235
+ "aggregator.cove_gate_desc": "Formulates API existence & method signature verification questions, disqualifying candidates that use hallucinated methods.",
236
+ "aggregator.profiler_gate_title": "\u26A1 Automated Runtime Memory & CPU Profiler Gate",
237
+ "aggregator.profiler_gate_desc": "Runs isolated micro-benchmarks measuring execution latency, throughput (ops/sec), and heap memory usage.",
238
+ "aggregator.mutation_gate_title": "\u{1F9EC} Runtime Mutation Testing Gate for Candidate Tests",
239
+ "aggregator.mutation_gate_desc": "Injects synthetic faults into candidate code to test test-suite robustness, eliminating tautological and weak tests.",
240
+ "aggregator.privacy_gate_title": "\u{1F6E1}\uFE0F Hybrid Private Edge MoA (Local Edge Proposers + Cloud Judge)",
241
+ "aggregator.privacy_gate_desc": "Scans and redacts proprietary secrets, local paths, and private network IPs before forwarding proposals to external cloud judges.",
242
+ "aggregator.speculative_gate_title": "\u26A1 Heterogeneous Speculative Decoding (Local Draft + Cloud Verifier)",
243
+ "aggregator.speculative_gate_desc": "Lightweight local edge model generates high-speed token drafts verified and completed in one fast pass by the cloud model."
236
244
  };
237
245
  const zh = {
238
246
  title: "\u667A\u80FD\u4F53\u6DF7\u5408 (MoA)",
@@ -455,7 +463,15 @@ window.__ModuleLoader__.load({
455
463
  "aggregator.security_gate_title": "\u{1F6E1}\uFE0F \u5BF9\u6297\u6027\u767D\u5E3D\u5B89\u5168\u6A21\u7CCA\u6D4B\u8BD5\u5173\u5361",
456
464
  "aggregator.security_gate_desc": "\u7EA2\u961F\u5B89\u5168\u5BA1\u8BA1\u5668\u68C0\u67E5\u5019\u9009\u65B9\u6848\u4E2D\u7684 SQLi\u3001XSS\u3001ReDoS\u3001\u547D\u4EE4\u6CE8\u5165\u548C\u8DEF\u5F84\u7A7F\u8D8A\u6F0F\u6D1E\uFF0C\u5E76\u5728\u53D1\u73B0\u4E25\u91CD\u6F0F\u6D1E\u65F6\u89E6\u53D1 SECURITY_VETO \u5426\u51B3\u3002",
457
465
  "aggregator.cove_gate_title": "\u{1F50D} \u9A8C\u8BC1\u94FE (CoVe) \u4EE3\u7801\u4E8B\u5B9E\u4E0E API \u6838\u67E5\u5173\u5361",
458
- "aggregator.cove_gate_desc": "\u751F\u6210 API \u5B58\u5728\u6027\u4E0E\u65B9\u6CD5\u7B7E\u540D\u6838\u9A8C\u95EE\u9898\uFF0C\u81EA\u52A8\u60E9\u7F5A\u6216\u53D6\u6D88\u4F7F\u7528\u865A\u6784\u5E7B\u89C9\u65B9\u6CD5\u7684\u5019\u9009\u65B9\u6848\u3002"
466
+ "aggregator.cove_gate_desc": "\u751F\u6210 API \u5B58\u5728\u6027\u4E0E\u65B9\u6CD5\u7B7E\u540D\u6838\u9A8C\u95EE\u9898\uFF0C\u81EA\u52A8\u60E9\u7F5A\u6216\u53D6\u6D88\u4F7F\u7528\u865A\u6784\u5E7B\u89C9\u65B9\u6CD5\u7684\u5019\u9009\u65B9\u6848\u3002",
467
+ "aggregator.profiler_gate_title": "\u26A1 \u81EA\u52A8\u8FD0\u884C\u65F6 CPU \u4E0E\u5185\u5B58\u6027\u80FD\u5206\u6790\u5173\u5361",
468
+ "aggregator.profiler_gate_desc": "\u5728\u9694\u79BB\u6C99\u7BB1\u4E2D\u8FD0\u884C\u5FAE\u57FA\u51C6\u6D4B\u8BD5\uFF0C\u7CBE\u786E\u5EA6\u91CF\u6267\u884C\u5EF6\u8FDF\u3001\u541E\u5410\u91CF (ops/sec) \u548C\u5806\u5185\u5B58\u5206\u914D\u3002",
469
+ "aggregator.mutation_gate_title": "\u{1F9EC} \u5019\u9009\u6D4B\u8BD5\u5957\u4EF6\u8FD0\u884C\u65F6\u53D8\u5F02\u6D4B\u8BD5\u5173\u5361",
470
+ "aggregator.mutation_gate_desc": "\u5411\u4EE3\u7801\u6CE8\u5165\u5408\u6210\u6545\u969C\u5E76\u9A8C\u8BC1\u6D4B\u8BD5\u7684\u6355\u83B7\u80FD\u529B\uFF0C\u5254\u9664\u540C\u4E49\u53CD\u590D\u4E0E\u65E0\u6548\u7684\u865A\u5047\u6D4B\u8BD5\u3002",
471
+ "aggregator.privacy_gate_title": "\u{1F6E1}\uFE0F \u6DF7\u5408\u9690\u79C1\u8FB9\u7F18 MoA (\u672C\u5730\u751F\u6210 + \u4E91\u7AEF\u88C1\u5224)",
472
+ "aggregator.privacy_gate_desc": "\u5728\u5C06\u5019\u9009\u63D0\u6848\u53D1\u9001\u7ED9\u5916\u90E8\u4E91\u7AEF\u88C1\u5224\u4E4B\u524D\uFF0C\u626B\u63CF\u5E76\u8131\u654F\u4E13\u6709\u5BC6\u94A5\u3001\u672C\u5730\u7EDD\u5BF9\u8DEF\u5F84\u4E0E\u79C1\u6709\u7F51\u7EDC IP\u3002",
473
+ "aggregator.speculative_gate_title": "\u26A1 \u5F02\u6784\u6295\u673A\u89E3\u7801 (\u672C\u5730\u8349\u7A3F + \u4E91\u7AEF\u9A8C\u8BC1)",
474
+ "aggregator.speculative_gate_desc": "\u8F7B\u91CF\u7EA7\u672C\u5730\u8FB9\u7F18\u6A21\u578B\u751F\u6210\u9AD8\u901F\u8349\u7A3F\uFF0C\u7531\u5F3A\u5927\u7684\u4E91\u7AEF\u6A21\u578B\u4E00\u6B21\u6027\u5FEB\u901F\u9A8C\u8BC1\u5E76\u5B8C\u5584\u3002"
459
475
  };
460
476
  function makeT(dict, fallback) {
461
477
  return function t(key, vars) {
@@ -2660,6 +2676,90 @@ window.__ModuleLoader__.load({
2660
2676
  t("aggregator.cove_gate_desc") || "Formulates API existence & method signature verification questions, disqualifying candidates that use hallucinated methods."
2661
2677
  )
2662
2678
  ),
2679
+ /* Automated Runtime Memory & CPU Profiler Gate (#315) */
2680
+ React.createElement(
2681
+ "div",
2682
+ { style: { display: "flex", flexDirection: "column", gap: 6, padding: "8px 12px", background: "var(--dsw-alias-bg-layer-2)", borderRadius: 8, border: "1px solid var(--dsw-alias-border-l2)", marginTop: 8 } },
2683
+ React.createElement(
2684
+ "label",
2685
+ { style: { display: "flex", alignItems: "center", gap: 8, cursor: "pointer", fontSize: 13, fontWeight: 500 } },
2686
+ React.createElement("input", {
2687
+ type: "checkbox",
2688
+ checked: Boolean(currentPreset.profiler_gate_enabled),
2689
+ onChange: (e) => updateCurrentPreset((p) => ({ ...p, profiler_gate_enabled: e.target.checked })),
2690
+ style: { accentColor: "var(--dsw-alias-state-brand-primary, var(--dsw-alias-label-primary))", cursor: "pointer" }
2691
+ }),
2692
+ t("aggregator.profiler_gate_title") || "\u26A1 Automated Runtime Memory & CPU Profiler Gate"
2693
+ ),
2694
+ React.createElement(
2695
+ "div",
2696
+ { style: { fontSize: 12, color: "var(--dsw-alias-label-secondary)", marginLeft: 22 } },
2697
+ t("aggregator.profiler_gate_desc") || "Runs isolated micro-benchmarks measuring execution latency, throughput (ops/sec), and heap memory usage."
2698
+ )
2699
+ ),
2700
+ /* Runtime Mutation Testing Gate (#316) */
2701
+ React.createElement(
2702
+ "div",
2703
+ { style: { display: "flex", flexDirection: "column", gap: 6, padding: "8px 12px", background: "var(--dsw-alias-bg-layer-2)", borderRadius: 8, border: "1px solid var(--dsw-alias-border-l2)", marginTop: 8 } },
2704
+ React.createElement(
2705
+ "label",
2706
+ { style: { display: "flex", alignItems: "center", gap: 8, cursor: "pointer", fontSize: 13, fontWeight: 500 } },
2707
+ React.createElement("input", {
2708
+ type: "checkbox",
2709
+ checked: Boolean(currentPreset.mutation_testing_enabled),
2710
+ onChange: (e) => updateCurrentPreset((p) => ({ ...p, mutation_testing_enabled: e.target.checked })),
2711
+ style: { accentColor: "var(--dsw-alias-state-brand-primary, var(--dsw-alias-label-primary))", cursor: "pointer" }
2712
+ }),
2713
+ t("aggregator.mutation_gate_title") || "\u{1F9EC} Runtime Mutation Testing Gate for Candidate Tests"
2714
+ ),
2715
+ React.createElement(
2716
+ "div",
2717
+ { style: { fontSize: 12, color: "var(--dsw-alias-label-secondary)", marginLeft: 22 } },
2718
+ t("aggregator.mutation_gate_desc") || "Injects synthetic faults into candidate code to test test-suite robustness, eliminating tautological and weak tests."
2719
+ )
2720
+ ),
2721
+ /* Hybrid Edge Privacy Gate (#306) */
2722
+ React.createElement(
2723
+ "div",
2724
+ { style: { display: "flex", flexDirection: "column", gap: 4 } },
2725
+ React.createElement(
2726
+ "label",
2727
+ { style: { display: "flex", alignItems: "center", gap: 8, fontSize: 13, fontWeight: 600, color: "var(--dsw-alias-label-primary)", cursor: "pointer" } },
2728
+ React.createElement("input", {
2729
+ type: "checkbox",
2730
+ checked: Boolean(currentPreset.hybrid_edge_privacy_enabled),
2731
+ onChange: (e) => updateCurrentPreset((p) => ({ ...p, hybrid_edge_privacy_enabled: e.target.checked })),
2732
+ style: { accentColor: "var(--dsw-alias-state-brand-primary, var(--dsw-alias-label-primary))", cursor: "pointer" }
2733
+ }),
2734
+ t("aggregator.privacy_gate_title") || "\u{1F6E1}\uFE0F Hybrid Private Edge MoA (Local Edge Proposers + Cloud Judge)"
2735
+ ),
2736
+ React.createElement(
2737
+ "div",
2738
+ { style: { fontSize: 12, color: "var(--dsw-alias-label-secondary)", marginLeft: 22 } },
2739
+ t("aggregator.privacy_gate_desc") || "Scans and redacts proprietary secrets, local paths, and private network IPs before forwarding proposals to external cloud judges."
2740
+ )
2741
+ ),
2742
+ /* Heterogeneous Speculative Decoding (#317) */
2743
+ React.createElement(
2744
+ "div",
2745
+ { style: { display: "flex", flexDirection: "column", gap: 4 } },
2746
+ React.createElement(
2747
+ "label",
2748
+ { style: { display: "flex", alignItems: "center", gap: 8, fontSize: 13, fontWeight: 600, color: "var(--dsw-alias-label-primary)", cursor: "pointer" } },
2749
+ React.createElement("input", {
2750
+ type: "checkbox",
2751
+ checked: Boolean(currentPreset.speculative_draft_enabled),
2752
+ onChange: (e) => updateCurrentPreset((p) => ({ ...p, speculative_draft_enabled: e.target.checked })),
2753
+ style: { accentColor: "var(--dsw-alias-state-brand-primary, var(--dsw-alias-label-primary))", cursor: "pointer" }
2754
+ }),
2755
+ t("aggregator.speculative_gate_title") || "\u26A1 Heterogeneous Speculative Decoding (Local Draft + Cloud Verifier)"
2756
+ ),
2757
+ React.createElement(
2758
+ "div",
2759
+ { style: { fontSize: 12, color: "var(--dsw-alias-label-secondary)", marginLeft: 22 } },
2760
+ t("aggregator.speculative_gate_desc") || "Lightweight local edge model generates high-speed token drafts verified and completed in one fast pass by the cloud model."
2761
+ )
2762
+ ),
2663
2763
  /* Fallback Judges Chain */
2664
2764
  React.createElement(
2665
2765
  "div",
package/lib/events.js CHANGED
@@ -13,6 +13,10 @@ export const MOA_EVENTS = {
13
13
  PREFERENCES_APPLIED: 'moa/preferences-applied',
14
14
  SECURITY_AUDIT: 'moa/security-audit',
15
15
  COVE_VERIFIED: 'moa/cove-verified',
16
+ PROFILER_COMPLETE: 'moa/profiler-complete',
17
+ MUTATION_COMPLETE: 'moa/mutation-complete',
18
+ PRIVACY_SANITIZED: 'moa/privacy-sanitized',
19
+ SPECULATIVE_DRAFT: 'moa/speculative-draft',
16
20
  CRITIQUE_COMPLETE: 'moa/critique-complete',
17
21
  JUDGE_VERDICT: 'moa/judge-verdict',
18
22
  CONTEXT_PRUNING: 'moa/context-pruning',
package/lib/moa-schema.js CHANGED
@@ -95,6 +95,18 @@ export const PresetSchema = z.object({
95
95
  security_gate_strict_veto: z.boolean().default(true),
96
96
  cove_verification_enabled: z.boolean().default(false),
97
97
  cove_max_questions: z.number().default(5),
98
+ profiler_gate_enabled: z.boolean().default(false),
99
+ profiler_iterations: z.number().default(100),
100
+ profiler_timeout_ms: z.number().default(2000),
101
+ mutation_testing_enabled: z.boolean().default(false),
102
+ min_mutation_score: z.number().default(0.70),
103
+ hybrid_edge_privacy_enabled: z.boolean().default(false),
104
+ cloud_judge_sanitization_enabled: z.boolean().default(true),
105
+ redact_paths: z.boolean().default(true),
106
+ redact_secrets: z.boolean().default(true),
107
+ redact_ips: z.boolean().default(true),
108
+ speculative_draft_enabled: z.boolean().default(false),
109
+ speculative_block_size: z.number().default(256),
98
110
  })
99
111
 
100
112
  export const Config = z.object({
@@ -136,6 +148,18 @@ export const Config = z.object({
136
148
  security_gate_strict_veto: z.boolean().default(true).volatile(),
137
149
  cove_verification_enabled: z.boolean().default(false).volatile(),
138
150
  cove_max_questions: z.number().default(5).volatile(),
151
+ profiler_gate_enabled: z.boolean().default(false).volatile(),
152
+ profiler_iterations: z.number().default(100).volatile(),
153
+ profiler_timeout_ms: z.number().default(2000).volatile(),
154
+ mutation_testing_enabled: z.boolean().default(false).volatile(),
155
+ min_mutation_score: z.number().default(0.70).volatile(),
156
+ hybrid_edge_privacy_enabled: z.boolean().default(false).volatile(),
157
+ cloud_judge_sanitization_enabled: z.boolean().default(true).volatile(),
158
+ redact_paths: z.boolean().default(true).volatile(),
159
+ redact_secrets: z.boolean().default(true).volatile(),
160
+ redact_ips: z.boolean().default(true).volatile(),
161
+ speculative_draft_enabled: z.boolean().default(false).volatile(),
162
+ speculative_block_size: z.number().default(256).volatile(),
139
163
  smart_routing_model: ModelSlotSchema.default({ provider: '', model: '' }).volatile(),
140
164
  prices: z.dict(PriceRow).default({}).volatile(),
141
165
  presets: z.array(PresetSchema).default([
@@ -1,3 +1,4 @@
1
+ import { createSpeculativeCallLlm } from './speculative-verifier.js'
1
2
  import { createCachingCallLlm } from './semantic-cache.js'
2
3
  import { runSpeculativeMoa } from '../moa-speculative.js'
3
4
  import { runAdversarialTddLoop } from '../moa-adversarial-tdd.js'
@@ -47,6 +48,23 @@ export async function stageCandidateRound1({
47
48
  ctx,
48
49
  })
49
50
  : callLlm
51
+ let candidateCallLlm = effectiveCallLlm
52
+ const draftModelSlot = typeof preset?.speculative_draft_model === 'object' && preset?.speculative_draft_model?.model
53
+ ? preset.speculative_draft_model
54
+ : typeof preset?.speculative_draft_model === 'string' && preset?.speculative_draft_model
55
+ ? { provider: 'ollama', model: preset.speculative_draft_model }
56
+ : null
57
+
58
+ if (preset?.speculative_draft_enabled && draftModelSlot) {
59
+ candidateCallLlm = createSpeculativeCallLlm(effectiveCallLlm, {
60
+ draftSlot: draftModelSlot,
61
+ blockSize: preset.speculative_block_size || 256,
62
+ ctx,
63
+ runId,
64
+ onProgress,
65
+ })
66
+ }
67
+
50
68
  const lastMsgText = enrichedMessages[enrichedMessages.length - 1]?.content || ''
51
69
  const adaptiveRefs = preset?.adaptive_personas_enabled
52
70
  ? assignAdaptivePersonas(effectiveReferences, typeof lastMsgText === 'string' ? lastMsgText : '').references
@@ -75,7 +93,7 @@ export async function stageCandidateRound1({
75
93
 
76
94
  if (!referenceOutputs) {
77
95
  referenceOutputs = preset?.subagent_swarm_enabled
78
- ? await runSubagentSwarm(adaptiveRefs, enrichedMessages, { temperature: refTemp, maxTokens, signal, cwd, runId }, effectiveCallLlm, onProgress)
96
+ ? await runSubagentSwarm(adaptiveRefs, enrichedMessages, { temperature: refTemp, maxTokens, signal, cwd, runId }, candidateCallLlm, onProgress)
79
97
  : await runReferencesParallel(
80
98
  adaptiveRefs,
81
99
  enrichedMessages,
@@ -93,7 +111,7 @@ export async function stageCandidateRound1({
93
111
  maxRetries: candidateRetries,
94
112
  signal,
95
113
  },
96
- effectiveCallLlm,
114
+ candidateCallLlm,
97
115
  onProgress
98
116
  )
99
117
  }
@@ -0,0 +1,323 @@
1
+ /**
2
+ * DeepSeek Harness Mixture of Agents (MoA) — Mutation Testing Gate for Candidate Tests (#316)
3
+ *
4
+ * Injects deterministic AST-level fault mutations into candidate implementations and evaluates
5
+ * candidate test suites against mutants to calculate Mutation Score (killed vs survived mutants),
6
+ * eliminating false-positive and tautological tests before Judge arbitration.
7
+ */
8
+
9
+ import vm from 'node:vm'
10
+ import { MOA_EVENTS, emitEvent } from '../events.js'
11
+ import { extractExecutableCode } from './runtime-profiler.js'
12
+
13
+ /**
14
+ * Mutation operator patterns.
15
+ */
16
+ export const MUTATION_OPERATORS = [
17
+ {
18
+ name: 'invert_strict_equality',
19
+ regex: /===/g,
20
+ replacement: '!==',
21
+ description: 'Replaced strict equality (===) with inequality (!==)',
22
+ },
23
+ {
24
+ name: 'invert_relational_lt',
25
+ regex: /<(?!=)/g,
26
+ replacement: '>=',
27
+ description: 'Replaced less-than (<) with greater-or-equal (>=)',
28
+ },
29
+ {
30
+ name: 'invert_relational_gt',
31
+ regex: />(?!=)/g,
32
+ replacement: '<=',
33
+ description: 'Replaced greater-than (>) with less-or-equal (<=)',
34
+ },
35
+ {
36
+ name: 'invert_logical_and',
37
+ regex: /&&/g,
38
+ replacement: '||',
39
+ description: 'Replaced logical AND (&&) with logical OR (||)',
40
+ },
41
+ {
42
+ name: 'invert_arithmetic_plus',
43
+ regex: /(?<=\w\s*)\+(?=\s*\w)/g,
44
+ replacement: '-',
45
+ description: 'Replaced addition (+) with subtraction (-)',
46
+ },
47
+ {
48
+ name: 'invert_boolean_true',
49
+ regex: /\btrue\b/g,
50
+ replacement: 'false',
51
+ description: 'Replaced literal true with false',
52
+ },
53
+ {
54
+ name: 'mutate_return_zero',
55
+ regex: /return\s+0;/g,
56
+ replacement: 'return 1;',
57
+ description: 'Mutated return 0 to return 1',
58
+ },
59
+ ]
60
+
61
+ /**
62
+ * Generates an array of mutant variations from a single source code string.
63
+ *
64
+ * @param {string} sourceCode
65
+ * @param {object} [options]
66
+ * @returns {Array<object>} Up to maxMutants mutant objects
67
+ */
68
+ export function generateMutants(sourceCode, options = {}) {
69
+ const maxMutants = Math.min(10, Math.max(1, options.maxMutants || 5))
70
+ const mutants = []
71
+ if (!sourceCode || typeof sourceCode !== 'string') return mutants
72
+
73
+ for (const op of MUTATION_OPERATORS) {
74
+ let match
75
+ const re = new RegExp(op.regex.source, op.regex.flags)
76
+ while ((match = re.exec(sourceCode)) !== null) {
77
+ const idx = match.index
78
+ const len = match[0].length
79
+ const mutantCode = sourceCode.slice(0, idx) + op.replacement + sourceCode.slice(idx + len)
80
+
81
+ mutants.push({
82
+ id: mutants.length + 1,
83
+ operator: op.name,
84
+ description: op.description,
85
+ originalSnippet: match[0],
86
+ replacementSnippet: op.replacement,
87
+ mutantCode,
88
+ })
89
+
90
+ if (mutants.length >= maxMutants) return mutants
91
+ }
92
+ }
93
+
94
+ return mutants
95
+ }
96
+
97
+ /**
98
+ * Runs a test script against an implementation in an isolated VM context.
99
+ *
100
+ * @param {string} codeToExecute
101
+ * @param {number} [timeoutMs=1000]
102
+ * @returns {{ ok: boolean, error: string | null }}
103
+ */
104
+ function executeScript(codeToExecute, timeoutMs = 1000) {
105
+ const sandbox = {
106
+ console: { log: () => {}, warn: () => {}, error: () => {} },
107
+ assert: (cond, msg) => { if (!cond) throw new Error(msg || 'Assertion failed'); },
108
+ Math, Date, Array, Object, String, Number, Boolean, Set, Map, JSON,
109
+ }
110
+ sandbox.assert.equal = (a, b) => { if (a !== b) throw new Error(`Assertion failed: ${a} !== ${b}`); }
111
+ sandbox.assert.strictEqual = (a, b) => { if (a !== b) throw new Error(`Assertion failed: ${a} !== ${b}`); }
112
+ sandbox.assert.ok = (v) => { if (!v) throw new Error('Assertion failed: falsy value'); }
113
+
114
+ try {
115
+ const context = vm.createContext(sandbox)
116
+ const script = new vm.Script(codeToExecute)
117
+ script.runInContext(context, { timeout: timeoutMs })
118
+ return { ok: true, error: null }
119
+ } catch (err) {
120
+ return { ok: false, error: err.message }
121
+ }
122
+ }
123
+
124
+ /**
125
+ * Evaluates a candidate's implementation against generated mutants.
126
+ *
127
+ * @param {string} rawText - Candidate text containing code or tests
128
+ * @param {object} [options]
129
+ * @returns {object} Mutation testing report
130
+ */
131
+ export function auditCandidateMutations(rawText, options = {}) {
132
+ const { minMutationScore = 0.7, maxMutants = 5, timeoutMs = 1000 } = options
133
+ const code = extractExecutableCode(rawText)
134
+
135
+ if (!code || !code.trim()) {
136
+ return {
137
+ passed: true,
138
+ mutationScore: 100,
139
+ totalMutants: 0,
140
+ killedCount: 0,
141
+ survivedCount: 0,
142
+ mutants: [],
143
+ summary: 'Mutation Testing: No code to mutate.',
144
+ }
145
+ }
146
+
147
+ // 1. Verify baseline executes cleanly
148
+ const baseline = executeScript(code, timeoutMs)
149
+ if (!baseline.ok) {
150
+ return {
151
+ passed: false,
152
+ mutationScore: 0,
153
+ totalMutants: 0,
154
+ killedCount: 0,
155
+ survivedCount: 0,
156
+ mutants: [],
157
+ error: `Baseline test execution failed: ${baseline.error}`,
158
+ summary: `Baseline execution failed (${baseline.error}).`,
159
+ }
160
+ }
161
+
162
+ // 2. Generate mutations on the code
163
+ const mutants = generateMutants(code, { maxMutants })
164
+ if (mutants.length === 0) {
165
+ return {
166
+ passed: true,
167
+ mutationScore: 100,
168
+ totalMutants: 0,
169
+ killedCount: 0,
170
+ survivedCount: 0,
171
+ mutants: [],
172
+ summary: 'Mutation Testing: Code contains no mutable operators.',
173
+ }
174
+ }
175
+
176
+ // 3. Test each mutant
177
+ let killedCount = 0
178
+ let survivedCount = 0
179
+ const evaluatedMutants = []
180
+
181
+ for (const m of mutants) {
182
+ const res = executeScript(m.mutantCode, timeoutMs)
183
+ if (!res.ok) {
184
+ // Mutant was killed (good! The test broke as expected)
185
+ killedCount++
186
+ evaluatedMutants.push({
187
+ ...m,
188
+ status: 'KILLED',
189
+ reason: res.error,
190
+ })
191
+ } else {
192
+ // Mutant survived (bad! Code was broken but test passed anyway)
193
+ survivedCount++
194
+ evaluatedMutants.push({
195
+ ...m,
196
+ status: 'SURVIVED',
197
+ reason: 'Test suite passed despite injected fault.',
198
+ })
199
+ }
200
+ }
201
+
202
+ const mutationRatio = Number((killedCount / mutants.length).toFixed(2))
203
+ const mutationScore = Math.round(mutationRatio * 100)
204
+ const passed = mutationRatio >= minMutationScore
205
+
206
+ const summary = passed
207
+ ? `🧬 Mutation Testing: PASSED (Score ${mutationScore}%). Killed ${killedCount}/${mutants.length} mutant(s).`
208
+ : `⚠️ Mutation Testing: WEAK TESTS (Score ${mutationScore}% < ${Math.round(minMutationScore * 100)}%). ${survivedCount} mutant(s) survived undetected.`
209
+
210
+ return {
211
+ passed,
212
+ mutationScore,
213
+ mutationRatio,
214
+ totalMutants: mutants.length,
215
+ killedCount,
216
+ survivedCount,
217
+ mutants: evaluatedMutants,
218
+ summary,
219
+ }
220
+ }
221
+
222
+ /**
223
+ * Runs the Mutation Testing Gate across all candidate solutions.
224
+ *
225
+ * @param {Array<object>} candidates
226
+ * @param {object} [options]
227
+ * @returns {object} Mutation gate report
228
+ */
229
+ export function runMutationGate(candidates, options = {}) {
230
+ const { ctx, runId, onProgress, minMutationScore = 0.70 } = options
231
+ const results = []
232
+
233
+ if (!Array.isArray(candidates) || candidates.length === 0) {
234
+ return {
235
+ results: [],
236
+ passedCount: 0,
237
+ mutationDirective: null,
238
+ }
239
+ }
240
+
241
+ for (let i = 0; i < candidates.length; i++) {
242
+ const cand = candidates[i]
243
+ const text = typeof cand === 'string' ? cand : cand.output || cand.text || cand.code || ''
244
+ const report = auditCandidateMutations(text, { minMutationScore, ...options })
245
+
246
+ if (typeof cand === 'object' && cand !== null) {
247
+ cand.mutationReport = report
248
+ cand.mutationScore = report.mutationScore
249
+ }
250
+
251
+ results.push({
252
+ candidateIndex: i,
253
+ model: cand?.model || `Candidate ${i + 1}`,
254
+ report,
255
+ })
256
+
257
+ if (ctx) {
258
+ emitEvent(ctx, MOA_EVENTS.MUTATION_COMPLETE, {
259
+ runId,
260
+ candidateIndex: i,
261
+ mutationScore: report.mutationScore,
262
+ killedCount: report.killedCount,
263
+ totalMutants: report.totalMutants,
264
+ passed: report.passed,
265
+ })
266
+ }
267
+ }
268
+
269
+ const passedCount = results.filter((r) => r.report.passed).length
270
+
271
+ if (typeof onProgress === 'function' && results.length > 0) {
272
+ const summaryList = results.map((r) => `Ref ${r.candidateIndex + 1}: ${r.report.mutationScore}%`).join(', ')
273
+ onProgress(`🧬 *Mutation Testing Gate*: ${summaryList}\n`)
274
+ }
275
+
276
+ const mutationDirective = formatMutationDirective(results)
277
+
278
+ return {
279
+ results,
280
+ passedCount,
281
+ mutationDirective,
282
+ }
283
+ }
284
+
285
+ /**
286
+ * Formats mutation testing results into a prompt directive for the Judge.
287
+ *
288
+ * @param {Array<object>} results
289
+ * @returns {string|null}
290
+ */
291
+ export function formatMutationDirective(results) {
292
+ if (!Array.isArray(results) || results.length === 0) return null
293
+
294
+ const lines = [
295
+ '### 🧬 RUNTIME MUTATION TESTING & FAULT INJECTION REPORT',
296
+ 'Evaluation of test robustness against synthetic fault injection (killing code mutants):',
297
+ ]
298
+
299
+ for (const r of results) {
300
+ const label = `Candidate ${r.candidateIndex + 1}`
301
+ const rep = r.report
302
+ if (rep.totalMutants === 0) {
303
+ lines.push(`- **${label}**: ℹ️ No mutable operators found in test suite.`)
304
+ } else if (rep.passed) {
305
+ lines.push(
306
+ `- **${label}**: ✅ **RESILIENT TESTS** (Score ${rep.mutationScore}%) — Killed ${rep.killedCount}/${rep.totalMutants} fault mutants.`
307
+ )
308
+ } else {
309
+ lines.push(
310
+ `- **${label}**: ⚠️ **WEAK/TAUTOLOGICAL TESTS** (Score ${rep.mutationScore}%) — ${rep.survivedCount}/${rep.totalMutants} mutant(s) survived! The tests pass even when logic is inverted.`
311
+ )
312
+ for (const m of (rep.mutants || []).filter((m) => m.status === 'SURVIVED')) {
313
+ lines.push(` * ${m.description} [Survived]`)
314
+ }
315
+ }
316
+ }
317
+
318
+ lines.push(
319
+ '\n**Judge Quality Mandate**: Prefer solutions whose test suites have a high Mutation Score. Solutions with surviving mutants have weak or tautological tests that do not genuinely verify behavior.'
320
+ )
321
+
322
+ return lines.join('\n')
323
+ }