@goodandready/dsh-moa 0.2.45 → 0.2.47
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +2 -2
- package/README.ru.md +2 -2
- package/lib/client.js +7 -1
- package/lib/file-workspace.js +12 -12
- package/lib/index.js +12 -1
- package/lib/moa-budget.js +40 -2
- package/lib/moa-llm.js +1 -1
- package/lib/moa-multi-judge.js +64 -0
- package/lib/moa-runner.js +37 -35
- package/lib/moa-test-gate.js +82 -10
- package/lib/routes.js +47 -79
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -150,7 +150,7 @@ Inspect line-by-line differences between candidate proposals and the curator's s
|
|
|
150
150
|
Before promoting any winning candidate files over the workspace root, `dsh-moa` invokes the local `dsh-time-machine` service to create a shadow Git checkpoint (`moa-pre-promotion: candidate-N`). If `dsh-time-machine` is absent or unreachable, file promotion is blocked (fail-closed) to protect uncheckpointed workspace files against unintended overwrites (override with `force: true` only when automated checkpoints are explicitly unneeded).
|
|
151
151
|
|
|
152
152
|
### 9. Automated Test Execution Gate
|
|
153
|
-
When `test_gate_enabled: true` and `test_command` (e.g. `npm test` or `pytest`) are configured, candidate code is executed in an ephemeral sandbox overlay. The judge receives concrete test outcomes, durations, and output logs to ground decisions in objective verification.
|
|
153
|
+
When `test_gate_enabled: true` and `test_command` (e.g. `npm test` or `pytest`) are configured, candidate code is executed in an ephemeral sandbox overlay. The test sandbox uses process containment fences and path validation to block unauthorized host file access and computed shell evasion (base64/eval/pipes), while allowing safe HTTP route literals. The judge receives concrete test outcomes, durations, and output logs to ground decisions in objective verification.
|
|
154
154
|
|
|
155
155
|
### 10. Multi-Judge Panel & Consensus Voting
|
|
156
156
|
When `multi_judge_enabled: true`, candidate solutions are independently evaluated by a panel of judge models. Winner selection supports `majority`, `highest_score`, or `unanimous` consensus strategies.
|
|
@@ -165,7 +165,7 @@ When invoking `/moa` without explicit preset flags, the router classifies prompt
|
|
|
165
165
|
As each candidate completes, live token counts and USD costs are streamed directly into the chat based on catalog pricing and vendor rates.
|
|
166
166
|
|
|
167
167
|
### 14. Cost Budget Guardrails (Trim & Abort Modes)
|
|
168
|
-
`budget_guard_enabled` and `max_budget_usd` guard against accidental spend. Mode `trim` automatically reduces the candidate pool to fit the budget, while `abort` cancels execution before token consumption.
|
|
168
|
+
`budget_guard_enabled` and `max_budget_usd` guard against accidental spend. Upfront and runtime phase guards account for candidate generation, multi-judge panel voting, and downstream synthesis prompt overhead including custom evaluation criteria (`judge_criteria`). Mode `trim` automatically reduces the candidate pool to fit the budget, while `abort` cancels execution before token consumption. In abort payloads, all panel and candidate token metrics are fully preserved.
|
|
169
169
|
|
|
170
170
|
### 15. Temperature Gradient Exploration & Per-Candidate Temperature
|
|
171
171
|
Supports individual candidate temperatures (`slot.temperature`) and `temperature_gradient_enabled` to distribute temperatures (0.2 → 0.9) across candidates for maximum architectural diversity.
|
package/README.ru.md
CHANGED
|
@@ -150,7 +150,7 @@ graph TD
|
|
|
150
150
|
Перед промоушном файлов победителя в корень рабочей области `dsh-moa` автоматически обращается к сервису `dsh-time-machine` для создания снимка (`moa-pre-promotion: candidate-N`). Если плагин недоступен или создание снимка завершилось ошибкой, перезапись файлов блокируется во избежание потери несохранённых данных (переопределяется параметром `force: true`, только если создание снимков намеренно отключено).
|
|
151
151
|
|
|
152
152
|
### 9. Автоматический шлюз выполнения тестов (Automated Test Execution Gate)
|
|
153
|
-
При включении `test_gate_enabled: true` и указании тестовой команды (`test_command`, например `npm test` или `pytest`) каждый кандидат запускается в изолированном оверлейном
|
|
153
|
+
При включении `test_gate_enabled: true` и указании тестовой команды (`test_command`, например `npm test` или `pytest`) каждый кандидат запускается в изолированном оверлейном окружении с защитой от выхода за пределы песочницы (блокировка вычисляемых shell-команд base64/eval/pipes при сохранении безопасных строковых REST-маршрутов). Судья получает объективный вердикт с кодами возврата, временем исполнения и логами тестов до вынесения итогового решения.
|
|
154
154
|
|
|
155
155
|
### 10. Коллегия судей и консенсусное голосование (Multi-Judge Panel)
|
|
156
156
|
При активации опции `multi_judge_enabled` решения кандидатов оценивает независимая панель судейских моделей с поддержкой стратегий голосования: `majority` (большинство), `highest_score` (наивысший средний балл) или `unanimous` (единогласный консенсус).
|
|
@@ -165,7 +165,7 @@ graph TD
|
|
|
165
165
|
По мере завершения каждого кандидата плагин в реальном времени транслирует в чат накопленный расход токенов и расчетную стоимость в USD на базе встроенного каталога тарифов OpenRouter и прямых цен провайдеров.
|
|
166
166
|
|
|
167
167
|
### 14. Бюджетные ограничения расходов (Cost Budget Guardrails)
|
|
168
|
-
Опция `budget_guard_enabled` с лимитом `max_budget_usd` защищает от непредвиденных
|
|
168
|
+
Опция `budget_guard_enabled` с лимитом `max_budget_usd` защищает от непредвиденных трат. Упреждающие и пофазные проверки учитывают генерацию кандидатов, панель судей и оверхед промпта синтеза с учётом пользовательских критериев (`judge_criteria`). Режим `trim` автоматически сокращает пул кандидатов до самых экономичных моделей, укладывающихся в бюджет, а режим `abort` прерывает выполнение до начала генерации. При прерывании полностью сохраняются метрики токенов кандидатов и мульти-судейской панели.
|
|
169
169
|
|
|
170
170
|
### 15. Градиент температурной диверсификации и точечная температура
|
|
171
171
|
Поддерживается как индивидуальная настройка температуры для каждого кандидата (`slot.temperature`), так и опция `temperature_gradient_enabled`, автоматически распределяющая температуры кандидатов по шкале от 0.2 до 0.9 для достижения максимального разнообразия инженерных подходов.
|
package/lib/client.js
CHANGED
|
@@ -119,6 +119,7 @@ window.__ModuleLoader__.load({
|
|
|
119
119
|
'actions.save': 'Save Configuration',
|
|
120
120
|
'actions.saving': 'Saving…',
|
|
121
121
|
'actions.saved': 'Configuration saved successfully!',
|
|
122
|
+
'actions.saved_memory_only': 'Saved to session memory only (not persisted to disk)',
|
|
122
123
|
'actions.retry': 'Reload Configuration',
|
|
123
124
|
'badge.disabled': 'MoA Disabled',
|
|
124
125
|
'config.enabled_label': 'Enable MoA (slash command and turn routing)',
|
|
@@ -297,6 +298,7 @@ window.__ModuleLoader__.load({
|
|
|
297
298
|
'actions.save': '保存配置',
|
|
298
299
|
'actions.saving': '正在保存…',
|
|
299
300
|
'actions.saved': '配置已成功保存!',
|
|
301
|
+
'actions.saved_memory_only': '已保存到会话内存(未持久化到磁盘)',
|
|
300
302
|
'actions.retry': '重新加载配置',
|
|
301
303
|
'badge.disabled': 'MoA 已禁用',
|
|
302
304
|
'config.enabled_label': '启用 MoA(斜杠命令与轮次拦截路由)',
|
|
@@ -2566,7 +2568,11 @@ window.__ModuleLoader__.load({
|
|
|
2566
2568
|
.then((r) => r.json())
|
|
2567
2569
|
.then((res) => {
|
|
2568
2570
|
if (res && res.ok) {
|
|
2569
|
-
|
|
2571
|
+
if (res.persisted === false) {
|
|
2572
|
+
setSaveStatus(t('actions.saved_memory_only'))
|
|
2573
|
+
} else {
|
|
2574
|
+
setSaveStatus(t('actions.saved'))
|
|
2575
|
+
}
|
|
2570
2576
|
if (Array.isArray(res.presets)) setPresets(res.presets)
|
|
2571
2577
|
if (res.defaultPreset) setDefaultPreset(res.defaultPreset)
|
|
2572
2578
|
setTimeout(() => setSaveStatus(''), 3000)
|
package/lib/file-workspace.js
CHANGED
|
@@ -302,6 +302,7 @@ export async function createPrePromotionCheckpoint(baseDir, candidateTarget, opt
|
|
|
302
302
|
headers: { 'Content-Type': 'application/json' },
|
|
303
303
|
body: JSON.stringify({ label, cwd: baseDir }),
|
|
304
304
|
signal: AbortSignal.timeout(options.timeoutMs ?? 1500),
|
|
305
|
+
redirect: 'error',
|
|
305
306
|
})
|
|
306
307
|
if (res.ok) {
|
|
307
308
|
const data = await res.json()
|
|
@@ -563,15 +564,15 @@ export function computeLineDiff(oldText = '', newText = '') {
|
|
|
563
564
|
const subOld = oldLines.slice(start, oldEnd + 1)
|
|
564
565
|
const subNew = newLines.slice(start, newEnd + 1)
|
|
565
566
|
const subN = subOld.length, subM = subNew.length
|
|
566
|
-
|
|
567
|
-
|
|
568
|
-
|
|
569
|
-
else if (subM === 0)
|
|
570
|
-
|
|
571
|
-
|
|
572
|
-
|
|
573
|
-
|
|
574
|
-
)
|
|
567
|
+
let middleItems
|
|
568
|
+
if (subN === 0) {
|
|
569
|
+
middleItems = subNew.map((line, j) => ({ type: 'added', line, newNum: start + j + 1 }))
|
|
570
|
+
} else if (subM === 0) {
|
|
571
|
+
middleItems = subOld.map((line, i) => ({ type: 'removed', line, oldNum: start + i + 1 }))
|
|
572
|
+
} else if (subN > 2000 || subM > 2000) {
|
|
573
|
+
middleItems = new Array(subN + subM)
|
|
574
|
+
for (let i = 0; i < subN; i++) middleItems[i] = { type: 'removed', line: subOld[i], oldNum: start + i + 1 }
|
|
575
|
+
for (let j = 0; j < subM; j++) middleItems[subN + j] = { type: 'added', line: subNew[j], newNum: start + j + 1 }
|
|
575
576
|
} else {
|
|
576
577
|
const dp = Array.from({ length: subN + 1 }, () => new Uint16Array(subM + 1))
|
|
577
578
|
for (let i = 0; i < subN; i++) {
|
|
@@ -587,8 +588,7 @@ export function computeLineDiff(oldText = '', newText = '') {
|
|
|
587
588
|
stack.push({ type: 'removed', line: subOld[--i], oldNum: start + i + 1 })
|
|
588
589
|
}
|
|
589
590
|
}
|
|
590
|
-
middleItems
|
|
591
|
+
middleItems = stack.reverse()
|
|
591
592
|
}
|
|
592
|
-
|
|
593
|
-
return [...prefixItems, ...middleItems, ...suffixItems]
|
|
593
|
+
return prefixItems.concat(middleItems, suffixItems)
|
|
594
594
|
}
|
package/lib/index.js
CHANGED
|
@@ -95,10 +95,21 @@ export function apply(ctx, config) {
|
|
|
95
95
|
|
|
96
96
|
const saveConfig = async (newFields) => {
|
|
97
97
|
const plain = plainConfig(newFields)
|
|
98
|
+
let persisted = false
|
|
98
99
|
if (settingsService && typeof settingsService.replace === 'function') {
|
|
99
100
|
await settingsService.replace(NS, plain)
|
|
101
|
+
persisted = true
|
|
100
102
|
} else if (settingsService && typeof settingsService.update === 'function') {
|
|
101
103
|
await settingsService.update(NS, plain)
|
|
104
|
+
persisted = true
|
|
105
|
+
}
|
|
106
|
+
if (config && typeof config === 'object') {
|
|
107
|
+
for (const [k, v] of Object.entries(plain)) {
|
|
108
|
+
if (config[k] && typeof config[k] === 'object' && typeof config[k].get === 'function') continue
|
|
109
|
+
const desc = Object.getOwnPropertyDescriptor(config, k)
|
|
110
|
+
if (desc && desc.get && !desc.set) continue
|
|
111
|
+
bestEffort('update in-memory config key', () => { config[k] = v }, ctx.logger)
|
|
112
|
+
}
|
|
102
113
|
}
|
|
103
114
|
let liveCandidate = ctx.fiber?.config
|
|
104
115
|
if (!liveCandidate && ('config' in ctx)) {
|
|
@@ -110,7 +121,7 @@ export function apply(ctx, config) {
|
|
|
110
121
|
baseAtOverride[key] = JSON.stringify(currentLive[key])
|
|
111
122
|
}
|
|
112
123
|
localOverrides = { ...(localOverrides || {}), ...plain }
|
|
113
|
-
return live()
|
|
124
|
+
return { ok: true, persisted, ...live() }
|
|
114
125
|
}
|
|
115
126
|
|
|
116
127
|
const callLlm = createLlmCaller(ctx)
|
package/lib/moa-budget.js
CHANGED
|
@@ -66,6 +66,35 @@ export function estimateRound2Cost(candidates = [], prices = {}, expectedOutputT
|
|
|
66
66
|
return r2Total
|
|
67
67
|
}
|
|
68
68
|
|
|
69
|
+
/**
|
|
70
|
+
* Estimates upfront token cost for multi-judge consensus panel (#158).
|
|
71
|
+
*/
|
|
72
|
+
export function estimatePanelCost(judges = [], numCandidates = 1, prices = {}, maxTokens = 2048, promptTokens = 500) {
|
|
73
|
+
if (!Array.isArray(judges) || judges.length === 0 || numCandidates <= 1) return 0
|
|
74
|
+
let totalCost = 0
|
|
75
|
+
const inputTokens = promptTokens + (maxTokens * numCandidates) + 150
|
|
76
|
+
const outputTokens = Math.min(maxTokens, 1024)
|
|
77
|
+
for (const j of judges) {
|
|
78
|
+
if (j?.provider && j?.model) {
|
|
79
|
+
const cost = estimateTokenCost(j, { inputTokens, outputTokens }, prices)?.costUsd || 0
|
|
80
|
+
totalCost += cost
|
|
81
|
+
}
|
|
82
|
+
}
|
|
83
|
+
return totalCost
|
|
84
|
+
}
|
|
85
|
+
|
|
86
|
+
/**
|
|
87
|
+
* Calculates accurate synthesis prompt tokens accounting for full user prompt,
|
|
88
|
+
* judge criteria, and reference candidate texts (#158).
|
|
89
|
+
*/
|
|
90
|
+
export function calcSynthesisPromptTokens(userPrompt = '', referenceOutputs = [], judgeCriteria = '', overhead = 2600) {
|
|
91
|
+
const promptChars = (typeof userPrompt === 'string' ? userPrompt.length : 0) +
|
|
92
|
+
(typeof judgeCriteria === 'string' ? judgeCriteria.length : 0) +
|
|
93
|
+
(Array.isArray(referenceOutputs) ? referenceOutputs.reduce((acc, r) => acc + (r.text?.length || 0), 0) : 0) +
|
|
94
|
+
overhead
|
|
95
|
+
return Math.max(50, Math.ceil(promptChars / 4))
|
|
96
|
+
}
|
|
97
|
+
|
|
69
98
|
/**
|
|
70
99
|
* Phase-by-phase runtime budget check (#158).
|
|
71
100
|
* Checks if current spent + next phase estimate would exceed budget.
|
|
@@ -108,6 +137,7 @@ export function applyBudgetGuardrails({
|
|
|
108
137
|
promptLength = 1000,
|
|
109
138
|
projectContext = '',
|
|
110
139
|
messages = [],
|
|
140
|
+
judgeCriteria = '',
|
|
111
141
|
maxTokens = 2048,
|
|
112
142
|
peerCritiqueEnabled = false,
|
|
113
143
|
aggregator = null,
|
|
@@ -122,8 +152,11 @@ export function applyBudgetGuardrails({
|
|
|
122
152
|
}
|
|
123
153
|
}
|
|
124
154
|
|
|
125
|
-
// Calculate full input tokens from userPrompt length + messages context + system prompt overhead
|
|
155
|
+
// Calculate full input tokens from userPrompt length + judgeCriteria + messages context + system prompt overhead
|
|
126
156
|
let contextChars = Number(promptLength) || 1000
|
|
157
|
+
if (typeof judgeCriteria === 'string' && judgeCriteria.length > 0 && !String(promptLength).includes(String(judgeCriteria.length))) {
|
|
158
|
+
contextChars += judgeCriteria.length
|
|
159
|
+
}
|
|
127
160
|
if (typeof projectContext === 'string' && projectContext.length > 0 && !String(promptLength).includes(String(projectContext.length))) {
|
|
128
161
|
contextChars += projectContext.length
|
|
129
162
|
}
|
|
@@ -245,8 +278,11 @@ export function createBudgetAbortPayload({
|
|
|
245
278
|
isFastMode = false,
|
|
246
279
|
collectedCtx = {},
|
|
247
280
|
startTime = Date.now(),
|
|
281
|
+
multiJudgeResult = null,
|
|
248
282
|
}) {
|
|
249
283
|
const aggLabel = (primaryJudge?.provider && primaryJudge?.model) ? `${primaryJudge.provider}:${primaryJudge.model}` : "none";
|
|
284
|
+
const refTokens = referenceOutputs.reduce((acc, r) => acc + (r.usage?.totalTokens || 0), 0);
|
|
285
|
+
const panelTokens = multiJudgeResult?.totalUsage?.totalTokens || 0;
|
|
250
286
|
return {
|
|
251
287
|
kind: "failure",
|
|
252
288
|
content: `🛑 Budget Guardrail Abort: ${reason}`,
|
|
@@ -259,11 +295,13 @@ export function createBudgetAbortPayload({
|
|
|
259
295
|
winnerModel: "none",
|
|
260
296
|
promotedFiles: [],
|
|
261
297
|
usage: {
|
|
262
|
-
totalTokens:
|
|
298
|
+
totalTokens: refTokens + panelTokens,
|
|
263
299
|
totalCostUsd: Number(currentSpentUsd.toFixed(5)),
|
|
264
300
|
candidates: referenceOutputs.map((r) => ({ label: r.label, usage: r.usage, costUsd: r.costUsd })),
|
|
265
301
|
aggregator: { inputTokens: 0, outputTokens: 0, totalTokens: 0, costUsd: 0 },
|
|
302
|
+
...(multiJudgeResult ? { multiJudge: multiJudgeResult.totalUsage || { totalTokens: panelTokens } } : {}),
|
|
266
303
|
},
|
|
304
|
+
multiJudgeResult: multiJudgeResult || undefined,
|
|
267
305
|
skippedFiles: collectedCtx?.skippedFiles || 0,
|
|
268
306
|
skippedList: collectedCtx?.skippedList || [],
|
|
269
307
|
durationMs: Date.now() - startTime,
|
package/lib/moa-llm.js
CHANGED
|
@@ -26,7 +26,7 @@ function combineSignals(signal, timeoutSignal) {
|
|
|
26
26
|
*/
|
|
27
27
|
|
|
28
28
|
export function createLlmCaller(ctx) {
|
|
29
|
-
return async ({ provider, model, messages, temperature = 0.6, maxTokens =
|
|
29
|
+
return async ({ provider, model, messages, temperature = 0.6, maxTokens = 2048, timeoutMs, onStreamDelta, signal, stop, reasoningEffort }) => {
|
|
30
30
|
if (!ctx.llm) {
|
|
31
31
|
throw new Error('ctx.llm is not available in cordis context')
|
|
32
32
|
}
|
package/lib/moa-multi-judge.js
CHANGED
|
@@ -5,6 +5,7 @@
|
|
|
5
5
|
|
|
6
6
|
import { slotLabel } from './moa-prompts.js'
|
|
7
7
|
import { estimateTokenCost } from './pricing.js'
|
|
8
|
+
import { estimatePanelCost, estimateDownstreamCost, checkPhaseBudget, calcSynthesisPromptTokens } from './moa-budget.js'
|
|
8
9
|
|
|
9
10
|
/**
|
|
10
11
|
* Extracts code fences and block-level assets from markdown text.
|
|
@@ -160,6 +161,7 @@ export async function executeMultiJudgePanel({
|
|
|
160
161
|
onProgress,
|
|
161
162
|
prices = {},
|
|
162
163
|
blind = false,
|
|
164
|
+
maxTokens = 2048,
|
|
163
165
|
}) {
|
|
164
166
|
const activeCandidates = (candidates || []).filter((c) => c && c.ok)
|
|
165
167
|
if (activeCandidates.length === 0) {
|
|
@@ -240,6 +242,7 @@ CONSENSUS_REASONING: [one concise sentence explaining your pick]
|
|
|
240
242
|
{ role: 'system', content: 'You are an objective AI code evaluation judge.' },
|
|
241
243
|
{ role: 'user', content: judgePrompt },
|
|
242
244
|
],
|
|
245
|
+
maxTokens,
|
|
243
246
|
signal: abortCtrl.signal,
|
|
244
247
|
})
|
|
245
248
|
|
|
@@ -392,3 +395,64 @@ CONSENSUS_REASONING: [one concise sentence explaining your pick]
|
|
|
392
395
|
consensusReport,
|
|
393
396
|
}
|
|
394
397
|
}
|
|
398
|
+
|
|
399
|
+
|
|
400
|
+
/**
|
|
401
|
+
* Runs the Multi-Judge consensus phase with budget pre-checking (#158, #168).
|
|
402
|
+
*/
|
|
403
|
+
export async function runMultiJudgePhase({
|
|
404
|
+
preset = {},
|
|
405
|
+
successfulRefs = [],
|
|
406
|
+
primaryJudge,
|
|
407
|
+
userPrompt = '',
|
|
408
|
+
judgeCriteria = '',
|
|
409
|
+
prices = {},
|
|
410
|
+
maxTokens = 2048,
|
|
411
|
+
callLlm,
|
|
412
|
+
signal,
|
|
413
|
+
onProgress,
|
|
414
|
+
currentSpent = 0,
|
|
415
|
+
}) {
|
|
416
|
+
if (!preset?.multi_judge_enabled || successfulRefs.length <= 1) {
|
|
417
|
+
return { skipped: true, result: null }
|
|
418
|
+
}
|
|
419
|
+
const panelJudges = (preset.judge_models && preset.judge_models.length > 0) ? preset.judge_models : [primaryJudge]
|
|
420
|
+
const panelMaxTokens = Math.min(typeof preset.max_tokens === 'number' && preset.max_tokens > 0 ? preset.max_tokens : 2048, 2048)
|
|
421
|
+
|
|
422
|
+
if (preset?.budget_guard_enabled && preset?.max_budget_usd > 0) {
|
|
423
|
+
const promptTokens = calcSynthesisPromptTokens(userPrompt, successfulRefs, judgeCriteria)
|
|
424
|
+
const panelEst = estimatePanelCost(panelJudges, successfulRefs.length, prices, panelMaxTokens, promptTokens)
|
|
425
|
+
const downEst = estimateDownstreamCost(primaryJudge, [], successfulRefs.length, prices, maxTokens, promptTokens)
|
|
426
|
+
const phaseCheck = checkPhaseBudget({
|
|
427
|
+
phaseName: 'Multi-Judge Panel',
|
|
428
|
+
currentSpentUsd: currentSpent,
|
|
429
|
+
nextPhaseCostUsd: panelEst + downEst,
|
|
430
|
+
maxBudgetUsd: preset.max_budget_usd,
|
|
431
|
+
action: preset.budget_action || 'trim',
|
|
432
|
+
enabled: true,
|
|
433
|
+
})
|
|
434
|
+
if (!phaseCheck.allowed) {
|
|
435
|
+
if (preset.budget_action === 'abort') {
|
|
436
|
+
return { abort: true, reason: phaseCheck.reason }
|
|
437
|
+
}
|
|
438
|
+
if (typeof onProgress === 'function') {
|
|
439
|
+
onProgress(`✂️ *Skipping Multi-Judge Panel to stay within budget ($${preset.max_budget_usd.toFixed(4)})*\n`)
|
|
440
|
+
}
|
|
441
|
+
return { skipped: true, result: null }
|
|
442
|
+
}
|
|
443
|
+
}
|
|
444
|
+
|
|
445
|
+
const result = await executeMultiJudgePanel({
|
|
446
|
+
judges: panelJudges,
|
|
447
|
+
candidates: successfulRefs,
|
|
448
|
+
userPrompt,
|
|
449
|
+
callLlm,
|
|
450
|
+
strategy: preset.judge_voting_strategy || 'majority',
|
|
451
|
+
prices: prices || {},
|
|
452
|
+
blind: Boolean(preset.blind_evaluation),
|
|
453
|
+
maxTokens: panelMaxTokens,
|
|
454
|
+
signal,
|
|
455
|
+
onProgress,
|
|
456
|
+
})
|
|
457
|
+
return { skipped: false, result }
|
|
458
|
+
}
|
package/lib/moa-runner.js
CHANGED
|
@@ -8,8 +8,8 @@ import { slotLabel, isBroadPromptRequiringQuestions, buildQuestionSynthesisPromp
|
|
|
8
8
|
import { parseWinnerIndex, parseRecommendedAssembler } from './moa-parser.js'
|
|
9
9
|
import { REFERENCE_SYSTEM_PROMPT, callWithTransientRetry, runReferencesParallel } from './moa-candidates.js'
|
|
10
10
|
import { executeTestGateForCandidates } from './moa-test-gate.js'
|
|
11
|
-
import { executeMultiJudgePanel, buildCompositeBlockDirectives } from './moa-multi-judge.js'
|
|
12
|
-
import { applyBudgetGuardrails, checkPhaseBudget, estimateRound2Cost, estimateDownstreamCost, createBudgetAbortPayload } from './moa-budget.js'
|
|
11
|
+
import { executeMultiJudgePanel, buildCompositeBlockDirectives, runMultiJudgePhase } from './moa-multi-judge.js'
|
|
12
|
+
import { applyBudgetGuardrails, checkPhaseBudget, estimateRound2Cost, estimateDownstreamCost, calcSynthesisPromptTokens, createBudgetAbortPayload } from './moa-budget.js'
|
|
13
13
|
import { extractPriorTurnBaseline, pruneMultiTurnMessages } from './moa-context.js'
|
|
14
14
|
import { generateMoABenchmarkReport } from './moa-report.js'
|
|
15
15
|
// Re-exports for consumers & backward compatibility
|
|
@@ -27,13 +27,8 @@ export async function runMoAPipeline({
|
|
|
27
27
|
messages = [],
|
|
28
28
|
preset,
|
|
29
29
|
callLlm,
|
|
30
|
-
cwd = process.cwd(),
|
|
31
|
-
|
|
32
|
-
onProgress = null,
|
|
33
|
-
historyFilePath = null,
|
|
34
|
-
liveCanvas = null,
|
|
35
|
-
onStreamDelta = null,
|
|
36
|
-
signal = null,
|
|
30
|
+
cwd = process.cwd(), prices = {}, onProgress = null,
|
|
31
|
+
historyFilePath = null, liveCanvas = null, onStreamDelta = null, signal = null,
|
|
37
32
|
testGateEnabled = null, testCommand = null, execFn = null,
|
|
38
33
|
webServerPort = null, timeMachineEngine = null, timeMachineUrl = null,
|
|
39
34
|
force = isTestEnvironment() ? true : false, checkpoint = true, checkpointFn = null, runId: customRunId = null,
|
|
@@ -85,7 +80,8 @@ export async function runMoAPipeline({
|
|
|
85
80
|
maxBudgetUsd: preset?.max_budget_usd,
|
|
86
81
|
action: preset?.budget_action || 'trim',
|
|
87
82
|
prices,
|
|
88
|
-
promptLength: (userPrompt?.length || 1000) + (projectContext ? projectContext.length : 0),
|
|
83
|
+
promptLength: (userPrompt?.length || 1000) + (judgeCriteria?.length || 0) + (projectContext ? projectContext.length : 0),
|
|
84
|
+
judgeCriteria,
|
|
89
85
|
projectContext,
|
|
90
86
|
messages,
|
|
91
87
|
maxTokens,
|
|
@@ -316,11 +312,7 @@ export async function runMoAPipeline({
|
|
|
316
312
|
const currentSpent = referenceOutputs.reduce((acc, r) => acc + (r.costUsd || 0), 0)
|
|
317
313
|
const r2Next = estimateRound2Cost(successfulRefs, prices, maxTokens)
|
|
318
314
|
const downNext = estimateDownstreamCost(primaryJudge, preset?.judge_models, successfulRefs.length, prices, maxTokens)
|
|
319
|
-
const phaseCheck = checkPhaseBudget({
|
|
320
|
-
phaseName: 'Consilium Round 2', currentSpentUsd: currentSpent,
|
|
321
|
-
nextPhaseCostUsd: r2Next + downNext, maxBudgetUsd: preset.max_budget_usd,
|
|
322
|
-
action: preset.budget_action || 'trim', enabled: true,
|
|
323
|
-
})
|
|
315
|
+
const phaseCheck = checkPhaseBudget({ phaseName: 'Consilium Round 2', currentSpentUsd: currentSpent, nextPhaseCostUsd: r2Next + downNext, maxBudgetUsd: preset.max_budget_usd, action: preset.budget_action || 'trim', enabled: true })
|
|
324
316
|
if (!phaseCheck.allowed) {
|
|
325
317
|
if (phaseCheck.action === 'abort') {
|
|
326
318
|
await cleanMoaWorkspaces(cwd, { runId })
|
|
@@ -393,19 +385,15 @@ export async function runMoAPipeline({
|
|
|
393
385
|
return { error: new Error('Turn aborted') }
|
|
394
386
|
}
|
|
395
387
|
|
|
396
|
-
|
|
397
|
-
|
|
398
|
-
|
|
399
|
-
|
|
400
|
-
|
|
401
|
-
|
|
402
|
-
|
|
403
|
-
prices: prices || {},
|
|
404
|
-
blind: Boolean(preset.blind_evaluation),
|
|
405
|
-
signal,
|
|
406
|
-
onProgress,
|
|
407
|
-
})
|
|
388
|
+
const currentRefSpent = referenceOutputs.reduce((acc, r) => acc + (r.costUsd || 0), 0)
|
|
389
|
+
const panelPhase = await runMultiJudgePhase({
|
|
390
|
+
preset, successfulRefs, primaryJudge, userPrompt, judgeCriteria, prices, maxTokens, callLlm, signal, onProgress, currentSpent: currentRefSpent,
|
|
391
|
+
})
|
|
392
|
+
if (panelPhase.abort) {
|
|
393
|
+
await cleanMoaWorkspaces(cwd, { runId })
|
|
394
|
+
return createBudgetAbortPayload({ reason: panelPhase.reason, primaryJudge, referenceOutputs, currentSpentUsd: currentRefSpent, preset, isRefinement, isFastMode, collectedCtx, startTime })
|
|
408
395
|
}
|
|
396
|
+
multiJudgeResult = panelPhase.result
|
|
409
397
|
|
|
410
398
|
// 7d. Feature 3: Composite Block Merge Directive
|
|
411
399
|
const compositeMergeDirective = (preset?.composite_merge_enabled && successfulRefs.length > 1)
|
|
@@ -413,17 +401,14 @@ export async function runMoAPipeline({
|
|
|
413
401
|
: null
|
|
414
402
|
|
|
415
403
|
// 8. Synthesis phase via primary judge or fallback chain
|
|
404
|
+
const synthPromptTokens = calcSynthesisPromptTokens(userPrompt, successfulRefs, judgeCriteria)
|
|
416
405
|
if (preset?.budget_guard_enabled && preset?.max_budget_usd > 0 && !isFastMode) {
|
|
417
406
|
const currentSpent = referenceOutputs.reduce((acc, r) => acc + (r.costUsd || 0), 0) + (multiJudgeResult?.totalCostUsd || 0)
|
|
418
|
-
const downNext = estimateDownstreamCost(primaryJudge, [], successfulRefs.length, prices, maxTokens)
|
|
419
|
-
const phaseCheck = checkPhaseBudget({
|
|
420
|
-
|
|
421
|
-
nextPhaseCostUsd: downNext, maxBudgetUsd: preset.max_budget_usd,
|
|
422
|
-
action: preset.budget_action || 'trim', enabled: true,
|
|
423
|
-
})
|
|
424
|
-
if (!phaseCheck.allowed) {
|
|
407
|
+
const downNext = estimateDownstreamCost(primaryJudge, [], successfulRefs.length, prices, maxTokens, synthPromptTokens)
|
|
408
|
+
const phaseCheck = checkPhaseBudget({ phaseName: 'Judge Synthesis', currentSpentUsd: currentSpent, nextPhaseCostUsd: downNext, maxBudgetUsd: preset.max_budget_usd, action: preset.budget_action || 'trim', enabled: true })
|
|
409
|
+
if (!phaseCheck.allowed && fallbackJudges.length === 0) {
|
|
425
410
|
await cleanMoaWorkspaces(cwd, { runId })
|
|
426
|
-
return createBudgetAbortPayload({ reason: phaseCheck.reason, primaryJudge, referenceOutputs, currentSpentUsd: currentSpent, preset, isRefinement, isFastMode, collectedCtx, startTime })
|
|
411
|
+
return createBudgetAbortPayload({ reason: phaseCheck.reason, primaryJudge, referenceOutputs, currentSpentUsd: currentSpent, preset, isRefinement, isFastMode, collectedCtx, startTime, multiJudgeResult })
|
|
427
412
|
}
|
|
428
413
|
}
|
|
429
414
|
const synthesisPrompt = buildSynthesisPrompt(userPrompt, referenceOutputs, judgeCriteria, {
|
|
@@ -443,6 +428,18 @@ export async function runMoAPipeline({
|
|
|
443
428
|
const currentJudge = judgesChain[jIdx]
|
|
444
429
|
const currentLabel = slotLabel(currentJudge)
|
|
445
430
|
|
|
431
|
+
if (preset?.budget_guard_enabled && preset?.max_budget_usd > 0 && !isFastMode) {
|
|
432
|
+
const currentSpent = referenceOutputs.reduce((acc, r) => acc + (r.costUsd || 0), 0) + (multiJudgeResult?.totalCostUsd || 0)
|
|
433
|
+
const judgeCost = estimateDownstreamCost(currentJudge, [], successfulRefs.length, prices, maxTokens, synthPromptTokens)
|
|
434
|
+
const phaseCheck = checkPhaseBudget({ phaseName: `Judge Synthesis (${currentLabel})`, currentSpentUsd: currentSpent, nextPhaseCostUsd: judgeCost, maxBudgetUsd: preset.max_budget_usd, action: preset.budget_action || 'trim', enabled: true })
|
|
435
|
+
if (!phaseCheck.allowed) {
|
|
436
|
+
lastJudgeError = new Error(phaseCheck.reason)
|
|
437
|
+
logger.warn(`[dsh-moa] Judge ${currentLabel} budget check failed: ${phaseCheck.reason}`)
|
|
438
|
+
if (typeof onProgress === 'function') onProgress(`⚠️ *Judge ${currentLabel} skipped: exceeds budget limit.*\n`)
|
|
439
|
+
continue
|
|
440
|
+
}
|
|
441
|
+
}
|
|
442
|
+
|
|
446
443
|
if (typeof onProgress === 'function') {
|
|
447
444
|
const modeTitle = isCuratorSynthesis ? 'Curator' : 'Judge'
|
|
448
445
|
const fallbackBadge = jIdx > 0 ? ` (Fallback #${jIdx})` : ''
|
|
@@ -487,6 +484,11 @@ export async function runMoAPipeline({
|
|
|
487
484
|
}
|
|
488
485
|
|
|
489
486
|
if (!judgeSuccess) {
|
|
487
|
+
if (preset?.budget_guard_enabled && preset?.max_budget_usd > 0 && !isFastMode && (lastJudgeError?.message?.includes('exceeds max budget') || lastJudgeError?.message?.includes('Budget Guardrail') || preset?.budget_action === 'abort')) {
|
|
488
|
+
await cleanMoaWorkspaces(cwd, { runId })
|
|
489
|
+
const currentSpent = referenceOutputs.reduce((acc, r) => acc + (r.costUsd || 0), 0) + (multiJudgeResult?.totalCostUsd || 0)
|
|
490
|
+
return createBudgetAbortPayload({ reason: lastJudgeError?.message || 'Budget exceeded before judge synthesis', primaryJudge, referenceOutputs, currentSpentUsd: currentSpent, preset, isRefinement, isFastMode, collectedCtx, startTime, multiJudgeResult })
|
|
491
|
+
}
|
|
490
492
|
const firstErr = lastJudgeError ? (lastJudgeError.message || String(lastJudgeError)) : 'unknown error'
|
|
491
493
|
synthesizedText = `⚠️ *[Aggregator error: ${firstErr}. Fallback candidate outputs:]*\n\n` +
|
|
492
494
|
successfulRefs.map((r, i) => `### Candidate ${i + 1} (${r.label})\n${r.text}`).join('\n\n')
|
package/lib/moa-test-gate.js
CHANGED
|
@@ -43,8 +43,12 @@ export function resolveCandidateDir(baseDir, candidateIndex, options = {}) {
|
|
|
43
43
|
}
|
|
44
44
|
}
|
|
45
45
|
|
|
46
|
+
const COMPUTED_SHELL_EVASION = /(?:base64\s+(?:-d|--decode|-D)|\beval\s+["\']?\$|\bxxd\s+-r|\bprintf\s+[^|\n]+\|\s*(?:ba|z|da)?sh|\bopenssl\s+enc\s+-d)/i
|
|
47
|
+
|
|
48
|
+
const UNIX_ROOT_PREFIX = /^\/(?:home|root|etc|var|tmp|opt|mnt|srv|media|proc|sys|dev|usr|bin|lib|lib64|run\/(?:user|systemd|lock|credentials|secrets|udev|initramfs|mount|shm|dbus|sshd|motd)|sbin|boot|Users\/(?:[a-zA-Z0-9_\-.]+)|Applications|Library|System|Volumes|private)(?:\/|$)/
|
|
49
|
+
|
|
46
50
|
/**
|
|
47
|
-
* Validates that an executed script or command does not reference files outside allowed roots (#161).
|
|
51
|
+
* Validates that an executed script or command does not reference files outside allowed roots (#161, #225).
|
|
48
52
|
*/
|
|
49
53
|
function scanPathTokensForLeak(text, allowedRoots, executionDir) {
|
|
50
54
|
if (!text || typeof text !== 'string') return null
|
|
@@ -56,7 +60,10 @@ function scanPathTokensForLeak(text, allowedRoots, executionDir) {
|
|
|
56
60
|
if (/^\/(usr\/(bin|lib|include)|bin|lib|lib64|etc|dev)(\/|$)/.test(raw) && !/\.(txt|json|env|npmrc|key|ssh|md)/i.test(raw)) {
|
|
57
61
|
continue
|
|
58
62
|
}
|
|
59
|
-
|
|
63
|
+
const isAbs = path.isAbsolute(raw)
|
|
64
|
+
const isUp = raw.startsWith('..')
|
|
65
|
+
if (!isAbs && !isUp) continue
|
|
66
|
+
if (isAbs && process.platform !== 'win32' && !UNIX_ROOT_PREFIX.test(raw)) continue
|
|
60
67
|
const resolved = path.resolve(executionDir, raw)
|
|
61
68
|
const allowed = allowedRoots.some((r) => r && (resolved === r || resolved.startsWith(r + path.sep)))
|
|
62
69
|
if (!allowed) {
|
|
@@ -173,10 +180,10 @@ export async function runCandidateTestGate({
|
|
|
173
180
|
}
|
|
174
181
|
|
|
175
182
|
// 2. Scan script files referenced by command or present in candidate workspace (#161)
|
|
176
|
-
const candFiles = fsSync.existsSync(executionDir) ? fsSync.readdirSync(executionDir) : []
|
|
183
|
+
const candFiles = fsSync.existsSync(executionDir) ? fsSync.readdirSync(executionDir, { recursive: true }) : []
|
|
177
184
|
const scriptTokens = new Set([
|
|
178
185
|
...cmdTokens.map((t) => t.replace(/^["']|["']$/g, '')),
|
|
179
|
-
...candFiles,
|
|
186
|
+
...candFiles.map((f) => String(f)),
|
|
180
187
|
])
|
|
181
188
|
|
|
182
189
|
for (const scriptName of scriptTokens) {
|
|
@@ -184,6 +191,18 @@ export async function runCandidateTestGate({
|
|
|
184
191
|
if (fsSync.existsSync(scriptPath) && !fsSync.statSync(scriptPath).isDirectory()) {
|
|
185
192
|
try {
|
|
186
193
|
const scriptText = fsSync.readFileSync(scriptPath, 'utf8')
|
|
194
|
+
if (COMPUTED_SHELL_EVASION.test(scriptText)) {
|
|
195
|
+
const durationMs = Date.now() - startTime
|
|
196
|
+
return {
|
|
197
|
+
candidateIndex,
|
|
198
|
+
passed: false,
|
|
199
|
+
exitCode: 1,
|
|
200
|
+
durationMs,
|
|
201
|
+
command: resolvedCmd,
|
|
202
|
+
output: 'Access denied: test script contains obfuscated or computed command execution',
|
|
203
|
+
summary: `FAIL (code 1, ${durationMs}ms)`,
|
|
204
|
+
}
|
|
205
|
+
}
|
|
187
206
|
const leaked = scanPathTokensForLeak(scriptText, allowedRoots, executionDir)
|
|
188
207
|
if (leaked) {
|
|
189
208
|
const durationMs = Date.now() - startTime
|
|
@@ -197,7 +216,9 @@ export async function runCandidateTestGate({
|
|
|
197
216
|
summary: `FAIL (code 1, ${durationMs}ms)`,
|
|
198
217
|
}
|
|
199
218
|
}
|
|
200
|
-
} catch {
|
|
219
|
+
} catch (scanErr) {
|
|
220
|
+
logger.debug('[dsh-moa:testGate] candidate script read error:', scanErr?.message || scanErr)
|
|
221
|
+
}
|
|
201
222
|
}
|
|
202
223
|
}
|
|
203
224
|
|
|
@@ -207,6 +228,18 @@ export async function runCandidateTestGate({
|
|
|
207
228
|
try {
|
|
208
229
|
const pkgObj = JSON.parse(fsSync.readFileSync(candPkgFile, 'utf8'))
|
|
209
230
|
const scriptsContent = Object.values(pkgObj.scripts || {}).join('\n')
|
|
231
|
+
if (COMPUTED_SHELL_EVASION.test(scriptsContent)) {
|
|
232
|
+
const durationMs = Date.now() - startTime
|
|
233
|
+
return {
|
|
234
|
+
candidateIndex,
|
|
235
|
+
passed: false,
|
|
236
|
+
exitCode: 1,
|
|
237
|
+
durationMs,
|
|
238
|
+
command: resolvedCmd,
|
|
239
|
+
output: 'Access denied: package.json test script contains obfuscated or computed command execution',
|
|
240
|
+
summary: `FAIL (code 1, ${durationMs}ms)`,
|
|
241
|
+
}
|
|
242
|
+
}
|
|
210
243
|
const leaked = scanPathTokensForLeak(scriptsContent, allowedRoots, executionDir)
|
|
211
244
|
if (leaked) {
|
|
212
245
|
const durationMs = Date.now() - startTime
|
|
@@ -220,7 +253,9 @@ export async function runCandidateTestGate({
|
|
|
220
253
|
summary: `FAIL (code 1, ${durationMs}ms)`,
|
|
221
254
|
}
|
|
222
255
|
}
|
|
223
|
-
} catch {
|
|
256
|
+
} catch (pkgErr) {
|
|
257
|
+
logger.debug('[dsh-moa:testGate] candidate package.json read error:', pkgErr?.message || pkgErr)
|
|
258
|
+
}
|
|
224
259
|
}
|
|
225
260
|
|
|
226
261
|
// Setup isolated HOME & node roots
|
|
@@ -240,6 +275,8 @@ export async function runCandidateTestGate({
|
|
|
240
275
|
...(candidateDir && candidateDir !== executionDir ? [candidateDir] : []),
|
|
241
276
|
nodeRoot,
|
|
242
277
|
nodeBinDir,
|
|
278
|
+
...(process.env.NVM_DIR ? [process.env.NVM_DIR] : []),
|
|
279
|
+
...(nodeRoot.includes("/.nvm/") ? [path.resolve(nodeRoot, "../..")] : []),
|
|
243
280
|
...(process.platform === 'win32'
|
|
244
281
|
? [process.env.SYSTEMROOT || 'C:\\Windows']
|
|
245
282
|
: ['/usr', '/lib', '/lib64', '/etc']),
|
|
@@ -251,19 +288,40 @@ export async function runCandidateTestGate({
|
|
|
251
288
|
const cp = require('node:child_process');
|
|
252
289
|
const path = require('node:path');
|
|
253
290
|
const allowed = ${JSON.stringify(allowedReads)};
|
|
254
|
-
|
|
291
|
+
const COMPUTED_SHELL_EVASION = /(?:base64\\s+(?:-d|--decode|-D)|\\beval\\s+["\']?\\$|\\bxxd\\s+-r|\\bprintf\\s+[^|\\n]+\\|\\s*(?:ba|z|da)?sh|\\bopenssl\\s+enc\\s+-d)/i;
|
|
292
|
+
const UNIX_ROOT_PREFIX = /^\\/(?:home|root|etc|var|tmp|opt|mnt|srv|media|proc|sys|dev|usr|bin|lib|lib64|run\\/(?:user|systemd|lock|credentials|secrets|udev|initramfs|mount|shm|dbus|sshd|motd)|sbin|boot|Users\\/(?:[a-zA-Z0-9_\\-.]+)|Applications|Library|System|Volumes|private)(?:\\/|$)/;
|
|
293
|
+
function checkSinglePath(p) {
|
|
255
294
|
if (!p || typeof p !== 'string') return true;
|
|
256
295
|
if (/^\\/(usr\\/(bin|lib|include)|bin|lib|lib64|etc|dev)(\\/|$)/.test(p) && !/\\.(txt|json|env|npmrc|key|ssh|md)/i.test(p)) return true;
|
|
296
|
+
const isAbs = path.isAbsolute(p);
|
|
297
|
+
const isUp = p.startsWith('..');
|
|
298
|
+
if (!isAbs && !isUp) return true;
|
|
299
|
+
if (isAbs && process.platform !== 'win32' && !UNIX_ROOT_PREFIX.test(p)) return true;
|
|
257
300
|
const res = path.resolve(process.cwd(), p);
|
|
258
301
|
return allowed.some(root => root && (res === root || res.startsWith(root + path.sep)));
|
|
259
302
|
}
|
|
303
|
+
function scanStringForPaths(str) {
|
|
304
|
+
if (!str || typeof str !== 'string') return true;
|
|
305
|
+
if (!checkSinglePath(str)) return false;
|
|
306
|
+
const matches = str.match(/(?:[a-zA-Z]:[\\\\/][^\\s"'<>]+|(?<![\\w.@/\\\\])\\/(?:[a-zA-Z0-9_\\-.]+\\/)*[a-zA-Z0-9_\\-.]+|(?<![\\w.@/\\\\])\\.\\.[\\\\/][^\\s"'<>]*)/g) || [];
|
|
307
|
+
for (const m of matches) {
|
|
308
|
+
if (!checkSinglePath(m)) return false;
|
|
309
|
+
}
|
|
310
|
+
return true;
|
|
311
|
+
}
|
|
260
312
|
function guardArgs(cmd, args) {
|
|
261
|
-
|
|
262
|
-
|
|
313
|
+
const full = [cmd, ...(Array.isArray(args) ? args : [])].join(' ');
|
|
314
|
+
if (COMPUTED_SHELL_EVASION.test(full)) {
|
|
315
|
+
throw new Error('ERR_ACCESS_DENIED: child process computed execution pattern blocked: ' + full);
|
|
316
|
+
}
|
|
317
|
+
if (cmd && !scanStringForPaths(String(cmd))) {
|
|
318
|
+
throw new Error('ERR_ACCESS_DENIED: child process command outside sandbox: ' + cmd);
|
|
263
319
|
}
|
|
264
320
|
if (Array.isArray(args)) {
|
|
265
321
|
for (const a of args) {
|
|
266
|
-
if (!
|
|
322
|
+
if (a && !scanStringForPaths(String(a))) {
|
|
323
|
+
throw new Error('ERR_ACCESS_DENIED: child process argument outside sandbox: ' + a);
|
|
324
|
+
}
|
|
267
325
|
}
|
|
268
326
|
}
|
|
269
327
|
}
|
|
@@ -297,6 +355,20 @@ cp.execSync = function(cmd, opts) {
|
|
|
297
355
|
guardArgs(cmd, [cmd]);
|
|
298
356
|
return origExecSync.apply(this, arguments);
|
|
299
357
|
};
|
|
358
|
+
const fsMod = require('node:fs');
|
|
359
|
+
function guardFs(p, isWrite) {
|
|
360
|
+
if (!p || typeof p !== 'string') return;
|
|
361
|
+
const isAbs = path.isAbsolute(p);
|
|
362
|
+
const isUp = p.startsWith('..');
|
|
363
|
+
if (!isAbs && !isUp) return;
|
|
364
|
+
const resolved = path.resolve(process.cwd(), p);
|
|
365
|
+
const checkRoots = isWrite ? [process.cwd(), process.env.HOME].filter(Boolean) : allowed;
|
|
366
|
+
const ok = checkRoots.some(r => r && (resolved === r || resolved.startsWith(r + path.sep)));
|
|
367
|
+
if (!ok) throw new Error('ERR_ACCESS_DENIED: fs access outside sandbox: ' + p);
|
|
368
|
+
}
|
|
369
|
+
const origRF = fsMod.readFileSync; fsMod.readFileSync = function(p) { guardFs(p, false); return origRF.apply(this, arguments); };
|
|
370
|
+
const origWF = fsMod.writeFileSync; fsMod.writeFileSync = function(p) { guardFs(p, true); return origWF.apply(this, arguments); };
|
|
371
|
+
const origAF = fsMod.appendFileSync; fsMod.appendFileSync = function(p) { guardFs(p, true); return origAF.apply(this, arguments); };
|
|
300
372
|
`
|
|
301
373
|
fsSync.writeFileSync(fenceFile, fenceCode, 'utf8')
|
|
302
374
|
|
package/lib/routes.js
CHANGED
|
@@ -37,10 +37,7 @@ ctx.effect(() => {
|
|
|
37
37
|
kind: 'exact',
|
|
38
38
|
path: '/dsh-moa/status',
|
|
39
39
|
handler: (req, res) => {
|
|
40
|
-
if (req.method !== 'GET') {
|
|
41
|
-
writeJson(res, 405, { ok: false, error: 'GET required' })
|
|
42
|
-
return
|
|
43
|
-
}
|
|
40
|
+
if (req.method !== 'GET') return writeJson(res, 405, { ok: false, error: 'GET required' })
|
|
44
41
|
try {
|
|
45
42
|
const cfg = live()
|
|
46
43
|
writeJson(res, 200, {
|
|
@@ -49,9 +46,7 @@ ctx.effect(() => {
|
|
|
49
46
|
defaultPreset: cfg.default_preset || 'default',
|
|
50
47
|
presetsCount: (cfg.presets || []).length,
|
|
51
48
|
})
|
|
52
|
-
} catch (err) {
|
|
53
|
-
writeJson(res, 500, { ok: false, error: err?.message || String(err) })
|
|
54
|
-
}
|
|
49
|
+
} catch (err) { writeJson(res, 500, { ok: false, error: err?.message || String(err) }) }
|
|
55
50
|
},
|
|
56
51
|
})
|
|
57
52
|
}, 'dsh-moa: status route')
|
|
@@ -62,10 +57,7 @@ ctx.effect(() => {
|
|
|
62
57
|
kind: 'exact',
|
|
63
58
|
path: '/dsh-moa/models',
|
|
64
59
|
handler: async (req, res) => {
|
|
65
|
-
if (req.method !== 'GET') {
|
|
66
|
-
writeJson(res, 405, { ok: false, error: 'GET required' })
|
|
67
|
-
return
|
|
68
|
-
}
|
|
60
|
+
if (req.method !== 'GET') return writeJson(res, 405, { ok: false, error: 'GET required' })
|
|
69
61
|
if (_modelsCache.models.length > 0 && Date.now() - _modelsCache.timestamp < MODELS_CACHE_TTL) {
|
|
70
62
|
writeJson(res, 200, { ok: true, models: _modelsCache.models, cached: true })
|
|
71
63
|
return
|
|
@@ -201,21 +193,21 @@ ctx.effect(() => {
|
|
|
201
193
|
}
|
|
202
194
|
|
|
203
195
|
// Persist via saveConfig (in-memory and DSH profile settings)
|
|
196
|
+
let persistResult
|
|
204
197
|
if (typeof saveConfig === 'function') {
|
|
205
|
-
try { await saveConfig(validated) }
|
|
198
|
+
try { persistResult = await saveConfig(validated) }
|
|
206
199
|
catch (perr) { return writeJson(res, 500, { ok: false, error: 'Settings persistence failed: ' + (perr?.message || String(perr)) }) }
|
|
207
200
|
}
|
|
208
201
|
|
|
209
202
|
const unwrapped = typeof plainConfig === 'function' ? plainConfig(validated) : validated
|
|
210
203
|
writeJson(res, 200, {
|
|
211
204
|
ok: true,
|
|
205
|
+
persisted: persistResult?.persisted ?? false,
|
|
212
206
|
enabled: unwrapped.enabled !== false,
|
|
213
207
|
defaultPreset: unwrapped.default_preset,
|
|
214
208
|
presets: unwrapped.presets,
|
|
215
209
|
})
|
|
216
|
-
} catch (err) {
|
|
217
|
-
writeJson(res, 400, { ok: false, error: err?.message || String(err) })
|
|
218
|
-
}
|
|
210
|
+
} catch (err) { writeJson(res, 400, { ok: false, error: err?.message || String(err) }) }
|
|
219
211
|
return
|
|
220
212
|
}
|
|
221
213
|
|
|
@@ -230,19 +222,14 @@ ctx.effect(() => {
|
|
|
230
222
|
kind: 'exact',
|
|
231
223
|
path: '/dsh-moa/history',
|
|
232
224
|
handler: async (req, res) => {
|
|
233
|
-
if (req.method !== 'GET') {
|
|
234
|
-
writeJson(res, 405, { ok: false, error: 'GET required' })
|
|
235
|
-
return
|
|
236
|
-
}
|
|
225
|
+
if (req.method !== 'GET') return writeJson(res, 405, { ok: false, error: 'GET required' })
|
|
237
226
|
try {
|
|
238
227
|
const u = new URL(req.url, 'http://localhost')
|
|
239
228
|
const limit = Math.max(1, Math.min(100, parseInt(u.searchParams.get('limit') || '20', 10) || 20))
|
|
240
229
|
const offset = Math.max(0, parseInt(u.searchParams.get('offset') || '0', 10) || 0)
|
|
241
230
|
const data = getMoaHistory(limit, offset)
|
|
242
231
|
writeJson(res, 200, { ok: true, ...data })
|
|
243
|
-
} catch (err) {
|
|
244
|
-
writeJson(res, 500, { ok: false, error: err?.message || String(err) })
|
|
245
|
-
}
|
|
232
|
+
} catch (err) { writeJson(res, 500, { ok: false, error: err?.message || String(err) }) }
|
|
246
233
|
},
|
|
247
234
|
})
|
|
248
235
|
}, 'dsh-moa: history route')
|
|
@@ -253,10 +240,7 @@ ctx.effect(() => {
|
|
|
253
240
|
kind: 'exact',
|
|
254
241
|
path: '/dsh-moa/leaderboard',
|
|
255
242
|
handler: async (req, res) => {
|
|
256
|
-
if (req.method !== 'GET') {
|
|
257
|
-
writeJson(res, 405, { ok: false, error: 'GET required' })
|
|
258
|
-
return
|
|
259
|
-
}
|
|
243
|
+
if (req.method !== 'GET') return writeJson(res, 405, { ok: false, error: 'GET required' })
|
|
260
244
|
try {
|
|
261
245
|
const url = new URL(req.url || '', 'http://127.0.0.1')
|
|
262
246
|
const preset = url.searchParams.get('preset') || null
|
|
@@ -274,9 +258,7 @@ ctx.effect(() => {
|
|
|
274
258
|
|
|
275
259
|
const data = getMoaLeaderboard(undefined, { preset })
|
|
276
260
|
writeJson(res, 200, { ok: true, ...data })
|
|
277
|
-
} catch (err) {
|
|
278
|
-
writeJson(res, 500, { ok: false, error: err?.message || String(err) })
|
|
279
|
-
}
|
|
261
|
+
} catch (err) { writeJson(res, 500, { ok: false, error: err?.message || String(err) }) }
|
|
280
262
|
},
|
|
281
263
|
})
|
|
282
264
|
}, 'dsh-moa: leaderboard route')
|
|
@@ -287,10 +269,7 @@ ctx.effect(() => {
|
|
|
287
269
|
kind: 'exact',
|
|
288
270
|
path: '/dsh-moa/promote',
|
|
289
271
|
handler: async (req, res) => {
|
|
290
|
-
if (req.method !== 'POST') {
|
|
291
|
-
writeJson(res, 405, { ok: false, error: 'POST required' })
|
|
292
|
-
return
|
|
293
|
-
}
|
|
272
|
+
if (req.method !== 'POST') return writeJson(res, 405, { ok: false, error: 'POST required' })
|
|
294
273
|
if (!isSafeWriteRequest(req)) {
|
|
295
274
|
writeJson(res, 403, { ok: false, error: 'Forbidden: cross-site request rejected' })
|
|
296
275
|
return
|
|
@@ -303,26 +282,35 @@ ctx.effect(() => {
|
|
|
303
282
|
writeJson(res, 400, { ok: false, error: 'Valid candidateIndex required (>= 1)' })
|
|
304
283
|
return
|
|
305
284
|
}
|
|
306
|
-
const cwd = body?.cwd || process.cwd()
|
|
307
285
|
const runId = body?.runId
|
|
286
|
+
let recordedRun = null
|
|
308
287
|
if (runId) {
|
|
309
|
-
|
|
310
|
-
if (recordedRun
|
|
311
|
-
writeJson(res,
|
|
288
|
+
recordedRun = getMoaRunById(runId)
|
|
289
|
+
if (!recordedRun) {
|
|
290
|
+
writeJson(res, 404, { ok: false, error: `Run '${runId}' not found` })
|
|
291
|
+
return
|
|
292
|
+
}
|
|
293
|
+
} else {
|
|
294
|
+
recordedRun = getMoaHistory(1, 0)?.runs?.[0] || null
|
|
295
|
+
if (!recordedRun) {
|
|
296
|
+
writeJson(res, 400, { ok: false, error: 'No recorded MoA run found' })
|
|
312
297
|
return
|
|
313
298
|
}
|
|
314
299
|
}
|
|
315
|
-
|
|
316
|
-
|
|
300
|
+
if (body?.cwd && path.resolve(body.cwd) !== path.resolve(recordedRun.cwd)) {
|
|
301
|
+
writeJson(res, 403, { ok: false, error: 'Forbidden: cwd does not match recorded run cwd' })
|
|
302
|
+
return
|
|
303
|
+
}
|
|
304
|
+
const effectiveCwd = recordedRun.cwd
|
|
305
|
+
const promotedFiles = await promoteCandidateWorkspace(effectiveCwd, candidateIndex, {
|
|
306
|
+
runId: recordedRun.id,
|
|
317
307
|
keepMoa: true,
|
|
318
308
|
port: ctx.webServer?.port,
|
|
319
309
|
timeMachineEngine: timeMachineEngine || ctx.reflect?.get?.('timeMachineEngine', false),
|
|
320
310
|
force: Boolean(body?.force),
|
|
321
311
|
})
|
|
322
312
|
writeJson(res, 200, { ok: true, candidateIndex, promotedFiles, checkpoint: promotedFiles?.checkpoint || null })
|
|
323
|
-
} catch (err) {
|
|
324
|
-
writeJson(res, 500, { ok: false, error: err?.message || String(err) })
|
|
325
|
-
}
|
|
313
|
+
} catch (err) { writeJson(res, 500, { ok: false, error: err?.message || String(err) }) }
|
|
326
314
|
},
|
|
327
315
|
})
|
|
328
316
|
}, 'dsh-moa: promote candidate route')
|
|
@@ -333,10 +321,7 @@ ctx.effect(() => {
|
|
|
333
321
|
kind: 'exact',
|
|
334
322
|
path: '/dsh-moa/diff',
|
|
335
323
|
handler: async (req, res) => {
|
|
336
|
-
if (req.method !== 'GET') {
|
|
337
|
-
writeJson(res, 405, { ok: false, error: 'GET required' })
|
|
338
|
-
return
|
|
339
|
-
}
|
|
324
|
+
if (req.method !== 'GET') return writeJson(res, 405, { ok: false, error: 'GET required' })
|
|
340
325
|
try {
|
|
341
326
|
const u = new URL(req.url, 'http://localhost')
|
|
342
327
|
const rawRunId = u.searchParams.get('runId')
|
|
@@ -349,14 +334,18 @@ ctx.effect(() => {
|
|
|
349
334
|
return
|
|
350
335
|
}
|
|
351
336
|
} else {
|
|
352
|
-
run = getMoaHistory(1, 0)
|
|
337
|
+
run = getMoaHistory(1, 0)?.runs?.[0] || null
|
|
338
|
+
if (!run) {
|
|
339
|
+
writeJson(res, 404, { ok: false, error: 'No recorded MoA run found' })
|
|
340
|
+
return
|
|
341
|
+
}
|
|
353
342
|
}
|
|
354
|
-
if (
|
|
343
|
+
if (queryCwd && path.resolve(queryCwd) !== path.resolve(run.cwd)) {
|
|
355
344
|
writeJson(res, 403, { ok: false, error: 'Forbidden: cwd does not match recorded run cwd' })
|
|
356
345
|
return
|
|
357
346
|
}
|
|
358
|
-
const runId = run
|
|
359
|
-
const cwd = run
|
|
347
|
+
const runId = run.id
|
|
348
|
+
const cwd = run.cwd
|
|
360
349
|
const sanitizeTarget = (val, def) => {
|
|
361
350
|
const clean = String(val || '').trim().replace(/[^a-zA-Z0-9_-]/g, '')
|
|
362
351
|
return clean || def
|
|
@@ -393,24 +382,18 @@ ctx.effect(() => {
|
|
|
393
382
|
toSize: toItem?.size || 0,
|
|
394
383
|
diff,
|
|
395
384
|
})
|
|
396
|
-
} catch (err) {
|
|
397
|
-
writeJson(res, 500, { ok: false, error: err?.message || String(err) })
|
|
398
|
-
}
|
|
385
|
+
} catch (err) { writeJson(res, 500, { ok: false, error: err?.message || String(err) }) }
|
|
399
386
|
},
|
|
400
387
|
})
|
|
401
388
|
}, 'dsh-moa: candidate diff route')
|
|
402
389
|
|
|
403
|
-
|
|
404
390
|
// Route: /dsh-moa/history/export
|
|
405
391
|
ctx.effect(() => {
|
|
406
392
|
return ctx.webServer.register({
|
|
407
393
|
kind: 'exact',
|
|
408
394
|
path: '/dsh-moa/history/export',
|
|
409
395
|
handler: async (req, res) => {
|
|
410
|
-
if (req.method !== 'GET') {
|
|
411
|
-
writeJson(res, 405, { ok: false, error: 'GET required' })
|
|
412
|
-
return
|
|
413
|
-
}
|
|
396
|
+
if (req.method !== 'GET') return writeJson(res, 405, { ok: false, error: 'GET required' })
|
|
414
397
|
try {
|
|
415
398
|
const url = new URL(req.url || '', 'http://127.0.0.1')
|
|
416
399
|
const preset = url.searchParams.get('preset') || null
|
|
@@ -423,9 +406,7 @@ ctx.effect(() => {
|
|
|
423
406
|
'Content-Disposition': `attachment; filename="moa-history.${isCsv ? 'csv' : 'json'}"`,
|
|
424
407
|
})
|
|
425
408
|
res.end(data)
|
|
426
|
-
} catch (err) {
|
|
427
|
-
writeJson(res, 500, { ok: false, error: err?.message || String(err) })
|
|
428
|
-
}
|
|
409
|
+
} catch (err) { writeJson(res, 500, { ok: false, error: err?.message || String(err) }) }
|
|
429
410
|
},
|
|
430
411
|
})
|
|
431
412
|
}, 'dsh-moa: history export route')
|
|
@@ -436,10 +417,7 @@ ctx.effect(() => {
|
|
|
436
417
|
kind: 'prefix',
|
|
437
418
|
path: '/dsh-moa/runs/',
|
|
438
419
|
handler: (req, res) => {
|
|
439
|
-
if (req.method !== 'GET') {
|
|
440
|
-
writeJson(res, 405, { ok: false, error: 'GET required' })
|
|
441
|
-
return
|
|
442
|
-
}
|
|
420
|
+
if (req.method !== 'GET') return writeJson(res, 405, { ok: false, error: 'GET required' })
|
|
443
421
|
const urlPath = req.url.split('?')[0]
|
|
444
422
|
const parts = urlPath.split('/').filter(Boolean)
|
|
445
423
|
const runId = parts[2] || ''
|
|
@@ -484,10 +462,7 @@ ctx.effect(() => {
|
|
|
484
462
|
kind: 'exact',
|
|
485
463
|
path: '/dsh-moa/route-preset',
|
|
486
464
|
handler: async (req, res) => {
|
|
487
|
-
if (req.method !== 'POST') {
|
|
488
|
-
writeJson(res, 405, { ok: false, error: 'POST required' })
|
|
489
|
-
return
|
|
490
|
-
}
|
|
465
|
+
if (req.method !== 'POST') return writeJson(res, 405, { ok: false, error: 'POST required' })
|
|
491
466
|
if (!isSafeWriteRequest(req)) {
|
|
492
467
|
writeJson(res, 403, { ok: false, error: 'Forbidden: cross-site request rejected' })
|
|
493
468
|
return
|
|
@@ -513,9 +488,7 @@ ctx.effect(() => {
|
|
|
513
488
|
enabled: cfg.enabled !== false,
|
|
514
489
|
})
|
|
515
490
|
writeJson(res, 200, { ok: true, ...result })
|
|
516
|
-
} catch (err) {
|
|
517
|
-
writeJson(res, 500, { ok: false, error: err?.message || String(err) })
|
|
518
|
-
}
|
|
491
|
+
} catch (err) { writeJson(res, 500, { ok: false, error: err?.message || String(err) }) }
|
|
519
492
|
},
|
|
520
493
|
})
|
|
521
494
|
}, 'dsh-moa: route preset endpoint')
|
|
@@ -529,10 +502,7 @@ ctx.effect(() => {
|
|
|
529
502
|
kind: 'exact',
|
|
530
503
|
path: '/dsh-moa/run',
|
|
531
504
|
handler: async (req, res) => {
|
|
532
|
-
if (req.method !== 'POST') {
|
|
533
|
-
writeJson(res, 405, { ok: false, error: 'POST required' })
|
|
534
|
-
return
|
|
535
|
-
}
|
|
505
|
+
if (req.method !== 'POST') return writeJson(res, 405, { ok: false, error: 'POST required' })
|
|
536
506
|
if (!isSafeWriteRequest(req)) {
|
|
537
507
|
writeJson(res, 403, { ok: false, error: 'Forbidden: cross-site request rejected' })
|
|
538
508
|
return
|
|
@@ -584,9 +554,7 @@ ctx.effect(() => {
|
|
|
584
554
|
}
|
|
585
555
|
|
|
586
556
|
writeJson(res, 200, { ok: true, ...result })
|
|
587
|
-
} catch (err) {
|
|
588
|
-
writeJson(res, 500, { ok: false, error: err?.message || String(err) })
|
|
589
|
-
}
|
|
557
|
+
} catch (err) { writeJson(res, 500, { ok: false, error: err?.message || String(err) }) }
|
|
590
558
|
},
|
|
591
559
|
})
|
|
592
560
|
}, 'dsh-moa: run pipeline route')
|