dsh-omni-router 2.1.0 → 2.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +2 -1
- package/README.zh-CN.md +2 -1
- package/package.json +2 -2
- package/src/capability-auditor.mjs +90 -0
- package/src/capability-performance.mjs +103 -0
- package/src/capability-provisioner.mjs +223 -0
- package/src/capability-quality.mjs +90 -0
- package/src/capability-solver.mjs +88 -0
- package/src/omni-router.mjs +132 -0
- package/test/capability-auditor.test.mjs +43 -0
- package/test/capability-performance.test.mjs +64 -0
- package/test/capability-provisioner.test.mjs +96 -0
- package/test/capability-quality.test.mjs +62 -0
- package/test/capability-solver.test.mjs +51 -0
package/README.md
CHANGED
|
@@ -73,6 +73,7 @@
|
|
|
73
73
|
- **Stage 4(部分)**:`compileDagToWorkflow` 将 Mission DAG 编译为 DSH-native workflow outline。
|
|
74
74
|
- **P1/P3 能力**:Real capability sandbox(已接入 `omni_mission_run`,按 role + capability 生成 toolFilter)、Capability Resolver v2(risk/reliability 综合打分)、OmniBench v2 runner(manifest schema + prepare + prompt generation + `--exec` 本地执行/结果采集 + DAG-to-plan compile)。
|
|
75
75
|
- **2.1 收敛增强**:Mission Resume(`omni_mission_resume` 跨 session 继续 DAG,`runDagLoop` 支持 `onProgress` 周期保存)、Harness evidence extraction(`extractHarnessEvidence` 从 commands/tests/toolCalls/EVIDENCE_JSON 提取机器可验证证据)、context token budget(`maxContextTokens` 真正限制检索输出)、semantic failure retrieval + evidence-backed skill distillation、OmniBench v2 real runner。
|
|
76
|
+
- **2.2 Capability Auto-Provisioning(优化7)**:Capability Auditor(baseline/task-time audit)、Gap Detection、Plugin/Skill Quality Scoring、Minimal Capability Set Solver、Discovery Adapters(marketplace/hub/static registry)、Trusted Auto Provisioning(recommend/auto-trusted/manual)、Post-install probe、Install rollback transaction、Capability performance learning / redundancy detection。
|
|
76
77
|
- **TaskDecision**:`createTaskDecision` 生成唯一决策对象,Policy/Runtime 统一消费,避免重复 classify 分叉。
|
|
77
78
|
- **Evidence Protocol**:`src/evidence.mjs` 用结构化 command/file/test/finding 证据判定 PASS/FAIL;`isQaPass` 已支持从 QA 输出解析 JSON evidence,不再只信文本。
|
|
78
79
|
- **Task Compiler**:`compileTask` 生成 objective / constraints / non-goals / acceptance / hidden assumptions / ambiguities / invariants / risk / artifacts;`compileTaskWithLLM` 可用 LLM 增强。
|
|
@@ -154,7 +155,7 @@ Then restart DSH and select **Omni Router** in a new session.
|
|
|
154
155
|
- `/omni direct` — enter direct mode.
|
|
155
156
|
- `/omni mode spec|react|balanced` — set thinking mode.
|
|
156
157
|
- `/omni reroute plan|direct` — adaptively reroute current task.
|
|
157
|
-
- Model tools: `omni_status`, `omni_plan`, `omni_direct`, `omni_mode`, `omni_reroute`, `omni_delegate`, `omni_memory`, `omni_benchmark`, `omni_mission_run`, `omni_mission_resume`, `omni_visual_check`.
|
|
158
|
+
- Model tools: `omni_status`, `omni_plan`, `omni_direct`, `omni_mode`, `omni_reroute`, `omni_delegate`, `omni_memory`, `omni_benchmark`, `omni_mission_run`, `omni_mission_resume`, `omni_capability_audit`, `omni_capability_provision`, `omni_capability_probe`, `omni_visual_check`.
|
|
158
159
|
|
|
159
160
|
## Configuration
|
|
160
161
|
|
package/README.zh-CN.md
CHANGED
|
@@ -73,6 +73,7 @@
|
|
|
73
73
|
- **Stage 4(部分)**:`compileDagToWorkflow` 将 Mission DAG 编译为 DSH-native workflow outline。
|
|
74
74
|
- **P1/P3 能力**:Real capability sandbox(已接入 `omni_mission_run`,按 role + capability 生成 toolFilter)、Capability Resolver v2(risk/reliability 综合打分)、OmniBench v2 runner(manifest schema + prepare + prompt generation + `--exec` 本地执行/结果采集 + DAG-to-plan compile)。
|
|
75
75
|
- **2.1 收敛增强**:Mission Resume(`omni_mission_resume` 跨 session 继续 DAG,`runDagLoop` 支持 `onProgress` 周期保存)、Harness evidence extraction(`extractHarnessEvidence` 从 commands/tests/toolCalls/EVIDENCE_JSON 提取机器可验证证据)、context token budget(`maxContextTokens` 真正限制检索输出)、semantic failure retrieval + evidence-backed skill distillation、OmniBench v2 real runner。
|
|
76
|
+
- **2.2 Capability Auto-Provisioning(优化7)**:Capability Auditor(baseline/task-time audit)、Gap Detection、Plugin/Skill Quality Scoring、Minimal Capability Set Solver、Discovery Adapters(marketplace/hub/static registry)、Trusted Auto Provisioning(recommend/auto-trusted/manual)、Post-install probe、Install rollback transaction、Capability performance learning / redundancy detection。
|
|
76
77
|
- **TaskDecision**:`createTaskDecision` 生成唯一决策对象,Policy/Runtime 统一消费,避免重复 classify 分叉。
|
|
77
78
|
- **Evidence Protocol**:`src/evidence.mjs` 用结构化 command/file/test/finding 证据判定 PASS/FAIL;`isQaPass` 已支持从 QA 输出解析 JSON evidence,不再只信文本。
|
|
78
79
|
- **Task Compiler**:`compileTask` 生成 objective / constraints / non-goals / acceptance / hidden assumptions / ambiguities / invariants / risk / artifacts;`compileTaskWithLLM` 可用 LLM 增强。
|
|
@@ -154,7 +155,7 @@ node scripts/install-preset.mjs
|
|
|
154
155
|
- `/omni direct` — 进入直接执行模式。
|
|
155
156
|
- `/omni mode spec|react|balanced` — 设置思维模式。
|
|
156
157
|
- `/omni reroute plan|direct` — 动态切换当前任务路由。
|
|
157
|
-
- 模型工具:`omni_status` / `omni_plan` / `omni_direct` / `omni_mode` / `omni_reroute` / `omni_delegate` / `omni_memory` / `omni_benchmark` / `omni_mission_run` / `omni_mission_resume` / `omni_visual_check`。
|
|
158
|
+
- 模型工具:`omni_status` / `omni_plan` / `omni_direct` / `omni_mode` / `omni_reroute` / `omni_delegate` / `omni_memory` / `omni_benchmark` / `omni_mission_run` / `omni_mission_resume` / `omni_capability_audit` / `omni_capability_provision` / `omni_capability_probe` / `omni_visual_check`。
|
|
158
159
|
|
|
159
160
|
## 配置
|
|
160
161
|
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "dsh-omni-router",
|
|
3
|
-
"version": "2.
|
|
3
|
+
"version": "2.2.0",
|
|
4
4
|
"description": "Omni - a reliability and orchestration control plane for DeepSeek Harness, designed to turn fast models into dependable long-horizon software-engineering agents.",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"main": "./lib/index.js",
|
|
@@ -41,7 +41,7 @@
|
|
|
41
41
|
}
|
|
42
42
|
},
|
|
43
43
|
"scripts": {
|
|
44
|
-
"test": "node test/omni-router.test.mjs && node test/agent-chain.test.mjs && node test/skill-suggest.test.mjs && node test/methodology.test.mjs && node test/compat.test.mjs && node test/project-brain.test.mjs && node test/mission-planner.test.mjs && node test/agent-runtime.test.mjs && node test/judge.test.mjs && node test/memory.test.mjs && node test/memory-v3.test.mjs && node test/memory-engine.test.mjs && node test/mission-resume.test.mjs && node test/engineering-benchmark.test.mjs && node test/benchmark-runner.test.mjs && node test/benchmark-results.test.mjs && node test/continuous-eval.test.mjs && node --no-warnings test/project-brain-v2.test.mjs && node --no-warnings test/project-brain-v3.test.mjs && node test/visual-qa.test.mjs && node test/evidence.test.mjs && node test/evidence-store.test.mjs && node test/evidence-engine.test.mjs && node test/task-decision.test.mjs && node test/task-compiler.test.mjs && node test/capability-brain.test.mjs && node test/capability-manifest.test.mjs && node test/capability-sandbox.test.mjs && node test/planner-dag.test.mjs && node test/dynamic-context.test.mjs && node test/mission-dag.test.mjs && node test/failure-taxonomy.test.mjs && node test/context-expansion.test.mjs && node test/hybrid-retrieval.test.mjs && node test/dependency-graph.test.mjs && node test/ast-provider.test.mjs && node test/experience-router.test.mjs && node test/benchmark-tasks.test.mjs && node test/bundle-installer.test.mjs && node test/omnibench-runner.test.mjs",
|
|
44
|
+
"test": "node test/omni-router.test.mjs && node test/agent-chain.test.mjs && node test/skill-suggest.test.mjs && node test/methodology.test.mjs && node test/compat.test.mjs && node test/project-brain.test.mjs && node test/mission-planner.test.mjs && node test/agent-runtime.test.mjs && node test/judge.test.mjs && node test/memory.test.mjs && node test/memory-v3.test.mjs && node test/memory-engine.test.mjs && node test/mission-resume.test.mjs && node test/engineering-benchmark.test.mjs && node test/benchmark-runner.test.mjs && node test/benchmark-results.test.mjs && node test/continuous-eval.test.mjs && node --no-warnings test/project-brain-v2.test.mjs && node --no-warnings test/project-brain-v3.test.mjs && node test/visual-qa.test.mjs && node test/evidence.test.mjs && node test/evidence-store.test.mjs && node test/evidence-engine.test.mjs && node test/task-decision.test.mjs && node test/task-compiler.test.mjs && node test/capability-brain.test.mjs && node test/capability-manifest.test.mjs && node test/capability-sandbox.test.mjs && node test/capability-auditor.test.mjs && node test/capability-quality.test.mjs && node test/capability-solver.test.mjs && node test/capability-provisioner.test.mjs && node test/capability-performance.test.mjs && node test/planner-dag.test.mjs && node test/dynamic-context.test.mjs && node test/mission-dag.test.mjs && node test/failure-taxonomy.test.mjs && node test/context-expansion.test.mjs && node test/hybrid-retrieval.test.mjs && node test/dependency-graph.test.mjs && node test/ast-provider.test.mjs && node test/experience-router.test.mjs && node test/benchmark-tasks.test.mjs && node test/bundle-installer.test.mjs && node test/omnibench-runner.test.mjs",
|
|
45
45
|
"benchmark": "node benchmark/run.mjs",
|
|
46
46
|
"benchmark:engineering": "node benchmark/engineering-run.mjs",
|
|
47
47
|
"benchmark:compare": "node benchmark/compare.mjs",
|
|
@@ -0,0 +1,90 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* Capability Auditor.
|
|
3
|
+
*
|
|
4
|
+
* Answers "what does the current Harness environment lack?" for both a
|
|
5
|
+
* baseline (first-run / after DSH update) and a specific task. It is the
|
|
6
|
+
* front door of the Capability Auto-Provisioning loop.
|
|
7
|
+
*/
|
|
8
|
+
|
|
9
|
+
export const OMNI_CODING_BASELINE = [
|
|
10
|
+
'repository.read',
|
|
11
|
+
'repository.search',
|
|
12
|
+
'source.write',
|
|
13
|
+
'shell',
|
|
14
|
+
'test.run',
|
|
15
|
+
'debugging',
|
|
16
|
+
'verification',
|
|
17
|
+
'code.review',
|
|
18
|
+
'git.workflow',
|
|
19
|
+
]
|
|
20
|
+
|
|
21
|
+
const SEVERITY = {
|
|
22
|
+
'repository.read': 'high',
|
|
23
|
+
'repository.search': 'high',
|
|
24
|
+
'source.write': 'high',
|
|
25
|
+
'shell': 'high',
|
|
26
|
+
'test.run': 'high',
|
|
27
|
+
'debugging': 'high',
|
|
28
|
+
'verification': 'high',
|
|
29
|
+
'code.review': 'medium',
|
|
30
|
+
'git.workflow': 'medium',
|
|
31
|
+
'browser.navigation': 'medium',
|
|
32
|
+
'browser.interaction': 'medium',
|
|
33
|
+
'browser.screenshot': 'medium',
|
|
34
|
+
'frontend.validation': 'medium',
|
|
35
|
+
'github.remote': 'medium',
|
|
36
|
+
'gitlab.remote': 'medium',
|
|
37
|
+
'db.inspection': 'low',
|
|
38
|
+
'container.run': 'low',
|
|
39
|
+
'security.review': 'medium',
|
|
40
|
+
'performance.analysis': 'low',
|
|
41
|
+
}
|
|
42
|
+
|
|
43
|
+
export function severityForCapability(capability) {
|
|
44
|
+
return SEVERITY[capability] || 'low'
|
|
45
|
+
}
|
|
46
|
+
|
|
47
|
+
export function auditCapabilities(brain, required = []) {
|
|
48
|
+
const availableSet = new Set()
|
|
49
|
+
for (const provider of brain?.capabilities || []) {
|
|
50
|
+
for (const cap of provider.capabilities || []) availableSet.add(cap)
|
|
51
|
+
}
|
|
52
|
+
const requiredList = [...new Set(required.map((c) => String(c).trim()).filter(Boolean))]
|
|
53
|
+
const available = requiredList.filter((c) => availableSet.has(c))
|
|
54
|
+
const missing = requiredList.filter((c) => !availableSet.has(c))
|
|
55
|
+
return {
|
|
56
|
+
required: requiredList,
|
|
57
|
+
available,
|
|
58
|
+
missing,
|
|
59
|
+
coverage: requiredList.length ? Math.round((available.length / requiredList.length) * 1000) / 1000 : 1,
|
|
60
|
+
gaps: missing.map((capability) => ({
|
|
61
|
+
capability,
|
|
62
|
+
severity: severityForCapability(capability),
|
|
63
|
+
})),
|
|
64
|
+
}
|
|
65
|
+
}
|
|
66
|
+
|
|
67
|
+
export function baselineAudit(brain, baseline = OMNI_CODING_BASELINE) {
|
|
68
|
+
return auditCapabilities(brain, baseline)
|
|
69
|
+
}
|
|
70
|
+
|
|
71
|
+
export function taskTimeAudit(brain, requirements = []) {
|
|
72
|
+
return auditCapabilities(brain, requirements)
|
|
73
|
+
}
|
|
74
|
+
|
|
75
|
+
export function formatCapabilityAudit(audit = {}) {
|
|
76
|
+
const lines = [
|
|
77
|
+
`Capability audit: ${audit.available?.length || 0}/${audit.required?.length || 0} available (coverage ${audit.coverage ?? 0})`,
|
|
78
|
+
]
|
|
79
|
+
if (audit.required?.length) lines.push(`Required: ${audit.required.join(', ')}`)
|
|
80
|
+
if (audit.available?.length) lines.push(`Available: ${audit.available.join(', ')}`)
|
|
81
|
+
if (audit.missing?.length) {
|
|
82
|
+
lines.push(`Missing (${audit.missing.length}):`)
|
|
83
|
+
for (const gap of audit.gaps || []) {
|
|
84
|
+
lines.push(`- ${gap.capability} [${gap.severity}]`)
|
|
85
|
+
}
|
|
86
|
+
} else {
|
|
87
|
+
lines.push('No missing capabilities.')
|
|
88
|
+
}
|
|
89
|
+
return lines.join('\n')
|
|
90
|
+
}
|
|
@@ -0,0 +1,103 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* Capability Performance Learning.
|
|
3
|
+
*
|
|
4
|
+
* Tracks whether a provisioned plugin/skill actually improves task success,
|
|
5
|
+
* reduces false completion, or only adds token/catalog overhead. This feeds
|
|
6
|
+
* demotion/removal decisions and benchmark plugin-contribution analysis.
|
|
7
|
+
*/
|
|
8
|
+
|
|
9
|
+
export function createPerformanceRegistry(initial = {}) {
|
|
10
|
+
return {
|
|
11
|
+
providers: { ...(initial.providers || {}) },
|
|
12
|
+
}
|
|
13
|
+
}
|
|
14
|
+
|
|
15
|
+
export function recordProvisionOutcome(registry, providerId, outcome = {}) {
|
|
16
|
+
const previous = registry.providers[providerId] || {
|
|
17
|
+
runs: 0,
|
|
18
|
+
successBefore: null,
|
|
19
|
+
successAfter: null,
|
|
20
|
+
falseCompletionBefore: null,
|
|
21
|
+
falseCompletionAfter: null,
|
|
22
|
+
tokensBefore: null,
|
|
23
|
+
tokensAfter: null,
|
|
24
|
+
toolErrorsBefore: null,
|
|
25
|
+
toolErrorsAfter: null,
|
|
26
|
+
}
|
|
27
|
+
const next = {
|
|
28
|
+
...previous,
|
|
29
|
+
runs: (previous.runs || 0) + 1,
|
|
30
|
+
successBefore: outcome.successBefore ?? previous.successBefore,
|
|
31
|
+
successAfter: outcome.successAfter ?? previous.successAfter,
|
|
32
|
+
falseCompletionBefore: outcome.falseCompletionBefore ?? previous.falseCompletionBefore,
|
|
33
|
+
falseCompletionAfter: outcome.falseCompletionAfter ?? previous.falseCompletionAfter,
|
|
34
|
+
tokensBefore: outcome.tokensBefore ?? previous.tokensBefore,
|
|
35
|
+
tokensAfter: outcome.tokensAfter ?? previous.tokensAfter,
|
|
36
|
+
toolErrorsBefore: outcome.toolErrorsBefore ?? previous.toolErrorsBefore,
|
|
37
|
+
toolErrorsAfter: outcome.toolErrorsAfter ?? previous.toolErrorsAfter,
|
|
38
|
+
lastUpdated: new Date().toISOString(),
|
|
39
|
+
}
|
|
40
|
+
return {
|
|
41
|
+
...registry,
|
|
42
|
+
providers: {
|
|
43
|
+
...registry.providers,
|
|
44
|
+
[providerId]: next,
|
|
45
|
+
},
|
|
46
|
+
}
|
|
47
|
+
}
|
|
48
|
+
|
|
49
|
+
export function evaluateProviderValue(record = {}) {
|
|
50
|
+
const hasBefore = record.successBefore !== null && record.successBefore !== undefined
|
|
51
|
+
const hasAfter = record.successAfter !== null && record.successAfter !== undefined
|
|
52
|
+
if (!hasBefore || !hasAfter) return { value: 0, label: 'unknown' }
|
|
53
|
+
|
|
54
|
+
const successDelta = record.successAfter - record.successBefore
|
|
55
|
+
const falseCompletionDelta = (record.falseCompletionBefore ?? 0) - (record.falseCompletionAfter ?? 0)
|
|
56
|
+
const tokenRatio = record.tokensBefore ? Math.max(0, ((record.tokensAfter ?? 0) - record.tokensBefore) / record.tokensBefore) : 0
|
|
57
|
+
const toolErrorDelta = (record.toolErrorsAfter ?? 0) - (record.toolErrorsBefore ?? 0)
|
|
58
|
+
|
|
59
|
+
const value = (
|
|
60
|
+
successDelta * 0.5 +
|
|
61
|
+
falseCompletionDelta * 0.3 -
|
|
62
|
+
tokenRatio * 0.1 -
|
|
63
|
+
Math.max(0, toolErrorDelta) * 0.1
|
|
64
|
+
)
|
|
65
|
+
const rounded = Math.round(value * 1000) / 1000
|
|
66
|
+
let label = 'neutral'
|
|
67
|
+
if (rounded >= 0.05) label = 'high'
|
|
68
|
+
else if (rounded > 0) label = 'medium'
|
|
69
|
+
else if (rounded < 0) label = 'negative'
|
|
70
|
+
return { value: rounded, label, successDelta, falseCompletionDelta, tokenRatio, toolErrorDelta }
|
|
71
|
+
}
|
|
72
|
+
|
|
73
|
+
export function recommendDemotion(registry, providerId, options = {}) {
|
|
74
|
+
const record = registry.providers[providerId]
|
|
75
|
+
if (!record) return { providerId, recommendation: 'unknown', reason: 'no performance record' }
|
|
76
|
+
const value = evaluateProviderValue(record)
|
|
77
|
+
const usageDays = Number(options.usageDays ?? 0)
|
|
78
|
+
const uniqueCapabilities = Number(options.uniqueCapabilities ?? 0)
|
|
79
|
+
const coveredBy = options.coveredBy || []
|
|
80
|
+
|
|
81
|
+
if (value.label === 'negative' && usageDays >= 30 && uniqueCapabilities === 0) {
|
|
82
|
+
return { providerId, recommendation: 'remove', reason: 'negative value, no unique capability, low usage', value }
|
|
83
|
+
}
|
|
84
|
+
if (value.label === 'negative') {
|
|
85
|
+
return { providerId, recommendation: 'demote', reason: 'negative measured value', value }
|
|
86
|
+
}
|
|
87
|
+
if (usageDays >= 90 && uniqueCapabilities === 0 && coveredBy.length > 0) {
|
|
88
|
+
return { providerId, recommendation: 'remove', reason: 'redundant and unused for 90+ days', value }
|
|
89
|
+
}
|
|
90
|
+
if (value.label === 'high' || value.label === 'medium') {
|
|
91
|
+
return { providerId, recommendation: 'keep', reason: 'positive measured value', value }
|
|
92
|
+
}
|
|
93
|
+
return { providerId, recommendation: 'watch', reason: 'insufficient signal', value }
|
|
94
|
+
}
|
|
95
|
+
|
|
96
|
+
export function formatPerformanceRegistry(registry = {}) {
|
|
97
|
+
const lines = []
|
|
98
|
+
for (const [id, record] of Object.entries(registry.providers || {})) {
|
|
99
|
+
const value = evaluateProviderValue(record)
|
|
100
|
+
lines.push(`- ${id}: ${value.label} (${value.value}) runs=${record.runs || 0}`)
|
|
101
|
+
}
|
|
102
|
+
return lines.length ? lines.join('\n') : '(empty performance registry)'
|
|
103
|
+
}
|
|
@@ -0,0 +1,223 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* Capability Provisioner.
|
|
3
|
+
*
|
|
4
|
+
* Discovers candidate plugins/skills/MCPs for missing capabilities, evaluates
|
|
5
|
+
* them, selects a minimal set, installs through DSH's existing mechanisms, and
|
|
6
|
+
* verifies/rolls back the result. Omni does not become a marketplace: it
|
|
7
|
+
* delegates discovery to installed marketplace/hub tools, community adapters,
|
|
8
|
+
* or explicit commands.
|
|
9
|
+
*/
|
|
10
|
+
|
|
11
|
+
import { scorePluginCandidate } from './capability-quality.mjs'
|
|
12
|
+
import { solveMinimalSet } from './capability-solver.mjs'
|
|
13
|
+
|
|
14
|
+
export const TRUST_MODES = ['recommend', 'auto-trusted', 'manual']
|
|
15
|
+
|
|
16
|
+
export function canAutoInstall(candidate = {}, mode = 'auto-trusted', trustedSources = []) {
|
|
17
|
+
if (mode === 'manual' || mode === 'recommend') return false
|
|
18
|
+
const source = candidate.source || candidate.package || ''
|
|
19
|
+
const trusted = candidate.verified === true || candidate.trustedSource === true || trustedSources.includes(source)
|
|
20
|
+
const risk = candidate.risk || 'low'
|
|
21
|
+
if (risk === 'high' || risk === 'critical') return false
|
|
22
|
+
return trusted
|
|
23
|
+
}
|
|
24
|
+
|
|
25
|
+
export function createProvisionTransaction({
|
|
26
|
+
package: pkg,
|
|
27
|
+
version,
|
|
28
|
+
source,
|
|
29
|
+
reason,
|
|
30
|
+
profile,
|
|
31
|
+
beforeProfile,
|
|
32
|
+
installCommand,
|
|
33
|
+
rollbackCommand,
|
|
34
|
+
} = {}) {
|
|
35
|
+
return {
|
|
36
|
+
id: `txn-${Date.now().toString(36)}-${Math.random().toString(36).slice(2, 6)}`,
|
|
37
|
+
package: pkg,
|
|
38
|
+
version,
|
|
39
|
+
source,
|
|
40
|
+
reason,
|
|
41
|
+
profile,
|
|
42
|
+
beforeProfile,
|
|
43
|
+
installCommand,
|
|
44
|
+
rollbackCommand,
|
|
45
|
+
installedAt: new Date().toISOString(),
|
|
46
|
+
status: 'pending',
|
|
47
|
+
}
|
|
48
|
+
}
|
|
49
|
+
|
|
50
|
+
export async function probeCapability(provider = {}, { tools = [], skills = [], probe } = {}) {
|
|
51
|
+
const checks = []
|
|
52
|
+
for (const tool of provider.expectedTools || []) {
|
|
53
|
+
const ok = tools.includes(tool)
|
|
54
|
+
checks.push({ type: 'tool', name: tool, ok })
|
|
55
|
+
}
|
|
56
|
+
for (const skill of provider.expectedSkills || []) {
|
|
57
|
+
const ok = skills.includes(skill)
|
|
58
|
+
checks.push({ type: 'skill', name: skill, ok })
|
|
59
|
+
}
|
|
60
|
+
if (typeof probe === 'function') {
|
|
61
|
+
try {
|
|
62
|
+
const ok = await probe(provider)
|
|
63
|
+
checks.push({ type: 'probe', name: provider.id || provider.package || 'probe', ok: !!ok })
|
|
64
|
+
} catch {
|
|
65
|
+
checks.push({ type: 'probe', name: provider.id || provider.package || 'probe', ok: false })
|
|
66
|
+
}
|
|
67
|
+
}
|
|
68
|
+
const ok = checks.length === 0 || checks.every((c) => c.ok)
|
|
69
|
+
return { ok, checks }
|
|
70
|
+
}
|
|
71
|
+
|
|
72
|
+
export async function rollbackProvision(txn = {}, execute) {
|
|
73
|
+
if (!txn?.rollbackCommand && typeof execute !== 'function') {
|
|
74
|
+
return { ok: false, reason: 'no rollback executor or command', txn }
|
|
75
|
+
}
|
|
76
|
+
try {
|
|
77
|
+
if (typeof execute === 'function') {
|
|
78
|
+
const result = await execute({ type: 'rollback', txn })
|
|
79
|
+
return { ok: result !== false, txn: { ...txn, status: result === false ? 'rollback_failed' : 'rolled_back' } }
|
|
80
|
+
}
|
|
81
|
+
// execute as command runner function (command, cwd?) — kept for adapters.
|
|
82
|
+
const result = await execute(txn.rollbackCommand, txn)
|
|
83
|
+
return { ok: result !== false, txn: { ...txn, status: result === false ? 'rollback_failed' : 'rolled_back' } }
|
|
84
|
+
} catch (error) {
|
|
85
|
+
return { ok: false, reason: error?.message || String(error), txn: { ...txn, status: 'rollback_failed' } }
|
|
86
|
+
}
|
|
87
|
+
}
|
|
88
|
+
|
|
89
|
+
export async function discoverCandidates(missing = [], adapters = [], options = {}) {
|
|
90
|
+
const seen = new Map()
|
|
91
|
+
for (const adapter of adapters) {
|
|
92
|
+
let candidates = []
|
|
93
|
+
try {
|
|
94
|
+
if (typeof adapter?.search === 'function') {
|
|
95
|
+
candidates = await adapter.search(missing, options) || []
|
|
96
|
+
} else if (typeof adapter === 'function') {
|
|
97
|
+
candidates = await adapter(missing, options) || []
|
|
98
|
+
}
|
|
99
|
+
} catch {
|
|
100
|
+
candidates = []
|
|
101
|
+
}
|
|
102
|
+
for (const candidate of candidates) {
|
|
103
|
+
const key = candidate.id || candidate.package || candidate.name
|
|
104
|
+
if (!key) continue
|
|
105
|
+
if (!seen.has(key)) seen.set(key, { ...candidate, _adapter: adapter.id || adapter.name || 'unknown' })
|
|
106
|
+
}
|
|
107
|
+
}
|
|
108
|
+
return [...seen.values()]
|
|
109
|
+
}
|
|
110
|
+
|
|
111
|
+
export function evaluateProvisionPlan(missing = [], candidates = [], brain = { capabilities: [] }, options = {}) {
|
|
112
|
+
const maxPlugins = Number(options.maxPlugins ?? 2)
|
|
113
|
+
const minScore = Number(options.minScore ?? 0.4)
|
|
114
|
+
const scored = candidates.map((c) => scorePluginCandidate(c, missing, brain, options))
|
|
115
|
+
const solution = solveMinimalSet(missing, candidates, { brain, maxPlugins, minScore })
|
|
116
|
+
return {
|
|
117
|
+
missing,
|
|
118
|
+
candidates: scored.sort((a, b) => b.score - a.score),
|
|
119
|
+
solution,
|
|
120
|
+
selected: solution.scored,
|
|
121
|
+
requiresApproval: solution.scored.filter((s) => !canAutoInstall(s.candidate, options.mode || 'auto-trusted', options.trustedSources || [])),
|
|
122
|
+
}
|
|
123
|
+
}
|
|
124
|
+
|
|
125
|
+
export async function provisionCapabilities(plan = {}, {
|
|
126
|
+
execute,
|
|
127
|
+
profile,
|
|
128
|
+
mode = 'auto-trusted',
|
|
129
|
+
trustedSources = [],
|
|
130
|
+
probeTools = [],
|
|
131
|
+
probeSkills = [],
|
|
132
|
+
probe,
|
|
133
|
+
} = {}) {
|
|
134
|
+
const results = []
|
|
135
|
+
for (const scored of plan.selected || []) {
|
|
136
|
+
const candidate = scored.candidate
|
|
137
|
+
const txn = createProvisionTransaction({
|
|
138
|
+
package: candidate.package || candidate.id,
|
|
139
|
+
version: candidate.version,
|
|
140
|
+
source: candidate.source,
|
|
141
|
+
reason: plan.missing?.join(', ') || '',
|
|
142
|
+
profile,
|
|
143
|
+
beforeProfile: plan.beforeProfile,
|
|
144
|
+
installCommand: candidate.installCommand,
|
|
145
|
+
rollbackCommand: candidate.rollbackCommand,
|
|
146
|
+
})
|
|
147
|
+
if (!canAutoInstall(candidate, mode, trustedSources)) {
|
|
148
|
+
results.push({ candidate, status: 'needs_approval', txn })
|
|
149
|
+
continue
|
|
150
|
+
}
|
|
151
|
+
try {
|
|
152
|
+
if (typeof execute === 'function') {
|
|
153
|
+
const installResult = await execute({ type: 'install', candidate, profile, txn })
|
|
154
|
+
if (installResult === false) {
|
|
155
|
+
results.push({ candidate, status: 'install_failed', txn: { ...txn, status: 'install_failed' } })
|
|
156
|
+
continue
|
|
157
|
+
}
|
|
158
|
+
} else if (candidate.installCommand) {
|
|
159
|
+
// No executor provided; command-based install is left to the caller.
|
|
160
|
+
results.push({ candidate, status: 'install_command_ready', txn })
|
|
161
|
+
continue
|
|
162
|
+
} else {
|
|
163
|
+
results.push({ candidate, status: 'no_executor', txn })
|
|
164
|
+
continue
|
|
165
|
+
}
|
|
166
|
+
const probeResult = await probeCapability(candidate, { tools: probeTools, skills: probeSkills, probe })
|
|
167
|
+
if (!probeResult.ok) {
|
|
168
|
+
const rollback = await rollbackProvision({ ...txn, status: 'probe_failed' }, execute)
|
|
169
|
+
results.push({ candidate, status: rollback.ok ? 'rolled_back' : 'rollback_failed', txn: rollback.txn, probe: probeResult })
|
|
170
|
+
continue
|
|
171
|
+
}
|
|
172
|
+
results.push({ candidate, status: 'ready', txn: { ...txn, status: 'ready' }, probe: probeResult })
|
|
173
|
+
} catch (error) {
|
|
174
|
+
results.push({ candidate, status: 'error', error: error?.message || String(error), txn: { ...txn, status: 'error' } })
|
|
175
|
+
}
|
|
176
|
+
}
|
|
177
|
+
return { results, summary: summarizeProvisionResults(results) }
|
|
178
|
+
}
|
|
179
|
+
|
|
180
|
+
export function summarizeProvisionResults(results = []) {
|
|
181
|
+
const counts = {}
|
|
182
|
+
for (const r of results) counts[r.status] = (counts[r.status] || 0) + 1
|
|
183
|
+
return counts
|
|
184
|
+
}
|
|
185
|
+
|
|
186
|
+
export function formatProvisionResult(result = {}) {
|
|
187
|
+
const lines = []
|
|
188
|
+
for (const r of result.results || []) {
|
|
189
|
+
lines.push(`- ${r.candidate?.id || r.candidate?.package}: ${r.status}${r.error ? ` (${r.error})` : ''}`)
|
|
190
|
+
if (r.probe?.checks?.length) {
|
|
191
|
+
for (const check of r.probe.checks) lines.push(` ${check.ok ? '✓' : '✗'} ${check.type} ${check.name}`)
|
|
192
|
+
}
|
|
193
|
+
}
|
|
194
|
+
return lines.join('\n')
|
|
195
|
+
}
|
|
196
|
+
|
|
197
|
+
// --- Discovery adapters -----------------------------------------------------
|
|
198
|
+
|
|
199
|
+
export function createMarketplaceAdapter({ search, id = 'marketplace' } = {}) {
|
|
200
|
+
return {
|
|
201
|
+
id,
|
|
202
|
+
async search(missing, options) {
|
|
203
|
+
if (typeof search !== 'function') return []
|
|
204
|
+
return search(missing, options)
|
|
205
|
+
},
|
|
206
|
+
}
|
|
207
|
+
}
|
|
208
|
+
|
|
209
|
+
export function createHubAdapter({ search, id = 'hub' } = {}) {
|
|
210
|
+
return createMarketplaceAdapter({ search, id })
|
|
211
|
+
}
|
|
212
|
+
|
|
213
|
+
export function createStaticRegistryAdapter(registry = [], id = 'registry') {
|
|
214
|
+
return {
|
|
215
|
+
id,
|
|
216
|
+
async search(missing, options = {}) {
|
|
217
|
+
const missingSet = new Set(missing)
|
|
218
|
+
return registry
|
|
219
|
+
.filter((candidate) => (candidate.provides || []).some((c) => missingSet.has(c)))
|
|
220
|
+
.map((candidate) => ({ ...candidate, source: candidate.source || 'static-registry' }))
|
|
221
|
+
},
|
|
222
|
+
}
|
|
223
|
+
}
|
|
@@ -0,0 +1,90 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* Plugin / Skill Quality Scoring.
|
|
3
|
+
*
|
|
4
|
+
* Ranks candidate capability providers by capability fit, reliability,
|
|
5
|
+
* security/trust, maintenance, community adoption, performance, and an overlap
|
|
6
|
+
* penalty that keeps the tool catalog minimal.
|
|
7
|
+
*/
|
|
8
|
+
|
|
9
|
+
const WEIGHTS = {
|
|
10
|
+
fit: 0.30,
|
|
11
|
+
reliability: 0.20,
|
|
12
|
+
trust: 0.20,
|
|
13
|
+
maintenance: 0.10,
|
|
14
|
+
community: 0.10,
|
|
15
|
+
performance: 0.05,
|
|
16
|
+
}
|
|
17
|
+
|
|
18
|
+
export function capabilityFit(candidate = {}, required = []) {
|
|
19
|
+
const provides = new Set(candidate.provides || [])
|
|
20
|
+
if (!required.length) return 0
|
|
21
|
+
const covered = required.filter((r) => provides.has(r)).length
|
|
22
|
+
return covered / required.length
|
|
23
|
+
}
|
|
24
|
+
|
|
25
|
+
export function overlapWithExisting(candidate = {}, brain = { capabilities: [] }) {
|
|
26
|
+
const existing = new Set()
|
|
27
|
+
for (const provider of brain.capabilities || []) {
|
|
28
|
+
for (const cap of provider.capabilities || []) existing.add(cap)
|
|
29
|
+
}
|
|
30
|
+
const provides = candidate.provides || []
|
|
31
|
+
if (!provides.length) return 0
|
|
32
|
+
const overlapped = provides.filter((c) => existing.has(c)).length
|
|
33
|
+
return overlapped / provides.length
|
|
34
|
+
}
|
|
35
|
+
|
|
36
|
+
export function scorePluginCandidate(candidate = {}, required = [], brain = { capabilities: [] }, options = {}) {
|
|
37
|
+
const fit = capabilityFit(candidate, required)
|
|
38
|
+
const reliability = Number(candidate.reliability ?? 0.8)
|
|
39
|
+
const trust = candidate.verified === true || candidate.trustedSource === true
|
|
40
|
+
? 1
|
|
41
|
+
: Number(candidate.trust ?? 0.4)
|
|
42
|
+
const maintenance = Number(candidate.maintenance ?? 0.5)
|
|
43
|
+
const community = Number(candidate.community ?? 0.5)
|
|
44
|
+
const performance = Number(candidate.performance ?? (1 - Number(candidate.contextCost ?? 0.2)))
|
|
45
|
+
|
|
46
|
+
const overlap = options.overlap ?? overlapWithExisting(candidate, brain)
|
|
47
|
+
const overlapPenalty = 0.05 + Math.min(0.25, overlap * 0.25)
|
|
48
|
+
|
|
49
|
+
const raw = (
|
|
50
|
+
fit * WEIGHTS.fit +
|
|
51
|
+
reliability * WEIGHTS.reliability +
|
|
52
|
+
trust * WEIGHTS.trust +
|
|
53
|
+
maintenance * WEIGHTS.maintenance +
|
|
54
|
+
community * WEIGHTS.community +
|
|
55
|
+
performance * WEIGHTS.performance
|
|
56
|
+
) - overlapPenalty
|
|
57
|
+
|
|
58
|
+
const score = Math.max(0, Math.min(1, Math.round(raw * 1000) / 1000))
|
|
59
|
+
return {
|
|
60
|
+
candidate,
|
|
61
|
+
fit: Math.round(fit * 1000) / 1000,
|
|
62
|
+
reliability,
|
|
63
|
+
trust,
|
|
64
|
+
maintenance,
|
|
65
|
+
community,
|
|
66
|
+
performance,
|
|
67
|
+
overlap: Math.round(overlap * 1000) / 1000,
|
|
68
|
+
overlapPenalty: Math.round(overlapPenalty * 1000) / 1000,
|
|
69
|
+
score,
|
|
70
|
+
}
|
|
71
|
+
}
|
|
72
|
+
|
|
73
|
+
export function installUtility(candidate = {}, required = [], brain = { capabilities: [] }, options = {}) {
|
|
74
|
+
const quality = scorePluginCandidate(candidate, required, brain, options)
|
|
75
|
+
const gain = 1 - quality.overlap
|
|
76
|
+
const taskFrequency = Number(options.taskFrequency ?? 1)
|
|
77
|
+
const latencyPenalty = Number(options.latencyPenalty ?? candidate.latency ?? 0)
|
|
78
|
+
const contextCost = Number(options.contextCost ?? candidate.contextCost ?? 0)
|
|
79
|
+
const maintenanceCost = Number(options.maintenanceCost ?? (1 - quality.maintenance) * 0.1)
|
|
80
|
+
const riskPenalty = Number(options.riskPenalty ?? (candidate.risk === 'high' ? 0.15 : candidate.risk === 'critical' ? 0.3 : 0))
|
|
81
|
+
const utility = quality.score * gain * taskFrequency - latencyPenalty - contextCost - maintenanceCost - riskPenalty
|
|
82
|
+
return {
|
|
83
|
+
...quality,
|
|
84
|
+
utility: Math.round(utility * 1000) / 1000,
|
|
85
|
+
}
|
|
86
|
+
}
|
|
87
|
+
|
|
88
|
+
export function formatCandidateScore(scored) {
|
|
89
|
+
return `${scored.candidate.id} score=${scored.score} fit=${scored.fit} overlap=${scored.overlap}${scored.utility !== undefined ? ` utility=${scored.utility}` : ''}`
|
|
90
|
+
}
|
|
@@ -0,0 +1,88 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* Minimal Capability Set Solver.
|
|
3
|
+
*
|
|
4
|
+
* Chooses the smallest reliable set of plugins/skills that covers the current
|
|
5
|
+
* task's required capabilities. This is a greedy weighted set cover: maximize
|
|
6
|
+
* marginal capability gain while minimizing plugin count and risk.
|
|
7
|
+
*/
|
|
8
|
+
|
|
9
|
+
import { scorePluginCandidate } from './capability-quality.mjs'
|
|
10
|
+
|
|
11
|
+
export function solveMinimalSet(required = [], candidates = [], options = {}) {
|
|
12
|
+
const maxPlugins = Number(options.maxPlugins ?? 2)
|
|
13
|
+
const minScore = Number(options.minScore ?? 0.4)
|
|
14
|
+
const brain = options.brain || { capabilities: [] }
|
|
15
|
+
const target = [...new Set(required.filter(Boolean))]
|
|
16
|
+
const scored = candidates
|
|
17
|
+
.map((c) => scorePluginCandidate(c, target, brain, options))
|
|
18
|
+
.filter((s) => s.score >= minScore && s.fit > 0)
|
|
19
|
+
.sort((a, b) => b.score - a.score)
|
|
20
|
+
|
|
21
|
+
const selected = []
|
|
22
|
+
const covered = new Set()
|
|
23
|
+
let remaining = target.filter((r) => !covered.has(r))
|
|
24
|
+
let totalScore = 0
|
|
25
|
+
|
|
26
|
+
while (remaining.length > 0 && selected.length < maxPlugins) {
|
|
27
|
+
let best = null
|
|
28
|
+
let bestMarginal = -1
|
|
29
|
+
for (const scoredCandidate of scored) {
|
|
30
|
+
if (selected.some((s) => s.candidate.id === scoredCandidate.candidate.id)) continue
|
|
31
|
+
const provides = new Set(scoredCandidate.candidate.provides || [])
|
|
32
|
+
const newly = remaining.filter((r) => provides.has(r)).length
|
|
33
|
+
if (newly === 0) continue
|
|
34
|
+
const marginal = (newly / target.length) * scoredCandidate.score
|
|
35
|
+
const riskPenalty = scoredCandidate.candidate.risk === 'high' ? 0.05 : scoredCandidate.candidate.risk === 'critical' ? 0.1 : 0
|
|
36
|
+
const value = marginal - riskPenalty
|
|
37
|
+
if (value > bestMarginal) {
|
|
38
|
+
bestMarginal = value
|
|
39
|
+
best = scoredCandidate
|
|
40
|
+
}
|
|
41
|
+
}
|
|
42
|
+
if (!best) break
|
|
43
|
+
selected.push(best)
|
|
44
|
+
totalScore += best.score
|
|
45
|
+
for (const cap of best.candidate.provides || []) covered.add(cap)
|
|
46
|
+
remaining = target.filter((r) => !covered.has(r))
|
|
47
|
+
}
|
|
48
|
+
|
|
49
|
+
return {
|
|
50
|
+
selected: selected.map((s) => s.candidate),
|
|
51
|
+
scored: selected,
|
|
52
|
+
covered: [...covered],
|
|
53
|
+
missing: target.filter((r) => !covered.has(r)),
|
|
54
|
+
totalScore: Math.round(totalScore * 1000) / 1000,
|
|
55
|
+
pluginCount: selected.length,
|
|
56
|
+
}
|
|
57
|
+
}
|
|
58
|
+
|
|
59
|
+
export function isRedundant(candidate = {}, brain = { capabilities: [] }) {
|
|
60
|
+
const providers = brain.capabilities || []
|
|
61
|
+
const candidateId = candidate.id
|
|
62
|
+
const provides = candidate.provides || candidate.capabilities || []
|
|
63
|
+
if (!provides.length) return false
|
|
64
|
+
return provides.every((cap) =>
|
|
65
|
+
providers.some((p) => p.id !== candidateId && (p.capabilities || []).includes(cap)),
|
|
66
|
+
)
|
|
67
|
+
}
|
|
68
|
+
|
|
69
|
+
export function findRedundantProviders(brain = { capabilities: [] }) {
|
|
70
|
+
return (brain.capabilities || [])
|
|
71
|
+
.filter((provider) => isRedundant(provider, brain))
|
|
72
|
+
.map((provider) => ({
|
|
73
|
+
id: provider.id,
|
|
74
|
+
capabilities: provider.capabilities || [],
|
|
75
|
+
reason: 'all capabilities covered by other providers',
|
|
76
|
+
}))
|
|
77
|
+
}
|
|
78
|
+
|
|
79
|
+
export function formatSolveResult(result = {}) {
|
|
80
|
+
const lines = [
|
|
81
|
+
`Minimal set: ${result.pluginCount ?? 0} provider(s), covered ${result.covered?.length || 0}/${result.required?.length || result.covered?.length || 0}`,
|
|
82
|
+
]
|
|
83
|
+
for (const s of result.scored || []) {
|
|
84
|
+
lines.push(`- ${s.candidate.id} (score=${s.score}, provides ${(s.candidate.provides || []).join(', ')})`)
|
|
85
|
+
}
|
|
86
|
+
if (result.missing?.length) lines.push(`Missing: ${result.missing.join(', ')}`)
|
|
87
|
+
return lines.join('\n')
|
|
88
|
+
}
|
package/src/omni-router.mjs
CHANGED
|
@@ -35,6 +35,8 @@ import { generateMissionDag, roleForTask } from './planner-dag.mjs'
|
|
|
35
35
|
import { autoPopulateCapabilityBrain, createCapabilityBrain, recordCapabilityOutcome } from './capability-brain.mjs'
|
|
36
36
|
import { loadCapabilityManifests } from './capability-manifest.mjs'
|
|
37
37
|
import { capabilityToolFilter } from './capability-sandbox.mjs'
|
|
38
|
+
import { baselineAudit, formatCapabilityAudit, taskTimeAudit } from './capability-auditor.mjs'
|
|
39
|
+
import { createStaticRegistryAdapter, discoverCandidates, evaluateProvisionPlan, formatProvisionResult, probeCapability, provisionCapabilities } from './capability-provisioner.mjs'
|
|
38
40
|
import { buildProgressiveContext } from './context-expansion.mjs'
|
|
39
41
|
import { buildDynamicContext } from './dynamic-context.mjs'
|
|
40
42
|
import { classifyFailure } from './failure-taxonomy.mjs'
|
|
@@ -1468,6 +1470,136 @@ export function apply(ctx, config = {}) {
|
|
|
1468
1470
|
return { act, observe, saveProgress, getCapabilityBrain: () => brain }
|
|
1469
1471
|
}
|
|
1470
1472
|
|
|
1473
|
+
registerTool({
|
|
1474
|
+
name: 'omni_capability_audit',
|
|
1475
|
+
description: 'Audit current Harness capabilities against the Omni Coding Baseline or task requirements.',
|
|
1476
|
+
parameters: {
|
|
1477
|
+
type: 'object',
|
|
1478
|
+
properties: {
|
|
1479
|
+
requirements: { type: 'array', items: { type: 'string' }, description: 'Optional task capability requirements' },
|
|
1480
|
+
baseline: { type: 'boolean', description: 'Run baseline audit when no requirements are given (default true)' },
|
|
1481
|
+
},
|
|
1482
|
+
},
|
|
1483
|
+
async execute(args) {
|
|
1484
|
+
const session = currentSession()
|
|
1485
|
+
const toolsService = ctx.get('tools') || ctx.tools
|
|
1486
|
+
const toolNames = await collectToolNames(toolsService)
|
|
1487
|
+
let brain = autoPopulateCapabilityBrain(createCapabilityBrain(), toolNames)
|
|
1488
|
+
brain = loadCapabilityManifests(brain, config.capabilityManifests || [])
|
|
1489
|
+
const requirements = Array.isArray(args?.requirements) ? args.requirements.map(String).filter(Boolean) : []
|
|
1490
|
+
const audit = requirements.length ? taskTimeAudit(brain, requirements) : baselineAudit(brain)
|
|
1491
|
+
return formatCapabilityAudit(audit)
|
|
1492
|
+
},
|
|
1493
|
+
})
|
|
1494
|
+
|
|
1495
|
+
registerTool({
|
|
1496
|
+
name: 'omni_capability_provision',
|
|
1497
|
+
description: 'Detect missing capabilities, discover candidate plugins/skills, evaluate a minimal set, and optionally provision after approval/dry-run.',
|
|
1498
|
+
parameters: {
|
|
1499
|
+
type: 'object',
|
|
1500
|
+
properties: {
|
|
1501
|
+
requirements: { type: 'array', items: { type: 'string' }, description: 'Task capability requirements (preferred)' },
|
|
1502
|
+
missing: { type: 'array', items: { type: 'string' }, description: 'Explicit missing capabilities (alternative to requirements)' },
|
|
1503
|
+
mode: { type: 'string', enum: ['recommend', 'auto-trusted', 'manual'], description: 'Provisioning trust mode (default auto-trusted)' },
|
|
1504
|
+
dryRun: { type: 'boolean', description: 'Only evaluate and print the plan (default true)' },
|
|
1505
|
+
maxPlugins: { type: 'number', description: 'Maximum plugins to install for this task (default 2)' },
|
|
1506
|
+
profile: { type: 'string', description: 'DSH profile to provision into (default web)' },
|
|
1507
|
+
},
|
|
1508
|
+
required: [],
|
|
1509
|
+
},
|
|
1510
|
+
async execute(args) {
|
|
1511
|
+
const session = currentSession()
|
|
1512
|
+
const toolsService = ctx.get('tools') || ctx.tools
|
|
1513
|
+
const toolNames = await collectToolNames(toolsService)
|
|
1514
|
+
let brain = autoPopulateCapabilityBrain(createCapabilityBrain(), toolNames)
|
|
1515
|
+
brain = loadCapabilityManifests(brain, config.capabilityManifests || [])
|
|
1516
|
+
const provisioning = config.capabilityProvisioning || {}
|
|
1517
|
+
const requirements = Array.isArray(args?.requirements) ? args.requirements.map(String).filter(Boolean) : []
|
|
1518
|
+
const explicitMissing = Array.isArray(args?.missing) ? args.missing.map(String).filter(Boolean) : []
|
|
1519
|
+
const missing = explicitMissing.length
|
|
1520
|
+
? explicitMissing
|
|
1521
|
+
: requirements.length
|
|
1522
|
+
? taskTimeAudit(brain, requirements).missing
|
|
1523
|
+
: baselineAudit(brain).missing
|
|
1524
|
+
|
|
1525
|
+
if (!missing.length) return 'No missing capabilities detected.'
|
|
1526
|
+
|
|
1527
|
+
const adapters = [
|
|
1528
|
+
createStaticRegistryAdapter(provisioning.registry || []),
|
|
1529
|
+
...(Array.isArray(provisioning.adapters) ? provisioning.adapters : []),
|
|
1530
|
+
]
|
|
1531
|
+
const candidates = await discoverCandidates(missing, adapters)
|
|
1532
|
+
if (!candidates.length) {
|
|
1533
|
+
return `No candidate providers found for: ${missing.join(', ')}`
|
|
1534
|
+
}
|
|
1535
|
+
|
|
1536
|
+
const mode = args?.mode || provisioning.mode || 'auto-trusted'
|
|
1537
|
+
const plan = evaluateProvisionPlan(missing, candidates, brain, {
|
|
1538
|
+
mode,
|
|
1539
|
+
maxPlugins: Number(args?.maxPlugins ?? provisioning.maxTaskPlugins ?? 2),
|
|
1540
|
+
trustedSources: provisioning.trustedSources || [],
|
|
1541
|
+
})
|
|
1542
|
+
|
|
1543
|
+
const lines = [
|
|
1544
|
+
`Missing (${missing.length}): ${missing.join(', ')}`,
|
|
1545
|
+
`Candidates (${candidates.length}):`,
|
|
1546
|
+
...candidates.map((c) => `- ${c.id || c.package}: ${(c.provides || []).join(', ')}${c.verified ? ' [verified]' : ''}`),
|
|
1547
|
+
'',
|
|
1548
|
+
`Selected minimal set (${plan.selected.length}):`,
|
|
1549
|
+
...plan.selected.map((s) => `- ${s.candidate.id} score=${s.score}`),
|
|
1550
|
+
`Requires approval: ${plan.requiresApproval.map((s) => s.candidate.id).join(', ') || '(none)'}`,
|
|
1551
|
+
`Still missing: ${plan.solution.missing.join(', ') || '(none)'}`,
|
|
1552
|
+
]
|
|
1553
|
+
|
|
1554
|
+
if (args?.dryRun !== false) {
|
|
1555
|
+
lines.push('', 'Dry run: no changes made. Set dryRun:false to provision (requires trusted mode + executor).')
|
|
1556
|
+
return lines.join('\n')
|
|
1557
|
+
}
|
|
1558
|
+
|
|
1559
|
+
const result = await provisionCapabilities(plan, {
|
|
1560
|
+
mode,
|
|
1561
|
+
profile: args?.profile || provisioning.profile || 'web',
|
|
1562
|
+
trustedSources: provisioning.trustedSources || [],
|
|
1563
|
+
execute: provisioning.execute,
|
|
1564
|
+
probeTools: toolNames,
|
|
1565
|
+
probeSkills: provisioning.probeSkills || [],
|
|
1566
|
+
})
|
|
1567
|
+
lines.push('', 'Provision result:')
|
|
1568
|
+
lines.push(formatProvisionResult(result))
|
|
1569
|
+
return lines.join('\n')
|
|
1570
|
+
},
|
|
1571
|
+
})
|
|
1572
|
+
|
|
1573
|
+
registerTool({
|
|
1574
|
+
name: 'omni_capability_probe',
|
|
1575
|
+
description: 'Probe whether an installed provider actually exposes its expected tools/skills.',
|
|
1576
|
+
parameters: {
|
|
1577
|
+
type: 'object',
|
|
1578
|
+
properties: {
|
|
1579
|
+
provider: { type: 'string', description: 'Provider id in the Capability Brain' },
|
|
1580
|
+
expectedTools: { type: 'array', items: { type: 'string' }, description: 'Tool names the provider should expose' },
|
|
1581
|
+
expectedSkills: { type: 'array', items: { type: 'string' }, description: 'Skill names the provider should expose' },
|
|
1582
|
+
},
|
|
1583
|
+
required: ['provider'],
|
|
1584
|
+
},
|
|
1585
|
+
async execute(args) {
|
|
1586
|
+
const session = currentSession()
|
|
1587
|
+
const toolsService = ctx.get('tools') || ctx.tools
|
|
1588
|
+
const toolNames = await collectToolNames(toolsService)
|
|
1589
|
+
let brain = autoPopulateCapabilityBrain(createCapabilityBrain(), toolNames)
|
|
1590
|
+
brain = loadCapabilityManifests(brain, config.capabilityManifests || [])
|
|
1591
|
+
const provider = (brain.capabilities || []).find((c) => c.id === args?.provider)
|
|
1592
|
+
if (!provider) return `Provider "${args?.provider}" not found in Capability Brain.`
|
|
1593
|
+
const probe = await probeCapability(
|
|
1594
|
+
{ ...provider, expectedTools: args?.expectedTools || [], expectedSkills: args?.expectedSkills || [] },
|
|
1595
|
+
{ tools: toolNames, skills: config.capabilityProvisioning?.probeSkills || [] },
|
|
1596
|
+
)
|
|
1597
|
+
const lines = [`Probe ${args.provider}: ${probe.ok ? 'READY' : 'BROKEN'}`]
|
|
1598
|
+
for (const check of probe.checks) lines.push(`- ${check.ok ? '✓' : '✗'} ${check.type} ${check.name}`)
|
|
1599
|
+
return lines.join('\n')
|
|
1600
|
+
},
|
|
1601
|
+
})
|
|
1602
|
+
|
|
1471
1603
|
registerTool({
|
|
1472
1604
|
name: 'omni_mission_run',
|
|
1473
1605
|
description: 'Run a Mission Planner loop with real subagents: Observe → Think → Act → Replan until completed or maxSteps.',
|
|
@@ -0,0 +1,43 @@
|
|
|
1
|
+
import test from 'node:test'
|
|
2
|
+
import assert from 'node:assert/strict'
|
|
3
|
+
|
|
4
|
+
import {
|
|
5
|
+
OMNI_CODING_BASELINE,
|
|
6
|
+
auditCapabilities,
|
|
7
|
+
baselineAudit,
|
|
8
|
+
formatCapabilityAudit,
|
|
9
|
+
taskTimeAudit,
|
|
10
|
+
} from '../src/capability-auditor.mjs'
|
|
11
|
+
import { createCapabilityBrain, registerCapability } from '../src/capability-brain.mjs'
|
|
12
|
+
|
|
13
|
+
test('baselineAudit reports missing coding baseline on empty brain', () => {
|
|
14
|
+
const audit = baselineAudit(createCapabilityBrain())
|
|
15
|
+
assert.ok(audit.missing.includes('repository.read'))
|
|
16
|
+
assert.ok(audit.missing.includes('verification'))
|
|
17
|
+
assert.equal(audit.coverage, 0)
|
|
18
|
+
})
|
|
19
|
+
|
|
20
|
+
test('auditCapabilities lists available and missing', () => {
|
|
21
|
+
let brain = createCapabilityBrain()
|
|
22
|
+
brain = registerCapability(brain, { id: 'fs-tools', capabilities: ['repository.read', 'repository.search', 'source.write'] })
|
|
23
|
+
const audit = auditCapabilities(brain, ['repository.read', 'source.write', 'test.run', 'browser.screenshot'])
|
|
24
|
+
assert.deepEqual(audit.available, ['repository.read', 'source.write'])
|
|
25
|
+
assert.deepEqual(audit.missing, ['test.run', 'browser.screenshot'])
|
|
26
|
+
assert.equal(audit.coverage, 0.5)
|
|
27
|
+
})
|
|
28
|
+
|
|
29
|
+
test('taskTimeAudit returns gaps with severity', () => {
|
|
30
|
+
const audit = taskTimeAudit(createCapabilityBrain(), ['browser.screenshot', 'verification'])
|
|
31
|
+
assert.equal(audit.gaps.length, 2)
|
|
32
|
+
assert.ok(audit.gaps.every((g) => g.severity))
|
|
33
|
+
})
|
|
34
|
+
|
|
35
|
+
test('formatCapabilityAudit renders human-readable summary', () => {
|
|
36
|
+
const text = formatCapabilityAudit(baselineAudit(createCapabilityBrain()))
|
|
37
|
+
assert.match(text, /Capability audit/)
|
|
38
|
+
assert.match(text, /Missing/)
|
|
39
|
+
})
|
|
40
|
+
|
|
41
|
+
test('OMNI_CODING_BASELINE is defined', () => {
|
|
42
|
+
assert.ok(OMNI_CODING_BASELINE.length >= 7)
|
|
43
|
+
})
|
|
@@ -0,0 +1,64 @@
|
|
|
1
|
+
import test from 'node:test'
|
|
2
|
+
import assert from 'node:assert/strict'
|
|
3
|
+
|
|
4
|
+
import {
|
|
5
|
+
createPerformanceRegistry,
|
|
6
|
+
evaluateProviderValue,
|
|
7
|
+
formatPerformanceRegistry,
|
|
8
|
+
recommendDemotion,
|
|
9
|
+
recordProvisionOutcome,
|
|
10
|
+
} from '../src/capability-performance.mjs'
|
|
11
|
+
|
|
12
|
+
test('recordProvisionOutcome stores before/after metrics', () => {
|
|
13
|
+
let registry = createPerformanceRegistry()
|
|
14
|
+
registry = recordProvisionOutcome(registry, 'plugin-a', {
|
|
15
|
+
successBefore: 0.73,
|
|
16
|
+
successAfter: 0.86,
|
|
17
|
+
falseCompletionBefore: 0.12,
|
|
18
|
+
falseCompletionAfter: 0.04,
|
|
19
|
+
tokensBefore: 1000,
|
|
20
|
+
tokensAfter: 1080,
|
|
21
|
+
})
|
|
22
|
+
assert.equal(registry.providers['plugin-a'].runs, 1)
|
|
23
|
+
const value = evaluateProviderValue(registry.providers['plugin-a'])
|
|
24
|
+
assert.equal(value.label, 'high')
|
|
25
|
+
})
|
|
26
|
+
|
|
27
|
+
test('evaluateProviderValue labels negative outcomes', () => {
|
|
28
|
+
const value = evaluateProviderValue({
|
|
29
|
+
successBefore: 0.8,
|
|
30
|
+
successAfter: 0.81,
|
|
31
|
+
falseCompletionBefore: 0.1,
|
|
32
|
+
falseCompletionAfter: 0.1,
|
|
33
|
+
tokensBefore: 1000,
|
|
34
|
+
tokensAfter: 1300,
|
|
35
|
+
toolErrorsBefore: 0,
|
|
36
|
+
toolErrorsAfter: 5,
|
|
37
|
+
})
|
|
38
|
+
assert.equal(value.label, 'negative')
|
|
39
|
+
})
|
|
40
|
+
|
|
41
|
+
test('recommendDemotion removes negative redundant providers', () => {
|
|
42
|
+
const registry = createPerformanceRegistry({
|
|
43
|
+
providers: {
|
|
44
|
+
'old-plugin': {
|
|
45
|
+
successBefore: 0.8,
|
|
46
|
+
successAfter: 0.78,
|
|
47
|
+
falseCompletionBefore: 0.1,
|
|
48
|
+
falseCompletionAfter: 0.1,
|
|
49
|
+
tokensBefore: 1000,
|
|
50
|
+
tokensAfter: 1200,
|
|
51
|
+
toolErrorsBefore: 0,
|
|
52
|
+
toolErrorsAfter: 2,
|
|
53
|
+
},
|
|
54
|
+
},
|
|
55
|
+
})
|
|
56
|
+
const rec = recommendDemotion(registry, 'old-plugin', { usageDays: 45, uniqueCapabilities: 1 })
|
|
57
|
+
assert.equal(rec.recommendation, 'demote')
|
|
58
|
+
})
|
|
59
|
+
|
|
60
|
+
test('formatPerformanceRegistry renders entries', () => {
|
|
61
|
+
let registry = createPerformanceRegistry()
|
|
62
|
+
registry = recordProvisionOutcome(registry, 'p', { successBefore: 0.5, successAfter: 0.9 })
|
|
63
|
+
assert.match(formatPerformanceRegistry(registry), /p/)
|
|
64
|
+
})
|
|
@@ -0,0 +1,96 @@
|
|
|
1
|
+
import test from 'node:test'
|
|
2
|
+
import assert from 'node:assert/strict'
|
|
3
|
+
|
|
4
|
+
import {
|
|
5
|
+
canAutoInstall,
|
|
6
|
+
createHubAdapter,
|
|
7
|
+
createMarketplaceAdapter,
|
|
8
|
+
createProvisionTransaction,
|
|
9
|
+
createStaticRegistryAdapter,
|
|
10
|
+
discoverCandidates,
|
|
11
|
+
evaluateProvisionPlan,
|
|
12
|
+
formatProvisionResult,
|
|
13
|
+
probeCapability,
|
|
14
|
+
provisionCapabilities,
|
|
15
|
+
rollbackProvision,
|
|
16
|
+
} from '../src/capability-provisioner.mjs'
|
|
17
|
+
import { createCapabilityBrain } from '../src/capability-brain.mjs'
|
|
18
|
+
|
|
19
|
+
test('canAutoInstall enforces trust mode and risk', () => {
|
|
20
|
+
const verified = { verified: true, risk: 'low' }
|
|
21
|
+
assert.equal(canAutoInstall(verified, 'auto-trusted'), true)
|
|
22
|
+
assert.equal(canAutoInstall(verified, 'manual'), false)
|
|
23
|
+
assert.equal(canAutoInstall({ verified: false, risk: 'low' }, 'auto-trusted'), false)
|
|
24
|
+
assert.equal(canAutoInstall({ verified: true, risk: 'high' }, 'auto-trusted'), false)
|
|
25
|
+
assert.equal(canAutoInstall({ source: 'trusted-registry', risk: 'low' }, 'auto-trusted', ['trusted-registry']), true)
|
|
26
|
+
})
|
|
27
|
+
|
|
28
|
+
test('static registry adapter discovers matching candidates', async () => {
|
|
29
|
+
const adapter = createStaticRegistryAdapter([
|
|
30
|
+
{ id: 'browser-kit', provides: ['browser.screenshot', 'frontend.validation'], verified: true, reliability: 0.9 },
|
|
31
|
+
{ id: 'git-kit', provides: ['github.read'], verified: true, reliability: 0.9 },
|
|
32
|
+
])
|
|
33
|
+
const candidates = await discoverCandidates(['browser.screenshot', 'github.read'], [adapter])
|
|
34
|
+
assert.equal(candidates.length, 2)
|
|
35
|
+
})
|
|
36
|
+
|
|
37
|
+
test('marketplace and hub adapters delegate search', async () => {
|
|
38
|
+
const marketplace = createMarketplaceAdapter({ search: async () => [{ id: 'mp', provides: ['test.run'] }] })
|
|
39
|
+
const hub = createHubAdapter({ search: async () => [{ id: 'hub', provides: ['debugging'] }] })
|
|
40
|
+
const candidates = await discoverCandidates(['test.run', 'debugging'], [marketplace, hub])
|
|
41
|
+
assert.deepEqual(candidates.map((c) => c.id).sort(), ['hub', 'mp'])
|
|
42
|
+
})
|
|
43
|
+
|
|
44
|
+
test('evaluateProvisionPlan selects minimal set and flags approval', () => {
|
|
45
|
+
const brain = createCapabilityBrain()
|
|
46
|
+
const candidates = [
|
|
47
|
+
{ id: 'trusted', provides: ['test.run', 'debugging'], verified: true, reliability: 0.9, risk: 'low' },
|
|
48
|
+
{ id: 'untrusted', provides: ['test.run', 'debugging'], verified: false, reliability: 0.9, risk: 'medium' },
|
|
49
|
+
]
|
|
50
|
+
const plan = evaluateProvisionPlan(['test.run', 'debugging'], candidates, brain, { mode: 'auto-trusted', maxPlugins: 1 })
|
|
51
|
+
assert.equal(plan.selected.length, 1)
|
|
52
|
+
assert.equal(plan.selected[0].candidate.id, 'trusted')
|
|
53
|
+
assert.equal(plan.requiresApproval.length, 0)
|
|
54
|
+
})
|
|
55
|
+
|
|
56
|
+
test('provisionCapabilities installs, probes, and rolls back on probe failure', async () => {
|
|
57
|
+
const plan = {
|
|
58
|
+
missing: ['test.run'],
|
|
59
|
+
selected: [{
|
|
60
|
+
candidate: { id: 'test-kit', package: 'test-kit', provides: ['test.run'], verified: true, risk: 'low', rollbackCommand: 'remove test-kit', expectedTools: ['test_runner'] },
|
|
61
|
+
score: 0.9,
|
|
62
|
+
}],
|
|
63
|
+
}
|
|
64
|
+
const calls = []
|
|
65
|
+
const result = await provisionCapabilities(plan, {
|
|
66
|
+
mode: 'auto-trusted',
|
|
67
|
+
execute: async ({ type, candidate, txn }) => {
|
|
68
|
+
calls.push(type)
|
|
69
|
+
if (type === 'install') return true
|
|
70
|
+
if (type === 'rollback') return true
|
|
71
|
+
return false
|
|
72
|
+
},
|
|
73
|
+
probeTools: [],
|
|
74
|
+
})
|
|
75
|
+
// probe fails because expected tool is missing, so rollback path runs
|
|
76
|
+
assert.equal(result.results[0].status, 'rolled_back')
|
|
77
|
+
assert.ok(calls.includes('install'))
|
|
78
|
+
assert.ok(calls.includes('rollback'))
|
|
79
|
+
})
|
|
80
|
+
|
|
81
|
+
test('probeCapability checks tools and skills', async () => {
|
|
82
|
+
const provider = { expectedTools: ['browser_screenshot'], expectedSkills: ['frontend-validation'] }
|
|
83
|
+
const ok = await probeCapability(provider, { tools: ['browser_screenshot'], skills: ['frontend-validation'] })
|
|
84
|
+
assert.equal(ok.ok, true)
|
|
85
|
+
const bad = await probeCapability(provider, { tools: [], skills: [] })
|
|
86
|
+
assert.equal(bad.ok, false)
|
|
87
|
+
})
|
|
88
|
+
|
|
89
|
+
test('createProvisionTransaction and rollbackProvision', async () => {
|
|
90
|
+
const txn = createProvisionTransaction({ package: 'x', rollbackCommand: 'remove x' })
|
|
91
|
+
assert.equal(txn.status, 'pending')
|
|
92
|
+
const rollback = await rollbackProvision(txn, async () => true)
|
|
93
|
+
assert.equal(rollback.ok, true)
|
|
94
|
+
assert.equal(rollback.txn.status, 'rolled_back')
|
|
95
|
+
assert.match(formatProvisionResult({ results: [] }), /^$/)
|
|
96
|
+
})
|
|
@@ -0,0 +1,62 @@
|
|
|
1
|
+
import test from 'node:test'
|
|
2
|
+
import assert from 'node:assert/strict'
|
|
3
|
+
|
|
4
|
+
import {
|
|
5
|
+
capabilityFit,
|
|
6
|
+
formatCandidateScore,
|
|
7
|
+
installUtility,
|
|
8
|
+
overlapWithExisting,
|
|
9
|
+
scorePluginCandidate,
|
|
10
|
+
} from '../src/capability-quality.mjs'
|
|
11
|
+
import { createCapabilityBrain, registerCapability } from '../src/capability-brain.mjs'
|
|
12
|
+
|
|
13
|
+
test('capabilityFit measures required coverage', () => {
|
|
14
|
+
const candidate = { provides: ['browser.navigation', 'browser.interaction'] }
|
|
15
|
+
assert.equal(capabilityFit(candidate, ['browser.navigation', 'browser.interaction', 'browser.screenshot']), 2 / 3)
|
|
16
|
+
})
|
|
17
|
+
|
|
18
|
+
test('overlapWithExisting penalizes duplicate capabilities', () => {
|
|
19
|
+
let brain = createCapabilityBrain()
|
|
20
|
+
brain = registerCapability(brain, { id: 'existing', capabilities: ['browser.navigation', 'browser.screenshot'] })
|
|
21
|
+
const candidate = { provides: ['browser.navigation', 'browser.screenshot', 'github.read'] }
|
|
22
|
+
assert.equal(overlapWithExisting(candidate, brain), 2 / 3)
|
|
23
|
+
})
|
|
24
|
+
|
|
25
|
+
test('scorePluginCandidate ranks trusted reliable providers higher', () => {
|
|
26
|
+
const required = ['browser.navigation', 'browser.interaction', 'browser.screenshot', 'github.read']
|
|
27
|
+
const good = scorePluginCandidate({
|
|
28
|
+
id: 'good',
|
|
29
|
+
provides: required,
|
|
30
|
+
verified: true,
|
|
31
|
+
reliability: 0.95,
|
|
32
|
+
maintenance: 0.9,
|
|
33
|
+
community: 0.8,
|
|
34
|
+
risk: 'low',
|
|
35
|
+
}, required, createCapabilityBrain())
|
|
36
|
+
const bad = scorePluginCandidate({
|
|
37
|
+
id: 'bad',
|
|
38
|
+
provides: required,
|
|
39
|
+
verified: false,
|
|
40
|
+
reliability: 0.4,
|
|
41
|
+
maintenance: 0.2,
|
|
42
|
+
community: 0.1,
|
|
43
|
+
risk: 'high',
|
|
44
|
+
}, required, createCapabilityBrain())
|
|
45
|
+
assert.ok(good.score > bad.score)
|
|
46
|
+
})
|
|
47
|
+
|
|
48
|
+
test('overlap penalty lowers score', () => {
|
|
49
|
+
let brain = createCapabilityBrain()
|
|
50
|
+
brain = registerCapability(brain, { id: 'existing', capabilities: ['browser.navigation', 'browser.interaction'] })
|
|
51
|
+
const candidate = { id: 'dup', provides: ['browser.navigation', 'browser.interaction'], verified: true, reliability: 0.9 }
|
|
52
|
+
const lowOverlap = scorePluginCandidate(candidate, ['browser.navigation', 'browser.interaction', 'github.read'], brain, { overlap: 0 })
|
|
53
|
+
const highOverlap = scorePluginCandidate(candidate, ['browser.navigation', 'browser.interaction', 'github.read'], brain, { overlap: 1 })
|
|
54
|
+
assert.ok(lowOverlap.score > highOverlap.score)
|
|
55
|
+
})
|
|
56
|
+
|
|
57
|
+
test('installUtility includes overhead penalties', () => {
|
|
58
|
+
const candidate = { id: 'x', provides: ['test.run'], verified: true, reliability: 0.9, risk: 'low' }
|
|
59
|
+
const utility = installUtility(candidate, ['test.run'], createCapabilityBrain(), { taskFrequency: 2, latencyPenalty: 0.1, contextCost: 0.1 })
|
|
60
|
+
assert.equal(typeof utility.utility, 'number')
|
|
61
|
+
assert.match(formatCandidateScore(utility), /x/)
|
|
62
|
+
})
|
|
@@ -0,0 +1,51 @@
|
|
|
1
|
+
import test from 'node:test'
|
|
2
|
+
import assert from 'node:assert/strict'
|
|
3
|
+
|
|
4
|
+
import {
|
|
5
|
+
findRedundantProviders,
|
|
6
|
+
formatSolveResult,
|
|
7
|
+
isRedundant,
|
|
8
|
+
solveMinimalSet,
|
|
9
|
+
} from '../src/capability-solver.mjs'
|
|
10
|
+
import { createCapabilityBrain, registerCapability } from '../src/capability-brain.mjs'
|
|
11
|
+
|
|
12
|
+
test('solveMinimalSet prefers one comprehensive provider over many', () => {
|
|
13
|
+
const required = ['browser.navigation', 'browser.interaction', 'browser.screenshot', 'github.read']
|
|
14
|
+
const candidates = [
|
|
15
|
+
{ id: 'a', provides: ['browser.navigation', 'browser.interaction'], verified: true, reliability: 0.9, risk: 'low' },
|
|
16
|
+
{ id: 'b', provides: ['browser.screenshot'], verified: true, reliability: 0.9, risk: 'low' },
|
|
17
|
+
{ id: 'c', provides: ['browser.navigation', 'browser.interaction', 'browser.screenshot', 'github.read'], verified: true, reliability: 0.95, risk: 'low' },
|
|
18
|
+
{ id: 'd', provides: ['github.read'], verified: true, reliability: 0.9, risk: 'low' },
|
|
19
|
+
]
|
|
20
|
+
const result = solveMinimalSet(required, candidates, { maxPlugins: 2, minScore: 0.4 })
|
|
21
|
+
assert.ok(result.selected.some((c) => c.id === 'c'))
|
|
22
|
+
assert.equal(result.missing.length, 0)
|
|
23
|
+
assert.ok(result.pluginCount <= 2)
|
|
24
|
+
assert.match(formatSolveResult(result), /Minimal set/)
|
|
25
|
+
})
|
|
26
|
+
|
|
27
|
+
test('solveMinimalSet stops at maxPlugins and reports missing', () => {
|
|
28
|
+
const result = solveMinimalSet(
|
|
29
|
+
['a', 'b', 'c'],
|
|
30
|
+
[{ id: 'only-a', provides: ['a'], verified: true, reliability: 0.9 }],
|
|
31
|
+
{ maxPlugins: 1, minScore: 0.4 },
|
|
32
|
+
)
|
|
33
|
+
assert.equal(result.pluginCount, 1)
|
|
34
|
+
assert.deepEqual(result.missing, ['b', 'c'])
|
|
35
|
+
})
|
|
36
|
+
|
|
37
|
+
test('isRedundant detects providers fully covered by others', () => {
|
|
38
|
+
let brain = createCapabilityBrain()
|
|
39
|
+
brain = registerCapability(brain, { id: 'mega', capabilities: ['browser.navigation', 'browser.interaction', 'browser.screenshot'] })
|
|
40
|
+
brain = registerCapability(brain, { id: 'mini', capabilities: ['browser.navigation', 'browser.interaction'] })
|
|
41
|
+
assert.equal(isRedundant({ id: 'mini', provides: ['browser.navigation', 'browser.interaction'] }, brain), true)
|
|
42
|
+
assert.equal(isRedundant({ id: 'mega', provides: ['browser.navigation', 'browser.interaction', 'browser.screenshot'] }, brain), false)
|
|
43
|
+
})
|
|
44
|
+
|
|
45
|
+
test('findRedundantProviders returns covered providers', () => {
|
|
46
|
+
let brain = createCapabilityBrain()
|
|
47
|
+
brain = registerCapability(brain, { id: 'mega', capabilities: ['browser.navigation', 'browser.interaction', 'browser.screenshot'] })
|
|
48
|
+
brain = registerCapability(brain, { id: 'mini', capabilities: ['browser.navigation', 'browser.interaction'] })
|
|
49
|
+
const redundant = findRedundantProviders(brain)
|
|
50
|
+
assert.ok(redundant.some((r) => r.id === 'mini'))
|
|
51
|
+
})
|