ennodia 0.2.0 → 0.3.0-rc.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +75 -119
- package/docs/evidence/building-ennodia.md +66 -0
- package/docs/evidence/docs-drift.md +73 -0
- package/docs/evidence/index.md +18 -0
- package/docs/evidence/measurement.md +40 -0
- package/docs/evidence/skill-trial.md +68 -0
- package/docs/getting-started.md +48 -25
- package/docs/guides/agent-skills.md +22 -8
- package/docs/guides/budgets-and-limits.md +1 -1
- package/docs/guides/pragmatic-mode.md +85 -0
- package/docs/guides/recipes.md +56 -0
- package/docs/guides/troubleshooting.md +48 -0
- package/docs/guides/understand-results.md +41 -0
- package/docs/index.md +27 -90
- package/docs/install.md +49 -16
- package/docs/reference/controlled-english.md +2 -3
- package/docs/reference/ennodia-io.md +27 -4
- package/docs/reference/mcp-tools.md +61 -8
- package/docs/reference/supported-harnesses.md +40 -1
- package/docs/roadmap.md +25 -0
- package/examples/docs-drift/README.md +8 -0
- package/examples/docs-drift/archive/migration.md +4 -0
- package/examples/docs-drift/check.ts +7 -0
- package/examples/docs-drift/config.ts +8 -0
- package/examples/docs-drift/demo.ts +54 -0
- package/examples/docs-drift/recorded-run.json +16 -0
- package/examples/skill-trial/README.md +8 -0
- package/examples/skill-trial/check.ts +15 -0
- package/examples/skill-trial/contract.md +7 -0
- package/examples/skill-trial/recorded-trial.json +42 -0
- package/examples/skill-trial/retry.ts +4 -0
- package/package.json +5 -2
- package/skills/pragmatic/SKILL.md +23 -0
- package/src/advisor.ts +4 -0
- package/src/budget.ts +12 -8
- package/src/cli.ts +3 -0
- package/src/compare.ts +13 -12
- package/src/compositional.ts +10 -4
- package/src/core.ts +53 -47
- package/src/diagnosis.ts +17 -3
- package/src/harnesses.ts +88 -29
- package/src/history.ts +4 -1
- package/src/index.ts +1 -0
- package/src/plan-advice.ts +5 -2
- package/src/planner.ts +24 -1
- package/src/pragmatic.ts +59 -0
- package/src/process.ts +18 -0
- package/src/runs.ts +19 -1
- package/src/server.ts +17 -13
- package/src/tasks.ts +106 -18
- package/src/version.ts +1 -1
package/README.md
CHANGED
|
@@ -1,156 +1,112 @@
|
|
|
1
|
-
<
|
|
2
|
-
|
|
3
|
-
<
|
|
4
|
-
<picture>
|
|
5
|
-
<source media="(prefers-color-scheme: dark)" srcset="https://github.com/cherninlab/ennodia/raw/main/docs/assets/logo-dark.svg">
|
|
6
|
-
<source media="(prefers-color-scheme: light)" srcset="https://github.com/cherninlab/ennodia/raw/main/docs/assets/logo.svg">
|
|
7
|
-
<img alt="Ennodia" src="https://github.com/cherninlab/ennodia/raw/main/docs/assets/logo.svg" width="235" height="50">
|
|
1
|
+
<picture>
|
|
2
|
+
<source media="(prefers-color-scheme: dark)" srcset="docs/assets/logo-dark.svg">
|
|
3
|
+
<img alt="Ennodia" src="docs/assets/logo.svg" width="235" height="50">
|
|
8
4
|
</picture>
|
|
9
|
-
</a>
|
|
10
5
|
|
|
11
|
-
|
|
6
|
+
**Keep your agent. Get another way forward.**
|
|
12
7
|
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
<a href="https://smithery.ai/servers/cherninlab/ennodia"><img alt="Smithery: Ennodia" src="https://img.shields.io/badge/Smithery-Ennodia-FF5601"></a>
|
|
17
|
-
</p>
|
|
8
|
+
When a task stalls, Ennodia lets your primary artificial intelligence (AI) agent try another agent, model, or skill.
|
|
9
|
+
The findings come back to the same conversation.
|
|
10
|
+
You keep your familiar interface and decide what to try next.
|
|
18
11
|
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
|
|
12
|
+
```text
|
|
13
|
+
Use Ennodia MCP to get a second opinion on this bug.
|
|
14
|
+
Give one available agent the failing test and our attempted fixes.
|
|
15
|
+
Request a different explanation and a way to test it.
|
|
16
|
+
Bring the findings back here.
|
|
17
|
+
```
|
|
22
18
|
|
|
23
|
-
|
|
19
|
+
## When it helps
|
|
24
20
|
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
|
|
21
|
+
- A task is stuck and you want to check another approach.
|
|
22
|
+
- A skill looks useful, but you want to try it in a separate task first.
|
|
23
|
+
- Two answers disagree and you need to identify the next check.
|
|
24
|
+
- An unsuccessful attempt can help you avoid repeating the same approach under the same conditions.
|
|
28
25
|
|
|
29
|
-
|
|
30
|
-
|
|
26
|
+
Your main agent checks the findings and integrates useful work.
|
|
27
|
+
Additional attempts consume time and resources. General time or token savings have not been established.
|
|
31
28
|
|
|
32
29
|
## Install
|
|
33
30
|
|
|
34
|
-
Send this address to your primary agent
|
|
35
|
-
installation:
|
|
31
|
+
Send this address to your primary agent:
|
|
36
32
|
|
|
37
33
|
```text
|
|
38
34
|
try-ennodia.cherninlab.com
|
|
39
35
|
```
|
|
40
|
-
To run Ennodia directly as a standard input/output (stdio) MCP server, use:
|
|
41
36
|
|
|
42
|
-
|
|
43
|
-
|
|
37
|
+
For manual setup, add this Model Context Protocol (MCP) server to your client:
|
|
38
|
+
|
|
39
|
+
```json
|
|
40
|
+
{
|
|
41
|
+
"mcpServers": {
|
|
42
|
+
"ennodia": {
|
|
43
|
+
"command": "npx",
|
|
44
|
+
"args": ["-y", "ennodia"]
|
|
45
|
+
}
|
|
46
|
+
}
|
|
47
|
+
}
|
|
44
48
|
```
|
|
45
|
-
To use Bun directly, run `bunx ennodia`.
|
|
46
49
|
|
|
47
|
-
|
|
48
|
-
|
|
50
|
+
Requirements: Bun `1.3.14` or newer, a compatible MCP client, and a supported agent with working provider access.
|
|
51
|
+
`npx` downloads Ennodia. Bun runs it. You can also use `bunx ennodia`.
|
|
49
52
|
|
|
50
|
-
|
|
51
|
-
[
|
|
53
|
+
[Get your first result](https://ennodia.cherninlab.com/docs/getting-started/) ·
|
|
54
|
+
[Installation for agents](https://ennodia.cherninlab.com/docs/install/)
|
|
52
55
|
|
|
53
|
-
##
|
|
56
|
+
## Use your installed agents and skills
|
|
54
57
|
|
|
55
|
-
|
|
56
|
-
|
|
57
|
-
|
|
58
|
-
|
|
59
|
-
- Shows status, elapsed time, logs, and failures
|
|
60
|
-
- Cancels tasks and runs explicitly
|
|
61
|
-
- Lets a Plan Advisor propose an inert, validated work plan
|
|
62
|
-
- Compares multiple completed outputs with a Judge
|
|
63
|
-
- Lets a Result Advisor combine the Judge findings into one answer
|
|
58
|
+
Supported agents: Codex CLI, Claude Code, OpenCode, Kilo Code, Kiro CLI, Cline, Hermes Agent, and Antigravity.
|
|
59
|
+
Model availability and tool permissions depend on the installed agent and its provider configuration.
|
|
60
|
+
Discovery finds executable commands. It does not verify authentication or model access.
|
|
61
|
+
Codex workers use a read-only sandbox by default.
|
|
64
62
|
|
|
65
|
-
|
|
66
|
-
|
|
63
|
+
Ennodia discovers native Agent Skills and can request them for specific tasks.
|
|
64
|
+
A separate task keeps the trial outside the main conversation.
|
|
65
|
+
Native agents can also select unrequested skills from their own environment.
|
|
67
66
|
|
|
68
|
-
|
|
69
|
-
|
|
67
|
+
[Recipes](https://ennodia.cherninlab.com/docs/guides/recipes/) ·
|
|
68
|
+
[Using skills](https://ennodia.cherninlab.com/docs/guides/agent-skills/) ·
|
|
69
|
+
[Understand results](https://ennodia.cherninlab.com/docs/guides/understand-results/)
|
|
70
70
|
|
|
71
|
-
|
|
72
|
-
minutes. Compare adds two serial model passes after the child agents finish.
|
|
71
|
+
## See it in practice
|
|
73
72
|
|
|
74
|
-
|
|
73
|
+
- [Building Ennodia with Ennodia](https://ennodia.cherninlab.com/docs/evidence/building-ennodia/): a worker patch, a failed login, and primary-agent corrections.
|
|
74
|
+
- [Find stale documentation](https://ennodia.cherninlab.com/docs/evidence/docs-drift/): a small review with known answers.
|
|
75
|
+
- [Try a skill separately](https://ennodia.cherninlab.com/docs/evidence/skill-trial/): two attempts against the same contract.
|
|
75
76
|
|
|
76
|
-
|
|
77
|
+
The existing saved-answer benchmark is a regression check, not a live speed or cost comparison.
|
|
78
|
+
[Measurement plan](https://ennodia.cherninlab.com/docs/evidence/measurement/)
|
|
77
79
|
|
|
78
|
-
|
|
79
|
-
harness, model, and skill assignments as inert data. Ennodia validates the
|
|
80
|
-
proposal against a frozen inventory. Plan Advisor cannot execute it. The
|
|
81
|
-
caller must use a separate call to start the validated plan once.
|
|
82
|
-
- **Judge** runs during Compare and maps agreements, contradictions, unique
|
|
83
|
-
evidence, omissions, and risks across completed outputs.
|
|
84
|
-
- **Result Advisor** follows the Judge. It uses the Judge findings and original
|
|
85
|
-
outputs to recommend an answer. If Judge analysis is not usable, it uses
|
|
86
|
-
`basis: "candidates-only"`. This status makes the degradation visible.
|
|
80
|
+
## Data and limits
|
|
87
81
|
|
|
88
|
-
|
|
89
|
-
|
|
90
|
-
|
|
91
|
-
Transfer Protocol (HTTP) and TypeScript interface. Apps can use it with
|
|
92
|
-
user-supplied keys and installed local agents:
|
|
93
|
-
|
|
94
|
-
```sh
|
|
95
|
-
npx -y @cherninlab/ennodia-io
|
|
96
|
-
```
|
|
82
|
+
Ennodia runs locally without an Ennodia-hosted service.
|
|
83
|
+
Selected agents can send material to their configured model providers.
|
|
84
|
+
Terminal run history is stored locally and can be disabled with `ENNODIA_HISTORY=0`.
|
|
97
85
|
|
|
98
|
-
|
|
99
|
-
|
|
100
|
-
|
|
86
|
+
Ennodia can limit child tasks and estimated input tokens.
|
|
87
|
+
Those estimates exclude internal harness work and are not provider bills.
|
|
88
|
+
Automatic agent updates and recommendations learned from past results remain future work.
|
|
101
89
|
|
|
102
|
-
|
|
90
|
+
[Data governance](https://ennodia.cherninlab.com/docs/concepts/data-governance/) ·
|
|
91
|
+
[Budgets](https://ennodia.cherninlab.com/docs/guides/budgets-and-limits/) ·
|
|
92
|
+
[Roadmap](https://ennodia.cherninlab.com/docs/roadmap/)
|
|
103
93
|
|
|
104
|
-
|
|
105
|
-
- Claude Code
|
|
106
|
-
- OpenCode
|
|
107
|
-
- Kilo Code
|
|
108
|
-
- Kiro CLI
|
|
109
|
-
- Cline CLI
|
|
110
|
-
- Hermes Agent
|
|
111
|
-
- Antigravity
|
|
94
|
+
## Technical reference and contribution
|
|
112
95
|
|
|
113
|
-
|
|
114
|
-
|
|
96
|
+
The main entrypoint is `ennodia_run`. The optional Plan Advisor proposes explicit tasks before execution.
|
|
97
|
+
Compare examines completed answers and returns advice with its evidence and limitations.
|
|
115
98
|
|
|
116
|
-
|
|
117
|
-
|
|
118
|
-
|
|
119
|
-
Ennodia can add a candidate after verification of a supported prompt-in and
|
|
120
|
-
text-out interface. Verification must not use permission-bypass flags or
|
|
121
|
-
provider-private APIs.
|
|
122
|
-
|
|
123
|
-
## Documentation
|
|
124
|
-
|
|
125
|
-
- [Installation for Agents](https://ennodia.cherninlab.com/docs/install/): agent-controlled installation
|
|
126
|
-
- [Quickstart](https://ennodia.cherninlab.com/docs/getting-started/): manual installation and local development
|
|
127
|
-
- [MCP Tools](https://ennodia.cherninlab.com/docs/reference/mcp-tools/): full tool parameter reference
|
|
128
|
-
- [How Ennodia Works](https://ennodia.cherninlab.com/docs/concepts/how-ennodia-works/): orchestration pipeline
|
|
129
|
-
- [Second Opinions](https://ennodia.cherninlab.com/docs/concepts/second-opinions/): replicate, decompose, and red-team patterns
|
|
130
|
-
- [Data Governance](https://ennodia.cherninlab.com/docs/concepts/data-governance/): local storage and data movement boundaries
|
|
131
|
-
- [Controlled English](https://ennodia.cherninlab.com/docs/reference/controlled-english/): ASD-STE100 rules and Ennodia technical terms
|
|
132
|
-
- [Comparisons](https://ennodia.cherninlab.com/docs/comparisons/): comparisons with adjacent tools
|
|
133
|
-
- [Benchmarks](https://ennodia.cherninlab.com/docs/reference/benchmarks/): deterministic bug-recall results
|
|
134
|
-
- [Better Audits](https://ennodia.cherninlab.com/docs/guides/running-better-audits/): prompt rubrics for Compare
|
|
135
|
-
|
|
136
|
-
## Benchmarks
|
|
137
|
-
|
|
138
|
-
The current benchmark is `multi-model-bug-recall`. It uses small TypeScript
|
|
139
|
-
review fixtures and committed bug oracles. Run the deterministic suite with:
|
|
99
|
+
[MCP tools](https://ennodia.cherninlab.com/docs/reference/mcp-tools/) ·
|
|
100
|
+
[Experimental Ennodia IO](https://ennodia.cherninlab.com/docs/reference/ennodia-io/) ·
|
|
101
|
+
[Contributing](CONTRIBUTING.md) · [MIT license](LICENSE)
|
|
140
102
|
|
|
141
103
|
```sh
|
|
142
|
-
bun
|
|
104
|
+
bun install
|
|
105
|
+
bun run verify
|
|
143
106
|
```
|
|
144
107
|
|
|
145
|
-
|
|
146
|
-
kept out of `bun run verify`.
|
|
147
|
-
|
|
148
|
-
The current dated fixture snapshot is published in
|
|
149
|
-
[Benchmarks](https://ennodia.cherninlab.com/docs/reference/benchmarks/): 4
|
|
150
|
-
cases, with `ennodia-parallel-compare` at 100% recall and 100% precision.
|
|
151
|
-
|
|
152
|
-
## Contribute
|
|
108
|
+
### Experimental Pragmatic mode
|
|
153
109
|
|
|
154
|
-
|
|
155
|
-
|
|
156
|
-
|
|
110
|
+
In `ennodia@next`, delegate bounded investigation or patch proposals to an explicitly selected model.
|
|
111
|
+
[Pragmatic mode](docs/guides/pragmatic-mode.md) keeps bulky evidence outside the main conversation and returns findings for verification.
|
|
112
|
+
Savings are not yet validated.
|
|
@@ -0,0 +1,66 @@
|
|
|
1
|
+
---
|
|
2
|
+
title: Building Ennodia with Ennodia
|
|
3
|
+
description: A real development case with a failed launch, a useful worker patch, and primary-agent review.
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# Building Ennodia with Ennodia
|
|
7
|
+
|
|
8
|
+
Development record: September 6, 2026. These changes belong to the next release candidate.
|
|
9
|
+
|
|
10
|
+
A local server could accept browser requests that started agent work without an intended origin check.
|
|
11
|
+
The primary agent had a reproduced defect and a focused implementation task.
|
|
12
|
+
It stayed in Codex and used Ennodia to dispatch the work.
|
|
13
|
+
|
|
14
|
+
## What happened
|
|
15
|
+
|
|
16
|
+
| Attempt | Observed result | Next action |
|
|
17
|
+
| --- | --- | --- |
|
|
18
|
+
| Claude Code with Sonnet | Authentication failed before the task ran. | Use another configured agent. |
|
|
19
|
+
| OpenCode with its configured model | Reproduced the problem, edited four files, and added seven regression tests. | Review the patch in the primary conversation. |
|
|
20
|
+
| Primary agent | Tightened malformed Host and Origin validation and preserved the native request size limit. | Run the combined verification suite. |
|
|
21
|
+
|
|
22
|
+
The worker tested both the request handler and a real listening server.
|
|
23
|
+
It also found that Bun can return an empty oversized-request error before the handler runs.
|
|
24
|
+
Its proposed size-limit adjustment could not guarantee a structured response for every oversized request.
|
|
25
|
+
The primary agent kept the hard limit and documented the actual behavior.
|
|
26
|
+
|
|
27
|
+
The combined suite passed after integration.
|
|
28
|
+
This demonstrates a useful contribution that still needed review and adaptation.
|
|
29
|
+
It does not establish how long the primary agent alone would have taken.
|
|
30
|
+
|
|
31
|
+
## What did not help
|
|
32
|
+
|
|
33
|
+
A separate Compare review used a smaller GPT model with the rigorous-review skill.
|
|
34
|
+
It reached the ten-minute task limit without returning a usable proposal.
|
|
35
|
+
The primary agent completed those fixes locally.
|
|
36
|
+
The record includes that failed attempt, rather than treating every delegation as a success.
|
|
37
|
+
|
|
38
|
+
The Claude failure indicates an expired login, not model quality.
|
|
39
|
+
The GPT attempt does not establish that the model or skill is generally unsuitable.
|
|
40
|
+
Its observed result applies to this task, configuration, and time limit.
|
|
41
|
+
|
|
42
|
+
A later broad documentation assignment through OpenCode also reached its ten-minute limit without producing edits.
|
|
43
|
+
The primary agent completed those pages directly.
|
|
44
|
+
This suggests testing smaller assignments and clearer execution controls. It does not establish a general provider limitation.
|
|
45
|
+
|
|
46
|
+
## Inspect and reproduce
|
|
47
|
+
|
|
48
|
+
The implementation and regressions are in the open repository:
|
|
49
|
+
|
|
50
|
+
- [HTTP implementation](https://github.com/cherninlab/ennodia/blob/main/packages/ennodia-io/src/io.ts)
|
|
51
|
+
- [HTTP regressions](https://github.com/cherninlab/ennodia/blob/main/packages/ennodia-io/src/io.test.ts)
|
|
52
|
+
|
|
53
|
+
Use the release candidate checkout that contains this page.
|
|
54
|
+
|
|
55
|
+
```sh
|
|
56
|
+
bun install
|
|
57
|
+
bun test packages/ennodia-io/src/io.test.ts
|
|
58
|
+
bun run verify
|
|
59
|
+
```
|
|
60
|
+
|
|
61
|
+
The tests use local fixtures. They do not call model providers.
|
|
62
|
+
A live repeat requires an authenticated supported agent.
|
|
63
|
+
Save the worker output and the primary agent’s final diff when repeating the experiment.
|
|
64
|
+
|
|
65
|
+
The practical benefit here was a reviewed contribution from another tool without moving the primary conversation.
|
|
66
|
+
No token-saving or speed comparison was measured.
|
|
@@ -0,0 +1,73 @@
|
|
|
1
|
+
---
|
|
2
|
+
title: Find stale documentation
|
|
3
|
+
description: Review current documentation without deleting useful historical records.
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# Find stale documentation
|
|
7
|
+
|
|
8
|
+
A current guide can disagree with code while a historical document remains correct for its period.
|
|
9
|
+
This exercise gives an additional agent a bounded review task.
|
|
10
|
+
|
|
11
|
+
The fixture lives in `examples/docs-drift` in the Ennodia checkout.
|
|
12
|
+
It contains deliberate documentation errors and a labeled migration record.
|
|
13
|
+
|
|
14
|
+
## Try it
|
|
15
|
+
|
|
16
|
+
The recording below shows an actual MCP run from the release candidate checkout.
|
|
17
|
+
The final frame includes ten extra seconds for reading.
|
|
18
|
+
|
|
19
|
+
<video controls preload="metadata" aria-label="Recorded Ennodia documentation review" style="width:100%; height:auto;" src="/demos/docs-drift.mp4"></video>
|
|
20
|
+
|
|
21
|
+
The [terminal recording](/demos/docs-drift.cast) preserves the captured output and timing.
|
|
22
|
+
The text below provides the same findings without video.
|
|
23
|
+
|
|
24
|
+
1. Use a local checkout of Ennodia.
|
|
25
|
+
2. Run the fixture’s behavior check.
|
|
26
|
+
|
|
27
|
+
```sh
|
|
28
|
+
bun examples/docs-drift/check.ts
|
|
29
|
+
```
|
|
30
|
+
|
|
31
|
+
3. Give this request to your primary agent.
|
|
32
|
+
|
|
33
|
+
```text
|
|
34
|
+
Use Ennodia MCP to review examples/docs-drift in a separate read-only task.
|
|
35
|
+
Compare the current README with config.ts. Identify each conflicting claim.
|
|
36
|
+
Distinguish current instructions from the labeled historical migration record.
|
|
37
|
+
Return proposed corrections with file references. Bring the findings back here.
|
|
38
|
+
```
|
|
39
|
+
|
|
40
|
+
4. Review each proposed correction against the source.
|
|
41
|
+
5. Update the current guide in your copy.
|
|
42
|
+
6. Keep the historical migration record labeled as history.
|
|
43
|
+
|
|
44
|
+
## Check the result
|
|
45
|
+
|
|
46
|
+
The current implementation uses port `4545`, the variable `APP_PORT`, and the response `{ "ready": true }`.
|
|
47
|
+
The README contains three conflicting current claims.
|
|
48
|
+
The migration record describes the previous release and does not need the same changes.
|
|
49
|
+
|
|
50
|
+
A useful review finds the three conflicts and preserves the historical distinction.
|
|
51
|
+
A model can still miss a conflict or recommend deleting useful history.
|
|
52
|
+
This exercise has known answers. It is not a general benchmark.
|
|
53
|
+
|
|
54
|
+
## Recorded development trial
|
|
55
|
+
|
|
56
|
+
On September 6, 2026, an Antigravity task through Ennodia found all three current documentation conflicts.
|
|
57
|
+
It also identified the archive as a historical record that did not need changes.
|
|
58
|
+
The primary agent checked each finding against the fixture source.
|
|
59
|
+
This is one successful trial on a small known-answer example, not a general performance result.
|
|
60
|
+
|
|
61
|
+
The fixture includes `recorded-run.json` with the answer, task IDs, timing, and verification notes.
|
|
62
|
+
Local file URL prefixes were normalized for sharing.
|
|
63
|
+
|
|
64
|
+
The video records a later successful repetition through the candidate's MCP server.
|
|
65
|
+
It uses the configured default Antigravity model, with no model override.
|
|
66
|
+
The opt-in script starts one real agent using your configured provider access:
|
|
67
|
+
|
|
68
|
+
```sh
|
|
69
|
+
bun examples/docs-drift/demo.ts --live antigravity
|
|
70
|
+
```
|
|
71
|
+
|
|
72
|
+
Replace `antigravity` with another installed, authenticated harness ID when needed.
|
|
73
|
+
The recording uses the [asciicast format](https://docs.asciinema.org/manual/asciicast/v2/).
|
|
@@ -0,0 +1,18 @@
|
|
|
1
|
+
---
|
|
2
|
+
title: Examples and evidence
|
|
3
|
+
description: Try small examples and inspect what Ennodia contributed to its own development.
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# Examples and evidence
|
|
7
|
+
|
|
8
|
+
Ennodia helps your primary agent try another approach and bring the findings back.
|
|
9
|
+
These examples show the workflow and its limits.
|
|
10
|
+
|
|
11
|
+
- [Building Ennodia with Ennodia](./building-ennodia.md) follows a real development task, including an authentication failure and the primary agent’s corrections.
|
|
12
|
+
- [Find stale documentation](./docs-drift.md) provides a small reproducible review exercise.
|
|
13
|
+
- [Try a skill separately](./skill-trial.md) compares two reviews of the same function.
|
|
14
|
+
- [Measurement plan](./measurement.md) defines the next comparative experiment.
|
|
15
|
+
|
|
16
|
+
The examples are not a speed or cost benchmark.
|
|
17
|
+
The [existing deterministic benchmark](../reference/benchmarks.md) checks saved answers against known findings.
|
|
18
|
+
It does not measure live task completion time.
|
|
@@ -0,0 +1,40 @@
|
|
|
1
|
+
---
|
|
2
|
+
title: How we will measure usefulness
|
|
3
|
+
description: A planned comparison of complete tasks, including unsuccessful attempts and human intervention.
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# How we will measure usefulness
|
|
7
|
+
|
|
8
|
+
Status: proposed experiment. Results are not available yet.
|
|
9
|
+
|
|
10
|
+
The primary comparison uses the same main agent with and without Ennodia.
|
|
11
|
+
A separate control can give the main agent the same additional models without Ennodia.
|
|
12
|
+
That control helps distinguish orchestration benefits from additional computation.
|
|
13
|
+
|
|
14
|
+
## Pilot design
|
|
15
|
+
|
|
16
|
+
- Select 20 tasks before running the experiment.
|
|
17
|
+
- Repeat each condition three times.
|
|
18
|
+
- Fix models, versions, source material, available skills, and resource limits.
|
|
19
|
+
- Define correctness checks before collecting answers.
|
|
20
|
+
- Keep evaluator answers outside agent materials.
|
|
21
|
+
- Retain failed, canceled, and incomplete attempts.
|
|
22
|
+
|
|
23
|
+
| Measure | Definition |
|
|
24
|
+
| --- | --- |
|
|
25
|
+
| Correctness | Fraction of tasks meeting the predefined checks. |
|
|
26
|
+
| Time | Elapsed time through planning, workers, comparison, verification, and integration. |
|
|
27
|
+
| Resources | Reported input, output, and cached tokens across all participants. Unknown usage stays unknown. |
|
|
28
|
+
| Human work | Interventions, manual transfers, and repeated setup steps. |
|
|
29
|
+
| Repeated attempts | Reuse of a previously unsuccessful approach under the same recorded conditions. |
|
|
30
|
+
|
|
31
|
+
Compare time together with correctness. Excluding failures from timing can hide a worse completion rate.
|
|
32
|
+
Report model tokens, estimated provider charges, and subscription payments separately.
|
|
33
|
+
A token estimate is not a measured provider bill.
|
|
34
|
+
|
|
35
|
+
[Terminal-Bench](https://github.com/harbor-framework/terminal-bench) is a candidate for terminal tasks.
|
|
36
|
+
[SWE-bench](https://github.com/SWE-bench/SWE-bench) provides a separate direction for repository bug fixes.
|
|
37
|
+
Pin the exact dataset version and evaluation environment.
|
|
38
|
+
Use small custom fixtures for skill trials and manual workflow measurements.
|
|
39
|
+
|
|
40
|
+
This pilot can reveal useful patterns. It cannot establish universal superiority.
|
|
@@ -0,0 +1,68 @@
|
|
|
1
|
+
---
|
|
2
|
+
title: Try a skill separately
|
|
3
|
+
description: Compare two reviews of one function while keeping the main conversation focused.
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# Try a skill separately
|
|
7
|
+
|
|
8
|
+
A skill can improve a review, add irrelevant work, or make no difference.
|
|
9
|
+
A small separate task gives you evidence before you use its recommendations.
|
|
10
|
+
|
|
11
|
+
The fixture lives in `examples/skill-trial` in the Ennodia checkout.
|
|
12
|
+
It contains a short function and a local contract with deliberate mismatches.
|
|
13
|
+
|
|
14
|
+
## Prepare two comparable attempts
|
|
15
|
+
|
|
16
|
+
1. Select one available agent and model.
|
|
17
|
+
2. Check that the requested skill is installed for that agent.
|
|
18
|
+
3. Give both attempts the same `retry.ts` and `contract.md` files.
|
|
19
|
+
4. Keep the evaluator `check.ts` out of the review materials.
|
|
20
|
+
5. Run the plain review and skill review as separate read-only tasks.
|
|
21
|
+
|
|
22
|
+
```text
|
|
23
|
+
Use Ennodia MCP for two separate read-only reviews of examples/skill-trial/retry.ts.
|
|
24
|
+
Provide contract.md as the source of truth. Use the same agent and model.
|
|
25
|
+
First run: request no additional skill.
|
|
26
|
+
Second run: request the installed source-grounded-audit skill.
|
|
27
|
+
Do not open check.ts until both reviews finish.
|
|
28
|
+
Return each contract mismatch with a concrete failing input.
|
|
29
|
+
Compare useful findings, unsupported claims, and time spent.
|
|
30
|
+
```
|
|
31
|
+
|
|
32
|
+
Your agent can use `ennodia_start_compositional` with one slice per attempt.
|
|
33
|
+
Set `skillIds: []` on the plain slice and `skillIds: ["source-grounded-audit"]` on the skill slice.
|
|
34
|
+
The [skill guide](/docs/guides/agent-skills/) describes installation and explicit selection.
|
|
35
|
+
|
|
36
|
+
A native agent can self-select other available skills.
|
|
37
|
+
An empty requested skill list does not guarantee a skill-free control.
|
|
38
|
+
For strict measurement, use separate controlled skill inventories and record the skills actually loaded.
|
|
39
|
+
|
|
40
|
+
## Validate the findings
|
|
41
|
+
|
|
42
|
+
Run the evaluator after saving both reviews:
|
|
43
|
+
|
|
44
|
+
```sh
|
|
45
|
+
bun examples/skill-trial/check.ts
|
|
46
|
+
```
|
|
47
|
+
|
|
48
|
+
The supplied implementation deliberately fails some cases.
|
|
49
|
+
Use each reported input to check a proposed fix in a disposable copy.
|
|
50
|
+
A useful finding needs to explain a contract mismatch, not just suggest a stylistic change.
|
|
51
|
+
Keep useful partial findings even if the skill adds other irrelevant advice.
|
|
52
|
+
|
|
53
|
+
This fixture supports a repeatable workflow demonstration.
|
|
54
|
+
No quality improvement is claimed for it.
|
|
55
|
+
|
|
56
|
+
## Recorded development trial
|
|
57
|
+
|
|
58
|
+
On September 6, 2026, two Antigravity tasks ran through Ennodia in separate temporary copies.
|
|
59
|
+
The plain review found the six failing evaluator cases and an additional alternative-number-notation case.
|
|
60
|
+
The requested-skill attempt produced no answer because native tool permissions blocked a required read.
|
|
61
|
+
|
|
62
|
+
The agent command exited with code zero despite the permission denial.
|
|
63
|
+
Ennodia initially labeled that task successful. This trial exposed a classification defect that the next release candidate fixes.
|
|
64
|
+
The denied attempt cannot establish the quality of the skill.
|
|
65
|
+
No permission bypass was used to force a successful result.
|
|
66
|
+
|
|
67
|
+
The fixture includes `recorded-trial.json` with both observed outcomes and the diagnostic excerpt.
|
|
68
|
+
Local paths were normalized, and CLI remediation instructions were omitted from the excerpt.
|
package/docs/getting-started.md
CHANGED
|
@@ -1,22 +1,36 @@
|
|
|
1
1
|
---
|
|
2
2
|
title: Quickstart
|
|
3
|
-
description:
|
|
3
|
+
description: Give your agent the Ennodia setup link, get your first useful result, or follow the manual installation instructions.
|
|
4
4
|
---
|
|
5
5
|
|
|
6
|
-
|
|
7
|
-
(MCP) client. To use an agent for installation and configuration, see
|
|
8
|
-
[Installation for Agents](/docs/install/).
|
|
6
|
+
Paste this address into a new or existing conversation with your agent:
|
|
9
7
|
|
|
10
|
-
|
|
8
|
+
```text
|
|
9
|
+
try-ennodia.cherninlab.com
|
|
10
|
+
```
|
|
11
|
+
|
|
12
|
+
Your agent can follow the setup instructions, check the requirements, and connect Ennodia to your client.
|
|
13
|
+
Continue in your preferred compatible chat app, command-line interface (CLI), or editor.
|
|
14
|
+
Ennodia connects through Model Context Protocol (MCP).
|
|
15
|
+
|
|
16
|
+
The [installation instructions for agents](/docs/install/) explain the setup steps.
|
|
17
|
+
Your agent will identify any client restart or reload needed to make the new tools available.
|
|
18
|
+
|
|
19
|
+
After setup, [try your first useful result](#your-first-useful-result).
|
|
20
|
+
For manual setup, follow the instructions below.
|
|
21
|
+
|
|
22
|
+
## Manual Setup
|
|
23
|
+
|
|
24
|
+
### Requirements
|
|
11
25
|
|
|
12
26
|
- Bun `1.3.14` or newer
|
|
13
27
|
- An MCP client that can launch a standard input/output (stdio) server
|
|
14
|
-
- At least one supported artificial intelligence (AI) command-line tool
|
|
15
|
-
|
|
28
|
+
- At least one supported artificial intelligence (AI) command-line tool with
|
|
29
|
+
working provider access
|
|
16
30
|
|
|
17
31
|
See [Supported Harnesses](/docs/reference/supported-harnesses/).
|
|
18
32
|
|
|
19
|
-
|
|
33
|
+
### Install from npm
|
|
20
34
|
|
|
21
35
|
Install the stable release from npm:
|
|
22
36
|
|
|
@@ -30,7 +44,7 @@ To use Bun directly, run `bunx ennodia`.
|
|
|
30
44
|
The JSR package `@cherninlab/ennodia` exposes TypeScript modules for import.
|
|
31
45
|
Use the npm package for the stdio MCP executable.
|
|
32
46
|
|
|
33
|
-
|
|
47
|
+
### Local Checkout
|
|
34
48
|
|
|
35
49
|
Use a checkout to modify Ennodia:
|
|
36
50
|
|
|
@@ -41,7 +55,7 @@ bun install
|
|
|
41
55
|
bun run verify
|
|
42
56
|
```
|
|
43
57
|
|
|
44
|
-
|
|
58
|
+
### MCP Client Config
|
|
45
59
|
|
|
46
60
|
```json
|
|
47
61
|
{
|
|
@@ -69,28 +83,37 @@ For a local checkout, use the source file:
|
|
|
69
83
|
|
|
70
84
|
Replace `/absolute/path/to/ennodia` with your local repository path.
|
|
71
85
|
|
|
72
|
-
|
|
86
|
+
Use your client's supported configuration method.
|
|
87
|
+
Preserve existing server entries.
|
|
88
|
+
Reload the tools or restart the client if its setup instructions require this step.
|
|
73
89
|
|
|
74
|
-
|
|
90
|
+
## Your first useful result
|
|
75
91
|
|
|
76
|
-
|
|
77
|
-
|
|
78
|
-
|
|
79
|
-
|
|
92
|
+
After installation, give your primary agent a small request:
|
|
93
|
+
|
|
94
|
+
```text
|
|
95
|
+
Use Ennodia MCP to get one additional opinion on this function.
|
|
96
|
+
Choose an available agent. Keep the review read-only.
|
|
97
|
+
Return one concrete issue or explain that no issue was established.
|
|
98
|
+
Bring the findings back to this conversation.
|
|
80
99
|
```
|
|
81
100
|
|
|
82
|
-
|
|
83
|
-
|
|
101
|
+
Start with one worker and a small set of files.
|
|
102
|
+
Discovery checks installed commands, not authentication, model access, or provider quota.
|
|
103
|
+
A live task establishes which configuration works in your environment.
|
|
104
|
+
|
|
105
|
+
Your agent can call `ennodia_list_harnesses`, then `ennodia_run` with a selected `harnessId`, `mode: "single"`, and `compare: false`.
|
|
106
|
+
Poll `ennodia_get_run` with the returned run identifier (ID).
|
|
107
|
+
Stop when the status is `succeeded`, `failed`, or `cancelled`.
|
|
84
108
|
|
|
85
|
-
|
|
86
|
-
|
|
87
|
-
|
|
109
|
+
Inspect the result before starting more work.
|
|
110
|
+
Use the [recipes](/docs/guides/recipes/) for a stuck task, a skill trial, or conflicting answers.
|
|
111
|
+
See [Understand Results](/docs/guides/understand-results/) when the attempt is incomplete or unhelpful.
|
|
88
112
|
|
|
89
|
-
|
|
90
|
-
|
|
113
|
+
Expect live runs to take time. A timeout budget is not a completion prediction.
|
|
114
|
+
Compare adds a Judge pass and a Result Advisor pass after child agents finish.
|
|
91
115
|
|
|
92
|
-
|
|
93
|
-
Advisor pass after child agents finish.
|
|
116
|
+
For a local development checkout, run `bun run verify` before relying on changes.
|
|
94
117
|
|
|
95
118
|
## Optional Team Advice
|
|
96
119
|
|