ennodia 0.1.0-rc.0

package/CONTRIBUTING.md

@@ -0,0 +1,21 @@
+# Contributing to Ennodia
+
+## Prerequisites
+
+- **Bun** 1.3.14+
+- Local AI tools are optional unless you are changing their adapters
+
+## Quickstart
+
+```sh
+bun install
+bun run verify
+```
+
+`verify` runs typechecking, Oxlint, unit tests, and local MCP smoke checks.
+
+Before changing release metadata or package contents, also run:
+
+```sh
+bun run release:check
+```

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026-present, Theo Chernin
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -0,0 +1,99 @@
+<div align="center">
+
+<a href="https://ennodia.cherninlab.com">
+<picture>
+  <source media="(prefers-color-scheme: dark)" srcset="docs/assets/logo-dark.svg">
+  <source media="(prefers-color-scheme: light)" srcset="docs/assets/logo.svg">
+  <img alt="Ennodia" src="docs/assets/logo.svg" width="235" height="50">
+</picture>
+</a>
+
+<p><strong>MCP server that lets one AI agent ask other agents for help</strong></p>
+
+<p>
+  <a href="LICENSE"><img alt="License: MIT" src="https://img.shields.io/badge/License-MIT-informational"></a>
+  <img alt="CI" src="https://github.com/cherninlab/ennodia/actions/workflows/ci.yml/badge.svg" />
+</p>
+
+</div>
+
+Ennodia is a local MCP server that lets one AI agent ask other agents for help while a task is still in progress. It routes the request, runs selected command line tools, tracks status and output, and can compare several answers before returning one result.
+
+It is built for workflows where no single model or agent should be trusted as the only reviewer.
+
+## Local Setup
+
+From a checkout:
+
+```sh
+git clone https://github.com/cherninlab/ennodia
+cd ennodia
+bun install
+bun run verify
+```
+
+After the npm package is published, the prerelease channel will be available as
+`bunx ennodia@next`. `npx` can also launch Ennodia, but only when `bun` is
+already available on `PATH`.
+
+## What Ennodia Does
+
+- Discovers available local AI tools
+- Plans which tool should handle a request
+- Starts and monitors child tasks
+- Shows status, timing, logs, and failures
+- Cancels tasks and runs explicitly
+- Compares multiple completed outputs
+- Synthesizes one answer from the comparison
+
+## Supported Harnesses
+
+Current adapters:
+
+- Codex CLI
+- Claude Code
+- OpenCode
+- Antigravity
+
+Adapters stay thin. Shared routing, tracing, task state, recovery, and Compare
+logic live in the core modules.
+
+## MCP Tools
+
+Common entrypoints:
+
+- `ennodia_list_harnesses` - show detected tools
+- `ennodia_plan` - preview routing for a prompt
+- `ennodia_start` - start direct child tasks without run-level synthesis
+- `ennodia_run` - plan, execute, optionally Compare, and return a final answer
+- `ennodia_get_run` - inspect run state, events, ETA, and final output
+- `ennodia_cancel_run` - cancel a running orchestration
+- `ennodia_start_compare` - compare completed task outputs or supplied responses
+
+Lower-level task tools are available for polling and cancellation.
+
+## Documentation
+
+- [Getting started](docs/getting-started.md)
+- [How Ennodia works](docs/in-depth/architecture.md)
+- [MCP tools](docs/reference/mcp-tools.md)
+- [Releasing Ennodia](docs/in-depth/releasing.md)
+
+## Benchmarks
+
+The first benchmark is `multi-model-bug-recall`: small TypeScript review
+fixtures scored against committed bug oracles. Run the deterministic fixture
+suite with:
+
+```sh
+bun run bench:bug-recall
+```
+
+Live harness runs are available through `bun run bench:bug-recall:live` and are
+kept out of `bun run verify`.
+
+## Contributing
+
+Ennodia is under active development. Bug reports and small, focused pull requests
+are welcome. See [CONTRIBUTING.md](CONTRIBUTING.md) for the local verification
+workflow.

package/bench/README.md

@@ -0,0 +1,32 @@
+# Benchmarks
+
+Ennodia's first release-candidate benchmark is `multi-model-bug-recall`: small
+TypeScript review fixtures with committed oracles. It measures whether
+parallel peer review plus Compare recovers more required findings than a solo
+harness answer.
+
+The default command is deterministic and uses committed reference outputs:
+
+```sh
+bun run bench:bug-recall
+```
+
+The live command starts a fresh Ennodia MCP server and runs real harnesses. It
+is intentionally not part of `bun run verify`.
+
+```sh
+bun run bench:bug-recall:live -- --fixture 001-missing-await
+```
+
+Useful flags:
+
+- `--fixture <id>` runs one fixture; repeat it to select several.
+- `--harness <id>` adds a solo baseline harness in live mode. Defaults to
+  `codex` and `claude-code`.
+- `--judge-harness <id>` and `--synthesizer-harness <id>` pin Compare roles.
+- `--timeout-ms <ms>` sets the per-run timeout for live harness calls.
+- `--out <dir>` writes results to a specific directory.
+
+Results are written under `bench/results/bug-recall/`. Raw benchmark output is
+gitignored by default because live runs include model output and environment
+metadata.

package/bench/bug-recall/run.ts

@@ -0,0 +1,331 @@
+import { mkdir, readdir, writeFile } from "node:fs/promises";
+import { basename, join } from "node:path";
+import { Client } from "@modelcontextprotocol/sdk/client/index.js";
+import { StdioClientTransport } from "@modelcontextprotocol/sdk/client/stdio.js";
+import {
+  buildPrompt,
+  DEFAULT_FIXTURES_ROOT,
+  formatConditionTable,
+  loadCases,
+  scoreBenchmark,
+  type BenchmarkCase,
+  type BenchmarkSummary,
+  type OutputRecord,
+} from "./scorer";
+import { ENNODIA_VERSION } from "../../src/version";
+
+type CliOptions = {
+  live: boolean;
+  fixtureIds: string[];
+  harnessIds: string[];
+  timeoutMs: number;
+  outDir?: string;
+  judgeHarnessId?: string;
+  synthesizerHarnessId?: string;
+};
+
+type McpTextContent = {
+  type: "text";
+  text: string;
+};
+
+type RunView = {
+  id: string;
+  status: "executing" | "comparing" | "succeeded" | "failed" | "cancelled";
+  finalAnswer?: string;
+  selectedHarnessIds?: string[];
+  taskIds?: string[];
+  compareId?: string;
+  elapsedMs?: number;
+  error?: string;
+};
+
+const DEFAULT_LIVE_HARNESSES = ["codex", "claude-code"];
+
+const options = parseArgs(Bun.argv.slice(2));
+const cases = await selectCases(await loadCases(), options.fixtureIds);
+const outputs = options.live
+  ? await runLiveBenchmark(cases, options)
+  : await loadFixtureOutputs(cases);
+const summary = await scoreBenchmark({
+  cases,
+  outputs,
+  mode: options.live ? "live" : "fixture",
+});
+const outDir = await resolveOutDir(options, summary);
+
+await writeSummary(outDir, summary, outputs);
+
+console.log(formatConditionTable(summary));
+console.log("");
+console.log(`Wrote benchmark results to ${outDir}`);
+
+async function loadFixtureOutputs(cases: BenchmarkCase[]): Promise<OutputRecord[]> {
+  const outputs: OutputRecord[] = [];
+
+  for (const benchmarkCase of cases) {
+    const responseDir = new URL(
+      `${benchmarkCase.id}/responses/`,
+      DEFAULT_FIXTURES_ROOT,
+    );
+    const entries = await readdir(responseDir, { withFileTypes: true });
+    const files = entries
+      .filter((entry) => entry.isFile() && entry.name.endsWith(".md"))
+      .map((entry) => entry.name)
+      .sort();
+
+    for (const file of files) {
+      outputs.push({
+        caseId: benchmarkCase.id,
+        condition: basename(file, ".md"),
+        text: await Bun.file(new URL(file, responseDir)).text(),
+        metadata: { mode: "fixture", source: file },
+      });
+    }
+  }
+
+  return outputs;
+}
+
+async function runLiveBenchmark(
+  cases: BenchmarkCase[],
+  options: CliOptions,
+): Promise<OutputRecord[]> {
+  const client = new Client({
+    name: "ennodia-benchmark",
+    version: ENNODIA_VERSION,
+  });
+  const transport = new StdioClientTransport({
+    command: "bun",
+    args: ["run", "src/index.ts"],
+    cwd: new URL("../../", import.meta.url).pathname,
+    stderr: "pipe",
+  });
+  await client.connect(transport);
+
+  try {
+    const outputs: OutputRecord[] = [];
+    const harnessIds = options.harnessIds.length > 0
+      ? options.harnessIds
+      : DEFAULT_LIVE_HARNESSES;
+
+    for (const benchmarkCase of cases) {
+      const prompt = buildPrompt(benchmarkCase);
+
+      for (const harnessId of harnessIds) {
+        const run = await startAndWaitForRun(client, {
+          prompt,
+          harnessId,
+          mode: "single",
+          compare: false,
+          timeoutMs: options.timeoutMs,
+        });
+        outputs.push(runToOutput(benchmarkCase.id, `${harnessId}-solo`, run));
+      }
+
+      const parallelRun = await startAndWaitForRun(client, {
+        prompt,
+        mode: "parallel",
+        compare: true,
+        timeoutMs: options.timeoutMs,
+        judgeHarnessId: options.judgeHarnessId,
+        synthesizerHarnessId: options.synthesizerHarnessId,
+        maxOutputChars: 16_000,
+      });
+      outputs.push(runToOutput(
+        benchmarkCase.id,
+        "ennodia-parallel-compare",
+        parallelRun,
+      ));
+    }
+
+    return outputs;
+  } finally {
+    await client.close();
+  }
+}
+
+async function startAndWaitForRun(
+  client: Client,
+  args: Record<string, unknown>,
+): Promise<RunView> {
+  const started = parseTextResult(await client.callTool({
+    name: "ennodia_run",
+    arguments: args,
+  })) as RunView;
+
+  let run = started;
+  while (run.status === "executing" || run.status === "comparing") {
+    await Bun.sleep(1_000);
+    run = parseTextResult(await client.callTool({
+      name: "ennodia_get_run",
+      arguments: {
+        runId: started.id,
+        includeEvents: true,
+        maxEvents: 80,
+        maxAnswerChars: 120_000,
+      },
+    })) as RunView;
+  }
+
+  if (run.status !== "succeeded") {
+    throw new Error(`Run ${started.id} ended as ${run.status}: ${run.error ?? ""}`);
+  }
+
+  return run;
+}
+
+function runToOutput(
+  caseId: string,
+  condition: string,
+  run: RunView,
+): OutputRecord {
+  return {
+    caseId,
+    condition,
+    text: run.finalAnswer ?? "",
+    metadata: {
+      mode: "live",
+      runId: run.id,
+      elapsedMs: run.elapsedMs,
+      selectedHarnessIds: run.selectedHarnessIds,
+      taskIds: run.taskIds,
+      compareId: run.compareId,
+    },
+  };
+}
+
+async function writeSummary(
+  outDir: string,
+  summary: BenchmarkSummary,
+  outputs: OutputRecord[],
+): Promise<void> {
+  await mkdir(outDir, { recursive: true });
+  await writeFile(
+    join(outDir, "summary.json"),
+    `${JSON.stringify({ ...summary, outputs }, null, 2)}\n`,
+  );
+  await writeFile(
+    join(outDir, "summary.md"),
+    [
+      `# ${summary.benchmark}`,
+      "",
+      `Mode: ${summary.mode}`,
+      "",
+      formatConditionTable(summary),
+      "",
+      "## Wins",
+      "",
+      ...summary.wins.map((win) =>
+        `- ${win.condition}: ${win.fixtures} fixture(s)`
+      ),
+      "",
+    ].join("\n"),
+  );
+}
+
+async function resolveOutDir(
+  options: CliOptions,
+  summary: BenchmarkSummary,
+): Promise<string> {
+  if (options.outDir) {
+    return options.outDir;
+  }
+
+  const stamp = summary.createdAt.replace(/[:.]/g, "-");
+  return join(
+    new URL("../results/bug-recall/", import.meta.url).pathname,
+    `${summary.mode}-${stamp}`,
+  );
+}
+
+async function selectCases(
+  cases: BenchmarkCase[],
+  fixtureIds: string[],
+): Promise<BenchmarkCase[]> {
+  if (fixtureIds.length === 0) {
+    return cases;
+  }
+
+  const selected = cases.filter((benchmarkCase) =>
+    fixtureIds.includes(benchmarkCase.id)
+  );
+  const missing = fixtureIds.filter((fixtureId) =>
+    !selected.some((benchmarkCase) => benchmarkCase.id === fixtureId)
+  );
+
+  if (missing.length > 0) {
+    throw new Error(`Unknown fixture id(s): ${missing.join(", ")}`);
+  }
+
+  return selected;
+}
+
+function parseArgs(args: string[]): CliOptions {
+  const options: CliOptions = {
+    live: false,
+    fixtureIds: [],
+    harnessIds: [],
+    timeoutMs: 300_000,
+  };
+
+  for (let index = 0; index < args.length; index += 1) {
+    const arg = args[index];
+
+    if (arg === "--live") {
+      options.live = true;
+    } else if (arg === "--fixture") {
+      options.fixtureIds.push(requireValue(args, ++index, arg));
+    } else if (arg === "--harness") {
+      options.harnessIds.push(requireValue(args, ++index, arg));
+    } else if (arg === "--timeout-ms") {
+      options.timeoutMs = Number(requireValue(args, ++index, arg));
+    } else if (arg === "--out") {
+      options.outDir = requireValue(args, ++index, arg);
+    } else if (arg === "--judge-harness") {
+      options.judgeHarnessId = requireValue(args, ++index, arg);
+    } else if (arg === "--synthesizer-harness") {
+      options.synthesizerHarnessId = requireValue(args, ++index, arg);
+    } else {
+      throw new Error(`Unknown option: ${arg}`);
+    }
+  }
+
+  if (!Number.isFinite(options.timeoutMs) || options.timeoutMs <= 0) {
+    throw new Error("--timeout-ms must be a positive number.");
+  }
+
+  return options;
+}
+
+function requireValue(args: string[], index: number, flag: string): string {
+  const value = args[index];
+  if (!value) {
+    throw new Error(`${flag} requires a value.`);
+  }
+
+  return value;
+}
+
+function parseTextResult(result: unknown): unknown {
+  const content = (result as { content?: unknown }).content;
+  if (!Array.isArray(content)) {
+    throw new Error("MCP response did not include content.");
+  }
+
+  const text = content
+    .filter(isMcpTextContent)
+    .map((item) => item.text)
+    .join("\n");
+
+  return JSON.parse(text);
+}
+
+function isMcpTextContent(value: unknown): value is McpTextContent {
+  return (
+    value !== null &&
+    typeof value === "object" &&
+    (value as { type?: unknown }).type === "text" &&
+    typeof (value as { text?: unknown }).text === "string"
+  );
+}