@m6d/cortex-cli 1.0.0

package/README.md

@@ -0,0 +1,135 @@
+# @m6d/cortex-cli
+
+The `cortex` command line: scaffold Cortex servers, and run the generators that otherwise need a
+hand-written script.
+
+```sh
+bunx @m6d/cortex-cli new my-app   # from nowhere — scaffolds a project
+bunx cortex swagger sync          # inside a project — reads its cortex.config.ts
+bunx cortex graph seed
+```
+
+Commands are noun-first — `cortex <resource> <verb>` — except `new`, which acts on no existing
+resource. Run `cortex --help`, then `cortex <resource> --help`.
+
+```sh
+bun add -d @m6d/cortex-cli
+```
+
+Requires [Bun](https://bun.sh). The binary re-execs itself under Bun when Node started it, and tells
+you how to install Bun when it is genuinely absent.
+
+## `cortex new`
+
+Scaffolds `./<name>`: a standalone project with `cortex.config.ts`, `index.ts`, `.env` and
+`.env.example`, inlined eslint/prettier/cspell/fallow config, and a `docker-compose.yml` when a
+selection contributes a service. It then runs `bun install`, and `git init` unless the target is
+already inside a work tree. An existing non-empty directory is an error.
+
+```sh
+bunx @m6d/cortex-cli new widget-backend --knowledge --redis
+```
+
+On a TTY it asks only what the flags left open — the name, the database, and one multi-select over
+the undecided features. With no TTY it never prompts and behaves as `--yes`, so a missing name is a
+hard error rather than a hang.
+
+| Flag                                       | Default    | Writes                                                                    |
+| ------------------------------------------ | ---------- | ------------------------------------------------------------------------- |
+| `--database <postgres\|mssql>`             | `postgres` | The `database` block. `postgres` also gets a compose service; `mssql` not |
+| `--attachments` / `--no-attachments`       | off        | `storage` + `vision` — MinIO uploads, described by a vision model         |
+| `--redis` / `--no-redis`                   | off        | `redis` — resumable streams across restarts                               |
+| `--auth` / `--no-auth`                     | off        | `auth` — JWT verification via JWKS                                        |
+| `--knowledge` / `--no-knowledge`           | off        | `neo4j` + `embedding` + `knowledge`, plus a `src/domains/` tree           |
+| `--control-center` / `--no-control-center` | off        | `controlCenter`                                                           |
+| `--yes`                                    |            | Accept the defaults instead of asking                                     |
+
+Unselected features are still written to `cortex.config.ts`, commented out, alongside `fastModel`,
+`reranker`, and `context` — the generated config documents what else exists. Every credential and
+URL is a placeholder in `.env`; none of them is prompted for.
+
+## `cortex swagger sync` / `check`
+
+Fetches every Swagger URL the config names — server-level and per-agent, deduped — and reconciles
+the `.endpoint.ts` files under the domains directory against it.
+
+```sh
+bunx cortex swagger sync
+bunx cortex swagger check
+bunx cortex swagger sync --domains-dir ./server/src/domains
+```
+
+`sync` rewrites each endpoint's params, body, and response schemas in place and prints only the
+files it changed; unchanged ones live in the footer count. `check` writes nothing, reports the drift
+it found, and exits 2 — which is what makes it a CI step.
+
+An endpoint file pointing at a route the spec no longer serves is `MISSING`. `sync` prints those
+last, so they are the thing you read, and still exits 0: writing is its job. `check` counts them as
+a finding, same as drift.
+
+`-d, --domains-dir <path>` is declared on the `swagger` group, so it works on either side of the
+verb. It defaults to `src/domains`, resolved relative to the config file.
+
+## `cortex graph seed`
+
+Loads every domain the config declares — server-level `knowledge.domains` and per-agent ones,
+deduped — and seeds Neo4j with the concepts, endpoints, services, and rules they declare, then
+embeds them. Reports per domain, because a domain is the unit you edit. No
+flags.
+
+```sh
+bunx cortex graph seed
+```
+
+Seeding is `MERGE`-based, so re-running is safe. If any statement failed, the graph is partially
+seeded and the command exits 1 — a half-seeded graph should not pass as success.
+
+## Global options
+
+| Option                | Description                                           |
+| --------------------- | ----------------------------------------------------- |
+| `-c, --config <path>` | Path to the config file. Default `./cortex.config.ts` |
+| `-v, --verbose`       | Print full errors and the commands being run          |
+| `-V, --version`       | Output the version number                             |
+| `-h, --help`          | Display help for command                              |
+
+Config discovery is that exact filename in the directory you run in, with no upward search. There is
+no `--cwd`: Bun auto-loads `.env` relative to the process cwd before any of our code runs, so
+reading another directory's config from here would pair it with the wrong environment. `cd` first,
+or point `--config` at the file.
+
+The config is inert data — a plain object literal the CLI imports. Any key that is present but
+evaluates to `undefined`, usually a missing environment variable, is listed by path and stops the
+run.
+
+`graph seed` runs standalone: it never constructs or serves your `@m6d/cortex-server` — it only
+imports your `cortex.config.ts` (whose own imports may pull the server's modules in, inertly).
+Its Neo4j client, its embedder and its `GRAPH_SCHEMA` come from the graph contract the server
+reads the graph back with (each package vendors its own copy at pack time; the server re-exports
+it) — one contract, shared from both ends. Seeding stamps the schema
+version it wrote onto the graph; if that graph already carries a different one, the CLI says so and
+seeds anyway, noting that nodes from the old schema are not removed. Skew is caught on the reading
+side, where it matters: the server checks the stamp on its first resolve and refuses a graph it
+cannot read.
+
+`swagger` writes `.endpoint.ts` source whose generated block has to satisfy the server's own
+`EndpointDef`, so a codegen that outran the installed version is a loud type error in your
+`bun run check`.
+
+Your `cortex.config.ts` imports `@m6d/cortex-server` for `defineAgent`, so a project without it
+installed fails on the import — reported as an unloadable config, with the missing package named.
+
+## Exit codes
+
+| Command         | 0                                             | 1                                   | 2                      |
+| --------------- | --------------------------------------------- | ----------------------------------- | ---------------------- |
+| `new`           | Files written — even if `bun install` failed  | Bad target, missing name, cancelled | —                      |
+| `swagger sync`  | Wrote what it could, `MISSING` files included | Couldn't run                        | —                      |
+| `swagger check` | Every endpoint file matched and in sync       | Couldn't run                        | Drift and/or `MISSING` |
+| `graph seed`    | Everything seeded                             | A statement failed, or couldn't run | —                      |
+
+_Couldn't run_ means no config file, an unloadable config (a missing `@m6d/cortex-server` in the
+project reads as one), undefined values in it, an absent domains directory, or an unreachable
+Swagger host or Neo4j.
+
+Design: [`docs/design/cortex-cli.md`](../../docs/design/cortex-cli.md).

package/bin/cortex.js

@@ -0,0 +1,52 @@
+#!/usr/bin/env node
+
+// Plain JavaScript, and a Node shebang, on purpose: `bunx` honours the shebang
+// unless you pass `--bun`, so Node parses this file on the normal path. A
+// `#!/usr/bin/env bun` shebang would give a user without Bun
+// `env: bun: No such file or directory` — unrecoverable, because none of our
+// code ever runs. See docs/design/cortex-cli.md §4.
+
+import { spawnSync } from "node:child_process";
+import { fileURLToPath } from "node:url";
+
+if (process.versions.bun) {
+    await import("../src/cli");
+} else {
+    const cli = fileURLToPath(new URL("../src/cli.ts", import.meta.url));
+    const args = [cli, ...process.argv.slice(2)];
+    let result = spawnSync("bun", args, { stdio: "inherit" });
+
+    // Node's CVE-2024-27980 fix refuses to spawn a `.cmd` file directly, which is
+    // exactly what an npm-installed Bun is on Windows: the user has Bun and still
+    // gets EINVAL. `shell: true` is the documented way through; every argument is
+    // quoted because cmd.exe splits on the spaces in a Windows path otherwise.
+    if (result.error?.code === "EINVAL" && process.platform === "win32") {
+        result = spawnSync("bun", args.map(quote), { stdio: "inherit", shell: true });
+    }
+
+    // Any spawn error at all means Bun did not run, and the reason matters less
+    // than the fix — a raw `Error: spawnSync bun EACCES` helps nobody.
+    if (result.error) {
+        process.stderr.write(
+            "\n■  cortex needs Bun.\n" +
+                (result.error.code === "ENOENT" ? "" : `\n   ${result.error.message}\n`) +
+                "\n     curl -fsSL https://bun.sh/install | bash\n" +
+                "\n   Then:  bunx @m6d/cortex-cli new\n\n",
+        );
+        process.exit(1);
+    }
+
+    // A signal-killed child reports `status: null`. Exiting 1 would turn the
+    // user's Ctrl-C into a generic failure, so the signal is re-raised on
+    // ourselves and the shell sees the 130/143 it expects. If the signal is
+    // ignored or blocked and we survive it, the exit below still runs.
+    if (result.signal) {
+        process.kill(process.pid, result.signal);
+    }
+
+    process.exit(result.status ?? 1);
+}
+
+function quote(argument) {
+    return `"${argument}"`;
+}

package/package.json

@@ -0,0 +1,37 @@
+{
+    "name": "@m6d/cortex-cli",
+    "version": "1.0.0",
+    "description": "Scaffold and operate Cortex servers",
+    "license": "MIT",
+    "type": "module",
+    "bin": {
+        "cortex": "./bin/cortex.js"
+    },
+    "files": [
+        "bin",
+        "src",
+        "!src/**/*.test.ts",
+        "tsconfig.json",
+        "README.md"
+    ],
+    "scripts": {
+        "check": "tsc --noEmit",
+        "prepack": "cp -R ../../internal/contracts src/contracts && rm src/contracts/package.json src/contracts/tsconfig.json",
+        "postpack": "rm -rf src/contracts",
+        "test": "bun test"
+    },
+    "dependencies": {
+        "@clack/prompts": "^1.7.0",
+        "@commander-js/extra-typings": "^15.0.0",
+        "commander": "^15.0.0",
+        "picocolors": "^1.1.1"
+    },
+    "devDependencies": {
+        "@cortex/contracts": "workspace:*",
+        "@types/bun": "latest",
+        "typescript": "~5.9.3"
+    },
+    "publishConfig": {
+        "access": "public"
+    }
+}

package/src/cli.ts

@@ -0,0 +1,63 @@
+import { Command, Help, Option } from "@commander-js/extra-typings";
+
+import pkg from "../package.json";
+import { registerGraph } from "@/commands/graph";
+import { registerNew } from "@/commands/new";
+import { registerSwagger } from "@/commands/swagger";
+import * as report from "@/ui/report";
+
+// Discovery is this exact name in the cwd, with no upward walk (design spec §6.2),
+// so the flag's default *is* the discovery rule. The second argument is what help
+// prints — without it commander would JSON.stringify the path and show quotes.
+const DEFAULT_CONFIG = "./cortex.config.ts";
+
+const program = new Command()
+    .name("cortex")
+    .description("Scaffold and operate Cortex servers.")
+    .addOption(
+        new Option("-c, --config <path>", "Path to cortex.config.ts").default(
+            DEFAULT_CONFIG,
+            DEFAULT_CONFIG,
+        ),
+    )
+    .option("-v, --verbose", "Print full errors and the commands being run")
+    .version(pkg.version, "-V, --version", "Output the version number")
+    .helpOption("-h, --help", "Display help for command")
+    // The implicit `help [command]` would print as a third, ungrouped section and
+    // break the two-group screen the prototype fixes. `cortex <cmd> --help` covers it.
+    .helpCommand(false)
+    // Commander appends ` [options]` to any command that has some. Left in, the
+    // first command to grow a flag widens the whole list and the two-group screen
+    // in cli-output.md §1 stops matching. `cortex <command> --help` lists them.
+    .configureHelp({
+        subcommandTerm: function (cmd) {
+            return new Help().subcommandTerm(cmd).replace(" [options]", "");
+        },
+    })
+    .addHelpText("after", "\nRun `cortex <resource> --help` to see what you can do with one.");
+
+// Each command owns its help group, so the two headings survive these files being
+// rewritten. Registration order fixes the order the groups print in.
+registerNew(program);
+registerSwagger(program);
+registerGraph(program);
+
+// Commander's own errors print and exit on their own, and every failure the
+// commands predict goes through `report.fail`. Anything still reaching here is a
+// bug — in the CLI, or in the project's server — and it gets the same grammar
+// rather than a raw Bun stack dump.
+await program.parseAsync().catch(function (error: unknown) {
+    const verbose = program.opts().verbose === true;
+    const rerun = process.argv.slice(2);
+    if (!verbose) rerun.push("--verbose");
+
+    report.fail({
+        what: "cortex hit an unexpected error",
+        error,
+        blocks: [
+            "That is a bug — the CLI should have explained this. Report it at\nhttps://github.com/boring91/cortex/issues with the output of:",
+            `  cortex ${rerun.join(" ")}`,
+        ],
+        verbose,
+    });
+});

package/src/commands/graph.ts

@@ -0,0 +1,253 @@
+import type { Command } from "@commander-js/extra-typings";
+import {
+    GRAPH_SCHEMA,
+    GRAPH_SCHEMA_VERSION,
+    createEmbedder,
+    createNeo4jClient,
+    type DomainDef,
+    type EmbeddingProviderConfig,
+    type Neo4jClient,
+    type Neo4jConfig,
+} from "@cortex/contracts/graph";
+import pc from "picocolors";
+
+import { configLabel, findConfig, loadConfig } from "@/config/load";
+import { validateConfig } from "@/config/validate";
+import { seedGraph } from "@/graph/seed";
+import * as report from "@/ui/report";
+
+const TITLE = "cortex graph seed";
+
+/** Domain names are longer than swagger's finding labels, so the body is wider. */
+const DOMAIN_COLUMN = 14;
+
+/** A failing statement is worth one line: enough to identify, not enough to drown. */
+const DETAIL_WIDTH = 60;
+
+/** Only the parts of a loaded config `graph seed` reads. */
+type ProjectConfig = {
+    neo4j?: Neo4jConfig;
+    embedding?: EmbeddingProviderConfig & { dimension: number };
+    knowledge?: { domains?: Record<string, DomainDef> };
+    agents?: Record<string, { knowledge?: { domains?: Record<string, DomainDef> } | null }>;
+};
+
+// Connection failures reach us as whatever `fetch` threw, so match on shape
+// rather than on a driver's error class.
+const UNREACHABLE =
+    /unable to connect|connection refused|econnrefused|failed to fetch|fetch failed|enotfound|getaddrinfo|timed out/i;
+
+export function registerGraph(program: Command) {
+    const graph = program
+        .command("graph")
+        .helpGroup("Resources:")
+        .summary("The Neo4j knowledge graph")
+        .description(
+            "The Neo4j knowledge graph.\n\nReads `knowledge.domains` and `neo4j` from your cortex.config.ts.",
+        )
+        .helpCommand(false);
+
+    // No flags. `--domain <name>` is flag-shaped design work, not a flag:
+    // `expandDomains` resolves cross-domain references and phase 1 creates every
+    // node before any edge, so a subset seed risks dangling edges.
+    graph
+        .command("seed")
+        .description("Load concepts, endpoints, services, and rules into Neo4j")
+        .action(async function () {
+            // The globals live on the root command, which reaches this file
+            // untyped — `Command` here is the parameter type, not cli.ts's
+            // inferred one.
+            const { config: path, verbose = false } = program.opts() as {
+                config: string;
+                verbose?: boolean;
+            };
+
+            await seed({
+                path,
+                explicit: program.getOptionValueSource("config") !== "default",
+                verbose,
+            });
+        });
+}
+
+async function seed(options: { path: string; explicit: boolean; verbose: boolean }) {
+    const { verbose } = options;
+
+    // Discovery runs before the box opens: with no config there is no run to
+    // report on, only the reason there isn't one.
+    const configPath = findConfig({ ...options, command: TITLE });
+
+    report.header(TITLE);
+    const loaded = await loadConfig(configPath, verbose);
+    validateConfig(loaded, configPath);
+    report.input("Config", configLabel(configPath));
+
+    const config = loaded as ProjectConfig;
+
+    // Inputs are reported as they resolve, and only when the config has them:
+    // an absent section is `runSeed`'s error to throw, not ours to predict.
+    if (config.neo4j?.url) report.input("Neo4j", config.neo4j.url);
+    const domains = collectDomains(config);
+    const names = Object.keys(domains);
+    if (names.length) report.input("Domains", `${names.length} — ${names.join(", ")}`);
+
+    const startedAt = Date.now();
+    const result = await runSeed(config, domains).catch(function (error: unknown): never {
+        return failSeed({ error, config, configPath, verbose });
+    });
+
+    // A domain is the unit the user edits, so it is the unit the report counts
+    // against: a wrong number next to `attendance` points at src/domains/attendance/.
+    report.blank();
+    for (const [name, domain] of Object.entries(result.domains)) {
+        const counts = [report.plural(domain.nodes, "node"), report.plural(domain.edges, "edge")];
+        if (domain.failed) counts.push(`${domain.failed} failed`);
+        report.row(name, counts.join(" · "), DOMAIN_COLUMN);
+
+        // The first failure only: enough to identify what broke, not enough to
+        // drown the run — the count above says how many followed it.
+        const failure = domain.failures[0];
+        if (failure) {
+            report.row("", `${pc.red("✗")} ${truncate(failure.error)}`, DOMAIN_COLUMN);
+            report.row("", `  ${truncate(oneLine(failure.statement))}`, DOMAIN_COLUMN);
+        }
+    }
+
+    // Embeddings are a second pass over the whole graph, so they get their own
+    // step rather than a row among the counts.
+    report.blank();
+    report.input(
+        "Embeddings",
+        `${report.plural(result.embeddings.concepts, "concept")}, dimension ${result.embeddings.dimension}`,
+    );
+    report.blank();
+
+    const counts = [report.plural(result.nodes, "node"), report.plural(result.edges, "edge")];
+    if (result.failed) counts.push(`${result.failed} failed`);
+    report.footer(`${counts.join(" · ")}  (${((Date.now() - startedAt) / 1000).toFixed(1)}s)`);
+
+    if (result.failed) {
+        report.note("The graph is partially seeded. Re-running is safe — seeding is MERGE-based.");
+        process.exit(1);
+    }
+}
+
+/** Cypher is generated multi-line and indented; a report row is neither. */
+function oneLine(statement: string) {
+    return statement.replace(/\s+/g, " ").trim();
+}
+
+function truncate(text: string) {
+    return text.length > DETAIL_WIDTH ? `${text.slice(0, DETAIL_WIDTH - 2)} …` : text;
+}
+
+/** Server level first, then per agent — an agent's domain of the same name wins. */
+function collectDomains(config: ProjectConfig) {
+    const all: Record<string, DomainDef> = { ...config.knowledge?.domains };
+
+    for (const agent of Object.values(config.agents ?? {})) {
+        Object.assign(all, agent.knowledge?.domains);
+    }
+
+    return all;
+}
+
+/**
+ * The whole pipeline is the CLI's, built on the graph contract it depends on:
+ * `@cortex/contracts/graph` is where the vocabulary, the Neo4j client and the embedder
+ * come from, and the version it seeds at is the version of that dependency.
+ */
+async function runSeed(config: ProjectConfig, domains: Record<string, DomainDef>) {
+    if (!Object.keys(domains).length) throw new Error("No domains found in knowledge config");
+    if (!config.neo4j) throw new Error("neo4j config is required for graph seed");
+    if (!config.embedding) throw new Error("embedding config is required for graph seed");
+
+    const embed = createEmbedder(config.embedding);
+    const neo4j = createNeo4jClient(config.neo4j, embed);
+
+    await warnOnRestamp(neo4j);
+
+    return await seedGraph(
+        {
+            neo4j,
+            embedding: { embed, dimension: config.embedding.dimension },
+            schema: GRAPH_SCHEMA,
+            schemaVersion: GRAPH_SCHEMA_VERSION,
+        },
+        domains,
+    );
+}
+
+/**
+ * Seeding is MERGE-based, so a graph written under an older schema is not cleaned
+ * up — its nodes stay, under labels nothing reads. Worth saying out loud; not
+ * worth blocking on, since re-seeding is exactly how that graph gets fixed.
+ */
+async function warnOnRestamp(neo4j: Neo4jClient) {
+    const rows: unknown = JSON.parse(
+        await neo4j.query(
+            `MATCH (g:${GRAPH_SCHEMA.label.marker}) RETURN g.${GRAPH_SCHEMA.prop.version} AS version`,
+        ),
+    );
+
+    // Anything but a row array means an unreachable or unreadable graph, which
+    // the seed itself is about to report far better than a warning could.
+    if (!Array.isArray(rows)) return;
+
+    const stamped = (rows as { version?: number }[])[0]?.version;
+    if (stamped === undefined || stamped === GRAPH_SCHEMA_VERSION) return;
+
+    report.blank();
+    report.row(
+        "",
+        `${pc.yellow("!")} Graph was written with schema v${stamped}; restamping as v${GRAPH_SCHEMA_VERSION}.`,
+        DOMAIN_COLUMN,
+    );
+    report.row("", "  Nodes from the old schema are not removed.", DOMAIN_COLUMN);
+}
+
+/**
+ * Section requirements are `runSeed`'s to enforce — it already throws
+ * `"neo4j config is required for graph seed"`. This renders what it threw.
+ */
+function failSeed(options: {
+    error: unknown;
+    config: ProjectConfig;
+    configPath: string;
+    verbose: boolean;
+}): never {
+    const { error, config, configPath, verbose } = options;
+    const label = configLabel(configPath);
+    const message = error instanceof Error ? error.message : String(error);
+
+    if (message === "No domains found in knowledge config") {
+        report.fail({
+            what: `${label} has no knowledge.domains`,
+            blocks: [
+                "`graph seed` loads domain definitions into Neo4j. Add them:",
+                "  knowledge: { domains: { leaves: leavesDomain } }",
+                "See https://github.com/boring91/cortex#file-structure-convention",
+            ],
+        });
+    }
+
+    const url = config.neo4j?.url;
+    if (url !== undefined && UNREACHABLE.test(message)) {
+        report.fail({
+            what: `Cannot reach Neo4j at ${url}`,
+            error,
+            blocks: [
+                "Is it running?  docker compose up -d neo4j",
+                `Otherwise check neo4j.url / neo4j.user / neo4j.password in\n${label}.`,
+            ],
+            verbose,
+        });
+    }
+
+    report.fail({
+        what: "Could not seed the graph",
+        error,
+        blocks: [`Check neo4j, embedding, and knowledge.domains in ${label}.`],
+        verbose,
+    });
+}