@m6d/cortex-cli 0.12.1 → 0.14.0

package/README.md

@@ -1,16 +1,17 @@
 # @m6d/cortex-cli
 
-The `cortex` command line: scaffold Cortex servers, and run the generators that otherwise need a
-hand-written script.
+The `cortex` command line: scaffold a Cortex project, and manage a Cortex deployment from the
+terminal.
 
 ```sh
 bunx @m6d/cortex-cli new my-app   # from nowhere — scaffolds a project
-bunx cortex swagger sync          # inside a project — reads its cortex.config.ts
-bunx cortex graph seed
+bunx cortex agents list           # against the Cortex you logged in to
+bunx cortex apply                 # make it match ./cortex.manifest.ts
 ```
 
-Commands are noun-first — `cortex <resource> <verb>` — except `new`, which acts on no existing
-resource. Run `cortex --help`, then `cortex <resource> --help`.
+Commands are noun-first, `cortex <resource> <procedure>`, except `new`, which acts on no existing
+resource, `setup` and `apply`, which act on a whole deployment, and `login`, `logout` and `status`,
+which act on your profiles. Run `cortex --help`, then `cortex <resource> --help`.
 
 ```sh
 bun add -d @m6d/cortex-cli
@@ -21,155 +22,219 @@ you how to install Bun when it is genuinely absent.
 
 ## `cortex new`
 
-Scaffolds `./<name>`: a standalone project with `cortex.config.ts`, `index.ts`, `.env` and
-`.env.example`, inlined eslint/prettier/cspell/fallow config, and a `docker-compose.yml` when a
-selection contributes a service. It then runs `bun install`, and `git init` unless the target is
-already inside a work tree. An existing non-empty directory is an error.
+Scaffolds `./<name>`: a backend for Cortex agents, and Cortex itself to run them on.
+
+- `server.ts` is a Cortex server built with `defineServer` from `@m6d/cortex`: one variable
+  (`userName`) and one native tool (`rollDice`), mounted at `/cortex` by `index.ts`.
+- `cortex.manifest.ts` declares a `helper` agent on that server, for `cortex apply`.
+- `docker-compose.yml` runs Cortex's app, worker and engine on the one `syscodeia/cortex` image,
+  with Postgres, Redis and MinIO. Only the app and the engine publish host ports. They and the
+  backend's port are picked at scaffold time from what was free.
+- `.env` holds every setting: Cortex's secrets, the backend's key and name, the model provider, and
+  the `CORTEX_URL` and `CORTEX_KEY` the CLI reaches this checkout's Cortex with. It is not
+  committed: `setup.ts` writes it from `.env.example` on every `bun install`, generating each
+  secret left blank, so a fresh clone gets its own secrets and a stack that never shares the
+  original's volumes. `.dockerignore` keeps both out of images.
+- `bun run dev` starts Cortex (`docker compose up`), the backend and `bun cortex setup --wait --yes` in
+  an mprocs dashboard. `bun run dev:agent` runs the same through `concurrently`, one labelled log for
+  coding agents and CI.
+- eslint, prettier, cspell, fallow and tsconfig are inlined, so `bun run verify` works standalone.
+
+It then runs `bun install`, and `git init` unless the target is already inside a work tree. An
+existing non-empty directory is an error. So is a name like `cortex` or `cortex.`, since the
+manifest takes its id from the name and Cortex keeps the id `cortex` for its own.
 
 ```sh
-bunx @m6d/cortex-cli new widget-backend --knowledge --redis
+bunx @m6d/cortex-cli new shop --datasets
 ```
 
-On a TTY it asks only what the flags left open — the name, the database, and one multi-select over
-the undecided features. With no TTY it never prompts and behaves as `--yes`, so a missing name is a
+On a TTY it asks only what the flags left open: the name, whether to set up the `helper` agent on
+the first `bun run dev`, and if so the model provider (name, base URL, API key, model). It checks
+the provider with one chat message to the model, and reuses it for embeddings when it serves the
+stack's embedding model. With no TTY it never prompts and behaves as `--yes`, so a missing name is a
 hard error rather than a hang.
 
-| Flag                                       | Default    | Writes                                                                    |
-| ------------------------------------------ | ---------- | ------------------------------------------------------------------------- |
-| `--database <postgres\|mssql>`             | `postgres` | The `database` block. `postgres` also gets a compose service; `mssql` not |
-| `--attachments` / `--no-attachments`       | off        | `storage` + `vision` — MinIO uploads, described by a vision model         |
-| `--redis` / `--no-redis`                   | off        | `redis` — resumable streams across restarts                               |
-| `--auth` / `--no-auth`                     | off        | `auth` — JWT verification via JWKS                                        |
-| `--knowledge` / `--no-knowledge`           | off        | `neo4j` + `embedding` + `knowledge`, plus a `src/domains/` tree           |
-| `--control-center` / `--no-control-center` | off        | `controlCenter`                                                           |
-| `--yes`                                    |            | Accept the defaults instead of asking                                     |
-
-Unselected features are still written to `cortex.config.ts`, commented out, alongside `reranker`
-and `context` — the generated config documents what else exists. Every credential and
-URL is a placeholder in `.env`; none of them is prompted for.
-
-## `cortex swagger sync` / `check`
-
-Fetches every Swagger URL the config names — server-level and per-agent, deduped — and reconciles
-the `.endpoint.ts` files under the domains directory against it.
-
-```sh
-bunx cortex swagger sync
-bunx cortex swagger check
-bunx cortex swagger sync --domains-dir ./server/src/domains
-```
-
-`sync` rewrites each endpoint's params, body, and response schemas in place and prints only the
-files it changed; unchanged ones live in the footer count. `check` writes nothing, reports the drift
-it found, and exits 2 — which is what makes it a CI step.
-
-An endpoint file pointing at a route the spec no longer serves is `MISSING`. `sync` prints those
-last, so they are the thing you read, and still exits 0: writing is its job. `check` counts them as
-a finding, same as drift.
-
-`-d, --domains-dir <path>` is declared on the `swagger` group, so it works on either side of the
-verb. It defaults to `src/domains`, resolved relative to the config file.
-
-## `cortex graph seed`
-
-Loads every domain the config declares — server-level `knowledge.domains` and per-agent ones,
-deduped — and seeds Neo4j with the concepts, endpoints, services, and rules they declare, then
-embeds them. Reports per domain, because a domain is the unit you edit. No
-flags.
+| Flag                     | Default | Writes                                                                                      |
+| ------------------------ | ------- | ------------------------------------------------------------------------------------------- |
+| `--datasets`             | off     | `datasets.ts`, an in-memory `orders` table served as a dataset host, and its manifest entry |
+| `--setup` / `--no-setup` | on      | `cortex setup` among the processes `bun run dev` starts                                     |
+| `--yes`                  |         | Accept the defaults instead of asking; setup then runs with no model provider               |
+
+The image runs at the Cortex release that was current when this CLI was published
+(`CORTEX_VERSION` in the shell picks another); the backend depends on the `@m6d/cortex` release of
+that time. A server name stops at Cortex's 100 characters, so a longer project name is
+registered cut to that length.
+
+## `cortex setup`
+
+Registers a `cortex new` project's backend in its Cortex, then applies its manifest. It reads the
+backend's name, URL and key (`CORTEX_SERVER_*`) and the model provider (`MODEL_PROVIDER_*`) from
+`.env`, and goes in the one order Cortex accepts: the server key's secret, the provider's key and
+the provider, the server on the provider's model, a sync of the server, then the apply. Each step
+creates what is missing and keeps what exists, so a second run changes nothing. A key's secret is
+named for its value (`openai-api-key-1a2b3c4d`), so a changed key gets a new secret the provider or
+server moves to, and whatever else presents the old one keeps it. A provider or server under
+`.env`'s name that calls another URL belongs to something else: setup leaves it alone and stops. So
+does a `MODEL_PROVIDER_*` set only in part, or a provider URL on this machine or a private network
+that `EGRESS_ALLOWED_HOSTS` in `.env` does not admit. A server with no model yet gets the provider's.
 
 ```sh
-bunx cortex graph seed
+bun cortex setup              # asks for a model provider when .env has none
+bun cortex setup --wait --yes # what `bun run dev` runs
 ```
 
-Seeding is `MERGE`-based, so re-running is safe. If any statement failed, the graph is partially
-seeded and the command exits 1 — a half-seeded graph should not pass as success.
+`--wait` waits up to 10 minutes for Cortex to answer, long enough for a first start to pull the
+image. Without `-p`, setup sets up the Cortex `.env` names: a different `CORTEX_URL` exported in the
+shell, which Bun lets win over `.env`, ends the run. The retried sync, a minute while Cortex cannot
+reach the backend, gives up at once on any other refusal. With no model provider and `--yes`, it
+registers the server and applies nothing: Cortex publishes no agent without a model. The run that
+first creates something says how to sign in, with the password only on a terminal.
 
-## `cortex cc`
+## Management commands
 
-Manage a Control Center without its console. The commands come from the
-Control Center's own management API (`/api/management/openapi.json`), fetched
+Manage a Cortex deployment from the terminal. The commands come from
+Cortex's own management API (`/api/management/openapi.json`), fetched
 on each run and cached, so the tree always matches the deployment you point
 at and a new procedure needs no CLI release.
 
 ```sh
-cortex cc login                          # asks for the URL and a management key
-cortex cc status
-cortex cc agents list
-cortex cc agents create --slug support --name Support --serverId <uuid>
-cortex cc tools create --json tool.json --publish
-cortex cc tools list -o json | jq '.items[].name'
-cortex cc logout
+cortex login                             # asks for the URL and a management key
+cortex status
+cortex agents list
+cortex agents create --slug support --name Support --serverId <uuid>
+cortex tools create --json tool.json --publish
+cortex tools list -o json | jq '.items[].name'
+cortex logout
 ```
 
-Profiles live in `~/.cortex/cc.json` (owner-only). `--profile <name>` picks
-one; `CORTEX_CC_URL` and `CORTEX_CC_KEY`, both set, win over the file for CI.
-`CORTEX_HOME` relocates the directory.
+Profiles live in `~/.cortex/profiles.json` (owner-only). `-p, --profile <name>`,
+a global option, picks one for any command, and names the one `login` saves
+and `logout` forgets (`default` otherwise). `CORTEX_URL` and `CORTEX_KEY`, both
+set, win over the current profile, for CI and for a project's `.env`; a
+profile `-p` names wins over them. `CORTEX_HOME` relocates the directory.
 
-The grammar is `cortex cc <router> <procedure> [id] [flags]`. Each top-level
-input field is a flag typed from the API's schema; nested values take JSON,
-and `--json <file|->` supplies a whole body that flags then override. A
-required field left out is asked for on a terminal and named as the missing
-flag otherwise. Listings take `--pageIndex`, `--pageSize`, `--q`, and `--all`
-to walk every page. `-o json` prints the raw result for pipes.
+The grammar is `cortex <router> <procedure> [id] [flags]`, with routers as
+the API names them (`datasetHosts`, `modelProviders`). Each top-level input
+field is a flag typed from the API's schema; nested values take JSON, and
+`--json <file|->` supplies a whole body that flags then override. A required
+field left out, `id` included, is asked for on a terminal and named as the
+missing flag or positional otherwise. Listings take `--pageIndex`, `--pageSize`, `--q`, and `--all` to
+walk every page. `-o json` prints the raw result for pipes.
 
 Writes to agents, tools, services, skills, triggers, and dataset hosts save
 drafts.
 `--publish` on a write the API marks publishable (`update`, and a `create`
 whose result is complete enough) runs that entity's publish procedure right
-after; without it the command says so and names the command. Live fields bind
-on the server's next config revalidation without a publish: a trigger's
-`credential` (a secret, so it is prompted for or piped in with `--json -`
-rather than passed as a flag) and a dataset host's url, access fields, and
-timeout. `cortex cc dataset-hosts sync <id>` reads a host's manifest, and a
-draft host publishes only after `dataset-hosts test` reached it once.
+after; without it the command says so and names the command. Fields the API
+marks live (`x-live-fields`) take effect without a publish, from agents'
+next turn, and the command names them: a trigger's
+`credentialName`, `fireCredentialName` and model, say, or a dataset host's
+`url`, `credentials` and `timeoutMs`. `cortex datasetHosts sync <id>` reads a host's manifest, and a
+draft host publishes only after `datasetHosts test` reached it once.
+
+## `cortex apply`
+
+Makes Cortex match a manifest, in one transaction: what is new is
+created, what changed is updated and republished, and what the manifest
+created and does not list is deleted. Rows it did not create are never
+touched, and a name one of them holds fails the apply.
+
+```sh
+cortex apply                             # ./cortex.manifest.ts
+cortex apply deploy/manifest.json --dry-run
+cortex apply -p prod
+```
+
+The path defaults to exactly `./cortex.manifest.ts` in the directory you run
+in. TypeScript and JSON load the same way: the file's default export is the
+manifest. `defineManifest` from `@m6d/cortex` types one as you write it:
+
+```ts
+import { defineManifest } from "@m6d/cortex";
+
+export default defineManifest({
+    id: "shop",
+    tools: [
+        {
+            type: "http",
+            name: "findOrder",
+            description: "Find one order by id.",
+            url: "https://shop.example.com/orders/{orderId}",
+            method: "GET",
+            readOnly: true,
+            inputSchema: {
+                type: "object",
+                properties: { orderId: { type: "string" } },
+                required: ["orderId"],
+            },
+            audience: { agents: ["support"] },
+        },
+    ],
+});
+```
+
+An `openapi` source makes every operation of an OpenAPI document a tool the
+manifest owns, so a backend with many endpoints needs no entry per tool:
+
+```ts
+openapi: [
+    {
+        name: "shop",
+        url: "https://shop.example.com/openapi.json",
+        credentials: [{ header: "Authorization", prefix: "Bearer ", credential: null }],
+        pinned: ["findOrder"],
+        // A GET runs at once; any other method waits for the end user's approval.
+        approval: { never: ["addOrderNote"], always: ["exportOrders"] },
+        audience: { agents: ["support"] },
+    },
+],
+```
+
+Every apply reads the document first, without the source's credentials. An
+operation it adds is created, a changed one updated, and a dropped one
+deleted. A source with credentials is read only over https, or from a host
+the deployment allows, since the document says where they go. A source needs the `tools` write permission even when nothing
+changes. Each tool is named after its `operationId`, made an identifier of at
+most 64 characters; an operation Cortex cannot import is left
+out. Two operations that come out with one name are told apart by a number
+(`getOrders`, `getOrders2`) in the order of their paths. The apply fails
+when `pinned` or `approval` names a tool no operation has (`unknown_tool`)
+or any name such a collision decided (`ambiguous_tool`), since a path the
+document adds or drops could move it to another endpoint. It also fails when an
+operation's name is already a listed tool's or another source's.
+
+The manifest is checked against the contract before anything is sent, then
+sent as written, so Cortex fills its own defaults. It goes to the
+management API (`manifests/sync`, or `manifests/preview` with `--dry-run`,
+which changes nothing) under the same profile the management commands use, so
+the key needs `publishing:publish` and the write permission of every kind the
+apply changes. The output lists each changed entry under its kind: `+`
+created, `~` updated, `-` deleted. A trigger the apply creates, or whose
+`fireCredential` it drops, gets a fire secret, printed once below the summary
+and never again.
+
+References between entries (mentions, service scopes, which role a
+credential or secret plays) are judged on the state the apply leaves, so
+the order the manifest lists them in does not matter. A dataset host's
+credentials are the exception: its catalog is read with them before the
+apply, so they are judged on the deployment as it stands. A credential a
+trigger fires with cannot go to a host in the same apply that drops
+it; drop it first, then apply again.
 
 ## Global options
 
-| Option          | Description                                  |
-| --------------- | -------------------------------------------- |
-| `-v, --verbose` | Print full errors and the commands being run |
-| `-V, --version` | Output the version number                    |
-| `-h, --help`    | Display help for command                     |
-
-`swagger` and `graph` read the project and take `-c, --config <path>` (default
-`./cortex.config.ts`) on the group or the verb. `new` and `cc` do not touch a
-project and have no such flag.
-
-Config discovery is that exact filename in the directory you run in, with no upward search. There is
-no `--cwd`: Bun auto-loads `.env` relative to the process cwd before any of our code runs, so
-reading another directory's config from here would pair it with the wrong environment. `cd` first,
-or point `--config` at the file.
-
-The config is inert data — a plain object literal the CLI imports. Any key that is present but
-evaluates to `undefined`, usually a missing environment variable, is listed by path and stops the
-run.
-
-`graph seed` runs standalone: it never constructs or serves your `@m6d/cortex-server` — it only
-imports your `cortex.config.ts` (whose own imports may pull the server's modules in, inertly).
-Its Neo4j client, its embedder and its `GRAPH_SCHEMA` come from the graph contract the server
-reads the graph back with (each package vendors its own copy at pack time; the server re-exports
-it) — one contract, shared from both ends. Seeding stamps the schema
-version it wrote onto the graph; if that graph already carries a different one, the CLI says so and
-seeds anyway, noting that nodes from the old schema are not removed. Skew is caught on the reading
-side, where it matters: the server checks the stamp on its first resolve and refuses a graph it
-cannot read.
-
-`swagger` writes `.endpoint.ts` source whose generated block has to satisfy the server's own
-`EndpointDef`, so a codegen that outran the installed version is a loud type error in your
-`bun run check`.
-
-Your `cortex.config.ts` imports `@m6d/cortex-server` for `defineAgent`, so a project without it
-installed fails on the import — reported as an unloadable config, with the missing package named.
+| Option                 | Description                                  |
+| ---------------------- | -------------------------------------------- |
+| `-v, --verbose`        | Print full errors and the commands being run |
+| `-p, --profile <name>` | Which saved profile to use                   |
+| `-V, --version`        | Output the version number                    |
+| `-h, --help`           | Display help for command                     |
 
 ## Exit codes
 
-| Command         | 0                                             | 1                                   | 2                      |
-| --------------- | --------------------------------------------- | ----------------------------------- | ---------------------- |
-| `new`           | Files written — even if `bun install` failed  | Bad target, missing name, cancelled | —                      |
-| `swagger sync`  | Wrote what it could, `MISSING` files included | Couldn't run                        | —                      |
-| `swagger check` | Every endpoint file matched and in sync       | Couldn't run                        | Drift and/or `MISSING` |
-| `graph seed`    | Everything seeded                             | A statement failed, or couldn't run | —                      |
-
-_Couldn't run_ means no config file, an unloadable config (a missing `@m6d/cortex-server` in the
-project reads as one), undefined values in it, an absent domains directory, or an unreachable
-Swagger host or Neo4j.
+| Command | 0                                                          | 1                                                   |
+| ------- | ---------------------------------------------------------- | --------------------------------------------------- |
+| `new`   | Files written — even if `bun install` failed               | Bad target, missing name, cancelled                 |
+| `setup` | Set up, or the server registered with no model to apply on | Not a project, or Cortex refused or was unreachable |
+| `apply` | Applied, or previewed with `--dry-run`                     | Invalid manifest, or refused                        |
+| others  | The procedure ran                                          | It failed, or couldn't run                          |

package/package.json

@@ -1,7 +1,7 @@
 {
     "name": "@m6d/cortex-cli",
-    "version": "0.12.1",
-    "description": "Scaffold and operate Cortex servers",
+    "version": "0.14.0",
+    "description": "Scaffold Cortex projects and manage a Cortex deployment",
     "license": "MIT",
     "type": "module",
     "bin": {
@@ -24,7 +24,8 @@
         "@clack/prompts": "^1.7.0",
         "@commander-js/extra-typings": "^15.0.0",
         "commander": "^15.0.0",
-        "picocolors": "^1.1.1"
+        "picocolors": "^1.1.1",
+        "zod": "^4.6.5"
     },
     "devDependencies": {
         "@cortex/contracts": "workspace:*",
@@ -35,7 +36,17 @@
         "access": "public"
     },
     "releaseWatchPaths": [
-        "internal/contracts",
-        "packages/cortex-server"
+        "apps/cortex",
+        "packages/cortex-typescript",
+        "internal/contracts/src/client-tools",
+        "internal/contracts/src/datasets",
+        "internal/contracts/src/egress-policy",
+        "internal/contracts/src/errors",
+        "internal/contracts/src/json",
+        "internal/contracts/src/manifest",
+        "internal/contracts/src/manifest-file",
+        "internal/contracts/src/server",
+        "internal/contracts/src/triggers",
+        "internal/contracts/src/wire"
     ]
 }

package/src/cli.ts

@@ -1,16 +1,17 @@
 import { Command, Help } from "@commander-js/extra-typings";
 
 import pkg from "../package.json";
-import { registerCc } from "@/commands/cc";
-import { registerGraph } from "@/commands/graph";
+import { registerApply } from "@/commands/apply";
+import { registerManagement } from "@/commands/management";
 import { registerNew } from "@/commands/new";
-import { registerSwagger } from "@/commands/swagger";
+import { registerSetup } from "@/commands/setup";
 import * as report from "@/ui/report";
 
 const program = new Command()
     .name("cortex")
-    .description("Scaffold and operate Cortex servers.")
+    .description("Scaffold Cortex projects and manage a Cortex deployment.")
     .option("-v, --verbose", "Print full errors and the commands being run")
+    .option("-p, --profile <name>", "Which saved profile to use")
     .version(pkg.version, "-V, --version", "Output the version number")
     .helpOption("-h, --help", "Display help for command")
     // The implicit `help [command]` would print as a third, ungrouped section and
@@ -19,25 +20,31 @@ const program = new Command()
     // Commander appends ` [options]` to any command that has some. Left in, the
     // first command to grow a flag widens the whole list and the two-group screen
     // stops matching. `cortex <command> --help` lists them.
+    // Every command's help lists the root's options too, `--profile` among them.
     .configureHelp({
+        showGlobalOptions: true,
         subcommandTerm: function (cmd) {
             return new Help().subcommandTerm(cmd).replace(" [options]", "");
         },
     })
-    .addHelpText("after", "\nRun `cortex <resource> --help` to see what you can do with one.");
+    .addHelpText(
+        "after",
+        "\nEvery other command comes from the Cortex you logged in to:\n`cortex <resource> <procedure>`. Run `cortex <resource> --help` to see a resource's.",
+    );
 
 // Each command owns its help group, so the two headings survive these files being
 // rewritten. Registration order fixes the order the groups print in.
 registerNew(program);
-registerSwagger(program);
-registerGraph(program);
-// The Control Center's commands come from its API, so this one may fetch
-// before parsing; it looks at argv to decide whether it needs to.
-await registerCc(program, process.argv.slice(2));
+registerSetup(program);
+registerApply(program);
+// The management commands come from Cortex's API, so this one may fetch
+// before parsing; it looks at argv to decide whether it needs to, which it
+// does only when argv names none of the commands registered before it.
+await registerManagement(program, process.argv.slice(2));
 
 // Commander's own errors print and exit on their own, and every failure the
-// commands predict goes through `report.fail`. Anything still reaching here is a
-// bug — in the CLI, or in the project's server — and it gets the same grammar
+// commands predict goes through `report.fail`. Anything reaching here is a
+// bug — in the CLI, or in Cortex — and it gets the same grammar
 // rather than a raw Bun stack dump.
 await program.parseAsync().catch(function (error: unknown) {
     const verbose = program.opts().verbose === true;