@maiaos/node 0.1.0-canary.2

package/README.md

@@ -0,0 +1,281 @@
+# Maia Node (`@maiaos/node`)
+
+[![npm canary](https://img.shields.io/npm/v/@maiaos/node/canary?label=npm%20canary)](https://www.npmjs.com/package/@maiaos/node)
+
+Maia Node is the local authority of Maia OS: one process per machine and
+profile that owns Sessions, turns, worker accounts, durable state, event order
+and receipts for swarms of AI coding agents such as Claude Code and Codex.
+
+Every Maia client (the `maia` CLI, the `maia mcp` MCP server, the
+`@maiaos/sdk` TypeScript SDK and the Desktop app) reaches the Node through its
+authenticated loopback API; none of them owns state. The Node validates and
+authorizes each operation, stores the result in SQLite, writes a receipt and
+runs workers as child processes under a native process host.
+
+## Run it
+
+In development, run a Node inside an isolated dev worktree, so it never touches
+your real `~/.maia`:
+
+```sh
+pnpm dev:worktree create try-maia
+cd ../maiaos-worktrees/try-maia
+pnpm dev:node          # builds the Node and runs it in the foreground
+pnpm maia status       # in a second terminal: the Node reports ready
+```
+
+The executable itself:
+
+```sh
+pnpm --filter @maiaos/node build
+node services/node/dist/bin/maia-node.mjs --help
+node services/node/dist/bin/maia-node.mjs --desktop --profile <name>
+```
+
+`--desktop` runs the authenticated local Node in the foreground, and
+`--profile` selects its runtime storage (`default` when omitted).
+For a Desktop-owned child, the native host sets `MAIA_DESKTOP_PARENT_PIPE=1`
+and retains its stdin writer. EOF or a pipe error interrupts the existing
+Node scope, including during startup, to drain and release runtime authority.
+A separate Effect deadline kills this process after 10 seconds if its drain
+stalls. A drain that finishes after SIGINT, SIGTERM or a closed pipe exits 0,
+so systemd logs no failure for a restart; a failed drain or startup keeps its
+non-zero code. Without that explicit flag, stdin closure does not stop the Node;
+attached local services and remote Nodes keep their own lifetimes. The
+self-kill deadline requires a responsive Node event loop.
+`maia-node remote probe|install|rollback|uninstall <ssh-host>` installs a
+pinned Node on a Linux host as a systemd user service, and `maia-node bridge`
+serves that host's Node API over SSH standard input and output, with no
+listener beyond loopback ([remote Node install](../../docs/operations.md#remote-node-install)).
+The local Node also registers remote Nodes (`node.remote.*`, S14): a preview
+probes and plans in a Node-scoped job, a confirmation bound to the plan
+digest records a receipt and runs the same install or uninstall, and the
+install pins `codex` and `claude` with a PATH of their directories, the Maia
+runtime and the system directories ([D-080](../../docs/decisions.md#d-080)).
+A local background service is planned (RL3). `@maiaos/node@canary` is
+published on npm for installation testing only; npm's automatic `latest` tag
+also points at a canary build.
+
+## Key concepts
+
+- **One authority.** The Node alone validates, authorizes, persists, executes,
+  orders events and writes receipts. Clients are projections: a cached client
+  state never overwrites newer Node state.
+- **Node API.** The Node binds an ephemeral port on `127.0.0.1` only and writes
+  an owner-private connection descriptor and bearer credential to its profile
+  runtime directory. Unary operations use `POST /api/v1/operations`;
+  observations use the `/api/v1/observations` WebSocket. A restart replaces the
+  credential.
+- **Live observation (S12a).** `observe.events` replays committed journal rows
+  from SQLite, which stays the source of cursors and replay. An idle
+  subscriber wakes as soon as a journal append commits: a TEMP trigger on
+  `domain_events` signals every append on the Node's connection, and a burst
+  coalesces into one wake. The five-second heartbeat proves liveness and
+  publishes filtered progress.
+- **Live Session items (S12b and S12c, source candidates).** A per-Session
+  writer assigns each Codex or Claude Code item record its owner sequence. The live lane coalesces
+  deltas for 33 ms and offers them to bounded attachment queues (256 records,
+  2 MiB); a full queue gets `resync`. The durable lane commits finished items
+  in one transaction per 400 ms (`session_stream_records`, migration v50), and
+  startup closes what a crashed epoch left open. `session.attach` on the
+  observation socket replays or snapshots, synchronizes the open items, then
+  streams live records.
+- **Profiles.** A profile has its own runtime directory, database and
+  observability files. Each dev worktree runs its own profile and `MAIA_HOME`.
+- **Sessions and executors.** A Session runs one vendor executor, Codex through
+  `codex app-server` or Claude Code through the Claude Agent SDK. A reading
+  Session runs read-only in its Workspace; a coding Session (`--base`) gets a
+  worktree of its own and writes only there, with `session diff` and
+  `session archive` to review and retire it
+  ([coding Sessions](../../docs/maia-swarm/external-executors.md#coding-sessions)).
+  A Session is bound to
+  one account Connection; when the caller names only the executor kind, the
+  Node picks the eligible account with the most usage left. At most 8 Sessions
+  run at once on a Node; each account allows 2 by default (1–16, set with
+  `maia login <claude|codex> --max-sessions` or `maia connection capacity set`).
+- **Accounts.** A Connection is a named Codex or Claude Code account, either a
+  Maia-managed account root or the system default login. The vendor CLI keeps
+  custody of the credential; Maia never reads it. Vendor usage-limit signals
+  mark an account `limited` until its reset time. A gateway Connection is an
+  AI gateway's base URL with its API key in an owner-only `gateway-key` file;
+  Claude Code and Codex Sessions on it receive the URL and key through their
+  launch environment.
+- **Process hosts.** On macOS a native helper owns each worker's process groups,
+  limits and cleanup proof (cooperative, not a sandbox). On Linux a helper puts
+  each process tree in its own cgroup v2 directory and confines it with
+  Landlock and seccomp; it needs a systemd user unit with `Delegate=yes` and
+  Linux 6.12 or later, and it bounds the tree, not the user. Both are source
+  candidates. A Windows Job Object helper exists in source but is unqualified:
+  Windows is unsupported in v1 and fails with `platform_unsupported`. A host
+  without qualified process authority fails closed. A vendor launch permit
+  records where its host keeps the tree (the macOS process group, the Linux
+  tree cgroups' parent), and after a crash the restarted Node frees the
+  vendor only once that record shows the tree ended
+  ([D-098](../../docs/decisions.md#d-098)).
+- **Mutations.** Every mutation checks a revision or command ID and returns a
+  receipt; replaying a command ID returns the first result instead of acting
+  twice. Project and container metadata, agent permissions, integration plans
+  and the voice relay also need a preview and an explicit confirmation.
+- **Port leases (S16).** `port.lease.acquire|renew|release|list` grant 1–8
+  loopback ports from `MAIA_PORT_RANGE` (default `41000-41999`) to a Session
+  or the operator. Leases are advisory: the Node never restricts a bind or
+  stops a process. A port is granted only when connects to `127.0.0.1` and
+  `[::1]` are refused and a loopback bind succeeds. Leases last 15 minutes
+  unless renewed, and end with their Session, at expiry or at Node start.
+  Codex and Claude Code Sessions that hold leases receive `MAIA_PORTS`.
+- **Emergency holds.** A person holds exact Projects or accounts with a
+  previewed, confirmed `hold.place`. Every new Session, start, resume, turn,
+  Run dispatch and voice relay on a held target fails with `held`, checked in
+  the transaction that admits it; account selection skips held accounts. A
+  `stop` hold also stops the Sessions running on its targets in the same
+  transaction, then ends their runtimes. Holds survive restarts, recovery never
+  resumes held work, and `hold.release` restores admission. See
+  [operations](../../docs/operations.md#emergency-holds-implemented-today-source-candidate).
+- **Worker credentials and fan-out guards (S18a, S18b).** Each Codex or
+  Claude Code launch gets its own Session credential (`session-credential/`),
+  revoked when its runtime ends; `Authorization: Maia-Session` admits that
+  Session's role operations (`application/session-request.ts`). Both roles
+  list Sessions only in the credential's Project, before pagination; an
+  explicit other Project is refused. Human and external-coordinator lists
+  retain whole-Node visibility ([D-092](../../docs/decisions.md#d-092)). The Linux
+  helper denies Claude Code trees reads of the runtime directory and auth
+  file; Codex's profile denies them to Codex commands
+  ([D-110](../../docs/decisions.md#d-110)).
+  `session-guard/` keeps each Session's role, parent and depth and the
+  per-Project depth, live-children, messages-per-minute and
+  leases-per-Session limits (`project.guard.*`), checked in the write that
+  creates, sends or leases. See
+  [vendor executors](../../docs/maia-swarm/external-executors.md#session-bound-worker-credential).
+- **Control dispatch.** An operator or coordinator request with a feature
+  descriptor goes through `application/feature-handlers.ts`: one
+  context-preserving adapter per registered Node handler, which forwards the
+  command ID and any attached credential, selects the executor's services and
+  fails closed when one is not composed (adopted-Session resume preflight,
+  legacy versus interactive `terminal.create`, voice relay availability).
+  Undescribed methods keep explicit routes in `application/control-request.ts`
+  until their descriptors land.
+- **Model providers.** `provider.pool.list` reads the owner-only `auth.json`
+  that `maia login` writes. The managed Pi worker leases one credential in
+  memory per Session and releases it after a cleanup proof.
+
+## Status
+
+Implemented as source candidates, not release-qualified:
+
+- the Node API, SQLite storage with checksummed migrations, and receipts with
+  command-ID replay;
+- Projects, execution Workspaces, Workspace containers and confirmed Project
+  metadata;
+- Sessions with Codex and Claude Code executors: create, start, stop, resume,
+  rename, observe, events, with titles (given at creation or taken from the
+  first turn); turns: send, wait, show, interrupt;
+- reference resolution (`reference.resolve`): an ID, ID prefix, name or
+  Session title, or the registered folder containing a path, to one exact ID
+  or its candidates;
+- Connections, sign-in, usage and integration approval of vendor executables;
+- record-only external questions (`interaction.*`), presentation reads, live
+  terminals on macOS, and the release-disabled voice orchestrator;
+- emergency holds on exact Projects and accounts (`hold.*`, S20);
+- the Run ledger over executor Sessions (`run.ledger.*`, S19): a task graph
+  whose dispatches are Codex or Claude Code Session turns, Task state derived
+  from dispatches, three failed dispatches stop a Task, and the
+  `session.observe` attention value.
+- Session-bound worker credentials, coding-worker MCP launch configuration,
+  roles and per-Project fan-out guards
+  (`project.guard.*`, S18a and S18b).
+
+Planned: Pi item adapters and the Desktop timeline
+(S12d) on the S12b stream, host and agent observability
+(S15), advisory leases (S16, in progress), Node and capability holds with `capability.revoke`, coding
+workers (S5), the installer and
+background service (RL3) and updates (RL4). The
+[roadmap](../../ROADMAP.md#v1-external-brain-and-vendor-executors) tracks each
+slice.
+
+## Direction: Control Nodes, Execution Nodes and principals
+
+The Node is designed to grow into an organization's agent hub. None of this is
+implemented:
+
+- one **Control Node** owns durable truth for team Workspaces, starting from an
+  explicit first-owner bootstrap
+  ([design](../../docs/team-workspaces-and-control-nodes.md));
+- every request resolves a server-created **principal** with explicit Project
+  **grants**; callers never supply identity, operator rights or grants
+  ([identity, authentication and tenancy](../../docs/maia-node/architecture.md#identity-authentication-and-tenancy));
+- enrolled **Execution Nodes** (your computer, a VPS, other servers) provide
+  capacity through one-time pairing and Node-issued certificates;
+- **A2A** peers map each accepted task to one durable Maia task and never
+  receive worker leases or mutation authority.
+
+## Development
+
+```sh
+pnpm --filter @maiaos/node build
+pnpm --filter @maiaos/node lint
+pnpm --filter @maiaos/node typecheck
+pnpm --filter @maiaos/node test
+pnpm test:resource-diagnostic        # from the repository root
+pnpm --filter @maiaos/node test:observation-benchmark
+```
+
+The observation benchmark commits through a real SQLite journal after seeded
+idle gaps of 0–3 s and times delivery to an SDK subscriber over loopback; it
+fails above a p95 of 50 ms and writes `.artifacts/benchmarks/`. Measured on an
+Apple M4 Pro (12 cores, macOS 27.2, Node 24.19.0) while parallel test runs
+loaded the machine (load average 5–32), 100 samples each:
+
+| Measure                                | Before S12a              | After S12a           |
+| -------------------------------------- | ------------------------ | -------------------- |
+| Commit to subscriber, p50 / p95 / max  | 3,410 / 4,874 / 4,957 ms | 1.9 / 4.7 / 19.8 ms  |
+| Idle CPU per 60 s, 8 subscribers       | 222 ms (165–279)         | 267 ms (207–334)     |
+| 10,000 commits, 142 forced resumptions | not measured             | none lost or doubled |
+
+The idle row is the median and range of six alternating runs each; the spread
+between runs is wider than the difference. The new wait races two Effect
+fibers where the old loop slept, about 60 µs per wait in a microbenchmark,
+roughly 6 ms of CPU per minute for eight idle subscribers.
+
+The S12b acceptance test (`src/runtime/node-api/observation/attach.integration.test.ts`)
+drives a synthetic harness into the real hub, the real Node API server, the
+SDK and the in-process `maia session attach` command, and times each delta
+until the CLI writes it. Same machine and load conditions:
+
+| Measure (S12b)                                    | Result                             |
+| ------------------------------------------------- | ---------------------------------- |
+| Harness delta to CLI line, 1 streaming Session    | p50 18.8 ms, p95 36.0 ms (150)     |
+| Harness delta to CLI line, 100 streaming Sessions | p50 21.1 ms, p95 39.3 ms (900)     |
+| Durable writes per streaming Session              | at most 2.78 per second            |
+| Catch-up of 1,000 records to `synchronized`       | p50 15.8 ms, p95 22.9 ms (10 runs) |
+| Viewer beside a stalled subscriber                | p95 36.1 ms; stalled one: `resync` |
+
+At 100 Sessions six viewers attach (the observation socket admits eight), and
+every Session streams a delta every 20 ms. S12c runs the same test with each
+delta sent as an Agent SDK partial message through the Claude adapter:
+
+| Measure (S12c, Claude adapter)                    | Result                         |
+| ------------------------------------------------- | ------------------------------ |
+| Harness delta to CLI line, 1 streaming Session    | p50 18.8 ms, p95 34.6 ms (150) |
+| Harness delta to CLI line, 100 streaming Sessions | p50 19.2 ms, p95 37.2 ms (900) |
+| Durable writes per streaming Session              | at most 2.55 per second        |
+
+Tests use deterministic clocks, identifiers, transports and disposable SQLite.
+Package tests get a throwaway `MAIA_HOME`; never point a test at your real
+`~/.maia`.
+
+## Documentation
+
+- [Node architecture](../../docs/maia-node/architecture.md): Sessions,
+  activations, supervision, events, observation streaming and recovery
+- [Operations](../../docs/operations.md): deployment, SQLite, identity, backup
+  and observability
+- [External brain and vendor executors](../../docs/maia-swarm/external-executors.md):
+  executor, account and containment decisions
+- [Release, update and compatibility](../../docs/release.md): the planned
+  installer, service and update flow
+- Scope READMEs: [harness adapters](./src/harness/README.md) and
+  [Project and Workspace storage](./src/project-workspace/README.md)
+
+## License
+
+No license has been published yet. All rights reserved.