kylon-cli 0.1.0-next.100

package/README.md

@@ -0,0 +1,472 @@
+# kylon-cli
+
+Gateway CLI for connecting local agent providers to a P2 workspace.
+
+Requires Node.js 22 or newer.
+
+## Install
+
+The CLI is published to npm as
+[`kylon-cli`](https://www.npmjs.com/package/kylon-cli). The Web UI
+generates two separate commands — an **install** step and a **run**
+step — each with its own Copy button. Operators paste the install
+once per host (and any time they want to upgrade) and the run
+whenever they want to start the daemon.
+
+### Step 1 — Install kylon (once per host, re-paste to upgrade)
+
+```bash
+npm install -g kylon-cli@latest
+```
+
+Installs the CLI globally so the `kylon` binary lands on PATH — the
+gateway daemon's child `kylon workspace …` calls (issued by the
+provider subprocess) resolve it there. Re-run to upgrade to whatever
+the dist-tag now points to. If the global install needs elevated
+permissions, prefix it with `sudo`.
+
+The dist-tag tracks the environment: production installs `@latest`,
+dev installs `@next` (newest prerelease). To pin a specific build,
+install `kylon-cli@X.Y.Z`.
+
+Requirements: Node.js 22+ (npm ships with Node). Linux / macOS only
+(Windows operators should use WSL).
+
+### Step 2 — Start the gateway daemon
+
+```bash
+kylon gateway run \
+  --server-url https://<origin>/api \
+  --provider codex \
+  --api-key <agent-api-key>
+```
+
+`gateway run` registers the session (writing `~/.kylon/gateway-session.json`
+with mode `0600`) and starts the daemon in one step. On the next
+invocation, if a session already exists, plain `kylon gateway run`
+without flags picks it up and jumps straight to start. See
+[Usage → Run](#run) for the full decision table. The daemon runs
+in the foreground until you stop it with `Ctrl+C`; hosts that want a
+supervised daemon typically wrap the same command in
+`launchd` / `systemd` / `supervisord`.
+
+### Local development (contributors)
+
+```bash
+pnpm install
+pnpm --filter kylon-cli build
+```
+
+See [Development → Exposing `kylon` to provider subprocesses](#exposing-kylon-to-provider-subprocesses)
+for the debugger-friendly tsx-based dev loop.
+
+### Release channels
+
+npm dist-tags are the version pointer — there is no server-side version
+policy. Every merge to `main` publishes a prerelease to the `next` tag
+(`.github/workflows/cli-publish.yml`); cutting a `kylon-cli-vX.Y.Z`
+release tag publishes a stable to `latest`. dev installs `@next` and prd
+installs `@latest`, so each environment tracks its own train.
+
+Cutting a stable is a manual, when-ready step: tag a validated build
+`kylon-cli-vX.Y.Z` and push the tag; CI publishes it to `latest`. Don't
+`npm publish` ad-hoc — releases go through the CI workflow so the OIDC
+publish path is exercised end-to-end.
+
+## Usage
+
+### Run
+
+`gateway run` is the recommended entrypoint for external agent operators.
+It composes `connect` + `start` into a single command.
+
+```bash
+kylon gateway run \
+  --server-url https://api.p2.ai \
+  --provider codex \
+  --api-key pak_xxxxx
+```
+
+Decision table:
+
+| Saved session? | Flags passed? | What `run` does |
+|---|---|---|
+| no | `--server-url` + `--api-key` + `--provider` | connect, persist session, start daemon |
+| no | any field missing | error — lists the missing flag |
+| yes | none | start the daemon from the saved session |
+| yes | any | reconnect with the overrides (falls back to saved values for fields you didn't pass), persist the refreshed session, start the daemon |
+
+Only CLI flags count as overrides. Setting `KYLON_API_KEY` in the
+environment does **not** trigger a reconnect on a saved session —
+it still feeds the normal API-key resolution inside `gateway start`.
+
+### Connect
+
+Register this machine as the gateway client for an external agent.
+Most operators should prefer `gateway run` above. Use `connect` on its
+own when scripting or when you need to register a session without
+immediately starting the daemon.
+
+```bash
+kylon gateway connect \
+  --server-url https://api.p2.ai \
+  --api-key pak_xxxxx \
+  --provider codex
+```
+
+The agent's API key identifies which agent the daemon will serve;
+channels are bound separately via the "invite agent into channel" flow
+in the web UI.
+
+### Bind
+
+Create or update a logical session binding for an agent. Run from the
+target working directory:
+
+```bash
+kylon gateway bind \
+  --agent agent_123 \
+  --provider codex
+```
+
+Switch an existing binding to a different directory:
+
+```bash
+cd /path/to/other/repo
+kylon gateway bind --agent agent_123 --workdir .
+```
+
+Bindings are keyed by `(gateway session, agent)`. The same agent
+behaves the same way regardless of which channel an assignment
+arrives on — see
+[`docs/journal_docs/05_27_gateway_routing_simplification.md`](../../docs/journal_docs/05_27_gateway_routing_simplification.md).
+
+### Commands
+
+| Command | Description |
+|---|---|
+| `kylon gateway run` | **Recommended.** Connect + start in one step; idempotent when a session already exists. |
+| `kylon gateway connect` | Register this machine as the gateway client for an agent (API key identifies which), without starting the daemon. |
+| `kylon gateway bind` | Create or update a logical session binding. |
+| `kylon gateway start` | Start the gateway daemon from a saved session — opens the SSE stream and executes assignments. Reads the API key from the saved session. |
+
+### Run Options
+
+| Flag | Description |
+|---|---|
+| `--server-url <url>` | P2 server URL (required for the first run; override otherwise) |
+| `--api-key <key>` | Agent API key, e.g. `pak_xxx` (required for first run; override otherwise. `KYLON_API_KEY` env var satisfies the first-run requirement but does not count as an override on subsequent runs) |
+| `--provider <name>` | Provider CLI: `codex`, `claude-code`, `hermes`, `openclaw`, `generic` (required for first run; override otherwise) |
+
+### Connect Options
+
+| Flag | Description |
+|---|---|
+| `--server-url <url>` | P2 server URL (required) |
+| `--api-key <key>` | Agent API key, e.g. `pak_xxx` (required, or set `KYLON_API_KEY`) |
+| `--provider <name>` | Provider CLI: `codex`, `claude-code`, `hermes`, `openclaw`, `generic` (required) |
+
+### Bind Options
+
+| Flag | Description |
+|---|---|
+| `--agent <id>` | Agent ID (required) |
+| `--provider <name>` | Provider CLI (required for new, optional for update) |
+| `--workdir <path>` | Working directory (default: current directory) |
+
+### Supported Providers
+
+- `codex` — OpenAI Codex CLI
+- `claude-code` — Anthropic Claude Code CLI
+- `hermes` — Hermes CLI using `hermes -z <prompt>`
+- `openclaw` — OpenClaw CLI using `openclaw agent --message <prompt> --json`
+- `generic` - provider-neutral wrapper. Runs a `kylon-provider`
+  executable on `PATH` and expects newline-delimited JSON events matching
+  `docs/journal_docs/05_25_external_agent_provider_adapter_contract.md`.
+
+## State Model
+
+The CLI uses a three-layer state model:
+
+- **GatewaySession** — authenticated connection to the P2 server (one per machine)
+- **LogicalSessionState** — per-`(gateway session, agent)` binding holding the current workdir and provider
+- **ProviderRuntimeEntry** — per-conversation provider resume cache, keyed by `(gateway session, channel, agent, scope, provider, workdir)` (disposable)
+
+Switching workdir updates the logical session without creating a new one. Provider runtimes are cached per workdir + conversation scope — switching back resumes the old runtime.
+
+State is persisted to `~/.kylon/` (or `$XDG_CONFIG_HOME/kylon/`).
+
+## Development
+
+> This section assumes you cloned `fre-so/p2` and ran `pnpm install` at the
+> repo root. All commands are run from anywhere in the monorepo unless
+> otherwise noted.
+
+### Inner loop
+
+```bash
+# type-check without emitting
+pnpm --filter kylon-cli typecheck
+
+# tsc build — emits dist/bin/kylon.js and other .js files
+pnpm --filter kylon-cli build
+
+# unit tests (no network, no DB)
+pnpm --filter kylon-cli test
+
+# run the locally built CLI
+node packages/cli/dist/bin/kylon.js --help
+```
+
+The Web agent-settings "Local dev" block generates the same
+`node packages/cli/dist/bin/kylon.js …` invocation from the logged-in
+agent's API key. If you change `src/bin/kylon.ts` or anything it
+imports, rerun `pnpm --filter kylon-cli build` before re-executing.
+
+### Exposing `kylon` to provider subprocesses
+
+`node packages/cli/dist/bin/kylon.js gateway run …` starts the daemon
+but leaves **no `kylon` binary on PATH**. When the provider subprocess
+(claude-code / codex) then tries `kylon workspace …` via its Bash tool,
+the shell fails with `command not found`. Two ways to fix this:
+
+**Option 1 — `pnpm link` (persistent).** Symlink kylon's published
+`bin.kylon` into pnpm's global bin dir:
+
+```bash
+pnpm --filter kylon-cli bundle            # or bundle:minify for a prod-shaped build
+pnpm --filter kylon-cli link --global
+
+which kylon
+# → ~/Library/pnpm/kylon (or similar) → packages/cli/dist/kylon-bundle.mjs
+```
+
+From here on, start the daemon via the linked binary instead of the
+raw `node dist/bin/kylon.js`:
+
+```bash
+kylon gateway run --server-url http://localhost:5173/api --provider codex --api-key pak_…
+```
+
+Every provider call to `kylon workspace …` resolves to the linked
+binary. Edit source → `pnpm --filter kylon-cli bundle` → next provider
+call picks it up (each workspace invocation is a fresh process;
+restart the daemon only for *daemon*-side edits). Clean up with
+`pnpm --filter kylon-cli unlink --global`.
+
+**Option 2 — `--dev-cli-shim` (ephemeral, IDE-friendly).** The daemon
+can install a temporary bash shim that execs the TS source through
+tsx. Pass `--dev-cli-shim <abs-path-to-p2-repo>` on `gateway run` or
+`gateway start`:
+
+```bash
+node packages/cli/dist/bin/kylon.js gateway run \
+  --server-url http://localhost:5173/api \
+  --provider codex \
+  --api-key pak_… \
+  --dev-cli-shim "$(pwd)"
+```
+
+The daemon prints the shim location on startup:
+
+```text
+[dev] kylon shim: /tmp/kylon-dev-shim-abc123/kylon
+[dev] provider calls will exec: node --import tsx /abs/path/packages/cli/src/bin/kylon.ts
+```
+
+The tmpdir is prepended to the daemon's `PATH`, so the provider
+subprocess (and the bash shell it spawns) resolves `kylon` to the
+shim. The shim execs `node --import tsx <src>`, so:
+
+- Every `kylon workspace …` invocation reads the current `src/*.ts` —
+  no bundle rebuild needed between edits.
+- A Node debugger attached to the daemon is inherited by each shim
+  invocation (they exec `node`, so `NODE_OPTIONS` and VS Code's
+  Auto-Attach loader pass through).
+- On SIGINT/SIGTERM the tmpdir is wiped.
+
+This flag is **only compiled into local (non-minified) builds**. The
+npm package and any `bundle:release` output reject `--dev-cli-shim` as
+an unknown argument and omit it from `--help`, so it can never
+accidentally ship.
+
+### Debugging both the daemon and `kylon workspace` calls
+
+Combine `--dev-cli-shim` with a Node debugger to step through the
+full chain — daemon → provider subprocess → `kylon workspace …` — in
+one IDE session.
+
+**VS Code**:
+
+1. Enable `Debug: Toggle Auto Attach → Always` (or `Only With Flag`).
+   VS Code prepends its `js-debug` bootloader to `NODE_OPTIONS`, which
+   every child Node process — including the ones launched by the
+   shim — inherits.
+2. Open an integrated terminal and run:
+
+   ```bash
+   node packages/cli/dist/bin/kylon.js gateway run \
+     --server-url http://localhost:5173/api \
+     --provider codex \
+     --api-key pak_… \
+     --dev-cli-shim "$(pwd)"
+   ```
+
+3. Set breakpoints in both `packages/cli/src/commands/gateway-start.ts`
+   (daemon) and `packages/cli/src/commands/workspace/*.ts` (child
+   commands). Both fire the next time the provider issues a workspace
+   call.
+
+**JetBrains / others**: export `NODE_OPTIONS=--inspect=0.0.0.0:0`
+before launching the daemon. Every subsequent Node process — daemon
+and every `kylon workspace …` invocation — opens its own inspector
+port. Attach your IDE to the process list.
+
+No debugger attach? The shim still works — source edits are picked up
+on the next provider call, but breakpoints just don't fire.
+
+### Bundles
+
+The release artifact is a single-file ESM bundle produced by
+`scripts/bundle.mjs`. Three variants:
+
+| Script | Output | Passes to `bundle.mjs` | Use when |
+|---|---|---|---|
+| `pnpm --filter kylon-cli bundle` | `dist/kylon-bundle.mjs` | _(none)_ | debugging the bundled shape while keeping readable names — not shipped |
+| `pnpm --filter kylon-cli bundle:minify` | `dist/kylon-bundle.mjs` | `--minify` | reproducing the pre-obfuscation size and behavior for a bisect |
+| `pnpm --filter kylon-cli bundle:release` | `dist/kylon-bundle.mjs` | `--minify --obfuscate` | what ships on npm |
+
+The `bundle:release` path runs esbuild with `--minify`, then passes the
+output through `javascript-obfuscator`. It's also what the `prepack` hook
+runs, so `pnpm pack` / `npm publish` always produce the obfuscated shape
+even if you forget to call `bundle:release` explicitly.
+
+Run the bundle directly to sanity-check it:
+
+```bash
+pnpm --filter kylon-cli bundle:release
+node packages/cli/dist/kylon-bundle.mjs --help
+```
+
+### Testing
+
+Unit tests run against Node's built-in test runner (`node:test`) plus
+`tsx`. They touch the filesystem inside temp dirs but never the
+network or a database, so `pnpm --filter kylon-cli test` is safe to
+run anywhere.
+
+E2E tests drive the CLI as a subprocess against a configurable mock
+or live server:
+
+```bash
+# headless e2e (mock provider processes spawned from scripts/mock-*.mjs)
+pnpm --filter kylon-cli test:e2e
+
+# live e2e against a real P2 environment — requires doppler secrets
+doppler run --project p2 --config prd -- pnpm --filter kylon-cli test:e2e:live
+```
+
+Live E2E tests provision a throwaway workspace via the REST API, so
+expect them to take several minutes and to leave audit trail rows in
+the target environment. Do not point them at production casually.
+
+Before shipping a release, also smoke-test the packaged artifact
+exactly as operators will receive it:
+
+```bash
+# 1. produce the release bundle + tarball
+pnpm --filter kylon-cli bundle:release
+cd packages/cli && pnpm pack --pack-destination /tmp/kylon-out
+
+# 2. install the tarball in a clean directory
+WORK=$(mktemp -d) && cd "$WORK"
+npm init -y >/dev/null
+npm install /tmp/kylon-out/kylon-cli-*.tgz
+
+# 3. the published shape should have only three entries
+ls node_modules/kylon-cli               # dist/  package.json  README.md
+ls node_modules/kylon-cli/dist          # kylon-bundle.mjs
+
+# 4. verify the binary is runnable
+node_modules/.bin/kylon --help
+
+# 5. confirm the bundle is actually obfuscated
+head -c 200 node_modules/kylon-cli/dist/kylon-bundle.mjs
+# expect `#!/usr/bin/env node` followed by hexadecimal identifier soup,
+# not recognizable function names or source strings
+```
+
+### Obfuscation
+
+`bundle:release` and `prepack` run `javascript-obfuscator` with a
+conservative preset chosen for runtime safety:
+
+- **On:** `compact`, `identifierNamesGenerator: "hexadecimal"`,
+  `stringArray` with `base64` encoding + rotate + shuffle, two
+  wrapper function layers.
+- **Off:** `controlFlowFlattening`, `deadCodeInjection`,
+  `selfDefending`, `debugProtection`, `unicodeEscapeSequence`. These
+  trade correctness and startup latency for marginal protection — do
+  not turn them on without measuring startup and rerunning the full
+  test:e2e suite.
+- `renameGlobals` stays off so Node built-ins keep their names.
+
+Startup stays under 100 ms on modern hardware; bundle grows from
+~85 KB (minify only) to ~230 KB (obfuscated). Obfuscation is a
+tampering and casual-reading deterrent, not a security control — the
+API server is the security boundary.
+
+## Release
+
+The CLI ships via **npm dist-tags** — there is no server-side version policy.
+`cli-verify.yml` validates on PRs and pushes; `cli-publish.yml` publishes via
+OIDC (no token): a prerelease to the `next` tag on every merge to `main`, and a
+stable to `latest` when a `kylon-cli-vX.Y.Z` tag is pushed. dev installs
+`@next`, prd installs `@latest`.
+
+### Versioning
+
+Use explicit semver in `packages/cli/package.json`:
+
+| Bump | When |
+|---|---|
+| patch (`0.1.0 → 0.1.1`) | bug fix, no new flags, no behavior change |
+| minor (`0.1.x → 0.2.0`) | backward-compatible capability (new command, new flag) |
+| major (`0.x.x → 1.0.0`) | breaking change to CLI contract or runtime behavior |
+
+Prerelease builds are versioned `0.1.0-next.<run>` automatically by
+`cli-publish.yml`; a stable release is whatever you tag — bump
+`packages/cli/package.json` to the target semver, then push the matching
+`kylon-cli-vX.Y.Z` tag.
+
+### Publishing
+
+- **Prerelease (automatic):** every merge to `main` touching the CLI publishes
+  `0.1.0-next.<run>` to the `next` tag via `cli-publish.yml` (OIDC, no
+  `NPM_TOKEN`). dev tracks this tag, so dev always dogfoods the newest build.
+- **Stable (manual, when ready):** bump `packages/cli/package.json` to the
+  target version, merge, then tag the commit on `main`:
+
+  ```bash
+  git pull
+  git tag kylon-cli-v0.1.1
+  git push origin kylon-cli-v0.1.1   # cli-publish.yml publishes it to `latest`
+  npm view kylon-cli dist-tags        # verify
+  ```
+
+  Do not `npm publish` ad-hoc — releases go through the workflow so the OIDC
+  provenance path is exercised end to end.
+
+### Rollback
+
+npm publishes are immutable; roll back by re-pointing the tag, not by
+unpublishing:
+
+```bash
+npm dist-tag add kylon-cli@<last-good> latest   # move the channel back
+npm deprecate kylon-cli@<bad> "Broken release — use <last-good>."
+```
+
+Operators on `@latest` pick up the re-pointed version the next time they
+`npm install -g kylon-cli@latest`; in-flight daemons keep their current CLI
+until they restart. Do not `npm unpublish`.