@corenel/sidecar 0.10.0 → 0.13.0

package/README.md

@@ -4,8 +4,9 @@ A small local capability server for [Prompd](https://prompd.app). Run it next to
 web app and the browser can reach your machine — most usefully, your **local LLMs**
 (Ollama, LM Studio, vLLM) **without CORS**, plus your filesystem and (opt-in) shell.
 
-It runs only while the command is open, binds to loopback by default, and pairs with
-a token the app reads once.
+It runs only while the command is open, binds to loopback by default, and pairs each
+device once with a single-use code; after that the device proves its own key, and the
+person using it proves who they are (see [Identity](#identity-and-who-may-connect)).
 
 ## Use
 
@@ -15,8 +16,10 @@ In Prompd, open **Connect a sidecar** and follow the steps, or run it directly:
 npx @corenel/sidecar --allow-origin https://prompd.app
 ```
 
-It prints a pairing token; paste that into the app. The `--allow-origin` flag must
-match the site you're connecting from (the app fills it in for you).
+It prints a pairing code and the sidecar's fingerprint; paste them into the app. The
+`--allow-origin` flag must match the site you're connecting from (the app fills it in
+for you). Sign in to the app first: only the account the sidecar is signed in as
+(`corenel-sidecar login`) may connect, or with `--mode team`, members of its org.
 
 ## Options
 
@@ -26,8 +29,12 @@ match the site you're connecting from (the app fills it in for you).
 | `--port <n>` | `4858` | Port to listen on. |
 | `--root <dir>` | cwd | Root the file tools are confined to. |
 | `--allow-shell` | off | Enable the `sidecar_shell_exec` tool (RCE on your machine — opt-in). |
-| `--host <addr>` | `127.0.0.1` | Bind address. Non-loopback is reachable off-box; keep the token secret. |
+| `--host <addr>` | `127.0.0.1` | Bind address. Non-loopback is reachable off-box and needs TLS (below, `--allow-lan`, a tunnel, or `--behind-tls-proxy`); the daemon refuses to serve plain `ws://` off the machine. |
+| `--behind-tls-proxy` | off | A proxy in front terminates TLS (e.g. a container ingress): allows a non-loopback bind without a certificate of its own. |
+| `--mode team` `--org <org id>` | solo | Team mode: members of the Clerk organization may connect (`corenel-sidecar config set org <org id>`; the ID starts with `org_`, from the Clerk dashboard). Solo: only the account the daemon is signed in as. |
 | `--tls-cert <f>` `--tls-key <f>` | _(none)_ | Serve `wss://` (needed for a remote sidecar from an https page). |
+| `--name <text>` | generated | The name the app shows for this sidecar (daemon.json `name`). Without it the name is generated, `<hostname> · <first root folder>`, plus ` :<port>` off the default port, so several sidecars on one machine are told apart. The banner prints it. |
+| `--env <name>` | _(none)_ | Layer `daemon.<name>.json` over `daemon.json` (same state dir), under flags. See [Config environments](#config-environments). |
 
 ## Daemon (preview)
 
@@ -143,7 +150,7 @@ Manual smoke (manual/client-commanded trigger):
    to use a state dir other than the default, or `--default-model <id>` to give
    agents with no `settings.model` a fallback.
 
-4. **Pair from the app** with the printed token, then command a manual run of the
+4. **Pair from the app** with the printed code, then command a manual run of the
    agent you created. Confirm the reply streams back in the app as it's generated,
    and that a new session appears under `<state dir>/crew/<name>/sessions/<runId>/`.
 
@@ -151,10 +158,249 @@ Entitlement for daemon runs is enforced at the gateway, not locally — a 401/40
 the run means the login token from step 1 doesn't carry daemon access, not a bug in
 the sidecar itself.
 
+## Config environments
+
+`daemon.json` (in the state dir) is the machine's own settings, written by
+`corenel setup`/`config`/`roots`. `--env <name>` (or `CORENEL_ENV=<name>`)
+layers a second file, `daemon.<name>.json`, on top of it, in the same state
+dir: `defaults < daemon.json < daemon.<name>.json < flags`. Every key present
+in the named file replaces the base value outright (an overlay `roots` array
+replaces the base array, never merges into it); a flag still wins over both.
+
+The first use is pointing a local checkout at a local API instead of the
+production one, without hand-editing `daemon.json` back and forth:
+
+```bash
+echo '{ "identityIssuer": "http://localhost:3010" }' > ~/.corenel/daemon.local.json
+corenel --env local
+```
+
+The name is restricted to `^[a-z0-9-]{1,32}$` (it becomes part of a filename) —
+anything else is refused rather than resolved into a path. Requesting a named
+environment that has no `daemon.<name>.json`, or one that is corrupt or
+carries a credential, fails the command rather than silently falling back to
+the base file: unlike a missing `daemon.json`, a named environment was asked
+for by name. `corenel doctor` and `corenel config get`/`list` (and `corenel
+roots list`/`corenel mcp list`) show the active environment and, per value,
+whether it came from the base file, the overlay, or a flag.
+
+**Writes.** An EXPLICIT `--env <name>` on `corenel config set|unset|add|remove`
+redirects that one write to `daemon.<name>.json` instead of `daemon.json` —
+this is how you create an environment file in the first place; a missing
+overlay is created (`{ "version": 1 }` plus the change) rather than refused.
+`CORENEL_ENV` alone does NOT redirect a write — a shell profile exporting it
+must not silently turn every `config set` into an overlay edit — so a write
+with only the env var set still lands on `daemon.json`, with one stderr note
+saying so. Every OTHER write on this plane (`corenel roots add`, `corenel mcp
+add`, `corenel setup`, and a folder added from the app) always lands on
+`daemon.json`, never an overlay, regardless of `--env`/`CORENEL_ENV`.
+
+```bash
+corenel config set --env local identityIssuer http://localhost:3010
+corenel config add --env local allowOrigins https://127.0.0.1:5174
+corenel --env local
+```
+
+**Caveat:** if the active overlay defines its own `roots`, a root added at
+runtime (`corenel roots add`, or a folder added from the app) still saves to
+`daemon.json` as always, but the overlay's `roots` replaces the base array
+wholesale on the next start under that same environment — so the just-added
+root is shadowed until it is removed from `daemon.<name>.json`, or added
+there directly. The add command's own output says so when this applies.
+
+**Discovery.** `corenel config envs` lists every `daemon.*.json` file in the
+state dir (never `daemon.json` itself — that is the base file, not an
+environment): for each one whose name passes the same rule, whether it parses
+and the keys it sets; a file whose name does not pass the rule is listed
+separately as "ignored (invalid name)" rather than silently skipped. Marks
+whichever one `--env`/`CORENEL_ENV` currently selects. Prints one note, and
+nothing else, when none exist.
+
+**In the app.** A connected daemon's active environment shows up next to its
+name in the connection UI, e.g. `laptop (env: local)` — the welcome
+handshake's `info.env`, present only when one is active.
+
+## Web push
+
+On by default — no flag needed to advertise it. When a permission ask's
+owner is not attached but has a subscribed device, the daemon pushes a
+notification and holds the ask for a bounded wait (`pushWaitMs`, default
+90s) before falling through to the run's unattended policy, giving the
+owner a chance to open the app and answer from wherever they are.
+
+Pushes are sent only for runs started from a connected browser or device —
+those asks carry the owning connection's `clientId`, which is what a
+subscription is filed under. Scheduled and trigger-fired crew runs are
+started by the daemon itself, have no owning connection, and **do not push
+yet**; their asks go to whoever is attached, or straight to the unattended
+policy.
+
+Every link in that chain — VAPID signing, the subscribe handshake, the
+persisted store, the push-and-wait timing, the service worker's
+`push`/`notificationclick` handlers — is built and unit-tested. Real
+delivery to a phone with the tab closed has not yet been observed; see
+`docs/competitive-landscape.md`'s "Push, precisely" note for the test
+list and what that gap means.
+
+- **`--no-push`** turns the whole feature off (`daemon.json`: `push: false`).
+  With no subscriptions, push already does nothing — this flag matters for
+  the PHI/self-host story, where even the metadata below must not leave the
+  network.
+- **Egress.** The push body is end-to-end encrypted (RFC 8291/8188) — Apple
+  or Google, whoever runs the browser's push service, cannot read it. But
+  they DO learn that a notification was sent, when, and to which
+  subscription. That is a real egress for a self-hosted or PHI deployment
+  and is why `--no-push` exists.
+  A daemon with no outbound internet cannot deliver, and the one
+  difference from a daemon without this feature is time: the send fails
+  silently (never throws into the run), but an owner with a subscription on
+  file still makes the ask wait (`pushWaitMs` under a deny/allow policy; a
+  `park` policy waits its own bound) before the run applies its unattended
+  policy — later than it otherwise would, never a hang.
+  A push failure of any kind — a throw, a timeout (each send is bounded at
+  10s), a refusal — never fails the run; the ask reaches the same outcome
+  it would have reached without push, only later by that bounded wait.
+- **Endpoints are restricted to known push services.** The daemon POSTs to
+  a subscription's endpoint, so an open-ended URL would let any paired
+  device make it send requests to any host it can reach. A
+  subscription is refused unless its endpoint is `https://` on the default
+  port, carries no credentials, and its host is — or is a subdomain of —
+  `fcm.googleapis.com` (Chrome/Chromium), `push.services.mozilla.com`
+  (Firefox), `push.apple.com` (Safari) or `notify.windows.com` (Edge/
+  Windows). The list is `WEB_PUSH_SERVICE_HOSTS` in `@corenel/protocol`;
+  stored subscriptions outside it are dropped (and logged) on load.
+- **Where things live**, both under `<stateDir>/sidecar/`:
+  - `vapid.json` — the daemon's own VAPID keypair, generated on first use,
+    mode `0600`, written atomically. Deliberately NOT `auth.json`: signing
+    out deletes `auth.json`, and every browser subscription is bound to
+    this public key. **Deleting `vapid.json` orphans every subscribed
+    device** — each one is bound to the key that no longer exists, and
+    there is no re-linking; they have to click "Notify me on this device"
+    again once the daemon mints a new key. A new key is minted only when the
+    file does not exist. A file that is not a valid keypair is moved aside
+    to `vapid.json.corrupt-<timestamp>` with a loud log before a new key is
+    minted; any other read failure (permissions, a locked file) disables
+    push for that start, with a log line, rather than minting over a key
+    that may be fine.
+  - `push-subscriptions.json` — subscriptions, deduped by endpoint, capped
+    at 10 per client and 500 total, pruned automatically on a `404`/`410`
+    from the push service (the browser dropped the subscription) and after
+    5 consecutive `401`/`403`s (the device subscribed to a key this daemon
+    no longer signs with). A corrupt file is moved aside
+    (`push-subscriptions.json.corrupt-<timestamp>`) and logged; an
+    unreadable one disables push for that start.
+- A subscription is bound to one daemon's key; subscribing to a second
+  daemon replaces it — pairing with a new daemon and clicking "Notify me on
+  this device" moves the browser's one push registration to that daemon.
+- The owner's OTHER devices only count as reachable when they share a
+  **verified** Clerk user id, never a claimed account alone — see
+  `docs/team-agents.md` §5.2. That id comes from the attendance registry,
+  which refuses to answer for a `clientId` seen verified as two different
+  users (a shared browser); such an owner reaches only its own
+  subscriptions. A browser re-sends its existing subscription on every
+  pairing, so the id it is filed under follows an account switch.
+
+## Identity and who may connect
+
+Pairing proves two things on every connection: the DEVICE (a key the daemon
+recorded when the device enrolled with a single-use pairing code; see the
+device enrollment spec) and the PERSON (a verified identity). Both are
+required; there is no switch to turn identity off. A connection that cannot
+prove both gets the same `denied: invalid credentials` as any other bad
+credential, and nothing is served before it is authenticated.
+
+**Who may connect** follows the existing `--mode team` switch:
+
+- **Solo** (the default): only the account this daemon is signed in as
+  (`corenel-sidecar login`, or `CORENEL_TOKEN`). The daemon learns that
+  account's Clerk user id at startup from `GET <endpoint>/auth/me` with its
+  own credential, re-reads it every 15 minutes and whenever its sign-in
+  changes, and FAILS CLOSED: until it knows, no one may connect, and the
+  banner's `who may connect:` line says why. An `org` set in solo mode is
+  ignored, with a log line saying so.
+- **Team** (`--mode team`): only members of ONE Clerk organization, named by
+  its ID (`org_...`, from the Clerk dashboard; never a slug, which can be
+  renamed or taken by another org) with `corenel-sidecar config set org <org id>`
+  (daemon.json `org`, or `--org <org id>`). Team mode without a valid org ID
+  refuses to start, naming that command. Membership is checked by corenel-api
+  on every connect against Clerk's membership list for the verified user (not
+  the session's active org), cached at most 60 s; the daemon admits an
+  assertion whose `org_id` claim is its own org ID. Any role counts. A device
+  belongs to the first member who connected on it. Every open connection is re-challenged every 15 minutes
+  and closed when the same user cannot re-assert membership, so a removed
+  member is out within 15 minutes. Over the wire a member may revoke only
+  their own devices (those recorded under their verified user), and may read
+  a run's output (reports, check output, files) or watch it only for runs
+  they started;
+  the operator's local `corenel-sidecar devices revoke` keeps full power.
+
+**The exchange.** Once the device key is proven (and, for an enrollment,
+before the code is burned), the daemon sends `identity-challenge` naming its
+own `sidecarId`, a fresh single-use nonce and, in team mode, its org. The
+client obtains a one-minute RS256 assertion from corenel-api's
+`POST /api/v1/pairing-assertions` (`{ aud, nonce, jkt, org_id? }`): a browser
+with its Clerk session token, the `corenel` CLI with its `cnl_cli_` login
+token, which must carry the `pair` scope (granted only when the person
+approving `corenel login` ticks "Also let this machine connect to your
+sidecars"; without it, log in again). That token goes to corenel-api and nowhere else; it never crosses the
+daemon socket. The client answers with `identity-assert`, and the daemon
+verifies the assertion locally against corenel-api's OIDC discovery and JWKS:
+signature, `iss`, `exp`, `aud` (this daemon), `nonce` (this challenge),
+`azp` (an allowlisted origin, or `corenel-cli` for a CLI assertion) and
+`cnf.jkt`, which must be the thumbprint of the device key THIS connection
+proved. That binding stops a live relay: a daemon that forwards another
+daemon's challenge to a browser paired with it gets an assertion bound to
+that browser's key for IT, which the other daemon refuses; and a client only
+answers a challenge from the daemon that proved the key it pinned.
+The whole identity leg is bounded at 10 s from the challenge; a missing,
+late, oversized or invalid assertion is a refusal.
+
+**What the client can say.** The refusal is uniform on the wire, but the
+client knows when it had no identity to offer and says so: not signed in,
+not a member of the org, or the sign-in service could not vouch in time. The
+device is kept in those cases; only a device the daemon refused is
+forgotten.
+
+- **Default issuer:** `https://api.corenel.ai`. Point at a local corenel-api
+  with `--identity-issuer <url>` (daemon.json `identityIssuer`). It must be
+  `https://`, except a loopback host, which may use `http://`, and its
+  `jwks_uri` follows the same rule (same origin under a loopback issuer).
+- **Egress:** one discovery fetch plus one JWKS fetch per hour to the issuer,
+  cached together; an unknown `kid` refetches at most once a minute. In solo
+  mode, one `GET <endpoint>/auth/me` at startup and every 15 minutes.
+- The assertion is never logged or stored; only the verified user id is kept,
+  in memory, for the life of the connection (and recorded on the device).
+- **[D]** corenel-api only stamps `azp` for request origins in its own CORS
+  allowlist, so a page served from an origin missing there is refused.
+- **[D]** Both web apps call `https://api.corenel.ai/api/v1/pairing-assertions`
+  directly (`VITE_PAIRING_ASSERTION_URL`), never through a front end's own
+  `/api` proxy, whose single egress address would share one rate-limit bucket.
+- The runbook for the signing key (`corenel-pairing-signing-key`) lives in
+  corenel-api's `docs/pairing-signing-key.md`.
+
+**Before a connection authenticates** the daemon keeps its cost bounded: a
+daemon-wide limiter refuses an address after 10 failed hellos in a minute
+(keyed by IPv4 address or IPv6 /64; off with `--behind-tls-proxy`, where
+every peer is the proxy), lets one address hold at most 4 unauthenticated
+connections, and holds at most 32 daemon-wide (HTTP 429 and 503 at the
+upgrade); a plain HTTP request to the port gets
+`426`; request headers must arrive within 10 s; a frame over 64 KiB before
+the welcome closes the socket unparsed; and a hello that does not pair within
+20 s is closed.
+
+**`corenel doctor` Identity section.** Computed from the EFFECTIVE config: who
+the daemon admits (solo, or the team org; team mode without an org is a
+failure), the issuer URL and whether it passes the https/loopback rule, a
+real discovery fetch (the same code the verifier uses), whether the
+document's `issuer` matches, whether `jwks_uri` is accepted, how many usable
+RS256 keys it lists (zero is a failure), the trusted origins, and a warning
+when `--allow-lan` is on that a LAN-allowed page is refused unless its origin
+is also in `allowOrigins`. Included in `--json`.
+
 ## Security
 
-- Loopback by default; a browser connection requires both the pairing token **and** an
-  allowed Origin.
+- Loopback by default; a browser connection requires a recorded device key, a verified
+  identity this daemon admits, **and** an allowed Origin.
 - The HTTP proxy (for local LLMs) only reaches **loopback** hosts — it can't be used as
   an SSRF pivot to internal services.
 - `sidecar_shell_exec` is off unless you pass `--allow-shell`.