@corenel/sidecar 0.9.0 → 0.11.0

package/README.md

@@ -28,6 +28,7 @@ match the site you're connecting from (the app fills it in for you).
 | `--allow-shell` | off | Enable the `sidecar_shell_exec` tool (RCE on your machine — opt-in). |
 | `--host <addr>` | `127.0.0.1` | Bind address. Non-loopback is reachable off-box; keep the token secret. |
 | `--tls-cert <f>` `--tls-key <f>` | _(none)_ | Serve `wss://` (needed for a remote sidecar from an https page). |
+| `--env <name>` | _(none)_ | Layer `daemon.<name>.json` over `daemon.json` (same state dir), under flags. See [Config environments](#config-environments). |
 
 ## Daemon (preview)
 
@@ -151,6 +152,275 @@ Entitlement for daemon runs is enforced at the gateway, not locally — a 401/40
 the run means the login token from step 1 doesn't carry daemon access, not a bug in
 the sidecar itself.
 
+## Config environments
+
+`daemon.json` (in the state dir) is the machine's own settings, written by
+`corenel setup`/`config`/`roots`. `--env <name>` (or `CORENEL_ENV=<name>`)
+layers a second file, `daemon.<name>.json`, on top of it, in the same state
+dir: `defaults < daemon.json < daemon.<name>.json < flags`. Every key present
+in the named file replaces the base value outright (an overlay `roots` array
+replaces the base array, never merges into it); a flag still wins over both.
+
+The first use is pointing a local checkout at a local API instead of the
+production one, without hand-editing `daemon.json` back and forth:
+
+```bash
+echo '{ "identityIssuer": "http://localhost:3010" }' > ~/.corenel/daemon.local.json
+corenel --env local
+```
+
+The name is restricted to `^[a-z0-9-]{1,32}$` (it becomes part of a filename) —
+anything else is refused rather than resolved into a path. Requesting a named
+environment that has no `daemon.<name>.json`, or one that is corrupt or
+carries a credential, fails the command rather than silently falling back to
+the base file: unlike a missing `daemon.json`, a named environment was asked
+for by name. `corenel doctor` and `corenel config get`/`list` (and `corenel
+roots list`/`corenel mcp list`) show the active environment and, per value,
+whether it came from the base file, the overlay, or a flag.
+
+**Writes.** An EXPLICIT `--env <name>` on `corenel config set|unset|add|remove`
+redirects that one write to `daemon.<name>.json` instead of `daemon.json` —
+this is how you create an environment file in the first place; a missing
+overlay is created (`{ "version": 1 }` plus the change) rather than refused.
+`CORENEL_ENV` alone does NOT redirect a write — a shell profile exporting it
+must not silently turn every `config set` into an overlay edit — so a write
+with only the env var set still lands on `daemon.json`, with one stderr note
+saying so. Every OTHER write on this plane (`corenel roots add`, `corenel mcp
+add`, `corenel setup`, and a folder added from the app) always lands on
+`daemon.json`, never an overlay, regardless of `--env`/`CORENEL_ENV`.
+
+```bash
+corenel config set --env local identityIssuer http://localhost:3010
+corenel config add --env local allowOrigins https://127.0.0.1:5174
+corenel --env local
+```
+
+**Caveat:** if the active overlay defines its own `roots`, a root added at
+runtime (`corenel roots add`, or a folder added from the app) still saves to
+`daemon.json` as always, but the overlay's `roots` replaces the base array
+wholesale on the next start under that same environment — so the just-added
+root is shadowed until it is removed from `daemon.<name>.json`, or added
+there directly. The add command's own output says so when this applies.
+
+**Discovery.** `corenel config envs` lists every `daemon.*.json` file in the
+state dir (never `daemon.json` itself — that is the base file, not an
+environment): for each one whose name passes the same rule, whether it parses
+and the keys it sets; a file whose name does not pass the rule is listed
+separately as "ignored (invalid name)" rather than silently skipped. Marks
+whichever one `--env`/`CORENEL_ENV` currently selects. Prints one note, and
+nothing else, when none exist.
+
+**In the app.** A connected daemon's active environment shows up next to its
+name in the connection UI, e.g. `laptop (env: local)` — the welcome
+handshake's `info.env`, present only when one is active.
+
+## Web push
+
+On by default — no flag needed to advertise it. When a permission ask's
+owner is not attached but has a subscribed device, the daemon pushes a
+notification and holds the ask for a bounded wait (`pushWaitMs`, default
+90s) before falling through to the run's unattended policy, giving the
+owner a chance to open the app and answer from wherever they are.
+
+Pushes are sent only for runs started from a connected browser or device —
+those asks carry the owning connection's `clientId`, which is what a
+subscription is filed under. Scheduled and trigger-fired crew runs are
+started by the daemon itself, have no owning connection, and **do not push
+yet**; their asks go to whoever is attached, or straight to the unattended
+policy.
+
+Every link in that chain — VAPID signing, the subscribe handshake, the
+persisted store, the push-and-wait timing, the service worker's
+`push`/`notificationclick` handlers — is built and unit-tested. Real
+delivery to a phone with the tab closed has not yet been observed; see
+`docs/competitive-landscape.md`'s "Push, precisely" note for the test
+list and what that gap means.
+
+- **`--no-push`** turns the whole feature off (`daemon.json`: `push: false`).
+  With no subscriptions, push already does nothing — this flag matters for
+  the PHI/self-host story, where even the metadata below must not leave the
+  network.
+- **Egress.** The push body is end-to-end encrypted (RFC 8291/8188) — Apple
+  or Google, whoever runs the browser's push service, cannot read it. But
+  they DO learn that a notification was sent, when, and to which
+  subscription. That is a real egress for a self-hosted or PHI deployment
+  and is why `--no-push` exists.
+  A daemon with no outbound internet cannot deliver, and the one
+  difference from a daemon without this feature is time: the send fails
+  silently (never throws into the run), but an owner with a subscription on
+  file still makes the ask wait (`pushWaitMs` under a deny/allow policy; a
+  `park` policy waits its own bound) before the run applies its unattended
+  policy — later than it otherwise would, never a hang.
+  A push failure of any kind — a throw, a timeout (each send is bounded at
+  10s), a refusal — never fails the run; the ask reaches the same outcome
+  it would have reached without push, only later by that bounded wait.
+- **Endpoints are restricted to known push services.** The daemon POSTs to
+  a subscription's endpoint, so an open-ended URL would let anyone holding a
+  pairing token make it send requests to any host it can reach. A
+  subscription is refused unless its endpoint is `https://` on the default
+  port, carries no credentials, and its host is — or is a subdomain of —
+  `fcm.googleapis.com` (Chrome/Chromium), `push.services.mozilla.com`
+  (Firefox), `push.apple.com` (Safari) or `notify.windows.com` (Edge/
+  Windows). The list is `WEB_PUSH_SERVICE_HOSTS` in `@corenel/protocol`;
+  stored subscriptions outside it are dropped (and logged) on load.
+- **Where things live**, both under `<stateDir>/sidecar/`:
+  - `vapid.json` — the daemon's own VAPID keypair, generated on first use,
+    mode `0600`, written atomically. Deliberately NOT `auth.json`: signing
+    out deletes `auth.json`, and every browser subscription is bound to
+    this public key. **Deleting `vapid.json` orphans every subscribed
+    device** — each one is bound to the key that no longer exists, and
+    there is no re-linking; they have to click "Notify me on this device"
+    again once the daemon mints a new key. A new key is minted only when the
+    file does not exist. A file that is not a valid keypair is moved aside
+    to `vapid.json.corrupt-<timestamp>` with a loud log before a new key is
+    minted; any other read failure (permissions, a locked file) disables
+    push for that start, with a log line, rather than minting over a key
+    that may be fine.
+  - `push-subscriptions.json` — subscriptions, deduped by endpoint, capped
+    at 10 per client and 500 total, pruned automatically on a `404`/`410`
+    from the push service (the browser dropped the subscription) and after
+    5 consecutive `401`/`403`s (the device subscribed to a key this daemon
+    no longer signs with). A corrupt file is moved aside
+    (`push-subscriptions.json.corrupt-<timestamp>`) and logged; an
+    unreadable one disables push for that start.
+- A subscription is bound to one daemon's key; subscribing to a second
+  daemon replaces it — pairing with a new daemon and clicking "Notify me on
+  this device" moves the browser's one push registration to that daemon.
+- The owner's OTHER devices only count as reachable when they share a
+  **verified** Clerk user id, never a claimed account alone — see
+  `docs/team-agents.md` §5.2. That id comes from the attendance registry,
+  which refuses to answer for a `clientId` seen verified as two different
+  users (a shared browser); such an owner reaches only its own
+  subscriptions. A browser re-sends its existing subscription on every
+  pairing, so the id it is filed under follows an account switch.
+
+## Identity
+
+On by default. The daemon's issuer is **corenel-api**, not Clerk — the browser
+never gives the daemon its Clerk session token at all (spec amendment
+2026-09-25b). Pairing runs challenge → API assertion → assert:
+
+1. On `hello.wantsIdentity`, the daemon sends `identity-challenge` naming its
+   own `sidecarId` and a fresh, single-use nonce.
+2. The browser exchanges its Clerk token for a short-lived RS256 assertion by
+   calling corenel-api's `POST /api/v1/pairing-assertions` — the Clerk token
+   goes there and nowhere else. The assertion's claims are `aud` (this
+   daemon's `sidecarId`) and `nonce` (the one just issued), which is what
+   closes replay: a captured assertion names one daemon and one nonce that
+   daemon has already consumed, so it verifies nowhere else and never again
+   **[V]** `src/prompd/sidecar/connect.identity.test.ts` ("answers the daemon
+   challenge by fetching the Clerk token, POSTing {aud, nonce, tkh} with it,
+   and sending the returned assertion -- never the Clerk token itself"), which
+   also asserts that the Clerk token never appears in any message this
+   connection ever sends, `hello` included **[V]** same test, and
+   `packages/harness/sidecar/host.identity.test.ts` ("omits wantsIdentity from
+   hello, and never sends identityToken, when not configured") confirms
+   `hello.identityToken` is gone from the wire.
+3. The browser sends `identity-assert` with that assertion. The daemon
+   verifies it against corenel-api's own OIDC discovery/JWKS, checking `aud`,
+   `nonce`, `azp`, `cnf.tkh` (below) and the existing signature/`iss`/`exp`
+   checks **[V]** `packages/sidecar/src/identity/oidcVerifier.test.ts`
+   (`'rejects: %s'` cases `'foreign app (azp)'`, `'missing azp'`, `'wrong
+   aud'`, `'missing aud'`, `'aud array without the audience'`, `'wrong
+   nonce'`, `'missing nonce'`, `'wrong cnf.tkh'`, `'missing cnf'`,
+   `'non-object cnf'`).
+
+**Relay binding (`cnf.tkh`).** `aud` and the nonce stop a captured assertion
+being replayed, but not a LIVE relay: a malicious daemon M that holds this
+daemon's (B's) pairing token connects to B, forwards B's own challenge to a
+victim browser paired with M, and relays the victim's fresh assertion back.
+So step 2 also sends `tkh`, the unpadded base64url SHA-256 of the pairing
+token the browser presented on ITS connection; corenel-api signs it in as
+`cnf: { tkh }`, and the daemon requires it to equal the hash of the token
+presented on the connection the assertion arrives on. The victim's browser
+hashed M's token, not B's, so the relayed assertion pairs unverified
+(`pairing not bound to this connection`) **[V]**
+`packages/sidecar/src/identity/pairingRelay.e2e.test.ts` (two real SidecarServers, the real
+verifier, a test-key API minting exactly what corenel-api mints: the relay
+pairs UNVERIFIED and a direct pairing still verifies; removing the `cnf`
+comparison turns it red), `oidcVerifier.test.ts` ("binds to the
+pairing token"), `packages/harness/sidecar/identityHandshake.test.ts` ("the
+verifier is handed the hash of the pairing token THIS connection presented,
+not another"). The API only ever sees the hash, never the pairing token.
+**[D] Residual, not stopped:** an attacker who hands the victim B's pairing
+token under the attacker's OWN URL (so the victim's browser really presents
+B's token, to M, which proxies it to B) gets an assertion bound to B's token.
+The protocol cannot tell the browser that the daemon at a URL is not the one
+that printed the token; pair a token only with the daemon that printed it.
+
+The pairing token (`hello.token`) is checked FIRST; the challenge is never
+sent before it passes **[V]** `packages/harness/sidecar/identityHandshake.test.ts`
+("never sends a challenge for a bad pairing token"). A missing, late, or
+invalid assertion — or corenel-api being unreachable, or the signing key being
+absent (corenel-api then answers 503 on discovery/assertion) — always pairs
+the connection **unverified**, never refuses it **[V]**
+`identityHandshake.test.ts` ("no assertion arrives within 3000ms: pairs
+unverified, reason \"timed out\"", "a LATE identity-assert, arriving after the
+3000ms bound already answered \"timed out\", is ignored", "a throwing
+verifier pairs unverified with reason \"error\""). The whole identity leg is
+bounded at 3 000 ms from the challenge.
+
+The verified Clerk user id (never the claimed `accounts` text, which stays
+display-only) is the only thing that widens routing to a second device or
+widens push delivery to a device that never subscribed itself — see
+`docs/team-agents.md` §5.2 (at the repository root).
+
+- **Default issuer:** `https://api.corenel.ai` **[V]** `packages/sidecar/src/args.test.ts`
+  ("identity verification is on by default against the API issuer"). Point at
+  a local corenel-api with `--identity-issuer <url>`, or the `daemon.json` key
+  `identityIssuer`.
+- **https rule, with a loopback exception:** the issuer must be `https://`,
+  except a loopback host (`localhost`, `127.0.0.1`, `[::1]`), which may use
+  `http://` — for local development against a corenel-api with no TLS cert to
+  offer **[V]** `args.test.ts` ("rejects http:// on a non-loopback host",
+  "accepts %s (loopback issuer, for local development)" for
+  `http://localhost:4000`, `http://127.0.0.1:4000`, `http://[::1]:4000`); a
+  loopback-lookalike host (`http://localhost.evil.com`, etc.) is still
+  rejected **[V]** `args.test.ts` ("rejects the loopback lookalike %s").
+- **`--no-identity`** turns verification off entirely (`daemon.json`:
+  `identity: false`); every connection then pairs unverified, exactly as
+  before this feature existed.
+- **Keys over https:** the discovery document's `jwks_uri` must be `https://`
+  too — except under a loopback `http://` issuer, where an `http://`
+  `jwks_uri` on the issuer's own origin (same host and port) is accepted, so
+  a local corenel-api works end to end **[V]** `oidcVerifier.test.ts`
+  ("loopback http issuer" describe block: same-origin http verifies; another
+  host, another loopback host or another port does not; an https issuer with
+  an http `jwks_uri` still fails).
+- **Egress:** one OIDC discovery fetch (`<issuer>/.well-known/openid-configuration`)
+  plus one JWKS fetch per hour to the configured issuer, cached together; an
+  unknown `kid` triggers at most one extra refetch per 60 seconds.
+- The Clerk token never reaches the daemon at all, and the assertion itself is
+  never logged and never stored — only the resulting user id is kept, in
+  memory, for the life of the connection.
+- **[D]** The browser's worst case for the whole exchange is roughly 6 s (up to
+  3 s to fetch a fresh Clerk token, plus up to 3 s for the POST to
+  corenel-api) against the daemon's fixed 3 s bound from the challenge — a
+  slow answer pairs unverified, not refused.
+- **[D]** corenel-api only stamps `azp` for request origins in its own CORS
+  allowlist. A page served from an origin missing there pairs unverified even
+  with a valid Clerk session; local https dev against a local corenel-api
+  needs that instance's `FRONTEND_URL` set to the dev origin.
+- **[D]** Both web apps call `https://api.corenel.ai/api/v1/pairing-assertions`
+  directly (`VITE_PAIRING_ASSERTION_URL` in each app's `vite.config.ts`),
+  never through a front end's own `/api` proxy: the route's pre-auth limiter is
+  per client IP, and behind a proxy every user of that front end would share
+  one address and one bucket.
+- The runbook for provisioning and rotating the signing key
+  (`corenel-pairing-signing-key`) lives in corenel-api's own
+  `docs/pairing-signing-key.md`, not in this repo.
+
+**`corenel doctor` Identity section.** Computed from the EFFECTIVE config
+(flags/file/overlay already layered), never a second guess: `identity:
+disabled` when `--no-identity`/`identity: false`; otherwise the issuer URL and
+whether it passes the https/loopback rule, a real discovery fetch (5 s
+timeout, the SAME code the verifier uses), whether the document's `issuer`
+matches, whether `jwks_uri` is accepted under the same https/loopback rule,
+how many usable RS256 keys it lists (zero is a failure), the trusted origins
+(canonical defaults + `--allow-origin`), and a warning when `--allow-lan` is
+on that a LAN-allowed page can connect but can never verify identity unless
+its origin is also in `allowOrigins`. Included in `--json`.
+
 ## Security
 
 - Loopback by default; a browser connection requires both the pairing token **and** an