@north-light/cli 0.3.473 → 0.3.474

package/docs/sdk/memory.md

@@ -1,47 +0,0 @@
----
-title: Memory
-description: Read the person's public documents — knowledge and preferences about you — from an app's server with client.memory.
----
-
-# `client.memory`
-
-`client.memory` is the SDK's read-only view of the person's documents: the `knowledge` and `preference` documents owned by the person on the canvas (the same objects `crtr canvas read` shows them). The `memory` name and the `/v1/memory` routes are the API's names for this. An app reads the person's `public` documents with `memory.list` and `memory.get`, and needs no scope to do it. It never sees a `friends` or `private` memory, and cannot tell one from a memory that does not exist: both answer `404 not_found`. This list is what the app's agents can read in memory about you, so an app can show the person exactly which of their memories its agents draw on.
-
-```ts
-const page = await client.memory.list({ store: 'user' });
-for (const item of page.items) console.log(item.name, item.preview);
-
-const doc = await client.memory.get({ store: 'user', name: page.items[0].name });
-console.log(doc.body);
-```
-
-`store` is `'user'` (memory about you) in this release; any other value answers `400 invalid_request` with `param: "store"`. An app whose grant holds `crtr:memory:manage:user` (a first-party app) sees every privacy level.
-
-Reading records nothing: no watch, no `read` event, no audit row. Both methods are `GET` requests and follow the SDK's retry rule for safe requests.
-
-## List
-
-`list({store, path?, kind?, limit?, cursor?}, options?)` sends `GET /v1/memory` and returns `Promise<MemoryListPage>`, `{items, next_cursor}`.
-
-| Field | Meaning |
-|---|---|
-| `path` | A folder: lists documents whose name starts with `<path>/`, at any depth |
-| `kind` | `knowledge` or `preference`; both when omitted |
-| `limit` | 1 to 100; default 50 |
-| `cursor` | The previous page's `next_cursor`, with the same `store`, `path` and `kind` |
-
-Items are ordered by `name`. Each `MemoryListItem` is `{name, folder, kind, preview, summary, privacy, revision, created_at, updated_at}`: `name` has no `user/` prefix, `folder` is the name up to its last `/` (`""` at the top level), and `updated_at` is the current revision's time. Folders are not items, and deleted and unlisted documents are left out. `next_cursor` is `null` on the last page.
-
-`listAll(params, options?)` takes the same fields without `cursor` and yields every item across pages, fetching the next page only as you iterate; stop early with `break`:
-
-```ts
-for await (const item of client.memory.listAll({ store: 'user', path: 'work' })) render(item);
-```
-
-## Get
-
-`get({store, name}, options?)` sends `GET /v1/memory/document` and returns `Promise<MemoryDocument>`: a list item's fields plus `body`, `size`, `truncated` and `grantee`.
-
-- `body` is the current markdown as stored; `[[links]]` are not resolved. A body over 1 MiB is cut to its longest whole-character prefix of at most 1 MiB, with `truncated: true`; `size` is always the full length in UTF-8 bytes.
-- `grantee` is `app:<id>` of the app whose agent wrote the first revision, or `null` when the person wrote it.
-- An absent, deleted, `friends` or `private` name rejects with `NotFoundError` (`code: 'not_found'`); a malformed field rejects with `BadRequestError`.

package/docs/sdk/testing.md

@@ -1,191 +0,0 @@
----
-title: Testing your app
-description: FakeCloud from @north-light/sdk/testing — a fake directory and runtime your real OAuth2Client and Crouter talk to through fetch.
----
-
-# Testing your app
-
-`@north-light/sdk/testing` exports `FakeCloud`, a fake Northlight: the directory (discovery, token refresh, JWKS) and each person's runtime (grant, runs, memory, providers, run notices). It answers through `fetch`. Your app's production path — `new OAuth2Client(...)`, `oauth.client(userId)`, `runs`, `memory`, `grant`, `providers`, `watchRuns`, `watchGrants`, `onRefusal`, `forwardRunEvents`, `verifyRunNotice` — runs against it unchanged. The only differences from production are the `fetch` and `issuer` you pass in.
-
-There is no mock `Crouter`. An object typed as `Crouter` would re-implement `RunReply`, `RunStream`, the watchers, retries and error mapping, so your tests would check a copy of the SDK instead of the SDK. FakeCloud fakes only the wire, so every SDK behaviour under test is the one you ship.
-
-The subpath is Node-only (it uses `node:crypto`), adds no dependency, and imports no test framework: it works with `node:test`, vitest, jest or anything else. Nothing listens on a port.
-
-## Setup
-
-```ts
-import { OAuth2Client, MemoryConnectionStore } from '@north-light/sdk';
-import { FakeCloud } from '@north-light/sdk/testing';
-
-const cloud = new FakeCloud({ clientId: 'doppel' });
-const store = new MemoryConnectionStore();                   // or your own ConnectionStore
-await cloud.connect(store, 'user:o1', { scopes: ['crtr:llm', 'crtr:act', 'sms:send'] });
-
-const oauth = new OAuth2Client({
-  clientId: 'doppel', issuer: cloud.issuer, fetch: cloud.fetch,
-  privateKey, redirectUri: 'https://app.test/callback', store,
-});
-const client = await oauth.client('user:o1');               // first call refreshes through the fake directory
-```
-
-`cloud.connect(store, userId, {scopes?, dependencies?})` makes the person's grant active (default scopes `crtr:llm`, `crtr:act`) and saves a `Connection` in your store with its access token already expired, so the first call refreshes through the directory exactly as in production.
-
-`cloud.crouter(userId)` returns a real `Crouter` on that person's runtime with a fresh token and no refresh. Refusals and refreshes never reach it. Use it only for code whose production signature takes a `Crouter`. Code that goes through `OAuth2Client` in production should go through `oauth.client` in tests too.
-
-Options: `new FakeCloud({clientId, issuer = 'https://directory.test', runtimeUrl = 'https://runtime.test' (or (userId) => url), runtimeId = 'runtime:test', runtimeClientId = 'runtime-client', runtimeVersion?, streamIdleMs = 60000, accountAppUriTemplate = '<issuer>/account/apps/{client_id}' (null omits it), runNotice?, isolatedRuns = 'available', clock = Date.now})`.
-
-`clock` is `{now(): number}`; every timestamp the fake writes reads it (run `created_at` and `settled_at`, grant `exp`, memory `created_at`/`updated_at`, access-token `iat`/`exp`, `calls[].at`, `directoryCalls[].at`) and the idempotency window runs on it. Pass `manualClock()` (from `@north-light/sdk/testing`; `now()`, `advance(ms)`) to step time by hand. Stream idle, long-poll waits and run-notice retries stay on real timers.
-
-`cloud.activate(userId, {scopes?, dependencies?})` makes the grant active with no stored connection, for an app that mints a per-user runtime token itself. The fake directory serves that mint at `POST <issuer>/token` with `grant_type=client_credentials&sub=<userId>` (the client assertion is not verified): 200 `{access_token, token_type: 'Bearer', expires_in: 600, runtimeUrl}` (the token's `exp` claim is 600 s after `iat`; the fake runtime does not enforce token expiry); a missing `sub` is 400 `unauthorized_client`; an unknown or revoked user is 400 `invalid_grant` (`no active grant`); `user.refuse('runtime_provisioning')` is 400 `invalid_grant` with `error_description: runtime_provisioning`. A mint is a `directory.token` call (`userId` is the `sub`), so `failNext('directory.token', …, {user})` applies to it.
-
-### Isolated runs
-
-The fake stands in for a Northlight guest, where the Linux root launcher always exists, so `runs.start({isolated: true})` starts a run by default and `run.params.isolated` is `true`. The checks `startRun` makes without the launcher are mirrored: `isolated` must be a boolean, `cwd` is accepted only with `isolated: true` and must be an absolute path inside the run's root folder, `/apps/<clientId>/nodes/<run id>` (else 400 `invalid_request`, `param: cwd`), and `worktree` is refused. Nothing in the fake enforces the sandbox itself (no network, no `files/` mount).
-
-`isolatedRuns: 'unavailable'` models a runtime without the launcher, such as a local daemon on a Mac: every isolated start answers 503 `isolated_unavailable` (`param: isolated`, retryable, so `runs.start` sends it three times before throwing). Use it to test your app's handling of that refusal.
-
-## Driving runs
-
-Nothing in the fake runs a model. A run stays `running` until your test drives it, either directly or through a responder:
-
-```ts
-cloud.runs.respond(({ run }) => run.reply(`echo: ${run.pending.at(-1)?.text ?? run.params.prompt}`));  // every user
-cloud.user('user:o1').runs.respond(null);                                          // per-user override
-```
-
-A responder gets `{run, kind: 'start' | 'message', messages}`. It is called after the fake answers a `run.start`, or a `run.message` with `start_turn` true, and never for an idempotent replay. A responder that throws is recorded in `cloud.responderErrors` and fails nothing else. An `async` responder can await your app's state mid-reply.
-
-`cloud.runs` and `cloud.user(id).runs` have one shape, `{all: FakeRun[], respond(fn)}`: `all` lists the runs in start order (a deleted run stays, with `deleted` set), so `expect(user.runs.all).toEqual([])` works. Each entry is a `FakeRun`:
-
-| Method | Effect |
-|---|---|
-| `reply(text, {chunks = 2, messageIds?})` | One turn: `node.turn.started`, text deltas, `node.output_text.done`, `node.turn.completed`. Delivers `messageIds` (default: every pending message). The run goes `idle` (`waiting_on_user` with an open question). |
-| `failTurn({code: 'model_error' \| 'content_refused', message})` | The same turn ending `stop_reason: 'error'`; `runs.get` shows `last_turn_error`; `runs.reply(...).collect()` ends `error`. |
-| `setActivity(activity, {at?})` | Fake-only: status `running` with this `activity` until the next event. The only way to reach `writing` or a streaming tool call. |
-| `settle(ending)` | `{result}`, `{declined: {reason, code, retryable}}`, `{failure: 'canceled' \| 'deadline_exceeded'}`, or `{runtimeError: reason}`. Emits `node.settled` with the outcome the runtime writes. |
-| `idle()` | Goes idle without a new turn (stopped without submitting). |
-| `ask({title, slots?})` | Opens a question (`waiting_on_user`); returns its id. Your app's `questions.answer` records the answer. |
-| `delete()` | As if the runtime deleted it: reads 404, streams close, its idempotency keys are dropped. |
-| `emit(type, fields, {nodeId?})` | Appends any run event; another `nodeId` is a child node's event. Status and activity re-derive from the log. |
-| `closeIdle()`, `expireEvents(upTo)` | Idle-close open streams now; drop old events so `stream_gap` is reachable. |
-
-Read-only members: `id`, `userId`, `params` (the `run.start` body, `isolated` and `cwd` included), `idempotencyKey`, `messages`, `pending`, `status`, `activity`, `outcome`, `result`, `lastTurnError`, `events`.
-
-What a turn's input is: `reply` and `failTurn` deliver `messageIds` (default: every pending message), and the turn's `node.turn.started` input text is those messages' text. A turn that delivers none (`messageIds: []`, or nothing pending) uses the run's `prompt` as its input on the run's first turn only, and an empty input after that; it still emits a full root turn with the text you pass. So `reply('', {messageIds: []})` is not a turn that answers nothing. To script a child node that works and answers nothing, or any event outside a root turn, use `emit(type, fields, {nodeId: 'child-1'})`: for example `run.emit('node.turn.started', {input: {text: 'x', message_ids: []}, entry_id: 'c1'}, {nodeId: 'child-1'})` followed by `node.turn.completed` with the same `nodeId`. Events with another `nodeId` never complete the app's `runs.reply`, which waits for the root's turn.
-
-Messages follow the runtime: `start_turn` true makes the run `running` at once and calls the responder; `start_turn` false only records it. A message to a run settled `canceled` or `deadline_exceeded` is refused 409 `run_settled`; to a run settled any other way it reopens the run. A deleted run answers 404.
-
-Idempotency follows the runtime too: the same `Idempotency-Key` with the same body replays the first answer (no new run, no responder call); the same key with a different body, or reused between `run.start` and `run.message`, is 409 `idempotency_conflict`. After a run is deleted, its start key creates a new run. A key is live for 24 hours (`IDEMPOTENCY_WINDOW_MS`): at exactly the window it reads as absent, and the next accept replaces the record. Step the window with `clock`.
-
-`run.openStreams` counts the run's open event streams. It rises when the SDK starts reading the response body, which is after `cloud.calls` records `run.events`, so poll `openStreams`, not `calls`, to know a stream is attached. It falls on idle, error, abort or revoke. Run notices are owed only while no stream is open.
-
-## Grants and setup state
-
-`cloud.user(id).grant`: `state` (the `GrantGetDTO`, or null), `set({dependencies?, resolved_scopes?, spend_cap?})` (a new `rev`), `setup(provider, state, {message?})` (a dependency change without a `rev` bump), `sync()` (unchanged content: no version change, no wake), `revoke(code = 'grant_revoked')`, `restore(patch?)`.
-
-`grant.get` with `wait` and `since` is held until the version changes, exactly as the runtime holds it, so `oauth.watchGrants` works unmodified. `user.reads` lists the person's `grant.get` calls and `user.holding` counts reads held now.
-
-## Refusals
-
-- `user.refuse(code)` leaves the grant alone: the directory refuses each refresh with `code` (`grant_removed`, `grant_suspended`, `app_in_testing`, `refresh_token_expired`, `refresh_token_reused`, `grant_revoked`), and tokens issued before the call get 401, so the next call refreshes and is refused. `refuse('ok')` clears it.
-- `grant.revoke('grant_removed')` deactivates the grant: calls answer 401 `grant_revoked`, held reads wake to it, open streams get an error frame, and the next refresh is refused with that code.
-- `user.expireTokens()` forces a refresh without a refusal.
-
-The real `OAuth2Client` handles all of it: your `onRefusal` runs with the refusal, the stored connection is removed, and the thrown error has `refusalOf(e)` set. `await cloud.refusalError(code)` returns that exact error for unit tests that need the value.
-
-## Testing mode
-
-An unverified third-party app is *in testing* on Northlight: only its admin and up to 100 named testers can sign in, and anyone else is refused. `cloud.testing` puts the fake's app in that state so your app can test the refusal.
-
-```ts
-cloud.user('user:tess').setAccount({ email: 'tess@example.com' });  // the identifiers the account holds
-cloud.testing.addTester({ email: 'Tess@Example.com' });             // or {phone: '+15551234567'}; false if already present
-cloud.testing.enable({ admin: 'user:me' });                         // in testing; revokes active grants of people not allowed
-
-const back = cloud.authorize('user:stranger', { redirectUri, state });   // the redirect the browser follows back to your app
-authorizationRefusalOf(back);   // {code: 'app_in_testing', error: 'access_denied', state}
-```
-
-- `cloud.testing.enable({admin?})` / `disable()` / `active`. `enable` returns the user ids whose active grants it revoked.
-- `addTester({email} | {phone})` stores the entry (email trimmed and lowercased, phone E.164). The 101st entry throws. `removeTester(ref)` returns `{removed, revoked}`: a removed tester's active grant is revoked, not deleted, and the fake keeps their runs.
-- `cloud.testing.allowed(userId)`: not in testing, the admin, or a tester entry matches the account: an `email` entry needs `emailVerified` (default true when `email` is set), a `phone` entry matches `setAccount({phone})`.
-- A refresh for an active grant whose person is not allowed answers `invalid_grant` `app_in_testing`, and so does a refresh after the grant was revoked this way. `oauth.client(id)` rejects with an `AuthenticationError` (`code: 'app_in_testing'`, `refusalOf(e).kind === 'paused'`) and `onRefusal` runs. Keep the person's data.
-- `cloud.authorize(userId, {redirectUri, state, scopes?})` stands in for the browser round trip. An allowed person gets an active grant and a redirect with `code` and `state`; give `code` to `oauth.exchangeCode`. A person who is not allowed gets `error=access_denied`, `error_description=app_in_testing` and `state`, and nothing is written. A person revoked by `removeTester` who is allowed again gets access back by `authorize`.
-- When a person's email or phone stops matching (`setAccount`), nothing is revoked; their next refresh and `authorize` are refused.
-
-In your callback handler, check `authorizationRefusalOf(url)` before reading `code`.
-
-## Memory and providers
-
-`user.memory.put({name, body, privacy = 'public', kind = 'knowledge', ...})` and `remove(name)`. Without `crtr:memory:manage:user` in the grant, only `public` documents are listed and others read as 404, as on the runtime. List items carry the full `MemoryListItem` fields.
-
-```ts
-cloud.providers.define('sms', { tools: { send: { group: 'send', handler: ({ userId, arguments: args }) => ({ sent: true }) } } });
-```
-
-A call is checked against the grant's `<provider>:<group>` scope (403 `scope_missing`). A handler that throws `FakeCloud.error({...})` answers that envelope; any other throw is 502 `provider_error`. `status: 'install_failed'` lists the provider with its `error` and answers calls 503 `provider_unavailable`.
-
-## Failures, gates and the directory
-
-`intercept(op, fn, {user?, times?})` runs `fn` before an operation; it may await, act, or return an answer. `failNext(op, answer, {user?, times = 1})` is the shorthand. `gate(op)` holds every matching request until you call the `release()` it returns. `op` is a runtime operation name (`run.start`, `grant.get`, `memory.list`, …), `'*'` for every runtime operation, or `directory.discovery`, `directory.token`, `directory.jwks`.
-
-| Answer | Runtime operation | `directory.discovery` | `directory.token` | `directory.jwks` |
-|---|---|---|---|---|
-| `'unreachable'` | fetch rejects; the SDK throws `APIConnectionError` | fetch rejects | fetch rejects; no `onRefusal` | fetch rejects; `verifyRunNotice` throws `RunNoticeError` |
-| a status number | that status with a runtime envelope | that status; discovery fails | that status; an `InternalServerError` for 5xx coded `invalid_request`, origin `directory`; connection kept | that status |
-| `FakeCloud.error({status, code, message?, origin?, retryable?, details?, param?})` | that envelope | not allowed | not allowed (use `user.refuse`) | not allowed |
-
-Retries are the SDK's, so a scripted failure the SDK retries is absorbed unless `times` covers every attempt. One SDK call makes these attempts (`maxRetries` defaults to 2; a call's `maxRetries` option changes it):
-
-| What the fake answers | Operations | SDK behaviour | `times` to surface it |
-|---|---|---|---|
-| 401 (any code) | every runtime operation called through `oauth.client` | refreshes the token once, then repeats the call once | 2 |
-| a status number ≥ 500, or 429 | GET and DELETE operations: `grant.get`, `run.get`, `run.list`, `run.delete`, `trace.get`, `question.get`, `memory.list`, `memory.get`, `provider.tools`, `profile.list` | retried `maxRetries` times with backoff (250 ms, 500 ms) | 3 |
-| `'unreachable'` | the same GET and DELETE operations | retried `maxRetries` times | 3 |
-| `FakeCloud.error({..., retryable: true})` (the default for a 503), or `'unreachable'` | `run.start`, `run.message` (keyed sends) | resent with the same `Idempotency-Key` `maxRetries` times, waiting 500 ms, 1 s (or the envelope's `retry_after_s`) | 3 |
-| a status number, or an envelope with `retryable: false` | `run.start`, `run.message` | not retried (a status number answers `retryable: false`) | 1 |
-| anything else | the other POSTs: `run.interrupt`, `run.cancel`, `run.rename`, `question.answer`, `provider.call` | never retried | 1 |
-| anything | `run.events` (the SDK's `RunStream`) | a retryable error reconnects from the cursor with backoff; a non-retryable one, `stream_error` or `stream_dropped`, ends the stream | `Infinity` for a retryable one |
-
-The rows compose: one 401 followed by three 500s on `grant.get` surfaces the 500 (the call repeated after the refresh has its own two retries), so a 401 surfaces only when the repeat gets one too. A test that wants an operation to fail no matter what scripts `times: Infinity` and removes it afterwards with `intercept`'s returned function, or calls `cloud.reset()`.
-
-Discovery is memoized per `OAuth2Client`. A test that fails discovery should use a fresh `FakeCloud` and `OAuth2Client`.
-
-A runtime outage on an ordinary call: `user.expireTokens(); cloud.failNext('directory.token', 503, {user})`. An unreachable runtime: `cloud.failNext('*', 'unreachable', {user, times: Infinity})`. With `runtimeVersion` older than the SDK's minimum, calls fail with `RuntimeVersionError` (426).
-
-## Run notices and forwarding
-
-```ts
-const cloud = new FakeCloud({ clientId: 'chat', runNotice: { url: 'https://app.test/run-notices', deliver: (request) => app.handleNotice(request) } });
-```
-
-The fake sends notices by the runtime's rules: one is owed when an event is appended with no `run.events` stream open, or when the last stream closes with events unread; a burst coalesces into one; up to three attempts; a 401 re-mints the token once; a 5xx, a thrown `deliver`, or no answer within 10 s is retried; any other status ends the send. A run whose person has no active grant is owed nothing. The waits default to 0 so tests stay fast. Pass the runtime's values (`retryWaitsMs: [3000, 5000]`, `ackGapMs: 5000`, `failGapMs: 60000`) to test timing. Every attempt is recorded in `cloud.notices`.
-
-In your route, pass the cloud's `fetch` to `verifyRunNotice`:
-
-```ts
-await verifyRunNotice(request, { runNoticeUrl, issuer: cloud.issuer, grantee: 'app:chat', fetch: cloud.fetch });
-```
-
-Keys are cached per `(issuer, fetch)`, and each cloud has its own signing key, so several clouds on one issuer verify independently. `forwardRunEvents` needs nothing special: it reads the fake's event stream, idle close included (`streamIdleMs`).
-
-## Two app processes
-
-Build two app instances (two `OAuth2Client`s) on the same `cloud`. They see the same runs, grants and held reads. If they keep separate stores, the second refresh with a superseded token is refused `refresh_token_reused`, as it would be in production; a shared store with `lock` prevents it.
-
-## Asserting
-
-- `cloud.calls`: every runtime request as `{userId, op, runId, questionId, method, path, query, body, idempotencyKey, status, at}`.
-  - `userId` is the bearer token's `sub` whenever it names a person this cloud knows (created by `connect` or `user(id)`), whatever their grant or refresh-token state, so a call refused 401 still carries it. It is `null` only for a missing or undecodable bearer, or a `sub` the cloud has never seen.
-  - `runId` is the run the call addresses: the path's id for `run.get`, `run.message`, `run.events`, `run.interrupt`, `run.cancel`, `run.delete`, `run.rename` and `trace.get` (as sent, even when no such run exists), and for `question.get` and `question.answer` the run holding the question (null when none of the caller's runs does). `questionId` is the path's id on the two question operations. Both are `null` on every other operation.
-  - `body` is a parsed copy for your assertions; mutating it from an intercept is not a supported way to change what the fake handles.
-- `cloud.directoryCalls` and `cloud.refreshes`: directory requests, with `answer` (`ok`, a refusal code, a status or `unreachable`).
-- `run.params`, `run.messages`, `run.idempotencyKey`: what your app sent.
-- `cloud.notices`: every notice attempt, with its decoded token claims and your route's status.
-- `cloud.reset()` clears runs, calls, intercepts, responders, memory, idempotency keys and notices, and keeps grants and refresh tokens so stored connections stay valid.
-
-## Fidelity
-
-FakeCloud is checked against the real runtime on every crouter release: one set of conformance scenarios runs on the fake and on the real app listener (run start and idempotency, messages, reply streams, long-poll, every settle outcome, delete, the event stream's edges, grant holds, memory, providers, run notices). The keyed-write scenarios K1–K8 (equal-key replay, conflict, concurrent equal keys accepted once, the 24h window, replay after cancel, a key bound to its operation, keys scoped to the grantee, concurrent notes) are exported as `@north-light/sdk/testing/conformance` (`keyedWriteScenarios`, `IDEMPOTENCY_WINDOW_MS`) and run on FakeCloud and on the real daemon; an app test double for the runtime should run them against its own target. Two behaviours are fake-only because the runtime cannot be scripted that way: a successful provider call (the real side needs an installed provider), and `setActivity`. An isolated run that starts is checked only against `startRun`'s documented checks: the real listener in the suite runs without the Linux root launcher, so on it the `isolatedRuns: 'unavailable'` refusal is what is conformance-checked.
-
-Not faked: sign-in (`authorizeUrl`, `exchangeCode`, ID tokens — start from `cloud.connect`), token revocation, the owner and local-daemon surface (`nodes`, `canvas`, `crons`, `files`, …), and several apps on one cloud. `trace.get`, `run.rename`, `run.interrupt` and `profile.list` answer the right shape only.