@semiont/sdk 0.5.0 → 0.5.2

package/README.md

@@ -6,17 +6,81 @@
 [![npm downloads](https://img.shields.io/npm/dm/@semiont/sdk.svg)](https://www.npmjs.com/package/@semiont/sdk)
 [![License](https://img.shields.io/npm/l/@semiont/sdk.svg)](https://github.com/The-AI-Alliance/semiont/blob/main/LICENSE)
 
-The developer-facing SDK for [Semiont](https://github.com/The-AI-Alliance/semiont). This package owns the high-level surface every Semiont consumer reaches for: a verb-oriented `SemiontClient`, per-KB sessions, RxJS view-models, and the helpers that wire them all together.
+The TypeScript SDK for [Semiont](https://github.com/The-AI-Alliance/semiont) — a programmable surface for **collaborative knowledge work**. Whether you're building a browser app where humans annotate documents and propose links, an AI agent that gathers context and matches candidate references, a daemon that ingests new sources, or a one-shot script that queries an established knowledge base, you reach the same verb namespaces, the same collaboration primitives, the same lifecycle observables.
 
-The SDK is **transport-agnostic** — it consumes the `ITransport` and `IContentTransport` contracts from [`@semiont/core`](https://github.com/The-AI-Alliance/semiont/tree/main/packages/core). For HTTP, the canonical wire adapter is re-exported here for convenience. For in-process operation, use `LocalTransport` from [`@semiont/make-meaning`](https://github.com/The-AI-Alliance/semiont/tree/main/packages/make-meaning).
+The eight flows — *frame, yield, mark, match, bind, gather, browse, beckon* — describe what participants *do* when they work with a shared corpus. The first seven act on content; Frame acts on the schema layer (the conceptual vocabulary the others operate within). The SDK exposes them uniformly across surfaces. A human in a browser hovers an annotation; an AI agent at the other end of the bus sees the hover and reacts; a daemon ingests new text and every connected participant sees the corpus grow live. Humans and AI agents are peers — the SDK does not distinguish.
+
+The SDK is **transport-agnostic**: it consumes the `ITransport` and `IContentTransport` contracts from [`@semiont/core`](https://github.com/The-AI-Alliance/semiont/tree/main/packages/core). For HTTP backends, the canonical wire adapter is re-exported here for convenience. For in-process operation (CLI, agentic worker, embedded use), use `LocalTransport` from [`@semiont/make-meaning`](https://github.com/The-AI-Alliance/semiont/tree/main/packages/make-meaning).
+
+> **Where this doc fits.** This README is the *typed-surface reference* — what's in `@semiont/sdk`, how the namespaces are organized, what return shapes to expect. For the *protocol-level architectural framing* (the eight flows, the three programmable surfaces — CLI, SDK, Skills — the core tenets, the per-flow contracts), start with [`docs/protocol/README.md`](https://github.com/The-AI-Alliance/semiont/blob/main/docs/protocol/README.md). Daemon authors stitching multiple packages together also want the [skill packs](https://github.com/The-AI-Alliance/semiont/tree/main/docs/protocol/skills) — `semiont-session` for watcher daemons, `semiont-worker` for job-claim daemons, `semiont-wiki` for the end-to-end annotation pipeline.
+
+## Three ideas that hold the surface together
+
+The SDK is wider than a typical client library because the domain is — collaborative knowledge work over a shared corpus, with humans and AI agents as peers. Three framings make the API tractable; once you've seen them, the rest is predictable.
+
+### 1. Eight verbs
+
+Every operation in the SDK belongs to one of eight *flows* — verbs that describe what a participant *does* with a shared corpus. The flows are the entire vocabulary of the protocol; learn them once and the surface stays small.
+
+| Verb | What it does | Example methods |
+|---|---|---|
+| **frame** | Define and evolve the schema vocabulary (entity types, future tag schemas, relation types) | `frame.addEntityType`, `frame.addEntityTypes` |
+| **yield** | Introduce new resources into the system | `yield.resource`, `yield.fromAnnotation`, `yield.cloneToken` |
+| **mark** | Add structured metadata to resources | `mark.annotation`, `mark.assist`, `mark.archive` |
+| **match** | Search the corpus for candidate resources | `match.search` |
+| **bind** | Resolve ambiguous references to specific resources | `bind.body`, `bind.initiate` |
+| **gather** | Assemble related context around an annotation | `gather.annotation` |
+| **browse** | Navigate, read, and observe | `browse.resource`, `browse.annotations`, `browse.click` |
+| **beckon** | Coordinate attention across participants | `beckon.hover`, `beckon.attention`, `beckon.sparkle` |
+
+Each flow is a namespace on `SemiontClient` (`client.mark.X(...)`, `client.gather.X(...)`, ...). The verb is the unit of mental model — a method call belongs to a flow, not to a noun. Frame is the schema-layer flow — content flows operate within the vocabulary Frame manages. Per-flow contracts live in [`docs/protocol/flows`](https://github.com/The-AI-Alliance/semiont/tree/main/docs/protocol/flows).
+
+### 2. Four return shapes — guess from the name
+
+Every method on every namespace returns one of exactly four shapes, and the method name tells you which. Internalize the convention once and you stop having to read return types:
+
+| Shape | Naming convention | Examples |
+|---|---|---|
+| **`Promise<T>`** — atomic backend op | past-tense or short noun | `mark.annotation`, `bind.body`, `auth.password` |
+| **`StreamObservable<T>`** — long-running stream | plain verb | `mark.assist`, `match.search`, `gather.annotation` |
+| **`CacheObservable<T>`** — live query | plain noun | `browse.resource`, `browse.annotations`, `browse.entityTypes` |
+| **`void`** — collaboration signal | imperative or progressive verb | `beckon.hover`, `bind.initiate`, `mark.changeShape` |
+
+Both Observable subclasses implement `PromiseLike<T>`, so consumers can `await` them directly without learning RxJS. Reach for `.subscribe(...)` when you want progress events, live updates, or to observe a collaboration signal another participant emitted. See [`docs/REACTIVE-MODEL.md`](https://github.com/The-AI-Alliance/semiont/blob/main/packages/sdk/docs/REACTIVE-MODEL.md) for the full design and method-by-method assignment.
+
+### 3. Collaboration primitives
+
+The fourth row above — the `void`-returning collaboration signals — is the SDK's distinctive contribution to multi-participant coordination. They look fire-and-forget at the call site; on the bus they fan out across every participant.
+
+A human in a browser hovers an annotation (`beckon.hover(annotationId)`); an AI agent at the other end of the bus sees `beckon:hover` and reacts. An agent emits a sparkle (`beckon.sparkle(annotationId)`); the human's UI lights up the indicated annotation. A frontend state unit emits `mark.changeShape('rectangle')`; a different participant subscribed to `mark:shape-changed` reacts.
+
+This is *protocol-level* coordination — not browser-app fluff, not bolted-on presence — and it sits on the same typed namespace surface as data operations. Observers reach the same signals via `session.subscribe(channel, handler)` or `client.bus.get(channel)`. Three legitimate paths to the bus are documented in [`docs/REACTIVE-MODEL.md`](https://github.com/The-AI-Alliance/semiont/blob/main/packages/sdk/docs/REACTIVE-MODEL.md#three-paths-to-the-bus).
 
 ## What's in the box
 
 - **`SemiontClient`** — the verb-oriented coordinator over a wire transport.
 - **Verb namespaces** — `browse`, `mark`, `bind`, `gather`, `match`, `yield`, `beckon`, `job`, `auth`, `admin`. Typed methods that wrap the bus protocol; consumers never touch raw channel strings.
-- **Session layer** — `SemiontSession` (per-KB authentication, token refresh, lifecycle), `SemiontBrowser` (tab-singleton coordinator), and `SessionStorage` adapters (`InMemorySessionStorage`, plus a web one in `@semiont/react-ui`).
-- **View-models** — RxJS-based MVVM factories the React layer mounts via `useViewModel`.
-- **Helpers** — `bus-request` (correlation-ID request/reply) and `cache` (per-key SWR cache).
+- **Collaboration primitives** — fire-and-forget signals on the verb namespaces (`beckon.hover`, `bind.initiate`, `mark.changeShape`, `browse.click`, ...) coordinate attention and intent across participants. Not afterthoughts, not browser-app fluff: they're how a multi-participant session stays coherent.
+- **Session layer** — `SemiontSession` (per-KB authentication, token refresh, lifecycle), `SemiontBrowser` (multi-KB orchestration), and `SessionStorage` adapters (`InMemorySessionStorage`, plus a browser-backed one in `@semiont/react-ui`).
+- **Flow state machines** — RxJS-based factories (`createMarkStateUnit`, `createGatherStateUnit`, `createMatchStateUnit`, `createYieldStateUnit`, `createBeckonStateUnit`) that wrap each long-running flow with `loading$` / `error$` / progress observables. UI-shape-agnostic — any consumer (browser, terminal, mobile, daemon) can subscribe.
+- **`WorkerBus`** — the transport-neutral channel-bus interface that worker-side adapters consume. Domain-specific worker adapters live with their domain — `createJobClaimAdapter` and `createJobQueueStateUnit` in [`@semiont/jobs`](https://github.com/The-AI-Alliance/semiont/tree/main/packages/jobs); `createSmelterActorStateUnit` in [`@semiont/make-meaning`](https://github.com/The-AI-Alliance/semiont/tree/main/packages/make-meaning) — and consume `WorkerBus` from here.
+- **Helpers** — `bus-request` (correlation-ID request/reply), the cache primitive backing live queries, and `createSearchPipeline` (debounced-search RxJS pipeline).
+
+Page-shaped state machines (admin tables, compose page, resource viewer page, etc.) live in [`@semiont/react-ui`](https://github.com/The-AI-Alliance/semiont/tree/main/packages/react-ui), alongside the components that render them. Those are framework-neutral but tied to the Semiont web frontend's specific page taxonomy; they don't apply to non-web consumers.
+
+## For non-web consumers (TUI, mobile, daemon, agent)
+
+`@semiont/sdk` is the only package a non-web Semiont consumer needs. From it you get:
+
+- `SemiontClient` + the eight verb namespaces (`frame`, `browse`, `mark`, `bind`, `gather`, `match`, `yield`, `beckon`)
+- Three infrastructure namespaces (`auth`, `admin`, `job`) when constructed with backend operations
+- `SemiontSession` for long-running token refresh + persistence
+- `SemiontBrowser` for multi-KB orchestration (transport-agnostic; takes a `SessionFactory`)
+- The five flow state machines above
+- The transport-neutral `WorkerBus` interface (worker adapters live in their domain packages — `@semiont/jobs`, `@semiont/make-meaning`)
+- Branded ID types, the unified error hierarchy, the `TransportErrorCode` neutral vocabulary
+
+Nothing page-shaped, nothing web-shell-shaped. A TUI, mobile reader, daemon, or AI agent installs `@semiont/sdk` alone (plus a transport package — `@semiont/api-client` for HTTP, `@semiont/make-meaning` for in-process).
 
 ## Installation
 
@@ -26,42 +90,54 @@ npm install @semiont/sdk
 
 ## Quick start (HTTP)
 
+For one-shot scripts, `SemiontClient.signInHttp(...)` is the credentials-first one-line construction:
+
 ```ts
-import {
-  SemiontSession,
-  InMemorySessionStorage,
-  setStoredSession,
-  type KnowledgeBase,
-} from '@semiont/sdk';
-import { accessToken } from '@semiont/core';
-import { firstValueFrom } from 'rxjs';
-import { filter } from 'rxjs/operators';
+import { SemiontClient } from '@semiont/sdk';
+
+const semiont = await SemiontClient.signInHttp({
+  baseUrl: 'http://localhost:4000',
+  email: 'me@example.com',
+  password: 'pwd',
+});
+
+const resources = await semiont.browse.resources({ limit: 10 });
+console.log(resources);
+
+semiont.dispose();
+```
+
+For long-running scripts that need to survive token expiry, use `SemiontSession.signInHttp(...)` — same credentials shape, plus proactive refresh, validation, storage-adapter wiring, and disposal. `kb` is required; its `id` is the storage key for this session, so distinct scripts must use distinct ids:
+
+```ts
+import { SemiontSession, InMemorySessionStorage, type KnowledgeBase } from '@semiont/sdk';
 
 const kb: KnowledgeBase = {
-  id: 'local',
-  label: 'Local Backend',
-  protocol: 'http',
-  host: 'localhost',
-  port: 4000,
+  id: 'my-watcher',
+  label: 'My Watcher',
   email: 'me@example.com',
+  endpoint: { kind: 'http', host: 'localhost', port: 4000, protocol: 'http' },
 };
 
-const storage = new InMemorySessionStorage();
-setStoredSession(storage, kb.id, {
-  access: accessToken('your-jwt'),
-  refresh: '',
+const session = await SemiontSession.signInHttp({
+  kb,
+  storage: new InMemorySessionStorage(),
+  baseUrl: 'http://localhost:4000',
+  email: 'me@example.com',
+  password: 'pwd',
 });
 
-const session = await SemiontSession.create({ kb, storage });
+// session.client is the same SemiontClient surface; the session manages
+// the token$ lifecycle around it (default refresh callback wired automatically).
+const resources = await session.client.browse.resources({ limit: 10 });
 
-const resources = await firstValueFrom(
-  session.client.browse.resources({ limit: 10 }).pipe(
-    filter((r): r is NonNullable<typeof r> => r !== undefined),
-  ),
-);
-console.log(resources);
+await session.dispose();
 ```
 
+`KnowledgeBase` is uniform regardless of transport kind; the variation lives in the nested `endpoint` (currently `{ kind: 'http', host, port, protocol }` or `{ kind: 'local', kbId }`). Code that doesn't construct transports — your scripts, the verb namespaces, the flow state machines — never inspects `endpoint`.
+
+If you already have an access token (CLI cached-token path, env-var token, embedded auth flow), use `SemiontClient.fromHttp({ baseUrl, token })` or `SemiontSession.fromHttp({ baseUrl, token, storage, kb, refresh, ... })` to skip the auth round-trip.
+
 ## Quick start (in-process)
 
 When you want the SDK without an HTTP backend — e.g. in a CLI, a unit test, or an Electron-style desktop app — wire `LocalTransport` directly to a knowledge system:
@@ -86,39 +162,55 @@ const client = new SemiontClient(
 );
 ```
 
-Same `SemiontClient`, same verb namespaces — no network involved.
+Same `SemiontClient`, same verb namespaces — no network involved. There is no `fromLocal` factory because the in-process transport's dependencies (knowledgeSystem, eventBus, userId) are not boilerplate the SDK can hide.
 
-## Verb namespaces
+## Worked examples
 
-All ten namespaces hang off `SemiontClient`. Each method either returns a `Promise` (one-shot RPC-style operations) or an `Observable` (streaming subscriptions). The bus is invisible to callers — channel strings, correlation IDs, and reconnection are internal.
+The eight verb namespaces hang off `SemiontClient`, plus three infrastructure namespaces (`auth`, `admin`, `job`) when the client was constructed with backend operations. Each example below uses one of the four return shapes from the table above; pick whichever matches what your call site needs.
 
 ```ts
-// Browse — read the knowledge graph.
-await client.browse.resources({ limit: 10 });
+// Browse — live queries; await yields the loaded value, subscribe yields
+// loading-then-loaded.
+const resources = await client.browse.resources({ limit: 10 });
 client.browse.resource(resourceId).subscribe(/* ... */);
 
-// Mark / Bind — create and modify annotations.
-const { annotationId } = await client.mark.annotation(rid, request);
+// Mark / Bind — atomic operations return Promise<T>.
+// `mark.annotation` takes the W3C-shaped input directly; the resourceId
+// is derived from `input.target.source` and returned as a branded id.
+const { annotationId } = await client.mark.annotation(annotationInput);
 await client.bind.body(rid, aid, [{ op: 'add', item: { /* W3C body */ } }]);
 
-// Gather / Match — assemble context and run semantic search.
-const ctx = await lastValueFrom(client.gather.annotation(aid, rid));
+// Gather / Match — bounded streams; await yields the final value, subscribe
+// yields every progress emission.
+const ctx = await client.gather.annotation(rid, aid);
 client.match.search(rid, refId, ctx, { limit: 10 }).subscribe(/* ... */);
 
-// Yield — author new resources.
+// Yield — author new resources. Returns an UploadObservable; await yields
+// { resourceId }, subscribe yields the upload-progress lifecycle.
 const { resourceId } = await client.yield.resource({
   name, file, format, storageUri,
 });
 
-// Beckon — UI signals (hover, focus, selection).
+// Beckon, Bind, Browse, Mark — collaboration signals (void). Fire-and-
+// forget; fan out to other participants over the bus.
 client.beckon.hover(annotationId);
+client.bind.initiate({ annotationId });
+client.browse.click(annotationId, 'linking');
 ```
 
-The verb-by-verb walkthroughs live in [docs/flows](https://github.com/The-AI-Alliance/semiont/tree/main/docs/flows).
+The verb-by-verb walkthroughs live in [docs/protocol/flows](https://github.com/The-AI-Alliance/semiont/tree/main/docs/protocol/flows). The per-namespace API reference with concrete examples for each method lives in [`docs/Usage.md`](https://github.com/The-AI-Alliance/semiont/blob/main/packages/sdk/docs/Usage.md).
+
+## Documentation
+
+- [`docs/Usage.md`](https://github.com/The-AI-Alliance/semiont/blob/main/packages/sdk/docs/Usage.md) — per-namespace tour with concrete examples for Browse, Mark, Bind, Gather, Match, Yield, Beckon, Auth, Admin, Job, plus SSE and error handling.
+- [`docs/REACTIVE-MODEL.md`](https://github.com/The-AI-Alliance/semiont/blob/main/packages/sdk/docs/REACTIVE-MODEL.md) — the Promise-shape-over-Observable design: how `await` works on the SDK's return values without learning RxJS, and where RxJS is still visible by design.
+- [`docs/STATE-UNITS.md`](https://github.com/The-AI-Alliance/semiont/blob/main/packages/sdk/docs/STATE-UNITS.md) — the foundational pattern behind the flow state machines, worker adapters, and search pipeline: closure-based factories, RxJS-shaped surface, dispose lifecycle, and the axioms every new state unit honors.
+- [`docs/CACHE-SEMANTICS.md`](https://github.com/The-AI-Alliance/semiont/blob/main/packages/sdk/docs/CACHE-SEMANTICS.md) — the cache primitive's behavioral contract.
+- [`docs/protocol/TRANSPORT-CONTRACT.md`](https://github.com/The-AI-Alliance/semiont/blob/main/docs/protocol/TRANSPORT-CONTRACT.md) — the transport interface every `ITransport` must honor.
 
 ## Behavioral contract
 
-The guarantees every `ITransport` implementation must honor — what `subscribe()` does on disconnect, what `LastEventId` replay must look like, what `puts` must be idempotent — are documented in [packages/core/docs/TRANSPORT-CONTRACT.md](https://github.com/The-AI-Alliance/semiont/blob/main/packages/core/docs/TRANSPORT-CONTRACT.md). HTTP-specific guarantees (the `/bus/emit` gateway, SSE reconnect, `Last-Event-ID` replay window) live alongside the backend at [apps/backend/docs/TRANSPORT.md](https://github.com/The-AI-Alliance/semiont/blob/main/apps/backend/docs/TRANSPORT.md).
+The guarantees every `ITransport` implementation must honor — what `subscribe()` does on disconnect, what `LastEventId` replay must look like, what `puts` must be idempotent — are documented in [docs/protocol/TRANSPORT-CONTRACT.md](https://github.com/The-AI-Alliance/semiont/blob/main/docs/protocol/TRANSPORT-CONTRACT.md). HTTP-specific guarantees (the `/bus/emit` gateway, SSE reconnect, `Last-Event-ID` replay window) live in [docs/protocol/TRANSPORT-HTTP.md](https://github.com/The-AI-Alliance/semiont/blob/main/docs/protocol/TRANSPORT-HTTP.md).
 
 When implementing a new transport (gRPC, WebSocket, IPC, …), implement those interfaces from `@semiont/core` directly — there is no inheritance from `HttpTransport`.
 
@@ -132,4 +224,4 @@ Apache-2.0 — see [LICENSE](https://github.com/The-AI-Alliance/semiont/blob/mai
 - [`@semiont/api-client`](https://github.com/The-AI-Alliance/semiont/tree/main/packages/api-client) — HTTP transport (`HttpTransport`, `HttpContentTransport`)
 - [`@semiont/make-meaning`](https://github.com/The-AI-Alliance/semiont/tree/main/packages/make-meaning) — in-process transport (`LocalTransport`) and the actor model behind it
 - [`@semiont/observability`](https://github.com/The-AI-Alliance/semiont/tree/main/packages/observability) — OpenTelemetry tracing the SDK propagates across the bus
-- [`@semiont/react-ui`](https://github.com/The-AI-Alliance/semiont/tree/main/packages/react-ui) — React bindings (`useViewModel`, web `SessionStorage`)
+- [`@semiont/react-ui`](https://github.com/The-AI-Alliance/semiont/tree/main/packages/react-ui) — React bindings (`useStateUnit`, web `SessionStorage`)