@semiont/sdk 0.5.1 → 0.5.3

package/README.md

@@ -6,17 +6,105 @@
 [![npm downloads](https://img.shields.io/npm/dm/@semiont/sdk.svg)](https://www.npmjs.com/package/@semiont/sdk)
 [![License](https://img.shields.io/npm/l/@semiont/sdk.svg)](https://github.com/The-AI-Alliance/semiont/blob/main/LICENSE)
 
-The developer-facing SDK for [Semiont](https://github.com/The-AI-Alliance/semiont). This package owns the high-level surface every Semiont consumer reaches for: a verb-oriented `SemiontClient`, per-KB sessions, RxJS view-models, and the helpers that wire them all together.
+The TypeScript SDK for [Semiont](https://github.com/The-AI-Alliance/semiont) — a programmable surface for **collaborative knowledge work**. Whether you're building a browser app where humans annotate documents and propose links, an AI agent that gathers context and matches candidate references, a daemon that ingests new sources, or a one-shot script that queries an established knowledge base, you reach the same verb namespaces, the same collaboration primitives, the same lifecycle observables.
 
-The SDK is **transport-agnostic** — it consumes the `ITransport` and `IContentTransport` contracts from [`@semiont/core`](https://github.com/The-AI-Alliance/semiont/tree/main/packages/core). For HTTP, the canonical wire adapter is re-exported here for convenience. For in-process operation, use `LocalTransport` from [`@semiont/make-meaning`](https://github.com/The-AI-Alliance/semiont/tree/main/packages/make-meaning).
+The eight flows — *frame, yield, mark, match, bind, gather, browse, beckon* — describe what participants *do* when they work with a shared corpus. The first seven act on content; Frame acts on the schema layer (the conceptual vocabulary the others operate within). The SDK exposes them uniformly across surfaces. A human in a browser hovers an annotation; an AI agent at the other end of the bus sees the hover and reacts; a daemon ingests new text and every connected participant sees the corpus grow live. Humans and AI agents are peers — the SDK does not distinguish.
+
+The SDK is **transport-agnostic**: it consumes the `ITransport` and `IContentTransport` contracts from [`@semiont/core`](https://github.com/The-AI-Alliance/semiont/tree/main/packages/core). For HTTP backends, the canonical wire adapter is re-exported here for convenience. For in-process operation (CLI, agentic worker, embedded use), use `LocalTransport` from [`@semiont/make-meaning`](https://github.com/The-AI-Alliance/semiont/tree/main/packages/make-meaning).
+
+> **Where this doc fits.** This README is the *typed-surface reference* — what's in `@semiont/sdk`, how the namespaces are organized, what return shapes to expect. For the *protocol-level architectural framing* (the eight flows, the three programmable surfaces — CLI, SDK, Skills — the core tenets, the per-flow contracts), start with [`docs/protocol/README.md`](https://github.com/The-AI-Alliance/semiont/blob/main/docs/protocol/README.md). Daemon authors stitching multiple packages together also want the [skill packs](https://github.com/The-AI-Alliance/semiont/tree/main/docs/protocol/skills) — `semiont-session` for watcher daemons, `semiont-worker` for job-claim daemons, `semiont-wiki` for the end-to-end annotation pipeline.
+
+## Four ideas that hold the surface together
+
+The SDK is wider than a typical client library because the domain is — collaborative knowledge work over a shared corpus, with humans and AI agents as peers. Four framings make the API tractable; once you've seen them, the rest is predictable.
+
+### 1. Eight verbs
+
+Every operation in the SDK belongs to one of eight *flows* — verbs that describe what a participant *does* with a shared corpus. The flows are the entire vocabulary of the protocol; learn them once and the surface stays small.
+
+| Verb | What it does | Example methods |
+|---|---|---|
+| **frame** | Define and evolve the schema vocabulary (entity types, tag schemas, future relation types) | `frame.addEntityType`, `frame.addEntityTypes`, `frame.addTagSchema` |
+| **yield** | Introduce new resources into the system | `yield.resource`, `yield.fromAnnotation`, `yield.cloneToken` |
+| **mark** | Add structured metadata to resources | `mark.annotation`, `mark.assist`, `mark.archive` |
+| **match** | Search the corpus for candidate resources | `match.search` |
+| **bind** | Resolve ambiguous references to specific resources | `bind.body`, `bind.initiate` |
+| **gather** | Assemble related context around an annotation | `gather.annotation` |
+| **browse** | Navigate, read, and observe | `browse.resource`, `browse.annotations`, `browse.entityTypes`, `browse.tagSchemas`, `browse.click` |
+| **beckon** | Coordinate attention across participants | `beckon.hover`, `beckon.attention`, `beckon.sparkle` |
+
+Each flow is a namespace on `SemiontClient` (`client.mark.X(...)`, `client.gather.X(...)`, ...). The verb is the unit of mental model — a method call belongs to a flow, not to a noun. Frame is the schema-layer flow — content flows operate within the vocabulary Frame manages. Per-flow contracts live in [`docs/protocol/flows`](https://github.com/The-AI-Alliance/semiont/tree/main/docs/protocol/flows).
+
+### 2. One call, two ways to consume
+
+Most data-fetching libraries make you choose between Promise-shaped and Observable-shaped at the import line. The SDK doesn't. Every long-lived value comes back as an `Observable` that *also* implements `PromiseLike<T>` — `await` it for the final value, `.subscribe(...)` it for progress events or live updates, from the same call.
+
+```ts
+// One-shot consumer — never imports rxjs
+const resource = await client.browse.resource(rId);
+const result   = await client.match.search(rId, refId, ctx);
+
+// Reactive consumer — same call, .subscribe instead of await
+client.browse.resource(rId).subscribe((r) => {
+  if (r === undefined) showSkeleton();
+  else render(r);
+});
+client.match.search(rId, refId, ctx).subscribe((event) => {
+  if (event.kind === 'progress') updateProgress(event.data);
+});
+```
+
+A script that just wants the final value never imports anything from `rxjs`. A browser app rendering loading state subscribes to the same shape. The reactive substrate is preserved as a load-bearing architectural choice; the user-facing surface looks Promise-shaped when that's all the caller needs.
+
+Methods return one of: `Promise<T>` (atomic backend ops), an awaitable `Observable<T>` subclass (`StreamObservable<T>` for bounded progress streams, `CacheObservable<T>` for live queries with stale-while-revalidate), or `void` (collaboration signals — see #3). Per-method assignments and the typing discipline are in [`docs/REACTIVE-MODEL.md`](https://github.com/The-AI-Alliance/semiont/blob/main/packages/sdk/docs/REACTIVE-MODEL.md).
+
+### 3. Collaboration primitives
+
+The `void`-returning third category — collaboration signals — is the SDK's distinctive contribution to multi-participant coordination. They look fire-and-forget at the call site; on the bus they fan out across every participant.
+
+A human in a browser hovers an annotation (`beckon.hover(annotationId)`); an AI agent at the other end of the bus sees `beckon:hover` and reacts. An agent emits a sparkle (`beckon.sparkle(annotationId)`); the human's UI lights up the indicated annotation. A frontend state unit emits `mark.changeShape('rectangle')`; a different participant subscribed to `mark:shape-changed` reacts.
+
+This is *protocol-level* coordination — not browser-app fluff, not bolted-on presence — and it sits on the same typed namespace surface as data operations. Observers reach the same signals via `session.subscribe(channel, handler)` or `client.bus.get(channel)`. Three legitimate paths to the bus are documented in [`docs/REACTIVE-MODEL.md`](https://github.com/The-AI-Alliance/semiont/blob/main/packages/sdk/docs/REACTIVE-MODEL.md#three-paths-to-the-bus).
+
+### 4. Transport agnosticism
+
+`SemiontClient` is constructed against the `ITransport` and `IContentTransport` interfaces from `@semiont/core` — not against any particular wire. The same SDK surface runs over HTTP, in-process, or any future transport that satisfies the interface. None of the eight verb namespaces or the flow state machines reach for transport-specific features.
+
+```ts
+// HTTP — connect to a remote backend
+const client = await SemiontClient.signInHttp({ baseUrl, email, password });
+
+// In-process — same surface, no network
+const client = new SemiontClient(localTransport, localContentTransport);
+```
+
+`KnowledgeBase` carries a uniform shape regardless of transport (only the nested `endpoint` varies — `{ kind: 'http', host, port, protocol }` or `{ kind: 'local', kbId }`). Code that doesn't *construct* transports — your scripts, the verb namespaces, the flow state machines — never inspects which kind it has. Tests can run against an in-process transport for speed; daemons can embed the backend; the same domain code drives both.
 
 ## What's in the box
 
 - **`SemiontClient`** — the verb-oriented coordinator over a wire transport.
 - **Verb namespaces** — `browse`, `mark`, `bind`, `gather`, `match`, `yield`, `beckon`, `job`, `auth`, `admin`. Typed methods that wrap the bus protocol; consumers never touch raw channel strings.
-- **Session layer** — `SemiontSession` (per-KB authentication, token refresh, lifecycle), `SemiontBrowser` (tab-singleton coordinator), and `SessionStorage` adapters (`InMemorySessionStorage`, plus a web one in `@semiont/react-ui`).
-- **View-models** — RxJS-based MVVM factories the React layer mounts via `useViewModel`.
-- **Helpers** — `bus-request` (correlation-ID request/reply) and `cache` (per-key SWR cache).
+- **Collaboration primitives** — fire-and-forget signals on the verb namespaces (`beckon.hover`, `bind.initiate`, `mark.changeShape`, `browse.click`, ...) coordinate attention and intent across participants. Not afterthoughts, not browser-app fluff: they're how a multi-participant session stays coherent.
+- **Session layer** — `SemiontSession` (per-KB authentication, token refresh, lifecycle), `SemiontBrowser` (multi-KB orchestration), and `SessionStorage` adapters (`InMemorySessionStorage`, plus a browser-backed one in `@semiont/react-ui`).
+- **Flow state machines** — RxJS-based factories (`createMarkStateUnit`, `createGatherStateUnit`, `createMatchStateUnit`, `createYieldStateUnit`, `createBeckonStateUnit`) that wrap each long-running flow with `loading$` / `error$` / progress observables. UI-shape-agnostic — any consumer (browser, terminal, mobile, daemon) can subscribe.
+- **`WorkerBus`** — the transport-neutral channel-bus interface that worker-side adapters consume. Domain-specific worker adapters live with their domain — `createJobClaimAdapter` and `createJobQueueStateUnit` in [`@semiont/jobs`](https://github.com/The-AI-Alliance/semiont/tree/main/packages/jobs); `createSmelterActorStateUnit` in [`@semiont/make-meaning`](https://github.com/The-AI-Alliance/semiont/tree/main/packages/make-meaning) — and consume `WorkerBus` from here.
+- **Helpers** — `bus-request` (correlation-ID request/reply), the cache primitive backing live queries, and `createSearchPipeline` (debounced-search RxJS pipeline).
+
+Page-shaped state machines (admin tables, compose page, resource viewer page, etc.) live in [`@semiont/react-ui`](https://github.com/The-AI-Alliance/semiont/tree/main/packages/react-ui), alongside the components that render them. Those are framework-neutral but tied to the Semiont web frontend's specific page taxonomy; they don't apply to non-web consumers.
+
+## For non-web consumers (TUI, mobile, daemon, agent)
+
+`@semiont/sdk` is the only package a non-web Semiont consumer needs. From it you get:
+
+- `SemiontClient` + the eight verb namespaces (`frame`, `browse`, `mark`, `bind`, `gather`, `match`, `yield`, `beckon`)
+- Three infrastructure namespaces (`auth`, `admin`, `job`) when constructed with backend operations
+- `SemiontSession` for long-running token refresh + persistence
+- `SemiontBrowser` for multi-KB orchestration (transport-agnostic; takes a `SessionFactory`)
+- The five flow state machines above
+- The transport-neutral `WorkerBus` interface (worker adapters live in their domain packages — `@semiont/jobs`, `@semiont/make-meaning`)
+- Branded ID types, the unified error hierarchy, the `TransportErrorCode` neutral vocabulary
+
+Nothing page-shaped, nothing web-shell-shaped. A TUI, mobile reader, daemon, or AI agent installs `@semiont/sdk` alone (plus a transport package — `@semiont/api-client` for HTTP, `@semiont/make-meaning` for in-process).
 
 ## Installation
 
@@ -26,12 +114,12 @@ npm install @semiont/sdk
 
 ## Quick start (HTTP)
 
-For one-shot scripts, `SemiontClient.signIn(...)` is the credentials-first one-line construction:
+For one-shot scripts, `SemiontClient.signInHttp(...)` is the credentials-first one-line construction:
 
 ```ts
 import { SemiontClient } from '@semiont/sdk';
 
-const semiont = await SemiontClient.signIn({
+const semiont = await SemiontClient.signInHttp({
   baseUrl: 'http://localhost:4000',
   email: 'me@example.com',
   password: 'pwd',
@@ -43,7 +131,7 @@ console.log(resources);
 semiont.dispose();
 ```
 
-For long-running scripts that need to survive token expiry, use `SemiontSession.signIn(...)` — same credentials shape, plus proactive refresh, validation, storage-adapter wiring, and disposal. `kb` is required; its `id` is the storage key for this session, so distinct scripts must use distinct ids:
+For long-running scripts that need to survive token expiry, use `SemiontSession.signInHttp(...)` — same credentials shape, plus proactive refresh, validation, storage-adapter wiring, and disposal. `kb` is required; its `id` is the storage key for this session, so distinct scripts must use distinct ids:
 
 ```ts
 import { SemiontSession, InMemorySessionStorage, type KnowledgeBase } from '@semiont/sdk';
@@ -51,13 +139,11 @@ import { SemiontSession, InMemorySessionStorage, type KnowledgeBase } from '@sem
 const kb: KnowledgeBase = {
   id: 'my-watcher',
   label: 'My Watcher',
-  protocol: 'http',
-  host: 'localhost',
-  port: 4000,
   email: 'me@example.com',
+  endpoint: { kind: 'http', host: 'localhost', port: 4000, protocol: 'http' },
 };
 
-const session = await SemiontSession.signIn({
+const session = await SemiontSession.signInHttp({
   kb,
   storage: new InMemorySessionStorage(),
   baseUrl: 'http://localhost:4000',
@@ -72,6 +158,8 @@ const resources = await session.client.browse.resources({ limit: 10 });
 await session.dispose();
 ```
 
+`KnowledgeBase` is uniform regardless of transport kind; the variation lives in the nested `endpoint` (currently `{ kind: 'http', host, port, protocol }` or `{ kind: 'local', kbId }`). Code that doesn't construct transports — your scripts, the verb namespaces, the flow state machines — never inspects `endpoint`.
+
 If you already have an access token (CLI cached-token path, env-var token, embedded auth flow), use `SemiontClient.fromHttp({ baseUrl, token })` or `SemiontSession.fromHttp({ baseUrl, token, storage, kb, refresh, ... })` to skip the auth round-trip.
 
 ## Quick start (in-process)
@@ -100,9 +188,9 @@ const client = new SemiontClient(
 
 Same `SemiontClient`, same verb namespaces — no network involved. There is no `fromLocal` factory because the in-process transport's dependencies (knowledgeSystem, eventBus, userId) are not boilerplate the SDK can hide.
 
-## Verb namespaces
+## Worked examples
 
-All ten namespaces hang off `SemiontClient`. Methods that return data return either a `Promise<T>` (atomic ops like `mark.archive`) or an awaitable Observable subclass — `StreamObservable<T>` for streams (`mark.assist`, `gather.annotation`, `match.search`, `yield.fromAnnotation`) and `CacheObservable<T>` for live queries (`browse.*`). Both subclasses implement `PromiseLike<T>`, so consumers can `await` them directly. Reactive consumers can `.subscribe(...)` exactly as with a plain Observable. The bus is invisible to callers — channel strings, correlation IDs, and reconnection are internal.
+The eight verb namespaces hang off `SemiontClient`, plus three infrastructure namespaces (`auth`, `admin`, `job`) when the client was constructed with backend operations. Each example below uses one of the return shapes mentioned above; the per-method assignment table is in [`docs/REACTIVE-MODEL.md`](https://github.com/The-AI-Alliance/semiont/blob/main/packages/sdk/docs/REACTIVE-MODEL.md#method-by-method-assignment).
 
 ```ts
 // Browse — live queries; await yields the loaded value, subscribe yields
@@ -111,30 +199,36 @@ const resources = await client.browse.resources({ limit: 10 });
 client.browse.resource(resourceId).subscribe(/* ... */);
 
 // Mark / Bind — atomic operations return Promise<T>.
-const { annotationId } = await client.mark.annotation(rid, request);
+// `mark.annotation` takes the W3C-shaped input directly; the resourceId
+// is derived from `input.target.source` and returned as a branded id.
+const { annotationId } = await client.mark.annotation(annotationInput);
 await client.bind.body(rid, aid, [{ op: 'add', item: { /* W3C body */ } }]);
 
 // Gather / Match — bounded streams; await yields the final value, subscribe
 // yields every progress emission.
-const ctx = await client.gather.annotation(aid, rid);
+const ctx = await client.gather.annotation(rid, aid);
 client.match.search(rid, refId, ctx, { limit: 10 }).subscribe(/* ... */);
 
-// Yield — author new resources.
+// Yield — author new resources. Returns an UploadObservable; await yields
+// { resourceId }, subscribe yields the upload-progress lifecycle.
 const { resourceId } = await client.yield.resource({
   name, file, format, storageUri,
 });
 
-// Beckon — UI signals (hover, focus, selection).
+// Beckon, Bind, Browse, Mark — collaboration signals (void). Fire-and-
+// forget; fan out to other participants over the bus.
 client.beckon.hover(annotationId);
+client.bind.initiate({ annotationId });
+client.browse.click(annotationId, 'linking');
 ```
 
-The verb-by-verb walkthroughs live in [docs/flows](https://github.com/The-AI-Alliance/semiont/tree/main/docs/flows).
-
-`.pipe(...)` returns a plain `Observable<T>` — once you compose with RxJS operators you've explicitly entered RxJS land, and `lastValueFrom` from `rxjs` is the right bridge. The `firstValueFrom`/`lastValueFrom` re-exports from `@semiont/sdk` stay available for that case.
+The verb-by-verb walkthroughs live in [docs/protocol/flows](https://github.com/The-AI-Alliance/semiont/tree/main/docs/protocol/flows). The per-namespace API reference with concrete examples for each method lives in [`docs/Usage.md`](https://github.com/The-AI-Alliance/semiont/blob/main/packages/sdk/docs/Usage.md).
 
 ## Documentation
 
 - [`docs/Usage.md`](https://github.com/The-AI-Alliance/semiont/blob/main/packages/sdk/docs/Usage.md) — per-namespace tour with concrete examples for Browse, Mark, Bind, Gather, Match, Yield, Beckon, Auth, Admin, Job, plus SSE and error handling.
+- [`docs/REACTIVE-MODEL.md`](https://github.com/The-AI-Alliance/semiont/blob/main/packages/sdk/docs/REACTIVE-MODEL.md) — the Promise-shape-over-Observable design: how `await` works on the SDK's return values without learning RxJS, and where RxJS is still visible by design.
+- [`docs/STATE-UNITS.md`](https://github.com/The-AI-Alliance/semiont/blob/main/packages/sdk/docs/STATE-UNITS.md) — the foundational pattern behind the flow state machines, worker adapters, and search pipeline: closure-based factories, RxJS-shaped surface, dispose lifecycle, and the axioms every new state unit honors.
 - [`docs/CACHE-SEMANTICS.md`](https://github.com/The-AI-Alliance/semiont/blob/main/packages/sdk/docs/CACHE-SEMANTICS.md) — the cache primitive's behavioral contract.
 - [`docs/protocol/TRANSPORT-CONTRACT.md`](https://github.com/The-AI-Alliance/semiont/blob/main/docs/protocol/TRANSPORT-CONTRACT.md) — the transport interface every `ITransport` must honor.
 
@@ -154,4 +248,4 @@ Apache-2.0 — see [LICENSE](https://github.com/The-AI-Alliance/semiont/blob/mai
 - [`@semiont/api-client`](https://github.com/The-AI-Alliance/semiont/tree/main/packages/api-client) — HTTP transport (`HttpTransport`, `HttpContentTransport`)
 - [`@semiont/make-meaning`](https://github.com/The-AI-Alliance/semiont/tree/main/packages/make-meaning) — in-process transport (`LocalTransport`) and the actor model behind it
 - [`@semiont/observability`](https://github.com/The-AI-Alliance/semiont/tree/main/packages/observability) — OpenTelemetry tracing the SDK propagates across the bus
-- [`@semiont/react-ui`](https://github.com/The-AI-Alliance/semiont/tree/main/packages/react-ui) — React bindings (`useViewModel`, web `SessionStorage`)
+- [`@semiont/react-ui`](https://github.com/The-AI-Alliance/semiont/tree/main/packages/react-ui) — React bindings (`useStateUnit`, web `SessionStorage`)