@happyvertical/smrt-chat 0.55.9 → 0.55.11

package/AGENTS.md

@@ -5,7 +5,9 @@ Chat rooms, threads, and agent sessions with app-controlled tool whitelisting.
 ## Dev Server
 
 `pnpm --dir packages/chat dev` runs a package-local SvelteKit workbench. The root
-route is an interactive chat surface with a dev-only `/api/dev-chat` endpoint:
+route separates **Character setup** and **Chat dev** into tabs; switching tabs
+retains the current character rig and chat state. The chat tab is an interactive
+surface with a dev-only `/api/dev-chat` endpoint:
 it uses `@happyvertical/ai` when local provider credentials are present and
 falls back to a deterministic local assistant otherwise. `/api/dev-chat-stream`
 is its SSE companion (#1936) — the same provider/local-fallback resolution wired
@@ -21,12 +23,10 @@ over `src/routes/chat-dev.intents.ts`, #2588). Their dev-config dependencies
 stylesheet import) and the stage→apply tool-call gotchas are in
 [agents/dev-workbench.md](agents/dev-workbench.md).
 
-The root workbench also has a dev-only voice conversation mode. It reads voice
-gateway connection details through `/api/dev-voice/config`, streams browser mic
-audio to `WS /ws/voice` as PCM16 mono, appends gateway transcripts/responses to
-the chat, and plays returned TTS audio. Exposing
-`SMRT_CHAT_DEV_VOICE_GATEWAY_TOKEN` to the browser requires
-`SMRT_CHAT_DEV_VOICE_GATEWAY_EXPOSE_TOKEN=true`; keep that local-only.
+Dev-only voice and opt-in local character persistence are documented in
+[agents/dev-workbench.md](agents/dev-workbench.md#voice-and-local-character-persistence).
+These endpoints are loopback-only demos; production hosts supply authenticated
+principals, tenancy, and authorization to the owning services.
 
 ## Models
 
@@ -122,6 +122,16 @@ reads/saves run under the principal's tenant (`withPrincipalTenant`), refuse a
 mismatching ambient tenant, and filter by the principal tenant explicitly. Contract and authority model:
 [`packages/reports/agents/runtime-reports.md`](../reports/agents/runtime-reports.md).
 
+## Overview tools (#3727)
+
+`createOverviewTools({ host })` (`overview-tools.ts`) returns
+`overviews.describe`, `overviews.apply` (an atomic batch of structured widget
+operations) and `overviews.undo` (single step, keyed by tenant, user and page).
+The host opens a page per `PrincipalRun` (smrt-svelte's
+`createOverviewAssistantSurface`, typed structurally here); validation and
+`canCustomize` are the surface's. Contract:
+[agents/overview-tools.md](agents/overview-tools.md).
+
 ## Conversational Harness (L3, #1891)
 
 The `AgentSession` runtime depends on personas, agents, and users. Keep those
@@ -138,14 +148,15 @@ Voice is an input mode for the existing persona chat harness, not a separate cha
 
 The gateway bearer token proves only "this request came from the gateway"; it never authorizes the end user. The short-lived `VoiceSession` binding is the user/session proof, and untrusted gateway metadata must be validated against that binding before any chat write or tool loop. Tool execution remains fail-closed through the persona allow-list mirrored onto `AgentSession` by `bindPersonaToSession()`.
 
-## Token Streaming (SSE, #1936)
+## Token streaming
 
-`chat-stream.ts` is the SSE seam for embeddable conversational UIs:
-`runChatConversationStream()` (engine), `createChatStreamHandler({ authorize })`
-(Fetch handler; `authorize` is the sole trust boundary), and the browser-safe
-`SmrtChatBackend` (`./client`). Persona-bound streams reuse the harness gates
-unchanged, and `binding.extraTools` is resolved server-side. Full contract:
-[agents/token-streaming.md](agents/token-streaming.md).
+`chat-stream.ts` serves browser-safe SSE through `./client`. The host's
+`authorize(request, body)` is the sole trust boundary: resolve identity, tenant,
+capabilities, and generation limits server-side; never authorize from request
+session metadata. Persona turns reuse the principal and fail-closed tool gates.
+The engine and browser-safe `SmrtChatBackend` keep custom-tool resolution
+server-side. See [agents/token-streaming.md](agents/token-streaming.md) for the
+event contract, CORS policy, and streaming behavior.
 
 ## Streamed assistant turns with browser tools (#2908)
 

package/README.md

@@ -29,6 +29,18 @@ trusting browser data or throwing raw transport errors.
 
 ## Usage
 
+### Team Chat recipe
+
+`ChatRoomsRecipe` declares the `chat.rooms` unit for app navigation and
+generated read-only room, message, membership, thread, and reaction views.
+It places the Chat Rooms entry in the Team section under the Team collaboration
+card, using the shell's `users` icon.
+Hosts can mount its exported `ChatLayout` at `/chat/rooms`; the recipe also
+points demo hosts at the browser-safe static chat fixture from `./playground`.
+Its [user help](src/chat-rooms.recipe.md) describes how to find rooms and
+follow conversations. The package playground previews use static mock fixtures;
+they do not connect to a live chat service.
+
 ### Local dev server
 
 Run the chat package workbench directly:

package/agents/dev-workbench.md

@@ -69,3 +69,72 @@ are non-obvious and cost time to find:
   programmatic `element.click()` (`event.isTrusted === false`) for `apply`, which
   is the stage→apply consent split doing its job. Verification must use a real
   input event, not `evaluate(() => button.click())`.
+
+## Voice and local character persistence
+
+The root workbench also has a dev-only voice conversation mode. It reads voice
+gateway connection details through `/api/dev-voice/config`, streams browser mic
+audio to `WS /ws/voice` as PCM16 mono, appends gateway transcripts/responses to
+the chat, and plays returned TTS audio. Exposing
+`SMRT_CHAT_DEV_VOICE_GATEWAY_TOKEN` to the browser requires
+`SMRT_CHAT_DEV_VOICE_GATEWAY_EXPOSE_TOKEN=true`; keep that local-only.
+
+### Local character persistence
+
+`/api/dev-character-persistence` is an opt-in loopback-only demo bridge for
+the photographic character workbench. It is disabled unless
+`SMRT_CHAT_DEV_CHARACTER_PERSISTENCE=true`; it accepts neither actor, tenant,
+nor profile ids from the browser. Its server configuration names one
+pre-provisioned profile and tenant plus a temporary SQLite database and asset
+directory, all under the operating system temporary directory. First run
+`DATABASE_URL=<the same SQLite path> pnpm smrt db:migrate`, then run
+`pnpm exec tsx src/scripts/provision-dev-character-persistence.ts` with the
+`SMRT_CHAT_DEV_CHARACTER_*` variables set. The route never creates schema.
+Use fresh UUIDs for `SMRT_CHAT_DEV_CHARACTER_PROFILE_ID` and
+`SMRT_CHAT_DEV_CHARACTER_TENANT_ID`; keep the database and asset directory in
+the OS temporary directory, for example `/tmp/smrt-character-setup/` on Linux
+or the path reported by `node -p "require('node:os').tmpdir()"` on macOS.
+
+This is not production authentication. A production host must provide its
+authenticated principal, tenant, and authorization policy to
+`PhotoCutoutProfileStore`; it must not copy this fixed local identity pattern.
+
+### Local helper control panel
+
+`/api/dev-helper` uses the same loopback-only identity and durable Profile
+metadata store. Provisioning also creates its application-scoped metafield;
+the route never creates schema or accepts actor, tenant, policy, provider, or
+asset URL fields from the browser. It resolves the current gallery, allowed
+voice, and helper policy on every load/save/reset. Its whole-value persistence
+semantic is last-successful-write-wins, so a production host with concurrent
+editors must serialize writes per profile/application or supply a stronger
+adapter. The speech route resolves the saved offered voice server-side; clients
+submit only text. Helper preference loading never starts audio or a microphone.
+
+### Character conversation lifecycle
+
+The Character conversation tab refreshes the persisted rig each time it becomes
+active, retaining its AssistantDock controller and history. Leaving the tab stops
+microphone input and reply playback; listening mode requires an explicit restart.
+An outstanding turn keeps both the listening input and full dock composer busy
+across mode changes and restarts until it settles. Both inputs share the retained
+controller’s pending-send state; microphone Stop remains reachable. Polling
+retains active sends until their transport completes, even when older history
+contains the same message text.
+Late persistence loads and cancelled speech responses cannot replace the current
+rig or audio. Spoken captions begin only when SDK playback actually starts.
+Listening mode also keeps the newest assistant reply in a dedicated reply
+region, so a muted or failed spoken reply is still readable while history stays
+hidden. A speech failure is shown beside the conversation controls; disabling
+spoken replies aborts its outstanding request and playback. While the tab is
+inactive, its fixed dock is hidden and polling is paused.
+
+The development workbench links the SDK speech and animation packages from
+sibling worktrees. Keep those worktrees available until their released package
+versions replace the links; a fresh browser or Vite load cannot resolve a
+linked package whose sibling checkout is absent.
+
+The development transport sends at most 24 context messages within a 16 KiB UTF-8
+JSON body, dropping oldest context first. Failed turns are not committed to its
+history, so retrying does not duplicate them. Draft proposals use one shared
+200-character limit in the model tool, route, preview, and execution boundary.

package/agents/helper-preferences.md

@@ -0,0 +1,65 @@
+# Helper preferences server contract
+
+`HelperPreferencesService` is presentation policy, never an identity, persona,
+tool, or authorization authority. A host creates `HelperContext` from its
+authenticated request and supplies `authorize`, `resolvePolicy`, registered
+`styleIds`, and `validateOffering`. Browser payloads never supply those values.
+
+Use `createHelperProfilePreferenceStore({ db, applicationId, metafieldSlug })` only with an
+already provisioned, application-scoped ProfileMetafield. The adapter checks
+the context profile and tenant on every operation, fails duplicate rows, and
+persists one complete versioned preference value. `clear()` writes the explicit
+`{ version: 1, cleared: true }` marker. Its durable guarantee is complete
+last-successful-write-wins values **only when the host serializes writes for a
+profile/application across instances**; it does not offer distributed locking
+or compare-and-swap. Hosts requiring either must supply their own store adapter.
+
+The service re-resolves policy and validates saved values on each load, save,
+and reset. Invalid old values produce a recoverable snapshot without silently
+overwriting metadata. Owner-assigned policies ignore personal state and deny
+save/reset. An offering validator must re-check saved asset ownership, tenant,
+relationship, and payload before it can become effective.
+
+When an editable policy has no valid effective helper but can preserve its
+locked baseline fields while selecting an offered replacement, the snapshot
+contains `recoveryDraft`. It is an edit candidate only: UI must submit it to
+`save()` and wait for the returned snapshot before applying it.
+
+## Browser DTO import
+
+Browser clients import the DTO, parser, and field list from
+`@happyvertical/smrt-chat/helper-preferences`. This public subpath deliberately
+contains no Profile/SQL adapter or `HelperPreferencesService`; authenticated
+servers import those server APIs from the package root.
+
+```ts
+import {
+  helperPreferenceFields,
+  parseHelperPreferences,
+  type HelperClient,
+  type HelperPreferences,
+} from '@happyvertical/smrt-chat/helper-preferences';
+```
+
+## Svelte helper controls
+
+`HelperControlPanel` distinguishes a local draft from the applied server
+snapshot. Selecting a gallery item, changing a control, or completing photo
+setup creates an unsaved draft; only the snapshot returned by `save()` is
+applied through `onchanged`. Photo setup returns to the settings panel, selects
+the newly returned offering in the draft, and restores focus to its gallery
+choice so the next action is explicit: **Save changes**.
+
+Gallery choices use roving radio keyboard behavior (Arrow keys, Home, End).
+Hosts should provide distinct safe offering labels, such as a user-provided
+name with a saved timestamp, rather than raw duplicate filenames.
+
+`FloatingAssistant` accepts an optional bottom-corner placement. Conversation
+hosts pass the effective saved placement to keep the dock with its selected
+helper. Opening listening mode may reveal the existing assistant surface, but
+must never start microphone capture or audio playback; those remain explicit
+user actions.
+
+`FloatingAssistant.hideIdleControls` is opt-in and defaults to false for existing hosts.
+The character listening workbench enables it to hide idle controls chrome while
+keeping active runs, errors and confirmation surfaces reachable.

package/agents/overview-tools.md

@@ -0,0 +1,172 @@
+# Overview assistant tools (#3727 phase 4)
+
+`createOverviewTools({ host, undoStore?, audit?, maxOperations? })`
+(`src/overview-tools.ts`, package index) returns three `PrincipalTool`s for the
+persona / assistant-route `extraTools` seam. They let the assistant add and
+configure widgets on a customizable overview page ("add a chart of overdue
+invoices") with structured operations, and undo its last change. The overview
+model, widget registry and option schemas are smrt-svelte's
+([overview-surfaces.md](../../smrt-svelte/agents/overview-surfaces.md)).
+
+| Slug (function name) | Arguments | Result |
+|---|---|---|
+| `overviews.describe` (`overviews-describe`) | `{ page? }` | `{ page, overview }`: allowed widget types with option fields (type, required, default, range, `choices`, page `models`), span range, the current widgets in order, `canCustomize`. Without `page`: `{ pages }` from `host.pages` |
+| `overviews.apply` (`overviews-apply`) | `{ page, operations }` | `{ page, changed, applied: [{ index, op, id }], undoToken, overview }`; `changed: false` and `undoToken: null` for a batch that changes nothing |
+| `overviews.undo` (`overviews-undo`) | `{ page, undoToken }` | `{ page, undone: true, overview }` |
+
+Operations (1 to `maxOperations`, default 20): `add { type, span?, options?,
+index? }`, `configure { id, options }` (merged onto the current options; `null`
+clears a key to its default), `move { id, index }`, `resize { id, span }`,
+`remove { id }`. Option values are JSON primitives over the widget's closed
+field vocabulary; there is no query, filter expression, URL or SQL anywhere.
+
+## Authority
+
+- **The host decides, per principal.** `host.open(run, pageId)` returns the
+  page for that `PrincipalRun` or `null` (unknown and not offered answer the
+  same 404). Build it with smrt-svelte's `createOverviewAssistantSurface`
+  (`./overview/server`) from the page's `OverviewDefinition`, the
+  principal's own stored override, `canCustomize` from the principal's role,
+  and `persist`. Never derive any of these from model input except the page id.
+- **Tenant.** Like the runtime-report tools, with tenancy on a run without
+  `run.context.tenantId` is refused (403 `tenant_required`), `host.open` and
+  every surface call (including `persist`) run under the principal's tenant
+  (entered when no tenant context is active), and an ambient tenant that
+  differs from the principal's is refused (`DataSurfaceDeniedError`) unless
+  the caller is in a system-context or super-admin path. The host scopes its
+  reads and writes by `run.context.tenantId`.
+- **Validation is the overview model's.** The surface plans each batch with
+  `planOverviewOperations` and checks the result with `checkOverviewOverride`
+  (allowed set and `allowedIn`, page-confined `models`, option schemas, span
+  ranges, the widget cap), so the assistant can only store an override the
+  page's own save endpoint accepts. A batch is atomic.
+- **Errors** are `OverviewToolError` with `status`, `code`, `publicMessage`,
+  `issues`. `runToolLoop` classifies 404/409/422 as `invalid_request` (the model
+  sees `publicMessage`, which lists every issue as `#<n> <op>: <message>`, and is
+  told to fix and retry) and 403 as `not_permitted`. Codes: `invalid_page`,
+  `invalid_operations` (422), `unknown_page` (404), `tenant_required` (403),
+  `not_allowed` (403,
+  `canCustomize` false, on apply and on undo), `conflict` (409, apply lost a
+  race with another save), `nothing_to_undo`, `changed_since`,
+  `cannot_restore` (409).
+- **Writes are conditional.** `surface.persist(override)` answers
+  `OverviewToolPersistResult` (`{ ok: true }` or `{ ok: false, reason:
+  'conflict' | 'not_allowed' | 'invalid', issues? }`) and must write only if
+  the stored row is still the revision the surface was opened on. A save that
+  lands between `host.open` and the write is a `conflict`: apply answers 409
+  `conflict` (describe again, rebuild the batch), undo answers 409
+  `changed_since` and keeps its entry. smrt-svelte's surface takes the loaded
+  `revision` and passes `{ revision }` to its `persist` option, so the
+  revision-guarded `save` of `@happyvertical/smrt-preferences`'
+  `createOverviewStore` plugs in directly (see below).
+- **Undo** is single-step. Apply stores `{ token, before, after }` under
+  `[tenantId, userId, pageId]` of the calling run; undo looks it up under the
+  calling run only, so another principal's (or tenant's) token is simply not
+  found. It refuses when the stored override is no longer `after` (someone
+  changed the page since), re-checks `before` with `check`, persists it
+  conditionally, and deletes the entry only after that write succeeded.
+  Removal is compare-and-delete: `OverviewUndoStore.delete(key, token)` is a
+  no-op unless the stored entry still carries that token, so an undo that
+  finishes after a newer batch stored its entry never removes that entry.
+  Storing is a conditional set: an entry's `createdAt` is stamped right after
+  its batch's conditional write succeeded (strictly increasing per process),
+  and `set` is a no-op when the stored entry is newer, so an older apply whose
+  entry write lands late never replaces a newer batch's entry. Ordering by
+  write-success time rather than serializing per key works across replicas
+  sharing one store (a lock would only cover one process), and the revision
+  guard already serializes the writes themselves; revisions are opaque here,
+  so they cannot order entries. Across replicas the stamps rely on clocks
+  agreeing to within the time between two applies by the same person on the
+  same page.
+- **The undo store must outlive a request and a turn.** Without `undoStore`
+  the tools use `defaultOverviewUndoStore()`, one in-memory store per process
+  (30-minute TTL, 1000 keys), so tools built per turn
+  (`extraTools: (context) => createOverviewTools(...)`) still undo an earlier
+  turn's batch. Never pass a store created inside the per-turn factory. A
+  multi-replica host passes one shared `OverviewUndoStore`, created once,
+  whose `set` (newer `createdAt` wins) and `delete(key, token)` are atomic
+  conditional operations.
+- **After a successful write the call never fails.** The undo entry is written
+  after the write; if the store throws, apply still answers `changed: true`
+  with `undoToken: null` and a `note` not to apply the batch again (a lost entry
+  only costs the Undo, while an entry written first could later restore over a
+  value the batch never wrote). Audit is best effort; failures go to
+  `onError` (default `console.warn`).
+
+## Wiring to persistence (phase 3)
+
+The surface is the only seam; this package has no table and no dependency on
+smrt-svelte or `@happyvertical/smrt-preferences` (smrt-svelte depends on chat
+for tests, so the types are structural). Phase 3 stores overview layouts in
+`@happyvertical/smrt-preferences` (table `_smrt_ui_preferences`, preference
+kind `overview`); its overview layer is `createOverviewStore({ db })`, built
+over the generic `createPreferenceStore({ db })` (pass `preferences` to share
+one). Every store call acts as the ambient principal, which is the run's
+principal inside the tool loop, under the tenant the tools enter.
+
+```ts
+import { createOverviewTools } from '@happyvertical/smrt-chat';
+import {
+  createOverviewStore,
+  withTenantDefaults,
+} from '@happyvertical/smrt-preferences';
+import { createOverviewAssistantSurface } from '@happyvertical/smrt-svelte/overview/server';
+
+// Module scope: created once, shared by every turn and request.
+const overviewUndo = createSharedOverviewUndoStore(); // or omit for the process default
+
+const overviewTools = (db: DatabaseInterface /* per turn */) => {
+  const store = createOverviewStore({ db });
+  return createOverviewTools({
+    undoStore: overviewUndo,
+    host: {
+      async open(run, pageId) {
+        const definition = overviewDefinitions.get(pageId);
+        if (!definition) return null;
+        const state = await store.load(definition, registry);
+        if (!state.user) return null;
+        return createOverviewAssistantSurface({
+          // The user tier is a delta over the tenant default.
+          definition: withTenantDefaults(definition, state.tenant.document),
+          registry,
+          override: state.user.override,
+          revision: state.user.revision,
+          canCustomize: state.canCustomize.user,
+          // The page definition: the store merges the tenant tier itself.
+          persist: (override, { revision }) =>
+            store.save(definition, registry, { scope: 'user', override, revision }),
+        });
+      },
+    },
+  });
+};
+```
+
+`createSharedOverviewUndoStore` stands for the host's own shared store (any
+`OverviewUndoStore`). How the store's answer maps onto the tools:
+
+| `store.save(definition, registry, { scope, override, revision })` (`OverviewSaveResult`) | Surface / tool |
+|---|---|
+| `{ ok: true, override, revision }` | the surface follows the new `revision`; apply answers `changed: true` |
+| `{ ok: false, reason: 'conflict' }` (the loaded `revision` is stale, or a first write raced another; nothing is overwritten) | apply: 409 `conflict`; undo: 409 `changed_since`, entry kept |
+| `{ ok: false, reason: 'not_allowed' }` (no `overviews.personalize` / `overviews.customize`) | 403 `not_allowed` |
+| `{ ok: false, reason: 'invalid', issues }` (`checkOverviewOverride` refused it) | 422 `invalid_operations` |
+
+`revision` is `null` when the principal had no row yet; the first write then
+creates it, and a concurrent first write answers `conflict`. Add the three
+slugs to the persona's / route's `allowedTools`.
+
+## In the browser
+
+When the overview is mounted on the page the dock runs on, use the browser
+tools instead: smrt-svelte's `createOverviewAssistant(controller)` +
+`useOverviewAssistantTools` register `overview_describe`, `overview_apply` and
+`overview_undo` as page tools (add `overview_*` to the route's
+`clientToolAllowList`). A batch lands through `controller.restore`, so the grid
+updates live and the host's `onchange` persists it, and `OverviewAssistantUndo`
+(or a toast) offers Undo. Same operations, same validation, same error shape
+(returned as JSON, not thrown).
+
+Tests: `src/overview-tools.test.ts` (contract against a structural surface) and
+`packages/smrt-svelte/src/components/overview/__tests__/assistant-tools.integration.test.ts`
+(these tools over the real model).

package/agents/token-streaming.md

@@ -1,7 +1,8 @@
 # Token Streaming (SSE, #1936)
 
-Moved verbatim from `packages/chat/AGENTS.md` to keep the instruction chain under
-the 32 KB cap (`pnpm check:agents-chain`). The summary stays in AGENTS.md.
+Reference linked from [../AGENTS.md](../AGENTS.md). This contract was moved from
+`packages/chat/AGENTS.md` to keep the instruction chain under the 32 KB cap
+(`pnpm check:agents-chain`); AGENTS.md retains the cap-oriented summary.
 
 `chat-stream.ts` is the SSE seam for embeddable conversational UIs (first consumer: the Happy chat widget, `animation#5`): a client POSTs the conversation so far and receives a `text/event-stream` of `data: <json>` frames — `token` deltas as the model generates, then a final `done` frame with the message. The wire `ChatStreamEvent` union also declares `emotion` and `control` (#1921 host-page control commands) lanes for forward compatibility; the v1 engine emits `token`/`done`/`error`.
 

package/dist/chunks/{ChatService-C9UxuJaw.js → ChatService-CFaaDHyU.js}

@@ -1,7 +1,7 @@
 import { ObjectRegistry, SmrtCollection, SmrtObject, crossPackageRef, field, foreignKey, isPostgresDatabase, smrt } from "@happyvertical/smrt-core";
 import { TenantScoped, tenantId } from "@happyvertical/smrt-tenancy";
-import { createHash } from "node:crypto";
 import { ProfileCollection, resolveAgentProfileId } from "@happyvertical/smrt-profiles";
+import { createHash } from "node:crypto";
 //#region \0rolldown/runtime.js
 var __defProp$7 = Object.defineProperty;
 var __exportAll = (all, no_symbols) => {
@@ -2225,4 +2225,4 @@ function sendAgentReply(service, params) {
 //#endregion
 export { sendAgentReply as a, ChatThread as c, ChatParticipant as d, ChatMessage as f, clientRequestMessageId as i, ChatRoom as l, ChatService as n, VoiceSessionCollection as o, AgentSession as p, ChatService_exports as r, VoiceSession as s, ChatClientRequestConflictError as t, ChatReaction as u };
 
-//# sourceMappingURL=ChatService-C9UxuJaw.js.map
+//# sourceMappingURL=ChatService-CFaaDHyU.js.map