@memberjunction/realtime-runtime 0.0.0 → 6.2.0-edge.0

package/LICENSE

@@ -0,0 +1,183 @@
+Business Source License 1.1
+
+License text copyright (c) 2024 MariaDB plc, All Rights Reserved.
+"Business Source License" is a trademark of MariaDB plc.
+
+-----------------------------------------------------------------------------
+
+Parameters
+
+Licensor:             Blue Cypress, Inc.
+
+Licensed Work:        MemberJunction.
+                      The Licensed Work is (c) 2023-2026 Blue Cypress, Inc.
+
+Additional Use Grant: Subject to the terms of this License, Licensor grants
+                      you the following additional rights to make Production
+                      Use of the Licensed Work.
+
+                      1. Internal Use
+
+                      You may make production use of the Licensed Work for
+                      your own internal business or organizational operations.
+
+                      2. Nonprofit Use
+
+                      If you are a Nonprofit, you may make production use of
+                      the Licensed Work for the operations and activities of
+                      your Organizational Family.
+
+                      3. MemberJunction Certified Program Use
+
+                      If you are authorized by Licensor under the
+                      MemberJunction Certified Program to provide professional
+                      services using the Licensed Work, you may make
+                      production use of the Licensed Work in providing such
+                      professional services to a client, provided that:
+
+                      (a) the Licensed Work is deployed in, and the applicable
+                      production use occurs within, an environment owned,
+                      leased, licensed, subscribed to, or otherwise controlled
+                      by that client; and
+
+                      (b) the production use is for that client's own internal
+                      business or organizational operations or is otherwise
+                      independently permitted to that client under this
+                      Additional Use Grant.
+
+                      4. Definitions Applicable to the Additional Use Grant
+
+                      "Affiliate" means, with respect to a specified Person,
+                      any other Person that directly or indirectly Controls,
+                      is Controlled by, or is under common Control with such
+                      specified Person.
+
+                      "Control" (including the terms "Controls," "Controlled
+                      by," and "under common Control with") means the direct
+                      or indirect possession of the power to direct or cause
+                      the direction of the management and policies of a
+                      Person, whether through ownership of voting interests,
+                      by contract, or otherwise.
+
+                      "Organizational Family" means, with respect to a Person,
+                      (a) such Person and its Affiliates, and (b) any
+                      nonprofit organization, governmental entity, chapter,
+                      division, local affiliate, regional affiliate, state
+                      affiliate, national affiliate, or other entity that is
+                      formally affiliated with such Person through governing
+                      documents, a charter, bylaws, a membership agreement, or
+                      another written organizational instrument, and is
+                      recognized under such documents as part of the same
+                      organizational structure.
+
+                      "Nonprofit" means a Person recognized by the Internal
+                      Revenue Service as exempt from federal income taxation
+                      under Section 501(c)(3), 501(c)(4), 501(c)(5), or
+                      501(c)(6) of the Internal Revenue Code, or a foreign
+                      organization recognized under substantially equivalent
+                      laws.
+
+                      A Person claiming eligibility as a Nonprofit shall, upon
+                      Licensor's reasonable request, provide documentation
+                      reasonably sufficient to demonstrate that it qualifies
+                      as a Nonprofit. If such Person materially misrepresents,
+                      or is unable to demonstrate, its qualification as a
+                      Nonprofit, the rights granted to such Person under
+                      Section 2 of this Additional Use Grant shall terminate.
+
+                      "MemberJunction Certified Program" means Licensor's
+                      then-current program for certifying and authorizing a
+                      Person to provide professional services using the
+                      Licensed Work.
+
+                      "Person" means any individual, corporation, limited
+                      liability company, partnership, association, nonprofit
+                      organization, governmental entity, or other legal or
+                      organizational entity.
+
+Change Date:          Four (4) years from the date the Licensed Work is first
+                      made available.
+
+Change License:       MIT License.
+
+For information about alternative licensing arrangements for the Licensed
+Work, please contact Blue Cypress, Inc.
+
+-----------------------------------------------------------------------------
+
+Terms
+
+The Licensor hereby grants you the right to copy, modify, create derivative
+works, redistribute, and make non-production use of the Licensed Work. The
+Licensor may make an Additional Use Grant, above, permitting limited
+production use.
+
+Effective on the Change Date, or the fourth anniversary of the first publicly
+available distribution of a specific version of the Licensed Work under this
+License, whichever comes first, the Licensor hereby grants you rights under
+the terms of the Change License, and the rights granted in the paragraph
+above terminate.
+
+If your use of the Licensed Work does not comply with the requirements
+currently in effect as described in this License, you must purchase a
+commercial license from the Licensor, its affiliated entities, or authorized
+resellers, or you must refrain from using the Licensed Work.
+
+All copies of the original and modified Licensed Work, and derivative works
+of the Licensed Work, are subject to this License. This License applies
+separately for each version of the Licensed Work and the Change Date may vary
+for each version of the Licensed Work released by Licensor.
+
+You must conspicuously display this License on each original or modified copy
+of the Licensed Work. If you receive the Licensed Work in original or
+modified form from a third party, the terms and conditions set forth in this
+License apply to your use of that work.
+
+Any use of the Licensed Work in violation of this License will automatically
+terminate your rights under this License for the current and all other
+versions of the Licensed Work.
+
+This License does not grant you any right in any trademark or logo of
+Licensor or its affiliates (provided that you may use a trademark or logo of
+Licensor as expressly required by this License).
+
+TO THE EXTENT PERMITTED BY APPLICABLE LAW, THE LICENSED WORK IS PROVIDED ON
+AN "AS IS" BASIS. LICENSOR HEREBY DISCLAIMS ALL WARRANTIES AND CONDITIONS,
+EXPRESS OR IMPLIED, INCLUDING (WITHOUT LIMITATION) WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, NON-INFRINGEMENT, AND
+TITLE.
+
+MariaDB hereby grants you permission to use this License's text to license
+your works, and to refer to it using the trademark "Business Source License",
+as long as you comply with the Covenants of Licensor below.
+
+-----------------------------------------------------------------------------
+
+Covenants of Licensor
+
+In consideration of the right to use this License's text and the "Business
+Source License" name and trademark, Licensor covenants to MariaDB, and to all
+other recipients of the licensed work to be provided by Licensor:
+
+1. To specify as the Change License the GPL Version 2.0 or any later version,
+   or a license that is compatible with GPL Version 2.0 or a later version,
+   where "compatible" means that software provided under the Change License
+   can be included in a program with software provided under GPL Version 2.0
+   or a later version. Licensor may specify additional Change Licenses without
+   limitation.
+
+2. To either: (a) specify an additional grant of rights to use that does not
+   impose any additional restriction on the right granted in this License, as
+   the Additional Use Grant; or (b) insert the text "None".
+
+3. To specify a Change Date.
+
+4. Not to modify this License in any other way.
+
+-----------------------------------------------------------------------------
+
+Notice
+
+The Business Source License (this document, or the "License") is not an Open
+Source license. However, the Licensed Work will eventually be made available
+under an Open Source License, as stated in this License.

package/dist/channels/base-realtime-channel-client.d.ts

@@ -0,0 +1,402 @@
+import type { Observable } from 'rxjs';
+import { IMetadataProvider } from '@memberjunction/core';
+import { JSONValue, RealtimeToolDefinition, RealtimeTrackDescriptor, RealtimeTrackDirection } from '@memberjunction/ai';
+import type { BaseRealtimeClient } from '@memberjunction/ai-realtime-client';
+import type { AppContextSnapshot } from '@memberjunction/ai-core-plus';
+/**
+ * A UI framework's component-class reference, as far as this runtime is concerned.
+ *
+ * This was `Type<T>` from `@angular/core`. That import was the *only* Angular tie in this file —
+ * and it was never a real dependency: the runtime receives a component class from a channel plugin
+ * and hands it to the host to render. It never constructs one, never reads a property off one, and
+ * never cares what framework produced it.
+ *
+ * It is deliberately opaque. Trying to describe a component class *structurally* here would be
+ * dishonest precision: each framework's constructor contract differs, and narrowing to any one of
+ * them re-couples the runtime to that framework. The runtime only ever carries this value from the
+ * plugin to the host, so "a class" is the whole of what it needs to know — the host, which does
+ * know what a component is, narrows it at the single point where it instantiates one.
+ *
+ * Angular's `Type<T>` satisfies this unchanged, so no existing plugin needs editing. Without it,
+ * every interactive channel — the whiteboard, the remote browser, the media surface — is
+ * permanently Angular-only, which is exactly why a non-Angular host could not offer channels.
+ */
+export type RealtimeSurfaceComponentType = Function;
+/**
+ * Host services handed to a {@link BaseRealtimeChannelClient} at {@link BaseRealtimeChannelClient.Initialize}.
+ *
+ * The context is the plugin's ONLY line back to the live session — channels never talk to
+ * `RealtimeSessionService` (or any host component) directly, which is what keeps them drop-in
+ * plugins. Every member is host-implemented:
+ *
+ *  - the SESSION SERVICE supplies {@link SendContextNote} (perception feed into the live
+ *    model), {@link RequestSave} (debounced state-of-record persistence onto the session's
+ *    `MJ: AI Agent Session Channels` row) and {@link AgentName};
+ *  - the OVERLAY SHELL wires {@link SetFocusMode} (through the service's focus stream) so a
+ *    channel surface can request the focus layout — main call column collapsed, surface
+ *    panel filling the overlay, floating call pill riding on top.
+ */
+export interface RealtimeChannelContext {
+    /** Display name of the agent the live session fronts (e.g. `"Sage"`), fixed at session start. */
+    AgentName: string;
+    /**
+     * The MJ metadata provider the live session runs on — the SAME `IMetadataProvider` instance that
+     * authenticates the session (NOT necessarily the global default). A channel whose surface renders
+     * MemberJunction-backed data (e.g. the Media channel streaming an `MJ: Files` record through
+     * `mj-storage-media-player`) threads THIS provider into its surface / GraphQL calls so it stays
+     * multi-provider safe. `null` only in degenerate/early states; channels fall back to the global
+     * default when absent.
+     */
+    Provider: IMetadataProvider | null;
+    /**
+     * Feeds a background context note into the live realtime model (no spoken reply is
+     * requested) — the PERCEPTION direction of the channel: serialized state deltas flow here
+     * so the agent stays aware of what's on the surface. No-op when the session isn't live.
+     */
+    SendContextNote(text: string): void;
+    /**
+     * Asks the host to persist `stateJson` as this channel's state of record. The host
+     * DEBOUNCES (a change burst becomes one save) and flushes any pending save at session
+     * teardown — the plugin just calls this on every state mutation and never schedules
+     * timers itself. Best-effort: persistence failures are logged host-side, never thrown.
+     */
+    RequestSave(stateJson: string): void;
+    /**
+     * Requests (or releases) the FOCUS layout for this channel's surface: the overlay
+     * collapses the main call column so the surface owns the screen, with a compact floating
+     * call pill keeping mute / thread / end reachable. Any channel may request it; the host
+     * tracks which channel holds focus and routes the pill's "exit" back to it via
+     * {@link BaseRealtimeChannelClient.RequestFocusExit}.
+     */
+    SetFocusMode(on: boolean): void;
+    /**
+     * Asks the live model to SPEAK a response to the supplied instructions RIGHT NOW —
+     * the channel's "react to this" path, e.g. a widget submission the user expects an
+     * audible reaction to ({@link SendContextNote} deliberately never triggers speech).
+     * Rides the realtime client's spoken-update channel, so on some providers the spoken
+     * reply is narration-kind (ephemeral, not persisted as a caption). OPTIONAL member:
+     * older host contexts may not supply it — plugins must call it null-safely.
+     */
+    RequestSpokenResponse?(instructions: string): void;
+    /**
+     * Persists a snapshot of the channel's state as a first-class versioned artifact
+     * (`MJ: Artifacts` + version, linked into conversation history when possible) — e.g. the
+     * whiteboard's "Save to artifacts". Distinct from {@link RealtimeChannelContext.RequestSave},
+     * which maintains the session's rolling state of record. Best-effort: resolves to the
+     * created Artifact ID, or `null` on failure (logged host-side, never thrown). Works during
+     * the call AND right after it ends (the host retains the session id for late saves).
+     */
+    SaveAsArtifact(name: string, contentJson: string): Promise<string | null>;
+    /**
+     * The live `MJ: AI Agent Sessions` id this channel belongs to, or `null` before the
+     * session has minted / after it has torn down. A channel whose tools or surface drive a
+     * SERVER-SIDE resource (e.g. the Remote Browser channel's server-hosted browser) passes
+     * this as the `agentSessionID` argument to its own GraphQL resolvers via
+     * {@link ExecuteServerAction}. Most channels (whiteboard, shared doc) keep all state
+     * client-side and never read it.
+     */
+    AgentSessionID: string | null;
+    /**
+     * Executes a CHANNEL-SPECIFIC GraphQL operation against the session's MJ server — the
+     * escape hatch for channels backed by a SERVER-SIDE resource that the generic
+     * {@link RequestSave} / {@link SaveAsArtifact} contract doesn't cover (e.g. the Remote
+     * Browser channel driving a server-hosted browser through its own
+     * `ExecuteRemoteBrowserAction` mutation + `RemoteBrowserSnapshot` query).
+     *
+     * The host runs the operation through the SAME provider the live session uses, so the
+     * request rides the authenticated session. Best-effort and tolerant: a transport or
+     * server error resolves to `null` (logged host-side) rather than throwing, so a channel
+     * can map the failure to a model-readable result string without `try/catch`.
+     *
+     * @typeParam TResult The expected shape of the GraphQL operation's data payload.
+     * @param query The GraphQL query/mutation document.
+     * @param variables The operation variables (all JSON-serializable).
+     * @returns The operation's `data` payload, or `null` on any failure / when no session is live.
+     */
+    ExecuteServerAction<TResult>(query: string, variables: Record<string, JSONValue>): Promise<TResult | null>;
+    /**
+     * OPTIONAL — the live app-context stream (where the user is, what they see, and the available
+     * client-tool + agent manifest), pushed by the host (Explorer) at session start and on subsequent
+     * changes. The headless `ClientContextChannel` subscribes to this and streams deltas to the model
+     * via {@link SendContextNote}. Absent on hosts that don't supply app context (e.g. custom apps);
+     * channels must read it null-safely.
+     */
+    AppContext$?: Observable<AppContextSnapshot | null>;
+    /**
+     * OPTIONAL — executes a host-registered surface CLIENT TOOL by name (the handlers the host wired
+     * from the active surface's `SetAgentClientTools`). The headless `ClientContextChannel`'s
+     * `ContextTool` proxy routes the model's `{ action, params }` here so a surface tool runs in the
+     * browser with no server round-trip. Best-effort and tolerant — resolves to a structured result
+     * (never throws); `Success: false` for an unknown tool or a thrown handler. Absent on hosts that
+     * register no client tools.
+     *
+     * @param name The client-tool name (the model's `action`).
+     * @param params The tool parameters (the model's `params`).
+     * @returns A structured result the channel serializes back to the model.
+     */
+    ExecuteClientTool?(name: string, params: Record<string, unknown>): Promise<{
+        Success: boolean;
+        Result?: unknown;
+        ErrorMessage?: string;
+    }>;
+    /**
+     * OPTIONAL — sends a visual frame into the live session's inbound video track
+     * (e.g. from Whiteboard or Remote Browser video bridges). No-op when the session
+     * has not established an inbound video track or is not live.
+     *
+     * @param base64Image The image data (base64-encoded JPEG/PNG).
+     * @param mimeType The image MIME type (defaults to 'image/jpeg').
+     */
+    SendVideoFrame?(base64Image: string, mimeType?: string): void;
+    /**
+     * OPTIONAL — checks whether a media track is currently established on the live session.
+     */
+    IsTrackEstablished?(modality: string, direction: RealtimeTrackDirection): boolean;
+    /**
+     * OPTIONAL — the underlying {@link BaseRealtimeClient} driving the media and transport planes.
+     */
+    Client?: BaseRealtimeClient | null;
+}
+/**
+ * The first-run INTRO content for an interactive channel — the concise "what is this surface
+ * and how do I use it" copy the overlay shows the very first time a user opens this channel's
+ * tab (persisted "seen" per user, so it's shown ONCE per channel per user).
+ *
+ * A channel opts in by overriding {@link BaseRealtimeChannelClient.GetOnboardingDetails}; the
+ * default returns `null`, so the base Voice/text channel (which has no plugin at all) AND any
+ * plugin that doesn't override it show nothing.
+ */
+export interface ChannelOnboardingDetails {
+    /** Short title, usually the surface name (e.g. `"Whiteboard"`). */
+    Heading: string;
+    /** One or two sentences: what the surface is and what the user can expect to see on it. */
+    Description: string;
+    /** Optional quick-tip bullets (kept to 2-3 short, scannable lines). */
+    Tips?: string[];
+    /** Optional Font Awesome icon class for the intro panel (e.g. `'fa-solid fa-chalkboard'`). */
+    IconClass?: string;
+}
+/**
+ * Base class for CLIENT-SIDE interactive-channel plugins (per
+ * `plans/ai-agent-sessions.md` → "Interactive Channels" / "Pluggable Channel Interfaces").
+ *
+ * An interactive channel is a bidirectional surface the session's single realtime agent
+ * both PERCEIVES and ACTS UPON (whiteboard, shared doc, map, …). A concrete plugin
+ * contributes everything the channel needs, so the session service / call overlay carry
+ * ZERO channel-specific wiring:
+ *
+ *  1. a CLIENT-EXECUTED TOOL SET ({@link GetToolDefinitions}, declared to the realtime
+ *     model at session mint) plus the local executor ({@link ApplyAgentTool}) the host
+ *     routes `{@link ToolNamePrefix}*` calls to — the ACTION direction;
+ *  2. a STATE→CONTEXT SERIALIZER policy — the plugin owns its state engine and pushes
+ *     coalesced deltas through {@link RealtimeChannelContext.SendContextNote} — the
+ *     PERCEPTION direction;
+ *  3. an OPTIONAL ANGULAR SURFACE ({@link GetSurfaceComponent}) the overlay creates dynamically
+ *     in a channel tab, handed back through {@link BindSurface} so the plugin wires its own
+ *     inputs/outputs (the host never knows the component's API). A channel may be **server-only**
+ *     (no rendered surface) — e.g. a bridge-contributed meeting-controls or native-whiteboard
+ *     channel whose surface lives on the external platform, not in MJ. Such a channel returns
+ *     `null` from {@link GetSurfaceComponent} ({@link HasSurface} is `false`) and the overlay
+ *     simply skips its tab while still wiring its tools + perception;
+ *  4. a STATE OF RECORD ({@link SerializeState}, persisted via
+ *     {@link RealtimeChannelContext.RequestSave} under {@link ChannelName}).
+ *
+ * ### Registration & resolution (mirrors the realtime model drivers)
+ * Concrete plugins are `@RegisterClass(BaseRealtimeChannelClient, '<ClientPluginClass>')`
+ * and are resolved at session start from the `MJ: AI Agent Channels` registry: each ACTIVE
+ * row's `ClientPluginClass` is the ClassFactory key (exactly how `BaseRealtimeClient`
+ * drivers resolve by provider key). Ship a `Load<YourChannel>()` no-op alongside the class
+ * and call it from a static code path to defeat tree-shaking.
+ *
+ * ### Lifecycle — ONE INSTANCE PER SESSION (not a singleton)
+ * `ClassFactory.CreateInstance` → {@link Initialize}(ctx) → zero or more
+ * {@link BindSurface}/{@link UnbindSurface} cycles (the surface pane is created/destroyed
+ * with the overlay's tab panel, e.g. collapse/expand) → {@link Dispose} at teardown.
+ * {@link ApplyAgentTool} MUST work with NO surface bound (apply to the state engine
+ * directly; skip the UI garnish) — tool calls can arrive while the panel is collapsed.
+ *
+ * @typeParam TSurface The plugin's Angular surface component type. The host only ever
+ *   sees the default (`object`) — the typed parameter exists so concrete plugins get a
+ *   fully typed {@link BindSurface} without casts.
+ */
+export declare abstract class BaseRealtimeChannelClient<TSurface extends object = object> {
+    /**
+     * The host context, available from {@link Initialize} until {@link Dispose}.
+     * `null` outside that window — guard with `?.` in any code that can run early/late.
+     */
+    protected Context: RealtimeChannelContext | null;
+    /**
+     * The channel definition name — MUST match the `MJ: AI Agent Channels` row's `Name`
+     * (e.g. `'Whiteboard'`). Used as the persistence key for {@link SerializeState} saves
+     * and as the channel tab's stable key.
+     */
+    abstract get ChannelName(): string;
+    /**
+     * The shared name prefix of every tool this channel exposes (e.g. `'Whiteboard_'`).
+     * The host registers ONE local-execution route per plugin: tool calls whose name starts
+     * with this prefix go to {@link ApplyAgentTool} instead of the server relay.
+     */
+    abstract get ToolNamePrefix(): string;
+    /** Label for the channel's tab on the overlay's surface panel (e.g. `'Whiteboard'`). */
+    abstract get TabTitle(): string;
+    /** Font Awesome icon class for the channel's tab (e.g. `'fa-solid fa-chalkboard'`). */
+    abstract get TabIcon(): string;
+    /**
+     * OPTIONAL accent color for the channel's tab (a CSS color string, e.g. an `hsl()` /
+     * token). When a plugin supplies one, the overlay paints the tab's dot + active underline
+     * with it; when omitted (the default `null`), the overlay derives a stable, deterministic
+     * color from the {@link ChannelName} so every channel still reads as a distinct, colored
+     * surface. A channel only overrides this to enforce a specific brand accent.
+     */
+    get TabColor(): string | null;
+    /**
+     * The channel's CLIENT-EXECUTED tool declarations, aggregated by the session service
+     * into the `clientTools` set declared to the realtime model at session mint. The server
+     * only DECLARES these — execution stays in the browser via {@link ApplyAgentTool}.
+     */
+    abstract GetToolDefinitions(): RealtimeToolDefinition[];
+    /**
+     * Executes ONE agent tool call locally (the ACTION direction) and returns the result
+     * JSON string fed back to the model as the `tool_response`. Called for every tool whose
+     * name starts with {@link ToolNamePrefix}. Must work both WITH a bound surface (apply +
+     * UI garnish) and WITHOUT one (apply to the state engine directly — the tab pane may not
+     * exist, e.g. the surface panel is collapsed). Should not throw: return a
+     * `{ success: false, error }` payload so the model can narrate the failure (the host
+     * additionally wraps anything thrown).
+     */
+    abstract ApplyAgentTool(toolName: string, argsJson: string): string | Promise<string>;
+    /**
+     * The Angular component the overlay creates dynamically as this channel's tab pane, or `null`
+     * for a **server-only** channel that renders no MJ surface (its surface, if any, lives on the
+     * external platform — e.g. a bridge-contributed native whiteboard or meeting-controls channel).
+     *
+     * When this returns `null`, the overlay renders NO tab for the channel and never calls
+     * {@link BindSurface}/{@link UnbindSurface} — but the channel's tools ({@link GetToolDefinitions} /
+     * {@link ApplyAgentTool}) and perception ({@link RealtimeChannelContext.SendContextNote}) still run.
+     * A created surface instance is handed straight back via {@link BindSurface}; the host treats it as
+     * opaque.
+     *
+     * Default: `null` (server-only). A channel with a rendered surface overrides this to return its
+     * component type.
+     */
+    GetSurfaceComponent(): RealtimeSurfaceComponentType | null;
+    /**
+     * Whether this channel has a rendered MJ surface ({@link GetSurfaceComponent} returns non-null).
+     * The overlay uses this to decide whether to register a surface tab; server-only channels are
+     * `false`. Override only if surface availability must be decided WITHOUT constructing the type
+     * (the default calls {@link GetSurfaceComponent} once).
+     */
+    HasSurface(): boolean;
+    /**
+     * The channel's FIRST-RUN INTRO content, or `null` when the channel offers no onboarding.
+     * The overlay shows this once per channel per user — the first time the user opens this
+     * channel's surface tab — and remembers "seen" via the user's settings (NOT localStorage),
+     * so it never re-appears on later sessions or other devices.
+     *
+     * Default: `null` (no intro). The base Voice/text channel has no plugin at all, so it never
+     * shows an intro; an interactive channel with a surface worth explaining (whiteboard, remote
+     * browser, …) overrides this to return its {@link ChannelOnboardingDetails}. A plugin that
+     * doesn't override it simply shows nothing — onboarding is strictly opt-in.
+     */
+    GetOnboardingDetails(): ChannelOnboardingDetails | null;
+    /**
+     * Called by the host right after it created the surface component (and BEFORE the
+     * component's first change detection, so inputs set here are visible in its `ngOnInit`).
+     * The plugin — which knows its own component type — sets inputs (state engine, agent
+     * name, …) and subscribes outputs here, wiring perception/garnish flows back through
+     * {@link Context}. May be called again with a NEW instance after an
+     * {@link UnbindSurface} (the pane is destroyed/recreated with the tab panel).
+     */
+    abstract BindSurface(instance: TSurface): void;
+    /**
+     * Called by the host when the surface component is being destroyed (tab panel
+     * collapsed / overlay torn down). Drop the instance reference and unsubscribe any
+     * output subscriptions — after this, {@link ApplyAgentTool} runs in its no-surface
+     * mode. Default: no-op.
+     */
+    UnbindSurface(): void;
+    /**
+     * Binds the host context and invokes the {@link OnInitialize} hook. Called exactly once
+     * per session, right after ClassFactory instantiation and before any tool call or
+     * surface bind.
+     */
+    Initialize(ctx: RealtimeChannelContext): void;
+    /**
+     * Subclass hook invoked from {@link Initialize} once {@link Context} is bound — wire
+     * state-engine subscriptions (e.g. state change → `Context.RequestSave(...)`) here.
+     * Default: no-op.
+     */
+    protected OnInitialize(): void;
+    /**
+     * Subclass hook invoked once the realtime session is connected and live (the client driver
+     * is created, connected, and media tracks negotiated). Channels that establish media bridges
+     * (e.g. video streaming) can start them here when `Context.Client` is available.
+     * Default: no-op.
+     */
+    OnSessionStarted(): void;
+    /**
+     * Max time {@link ResolveAgentSessionId} waits for the session id to bind before giving up, and the
+     * poll interval it re-checks on. Protected so tests can shrink the wait; production keeps the
+     * defaults (the real mint race is sub-second, 8s is generous headroom).
+     */
+    protected SessionIdWaitTimeoutMs: number;
+    protected SessionIdWaitIntervalMs: number;
+    /**
+     * Resolves the live {@link RealtimeChannelContext.AgentSessionID}, briefly WAITING for it when it
+     * isn't bound yet rather than giving up instantly. `AgentSessionID` is a live getter over the
+     * session service's current id: it reads `null` in the window BEFORE the session mints (the
+     * realtime model can fire a tool call the very first beat it connects, before `mintSession`
+     * resolves) and again AFTER teardown. Server-backed tool paths (e.g. the Remote Browser channel's
+     * `browser_*` tools) call this instead of reading `Context?.AgentSessionID` synchronously, so a tool
+     * invoked a beat early WAITS for the session to come live — defense-in-depth against the
+     * "session id missing" race — instead of returning a hard failure to the model.
+     *
+     * Returns the id as soon as it's non-null (the common path resolves immediately, no delay), or
+     * `null` if it's still unbound after {@link SessionIdWaitTimeoutMs} — or the channel was
+     * {@link Dispose}d in the meantime (`Context` goes null, so we stop waiting on a torn-down session).
+     */
+    protected ResolveAgentSessionId(): Promise<string | null>;
+    /**
+     * Serializes the channel's current state of record (the payload persisted on the
+     * session's channel row), or `null` when the channel keeps no persistent state.
+     * Default: `null`.
+     */
+    SerializeState(): string | null;
+    /**
+     * Restores a PRIOR session's saved channel state (the payload a previous session
+     * persisted via {@link SerializeState} / {@link RealtimeChannelContext.RequestSave}).
+     * Invoked by the session host AFTER {@link Initialize} and BEFORE any surface binding,
+     * when a prior session's saved state exists for this channel.
+     *
+     * Returns `true` when the state was applied; `false` when the channel ignored it —
+     * either because it keeps no persistent state (this default) or because the payload was
+     * malformed/incompatible. Implementations MUST be tolerant: never throw on bad input,
+     * just return `false` and start fresh.
+     */
+    RestoreState(stateJson: string): boolean;
+    /**
+     * The focus pill's "exit" affordance, routed by the overlay to the channel that holds
+     * focus. Implementations should leave focus mode through their OWN surface (so surface
+     * toggles stay in sync), ultimately emitting `Context.SetFocusMode(false)`. The overlay
+     * defensively clears its layout flag as well, so a no-op default is safe.
+     */
+    RequestFocusExit(): void;
+    /**
+     * Media tracks this client channel can SOURCE — samples flowing into the model.
+     * Default `[]`.
+     */
+    GetSourcedTracks(): readonly RealtimeTrackDescriptor[];
+    /**
+     * Media tracks this client channel can SINK — samples flowing from the model OUT.
+     * Default `[]`.
+     */
+    GetSunkTracks(): readonly RealtimeTrackDescriptor[];
+    /**
+     * Tears the plugin down at session end: release the surface binding, unsubscribe
+     * state-engine subscriptions, then drop the context. Subclasses overriding this MUST
+     * call `super.Dispose()`. Any final state save has already been flushed by the host
+     * (the debounced {@link RealtimeChannelContext.RequestSave} pipeline) before disposal.
+     */
+    Dispose(): void;
+}
+//# sourceMappingURL=base-realtime-channel-client.d.ts.map

package/dist/channels/base-realtime-channel-client.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"base-realtime-channel-client.d.ts","sourceRoot":"","sources":["../../src/channels/base-realtime-channel-client.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EAAE,UAAU,EAAE,MAAM,MAAM,CAAC;AACvC,OAAO,EAAE,iBAAiB,EAAE,MAAM,sBAAsB,CAAC;AACzD,OAAO,EAAE,SAAS,EAAE,sBAAsB,EAAE,uBAAuB,EAAE,sBAAsB,EAAE,MAAM,oBAAoB,CAAC;AACxH,OAAO,KAAK,EAAE,kBAAkB,EAAE,MAAM,oCAAoC,CAAC;AAC7E,OAAO,KAAK,EAAE,kBAAkB,EAAE,MAAM,8BAA8B,CAAC;AAEvE;;;;;;;;;;;;;;;;;GAiBG;AACH,MAAM,MAAM,4BAA4B,GAAG,QAAQ,CAAC;AAEpD;;;;;;;;;;;;;GAaG;AACH,MAAM,WAAW,sBAAsB;IACrC,iGAAiG;IACjG,SAAS,EAAE,MAAM,CAAC;IAElB;;;;;;;OAOG;IACH,QAAQ,EAAE,iBAAiB,GAAG,IAAI,CAAC;IAEnC;;;;OAIG;IACH,eAAe,CAAC,IAAI,EAAE,MAAM,GAAG,IAAI,CAAC;IAEpC;;;;;OAKG;IACH,WAAW,CAAC,SAAS,EAAE,MAAM,GAAG,IAAI,CAAC;IAErC;;;;;;OAMG;IACH,YAAY,CAAC,EAAE,EAAE,OAAO,GAAG,IAAI,CAAC;IAEhC;;;;;;;OAOG;IACH,qBAAqB,CAAC,CAAC,YAAY,EAAE,MAAM,GAAG,IAAI,CAAC;IAEnD;;;;;;;OAOG;IACH,cAAc,CAAC,IAAI,EAAE,MAAM,EAAE,WAAW,EAAE,MAAM,GAAG,OAAO,CAAC,MAAM,GAAG,IAAI,CAAC,CAAC;IAE1E;;;;;;;OAOG;IACH,cAAc,EAAE,MAAM,GAAG,IAAI,CAAC;IAE9B;;;;;;;;;;;;;;;;OAgBG;IACH,mBAAmB,CAAC,OAAO,EAAE,KAAK,EAAE,MAAM,EAAE,SAAS,EAAE,MAAM,CAAC,MAAM,EAAE,SAAS,CAAC,GAAG,OAAO,CAAC,OAAO,GAAG,IAAI,CAAC,CAAC;IAE3G;;;;;;OAMG;IACH,WAAW,CAAC,EAAE,UAAU,CAAC,kBAAkB,GAAG,IAAI,CAAC,CAAC;IAEpD;;;;;;;;;;;OAWG;IACH,iBAAiB,CAAC,CAChB,IAAI,EAAE,MAAM,EACZ,MAAM,EAAE,MAAM,CAAC,MAAM,EAAE,OAAO,CAAC,GAC9B,OAAO,CAAC;QAAE,OAAO,EAAE,OAAO,CAAC;QAAC,MAAM,CAAC,EAAE,OAAO,CAAC;QAAC,YAAY,CAAC,EAAE,MAAM,CAAA;KAAE,CAAC,CAAC;IAE1E;;;;;;;OAOG;IACH,cAAc,CAAC,CAAC,WAAW,EAAE,MAAM,EAAE,QAAQ,CAAC,EAAE,MAAM,GAAG,IAAI,CAAC;IAE9D;;OAEG;IACH,kBAAkB,CAAC,CAAC,QAAQ,EAAE,MAAM,EAAE,SAAS,EAAE,sBAAsB,GAAG,OAAO,CAAC;IAElF;;OAEG;IACH,MAAM,CAAC,EAAE,kBAAkB,GAAG,IAAI,CAAC;CACpC;AAED;;;;;;;;GAQG;AACH,MAAM,WAAW,wBAAwB;IACvC,mEAAmE;IACnE,OAAO,EAAE,MAAM,CAAC;IAChB,2FAA2F;IAC3F,WAAW,EAAE,MAAM,CAAC;IACpB,uEAAuE;IACvE,IAAI,CAAC,EAAE,MAAM,EAAE,CAAC;IAChB,8FAA8F;IAC9F,SAAS,CAAC,EAAE,MAAM,CAAC;CACpB;AAED;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;GA0CG;AACH,8BAAsB,yBAAyB,CAAC,QAAQ,SAAS,MAAM,GAAG,MAAM;IAC9E;;;OAGG;IACH,SAAS,CAAC,OAAO,EAAE,sBAAsB,GAAG,IAAI,CAAQ;IAExD;;;;OAIG;IACH,aAAoB,WAAW,IAAI,MAAM,CAAC;IAE1C;;;;OAIG;IACH,aAAoB,cAAc,IAAI,MAAM,CAAC;IAE7C,wFAAwF;IACxF,aAAoB,QAAQ,IAAI,MAAM,CAAC;IAEvC,uFAAuF;IACvF,aAAoB,OAAO,IAAI,MAAM,CAAC;IAEtC;;;;;;OAMG;IACH,IAAW,QAAQ,IAAI,MAAM,GAAG,IAAI,CAEnC;IAED;;;;OAIG;aACa,kBAAkB,IAAI,sBAAsB,EAAE;IAE9D;;;;;;;;OAQG;aACa,cAAc,CAAC,QAAQ,EAAE,MAAM,EAAE,QAAQ,EAAE,MAAM,GAAG,MAAM,GAAG,OAAO,CAAC,MAAM,CAAC;IAE5F;;;;;;;;;;;;;OAaG;IACI,mBAAmB,IAAI,4BAA4B,GAAG,IAAI;IAIjE;;;;;OAKG;IACI,UAAU,IAAI,OAAO;IAI5B;;;;;;;;;;OAUG;IACI,oBAAoB,IAAI,wBAAwB,GAAG,IAAI;IAI9D;;;;;;;OAOG;aACa,WAAW,CAAC,QAAQ,EAAE,QAAQ,GAAG,IAAI;IAErD;;;;;OAKG;IACI,aAAa,IAAI,IAAI;IAI5B;;;;OAIG;IACI,UAAU,CAAC,GAAG,EAAE,sBAAsB,GAAG,IAAI;IAKpD;;;;OAIG;IACH,SAAS,CAAC,YAAY,IAAI,IAAI;IAI9B;;;;;OAKG;IACI,gBAAgB,IAAI,IAAI;IAI/B;;;;OAIG;IACH,SAAS,CAAC,sBAAsB,SAAQ;IACxC,SAAS,CAAC,uBAAuB,SAAO;IAExC;;;;;;;;;;;;;OAaG;cACa,qBAAqB,IAAI,OAAO,CAAC,MAAM,GAAG,IAAI,CAAC;IAoB/D;;;;OAIG;IACI,cAAc,IAAI,MAAM,GAAG,IAAI;IAItC;;;;;;;;;;OAUG;IACI,YAAY,CAAC,SAAS,EAAE,MAAM,GAAG,OAAO;IAI/C;;;;;OAKG;IACI,gBAAgB,IAAI,IAAI;IAI/B;;;OAGG;IACI,gBAAgB,IAAI,SAAS,uBAAuB,EAAE;IAI7D;;;OAGG;IACI,aAAa,IAAI,SAAS,uBAAuB,EAAE;IAI1D;;;;;OAKG;IACI,OAAO,IAAI,IAAI;CAIvB"}