@pouchy_ai/companion-sdk 0.10.0

package/CHANGELOG.md

@@ -0,0 +1,86 @@
+# Changelog
+
+All notable changes to `@pouchy_ai/companion-sdk` are documented in this file.
+
+The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
+This package follows [Semantic Versioning](https://semver.org/spec/v2.0.0.html)
+with the usual pre-1.0 caveat: while the major version is `0`, a **minor** bump
+(`0.x.0`) may contain breaking changes and a **patch** bump (`0.x.y`) is reserved
+for backwards-compatible fixes and additions. The wire protocol has its own
+version (`PROTOCOL_VERSION`, currently `1`) guarded by `protocol.drift.test.ts`;
+a protocol bump is always called out explicitly here.
+
+## [Unreleased]
+
+## [0.10.0] - 2026-06-28
+
+Completes the SDK surface for all six companion capabilities, with Instant UI and
+cross-app A2A added. All changes are **additive** (new event types are ignored by
+older receivers — `PROTOCOL_VERSION` stays `1`).
+
+### Added
+
+- **Instant UI.** The companion can render an interactive panel on the host:
+  `companion.ui_action` (`onRender`) carries a platform-neutral genui interface
+  (`{ title?, nodes, state?, computed?, reportChanges? }`), and
+  `companion.ui_update` (`onInterfaceUpdate`) applies live `{key,value}` writes to
+  an already-rendered panel. Gated by the new `ui.render` scope. The same payload
+  renders on web, native iOS/Android, or a CLI — reference renderers in
+  `examples/web-instant-ui`, `examples/native-instant-ui` (SwiftUI + Jetpack
+  Compose). New types `RenderInterfacePayload` / `InterfaceUpdatePayload`. See
+  `docs/companion-instant-ui.md`.
+- **Cross-app A2A.** `companion.social_message` (`onSocialMessage`) delivers an
+  inbound A2A friend message to any embed whose token holds `social.message`, not
+  just the first-party app. New type `SocialMessagePayload`.
+- **Confirmation observation + biometric cue.** `onConfirmRequest`
+  (`companion.confirm_request`) lets an embed surface a pending sensitive-op
+  approval (observe-only; approval stays first-party). The payload now carries an
+  advisory `stepUp` flag (true for money ops) so a first-party surface can gate
+  approval behind a passkey / Face ID. New type `ConfirmRequestPayload`.
+- **Typed event helpers** for the remaining outbound events: `onAudio`
+  (`companion.audio`), `onExpression` (`companion.expression`), `onUsage`
+  (`control.usage`). New types `AudioClipPayload` / `ExpressionPayload` /
+  `UsagePayload`.
+- **Capabilities map** doc (`docs/companion-capabilities.md`) — the six
+  capabilities, their SDK surface, and the beyond-the-sandbox model.
+
+### Protocol
+
+- Outbound types added: `companion.ui_action`* / `companion.ui_update` /
+  `companion.social_message`. (*`ui_action` was reserved previously; now emitted.)
+  Non-breaking — receivers ignore unknown types; `PROTOCOL_VERSION` unchanged (`1`).
+
+## [0.9.0] - 2026-06-23
+
+First publicly published release (public npm, `publishConfig.access: "public"`,
+proprietary license — see `LICENSE`). No code changes versus the previously
+self-hosted/CDN build; this entry captures the surface that ships publicly.
+
+### Added
+
+- **Public distribution.** Installable from the public npm registry as
+  `@pouchy_ai/companion-sdk`; also resolvable via `esm.sh` / `jsDelivr` for
+  no-bundler / import-map consumers, and self-hosted at
+  `https://www.pouchy.ai/sdk/companion-sdk.js`.
+- **Chat.** Text and multimodal (text + images) messaging.
+- **Voice.** Live WebRTC call (`connectCall`), provider-agnostic across OpenAI
+  Realtime (zero extra deps) and ElevenLabs Convai (optional
+  `@elevenlabs/client` peer dependency).
+- **Tools.** Bidirectional tool calling — the companion invokes app-declared
+  actions and the app reports results back.
+- **Memory.** Scope-gated read/write into the user's recall namespace.
+- **Live world-state.** CloudEvents-shaped "play-along" channel
+  (`sendWorldState`) for streaming contextual data to the companion.
+- **Representative mode (代聊).** Visitor-facing sessions answered on the owner's
+  behalf using only screened context (`represent` scope + optional
+  `expose:*` / `represent:*` scopes).
+- **Avatar.** VRM (3D) and 2D portrait retrieval.
+- **Transports.** REST + Server-Sent Events by default; optional WebSocket
+  (`stream: 'websocket'`) with automatic SSE fallback.
+- **Auth.** Personal Access Tokens (`pchy_…`) and "Login with Pouchy" OAuth 2.1
+  (Authorization Code + PKCE, rotating refresh tokens).
+- **Versioned wire protocol.** `PROTOCOL_VERSION = 1`, kept in lockstep with the
+  server by `protocol.drift.test.ts` (fails CI on divergence).
+
+[Unreleased]: https://github.com/oviswang/Pouchy/compare/companion-sdk-v0.9.0...HEAD
+[0.9.0]: https://github.com/oviswang/Pouchy/releases/tag/companion-sdk-v0.9.0

package/LICENSE

@@ -0,0 +1,41 @@
+Pouchy Companion SDK License
+Copyright (c) 2026 Pouchy.ai. All Rights Reserved.
+
+This license governs use of the "@pouchy_ai/companion-sdk" software package (the
+"SDK") as published to the npm registry. The SDK is proprietary software owned
+by Pouchy.ai. By installing, copying, or using the SDK you agree to these terms.
+
+1. GRANT. Subject to these terms, Pouchy.ai grants you a worldwide,
+   non-exclusive, non-transferable, royalty-free, revocable license to install
+   and use the SDK, as distributed, solely to build and operate applications
+   that integrate with Pouchy.ai's official services and APIs.
+
+2. RESTRICTIONS. Except to the extent a restriction below is prohibited by
+   applicable law, you may not:
+   (a) modify, adapt, or create derivative works of the SDK's source for
+       redistribution;
+   (b) redistribute, sublicense, sell, rent, or lease the SDK as a standalone
+       product, or republish it under a different name;
+   (c) reverse engineer, decompile, or disassemble the SDK except to the extent
+       necessary for interoperability and expressly permitted by law;
+   (d) use the SDK to build or operate a service that competes with Pouchy.ai's
+       services, or to access Pouchy.ai's services in violation of their terms;
+   (e) remove or alter any copyright, trademark, or other proprietary notices.
+
+3. RESERVATION OF RIGHTS. All rights not expressly granted are reserved by
+   Pouchy.ai. No rights are granted in Pouchy.ai's source repositories,
+   trademarks, or services beyond what is needed to use the SDK as described.
+
+4. TERMINATION. This license terminates automatically if you breach it. On
+   termination you must stop using and delete all copies of the SDK.
+
+5. NO WARRANTY. THE SDK IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+   EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+   MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
+
+6. LIMITATION OF LIABILITY. IN NO EVENT SHALL POUCHY.AI BE LIABLE FOR ANY CLAIM,
+   DAMAGES, OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT, OR
+   OTHERWISE, ARISING FROM, OUT OF, OR IN CONNECTION WITH THE SDK OR THE USE OR
+   OTHER DEALINGS IN THE SDK.
+
+For any other use, or for written permission, contact legal@pouchy.ai.

package/README.md

@@ -0,0 +1,136 @@
+# @pouchy_ai/companion-sdk
+
+Embed the Pouchy companion — chat, voice, tools, memory, live world-state,
+**Instant UI**, and **agent-to-agent messaging** — in any app, game, or site.
+
+All six companion capabilities are reachable over the SDK (memory · reasoning ·
+skills · A2A/social · wallet · instant UI), and each rides a **platform-neutral**
+contract — so the same companion runs on web, native iOS/Android, or a CLI. See
+[`docs/companion-capabilities.md`](../../docs/companion-capabilities.md).
+
+## Install
+
+```sh
+npm i @pouchy_ai/companion-sdk
+```
+
+No bundler? Pull it from a CDN via an import map instead — see
+[`docs/companion-integration-faq.md`](../../docs/companion-integration-faq.md).
+Changes per release are tracked in [`CHANGELOG.md`](./CHANGELOG.md).
+
+```ts
+import { createCompanion } from '@pouchy_ai/companion-sdk';
+
+const companion = createCompanion({ baseUrl: 'https://www.pouchy.ai', token: PAT });
+await companion.connect();
+companion.onMessage((t) => console.log(t));
+companion.start();
+await companion.sendText('hey');
+await companion.connectCall();              // live voice (browser)
+companion.sendWorldState({ type: 'game.player.hp', data: { hp: 12 }, retained: true });
+```
+
+- **Guide:** [`docs/companion-quickstart.md`](../../docs/companion-quickstart.md)
+- **Spec:** [`docs/companion-sdk-protocol.md`](../../docs/companion-sdk-protocol.md)
+- **API reference:** [`docs/companion-api-reference.md`](../../docs/companion-api-reference.md)
+- **Capabilities map:** [`docs/companion-capabilities.md`](../../docs/companion-capabilities.md)
+- **Instant UI renderer contract:** [`docs/companion-instant-ui.md`](../../docs/companion-instant-ui.md)
+
+## Events
+
+Subscribe with `on(type, fn)` (or `'*'`), or these typed convenience helpers:
+
+| Helper | Event | Use |
+| --- | --- | --- |
+| `onMessage(fn)` | `companion.message` | streamed assistant text |
+| `onToolCall(fn)` | `companion.tool_call` | the companion asks your app to run a declared tool → `sendToolResult` |
+| `onRender(fn)` | `companion.ui_action` | **Instant UI** — draw `payload.interface` (platform-neutral genui schema) with your own renderer (web / iOS / Android / CLI). Needs `ui.render` |
+| `onInterfaceUpdate(fn)` | `companion.ui_update` | live `{key,value}` update to an already-rendered panel (no rebuild) |
+| `onSocialMessage(fn)` | `companion.social_message` | inbound A2A friend message, delivered cross-app. Needs `social.message` |
+| `onConfirmRequest(fn)` | `companion.confirm_request` | observe a sensitive-op approval (`stepUp:true` ⇒ the first-party surface should gate it with a biometric). Observe-only — approval is first-party |
+| `onAudio(fn)` | `companion.audio` | TTS clip (non-call modality) |
+| `onExpression(fn)` | `companion.expression` | avatar viseme / expression / gesture |
+| `onUsage(fn)` | `control.usage` | per-token metering echo |
+| `onError(fn)` | `control.error` | agent / stream errors |
+
+### Instant UI
+
+```ts
+import { renderInterface } from './instant-ui'; // your renderer (web example in examples/web-instant-ui)
+let panel: ReturnType<typeof renderInterface> | null = null;
+companion.onRender(({ interface: ui }) => { panel = renderInterface(ui, mountEl); });
+companion.onInterfaceUpdate(({ update }) => panel?.applyUpdate(update));
+```
+
+Reference renderers ship in [`examples/`](../../examples): `web-instant-ui` (vanilla),
+`native-instant-ui` (SwiftUI + Jetpack Compose), and `cli-skills` (terminal +
+local/device skills).
+
+## Representative mode (代聊)
+
+Pass a `visitor` and the session flips from **owner-facing** to **representative**:
+the companion answers *that visitor* **on the owner's behalf** (代聊 /
+customer-service) using only **screened** owner context — never the owner's private
+memory, system prompt, or PII. Works over text and voice.
+
+```ts
+const c = createCompanion({
+  baseUrl: 'https://www.pouchy.ai',
+  token: OWNER_PAT,                 // must hold the `represent` scope
+  surface: 'support-widget',
+  appContext: { name: 'AcmeShop', description: 'Order support' },
+  visitor: { id: stableVisitorId, displayName: 'Sam' }  // stable per end-user
+});
+const ack = await c.connect();      // → { representative: true, visitorPaired, … }
+await c.sendText('do you ship to Canada?');
+const call = await c.connectCall(); // same, over voice
+```
+
+Scopes (granted by the owner's **"Let it represent you to visitors"** key toggle):
+
+| Scope | Adds |
+| --- | --- |
+| `represent` | **required** — open a visitor-facing session |
+| `expose:knowledge` | answer from the owner's knowledge base ("My materials") |
+| `expose:facts` | share a wider set of screened facts |
+| `represent:remember` | durable per-visitor notes across visits (isolated store) |
+| `represent:pair` | `c.pairVisitor(visitorPAT)` — pair a visitor who's also a Pouchy user → unlock A2A |
+
+Without `represent`, supplying a `visitor` is rejected (`403`). First-party tools
+(wallet / skills / social) are withheld; your own declared tools still work. Full
+walkthrough: **[API reference §3.3–3.4](../../docs/companion-api-reference.md)** and
+the [source README](../../src/lib/companion-sdk/README.md#representative-mode-代聊).
+
+## Voice (optional dependency)
+
+`connectCall()` for the **ElevenLabs Convai** provider needs the optional peer
+dependency:
+
+```sh
+npm i @elevenlabs/client
+```
+
+OpenAI Realtime needs no extra dependency (self-contained WebRTC).
+
+## Build & publish
+
+The source lives in the app repo at `src/lib/companion-sdk` and is decoupled from
+any `$lib` import, so this package builds with plain `tsc`:
+
+```sh
+cd packages/companion-sdk
+npm run build      # tsc → dist/ (ESM + .d.ts)
+npm publish        # public scope (publishConfig.access: "public"); runs build via prepublishOnly
+```
+
+The wire contract (`protocol.ts`) is a self-contained mirror of the server's;
+`protocol.drift.test.ts` (run by the app's vitest) fails CI if they diverge.
+
+Full release steps (version bump, `CHANGELOG.md`, drift test, tag) live in
+[`PUBLISHING.md`](./PUBLISHING.md).
+
+## License
+
+Proprietary — © 2026 Pouchy.ai. The published package may be installed and used
+to build integrations with Pouchy's official services; all other rights are
+reserved. See [`LICENSE`](./LICENSE).

package/dist/call.d.ts

@@ -0,0 +1,59 @@
+import type { CallCredentials } from './client.js';
+/** Lets the live voice call invoke the app's declared tools (pause_game,
+ *  set_feature, …). The companion's voice LLM calls a tool → invoke() runs the
+ *  app's onToolCall handler and resolves with a result string fed back to the
+ *  model. Wired by connectCall from the tools declared at createCompanion(). */
+export interface VoiceToolBridge {
+    tools: {
+        name: string;
+        description?: string;
+        parameters?: Record<string, unknown>;
+    }[];
+    invoke: (name: string, argsJson: string) => Promise<string>;
+}
+/** Universal "host control" verbs the companion can call on ANY surface (web /
+ *  game / app / hardware), regardless of provider. They're generic on purpose —
+ *  the SPECIFIC intent rides in the string params, which YOUR surface interprets
+ *  — so a single fixed schema set serves every embed AND can be pre-registered on
+ *  the (otherwise static) ElevenLabs Convai agent. `invoke_action` is the
+ *  catch-all trigger; the SDK unwraps it to the named action so it reaches your
+ *  onToolCall just like a declared tool. The others pass through by name. */
+export declare const HOST_CONTROL_TOOLS: VoiceToolBridge['tools'];
+/** Names of the host-control verbs (for routing). */
+export declare const HOST_CONTROL_TOOL_NAMES: readonly string[];
+/** Avatar gesture/expression tools the embodied ("形象") companion calls to emote
+ *  — drive a VRM/3D body's gestures and facial expression. These are offered on
+ *  EVERY embed (like the host verbs) so the companion can express itself without
+ *  each third party re-declaring them, and — crucially — so the SDK can register
+ *  a no-op handler for them on the ElevenLabs Convai path, where the shared agent
+ *  is pre-configured to call play_gesture / play_expression regardless of what the
+ *  embed declares. An integrator that actually renders an avatar opts in by
+ *  declaring (or handling) the same tool name and the SDK routes the call to them;
+ *  otherwise it answers with a silent no-op success (see invokeVoiceTool / emit),
+ *  so a pure audio or 2D embed can ignore avatars entirely and the companion's
+ *  emote calls never error. Enum values mirror the canonical GESTURE_CATEGORIES /
+ *  EXPRESSION_INTENTS the avatar runtime understands (kept inline so the SDK stays
+ *  self-contained). */
+export declare const AVATAR_VISUAL_TOOLS: VoiceToolBridge['tools'];
+/** Names of the avatar visual tools (for routing + the no-op fallback). */
+export declare const AVATAR_VISUAL_TOOL_NAMES: readonly string[];
+export interface CompanionCallOptions {
+    /** Element to route remote audio to. One is created if omitted. */
+    audioElement?: HTMLAudioElement;
+    onTranscript?: (e: {
+        role: 'user' | 'assistant';
+        text: string;
+    }) => void;
+    onSpeakingChange?: (speaking: boolean) => void;
+    onError?: (err: Error) => void;
+}
+export interface CompanionCall {
+    /** Hang up. Idempotent. */
+    close: () => void;
+    /** Inject an out-of-band context line. `speak: true` makes the companion react
+     *  now; false stages it for the next turn. */
+    injectEvent: (text: string, speak?: boolean) => void;
+    provider: CallCredentials['provider'];
+}
+/** Open a live call from pre-minted credentials. */
+export declare function openCompanionCall(creds: CallCredentials, opts?: CompanionCallOptions, bridge?: VoiceToolBridge): Promise<CompanionCall>;