@clowder-ai/companion 0.1.0-alpha.3 → 0.1.0-alpha.30

package/NOTICE

@@ -1,7 +1,12 @@
-Companion imagery and theme tokens originate in Clowder AI (MIT).
-See source-lock.json for the exact public source revision and content hashes.
+The original companion atlases and theme tokens originate in Clowder AI (MIT).
+See source-lock.json for their exact public source revision and content hashes.
 The legacy ragdoll-v1 and yarn-ball stills are AI-assisted derivatives of the
 listed public reference, restoring the complete cat silhouette and transparent
 background. Their reference and resulting hashes are recorded separately.
+The living Xianxian layers and clips are F317 assets: CVO-generated source video,
+keyframes by codex-astra, and deterministic extraction/alignment by opus55.
+source-lock.json pins the 20 bundled distributable files, their geometry file,
+and the source package manifest digest. Source video and private workspace paths
+are not bundled.
 The conversation transport and controls continue the F317 companion implementation
 by the Clowder AI contributors. No private runtime data or pet metadata is included.

package/README.md

@@ -10,22 +10,71 @@ visible and ends the call in one click. Screen sharing has its own explicit
 picker and termination badge. There is no microphone action on a double-click,
 page load, body tap or drag.
 
+When a compatible Host advertises receive-only audio, **只听** opens the same
+conversation with playback but never requests or toggles a microphone. Older
+Hosts omit that capability, so the control stays hidden and ordinary voice keeps
+the exact legacy connect request. Text remains available independently.
+
 Typing works with voice off. The Host uses the existing owner conversation,
 configured duty cat, ordinary message routing and idempotency. The small history
 view reads a bounded recent subset; **完整聊天** opens the canonical conversation.
 No second transcript database, credentials or selectable Host identity live here.
+When the Host supplies a valid per-message identity snapshot, each history row
+keeps its real author while separately showing the partner, Live carrier and
+deep cat saved with that message. Legacy rows omit this context instead of
+borrowing today's selected companion.
 
 ## Host integration
 
-This surface requires the additive desktop bridge **1.1.0**. The Host computes
+This surface requires the additive desktop bridge **1.6.0**. The Host computes
 placement within the display work area, keeps the cat as its stable anchor, and
-owns bounded drag gestures and transparent-area hit testing. Older 1.0-only
-Hosts reject this manifest before installation. Media admission and capture
+owns bounded drag gestures and transparent-area hit testing. Hosts below 1.6
+reject this manifest before installation. Media admission and capture
 remain in the trusted Host; this package receives neither streams nor provider
 SDP. Hiding releases media; the existing Clowder **聊聊** entry restores the cat.
+Bridge 1.4 adds one untrusted first-visible-subtitle observation containing only
+an epoch-millisecond time, the current call fence and a Host-correlatable source
+reference. It does not claim compositor paint, an unoccluded window or reading.
+Message and native-item references use the same bounded, single-line opaque ID
+contract as the transcript surfaces that supply them. A Host
+`invalid_request` rejects and remembers that candidate for the current call so
+another current reference may be observed without retrying the rejected one.
+Other failures retry after 1 and 2 seconds, then leave the call's
+first-visible result unknown; later text never backfills a manufactured success.
+Bridge 1.5 adds Host-bound conversation pages and versioned body continuations.
+The renderer can only return opaque cursors issued by the Host; it cannot select
+an owner, thread, message, call or byte offset. History progress, response phase
+and body completeness remain separate, so a processing draft or truncated body
+cannot masquerade as a completed reply.
+
+It also binds the one-line composer to the Host-owned durable draft. Versioned
+text submission carries the Host-issued conversation `scopeRef`, draft revision
+and caller message identity. An unknown settlement is read back but never
+automatically resent; an explicit retry keeps the original text, identity,
+revision and route. If the canonical thread changes, the old unconfirmed input
+stays visible as invalidated local text and is never replayed into the new scope.
+The legacy text and conversation methods remain unchanged for older pairs.
+
+Bridge 1.6 separates passive call speech from the bounded nameplate and control
+panel, and adds an optional application-assistance projection. Assistance is
+off by default; settling a prompt only opens the ordinary menu and grants no
+media, screen-selection or call authority.
+
+Xianxian uses one layered sitting body or one VP9 clip at a time. Expiring Host
+facts with task, result and event identities select tool work, workspace fetch,
+screen reading and applied-result delivery. Native travel is independent: the
+renderer observes the actual desktop window coordinates, so a running clip can
+appear only after the Host window really moves. Dragging suppresses that
+observer and freezes a visibly held candidate instead of renaming the running
+clip. Pounce requires an explicit play gesture by default; optional automatic
+play is bounded by idle and cooldown timers. Reduced motion keeps semantic
+poses static; a compatible Host also revokes its native movement lease.
+Other cats retain their own skins. The v3 living assets and their hashes are
+pinned in `source-lock.json`; the installable renderer carries all 21 assets
+itself, including the mid-edge `peek_fade` variant.
 
-The original animated skin bytes remain unchanged. Public static skin derivatives
-and the shared color tokens retain their exact provenance in `source-lock.json`.
+The Host keeps a small transparent margin left of the pet hit target so the
+pounce's tail is visible without turning empty space into a click target.
 
 ## Verification
 

package/manifest.json

@@ -1,6 +1,6 @@
 {
   "pluginId": "official.companion",
-  "version": "0.1.0-alpha.3",
+  "version": "0.1.0-alpha.30",
   "contractVersion": "0.1.0",
   "name": "猫猫球",
   "icon": {
@@ -37,10 +37,10 @@
       "type": "desktop-window",
       "id": "companion",
       "role": "companion",
-      "bridgeVersion": "1.1.0",
+      "bridgeVersion": "1.6.0",
       "surface": {
         "entrypoint": "renderer/index.html",
-        "integrity": "sha256-xYeE8V8E5IA/XsLIkbLATsVze+7Fea6eGkjJIdUTKzM="
+        "integrity": "sha256-dR1cqLYEPsY28TJ6ureljiV5CP23wubn70G7pePPkNg="
       },
       "presentation": {
         "width": 300,

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@clowder-ai/companion",
-  "version": "0.1.0-alpha.3",
+  "version": "0.1.0-alpha.30",
   "description": "The desktop voice body for your Clowder companion",
   "license": "MIT",
   "type": "module",
@@ -21,13 +21,15 @@
   "scripts": {
     "build": "node scripts/build.mjs",
     "test": "node --test test/*.test.mjs",
+    "test:browser": "pnpm build && node --test test/*.browser.mjs",
     "typecheck": "node scripts/check.mjs",
     "lint": "node scripts/check.mjs",
     "prepack": "pnpm build"
   },
   "devDependencies": {
-    "@clowder-ai/plugin-contract": "0.1.0-beta.17",
-    "@clowder-ai/plugin-sdk": "0.1.0-beta.13"
+    "@clowder-ai/plugin-contract": "0.1.0-beta.29",
+    "@clowder-ai/plugin-sdk": "0.1.0-beta.25",
+    "playwright": "1.61.1"
   },
   "repository": {
     "type": "git",

package/plugin.yaml

@@ -1,6 +1,6 @@
 {
   "pluginId": "official.companion",
-  "version": "0.1.0-alpha.3",
+  "version": "0.1.0-alpha.30",
   "contractVersion": "0.1.0",
   "name": "猫猫球",
   "icon": {
@@ -37,10 +37,10 @@
       "type": "desktop-window",
       "id": "companion",
       "role": "companion",
-      "bridgeVersion": "1.1.0",
+      "bridgeVersion": "1.6.0",
       "surface": {
         "entrypoint": "renderer/index.html",
-        "integrity": "sha256-xYeE8V8E5IA/XsLIkbLATsVze+7Fea6eGkjJIdUTKzM="
+        "integrity": "sha256-dR1cqLYEPsY28TJ6ureljiV5CP23wubn70G7pePPkNg="
       },
       "presentation": {
         "width": 300,

package/renderer/app-assist-controller.mjs

@@ -0,0 +1,125 @@
+const ACTIONS = new Set(['dismiss', 'timeout', 'accept']);
+
+/**
+ * Renderer projection only. The Host owns offer creation, expiry and clearing;
+ * this controller merely keeps stale reads and replies from reviving UI or
+ * turning an old accept into a new action.
+ */
+export class AppAssistController {
+  constructor({
+    client,
+    changed = () => {},
+    accepted = () => {},
+    scheduleTimeout = setTimeout,
+    cancelTimeout = clearTimeout,
+    timeoutMs = 15_000,
+  }) {
+    this.client = client;
+    this.changed = changed;
+    this.accepted = accepted;
+    this.scheduleTimeout = scheduleTimeout;
+    this.cancelTimeout = cancelTimeout;
+    this.timeoutMs = timeoutMs;
+    this.operation = 0;
+    this.timer = null;
+    this.timerOfferRef = null;
+    this.state = {
+      scopeRef: null,
+      revision: -1,
+      status: null,
+      offer: null,
+      hiddenRef: null,
+      pending: null,
+    };
+  }
+
+  publish(patch) {
+    this.state = { ...this.state, ...patch };
+    this.changed(this.state);
+  }
+
+  clearTimer() {
+    if (this.timer !== null) this.cancelTimeout(this.timer);
+    this.timer = null;
+    this.timerOfferRef = null;
+  }
+
+  armTimer(offerRef) {
+    if (this.timer !== null && this.timerOfferRef === offerRef) return;
+    this.clearTimer();
+    this.timerOfferRef = offerRef;
+    this.timer = this.scheduleTimeout(() => {
+      this.timer = null;
+      this.timerOfferRef = null;
+      if (this.state.offer?.offerRef === offerRef) void this.settle('timeout');
+    }, this.timeoutMs);
+    this.timer?.unref?.();
+  }
+
+  consume(snapshot) {
+    if (!snapshot || typeof snapshot.scopeRef !== 'string' || !Number.isSafeInteger(snapshot.revision)) return false;
+    if (this.state.scopeRef !== null && snapshot.scopeRef !== this.state.scopeRef) return false;
+    if (snapshot.revision <= this.state.revision) return false;
+
+    const scopeRef = this.state.scopeRef ?? snapshot.scopeRef;
+    const incomingRef = snapshot.offer?.offerRef ?? null;
+    const sameHiddenOffer = incomingRef !== null && incomingRef === this.state.hiddenRef;
+    const hiddenRef = incomingRef === null || (this.state.hiddenRef && incomingRef !== this.state.hiddenRef)
+      ? null
+      : this.state.hiddenRef;
+    const offer = sameHiddenOffer ? null : snapshot.offer;
+    const pending = incomingRef && this.state.pending?.offerRef !== incomingRef ? null : this.state.pending;
+    this.publish({ scopeRef, revision: snapshot.revision, status: snapshot.status, offer, hiddenRef, pending });
+    if (offer) this.armTimer(offer.offerRef);
+    else this.clearTimer();
+    return true;
+  }
+
+  async settle(action) {
+    if (!ACTIONS.has(action) || !this.state.offer || !this.state.scopeRef) return false;
+    const pending = {
+      id: ++this.operation,
+      scopeRef: this.state.scopeRef,
+      revision: this.state.revision,
+      offerRef: this.state.offer.offerRef,
+      action,
+    };
+    this.clearTimer();
+    this.publish({ offer: null, hiddenRef: pending.offerRef, pending });
+
+    let reply;
+    try {
+      reply = await this.client.settleAppAssist(pending.scopeRef, pending.offerRef, action);
+    } catch {
+      if (this.state.pending?.id === pending.id) this.publish({ pending: null });
+      return false;
+    }
+    if (this.state.pending?.id !== pending.id) return false;
+    const matching = reply?.kind === 'app-assist-settlement'
+      && reply.scopeRef === pending.scopeRef
+      && reply.offerRef === pending.offerRef
+      && reply.action === pending.action
+      && Number.isSafeInteger(reply.revision)
+      && reply.revision > pending.revision
+      && reply.revision >= this.state.revision
+      && this.state.status === 'ready';
+    if (!matching || reply.outcome !== 'settled') {
+      this.publish({ pending: null });
+      return false;
+    }
+
+    this.publish({ revision: reply.revision, hiddenRef: null, pending: null });
+    if (action === 'accept') this.accepted({
+      scopeRef: reply.scopeRef,
+      revision: reply.revision,
+      offerRef: reply.offerRef,
+    });
+    return true;
+  }
+
+  close() {
+    this.operation++;
+    this.clearTimer();
+    this.state = { ...this.state, offer: null, pending: null };
+  }
+}

package/renderer/call-controls-presentation.mjs

@@ -0,0 +1,179 @@
+/** Presentation-only state for the active call surface. Media remains owned by explicit actions. */
+export class CallControlsPresentation {
+  constructor({
+    document,
+    root,
+    pet,
+    input,
+    composers = [],
+    send = async () => false,
+    accepted = () => {},
+    changed = () => {},
+    collapseDelay = 1800,
+    schedule = (callback, delay) => setTimeout(callback, delay),
+    cancel = handle => clearTimeout(handle),
+  }) {
+    this.document = document;
+    this.root = root;
+    this.pet = pet;
+    this.input = input;
+    this.composers = composers;
+    this.send = send;
+    this.accepted = accepted;
+    this.changed = changed;
+    this.collapseDelay = collapseDelay;
+    this.schedule = schedule;
+    this.cancel = cancel;
+    this.presentation = 'lite';
+    this.panel = 'none';
+    this.active = false;
+    this.mediaLocked = false;
+    this.petHovered = false;
+    this.controlsHovered = false;
+    this.timer = undefined;
+
+    this.onPetEnter = () => {
+      this.petHovered = true;
+      if (this.active) this.reveal();
+    };
+    this.onPetLeave = () => {
+      this.petHovered = false;
+      this.armCollapse();
+    };
+    this.onControlsEnter = () => {
+      this.controlsHovered = true;
+      this.clearTimer();
+    };
+    this.onControlsLeave = () => {
+      this.controlsHovered = false;
+      this.armCollapse();
+    };
+    this.onFocusIn = () => this.clearTimer();
+    this.onFocusOut = () => queueMicrotask(() => this.armCollapse());
+    this.onInput = () => this.reevaluate();
+
+    root.dataset.presentation = this.presentation;
+    pet.addEventListener('pointerenter', this.onPetEnter);
+    pet.addEventListener('pointerleave', this.onPetLeave);
+    root.addEventListener('pointerenter', this.onControlsEnter);
+    root.addEventListener('pointerleave', this.onControlsLeave);
+    root.addEventListener('focusin', this.onFocusIn);
+    root.addEventListener('focusout', this.onFocusOut);
+    input.addEventListener('input', this.onInput);
+    for (const composer of composers) {
+      composer.submit = event => this.submit(event, composer.input);
+      composer.form.onsubmit = composer.submit;
+    }
+  }
+
+  setComposerState(identity, connecting) {
+    for (const { form, input } of this.composers) {
+      input.placeholder = identity ? `发消息给${identity.displayName}…` : '发消息给猫猫…';
+      form.querySelector('button').disabled = !identity || connecting;
+    }
+  }
+
+  async submit(event, input) {
+    event.preventDefault();
+    const text = input.value;
+    if (!await this.send(text)) return;
+    if (input.value === text) input.value = '';
+    this.reevaluate();
+    this.accepted();
+  }
+
+  setActive(active) {
+    if (this.active === active) return;
+    this.active = active;
+    this.clearTimer();
+    if (!active) {
+      this.mediaLocked = false;
+      this.setPresentation('lite');
+      return;
+    }
+    this.setPresentation(this.mediaLocked ? 'controls' : 'lite');
+  }
+
+  setPanel(panel) {
+    this.panel = panel;
+    if (panel === 'actions') this.armCollapse();
+    else this.clearTimer();
+  }
+
+  setMediaLocked(locked) {
+    this.mediaLocked = locked;
+    if (!this.active) return;
+    if (locked) this.reveal();
+    else this.armCollapse();
+  }
+
+  activate() {
+    if (!this.active) return false;
+    this.reveal();
+    return true;
+  }
+
+  escape(panel) {
+    if (!this.active || panel !== 'actions') return false;
+    this.clearTimer();
+    if (!this.mediaLocked) this.setPresentation('lite');
+    return true;
+  }
+
+  reveal() {
+    if (!this.active) return;
+    this.clearTimer();
+    this.setPresentation('controls');
+  }
+
+  reevaluate() {
+    if (this.canAutoCollapse()) this.armCollapse();
+    else this.clearTimer();
+  }
+
+  armCollapse() {
+    this.clearTimer();
+    if (!this.canAutoCollapse()) return;
+    this.timer = this.schedule(() => {
+      this.timer = undefined;
+      if (this.canAutoCollapse()) this.setPresentation('lite');
+    }, this.collapseDelay);
+  }
+
+  canAutoCollapse() {
+    return this.active
+      && this.presentation === 'controls'
+      && !this.mediaLocked
+      && !this.petHovered
+      && !this.controlsHovered
+      && !this.root.contains(this.document.activeElement)
+      && this.input.value.trim() === '';
+  }
+
+  setPresentation(next) {
+    if (this.presentation === next) return;
+    this.presentation = next;
+    this.root.dataset.presentation = next;
+    this.changed(next);
+  }
+
+  clearTimer() {
+    if (this.timer === undefined) return;
+    this.cancel(this.timer);
+    this.timer = undefined;
+  }
+
+  close() {
+    this.clearTimer();
+    this.pet.removeEventListener('pointerenter', this.onPetEnter);
+    this.pet.removeEventListener('pointerleave', this.onPetLeave);
+    this.root.removeEventListener('pointerenter', this.onControlsEnter);
+    this.root.removeEventListener('pointerleave', this.onControlsLeave);
+    this.root.removeEventListener('focusin', this.onFocusIn);
+    this.root.removeEventListener('focusout', this.onFocusOut);
+    this.input.removeEventListener('input', this.onInput);
+    for (const composer of this.composers) {
+      if (composer.form.onsubmit === composer.submit) composer.form.onsubmit = null;
+    }
+  }
+}

package/renderer/call-duration.mjs

@@ -0,0 +1,59 @@
+function formatDuration(elapsedMs) {
+  const seconds = Math.max(0, Math.floor(elapsedMs / 1000));
+  const hours = Math.floor(seconds / 3600);
+  const minutes = Math.floor((seconds % 3600) / 60);
+  const remainder = seconds % 60;
+  const clock = `${String(minutes).padStart(2, '0')}:${String(remainder).padStart(2, '0')}`;
+  return hours ? `${hours}:${clock}` : clock;
+}
+
+/** Presentation clock anchored only after the current media connection is confirmed. */
+export class CallDuration {
+  constructor(node, {
+    now = Date.now,
+    schedule = callback => {
+      const handle = setInterval(callback, 1000);
+      handle.unref?.();
+      return handle;
+    },
+    cancel = handle => clearInterval(handle),
+  } = {}) {
+    Object.assign(this, { node, now, schedule, cancel });
+    this.startedAt = null;
+    this.timer = undefined;
+  }
+
+  set(startedAt) {
+    startedAt = Number.isFinite(startedAt) ? startedAt : null;
+    if (startedAt === this.startedAt) {
+      if (startedAt !== null) this.render();
+      return;
+    }
+    this.clear();
+    this.startedAt = startedAt;
+    this.node.hidden = startedAt === null;
+    if (startedAt === null) {
+      this.node.textContent = '';
+      return;
+    }
+    this.render();
+    this.timer = this.schedule(() => this.render());
+  }
+
+  render() {
+    if (this.startedAt === null) return;
+    this.node.textContent = formatDuration(this.now() - this.startedAt);
+  }
+
+  clear() {
+    if (this.timer === undefined) return;
+    this.cancel(this.timer);
+    this.timer = undefined;
+  }
+
+  close() {
+    this.clear();
+  }
+}
+
+export { formatDuration };