humanish 0.101.0 → 0.103.0

package/docs/goals/current.md

@@ -1,9 +1,10 @@
 # Current Goals
 
-Status date: 2026-09-25. Release baseline: `0.101.0`.
+Status date: 2026-09-26. Source baseline: `0.103.0`.
 
-This page guides work on current merged source. Published behavior is described
-in the [release notes](../release/0.101.0-codex-ui-tools.md).
+This page guides work on current source. Version `0.103.0` adds optional desktop
+video and audio; its [release note](../release/0.103.0-desktop-recording.md)
+records the supported routes, playback behavior and local-only evidence limits.
 The [September 9 history](https://github.com/danielgwilson/humanish/blob/main/docs/goals/current-history-2026-09-09.md)
 preserves the former status log; its queues do not supersede this page.
 
@@ -88,9 +89,9 @@ requires decision-equivalent retained evidence and a real deletion branch.
 No first-party deletion branch has met that gate. Public demonstrations do not
 substitute for it.
 
-## Current Program Truth (source `0.101.0`)
+## Current Program Truth (source `0.103.0`)
 
-| Surface | Available in merged source | Remaining boundary |
+| Surface | Available in source | Remaining boundary |
 | --- | --- | --- |
 | Study authoring | YAML labs, lane/roster composition, model settings and study/per-lane caps | Route support varies; declarations are not promises of engine parity |
 | Actors | Seven first-party descriptors; computer-use, scripted-browser and terminal-product dispatch | No supported public out-of-tree actor-registration API |
@@ -102,8 +103,8 @@ substitute for it.
 | Study findings | Default post-run analysis on supported live routes with a separate disclosed $3 admission estimate limit and opt-out; explicit `analyze`, fairer evidence selection, concern review and versioned findings with exact source links; explicit restricted Codex account analysis on qualified Linux x64 and Apple Silicon Mac profiles | Account dollars/output-token caps are unavailable; Keychain/other CLI profiles are unqualified. Model interpretation needs review; selection limits coverage; opening Observer never dispatches analysis |
 | TUI and serving | Detached starts, run stopping, reclamation, Observer attachment, loopback serving, run library and AgentMail setup, authentication and lab configuration | Stopping a process does not itself prove sandbox cleanup; TUI views over CLI `stats`/`export` remain follow-ups |
 | Off-app communication | Recipient-scoped local capture and fresh real AgentMail receiving, supported inline raster images, bounded collection and host-owned recovery | Real mail uses isolated participant surfaces and remains local-only for publication. Hosted mail/model processing, bounded fidelity and interrupted-run recovery are explicit; local-agent, borrowed inboxes and SMS are unsupported |
-| Local browsers | Shared study loop on Linux x64/Docker/KVM and M3-or-newer Mac/Lima; pinned runtime download, Codex account participants and automatic analysis | Browser-only, explicit local labs; local inbox/media and automatic onboarding remain follow-ups; larger cohorts and sleep/wake are unqualified |
-| Mobile and media | Hosted viewport/emulation, desktop geometry checks, bounded dwell and declared camera feed; a synthetic video-only call with separate hosted peers is proven | Audio, TURN, provider-specific rooms, physical-device and touch fidelity remain unproven; unsupported media declarations are rejected |
+| Local browsers | Shared study loop on Linux x64/Docker/KVM and M3-or-newer Mac/Lima; pinned runtime download, Codex account participants and automatic analysis | Explicit local labs; local inbox and automatic onboarding remain follow-ups; larger cohorts and sleep/wake are unqualified |
+| Mobile and media | Hosted viewport/emulation, desktop geometry checks, bounded dwell and declared camera feed; optional camera and speech passed two-person Linux and Mac conversations with validated analysis; hosted audio-only uses the same worker | Hosted camera+speech is unsupported; English recognition and robotic voice have fidelity limits; raw AV recording, forced TURN, physical-device and touch fidelity are unproven |
 
 Use the [task support matrix](../architecture/task-protocol-support.md),
 [actor registry](https://github.com/danielgwilson/humanish/blob/main/src/actor-registry.ts)
@@ -123,8 +124,11 @@ ordinary TAP/NAT networking. Continue managed-local work from this complete stud
 path; the earlier offline owner/service qualification experiments are historical
 fixtures, not an installation architecture or a prerequisite queue. Explicit
 Linux local labs now use the installed CLI/TUI, with a verified runtime download
-before the first live run. Mac support is also shipped; inbox integration and
-optional media remain unfinished. Existing hosted labs retain their behavior.
+before the first live run. Mac browser support is also shipped; inbox integration remains unfinished.
+Optional media passed Linux and Mac conversations with receiving-side evidence
+and owned-resource cleanup. The original Mac recording produced validated
+analysis after its citation instructions were clarified. Existing hosted labs
+retain their behavior.
 
 ## Gates And Deferred Work
 

package/docs/ramp/README.md

@@ -2,7 +2,7 @@
 
 Status: public-safe contributor and agent ramp.
 
-Package/source version in this tree: `0.101.0` (2026-09-25). The Observer is phone-usable as a stated requirement (observer/AGENTS.md); interactive primitives start from Base UI. The Observer renderer is the observer/ workspace artifact only; the legacy string-concat renderer was deleted at cutover (#426), and rollback is a version pin to 0.42.0. The containment boundary introduced in
+Package/source version in this tree: `0.103.0` (2026-09-26). The Observer is phone-usable as a stated requirement (observer/AGENTS.md); interactive primitives start from Base UI. The Observer renderer is the observer/ workspace artifact only; the legacy string-concat renderer was deleted at cutover (#426), and rollback is a version pin to `0.42.0`. The containment boundary introduced in
 `0.15.1` remains in force: managed run and output paths bind to validated
 physical filesystem identities, and stored provider IDs are evidence, not
 cleanup authority. The bundled OSS meta-lab is dry-run only until
@@ -54,6 +54,16 @@ If a change does not improve one of those loops, it probably belongs elsewhere.
 
 ## Current State
 
+The [0.103.0 release note](../release/0.103.0-desktop-recording.md) describes
+optional desktop video and audio, playback on the shared Observer clock, and
+local-only media evidence. Screenshots remain the default.
+
+The [0.102.0 release note](../release/0.102.0-participant-media.md) describes
+optional native camera and speech. Linux and Mac conversations passed, including
+spoken clarification of a misheard answer. The retained Mac recording also
+produced a validated report after the analysis citation instructions were
+clarified; the validator and original evidence remain unchanged.
+
 The [0.101.0 release note](../release/0.101.0-codex-ui-tools.md) describes native
 Codex UI tools shared by local Firecracker and hosted E2B participants.
 

package/docs/release/0.102.0-participant-media.md

@@ -0,0 +1,43 @@
+# 0.102.0 — Optional participant camera and speech
+
+Codex participants can listen and speak while using a conferencing app. The same
+continuing conversation handles browser actions and spoken replies on local
+Firecracker and hosted E2B desktops. No separate conversational agent or speech
+API key is required. Codex inference still uses the selected remote account.
+
+Declare `execution.desktop.media.microphone.source: speech` for speech, and
+`camera.source: synthetic` for a changing test-pattern camera. Local desktops
+support both through native browser devices and ordinary permissions. Humanish
+downloads the optional media runtime when needed; `humanish runtime setup --media`
+can prepare it ahead of a study. Browser-only studies retain the smaller runtime
+and start no media processes.
+
+Hosted speech automatically selects a public E2B template and uses the same
+worker. E2B's existing fake-camera flags also replace its microphone, so hosted
+camera and speech cannot be combined. Explicit compatible template overrides
+remain supported. Hosted speech requires `@e2b/desktop` 2.3.2 or newer for the
+background command handle's streaming stdin methods.
+
+Recordings and analysis retain spoken text and transcriptions from actual speaker
+audio under the existing redaction rules. Observer links findings to those
+entries and nearby screenshots. Raw audio/video is not recorded. The initial
+voice is robotic, recognition is English, and clarification may be necessary;
+this is turn-taking rather than interruptible realtime voice conversation.
+
+Linux acceptance includes two concurrent participants entering a real conference
+through different application roles, receiving counterpart video, and exchanging
+previously undisclosed information by voice. Automatic analysis and owned-resource
+cleanup completed. The Whisper `base.en` runtime also passed a Mac
+conversation, including reciprocal video, a fresh fact and question, and spoken
+correction and confirmation of a misheard answer. Recognition still makes
+meaningful errors; this pass depended on clarification. Analysis of the retained
+Mac recording validated both participant reviews and a shared finding about the
+corrected misunderstanding, with all selected captures and earlier attempts
+preserved.
+
+Analysis now retains specific, safe validation failure codes. Its citation
+instructions distinguish each participant's own evidence from cross-participant
+observations in findings. The evidence validator remains unchanged.
+
+See [participant media](../architecture/participant-media.md) for configuration and
+the [hosted template recipe](../../runtime/browser-media/E2B.md) for maintenance.

package/docs/release/0.103.0-desktop-recording.md

@@ -0,0 +1,29 @@
+# 0.103.0 — Optional desktop video and audio
+
+Computer-use studies can retain an MP4 alongside screenshots, actions,
+participant feedback and analysis:
+
+```yaml
+execution:
+  desktop:
+    recording: { audio: true }
+```
+
+Set `audio: false` for screen-only video. Omit `recording` for the existing
+lightweight captures. Recording does not enable camera or speech features.
+
+Observer plays the selected participant's video on the existing study timeline.
+Opening a participant, scrubbing and returning to the grid preserve the cursor.
+Audio starts muted; the selected participant has an explicit audio control.
+Unavailable video falls back to screenshots where they exist.
+
+Local Firecracker and hosted E2B use the same capture settings and evidence
+format. Local capture uses the optional media runtime archive. Recorder failure
+preserves the study; a playable interrupted or size-limited recording is marked
+partial. The per-participant file limit is 128 MiB.
+
+Continuous media stays local-only. Screenshot redaction does not redact video
+or audio. Portable HTML exports explicitly omit continuous media; analysis
+continues to inspect screenshots and text. See the
+[recording guide](../architecture/desktop-recording.md) for supported routes,
+capture semantics and export behavior.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "humanish",
-  "version": "0.101.0",
+  "version": "0.103.0",
   "description": "Open-source-safe CLI for persona simulation, observer review, and public-safe feedback drafts.",
   "author": "Daniel G Wilson <daniel@danielgwilson.com>",
   "keywords": [
@@ -50,7 +50,7 @@
   },
   "scripts": {
     "build": "tsc -p tsconfig.build.json && pnpm --filter humanish-observer build && pnpm --filter humanish-tui build && node scripts/finish-build.mjs",
-    "check": "pnpm typecheck && pnpm test && pnpm tui:test && pnpm build && pnpm cli:startup:test && pnpm cli:preflight:test && pnpm tui:smoke",
+    "check": "pnpm typecheck && pnpm test && pnpm tui:test && pnpm build && pnpm guest-media:startup:proof && pnpm cli:startup:test && pnpm cli:preflight:test && pnpm tui:smoke",
     "cli:startup:test": "node scripts/terminal-startup-exit-proof.mjs",
     "cli:preflight:test": "node scripts/task-route-preflight-proof.mjs",
     "humanish": "tsx src/cli.ts",
@@ -84,7 +84,8 @@
     "observer:chrome:proof": "node scripts/observer-chrome-proof.mjs",
     "observer:reliability:proof": "node scripts/observer-reliability-proof.mjs",
     "tui:connections:proof": "python3 scripts/tui-connections-proof.py",
-    "guest-desktop:proof": "node scripts/guest-desktop-proof.mjs"
+    "guest-desktop:proof": "node scripts/guest-desktop-proof.mjs",
+    "guest-media:startup:proof": "node scripts/guest-media-worker-startup-proof.mjs"
   },
   "repository": {
     "type": "git",
@@ -105,7 +106,7 @@
   },
   "peerDependencies": {
     "@anthropic-ai/claude-agent-sdk": "^0.3.0",
-    "@e2b/desktop": "^2.2.3"
+    "@e2b/desktop": "^2.3.2"
   },
   "peerDependenciesMeta": {
     "@anthropic-ai/claude-agent-sdk": {

package/skills/humanish/SKILL.md

@@ -115,8 +115,8 @@ request. Inspect a failed attempt before explicitly retrying `--provider codex
 
 ## Local browser setup
 
-On Linux x64 with a local rootful Docker Engine, KVM and TUN, an `app-url` lab can
-set `execution.target: local` and `actors[0].type: local-agent` with
+On Linux x64 with local rootful Docker, KVM and TUN, or a supported M3-or-newer
+Mac with native ARM64 Node and Lima 2.2+, an `app-url` lab can set `execution.target: local` and `actors[0].type: local-agent` with
 `localAgent: codex`. It uses the supported Codex ChatGPT login, not E2B or an
 OpenAI API key. Inference is remote and consumes account quota. Existing hosted
 labs stay hosted; never silently change their execution or billing provider.
@@ -128,9 +128,18 @@ the pinned runtime; a live local run also prepares it automatically. The normal
 See [the complete example and limits](../../docs/architecture/local-browser-runtime.md).
 
 Local browsers currently require a loopback app URL with an explicit port above
-1023, use a 960×720 Chromium desktop, and reject inbox/media declarations. Mac
-setup is not integrated. Do not claim that installing the CLI also installs
-Docker or makes these prerequisites available.
+1023 and use a 960×720 Chromium desktop. Local inbox declarations remain
+unsupported. Optional camera and spoken conversation require a separate media
+runtime; follow [participant media](../../docs/architecture/participant-media.md)
+for setup and provider limits. Speech uses the same continuing Codex participant,
+not a second conversation. Do not claim that installing the CLI also installs
+Docker or Lima, or qualifies a machine for arbitrary participant counts.
+
+For continuous desktop playback, independent computer-use lanes can opt into
+`execution.desktop.recording: { audio: true }` (`false` for screen-only video).
+Screenshots remain the default. Recording does not enable camera or speech;
+the raw media stays local-only. See [desktop recording](../../docs/architecture/desktop-recording.md)
+for runtime requirements and export limits.
 
 ## Format Stack