@bobfrankston/msgapidefs 0.1.38 → 0.1.39

package/README.md

@@ -150,6 +150,7 @@ nothing about capability.
 | `tcp.close()` | ✅ | ✅ | ✅ | Close connection |
 | **HTTP Fetch** | | | | **`window.msgapi.http.*`** |
 | `http.fetch()` | ✅ | ✅ | ✅ | Native HTTP — bypasses CORS/mixed-content |
+| **Media capture (mic/camera)** | ❌ | ❌ | ❌ | Not an msgapi API — `getUserMedia` is the engine's. What's missing is the permission grant, which no host makes: wry's WebKitGTK backend has no `permission-request` handling (WebKitGTK denies by default), msger ignores WebView2's `PermissionRequested`, msga lacks `RECORD_AUDIO`. Deferred; direction is an external speech package, not in-page capture — see [msgapi-plan.md](./msgapi-plan.md#not-planned-recorded-so-it-doesnt-get-re-investigated). |
 | **Host Launcher Features** | | | | *Not part of `window.msgapi`; CLI / launcher-side* |
 | Markdown rendering (`.md`/`.markdown` auto-render) | ✅ | ✅ | ❌ | Local `file://` URL → rendered HTML via `msgcommon/markdown` (marked + light/dark CSS). msga could match it in `MainPage.xaml.cs` URL interception — not done yet. |
 | Unpacked Chrome extension loading (`-extension <dir>`) | ✅ | ❌ | ❌ | msgview-only via `session.extensions.loadExtension`. wry has no public extension API; WebKitGTK has none at all. **Not the add-in mechanism here** — app add-ins are `window.msgapi` + injected init scripts (`initScript` / `initScriptPath`), which work in msger and msga without Chromium. Chrome-extension-specific needs go to Chrome. |

package/msgapi-plan.md

@@ -242,6 +242,53 @@ interface MsgAPI {
 
 ---
 
+## Not Planned (recorded so it doesn't get re-investigated)
+
+### Microphone / camera capture — deferred, direction is an external package
+
+**Status 2026-08-01: not in msgapi, and no host grants the permission.** Recorded
+after the wallclock app showed *"Initialization failed: Microphone constraints not
+supported"* under msger on a Pi 5.
+
+**Why it isn't an msgapi API.** The mic is reached with the plain web API
+`navigator.mediaDevices.getUserMedia` — the engine already provides it. msgapi
+exists for what a browser *can't* do. What's actually missing is a **permission
+decision**, and none of the hosts makes one:
+
+| host | today |
+|------|-------|
+| msger / WebKitGTK (Linux, Pi) | wry 0.54's WebKitGTK backend has no permission-request handling at all; WebKitGTK denies an unhandled `permission-request` by default |
+| msger / WebView2 (Windows) | msger never handles `PermissionRequested`; whatever WebView2 defaults to applies |
+| msga (Android) | manifest doesn't declare `RECORD_AUDIO` (the only hits are inside `Mono.Android.dll`); no `onPermissionRequest` handler |
+
+**Unresolved:** whether the Pi failure is the permission gate or the app's
+constraints — WebKitGTK is pickier than Chromium about `echoCancellation`,
+`sampleRate`, etc. One probe settles it: call `getUserMedia({audio: true})` with
+bare constraints via `initScript` and read the error name — `NotAllowedError`
+means permission, `OverconstrainedError` means constraints.
+
+**If it ever becomes a priority**, the work is:
+
+1. **msger (Rust)** — connect WebKitGTK's `permission-request` signal and WebView2's
+   `PermissionRequested` event, granting media for content the caller trusts.
+   Opt-in per the no-regression rule in CLAUDE.md (e.g. `permissions:
+   ["microphone","camera"]` / `-allow mic`), defaulting to today's behavior so no
+   existing caller changes.
+2. **msga** — `RECORD_AUDIO` in the manifest, the Android runtime permission
+   request, and a `WebChromeClient.onPermissionRequest` handler.
+3. **msgapidefs** — a matrix row for which host grants what. No new
+   `window.msgapi` call is needed.
+
+**Direction (Bob, 2026-08-01): don't build this speculatively.** The original
+motivation was speech processing, which never worked well enough in-browser to be
+worth it. The likely future is delegating to an **external speech package** —
+msger/msga hand off to a native or networked processor (reachable today through
+`msgapi.shell` / `msgapi.http` / `msgapi.udp`) rather than capturing audio in the
+page. In-page mic support is only worth building if a concrete use case shows up
+that genuinely needs the audio inside the WebView.
+
+---
+
 ## Implementation Priorities
 
 ### Phase 1: msga msgapi + UDP ✅ Done

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@bobfrankston/msgapidefs",
-  "version": "0.1.38",
+  "version": "0.1.39",
   "description": "TypeScript definitions for msgapi JavaScript API (msgview/msger)",
   "type": "module",
   "main": "./msgapidefs.js",