@live-assistant/audio 0.1.0 → 0.3.0

package/README.md

@@ -1,9 +1,89 @@
 # @live-assistant/audio
 
-`Microphone` and `PcmPlayer` for iOS, Android (`react-native-audio-api`, a peer dependency) and the web (Web Audio). The platform is picked through file extensions.
+`Microphone` and `PcmPlayer` for iOS, Android and the web. The platform is picked
+by file extension, so you import one name and get the right implementation.
 
-- Frames are resampled to the rate the session asks for, not the rate the recorder claims to deliver.
-- `level()` on both classes feeds animations. The player's level follows the playhead, and `remainingSeconds()` says how much audio has not been heard yet.
-- On iOS the session runs in `voiceChat` mode, so the app's own output is cancelled. Android has no echo cancellation here (`cancelsEcho` is false), so the controller holds the microphone shut while the assistant is audible.
+```sh
+npm install @live-assistant/audio react-native-audio-api
+```
 
-See the [overview](../README.md).
+## Native setup — one line
+
+The library ships an Expo config plugin, so this is the whole of it:
+
+```json
+["@live-assistant/audio", { "microphonePermission": "Acme uses your microphone so you can talk to the assistant." }]
+```
+
+(If you installed `@live-assistant/react-native`, name that instead — same
+plugin.)
+
+It writes what a voice assistant needs, checked by running `expo prebuild` and
+reading the **generated** files rather than the config: the microphone usage
+description set to your sentence, **no** `UIBackgroundModes`,
+`android.permission.RECORD_AUDIO` present, and no foreground service.
+
+Left to `react-native-audio-api`'s own plugin defaults you would get
+`UIBackgroundModes: ["audio"]` — background audio nothing here plays, which App
+Review rejects under guideline 2.5.4 — and **no** microphone usage description,
+so iOS terminates the app at the first request. That is why this plugin exists.
+**Do not list `react-native-audio-api` in `plugins` as well**: its plugin runs
+once, and whichever is listed first wins.
+
+Two more things the first run depends on:
+
+- **Expo Go cannot load this.** `react-native-audio-api` is a native module.
+  Build a development build (`npx expo prebuild && npx expo run:ios`) or use EAS.
+- **Bare React Native**: add `NSMicrophoneUsageDescription` to
+  `ios/<App>/Info.plist` yourself; `RECORD_AUDIO` arrives through the module's
+  manifest merge.
+
+## On the web, with or without React Native
+
+Nothing native is involved: the `.web` halves use Web Audio.
+
+- In an **Expo / React Native Web** app, Metro picks them by file extension.
+- In a **plain React** app — Vite, Next, webpack, esbuild — the package's
+  `browser` field points at them, so a browser build never reaches the native
+  module. Checked by bundling an installed copy for `platform=browser`:
+  `react-native-audio-api` does not appear in the output.
+
+Either way `getUserMedia` needs a **secure context**, so serve from `https://` or
+`localhost`. Anything else answers `microphone_unavailable`.
+
+**Start the session from a user gesture.** A browser leaves
+`AudioContext.resume()` pending until the page has been interacted with, so
+`PcmPlayer.prepare()` called on page load never settles — and neither does
+`start()`. Pressing the orb, or any button, is the gesture. Verified by running
+the web halves in a plain browser build: from a click,
+`prepare(24000)` returns ok, a second of audio reports `remainingSeconds=1.00`
+and a real `level()`, and `flush()` takes it back to zero.
+
+## What it does that a recorder does not
+
+- **It promises a sample rate rather than requesting one.** Platform recorders
+  treat the rate as a preference and hand back whatever the hardware runs at;
+  frames labelled with a rate they are not transcribe as fast noise instead of
+  failing. Resampling to the promised rate happens here, so no caller repeats it.
+- **`level()` on both classes feeds animations**, 0–1, pulled on an animation
+  clock rather than pushed. The player's level follows the **playhead**, so a
+  glow moves with what is heard, not with what has arrived.
+- **`remainingSeconds()`** says how much audio has not been heard yet — the
+  number the echo gate and the interruption flush are built on.
+- **Echo differs by platform.** iOS runs the session in `voiceChat` mode, which
+  cancels the app's own output, so `cancelsEcho` is true and the user can
+  interrupt freely. Android has none, so `cancelsEcho` is false and
+  `AssistantController` holds the microphone shut while the assistant is
+  audible. Do not send audio yourself during `speaking`.
+
+## Failures
+
+`microphone_denied` (the user said no, or iOS never asked because the usage
+description is missing) · `microphone_unavailable` (no native module, or not a
+secure context on the web) · `player_unavailable`.
+
+Codes, never sentences: map them to words in your own app.
+
+See the [overview](https://github.com/Recipely-Team/live-assistant#readme) for how this fits together.
+
+A working app that puts this together: [`examples/expo-app`](https://github.com/Recipely-Team/live-assistant/tree/main/examples/expo-app).

package/app.plugin.js

@@ -0,0 +1,40 @@
+/**
+ * The native configuration this library needs, so an app does not have to know
+ * it.
+ *
+ * @remarks
+ * - **Why this exists.** `react-native-audio-api` ships a config plugin whose
+ *   defaults are wrong for a voice assistant, and both mistakes are invisible
+ *   until late: it declares `UIBackgroundModes: ["audio"]`, which App Review
+ *   rejects under guideline 2.5.4 when nothing plays in the background, and it
+ *   writes no `NSMicrophoneUsageDescription`, so iOS terminates the app the
+ *   moment it asks for the microphone. Every integrator would have had to
+ *   discover both.
+ * - **What it produces**, verified by running `expo prebuild` and reading the
+ *   generated files: `NSMicrophoneUsageDescription` set to your sentence, no
+ *   `UIBackgroundModes`, `android.permission.RECORD_AUDIO` present, and no
+ *   foreground service.
+ * - **Use this INSTEAD of listing `react-native-audio-api` yourself.** That
+ *   plugin runs once; if the app lists it first, this one is skipped and the
+ *   defaults win.
+ * - An app that genuinely plays audio while backgrounded should configure
+ *   `react-native-audio-api` directly and not use this plugin — and be ready to
+ *   justify the background mode to App Review.
+ */
+// `react-native-audio-api/app.plugin` re-exports a transpiled ES module, so the
+// function is under `default`. Taking the module itself hands Expo an object and
+// prebuild dies with "withAudioAPI is not a function".
+const audioApiPlugin = require('react-native-audio-api/app.plugin');
+const withAudioAPI = audioApiPlugin.default || audioApiPlugin;
+
+const DEFAULT_MICROPHONE_PERMISSION =
+  'This app uses your microphone so you can talk to the assistant.';
+
+module.exports = (config, props) =>
+  withAudioAPI(config, {
+    iosMicrophonePermission:
+      (props && props.microphonePermission) || DEFAULT_MICROPHONE_PERMISSION,
+    iosBackgroundMode: false,
+    androidForegroundService: false,
+    androidPermissions: [],
+  });

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@live-assistant/audio",
-  "version": "0.1.0",
+  "version": "0.3.0",
   "description": "Microphone capture and streaming PCM playback for @live-assistant/core, on iOS, Android and the web, with levels an animation can read every frame.",
   "license": "MIT",
   "author": "Recep Tayyip Ekşi",
@@ -23,11 +23,16 @@
   ],
   "main": "dist/index.js",
   "types": "dist/index.d.ts",
+  "browser": {
+    "./dist/microphone.js": "./dist/microphone.web.js",
+    "./dist/pcm-player.js": "./dist/pcm-player.web.js"
+  },
   "files": [
     "dist",
     "src",
     "README.md",
     "LICENSE",
+    "app.plugin.js",
     "!src/**/__tests__",
     "!src/**/__fixtures__"
   ],
@@ -40,7 +45,7 @@
     "access": "public"
   },
   "dependencies": {
-    "@live-assistant/core": "0.1.0"
+    "@live-assistant/core": "0.3.0"
   },
   "peerDependencies": {
     "react-native": ">=0.76",