@trillboards/edge-platform-rtsp 0.2.1 → 0.2.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +86 -0
- package/dist/MacAudioProvider.d.ts +102 -0
- package/dist/MacAudioProvider.d.ts.map +1 -0
- package/dist/MacAudioProvider.js +218 -0
- package/dist/MacAudioProvider.js.map +1 -0
- package/dist/RTSPAudioProvider.d.ts +299 -0
- package/dist/RTSPAudioProvider.d.ts.map +1 -0
- package/dist/RTSPAudioProvider.js +644 -0
- package/dist/RTSPAudioProvider.js.map +1 -0
- package/dist/RTSPCameraProvider.d.ts +2 -0
- package/dist/RTSPCameraProvider.d.ts.map +1 -1
- package/dist/RTSPCameraProvider.js +3 -1
- package/dist/RTSPCameraProvider.js.map +1 -1
- package/dist/index.d.ts +4 -0
- package/dist/index.d.ts.map +1 -1
- package/dist/index.js +14 -1
- package/dist/index.js.map +1 -1
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -12,6 +12,11 @@ This is the package to use whenever a camera is reachable over RTSP, regardless
|
|
|
12
12
|
of the host OS. (V4L2 / DirectShow remain in `@trillboards/edge-platform-linux`
|
|
13
13
|
and `@trillboards/edge-platform-windows` for locally-attached USB/CSI cameras.)
|
|
14
14
|
|
|
15
|
+
It also provides the two ffmpeg-based **audio** providers — `RTSPAudioProvider`
|
|
16
|
+
(continuous audio off a camera/NVR channel) and `MacAudioProvider` (the macOS
|
|
17
|
+
local microphone) — both implementing `AudioProvider` from
|
|
18
|
+
`@trillboards/edge-core`.
|
|
19
|
+
|
|
15
20
|
## Install
|
|
16
21
|
|
|
17
22
|
```bash
|
|
@@ -43,6 +48,87 @@ An NVR typically exposes one RTSP URL per channel
|
|
|
43
48
|
(`…/Channels/101`, `…/Channels/201`, …). Create one `RTSPCameraProvider` per
|
|
44
49
|
channel — one feed = one screen = one sensing profile.
|
|
45
50
|
|
|
51
|
+
## Audio
|
|
52
|
+
|
|
53
|
+
### `RTSPAudioProvider` — continuous audio off a camera / NVR channel
|
|
54
|
+
|
|
55
|
+
Frame capture is one-shot per frame. Audio is not: it is a **long-lived ffmpeg
|
|
56
|
+
session** feeding a bounded ring buffer, because sampling a few seconds per
|
|
57
|
+
cadence misses most of what audio exists to catch.
|
|
58
|
+
|
|
59
|
+
That difference is why audio has its **own admission lane**. Long-lived audio
|
|
60
|
+
sessions are counted per *physical recorder* (`host:port` — credentials and
|
|
61
|
+
channel path excluded, so 32 NVR channels share one budget) in a pool **separate
|
|
62
|
+
from video pulls**, capped by `maxAudioSessionsPerSource` (**default 1**). Audio
|
|
63
|
+
therefore can never consume a frame-capture slot, and a 32-channel NVR opens one
|
|
64
|
+
continuous audio session rather than 32.
|
|
65
|
+
|
|
66
|
+
```ts
|
|
67
|
+
import { RTSPAudioProvider } from '@trillboards/edge-platform-rtsp';
|
|
68
|
+
|
|
69
|
+
const audio = new RTSPAudioProvider({
|
|
70
|
+
rtspUrl: 'rtsp://user:pass@192.168.1.100:554/Streaming/Channels/101',
|
|
71
|
+
sampleRate: 16000, // mono Float32 out
|
|
72
|
+
});
|
|
73
|
+
|
|
74
|
+
// Continuous stream (what the SDK's onset detector attaches to):
|
|
75
|
+
const off = audio.onPcm((pcm, atMs) => { /* … */ });
|
|
76
|
+
|
|
77
|
+
await audio.open(); // may reject — see below
|
|
78
|
+
const recent = await audio.getAudioChunk(1000); // AudioProvider pull contract
|
|
79
|
+
off();
|
|
80
|
+
await audio.close(); // frees the recorder's audio slot
|
|
81
|
+
```
|
|
82
|
+
|
|
83
|
+
`open()` rejects with a **typed, non-fatal** error in the two normal cases, and
|
|
84
|
+
callers should treat both as "this feed has no audio" rather than as a fault:
|
|
85
|
+
|
|
86
|
+
| Error | `code` | Meaning |
|
|
87
|
+
|---|---|---|
|
|
88
|
+
| `RtspAudioAdmissionError` | `RTSP_AUDIO_SESSION_LIMIT` | This recorder already runs its allowed audio session(s). Never queues. |
|
|
89
|
+
| `RtspAudioNoTrackError` | `RTSP_AUDIO_NO_TRACK` | The stream carries no audio track (most CCTV channels). The slot is released immediately so a camera on the same recorder that *does* have a microphone can use it. |
|
|
90
|
+
|
|
91
|
+
Any other failure (dropped TCP, recorder reboot) is transient and retried — a
|
|
92
|
+
running session with capped exponential backoff and no permanent retirement, and
|
|
93
|
+
an *opening* one for `openValidationAttempts` tries (default 4). Supervising only
|
|
94
|
+
the running case would leave a recorder that is merely slow at boot costing that
|
|
95
|
+
feed its audio for the life of the process. `getStats()` derives liveness on every call — there is
|
|
96
|
+
deliberately **no cached capability boolean**.
|
|
97
|
+
|
|
98
|
+
| Option | Default | Notes |
|
|
99
|
+
|---|---|---|
|
|
100
|
+
| `rtspUrl` | (required) | Credentials never reach `argv` (written to a 0600 ffconcat file). |
|
|
101
|
+
| `sampleRate` | `16000` | Output rate, mono. |
|
|
102
|
+
| `bufferSeconds` | `30` | Ring depth; also litert-lm's max clip length. |
|
|
103
|
+
| `maxAudioSessionsPerSource` | `1` | Long-lived audio sessions per recorder `host:port`. |
|
|
104
|
+
| `transport` | `'tcp'` | As the camera provider. |
|
|
105
|
+
| `openValidationAttempts` | `4` | Transient misses tolerated while opening. A no-audio-track stream is never retried. |
|
|
106
|
+
| `restartDelayMs` / `maxRestartDelayMs` | `1000` / `60000` | Backoff bounds for transient restarts. |
|
|
107
|
+
|
|
108
|
+
### `MacAudioProvider` — macOS local microphone
|
|
109
|
+
|
|
110
|
+
Captures via ffmpeg's `avfoundation` input. It exists because `loadAudio()` on
|
|
111
|
+
darwin previously fell through to the Linux `PulseAudioProvider`, whose `require`
|
|
112
|
+
*succeeds* on macOS and only throws later inside `open()` — into a swallowed
|
|
113
|
+
catch, so the Mac heard nothing and reported no fault.
|
|
114
|
+
|
|
115
|
+
macOS gates the microphone behind TCC, and that is reported as itself rather than
|
|
116
|
+
as broken hardware:
|
|
117
|
+
|
|
118
|
+
| Error | `code` |
|
|
119
|
+
|---|---|
|
|
120
|
+
| `MacAudioPermissionError` | `MAC_AUDIO_TCC_DENIED` — grant Microphone access in System Settings |
|
|
121
|
+
| `MacAudioNoDeviceError` | `MAC_AUDIO_NO_DEVICE` — no capture device at all |
|
|
122
|
+
|
|
123
|
+
```ts
|
|
124
|
+
import { MacAudioProvider } from '@trillboards/edge-platform-rtsp';
|
|
125
|
+
|
|
126
|
+
const mic = new MacAudioProvider(); // auto-selects the first audio device
|
|
127
|
+
await mic.open();
|
|
128
|
+
const pcm = await mic.getAudioChunk(1000);
|
|
129
|
+
await mic.close();
|
|
130
|
+
```
|
|
131
|
+
|
|
46
132
|
## Config
|
|
47
133
|
|
|
48
134
|
| Option | Default | Notes |
|
|
@@ -0,0 +1,102 @@
|
|
|
1
|
+
/// <reference types="node" />
|
|
2
|
+
import { spawn } from 'child_process';
|
|
3
|
+
import type { AudioProvider } from '@trillboards/edge-core';
|
|
4
|
+
/**
|
|
5
|
+
* MacAudioProvider — local microphone capture on macOS via ffmpeg's
|
|
6
|
+
* `avfoundation` input.
|
|
7
|
+
*
|
|
8
|
+
* ── WHY THIS EXISTS. `EdgeAgent.loadAudio()` branches on `platform === 'windows'`
|
|
9
|
+
* and sends EVERYTHING else to `PulseAudioProvider`. On darwin that `require`
|
|
10
|
+
* SUCCEEDS — the package is installed — so the agent holds a provider that
|
|
11
|
+
* looks fine and only fails later, inside `open()`, when it cannot find
|
|
12
|
+
* `parecord` or `arecord` (neither exists on macOS). That throw lands in a
|
|
13
|
+
* swallowed catch, so a Mac connector reports no audio fault and simply never
|
|
14
|
+
* hears anything. It is the same shape as the camera path, where darwin
|
|
15
|
+
* collapses to `'linux'` at `config.ts:602`, `V4L2CameraProvider` throws, and
|
|
16
|
+
* `EdgeAgent.ts:153-155` downgrades it to a warning
|
|
17
|
+
* (docs/runbooks/cctv-connector-macos-readiness.md).
|
|
18
|
+
*
|
|
19
|
+
* ── WHY IT LIVES IN THIS PACKAGE, which is named for RTSP. Deliberate, and the
|
|
20
|
+
* alternative was worse. A new `edge-platform-macos` package would add a
|
|
21
|
+
* workspace member, a publish lane and a new `optionalDependencies` entry on
|
|
22
|
+
* `edge-sdk` — and a dependency-graph change IS a payload change under the
|
|
23
|
+
* `SDK_VERSION` lockstep contract, which is exactly what broke the Windows
|
|
24
|
+
* release at 0.2.40. This package is already an optional dependency of
|
|
25
|
+
* `edge-sdk`, already ffmpeg-only, and already declares itself cross-platform
|
|
26
|
+
* ("works on Linux, macOS, Windows, and Docker"). Adding one more ffmpeg
|
|
27
|
+
* capture source to it changes no graph and needs no version bump.
|
|
28
|
+
*
|
|
29
|
+
* ── SHORT-LIVED, ON PURPOSE. This mirrors `WindowsAudioProvider` and
|
|
30
|
+
* `PulseAudioProvider`: one ffmpeg per `getAudioChunk` call. The CONTINUOUS
|
|
31
|
+
* path — the one the onset detector needs — is `RTSPAudioProvider`, because on
|
|
32
|
+
* the CCTV connector the audio that matters comes off the camera/NVR, not off
|
|
33
|
+
* the host's own microphone. This provider exists to make the darwin branch
|
|
34
|
+
* honest, not to become a second streaming stack.
|
|
35
|
+
*
|
|
36
|
+
* ── TCC IS THE REAL FAILURE MODE, AND IT IS REPORTED AS ITSELF. macOS gates
|
|
37
|
+
* microphone access behind TCC. When the grant is missing or denied,
|
|
38
|
+
* avfoundation does not return silence — it fails, and today the connector has
|
|
39
|
+
* ZERO TCC awareness anywhere, so any such failure surfaces as a generic
|
|
40
|
+
* "audio unavailable" indistinguishable from "this Mac has no microphone"
|
|
41
|
+
* (the same runbook names this as the genuine gap, and notes `diagnose.ts`
|
|
42
|
+
* returning `pass` for checks it merely skipped). `MacAudioPermissionError`
|
|
43
|
+
* carries `code = 'MAC_AUDIO_TCC_DENIED'` so an operator is told to grant
|
|
44
|
+
* Microphone access rather than sent to look for broken hardware.
|
|
45
|
+
*/
|
|
46
|
+
/** The microphone TCC grant is missing or denied. Actionable, not a hardware fault. */
|
|
47
|
+
export declare class MacAudioPermissionError extends Error {
|
|
48
|
+
readonly code = "MAC_AUDIO_TCC_DENIED";
|
|
49
|
+
constructor(detail: string);
|
|
50
|
+
}
|
|
51
|
+
/** No capture device at all — genuinely different from a denied grant. */
|
|
52
|
+
export declare class MacAudioNoDeviceError extends Error {
|
|
53
|
+
readonly code = "MAC_AUDIO_NO_DEVICE";
|
|
54
|
+
constructor(detail: string);
|
|
55
|
+
}
|
|
56
|
+
export interface MacAudioConfig {
|
|
57
|
+
/**
|
|
58
|
+
* avfoundation audio device INDEX as listed by
|
|
59
|
+
* `ffmpeg -f avfoundation -list_devices true -i ""`. Auto-detected if omitted.
|
|
60
|
+
*/
|
|
61
|
+
deviceIndex?: string;
|
|
62
|
+
/** Sample rate in Hz (default: 16000). */
|
|
63
|
+
sampleRate?: number;
|
|
64
|
+
/** Channels (default: 1 — mono). */
|
|
65
|
+
channels?: number;
|
|
66
|
+
/** Path to ffmpeg binary (default: 'ffmpeg'). */
|
|
67
|
+
ffmpegPath?: string;
|
|
68
|
+
/** Injected spawn, for tests. */
|
|
69
|
+
spawnFn?: typeof spawn;
|
|
70
|
+
/** Injected device enumeration, for tests. */
|
|
71
|
+
listDevicesFn?: () => string;
|
|
72
|
+
}
|
|
73
|
+
export declare class MacAudioProvider implements AudioProvider {
|
|
74
|
+
private readonly configDeviceIndex;
|
|
75
|
+
private readonly sampleRate;
|
|
76
|
+
private readonly channels;
|
|
77
|
+
private readonly ffmpegPath;
|
|
78
|
+
private readonly spawnFn;
|
|
79
|
+
private readonly listDevicesFn;
|
|
80
|
+
private opened;
|
|
81
|
+
private resolvedDeviceIndex;
|
|
82
|
+
constructor(config?: MacAudioConfig);
|
|
83
|
+
getSampleRate(): number;
|
|
84
|
+
isOpen(): boolean;
|
|
85
|
+
/** Resolved avfoundation device index, or null before `open()`. */
|
|
86
|
+
getDeviceIndex(): string | null;
|
|
87
|
+
open(deviceId?: string): Promise<void>;
|
|
88
|
+
getAudioChunk(durationMs: number): Promise<Float32Array>;
|
|
89
|
+
close(): Promise<void>;
|
|
90
|
+
/**
|
|
91
|
+
* Parse the audio half of `-list_devices true`. ffmpeg prints video devices
|
|
92
|
+
* and audio devices under separate headers, and the indices RESTART at 0 for
|
|
93
|
+
* audio — so a parser that ignores the headers happily returns a camera index
|
|
94
|
+
* and opens the wrong TCC grant.
|
|
95
|
+
*/
|
|
96
|
+
static parseAudioDevices(listing: string): Array<{
|
|
97
|
+
index: string;
|
|
98
|
+
name: string;
|
|
99
|
+
}>;
|
|
100
|
+
private listDevicesViaFfmpeg;
|
|
101
|
+
}
|
|
102
|
+
//# sourceMappingURL=MacAudioProvider.d.ts.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"MacAudioProvider.d.ts","sourceRoot":"","sources":["../src/MacAudioProvider.ts"],"names":[],"mappings":";AAAA,OAAO,EAAE,KAAK,EAAY,MAAM,eAAe,CAAC;AAChD,OAAO,KAAK,EAAE,aAAa,EAAE,MAAM,wBAAwB,CAAC;AAE5D;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;GAyCG;AAEH,uFAAuF;AACvF,qBAAa,uBAAwB,SAAQ,KAAK;IAChD,QAAQ,CAAC,IAAI,0BAA0B;gBAC3B,MAAM,EAAE,MAAM;CAQ3B;AAED,0EAA0E;AAC1E,qBAAa,qBAAsB,SAAQ,KAAK;IAC9C,QAAQ,CAAC,IAAI,yBAAyB;gBAC1B,MAAM,EAAE,MAAM;CAI3B;AAaD,MAAM,WAAW,cAAc;IAC7B;;;OAGG;IACH,WAAW,CAAC,EAAE,MAAM,CAAC;IACrB,0CAA0C;IAC1C,UAAU,CAAC,EAAE,MAAM,CAAC;IACpB,oCAAoC;IACpC,QAAQ,CAAC,EAAE,MAAM,CAAC;IAClB,iDAAiD;IACjD,UAAU,CAAC,EAAE,MAAM,CAAC;IACpB,iCAAiC;IACjC,OAAO,CAAC,EAAE,OAAO,KAAK,CAAC;IACvB,8CAA8C;IAC9C,aAAa,CAAC,EAAE,MAAM,MAAM,CAAC;CAC9B;AAED,qBAAa,gBAAiB,YAAW,aAAa;IACpD,OAAO,CAAC,QAAQ,CAAC,iBAAiB,CAAS;IAC3C,OAAO,CAAC,QAAQ,CAAC,UAAU,CAAS;IACpC,OAAO,CAAC,QAAQ,CAAC,QAAQ,CAAS;IAClC,OAAO,CAAC,QAAQ,CAAC,UAAU,CAAS;IACpC,OAAO,CAAC,QAAQ,CAAC,OAAO,CAAe;IACvC,OAAO,CAAC,QAAQ,CAAC,aAAa,CAAe;IAC7C,OAAO,CAAC,MAAM,CAAS;IACvB,OAAO,CAAC,mBAAmB,CAAuB;gBAEtC,MAAM,GAAE,cAAmB;IASvC,aAAa,IAAI,MAAM;IAIvB,MAAM,IAAI,OAAO;IAIjB,mEAAmE;IACnE,cAAc,IAAI,MAAM,GAAG,IAAI;IAIzB,IAAI,CAAC,QAAQ,CAAC,EAAE,MAAM,GAAG,OAAO,CAAC,IAAI,CAAC;IAsBtC,aAAa,CAAC,UAAU,EAAE,MAAM,GAAG,OAAO,CAAC,YAAY,CAAC;IAmDxD,KAAK,IAAI,OAAO,CAAC,IAAI,CAAC;IAK5B;;;;;OAKG;IACH,MAAM,CAAC,iBAAiB,CAAC,OAAO,EAAE,MAAM,GAAG,KAAK,CAAC;QAAE,KAAK,EAAE,MAAM,CAAC;QAAC,IAAI,EAAE,MAAM,CAAA;KAAE,CAAC;IAmBjF,OAAO,CAAC,oBAAoB;CAe7B"}
|
|
@@ -0,0 +1,218 @@
|
|
|
1
|
+
"use strict";
|
|
2
|
+
Object.defineProperty(exports, "__esModule", { value: true });
|
|
3
|
+
exports.MacAudioProvider = exports.MacAudioNoDeviceError = exports.MacAudioPermissionError = void 0;
|
|
4
|
+
const child_process_1 = require("child_process");
|
|
5
|
+
/**
|
|
6
|
+
* MacAudioProvider — local microphone capture on macOS via ffmpeg's
|
|
7
|
+
* `avfoundation` input.
|
|
8
|
+
*
|
|
9
|
+
* ── WHY THIS EXISTS. `EdgeAgent.loadAudio()` branches on `platform === 'windows'`
|
|
10
|
+
* and sends EVERYTHING else to `PulseAudioProvider`. On darwin that `require`
|
|
11
|
+
* SUCCEEDS — the package is installed — so the agent holds a provider that
|
|
12
|
+
* looks fine and only fails later, inside `open()`, when it cannot find
|
|
13
|
+
* `parecord` or `arecord` (neither exists on macOS). That throw lands in a
|
|
14
|
+
* swallowed catch, so a Mac connector reports no audio fault and simply never
|
|
15
|
+
* hears anything. It is the same shape as the camera path, where darwin
|
|
16
|
+
* collapses to `'linux'` at `config.ts:602`, `V4L2CameraProvider` throws, and
|
|
17
|
+
* `EdgeAgent.ts:153-155` downgrades it to a warning
|
|
18
|
+
* (docs/runbooks/cctv-connector-macos-readiness.md).
|
|
19
|
+
*
|
|
20
|
+
* ── WHY IT LIVES IN THIS PACKAGE, which is named for RTSP. Deliberate, and the
|
|
21
|
+
* alternative was worse. A new `edge-platform-macos` package would add a
|
|
22
|
+
* workspace member, a publish lane and a new `optionalDependencies` entry on
|
|
23
|
+
* `edge-sdk` — and a dependency-graph change IS a payload change under the
|
|
24
|
+
* `SDK_VERSION` lockstep contract, which is exactly what broke the Windows
|
|
25
|
+
* release at 0.2.40. This package is already an optional dependency of
|
|
26
|
+
* `edge-sdk`, already ffmpeg-only, and already declares itself cross-platform
|
|
27
|
+
* ("works on Linux, macOS, Windows, and Docker"). Adding one more ffmpeg
|
|
28
|
+
* capture source to it changes no graph and needs no version bump.
|
|
29
|
+
*
|
|
30
|
+
* ── SHORT-LIVED, ON PURPOSE. This mirrors `WindowsAudioProvider` and
|
|
31
|
+
* `PulseAudioProvider`: one ffmpeg per `getAudioChunk` call. The CONTINUOUS
|
|
32
|
+
* path — the one the onset detector needs — is `RTSPAudioProvider`, because on
|
|
33
|
+
* the CCTV connector the audio that matters comes off the camera/NVR, not off
|
|
34
|
+
* the host's own microphone. This provider exists to make the darwin branch
|
|
35
|
+
* honest, not to become a second streaming stack.
|
|
36
|
+
*
|
|
37
|
+
* ── TCC IS THE REAL FAILURE MODE, AND IT IS REPORTED AS ITSELF. macOS gates
|
|
38
|
+
* microphone access behind TCC. When the grant is missing or denied,
|
|
39
|
+
* avfoundation does not return silence — it fails, and today the connector has
|
|
40
|
+
* ZERO TCC awareness anywhere, so any such failure surfaces as a generic
|
|
41
|
+
* "audio unavailable" indistinguishable from "this Mac has no microphone"
|
|
42
|
+
* (the same runbook names this as the genuine gap, and notes `diagnose.ts`
|
|
43
|
+
* returning `pass` for checks it merely skipped). `MacAudioPermissionError`
|
|
44
|
+
* carries `code = 'MAC_AUDIO_TCC_DENIED'` so an operator is told to grant
|
|
45
|
+
* Microphone access rather than sent to look for broken hardware.
|
|
46
|
+
*/
|
|
47
|
+
/** The microphone TCC grant is missing or denied. Actionable, not a hardware fault. */
|
|
48
|
+
class MacAudioPermissionError extends Error {
|
|
49
|
+
constructor(detail) {
|
|
50
|
+
super('macOS microphone access denied (TCC). Grant Microphone permission to the ' +
|
|
51
|
+
'Trillboards connector in System Settings > Privacy & Security > Microphone. ' +
|
|
52
|
+
`ffmpeg reported: ${detail}`);
|
|
53
|
+
this.code = 'MAC_AUDIO_TCC_DENIED';
|
|
54
|
+
this.name = 'MacAudioPermissionError';
|
|
55
|
+
}
|
|
56
|
+
}
|
|
57
|
+
exports.MacAudioPermissionError = MacAudioPermissionError;
|
|
58
|
+
/** No capture device at all — genuinely different from a denied grant. */
|
|
59
|
+
class MacAudioNoDeviceError extends Error {
|
|
60
|
+
constructor(detail) {
|
|
61
|
+
super(`No macOS audio capture device available. ${detail}`);
|
|
62
|
+
this.code = 'MAC_AUDIO_NO_DEVICE';
|
|
63
|
+
this.name = 'MacAudioNoDeviceError';
|
|
64
|
+
}
|
|
65
|
+
}
|
|
66
|
+
exports.MacAudioNoDeviceError = MacAudioNoDeviceError;
|
|
67
|
+
/**
|
|
68
|
+
* avfoundation's wording when TCC blocks the device. A denied grant surfaces
|
|
69
|
+
* either as an explicit permission line or as a device that opens and then
|
|
70
|
+
* fails to deliver — both are the SAME operator action, so both map here.
|
|
71
|
+
*/
|
|
72
|
+
const TCC_DENIED_RE = /(not authorized to capture audio|permission denied|operation not permitted|tcc|privacy settings|failed to open the audio device|abort\(\) called)/i;
|
|
73
|
+
const NO_DEVICE_RE = /(no audio capture devices|input device not found|error opening input device|avfoundation input device.*not found)/i;
|
|
74
|
+
class MacAudioProvider {
|
|
75
|
+
constructor(config = {}) {
|
|
76
|
+
this.opened = false;
|
|
77
|
+
this.resolvedDeviceIndex = null;
|
|
78
|
+
this.configDeviceIndex = config.deviceIndex ?? '';
|
|
79
|
+
this.sampleRate = config.sampleRate ?? 16000;
|
|
80
|
+
this.channels = config.channels ?? 1;
|
|
81
|
+
this.ffmpegPath = config.ffmpegPath ?? 'ffmpeg';
|
|
82
|
+
this.spawnFn = config.spawnFn ?? child_process_1.spawn;
|
|
83
|
+
this.listDevicesFn = config.listDevicesFn ?? (() => this.listDevicesViaFfmpeg());
|
|
84
|
+
}
|
|
85
|
+
getSampleRate() {
|
|
86
|
+
return this.sampleRate;
|
|
87
|
+
}
|
|
88
|
+
isOpen() {
|
|
89
|
+
return this.opened;
|
|
90
|
+
}
|
|
91
|
+
/** Resolved avfoundation device index, or null before `open()`. */
|
|
92
|
+
getDeviceIndex() {
|
|
93
|
+
return this.resolvedDeviceIndex;
|
|
94
|
+
}
|
|
95
|
+
async open(deviceId) {
|
|
96
|
+
const requested = deviceId ?? this.configDeviceIndex;
|
|
97
|
+
if (requested) {
|
|
98
|
+
this.resolvedDeviceIndex = requested;
|
|
99
|
+
this.opened = true;
|
|
100
|
+
return;
|
|
101
|
+
}
|
|
102
|
+
const listing = this.listDevicesFn();
|
|
103
|
+
if (TCC_DENIED_RE.test(listing)) {
|
|
104
|
+
throw new MacAudioPermissionError(firstMatchingLine(listing, TCC_DENIED_RE));
|
|
105
|
+
}
|
|
106
|
+
const devices = MacAudioProvider.parseAudioDevices(listing);
|
|
107
|
+
if (devices.length === 0) {
|
|
108
|
+
throw new MacAudioNoDeviceError('ffmpeg -f avfoundation -list_devices reported no audio inputs.');
|
|
109
|
+
}
|
|
110
|
+
this.resolvedDeviceIndex = devices[0].index;
|
|
111
|
+
this.opened = true;
|
|
112
|
+
}
|
|
113
|
+
async getAudioChunk(durationMs) {
|
|
114
|
+
if (!this.opened || this.resolvedDeviceIndex === null) {
|
|
115
|
+
throw new Error('Audio not opened. Call open() first.');
|
|
116
|
+
}
|
|
117
|
+
const durationSec = Math.max(0.01, durationMs / 1000);
|
|
118
|
+
return new Promise((resolve, reject) => {
|
|
119
|
+
const args = [
|
|
120
|
+
'-nostdin',
|
|
121
|
+
'-f', 'avfoundation',
|
|
122
|
+
// avfoundation input spec is "<video>:<audio>"; a leading colon means
|
|
123
|
+
// audio-only, which is what keeps this from opening the camera and
|
|
124
|
+
// tripping the SEPARATE camera TCC grant.
|
|
125
|
+
'-i', `:${this.resolvedDeviceIndex}`,
|
|
126
|
+
'-t', String(durationSec),
|
|
127
|
+
'-ac', String(this.channels),
|
|
128
|
+
'-ar', String(this.sampleRate),
|
|
129
|
+
'-f', 's16le',
|
|
130
|
+
'pipe:1',
|
|
131
|
+
];
|
|
132
|
+
const proc = this.spawnFn(this.ffmpegPath, args, { stdio: ['ignore', 'pipe', 'pipe'] });
|
|
133
|
+
const chunks = [];
|
|
134
|
+
let stderr = '';
|
|
135
|
+
proc.stdout?.on('data', (c) => chunks.push(c));
|
|
136
|
+
proc.stderr?.on('data', (c) => {
|
|
137
|
+
if (stderr.length < 8000)
|
|
138
|
+
stderr += c.toString();
|
|
139
|
+
});
|
|
140
|
+
proc.on('error', (err) => reject(err));
|
|
141
|
+
proc.on('close', (code) => {
|
|
142
|
+
if (TCC_DENIED_RE.test(stderr)) {
|
|
143
|
+
reject(new MacAudioPermissionError(firstMatchingLine(stderr, TCC_DENIED_RE)));
|
|
144
|
+
return;
|
|
145
|
+
}
|
|
146
|
+
if (NO_DEVICE_RE.test(stderr)) {
|
|
147
|
+
reject(new MacAudioNoDeviceError(firstMatchingLine(stderr, NO_DEVICE_RE)));
|
|
148
|
+
return;
|
|
149
|
+
}
|
|
150
|
+
const buf = Buffer.concat(chunks);
|
|
151
|
+
if (code !== 0 && buf.length === 0) {
|
|
152
|
+
reject(new Error(`ffmpeg avfoundation capture failed (exit ${code}): ${stderr.slice(-400)}`));
|
|
153
|
+
return;
|
|
154
|
+
}
|
|
155
|
+
const usable = buf.length - (buf.length % 2);
|
|
156
|
+
const out = new Float32Array(usable / 2);
|
|
157
|
+
for (let i = 0; i < out.length; i++)
|
|
158
|
+
out[i] = buf.readInt16LE(i * 2) / 32768;
|
|
159
|
+
resolve(out);
|
|
160
|
+
});
|
|
161
|
+
});
|
|
162
|
+
}
|
|
163
|
+
async close() {
|
|
164
|
+
this.opened = false;
|
|
165
|
+
this.resolvedDeviceIndex = null;
|
|
166
|
+
}
|
|
167
|
+
/**
|
|
168
|
+
* Parse the audio half of `-list_devices true`. ffmpeg prints video devices
|
|
169
|
+
* and audio devices under separate headers, and the indices RESTART at 0 for
|
|
170
|
+
* audio — so a parser that ignores the headers happily returns a camera index
|
|
171
|
+
* and opens the wrong TCC grant.
|
|
172
|
+
*/
|
|
173
|
+
static parseAudioDevices(listing) {
|
|
174
|
+
const out = [];
|
|
175
|
+
let inAudio = false;
|
|
176
|
+
for (const line of listing.split(/\r?\n/)) {
|
|
177
|
+
if (/AVFoundation video devices:/i.test(line)) {
|
|
178
|
+
inAudio = false;
|
|
179
|
+
continue;
|
|
180
|
+
}
|
|
181
|
+
if (/AVFoundation audio devices:/i.test(line)) {
|
|
182
|
+
inAudio = true;
|
|
183
|
+
continue;
|
|
184
|
+
}
|
|
185
|
+
if (!inAudio)
|
|
186
|
+
continue;
|
|
187
|
+
const m = line.match(/\[(\d+)\]\s+(.+?)\s*$/);
|
|
188
|
+
if (m)
|
|
189
|
+
out.push({ index: m[1], name: m[2] });
|
|
190
|
+
}
|
|
191
|
+
return out;
|
|
192
|
+
}
|
|
193
|
+
listDevicesViaFfmpeg() {
|
|
194
|
+
try {
|
|
195
|
+
// ffmpeg exits non-zero after listing devices (there is no real input),
|
|
196
|
+
// so the listing arrives on stderr and execSync throws. That is the
|
|
197
|
+
// SUCCESS path — read stderr off the error rather than treating it as one.
|
|
198
|
+
(0, child_process_1.execSync)(`"${this.ffmpegPath}" -f avfoundation -list_devices true -i "" 2>&1`, {
|
|
199
|
+
encoding: 'utf-8',
|
|
200
|
+
timeout: 10000,
|
|
201
|
+
});
|
|
202
|
+
return '';
|
|
203
|
+
}
|
|
204
|
+
catch (err) {
|
|
205
|
+
const e = err;
|
|
206
|
+
return String(e.stdout ?? '') + String(e.stderr ?? '') + String(e.message ?? '');
|
|
207
|
+
}
|
|
208
|
+
}
|
|
209
|
+
}
|
|
210
|
+
exports.MacAudioProvider = MacAudioProvider;
|
|
211
|
+
function firstMatchingLine(text, re) {
|
|
212
|
+
for (const line of text.split(/\r?\n/)) {
|
|
213
|
+
if (re.test(line))
|
|
214
|
+
return line.trim().slice(0, 300);
|
|
215
|
+
}
|
|
216
|
+
return text.trim().slice(0, 300);
|
|
217
|
+
}
|
|
218
|
+
//# sourceMappingURL=MacAudioProvider.js.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"MacAudioProvider.js","sourceRoot":"","sources":["../src/MacAudioProvider.ts"],"names":[],"mappings":";;;AAAA,iDAAgD;AAGhD;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;GAyCG;AAEH,uFAAuF;AACvF,MAAa,uBAAwB,SAAQ,KAAK;IAEhD,YAAY,MAAc;QACxB,KAAK,CACH,2EAA2E;YACzE,8EAA8E;YAC9E,oBAAoB,MAAM,EAAE,CAC/B,CAAC;QANK,SAAI,GAAG,sBAAsB,CAAC;QAOrC,IAAI,CAAC,IAAI,GAAG,yBAAyB,CAAC;IACxC,CAAC;CACF;AAVD,0DAUC;AAED,0EAA0E;AAC1E,MAAa,qBAAsB,SAAQ,KAAK;IAE9C,YAAY,MAAc;QACxB,KAAK,CAAC,4CAA4C,MAAM,EAAE,CAAC,CAAC;QAFrD,SAAI,GAAG,qBAAqB,CAAC;QAGpC,IAAI,CAAC,IAAI,GAAG,uBAAuB,CAAC;IACtC,CAAC;CACF;AAND,sDAMC;AAED;;;;GAIG;AACH,MAAM,aAAa,GACjB,oJAAoJ,CAAC;AAEvJ,MAAM,YAAY,GAChB,oHAAoH,CAAC;AAoBvH,MAAa,gBAAgB;IAU3B,YAAY,SAAyB,EAAE;QAH/B,WAAM,GAAG,KAAK,CAAC;QACf,wBAAmB,GAAkB,IAAI,CAAC;QAGhD,IAAI,CAAC,iBAAiB,GAAG,MAAM,CAAC,WAAW,IAAI,EAAE,CAAC;QAClD,IAAI,CAAC,UAAU,GAAG,MAAM,CAAC,UAAU,IAAI,KAAM,CAAC;QAC9C,IAAI,CAAC,QAAQ,GAAG,MAAM,CAAC,QAAQ,IAAI,CAAC,CAAC;QACrC,IAAI,CAAC,UAAU,GAAG,MAAM,CAAC,UAAU,IAAI,QAAQ,CAAC;QAChD,IAAI,CAAC,OAAO,GAAG,MAAM,CAAC,OAAO,IAAI,qBAAK,CAAC;QACvC,IAAI,CAAC,aAAa,GAAG,MAAM,CAAC,aAAa,IAAI,CAAC,GAAG,EAAE,CAAC,IAAI,CAAC,oBAAoB,EAAE,CAAC,CAAC;IACnF,CAAC;IAED,aAAa;QACX,OAAO,IAAI,CAAC,UAAU,CAAC;IACzB,CAAC;IAED,MAAM;QACJ,OAAO,IAAI,CAAC,MAAM,CAAC;IACrB,CAAC;IAED,mEAAmE;IACnE,cAAc;QACZ,OAAO,IAAI,CAAC,mBAAmB,CAAC;IAClC,CAAC;IAED,KAAK,CAAC,IAAI,CAAC,QAAiB;QAC1B,MAAM,SAAS,GAAG,QAAQ,IAAI,IAAI,CAAC,iBAAiB,CAAC;QACrD,IAAI,SAAS,EAAE,CAAC;YACd,IAAI,CAAC,mBAAmB,GAAG,SAAS,CAAC;YACrC,IAAI,CAAC,MAAM,GAAG,IAAI,CAAC;YACnB,OAAO;QACT,CAAC;QAED,MAAM,OAAO,GAAG,IAAI,CAAC,aAAa,EAAE,CAAC;QACrC,IAAI,aAAa,CAAC,IAAI,CAAC,OAAO,CAAC,EAAE,CAAC;YAChC,MAAM,IAAI,uBAAuB,CAAC,iBAAiB,CAAC,OAAO,EAAE,aAAa,CAAC,CAAC,CAAC;QAC/E,CAAC;QACD,MAAM,OAAO,GAAG,gBAAgB,CAAC,iBAAiB,CAAC,OAAO,CAAC,CAAC;QAC5D,IAAI,OAAO,CAAC,MAAM,KAAK,CAAC,EAAE,CAAC;YACzB,MAAM,IAAI,qBAAqB,CAC7B,gEAAgE,CACjE,CAAC;QACJ,CAAC;QACD,IAAI,CAAC,mBAAmB,GAAG,OAAO,CAAC,CAAC,CAAC,CAAC,KAAK,CAAC;QAC5C,IAAI,CAAC,MAAM,GAAG,IAAI,CAAC;IACrB,CAAC;IAED,KAAK,CAAC,aAAa,CAAC,UAAkB;QACpC,IAAI,CAAC,IAAI,CAAC,MAAM,IAAI,IAAI,CAAC,mBAAmB,KAAK,IAAI,EAAE,CAAC;YACtD,MAAM,IAAI,KAAK,CAAC,sCAAsC,CAAC,CAAC;QAC1D,CAAC;QACD,MAAM,WAAW,GAAG,IAAI,CAAC,GAAG,CAAC,IAAI,EAAE,UAAU,GAAG,IAAI,CAAC,CAAC;QAEtD,OAAO,IAAI,OAAO,CAAe,CAAC,OAAO,EAAE,MAAM,EAAE,EAAE;YACnD,MAAM,IAAI,GAAG;gBACX,UAAU;gBACV,IAAI,EAAE,cAAc;gBACpB,sEAAsE;gBACtE,mEAAmE;gBACnE,0CAA0C;gBAC1C,IAAI,EAAE,IAAI,IAAI,CAAC,mBAAmB,EAAE;gBACpC,IAAI,EAAE,MAAM,CAAC,WAAW,CAAC;gBACzB,KAAK,EAAE,MAAM,CAAC,IAAI,CAAC,QAAQ,CAAC;gBAC5B,KAAK,EAAE,MAAM,CAAC,IAAI,CAAC,UAAU,CAAC;gBAC9B,IAAI,EAAE,OAAO;gBACb,QAAQ;aACT,CAAC;YACF,MAAM,IAAI,GAAG,IAAI,CAAC,OAAO,CAAC,IAAI,CAAC,UAAU,EAAE,IAAI,EAAE,EAAE,KAAK,EAAE,CAAC,QAAQ,EAAE,MAAM,EAAE,MAAM,CAAC,EAAE,CAAC,CAAC;YACxF,MAAM,MAAM,GAAa,EAAE,CAAC;YAC5B,IAAI,MAAM,GAAG,EAAE,CAAC;YAEhB,IAAI,CAAC,MAAM,EAAE,EAAE,CAAC,MAAM,EAAE,CAAC,CAAS,EAAE,EAAE,CAAC,MAAM,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC,CAAC;YACvD,IAAI,CAAC,MAAM,EAAE,EAAE,CAAC,MAAM,EAAE,CAAC,CAAS,EAAE,EAAE;gBACpC,IAAI,MAAM,CAAC,MAAM,GAAG,IAAK;oBAAE,MAAM,IAAI,CAAC,CAAC,QAAQ,EAAE,CAAC;YACpD,CAAC,CAAC,CAAC;YACH,IAAI,CAAC,EAAE,CAAC,OAAO,EAAE,CAAC,GAAU,EAAE,EAAE,CAAC,MAAM,CAAC,GAAG,CAAC,CAAC,CAAC;YAC9C,IAAI,CAAC,EAAE,CAAC,OAAO,EAAE,CAAC,IAAmB,EAAE,EAAE;gBACvC,IAAI,aAAa,CAAC,IAAI,CAAC,MAAM,CAAC,EAAE,CAAC;oBAC/B,MAAM,CAAC,IAAI,uBAAuB,CAAC,iBAAiB,CAAC,MAAM,EAAE,aAAa,CAAC,CAAC,CAAC,CAAC;oBAC9E,OAAO;gBACT,CAAC;gBACD,IAAI,YAAY,CAAC,IAAI,CAAC,MAAM,CAAC,EAAE,CAAC;oBAC9B,MAAM,CAAC,IAAI,qBAAqB,CAAC,iBAAiB,CAAC,MAAM,EAAE,YAAY,CAAC,CAAC,CAAC,CAAC;oBAC3E,OAAO;gBACT,CAAC;gBACD,MAAM,GAAG,GAAG,MAAM,CAAC,MAAM,CAAC,MAAM,CAAC,CAAC;gBAClC,IAAI,IAAI,KAAK,CAAC,IAAI,GAAG,CAAC,MAAM,KAAK,CAAC,EAAE,CAAC;oBACnC,MAAM,CAAC,IAAI,KAAK,CAAC,4CAA4C,IAAI,MAAM,MAAM,CAAC,KAAK,CAAC,CAAC,GAAG,CAAC,EAAE,CAAC,CAAC,CAAC;oBAC9F,OAAO;gBACT,CAAC;gBACD,MAAM,MAAM,GAAG,GAAG,CAAC,MAAM,GAAG,CAAC,GAAG,CAAC,MAAM,GAAG,CAAC,CAAC,CAAC;gBAC7C,MAAM,GAAG,GAAG,IAAI,YAAY,CAAC,MAAM,GAAG,CAAC,CAAC,CAAC;gBACzC,KAAK,IAAI,CAAC,GAAG,CAAC,EAAE,CAAC,GAAG,GAAG,CAAC,MAAM,EAAE,CAAC,EAAE;oBAAE,GAAG,CAAC,CAAC,CAAC,GAAG,GAAG,CAAC,WAAW,CAAC,CAAC,GAAG,CAAC,CAAC,GAAG,KAAK,CAAC;gBAC7E,OAAO,CAAC,GAAG,CAAC,CAAC;YACf,CAAC,CAAC,CAAC;QACL,CAAC,CAAC,CAAC;IACL,CAAC;IAED,KAAK,CAAC,KAAK;QACT,IAAI,CAAC,MAAM,GAAG,KAAK,CAAC;QACpB,IAAI,CAAC,mBAAmB,GAAG,IAAI,CAAC;IAClC,CAAC;IAED;;;;;OAKG;IACH,MAAM,CAAC,iBAAiB,CAAC,OAAe;QACtC,MAAM,GAAG,GAA2C,EAAE,CAAC;QACvD,IAAI,OAAO,GAAG,KAAK,CAAC;QACpB,KAAK,MAAM,IAAI,IAAI,OAAO,CAAC,KAAK,CAAC,OAAO,CAAC,EAAE,CAAC;YAC1C,IAAI,8BAA8B,CAAC,IAAI,CAAC,IAAI,CAAC,EAAE,CAAC;gBAC9C,OAAO,GAAG,KAAK,CAAC;gBAChB,SAAS;YACX,CAAC;YACD,IAAI,8BAA8B,CAAC,IAAI,CAAC,IAAI,CAAC,EAAE,CAAC;gBAC9C,OAAO,GAAG,IAAI,CAAC;gBACf,SAAS;YACX,CAAC;YACD,IAAI,CAAC,OAAO;gBAAE,SAAS;YACvB,MAAM,CAAC,GAAG,IAAI,CAAC,KAAK,CAAC,uBAAuB,CAAC,CAAC;YAC9C,IAAI,CAAC;gBAAE,GAAG,CAAC,IAAI,CAAC,EAAE,KAAK,EAAE,CAAC,CAAC,CAAC,CAAC,EAAE,IAAI,EAAE,CAAC,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC;QAC/C,CAAC;QACD,OAAO,GAAG,CAAC;IACb,CAAC;IAEO,oBAAoB;QAC1B,IAAI,CAAC;YACH,wEAAwE;YACxE,oEAAoE;YACpE,2EAA2E;YAC3E,IAAA,wBAAQ,EAAC,IAAI,IAAI,CAAC,UAAU,iDAAiD,EAAE;gBAC7E,QAAQ,EAAE,OAAO;gBACjB,OAAO,EAAE,KAAM;aAChB,CAAC,CAAC;YACH,OAAO,EAAE,CAAC;QACZ,CAAC;QAAC,OAAO,GAAG,EAAE,CAAC;YACb,MAAM,CAAC,GAAG,GAA6D,CAAC;YACxE,OAAO,MAAM,CAAC,CAAC,CAAC,MAAM,IAAI,EAAE,CAAC,GAAG,MAAM,CAAC,CAAC,CAAC,MAAM,IAAI,EAAE,CAAC,GAAG,MAAM,CAAC,CAAC,CAAC,OAAO,IAAI,EAAE,CAAC,CAAC;QACnF,CAAC;IACH,CAAC;CACF;AAtJD,4CAsJC;AAED,SAAS,iBAAiB,CAAC,IAAY,EAAE,EAAU;IACjD,KAAK,MAAM,IAAI,IAAI,IAAI,CAAC,KAAK,CAAC,OAAO,CAAC,EAAE,CAAC;QACvC,IAAI,EAAE,CAAC,IAAI,CAAC,IAAI,CAAC;YAAE,OAAO,IAAI,CAAC,IAAI,EAAE,CAAC,KAAK,CAAC,CAAC,EAAE,GAAG,CAAC,CAAC;IACtD,CAAC;IACD,OAAO,IAAI,CAAC,IAAI,EAAE,CAAC,KAAK,CAAC,CAAC,EAAE,GAAG,CAAC,CAAC;AACnC,CAAC"}
|