matterbridge-example-camera 0.0.8-dev-20260726-dffb00d → 0.0.8-dev-20260726-61cad4c
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +13 -0
- package/README.md +38 -6
- package/dist/module.js +8 -0
- package/dist/webrtc/weriftSession.js +116 -16
- package/matterbridge-example-camera.config.json +1 -0
- package/matterbridge-example-camera.schema.json +13 -1
- package/npm-shrinkwrap.json +2 -2
- package/package.json +1 -1
package/CHANGELOG.md
CHANGED
|
@@ -40,6 +40,19 @@ If you like this project and find it useful, please consider giving it a star on
|
|
|
40
40
|
- [tests]: Add `rtsp` video source and `videoResolution` precedence (`auto` vs. fixed, webcam and rtsp) coverage in `vitest/webrtc/weriftSession.test.ts`; extend `vitest/module.test.ts` with the `videoGenerator` normalization test and a test for the `videoResolution` default of `auto`.
|
|
41
41
|
- [webrtc]: Add `WeriftWebRtcSession.closeAll()`, a static helper that closes every active session (peer connection plus any injected ffmpeg generators). `onShutdown` now calls it so a graceful platform shutdown cleans up leftover WebRTC sessions itself instead of relying solely on the `process.on('exit', ...)` fallback.
|
|
42
42
|
- [tests]: Add `closeAll` coverage in `vitest/webrtc/weriftSession.test.ts`, covering closing all active sessions and the no-active-sessions case.
|
|
43
|
+
- [platform]: Add an `audioGenerator` config property (`none`/`test`, defaulting to `none`) replacing the undocumented `MATTERBRIDGE_CAMERA_DISABLE_TEST_AUDIO=1` env var toggle, mirroring `videoGenerator`'s pattern via the new `MATTERBRIDGE_CAMERA_AUDIO_SOURCE` env var. Test-voice audio injection is now opt-in instead of on by default.
|
|
44
|
+
- [docs]: Document `audioGenerator` in the README and schema.
|
|
45
|
+
- [tests]: Extend `vitest/module.test.ts` with `audioGenerator` normalization/apply coverage; update `vitest/webrtc/weriftSession.test.ts` for the new `MATTERBRIDGE_CAMERA_AUDIO_SOURCE` gate.
|
|
46
|
+
- [webrtc]: Add a periodic diagnostics log (`WeriftWebRtcSession.logDiagnosticsSnapshot`, every 10s) reporting `connectionState`, `iceConnectionState`, `iceGatheringState`, `signalingState`, per-transport ICE/DTLS states, and `getStats()`-derived packet-flow counters (nominated candidate-pair and outbound-rtp packets, with deltas since the previous tick). Needed because werift's ICE consent-freshness check (RFC 7675) can latch `iceConnectionState` at `disconnected` forever after a single missed keepalive, even on a healthy, actively-streaming session — the packet-flow deltas are unaffected by that bug and are a reliable "is media still moving" signal. A dedicated log line also fires immediately on any DTLS transport state change instead of waiting for the next tick.
|
|
47
|
+
- [webrtc]: Auto-close a `WeriftWebRtcSession` (peer connection plus any injected ffmpeg generators) when one of its DTLS transports reaches `closed` or `failed` on its own — a real, one-way teardown signal from the peer, unlike `iceConnectionState`'s `disconnected`, which must never trigger a close by itself. Fixes an orphaned-session leak: a controller that abandons a live view without ever sending `EndSession` (e.g. just closing the window) previously left the peer connection and its ffmpeg processes running indefinitely, piling up on every reconnect.
|
|
48
|
+
- [tests]: Add DTLS-triggered auto-close coverage in `vitest/webrtc/weriftSession.test.ts` (`closed`, `failed`, and a regression test ensuring a normal `close()` doesn't re-enter itself via the same DTLS state change).
|
|
49
|
+
- [webrtc]: Add `microphone` and `rtsp` options to `audioGenerator`, mirroring `videoGenerator`'s pattern: `microphone` captures from a local capture device via `ffmpeg -f alsa/avfoundation/dshow`, and `rtsp` pulls just the audio from a real RTSP camera stream (dropping any video) via `ffmpeg -rtsp_transport tcp -i <url> -vn`. Add the corresponding `audioSource` config property and `MATTERBRIDGE_CAMERA_AUDIO_SOURCE_DEVICE` env var (device identifier for `microphone`, or the RTSP url for `rtsp`), mirroring `videoSource`/`MATTERBRIDGE_CAMERA_VIDEO_SOURCE_DEVICE`. The `-af volume=6dB` boost is now only applied to the `test` clip (recorded quietly), not to `microphone`/`rtsp` capture.
|
|
50
|
+
- [docs]: Document the `microphone` and `rtsp` audio generators and the new `audioSource` property in the README and schema, with example configurations.
|
|
51
|
+
- [tests]: Extend `vitest/module.test.ts` and `vitest/webrtc/weriftSession.test.ts` with `microphone`/`rtsp` audio source coverage.
|
|
52
|
+
|
|
53
|
+
### Changed
|
|
54
|
+
|
|
55
|
+
- [webrtc]: Replace inline `error instanceof Error ? error.message : String(error)` error-message extraction with `getErrorMessage` from `matterbridge/utils` throughout `weriftSession.ts`; use `fireAndForget` for the auto-close's fire-and-forget `close()` call, matching the pattern already used elsewhere in the plugin.
|
|
43
56
|
|
|
44
57
|
### Fixed
|
|
45
58
|
|
package/README.md
CHANGED
|
@@ -186,9 +186,9 @@ So, to let either Intercom 1 or Intercom 2 start a call, on the fabric they shar
|
|
|
186
186
|
|
|
187
187
|
With both directions in place, either Intercom can call the other; a call initiated the other way only needs its own binding/ACL pair, already covered above since both were set up symmetrically.
|
|
188
188
|
|
|
189
|
-
## WebRTC video injection
|
|
189
|
+
## WebRTC video and audio injection
|
|
190
190
|
|
|
191
|
-
`WeriftWebRtcSession` (see `src/webrtc/weriftSession.ts`) wraps a real werift `RTCPeerConnection` for each WebRtcTransportProvider session (see `MatterbridgeWebRtcTransportProviderServer` in `src/behaviors/webRtcTransportProviderServer.ts`), so the session's SDP offer/answer and ICE candidates are handled by a real WebRTC peer connection instead of being just recorded. It can also inject a real ffmpeg-generated video track into the negotiated connection, so the end-to-end media path can be validated without a real camera capture pipeline.
|
|
191
|
+
`WeriftWebRtcSession` (see `src/webrtc/weriftSession.ts`) wraps a real werift `RTCPeerConnection` for each WebRtcTransportProvider session (see `MatterbridgeWebRtcTransportProviderServer` in `src/behaviors/webRtcTransportProviderServer.ts`), so the session's SDP offer/answer and ICE candidates are handled by a real WebRTC peer connection instead of being just recorded. It can also inject a real ffmpeg-generated video and/or audio track into the negotiated connection, so the end-to-end media path can be validated without a real camera/microphone capture pipeline.
|
|
192
192
|
|
|
193
193
|
The platform configuration controls WebRTC video injection with these properties:
|
|
194
194
|
|
|
@@ -200,11 +200,43 @@ The platform configuration controls WebRTC video injection with these properties
|
|
|
200
200
|
|
|
201
201
|
- `videoBitrate` is required and accepts a number (the target encoder bitrate in kbps). It defaults to `1000`. It applies to the `webcam` and `rtsp` sources only; the `test` pattern is always encoded at a fixed 1000 kbps regardless of this setting. Since the encoder always uses `-preset ultrafast -tune zerolatency` for low-latency live view (lower compression efficiency than slower presets), suggested values are higher than typical streaming guides: ~800 for `640x480`, ~2000 for `1280x720`, ~2500 for `1920x1080`.
|
|
202
202
|
|
|
203
|
+
The platform configuration controls WebRTC audio injection with these properties:
|
|
204
|
+
|
|
205
|
+
- `audioGenerator` is required and accepts `none`, `test`, `microphone`, or `rtsp`. It defaults to `none`, which negotiates the audio transceiver (e.g. for an Intercom's "Listen" live view) without attaching a track. `test` injects a recorded test-voice clip, looped, so the end-to-end audio path can be validated without a real microphone capture pipeline. `microphone` captures from the configured local microphone, and `rtsp` pulls just the audio from the RTSP url configured in `audioSource` (any video in the stream is dropped).
|
|
206
|
+
|
|
207
|
+
- `audioSource` is optional and has no default. For the `microphone` generator it contains the OS-specific ffmpeg device identifier — e.g. an ALSA device such as `hw:0,0` on Linux, an avfoundation audio index such as `0` on macOS, or a device name such as `Microphone Array` on Windows (dshow). For the `rtsp` generator it instead holds the RTSP url, e.g. `rtsp://user:password@host:554/path` (it can be a different camera/url than `videoSource`). Selecting the `microphone` or `rtsp` generator without this property falls back to the test-voice clip with a warning.
|
|
208
|
+
|
|
209
|
+
Use ffmpeg itself to list the [available capture devices](https://trac.ffmpeg.org/wiki/Capture/Webcam) and find the right value for `audioSource`:
|
|
210
|
+
|
|
211
|
+
- Linux (alsa): `arecord -l`, or `ffmpeg -f alsa -i default` to try the default device.
|
|
212
|
+
- macOS (avfoundation): `ffmpeg -f avfoundation -list_devices true -i dummy` — audio devices are listed with their index under "AVFoundation audio devices", e.g. `[0] MacBook Pro Microphone`; use that index (e.g. `0`) as the device value.
|
|
213
|
+
- Windows (dshow): `ffmpeg -f dshow -list_devices true -i dummy` — audio devices are listed by name under "DirectShow audio devices", e.g. `"Microphone Array"`; use that exact name as the device value.
|
|
214
|
+
|
|
215
|
+
Example configuration for a real Linux microphone:
|
|
216
|
+
|
|
217
|
+
```json
|
|
218
|
+
{
|
|
219
|
+
"audioGenerator": "microphone",
|
|
220
|
+
"audioSource": "hw:0,0"
|
|
221
|
+
}
|
|
222
|
+
```
|
|
223
|
+
|
|
224
|
+
Example, pulling the audio from the same RTSP camera used for `videoSource`:
|
|
225
|
+
|
|
226
|
+
```json
|
|
227
|
+
{
|
|
228
|
+
"videoGenerator": "rtsp",
|
|
229
|
+
"videoSource": "rtsp://admin:password@192.168.1.100:554/ch1/main",
|
|
230
|
+
"audioGenerator": "rtsp",
|
|
231
|
+
"audioSource": "rtsp://admin:password@192.168.1.100:554/ch1/main"
|
|
232
|
+
}
|
|
233
|
+
```
|
|
234
|
+
|
|
203
235
|
With `videoResolution: "auto"`, a real client's resolution/quality picker (e.g. in Home Assistant) drives the capture/output resolution: it allocates a video stream with `CameraAvStreamManagement.VideoStreamAllocate` before soliciting or providing a WebRTC offer, and `MatterbridgeWebRtcTransportProviderServer` looks up that stream's `maxResolution` to select the resolution for the session. For `webcam` this is the capture resolution passed to ffmpeg; for `rtsp` the camera keeps streaming at its own native resolution and ffmpeg scales it to match.
|
|
204
236
|
|
|
205
|
-
Requires `ffmpeg` to be installed. The resolver checks the system command and common installation directories on Linux, macOS, and Windows.
|
|
237
|
+
Requires [`ffmpeg`](https://ffmpeg.org/) to be installed. The resolver checks the system command and common installation directories on Linux, macOS, and Windows.
|
|
206
238
|
|
|
207
|
-
Use ffmpeg itself to list the available capture devices and find the right value for `videoSource`:
|
|
239
|
+
Use ffmpeg itself to list the [available capture devices](https://trac.ffmpeg.org/wiki/Capture/Webcam) and find the right value for `videoSource`:
|
|
208
240
|
|
|
209
241
|
- Linux (v4l2): `v4l2-ctl --list-devices` (from `v4l-utils`), or `ls /dev/video*`.
|
|
210
242
|
- macOS (avfoundation): `ffmpeg -f avfoundation -list_devices true -i dummy` — video devices are listed with their index, e.g. `[0] FaceTime HD Camera`; use that index (e.g. `0`) as the device value.
|
|
@@ -242,7 +274,7 @@ Finding your camera's RTSP url for `videoSource` when `videoGenerator` is `rtsp`
|
|
|
242
274
|
- Dahua: `/cam/realmonitor?channel=1&subtype=0`
|
|
243
275
|
- Reolink: `/h264Preview_01_main`
|
|
244
276
|
|
|
245
|
-
Verify the url works standalone before adding it to the config, e.g. `ffprobe rtsp://admin:password@192.168.1.100:554/ch1/main`, or open it in VLC via "Open Network Stream". Note that the url (with credentials in plaintext) ends up stored in Matterbridge's plugin config on disk, like any other password-bearing config value.
|
|
277
|
+
Verify the url works standalone before adding it to the config, e.g. `ffprobe rtsp://admin:password@192.168.1.100:554/ch1/main` or `ffplay rtsp://admin:password@192.168.1.100:554/ch1/main`, or open it in VLC via "Open Network Stream". Note that the url (with credentials in plaintext) ends up stored in Matterbridge's plugin config on disk, like any other password-bearing config value.
|
|
246
278
|
|
|
247
279
|
Example, pulling from a real RTSP camera, scaled to a fixed 1280x720 and re-encoded at `videoBitrate` (the camera's own frame rate is kept):
|
|
248
280
|
|
|
@@ -330,7 +362,7 @@ npm run test -- vitest/werift.test.ts
|
|
|
330
362
|
The `assets` directory contains deterministic three-second media fixtures for extending the werift test to real media tracks:
|
|
331
363
|
|
|
332
364
|
- `test-video.h264`: raw H.264 Constrained Baseline video, 640×360 at 15 FPS, with a moving test pattern. Use this elementary stream when implementing H.264 NAL-unit parsing and RTP packetization.
|
|
333
|
-
- `test-audio.opus`: Ogg container with mono Opus audio at 48 kHz and 64 kbit/s, containing a
|
|
365
|
+
- `test-audio.opus`: Ogg container with mono Opus audio at 48 kHz and 64 kbit/s, containing a recorded voice track. Use the Opus packets for an audio RTP track; the Ogg container itself is not sent over WebRTC.
|
|
334
366
|
- `test-camera.mp4`: playable reference containing the same 640×360 H.264 test pattern and a mono 1 kHz AAC track. The werift test transfers the complete file over its SCTP data channel and verifies its integrity. This exercises binary file transport, not a WebRTC video RTP track.
|
|
335
367
|
- `camera-color-1920-1080.jpeg`: 1920×1080 simplified ffmpeg-generated color-rectangle snapshot.
|
|
336
368
|
- `camera-color-1280-720.jpeg`: 1280×720 simplified ffmpeg-generated color-rectangle snapshot.
|
package/dist/module.js
CHANGED
|
@@ -35,6 +35,7 @@ export class ExampleMatterbridgeCameraPlatform extends MatterbridgeDynamicPlatfo
|
|
|
35
35
|
this.config.videoGenerator ??= 'none';
|
|
36
36
|
this.config.videoResolution ??= 'auto';
|
|
37
37
|
this.config.videoBitrate ??= 1000;
|
|
38
|
+
this.config.audioGenerator ??= 'none';
|
|
38
39
|
this.config.animationInterval ??= 60;
|
|
39
40
|
this.config.debug ??= false;
|
|
40
41
|
this.config.unregisterOnShutdown ??= false;
|
|
@@ -45,12 +46,19 @@ export class ExampleMatterbridgeCameraPlatform extends MatterbridgeDynamicPlatfo
|
|
|
45
46
|
process.env.MATTERBRIDGE_CAMERA_VIDEO_SOURCE_DEVICE = this.config.videoSource;
|
|
46
47
|
process.env.MATTERBRIDGE_CAMERA_VIDEO_RESOLUTION = this.config.videoResolution;
|
|
47
48
|
process.env.MATTERBRIDGE_CAMERA_VIDEO_BITRATE = String(this.config.videoBitrate);
|
|
49
|
+
process.env.MATTERBRIDGE_CAMERA_AUDIO_SOURCE = this.config.audioGenerator;
|
|
50
|
+
if (this.config.audioSource === undefined)
|
|
51
|
+
delete process.env.MATTERBRIDGE_CAMERA_AUDIO_SOURCE_DEVICE;
|
|
52
|
+
else
|
|
53
|
+
process.env.MATTERBRIDGE_CAMERA_AUDIO_SOURCE_DEVICE = this.config.audioSource;
|
|
48
54
|
this.log.debug(`Platform ${this.config.name} config:\n${RESET}${inspect(this.config, { depth: 10, colors: true })}`);
|
|
49
55
|
this.log.debug(`Platform ${this.config.name} environment variables:\n${RESET}${inspect({
|
|
50
56
|
MATTERBRIDGE_CAMERA_VIDEO_SOURCE: process.env.MATTERBRIDGE_CAMERA_VIDEO_SOURCE,
|
|
51
57
|
MATTERBRIDGE_CAMERA_VIDEO_SOURCE_DEVICE: process.env.MATTERBRIDGE_CAMERA_VIDEO_SOURCE_DEVICE,
|
|
52
58
|
MATTERBRIDGE_CAMERA_VIDEO_RESOLUTION: process.env.MATTERBRIDGE_CAMERA_VIDEO_RESOLUTION,
|
|
53
59
|
MATTERBRIDGE_CAMERA_VIDEO_BITRATE: process.env.MATTERBRIDGE_CAMERA_VIDEO_BITRATE,
|
|
60
|
+
MATTERBRIDGE_CAMERA_AUDIO_SOURCE: process.env.MATTERBRIDGE_CAMERA_AUDIO_SOURCE,
|
|
61
|
+
MATTERBRIDGE_CAMERA_AUDIO_SOURCE_DEVICE: process.env.MATTERBRIDGE_CAMERA_AUDIO_SOURCE_DEVICE,
|
|
54
62
|
}, { depth: 10, colors: true })}`);
|
|
55
63
|
this.log.info(`Platform ${this.config.name} initialized successfully`);
|
|
56
64
|
}
|
|
@@ -5,6 +5,7 @@ import { access, readdir } from 'node:fs/promises';
|
|
|
5
5
|
import path from 'node:path';
|
|
6
6
|
import { fileURLToPath } from 'node:url';
|
|
7
7
|
import { AnsiLogger, MAGENTA } from 'matterbridge/logger';
|
|
8
|
+
import { fireAndForget, getErrorMessage } from 'matterbridge/utils';
|
|
8
9
|
import { RTCPeerConnection, RTCRtpCodecParameters, useH264, useOPUS, usePCMU, useVP8 } from 'werift';
|
|
9
10
|
import { navigator } from 'werift/nonstandard';
|
|
10
11
|
export class WeriftWebRtcSession {
|
|
@@ -19,6 +20,10 @@ export class WeriftWebRtcSession {
|
|
|
19
20
|
testAudioGenerator;
|
|
20
21
|
testAudioUdpDisposer;
|
|
21
22
|
testAudioAttached = false;
|
|
23
|
+
diagnosticsInterval;
|
|
24
|
+
static DIAGNOSTICS_INTERVAL_MS = 10_000;
|
|
25
|
+
lastDiagnosticsCounters;
|
|
26
|
+
dtlsTransportsWithStateLogging = new Set();
|
|
22
27
|
constructor(webRtcSessionId) {
|
|
23
28
|
this.webRtcSessionId = webRtcSessionId;
|
|
24
29
|
this.peerConnection = new RTCPeerConnection({ codecs: { audio: [useOPUS(), usePCMU()], video: [useH264(), useVP8()] } });
|
|
@@ -47,6 +52,7 @@ export class WeriftWebRtcSession {
|
|
|
47
52
|
this.log.info(`Peer connection state: ${state}`);
|
|
48
53
|
});
|
|
49
54
|
this.log.debug(`Created RTCPeerConnection with codecs: audio=[OPUS, PCMU], video=[H264, VP8] for session ${webRtcSessionId}`);
|
|
55
|
+
this.diagnosticsInterval = setInterval(() => void this.logDiagnosticsSnapshot(), WeriftWebRtcSession.DIAGNOSTICS_INTERVAL_MS);
|
|
50
56
|
WeriftWebRtcSession.activeSessions.add(this);
|
|
51
57
|
if (!WeriftWebRtcSession.exitHandlerRegistered) {
|
|
52
58
|
WeriftWebRtcSession.exitHandlerRegistered = true;
|
|
@@ -314,7 +320,7 @@ export class WeriftWebRtcSession {
|
|
|
314
320
|
this.log.debug(`Spawning ffmpeg: ${ffmpegCommand} ${ffmpegArgs.join(' ')}`);
|
|
315
321
|
const generator = spawn(ffmpegCommand, ffmpegArgs);
|
|
316
322
|
generator.once('error', (error) => {
|
|
317
|
-
this.log.warn(`Video generator failed: ${
|
|
323
|
+
this.log.warn(`Video generator failed: ${getErrorMessage(error)}`);
|
|
318
324
|
});
|
|
319
325
|
this.testVideoUdpDisposer = disposer;
|
|
320
326
|
this.testVideoGenerator = generator;
|
|
@@ -322,7 +328,7 @@ export class WeriftWebRtcSession {
|
|
|
322
328
|
this.log.info(`Attached ${videoInput.description} video track (ffmpeg=${ffmpegCommand}, codec=${selectedMimeType}, payloadType=${selectedPayloadType}, sourcePort=${udpPort})`);
|
|
323
329
|
}
|
|
324
330
|
catch (error) {
|
|
325
|
-
this.log.warn(`Failed to attach ${videoInput.description} video track: ${
|
|
331
|
+
this.log.warn(`Failed to attach ${videoInput.description} video track: ${getErrorMessage(error)}`);
|
|
326
332
|
}
|
|
327
333
|
}
|
|
328
334
|
preferVideoCodecOnTransceivers(mimeType) {
|
|
@@ -356,13 +362,61 @@ export class WeriftWebRtcSession {
|
|
|
356
362
|
}
|
|
357
363
|
}
|
|
358
364
|
static TEST_VOICE_PATH = fileURLToPath(new URL('../../assets/test-voice.opus', import.meta.url));
|
|
359
|
-
|
|
365
|
+
getConfiguredAudioSource() {
|
|
366
|
+
const source = process.env.MATTERBRIDGE_CAMERA_AUDIO_SOURCE?.trim().toLowerCase() ?? 'none';
|
|
367
|
+
switch (source) {
|
|
368
|
+
case 'none':
|
|
369
|
+
case 'test':
|
|
370
|
+
case 'microphone':
|
|
371
|
+
case 'rtsp':
|
|
372
|
+
return source;
|
|
373
|
+
default:
|
|
374
|
+
this.log.warn(`Unsupported MATTERBRIDGE_CAMERA_AUDIO_SOURCE "${source}" (supported: test, microphone, rtsp, none); falling back to none`);
|
|
375
|
+
return 'none';
|
|
376
|
+
}
|
|
377
|
+
}
|
|
378
|
+
buildFfmpegAudioInputArgs(audioSource) {
|
|
379
|
+
const testVoiceInput = {
|
|
380
|
+
args: ['-re', '-stream_loop', '-1', '-i', WeriftWebRtcSession.TEST_VOICE_PATH],
|
|
381
|
+
description: 'recorded test-voice clip',
|
|
382
|
+
volumeFilter: 'volume=6dB',
|
|
383
|
+
};
|
|
384
|
+
if (audioSource === 'test')
|
|
385
|
+
return testVoiceInput;
|
|
386
|
+
const device = process.env.MATTERBRIDGE_CAMERA_AUDIO_SOURCE_DEVICE;
|
|
387
|
+
if (!device) {
|
|
388
|
+
this.log.warn(`MATTERBRIDGE_CAMERA_AUDIO_SOURCE=${audioSource} requires MATTERBRIDGE_CAMERA_AUDIO_SOURCE_DEVICE to be set; falling back to the recorded test-voice clip`);
|
|
389
|
+
return testVoiceInput;
|
|
390
|
+
}
|
|
391
|
+
if (audioSource === 'rtsp') {
|
|
392
|
+
const description = `RTSP camera audio (${device})`;
|
|
393
|
+
this.log.debug(`RTSP audio capture params: url=${device}, description="${description}"`);
|
|
394
|
+
return { args: ['-rtsp_transport', 'tcp', '-i', device], description };
|
|
395
|
+
}
|
|
396
|
+
const description = `local microphone (${device})`;
|
|
397
|
+
this.log.debug(`Microphone capture params: device=${device}, description="${description}"`);
|
|
398
|
+
switch (process.platform) {
|
|
399
|
+
case 'linux':
|
|
400
|
+
return { args: ['-f', 'alsa', '-i', device], description };
|
|
401
|
+
case 'darwin':
|
|
402
|
+
return { args: ['-f', 'avfoundation', '-i', `:${device}`], description };
|
|
403
|
+
case 'win32':
|
|
404
|
+
return { args: ['-f', 'dshow', '-i', `audio=${device}`], description };
|
|
405
|
+
default:
|
|
406
|
+
this.log.warn(`Microphone capture via ffmpeg is not supported on platform "${process.platform}"; falling back to the recorded test-voice clip`);
|
|
407
|
+
return testVoiceInput;
|
|
408
|
+
}
|
|
409
|
+
}
|
|
410
|
+
async generateAudioTrack(codec) {
|
|
360
411
|
if (this.testAudioAttached)
|
|
361
412
|
return;
|
|
362
|
-
|
|
363
|
-
|
|
413
|
+
const audioSource = this.getConfiguredAudioSource();
|
|
414
|
+
if (audioSource === 'none') {
|
|
415
|
+
this.log.debug('Audio injection disabled by MATTERBRIDGE_CAMERA_AUDIO_SOURCE=none');
|
|
364
416
|
return;
|
|
365
417
|
}
|
|
418
|
+
const audioInput = this.buildFfmpegAudioInputArgs(audioSource);
|
|
419
|
+
this.log.debug(`Attempting to attach ${audioInput.description} audio track`);
|
|
366
420
|
const ffmpegCommand = await this.resolveCommand('ffmpeg');
|
|
367
421
|
if (!ffmpegCommand) {
|
|
368
422
|
this.log.warn('Cannot inject audio stream: missing dependency ffmpeg');
|
|
@@ -383,14 +437,9 @@ export class WeriftWebRtcSession {
|
|
|
383
437
|
'-hide_banner',
|
|
384
438
|
'-loglevel',
|
|
385
439
|
'error',
|
|
386
|
-
|
|
387
|
-
'-stream_loop',
|
|
388
|
-
'-1',
|
|
389
|
-
'-i',
|
|
390
|
-
WeriftWebRtcSession.TEST_VOICE_PATH,
|
|
440
|
+
...audioInput.args,
|
|
391
441
|
'-vn',
|
|
392
|
-
'-af',
|
|
393
|
-
'volume=6dB',
|
|
442
|
+
...(audioInput.volumeFilter ? ['-af', audioInput.volumeFilter] : []),
|
|
394
443
|
'-c:a',
|
|
395
444
|
'libopus',
|
|
396
445
|
'-b:a',
|
|
@@ -408,15 +457,15 @@ export class WeriftWebRtcSession {
|
|
|
408
457
|
this.log.debug(`Spawning ffmpeg: ${ffmpegCommand} ${ffmpegArgs.join(' ')}`);
|
|
409
458
|
const generator = spawn(ffmpegCommand, ffmpegArgs);
|
|
410
459
|
generator.once('error', (error) => {
|
|
411
|
-
this.log.warn(`Audio generator failed: ${
|
|
460
|
+
this.log.warn(`Audio generator failed: ${getErrorMessage(error)}`);
|
|
412
461
|
});
|
|
413
462
|
this.testAudioUdpDisposer = disposer;
|
|
414
463
|
this.testAudioGenerator = generator;
|
|
415
464
|
this.testAudioAttached = true;
|
|
416
|
-
this.log.info(`Attached
|
|
465
|
+
this.log.info(`Attached ${audioInput.description} audio track (ffmpeg=${ffmpegCommand}, codec=${selectedMimeType}, payloadType=${selectedPayloadType}, sourcePort=${udpPort})`);
|
|
417
466
|
}
|
|
418
467
|
catch (error) {
|
|
419
|
-
this.log.warn(`Failed to attach
|
|
468
|
+
this.log.warn(`Failed to attach ${audioInput.description} audio track: ${getErrorMessage(error)}`);
|
|
420
469
|
}
|
|
421
470
|
}
|
|
422
471
|
cleanupTestVideoArtifacts() {
|
|
@@ -457,6 +506,7 @@ export class WeriftWebRtcSession {
|
|
|
457
506
|
await this.peerConnection.setLocalDescription(offer);
|
|
458
507
|
const sdp = this.peerConnection.localDescription.sdp;
|
|
459
508
|
this.log.info(`Created local SDP offer (${this.summarizeSdp(sdp)})`);
|
|
509
|
+
this.logDtlsTransportStateChanges();
|
|
460
510
|
return sdp;
|
|
461
511
|
}
|
|
462
512
|
async createAnswer(offerSdp, videoResolution) {
|
|
@@ -480,7 +530,7 @@ export class WeriftWebRtcSession {
|
|
|
480
530
|
const preferredAudioCodec = this.getPreferredInjectableAudioCodec();
|
|
481
531
|
if (preferredAudioCodec) {
|
|
482
532
|
this.preferAudioCodecOnTransceivers(preferredAudioCodec.mimeType.toLowerCase());
|
|
483
|
-
await this.
|
|
533
|
+
await this.generateAudioTrack(preferredAudioCodec);
|
|
484
534
|
}
|
|
485
535
|
else {
|
|
486
536
|
this.log.warn('No injectable audio codec available on negotiated transceivers (supported: Opus)');
|
|
@@ -493,19 +543,69 @@ export class WeriftWebRtcSession {
|
|
|
493
543
|
await this.peerConnection.setLocalDescription(answer);
|
|
494
544
|
const sdp = this.peerConnection.localDescription.sdp;
|
|
495
545
|
this.log.info(`Created local SDP answer (${this.summarizeSdp(sdp)})`);
|
|
546
|
+
this.logDtlsTransportStateChanges();
|
|
496
547
|
return sdp;
|
|
497
548
|
}
|
|
498
549
|
async applyAnswer(answerSdp) {
|
|
499
550
|
this.log.debug(`ApplyAnswer requested (${this.summarizeSdp(answerSdp)})`);
|
|
500
551
|
await this.peerConnection.setRemoteDescription({ type: 'answer', sdp: answerSdp });
|
|
501
552
|
this.log.info(`Remote SDP answer applied (signalingState=${this.peerConnection.signalingState})`);
|
|
553
|
+
this.logDtlsTransportStateChanges();
|
|
502
554
|
}
|
|
503
555
|
async addIceCandidate(candidate, sdpMid, sdpMLineIndex) {
|
|
504
556
|
this.log.debug(`Applying ICE candidate (mid=${sdpMid ?? 'null'}, mLine=${sdpMLineIndex ?? 'null'}, endOfCandidates=${candidate.trim() === ''})`);
|
|
505
557
|
await this.peerConnection.addIceCandidate({ candidate, sdpMid: sdpMid ?? undefined, sdpMLineIndex: sdpMLineIndex ?? undefined });
|
|
506
558
|
}
|
|
559
|
+
closing = false;
|
|
560
|
+
logDtlsTransportStateChanges() {
|
|
561
|
+
for (const transport of this.peerConnection.dtlsTransports) {
|
|
562
|
+
if (this.dtlsTransportsWithStateLogging.has(transport))
|
|
563
|
+
continue;
|
|
564
|
+
this.dtlsTransportsWithStateLogging.add(transport);
|
|
565
|
+
transport.onStateChange.subscribe((state) => {
|
|
566
|
+
this.log.info(`DTLS transport state: ${state}`);
|
|
567
|
+
if ((state === 'closed' || state === 'failed') && !this.closing) {
|
|
568
|
+
this.log.info(`Closing session ${this.webRtcSessionId}: DTLS transport reached ${state}`);
|
|
569
|
+
fireAndForget(this.close(), this.log, `Failed to auto-close session ${this.webRtcSessionId} after DTLS transport reached ${state}`);
|
|
570
|
+
}
|
|
571
|
+
});
|
|
572
|
+
}
|
|
573
|
+
}
|
|
574
|
+
async logDiagnosticsSnapshot() {
|
|
575
|
+
const iceStates = this.peerConnection.iceTransports.map((transport) => transport.state).join(',') || 'none';
|
|
576
|
+
const dtlsStates = this.peerConnection.dtlsTransports.map((transport) => transport.state).join(',') || 'none';
|
|
577
|
+
const stats = [...(await this.peerConnection.getStats()).values()];
|
|
578
|
+
const nominatedPair = stats
|
|
579
|
+
.filter((stat) => stat.type === 'candidate-pair')
|
|
580
|
+
.map((stat) => stat)
|
|
581
|
+
.find((pair) => pair.nominated === true);
|
|
582
|
+
const outboundRtp = stats.filter((stat) => stat.type === 'outbound-rtp').map((stat) => stat);
|
|
583
|
+
const candidatePairPacketsSent = nominatedPair?.packetsSent ?? 0;
|
|
584
|
+
const candidatePairPacketsReceived = nominatedPair?.packetsReceived ?? 0;
|
|
585
|
+
const outboundRtpPacketsSent = outboundRtp.reduce((sum, stat) => sum + (stat.packetsSent ?? 0), 0);
|
|
586
|
+
const previous = this.lastDiagnosticsCounters;
|
|
587
|
+
this.lastDiagnosticsCounters = { candidatePairPacketsSent, candidatePairPacketsReceived, outboundRtpPacketsSent };
|
|
588
|
+
const delta = (current, previousValue) => previousValue === undefined ? 'n/a' : `${current - previousValue >= 0 ? '+' : ''}${current - previousValue}`;
|
|
589
|
+
const outboundRtpSummary = outboundRtp.map((stat) => `${stat.kind ?? 'unknown'}:packetsSent=${stat.packetsSent ?? 0}`).join(', ') || 'none';
|
|
590
|
+
this.log.debug([
|
|
591
|
+
'Diagnostics:',
|
|
592
|
+
`- connectionState=${this.peerConnection.connectionState}`,
|
|
593
|
+
`- iceConnectionState=${this.peerConnection.iceConnectionState}`,
|
|
594
|
+
`- iceGatheringState=${this.peerConnection.iceGatheringState}`,
|
|
595
|
+
`- signalingState=${this.peerConnection.signalingState}`,
|
|
596
|
+
`- iceTransportStates=[${iceStates}]`,
|
|
597
|
+
`- dtlsTransportStates=[${dtlsStates}]`,
|
|
598
|
+
`- nominatedPair(packetsSent=${candidatePairPacketsSent} Δ${delta(candidatePairPacketsSent, previous?.candidatePairPacketsSent)}, packetsReceived=${candidatePairPacketsReceived} Δ${delta(candidatePairPacketsReceived, previous?.candidatePairPacketsReceived)})`,
|
|
599
|
+
`- outboundRtp[${outboundRtpSummary}] (totalPacketsSent Δ${delta(outboundRtpPacketsSent, previous?.outboundRtpPacketsSent)})`,
|
|
600
|
+
].join('\n'));
|
|
601
|
+
}
|
|
507
602
|
async close() {
|
|
603
|
+
if (this.closing)
|
|
604
|
+
return;
|
|
605
|
+
this.closing = true;
|
|
508
606
|
this.log.debug('Closing RTCPeerConnection');
|
|
607
|
+
clearInterval(this.diagnosticsInterval);
|
|
608
|
+
this.diagnosticsInterval = undefined;
|
|
509
609
|
await this.peerConnection.close();
|
|
510
610
|
this.cleanupTestVideoArtifacts();
|
|
511
611
|
this.cleanupTestAudioArtifacts();
|
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
"title": "Matterbridge example camera plugin",
|
|
3
3
|
"description": "matterbridge-example-camera v. 1.0.0 by https://github.com/Luligu",
|
|
4
4
|
"type": "object",
|
|
5
|
-
"required": ["videoGenerator", "videoResolution"],
|
|
5
|
+
"required": ["videoGenerator", "videoResolution", "audioGenerator"],
|
|
6
6
|
"properties": {
|
|
7
7
|
"name": {
|
|
8
8
|
"description": "Plugin Name",
|
|
@@ -67,6 +67,18 @@
|
|
|
67
67
|
"type": "number",
|
|
68
68
|
"default": 1000
|
|
69
69
|
},
|
|
70
|
+
"audioGenerator": {
|
|
71
|
+
"title": "Audio Generator",
|
|
72
|
+
"description": "Select the WebRTC audio generator: none disables track injection (only the audio transceiver is negotiated), test injects a recorded test-voice clip, looped, e.g. for an Intercom's Listen live view, microphone captures from the configured local microphone, and rtsp pulls the audio from the RTSP url configured in Audio Source.",
|
|
73
|
+
"type": "string",
|
|
74
|
+
"enum": ["none", "test", "microphone", "rtsp"],
|
|
75
|
+
"default": "none"
|
|
76
|
+
},
|
|
77
|
+
"audioSource": {
|
|
78
|
+
"title": "Audio Source",
|
|
79
|
+
"description": "OS-specific ffmpeg microphone identifier, such as hw:0,0 on Linux (alsa), 0 on macOS (avfoundation audio index), or Microphone Array on Windows (dshow). When Audio Generator is rtsp, this holds the RTSP url instead, e.g. rtsp://user:password@host:554/path.",
|
|
80
|
+
"type": "string"
|
|
81
|
+
},
|
|
70
82
|
"debug": {
|
|
71
83
|
"title": "Enable Debug",
|
|
72
84
|
"description": "Enable the debug for the plugin (development only)",
|
package/npm-shrinkwrap.json
CHANGED
|
@@ -1,12 +1,12 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "matterbridge-example-camera",
|
|
3
|
-
"version": "0.0.8-dev-20260726-
|
|
3
|
+
"version": "0.0.8-dev-20260726-61cad4c",
|
|
4
4
|
"lockfileVersion": 3,
|
|
5
5
|
"requires": true,
|
|
6
6
|
"packages": {
|
|
7
7
|
"": {
|
|
8
8
|
"name": "matterbridge-example-camera",
|
|
9
|
-
"version": "0.0.8-dev-20260726-
|
|
9
|
+
"version": "0.0.8-dev-20260726-61cad4c",
|
|
10
10
|
"license": "Apache-2.0",
|
|
11
11
|
"dependencies": {
|
|
12
12
|
"node-ansi-logger": "3.3.0",
|
package/package.json
CHANGED