matterbridge-example-camera 0.0.8-dev-20260726-dffb00d → 0.0.8-dev-20260726-61cad4c

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -40,6 +40,19 @@ If you like this project and find it useful, please consider giving it a star on
40
40
  - [tests]: Add `rtsp` video source and `videoResolution` precedence (`auto` vs. fixed, webcam and rtsp) coverage in `vitest/webrtc/weriftSession.test.ts`; extend `vitest/module.test.ts` with the `videoGenerator` normalization test and a test for the `videoResolution` default of `auto`.
41
41
  - [webrtc]: Add `WeriftWebRtcSession.closeAll()`, a static helper that closes every active session (peer connection plus any injected ffmpeg generators). `onShutdown` now calls it so a graceful platform shutdown cleans up leftover WebRTC sessions itself instead of relying solely on the `process.on('exit', ...)` fallback.
42
42
  - [tests]: Add `closeAll` coverage in `vitest/webrtc/weriftSession.test.ts`, covering closing all active sessions and the no-active-sessions case.
43
+ - [platform]: Add an `audioGenerator` config property (`none`/`test`, defaulting to `none`) replacing the undocumented `MATTERBRIDGE_CAMERA_DISABLE_TEST_AUDIO=1` env var toggle, mirroring `videoGenerator`'s pattern via the new `MATTERBRIDGE_CAMERA_AUDIO_SOURCE` env var. Test-voice audio injection is now opt-in instead of on by default.
44
+ - [docs]: Document `audioGenerator` in the README and schema.
45
+ - [tests]: Extend `vitest/module.test.ts` with `audioGenerator` normalization/apply coverage; update `vitest/webrtc/weriftSession.test.ts` for the new `MATTERBRIDGE_CAMERA_AUDIO_SOURCE` gate.
46
+ - [webrtc]: Add a periodic diagnostics log (`WeriftWebRtcSession.logDiagnosticsSnapshot`, every 10s) reporting `connectionState`, `iceConnectionState`, `iceGatheringState`, `signalingState`, per-transport ICE/DTLS states, and `getStats()`-derived packet-flow counters (nominated candidate-pair and outbound-rtp packets, with deltas since the previous tick). Needed because werift's ICE consent-freshness check (RFC 7675) can latch `iceConnectionState` at `disconnected` forever after a single missed keepalive, even on a healthy, actively-streaming session — the packet-flow deltas are unaffected by that bug and are a reliable "is media still moving" signal. A dedicated log line also fires immediately on any DTLS transport state change instead of waiting for the next tick.
47
+ - [webrtc]: Auto-close a `WeriftWebRtcSession` (peer connection plus any injected ffmpeg generators) when one of its DTLS transports reaches `closed` or `failed` on its own — a real, one-way teardown signal from the peer, unlike `iceConnectionState`'s `disconnected`, which must never trigger a close by itself. Fixes an orphaned-session leak: a controller that abandons a live view without ever sending `EndSession` (e.g. just closing the window) previously left the peer connection and its ffmpeg processes running indefinitely, piling up on every reconnect.
48
+ - [tests]: Add DTLS-triggered auto-close coverage in `vitest/webrtc/weriftSession.test.ts` (`closed`, `failed`, and a regression test ensuring a normal `close()` doesn't re-enter itself via the same DTLS state change).
49
+ - [webrtc]: Add `microphone` and `rtsp` options to `audioGenerator`, mirroring `videoGenerator`'s pattern: `microphone` captures from a local capture device via `ffmpeg -f alsa/avfoundation/dshow`, and `rtsp` pulls just the audio from a real RTSP camera stream (dropping any video) via `ffmpeg -rtsp_transport tcp -i <url> -vn`. Add the corresponding `audioSource` config property and `MATTERBRIDGE_CAMERA_AUDIO_SOURCE_DEVICE` env var (device identifier for `microphone`, or the RTSP url for `rtsp`), mirroring `videoSource`/`MATTERBRIDGE_CAMERA_VIDEO_SOURCE_DEVICE`. The `-af volume=6dB` boost is now only applied to the `test` clip (recorded quietly), not to `microphone`/`rtsp` capture.
50
+ - [docs]: Document the `microphone` and `rtsp` audio generators and the new `audioSource` property in the README and schema, with example configurations.
51
+ - [tests]: Extend `vitest/module.test.ts` and `vitest/webrtc/weriftSession.test.ts` with `microphone`/`rtsp` audio source coverage.
52
+
53
+ ### Changed
54
+
55
+ - [webrtc]: Replace inline `error instanceof Error ? error.message : String(error)` error-message extraction with `getErrorMessage` from `matterbridge/utils` throughout `weriftSession.ts`; use `fireAndForget` for the auto-close's fire-and-forget `close()` call, matching the pattern already used elsewhere in the plugin.
43
56
 
44
57
  ### Fixed
45
58
 
package/README.md CHANGED
@@ -186,9 +186,9 @@ So, to let either Intercom 1 or Intercom 2 start a call, on the fabric they shar
186
186
 
187
187
  With both directions in place, either Intercom can call the other; a call initiated the other way only needs its own binding/ACL pair, already covered above since both were set up symmetrically.
188
188
 
189
- ## WebRTC video injection
189
+ ## WebRTC video and audio injection
190
190
 
191
- `WeriftWebRtcSession` (see `src/webrtc/weriftSession.ts`) wraps a real werift `RTCPeerConnection` for each WebRtcTransportProvider session (see `MatterbridgeWebRtcTransportProviderServer` in `src/behaviors/webRtcTransportProviderServer.ts`), so the session's SDP offer/answer and ICE candidates are handled by a real WebRTC peer connection instead of being just recorded. It can also inject a real ffmpeg-generated video track into the negotiated connection, so the end-to-end media path can be validated without a real camera capture pipeline.
191
+ `WeriftWebRtcSession` (see `src/webrtc/weriftSession.ts`) wraps a real werift `RTCPeerConnection` for each WebRtcTransportProvider session (see `MatterbridgeWebRtcTransportProviderServer` in `src/behaviors/webRtcTransportProviderServer.ts`), so the session's SDP offer/answer and ICE candidates are handled by a real WebRTC peer connection instead of being just recorded. It can also inject a real ffmpeg-generated video and/or audio track into the negotiated connection, so the end-to-end media path can be validated without a real camera/microphone capture pipeline.
192
192
 
193
193
  The platform configuration controls WebRTC video injection with these properties:
194
194
 
@@ -200,11 +200,43 @@ The platform configuration controls WebRTC video injection with these properties
200
200
 
201
201
  - `videoBitrate` is required and accepts a number (the target encoder bitrate in kbps). It defaults to `1000`. It applies to the `webcam` and `rtsp` sources only; the `test` pattern is always encoded at a fixed 1000 kbps regardless of this setting. Since the encoder always uses `-preset ultrafast -tune zerolatency` for low-latency live view (lower compression efficiency than slower presets), suggested values are higher than typical streaming guides: ~800 for `640x480`, ~2000 for `1280x720`, ~2500 for `1920x1080`.
202
202
 
203
+ The platform configuration controls WebRTC audio injection with these properties:
204
+
205
+ - `audioGenerator` is required and accepts `none`, `test`, `microphone`, or `rtsp`. It defaults to `none`, which negotiates the audio transceiver (e.g. for an Intercom's "Listen" live view) without attaching a track. `test` injects a recorded test-voice clip, looped, so the end-to-end audio path can be validated without a real microphone capture pipeline. `microphone` captures from the configured local microphone, and `rtsp` pulls just the audio from the RTSP url configured in `audioSource` (any video in the stream is dropped).
206
+
207
+ - `audioSource` is optional and has no default. For the `microphone` generator it contains the OS-specific ffmpeg device identifier — e.g. an ALSA device such as `hw:0,0` on Linux, an avfoundation audio index such as `0` on macOS, or a device name such as `Microphone Array` on Windows (dshow). For the `rtsp` generator it instead holds the RTSP url, e.g. `rtsp://user:password@host:554/path` (it can be a different camera/url than `videoSource`). Selecting the `microphone` or `rtsp` generator without this property falls back to the test-voice clip with a warning.
208
+
209
+ Use ffmpeg itself to list the [available capture devices](https://trac.ffmpeg.org/wiki/Capture/Webcam) and find the right value for `audioSource`:
210
+
211
+ - Linux (alsa): `arecord -l`, or `ffmpeg -f alsa -i default` to try the default device.
212
+ - macOS (avfoundation): `ffmpeg -f avfoundation -list_devices true -i dummy` — audio devices are listed with their index under "AVFoundation audio devices", e.g. `[0] MacBook Pro Microphone`; use that index (e.g. `0`) as the device value.
213
+ - Windows (dshow): `ffmpeg -f dshow -list_devices true -i dummy` — audio devices are listed by name under "DirectShow audio devices", e.g. `"Microphone Array"`; use that exact name as the device value.
214
+
215
+ Example configuration for a real Linux microphone:
216
+
217
+ ```json
218
+ {
219
+ "audioGenerator": "microphone",
220
+ "audioSource": "hw:0,0"
221
+ }
222
+ ```
223
+
224
+ Example, pulling the audio from the same RTSP camera used for `videoSource`:
225
+
226
+ ```json
227
+ {
228
+ "videoGenerator": "rtsp",
229
+ "videoSource": "rtsp://admin:password@192.168.1.100:554/ch1/main",
230
+ "audioGenerator": "rtsp",
231
+ "audioSource": "rtsp://admin:password@192.168.1.100:554/ch1/main"
232
+ }
233
+ ```
234
+
203
235
  With `videoResolution: "auto"`, a real client's resolution/quality picker (e.g. in Home Assistant) drives the capture/output resolution: it allocates a video stream with `CameraAvStreamManagement.VideoStreamAllocate` before soliciting or providing a WebRTC offer, and `MatterbridgeWebRtcTransportProviderServer` looks up that stream's `maxResolution` to select the resolution for the session. For `webcam` this is the capture resolution passed to ffmpeg; for `rtsp` the camera keeps streaming at its own native resolution and ffmpeg scales it to match.
204
236
 
205
- Requires `ffmpeg` to be installed. The resolver checks the system command and common installation directories on Linux, macOS, and Windows.
237
+ Requires [`ffmpeg`](https://ffmpeg.org/) to be installed. The resolver checks the system command and common installation directories on Linux, macOS, and Windows.
206
238
 
207
- Use ffmpeg itself to list the available capture devices and find the right value for `videoSource`:
239
+ Use ffmpeg itself to list the [available capture devices](https://trac.ffmpeg.org/wiki/Capture/Webcam) and find the right value for `videoSource`:
208
240
 
209
241
  - Linux (v4l2): `v4l2-ctl --list-devices` (from `v4l-utils`), or `ls /dev/video*`.
210
242
  - macOS (avfoundation): `ffmpeg -f avfoundation -list_devices true -i dummy` — video devices are listed with their index, e.g. `[0] FaceTime HD Camera`; use that index (e.g. `0`) as the device value.
@@ -242,7 +274,7 @@ Finding your camera's RTSP url for `videoSource` when `videoGenerator` is `rtsp`
242
274
  - Dahua: `/cam/realmonitor?channel=1&subtype=0`
243
275
  - Reolink: `/h264Preview_01_main`
244
276
 
245
- Verify the url works standalone before adding it to the config, e.g. `ffprobe rtsp://admin:password@192.168.1.100:554/ch1/main`, or open it in VLC via "Open Network Stream". Note that the url (with credentials in plaintext) ends up stored in Matterbridge's plugin config on disk, like any other password-bearing config value.
277
+ Verify the url works standalone before adding it to the config, e.g. `ffprobe rtsp://admin:password@192.168.1.100:554/ch1/main` or `ffplay rtsp://admin:password@192.168.1.100:554/ch1/main`, or open it in VLC via "Open Network Stream". Note that the url (with credentials in plaintext) ends up stored in Matterbridge's plugin config on disk, like any other password-bearing config value.
246
278
 
247
279
  Example, pulling from a real RTSP camera, scaled to a fixed 1280x720 and re-encoded at `videoBitrate` (the camera's own frame rate is kept):
248
280
 
@@ -330,7 +362,7 @@ npm run test -- vitest/werift.test.ts
330
362
  The `assets` directory contains deterministic three-second media fixtures for extending the werift test to real media tracks:
331
363
 
332
364
  - `test-video.h264`: raw H.264 Constrained Baseline video, 640×360 at 15 FPS, with a moving test pattern. Use this elementary stream when implementing H.264 NAL-unit parsing and RTP packetization.
333
- - `test-audio.opus`: Ogg container with mono Opus audio at 48 kHz and 64 kbit/s, containing a 1 kHz test tone. Use the Opus packets for an audio RTP track; the Ogg container itself is not sent over WebRTC.
365
+ - `test-audio.opus`: Ogg container with mono Opus audio at 48 kHz and 64 kbit/s, containing a recorded voice track. Use the Opus packets for an audio RTP track; the Ogg container itself is not sent over WebRTC.
334
366
  - `test-camera.mp4`: playable reference containing the same 640×360 H.264 test pattern and a mono 1 kHz AAC track. The werift test transfers the complete file over its SCTP data channel and verifies its integrity. This exercises binary file transport, not a WebRTC video RTP track.
335
367
  - `camera-color-1920-1080.jpeg`: 1920×1080 simplified ffmpeg-generated color-rectangle snapshot.
336
368
  - `camera-color-1280-720.jpeg`: 1280×720 simplified ffmpeg-generated color-rectangle snapshot.
package/dist/module.js CHANGED
@@ -35,6 +35,7 @@ export class ExampleMatterbridgeCameraPlatform extends MatterbridgeDynamicPlatfo
35
35
  this.config.videoGenerator ??= 'none';
36
36
  this.config.videoResolution ??= 'auto';
37
37
  this.config.videoBitrate ??= 1000;
38
+ this.config.audioGenerator ??= 'none';
38
39
  this.config.animationInterval ??= 60;
39
40
  this.config.debug ??= false;
40
41
  this.config.unregisterOnShutdown ??= false;
@@ -45,12 +46,19 @@ export class ExampleMatterbridgeCameraPlatform extends MatterbridgeDynamicPlatfo
45
46
  process.env.MATTERBRIDGE_CAMERA_VIDEO_SOURCE_DEVICE = this.config.videoSource;
46
47
  process.env.MATTERBRIDGE_CAMERA_VIDEO_RESOLUTION = this.config.videoResolution;
47
48
  process.env.MATTERBRIDGE_CAMERA_VIDEO_BITRATE = String(this.config.videoBitrate);
49
+ process.env.MATTERBRIDGE_CAMERA_AUDIO_SOURCE = this.config.audioGenerator;
50
+ if (this.config.audioSource === undefined)
51
+ delete process.env.MATTERBRIDGE_CAMERA_AUDIO_SOURCE_DEVICE;
52
+ else
53
+ process.env.MATTERBRIDGE_CAMERA_AUDIO_SOURCE_DEVICE = this.config.audioSource;
48
54
  this.log.debug(`Platform ${this.config.name} config:\n${RESET}${inspect(this.config, { depth: 10, colors: true })}`);
49
55
  this.log.debug(`Platform ${this.config.name} environment variables:\n${RESET}${inspect({
50
56
  MATTERBRIDGE_CAMERA_VIDEO_SOURCE: process.env.MATTERBRIDGE_CAMERA_VIDEO_SOURCE,
51
57
  MATTERBRIDGE_CAMERA_VIDEO_SOURCE_DEVICE: process.env.MATTERBRIDGE_CAMERA_VIDEO_SOURCE_DEVICE,
52
58
  MATTERBRIDGE_CAMERA_VIDEO_RESOLUTION: process.env.MATTERBRIDGE_CAMERA_VIDEO_RESOLUTION,
53
59
  MATTERBRIDGE_CAMERA_VIDEO_BITRATE: process.env.MATTERBRIDGE_CAMERA_VIDEO_BITRATE,
60
+ MATTERBRIDGE_CAMERA_AUDIO_SOURCE: process.env.MATTERBRIDGE_CAMERA_AUDIO_SOURCE,
61
+ MATTERBRIDGE_CAMERA_AUDIO_SOURCE_DEVICE: process.env.MATTERBRIDGE_CAMERA_AUDIO_SOURCE_DEVICE,
54
62
  }, { depth: 10, colors: true })}`);
55
63
  this.log.info(`Platform ${this.config.name} initialized successfully`);
56
64
  }
@@ -5,6 +5,7 @@ import { access, readdir } from 'node:fs/promises';
5
5
  import path from 'node:path';
6
6
  import { fileURLToPath } from 'node:url';
7
7
  import { AnsiLogger, MAGENTA } from 'matterbridge/logger';
8
+ import { fireAndForget, getErrorMessage } from 'matterbridge/utils';
8
9
  import { RTCPeerConnection, RTCRtpCodecParameters, useH264, useOPUS, usePCMU, useVP8 } from 'werift';
9
10
  import { navigator } from 'werift/nonstandard';
10
11
  export class WeriftWebRtcSession {
@@ -19,6 +20,10 @@ export class WeriftWebRtcSession {
19
20
  testAudioGenerator;
20
21
  testAudioUdpDisposer;
21
22
  testAudioAttached = false;
23
+ diagnosticsInterval;
24
+ static DIAGNOSTICS_INTERVAL_MS = 10_000;
25
+ lastDiagnosticsCounters;
26
+ dtlsTransportsWithStateLogging = new Set();
22
27
  constructor(webRtcSessionId) {
23
28
  this.webRtcSessionId = webRtcSessionId;
24
29
  this.peerConnection = new RTCPeerConnection({ codecs: { audio: [useOPUS(), usePCMU()], video: [useH264(), useVP8()] } });
@@ -47,6 +52,7 @@ export class WeriftWebRtcSession {
47
52
  this.log.info(`Peer connection state: ${state}`);
48
53
  });
49
54
  this.log.debug(`Created RTCPeerConnection with codecs: audio=[OPUS, PCMU], video=[H264, VP8] for session ${webRtcSessionId}`);
55
+ this.diagnosticsInterval = setInterval(() => void this.logDiagnosticsSnapshot(), WeriftWebRtcSession.DIAGNOSTICS_INTERVAL_MS);
50
56
  WeriftWebRtcSession.activeSessions.add(this);
51
57
  if (!WeriftWebRtcSession.exitHandlerRegistered) {
52
58
  WeriftWebRtcSession.exitHandlerRegistered = true;
@@ -314,7 +320,7 @@ export class WeriftWebRtcSession {
314
320
  this.log.debug(`Spawning ffmpeg: ${ffmpegCommand} ${ffmpegArgs.join(' ')}`);
315
321
  const generator = spawn(ffmpegCommand, ffmpegArgs);
316
322
  generator.once('error', (error) => {
317
- this.log.warn(`Video generator failed: ${error instanceof Error ? error.message : String(error)}`);
323
+ this.log.warn(`Video generator failed: ${getErrorMessage(error)}`);
318
324
  });
319
325
  this.testVideoUdpDisposer = disposer;
320
326
  this.testVideoGenerator = generator;
@@ -322,7 +328,7 @@ export class WeriftWebRtcSession {
322
328
  this.log.info(`Attached ${videoInput.description} video track (ffmpeg=${ffmpegCommand}, codec=${selectedMimeType}, payloadType=${selectedPayloadType}, sourcePort=${udpPort})`);
323
329
  }
324
330
  catch (error) {
325
- this.log.warn(`Failed to attach ${videoInput.description} video track: ${error instanceof Error ? error.message : String(error)}`);
331
+ this.log.warn(`Failed to attach ${videoInput.description} video track: ${getErrorMessage(error)}`);
326
332
  }
327
333
  }
328
334
  preferVideoCodecOnTransceivers(mimeType) {
@@ -356,13 +362,61 @@ export class WeriftWebRtcSession {
356
362
  }
357
363
  }
358
364
  static TEST_VOICE_PATH = fileURLToPath(new URL('../../assets/test-voice.opus', import.meta.url));
359
- async ensureTestAudioTrack(codec) {
365
+ getConfiguredAudioSource() {
366
+ const source = process.env.MATTERBRIDGE_CAMERA_AUDIO_SOURCE?.trim().toLowerCase() ?? 'none';
367
+ switch (source) {
368
+ case 'none':
369
+ case 'test':
370
+ case 'microphone':
371
+ case 'rtsp':
372
+ return source;
373
+ default:
374
+ this.log.warn(`Unsupported MATTERBRIDGE_CAMERA_AUDIO_SOURCE "${source}" (supported: test, microphone, rtsp, none); falling back to none`);
375
+ return 'none';
376
+ }
377
+ }
378
+ buildFfmpegAudioInputArgs(audioSource) {
379
+ const testVoiceInput = {
380
+ args: ['-re', '-stream_loop', '-1', '-i', WeriftWebRtcSession.TEST_VOICE_PATH],
381
+ description: 'recorded test-voice clip',
382
+ volumeFilter: 'volume=6dB',
383
+ };
384
+ if (audioSource === 'test')
385
+ return testVoiceInput;
386
+ const device = process.env.MATTERBRIDGE_CAMERA_AUDIO_SOURCE_DEVICE;
387
+ if (!device) {
388
+ this.log.warn(`MATTERBRIDGE_CAMERA_AUDIO_SOURCE=${audioSource} requires MATTERBRIDGE_CAMERA_AUDIO_SOURCE_DEVICE to be set; falling back to the recorded test-voice clip`);
389
+ return testVoiceInput;
390
+ }
391
+ if (audioSource === 'rtsp') {
392
+ const description = `RTSP camera audio (${device})`;
393
+ this.log.debug(`RTSP audio capture params: url=${device}, description="${description}"`);
394
+ return { args: ['-rtsp_transport', 'tcp', '-i', device], description };
395
+ }
396
+ const description = `local microphone (${device})`;
397
+ this.log.debug(`Microphone capture params: device=${device}, description="${description}"`);
398
+ switch (process.platform) {
399
+ case 'linux':
400
+ return { args: ['-f', 'alsa', '-i', device], description };
401
+ case 'darwin':
402
+ return { args: ['-f', 'avfoundation', '-i', `:${device}`], description };
403
+ case 'win32':
404
+ return { args: ['-f', 'dshow', '-i', `audio=${device}`], description };
405
+ default:
406
+ this.log.warn(`Microphone capture via ffmpeg is not supported on platform "${process.platform}"; falling back to the recorded test-voice clip`);
407
+ return testVoiceInput;
408
+ }
409
+ }
410
+ async generateAudioTrack(codec) {
360
411
  if (this.testAudioAttached)
361
412
  return;
362
- if (process.env.MATTERBRIDGE_CAMERA_DISABLE_TEST_AUDIO === '1') {
363
- this.log.debug('Test audio injection disabled by MATTERBRIDGE_CAMERA_DISABLE_TEST_AUDIO=1');
413
+ const audioSource = this.getConfiguredAudioSource();
414
+ if (audioSource === 'none') {
415
+ this.log.debug('Audio injection disabled by MATTERBRIDGE_CAMERA_AUDIO_SOURCE=none');
364
416
  return;
365
417
  }
418
+ const audioInput = this.buildFfmpegAudioInputArgs(audioSource);
419
+ this.log.debug(`Attempting to attach ${audioInput.description} audio track`);
366
420
  const ffmpegCommand = await this.resolveCommand('ffmpeg');
367
421
  if (!ffmpegCommand) {
368
422
  this.log.warn('Cannot inject audio stream: missing dependency ffmpeg');
@@ -383,14 +437,9 @@ export class WeriftWebRtcSession {
383
437
  '-hide_banner',
384
438
  '-loglevel',
385
439
  'error',
386
- '-re',
387
- '-stream_loop',
388
- '-1',
389
- '-i',
390
- WeriftWebRtcSession.TEST_VOICE_PATH,
440
+ ...audioInput.args,
391
441
  '-vn',
392
- '-af',
393
- 'volume=6dB',
442
+ ...(audioInput.volumeFilter ? ['-af', audioInput.volumeFilter] : []),
394
443
  '-c:a',
395
444
  'libopus',
396
445
  '-b:a',
@@ -408,15 +457,15 @@ export class WeriftWebRtcSession {
408
457
  this.log.debug(`Spawning ffmpeg: ${ffmpegCommand} ${ffmpegArgs.join(' ')}`);
409
458
  const generator = spawn(ffmpegCommand, ffmpegArgs);
410
459
  generator.once('error', (error) => {
411
- this.log.warn(`Audio generator failed: ${error instanceof Error ? error.message : String(error)}`);
460
+ this.log.warn(`Audio generator failed: ${getErrorMessage(error)}`);
412
461
  });
413
462
  this.testAudioUdpDisposer = disposer;
414
463
  this.testAudioGenerator = generator;
415
464
  this.testAudioAttached = true;
416
- this.log.info(`Attached test-voice audio track (ffmpeg=${ffmpegCommand}, codec=${selectedMimeType}, payloadType=${selectedPayloadType}, sourcePort=${udpPort})`);
465
+ this.log.info(`Attached ${audioInput.description} audio track (ffmpeg=${ffmpegCommand}, codec=${selectedMimeType}, payloadType=${selectedPayloadType}, sourcePort=${udpPort})`);
417
466
  }
418
467
  catch (error) {
419
- this.log.warn(`Failed to attach test-voice audio track: ${error instanceof Error ? error.message : String(error)}`);
468
+ this.log.warn(`Failed to attach ${audioInput.description} audio track: ${getErrorMessage(error)}`);
420
469
  }
421
470
  }
422
471
  cleanupTestVideoArtifacts() {
@@ -457,6 +506,7 @@ export class WeriftWebRtcSession {
457
506
  await this.peerConnection.setLocalDescription(offer);
458
507
  const sdp = this.peerConnection.localDescription.sdp;
459
508
  this.log.info(`Created local SDP offer (${this.summarizeSdp(sdp)})`);
509
+ this.logDtlsTransportStateChanges();
460
510
  return sdp;
461
511
  }
462
512
  async createAnswer(offerSdp, videoResolution) {
@@ -480,7 +530,7 @@ export class WeriftWebRtcSession {
480
530
  const preferredAudioCodec = this.getPreferredInjectableAudioCodec();
481
531
  if (preferredAudioCodec) {
482
532
  this.preferAudioCodecOnTransceivers(preferredAudioCodec.mimeType.toLowerCase());
483
- await this.ensureTestAudioTrack(preferredAudioCodec);
533
+ await this.generateAudioTrack(preferredAudioCodec);
484
534
  }
485
535
  else {
486
536
  this.log.warn('No injectable audio codec available on negotiated transceivers (supported: Opus)');
@@ -493,19 +543,69 @@ export class WeriftWebRtcSession {
493
543
  await this.peerConnection.setLocalDescription(answer);
494
544
  const sdp = this.peerConnection.localDescription.sdp;
495
545
  this.log.info(`Created local SDP answer (${this.summarizeSdp(sdp)})`);
546
+ this.logDtlsTransportStateChanges();
496
547
  return sdp;
497
548
  }
498
549
  async applyAnswer(answerSdp) {
499
550
  this.log.debug(`ApplyAnswer requested (${this.summarizeSdp(answerSdp)})`);
500
551
  await this.peerConnection.setRemoteDescription({ type: 'answer', sdp: answerSdp });
501
552
  this.log.info(`Remote SDP answer applied (signalingState=${this.peerConnection.signalingState})`);
553
+ this.logDtlsTransportStateChanges();
502
554
  }
503
555
  async addIceCandidate(candidate, sdpMid, sdpMLineIndex) {
504
556
  this.log.debug(`Applying ICE candidate (mid=${sdpMid ?? 'null'}, mLine=${sdpMLineIndex ?? 'null'}, endOfCandidates=${candidate.trim() === ''})`);
505
557
  await this.peerConnection.addIceCandidate({ candidate, sdpMid: sdpMid ?? undefined, sdpMLineIndex: sdpMLineIndex ?? undefined });
506
558
  }
559
+ closing = false;
560
+ logDtlsTransportStateChanges() {
561
+ for (const transport of this.peerConnection.dtlsTransports) {
562
+ if (this.dtlsTransportsWithStateLogging.has(transport))
563
+ continue;
564
+ this.dtlsTransportsWithStateLogging.add(transport);
565
+ transport.onStateChange.subscribe((state) => {
566
+ this.log.info(`DTLS transport state: ${state}`);
567
+ if ((state === 'closed' || state === 'failed') && !this.closing) {
568
+ this.log.info(`Closing session ${this.webRtcSessionId}: DTLS transport reached ${state}`);
569
+ fireAndForget(this.close(), this.log, `Failed to auto-close session ${this.webRtcSessionId} after DTLS transport reached ${state}`);
570
+ }
571
+ });
572
+ }
573
+ }
574
+ async logDiagnosticsSnapshot() {
575
+ const iceStates = this.peerConnection.iceTransports.map((transport) => transport.state).join(',') || 'none';
576
+ const dtlsStates = this.peerConnection.dtlsTransports.map((transport) => transport.state).join(',') || 'none';
577
+ const stats = [...(await this.peerConnection.getStats()).values()];
578
+ const nominatedPair = stats
579
+ .filter((stat) => stat.type === 'candidate-pair')
580
+ .map((stat) => stat)
581
+ .find((pair) => pair.nominated === true);
582
+ const outboundRtp = stats.filter((stat) => stat.type === 'outbound-rtp').map((stat) => stat);
583
+ const candidatePairPacketsSent = nominatedPair?.packetsSent ?? 0;
584
+ const candidatePairPacketsReceived = nominatedPair?.packetsReceived ?? 0;
585
+ const outboundRtpPacketsSent = outboundRtp.reduce((sum, stat) => sum + (stat.packetsSent ?? 0), 0);
586
+ const previous = this.lastDiagnosticsCounters;
587
+ this.lastDiagnosticsCounters = { candidatePairPacketsSent, candidatePairPacketsReceived, outboundRtpPacketsSent };
588
+ const delta = (current, previousValue) => previousValue === undefined ? 'n/a' : `${current - previousValue >= 0 ? '+' : ''}${current - previousValue}`;
589
+ const outboundRtpSummary = outboundRtp.map((stat) => `${stat.kind ?? 'unknown'}:packetsSent=${stat.packetsSent ?? 0}`).join(', ') || 'none';
590
+ this.log.debug([
591
+ 'Diagnostics:',
592
+ `- connectionState=${this.peerConnection.connectionState}`,
593
+ `- iceConnectionState=${this.peerConnection.iceConnectionState}`,
594
+ `- iceGatheringState=${this.peerConnection.iceGatheringState}`,
595
+ `- signalingState=${this.peerConnection.signalingState}`,
596
+ `- iceTransportStates=[${iceStates}]`,
597
+ `- dtlsTransportStates=[${dtlsStates}]`,
598
+ `- nominatedPair(packetsSent=${candidatePairPacketsSent} Δ${delta(candidatePairPacketsSent, previous?.candidatePairPacketsSent)}, packetsReceived=${candidatePairPacketsReceived} Δ${delta(candidatePairPacketsReceived, previous?.candidatePairPacketsReceived)})`,
599
+ `- outboundRtp[${outboundRtpSummary}] (totalPacketsSent Δ${delta(outboundRtpPacketsSent, previous?.outboundRtpPacketsSent)})`,
600
+ ].join('\n'));
601
+ }
507
602
  async close() {
603
+ if (this.closing)
604
+ return;
605
+ this.closing = true;
508
606
  this.log.debug('Closing RTCPeerConnection');
607
+ clearInterval(this.diagnosticsInterval);
608
+ this.diagnosticsInterval = undefined;
509
609
  await this.peerConnection.close();
510
610
  this.cleanupTestVideoArtifacts();
511
611
  this.cleanupTestAudioArtifacts();
@@ -6,6 +6,7 @@
6
6
  "videoGenerator": "none",
7
7
  "videoResolution": "auto",
8
8
  "videoBitrate": 1000,
9
+ "audioGenerator": "none",
9
10
  "animationInterval": 60,
10
11
  "debug": false,
11
12
  "unregisterOnShutdown": false
@@ -2,7 +2,7 @@
2
2
  "title": "Matterbridge example camera plugin",
3
3
  "description": "matterbridge-example-camera v. 1.0.0 by https://github.com/Luligu",
4
4
  "type": "object",
5
- "required": ["videoGenerator", "videoResolution"],
5
+ "required": ["videoGenerator", "videoResolution", "audioGenerator"],
6
6
  "properties": {
7
7
  "name": {
8
8
  "description": "Plugin Name",
@@ -67,6 +67,18 @@
67
67
  "type": "number",
68
68
  "default": 1000
69
69
  },
70
+ "audioGenerator": {
71
+ "title": "Audio Generator",
72
+ "description": "Select the WebRTC audio generator: none disables track injection (only the audio transceiver is negotiated), test injects a recorded test-voice clip, looped, e.g. for an Intercom's Listen live view, microphone captures from the configured local microphone, and rtsp pulls the audio from the RTSP url configured in Audio Source.",
73
+ "type": "string",
74
+ "enum": ["none", "test", "microphone", "rtsp"],
75
+ "default": "none"
76
+ },
77
+ "audioSource": {
78
+ "title": "Audio Source",
79
+ "description": "OS-specific ffmpeg microphone identifier, such as hw:0,0 on Linux (alsa), 0 on macOS (avfoundation audio index), or Microphone Array on Windows (dshow). When Audio Generator is rtsp, this holds the RTSP url instead, e.g. rtsp://user:password@host:554/path.",
80
+ "type": "string"
81
+ },
70
82
  "debug": {
71
83
  "title": "Enable Debug",
72
84
  "description": "Enable the debug for the plugin (development only)",
@@ -1,12 +1,12 @@
1
1
  {
2
2
  "name": "matterbridge-example-camera",
3
- "version": "0.0.8-dev-20260726-dffb00d",
3
+ "version": "0.0.8-dev-20260726-61cad4c",
4
4
  "lockfileVersion": 3,
5
5
  "requires": true,
6
6
  "packages": {
7
7
  "": {
8
8
  "name": "matterbridge-example-camera",
9
- "version": "0.0.8-dev-20260726-dffb00d",
9
+ "version": "0.0.8-dev-20260726-61cad4c",
10
10
  "license": "Apache-2.0",
11
11
  "dependencies": {
12
12
  "node-ansi-logger": "3.3.0",
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "matterbridge-example-camera",
3
- "version": "0.0.8-dev-20260726-dffb00d",
3
+ "version": "0.0.8-dev-20260726-61cad4c",
4
4
  "description": "Matterbridge camera example plugin",
5
5
  "author": "https://github.com/Luligu",
6
6
  "license": "Apache-2.0",