node-av-light 0.0.0-stage → 7.1.0-beta.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE.md +22 -0
- package/README.md +710 -2
- package/dist/api/audio-frame-buffer.d.ts +636 -0
- package/dist/api/audio-frame-buffer.js +1027 -0
- package/dist/api/audio-frame-buffer.js.map +1 -0
- package/dist/api/bitstream-filter.d.ts +938 -0
- package/dist/api/bitstream-filter.js +1427 -0
- package/dist/api/bitstream-filter.js.map +1 -0
- package/dist/api/constants.d.ts +44 -0
- package/dist/api/constants.js +45 -0
- package/dist/api/constants.js.map +1 -0
- package/dist/api/data/test_av1.ivf +0 -0
- package/dist/api/data/test_h264.h264 +0 -0
- package/dist/api/data/test_hevc.h265 +0 -0
- package/dist/api/data/test_mjpeg.mjpeg +0 -0
- package/dist/api/data/test_vp8.ivf +0 -0
- package/dist/api/data/test_vp9.ivf +0 -0
- package/dist/api/decoder.d.ts +1413 -0
- package/dist/api/decoder.js +2367 -0
- package/dist/api/decoder.js.map +1 -0
- package/dist/api/demuxer.d.ts +1295 -0
- package/dist/api/demuxer.js +2344 -0
- package/dist/api/demuxer.js.map +1 -0
- package/dist/api/device.d.ts +589 -0
- package/dist/api/device.js +964 -0
- package/dist/api/device.js.map +1 -0
- package/dist/api/encoder-pool.d.ts +262 -0
- package/dist/api/encoder-pool.js +397 -0
- package/dist/api/encoder-pool.js.map +1 -0
- package/dist/api/encoder.d.ts +1509 -0
- package/dist/api/encoder.js +2619 -0
- package/dist/api/encoder.js.map +1 -0
- package/dist/api/filter-complex.d.ts +828 -0
- package/dist/api/filter-complex.js +1661 -0
- package/dist/api/filter-complex.js.map +1 -0
- package/dist/api/filter-presets.d.ts +800 -0
- package/dist/api/filter-presets.js +1554 -0
- package/dist/api/filter-presets.js.map +1 -0
- package/dist/api/filter.d.ts +1339 -0
- package/dist/api/filter.js +2075 -0
- package/dist/api/filter.js.map +1 -0
- package/dist/api/fmp4-stream.d.ts +797 -0
- package/dist/api/fmp4-stream.js +1428 -0
- package/dist/api/fmp4-stream.js.map +1 -0
- package/dist/api/hardware.d.ts +677 -0
- package/dist/api/hardware.js +1300 -0
- package/dist/api/hardware.js.map +1 -0
- package/dist/api/index.d.ts +21 -0
- package/dist/api/index.js +38 -0
- package/dist/api/index.js.map +1 -0
- package/dist/api/io-stream.d.ts +308 -0
- package/dist/api/io-stream.js +358 -0
- package/dist/api/io-stream.js.map +1 -0
- package/dist/api/muxer.d.ts +1311 -0
- package/dist/api/muxer.js +2672 -0
- package/dist/api/muxer.js.map +1 -0
- package/dist/api/pipeline.d.ts +801 -0
- package/dist/api/pipeline.js +1344 -0
- package/dist/api/pipeline.js.map +1 -0
- package/dist/api/probe.d.ts +128 -0
- package/dist/api/probe.js +227 -0
- package/dist/api/probe.js.map +1 -0
- package/dist/api/rtp-stream.d.ts +497 -0
- package/dist/api/rtp-stream.js +1131 -0
- package/dist/api/rtp-stream.js.map +1 -0
- package/dist/api/scaler.d.ts +561 -0
- package/dist/api/scaler.js +918 -0
- package/dist/api/scaler.js.map +1 -0
- package/dist/api/types.d.ts +17 -0
- package/dist/api/types.js +2 -0
- package/dist/api/types.js.map +1 -0
- package/dist/api/utilities/async-queue.d.ts +149 -0
- package/dist/api/utilities/async-queue.js +254 -0
- package/dist/api/utilities/async-queue.js.map +1 -0
- package/dist/api/utilities/audio-sample.d.ts +117 -0
- package/dist/api/utilities/audio-sample.js +112 -0
- package/dist/api/utilities/audio-sample.js.map +1 -0
- package/dist/api/utilities/audio-transcode.d.ts +20 -0
- package/dist/api/utilities/audio-transcode.js +27 -0
- package/dist/api/utilities/audio-transcode.js.map +1 -0
- package/dist/api/utilities/channel-layout.d.ts +76 -0
- package/dist/api/utilities/channel-layout.js +80 -0
- package/dist/api/utilities/channel-layout.js.map +1 -0
- package/dist/api/utilities/codec-format.d.ts +87 -0
- package/dist/api/utilities/codec-format.js +117 -0
- package/dist/api/utilities/codec-format.js.map +1 -0
- package/dist/api/utilities/context-options.d.ts +25 -0
- package/dist/api/utilities/context-options.js +23 -0
- package/dist/api/utilities/context-options.js.map +1 -0
- package/dist/api/utilities/electron-shared-texture.d.ts +368 -0
- package/dist/api/utilities/electron-shared-texture.js +540 -0
- package/dist/api/utilities/electron-shared-texture.js.map +1 -0
- package/dist/api/utilities/image.d.ts +207 -0
- package/dist/api/utilities/image.js +213 -0
- package/dist/api/utilities/image.js.map +1 -0
- package/dist/api/utilities/index.d.ts +14 -0
- package/dist/api/utilities/index.js +27 -0
- package/dist/api/utilities/index.js.map +1 -0
- package/dist/api/utilities/live-decoding.d.ts +15 -0
- package/dist/api/utilities/live-decoding.js +32 -0
- package/dist/api/utilities/live-decoding.js.map +1 -0
- package/dist/api/utilities/media-type.d.ts +49 -0
- package/dist/api/utilities/media-type.js +53 -0
- package/dist/api/utilities/media-type.js.map +1 -0
- package/dist/api/utilities/packet-age-watchdog.d.ts +198 -0
- package/dist/api/utilities/packet-age-watchdog.js +594 -0
- package/dist/api/utilities/packet-age-watchdog.js.map +1 -0
- package/dist/api/utilities/pixel-format.d.ts +89 -0
- package/dist/api/utilities/pixel-format.js +97 -0
- package/dist/api/utilities/pixel-format.js.map +1 -0
- package/dist/api/utilities/sample-format.d.ts +129 -0
- package/dist/api/utilities/sample-format.js +141 -0
- package/dist/api/utilities/sample-format.js.map +1 -0
- package/dist/api/utilities/scale-size.d.ts +51 -0
- package/dist/api/utilities/scale-size.js +55 -0
- package/dist/api/utilities/scale-size.js.map +1 -0
- package/dist/api/utilities/scheduler.d.ts +138 -0
- package/dist/api/utilities/scheduler.js +98 -0
- package/dist/api/utilities/scheduler.js.map +1 -0
- package/dist/api/utilities/streaming.d.ts +186 -0
- package/dist/api/utilities/streaming.js +309 -0
- package/dist/api/utilities/streaming.js.map +1 -0
- package/dist/api/utilities/timestamp.d.ts +193 -0
- package/dist/api/utilities/timestamp.js +206 -0
- package/dist/api/utilities/timestamp.js.map +1 -0
- package/dist/api/utilities/video-transcode.d.ts +218 -0
- package/dist/api/utilities/video-transcode.js +624 -0
- package/dist/api/utilities/video-transcode.js.map +1 -0
- package/dist/api/utilities/whisper-model.d.ts +330 -0
- package/dist/api/utilities/whisper-model.js +587 -0
- package/dist/api/utilities/whisper-model.js.map +1 -0
- package/dist/api/utils.d.ts +19 -0
- package/dist/api/utils.js +39 -0
- package/dist/api/utils.js.map +1 -0
- package/dist/api/webrtc-stream.d.ts +294 -0
- package/dist/api/webrtc-stream.js +546 -0
- package/dist/api/webrtc-stream.js.map +1 -0
- package/dist/api/whisper.d.ts +326 -0
- package/dist/api/whisper.js +402 -0
- package/dist/api/whisper.js.map +1 -0
- package/dist/constants/bsf-options.d.ts +364 -0
- package/dist/constants/bsf-options.js +7 -0
- package/dist/constants/bsf-options.js.map +1 -0
- package/dist/constants/channel-layouts.d.ts +53 -0
- package/dist/constants/channel-layouts.js +57 -0
- package/dist/constants/channel-layouts.js.map +1 -0
- package/dist/constants/constants.d.ts +2435 -0
- package/dist/constants/constants.js +1989 -0
- package/dist/constants/constants.js.map +1 -0
- package/dist/constants/decoders.d.ts +651 -0
- package/dist/constants/decoders.js +641 -0
- package/dist/constants/decoders.js.map +1 -0
- package/dist/constants/encoders.d.ts +314 -0
- package/dist/constants/encoders.js +309 -0
- package/dist/constants/encoders.js.map +1 -0
- package/dist/constants/filter-options.d.ts +12099 -0
- package/dist/constants/filter-options.js +7 -0
- package/dist/constants/filter-options.js.map +1 -0
- package/dist/constants/format-options.d.ts +4319 -0
- package/dist/constants/format-options.js +7 -0
- package/dist/constants/format-options.js.map +1 -0
- package/dist/constants/formats.d.ts +18 -0
- package/dist/constants/formats.js +7 -0
- package/dist/constants/formats.js.map +1 -0
- package/dist/constants/hardware.d.ts +26 -0
- package/dist/constants/hardware.js +27 -0
- package/dist/constants/hardware.js.map +1 -0
- package/dist/constants/index.d.ts +10 -0
- package/dist/constants/index.js +6 -0
- package/dist/constants/index.js.map +1 -0
- package/dist/constants/options.d.ts +6217 -0
- package/dist/constants/options.js +7 -0
- package/dist/constants/options.js.map +1 -0
- package/dist/ffmpeg/index.d.ts +101 -0
- package/dist/ffmpeg/index.js +161 -0
- package/dist/ffmpeg/index.js.map +1 -0
- package/dist/ffmpeg/version.d.ts +6 -0
- package/dist/ffmpeg/version.js +7 -0
- package/dist/ffmpeg/version.js.map +1 -0
- package/dist/index.d.ts +4 -0
- package/dist/index.js +9 -0
- package/dist/index.js.map +1 -0
- package/dist/lib/audio-fifo.d.ts +399 -0
- package/dist/lib/audio-fifo.js +431 -0
- package/dist/lib/audio-fifo.js.map +1 -0
- package/dist/lib/binding.d.ts +242 -0
- package/dist/lib/binding.js +102 -0
- package/dist/lib/binding.js.map +1 -0
- package/dist/lib/bitstream-filter-context.d.ts +381 -0
- package/dist/lib/bitstream-filter-context.js +443 -0
- package/dist/lib/bitstream-filter-context.js.map +1 -0
- package/dist/lib/bitstream-filter.d.ts +140 -0
- package/dist/lib/bitstream-filter.js +154 -0
- package/dist/lib/bitstream-filter.js.map +1 -0
- package/dist/lib/codec-context.d.ts +1069 -0
- package/dist/lib/codec-context.js +1336 -0
- package/dist/lib/codec-context.js.map +1 -0
- package/dist/lib/codec-parameters.d.ts +664 -0
- package/dist/lib/codec-parameters.js +813 -0
- package/dist/lib/codec-parameters.js.map +1 -0
- package/dist/lib/codec-parser.d.ts +201 -0
- package/dist/lib/codec-parser.js +213 -0
- package/dist/lib/codec-parser.js.map +1 -0
- package/dist/lib/codec.d.ts +617 -0
- package/dist/lib/codec.js +746 -0
- package/dist/lib/codec.js.map +1 -0
- package/dist/lib/device.d.ts +294 -0
- package/dist/lib/device.js +326 -0
- package/dist/lib/device.js.map +1 -0
- package/dist/lib/dictionary.d.ts +333 -0
- package/dist/lib/dictionary.js +372 -0
- package/dist/lib/dictionary.js.map +1 -0
- package/dist/lib/error.d.ts +311 -0
- package/dist/lib/error.js +395 -0
- package/dist/lib/error.js.map +1 -0
- package/dist/lib/fifo.d.ts +416 -0
- package/dist/lib/fifo.js +453 -0
- package/dist/lib/fifo.js.map +1 -0
- package/dist/lib/filter-context.d.ts +714 -0
- package/dist/lib/filter-context.js +783 -0
- package/dist/lib/filter-context.js.map +1 -0
- package/dist/lib/filter-graph-segment.d.ts +161 -0
- package/dist/lib/filter-graph-segment.js +172 -0
- package/dist/lib/filter-graph-segment.js.map +1 -0
- package/dist/lib/filter-graph.d.ts +641 -0
- package/dist/lib/filter-graph.js +704 -0
- package/dist/lib/filter-graph.js.map +1 -0
- package/dist/lib/filter-inout.d.ts +202 -0
- package/dist/lib/filter-inout.js +261 -0
- package/dist/lib/filter-inout.js.map +1 -0
- package/dist/lib/filter.d.ts +243 -0
- package/dist/lib/filter.js +272 -0
- package/dist/lib/filter.js.map +1 -0
- package/dist/lib/format-context.d.ts +1335 -0
- package/dist/lib/format-context.js +1455 -0
- package/dist/lib/format-context.js.map +1 -0
- package/dist/lib/frame-utils.d.ts +116 -0
- package/dist/lib/frame-utils.js +98 -0
- package/dist/lib/frame-utils.js.map +1 -0
- package/dist/lib/frame.d.ts +1315 -0
- package/dist/lib/frame.js +1535 -0
- package/dist/lib/frame.js.map +1 -0
- package/dist/lib/hardware-device-context.d.ts +362 -0
- package/dist/lib/hardware-device-context.js +383 -0
- package/dist/lib/hardware-device-context.js.map +1 -0
- package/dist/lib/hardware-frames-context.d.ts +418 -0
- package/dist/lib/hardware-frames-context.js +471 -0
- package/dist/lib/hardware-frames-context.js.map +1 -0
- package/dist/lib/index.d.ts +35 -0
- package/dist/lib/index.js +60 -0
- package/dist/lib/index.js.map +1 -0
- package/dist/lib/input-format.d.ts +249 -0
- package/dist/lib/input-format.js +306 -0
- package/dist/lib/input-format.js.map +1 -0
- package/dist/lib/io-context.d.ts +698 -0
- package/dist/lib/io-context.js +771 -0
- package/dist/lib/io-context.js.map +1 -0
- package/dist/lib/log.d.ts +174 -0
- package/dist/lib/log.js +184 -0
- package/dist/lib/log.js.map +1 -0
- package/dist/lib/native-types.d.ts +1024 -0
- package/dist/lib/native-types.js +2 -0
- package/dist/lib/native-types.js.map +1 -0
- package/dist/lib/option.d.ts +953 -0
- package/dist/lib/option.js +1607 -0
- package/dist/lib/option.js.map +1 -0
- package/dist/lib/output-format.d.ts +222 -0
- package/dist/lib/output-format.js +259 -0
- package/dist/lib/output-format.js.map +1 -0
- package/dist/lib/packet.d.ts +521 -0
- package/dist/lib/packet.js +613 -0
- package/dist/lib/packet.js.map +1 -0
- package/dist/lib/rational.d.ts +251 -0
- package/dist/lib/rational.js +278 -0
- package/dist/lib/rational.js.map +1 -0
- package/dist/lib/software-resample-context.d.ts +552 -0
- package/dist/lib/software-resample-context.js +592 -0
- package/dist/lib/software-resample-context.js.map +1 -0
- package/dist/lib/software-scale-context.d.ts +344 -0
- package/dist/lib/software-scale-context.js +366 -0
- package/dist/lib/software-scale-context.js.map +1 -0
- package/dist/lib/stream.d.ts +383 -0
- package/dist/lib/stream.js +524 -0
- package/dist/lib/stream.js.map +1 -0
- package/dist/lib/sync-queue.d.ts +179 -0
- package/dist/lib/sync-queue.js +197 -0
- package/dist/lib/sync-queue.js.map +1 -0
- package/dist/lib/types.d.ts +34 -0
- package/dist/lib/types.js +2 -0
- package/dist/lib/types.js.map +1 -0
- package/dist/lib/utilities.d.ts +1198 -0
- package/dist/lib/utilities.js +1302 -0
- package/dist/lib/utilities.js.map +1 -0
- package/dist/utils/electron.d.ts +49 -0
- package/dist/utils/electron.js +63 -0
- package/dist/utils/electron.js.map +1 -0
- package/dist/utils/index.d.ts +6 -0
- package/dist/utils/index.js +7 -0
- package/dist/utils/index.js.map +1 -0
- package/dist/utils/libc.d.ts +11 -0
- package/dist/utils/libc.js +35 -0
- package/dist/utils/libc.js.map +1 -0
- package/dist/utils/variant.d.ts +26 -0
- package/dist/utils/variant.js +31 -0
- package/dist/utils/variant.js.map +1 -0
- package/dist/webrtc/index.d.ts +4 -0
- package/dist/webrtc/index.js +7 -0
- package/dist/webrtc/index.js.map +1 -0
- package/install/check.js +124 -0
- package/install/ffmpeg.js +66 -0
- package/package.json +167 -4
package/README.md
CHANGED
|
@@ -1,3 +1,711 @@
|
|
|
1
|
-
#
|
|
1
|
+
# node-av-light
|
|
2
2
|
|
|
3
|
-
This
|
|
3
|
+
This is the light build of node-av, with the same API. FFmpeg is built with GPL components (x264, x265), so it is licensed under the GPL v3 or later. It also links libfdk_aac, which is licensed under the Fraunhofer FDK AAC license. See [Variants](#variants) below for what it contains.
|
|
4
|
+
|
|
5
|
+
Install it under the name node-av to keep the `node-av/...` imports:
|
|
6
|
+
|
|
7
|
+
```bash
|
|
8
|
+
npm install node-av@npm:node-av-light
|
|
9
|
+
```
|
|
10
|
+
|
|
11
|
+
---
|
|
12
|
+
|
|
13
|
+
<p align="center">
|
|
14
|
+
<img src="https://github.com/seydx/node-av/blob/main/docs/logo.png?raw=true" width="250px">
|
|
15
|
+
</p>
|
|
16
|
+
|
|
17
|
+
# NodeAV
|
|
18
|
+
|
|
19
|
+
[](https://www.npmjs.com/package/node-av)
|
|
20
|
+
[](https://www.npmjs.com/package/node-av)
|
|
21
|
+
[](https://socket.dev/npm/package/node-av)
|
|
22
|
+
[](https://ffmpeg.org)
|
|
23
|
+
[](https://github.com/seydx/node-av)
|
|
24
|
+
|
|
25
|
+
Native Node.js bindings for FFmpeg with full TypeScript support - including type-safe options (autocomplete + validation) for every codec, format, filter, and bitstream filter, generated from FFmpeg's own metadata. Provides direct access to FFmpeg's C APIs through N-API. Includes both raw FFmpeg bindings for full control and higher-level abstractions. Automatic resource management via Disposable pattern, hardware acceleration support and prebuilt binaries for Windows, Linux, and macOS.
|
|
26
|
+
|
|
27
|
+
📚 **[Documentation](https://seydx.github.io/node-av)**
|
|
28
|
+
|
|
29
|
+
## Table of Contents
|
|
30
|
+
|
|
31
|
+
- [Installation](#installation)
|
|
32
|
+
- [Quick Start](#quick-start)
|
|
33
|
+
- [Low-Level API](#low-level-api)
|
|
34
|
+
- [High-Level API](#high-level-api)
|
|
35
|
+
- [Pipeline API](#pipeline-api)
|
|
36
|
+
- [Hardware Acceleration](#hardware-acceleration)
|
|
37
|
+
- [Auto-Detection](#auto-detection)
|
|
38
|
+
- [Specific Hardware](#specific-hardware)
|
|
39
|
+
- [Stream Processing](#stream-processing)
|
|
40
|
+
- [From Files or Network](#from-files-or-network)
|
|
41
|
+
- [From Buffers](#from-buffers)
|
|
42
|
+
- [Raw Media Processing](#raw-media-processing)
|
|
43
|
+
- [Device Capture](#device-capture)
|
|
44
|
+
- [FFmpeg Binary Access](#ffmpeg-binary-access)
|
|
45
|
+
- [Resource Management](#resource-management)
|
|
46
|
+
- [Imports and Tree Shaking](#imports-and-tree-shaking)
|
|
47
|
+
- [Key Features](#key-features)
|
|
48
|
+
- [Performance](#performance)
|
|
49
|
+
- [Benchmarks](#benchmarks)
|
|
50
|
+
- [Sync vs Async Operations](#sync-vs-async-operations)
|
|
51
|
+
- [Memory Safety Considerations](#memory-safety-considerations)
|
|
52
|
+
- [Electron](#electron)
|
|
53
|
+
- [Examples](#examples)
|
|
54
|
+
- [Prebuilt Binaries](#prebuilt-binaries)
|
|
55
|
+
- [Cross-Platform Packaging (CI/CD)](#cross-platform-packaging-cicd)
|
|
56
|
+
- [Variants](#variants)
|
|
57
|
+
- [Troubleshooting](#troubleshooting)
|
|
58
|
+
- [License](#license)
|
|
59
|
+
- [Contributing](#contributing)
|
|
60
|
+
- [Support](#support)
|
|
61
|
+
- [See Also](#see-also)
|
|
62
|
+
|
|
63
|
+
## Installation
|
|
64
|
+
|
|
65
|
+
```bash
|
|
66
|
+
npm install node-av
|
|
67
|
+
```
|
|
68
|
+
|
|
69
|
+
The FFmpeg command-line tool is a separate, optional package (see [FFmpeg Binary Access](#ffmpeg-binary-access)):
|
|
70
|
+
|
|
71
|
+
```bash
|
|
72
|
+
npm install @seydx/node-av-ffmpeg
|
|
73
|
+
```
|
|
74
|
+
|
|
75
|
+
node-av links a GPL build of FFmpeg. For an LGPL build or a smaller one, see [Variants](#variants).
|
|
76
|
+
|
|
77
|
+
## Quick Start
|
|
78
|
+
|
|
79
|
+
### Low-Level API
|
|
80
|
+
|
|
81
|
+
Direct access to FFmpeg's C APIs with minimal abstractions. Perfect when you need full control over FFmpeg functionality.
|
|
82
|
+
|
|
83
|
+
```typescript
|
|
84
|
+
import { AVERROR_EOF, AVMEDIA_TYPE_VIDEO } from 'node-av/constants';
|
|
85
|
+
import { Codec, CodecContext, FFmpegError, FormatContext, Frame, Packet, Rational } from 'node-av/lib';
|
|
86
|
+
|
|
87
|
+
// Open input file
|
|
88
|
+
await using ifmtCtx = new FormatContext();
|
|
89
|
+
|
|
90
|
+
let ret = await ifmtCtx.openInput('input.mp4');
|
|
91
|
+
FFmpegError.throwIfError(ret, 'Could not open input file');
|
|
92
|
+
|
|
93
|
+
ret = await ifmtCtx.findStreamInfo();
|
|
94
|
+
FFmpegError.throwIfError(ret, 'Could not find stream info');
|
|
95
|
+
|
|
96
|
+
// Find video stream
|
|
97
|
+
const videoStreamIndex = ifmtCtx.findBestStream(AVMEDIA_TYPE_VIDEO);
|
|
98
|
+
const videoStream = ifmtCtx.streams?.[videoStreamIndex];
|
|
99
|
+
|
|
100
|
+
if (!videoStream) {
|
|
101
|
+
throw new Error('No video stream found');
|
|
102
|
+
}
|
|
103
|
+
|
|
104
|
+
// Create codec
|
|
105
|
+
const codec = Codec.findDecoder(videoStream.codecpar.codecId);
|
|
106
|
+
if (!codec) {
|
|
107
|
+
throw new Error('Codec not found');
|
|
108
|
+
}
|
|
109
|
+
|
|
110
|
+
// Allocate codec context for the decoder
|
|
111
|
+
using decoderCtx = new CodecContext();
|
|
112
|
+
decoderCtx.allocContext3(codec);
|
|
113
|
+
|
|
114
|
+
ret = decoderCtx.parametersToContext(videoStream.codecpar);
|
|
115
|
+
FFmpegError.throwIfError(ret, 'Could not copy codec parameters to decoder context');
|
|
116
|
+
|
|
117
|
+
// Open decoder context
|
|
118
|
+
ret = await decoderCtx.open2(codec, null);
|
|
119
|
+
FFmpegError.throwIfError(ret, 'Could not open codec');
|
|
120
|
+
|
|
121
|
+
// Process packets
|
|
122
|
+
using packet = new Packet();
|
|
123
|
+
packet.alloc();
|
|
124
|
+
|
|
125
|
+
using frame = new Frame();
|
|
126
|
+
frame.alloc();
|
|
127
|
+
|
|
128
|
+
while (true) {
|
|
129
|
+
let ret = await ifmtCtx.readFrame(packet);
|
|
130
|
+
if (ret < 0) {
|
|
131
|
+
break;
|
|
132
|
+
}
|
|
133
|
+
|
|
134
|
+
if (packet.streamIndex === videoStreamIndex) {
|
|
135
|
+
// Send packet to decoder
|
|
136
|
+
ret = await decoderCtx.sendPacket(packet);
|
|
137
|
+
if (ret < 0 && ret !== AVERROR_EOF) {
|
|
138
|
+
FFmpegError.throwIfError(ret, 'Error sending packet to decoder');
|
|
139
|
+
}
|
|
140
|
+
|
|
141
|
+
// Receive decoded frames
|
|
142
|
+
while (true) {
|
|
143
|
+
const ret = await decoderCtx.receiveFrame(frame);
|
|
144
|
+
if (ret === AVERROR_EOF || ret < 0) {
|
|
145
|
+
break;
|
|
146
|
+
}
|
|
147
|
+
|
|
148
|
+
console.log(`Decoded frame ${frame.pts}, size: ${frame.width}x${frame.height}`);
|
|
149
|
+
|
|
150
|
+
// Process frame data...
|
|
151
|
+
|
|
152
|
+
frame.unref();
|
|
153
|
+
}
|
|
154
|
+
}
|
|
155
|
+
|
|
156
|
+
packet.unref();
|
|
157
|
+
}
|
|
158
|
+
|
|
159
|
+
// Flush the decoder - codecs buffer frames internally (B-frames, threading),
|
|
160
|
+
// so the last frames only come out after signaling EOF with a null packet.
|
|
161
|
+
ret = await decoderCtx.sendPacket(null);
|
|
162
|
+
if (ret < 0 && ret !== AVERROR_EOF) {
|
|
163
|
+
FFmpegError.throwIfError(ret, 'Failed to flush decoder');
|
|
164
|
+
}
|
|
165
|
+
|
|
166
|
+
while (true) {
|
|
167
|
+
const ret = await decoderCtx.receiveFrame(frame);
|
|
168
|
+
if (ret === AVERROR_EOF || ret < 0) {
|
|
169
|
+
break;
|
|
170
|
+
}
|
|
171
|
+
|
|
172
|
+
console.log(`Decoded frame ${frame.pts}, size: ${frame.width}x${frame.height}`);
|
|
173
|
+
|
|
174
|
+
// Process frame data...
|
|
175
|
+
|
|
176
|
+
frame.unref();
|
|
177
|
+
}
|
|
178
|
+
```
|
|
179
|
+
|
|
180
|
+
### High-Level API
|
|
181
|
+
|
|
182
|
+
Higher-level abstractions for common tasks like decoding, encoding, filtering, and transcoding. Easier to use while still providing access to low-level details when needed.
|
|
183
|
+
|
|
184
|
+
```typescript
|
|
185
|
+
import { Decoder, Demuxer, Encoder, HardwareContext, Muxer } from 'node-av/api';
|
|
186
|
+
import { FF_ENCODER_LIBX264 } from 'node-av/constants';
|
|
187
|
+
|
|
188
|
+
// Open Demuxer
|
|
189
|
+
await using input = await Demuxer.open('input.mp4');
|
|
190
|
+
|
|
191
|
+
// Get video stream
|
|
192
|
+
const videoStream = input.video()!;
|
|
193
|
+
|
|
194
|
+
// Optional, setup hardware acceleration
|
|
195
|
+
using hw = HardwareContext.auto();
|
|
196
|
+
|
|
197
|
+
// Create decoder
|
|
198
|
+
using decoder = await Decoder.create(videoStream, {
|
|
199
|
+
hardware: hw, // Optional, use hardware acceleration if available
|
|
200
|
+
});
|
|
201
|
+
|
|
202
|
+
// Create encoder
|
|
203
|
+
using encoder = await Encoder.create(FF_ENCODER_LIBX264, {
|
|
204
|
+
decoder, // Optional, copy settings from decoder
|
|
205
|
+
});
|
|
206
|
+
|
|
207
|
+
// Open Muxer
|
|
208
|
+
await using output = await Muxer.open('output.mp4', {
|
|
209
|
+
input, // Optional, used to copy global headers and metadata
|
|
210
|
+
});
|
|
211
|
+
|
|
212
|
+
// Add stream to output
|
|
213
|
+
const outputIndex = output.addStream(encoder, {
|
|
214
|
+
inputStream: videoStream, // Optional, copy settings from input stream
|
|
215
|
+
});
|
|
216
|
+
|
|
217
|
+
// Create processing generators
|
|
218
|
+
const inputGenerator = input.packets(videoStream.index);
|
|
219
|
+
const decoderGenerator = decoder.frames(inputGenerator);
|
|
220
|
+
const encoderGenerator = encoder.packets(decoderGenerator);
|
|
221
|
+
|
|
222
|
+
// Process packets
|
|
223
|
+
for await (using packet of encoderGenerator) {
|
|
224
|
+
await output.writePacket(packet, outputIndex);
|
|
225
|
+
}
|
|
226
|
+
|
|
227
|
+
// Done
|
|
228
|
+
```
|
|
229
|
+
|
|
230
|
+
### Pipeline API
|
|
231
|
+
|
|
232
|
+
A simple way to chain together multiple processing steps like decoding, filtering, encoding, and muxing.
|
|
233
|
+
|
|
234
|
+
```typescript
|
|
235
|
+
import { Decoder, Demuxer, Encoder, HardwareContext, Muxer, pipeline } from 'node-av/api';
|
|
236
|
+
import { FF_ENCODER_LIBX264 } from 'node-av/constants';
|
|
237
|
+
|
|
238
|
+
// Simple transcode pipeline: input → decoder → encoder → output
|
|
239
|
+
|
|
240
|
+
// Open Demuxer
|
|
241
|
+
await using input = await Demuxer.open('input.mp4');
|
|
242
|
+
|
|
243
|
+
// Get video stream
|
|
244
|
+
const videoStream = input.video()!;
|
|
245
|
+
|
|
246
|
+
// Optional, setup hardware acceleration
|
|
247
|
+
using hw = HardwareContext.auto();
|
|
248
|
+
|
|
249
|
+
// Create decoder
|
|
250
|
+
using decoder = await Decoder.create(videoStream, {
|
|
251
|
+
hardware: hw, // Optional, use hardware acceleration if available
|
|
252
|
+
});
|
|
253
|
+
|
|
254
|
+
// Create encoder
|
|
255
|
+
using encoder = await Encoder.create(FF_ENCODER_LIBX264, {
|
|
256
|
+
decoder, // Optional, copy settings from decoder
|
|
257
|
+
});
|
|
258
|
+
|
|
259
|
+
// Open Muxer
|
|
260
|
+
await using output = await Muxer.open('output.mp4', {
|
|
261
|
+
input, // Optional, used to copy global headers and metadata
|
|
262
|
+
});
|
|
263
|
+
|
|
264
|
+
const control = pipeline(input, decoder, encoder, output);
|
|
265
|
+
await control.completion;
|
|
266
|
+
```
|
|
267
|
+
|
|
268
|
+
## Hardware Acceleration
|
|
269
|
+
|
|
270
|
+
The library supports all hardware acceleration methods available in FFmpeg. The specific hardware types available depend on your FFmpeg build and system configuration.
|
|
271
|
+
|
|
272
|
+
### Auto-Detection
|
|
273
|
+
|
|
274
|
+
```typescript
|
|
275
|
+
import { HardwareContext } from 'node-av/api';
|
|
276
|
+
import { FF_ENCODER_LIBX264 } from 'node-av/constants';
|
|
277
|
+
|
|
278
|
+
// Automatically detect best available hardware
|
|
279
|
+
const hw = HardwareContext.auto();
|
|
280
|
+
console.log(`Using hardware: ${hw.deviceTypeName}`);
|
|
281
|
+
|
|
282
|
+
// Use with decoder
|
|
283
|
+
const decoder = await Decoder.create(stream, {
|
|
284
|
+
hardware: hw
|
|
285
|
+
});
|
|
286
|
+
|
|
287
|
+
// Use with encoder (use hardware-specific codec)
|
|
288
|
+
const encoderCodec = hw?.getEncoderCodec('h264') ?? FF_ENCODER_LIBX264;
|
|
289
|
+
const encoder = await Encoder.create(encoderCodec, {
|
|
290
|
+
decoder,
|
|
291
|
+
});
|
|
292
|
+
```
|
|
293
|
+
|
|
294
|
+
### Specific Hardware
|
|
295
|
+
|
|
296
|
+
```typescript
|
|
297
|
+
import { AV_HWDEVICE_TYPE_CUDA, AV_HWDEVICE_TYPE_VAAPI } from 'node-av/constants';
|
|
298
|
+
|
|
299
|
+
// Use specific hardware type
|
|
300
|
+
const cuda = HardwareContext.create(AV_HWDEVICE_TYPE_CUDA);
|
|
301
|
+
const vaapi = HardwareContext.create(AV_HWDEVICE_TYPE_VAAPI, '/dev/dri/renderD128');
|
|
302
|
+
```
|
|
303
|
+
|
|
304
|
+
## Stream Processing
|
|
305
|
+
|
|
306
|
+
### From Files or Network
|
|
307
|
+
|
|
308
|
+
```typescript
|
|
309
|
+
const media = await Demuxer.open('input.mp4');
|
|
310
|
+
|
|
311
|
+
// or
|
|
312
|
+
|
|
313
|
+
const media = await Demuxer.open('rtsp://example.com/stream');
|
|
314
|
+
```
|
|
315
|
+
|
|
316
|
+
### From Buffers
|
|
317
|
+
|
|
318
|
+
```typescript
|
|
319
|
+
import { readFile } from 'fs/promises';
|
|
320
|
+
|
|
321
|
+
const buffer = await readFile('input.mp4');
|
|
322
|
+
const media = await Demuxer.open(buffer);
|
|
323
|
+
```
|
|
324
|
+
|
|
325
|
+
### Custom I/O Callbacks
|
|
326
|
+
|
|
327
|
+
```typescript
|
|
328
|
+
import type { IOInputCallbacks, IOOutputCallbacks } from 'node-av/api';
|
|
329
|
+
|
|
330
|
+
// Custom input source
|
|
331
|
+
const inputCallbacks: IOInputCallbacks = {
|
|
332
|
+
read: (size: number) => {
|
|
333
|
+
// Read from your custom source
|
|
334
|
+
return buffer; // or null for EOF
|
|
335
|
+
},
|
|
336
|
+
seek: (offset: bigint, whence: number) => {
|
|
337
|
+
// Seek in your custom source
|
|
338
|
+
return newPosition;
|
|
339
|
+
}
|
|
340
|
+
};
|
|
341
|
+
|
|
342
|
+
await using input = await Demuxer.open(inputCallbacks, {
|
|
343
|
+
format: 'mp4'
|
|
344
|
+
});
|
|
345
|
+
```
|
|
346
|
+
|
|
347
|
+
### Raw Media Processing
|
|
348
|
+
|
|
349
|
+
```typescript
|
|
350
|
+
// Raw video input
|
|
351
|
+
const rawVideo = await Demuxer.open({
|
|
352
|
+
type: 'video',
|
|
353
|
+
input: 'input.yuv',
|
|
354
|
+
width: 1280,
|
|
355
|
+
height: 720,
|
|
356
|
+
pixelFormat: AV_PIX_FMT_YUV420P,
|
|
357
|
+
frameRate: { num: 30, den: 1 }
|
|
358
|
+
});
|
|
359
|
+
|
|
360
|
+
// Raw audio input
|
|
361
|
+
const rawAudio = await Demuxer.open({
|
|
362
|
+
type: 'audio',
|
|
363
|
+
input: 'input.pcm',
|
|
364
|
+
sampleRate: 48000,
|
|
365
|
+
channels: 2,
|
|
366
|
+
sampleFormat: AV_SAMPLE_FMT_S16
|
|
367
|
+
}, {
|
|
368
|
+
format: 's16le'
|
|
369
|
+
});
|
|
370
|
+
```
|
|
371
|
+
|
|
372
|
+
## Device Capture
|
|
373
|
+
|
|
374
|
+
Capture video, audio, and screen content directly from system devices. Platform-specific formats are handled automatically.
|
|
375
|
+
|
|
376
|
+
```typescript
|
|
377
|
+
import { DeviceAPI } from 'node-av/api';
|
|
378
|
+
|
|
379
|
+
// List devices and query capabilities
|
|
380
|
+
const devices = await DeviceAPI.list();
|
|
381
|
+
const modes = await DeviceAPI.modes(devices.find(d => d.type === 'video')!.name);
|
|
382
|
+
|
|
383
|
+
// Camera
|
|
384
|
+
await using camera = await DeviceAPI.openCamera({ videoDevice: 0, width: 1280, height: 720, frameRate: 30 });
|
|
385
|
+
|
|
386
|
+
// Microphone
|
|
387
|
+
await using mic = await DeviceAPI.openMicrophone({ audioDevice: 0, sampleRate: 48000, channels: 2 });
|
|
388
|
+
|
|
389
|
+
// Combined video + audio (macOS/Windows)
|
|
390
|
+
await using device = await DeviceAPI.openDevice({ videoDevice: 0, audioDevice: 0, width: 1280, height: 720, frameRate: 30 });
|
|
391
|
+
|
|
392
|
+
// Screen capture
|
|
393
|
+
await using screen = await DeviceAPI.openScreen({ frameRate: 30, drawMouse: true });
|
|
394
|
+
|
|
395
|
+
// Screen capture with system audio (macOS 13.0+)
|
|
396
|
+
await using screen2 = await DeviceAPI.openScreen({
|
|
397
|
+
frameRate: 30,
|
|
398
|
+
avfoundation: { captureSystemAudio: true, audioSampleRate: 48000, audioChannels: 2 },
|
|
399
|
+
});
|
|
400
|
+
```
|
|
401
|
+
|
|
402
|
+
| Platform | Video | Audio | Screen |
|
|
403
|
+
|----------|-------|-------|--------|
|
|
404
|
+
| macOS | AVFoundation | AVFoundation | AVFoundation (ScreenCaptureKit) |
|
|
405
|
+
| Linux | V4L2 | ALSA | x11grab |
|
|
406
|
+
| Windows | DirectShow | DirectShow | GDIGrab |
|
|
407
|
+
|
|
408
|
+
## FFmpeg Binary Access
|
|
409
|
+
|
|
410
|
+
Need the FFmpeg command-line tool as well? Install `@seydx/node-av-ffmpeg` next to `node-av`. npm installs only the binary for your platform, built from the same FFmpeg as the native bindings, and `node-av/ffmpeg` finds it:
|
|
411
|
+
|
|
412
|
+
```bash
|
|
413
|
+
npm install node-av @seydx/node-av-ffmpeg
|
|
414
|
+
```
|
|
415
|
+
|
|
416
|
+
```typescript
|
|
417
|
+
import { ffmpegPath, isFfmpegAvailable } from 'node-av/ffmpeg';
|
|
418
|
+
import { execFile } from 'node:child_process';
|
|
419
|
+
import { promisify } from 'node:util';
|
|
420
|
+
|
|
421
|
+
const execFileAsync = promisify(execFile);
|
|
422
|
+
|
|
423
|
+
// Check if FFmpeg binary is available
|
|
424
|
+
if (isFfmpegAvailable()) {
|
|
425
|
+
console.log('FFmpeg binary found at:', ffmpegPath());
|
|
426
|
+
|
|
427
|
+
// Use FFmpeg binary directly
|
|
428
|
+
const { stdout } = await execFileAsync(ffmpegPath(), ['-version']);
|
|
429
|
+
console.log(stdout);
|
|
430
|
+
} else {
|
|
431
|
+
console.log('FFmpeg binary not available - run: npm install @seydx/node-av-ffmpeg');
|
|
432
|
+
}
|
|
433
|
+
|
|
434
|
+
// Direct usage example
|
|
435
|
+
async function convertVideo(input: string, output: string) {
|
|
436
|
+
const args = [
|
|
437
|
+
'-i', input,
|
|
438
|
+
'-c:v', 'libx264',
|
|
439
|
+
'-crf', '23',
|
|
440
|
+
'-c:a', 'aac',
|
|
441
|
+
output
|
|
442
|
+
];
|
|
443
|
+
|
|
444
|
+
await execFileAsync(ffmpegPath(), args);
|
|
445
|
+
}
|
|
446
|
+
```
|
|
447
|
+
|
|
448
|
+
`ffmpegPath()` throws if the package is not installed, so check `isFfmpegAvailable()` first when the CLI is optional. In Electron apps, unpack `node_modules/@seydx/**` from the ASAR archive so the binary can be executed.
|
|
449
|
+
|
|
450
|
+
Every [variant](#variants) has its own CLI package, built from the same FFmpeg as its native binding: `@seydx/node-av-ffmpeg-lgpl` for node-av-lgpl and `@seydx/node-av-ffmpeg-light` for node-av-light. `ffmpegPath()` looks for the one of the installed variant.
|
|
451
|
+
|
|
452
|
+
## Resource Management
|
|
453
|
+
|
|
454
|
+
The library supports automatic resource cleanup using the Disposable pattern:
|
|
455
|
+
|
|
456
|
+
```typescript
|
|
457
|
+
// Automatic cleanup with 'using'
|
|
458
|
+
{
|
|
459
|
+
await using media = await Demuxer.open('input.mp4');
|
|
460
|
+
using decoder = await Decoder.create(media.video());
|
|
461
|
+
// Resources automatically cleaned up at end of scope
|
|
462
|
+
}
|
|
463
|
+
|
|
464
|
+
// Manual cleanup
|
|
465
|
+
const media = await Demuxer.open('input.mp4');
|
|
466
|
+
try {
|
|
467
|
+
// Process media
|
|
468
|
+
} finally {
|
|
469
|
+
await media.close();
|
|
470
|
+
}
|
|
471
|
+
```
|
|
472
|
+
|
|
473
|
+
## Imports and Tree Shaking
|
|
474
|
+
|
|
475
|
+
The library provides multiple entry points for optimal tree shaking:
|
|
476
|
+
|
|
477
|
+
```typescript
|
|
478
|
+
// High-Level API only - Recommended for most use cases
|
|
479
|
+
import { Demuxer, Muxer, Decoder, Encoder } from 'node-av/api';
|
|
480
|
+
|
|
481
|
+
// Low-Level API only - Direct FFmpeg bindings
|
|
482
|
+
import { FormatContext, CodecContext, Frame, Packet } from 'node-av/lib';
|
|
483
|
+
|
|
484
|
+
// Constants only - When you just need FFmpeg constants
|
|
485
|
+
import { AV_PIX_FMT_YUV420P, AV_CODEC_ID_H264 } from 'node-av/constants';
|
|
486
|
+
|
|
487
|
+
// Channel layouts only - For audio channel configurations
|
|
488
|
+
import { AV_CHANNEL_LAYOUT_STEREO, AV_CHANNEL_LAYOUT_5POINT1 } from 'node-av/layouts';
|
|
489
|
+
|
|
490
|
+
// Default export - Includes everything
|
|
491
|
+
import * as ffmpeg from 'node-av';
|
|
492
|
+
```
|
|
493
|
+
|
|
494
|
+
## Key Features
|
|
495
|
+
|
|
496
|
+
Beyond basic transcoding, NodeAV provides advanced media processing capabilities:
|
|
497
|
+
|
|
498
|
+
**Fully Typed FFmpeg Options**
|
|
499
|
+
Every FFmpeg option is typed - across codecs, formats, ~580 filters, and bitstream filters. Keys autocomplete, enum values are validated, and typos become compile-time errors, all generated directly from FFmpeg's `AVOption` metadata (including each option's description and a link to the FFmpeg docs as JSDoc). No more grepping the FFmpeg docs for flag names.
|
|
500
|
+
|
|
501
|
+
**Speech Recognition with Whisper**
|
|
502
|
+
Integrate automatic speech-to-text transcription using OpenAI's Whisper model through the whisper.cpp implementation. The library handles automatic model downloading from HuggingFace, supports multiple model sizes (tiny, base, small, medium, large) for different accuracy/performance tradeoffs, and provides hardware-accelerated inference through Metal (macOS), Vulkan (cross-platform), or OpenCL backends. Transcription results include precise timestamps and can be processed in real-time from any audio source.
|
|
503
|
+
|
|
504
|
+
**Advanced Video Filtering with FilterComplexAPI**
|
|
505
|
+
Build sophisticated video processing pipelines using FFmpeg's complete filter ecosystem. The FilterComplexAPI provides direct access to complex filtergraphs with multiple inputs and outputs, enabling advanced operations like picture-in-picture overlays, multi-stream composition (side-by-side, grid layouts), real-time video effects, and custom processing chains. All filters support hardware acceleration where available, and filter configurations can be dynamically constructed based on runtime requirements.
|
|
506
|
+
|
|
507
|
+
**Browser Streaming**
|
|
508
|
+
Stream any media source directly to web browsers through fragmented MP4 (fMP4) or WebRTC protocols. The library can process inputs from RTSP cameras, local files, network streams, or custom sources and package them for browser consumption with minimal latency. Complete examples demonstrate both Media Source Extensions (MSE) based playback for on-demand content and WebRTC integration for real-time streaming scenarios.
|
|
509
|
+
|
|
510
|
+
**RTSP Backchannel / Talkback**
|
|
511
|
+
Implements bidirectional RTSP communication for IP camera integration. The library provides native support for RTSP backchannel streams, enabling audio transmission to camera devices. Transport is handled automatically with support for both TCP (interleaved mode) and UDP protocols, with proper RTP packet formatting and stream synchronization.
|
|
512
|
+
|
|
513
|
+
**Image Scaling & Snapshots**
|
|
514
|
+
The `Scaler` turns decoded frames into raw pixel buffers (`rgb`/`rgba`/`gray`/`nv12`/`yuv420p`) or JPEG/PNG images for detection, thumbnail, and snapshot workloads. One reusable instance pools its swscale contexts, GPU filter graphs, and encoders for zero per-frame allocation; hardware frames are cropped/scaled on the GPU with only the small result downloaded. Recurring resolutions can also be served directly by `EncoderPool`.
|
|
515
|
+
|
|
516
|
+
See the [Examples](#examples) section for complete implementations.
|
|
517
|
+
|
|
518
|
+
## Performance
|
|
519
|
+
|
|
520
|
+
NodeAV executes all media operations directly through FFmpeg's native C libraries. The Node.js bindings add minimal overhead - mostly just the JavaScript-to-C boundary crossings. During typical operations like transcoding or filtering, most processing time is spent in FFmpeg's optimized C code.
|
|
521
|
+
|
|
522
|
+
### Benchmarks
|
|
523
|
+
|
|
524
|
+
Performance comparison with FFmpeg CLI (4K 60fps, 30s test files on Apple M3 Max):
|
|
525
|
+
|
|
526
|
+
| Operation | FFmpeg CLI (FPS) | node-av (FPS) | FFmpeg CLI (Time) | node-av (Time) | Diff |
|
|
527
|
+
|-----------|------------------|---------------|-------------------|----------------|------|
|
|
528
|
+
| SW H.264 Transcode | 96 fps | 96 fps | 18.7s | 18.7s | ≈0% |
|
|
529
|
+
| SW H.265 Transcode | 40 fps | 41 fps | 44.5s | 43.7s | **+1.5%** |
|
|
530
|
+
| HW H.264 Transcode | 55 fps | 55 fps | 33.0s | 32.8s | **+0.5%** |
|
|
531
|
+
| Stream Copy (Remux) | 48k fps | 31k fps | 38ms | 106ms | -35% |
|
|
532
|
+
|
|
533
|
+
**Memory Usage:**
|
|
534
|
+
| Operation | FFmpeg CLI | node-av | Difference |
|
|
535
|
+
|-----------|-----------|---------|------------|
|
|
536
|
+
| H.264 Transcode (4K) | 3.6 GB | 3.4 GB | **-5%** |
|
|
537
|
+
| Stream Copy | 28 MB | 1 MB | **-96%** |
|
|
538
|
+
|
|
539
|
+
📊 **[Full benchmark results](https://github.com/seydx/node-av/tree/main/BENCHMARK.md)**
|
|
540
|
+
|
|
541
|
+
### Sync vs Async Operations
|
|
542
|
+
|
|
543
|
+
Every async method in NodeAV has a corresponding synchronous variant with the `Sync` suffix:
|
|
544
|
+
|
|
545
|
+
- **Async methods** (default) - Non-blocking operations using N-API's AsyncWorker. Methods like `decode()`, `encode()`, `read()`, `packets()` return Promises or AsyncGenerators.
|
|
546
|
+
|
|
547
|
+
- **Sync methods** - Direct FFmpeg calls without AsyncWorker overhead. Same methods with `Sync` suffix: `decodeSync()`, `encodeSync()`, `readSync()`, `packetsSync()`.
|
|
548
|
+
|
|
549
|
+
The key difference: Async methods don't block the Node.js event loop, allowing other operations to run concurrently. Sync methods block until completion but avoid AsyncWorker overhead, making them faster for sequential processing.
|
|
550
|
+
|
|
551
|
+
## Memory Safety Considerations
|
|
552
|
+
|
|
553
|
+
NodeAV provides direct bindings to FFmpeg's C APIs, which work with raw memory pointers. The high-level API adds safety abstractions and automatic resource management, but incorrect usage can still cause crashes. Common issues include mismatched video dimensions, incompatible pixel formats, or improper frame buffer handling. The library validates parameters where possible, but can't guarantee complete memory safety without limiting functionality. When using the low-level API, pay attention to parameter consistency, resource cleanup, and format compatibility. Following the documented patterns helps avoid memory-related issues.
|
|
554
|
+
|
|
555
|
+
## Worker Threads and the libuv Thread Pool
|
|
556
|
+
|
|
557
|
+
NodeAV works inside `worker_threads`, with two caveats:
|
|
558
|
+
|
|
559
|
+
- **Prefer graceful shutdown over `worker.terminate()`.** Terminating a worker while native calls are in flight is mitigated (pending completions are dropped safely instead of aborting), but a termination that lands exactly inside a running synchronous native call can still abort the process — a limitation of Node-API error handling during isolate teardown. Signal the worker to close its NodeAV resources and exit on its own. For test runners, process isolation (e.g. vitest `pool: 'forks'`) is the most robust choice.
|
|
560
|
+
- **Async operations share the libuv thread pool** (default: 4 threads, process-wide across all workers). A custom IO read callback that never settles parks one pool thread until it resolves or its input is closed or interrupted (a closing `Demuxer` or an exiting process or worker fails it with `AVERROR_EXIT`); enough of them stall every async NodeAV operation in the entire process. Make custom IO callbacks always settle (return `null` for EOF, throw on error), and raise `UV_THREADPOOL_SIZE` when running many parallel live inputs.
|
|
561
|
+
|
|
562
|
+
## Electron
|
|
563
|
+
|
|
564
|
+
NodeAV fully supports Electron applications. The prebuilt binaries are ABI-compatible with Electron, so no native rebuild is required during packaging. Both the native bindings and the bundled FFmpeg CLI binaries work seamlessly within Electron's main process.
|
|
565
|
+
|
|
566
|
+
NodeAV also supports Electron's [offscreen rendering with shared textures](https://www.electronjs.org/docs/latest/tutorial/offscreen-rendering), enabling zero-copy GPU texture import for hardware-accelerated encoding. See the [Electron Builder](https://github.com/seydx/node-av/tree/main/examples/electron/builder) and [Electron Forge](https://github.com/seydx/node-av/tree/main/examples/electron/forge) examples for complete implementations.
|
|
567
|
+
|
|
568
|
+
## Examples
|
|
569
|
+
|
|
570
|
+
The repository ships 60+ runnable examples covering both API levels — high-level (`api-*`: transcoding, hardware acceleration, RTSP, fMP4/DASH, RTP, Whisper, device capture, …) and low-level ports of the official FFmpeg C examples.
|
|
571
|
+
|
|
572
|
+
**[→ Browse all examples](https://github.com/seydx/node-av/tree/main/examples)**
|
|
573
|
+
|
|
574
|
+
## Prebuilt Binaries
|
|
575
|
+
|
|
576
|
+
Prebuilt binaries are available for multiple platforms:
|
|
577
|
+
|
|
578
|
+
- **macOS**: x64, ARM64
|
|
579
|
+
- **Linux**: x64, ARM64 (glibc and musl, e.g. Alpine)
|
|
580
|
+
- **Windows**: x64, ARM64
|
|
581
|
+
|
|
582
|
+
Each [variant](#variants) is built for all of them.
|
|
583
|
+
|
|
584
|
+
### Cross-Platform Packaging (CI/CD)
|
|
585
|
+
|
|
586
|
+
By default npm installs the native binding (and the FFmpeg binary, if
|
|
587
|
+
`@seydx/node-av-ffmpeg` is installed) for the host platform. When packaging for
|
|
588
|
+
a different target (e.g. building a Windows Electron app from a Linux CI
|
|
589
|
+
runner), pass npm's standard `--os` / `--cpu` / `--libc` flags:
|
|
590
|
+
|
|
591
|
+
```bash
|
|
592
|
+
# Install Windows x64 binaries while running on Linux
|
|
593
|
+
npm install --os=win32 --cpu=x64
|
|
594
|
+
|
|
595
|
+
# Install the musl binaries for an Alpine image while running on a glibc Linux
|
|
596
|
+
npm install --os=linux --cpu=x64 --libc=musl
|
|
597
|
+
```
|
|
598
|
+
|
|
599
|
+
npm filters the platform-specific `optionalDependencies` of both packages by
|
|
600
|
+
these flags.
|
|
601
|
+
|
|
602
|
+
## Variants
|
|
603
|
+
|
|
604
|
+
node-av is published in three variants with the same API. They differ in the FFmpeg they link, and with it in its license:
|
|
605
|
+
|
|
606
|
+
| Package | FFmpeg license | FFmpeg CLI | Contents |
|
|
607
|
+
| --- | --- | --- | --- |
|
|
608
|
+
| `node-av` | GPL v3 or later | `@seydx/node-av-ffmpeg` | All of FFmpeg, including the x264 and x265 encoders and the GPL-only filters |
|
|
609
|
+
| `node-av-lgpl` | LGPL v3 or later | `@seydx/node-av-ffmpeg-lgpl` | Everything except the GPL components and libfdk_aac: no libx264, libx265, chromaprint or GPL-only filters (`hqdn3d`, `delogo`, `eq`, ...). H.264 and H.265 are encoded only with a hardware encoder, AAC through FFmpeg's own encoder |
|
|
610
|
+
| `node-av-light` | GPL v3 or later | `@seydx/node-av-ffmpeg-light` | A smaller build: all of FFmpeg's own codecs, formats and filters, every hardware backend, x264, x265, Opus and libfdk_aac, but no Whisper, libplacebo, libvpx, SVT-AV1, JPEG XL, WebP, Vorbis, Theora, libass, zimg, SRT, Blu-ray, teletext or tracker music |
|
|
611
|
+
|
|
612
|
+
To keep the `node-av/...` imports with another variant, install it under the name node-av, together with its FFmpeg CLI if you need one:
|
|
613
|
+
|
|
614
|
+
```bash
|
|
615
|
+
npm install node-av@npm:node-av-lgpl
|
|
616
|
+
npm install @seydx/node-av-ffmpeg-lgpl
|
|
617
|
+
```
|
|
618
|
+
|
|
619
|
+
The JavaScript code is MIT licensed in every variant. The native binding and the FFmpeg CLI link FFmpeg statically, so they fall under the FFmpeg license in the table. With node-av and node-av-light, an application you distribute with them falls under the GPL as well. node-av-lgpl is plain LGPL and can be used in closed-source applications under the usual LGPL terms.
|
|
620
|
+
|
|
621
|
+
> [!NOTE]
|
|
622
|
+
> **node-av and node-av-light include libfdk_aac**, the Fraunhofer AAC encoder and decoder. It is built from [fdk-aac-stripped](https://gitlab.freedesktop.org/wtaymans/fdk-aac-stripped), fdk-aac with all patent-encumbered parts (HE-AAC, HE-AACv2, xHE-AAC) removed, and keeps the [Fraunhofer FDK AAC license](https://github.com/mstorsjo/fdk-aac/blob/master/NOTICE). [Fedora Legal](https://lists.fedoraproject.org/archives/list/legal@lists.fedoraproject.org/thread/OVW25JRWOKOLVMW3XGUX7E4OXFUR2RCG/) considers this stripped version GPL-compatible, and Fedora ships it. Upstream FFmpeg still marks libfdk_aac as nonfree in GPL builds; the Jellyfin FFmpeg patches node-av is built with drop that flag ([jellyfin/jellyfin-ffmpeg#61](https://github.com/jellyfin/jellyfin-ffmpeg/pull/61)). node-av-lgpl leaves it out and stays plain LGPL. See [#134](https://github.com/seydx/node-av/issues/134) for the background.
|
|
623
|
+
|
|
624
|
+
## Troubleshooting
|
|
625
|
+
|
|
626
|
+
### Hardware Acceleration on Linux (Intel/VAAPI)
|
|
627
|
+
|
|
628
|
+
For hardware-accelerated video processing with Intel GPUs on Linux, you need to install specific system packages. The FFmpeg binaries included with this library are built with **libva 2.20**, which requires **Ubuntu 24.04+** or **Debian 13+** as minimum OS versions.
|
|
629
|
+
|
|
630
|
+
#### Installation Steps
|
|
631
|
+
|
|
632
|
+
1. **Add Kisak-Mesa PPA** (recommended for newer Mesa versions with better hardware support):
|
|
633
|
+
|
|
634
|
+
```bash
|
|
635
|
+
sudo add-apt-repository ppa:kisak/kisak-mesa
|
|
636
|
+
sudo apt update
|
|
637
|
+
```
|
|
638
|
+
|
|
639
|
+
2. **Install required packages**:
|
|
640
|
+
|
|
641
|
+
```bash
|
|
642
|
+
sudo apt install libmfx-gen1.2 mesa-va-drivers mesa-vulkan-drivers libva2 libva-drm2 vainfo libvulkan1 vulkan-tools
|
|
643
|
+
```
|
|
644
|
+
|
|
645
|
+
After installation, verify hardware acceleration is working:
|
|
646
|
+
|
|
647
|
+
```bash
|
|
648
|
+
# Check VAAPI support
|
|
649
|
+
vainfo
|
|
650
|
+
|
|
651
|
+
# Check Vulkan support
|
|
652
|
+
vulkaninfo
|
|
653
|
+
|
|
654
|
+
# Should show available profiles and entrypoints for your Intel GPU
|
|
655
|
+
```
|
|
656
|
+
|
|
657
|
+
**Note**: If you're running an older Ubuntu version (< 24.04) or Debian version (< 13), you'll need to upgrade your OS to use hardware acceleration with this library.
|
|
658
|
+
|
|
659
|
+
#### Alpine Linux
|
|
660
|
+
|
|
661
|
+
```bash
|
|
662
|
+
# VAAPI, QSV and Vulkan
|
|
663
|
+
apk add libva intel-media-driver onevpl-intel-gpu vulkan-loader mesa-vulkan-intel
|
|
664
|
+
|
|
665
|
+
# OpenCL (OpenCL filters, Whisper)
|
|
666
|
+
apk add intel-compute-runtime intel-graphics-compiler
|
|
667
|
+
```
|
|
668
|
+
|
|
669
|
+
The OpenCL packages need Alpine 3.23 or newer. Alpine's `intel-compute-runtime` does not pull in `intel-graphics-compiler`, so install both. Without the compiler, OpenCL cannot build any kernels. Use `vainfo` (`libva-utils`) and `vulkaninfo` (`vulkan-tools`) to verify the setup.
|
|
670
|
+
|
|
671
|
+
## License
|
|
672
|
+
|
|
673
|
+
The node-av source code is licensed under the MIT License. See the LICENSE file for details.
|
|
674
|
+
|
|
675
|
+
The prebuilt native bindings and FFmpeg CLI packages link FFmpeg statically and carry its license: the GPL v3 or later for node-av and node-av-light, the LGPL v3 or later for node-av-lgpl. node-av and node-av-light also link libfdk_aac under the Fraunhofer FDK AAC license. Each of these packages ships the license texts and a LICENSE.md that links the exact sources it was built from. See [Variants](#variants) for what each one contains and [FFmpeg License](https://ffmpeg.org/legal.html) for FFmpeg's terms.
|
|
676
|
+
|
|
677
|
+
## Contributing
|
|
678
|
+
|
|
679
|
+
Contributions are welcome! Please read [CONTRIBUTING.md](https://github.com/seydx/node-av/tree/main/CONTRIBUTING.md) for development setup, code standards, and contribution guidelines before submitting pull requests.
|
|
680
|
+
|
|
681
|
+
## Support
|
|
682
|
+
|
|
683
|
+
For issues and questions, please use the GitHub issue tracker.
|
|
684
|
+
|
|
685
|
+
## See Also
|
|
686
|
+
|
|
687
|
+
- [FFmpeg Documentation](https://ffmpeg.org/documentation.html)
|
|
688
|
+
- [FFmpeg Doxygen](https://ffmpeg.org/doxygen/trunk/)
|
|
689
|
+
- [Jellyfin FFmpeg](https://github.com/seydx/jellyfin-ffmpeg)
|
|
690
|
+
- [FFmpeg MSVC](https://github.com/seydx/ffmpeg-msvc-prebuilt)
|
|
691
|
+
|
|
692
|
+
## Star History
|
|
693
|
+
|
|
694
|
+
<picture>
|
|
695
|
+
<source
|
|
696
|
+
media="(prefers-color-scheme: dark)"
|
|
697
|
+
srcset="
|
|
698
|
+
https://star-history.dera.page/svg?repos=seydx/node-av&type=Date&theme=dark
|
|
699
|
+
"
|
|
700
|
+
/>
|
|
701
|
+
<source
|
|
702
|
+
media="(prefers-color-scheme: light)"
|
|
703
|
+
srcset="
|
|
704
|
+
https://star-history.dera.page/svg?repos=seydx/node-av&type=Date
|
|
705
|
+
"
|
|
706
|
+
/>
|
|
707
|
+
<img
|
|
708
|
+
alt="Star History Chart"
|
|
709
|
+
src="https://star-history.dera.page/svg?repos=seydx/node-av&type=Date"
|
|
710
|
+
/>
|
|
711
|
+
</picture>
|