tunelet 0.2.0 → 0.3.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +405 -157
- package/package.json +4 -3
- package/pkg/tunelet.d.ts +1279 -0
- package/pkg/tunelet.js +968 -0
- package/pkg/tunelet_bg.wasm +0 -0
package/README.md
CHANGED
|
@@ -1,32 +1,208 @@
|
|
|
1
1
|
# tunelet
|
|
2
2
|
|
|
3
3
|
tunelet is a Rust library for vocal pitch correction, choir, harmony,
|
|
4
|
-
formant control and effects.
|
|
5
|
-
|
|
6
|
-
|
|
7
|
-
|
|
4
|
+
formant control and effects. It is also available as an npm package for
|
|
5
|
+
browsers and Node.js.
|
|
6
|
+
|
|
7
|
+
[](https://crates.io/crates/tunelet)
|
|
8
|
+
[](https://www.npmjs.com/package/tunelet)
|
|
9
|
+
[](https://docs.rs/tunelet)
|
|
10
|
+
[](LICENSE)
|
|
11
|
+
|
|
12
|
+
**[Website](https://tunelet.byteland.app)** ·
|
|
13
|
+
[crates.io](https://crates.io/crates/tunelet) ·
|
|
14
|
+
[npm](https://www.npmjs.com/package/tunelet) ·
|
|
15
|
+
[GitHub Packages](https://github.com/ByteLandTechnology/tunelet/pkgs/npm/tunelet) ·
|
|
16
|
+
[API documentation](https://docs.rs/tunelet)
|
|
17
|
+
|
|
18
|
+

|
|
19
|
+
|
|
20
|
+
## Contents
|
|
21
|
+
|
|
22
|
+
- [Features](#features)
|
|
23
|
+
- [Installation](#installation)
|
|
24
|
+
- [Quick start](#quick-start)
|
|
25
|
+
- [Signal chain](#signal-chain)
|
|
26
|
+
- [Process audio](#process-audio)
|
|
27
|
+
- [Timed events](#timed-events)
|
|
28
|
+
- [Lifecycle](#lifecycle)
|
|
29
|
+
- [Feedback and telemetry](#feedback-and-telemetry)
|
|
30
|
+
- [Real-time rules](#real-time-rules)
|
|
31
|
+
- [WebAssembly and TypeScript](#webassembly-and-typescript)
|
|
32
|
+
- [Build from source](#build-from-source)
|
|
33
|
+
- [License](#license)
|
|
34
|
+
|
|
35
|
+
## Features
|
|
36
|
+
|
|
37
|
+
- **Pitch correction**: The processor moves the voice to the nearest note in
|
|
38
|
+
a scale. A MIDI keyboard can also select the notes.
|
|
39
|
+
- **Choir**: The processor adds a maximum of 15 voices with random pitch
|
|
40
|
+
drift and delay.
|
|
41
|
+
- **Harmony**: The processor adds a maximum of four voices from intervals,
|
|
42
|
+
chords or MIDI notes.
|
|
43
|
+
- **Formant control**: An LPC model changes the width and the length of the
|
|
44
|
+
vocal tract.
|
|
45
|
+
- **Effects**: Punch, Warm, Alien and six reverb modes are available.
|
|
46
|
+
- **Sample-exact output**: All stages use fixed-point arithmetic. The
|
|
47
|
+
same input, configuration and seed give the same samples.
|
|
48
|
+
- **Real-time safety**: After construction, the audio calls do not allocate
|
|
49
|
+
memory, acquire locks or do I/O.
|
|
50
|
+
- **No dependencies**: The default Rust build has no dependencies and uses
|
|
51
|
+
only safe Rust.
|
|
52
|
+
|
|
53
|
+
The input is mono signed 16-bit PCM at 44.1 kHz or 48 kHz. The output is
|
|
54
|
+
stereo or mono 16-bit PCM.
|
|
55
|
+
|
|
56
|
+
## Installation
|
|
57
|
+
|
|
58
|
+
tunelet is available from these registries:
|
|
59
|
+
|
|
60
|
+
| Registry | Package | Use |
|
|
61
|
+
| --------------- | ----------------------------------------------------------------------------------------------- | ----------------- |
|
|
62
|
+
| crates.io | [`tunelet`](https://crates.io/crates/tunelet) | Rust |
|
|
63
|
+
| npm | [`tunelet`](https://www.npmjs.com/package/tunelet) | Browsers, Node.js |
|
|
64
|
+
| GitHub Packages | [`@bytelandtechnology/tunelet`](https://github.com/ByteLandTechnology/tunelet/pkgs/npm/tunelet) | Browsers, Node.js |
|
|
65
|
+
|
|
66
|
+
For Rust, add the crate:
|
|
8
67
|
|
|
9
|
-
|
|
68
|
+
```sh
|
|
69
|
+
cargo add tunelet
|
|
70
|
+
```
|
|
71
|
+
|
|
72
|
+
For browsers and Node.js, install the npm package:
|
|
73
|
+
|
|
74
|
+
```sh
|
|
75
|
+
npm install tunelet
|
|
76
|
+
```
|
|
77
|
+
|
|
78
|
+
The GitHub Packages registry has the same package with a scoped name. To
|
|
79
|
+
use it, add this line to the `.npmrc` file of your project:
|
|
10
80
|
|
|
11
|
-
|
|
12
|
-
|
|
13
|
-
|
|
14
|
-
4. Alien
|
|
15
|
-
5. Reverb
|
|
16
|
-
6. Output gain
|
|
81
|
+
```ini
|
|
82
|
+
@bytelandtechnology:registry=https://npm.pkg.github.com
|
|
83
|
+
```
|
|
17
84
|
|
|
18
|
-
|
|
19
|
-
|
|
85
|
+
Then install the package. GitHub Packages needs a token with the
|
|
86
|
+
`read:packages` scope, also for public packages.
|
|
20
87
|
|
|
21
|
-
|
|
88
|
+
```sh
|
|
89
|
+
npm install @bytelandtechnology/tunelet
|
|
90
|
+
```
|
|
22
91
|
|
|
23
|
-
|
|
92
|
+
If you use the scoped package, import from `@bytelandtechnology/tunelet`.
|
|
93
|
+
|
|
94
|
+
## Quick start
|
|
95
|
+
|
|
96
|
+
### Rust
|
|
97
|
+
|
|
98
|
+
```rust
|
|
99
|
+
use tunelet::{
|
|
100
|
+
CorrectionScale, NoteMask, PitchClass, PrepareConfig, Processor, ProcessorConfig,
|
|
101
|
+
SampleRate,
|
|
102
|
+
};
|
|
103
|
+
|
|
104
|
+
fn main() -> Result<(), tunelet::Error> {
|
|
105
|
+
// 1. Set the sample rate and the largest block.
|
|
106
|
+
let rate = SampleRate::try_from(48_000)?;
|
|
107
|
+
let preparation = PrepareConfig::new(rate, 1024)?;
|
|
108
|
+
|
|
109
|
+
// 2. Start from the default configuration and change fields.
|
|
110
|
+
let mut config = ProcessorConfig::default();
|
|
111
|
+
config.resynthesis.correction.notes =
|
|
112
|
+
NoteMask::from_key_scale(PitchClass::A, CorrectionScale::Minor);
|
|
113
|
+
config.resynthesis.correction.retune_ms = 20.0;
|
|
114
|
+
|
|
115
|
+
// 3. Construct the processor. Only this step allocates memory.
|
|
116
|
+
let mut processor = Processor::new(preparation, config)?;
|
|
117
|
+
|
|
118
|
+
// 4. Process blocks. Each input sample gives one stereo frame.
|
|
119
|
+
let input = [0i16; 1024];
|
|
120
|
+
let mut output = [[0i16; 2]; 1024];
|
|
121
|
+
processor.process(&input, &mut output, &[])?;
|
|
122
|
+
|
|
123
|
+
// 5. After the last block, get the tail.
|
|
124
|
+
processor.begin_drain();
|
|
125
|
+
loop {
|
|
126
|
+
let frames = processor.drain(&mut output)?;
|
|
127
|
+
if frames == 0 {
|
|
128
|
+
break;
|
|
129
|
+
}
|
|
130
|
+
// Use only output[..frames].
|
|
131
|
+
}
|
|
132
|
+
Ok(())
|
|
133
|
+
}
|
|
134
|
+
```
|
|
24
135
|
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
|
|
136
|
+
### TypeScript
|
|
137
|
+
|
|
138
|
+
```ts
|
|
139
|
+
import { loadTunelet, Tunelet, defaultConfig } from "tunelet";
|
|
140
|
+
|
|
141
|
+
await loadTunelet();
|
|
142
|
+
const config = defaultConfig();
|
|
143
|
+
config.resynthesis.pitchRatio = 1.25;
|
|
144
|
+
const processor = new Tunelet(48000, 1024, config);
|
|
145
|
+
|
|
146
|
+
const input = new Int16Array(1024);
|
|
147
|
+
const left = new Int16Array(1024);
|
|
148
|
+
const right = new Int16Array(1024);
|
|
149
|
+
// Fill input with mono PCM before each call; reuse all three arrays.
|
|
150
|
+
processor.process(input, left, right);
|
|
151
|
+
|
|
152
|
+
// After all processing and optional drain calls:
|
|
153
|
+
processor.free();
|
|
28
154
|
```
|
|
29
155
|
|
|
156
|
+
## Signal chain
|
|
157
|
+
|
|
158
|
+
The processor sends each sample through these stages, in this order. The
|
|
159
|
+
default configuration enables chromatic pitch correction and one choir
|
|
160
|
+
voice. It disables the effects and the formant stage.
|
|
161
|
+
|
|
162
|
+
| Stage | Function | Default | Configuration field |
|
|
163
|
+
| -------------- | ----------------------------------------------------------- | -------------- | ------------------------ |
|
|
164
|
+
| Punch | Input gain and transient enhancement | Disabled | `punch` |
|
|
165
|
+
| Warm | Asymmetric saturation and a 50 Hz high-pass filter | Disabled | `warm` |
|
|
166
|
+
| Pitch tracking | Finds the pitch of the voice | Enabled | `resynthesis.pitch` |
|
|
167
|
+
| Correction | Moves the pitch to the nearest permitted note | Chromatic | `resynthesis.correction` |
|
|
168
|
+
| Voices | Renders a choir (1 to 16 voices) or harmony (lead and 0–4) | Choir, 1 voice | `resynthesis.mode` |
|
|
169
|
+
| Formant | Changes the vocal tract with an LPC model and aspiration | Disabled | `resynthesis.formant` |
|
|
170
|
+
| Alien | Plays recent audio in reverse loops | Disabled | `alien` |
|
|
171
|
+
| Reverb | Echo, Schroeder, SmallRoom, Binaural, MediumHall, LargeHall | Disabled | `reverb` |
|
|
172
|
+
| Output gain | Linear gain, 0..=64, before `-32767..=32767` clipping | 1 | `output_gain` |
|
|
173
|
+
|
|
174
|
+
The internal Q14 stages use `i64` products and saturate at the `i32`
|
|
175
|
+
limits. Decrease `output_gain` to prevent `-32767..=32767` clipping on
|
|
176
|
+
multi-voice mixes.
|
|
177
|
+
|
|
178
|
+
`resynthesis.dry_mix` sets the dry fraction at the resynthesis stage. The
|
|
179
|
+
dry path starts after Warm and has a delay of 1,543 frames.
|
|
180
|
+
|
|
181
|
+
### Harmony sources
|
|
182
|
+
|
|
183
|
+
Set `resynthesis.mode` to `Harmony`. Then select the source of the voices
|
|
184
|
+
with `resynthesis.harmony.source`:
|
|
185
|
+
|
|
186
|
+
| Source | The voices follow |
|
|
187
|
+
| --------------- | ------------------------------------------------------ |
|
|
188
|
+
| `FixedInterval` | A fixed number of semitones from the singer |
|
|
189
|
+
| `SmartInterval` | An interval in scale steps, in a key and a scale |
|
|
190
|
+
| `ChordByDegree` | A chord on a scale degree |
|
|
191
|
+
| `ChordByName` | A named chord, for example `MinorSeventh` |
|
|
192
|
+
| `ChordViaMidi` | The active MIDI notes, as a chord |
|
|
193
|
+
| `MidiOmni` | MIDI notes from all channels, on the first free voice |
|
|
194
|
+
| `MidiChannels` | MIDI channel `n` for voice `n` (channels 0–3), default |
|
|
195
|
+
|
|
196
|
+
`resynthesis.harmony.voices` sets the number of interval voices, in `0..=4`.
|
|
197
|
+
The default is 4. `FixedInterval` and `SmartInterval` use the first `voices`
|
|
198
|
+
voices. The other voices release with the usual gain envelope. The chord
|
|
199
|
+
and MIDI sources ignore this field.
|
|
200
|
+
|
|
201
|
+
`resynthesis.harmony_gain` scales the four harmony voices in mono and in
|
|
202
|
+
stereo. It does not scale the lead voice.
|
|
203
|
+
|
|
204
|
+
## Process audio
|
|
205
|
+
|
|
30
206
|
To prepare a processor, do these steps:
|
|
31
207
|
|
|
32
208
|
1. Use `SampleRate::try_from(hz)` to check the input rate.
|
|
@@ -35,7 +211,10 @@ To prepare a processor, do these steps:
|
|
|
35
211
|
3. Create a `Processor` with that preparation and a `ProcessorConfig`.
|
|
36
212
|
|
|
37
213
|
Configuration structures implement `Default`. To select voices and effects,
|
|
38
|
-
edit their fields.
|
|
214
|
+
edit their fields. To change the sample rate or the block limit, make a new
|
|
215
|
+
processor.
|
|
216
|
+
|
|
217
|
+
Select the process function for your output layout:
|
|
39
218
|
|
|
40
219
|
| Output layout | Function for input | Function for the tail |
|
|
41
220
|
| ------------------------------ | ----------------------- | --------------------- |
|
|
@@ -43,55 +222,87 @@ edit their fields.
|
|
|
43
222
|
| Separate left and right slices | `process_planar` | `drain_planar` |
|
|
44
223
|
| Mono slice | `process_mono_in_place` | `drain_mono` |
|
|
45
224
|
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
|
|
49
|
-
|
|
225
|
+
These rules apply to each call:
|
|
226
|
+
|
|
227
|
+
- Each input sample gives one output frame.
|
|
228
|
+
- The channel lengths must be equal.
|
|
229
|
+
- The block must not be longer than the prepared block limit.
|
|
230
|
+
- Short blocks and empty blocks are permitted.
|
|
231
|
+
- If the call fails, the output and the state do not change.
|
|
232
|
+
|
|
233
|
+
## Timed events
|
|
234
|
+
|
|
235
|
+
To apply a control immediately before a specific frame, give `TimedEvent`
|
|
236
|
+
values to the process call.
|
|
237
|
+
|
|
238
|
+

|
|
50
239
|
|
|
51
|
-
|
|
52
|
-
values. These rules apply to the offsets:
|
|
240
|
+
These rules apply to the offsets:
|
|
53
241
|
|
|
54
242
|
- Offsets start at zero in each block.
|
|
55
243
|
- Each offset must be in the block.
|
|
56
244
|
- An offset must not be less than the offset before it.
|
|
57
245
|
|
|
58
246
|
Events with equal offsets execute in slice order. An empty block accepts no
|
|
59
|
-
events.
|
|
60
|
-
|
|
247
|
+
events. A stage event replaces all settings of that stage. To change one
|
|
248
|
+
field, start from `processor.config()`.
|
|
61
249
|
|
|
62
|
-
|
|
250
|
+
Harmony MIDI uses `ControlEvent::Midi`. MIDI for the correction keyboard uses
|
|
251
|
+
`ControlEvent::CorrectionMidi`. For example, this event starts a harmony note
|
|
252
|
+
at frame 256:
|
|
63
253
|
|
|
64
|
-
|
|
65
|
-
|
|
254
|
+
```rust
|
|
255
|
+
use tunelet::{ControlEvent, MidiEvent, TimedEvent};
|
|
66
256
|
|
|
67
|
-
|
|
68
|
-
|
|
69
|
-
|
|
70
|
-
|
|
71
|
-
|
|
72
|
-
|
|
73
|
-
|
|
257
|
+
let events = [TimedEvent {
|
|
258
|
+
offset: 256,
|
|
259
|
+
event: ControlEvent::Midi(MidiEvent::NoteOn { channel: 0, note: 64, velocity: 100 }),
|
|
260
|
+
}];
|
|
261
|
+
processor.process(&input, &mut output, &events)?;
|
|
262
|
+
```
|
|
263
|
+
|
|
264
|
+
## Lifecycle
|
|
265
|
+
|
|
266
|
+

|
|
267
|
+
|
|
268
|
+
### Construction
|
|
74
269
|
|
|
75
|
-
|
|
270
|
+
Construction validates the configuration and allocates all the storage of
|
|
271
|
+
the processor. After construction, no call allocates memory.
|
|
76
272
|
|
|
77
|
-
|
|
273
|
+
### Drain
|
|
274
|
+
|
|
275
|
+
The drain gives the tail of the reverb and the voices after the last input
|
|
276
|
+
block. To get the tail, do these steps:
|
|
78
277
|
|
|
79
278
|
1. Call `begin_drain()` one time.
|
|
80
|
-
2.
|
|
279
|
+
2. Call `drain`, `drain_planar` or `drain_mono` until the call returns zero.
|
|
280
|
+
3. From each drain call, use only the frames that the call returns.
|
|
81
281
|
|
|
82
282
|
The drain fills the unused output with zeros. The drain has a finite budget,
|
|
83
|
-
which can include silence.
|
|
84
|
-
feedback. A successful call that processes audio or
|
|
85
|
-
cancels the drain. `reset()` also cancels the drain.
|
|
283
|
+
`tail_frames()`, which can include silence. The last drain call resets the
|
|
284
|
+
audio histories and the feedback. A successful call that processes audio or
|
|
285
|
+
sets the configuration cancels the drain. `reset()` also cancels the drain.
|
|
286
|
+
|
|
287
|
+
### Reset
|
|
288
|
+
|
|
289
|
+
`reset()` keeps the configuration, the random stream and some controller and
|
|
290
|
+
filter state. If you need a fully new state, construct a new processor.
|
|
291
|
+
|
|
292
|
+
## Feedback and telemetry
|
|
293
|
+
|
|
294
|
+
These calls read the state of the processor. They do not change the audio.
|
|
86
295
|
|
|
87
|
-
|
|
88
|
-
|
|
89
|
-
|
|
90
|
-
|
|
91
|
-
|
|
296
|
+
| Call | Data |
|
|
297
|
+
| -------------------- | ----------------------------------------------------------------- |
|
|
298
|
+
| `feedback()` | The pitch, the stereo peak, the correction ratio, the frame count |
|
|
299
|
+
| `take_feedback()` | The same data as `feedback()`, and clears the peak window |
|
|
300
|
+
| `telemetry()` | Data for meters and displays (refer to the list below) |
|
|
301
|
+
| `take_telemetry()` | The same data as `telemetry()`, and clears its three peak windows |
|
|
302
|
+
| `latency_frames()` | 1,593 frames of nominal resynthesis latency at both sample rates |
|
|
303
|
+
| `formant_envelope()` | The LPC envelopes before and after the formant controls |
|
|
92
304
|
|
|
93
|
-
|
|
94
|
-
audio. The data includes these items:
|
|
305
|
+
The telemetry includes these items:
|
|
95
306
|
|
|
96
307
|
- The pitch ratio and the gain of each voice.
|
|
97
308
|
- The peaks of the input, of the two output channels and of the reverb wet
|
|
@@ -99,45 +310,55 @@ audio. The data includes these items:
|
|
|
99
310
|
- The pitch clarity, which is a confidence score from 0 to 1.
|
|
100
311
|
- The current gains of Punch and Warm, and the position in the Alien loop.
|
|
101
312
|
|
|
102
|
-
|
|
103
|
-
|
|
313
|
+
The peak windows of the telemetry are independent of the peak window of the
|
|
314
|
+
feedback. The latency is nominal. Dry mixes, resampled voices and effects
|
|
315
|
+
have different timing.
|
|
104
316
|
|
|
105
317
|
`formant_envelope()` writes two LPC envelopes into slices of equal length:
|
|
106
|
-
the vocal-tract envelope from the analysis, and the envelope after the width
|
|
107
|
-
and stretch controls. The values are linear and have no absolute level. The
|
|
108
|
-
formant stage must be enabled. The envelopes are those of the model that the
|
|
109
|
-
audio used for the last frame. The Rust method returns `Result<bool, Error>`.
|
|
110
|
-
If an argument is not valid, the method returns an error and does not change
|
|
111
|
-
the slices. If no model is available, the method returns `Ok(false)` and
|
|
112
|
-
fills both slices with zeros. Valid envelopes return `Ok(true)`.
|
|
113
318
|
|
|
114
|
-
|
|
115
|
-
The
|
|
116
|
-
voices. The other voices release with the usual gain envelope. The chord
|
|
117
|
-
and MIDI sources ignore this field.
|
|
319
|
+
- The vocal-tract envelope from the analysis.
|
|
320
|
+
- The envelope after the width and stretch controls.
|
|
118
321
|
|
|
119
|
-
|
|
120
|
-
|
|
322
|
+
The values are linear and have no absolute level. The formant stage must be
|
|
323
|
+
enabled. The envelopes are those of the model that the audio used for the
|
|
324
|
+
last frame. The method returns `Result<bool, Error>`:
|
|
121
325
|
|
|
122
|
-
|
|
123
|
-
|
|
326
|
+
| Result | Description |
|
|
327
|
+
| ----------- | ---------------------------------------------------------------- |
|
|
328
|
+
| `Ok(true)` | The slices contain valid envelopes. |
|
|
329
|
+
| `Ok(false)` | No model is available. The method fills both slices with zeros. |
|
|
330
|
+
| `Err(_)` | An argument is not valid. The method does not change the slices. |
|
|
124
331
|
|
|
125
|
-
|
|
126
|
-
cargo doc --locked --no-deps --open
|
|
127
|
-
```
|
|
332
|
+
## Real-time rules
|
|
128
333
|
|
|
129
|
-
|
|
334
|
+
Construction allocates the storage of the processor. These operations do not
|
|
335
|
+
allocate memory, acquire locks or do I/O:
|
|
130
336
|
|
|
131
|
-
|
|
337
|
+
- Processing
|
|
338
|
+
- Configuration
|
|
339
|
+
- MIDI
|
|
340
|
+
- Feedback
|
|
341
|
+
- Telemetry
|
|
342
|
+
- Reset
|
|
343
|
+
- Drain
|
|
344
|
+
|
|
345
|
+
All operations that can fail do their validation before they change the
|
|
346
|
+
state or the output. The crate forbids unsafe Rust.
|
|
347
|
+
|
|
348
|
+
The API documentation gives the parameter ranges, the MIDI behavior and the
|
|
349
|
+
reset semantics. Read it on [docs.rs](https://docs.rs/tunelet), or run this
|
|
350
|
+
command:
|
|
132
351
|
|
|
133
352
|
```sh
|
|
134
|
-
cargo
|
|
135
|
-
cargo fmt --check
|
|
136
|
-
cargo clippy --locked --lib -- -D warnings
|
|
353
|
+
cargo doc --locked --no-deps --open
|
|
137
354
|
```
|
|
138
355
|
|
|
139
356
|
## WebAssembly and TypeScript
|
|
140
357
|
|
|
358
|
+
For `wasm32-unknown-unknown`, the optional `wasm` feature of the crate gives
|
|
359
|
+
JavaScript bindings and generated TypeScript declarations. The npm package
|
|
360
|
+
contains these bindings.
|
|
361
|
+
|
|
141
362
|
The npm package is one ES module for both browsers and Node.js. It contains
|
|
142
363
|
these items:
|
|
143
364
|
|
|
@@ -145,29 +366,14 @@ these items:
|
|
|
145
366
|
- The generated bindings.
|
|
146
367
|
- A shared `loadTunelet` entry point.
|
|
147
368
|
|
|
148
|
-
|
|
149
|
-
|
|
150
|
-
```ts
|
|
151
|
-
import { loadTunelet, Tunelet, defaultConfig } from "tunelet";
|
|
152
|
-
|
|
153
|
-
await loadTunelet();
|
|
154
|
-
const config = defaultConfig();
|
|
155
|
-
config.resynthesis.pitchRatio = 1.25;
|
|
156
|
-
const processor = new Tunelet(48000, 1024, config);
|
|
157
|
-
|
|
158
|
-
const input = new Int16Array(1024);
|
|
159
|
-
const left = new Int16Array(1024);
|
|
160
|
-
const right = new Int16Array(1024);
|
|
161
|
-
// Fill input with mono PCM before each call; reuse all three arrays.
|
|
162
|
-
processor.process(input, left, right);
|
|
369
|
+
### Load the module
|
|
163
370
|
|
|
164
|
-
|
|
165
|
-
|
|
166
|
-
|
|
371
|
+
Both environments use the same imports and the same initialization call. An
|
|
372
|
+
import of the package does not load the binary. Call `loadTunelet()` before
|
|
373
|
+
you use `Tunelet` or `defaultConfig()`.
|
|
167
374
|
|
|
168
|
-
Node.js reads the packaged binary from a file URL relative to the module.
|
|
169
|
-
Browsers fetch the served Wasm asset.
|
|
170
|
-
the binary. Call `loadTunelet()` before you use `Tunelet` or `defaultConfig()`.
|
|
375
|
+
- Node.js reads the packaged binary from a file URL relative to the module.
|
|
376
|
+
- Browsers fetch the served Wasm asset.
|
|
171
377
|
|
|
172
378
|
Concurrent calls share one initialization promise. The package keeps the
|
|
173
379
|
result of a successful initialization. If a load fails, the promise rejects,
|
|
@@ -193,6 +399,8 @@ initializer and `initSync`. The package root re-exports `initSync` too. Use
|
|
|
193
399
|
only one initialization entry point for each module instance. The package
|
|
194
400
|
gives only ESM exports. CommonJS applications can use `await import("tunelet")`.
|
|
195
401
|
|
|
402
|
+
### Deployment
|
|
403
|
+
|
|
196
404
|
Browser bundlers must keep or emit the Wasm asset that
|
|
197
405
|
`new URL(..., import.meta.url)` refers to. For a browser deployment without a
|
|
198
406
|
bundler, do these steps:
|
|
@@ -203,44 +411,7 @@ bundler, do these steps:
|
|
|
203
411
|
Serve the assets over HTTP(S). The package does not support a local `file:`
|
|
204
412
|
page that opens the JavaScript.
|
|
205
413
|
|
|
206
|
-
###
|
|
207
|
-
|
|
208
|
-
Install Node.js and `wasm-pack`. Then run these commands to add the
|
|
209
|
-
`wasm32-unknown-unknown` Rust target, install the development dependencies
|
|
210
|
-
and make the archive:
|
|
211
|
-
|
|
212
|
-
```sh
|
|
213
|
-
rustup target add wasm32-unknown-unknown
|
|
214
|
-
npm ci
|
|
215
|
-
npm pack
|
|
216
|
-
```
|
|
217
|
-
|
|
218
|
-
`npm pack` builds the Wasm bindings and the TypeScript entry point. Then it
|
|
219
|
-
creates `tunelet-<version>.tgz`. To use this archive in a browser project or
|
|
220
|
-
a Node.js project, install it with this command:
|
|
221
|
-
|
|
222
|
-
```sh
|
|
223
|
-
npm install /path/to/tunelet-<version>.tgz
|
|
224
|
-
```
|
|
225
|
-
|
|
226
|
-
For local use, it is not necessary to publish the package to npm. To build
|
|
227
|
-
without an archive, run `npm run build`.
|
|
228
|
-
|
|
229
|
-
The build writes one `web`-target binding into `pkg/` and the shared entry
|
|
230
|
-
into `dist/`. Git ignores both directories. The archive contains only these
|
|
231
|
-
items:
|
|
232
|
-
|
|
233
|
-
- The runtime entry.
|
|
234
|
-
- The declarations.
|
|
235
|
-
- The bindings.
|
|
236
|
-
- The binary.
|
|
237
|
-
- The package metadata.
|
|
238
|
-
- The README.
|
|
239
|
-
|
|
240
|
-
The `build:wasm` script uses `--no-opt`. This flag disables the separate
|
|
241
|
-
Binaryen pass, but Rust still builds in release mode. The TypeScript and
|
|
242
|
-
Node.js type declarations are only development dependencies. Native Rust
|
|
243
|
-
builds use Cargo and do not need Node.js or npm.
|
|
414
|
+
### TypeScript API
|
|
244
415
|
|
|
245
416
|
| Operation | TypeScript API |
|
|
246
417
|
| --------------------------------- | -------------------------------------------------------- |
|
|
@@ -250,16 +421,27 @@ builds use Cargo and do not need Node.js or npm.
|
|
|
250
421
|
| Configuration | `config()`, `setConfig(config)`, `defaultConfig()` |
|
|
251
422
|
| Information about the preparation | `sampleRate`, `maxBlockFrames`, `latencyFrames` |
|
|
252
423
|
| Feedback | `feedback()`, `takeFeedback()` |
|
|
424
|
+
| Telemetry | `telemetry()`, `takeTelemetry()` |
|
|
425
|
+
| Formant envelopes | `formantEnvelope(maxFrequencyHz, input, output)` |
|
|
253
426
|
| Finite tail | `beginDrain()`, `drain(left, right)`, `drainMono(audio)` |
|
|
254
427
|
| Configured tail budget | `tailFrames()` |
|
|
255
428
|
| Reset or release memory | `reset()`, `free()` |
|
|
256
429
|
|
|
430
|
+
### Audio buffers
|
|
431
|
+
|
|
257
432
|
All audio buffers are `Int16Array` buffers, and they must fit in the prepared
|
|
258
|
-
frame limit.
|
|
259
|
-
|
|
260
|
-
|
|
261
|
-
|
|
262
|
-
|
|
433
|
+
frame limit. These rules apply:
|
|
434
|
+
|
|
435
|
+
- Stereo output arrays must not overlap.
|
|
436
|
+
- The input can overlap an output, because the binding copies the input
|
|
437
|
+
first.
|
|
438
|
+
- Buffers must stay attached during each call.
|
|
439
|
+
|
|
440
|
+
Web Audio uses Float32 samples, which you must convert to and from 16-bit
|
|
441
|
+
PCM. This package does not install an AudioWorklet or an integration with an
|
|
442
|
+
audio device.
|
|
443
|
+
|
|
444
|
+
### Configuration objects
|
|
263
445
|
|
|
264
446
|
Configuration objects contain all required fields. They use camelCase names
|
|
265
447
|
and show enums as string literals. To change the configuration, do these steps:
|
|
@@ -272,18 +454,35 @@ If a required field is absent or a value is not valid, the call throws a
|
|
|
272
454
|
JavaScript `Error`. The optional `resynthesis.harmony.voices` field defaults
|
|
273
455
|
to 4 if it is absent. The conversion ignores extra object properties. The
|
|
274
456
|
generated declarations contain the parameter documentation from the Rust
|
|
275
|
-
types.
|
|
276
|
-
|
|
457
|
+
types.
|
|
458
|
+
|
|
459
|
+
`resynthesis.randomSeed` and the `processedFrames` field of feedback and
|
|
460
|
+
telemetry use `bigint`. Frame offsets, block lengths and latency use
|
|
277
461
|
`number`.
|
|
278
462
|
|
|
463
|
+
### Events
|
|
464
|
+
|
|
465
|
+
The optional event argument is a `TimedEvent[]`. Controls use a `type`
|
|
466
|
+
discriminant and a `value` payload. The reset control has no payload. For
|
|
467
|
+
example, this event changes the final gain before the first frame:
|
|
468
|
+
|
|
469
|
+
```ts
|
|
470
|
+
{ offset: 0, event: { type: "outputGain", value: 0.5 } }
|
|
471
|
+
```
|
|
472
|
+
|
|
473
|
+
A harmony Note On event has this form:
|
|
474
|
+
|
|
475
|
+
```ts
|
|
476
|
+
{ offset: 0, event: { type: "midi", value: { type: "noteOn", channel: 0, note: 60, velocity: 100 } } }
|
|
477
|
+
```
|
|
478
|
+
|
|
479
|
+
Use `correctionMidi` for the separate correction keyboard. The native rules
|
|
480
|
+
for event order and for the validation of the whole block also apply to these
|
|
481
|
+
bindings.
|
|
482
|
+
|
|
483
|
+
### Feedback, telemetry and formant envelopes
|
|
484
|
+
|
|
279
485
|
`telemetry()` and `takeTelemetry()` return a `Telemetry` object.
|
|
280
|
-
`formantEnvelope(maxFrequencyHz, input, output)` fills two `Float32Array`
|
|
281
|
-
buffers of equal length and returns `false` if no envelope is available.
|
|
282
|
-
In that case, it fills both buffers with zeros. If the lengths are not equal
|
|
283
|
-
or the frequency is not valid, it throws and does not change the buffers. The
|
|
284
|
-
frequency must be finite, more than zero and not more than half of the sample
|
|
285
|
-
rate. The class keeps a buffer for the points, so only the first call and a
|
|
286
|
-
call with more points than before reserve memory.
|
|
287
486
|
|
|
288
487
|
In feedback, `targetMidiNote` is `undefined` in these cases, which are also
|
|
289
488
|
the cases where `voiced` is `false`:
|
|
@@ -293,16 +492,15 @@ the cases where `voiced` is `false`:
|
|
|
293
492
|
- The estimated frequency is outside the inclusive reporting bounds
|
|
294
493
|
`resynthesis.pitch.minFrequencyHz` and `resynthesis.pitch.maxFrequencyHz`.
|
|
295
494
|
|
|
296
|
-
|
|
297
|
-
|
|
298
|
-
|
|
299
|
-
the
|
|
495
|
+
`formantEnvelope(maxFrequencyHz, input, output)` fills two `Float32Array`
|
|
496
|
+
buffers of equal length and returns `false` if no envelope is available.
|
|
497
|
+
In that case, it fills both buffers with zeros. If the lengths are not equal
|
|
498
|
+
or the frequency is not valid, it throws and does not change the buffers. The
|
|
499
|
+
frequency must be finite, more than zero and not more than half of the sample
|
|
500
|
+
rate. The class keeps a buffer for the points, so only the first call and a
|
|
501
|
+
call with more points than before reserve memory.
|
|
300
502
|
|
|
301
|
-
|
|
302
|
-
`{ offset: 0, event: { type: "midi", value: { type: "noteOn", channel: 0, note: 60, velocity: 100 } } }`.
|
|
303
|
-
Use `correctionMidi` for the separate correction keyboard. The native rules
|
|
304
|
-
for event order and for the validation of the whole block also apply to these
|
|
305
|
-
bindings.
|
|
503
|
+
### Memory allocation
|
|
306
504
|
|
|
307
505
|
The constructor allocates reusable PCM buffers. A successful call that
|
|
308
506
|
processes audio without events does not allocate new Rust PCM storage.
|
|
@@ -320,6 +518,56 @@ use, and it does not guarantee hard real-time behavior for allocation or
|
|
|
320
518
|
scheduling. Drain completion and reset keep the same state as the equivalent
|
|
321
519
|
Rust functions.
|
|
322
520
|
|
|
521
|
+
## Build from source
|
|
522
|
+
|
|
523
|
+
### Rust crate
|
|
524
|
+
|
|
525
|
+
To build the crate and check the format and the lints, run these commands:
|
|
526
|
+
|
|
527
|
+
```sh
|
|
528
|
+
cargo build --locked --release
|
|
529
|
+
cargo fmt --check
|
|
530
|
+
cargo clippy --locked --lib -- -D warnings
|
|
531
|
+
```
|
|
532
|
+
|
|
533
|
+
Native Rust builds use Cargo and do not need Node.js or npm.
|
|
534
|
+
|
|
535
|
+
### npm package
|
|
536
|
+
|
|
537
|
+
Install Node.js and `wasm-pack`. Then run these commands to add the
|
|
538
|
+
`wasm32-unknown-unknown` Rust target, install the development dependencies
|
|
539
|
+
and make the archive:
|
|
540
|
+
|
|
541
|
+
```sh
|
|
542
|
+
rustup target add wasm32-unknown-unknown
|
|
543
|
+
npm ci
|
|
544
|
+
npm pack
|
|
545
|
+
```
|
|
546
|
+
|
|
547
|
+
`npm pack` builds the Wasm bindings and the TypeScript entry point. Then it
|
|
548
|
+
creates `tunelet-<version>.tgz`. To use this archive in a browser project or
|
|
549
|
+
a Node.js project, install it with this command:
|
|
550
|
+
|
|
551
|
+
```sh
|
|
552
|
+
npm install /path/to/tunelet-<version>.tgz
|
|
553
|
+
```
|
|
554
|
+
|
|
555
|
+
To build without an archive, run `npm run build`. The build writes one
|
|
556
|
+
`web`-target binding into `pkg/` and the shared entry into `dist/`. Git
|
|
557
|
+
ignores both directories. The archive contains only these items:
|
|
558
|
+
|
|
559
|
+
- The runtime entry.
|
|
560
|
+
- The declarations.
|
|
561
|
+
- The bindings.
|
|
562
|
+
- The binary.
|
|
563
|
+
- The package metadata.
|
|
564
|
+
- The README.
|
|
565
|
+
|
|
566
|
+
The `build:wasm` script builds the crate in release mode with LTO and one
|
|
567
|
+
codegen unit. Then `wasm-pack` runs Binaryen `wasm-opt` on
|
|
568
|
+
`pkg/tunelet_bg.wasm`. The TypeScript and Node.js type declarations are only
|
|
569
|
+
development dependencies.
|
|
570
|
+
|
|
323
571
|
## License
|
|
324
572
|
|
|
325
573
|
tunelet is available under the MIT License. Refer to [LICENSE](LICENSE).
|