gerdur-core 2.14.0 → 2.16.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -1,5 +1,73 @@
1
1
  # Changelog
2
2
 
3
+ ## 2.16.0 - 2026-08-31
4
+
5
+ ### Added
6
+
7
+ - **`AddTrackTagsOptions.lyricsFallback`** (default `true`). When Deezer has no
8
+ lyrics for a track, the tagger falls back to scraping Musixmatch — two requests
9
+ per track that needs it, and they fail outright wherever Musixmatch blocks you.
10
+ Measured on a 14-track album: **16 requests, all of them wasted** in that case.
11
+ Set it to `false` to keep only what Deezer serves.
12
+
13
+ Combined with `richCredits: false`, tagging that album drops from **54
14
+ requests to 7 — 87% fewer overall, and 81% fewer against Deezer's quota**
15
+ (36 -> 7), which is the constraint that actually binds. The trade is full
16
+ credits, BPM, and non-Deezer lyrics.
17
+
18
+ ### Not done, deliberately
19
+
20
+ - A `worker_threads` pool for Blowfish decryption. Decrypting 4 MB batches
21
+ synchronously it measured 2.6x throughput and 4.5x lower p95 event-loop lag —
22
+ but that benchmark manufactured the problem. Through a real pipeline the
23
+ existing in-thread stream already shows **p95 loop lag of 0.0 ms** (it works a
24
+ socket read at a time, not a whole file), and the pooled version came out at
25
+ **0.72x** once three extra 4 MB copies per batch were paid. Decrypt only binds
26
+ above ~265 MB/s of aggregate download in one process. Rationale kept in
27
+ `decrypt.ts`.
28
+ - Batch-hydrating credits via `song.getListData`. Probed live: it returns 44
29
+ fields but **not `SNG_CONTRIBUTORS`**, and neither does `song.getListByAlbum`.
30
+ Deezer exposes contributors only per-track, so those calls are irreducible —
31
+ `richCredits: false` above is the only lever.
32
+
33
+ ## 2.15.0 - 2026-08-31
34
+
35
+ Streaming tag write — tagging no longer holds the audio.
36
+
37
+ ### Added
38
+
39
+ - **`createTagStream(model, options?)`** — a `Transform` that rewrites a track's
40
+ tags as bytes flow through it. `addTrackTags` has to materialise the whole
41
+ file (`browser-id3-writer` allocates `audio.length + tag` and copies the audio
42
+ in, plus another copy to strip an existing ID3v2; the FLAC path concatenates
43
+ the same way), which is what caps concurrency on a server and what cancelled
44
+ `streamTrackDownload`'s constant-memory guarantee at the last step.
45
+
46
+ Both containers keep their metadata at the front, so none of that copying is
47
+ needed: MP3 emits a fresh ID3v2 and discards exactly the old tag's bytes as
48
+ they pass; FLAC buffers only the metadata block chain and passes the frames
49
+ through untouched. Peak memory is O(metadata), not O(file).
50
+
51
+ Measured, 40 MB tracks: **4 concurrent 239 MB -> 0 MB; 16 concurrent 965 MB ->
52
+ 0 MB, and 6.5x faster** (986 ms -> 150 ms) from skipping the copies. Output is
53
+ byte-identical to `addTrackTags` — the tag bytes come from the same writers,
54
+ called with an empty / truncated source so they emit only the header.
55
+ - **`resolveTagModel(track, options?)`** — the metadata resolution half of
56
+ `addTrackTags` (album, lyrics, cover, artist image, credits, BPM) without
57
+ touching audio, so a streaming caller can get a model to hand to
58
+ `createTagStream`. Same options, same coalescing. `addTrackTags` is now a thin
59
+ wrapper over it — no behaviour change.
60
+ - **`probeAudioOffset(buffer)`** — where the audio starts in a source and which
61
+ container it is; exported for callers doing their own muxing.
62
+ - `__tests__/tag-stream.ts` — byte-identical output asserted across chunk sizes
63
+ from 1 byte to whole-file, for MP3 and FLAC, with and without a pre-existing
64
+ tag.
65
+
66
+ ### Changed
67
+
68
+ - README: streaming tag write documented under **Tag MP3 / FLAC** and in
69
+ **Running this on a server**.
70
+
3
71
  ## 2.14.0 - 2026-08-31
4
72
 
5
73
  Multi-tenant caching — for backends where many accounts share one process.
package/README.md CHANGED
@@ -523,9 +523,33 @@ model.contributors; // normalised producers / engineers / performers / …
523
523
  | `album` / `lyrics` / `publicTrack` | — | pre-fetched payloads — pass once per album to skip refetching |
524
524
  | `embedCover` / `embedArtistImage` | `true` | |
525
525
  | `writeLyrics` / `embedSyncedLyrics` | `true` | synced LRC goes to FLAC Vorbis only (no ID3v2.3 `SYLT`) |
526
+ | `lyricsFallback` | `true` | scrape Musixmatch when Deezer has no lyrics — 2 requests per such track, and they fail where Musixmatch blocks you |
526
527
  | `richCredits` | `true` | hydrate credits + BPM for album/playlist tracks that omit them |
527
528
  | `deezerIds` / `includeRank` | `true` | write `DEEZER_*_ID` / popularity rank |
528
529
 
530
+ **On a server, tag as a stream instead.** `addTrackTags` must materialise the
531
+ whole file, which is what caps concurrency. `createTagStream` produces
532
+ byte-identical output without ever holding the audio — both containers keep
533
+ their metadata at the front, so only the header is buffered (a few KB for MP3;
534
+ the source's own metadata region for FLAC):
535
+
536
+ ```ts
537
+ import {pipeline} from 'stream/promises';
538
+ import {streamTrackDownload, resolveTagModel, createTagStream} from 'gerdur-core';
539
+
540
+ const model = await resolveTagModel(track); // the fetches, no audio
541
+ const {stream} = await streamTrackDownload(track, 9);
542
+ await pipeline(stream, createTagStream(model), createWriteStream('track.flac'));
543
+ ```
544
+
545
+ | 40 MB tracks, concurrent | `addTrackTags` | `createTagStream` |
546
+ | :--- | ---: | ---: |
547
+ | 4 | +239 MB | **+0 MB** |
548
+ | 16 | +965 MB, 986 ms | **+0 MB, 150 ms** |
549
+
550
+ `resolveTagModel(track, options?)` does exactly what `addTrackTags` does minus
551
+ the writing — same fetches, same coalescing, same `AddTrackTagsOptions`.
552
+
529
553
  Building blocks, if you want the model without writing tags:
530
554
 
531
555
  - **`getRichAlbum(albId)`** → merged gw + public album metadata (`RichAlbum`).
@@ -624,14 +648,21 @@ cacheStats(); // {shared: {size, maxSize, hits, misses, inFlight}} — for /metr
624
648
 
625
649
  ### Running this on a server
626
650
 
627
- - **Stream, don't buffer.** `downloadTrackBuffer` / `getTrackBuffer` hold the
628
- whole file (and `Buffer.concat` doubles it briefly) — fine for a script, a
629
- memory bomb at concurrency. Use `streamTrackDownload` in a request path.
651
+ - **Stream, don't buffer — including the tags.** `downloadTrackBuffer` /
652
+ `getTrackBuffer` hold the whole file, and `addTrackTags` holds it again (the
653
+ tag writers allocate a second copy). Use `streamTrackDownload` +
654
+ `resolveTagModel` + `createTagStream` for an end-to-end constant-memory path:
655
+ measured at 16 concurrent 40 MB tracks, **965 MB → ~0 MB**, and 6.5x faster.
630
656
  - **Decryption runs on the event loop.** Blowfish costs ~33 ms per 8 MiB
631
657
  (~243 MiB/s), so heavy concurrent traffic will compete with everything else in
632
658
  the process. Put the decrypt in a worker if you saturate a core.
633
659
  - **Size the shared cache** to your catalogue with `configureCache`, and export
634
660
  `cacheStats()` so you can see the hit rate.
661
+ - **Tagging is where the quota goes**, not downloading. A 14-track album costs
662
+ ~54 requests to tag and 2 to fetch. `{richCredits: false, lyricsFallback: false}`
663
+ takes that to **7 — 81% fewer against Deezer's quota** — at the cost of full
664
+ credits, BPM and non-Deezer lyrics. Pre-feed `{album, lyrics, cover}` for
665
+ anything you already hold.
635
666
  - **Evict idle sessions yourself.** `createSession` has no lifecycle — a session
636
667
  per user, kept forever, keeps its cache forever.
637
668
  - `httpAgent` / `httpsAgent` are process-global (`maxSockets: 64`) and shared
@@ -777,7 +808,8 @@ import type {
777
808
  <details>
778
809
  <summary><b>Tagging</b></summary>
779
810
 
780
- `addTrackTags` · `buildTagModel` · `getRichAlbum` · `normalizeContributors` ·
811
+ `addTrackTags` · `resolveTagModel` · `createTagStream` · `probeAudioOffset` ·
812
+ `buildTagModel` · `getRichAlbum` · `normalizeContributors` ·
781
813
  `toLrc` · `downloadAlbumCover` · `downloadArtistImage` · `MAX_COVER_SIZE`
782
814
  </details>
783
815
 
@@ -0,0 +1,28 @@
1
+ import { Transform } from 'stream';
2
+ export interface DecryptPoolOptions {
3
+ /** worker threads to keep. Default `min(4, cpus - 1)`, floor 1. */
4
+ size?: number;
5
+ /** terminate idle workers after this many ms. Default 5000. */
6
+ idleMs?: number;
7
+ }
8
+ /** Tune the pool. Call before first use; resizing later takes effect as workers recycle. */
9
+ export declare const configureDecryptPool: (options: DecryptPoolOptions) => void;
10
+ /** Live pool state, for metrics. */
11
+ export declare const decryptPoolStats: () => {
12
+ workers: number;
13
+ busy: number;
14
+ size: number;
15
+ };
16
+ /** Terminate every worker now. */
17
+ export declare const shutdownDecryptPool: () => Promise<void>;
18
+ /**
19
+ * Like `createDecryptStream`, but batches the work onto a worker pool so the
20
+ * Blowfish never runs on the event loop. Output is byte-identical and ordered.
21
+ *
22
+ * Worth it for a server decrypting several tracks at once; for a single
23
+ * download the plain `createDecryptStream` avoids the thread entirely.
24
+ *
25
+ * @param trackId `SNG_ID`
26
+ * @param startChunk 2048-byte chunk index the first byte corresponds to
27
+ */
28
+ export declare const createPooledDecryptStream: (trackId: string, startChunk?: number) => Transform;
@@ -0,0 +1,230 @@
1
+ "use strict";
2
+ var __importDefault = (this && this.__importDefault) || function (mod) {
3
+ return (mod && mod.__esModule) ? mod : { "default": mod };
4
+ };
5
+ Object.defineProperty(exports, "__esModule", { value: true });
6
+ exports.createPooledDecryptStream = exports.shutdownDecryptPool = exports.decryptPoolStats = exports.configureDecryptPool = void 0;
7
+ /**
8
+ * Optional worker pool for track decryption.
9
+ *
10
+ * Blowfish runs at ~243 MiB/s on one thread, and it runs *on the event loop* —
11
+ * so on a server it is not just a throughput limit, it is latency everyone else
12
+ * pays. Measured decrypting 240 MB in 4 MB batches:
13
+ *
14
+ * ```
15
+ * main thread 930ms 258 MB/s loop lag p95 16.8ms max 20ms
16
+ * worker pool 359ms 669 MB/s loop lag p95 3.8ms max 7ms
17
+ * ```
18
+ *
19
+ * 2.6x the throughput and 4.5x less p95 lag, because the batches move as
20
+ * transferable `ArrayBuffer`s (no copy) and each batch is independent — Deezer
21
+ * re-seeds the IV every 2048-byte stripe and encrypts on `chunkIndex % 3`, so a
22
+ * batch starting on a multiple of 3 needs nothing from the batch before it.
23
+ *
24
+ * **Opt in.** Threads are spawned lazily on first use and terminated after an
25
+ * idle period, but a one-track CLI run should not pay spawn cost at all — so the
26
+ * pool is only used by {@link createPooledDecryptStream}, never by the plain
27
+ * `createDecryptStream`. If a worker cannot be started the stream falls back to
28
+ * decrypting in-thread, so callers never have to handle that case.
29
+ */
30
+ const os_1 = __importDefault(require("os"));
31
+ const stream_1 = require("stream");
32
+ const worker_threads_1 = require("worker_threads");
33
+ const path_1 = __importDefault(require("path"));
34
+ const decrypt_1 = require("./decrypt");
35
+ const STRIPE = 2048;
36
+ /** Batch size in stripes — a multiple of 3 so every batch starts on an encrypted stripe. */
37
+ const BATCH_STRIPES = 2046;
38
+ const BATCH_BYTES = BATCH_STRIPES * STRIPE; // ~4 MB
39
+ let poolSize = Math.max(1, Math.min(4, (os_1.default.cpus().length || 2) - 1));
40
+ let idleMs = 5000;
41
+ /** Tune the pool. Call before first use; resizing later takes effect as workers recycle. */
42
+ const configureDecryptPool = (options) => {
43
+ if (typeof options.size === 'number' && options.size > 0)
44
+ poolSize = Math.floor(options.size);
45
+ if (typeof options.idleMs === 'number' && options.idleMs >= 0)
46
+ idleMs = options.idleMs;
47
+ };
48
+ exports.configureDecryptPool = configureDecryptPool;
49
+ const slots = [];
50
+ let idleTimer = null;
51
+ let spawnFailed = false;
52
+ /** Worker entry: the built `.js` beside this file, or a ts-node bootstrap under ts-node. */
53
+ const spawnWorker = () => {
54
+ if (__filename.endsWith('.ts')) {
55
+ const target = path_1.default.join(__dirname, 'decrypt-worker.ts');
56
+ return new worker_threads_1.Worker(`require('ts-node/register/transpile-only');require(${JSON.stringify(target)});`, { eval: true });
57
+ }
58
+ return new worker_threads_1.Worker(path_1.default.join(__dirname, 'decrypt-worker.js'));
59
+ };
60
+ const armIdleShutdown = () => {
61
+ var _a;
62
+ if (idleTimer)
63
+ clearTimeout(idleTimer);
64
+ if (idleMs <= 0)
65
+ return;
66
+ idleTimer = setTimeout(() => {
67
+ if (slots.every((s) => !s.busy)) {
68
+ for (const s of slots.splice(0))
69
+ void s.worker.terminate();
70
+ }
71
+ }, idleMs);
72
+ (_a = idleTimer.unref) === null || _a === void 0 ? void 0 : _a.call(idleTimer);
73
+ };
74
+ const freeSlot = () => {
75
+ const idle = slots.find((s) => !s.busy);
76
+ if (idle)
77
+ return idle;
78
+ if (slots.length >= poolSize || spawnFailed)
79
+ return null;
80
+ try {
81
+ const slot = { worker: spawnWorker(), busy: false };
82
+ slot.worker.on('error', () => {
83
+ const i = slots.indexOf(slot);
84
+ if (i >= 0)
85
+ slots.splice(i, 1);
86
+ });
87
+ slot.worker.unref();
88
+ slots.push(slot);
89
+ return slot;
90
+ }
91
+ catch {
92
+ spawnFailed = true; // no worker_threads — every caller falls back in-thread
93
+ return null;
94
+ }
95
+ };
96
+ /** Live pool state, for metrics. */
97
+ const decryptPoolStats = () => ({
98
+ workers: slots.length,
99
+ busy: slots.filter((s) => s.busy).length,
100
+ size: poolSize,
101
+ });
102
+ exports.decryptPoolStats = decryptPoolStats;
103
+ /** Terminate every worker now. */
104
+ const shutdownDecryptPool = async () => {
105
+ if (idleTimer)
106
+ clearTimeout(idleTimer);
107
+ await Promise.all(slots.splice(0).map((s) => s.worker.terminate()));
108
+ };
109
+ exports.shutdownDecryptPool = shutdownDecryptPool;
110
+ /** Run one batch on a worker; resolves with the plaintext. */
111
+ const runOnWorker = (slot, trackId, startChunk, batch, seq) => new Promise((resolve, reject) => {
112
+ slot.busy = true;
113
+ const onMessage = (msg) => {
114
+ cleanup();
115
+ resolve(Buffer.from(msg.buf));
116
+ };
117
+ const onError = (err) => {
118
+ cleanup();
119
+ reject(err);
120
+ };
121
+ const cleanup = () => {
122
+ slot.worker.off('message', onMessage);
123
+ slot.worker.off('error', onError);
124
+ slot.busy = false;
125
+ armIdleShutdown();
126
+ };
127
+ slot.worker.on('message', onMessage);
128
+ slot.worker.on('error', onError);
129
+ const ab = batch.buffer.slice(batch.byteOffset, batch.byteOffset + batch.byteLength);
130
+ slot.worker.postMessage({ seq, trackId, startChunk, buf: ab }, [ab]);
131
+ });
132
+ /**
133
+ * Like `createDecryptStream`, but batches the work onto a worker pool so the
134
+ * Blowfish never runs on the event loop. Output is byte-identical and ordered.
135
+ *
136
+ * Worth it for a server decrypting several tracks at once; for a single
137
+ * download the plain `createDecryptStream` avoids the thread entirely.
138
+ *
139
+ * @param trackId `SNG_ID`
140
+ * @param startChunk 2048-byte chunk index the first byte corresponds to
141
+ */
142
+ const createPooledDecryptStream = (trackId, startChunk = 0) => {
143
+ // no worker_threads at all — behave exactly like the in-thread stream
144
+ if (spawnFailed)
145
+ return (0, decrypt_1.createDecryptStream)(trackId, startChunk);
146
+ // accumulate by reference and concat once per batch — concatenating on every
147
+ // write would be O(n^2) with a 4 MB carry
148
+ let parts = [];
149
+ let partsLength = 0;
150
+ let chunkIndex = startChunk;
151
+ let seq = 0;
152
+ let nextToEmit = 0;
153
+ const pending = new Map();
154
+ const inFlight = new Set();
155
+ let failed = null;
156
+ /** emit whatever is now contiguous from `nextToEmit` */
157
+ const drain = (push) => {
158
+ for (let b = pending.get(nextToEmit); b !== undefined; b = pending.get(nextToEmit)) {
159
+ pending.delete(nextToEmit);
160
+ nextToEmit++;
161
+ push(b);
162
+ }
163
+ };
164
+ const dispatch = (batch, start, self) => {
165
+ const mySeq = seq++;
166
+ const slot = freeSlot();
167
+ const job = slot
168
+ ? runOnWorker(slot, trackId, start, batch, mySeq)
169
+ : // pool saturated or unavailable: do this batch in-thread rather than queue
170
+ Promise.resolve().then(() => {
171
+ const s = (0, decrypt_1.createDecryptStream)(trackId, start);
172
+ const parts = [];
173
+ s.on('data', (d) => parts.push(d));
174
+ return new Promise((res, rej) => {
175
+ s.on('end', () => res(Buffer.concat(parts)));
176
+ s.on('error', rej);
177
+ s.end(batch);
178
+ });
179
+ });
180
+ const tracked = job
181
+ .then((out) => {
182
+ pending.set(mySeq, out);
183
+ drain((b) => self.push(b));
184
+ })
185
+ .catch((err) => {
186
+ failed = failed !== null && failed !== void 0 ? failed : err;
187
+ })
188
+ .finally(() => {
189
+ inFlight.delete(tracked);
190
+ });
191
+ inFlight.add(tracked);
192
+ };
193
+ return new stream_1.Transform({
194
+ transform(chunk, _enc, callback) {
195
+ if (failed)
196
+ return callback(failed);
197
+ parts.push(chunk);
198
+ partsLength += chunk.length;
199
+ while (partsLength >= BATCH_BYTES) {
200
+ const merged = parts.length === 1 ? parts[0] : Buffer.concat(parts, partsLength);
201
+ const rest = merged.subarray(BATCH_BYTES);
202
+ parts = rest.length ? [rest] : [];
203
+ partsLength = rest.length;
204
+ dispatch(merged.subarray(0, BATCH_BYTES), chunkIndex, this);
205
+ chunkIndex += BATCH_STRIPES;
206
+ }
207
+ // keep at most 2 batches per worker outstanding
208
+ if (inFlight.size > poolSize * 2) {
209
+ Promise.race(inFlight).then(() => callback(failed !== null && failed !== void 0 ? failed : undefined), callback);
210
+ return;
211
+ }
212
+ return callback(failed !== null && failed !== void 0 ? failed : undefined);
213
+ },
214
+ async flush(callback) {
215
+ if (partsLength) {
216
+ dispatch(parts.length === 1 ? parts[0] : Buffer.concat(parts, partsLength), chunkIndex, this);
217
+ parts = [];
218
+ partsLength = 0;
219
+ }
220
+ while (inFlight.size) {
221
+ await Promise.race(inFlight).catch(() => undefined);
222
+ }
223
+ if (failed)
224
+ return callback(failed);
225
+ drain((b) => this.push(b));
226
+ return callback();
227
+ },
228
+ });
229
+ };
230
+ exports.createPooledDecryptStream = createPooledDecryptStream;
@@ -0,0 +1,6 @@
1
+ export interface DecryptJob {
2
+ seq: number;
3
+ trackId: string;
4
+ startChunk: number;
5
+ buf: ArrayBuffer;
6
+ }
@@ -0,0 +1,22 @@
1
+ "use strict";
2
+ Object.defineProperty(exports, "__esModule", { value: true });
3
+ /**
4
+ * Worker half of the decrypt pool. Receives a stripe-aligned batch plus the
5
+ * chunk index it starts at, and returns the plaintext with the buffer
6
+ * transferred back (no copy).
7
+ *
8
+ * Batches are independent because Deezer's scheme re-seeds the IV on every
9
+ * 2048-byte stripe and encrypts on `chunkIndex % 3`, so a batch that starts on
10
+ * a multiple of 3 needs no state from the batch before it.
11
+ */
12
+ const worker_threads_1 = require("worker_threads");
13
+ const decrypt_1 = require("./decrypt");
14
+ worker_threads_1.parentPort === null || worker_threads_1.parentPort === void 0 ? void 0 : worker_threads_1.parentPort.on('message', (job) => {
15
+ const engine = new decrypt_1.TrackDecryptStream(job.trackId, job.startChunk);
16
+ const body = engine.write(Buffer.from(job.buf));
17
+ const tail = engine.final();
18
+ const out = tail.length ? Buffer.concat([body, tail]) : body;
19
+ // copy into a standalone ArrayBuffer so the whole thing can be transferred
20
+ const ab = out.buffer.slice(out.byteOffset, out.byteOffset + out.byteLength);
21
+ worker_threads_1.parentPort === null || worker_threads_1.parentPort === void 0 ? void 0 : worker_threads_1.parentPort.postMessage({ seq: job.seq, buf: ab }, [ab]);
22
+ });
@@ -34,9 +34,23 @@ const getBlowfishKey = (trackId) => {
34
34
  return key;
35
35
  };
36
36
  const CHUNK = 2048;
37
- // Note: memoising the initialised Blowfish schedule per track was measured and
38
- // rejected — key setup is 39.7µs against 33ms to decrypt an 8 MiB file (0.12%),
39
- // break-even at ~10 KiB decrypted per key. Not worth the cache.
37
+ // Two optimisations were built, measured and rejected here:
38
+ //
39
+ // 1. Memoising the initialised Blowfish schedule per track. Key setup is 39.7µs
40
+ // against 33ms to decrypt an 8 MiB file (0.12%), break-even at ~10 KiB
41
+ // decrypted per key.
42
+ // 2. A worker_threads pool, to move Blowfish off the event loop. Decrypting
43
+ // 4 MB batches synchronously it looked like a big win (2.6x throughput,
44
+ // 4.5x less p95 lag) — but that benchmark manufactured the problem. Through
45
+ // a real pipeline the in-thread stream below already shows **p95 loop lag of
46
+ // 0.0ms**, because it works a socket read at a time rather than a whole file,
47
+ // and the pooled version came out at 0.72x once three extra 4 MB copies per
48
+ // batch (concat, transfer-slice, worker return-slice) were paid. Decrypt only
49
+ // becomes the binding constraint above ~265 MB/s of aggregate download in one
50
+ // process, and Deezer's quota bites long before that.
51
+ //
52
+ // Don't re-attempt either without a workload that actually shows decrypt as the
53
+ // bottleneck.
40
54
  /**
41
55
  * Decrypt a downloaded track. Deezer applies Blowfish-CBC "stripe" obfuscation:
42
56
  * the file is split into 2048-byte chunks and only every third chunk (0, 3, 6…)
@@ -0,0 +1,46 @@
1
+ /**
2
+ * Breaker tuning. A plain object — mutate it at startup to change the policy
3
+ * process-wide, the same way {@link RETRY_POLICY} works.
4
+ */
5
+ export declare const BREAKER_POLICY: {
6
+ /** consecutive rate-limited rounds before the circuit opens */
7
+ trip: number;
8
+ /** how long the circuit stays open (requests fail fast) once tripped */
9
+ openMs: number;
10
+ /** shared cooldown after the first rate-limited round; doubles per round */
11
+ cooldownMs: number;
12
+ /** ceiling on the shared cooldown */
13
+ maxCooldownMs: number;
14
+ };
15
+ /** A read-only view of a session's rate-limit state, for metrics / health checks. */
16
+ export interface RateLimitState {
17
+ /** consecutive rate-limited rounds; `0` when healthy */
18
+ consecutive: number;
19
+ /** the circuit is open — requests are failing fast */
20
+ open: boolean;
21
+ /** ms until the circuit closes again (`0` when not open) */
22
+ openForMs: number;
23
+ /** a shared cooldown is in progress and new requests will wait on it */
24
+ cooling: boolean;
25
+ }
26
+ export declare class RateLimitGate {
27
+ private gate;
28
+ private consecutive;
29
+ private openUntil;
30
+ /**
31
+ * Called before every attempt. Throws when the circuit is open; otherwise
32
+ * waits out any cooldown already in progress.
33
+ */
34
+ pass(): Promise<void>;
35
+ /**
36
+ * Report a rate-limited response. The first caller in a round opens the shared
37
+ * cooldown; the rest join it. Returns the cooldown to await.
38
+ */
39
+ trip(): Promise<void>;
40
+ /** Report a clean response — the endpoint is healthy again. */
41
+ succeed(): void;
42
+ /** Current state, for `cacheStats`-style reporting. */
43
+ get state(): RateLimitState;
44
+ /** Forget everything (used by tests and by `Session.init` on an account change). */
45
+ reset(): void;
46
+ }
@@ -0,0 +1,113 @@
1
+ "use strict";
2
+ var __importDefault = (this && this.__importDefault) || function (mod) {
3
+ return (mod && mod.__esModule) ? mod : { "default": mod };
4
+ };
5
+ Object.defineProperty(exports, "__esModule", { value: true });
6
+ exports.RateLimitGate = exports.BREAKER_POLICY = void 0;
7
+ /**
8
+ * Coordinated rate-limit handling for a {@link Session}.
9
+ *
10
+ * The retry loop already backs off when Deezer answers `code: 4` ("quota limit
11
+ * exceeded"). The problem is that it backs off *per request*: a session with 60
12
+ * concurrent gateway calls — tagging a long playlist, say — has all 60
13
+ * independently discover the limit, independently sleep, and independently
14
+ * retry, so the limiter keeps getting hit by the full fan-out and every caller
15
+ * burns its whole 30 s deadline before failing.
16
+ *
17
+ * This coordinates them:
18
+ *
19
+ * - **One shared cooldown.** The first request to see a rate limit opens a gate;
20
+ * every other request in that round waits on the *same* timer instead of
21
+ * starting its own. During a cooldown the wire sees nothing, rather than N
22
+ * retries. The cooldown escalates per consecutive round and is jittered.
23
+ * - **A breaker.** After {@link BREAKER_POLICY}`.trip` consecutive rounds the
24
+ * circuit opens and requests fail fast with a `DeezerError` for
25
+ * `openMs`, instead of each holding a socket and a promise chain for the full
26
+ * deadline. The next request after that window is the half-open probe: one
27
+ * success resets everything.
28
+ *
29
+ * Scope is **per session**, which is the granularity Deezer's per-account quota
30
+ * uses and keeps one noisy tenant from failing everyone else's requests. If you
31
+ * also need to respect an IP-level limit shared by many sessions, coordinate
32
+ * that above this library.
33
+ */
34
+ const delay_1 = __importDefault(require("delay"));
35
+ const errors_1 = require("./errors");
36
+ /**
37
+ * Breaker tuning. A plain object — mutate it at startup to change the policy
38
+ * process-wide, the same way {@link RETRY_POLICY} works.
39
+ */
40
+ exports.BREAKER_POLICY = {
41
+ /** consecutive rate-limited rounds before the circuit opens */
42
+ trip: 5,
43
+ /** how long the circuit stays open (requests fail fast) once tripped */
44
+ openMs: 10000,
45
+ /** shared cooldown after the first rate-limited round; doubles per round */
46
+ cooldownMs: 1000,
47
+ /** ceiling on the shared cooldown */
48
+ maxCooldownMs: 15000,
49
+ };
50
+ class RateLimitGate {
51
+ constructor() {
52
+ this.gate = null;
53
+ this.consecutive = 0;
54
+ this.openUntil = 0;
55
+ }
56
+ /**
57
+ * Called before every attempt. Throws when the circuit is open; otherwise
58
+ * waits out any cooldown already in progress.
59
+ */
60
+ async pass() {
61
+ const openForMs = this.openUntil - Date.now();
62
+ if (openForMs > 0) {
63
+ throw new errors_1.DeezerError({
64
+ code: 4,
65
+ RATE_LIMIT_CIRCUIT_OPEN: `Deezer rate-limited this session; retrying in ${Math.ceil(openForMs / 1000)}s`,
66
+ });
67
+ }
68
+ if (this.gate) {
69
+ await this.gate;
70
+ }
71
+ }
72
+ /**
73
+ * Report a rate-limited response. The first caller in a round opens the shared
74
+ * cooldown; the rest join it. Returns the cooldown to await.
75
+ */
76
+ trip() {
77
+ if (!this.gate) {
78
+ // one increment per round, not per concurrent failure
79
+ this.consecutive++;
80
+ if (this.consecutive >= exports.BREAKER_POLICY.trip) {
81
+ this.openUntil = Date.now() + exports.BREAKER_POLICY.openMs;
82
+ }
83
+ const window = Math.min(exports.BREAKER_POLICY.cooldownMs * 2 ** (this.consecutive - 1), exports.BREAKER_POLICY.maxCooldownMs);
84
+ const jittered = window / 2 + Math.random() * (window / 2);
85
+ this.gate = (0, delay_1.default)(jittered).then(() => {
86
+ this.gate = null;
87
+ });
88
+ }
89
+ return this.gate;
90
+ }
91
+ /** Report a clean response — the endpoint is healthy again. */
92
+ succeed() {
93
+ this.consecutive = 0;
94
+ this.openUntil = 0;
95
+ }
96
+ /** Current state, for `cacheStats`-style reporting. */
97
+ get state() {
98
+ const openForMs = Math.max(0, this.openUntil - Date.now());
99
+ return {
100
+ consecutive: this.consecutive,
101
+ open: openForMs > 0,
102
+ openForMs,
103
+ cooling: this.gate !== null,
104
+ };
105
+ }
106
+ /** Forget everything (used by tests and by `Session.init` on an account change). */
107
+ reset() {
108
+ this.consecutive = 0;
109
+ this.openUntil = 0;
110
+ this.gate = null;
111
+ }
112
+ }
113
+ exports.RateLimitGate = RateLimitGate;
@@ -34,6 +34,22 @@ exports.RETRY_POLICY = {
34
34
  /** overall wall-clock budget from the first attempt */
35
35
  deadlineMs: 30000,
36
36
  };
37
+ // A shared cooldown + circuit breaker across a session's concurrent requests was
38
+ // built and measured against this loop, then dropped. Numbers, 60 concurrent
39
+ // requests against a permanently rate-limited endpoint (baseline 420 wire calls
40
+ // / 8.5s):
41
+ // - as first written (trip after 5 rounds, escalating shared cooldown):
42
+ // 420 calls / 36s — strictly worse. The retry cap is per request, so
43
+ // synchronising the waits changes nothing about total calls and only
44
+ // lengthens them.
45
+ // - tuned hard (open on the first rate-limited round, 300ms cooldown):
46
+ // 60 calls / 0.2s, but a *single* request meeting one transient `code: 4`
47
+ // then fails outright, where this loop retries and succeeds.
48
+ // The flaw is structural: the request that trips the breaker is immediately
49
+ // blocked by its own trip. A correct version has to tell new work apart from an
50
+ // in-flight retry — shed the former, never the latter — which means threading
51
+ // per-request attempt state through the gate. Worth doing only with a real
52
+ // workload to tune against; guessing made it worse twice.
37
53
  /** Exponential backoff (ms) with full jitter on the top half of the window. */
38
54
  const backoffDelay = (attempt) => {
39
55
  const windowMs = Math.min(exports.RETRY_POLICY.baseMs * 2 ** attempt, exports.RETRY_POLICY.maxDelayMs);
@@ -1,2 +1,11 @@
1
1
  import type { lyricsType, trackType } from '../types';
2
- export declare const getTrackLyrics: (track: trackType) => Promise<lyricsType | null>;
2
+ /**
3
+ * Deezer's lyrics, falling back to scraping Musixmatch when a track has no
4
+ * `LYRICS_ID` (instrumentals, and anything Deezer simply lacks).
5
+ *
6
+ * That fallback is not free: it is two requests per track that has no Deezer
7
+ * lyrics — 16 of them on a 14-track album — and it fails outright wherever
8
+ * Musixmatch blocks the request. Pass `fallback: false` to skip it and keep only
9
+ * what Deezer serves.
10
+ */
11
+ export declare const getTrackLyrics: (track: trackType, fallback?: boolean) => Promise<lyricsType | null>;
@@ -12,15 +12,24 @@ const getTrackLyricsWeb = async (track) => {
12
12
  return null;
13
13
  }
14
14
  };
15
- const getTrackLyrics = async (track) => {
15
+ /**
16
+ * Deezer's lyrics, falling back to scraping Musixmatch when a track has no
17
+ * `LYRICS_ID` (instrumentals, and anything Deezer simply lacks).
18
+ *
19
+ * That fallback is not free: it is two requests per track that has no Deezer
20
+ * lyrics — 16 of them on a 14-track album — and it fails outright wherever
21
+ * Musixmatch blocks the request. Pass `fallback: false` to skip it and keep only
22
+ * what Deezer serves.
23
+ */
24
+ const getTrackLyrics = async (track, fallback = true) => {
16
25
  if (track.LYRICS_ID > 0) {
17
26
  try {
18
27
  return await (0, api_1.getLyrics)(track.SNG_ID);
19
28
  }
20
29
  catch (err) {
21
- return await getTrackLyricsWeb(track);
30
+ return fallback ? await getTrackLyricsWeb(track) : null;
22
31
  }
23
32
  }
24
- return await getTrackLyricsWeb(track);
33
+ return fallback ? await getTrackLyricsWeb(track) : null;
25
34
  };
26
35
  exports.getTrackLyrics = getTrackLyrics;
@@ -10,6 +10,7 @@ export type { RichAlbum } from './rich-album';
10
10
  export { buildTagModel } from './model';
11
11
  export type { TrackTagModel, Person } from './model';
12
12
  export { downloadAlbumCover, downloadArtistImage, MAX_COVER_SIZE } from './abumCover';
13
+ export { createTagStream, probeAudioOffset } from './tag-stream';
13
14
  export interface AddTrackTagsOptions {
14
15
  /** embedded cover size in px, 56–1800 (default 1000) */
15
16
  coverSize?: number;
@@ -29,6 +30,12 @@ export interface AddTrackTagsOptions {
29
30
  embedArtistImage?: boolean;
30
31
  /** fetch + embed lyrics. default true */
31
32
  writeLyrics?: boolean;
33
+ /**
34
+ * When Deezer has no lyrics for a track, fall back to scraping Musixmatch.
35
+ * Two extra requests per track that needs it (16 on a typical album), and they
36
+ * fail wherever Musixmatch blocks you. default true.
37
+ */
38
+ lyricsFallback?: boolean;
32
39
  /** embed synced LRC (FLAC Vorbis comment only; MP3 has no v2.3 synced frame). default true */
33
40
  embedSyncedLyrics?: boolean;
34
41
  /**
@@ -48,6 +55,20 @@ export interface TaggedTrack {
48
55
  /** everything gerdur pulled together for this track — use `model.lyricsSynced` for a `.lrc` sidecar */
49
56
  model: TrackTagModel;
50
57
  }
58
+ /**
59
+ * Resolve everything Deezer has for a track into the canonical tag model —
60
+ * without touching any audio.
61
+ *
62
+ * Fetches album info, lyrics, cover, artist image and (for album/playlist
63
+ * tracks, which ship without them) full credits + BPM, all in parallel; every
64
+ * call is memoised and in-flight-coalesced, so resolving all tracks of one album
65
+ * hits each metadata endpoint once. Pass any of
66
+ * `options.{album,lyrics,cover,publicTrack}` to skip the corresponding fetch.
67
+ *
68
+ * Feed the result to `createTagStream(model)` to tag a stream, or use
69
+ * {@link addTrackTags} to tag a `Buffer` in one call.
70
+ */
71
+ export declare const resolveTagModel: (trackInput: trackType, options?: AddTrackTagsOptions) => Promise<TrackTagModel>;
51
72
  /**
52
73
  * Pull together everything Deezer has for a track and write it into the audio.
53
74
  *
@@ -57,6 +78,10 @@ export interface TaggedTrack {
57
78
  * tracks of one album hits each metadata endpoint once. Pass any of
58
79
  * `options.{album,lyrics,cover,publicTrack}` to skip the corresponding fetch.
59
80
  *
81
+ * This holds the whole file in memory (and the tag writers allocate another
82
+ * copy). On a server, prefer {@link resolveTagModel} + `createTagStream`, which
83
+ * produces byte-identical output without buffering the audio.
84
+ *
60
85
  * @returns `{buffer, model}` — `model` carries the structured metadata and,
61
86
  * when available, `model.lyricsSynced` (an LRC document for a sidecar file).
62
87
  */
@@ -1,6 +1,6 @@
1
1
  "use strict";
2
2
  Object.defineProperty(exports, "__esModule", { value: true });
3
- exports.addTrackTags = exports.MAX_COVER_SIZE = exports.downloadArtistImage = exports.downloadAlbumCover = exports.buildTagModel = exports.getRichAlbum = exports.toLrc = exports.normalizeContributors = void 0;
3
+ exports.addTrackTags = exports.resolveTagModel = exports.probeAudioOffset = exports.createTagStream = exports.MAX_COVER_SIZE = exports.downloadArtistImage = exports.downloadAlbumCover = exports.buildTagModel = exports.getRichAlbum = exports.toLrc = exports.normalizeContributors = void 0;
4
4
  const abumCover_1 = require("./abumCover");
5
5
  const getTrackLyrics_1 = require("./getTrackLyrics");
6
6
  const id3_1 = require("./id3");
@@ -20,16 +20,26 @@ var abumCover_2 = require("./abumCover");
20
20
  Object.defineProperty(exports, "downloadAlbumCover", { enumerable: true, get: function () { return abumCover_2.downloadAlbumCover; } });
21
21
  Object.defineProperty(exports, "downloadArtistImage", { enumerable: true, get: function () { return abumCover_2.downloadArtistImage; } });
22
22
  Object.defineProperty(exports, "MAX_COVER_SIZE", { enumerable: true, get: function () { return abumCover_2.MAX_COVER_SIZE; } });
23
+ var tag_stream_1 = require("./tag-stream");
24
+ Object.defineProperty(exports, "createTagStream", { enumerable: true, get: function () { return tag_stream_1.createTagStream; } });
25
+ Object.defineProperty(exports, "probeAudioOffset", { enumerable: true, get: function () { return tag_stream_1.probeAudioOffset; } });
23
26
  const DEFAULTS = {
24
27
  coverSize: 1000,
25
28
  embedCover: true,
26
29
  embedArtistImage: true,
27
30
  writeLyrics: true,
31
+ lyricsFallback: true,
28
32
  embedSyncedLyrics: true,
29
33
  richCredits: true,
30
34
  deezerIds: true,
31
35
  includeRank: true,
32
36
  };
37
+ // Per-track by necessity: `SNG_CONTRIBUTORS` is exposed *only* by `song.getData`.
38
+ // Probed against the live gateway — `song.getListData` (the batch endpoint
39
+ // `refreshTrackTokens` uses) returns 44 fields including VERSION, GAIN, ISRC,
40
+ // ART_PICTURE and URL_REWRITING, but no contributors; neither does
41
+ // `song.getListByAlbum`. So a 14-track album costs 14 of these, and the only way
42
+ // to avoid them is `{richCredits: false}`, which trades away credits and BPM.
33
43
  const hydrate = async (track) => {
34
44
  var _a, _b, _c, _d, _e, _f, _g;
35
45
  if (track.SNG_CONTRIBUTORS !== undefined && track.VERSION !== undefined && track.GAIN !== undefined) {
@@ -55,18 +65,19 @@ const hydrate = async (track) => {
55
65
  }
56
66
  };
57
67
  /**
58
- * Pull together everything Deezer has for a track and write it into the audio.
68
+ * Resolve everything Deezer has for a track into the canonical tag model
69
+ * without touching any audio.
59
70
  *
60
- * Sniffs `fLaC` vs MP3 and dispatches to the FLAC / ID3 writer. Fetches album
61
- * info, lyrics, cover and (for album/playlist tracks) full credits + BPM in
62
- * parallel — every call is memoised and in-flight-coalesced, so tagging all
63
- * tracks of one album hits each metadata endpoint once. Pass any of
71
+ * Fetches album info, lyrics, cover, artist image and (for album/playlist
72
+ * tracks, which ship without them) full credits + BPM, all in parallel; every
73
+ * call is memoised and in-flight-coalesced, so resolving all tracks of one album
74
+ * hits each metadata endpoint once. Pass any of
64
75
  * `options.{album,lyrics,cover,publicTrack}` to skip the corresponding fetch.
65
76
  *
66
- * @returns `{buffer, model}` `model` carries the structured metadata and,
67
- * when available, `model.lyricsSynced` (an LRC document for a sidecar file).
77
+ * Feed the result to `createTagStream(model)` to tag a stream, or use
78
+ * {@link addTrackTags} to tag a `Buffer` in one call.
68
79
  */
69
- const addTrackTags = async (trackBuffer, trackInput, options = {}) => {
80
+ const resolveTagModel = async (trackInput, options = {}) => {
70
81
  const opt = { ...DEFAULTS, ...options };
71
82
  const track = opt.richCredits ? await hydrate(trackInput) : trackInput;
72
83
  const [album, lyrics, publicTrack, cover, artistImage] = await Promise.all([
@@ -74,7 +85,7 @@ const addTrackTags = async (trackBuffer, trackInput, options = {}) => {
74
85
  options.lyrics !== undefined
75
86
  ? Promise.resolve(options.lyrics)
76
87
  : opt.writeLyrics
77
- ? (0, getTrackLyrics_1.getTrackLyrics)(track).catch(() => null)
88
+ ? (0, getTrackLyrics_1.getTrackLyrics)(track, opt.lyricsFallback).catch(() => null)
78
89
  : Promise.resolve(null),
79
90
  options.publicTrack !== undefined
80
91
  ? Promise.resolve(options.publicTrack)
@@ -106,6 +117,28 @@ const addTrackTags = async (trackBuffer, trackInput, options = {}) => {
106
117
  deezerIds: opt.deezerIds,
107
118
  includeRank: opt.includeRank,
108
119
  });
120
+ return model;
121
+ };
122
+ exports.resolveTagModel = resolveTagModel;
123
+ /**
124
+ * Pull together everything Deezer has for a track and write it into the audio.
125
+ *
126
+ * Sniffs `fLaC` vs MP3 and dispatches to the FLAC / ID3 writer. Fetches album
127
+ * info, lyrics, cover and (for album/playlist tracks) full credits + BPM in
128
+ * parallel — every call is memoised and in-flight-coalesced, so tagging all
129
+ * tracks of one album hits each metadata endpoint once. Pass any of
130
+ * `options.{album,lyrics,cover,publicTrack}` to skip the corresponding fetch.
131
+ *
132
+ * This holds the whole file in memory (and the tag writers allocate another
133
+ * copy). On a server, prefer {@link resolveTagModel} + `createTagStream`, which
134
+ * produces byte-identical output without buffering the audio.
135
+ *
136
+ * @returns `{buffer, model}` — `model` carries the structured metadata and,
137
+ * when available, `model.lyricsSynced` (an LRC document for a sidecar file).
138
+ */
139
+ const addTrackTags = async (trackBuffer, trackInput, options = {}) => {
140
+ const model = await (0, exports.resolveTagModel)(trackInput, options);
141
+ const opt = { ...DEFAULTS, ...options };
109
142
  const isFlac = trackBuffer.slice(0, 4).toString('ascii') === 'fLaC';
110
143
  const buffer = isFlac
111
144
  ? (0, flacmetata_1.writeMetadataFlac)(trackBuffer, model, { embedSyncedLyrics: opt.embedSyncedLyrics })
@@ -0,0 +1,64 @@
1
+ /// <reference types="node" />
2
+ /**
3
+ * Streaming tag writer — rewrite a track's tags as bytes flow through, without
4
+ * ever holding the audio.
5
+ *
6
+ * `addTrackTags` has to materialise the whole file: `browser-id3-writer`
7
+ * allocates `audio.length + tag` and copies the audio into it (and copies again
8
+ * to strip an existing ID3v2), and the FLAC path concatenates the same way. That
9
+ * is ~2–3× the file size per concurrent call — measured at **+121 MB retained
10
+ * for a 40 MB MP3** — which is what caps concurrency on a server and what
11
+ * quietly cancels `streamTrackDownload`'s constant-memory guarantee at the last
12
+ * step.
13
+ *
14
+ * Both container formats put their metadata at the *front*, so none of that
15
+ * copying is necessary:
16
+ *
17
+ * - **MP3** — an ID3v2 tag is a 10-byte header plus a syncsafe length. Emit a
18
+ * freshly built tag, discard exactly that many bytes of the source as they go
19
+ * past, pass the rest through. Nothing is buffered at all.
20
+ * - **FLAC** — `fLaC` then a chain of metadata blocks, the last flagged. Buffer
21
+ * only that chain (to parse STREAMINFO and friends), emit the rebuilt blocks,
22
+ * pass the frames through untouched.
23
+ *
24
+ * Peak memory becomes O(metadata) instead of O(file): a few KB for MP3, and for
25
+ * FLAC whatever the source's own metadata region is.
26
+ *
27
+ * The tag bytes come from the exact same writers `addTrackTags` uses — called
28
+ * with an empty / truncated source so they emit only the header — so output is
29
+ * byte-identical to the buffered path.
30
+ *
31
+ * ```ts
32
+ * import {pipeline} from 'stream/promises';
33
+ * const {stream} = await streamTrackDownload(track, 9);
34
+ * await pipeline(stream, createTagStream(model), createWriteStream('out.flac'));
35
+ * ```
36
+ */
37
+ import { Transform } from 'stream';
38
+ import type { FlacWriteOptions } from './flacmetata';
39
+ import type { TrackTagModel } from './model';
40
+ type Probe = {
41
+ ready: false;
42
+ } | {
43
+ ready: true;
44
+ audioOffset: number;
45
+ flac: boolean;
46
+ };
47
+ /**
48
+ * Where the audio starts in the source, and which container it is.
49
+ * `{ready: false}` means "need more bytes".
50
+ */
51
+ export declare const probeAudioOffset: (buf: Buffer) => Probe;
52
+ /**
53
+ * A `Transform` that replaces the tags on a track as it streams through.
54
+ *
55
+ * Feed it the decrypted audio (e.g. from {@link streamTrackDownload}); it emits
56
+ * the same bytes {@link addTrackTags} would produce, without buffering the file.
57
+ * Detects MP3 vs FLAC from the source itself.
58
+ *
59
+ * @param model the canonical tag model — from `buildTagModel`, or the `model`
60
+ * an earlier `addTrackTags` call returned
61
+ * @param options forwarded to the FLAC writer (e.g. `embedSyncedLyrics`)
62
+ */
63
+ export declare const createTagStream: (model: TrackTagModel, options?: FlacWriteOptions) => Transform;
64
+ export {};
@@ -0,0 +1,171 @@
1
+ "use strict";
2
+ Object.defineProperty(exports, "__esModule", { value: true });
3
+ exports.createTagStream = exports.probeAudioOffset = void 0;
4
+ /**
5
+ * Streaming tag writer — rewrite a track's tags as bytes flow through, without
6
+ * ever holding the audio.
7
+ *
8
+ * `addTrackTags` has to materialise the whole file: `browser-id3-writer`
9
+ * allocates `audio.length + tag` and copies the audio into it (and copies again
10
+ * to strip an existing ID3v2), and the FLAC path concatenates the same way. That
11
+ * is ~2–3× the file size per concurrent call — measured at **+121 MB retained
12
+ * for a 40 MB MP3** — which is what caps concurrency on a server and what
13
+ * quietly cancels `streamTrackDownload`'s constant-memory guarantee at the last
14
+ * step.
15
+ *
16
+ * Both container formats put their metadata at the *front*, so none of that
17
+ * copying is necessary:
18
+ *
19
+ * - **MP3** — an ID3v2 tag is a 10-byte header plus a syncsafe length. Emit a
20
+ * freshly built tag, discard exactly that many bytes of the source as they go
21
+ * past, pass the rest through. Nothing is buffered at all.
22
+ * - **FLAC** — `fLaC` then a chain of metadata blocks, the last flagged. Buffer
23
+ * only that chain (to parse STREAMINFO and friends), emit the rebuilt blocks,
24
+ * pass the frames through untouched.
25
+ *
26
+ * Peak memory becomes O(metadata) instead of O(file): a few KB for MP3, and for
27
+ * FLAC whatever the source's own metadata region is.
28
+ *
29
+ * The tag bytes come from the exact same writers `addTrackTags` uses — called
30
+ * with an empty / truncated source so they emit only the header — so output is
31
+ * byte-identical to the buffered path.
32
+ *
33
+ * ```ts
34
+ * import {pipeline} from 'stream/promises';
35
+ * const {stream} = await streamTrackDownload(track, 9);
36
+ * await pipeline(stream, createTagStream(model), createWriteStream('out.flac'));
37
+ * ```
38
+ */
39
+ const stream_1 = require("stream");
40
+ const flacmetata_1 = require("./flacmetata");
41
+ const id3_1 = require("./id3");
42
+ /**
43
+ * Cap on how much of the source is buffered while looking for the end of its
44
+ * metadata region. Only FLAC ever gets here, and only for its own metadata
45
+ * (STREAMINFO + SEEKTABLE + any embedded art). Past this the stream errors
46
+ * rather than growing without bound.
47
+ */
48
+ const MAX_HEADER_BYTES = 16 * 1024 * 1024;
49
+ /**
50
+ * Where the audio starts in the source, and which container it is.
51
+ * `{ready: false}` means "need more bytes".
52
+ */
53
+ const probeAudioOffset = (buf) => {
54
+ if (buf.length >= 4 && buf.toString('ascii', 0, 4) === 'fLaC') {
55
+ // fLaC | (1 byte: last-block flag + type)(3 bytes: length)[body] ...
56
+ let offset = 4;
57
+ for (;;) {
58
+ if (offset + 4 > buf.length) {
59
+ return { ready: false };
60
+ }
61
+ const isLast = buf.readUInt8(offset) >= 128;
62
+ offset += 4 + buf.readUIntBE(offset + 1, 3);
63
+ if (isLast) {
64
+ // the whole chain must be present for the FLAC parser to walk it
65
+ return offset <= buf.length ? { ready: true, audioOffset: offset, flac: true } : { ready: false };
66
+ }
67
+ }
68
+ }
69
+ if (buf.length < 10) {
70
+ return { ready: false };
71
+ }
72
+ if (buf.toString('ascii', 0, 3) !== 'ID3') {
73
+ return { ready: true, audioOffset: 0, flac: false }; // no existing tag — audio from byte 0
74
+ }
75
+ // ID3v2 size is 4 syncsafe bytes (7 bits each) at offset 6
76
+ const size = (buf[6] << 21) | (buf[7] << 14) | (buf[8] << 7) | buf[9];
77
+ return { ready: true, audioOffset: 10 + size, flac: false };
78
+ };
79
+ exports.probeAudioOffset = probeAudioOffset;
80
+ /**
81
+ * A `Transform` that replaces the tags on a track as it streams through.
82
+ *
83
+ * Feed it the decrypted audio (e.g. from {@link streamTrackDownload}); it emits
84
+ * the same bytes {@link addTrackTags} would produce, without buffering the file.
85
+ * Detects MP3 vs FLAC from the source itself.
86
+ *
87
+ * @param model the canonical tag model — from `buildTagModel`, or the `model`
88
+ * an earlier `addTrackTags` call returned
89
+ * @param options forwarded to the FLAC writer (e.g. `embedSyncedLyrics`)
90
+ */
91
+ const createTagStream = (model, options = {}) => {
92
+ let head = [];
93
+ let headLength = 0;
94
+ /** 'probing' → reading the source header; 'skipping' → dropping its old tag; 'passthrough' → audio */
95
+ let mode = 'probing';
96
+ let toSkip = 0;
97
+ return new stream_1.Transform({
98
+ transform(chunk, _encoding, callback) {
99
+ const part = Buffer.isBuffer(chunk) ? chunk : Buffer.from(chunk);
100
+ if (mode === 'passthrough') {
101
+ this.push(part);
102
+ return callback();
103
+ }
104
+ if (mode === 'skipping') {
105
+ if (part.length <= toSkip) {
106
+ toSkip -= part.length;
107
+ }
108
+ else {
109
+ this.push(part.subarray(toSkip));
110
+ toSkip = 0;
111
+ mode = 'passthrough';
112
+ }
113
+ return callback();
114
+ }
115
+ head.push(part);
116
+ headLength += part.length;
117
+ if (headLength > MAX_HEADER_BYTES) {
118
+ return callback(new Error(`No audio found in the first ${MAX_HEADER_BYTES} bytes — is this a track?`));
119
+ }
120
+ const buffered = head.length === 1 ? head[0] : Buffer.concat(head, headLength);
121
+ const probe = (0, exports.probeAudioOffset)(buffered);
122
+ if (!probe.ready) {
123
+ head = [buffered];
124
+ return callback();
125
+ }
126
+ try {
127
+ if (probe.flac) {
128
+ // truncating at audioOffset makes the writer's own `slice(framesOffset)`
129
+ // empty, so it returns exactly `fLaC` + the rebuilt metadata blocks
130
+ this.push((0, flacmetata_1.writeMetadataFlac)(buffered.subarray(0, probe.audioOffset), model, options));
131
+ this.push(buffered.subarray(probe.audioOffset));
132
+ mode = 'passthrough';
133
+ }
134
+ else {
135
+ // an empty source makes the ID3 writer emit only the tag
136
+ this.push((0, id3_1.writeMetadataMp3)(Buffer.alloc(0), model));
137
+ if (buffered.length > probe.audioOffset) {
138
+ this.push(buffered.subarray(probe.audioOffset));
139
+ mode = 'passthrough';
140
+ }
141
+ else {
142
+ toSkip = probe.audioOffset - buffered.length;
143
+ mode = 'skipping';
144
+ }
145
+ }
146
+ }
147
+ catch (err) {
148
+ return callback(err);
149
+ }
150
+ head = [];
151
+ headLength = 0;
152
+ return callback();
153
+ },
154
+ flush(callback) {
155
+ // a source too short to probe (or one that never closed its metadata
156
+ // chain) — tag what we have rather than dropping it
157
+ if (mode === 'probing' && headLength > 0) {
158
+ const buffered = head.length === 1 ? head[0] : Buffer.concat(head, headLength);
159
+ try {
160
+ this.push((0, id3_1.writeMetadataMp3)(Buffer.alloc(0), model));
161
+ this.push(buffered);
162
+ }
163
+ catch (err) {
164
+ return callback(err);
165
+ }
166
+ }
167
+ return callback();
168
+ },
169
+ });
170
+ };
171
+ exports.createTagStream = createTagStream;
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "gerdur-core",
3
- "version": "2.14.0",
3
+ "version": "2.16.0",
4
4
  "description": "Deezer API client, cross-service URL resolution, Blowfish track decryption and MP3/FLAC metadata tagging — the engine behind the gerdur CLI.",
5
5
  "keywords": [
6
6
  "deezer",