gerdur-core 2.15.0 → 2.17.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -1,5 +1,61 @@
1
1
  # Changelog
2
2
 
3
+ ## 2.17.0 - 2026-08-31
4
+
5
+ The account's own library, over the authenticated gateway.
6
+
7
+ ### Added
8
+
9
+ - **`src/api/library.ts`** — the logged-in account's *private* library. Distinct
10
+ from the public-REST surface in `api/user.ts`, which needs a public profile and
11
+ only shows what that profile exposes. Found by probing the gateway: seven
12
+ methods answer here that the public API cannot reach.
13
+ - `getMyPlaylists(userId?, nb?, start?)` — **including private playlists**
14
+ - `getMyFavoriteTracks` / `getMyFavoriteAlbums` / `getMyFavoriteArtists`
15
+ - `getMyFavoritePlaylists` / `getMyFavoriteRadios` / `getMyFavoriteShows`
16
+ - `getMyFavoriteTrackIds()` — every loved track id in one small request, for
17
+ diffing a local library against the account
18
+ - `userId` defaults to the logged-in account. Responses are gateway shapes, so
19
+ tracks arrive with a `TRACK_TOKEN` and are directly downloadable; being
20
+ account-scoped they stay in the per-session cache and are never shared.
21
+ - **`getTrackMix(sngId, nb?, start?)`** (`song.getSearchTrackMix`) — a "more like
22
+ this" mix seeded from one track. Unlike the public radio endpoints it returns
23
+ full gateway tracks **with tokens already attached**: verified that
24
+ `resolveDownloadUrls` resolves them 3/3 with no `getTrackInfo` round trip.
25
+
26
+ Verified against a live account: `getMyPlaylists` 1/1, `getMyFavoriteArtists`
27
+ 7/7 with real names, `getTrackMix` 5/5 all carrying tokens.
28
+
29
+ ## 2.16.0 - 2026-08-31
30
+
31
+ ### Added
32
+
33
+ - **`AddTrackTagsOptions.lyricsFallback`** (default `true`). When Deezer has no
34
+ lyrics for a track, the tagger falls back to scraping Musixmatch — two requests
35
+ per track that needs it, and they fail outright wherever Musixmatch blocks you.
36
+ Measured on a 14-track album: **16 requests, all of them wasted** in that case.
37
+ Set it to `false` to keep only what Deezer serves.
38
+
39
+ Combined with `richCredits: false`, tagging that album drops from **54
40
+ requests to 7 — 87% fewer overall, and 81% fewer against Deezer's quota**
41
+ (36 -> 7), which is the constraint that actually binds. The trade is full
42
+ credits, BPM, and non-Deezer lyrics.
43
+
44
+ ### Not done, deliberately
45
+
46
+ - A `worker_threads` pool for Blowfish decryption. Decrypting 4 MB batches
47
+ synchronously it measured 2.6x throughput and 4.5x lower p95 event-loop lag —
48
+ but that benchmark manufactured the problem. Through a real pipeline the
49
+ existing in-thread stream already shows **p95 loop lag of 0.0 ms** (it works a
50
+ socket read at a time, not a whole file), and the pooled version came out at
51
+ **0.72x** once three extra 4 MB copies per batch were paid. Decrypt only binds
52
+ above ~265 MB/s of aggregate download in one process. Rationale kept in
53
+ `decrypt.ts`.
54
+ - Batch-hydrating credits via `song.getListData`. Probed live: it returns 44
55
+ fields but **not `SNG_CONTRIBUTORS`**, and neither does `song.getListByAlbum`.
56
+ Deezer exposes contributors only per-track, so those calls are irreducible —
57
+ `richCredits: false` above is the only lever.
58
+
3
59
  ## 2.15.0 - 2026-08-31
4
60
 
5
61
  Streaming tag write — tagging no longer holds the audio.
package/README.md CHANGED
@@ -46,6 +46,7 @@ the CLI and the file-writing layer on top.
46
46
  - [Search](#search)
47
47
  - [Browse and discover](#browse-and-discover)
48
48
  - [Flow, radios and a user's library](#flow-radios-and-a-users-library)
49
+ - [Your own library](#your-own-library)
49
50
  - [Podcasts](#podcasts)
50
51
  - [Preview clips](#preview-clips)
51
52
  - [Resolve a download URL](#resolve-a-download-url)
@@ -381,6 +382,33 @@ const {data: loved} = await getUserFavoriteTracks(me.USER_ID);
381
382
  const {data: eighties} = await getRadioTracks(38305); // "The '80s"
382
383
  ```
383
384
 
385
+ ### Your own library
386
+
387
+ `api/user.ts` above reads a **public** profile. These read what the *account*
388
+ can see — private playlists included — over the authenticated gateway, and
389
+ return gateway shapes, so tracks come with a `TRACK_TOKEN` and are immediately
390
+ downloadable.
391
+
392
+ | Function | Returns |
393
+ | :--- | :--- |
394
+ | `getMyPlaylists(userId?, nb?, start?)` | the account's own playlists, **including private ones** |
395
+ | `getMyFavoriteTracks(userId?, nb?, start?)` | loved tracks, as downloadable gw tracks |
396
+ | `getMyFavoriteTrackIds()` | every loved track id in one request — for diffing a local library |
397
+ | `getMyFavoriteAlbums` / `getMyFavoriteArtists` | favourited albums / artists |
398
+ | `getMyFavoritePlaylists` / `getMyFavoriteRadios` / `getMyFavoriteShows` | followed playlists / radios / shows |
399
+ | `getTrackMix(sngId, nb?, start?)` | a "more like this" mix seeded from a track — **tokens already attached** |
400
+
401
+ ```ts
402
+ import {getMyPlaylists, getTrackMix, resolveDownloadUrls} from 'gerdur-core';
403
+
404
+ const {data: playlists} = await getMyPlaylists(); // yours, private included
405
+ const {data: mix} = await getTrackMix('3135556', 20); // 20 tracks like this one
406
+ const urls = await resolveDownloadUrls(mix, [9, 3, 1]); // straight to download — no per-track lookup
407
+ ```
408
+
409
+ `userId` defaults to the logged-in account. These are account-scoped, so they
410
+ never enter the shared cross-session cache.
411
+
384
412
  ### Podcasts
385
413
 
386
414
  ```ts
@@ -523,6 +551,7 @@ model.contributors; // normalised producers / engineers / performers / …
523
551
  | `album` / `lyrics` / `publicTrack` | — | pre-fetched payloads — pass once per album to skip refetching |
524
552
  | `embedCover` / `embedArtistImage` | `true` | |
525
553
  | `writeLyrics` / `embedSyncedLyrics` | `true` | synced LRC goes to FLAC Vorbis only (no ID3v2.3 `SYLT`) |
554
+ | `lyricsFallback` | `true` | scrape Musixmatch when Deezer has no lyrics — 2 requests per such track, and they fail where Musixmatch blocks you |
526
555
  | `richCredits` | `true` | hydrate credits + BPM for album/playlist tracks that omit them |
527
556
  | `deezerIds` / `includeRank` | `true` | write `DEEZER_*_ID` / popularity rank |
528
557
 
@@ -657,6 +686,11 @@ cacheStats(); // {shared: {size, maxSize, hits, misses, inFlight}} — for /metr
657
686
  the process. Put the decrypt in a worker if you saturate a core.
658
687
  - **Size the shared cache** to your catalogue with `configureCache`, and export
659
688
  `cacheStats()` so you can see the hit rate.
689
+ - **Tagging is where the quota goes**, not downloading. A 14-track album costs
690
+ ~54 requests to tag and 2 to fetch. `{richCredits: false, lyricsFallback: false}`
691
+ takes that to **7 — 81% fewer against Deezer's quota** — at the cost of full
692
+ credits, BPM and non-Deezer lyrics. Pre-feed `{album, lyrics, cover}` for
693
+ anything you already hold.
660
694
  - **Evict idle sessions yourself.** `createSession` has no lifecycle — a session
661
695
  per user, kept forever, keeps its cache forever.
662
696
  - `httpAgent` / `httpsAgent` are process-global (`maxSockets: 64`) and shared
@@ -781,7 +815,10 @@ import type {
781
815
 
782
816
  `getUserFlow` · `getUserFavoriteTracks` · `getUserFavoriteAlbums` ·
783
817
  `getUserFavoriteArtists` · `getUserPlaylists` · `getUserRadios` ·
784
- `getUserChartTracks` · `getRadios` · `getRadioTracks` · `getRadioGenres`
818
+ `getUserChartTracks` · `getRadios` · `getRadioTracks` · `getRadioGenres` ·
819
+ `getMyPlaylists` · `getMyFavoriteTracks` · `getMyFavoriteTrackIds` ·
820
+ `getMyFavoriteAlbums` · `getMyFavoriteArtists` · `getMyFavoritePlaylists` ·
821
+ `getMyFavoriteRadios` · `getMyFavoriteShows` · `getTrackMix`
785
822
  </details>
786
823
 
787
824
  <details>
@@ -5,3 +5,4 @@ export * from './browse';
5
5
  export * from './preview';
6
6
  export * from './user';
7
7
  export * from './podcast';
8
+ export * from './library';
package/dist/api/index.js CHANGED
@@ -21,3 +21,4 @@ __exportStar(require("./browse"), exports);
21
21
  __exportStar(require("./preview"), exports);
22
22
  __exportStar(require("./user"), exports);
23
23
  __exportStar(require("./podcast"), exports);
24
+ __exportStar(require("./library"), exports);
@@ -0,0 +1,72 @@
1
+ import type { playlistInfoMinimal, trackType } from '../types';
2
+ /** Envelope the gateway wraps these listings in. */
3
+ export interface GwList<T> {
4
+ data: T[];
5
+ count: number;
6
+ total: number;
7
+ filtered_count?: number;
8
+ checksum?: string;
9
+ }
10
+ /** A favourited artist, as the library returns it. */
11
+ export interface FavoriteArtist {
12
+ ART_ID: string;
13
+ ART_NAME: string;
14
+ ART_PICTURE: string;
15
+ /** when it was favourited — `YYYY-MM-DD HH:MM:SS` */
16
+ DATE_ADD?: string;
17
+ NB_ALBUM?: number;
18
+ NB_FAN?: number;
19
+ ARTIST_IS_DUMMY?: boolean;
20
+ __TYPE__?: string;
21
+ }
22
+ /** A favourited album, as the library returns it. */
23
+ export interface FavoriteAlbum {
24
+ ALB_ID: string;
25
+ ALB_TITLE: string;
26
+ ALB_PICTURE: string;
27
+ ART_ID?: string;
28
+ ART_NAME?: string;
29
+ DATE_ADD?: string;
30
+ NUMBER_TRACK?: number;
31
+ PHYSICAL_RELEASE_DATE?: string;
32
+ __TYPE__?: string;
33
+ }
34
+ /**
35
+ * The account's own playlists (`playlist.getList`) — **including private ones**,
36
+ * which the public `/user/{id}/playlists` endpoint will not show you.
37
+ *
38
+ * @param userId defaults to the logged-in account
39
+ */
40
+ export declare const getMyPlaylists: (userId?: string | number, nb?: number, start?: number) => Promise<GwList<playlistInfoMinimal>>;
41
+ /**
42
+ * Loved tracks (`song.getFavorites`). Gateway track objects, so each carries a
43
+ * `TRACK_TOKEN` and can go straight to `resolveDownloadUrls`.
44
+ */
45
+ export declare const getMyFavoriteTracks: (userId?: string | number, nb?: number, start?: number) => Promise<GwList<trackType>>;
46
+ /**
47
+ * Just the ids of every loved track (`song.getFavoriteIds`) — one small request
48
+ * for the whole set, for diffing a local library against the account.
49
+ */
50
+ export declare const getMyFavoriteTrackIds: () => Promise<GwList<{
51
+ SNG_ID: string;
52
+ }>>;
53
+ /** Favourited albums (`album.getFavorites`). */
54
+ export declare const getMyFavoriteAlbums: (userId?: string | number, nb?: number, start?: number) => Promise<GwList<FavoriteAlbum>>;
55
+ /** Favourited artists (`artist.getFavorites`). */
56
+ export declare const getMyFavoriteArtists: (userId?: string | number, nb?: number, start?: number) => Promise<GwList<FavoriteArtist>>;
57
+ /** Playlists the account follows (`playlist.getFavorites`). */
58
+ export declare const getMyFavoritePlaylists: (userId?: string | number, nb?: number, start?: number) => Promise<GwList<playlistInfoMinimal>>;
59
+ /** Favourited radios (`radio.getFavorites`). */
60
+ export declare const getMyFavoriteRadios: (userId?: string | number, nb?: number, start?: number) => Promise<GwList<Record<string, any>>>;
61
+ /** Favourited podcast shows (`show.getFavorites`). */
62
+ export declare const getMyFavoriteShows: (userId?: string | number, nb?: number, start?: number) => Promise<GwList<Record<string, any>>>;
63
+ /**
64
+ * A "more like this" mix seeded from one track (`song.getSearchTrackMix`).
65
+ *
66
+ * Returns full gateway tracks **with `TRACK_TOKEN`s**, so unlike the public
67
+ * radio endpoints the results are immediately downloadable — no `getTrackInfo`
68
+ * round trip per hit.
69
+ *
70
+ * @param sngId the seed track's `SNG_ID`
71
+ */
72
+ export declare const getTrackMix: (sngId: string, nb?: number, start?: number) => Promise<GwList<trackType>>;
@@ -0,0 +1,65 @@
1
+ "use strict";
2
+ Object.defineProperty(exports, "__esModule", { value: true });
3
+ exports.getTrackMix = exports.getMyFavoriteShows = exports.getMyFavoriteRadios = exports.getMyFavoritePlaylists = exports.getMyFavoriteArtists = exports.getMyFavoriteAlbums = exports.getMyFavoriteTrackIds = exports.getMyFavoriteTracks = exports.getMyPlaylists = void 0;
4
+ /**
5
+ * The **logged-in account's own library**, over the authenticated gateway.
6
+ *
7
+ * This is not the same surface as `src/api/user.ts`. Those functions read the
8
+ * *public* REST profile (`/user/{id}/tracks` …), which needs the profile to be
9
+ * public and shows only what that profile exposes. These read what the account
10
+ * itself can see — including a private library — and return gateway shapes, so
11
+ * tracks arrive with a `TRACK_TOKEN` and are directly downloadable.
12
+ *
13
+ * Every response here is account-scoped, so it lives in the per-session cache
14
+ * and is never shared between sessions.
15
+ */
16
+ const request_1 = require("./request");
17
+ const api_1 = require("./api");
18
+ /** Resolve the caller's own user id when one wasn't supplied. */
19
+ const ownUserId = async (userId) => userId !== undefined ? String(userId) : String((await (0, api_1.getUser)()).USER_ID);
20
+ /**
21
+ * The account's own playlists (`playlist.getList`) — **including private ones**,
22
+ * which the public `/user/{id}/playlists` endpoint will not show you.
23
+ *
24
+ * @param userId defaults to the logged-in account
25
+ */
26
+ const getMyPlaylists = async (userId, nb = 50, start = 0) => (0, request_1.request)({ user_id: await ownUserId(userId), nb, start, tab: 'all' }, 'playlist.getList');
27
+ exports.getMyPlaylists = getMyPlaylists;
28
+ /**
29
+ * Loved tracks (`song.getFavorites`). Gateway track objects, so each carries a
30
+ * `TRACK_TOKEN` and can go straight to `resolveDownloadUrls`.
31
+ */
32
+ const getMyFavoriteTracks = async (userId, nb = 50, start = 0) => (0, request_1.request)({ user_id: await ownUserId(userId), nb, start }, 'song.getFavorites');
33
+ exports.getMyFavoriteTracks = getMyFavoriteTracks;
34
+ /**
35
+ * Just the ids of every loved track (`song.getFavoriteIds`) — one small request
36
+ * for the whole set, for diffing a local library against the account.
37
+ */
38
+ const getMyFavoriteTrackIds = () => (0, request_1.request)({}, 'song.getFavoriteIds');
39
+ exports.getMyFavoriteTrackIds = getMyFavoriteTrackIds;
40
+ /** Favourited albums (`album.getFavorites`). */
41
+ const getMyFavoriteAlbums = async (userId, nb = 50, start = 0) => (0, request_1.request)({ user_id: await ownUserId(userId), nb, start }, 'album.getFavorites');
42
+ exports.getMyFavoriteAlbums = getMyFavoriteAlbums;
43
+ /** Favourited artists (`artist.getFavorites`). */
44
+ const getMyFavoriteArtists = async (userId, nb = 50, start = 0) => (0, request_1.request)({ user_id: await ownUserId(userId), nb, start }, 'artist.getFavorites');
45
+ exports.getMyFavoriteArtists = getMyFavoriteArtists;
46
+ /** Playlists the account follows (`playlist.getFavorites`). */
47
+ const getMyFavoritePlaylists = async (userId, nb = 50, start = 0) => (0, request_1.request)({ user_id: await ownUserId(userId), nb, start }, 'playlist.getFavorites');
48
+ exports.getMyFavoritePlaylists = getMyFavoritePlaylists;
49
+ /** Favourited radios (`radio.getFavorites`). */
50
+ const getMyFavoriteRadios = async (userId, nb = 50, start = 0) => (0, request_1.request)({ user_id: await ownUserId(userId), nb, start }, 'radio.getFavorites');
51
+ exports.getMyFavoriteRadios = getMyFavoriteRadios;
52
+ /** Favourited podcast shows (`show.getFavorites`). */
53
+ const getMyFavoriteShows = async (userId, nb = 50, start = 0) => (0, request_1.request)({ user_id: await ownUserId(userId), nb, start }, 'show.getFavorites');
54
+ exports.getMyFavoriteShows = getMyFavoriteShows;
55
+ /**
56
+ * A "more like this" mix seeded from one track (`song.getSearchTrackMix`).
57
+ *
58
+ * Returns full gateway tracks **with `TRACK_TOKEN`s**, so unlike the public
59
+ * radio endpoints the results are immediately downloadable — no `getTrackInfo`
60
+ * round trip per hit.
61
+ *
62
+ * @param sngId the seed track's `SNG_ID`
63
+ */
64
+ const getTrackMix = (sngId, nb = 40, start = 0) => (0, request_1.request)({ sng_id: sngId, start, nb }, 'song.getSearchTrackMix');
65
+ exports.getTrackMix = getTrackMix;
@@ -0,0 +1,28 @@
1
+ import { Transform } from 'stream';
2
+ export interface DecryptPoolOptions {
3
+ /** worker threads to keep. Default `min(4, cpus - 1)`, floor 1. */
4
+ size?: number;
5
+ /** terminate idle workers after this many ms. Default 5000. */
6
+ idleMs?: number;
7
+ }
8
+ /** Tune the pool. Call before first use; resizing later takes effect as workers recycle. */
9
+ export declare const configureDecryptPool: (options: DecryptPoolOptions) => void;
10
+ /** Live pool state, for metrics. */
11
+ export declare const decryptPoolStats: () => {
12
+ workers: number;
13
+ busy: number;
14
+ size: number;
15
+ };
16
+ /** Terminate every worker now. */
17
+ export declare const shutdownDecryptPool: () => Promise<void>;
18
+ /**
19
+ * Like `createDecryptStream`, but batches the work onto a worker pool so the
20
+ * Blowfish never runs on the event loop. Output is byte-identical and ordered.
21
+ *
22
+ * Worth it for a server decrypting several tracks at once; for a single
23
+ * download the plain `createDecryptStream` avoids the thread entirely.
24
+ *
25
+ * @param trackId `SNG_ID`
26
+ * @param startChunk 2048-byte chunk index the first byte corresponds to
27
+ */
28
+ export declare const createPooledDecryptStream: (trackId: string, startChunk?: number) => Transform;
@@ -0,0 +1,230 @@
1
+ "use strict";
2
+ var __importDefault = (this && this.__importDefault) || function (mod) {
3
+ return (mod && mod.__esModule) ? mod : { "default": mod };
4
+ };
5
+ Object.defineProperty(exports, "__esModule", { value: true });
6
+ exports.createPooledDecryptStream = exports.shutdownDecryptPool = exports.decryptPoolStats = exports.configureDecryptPool = void 0;
7
+ /**
8
+ * Optional worker pool for track decryption.
9
+ *
10
+ * Blowfish runs at ~243 MiB/s on one thread, and it runs *on the event loop* —
11
+ * so on a server it is not just a throughput limit, it is latency everyone else
12
+ * pays. Measured decrypting 240 MB in 4 MB batches:
13
+ *
14
+ * ```
15
+ * main thread 930ms 258 MB/s loop lag p95 16.8ms max 20ms
16
+ * worker pool 359ms 669 MB/s loop lag p95 3.8ms max 7ms
17
+ * ```
18
+ *
19
+ * 2.6x the throughput and 4.5x less p95 lag, because the batches move as
20
+ * transferable `ArrayBuffer`s (no copy) and each batch is independent — Deezer
21
+ * re-seeds the IV every 2048-byte stripe and encrypts on `chunkIndex % 3`, so a
22
+ * batch starting on a multiple of 3 needs nothing from the batch before it.
23
+ *
24
+ * **Opt in.** Threads are spawned lazily on first use and terminated after an
25
+ * idle period, but a one-track CLI run should not pay spawn cost at all — so the
26
+ * pool is only used by {@link createPooledDecryptStream}, never by the plain
27
+ * `createDecryptStream`. If a worker cannot be started the stream falls back to
28
+ * decrypting in-thread, so callers never have to handle that case.
29
+ */
30
+ const os_1 = __importDefault(require("os"));
31
+ const stream_1 = require("stream");
32
+ const worker_threads_1 = require("worker_threads");
33
+ const path_1 = __importDefault(require("path"));
34
+ const decrypt_1 = require("./decrypt");
35
+ const STRIPE = 2048;
36
+ /** Batch size in stripes — a multiple of 3 so every batch starts on an encrypted stripe. */
37
+ const BATCH_STRIPES = 2046;
38
+ const BATCH_BYTES = BATCH_STRIPES * STRIPE; // ~4 MB
39
+ let poolSize = Math.max(1, Math.min(4, (os_1.default.cpus().length || 2) - 1));
40
+ let idleMs = 5000;
41
+ /** Tune the pool. Call before first use; resizing later takes effect as workers recycle. */
42
+ const configureDecryptPool = (options) => {
43
+ if (typeof options.size === 'number' && options.size > 0)
44
+ poolSize = Math.floor(options.size);
45
+ if (typeof options.idleMs === 'number' && options.idleMs >= 0)
46
+ idleMs = options.idleMs;
47
+ };
48
+ exports.configureDecryptPool = configureDecryptPool;
49
+ const slots = [];
50
+ let idleTimer = null;
51
+ let spawnFailed = false;
52
+ /** Worker entry: the built `.js` beside this file, or a ts-node bootstrap under ts-node. */
53
+ const spawnWorker = () => {
54
+ if (__filename.endsWith('.ts')) {
55
+ const target = path_1.default.join(__dirname, 'decrypt-worker.ts');
56
+ return new worker_threads_1.Worker(`require('ts-node/register/transpile-only');require(${JSON.stringify(target)});`, { eval: true });
57
+ }
58
+ return new worker_threads_1.Worker(path_1.default.join(__dirname, 'decrypt-worker.js'));
59
+ };
60
+ const armIdleShutdown = () => {
61
+ var _a;
62
+ if (idleTimer)
63
+ clearTimeout(idleTimer);
64
+ if (idleMs <= 0)
65
+ return;
66
+ idleTimer = setTimeout(() => {
67
+ if (slots.every((s) => !s.busy)) {
68
+ for (const s of slots.splice(0))
69
+ void s.worker.terminate();
70
+ }
71
+ }, idleMs);
72
+ (_a = idleTimer.unref) === null || _a === void 0 ? void 0 : _a.call(idleTimer);
73
+ };
74
+ const freeSlot = () => {
75
+ const idle = slots.find((s) => !s.busy);
76
+ if (idle)
77
+ return idle;
78
+ if (slots.length >= poolSize || spawnFailed)
79
+ return null;
80
+ try {
81
+ const slot = { worker: spawnWorker(), busy: false };
82
+ slot.worker.on('error', () => {
83
+ const i = slots.indexOf(slot);
84
+ if (i >= 0)
85
+ slots.splice(i, 1);
86
+ });
87
+ slot.worker.unref();
88
+ slots.push(slot);
89
+ return slot;
90
+ }
91
+ catch {
92
+ spawnFailed = true; // no worker_threads — every caller falls back in-thread
93
+ return null;
94
+ }
95
+ };
96
+ /** Live pool state, for metrics. */
97
+ const decryptPoolStats = () => ({
98
+ workers: slots.length,
99
+ busy: slots.filter((s) => s.busy).length,
100
+ size: poolSize,
101
+ });
102
+ exports.decryptPoolStats = decryptPoolStats;
103
+ /** Terminate every worker now. */
104
+ const shutdownDecryptPool = async () => {
105
+ if (idleTimer)
106
+ clearTimeout(idleTimer);
107
+ await Promise.all(slots.splice(0).map((s) => s.worker.terminate()));
108
+ };
109
+ exports.shutdownDecryptPool = shutdownDecryptPool;
110
+ /** Run one batch on a worker; resolves with the plaintext. */
111
+ const runOnWorker = (slot, trackId, startChunk, batch, seq) => new Promise((resolve, reject) => {
112
+ slot.busy = true;
113
+ const onMessage = (msg) => {
114
+ cleanup();
115
+ resolve(Buffer.from(msg.buf));
116
+ };
117
+ const onError = (err) => {
118
+ cleanup();
119
+ reject(err);
120
+ };
121
+ const cleanup = () => {
122
+ slot.worker.off('message', onMessage);
123
+ slot.worker.off('error', onError);
124
+ slot.busy = false;
125
+ armIdleShutdown();
126
+ };
127
+ slot.worker.on('message', onMessage);
128
+ slot.worker.on('error', onError);
129
+ const ab = batch.buffer.slice(batch.byteOffset, batch.byteOffset + batch.byteLength);
130
+ slot.worker.postMessage({ seq, trackId, startChunk, buf: ab }, [ab]);
131
+ });
132
+ /**
133
+ * Like `createDecryptStream`, but batches the work onto a worker pool so the
134
+ * Blowfish never runs on the event loop. Output is byte-identical and ordered.
135
+ *
136
+ * Worth it for a server decrypting several tracks at once; for a single
137
+ * download the plain `createDecryptStream` avoids the thread entirely.
138
+ *
139
+ * @param trackId `SNG_ID`
140
+ * @param startChunk 2048-byte chunk index the first byte corresponds to
141
+ */
142
+ const createPooledDecryptStream = (trackId, startChunk = 0) => {
143
+ // no worker_threads at all — behave exactly like the in-thread stream
144
+ if (spawnFailed)
145
+ return (0, decrypt_1.createDecryptStream)(trackId, startChunk);
146
+ // accumulate by reference and concat once per batch — concatenating on every
147
+ // write would be O(n^2) with a 4 MB carry
148
+ let parts = [];
149
+ let partsLength = 0;
150
+ let chunkIndex = startChunk;
151
+ let seq = 0;
152
+ let nextToEmit = 0;
153
+ const pending = new Map();
154
+ const inFlight = new Set();
155
+ let failed = null;
156
+ /** emit whatever is now contiguous from `nextToEmit` */
157
+ const drain = (push) => {
158
+ for (let b = pending.get(nextToEmit); b !== undefined; b = pending.get(nextToEmit)) {
159
+ pending.delete(nextToEmit);
160
+ nextToEmit++;
161
+ push(b);
162
+ }
163
+ };
164
+ const dispatch = (batch, start, self) => {
165
+ const mySeq = seq++;
166
+ const slot = freeSlot();
167
+ const job = slot
168
+ ? runOnWorker(slot, trackId, start, batch, mySeq)
169
+ : // pool saturated or unavailable: do this batch in-thread rather than queue
170
+ Promise.resolve().then(() => {
171
+ const s = (0, decrypt_1.createDecryptStream)(trackId, start);
172
+ const parts = [];
173
+ s.on('data', (d) => parts.push(d));
174
+ return new Promise((res, rej) => {
175
+ s.on('end', () => res(Buffer.concat(parts)));
176
+ s.on('error', rej);
177
+ s.end(batch);
178
+ });
179
+ });
180
+ const tracked = job
181
+ .then((out) => {
182
+ pending.set(mySeq, out);
183
+ drain((b) => self.push(b));
184
+ })
185
+ .catch((err) => {
186
+ failed = failed !== null && failed !== void 0 ? failed : err;
187
+ })
188
+ .finally(() => {
189
+ inFlight.delete(tracked);
190
+ });
191
+ inFlight.add(tracked);
192
+ };
193
+ return new stream_1.Transform({
194
+ transform(chunk, _enc, callback) {
195
+ if (failed)
196
+ return callback(failed);
197
+ parts.push(chunk);
198
+ partsLength += chunk.length;
199
+ while (partsLength >= BATCH_BYTES) {
200
+ const merged = parts.length === 1 ? parts[0] : Buffer.concat(parts, partsLength);
201
+ const rest = merged.subarray(BATCH_BYTES);
202
+ parts = rest.length ? [rest] : [];
203
+ partsLength = rest.length;
204
+ dispatch(merged.subarray(0, BATCH_BYTES), chunkIndex, this);
205
+ chunkIndex += BATCH_STRIPES;
206
+ }
207
+ // keep at most 2 batches per worker outstanding
208
+ if (inFlight.size > poolSize * 2) {
209
+ Promise.race(inFlight).then(() => callback(failed !== null && failed !== void 0 ? failed : undefined), callback);
210
+ return;
211
+ }
212
+ return callback(failed !== null && failed !== void 0 ? failed : undefined);
213
+ },
214
+ async flush(callback) {
215
+ if (partsLength) {
216
+ dispatch(parts.length === 1 ? parts[0] : Buffer.concat(parts, partsLength), chunkIndex, this);
217
+ parts = [];
218
+ partsLength = 0;
219
+ }
220
+ while (inFlight.size) {
221
+ await Promise.race(inFlight).catch(() => undefined);
222
+ }
223
+ if (failed)
224
+ return callback(failed);
225
+ drain((b) => this.push(b));
226
+ return callback();
227
+ },
228
+ });
229
+ };
230
+ exports.createPooledDecryptStream = createPooledDecryptStream;
@@ -0,0 +1,6 @@
1
+ export interface DecryptJob {
2
+ seq: number;
3
+ trackId: string;
4
+ startChunk: number;
5
+ buf: ArrayBuffer;
6
+ }
@@ -0,0 +1,22 @@
1
+ "use strict";
2
+ Object.defineProperty(exports, "__esModule", { value: true });
3
+ /**
4
+ * Worker half of the decrypt pool. Receives a stripe-aligned batch plus the
5
+ * chunk index it starts at, and returns the plaintext with the buffer
6
+ * transferred back (no copy).
7
+ *
8
+ * Batches are independent because Deezer's scheme re-seeds the IV on every
9
+ * 2048-byte stripe and encrypts on `chunkIndex % 3`, so a batch that starts on
10
+ * a multiple of 3 needs no state from the batch before it.
11
+ */
12
+ const worker_threads_1 = require("worker_threads");
13
+ const decrypt_1 = require("./decrypt");
14
+ worker_threads_1.parentPort === null || worker_threads_1.parentPort === void 0 ? void 0 : worker_threads_1.parentPort.on('message', (job) => {
15
+ const engine = new decrypt_1.TrackDecryptStream(job.trackId, job.startChunk);
16
+ const body = engine.write(Buffer.from(job.buf));
17
+ const tail = engine.final();
18
+ const out = tail.length ? Buffer.concat([body, tail]) : body;
19
+ // copy into a standalone ArrayBuffer so the whole thing can be transferred
20
+ const ab = out.buffer.slice(out.byteOffset, out.byteOffset + out.byteLength);
21
+ worker_threads_1.parentPort === null || worker_threads_1.parentPort === void 0 ? void 0 : worker_threads_1.parentPort.postMessage({ seq: job.seq, buf: ab }, [ab]);
22
+ });
@@ -34,9 +34,23 @@ const getBlowfishKey = (trackId) => {
34
34
  return key;
35
35
  };
36
36
  const CHUNK = 2048;
37
- // Note: memoising the initialised Blowfish schedule per track was measured and
38
- // rejected — key setup is 39.7µs against 33ms to decrypt an 8 MiB file (0.12%),
39
- // break-even at ~10 KiB decrypted per key. Not worth the cache.
37
+ // Two optimisations were built, measured and rejected here:
38
+ //
39
+ // 1. Memoising the initialised Blowfish schedule per track. Key setup is 39.7µs
40
+ // against 33ms to decrypt an 8 MiB file (0.12%), break-even at ~10 KiB
41
+ // decrypted per key.
42
+ // 2. A worker_threads pool, to move Blowfish off the event loop. Decrypting
43
+ // 4 MB batches synchronously it looked like a big win (2.6x throughput,
44
+ // 4.5x less p95 lag) — but that benchmark manufactured the problem. Through
45
+ // a real pipeline the in-thread stream below already shows **p95 loop lag of
46
+ // 0.0ms**, because it works a socket read at a time rather than a whole file,
47
+ // and the pooled version came out at 0.72x once three extra 4 MB copies per
48
+ // batch (concat, transfer-slice, worker return-slice) were paid. Decrypt only
49
+ // becomes the binding constraint above ~265 MB/s of aggregate download in one
50
+ // process, and Deezer's quota bites long before that.
51
+ //
52
+ // Don't re-attempt either without a workload that actually shows decrypt as the
53
+ // bottleneck.
40
54
  /**
41
55
  * Decrypt a downloaded track. Deezer applies Blowfish-CBC "stripe" obfuscation:
42
56
  * the file is split into 2048-byte chunks and only every third chunk (0, 3, 6…)
@@ -1,2 +1,11 @@
1
1
  import type { lyricsType, trackType } from '../types';
2
- export declare const getTrackLyrics: (track: trackType) => Promise<lyricsType | null>;
2
+ /**
3
+ * Deezer's lyrics, falling back to scraping Musixmatch when a track has no
4
+ * `LYRICS_ID` (instrumentals, and anything Deezer simply lacks).
5
+ *
6
+ * That fallback is not free: it is two requests per track that has no Deezer
7
+ * lyrics — 16 of them on a 14-track album — and it fails outright wherever
8
+ * Musixmatch blocks the request. Pass `fallback: false` to skip it and keep only
9
+ * what Deezer serves.
10
+ */
11
+ export declare const getTrackLyrics: (track: trackType, fallback?: boolean) => Promise<lyricsType | null>;
@@ -12,15 +12,24 @@ const getTrackLyricsWeb = async (track) => {
12
12
  return null;
13
13
  }
14
14
  };
15
- const getTrackLyrics = async (track) => {
15
+ /**
16
+ * Deezer's lyrics, falling back to scraping Musixmatch when a track has no
17
+ * `LYRICS_ID` (instrumentals, and anything Deezer simply lacks).
18
+ *
19
+ * That fallback is not free: it is two requests per track that has no Deezer
20
+ * lyrics — 16 of them on a 14-track album — and it fails outright wherever
21
+ * Musixmatch blocks the request. Pass `fallback: false` to skip it and keep only
22
+ * what Deezer serves.
23
+ */
24
+ const getTrackLyrics = async (track, fallback = true) => {
16
25
  if (track.LYRICS_ID > 0) {
17
26
  try {
18
27
  return await (0, api_1.getLyrics)(track.SNG_ID);
19
28
  }
20
29
  catch (err) {
21
- return await getTrackLyricsWeb(track);
30
+ return fallback ? await getTrackLyricsWeb(track) : null;
22
31
  }
23
32
  }
24
- return await getTrackLyricsWeb(track);
33
+ return fallback ? await getTrackLyricsWeb(track) : null;
25
34
  };
26
35
  exports.getTrackLyrics = getTrackLyrics;
@@ -30,6 +30,12 @@ export interface AddTrackTagsOptions {
30
30
  embedArtistImage?: boolean;
31
31
  /** fetch + embed lyrics. default true */
32
32
  writeLyrics?: boolean;
33
+ /**
34
+ * When Deezer has no lyrics for a track, fall back to scraping Musixmatch.
35
+ * Two extra requests per track that needs it (16 on a typical album), and they
36
+ * fail wherever Musixmatch blocks you. default true.
37
+ */
38
+ lyricsFallback?: boolean;
33
39
  /** embed synced LRC (FLAC Vorbis comment only; MP3 has no v2.3 synced frame). default true */
34
40
  embedSyncedLyrics?: boolean;
35
41
  /**
@@ -28,11 +28,18 @@ const DEFAULTS = {
28
28
  embedCover: true,
29
29
  embedArtistImage: true,
30
30
  writeLyrics: true,
31
+ lyricsFallback: true,
31
32
  embedSyncedLyrics: true,
32
33
  richCredits: true,
33
34
  deezerIds: true,
34
35
  includeRank: true,
35
36
  };
37
+ // Per-track by necessity: `SNG_CONTRIBUTORS` is exposed *only* by `song.getData`.
38
+ // Probed against the live gateway — `song.getListData` (the batch endpoint
39
+ // `refreshTrackTokens` uses) returns 44 fields including VERSION, GAIN, ISRC,
40
+ // ART_PICTURE and URL_REWRITING, but no contributors; neither does
41
+ // `song.getListByAlbum`. So a 14-track album costs 14 of these, and the only way
42
+ // to avoid them is `{richCredits: false}`, which trades away credits and BPM.
36
43
  const hydrate = async (track) => {
37
44
  var _a, _b, _c, _d, _e, _f, _g;
38
45
  if (track.SNG_CONTRIBUTORS !== undefined && track.VERSION !== undefined && track.GAIN !== undefined) {
@@ -78,7 +85,7 @@ const resolveTagModel = async (trackInput, options = {}) => {
78
85
  options.lyrics !== undefined
79
86
  ? Promise.resolve(options.lyrics)
80
87
  : opt.writeLyrics
81
- ? (0, getTrackLyrics_1.getTrackLyrics)(track).catch(() => null)
88
+ ? (0, getTrackLyrics_1.getTrackLyrics)(track, opt.lyricsFallback).catch(() => null)
82
89
  : Promise.resolve(null),
83
90
  options.publicTrack !== undefined
84
91
  ? Promise.resolve(options.publicTrack)
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "gerdur-core",
3
- "version": "2.15.0",
3
+ "version": "2.17.0",
4
4
  "description": "Deezer API client, cross-service URL resolution, Blowfish track decryption and MP3/FLAC metadata tagging — the engine behind the gerdur CLI.",
5
5
  "keywords": [
6
6
  "deezer",