@doitian/dsh-music 0.1.4 → 0.1.5

package/README.md

@@ -197,15 +197,109 @@ something:
   than missing the signal. Removing the track that is playing also advances
   playback, which a removal otherwise would not.
 
+### Skips
+
+A skip is a fourth, softer signal, and nobody has to press anything for it:
+**moving to the next track before 30 seconds, or before a quarter of the track
+when that is longer**, records one — from the transport row or the agent's
+`music_control next`; jumping to a queue row does not count. It is local, like
+a dislike.
+
+What does *not* count is as deliberate as what does: going back, a track ending
+or failing on its own, single-repeat mode (which stays on the track), and a
+track that never reported a position — paused, blocked by autoplay, or still
+loading — because a track nobody heard was not rejected. Nor does the next
+that the transport row's **✕** presses after a dislike: the dislike is already
+the record, and counting it as a skip too would weigh an early dislike more
+than a later one.
+
+The DJ reads skips at two grains:
+
+- **Per track.** One skip is a penalty, since it may have been the wrong
+  moment rather than the wrong song. A second takes the track out of the pool
+  for good, like a dislike.
+- **Per artist.** The skip marks the play it cut short in the history, and that
+  play stops counting toward the artist's affinity — otherwise abandoning an
+  artist's songs would keep making them a "favourite". Skipped plays then count
+  against the artist instead, and the model is shown the recently skipped
+  tracks and the artists skipped more than once.
+
 ## The AI DJ
 
 The DJ keeps the queue stocked. When the queue drops below
 `djAutoExtendBelow`, it gathers candidates and appends a batch:
 
-- similar songs for the current and recently played tracks (`simiSong`),
+- similar songs (`simiSong`) for the current track and recent plays the
+  listener heard through — never a disliked or skipped one, and topped up from
+  the likes when the history is thin,
+- personal FM (私人FM), two calls of three tracks, when signed in,
+- up to six liked tracks not played in the last three days, marked `[liked]`
+  for the model,
 - the daily recommendations and the anonymous new-song feed,
-- the official charts,
-- keyword search for the mood brief.
+- the four general charts — 飙升榜, 新歌榜, 原创榜, 热歌榜 — sampled together,
+- for a mood brief, its best-loved playlists, and a keyword song search.
+
+Personal FM stands in for heartbeat mode (心动模式), which would be the better
+personalised source but answers `code 500` on the plain web API for every seed;
+it needs the `weapi` transport this plugin does not implement. `simiSong`
+answers five tracks whatever `limit` asks for.
+
+**A brief finds music through playlists.** A song search only matches titles
+and lyrics, so "rainy afternoon jazz" mostly finds library tracks literally
+named *Rainy Afternoon Jazz*. Listeners name and tag playlists by mood (雨天,
+爵士, 深夜…), so the brief searches playlists, takes the most played among
+the six most relevant (skipping any under ten tracks), and samples across
+them.
+
+**The model rewrites the brief first.** Playlists are named in Chinese, so an
+English brief searched as written matches badly — live, "90s cantopop" found
+Notorious B.I.G. and Eminem. When a model route is available, one call turns
+the brief into up to three Chinese search terms the way listeners name
+playlists (`90年代 粤语金曲`, `港乐 经典`, `粤语老歌`), and the same brief then
+found 张国荣, 刘德华 and 许冠杰. Each term gets its own playlist, so one broad
+term cannot take every slot, and the song search uses the first.
+
+The rewrite is made **once per brief, not per plan**, and kept for a week: a
+brief means the same thing next week. It uses the curation route and the same
+session identity. Without a model, or when the rewrite answers nothing usable,
+the brief is searched as written — the plan never fails over it — and the
+rewrite is tried again after half an hour rather than on every plan. The terms
+used are reported as `searchedAs` on the plan and on `dj.lastPlan`, and the
+`music_dj` result says *Brief searched as: …*.
+
+### Caching, sampling and pacing
+
+Most sources change on the scale of hours or days, so each is cached for
+about as long as it stays the same, per account where it is personal:
+
+| Source | Kept for |
+|---|---|
+| similar songs (per seed), track details (per id) | 24 h |
+| a playlist's track list | 12 h |
+| a brief's rewrite into search terms | 7 days (30 min after a failed one) |
+| the charts, a brief's searches | 6 h |
+| the daily recommendations | 3 h |
+| the new-song feed | 1 h |
+| personal FM | never — every call answers a different three |
+
+**Every plan samples.** Each source contributes a random handful up to its
+quota — similar 14, daily 8, charts 8, new songs 6, likes 6, FM 6, and for a
+brief 15 from its playlists plus 8 from the song search — drawn only from
+tracks still eligible, and kept in the source's own order. Taking the first
+few would serve the same tracks until the source changed; sampling a cached
+list yields a different handful every plan for free. It also reaches past
+what NetEase sends in full: for a playlist the account does not own, only
+the first 20 tracks come with details, but every id does, so the DJ samples
+ids from the whole list and looks up just the ones it chose. The quotas also
+keep any one source from crowding the others out of the pool.
+
+**Requests are paced, not fired as a burst.** Each request starts a random
+0.25–0.75 s after the previous one, so a plan reads like someone clicking
+through the app rather than a crawler. Only the DJ's own probing is paced;
+searches and plays you ask for go out at once. Live, a first plan with a
+brief sends 14 requests over about 7 s; the next one, mostly from cache,
+sends 2. The DJ stocks the queue before it runs dry, so the wait is not
+heard.
 
 Then one of two tiers chooses:
 
@@ -214,9 +308,15 @@ Then one of two tiers chooses:
   (recently played, favourite artists, likes, dislikes) goes to
   `ctx.llm.stream()`, and the model returns the picks and a one-line vibe. This
   is the tier that reasons about a mood.
-- **Heuristic tier** — always available: candidates are scored by artist
-  affinity, novelty against play history, and explicit like/dislike feedback,
-  then interleaved so consecutive tracks do not share an artist.
+- **Heuristic tier** — always available: candidates are scored by mood-brief
+  match and artist affinity, then interleaved so consecutive tracks do not
+  share an artist. See [Heuristic tier](#heuristic-tier).
+
+Either way, the pool never holds a track that is disliked, recently played, or
+**already in the queue** — a batch is appended, so a pick the queue already
+holds would be dropped and the batch would come up short. The model is also
+told which track its picks will follow, and is held to the count it was asked
+for.
 
 **Every model call carries a session identity.** A leaf call cannot set headers
 — `GenerateOptions` has no `headers` field — so `sessionId` is the only identity
@@ -368,7 +468,28 @@ batch; it never leaves the queue empty.
 
 ### Heuristic tier
 
-Always available, used whenever the model tier is unavailable:
+Always available, used whenever the model tier is unavailable. Each candidate
+scores:
+
+| Signal | Weight |
+|---|---|
+| matches the mood brief (keyword search) | +3.0 |
+| from personal FM | +1.0 |
+| similar to the current or recent tracks | +0.6 |
+| a rested like | +0.5 |
+| per artist among the listener's most played (skipped plays excluded) | +2.2 |
+| per artist shared with the current track | +1.4 |
+| per artist play count in the last 60 plays, skips excluded | +0.15 each, capped at +1.0 |
+| per artist skip among the last 150 plays | −0.6 each, capped at −2.4 |
+| skipped once (twice excludes it) | −1.5 |
+| VIP-only | −0.4 |
+| jitter, so repeat plans differ | 0–0.8 |
+
+The brief outweighs a favourite artist on purpose: it is the listener's explicit
+ask, and without that weight a chart hit by a favourite would outrank every
+track that matches it. The ranked list is then interleaved starting from the
+track the batch is appended after, so no two adjacent tracks share an artist —
+including the seam between the old queue and the new batch.
 
 ## Configuration
 
@@ -419,6 +540,7 @@ browser (DSH web GUI, http://127.0.0.1:<port>)
                           ├─ lib/session.js   cookie store + QR login state machine
                           ├─ lib/state.js     queue, cursor, transport revisions
                           ├─ lib/dj.js        candidate pool + model/heuristic tiers
+                          ├─ lib/cache.js     source cache, pool sampling, request pacing
                           ├─ lib/likes.js     the account's like state, cached
                           └─ lib/router.js    JSON API, HTML, Range-capable audio proxy
 ```
@@ -529,8 +651,8 @@ down first. A restart brings both halves back into agreement.
 
 ```powershell
 npm run check        # node --check on every module
-npm test             # 114 deterministic tests: pure, like state, DJ, browser half
-npm run test:live    # 33 integration tests against the live NetEase API
+npm test             # 154 deterministic tests: pure, like state, DJ, browser half
+npm run test:live    # 34 integration tests against the live NetEase API
 npm run test:all     # both
 ```
 
@@ -546,11 +668,12 @@ network cases inside it.
 Or run one file directly:
 
 ```powershell
-node test/netease.test.mjs   # 32 pure: normalisation, quality ladder, likes, cookies, taste, player state
+node test/netease.test.mjs   # 34 pure: normalisation, quality ladder, likes, cookies, taste, skips, player state
+node test/cache.test.mjs     # 10 source cache: lifetimes, shared loads, sampling, pacing
 node test/likes.test.mjs     # 12 like-state cache: what counts as an answer, refusals, batching, writes
-node test/dj.test.mjs        # 37 AI DJ: model call identity, route resolution, failure reporting, queue invariants
+node test/dj.test.mjs        # 60 AI DJ: model call identity, route resolution, failure reporting, curation, skips, pool sources, caching, brief rewriting, queue invariants
 node test/client.test.mjs    # 38 browser half: the engine against a fake DOM, and the page it pairs with
-node test/host.test.mjs      # 33 integration: routes, streaming, curation, quality, taste
+node test/host.test.mjs      # 34 integration: routes, streaming, curation, quality, taste, skips
 ```
 
 The DJ tests drive `ctx.llm.stream()` with a stub that emits the documented
@@ -583,7 +706,7 @@ poll, and a track change replaces the pane. Two more cover the pane's shape: it
 is capped to a few lines, collapses to its header on demand, and remembers that
 choice across loads — while still fetching the lines, so expanding is instant.
 
-`npm test` runs the four deterministic files in sequence (`npm run test:all`
+`npm test` runs the five deterministic files in sequence (`npm run test:all`
 adds the live one), deliberately **not** `node --test <dir>`: the directory form forks one child process per file, which
 is blocked in sandboxed environments.
 

package/lib/cache.js

@@ -0,0 +1,156 @@
+/**
+ * How the DJ reads its sources: a time-bounded cache, the sampling it draws
+ * its pool with, and the pacing its requests are sent at.
+ *
+ * Most of the DJ's sources change on the scale of hours or days — a chart
+ * daily, similarity effectively never — so fetching each one on every plan
+ * spends requests to learn nothing. The cache keeps each source for as long as
+ * it plausibly stays the same, and the DJ samples from what it holds, so a
+ * long cached list still yields a different handful each time. What does go
+ * out is paced like a person browsing, not fired as a burst.
+ *
+ * @module dsh-music/cache
+ */
+
+/** Entries kept at most; the oldest are dropped first. */
+const MAX_ENTRIES = 2_000;
+
+export class TtlCache {
+  /**
+   * @param {object} [options]
+   * @param {() => number} [options.now] clock, injectable for tests.
+   * @param {number} [options.maxEntries]
+   */
+  constructor({ now = Date.now, maxEntries = MAX_ENTRIES } = {}) {
+    this.now = now;
+    this.maxEntries = maxEntries;
+    /** @type {Map<string, {value: unknown, expiresAt: number}>} */
+    this.entries = new Map();
+  }
+
+  /** The live value for `key`, or `undefined` when absent or expired. */
+  peek(key) {
+    const entry = this.entries.get(key);
+    if (!entry) return undefined;
+    if (entry.expiresAt <= this.now()) {
+      this.entries.delete(key);
+      return undefined;
+    }
+    return entry.value;
+  }
+
+  set(key, value, ttlMs) {
+    this.entries.delete(key);
+    this.entries.set(key, { value, expiresAt: this.now() + ttlMs });
+    while (this.entries.size > this.maxEntries) this.entries.delete(this.entries.keys().next().value);
+    return value;
+  }
+
+  /**
+   * The cached value for `key`, loading it when absent or expired.
+   *
+   * The pending load is what gets cached, so two plans racing for the same
+   * source share one request. A failed load is forgotten rather than cached:
+   * the next plan should try again, not inherit the outage for the whole TTL.
+   *
+   * @template T
+   * @param {string} key
+   * @param {number} ttlMs
+   * @param {() => Promise<T>} load
+   * @returns {Promise<T>}
+   */
+  get(key, ttlMs, load) {
+    const cached = this.peek(key);
+    if (cached !== undefined) return cached;
+    const pending = Promise.resolve().then(load);
+    this.set(key, pending, ttlMs);
+    pending.catch(() => {
+      if (this.entries.get(key)?.value === pending) this.entries.delete(key);
+    });
+    return pending;
+  }
+
+  clear() {
+    this.entries.clear();
+  }
+
+  get size() {
+    return this.entries.size;
+  }
+}
+
+/**
+ * A random `count` of `items`, kept in their original order.
+ *
+ * Random so a cached list still yields a different handful each plan — taking
+ * the first `count` would serve the same tracks until the source changes. In
+ * order, because a source's order carries meaning (chart rank, search
+ * relevance), and the pool keeps it.
+ *
+ * @template T
+ * @param {T[]} items
+ * @param {number} count
+ * @param {() => number} [random]
+ * @returns {T[]}
+ */
+export function sampleInOrder(items, count, random = Math.random) {
+  if (count >= items.length) return [...items];
+  if (count <= 0) return [];
+  const indices = items.map((_, index) => index);
+  for (let index = 0; index < count; index += 1) {
+    const swap = index + Math.floor(random() * (indices.length - index));
+    [indices[index], indices[swap]] = [indices[swap], indices[index]];
+  }
+  return indices
+    .slice(0, count)
+    .sort((a, b) => a - b)
+    .map((index) => items[index]);
+}
+
+/**
+ * Spaces out request starts by a randomised gap.
+ *
+ * A plan touches up to a dozen sources, and firing them at once is a crawler's
+ * signature: a tight burst at a machine-regular rhythm. Starts are spaced
+ * instead, each by its own random gap, so the traffic reads like someone
+ * clicking through the app. Only starts are spaced — a request already sent may
+ * still be in flight when the next starts, as on any page that is loading.
+ */
+export class Pacer {
+  /**
+   * @param {object} [options]
+   * @param {number} [options.minGapMs] the shortest gap between two starts.
+   * @param {number} [options.jitterMs] the random span added to each gap.
+   * @param {() => number} [options.now]
+   * @param {(ms: number) => Promise<void>} [options.sleep]
+   * @param {() => number} [options.random]
+   */
+  constructor({
+    minGapMs = 250,
+    jitterMs = 500,
+    now = Date.now,
+    sleep = (ms) => new Promise((resolve) => setTimeout(resolve, ms)),
+    random = Math.random,
+  } = {}) {
+    this.minGapMs = minGapMs;
+    this.jitterMs = jitterMs;
+    this.now = now;
+    this.sleep = sleep;
+    this.random = random;
+    /** The earliest the next request may start. */
+    this.nextAt = 0;
+  }
+
+  /**
+   * Run `task` at the next free start slot.
+   * @template T
+   * @param {() => Promise<T>} task
+   * @returns {Promise<T>}
+   */
+  run(task) {
+    const at = Math.max(this.now(), this.nextAt);
+    this.nextAt = at + this.minGapMs + this.random() * this.jitterMs;
+    const wait = at - this.now();
+    return (wait > 0 ? this.sleep(wait) : Promise.resolve()).then(task);
+  }
+}