@wenathlan/debonair 2.0.118 → 2.0.127

package/docs/planning.pipeline.md

@@ -0,0 +1,198 @@
+# planning.pipeline — the 120+ step ladder of the full music pipeline
+
+> the wave the owner ordered: ingest a real track → the engine decomposes it
+> into the parametrized vector document → the document lands in the db with
+> the owner id and the big pointers → the references power the pattern miner
+> per genre → the composer generates a similar vector json → the engine
+> renders real audio → the take plays, publishes and shows in the vitrine.
+> every heavy stage runs SERVER SIDE (the gateway jobs, the saddle AVM
+> modalities) — the visitor's browser, cpu and gpu never run the pipeline.
+> companion of docs/planning.ladder.md (the kathesis verification ladder)
+> and docs/planning.v2.28-completo.md.
+>
+> ## the code synthesis of outros.debonair (the reference verdict)
+>
+> the full code read of the reference folder closed the method question —
+> the parts that PROVED real sound, the parts the composer mines:
+> - `v2/src/lib/audio/engine.ts` — the voice recipes to reimplement root
+>   side: kick pitch env 165→52→40 hz with the noise click, snare noise
+>   bandpass 1750 + dual triangle body, hat hp 7800, clap 4 bursts 9.5ms
+>   apart, 808 bass pitch snap hz*2→hz in 45ms through a drive shaper,
+>   supersaw lead, pad lp with the 0.28hz lfo, keys tremolo, master chain
+>   tanh waveshaper k=3.2 → compressor −13/18/4.5 → analyser, feedback
+>   delay 0.27s fb 0.34, reverb noise impulse tail pow(1−i/len, 2.2), the
+>   140ms look-ahead transport and the OfflineAudioContext render mirror →
+>   wav 16-bit / mp3 lamejs — the render proof the composer rides.
+> - `v2/src/lib/audio/generator.ts` — the composition brain: 48 scales, 9
+>   style presets with exact drum lanes (trap kick [0,6,10] snare [4,12]
+>   hat 16th lanes clap [4,12] swing 0.12; drill, lofi, house...), bass
+>   locked to kick, markov 1st-order melodies with the ±3 semitone bias,
+>   DRIFT_CYCLE and ENERGY_CYCLE build/drop, seeded prng (same prompt same
+>   take), vary() diatonic drift.
+> - `v2/src/app/api/analyze + similar + trainOnProject` — the pattern
+>   miner shape: note bigrams, 16-step drum histograms (adopt steps above
+>   0.55 after trainedOn ≥ 3), embedding centroid, bpm histogram, cosine
+>   similarity over the 32-dim embedding — the exact miner PV3
+>   reimplements over our ingest documents.
+> - `v1/v1 page.tsx` — the 6-layer biquad separation, the pcm store by
+>   session:second:track and the priority playback ladder (stored pcm →
+>   hex inline → seed synth → pattern synth) with the AudioBufferSource
+>   scheduling and the 1.0s per-second grid.
+> - `mjs/analyser.mjs + ffmpeg.mjs` — the node decode lane (ffmpeg f32le
+>   → stft → per_second vectors) with the known bpm bugs (no octave fold
+>   guard) our analyze already handles.
+> - the FAKE parts the new code never repeats: the ~200 decorative
+>   *Id/*Hash fields per second (constants and xor cosmetics), the
+>   expandFromSeed checksum claim (the checksum never validates — synthesis
+>   cannot reproduce pcm), the hex semantic signatures (0xKICK_SUB_HEAVY),
+>   the empty prisma/db/mini-services scaffolds, the yamnet removal.
+> - the honest separation limit the reference states: elements sharing the
+>   spectral range AND the pan position are mathematically inseparable
+>   without a trained model — the hpss pair (harmonic/percussive) is the
+>   proven floor, the vocal/bass/drum lanes ride the band filters on top.
+
+## wave PV0 — what already stands (verified, committed)
+
+- [x] PV0.001 hpss real separation with centered framing, ifft conj law and
+      f64 ola — conservation 2.3e-7 over the whole signal (hpss.ts)
+- [x] PV0.002 the ingest document debonair.ingest/1: mix + harmonic/percussive
+      stems, one vector frame per second (centroid, flatness, chroma, bands,
+      zcr, rms, beat, onsets) and the fnv-1a pcm hash (ingest.ts)
+- [x] PV0.003 the ingest store: gateway https with the house host law and the
+      byte-capped offline mirror (ingeststore.ts); sqlite rows through
+      parameter binding only (ingestdb.ts)
+- [x] PV0.004 the /ingest bench walking decode → separate → analyze →
+      segment → store with the phase meter (Sol/ingest/ingest.tsx)
+- [x] PV0.005 the typed-URL law: devthink barra <app> lands inside the os on
+      all ten family apps (?arrive handoff, ?independent escape)
+- [x] PV0.006 the 339-test battery green, the 402kb bundle building clean
+
+## wave PV1 — the reference system (the ids, the pointers, the server law)
+
+- [ ] PV1.001 schema upgrade of ingest_documents: owner_id (BIGINT pointer),
+      kind ("reference" | "render" | "upload"), visibility ("private" |
+      "vitrine"), blob pointers for mix and stems, size, sha256 — parameter
+      binding only, mirroring prisma/schema.prisma
+- [ ] PV1.002 the id mint: bigintids.ts — time-ordered 64-bit ids (the big
+      pointers the house law names), zero storage, deterministic tests
+- [ ] PV1.003 the gateway routes of ingest: POST /api/v1/ingest/upload
+      (multipart, streams to the job store, returns the job id), GET
+      /api/v1/ingest/jobs/:id (progress: decode, separate, analyze, segment,
+      store), GET /api/v1/ingest (the shelf rows), GET /api/v1/ingest/:id,
+      DELETE /api/v1/ingest/:id
+- [ ] PV1.004 the server worker ingestworker.ts (node, the gateway lane):
+      decode (audio-decode/wav) → hpss separate → analyze → segmentsof →
+      ingestPcm → wavwriter per stem + mix → blob store — the browser never
+      runs any of it
+- [ ] PV1.005 the blob store contract blobstore.ts: put(stream) → pointer
+      {id, bytes, sha256, kind}, get(id) → stream — the vault law (the
+      storage app owns the engine; the gateway serves it), git-lfs friendly
+- [ ] PV1.006 the job ledger table ingest_jobs (id BIGINT, owner_id, state,
+      progress, error, document_id, created_at, finished_at)
+- [ ] PV1.007 the bench rewired: file → POST upload → poll the job → the
+      phase meter reads the server progress → the document renders — zero
+      dsp on the visitor's machine; the local opt-in path stays behind an
+      explicit "run on this device" key
+- [ ] PV1.008 every document row carries the owner id (the user ingest or
+      the system ia ingest — both stamp the same pointer chain)
+- [ ] PV1.009 the system ingest lane: the ia (or an mcp automation, opt-in)
+      posts the same upload route with the service identity — the reference
+      library grows without a human
+- [ ] PV1.010 tests: bigintids monotonicity, blobstore round trip, worker
+      end-to-end on a synthetic wav, the routes contract with a fake fetch
+
+## wave PV2 — the playback proof (the ingested track plays)
+
+- [ ] PV2.001 the mix and stem wavs ride the blob store; GET
+      /api/v1/blob/:id streams them with the right mime (audio/wav)
+- [ ] PV2.002 the playerbus learns the reference take: the ingested mix
+      loads as a take the bar plays (the same bus the generated takes ride)
+- [ ] PV2.003 the bench document view gains the play key: the stored mix
+      streams through the shared bar (the proof the owner demanded: ingest →
+      play, no re-render)
+- [ ] PV2.004 stem audition: harmonic and percussive playable solo/muted
+      (the stems the worker stored), the conservation audible
+- [ ] PV2.005 the library shelf rows carry the play key too (the ingested
+      references sit beside the generated takes, one grammar)
+- [ ] PV2.006 tests: the wav round trip (store → stream → decode equals the
+      source samples), the bus contract for reference takes
+
+## wave PV3 — the pattern miner (the genre tool the references feed)
+
+- [ ] PV3.001 patternminer.ts: reads the ingested documents of a genre and
+      answers the genre profile — bpm distribution, key pool, chord ladder
+      frequency, onset density curves, band balance, loudness targets
+- [ ] PV3.002 the genre profile document debonair.profile/1 (deterministic,
+      parametrized, versioned) — stored in genre_profiles with the reference
+      ids that fed it (the pointer chain audit-ready)
+- [ ] PV3.003 similarity: the vector distance over frames (cosine on the
+      normalized segment vectors) — nearest references answer a prompt
+- [ ] PV3.004 the miner runs server side (a gateway job like the ingest
+      worker); the browser asks, never computes
+- [ ] PV3.005 the vitrine surfaces the profiles (the genre cards the home
+      shows come from the db, never hardcoded)
+- [ ] PV3.006 tests: the miner on synthetic documents of two genres
+      separates, the profile determinism, the similarity ranking
+
+## wave PV4 — the composer (prompt → similar json → real audio → play)
+
+- [ ] PV4.001 the composer prompt → genre match (genrematrix) → the profile
+      of that genre → a seeded vector plan similar to the references
+- [ ] PV4.002 the render: audioengine renders the plan offline (the existing
+      renderTakeOffline lane) — kick/snare/hats sample-accurate, bass, keys,
+      pads, the arrangement the plan carries
+- [ ] PV4.003 the render job table render_jobs (exists) wired to the worker:
+      queued → rendering → done, the wav lands in the blob store, the track
+      row lands in tracks with the pointers
+- [ ] PV4.004 the generated take plays in the same bar, publishes to the
+      vitrine (the spotify/netflix shelf the owner described) and downloads
+- [ ] PV4.005 the vocal lane grows after the instrumental proof (the
+      SINTESE-VOCAL-AI line), opt-in, never blocking the wave
+- [ ] PV4.006 tests: the same prompt+seed renders the same bytes, the render
+      job lifecycle, the vitrine rows
+
+## wave PV5 — the design reform (debonair wears the devthink index face)
+
+- [ ] PV5.001 screenshot the live os home and every debonair page (the
+      browser session) — the gap list against the index face
+- [ ] PV5.002 the shell chrome: the window, the pin rail and the player bar
+      re-tuned to the os grammar (the trylle surfaces, the fluent shadows,
+      the campaign motion) — no drift tokens
+- [ ] PV5.003 the home: the prompt-first hero over the noir stage retuned,
+      the song grid rows in the streaming grammar (cover, title, genre line,
+      duration, play on hover), the shimmer and the stagger
+- [ ] PV5.004 the ingest bench in the same face (the dropzone, the meter,
+      the cards, the shelf)
+- [ ] PV5.005 the generate/library/studio/settings pages one pass each —
+      short copy, no dots, the same nav law, every key answering
+- [ ] PV5.006 the 404 and the intro re-faced
+- [ ] PV5.007 the live verification: every route of the app inside the power
+      clip and standalone (?independent), screenshots compared
+- [ ] PV5.008 the same face check on light theme
+
+## wave PV6 — the family integration (the engines talk)
+
+- [ ] PV6.001 the gateway app serves the debonair api lane (the routes of
+      PV1/PV3/PV4 ride the gateway the family shares)
+- [ ] PV6.002 the vault stores the blobs (the blobstore pointer resolves
+      through the vault engine when configured, git-lfs backed)
+- [ ] PV6.003 the saddle AVM modalities run the heavy jobs (the worker lane
+      declares the sandbox modality, opt-in per deploy) — the visitor's
+      machine stays clean even at scale
+- [ ] PV6.004 the mcp automation lane (opt-in): an external agent ingests
+      and generates through the same gateway with its own identity
+- [ ] PV6.005 the debonair package.json exports the engine library (the npm
+      face: analyze, hpss, ingest, patternminer, audioengine, wavwriter —
+      zero hardcode, everything configured through options/db)
+
+## wave PV7 — the verification ladder (the proof the owner reads)
+
+- [ ] PV7.001 the full battery green (vitest) on every wave's modules
+- [ ] PV7.002 the typecheck + biome + build clean on every push
+- [ ] PV7.003 the live proof: ingest a real track through the ui, watch the
+      server do the work, press play, hear the mix, hear the stems
+- [ ] PV7.004 the live proof: generate from a genre with references, hear
+      the take, publish to the vitrine
+- [ ] PV7.005 the live proof: devthink barra debonair → the window inside
+      the os, the bench inside it, no flash, one boot
+- [ ] PV7.006 the worklog and the ladder ticked, the wave committed

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@wenathlan/debonair",
-  "version": "2.0.118",
+  "version": "2.0.127",
   "repository": {
     "type": "git",
     "url": "https://github.com/wenathlan/devthink"
@@ -75,6 +75,10 @@
     "wouter": "^3.13.0",
     "z-ai-web-dev-sdk": "^0.0.18"
   },
+  "overrides": {
+    "esbuild": "^0.28.2",
+    "uuid": "^11.1.1"
+  },
   "devDependencies": {
     "@biomejs/biome": "^2.5.15",
     "@capacitor/android": "^8.5.2",

package/tests/bigintids.test.ts

@@ -0,0 +1,32 @@
+// # bigintids.test — the pointer mint the storage law names: ids sort by
+// creation (the high bits carry the clock), never collide within a tick
+// (the tail increments), self-describe their mint time and ride the strong
+// platform randomness. runnable with vitest: pnpm vitest run tests/bigintids.test.ts
+import assert from "node:assert/strict";
+import { describe, it } from "vitest";
+import { idtick, mintid } from "../bigintids.ts";
+
+describe("bigintids", () => {
+  it("mints ascending ids across ticks", () => {
+    const base = 1_700_000_000_000;
+    let previous = 0n;
+    for (let i = 0; i < 200; i++) {
+      const id = BigInt(mintid(base + i));
+      assert.ok(id > previous, `id ${id} must exceed ${previous}`);
+      previous = id;
+    }
+  });
+
+  it("never repeats inside one millisecond (the tail walks)", () => {
+    const base = 1_700_000_000_000;
+    const seen = new Set<string>();
+    for (let i = 0; i < 5000; i++) seen.add(mintid(base));
+    assert.equal(seen.size, 5000);
+  });
+
+  it("self-describes the mint tick", () => {
+    const at = 1_700_000_012_345;
+    const id = mintid(at);
+    assert.equal(idtick(id), at);
+  });
+});

package/tests/ingest.test.ts

@@ -0,0 +1,127 @@
+// # ingest.test — the decomposition pipeline contract: the hpss separation
+//   conserves the mix (harmonic + percussive == the original), the ingest
+//   document is deterministic for a given pcm and carries the vector
+//   segments the pattern miner reads (one frame per second, chroma, bands,
+//   onsets), the pcm hash is stable and the store guard refuses foreign
+//   schemas. runnable with vitest: pnpm vitest run tests/ingest.test.ts
+import assert from "node:assert/strict";
+import { describe, it } from "vitest";
+import { separate } from "../hpss.ts";
+import type { IngestDocument } from "../ingest.ts";
+import { INGEST_SCHEMA_VERSION, ingestPcm, onsetSecondsOf, pcmhash, segmentsof } from "../ingest.ts";
+
+/** A deterministic synthetic take: a 110 hz sawtooth bass (harmonic) over
+ * an 8 hz amplitude-gated click train (percussive), 3 seconds at 8 khz —
+ * small enough to analyze fast, structured enough to exercise both stems. */
+function synthetictake(): { mono: Float32Array; sampleRate: number } {
+  const sampleRate = 8000;
+  const length = sampleRate * 3;
+  const mono = new Float32Array(length);
+  for (let i = 0; i < length; i++) {
+    const t = i / sampleRate;
+    const bass = 0.4 * Math.sign(Math.sin(2 * Math.PI * 110 * t));
+    const gate = t % 0.5 < 0.02 ? 0.8 : 0;
+    mono[i] = Math.max(-1, Math.min(1, bass * 0.7 + gate));
+  }
+  return { mono, sampleRate };
+}
+
+describe("hpss separation", () => {
+  it("conserves the mix: harmonic + percussive equals the original", () => {
+    const { mono, sampleRate } = synthetictake();
+    const { harmonic, percussive } = separate(mono);
+    assert.equal(harmonic.length, mono.length);
+    let worst = 0;
+    // the centered framing keeps every input sample near a window peak, so
+    // the whole signal conserves to the f32 storage floor
+    for (let i = 0; i < mono.length; i++) {
+      worst = Math.max(worst, Math.abs(harmonic[i] + percussive[i] - mono[i]));
+    }
+    assert.ok(worst < 1e-3, `reconstruction error ${worst}`);
+    void sampleRate;
+  });
+
+  it("pushes the click train into the percussive stem and the sawtooth into the harmonic one", () => {
+    const { mono } = synthetictake();
+    const { harmonic, percussive } = separate(mono);
+    const energy = (pcm: Float32Array): number => {
+      let sum = 0;
+      for (let i = 0; i < pcm.length; i++) sum += pcm[i] * pcm[i];
+      return sum;
+    };
+    assert.ok(energy(harmonic) > energy(percussive), "the tonal sawtooth carries most of the energy");
+  });
+});
+
+describe("ingest document", () => {
+  const { mono, sampleRate } = synthetictake();
+  const doc = ingestPcm(mono, sampleRate, { name: "synthetic take", at: "2026-01-01T00:00:00.000Z" });
+
+  it("writes the one schema with the mix, two stems and per-second frames", () => {
+    assert.equal(doc.schema, INGEST_SCHEMA_VERSION);
+    assert.equal(doc.name, "synthetic take");
+    assert.equal(doc.stems.length, 2);
+    assert.equal(doc.stems[0].role, "harmonic");
+    assert.equal(doc.stems[1].role, "percussive");
+    assert.equal(doc.segments.length, 3);
+    for (const stem of doc.stems) assert.equal(stem.segments.length, 3);
+  });
+
+  it("is deterministic: the same pcm ingests to the same bytes", () => {
+    const again = ingestPcm(mono, sampleRate, { name: "synthetic take", at: "2026-01-01T00:00:00.000Z" });
+    assert.equal(JSON.stringify(again), JSON.stringify(doc));
+  });
+
+  it("carries the vector fields every frame needs", () => {
+    for (const frame of doc.segments) {
+      assert.equal(frame.s >= 0, true);
+      assert.equal(frame.bands.length, 3);
+      assert.equal(frame.chroma.length, 12);
+      const total = frame.bands.reduce((sum, value) => sum + value, 0);
+      assert.ok(Math.abs(total - 1) < 0.01, `bands sum ${total}`);
+      assert.ok(frame.rms >= 0 && frame.peak >= 0 && frame.zcr >= 0);
+      assert.ok(["", ...["c", "c#", "d", "d#", "e", "f", "f#", "g", "g#", "a", "a#", "b"]].includes(frame.note));
+    }
+  });
+
+  it("hashes the pcm deterministically and differently per stem", () => {
+    assert.equal(pcmhash(mono), doc.pcmhash);
+    assert.notEqual(doc.stems[0].pcmhash, doc.stems[1].pcmhash);
+    const scratch = new Float32Array(mono);
+    scratch[10] += 0.5;
+    assert.notEqual(pcmhash(scratch), doc.pcmhash);
+  });
+
+  it("detects the gated clicks as onsets of the percussive stem", () => {
+    const onsets = onsetSecondsOf(mono, sampleRate);
+    assert.ok(onsets.length >= 4, `onsets ${onsets.length}`);
+    for (let i = 1; i < onsets.length; i++) assert.ok(onsets[i] > onsets[i - 1], "ascending");
+  });
+
+  it("segments count the onsets that fall inside each second", () => {
+    const onsets = onsetSecondsOf(mono, sampleRate);
+    const frames = segmentsof(mono, sampleRate, onsets);
+    let counted = 0;
+    for (const frame of frames) counted += frame.onsets;
+    assert.equal(counted, onsets.length);
+  });
+});
+
+describe("the store guard", () => {
+  it("accepts only schema-matching documents (the browser guard shape)", () => {
+    const { mono, sampleRate } = synthetictake();
+    const doc = ingestPcm(mono, sampleRate, { name: "guard take", at: "2026-01-01T00:00:00.000Z" });
+    const broken = { schema: "other/1" } as unknown as IngestDocument;
+    // the guard lives in the store module: the shape check mirrors it here
+    const guard = (candidate: IngestDocument): boolean =>
+      !!candidate &&
+      candidate.schema === INGEST_SCHEMA_VERSION &&
+      typeof candidate.name === "string" &&
+      candidate.name.length > 0 &&
+      Array.isArray(candidate.stems) &&
+      candidate.stems.length > 0 &&
+      Array.isArray(candidate.segments);
+    assert.equal(guard(broken), false);
+    assert.equal(guard(doc), true);
+  });
+});

package/tests/tuning.test.ts

@@ -3,7 +3,7 @@
 // octave doubles. The recipes ride this one function; the tests pin it.
 import assert from "node:assert/strict";
 import { describe, it } from "vitest";
-import { A4_MIDI, CONCERT_PITCH, midiToHz } from "../Sol/studio/tuning.ts";
+import { A4_MIDI, CONCERT_PITCH, midiToHz } from "../tuning.ts";
 
 describe("tuning — midiToHz", () => {
   it("anchors A4 at the concert pitch", () => {