@kairotic/analysis 0.7.2 → 0.7.5

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -41,7 +41,8 @@ To watch what a sound does over time, several descriptors side by side:
41
41
 
42
42
  An improvisation into notation: `adc~` → `pitch~` → `transcribe` → `freqt`, with `onset~` on the right
43
43
  of `transcribe` (its second outlet, the strength: each note is then written with the weight of its attack); `rec`,
44
- play, `stop`; then `(track! :bN)` to find its beats and bars, `(view :bN :score)` on the freqt, or `(keep! …)` and
44
+ play, `stop`; then a `quantize` box (the freqt's second outlet on its right inlet; `:time :track` to find its beats, a bang)
45
+ writes it in measures as a tree for a `tree` box — or `(track! :bN)` and `(view :bN :score)` on the freqt to preview it, or `(keep! …)` and
45
46
  `mt`.
46
47
 
47
48
  The form of what is played, and onsets that adapt to it: `form~` on a `chroma~`, its fourth outlet into
package/dist/chroma.mjs CHANGED
@@ -1,7 +1,7 @@
1
1
  // Generated by packages/build.mjs from src/chroma.cljs
2
2
  export default {
3
3
  "class": "chroma~",
4
- "version": "0.7.2",
4
+ "version": "0.7.5",
5
5
  "author": "bonuoq",
6
6
  "license": "MPL-2.0",
7
7
  "doc": "How much of each pitch class the signal on its left has, every ms (40 by default): a list of 12 values from C to B (0…1, the strongest 1) on the left, and the strongest class as a word (:c, :c#, … :b) on the right when it changes. Silence sends nothing. Chords, keys and drones become data for mt and the patch. Computed with Meyda.",
package/dist/features.mjs CHANGED
@@ -1,7 +1,7 @@
1
1
  // Generated by packages/build.mjs from src/features.cljs
2
2
  export default {
3
3
  "class": "features~",
4
- "version": "0.7.2",
4
+ "version": "0.7.5",
5
5
  "author": "bonuoq",
6
6
  "license": "MPL-2.0",
7
7
  "doc": "Descriptors of the signal on its left, every ms (20 by default), one outlet for each word written after it, in order: rms, energy, zcr, loudness, centroid and spread (in Hz), flatness, rolloff (Hz), kurtosis, skewness, slope, crest, sharpness, perceptualSpread, mfcc (13 values) and chroma (12). Without words: rms centroid flatness. Each outlet sends only what changed. Computed with Meyda.",
package/dist/form.mjs CHANGED
@@ -1,9 +1,9 @@
1
1
  // Generated by packages/build.mjs from src/form.cljs
2
2
  export default {
3
3
  "class": "form~",
4
- "version": "0.7.2",
4
+ "version": "0.7.5",
5
5
  "author": "bonuoq",
6
6
  "license": "MPL-2.0",
7
- "doc": "How alike the moments of what arrives are, over the last seconds (20): a self-similarity matrix, drawn, of one frame every step seconds (0.25) — the mean of the lists that arrived in it (a chroma~, a features~ mfcc), or, for a stream of numbers (onset~'s third outlet, an rms), the shape of its values. On the left the novelty of each frame (0…1: how much the music changes there, by a checkerboard of kernel seconds (2) on each side, so it comes kernel seconds late); a bang on the second outlet where a segment begins (a novelty peak high for the window, two kernels after the last at least), and, on the fourth, [:segment {:t :at :length :period}] for the segment it ends; on the third, the period that repeats most in the window, in seconds (the bar, the phrase), when it changes. [:seconds x], [:step x] and [:kernel x] change them, clear forgets. Into onset~ (a [:segment …] makes it forget the usual change of the section before: adaptive onsets), a message box with [:threshold x], or a print. Self-similarity after Foote (1999, 2000, 2001); the same functions for a whole take or a tree are api/ssm, api/novelty, api/boundaries and api/note-frames.",
8
- "source": ";; form~ · how alike the moments are: a self-similarity matrix over the last seconds, its changes, its periods\n;; Part of @kairotic/analysis. Written against the Kairotic API (api/…): the matrix, novelty and beat spectrum\n;; are those of the core (kairotic.structure, issue #120), through api/ssm, api/novelty-at, api/beat-spectrum.\n;; What arrives (a chroma, descriptors, an onset envelope) is gathered into one frame every step seconds; each\n;; new frame is compared with all those of the window. A segment begins where the frames before are alike\n;; among themselves and unlike the ones after (Foote 2000), and the lag at which frames are most alike is the\n;; period that repeats (Foote & Uchihashi 2001).\n(let [G 20 ; the matrix drawn: G × G cells at most (the patch draws at most 600 shapes)\n bins 8 ; a stream of numbers: each frame is the shape of its step, in 8 slices\n settings (fn [args state]\n (let [o (:set state)\n seconds (api/clamp (or (:seconds o) (:seconds args) 20) 2 120)\n step (api/clamp (or (:step o) (:step args) 0.25) 0.02 4)\n kernel (api/clamp (or (:kernel o) (:kernel args) 2) (* 2 step) (/ seconds 3))]\n {:seconds seconds :step step :kernel kernel\n :L (max 1 (js/Math.round (/ kernel step)))\n :N (max 4 (js/Math.round (/ seconds step)))}))\n frame-of (fn [xs]\n ;; vectors: their mean; numbers: the values of the step, resampled into bins slices\n (if (sequential? (first xs))\n (let [d (reduce max 0 (map count xs))]\n (mapv (fn [j] (/ (reduce + (map (fn [v] (or (get v j) 0)) xs)) (count xs))) (range d)))\n (let [v (vec xs) n (count v)]\n (mapv (fn [j] (let [a (js/Math.floor (/ (* j n) bins)) b (max (inc a) (js/Math.floor (/ (* (inc j) n) bins)))\n s (subvec v (min a (dec n)) (min b n))]\n (/ (reduce + s) (max 1 (count s)))))\n (range bins)))))\n usable (fn [v] (cond (number? v) v\n (and (sequential? v) (seq v) (every? number? v)) (vec v)\n :else nil))\n ;; the matrix drawn: the mean of each block of frames\n cells (fn [S]\n (let [n (:n S) g (min G n) blk (fn [k] [(js/Math.floor (/ (* k n) g)) (js/Math.floor (/ (* (inc k) n) g))])]\n (vec (for [r (range g)]\n (let [[a b] (blk r)]\n (vec (for [c (range g)]\n (let [[x y] (blk c) cnt (* (- b a) (- y x))]\n (/ (reduce + (for [i (range a b) j (range x y)] (api/ssm-at S i j))) (max 1 cnt))))))))))\n mean-sd (fn [xs] (let [n (max 1 (count xs)) m (/ (reduce + xs) n)]\n [m (js/Math.sqrt (/ (reduce + (map (fn [x] (* (- x m) (- x m))) xs)) n))]))\n r3 (fn [x] (/ (js/Math.round (* 1000 x)) 1000))\n ;; a frame is complete: into the window, and what it tells\n close (fn [state {:keys [N L step kernel]}]\n (let [f (frame-of (:acc state))\n t (+ (:t0 state) (/ step 2))\n frames (vec (take-last N (conj (or (:frames state) []) [t f])))\n S (api/ssm (mapv second frames))\n n (:n S)\n i (- n 1 L)\n nov (when (>= i L) (api/novelty-at S L i))\n nt (when nov (first (frames i)))\n hist (if nov (vec (take-last N (conj (or (:nov state) []) [nt (max 0 nov)]))) (or (:nov state) []))\n ;; a boundary: the novelty of the frame before is a peak, high for this window, and the last\n ;; boundary is at least two kernels away\n vs (mapv second hist) k (count vs)\n [m sd] (mean-sd vs)\n peak (when (>= k 3)\n (let [[tp p] (hist (- k 2))]\n (when (and (> p (vs (- k 3))) (>= p (vs (dec k))) (> p (max 0.05 (+ m sd)))\n (or (nil? (:last state)) (>= (- tp (:last state)) (* 2 kernel))))\n tp)))\n bs (when (>= n 8) (api/beat-spectrum S (quot n 2)))\n ;; a period is a lag more alike than the lags before it (a real dip between): music that does\n ;; not change is alike at every lag, and has none\n lag (some (fn [[l x]] (when (and (>= l 2) (> (- x (reduce min (subvec bs 1 l))) 0.1)) l))\n (when bs (api/periods bs)))\n period (when lag (r3 (* lag step)))\n s (cond-> (assoc state :frames frames :nov hist\n :cells (cells S) :period (or period (:period state)))\n peak (assoc :last peak :marks (vec (take-last 8 (conj (or (:marks state) []) peak)))))]\n {:state s\n :out (cond-> []\n nov (conj [0 (r3 (max 0 nov))])\n (and period (not= period (:period state))) (conj [2 period])\n peak (conj [3 [:segment {:t (or (:last state) (first (first frames))) :at peak\n :length (r3 (- peak (or (:last state) (first (first frames)))))\n :period (or period (:period state))}]]\n [1 :bang]))}))]\n {;; ── about ──\n :class \"form~\"\n :license \"MPL-2.0\"\n :doc \"How alike the moments of what arrives are, over the last seconds (20): a self-similarity matrix, drawn, of one frame every step seconds (0.25) — the mean of the lists that arrived in it (a chroma~, a features~ mfcc), or, for a stream of numbers (onset~'s third outlet, an rms), the shape of its values. On the left the novelty of each frame (0…1: how much the music changes there, by a checkerboard of kernel seconds (2) on each side, so it comes kernel seconds late); a bang on the second outlet where a segment begins (a novelty peak high for the window, two kernels after the last at least), and, on the fourth, [:segment {:t :at :length :period}] for the segment it ends; on the third, the period that repeats most in the window, in seconds (the bar, the phrase), when it changes. [:seconds x], [:step x] and [:kernel x] change them, clear forgets. Into onset~ (a [:segment …] makes it forget the usual change of the section before: adaptive onsets), a message box with [:threshold x], or a print. Self-similarity after Foote (1999, 2000, 2001); the same functions for a whole take or a tree are api/ssm, api/novelty, api/boundaries and api/note-frames.\"\n :usage \"form~ [seconds] [step] [kernel]\"\n :io [1 4]\n :size [220 128]\n :args [[:seconds 20] [:step 0.25] [:kernel 2]]\n\n ;; ── logic ──\n ;; the state: {:frames [[t vector] …] :acc [values of the frame under way] :t0 its start :nov [[t value] …]\n ;; :last the last boundary :marks boundaries :cells the matrix drawn :period :set what messages changed}\n :receive (fn [{:keys [value state args at]}]\n (cond\n (= value :clear) {:state (select-keys state [:set]) :show nil}\n (and (vector? value) (= 2 (count value)) (#{:seconds :step :kernel} (first value)) (number? (second value)))\n (let [s (-> (select-keys state [:set]) (assoc-in [:set (first value)] (second value)))] {:state s :show nil})\n (and (usable value) (number? at))\n (let [o (settings args state)\n ;; the engine started again (its clock went back), or the kind of what arrives changed: from nothing\n state (if (or (and (:t0 state) (< at (- (:t0 state) 1)))\n (and (seq (:acc state)) (not= (number? (first (:acc state))) (number? (usable value)))))\n (select-keys state [:set]) state)\n v (usable value)]\n ;; a value a step after the frame under way began closes it, and begins the next\n (if (and (:t0 state) (seq (:acc state)) (>= (- at (:t0 state)) (:step o)))\n (let [{s :state out :out} (close state o)\n s (assoc s :acc [v] :t0 at)]\n (cond-> {:state s :show (select-keys s [:cells :nov :marks :period :frames])}\n (seq out) (assoc :out out)))\n {:state (cond-> (update state :acc (fnil conj []) v) (nil? (:t0 state)) (assoc :t0 at))}))\n :else {}))\n\n ;; ── view: the matrix (time runs down and to the right), the novelty under it, the period ──\n :draw (fn [{:keys [shown args state w h]}]\n (let [{:keys [step kernel seconds]} (settings args state)\n cs (:cells shown) g (count cs)\n side (- h 12) x0 6 y0 6 cw (if (pos? g) (/ side g) side)\n frames (:frames shown)\n t0 (first (first frames)) t1 (first (peek frames))\n px0 (+ x0 side 10) px1 (- w 6) pw (- px1 px0)\n tx (fn [t] (if (and t0 t1 (> t1 t0)) (+ px0 (* pw (/ (- t t0) (- t1 t0)))) px0))\n ny0 (+ y0 34) ny1 (- h 8)\n nov (filter (fn [[t]] (and t0 (>= t t0))) (:nov shown))\n ty (fn [v] (- ny1 (* (- ny1 ny0) (api/clamp v 0 1))))]\n (into\n [[:rect {:x 2 :y 2 :w (- w 4) :h (- h 4) :r 4 :stroke :grid :fill :bg}]\n [:rect {:x x0 :y y0 :w side :h side :stroke :grid :fill :bg}]\n [:text {:x px0 :y (+ y0 10) :size 10 :fill :ink} \"form\"]\n [:text {:x px0 :y (+ y0 23) :size 8 :fill :soft}\n (if-let [p (:period shown)] (str \"period \" (api/nice p) \" s\") (str (api/nice seconds) \" s · \" (api/nice step) \" · \" (api/nice kernel)))]\n [:line {:x1 px0 :y1 ny1 :x2 px1 :y2 ny1 :stroke :grid :weight 1}]]\n (concat\n (for [r (range g) c (range g) :let [v ((cs r) c)] :when (> v 0.1)]\n [:rect {:x (+ x0 (* c cw)) :y (+ y0 (* r cw)) :w (+ cw 0.3) :h (+ cw 0.3)\n :fill :accent :stroke nil :opacity (api/clamp (* v v) 0 1)}])\n (let [xys (mapv (fn [[t v]] [(api/clamp (tx t) px0 px1) (ty v)]) nov)]\n (when (> (count xys) 1) [[:path {:points xys :stroke :accent :weight 1.2}]]))\n (for [t (:marks shown) :when (and t0 (>= t t0))]\n [:line {:x1 (tx t) :y1 ny0 :x2 (tx t) :y2 ny1 :stroke :kairos :weight 1.2}])))))})\n"
7
+ "doc": "How alike the moments of what arrives are, over the last seconds (20): a self-similarity matrix, drawn, of one frame every step seconds (0.25) — the mean of the lists that arrived in it (a chroma~, a features~ mfcc), or, for a stream of numbers (onset~'s third outlet, an rms), the shape of its values. On the left the novelty of each frame (0…1: how much the music changes there, by a checkerboard of kernel seconds (2) on each side, so it comes kernel seconds late); a bang on the second outlet where a segment begins (a novelty peak high for the window, two kernels after the last at least), and, on the fourth, [:segment {:t :at :length :period}] for the segment it ends; on the third, the period that repeats most in the window, in seconds (the bar, the phrase), when it changes. [:seconds x], [:step x] and [:kernel x] change them, :clear forgets. Into onset~ (a [:segment …] makes it forget the usual change of the section before: adaptive onsets), a message box with [:threshold x], or a print. Self-similarity after Foote (1999, 2000, 2001); the same functions for a whole take or a tree are api/ssm, api/novelty, api/boundaries and api/note-frames.",
8
+ "source": ";; form~ · how alike the moments are: a self-similarity matrix over the last seconds, its changes, its periods\n;; Part of @kairotic/analysis. Written against the Kairotic API (api/…): the matrix, novelty and beat spectrum\n;; are those of the core (kairotic.structure, issue #120), through api/ssm, api/novelty-at, api/beat-spectrum.\n;; What arrives (a chroma, descriptors, an onset envelope) is gathered into one frame every step seconds; each\n;; new frame is compared with all those of the window. A segment begins where the frames before are alike\n;; among themselves and unlike the ones after (Foote 2000), and the lag at which frames are most alike is the\n;; period that repeats (Foote & Uchihashi 2001).\n(let [G 20 ; the matrix drawn: G × G cells at most (the patch draws at most 600 shapes)\n bins 8 ; a stream of numbers: each frame is the shape of its step, in 8 slices\n settings (fn [args state]\n (let [o (:set state)\n seconds (api/clamp (or (:seconds o) (:seconds args) 20) 2 120)\n step (api/clamp (or (:step o) (:step args) 0.25) 0.02 4)\n kernel (api/clamp (or (:kernel o) (:kernel args) 2) (* 2 step) (/ seconds 3))]\n {:seconds seconds :step step :kernel kernel\n :L (max 1 (js/Math.round (/ kernel step)))\n :N (max 4 (js/Math.round (/ seconds step)))}))\n frame-of (fn [xs]\n ;; vectors: their mean; numbers: the values of the step, resampled into bins slices\n (if (sequential? (first xs))\n (let [d (reduce max 0 (map count xs))]\n (mapv (fn [j] (/ (reduce + (map (fn [v] (or (get v j) 0)) xs)) (count xs))) (range d)))\n (let [v (vec xs) n (count v)]\n (mapv (fn [j] (let [a (js/Math.floor (/ (* j n) bins)) b (max (inc a) (js/Math.floor (/ (* (inc j) n) bins)))\n s (subvec v (min a (dec n)) (min b n))]\n (/ (reduce + s) (max 1 (count s)))))\n (range bins)))))\n usable (fn [v] (cond (number? v) v\n (and (sequential? v) (seq v) (every? number? v)) (vec v)\n :else nil))\n ;; the matrix drawn: the mean of each block of frames\n cells (fn [S]\n (let [n (:n S) g (min G n) blk (fn [k] [(js/Math.floor (/ (* k n) g)) (js/Math.floor (/ (* (inc k) n) g))])]\n (vec (for [r (range g)]\n (let [[a b] (blk r)]\n (vec (for [c (range g)]\n (let [[x y] (blk c) cnt (* (- b a) (- y x))]\n (/ (reduce + (for [i (range a b) j (range x y)] (api/ssm-at S i j))) (max 1 cnt))))))))))\n mean-sd (fn [xs] (let [n (max 1 (count xs)) m (/ (reduce + xs) n)]\n [m (js/Math.sqrt (/ (reduce + (map (fn [x] (* (- x m) (- x m))) xs)) n))]))\n r3 (fn [x] (/ (js/Math.round (* 1000 x)) 1000))\n ;; a frame is complete: into the window, and what it tells\n close (fn [state {:keys [N L step kernel]}]\n (let [f (frame-of (:acc state))\n t (+ (:t0 state) (/ step 2))\n frames (vec (take-last N (conj (or (:frames state) []) [t f])))\n S (api/ssm (mapv second frames))\n n (:n S)\n i (- n 1 L)\n nov (when (>= i L) (api/novelty-at S L i))\n nt (when nov (first (frames i)))\n hist (if nov (vec (take-last N (conj (or (:nov state) []) [nt (max 0 nov)]))) (or (:nov state) []))\n ;; a boundary: the novelty of the frame before is a peak, high for this window, and the last\n ;; boundary is at least two kernels away\n vs (mapv second hist) k (count vs)\n [m sd] (mean-sd vs)\n peak (when (>= k 3)\n (let [[tp p] (hist (- k 2))]\n (when (and (> p (vs (- k 3))) (>= p (vs (dec k))) (> p (max 0.05 (+ m sd)))\n (or (nil? (:last state)) (>= (- tp (:last state)) (* 2 kernel))))\n tp)))\n bs (when (>= n 8) (api/beat-spectrum S (quot n 2)))\n ;; a period is a lag more alike than the lags before it (a real dip between): music that does\n ;; not change is alike at every lag, and has none\n lag (some (fn [[l x]] (when (and (>= l 2) (> (- x (reduce min (subvec bs 1 l))) 0.1)) l))\n (when bs (api/periods bs)))\n period (when lag (r3 (* lag step)))\n s (cond-> (assoc state :frames frames :nov hist\n :cells (cells S) :period (or period (:period state)))\n peak (assoc :last peak :marks (vec (take-last 8 (conj (or (:marks state) []) peak)))))]\n {:state s\n :out (cond-> []\n nov (conj [0 (r3 (max 0 nov))])\n (and period (not= period (:period state))) (conj [2 period])\n peak (conj [3 [:segment {:t (or (:last state) (first (first frames))) :at peak\n :length (r3 (- peak (or (:last state) (first (first frames)))))\n :period (or period (:period state))}]]\n [1 :bang]))}))]\n {;; ── about ──\n :class \"form~\"\n :license \"MPL-2.0\"\n :doc \"How alike the moments of what arrives are, over the last seconds (20): a self-similarity matrix, drawn, of one frame every step seconds (0.25) — the mean of the lists that arrived in it (a chroma~, a features~ mfcc), or, for a stream of numbers (onset~'s third outlet, an rms), the shape of its values. On the left the novelty of each frame (0…1: how much the music changes there, by a checkerboard of kernel seconds (2) on each side, so it comes kernel seconds late); a bang on the second outlet where a segment begins (a novelty peak high for the window, two kernels after the last at least), and, on the fourth, [:segment {:t :at :length :period}] for the segment it ends; on the third, the period that repeats most in the window, in seconds (the bar, the phrase), when it changes. [:seconds x], [:step x] and [:kernel x] change them, :clear forgets. Into onset~ (a [:segment …] makes it forget the usual change of the section before: adaptive onsets), a message box with [:threshold x], or a print. Self-similarity after Foote (1999, 2000, 2001); the same functions for a whole take or a tree are api/ssm, api/novelty, api/boundaries and api/note-frames.\"\n :usage \"form~ [seconds] [step] [kernel]\"\n :io [1 4]\n :size [220 128]\n :args [[:seconds 20] [:step 0.25] [:kernel 2]]\n\n ;; ── logic ──\n ;; the state: {:frames [[t vector] …] :acc [values of the frame under way] :t0 its start :nov [[t value] …]\n ;; :last the last boundary :marks boundaries :cells the matrix drawn :period :set what messages changed}\n :receive (fn [{:keys [value state args at]}]\n (cond\n (= value :clear) {:state (select-keys state [:set]) :show nil}\n (and (vector? value) (= 2 (count value)) (#{:seconds :step :kernel} (first value)) (number? (second value)))\n (let [s (-> (select-keys state [:set]) (assoc-in [:set (first value)] (second value)))] {:state s :show nil})\n (and (usable value) (number? at))\n (let [o (settings args state)\n ;; the engine started again (its clock went back), or the kind of what arrives changed: from nothing\n state (if (or (and (:t0 state) (< at (- (:t0 state) 1)))\n (and (seq (:acc state)) (not= (number? (first (:acc state))) (number? (usable value)))))\n (select-keys state [:set]) state)\n v (usable value)]\n ;; a value a step after the frame under way began closes it, and begins the next\n (if (and (:t0 state) (seq (:acc state)) (>= (- at (:t0 state)) (:step o)))\n (let [{s :state out :out} (close state o)\n s (assoc s :acc [v] :t0 at)]\n (cond-> {:state s :show (select-keys s [:cells :nov :marks :period :frames])}\n (seq out) (assoc :out out)))\n {:state (cond-> (update state :acc (fnil conj []) v) (nil? (:t0 state)) (assoc :t0 at))}))\n :else {}))\n\n ;; ── view: the matrix (time runs down and to the right), the novelty under it, the period ──\n :draw (fn [{:keys [shown args state w h]}]\n (let [{:keys [step kernel seconds]} (settings args state)\n cs (:cells shown) g (count cs)\n side (- h 12) x0 6 y0 6 cw (if (pos? g) (/ side g) side)\n frames (:frames shown)\n t0 (first (first frames)) t1 (first (peek frames))\n px0 (+ x0 side 10) px1 (- w 6) pw (- px1 px0)\n tx (fn [t] (if (and t0 t1 (> t1 t0)) (+ px0 (* pw (/ (- t t0) (- t1 t0)))) px0))\n ny0 (+ y0 34) ny1 (- h 8)\n nov (filter (fn [[t]] (and t0 (>= t t0))) (:nov shown))\n ty (fn [v] (- ny1 (* (- ny1 ny0) (api/clamp v 0 1))))]\n (into\n [[:rect {:x 2 :y 2 :w (- w 4) :h (- h 4) :r 4 :stroke :grid :fill :bg}]\n [:rect {:x x0 :y y0 :w side :h side :stroke :grid :fill :bg}]\n [:text {:x px0 :y (+ y0 10) :size 10 :fill :ink} \"form\"]\n [:text {:x px0 :y (+ y0 23) :size 8 :fill :soft}\n (if-let [p (:period shown)] (str \"period \" (api/nice p) \" s\") (str (api/nice seconds) \" s · \" (api/nice step) \" · \" (api/nice kernel)))]\n [:line {:x1 px0 :y1 ny1 :x2 px1 :y2 ny1 :stroke :grid :weight 1}]]\n (concat\n (for [r (range g) c (range g) :let [v ((cs r) c)] :when (> v 0.1)]\n [:rect {:x (+ x0 (* c cw)) :y (+ y0 (* r cw)) :w (+ cw 0.3) :h (+ cw 0.3)\n :fill :accent :stroke nil :opacity (api/clamp (* v v) 0 1)}])\n (let [xys (mapv (fn [[t v]] [(api/clamp (tx t) px0 px1) (ty v)]) nov)]\n (when (> (count xys) 1) [[:path {:points xys :stroke :accent :weight 1.2}]]))\n (for [t (:marks shown) :when (and t0 (>= t t0))]\n [:line {:x1 (tx t) :y1 ny0 :x2 (tx t) :y2 ny1 :stroke :kairos :weight 1.2}])))))})\n"
9
9
  };
package/dist/index.mjs CHANGED
@@ -1,7 +1,7 @@
1
1
  // Generated by packages/build.mjs
2
2
  export default {
3
3
  "package": "@kairotic/analysis",
4
- "version": "0.7.2",
4
+ "version": "0.7.5",
5
5
  "api": 1,
6
6
  "author": "bonuoq",
7
7
  "license": "MPL-2.0",
@@ -21,8 +21,8 @@ export default {
21
21
  {
22
22
  "class": "form~",
23
23
  "license": "MPL-2.0",
24
- "doc": "How alike the moments of what arrives are, over the last seconds (20): a self-similarity matrix, drawn, of one frame every step seconds (0.25) — the mean of the lists that arrived in it (a chroma~, a features~ mfcc), or, for a stream of numbers (onset~'s third outlet, an rms), the shape of its values. On the left the novelty of each frame (0…1: how much the music changes there, by a checkerboard of kernel seconds (2) on each side, so it comes kernel seconds late); a bang on the second outlet where a segment begins (a novelty peak high for the window, two kernels after the last at least), and, on the fourth, [:segment {:t :at :length :period}] for the segment it ends; on the third, the period that repeats most in the window, in seconds (the bar, the phrase), when it changes. [:seconds x], [:step x] and [:kernel x] change them, clear forgets. Into onset~ (a [:segment …] makes it forget the usual change of the section before: adaptive onsets), a message box with [:threshold x], or a print. Self-similarity after Foote (1999, 2000, 2001); the same functions for a whole take or a tree are api/ssm, api/novelty, api/boundaries and api/note-frames.",
25
- "source": ";; form~ · how alike the moments are: a self-similarity matrix over the last seconds, its changes, its periods\n;; Part of @kairotic/analysis. Written against the Kairotic API (api/…): the matrix, novelty and beat spectrum\n;; are those of the core (kairotic.structure, issue #120), through api/ssm, api/novelty-at, api/beat-spectrum.\n;; What arrives (a chroma, descriptors, an onset envelope) is gathered into one frame every step seconds; each\n;; new frame is compared with all those of the window. A segment begins where the frames before are alike\n;; among themselves and unlike the ones after (Foote 2000), and the lag at which frames are most alike is the\n;; period that repeats (Foote & Uchihashi 2001).\n(let [G 20 ; the matrix drawn: G × G cells at most (the patch draws at most 600 shapes)\n bins 8 ; a stream of numbers: each frame is the shape of its step, in 8 slices\n settings (fn [args state]\n (let [o (:set state)\n seconds (api/clamp (or (:seconds o) (:seconds args) 20) 2 120)\n step (api/clamp (or (:step o) (:step args) 0.25) 0.02 4)\n kernel (api/clamp (or (:kernel o) (:kernel args) 2) (* 2 step) (/ seconds 3))]\n {:seconds seconds :step step :kernel kernel\n :L (max 1 (js/Math.round (/ kernel step)))\n :N (max 4 (js/Math.round (/ seconds step)))}))\n frame-of (fn [xs]\n ;; vectors: their mean; numbers: the values of the step, resampled into bins slices\n (if (sequential? (first xs))\n (let [d (reduce max 0 (map count xs))]\n (mapv (fn [j] (/ (reduce + (map (fn [v] (or (get v j) 0)) xs)) (count xs))) (range d)))\n (let [v (vec xs) n (count v)]\n (mapv (fn [j] (let [a (js/Math.floor (/ (* j n) bins)) b (max (inc a) (js/Math.floor (/ (* (inc j) n) bins)))\n s (subvec v (min a (dec n)) (min b n))]\n (/ (reduce + s) (max 1 (count s)))))\n (range bins)))))\n usable (fn [v] (cond (number? v) v\n (and (sequential? v) (seq v) (every? number? v)) (vec v)\n :else nil))\n ;; the matrix drawn: the mean of each block of frames\n cells (fn [S]\n (let [n (:n S) g (min G n) blk (fn [k] [(js/Math.floor (/ (* k n) g)) (js/Math.floor (/ (* (inc k) n) g))])]\n (vec (for [r (range g)]\n (let [[a b] (blk r)]\n (vec (for [c (range g)]\n (let [[x y] (blk c) cnt (* (- b a) (- y x))]\n (/ (reduce + (for [i (range a b) j (range x y)] (api/ssm-at S i j))) (max 1 cnt))))))))))\n mean-sd (fn [xs] (let [n (max 1 (count xs)) m (/ (reduce + xs) n)]\n [m (js/Math.sqrt (/ (reduce + (map (fn [x] (* (- x m) (- x m))) xs)) n))]))\n r3 (fn [x] (/ (js/Math.round (* 1000 x)) 1000))\n ;; a frame is complete: into the window, and what it tells\n close (fn [state {:keys [N L step kernel]}]\n (let [f (frame-of (:acc state))\n t (+ (:t0 state) (/ step 2))\n frames (vec (take-last N (conj (or (:frames state) []) [t f])))\n S (api/ssm (mapv second frames))\n n (:n S)\n i (- n 1 L)\n nov (when (>= i L) (api/novelty-at S L i))\n nt (when nov (first (frames i)))\n hist (if nov (vec (take-last N (conj (or (:nov state) []) [nt (max 0 nov)]))) (or (:nov state) []))\n ;; a boundary: the novelty of the frame before is a peak, high for this window, and the last\n ;; boundary is at least two kernels away\n vs (mapv second hist) k (count vs)\n [m sd] (mean-sd vs)\n peak (when (>= k 3)\n (let [[tp p] (hist (- k 2))]\n (when (and (> p (vs (- k 3))) (>= p (vs (dec k))) (> p (max 0.05 (+ m sd)))\n (or (nil? (:last state)) (>= (- tp (:last state)) (* 2 kernel))))\n tp)))\n bs (when (>= n 8) (api/beat-spectrum S (quot n 2)))\n ;; a period is a lag more alike than the lags before it (a real dip between): music that does\n ;; not change is alike at every lag, and has none\n lag (some (fn [[l x]] (when (and (>= l 2) (> (- x (reduce min (subvec bs 1 l))) 0.1)) l))\n (when bs (api/periods bs)))\n period (when lag (r3 (* lag step)))\n s (cond-> (assoc state :frames frames :nov hist\n :cells (cells S) :period (or period (:period state)))\n peak (assoc :last peak :marks (vec (take-last 8 (conj (or (:marks state) []) peak)))))]\n {:state s\n :out (cond-> []\n nov (conj [0 (r3 (max 0 nov))])\n (and period (not= period (:period state))) (conj [2 period])\n peak (conj [3 [:segment {:t (or (:last state) (first (first frames))) :at peak\n :length (r3 (- peak (or (:last state) (first (first frames)))))\n :period (or period (:period state))}]]\n [1 :bang]))}))]\n {;; ── about ──\n :class \"form~\"\n :license \"MPL-2.0\"\n :doc \"How alike the moments of what arrives are, over the last seconds (20): a self-similarity matrix, drawn, of one frame every step seconds (0.25) — the mean of the lists that arrived in it (a chroma~, a features~ mfcc), or, for a stream of numbers (onset~'s third outlet, an rms), the shape of its values. On the left the novelty of each frame (0…1: how much the music changes there, by a checkerboard of kernel seconds (2) on each side, so it comes kernel seconds late); a bang on the second outlet where a segment begins (a novelty peak high for the window, two kernels after the last at least), and, on the fourth, [:segment {:t :at :length :period}] for the segment it ends; on the third, the period that repeats most in the window, in seconds (the bar, the phrase), when it changes. [:seconds x], [:step x] and [:kernel x] change them, clear forgets. Into onset~ (a [:segment …] makes it forget the usual change of the section before: adaptive onsets), a message box with [:threshold x], or a print. Self-similarity after Foote (1999, 2000, 2001); the same functions for a whole take or a tree are api/ssm, api/novelty, api/boundaries and api/note-frames.\"\n :usage \"form~ [seconds] [step] [kernel]\"\n :io [1 4]\n :size [220 128]\n :args [[:seconds 20] [:step 0.25] [:kernel 2]]\n\n ;; ── logic ──\n ;; the state: {:frames [[t vector] …] :acc [values of the frame under way] :t0 its start :nov [[t value] …]\n ;; :last the last boundary :marks boundaries :cells the matrix drawn :period :set what messages changed}\n :receive (fn [{:keys [value state args at]}]\n (cond\n (= value :clear) {:state (select-keys state [:set]) :show nil}\n (and (vector? value) (= 2 (count value)) (#{:seconds :step :kernel} (first value)) (number? (second value)))\n (let [s (-> (select-keys state [:set]) (assoc-in [:set (first value)] (second value)))] {:state s :show nil})\n (and (usable value) (number? at))\n (let [o (settings args state)\n ;; the engine started again (its clock went back), or the kind of what arrives changed: from nothing\n state (if (or (and (:t0 state) (< at (- (:t0 state) 1)))\n (and (seq (:acc state)) (not= (number? (first (:acc state))) (number? (usable value)))))\n (select-keys state [:set]) state)\n v (usable value)]\n ;; a value a step after the frame under way began closes it, and begins the next\n (if (and (:t0 state) (seq (:acc state)) (>= (- at (:t0 state)) (:step o)))\n (let [{s :state out :out} (close state o)\n s (assoc s :acc [v] :t0 at)]\n (cond-> {:state s :show (select-keys s [:cells :nov :marks :period :frames])}\n (seq out) (assoc :out out)))\n {:state (cond-> (update state :acc (fnil conj []) v) (nil? (:t0 state)) (assoc :t0 at))}))\n :else {}))\n\n ;; ── view: the matrix (time runs down and to the right), the novelty under it, the period ──\n :draw (fn [{:keys [shown args state w h]}]\n (let [{:keys [step kernel seconds]} (settings args state)\n cs (:cells shown) g (count cs)\n side (- h 12) x0 6 y0 6 cw (if (pos? g) (/ side g) side)\n frames (:frames shown)\n t0 (first (first frames)) t1 (first (peek frames))\n px0 (+ x0 side 10) px1 (- w 6) pw (- px1 px0)\n tx (fn [t] (if (and t0 t1 (> t1 t0)) (+ px0 (* pw (/ (- t t0) (- t1 t0)))) px0))\n ny0 (+ y0 34) ny1 (- h 8)\n nov (filter (fn [[t]] (and t0 (>= t t0))) (:nov shown))\n ty (fn [v] (- ny1 (* (- ny1 ny0) (api/clamp v 0 1))))]\n (into\n [[:rect {:x 2 :y 2 :w (- w 4) :h (- h 4) :r 4 :stroke :grid :fill :bg}]\n [:rect {:x x0 :y y0 :w side :h side :stroke :grid :fill :bg}]\n [:text {:x px0 :y (+ y0 10) :size 10 :fill :ink} \"form\"]\n [:text {:x px0 :y (+ y0 23) :size 8 :fill :soft}\n (if-let [p (:period shown)] (str \"period \" (api/nice p) \" s\") (str (api/nice seconds) \" s · \" (api/nice step) \" · \" (api/nice kernel)))]\n [:line {:x1 px0 :y1 ny1 :x2 px1 :y2 ny1 :stroke :grid :weight 1}]]\n (concat\n (for [r (range g) c (range g) :let [v ((cs r) c)] :when (> v 0.1)]\n [:rect {:x (+ x0 (* c cw)) :y (+ y0 (* r cw)) :w (+ cw 0.3) :h (+ cw 0.3)\n :fill :accent :stroke nil :opacity (api/clamp (* v v) 0 1)}])\n (let [xys (mapv (fn [[t v]] [(api/clamp (tx t) px0 px1) (ty v)]) nov)]\n (when (> (count xys) 1) [[:path {:points xys :stroke :accent :weight 1.2}]]))\n (for [t (:marks shown) :when (and t0 (>= t t0))]\n [:line {:x1 (tx t) :y1 ny0 :x2 (tx t) :y2 ny1 :stroke :kairos :weight 1.2}])))))})\n"
24
+ "doc": "How alike the moments of what arrives are, over the last seconds (20): a self-similarity matrix, drawn, of one frame every step seconds (0.25) — the mean of the lists that arrived in it (a chroma~, a features~ mfcc), or, for a stream of numbers (onset~'s third outlet, an rms), the shape of its values. On the left the novelty of each frame (0…1: how much the music changes there, by a checkerboard of kernel seconds (2) on each side, so it comes kernel seconds late); a bang on the second outlet where a segment begins (a novelty peak high for the window, two kernels after the last at least), and, on the fourth, [:segment {:t :at :length :period}] for the segment it ends; on the third, the period that repeats most in the window, in seconds (the bar, the phrase), when it changes. [:seconds x], [:step x] and [:kernel x] change them, :clear forgets. Into onset~ (a [:segment …] makes it forget the usual change of the section before: adaptive onsets), a message box with [:threshold x], or a print. Self-similarity after Foote (1999, 2000, 2001); the same functions for a whole take or a tree are api/ssm, api/novelty, api/boundaries and api/note-frames.",
25
+ "source": ";; form~ · how alike the moments are: a self-similarity matrix over the last seconds, its changes, its periods\n;; Part of @kairotic/analysis. Written against the Kairotic API (api/…): the matrix, novelty and beat spectrum\n;; are those of the core (kairotic.structure, issue #120), through api/ssm, api/novelty-at, api/beat-spectrum.\n;; What arrives (a chroma, descriptors, an onset envelope) is gathered into one frame every step seconds; each\n;; new frame is compared with all those of the window. A segment begins where the frames before are alike\n;; among themselves and unlike the ones after (Foote 2000), and the lag at which frames are most alike is the\n;; period that repeats (Foote & Uchihashi 2001).\n(let [G 20 ; the matrix drawn: G × G cells at most (the patch draws at most 600 shapes)\n bins 8 ; a stream of numbers: each frame is the shape of its step, in 8 slices\n settings (fn [args state]\n (let [o (:set state)\n seconds (api/clamp (or (:seconds o) (:seconds args) 20) 2 120)\n step (api/clamp (or (:step o) (:step args) 0.25) 0.02 4)\n kernel (api/clamp (or (:kernel o) (:kernel args) 2) (* 2 step) (/ seconds 3))]\n {:seconds seconds :step step :kernel kernel\n :L (max 1 (js/Math.round (/ kernel step)))\n :N (max 4 (js/Math.round (/ seconds step)))}))\n frame-of (fn [xs]\n ;; vectors: their mean; numbers: the values of the step, resampled into bins slices\n (if (sequential? (first xs))\n (let [d (reduce max 0 (map count xs))]\n (mapv (fn [j] (/ (reduce + (map (fn [v] (or (get v j) 0)) xs)) (count xs))) (range d)))\n (let [v (vec xs) n (count v)]\n (mapv (fn [j] (let [a (js/Math.floor (/ (* j n) bins)) b (max (inc a) (js/Math.floor (/ (* (inc j) n) bins)))\n s (subvec v (min a (dec n)) (min b n))]\n (/ (reduce + s) (max 1 (count s)))))\n (range bins)))))\n usable (fn [v] (cond (number? v) v\n (and (sequential? v) (seq v) (every? number? v)) (vec v)\n :else nil))\n ;; the matrix drawn: the mean of each block of frames\n cells (fn [S]\n (let [n (:n S) g (min G n) blk (fn [k] [(js/Math.floor (/ (* k n) g)) (js/Math.floor (/ (* (inc k) n) g))])]\n (vec (for [r (range g)]\n (let [[a b] (blk r)]\n (vec (for [c (range g)]\n (let [[x y] (blk c) cnt (* (- b a) (- y x))]\n (/ (reduce + (for [i (range a b) j (range x y)] (api/ssm-at S i j))) (max 1 cnt))))))))))\n mean-sd (fn [xs] (let [n (max 1 (count xs)) m (/ (reduce + xs) n)]\n [m (js/Math.sqrt (/ (reduce + (map (fn [x] (* (- x m) (- x m))) xs)) n))]))\n r3 (fn [x] (/ (js/Math.round (* 1000 x)) 1000))\n ;; a frame is complete: into the window, and what it tells\n close (fn [state {:keys [N L step kernel]}]\n (let [f (frame-of (:acc state))\n t (+ (:t0 state) (/ step 2))\n frames (vec (take-last N (conj (or (:frames state) []) [t f])))\n S (api/ssm (mapv second frames))\n n (:n S)\n i (- n 1 L)\n nov (when (>= i L) (api/novelty-at S L i))\n nt (when nov (first (frames i)))\n hist (if nov (vec (take-last N (conj (or (:nov state) []) [nt (max 0 nov)]))) (or (:nov state) []))\n ;; a boundary: the novelty of the frame before is a peak, high for this window, and the last\n ;; boundary is at least two kernels away\n vs (mapv second hist) k (count vs)\n [m sd] (mean-sd vs)\n peak (when (>= k 3)\n (let [[tp p] (hist (- k 2))]\n (when (and (> p (vs (- k 3))) (>= p (vs (dec k))) (> p (max 0.05 (+ m sd)))\n (or (nil? (:last state)) (>= (- tp (:last state)) (* 2 kernel))))\n tp)))\n bs (when (>= n 8) (api/beat-spectrum S (quot n 2)))\n ;; a period is a lag more alike than the lags before it (a real dip between): music that does\n ;; not change is alike at every lag, and has none\n lag (some (fn [[l x]] (when (and (>= l 2) (> (- x (reduce min (subvec bs 1 l))) 0.1)) l))\n (when bs (api/periods bs)))\n period (when lag (r3 (* lag step)))\n s (cond-> (assoc state :frames frames :nov hist\n :cells (cells S) :period (or period (:period state)))\n peak (assoc :last peak :marks (vec (take-last 8 (conj (or (:marks state) []) peak)))))]\n {:state s\n :out (cond-> []\n nov (conj [0 (r3 (max 0 nov))])\n (and period (not= period (:period state))) (conj [2 period])\n peak (conj [3 [:segment {:t (or (:last state) (first (first frames))) :at peak\n :length (r3 (- peak (or (:last state) (first (first frames)))))\n :period (or period (:period state))}]]\n [1 :bang]))}))]\n {;; ── about ──\n :class \"form~\"\n :license \"MPL-2.0\"\n :doc \"How alike the moments of what arrives are, over the last seconds (20): a self-similarity matrix, drawn, of one frame every step seconds (0.25) — the mean of the lists that arrived in it (a chroma~, a features~ mfcc), or, for a stream of numbers (onset~'s third outlet, an rms), the shape of its values. On the left the novelty of each frame (0…1: how much the music changes there, by a checkerboard of kernel seconds (2) on each side, so it comes kernel seconds late); a bang on the second outlet where a segment begins (a novelty peak high for the window, two kernels after the last at least), and, on the fourth, [:segment {:t :at :length :period}] for the segment it ends; on the third, the period that repeats most in the window, in seconds (the bar, the phrase), when it changes. [:seconds x], [:step x] and [:kernel x] change them, :clear forgets. Into onset~ (a [:segment …] makes it forget the usual change of the section before: adaptive onsets), a message box with [:threshold x], or a print. Self-similarity after Foote (1999, 2000, 2001); the same functions for a whole take or a tree are api/ssm, api/novelty, api/boundaries and api/note-frames.\"\n :usage \"form~ [seconds] [step] [kernel]\"\n :io [1 4]\n :size [220 128]\n :args [[:seconds 20] [:step 0.25] [:kernel 2]]\n\n ;; ── logic ──\n ;; the state: {:frames [[t vector] …] :acc [values of the frame under way] :t0 its start :nov [[t value] …]\n ;; :last the last boundary :marks boundaries :cells the matrix drawn :period :set what messages changed}\n :receive (fn [{:keys [value state args at]}]\n (cond\n (= value :clear) {:state (select-keys state [:set]) :show nil}\n (and (vector? value) (= 2 (count value)) (#{:seconds :step :kernel} (first value)) (number? (second value)))\n (let [s (-> (select-keys state [:set]) (assoc-in [:set (first value)] (second value)))] {:state s :show nil})\n (and (usable value) (number? at))\n (let [o (settings args state)\n ;; the engine started again (its clock went back), or the kind of what arrives changed: from nothing\n state (if (or (and (:t0 state) (< at (- (:t0 state) 1)))\n (and (seq (:acc state)) (not= (number? (first (:acc state))) (number? (usable value)))))\n (select-keys state [:set]) state)\n v (usable value)]\n ;; a value a step after the frame under way began closes it, and begins the next\n (if (and (:t0 state) (seq (:acc state)) (>= (- at (:t0 state)) (:step o)))\n (let [{s :state out :out} (close state o)\n s (assoc s :acc [v] :t0 at)]\n (cond-> {:state s :show (select-keys s [:cells :nov :marks :period :frames])}\n (seq out) (assoc :out out)))\n {:state (cond-> (update state :acc (fnil conj []) v) (nil? (:t0 state)) (assoc :t0 at))}))\n :else {}))\n\n ;; ── view: the matrix (time runs down and to the right), the novelty under it, the period ──\n :draw (fn [{:keys [shown args state w h]}]\n (let [{:keys [step kernel seconds]} (settings args state)\n cs (:cells shown) g (count cs)\n side (- h 12) x0 6 y0 6 cw (if (pos? g) (/ side g) side)\n frames (:frames shown)\n t0 (first (first frames)) t1 (first (peek frames))\n px0 (+ x0 side 10) px1 (- w 6) pw (- px1 px0)\n tx (fn [t] (if (and t0 t1 (> t1 t0)) (+ px0 (* pw (/ (- t t0) (- t1 t0)))) px0))\n ny0 (+ y0 34) ny1 (- h 8)\n nov (filter (fn [[t]] (and t0 (>= t t0))) (:nov shown))\n ty (fn [v] (- ny1 (* (- ny1 ny0) (api/clamp v 0 1))))]\n (into\n [[:rect {:x 2 :y 2 :w (- w 4) :h (- h 4) :r 4 :stroke :grid :fill :bg}]\n [:rect {:x x0 :y y0 :w side :h side :stroke :grid :fill :bg}]\n [:text {:x px0 :y (+ y0 10) :size 10 :fill :ink} \"form\"]\n [:text {:x px0 :y (+ y0 23) :size 8 :fill :soft}\n (if-let [p (:period shown)] (str \"period \" (api/nice p) \" s\") (str (api/nice seconds) \" s · \" (api/nice step) \" · \" (api/nice kernel)))]\n [:line {:x1 px0 :y1 ny1 :x2 px1 :y2 ny1 :stroke :grid :weight 1}]]\n (concat\n (for [r (range g) c (range g) :let [v ((cs r) c)] :when (> v 0.1)]\n [:rect {:x (+ x0 (* c cw)) :y (+ y0 (* r cw)) :w (+ cw 0.3) :h (+ cw 0.3)\n :fill :accent :stroke nil :opacity (api/clamp (* v v) 0 1)}])\n (let [xys (mapv (fn [[t v]] [(api/clamp (tx t) px0 px1) (ty v)]) nov)]\n (when (> (count xys) 1) [[:path {:points xys :stroke :accent :weight 1.2}]]))\n (for [t (:marks shown) :when (and t0 (>= t t0))]\n [:line {:x1 (tx t) :y1 ny0 :x2 (tx t) :y2 ny1 :stroke :kairos :weight 1.2}])))))})\n"
26
26
  },
27
27
  {
28
28
  "class": "monitor",
@@ -46,7 +46,7 @@ export default {
46
46
  "class": "realize~",
47
47
  "license": "MPL-2.0",
48
48
  "doc": "Plays a faust~ from descriptors, as a score of them: loudness (dB, or a level 0…1 as features~ rms), brightness (Hz, the spectral centroid), noisiness (0…1, the spectral flatness) and pitch (Hz as pitch~ sends it, 0 for none; MIDI with `midi`). Its left outlet goes to the faust~: the faust~'s pitch or freq slider takes the pitch, its gain slider the loudness, and its other sliders are set to the settings of that faust~ that sound nearest to the brightness and noisiness written. It learns them first by playing steady settings of the faust~ offline and listening (300 by default), when the first values arrive or on learn; learn again after changing the faust~'s code. With features~ of the faust~'s sound on its last three inlets (rms, centroid, flatness) it listens to what it plays and asks for more where it falls short. On the right, how far the nearest setting is from what is written (above 1: the faust~ cannot sound like that).",
49
- "source": ";; realize~ · descriptors written → the sliders of a faust~ that sound like them (discussion #114, Study 37)\n;; Part of @kairotic/analysis. Written against the Kairotic API (api/…).\n;; What arrives is a frame of a score: a loudness, a brightness (the spectral centroid, Hz), a noisiness (the\n;; spectral flatness) and a pitch — what features~ and pitch~ hear, so another sound can drive it, or curves\n;; drawn on canvases read back (reader~ → snapshot~). What leaves is [:slider value] for the faust~ on its\n;; left outlet: its `pitch` (MIDI) or `freq` (Hz) slider takes the pitch, its `gain` (dB or a level) the\n;; loudness, and its other sliders, the timbre, are set to the blend of the settings of that faust~ that come\n;; nearest to the brightness and noisiness written (where a pitch is written, among the settings in which YIN\n;; would hear it). To know them, it learns the faust~ first (the task :learn: steady settings played offline\n;; and heard, @kairotic/faust-units' notation.mjs), when the first frame arrives or on `learn`.\n;; It can also listen to what it plays: features~ of the faust~'s sound on its last three inlets (loudness,\n;; brightness, noisiness heard). Then it asks for more where what is heard falls short of what is written,\n;; little by little (a correction that grows while the difference lasts, within limits), as the study's\n;; passes did offline.\n(let [meyda-url \"npm:meyda@5.6.3/dist/web/meyda.min.js\"\n origins [\"npm:@kairotic/faust-units@0.1.5/dist/notation.mjs\" \"kairotic/packages/faust-units/notation.mjs\"]\n silent -60 ; dB: below it nothing is written (a microphone in a room is often near -50)\n every 0.04 ; seconds: at most one new setting this often\n ;; listening: how much of each difference heard goes into the correction, and how far it may go\n rate 0.08\n leak 0.995 ; what it corrected fades unless it is still needed\n hold 0.25 ; seconds a sound has lasted before it is corrected (not its attack)\n limits {:loud 18 :lb 2 :nz 0.4}\n bank-size (fn [args] (js/Math.round (api/clamp (or (first (filter number? args)) 300) 40 2000)))\n midi-in? (fn [args] (some #{:midi} args))\n log2 (fn [x] (/ (js/Math.log x) js/Math.LN2))\n ;; a loudness as dB: a level (0 < v ≤ 1, as features~ rms) or dB already (≤ 0); 0 is silence\n db-of (fn [v] (cond (not (number? v)) nil (zero? v) -100 (and (pos? v) (<= v 1)) (api/db v) :else v))\n ;; the nearest settings of the bank to what is asked, blended: {:u [...] :loud0 dB :d distance}. The search\n ;; itself is the bank's query (notation.mjs: plain loops over typed arrays), asked for every frame it plays\n nearest (fn [bank {:keys [pitch lb nz voiced]}]\n (let [r ((:query bank) pitch lb nz (boolean voiced))]\n {:loud0 (aget r 0) :d (aget r 1) :u (vec (.slice r 2))}))\n ;; the values for the faust~, and only those that changed: [[0 [:cutoff 1800]] …]\n settle (fn [st at]\n (let [{:keys [bank]} st\n ;; a lane that never arrived keeps the middle of what the faust~ can do\n want (merge {:lb (first (:mid bank)) :nz (second (:mid bank))} (:want st))\n ;; what is asked of the bank: what is written, corrected by what was heard\n {cl :loud clb :lb cnz :nz} (:corr st)\n [lb-lo lb-hi] (or (:lbr bank) [0 20])\n ask (cond-> want\n (and cl (:loud want)) (update :loud + cl)\n clb (assoc :lb (api/clamp (+ (:lb want) clb) lb-lo lb-hi))\n cnz (assoc :nz (api/clamp (+ (:nz want) cnz) 0 1)))\n loud (:loud want)]\n (if (and (nil? loud) (nil? (:pitch want)) (nil? (get-in st [:want :lb])) (nil? (get-in st [:want :nz])))\n {:state (assoc st :t at)}\n (let [quiet? (and loud (< loud silent))\n {:keys [u loud0 d]} (if quiet? {:u (:u st) :loud0 0 :d (:reach st)} (nearest bank ask))\n ;; the timbre glides to where it goes (half the way every 40 ms), and keeps\n ;; gliding when nothing more arrives (a value written once, a canvas held)\n goal u\n u (if (and (:u st) (not quiet?)) (mapv (fn [a b] (+ a (* 0.5 (- b a)))) (:u st) u) u)\n moving? (and goal u (not quiet?) (some (fn [[a b]] (> (js/Math.abs (- a b)) 0.004)) (map vector u goal)))\n g (:gain bank) p (:pitch bank)\n gain-v (when (and g loud) (let [dB (if quiet? -100 (- (:loud ask) loud0))]\n (if (:db g) (api/clamp dB (:min g) (:max g))\n (api/clamp (js/Math.pow 10 (/ dB 20)) (:min g) (:max g)))))\n pitch-v (when (and p (:pitch want))\n (api/clamp (if (= \"hz\" (:kind p)) (api/midi->hz (:pitch want)) (:pitch want)) (:min p) (:max p)))\n vals (cond-> (if (and u (not quiet?))\n (into {} (map (fn [s x] [(:label s) (api/from-unit x (:min s) (:max s) (:log s))]) (:timbre bank) u))\n {})\n gain-v (assoc (:label g) gain-v)\n pitch-v (assoc (:label p) pitch-v))\n span (fn [label] (let [s (or (first (filter #(= label (:label %)) (:timbre bank))) (when (= label (:label g)) g) p)]\n (js/Math.abs (- (:max s) (:min s)))))\n changed (filter (fn [[k v]] (let [old (get (:sent st) k)]\n (or (nil? old) (> (js/Math.abs (- v old)) (* 0.002 (span k))))))\n vals)\n reach (when d (/ (js/Math.round (* 100 d)) 100))]\n {:state (-> st (assoc :t at :u (or u (:u st)) :reach reach :dirty (boolean moving?)\n :loud0 (if quiet? (:loud0 st) loud0)\n :tick (or (:tick st) (boolean moving?)))\n (update :sent merge (into {} changed)))\n :later (when (and moving? (not (:tick st))) [[(+ at every) 0 :realize-tick]])\n :out (cond-> (mapv (fn [[k v]] [0 [(keyword k) (api/nice v)]]) changed)\n (and reach (not= reach (:reach st))) (conj [1 reach]))\n :show {:status :ready :faust (:faust bank) :n (count (:entries bank)) :want want :reach reach\n :listening (some? (:corr st))}}))))\n ;; each request is numbered, so the answer to an older one is dropped\n learn (fn [st args]\n (let [k (inc (:k st 0))]\n {:state (assoc st :learning true :error nil :k k :corr nil)\n :show {:status :learning}\n :effects [[:learn {:n (bank-size args) :k k}]]}))]\n {;; ── about ──\n :class \"realize~\"\n :license \"MPL-2.0\"\n :doc \"Plays a faust~ from descriptors, as a score of them: loudness (dB, or a level 0…1 as features~ rms), brightness (Hz, the spectral centroid), noisiness (0…1, the spectral flatness) and pitch (Hz as pitch~ sends it, 0 for none; MIDI with `midi`). Its left outlet goes to the faust~: the faust~'s pitch or freq slider takes the pitch, its gain slider the loudness, and its other sliders are set to the settings of that faust~ that sound nearest to the brightness and noisiness written. It learns them first by playing steady settings of the faust~ offline and listening (300 by default), when the first values arrive or on learn; learn again after changing the faust~'s code. With features~ of the faust~'s sound on its last three inlets (rms, centroid, flatness) it listens to what it plays and asks for more where it falls short. On the right, how far the nearest setting is from what is written (above 1: the faust~ cannot sound like that).\"\n :usage \"realize~ [settings] [midi]\"\n :io [7 2]\n :size [236 46]\n\n ;; ── tasks: learning the faust~ it drives, outside its pure logic ──\n :tasks {:learn (fn [{:keys [n]} box]\n (let [f (first (filter (fn [w] (and (= \"faust~\" (:class w)) (zero? (:outlet w)))) (:wired box)))\n src (str (get-in f [:data :source]))\n ;; the compilation of its code, waited for (a faust~ just written is still compiling)\n compiled (fn compiled [tries]\n (let [r (api/compiled \"faust\" src)]\n (cond (= :ok (:status r)) (js/Promise.resolve r)\n (and (#{:compiling :empty} (:status r)) (pos? tries))\n (js/Promise. (fn [ok] (js/setTimeout (fn [] (ok nil)) 250)))\n :else (js/Promise.reject (js/Error. (str \"the faust~ does not compile: \" (or (:error r) \"no code\")))))))\n wait (fn wait [tries]\n (.then (compiled tries) (fn [r] (if r r (wait (dec tries))))))]\n (if-not f\n (js/Promise.reject (js/Error. \"connect its left outlet to a faust~: it learns the one it plays\"))\n (-> (wait 120)\n (.then (fn [r]\n (when (get-in r [:info :poly])\n (throw (js/Error. \"it plays a faust~ of one voice (this one declares nvoices)\")))\n (let [fac (.-factory (:value r))]\n (-> (js/Promise.all #js [(api/load-script meyda-url) (api/import-module origins)])\n (.then (fn [[_ m]]\n (.then (.learn m #js {:module (or (.-module fac) (.-code fac))\n :json (js/JSON.parse (.-json fac))\n :Meyda (.-Meyda js/window) :n n})\n (fn [b] [b (.index m b)]))))))))\n (.then (fn [[bjs query]]\n ;; the settings stay a JS array (the query reads them); the rest as data\n (let [es (.-entries bjs)\n col (fn [j] (map (fn [e] (aget e j)) (array-seq es)))\n avg (fn [j] (/ (reduce + (col j)) (.-length es)))]\n (-> (js->clj #js {:pitch (.-pitch bjs) :gain (.-gain bjs) :timbre (.-timbre bjs) :sd (.-sd bjs)}\n :keywordize-keys true)\n (assoc :query query :entries es :faust (name (:id f)) :mid [(avg 1) (avg 2)]\n :lbr [(reduce min (col 1)) (reduce max (col 1))])))))))))}\n\n ;; ── logic ──\n :receive (fn [{:keys [inlet value state args at]}]\n (let [st (or state {})]\n (cond\n ;; what learning found\n (= inlet :task)\n (cond\n (not= (:k (:arg value)) (:k st)) {}\n (:error value) {:state (assoc st :learning false :error (:error value))\n :show {:status :error :error (:error value)}}\n :else (let [st (assoc st :learning false :bank (:value value) :sent {} :u nil)]\n (settle st (or at (:t st) 0))))\n\n (#{:learn \"learn\" :bang} value) (learn st args)\n\n ;; a moment later: the last frame that arrived while it waited\n (= value :realize-tick) (if (and (:bank st) (:dirty st)) (settle (assoc st :tick false) at) {:state (assoc st :tick false)})\n\n ;; what is heard in what it plays (features~ of the faust~): the correction grows by a part of\n ;; the difference, while something is written and sounds\n (and (#{4 5 6} inlet) (number? value))\n (let [heard (case inlet 4 (db-of value) 5 (when (pos? value) (log2 value)) 6 value)\n k ({4 :loud 5 :lb 6 :nz} inlet)\n want (:want st)\n st (cond-> st (= k :loud) (assoc :heard-loud heard))\n sounding? (and (:bank st) (number? (:loud want)) (> (:loud want) silent)\n (> (or (:heard-loud st) -100) silent))]\n (if (and sounding? (number? heard) (number? (get want k)) (:since st) (>= (- at (:since st)) hold))\n ;; within what the faust~ can give (the bank's brightness, noisiness 0…1, the gain slider's\n ;; range): a correction that cannot be played does not grow, so it does not wind up\n (let [bank (:bank st) w (get want k) g (:gain bank) [lb-lo lb-hi] (or (:lbr bank) [0 20])\n [lo hi] (case k\n :lb [(- lb-lo w) (- lb-hi w)]\n :nz [(- w) (- 1 w)]\n :loud (if (and g (:db g) (:loud0 st))\n [(+ (:min g) (:loud0 st) (- w)) (+ (:max g) (:loud0 st) (- w))]\n [(- (limits k)) (limits k)]))\n c (-> (+ (* leak (get-in st [:corr k] 0)) (* rate (- w heard)))\n (api/clamp (max lo (- (limits k))) (min hi (limits k))))\n st (-> st (assoc-in [:corr k] c) (assoc :dirty true))]\n (if (or (nil? (:t st)) (>= (- at (:t st)) every))\n (settle st at)\n (if (:tick st) {:state st} {:state (assoc st :tick true) :later [[(+ (:t st) every) 0 :realize-tick]]})))\n {:state st}))\n\n (and (integer? inlet) (number? value))\n (let [midi (when (and (= inlet 3) (pos? value)) (if (midi-in? args) value (api/hz->midi value)))\n want (case inlet\n 0 (assoc (:want st) :loud (db-of value))\n 1 (if (pos? value) (assoc (:want st) :lb (log2 value)) (:want st))\n 2 (assoc (:want st) :nz (api/clamp value 0 1))\n 3 (if midi (assoc (:want st) :pitch midi :voiced true) (assoc (:want st) :voiced false))\n (:want st))\n ;; since when what is written sounds (its attack is not corrected)\n st (cond-> (assoc st :want want :dirty true)\n (= inlet 0) (assoc :since (if (> (or (:loud want) -100) silent) (or (:since st) at) nil)))]\n (cond\n (:bank st) (if (or (nil? (:t st)) (>= (- at (:t st)) every))\n (settle st at)\n (if (:tick st)\n {:state st}\n {:state (assoc st :tick true) :later [[(+ (:t st) every) 0 :realize-tick]]}))\n (or (:learning st) (:error st)) {:state st}\n :else (learn st args)))\n\n :else {})))\n\n ;; ── view: what it learnt, what is written now, and how far the faust~ is from it ──\n :draw (fn [{:keys [shown w h]}]\n (let [{:keys [status faust n want reach error]} shown\n line1 (case status\n :learning \"learning the faust~ it plays…\"\n :ready (str \"plays \" faust \" · \" n \" settings heard\" (when (:listening shown) \" · listening\"))\n :error (str \"✗ \" error)\n \"waits for a faust~ and a score\")\n fmt (fn [x d] (if (number? x) (.toFixed x d) \"–\"))\n line2 (when (= status :ready)\n (str (fmt (:loud want) 0) \" dB · \"\n (if (:lb want) (str (js/Math.round (js/Math.pow 2 (:lb want))) \" Hz\") \"–\") \" · \"\n (fmt (:nz want) 2)\n (when (and (:voiced want) (:pitch want)) (str \" · \" (api/midi-name (js/Math.round (:pitch want)))))))\n bw (- w 16) far (when (number? reach) (api/clamp (/ reach 2) 0 1))]\n (cond-> [[:rect {:x 2 :y 2 :w (- w 4) :h (- h 4) :r 4 :stroke :grid :fill :bg}]\n [:text {:x 8 :y 15 :size 9 :fill (if (= status :error) :kairos :ink)} line1]]\n line2 (conj [:text {:x 8 :y 28 :size 9 :fill :soft} line2])\n far (conj [:rect {:x 8 :y 35 :w bw :h 4 :r 2 :fill :grid :stroke nil}]\n [:rect {:x 8 :y 35 :w (max 2 (* bw far)) :h 4 :r 2 :fill (if (> reach 1) :kairos :accent) :stroke nil}]))))})\n"
49
+ "source": ";; realize~ · descriptors written → the sliders of a faust~ that sound like them (discussion #114, Study 37)\n;; Part of @kairotic/analysis. Written against the Kairotic API (api/…).\n;; What arrives is a frame of a score: a loudness, a brightness (the spectral centroid, Hz), a noisiness (the\n;; spectral flatness) and a pitch — what features~ and pitch~ hear, so another sound can drive it, or curves\n;; drawn on canvases read back (reader~ → snapshot~). What leaves is [:slider value] for the faust~ on its\n;; left outlet: its `pitch` (MIDI) or `freq` (Hz) slider takes the pitch, its `gain` (dB or a level) the\n;; loudness, and its other sliders, the timbre, are set to the blend of the settings of that faust~ that come\n;; nearest to the brightness and noisiness written (where a pitch is written, among the settings in which YIN\n;; would hear it). To know them, it learns the faust~ first (the task :learn: steady settings played offline\n;; and heard, @kairotic/faust-units' notation.mjs), when the first frame arrives or on `learn`.\n;; It can also listen to what it plays: features~ of the faust~'s sound on its last three inlets (loudness,\n;; brightness, noisiness heard). Then it asks for more where what is heard falls short of what is written,\n;; little by little (a correction that grows while the difference lasts, within limits), as the study's\n;; passes did offline.\n(let [meyda-url \"npm:meyda@5.6.3/dist/web/meyda.min.js\"\n origins [\"npm:@kairotic/faust-units@0.1.5/dist/notation.mjs\" \"kairotic/packages/faust-units/notation.mjs\"]\n silent -60 ; dB: below it nothing is written (a microphone in a room is often near -50)\n every 0.04 ; seconds: at most one new setting this often\n ;; listening: how much of each difference heard goes into the correction, and how far it may go\n rate 0.08\n leak 0.995 ; what it corrected fades unless it is still needed\n hold 0.25 ; seconds a sound has lasted before it is corrected (not its attack)\n limits {:loud 18 :lb 2 :nz 0.4}\n bank-size (fn [args] (js/Math.round (api/clamp (or (first (filter number? args)) 300) 40 2000)))\n midi-in? (fn [args] (some #{:midi} args))\n log2 (fn [x] (/ (js/Math.log x) js/Math.LN2))\n ;; a loudness as dB: a level (0 < v ≤ 1, as features~ rms) or dB already (≤ 0); 0 is silence\n db-of (fn [v] (cond (not (number? v)) nil (zero? v) -100 (and (pos? v) (<= v 1)) (api/db v) :else v))\n ;; the nearest settings of the bank to what is asked, blended: {:u [...] :loud0 dB :d distance}. The search\n ;; itself is the bank's query (notation.mjs: plain loops over typed arrays), asked for every frame it plays\n nearest (fn [bank {:keys [pitch lb nz voiced]}]\n (let [r ((:query bank) pitch lb nz (boolean voiced))]\n {:loud0 (aget r 0) :d (aget r 1) :u (vec (.slice r 2))}))\n ;; the values for the faust~, and only those that changed: [[0 [:cutoff 1800]] …]\n settle (fn [st at]\n (let [{:keys [bank]} st\n ;; a lane that never arrived keeps the middle of what the faust~ can do\n want (merge {:lb (first (:mid bank)) :nz (second (:mid bank))} (:want st))\n ;; what is asked of the bank: what is written, corrected by what was heard\n {cl :loud clb :lb cnz :nz} (:corr st)\n [lb-lo lb-hi] (or (:lbr bank) [0 20])\n ask (cond-> want\n (and cl (:loud want)) (update :loud + cl)\n clb (assoc :lb (api/clamp (+ (:lb want) clb) lb-lo lb-hi))\n cnz (assoc :nz (api/clamp (+ (:nz want) cnz) 0 1)))\n loud (:loud want)]\n (if (and (nil? loud) (nil? (:pitch want)) (nil? (get-in st [:want :lb])) (nil? (get-in st [:want :nz])))\n {:state (assoc st :t at)}\n (let [quiet? (and loud (< loud silent))\n {:keys [u loud0 d]} (if quiet? {:u (:u st) :loud0 0 :d (:reach st)} (nearest bank ask))\n ;; the timbre glides to where it goes (half the way every 40 ms), and keeps\n ;; gliding when nothing more arrives (a value written once, a canvas held)\n goal u\n u (if (and (:u st) (not quiet?)) (mapv (fn [a b] (+ a (* 0.5 (- b a)))) (:u st) u) u)\n moving? (and goal u (not quiet?) (some (fn [[a b]] (> (js/Math.abs (- a b)) 0.004)) (map vector u goal)))\n g (:gain bank) p (:pitch bank)\n gain-v (when (and g loud) (let [dB (if quiet? -100 (- (:loud ask) loud0))]\n (if (:db g) (api/clamp dB (:min g) (:max g))\n (api/clamp (js/Math.pow 10 (/ dB 20)) (:min g) (:max g)))))\n pitch-v (when (and p (:pitch want))\n (api/clamp (if (= \"hz\" (:kind p)) (api/midi->hz (:pitch want)) (:pitch want)) (:min p) (:max p)))\n vals (cond-> (if (and u (not quiet?))\n (into {} (map (fn [s x] [(:label s) (api/from-unit x (:min s) (:max s) (:log s))]) (:timbre bank) u))\n {})\n gain-v (assoc (:label g) gain-v)\n pitch-v (assoc (:label p) pitch-v))\n span (fn [label] (let [s (or (first (filter #(= label (:label %)) (:timbre bank))) (when (= label (:label g)) g) p)]\n (js/Math.abs (- (:max s) (:min s)))))\n changed (filter (fn [[k v]] (let [old (get (:sent st) k)]\n (or (nil? old) (> (js/Math.abs (- v old)) (* 0.002 (span k))))))\n vals)\n reach (when d (/ (js/Math.round (* 100 d)) 100))]\n {:state (-> st (assoc :t at :u (or u (:u st)) :reach reach :dirty (boolean moving?)\n :loud0 (if quiet? (:loud0 st) loud0)\n :tick (or (:tick st) (boolean moving?)))\n (update :sent merge (into {} changed)))\n :later (when (and moving? (not (:tick st))) [[(+ at every) 0 :realize-tick]])\n :out (cond-> (mapv (fn [[k v]] [0 [(keyword k) (api/nice v)]]) changed)\n (and reach (not= reach (:reach st))) (conj [1 reach]))\n :show {:status :ready :faust (:faust bank) :n (count (:entries bank)) :want want :reach reach\n :listening (some? (:corr st))}}))))\n ;; each request is numbered, so the answer to an older one is dropped\n learn (fn [st args]\n (let [k (inc (:k st 0))]\n {:state (assoc st :learning true :error nil :k k :corr nil)\n :show {:status :learning}\n :effects [[:learn {:n (bank-size args) :k k}]]}))]\n {;; ── about ──\n :class \"realize~\"\n :license \"MPL-2.0\"\n :doc \"Plays a faust~ from descriptors, as a score of them: loudness (dB, or a level 0…1 as features~ rms), brightness (Hz, the spectral centroid), noisiness (0…1, the spectral flatness) and pitch (Hz as pitch~ sends it, 0 for none; MIDI with `midi`). Its left outlet goes to the faust~: the faust~'s pitch or freq slider takes the pitch, its gain slider the loudness, and its other sliders are set to the settings of that faust~ that sound nearest to the brightness and noisiness written. It learns them first by playing steady settings of the faust~ offline and listening (300 by default), when the first values arrive or on learn; learn again after changing the faust~'s code. With features~ of the faust~'s sound on its last three inlets (rms, centroid, flatness) it listens to what it plays and asks for more where it falls short. On the right, how far the nearest setting is from what is written (above 1: the faust~ cannot sound like that).\"\n :usage \"realize~ [settings] [midi]\"\n :io [7 2]\n :size [236 46]\n\n ;; ── tasks: learning the faust~ it drives, outside its pure logic ──\n :tasks {:learn (fn [{:keys [n]} box]\n (let [f (first (filter (fn [w] (and (= \"faust~\" (:class w)) (zero? (:outlet w)))) (:wired box)))\n src (str (get-in f [:data :source]))\n ;; the compilation of its code, waited for (a faust~ just written is still compiling)\n compiled (fn compiled [tries]\n (let [r (api/compiled \"faust\" src)]\n (cond (= :ok (:status r)) (js/Promise.resolve r)\n (and (#{:compiling :empty} (:status r)) (pos? tries))\n (js/Promise. (fn [ok] (js/setTimeout (fn [] (ok nil)) 250)))\n :else (js/Promise.reject (js/Error. (str \"the faust~ does not compile: \" (or (:error r) \"no code\")))))))\n wait (fn wait [tries]\n (.then (compiled tries) (fn [r] (if r r (wait (dec tries))))))]\n (if-not f\n (js/Promise.reject (js/Error. \"connect its left outlet to a faust~: it learns the one it plays\"))\n (-> (wait 120)\n (.then (fn [r]\n (when (get-in r [:info :poly])\n (throw (js/Error. \"it plays a faust~ of one voice (this one declares nvoices)\")))\n (let [fac (.-factory (:value r))]\n (-> (js/Promise.all #js [(api/load-script meyda-url) (api/import-module origins)])\n (.then (fn [[_ m]]\n (.then (.learn m #js {:module (or (.-module fac) (.-code fac))\n :json (js/JSON.parse (.-json fac))\n :Meyda (.-Meyda js/window) :n n})\n (fn [b] [b (.index m b)]))))))))\n (.then (fn [[bjs query]]\n ;; the settings stay a JS array (the query reads them); the rest as data\n (let [es (.-entries bjs)\n col (fn [j] (map (fn [e] (aget e j)) (array-seq es)))\n avg (fn [j] (/ (reduce + (col j)) (.-length es)))]\n (-> (js->clj #js {:pitch (.-pitch bjs) :gain (.-gain bjs) :timbre (.-timbre bjs) :sd (.-sd bjs)}\n :keywordize-keys true)\n (assoc :query query :entries es :faust (name (:id f)) :mid [(avg 1) (avg 2)]\n :lbr [(reduce min (col 1)) (reduce max (col 1))])))))))))}\n\n ;; ── logic ──\n :receive (fn [{:keys [inlet value state args at]}]\n (let [st (or state {})]\n (cond\n ;; what learning found\n (= inlet :task)\n (cond\n (not= (:k (:arg value)) (:k st)) {}\n (:error value) {:state (assoc st :learning false :error (:error value))\n :show {:status :error :error (:error value)}}\n :else (let [st (assoc st :learning false :bank (:value value) :sent {} :u nil)]\n (settle st (or at (:t st) 0))))\n\n (#{:learn :bang} value) (learn st args)\n\n ;; a moment later: the last frame that arrived while it waited\n (= value :realize-tick) (if (and (:bank st) (:dirty st)) (settle (assoc st :tick false) at) {:state (assoc st :tick false)})\n\n ;; what is heard in what it plays (features~ of the faust~): the correction grows by a part of\n ;; the difference, while something is written and sounds\n (and (#{4 5 6} inlet) (number? value))\n (let [heard (case inlet 4 (db-of value) 5 (when (pos? value) (log2 value)) 6 value)\n k ({4 :loud 5 :lb 6 :nz} inlet)\n want (:want st)\n st (cond-> st (= k :loud) (assoc :heard-loud heard))\n sounding? (and (:bank st) (number? (:loud want)) (> (:loud want) silent)\n (> (or (:heard-loud st) -100) silent))]\n (if (and sounding? (number? heard) (number? (get want k)) (:since st) (>= (- at (:since st)) hold))\n ;; within what the faust~ can give (the bank's brightness, noisiness 0…1, the gain slider's\n ;; range): a correction that cannot be played does not grow, so it does not wind up\n (let [bank (:bank st) w (get want k) g (:gain bank) [lb-lo lb-hi] (or (:lbr bank) [0 20])\n [lo hi] (case k\n :lb [(- lb-lo w) (- lb-hi w)]\n :nz [(- w) (- 1 w)]\n :loud (if (and g (:db g) (:loud0 st))\n [(+ (:min g) (:loud0 st) (- w)) (+ (:max g) (:loud0 st) (- w))]\n [(- (limits k)) (limits k)]))\n c (-> (+ (* leak (get-in st [:corr k] 0)) (* rate (- w heard)))\n (api/clamp (max lo (- (limits k))) (min hi (limits k))))\n st (-> st (assoc-in [:corr k] c) (assoc :dirty true))]\n (if (or (nil? (:t st)) (>= (- at (:t st)) every))\n (settle st at)\n (if (:tick st) {:state st} {:state (assoc st :tick true) :later [[(+ (:t st) every) 0 :realize-tick]]})))\n {:state st}))\n\n (and (integer? inlet) (number? value))\n (let [midi (when (and (= inlet 3) (pos? value)) (if (midi-in? args) value (api/hz->midi value)))\n want (case inlet\n 0 (assoc (:want st) :loud (db-of value))\n 1 (if (pos? value) (assoc (:want st) :lb (log2 value)) (:want st))\n 2 (assoc (:want st) :nz (api/clamp value 0 1))\n 3 (if midi (assoc (:want st) :pitch midi :voiced true) (assoc (:want st) :voiced false))\n (:want st))\n ;; since when what is written sounds (its attack is not corrected)\n st (cond-> (assoc st :want want :dirty true)\n (= inlet 0) (assoc :since (if (> (or (:loud want) -100) silent) (or (:since st) at) nil)))]\n (cond\n (:bank st) (if (or (nil? (:t st)) (>= (- at (:t st)) every))\n (settle st at)\n (if (:tick st)\n {:state st}\n {:state (assoc st :tick true) :later [[(+ (:t st) every) 0 :realize-tick]]}))\n (or (:learning st) (:error st)) {:state st}\n :else (learn st args)))\n\n :else {})))\n\n ;; ── view: what it learnt, what is written now, and how far the faust~ is from it ──\n :draw (fn [{:keys [shown w h]}]\n (let [{:keys [status faust n want reach error]} shown\n line1 (case status\n :learning \"learning the faust~ it plays…\"\n :ready (str \"plays \" faust \" · \" n \" settings heard\" (when (:listening shown) \" · listening\"))\n :error (str \"✗ \" error)\n \"waits for a faust~ and a score\")\n fmt (fn [x d] (if (number? x) (.toFixed x d) \"–\"))\n line2 (when (= status :ready)\n (str (fmt (:loud want) 0) \" dB · \"\n (if (:lb want) (str (js/Math.round (js/Math.pow 2 (:lb want))) \" Hz\") \"–\") \" · \"\n (fmt (:nz want) 2)\n (when (and (:voiced want) (:pitch want)) (str \" · \" (api/midi-name (js/Math.round (:pitch want)))))))\n bw (- w 16) far (when (number? reach) (api/clamp (/ reach 2) 0 1))]\n (cond-> [[:rect {:x 2 :y 2 :w (- w 4) :h (- h 4) :r 4 :stroke :grid :fill :bg}]\n [:text {:x 8 :y 15 :size 9 :fill (if (= status :error) :kairos :ink)} line1]]\n line2 (conj [:text {:x 8 :y 28 :size 9 :fill :soft} line2])\n far (conj [:rect {:x 8 :y 35 :w bw :h 4 :r 2 :fill :grid :stroke nil}]\n [:rect {:x 8 :y 35 :w (max 2 (* bw far)) :h 4 :r 2 :fill (if (> reach 1) :kairos :accent) :stroke nil}]))))})\n"
50
50
  },
51
51
  {
52
52
  "class": "spectrum~",
@@ -57,8 +57,8 @@ export default {
57
57
  {
58
58
  "class": "transcribe",
59
59
  "license": "MPL-2.0",
60
- "doc": "Writes what pitch~ hears into a freqt, as strokes. Hz on the left (from pitch~: 0 when it hears none), rec to start from 0 (its first write clears the freqt) or rec t (msg rec 8) to start from beat t, keeping what is before it, or ovr t (msg ovr 8) to write over only what the take covers, keeping what comes after and stop to end; a bang on the right (from onset~) begins a new note even on the same pitch, and a number there (onset~'s strength, 0…1) does too and gives the note that weight: its stroke is written heavier (as an event's :amp), and the quantizer reads it as the note's :vel, which helps find the beat and the bar (issue #106). In notes mode the contour of each voiced stretch is cut into notes where it settles on another pitch (a slide, a vibrato or an ornament that passes does not start notes; an onset on the right does), each a flat stroke at its median pitch (the freqt's tuning, or its score view, rounds it), written when the voice stops; in glide mode a stroke follows the pitch as it moves. Times are beats at bpm (60: one beat a second). Notes shorter than ms (80) are left out. It writes into the freqt at most every 2 seconds, each time one event of the history. Connect its outlet to the freqt's inlet.",
61
- "source": ";; transcribe · what pitch~ hears, written as strokes of a freqt\n;; Part of @kairotic/analysis. Written against the Kairotic API (api/…). Pure logic: no audio of its own.\n(let [;; ── what it understands ──\n flush-every 2 ; seconds between two writes into the freqt (each write is one event of the history)\n max-points 64 ; points of a stroke in glide mode\n cents (fn [a b] (* 1200 (/ (js/Math.log (/ b a)) js/Math.LN2)))\n ;; the pitch a note holds longest: pitch~ only sends what changed, so a steady note is one point and\n ;; its attack many; each pitch weighs the time until the next one\n held (fn [pts t1]\n (let [ws (map (fn [[t f] [t2 _]] [f (max 0 (- t2 t))]) pts (concat (rest pts) [[t1 nil]]))\n total (reduce + (map second ws))\n sorted (sort-by first ws)]\n (if (pos? total)\n (loop [[[f w] & more] sorted acc 0]\n (let [acc (+ acc w)] (if (or (>= acc (/ total 2)) (empty? more)) f (recur more acc))))\n (second (last pts)))))\n mode-of (fn [args] (if (= :glide (:mode args)) :glide :notes))\n beats (fn [state at] (* (- at (:t0 state)) (/ (:bpm state) 60)))\n r3 (fn [x] (/ (js/Math.round (* x 1000)) 1000))\n ;; ── a note that ends: a stroke, if it lasted long enough ──\n stroke-of (fn [{:keys [pts]} t1 mode]\n (let [t0 (ffirst pts) hz (held pts t1)]\n (if (= mode :glide)\n (let [kept (reduce (fn [acc [t f]]\n (if (> (js/Math.abs (cents (second (peek acc)) f)) 10) (conj acc [t f]) acc))\n [(first pts)] (rest pts))\n kept (if (> (count kept) max-points)\n (vec (take-nth (js/Math.ceil (/ (count kept) max-points)) kept))\n kept)]\n (mapv (fn [[t f]] [(r3 t) (r3 f)]) (conj kept [t1 (second (peek pts))])))\n [[(r3 t0) (r3 hz)] [(r3 t1) (r3 hz)]])))\n end-note (fn [state t]\n (if-let [cur (:cur state)]\n (let [t0 (ffirst (:pts cur))\n long-enough (>= (/ (- t t0) (/ (:bpm state) 60)) (/ (:min-ms state) 1000))]\n (cond-> (dissoc state :cur)\n long-enough (update :done conj {:pts (stroke-of cur t (:mode state)) :acc (:acc cur)})\n long-enough (update :count inc)))\n state))\n ;; a note takes the strength of the onset that began it (a number on the right inlet, from onset~), if it\n ;; came at most 0.15 s before: onset~ often hears the attack before pitch~ hears the pitch\n start-note (fn [state t hz]\n (let [[at s] (:acc state) acc (when (and at (<= (- t at) (* 0.15 (/ (:bpm state) 60)))) s)]\n (cond-> (assoc state :cur (cond-> {:pts [[t hz]] :midi (api/hz->midi hz)} acc (assoc :acc acc)))\n acc (dissoc :acc))))\n ;; how a note with the strength of its attack is written: the weight of its stroke, as an event's :amp is\n ;; (0 … 1 → 0.4 … 2.8 times the usual); the quantizer reads it back as the note's :vel (issue #106)\n edit-of (fn [{:keys [pts acc]}] (if (number? acc) [:stroke pts {:w (/ (js/Math.round (* 100 (+ 0.4 (* 2.4 acc)))) 100)}] [:stroke pts]))\n ;; ── writing: the strokes that ended, as one [:batch …], at most every flush-every seconds ──\n flush (fn [state at force]\n (let [now (r3 (beats state at))]\n (cond\n ;; ovr at stop: what the take covered since the last write is overwritten, even by silence\n (and (empty? (:done state)) (not (and force (:ovr state) (> now (:cut-from state))))) [state nil nil]\n (or force (nil? (:sent state)) (>= (- at (:sent state)) flush-every))\n ;; the first write of a take clears the freqt (as part of the same event); a take over (ovr)\n ;; cuts what it covered since the last write, and keeps what comes after\n [(cond-> (assoc state :done [] :sent at :pending false :fresh false)\n (:ovr state) (assoc :cut-from now))\n [:batch (into (cond (:ovr state) [[:cut (:cut-from state) now]]\n (:fresh state) [(if (:from state) [:cut (:from state)] [:clear])]\n :else [])\n (mapv edit-of (:done state)))] nil]\n (:pending state) [state nil nil]\n :else [(assoc state :pending true) nil [(+ (:sent state) flush-every) 0 [:flush]]])))\n result (fn [state [st edit later]]\n (cond-> {:state st :show (select-keys st [:on :count])}\n edit (assoc :out [[0 edit]])\n later (assoc :later [later])))\n ;; ── notes mode: the contour of each voiced stretch, segmented (issue #106, measured on cante2midi) ──\n ;; What pitch~ sends while a voice sounds is kept as a run of points; when the voice stops (or the run is long)\n ;; the run is read as a contour every 10 ms, smoothed by a median of 100 ms (vibrato and jitter), and cut into\n ;; notes with hysteresis: a note goes on while the contour stays within a semitone of its pitch; a new one\n ;; begins only where the contour, away from it, has held still (within 1.1 semitones) for 70 ms, so a slide\n ;; or an ornament that passes does not start notes. An onset (inlet 1) cuts there too. Each note is written\n ;; at the median pitch of its frames. Silences of 30 ms at most are bridged.\n median (fn [xs] (let [v (vec (sort xs))] (nth v (quot (count v) 2))))\n segment (fn [run marks t-end k minlen]\n (let [dt (* 0.01 k) w 5 band 1 stab 1.1 hold-n 7\n ts (vec (range (ffirst run) t-end dt))\n ;; the contour on the grid: each frame, the last point at or before it (as MIDI)\n raw (loop [i 0 [p & more :as ps] run cur nil out (transient [])]\n (if (>= i (count ts))\n (persistent! out)\n (let [t (ts i)]\n (if (and p (<= (first p) (+ t 1e-9)))\n (recur i more (api/hz->midi (second p)) out)\n (recur (inc i) ps cur (conj! out cur))))))\n n (count raw)\n sm (mapv (fn [i] (median (subvec raw (max 0 (- i w)) (min n (+ i w 1))))) (range n))\n cut? (fn [i] (let [a (if (pos? i) (ts (dec i)) (- (ts 0) dt)) b (ts i)] (some (fn [[mt]] (and (> mt a) (<= mt b))) marks)))\n note (fn [s e ms] {:s s :e e :ms ms})\n notes (loop [i 0 cur nil pend [] out []]\n (if (>= i n)\n (if cur (conj out (assoc cur :e t-end)) out)\n (let [t (ts i) m (sm i)]\n (cond\n (nil? cur) (recur (inc i) (note t (+ t dt) [m]) [] out)\n ;; (not while the contour is away from the note and not settled yet: there the contour\n ;; decides; measured on cante2midi with onsets from its Bark bands)\n (and (empty? pend) (cut? i) (> t (:s cur))) (recur (inc i) (note t (+ t dt) [m]) [] (conj out (assoc cur :e t)))\n (<= (js/Math.abs (- m (median (take-last 24 (:ms cur))))) band)\n (recur (inc i) (assoc cur :e (+ t dt) :ms (into (conj (:ms cur) m) (map second pend))) [] out)\n :else\n (let [pend (conj pend [t m]) tail (take-last hold-n pend) tm (map second tail)]\n (if (and (>= (count pend) hold-n) (< (- (apply max tm) (apply min tm)) stab))\n ;; the new note begins where the contour left the old one; its pitch is that of where it settled\n (recur (inc i) (note (ffirst pend) (+ t dt) (vec tm)) [] (conj out (assoc cur :e (ffirst pend))))\n (recur (inc i) cur pend out)))))))]\n (->> notes\n (filter (fn [{:keys [s e]}] (>= (- e s) minlen)))\n (mapv (fn [{:keys [s e ms]}]\n (let [hz (api/midi->hz (median ms))\n ;; the strength of an onset heard at most 0.15 s before the note (or just after its start)\n acc (some (fn [[mt x]] (when (and (number? x) (<= (- s (* 0.15 k)) mt (+ s (* 0.05 k)))) x)) marks)]\n (cond-> {:pts [[(r3 s) (r3 hz)] [(r3 e) (r3 hz)]]} acc (assoc :acc acc))))))))\n ;; the run segmented into :done: all of it (the voice stopped at t-end), or all but its last note (a long run\n ;; still sounding: that note goes on in the run)\n settle (fn [state t-end all?]\n (let [run (:run state) k (/ (:bpm state) 60) minlen (* k (/ (:min-ms state) 1000))]\n (if (or (empty? run) (<= t-end (ffirst run)))\n (dissoc state :run :silent)\n (let [ns (segment run (:marks state) t-end k minlen)\n [keep-ns rest-from] (if (or all? (< (count ns) 2)) [ns nil] [(pop ns) (ffirst (:pts (peek ns)))])\n state (-> state (update :done into keep-ns) (update :count + (count keep-ns)))]\n (if rest-from\n (let [before (last (filter (fn [[t]] (<= t rest-from)) run))]\n (assoc state :run (into [[rest-from (second before)]] (filter (fn [[t]] (> t rest-from)) run))\n :marks (filterv (fn [[t]] (>= t (- rest-from (* 0.15 k)))) (:marks state))))\n (-> state (dissoc :run :silent) (assoc :marks (filterv (fn [[t]] (> t t-end)) (:marks state)))))))))\n ;; a silence that lasted more than 30 ms: the run ends where it began\n settled (fn [state t]\n (if (and (:silent state) (> (- t (:silent state)) (* 0.03 (/ (:bpm state) 60))))\n (settle state (:silent state) true)\n state))\n hear-notes (fn [state hz t]\n (let [k (/ (:bpm state) 60)\n state (settled state t)]\n (cond\n (not (pos? hz)) (if (and (seq (:run state)) (nil? (:silent state))) (assoc state :silent t) state)\n :else (let [state (-> state (dissoc :silent) (update :run (fnil conj []) [t hz]))]\n ;; a run of 8 s and more: what is settled is written, the last note goes on\n (if (> (- t (ffirst (:run state))) (* 8 k)) (settle state t false) state)))))\n ;; ── one value of pitch~ (Hz, 0 when it hears none) ──\n hear (fn [state hz at]\n (let [t (beats state at)\n cur (:cur state)\n state (cond\n (= :notes (:mode state)) (hear-notes state hz t)\n (not (pos? hz)) (end-note state t)\n (nil? cur) (start-note state t hz)\n :else (update-in state [:cur :pts] conj [t hz]))\n [st edit later] (flush state at false)]\n ;; the voice stopped: a look again in a little while, to write its notes even if nothing else arrives\n [st edit (or later (when (and (:silent st) (not (:pending st))) [(+ at 0.05) 0 [:flush]]))]))]\n {;; ── about ──\n :class \"transcribe\"\n :license \"MPL-2.0\"\n :doc \"Writes what pitch~ hears into a freqt, as strokes. Hz on the left (from pitch~: 0 when it hears none), rec to start from 0 (its first write clears the freqt) or rec t (msg rec 8) to start from beat t, keeping what is before it, or ovr t (msg ovr 8) to write over only what the take covers, keeping what comes after and stop to end; a bang on the right (from onset~) begins a new note even on the same pitch, and a number there (onset~'s strength, 0…1) does too and gives the note that weight: its stroke is written heavier (as an event's :amp), and the quantizer reads it as the note's :vel, which helps find the beat and the bar (issue #106). In notes mode the contour of each voiced stretch is cut into notes where it settles on another pitch (a slide, a vibrato or an ornament that passes does not start notes; an onset on the right does), each a flat stroke at its median pitch (the freqt's tuning, or its score view, rounds it), written when the voice stops; in glide mode a stroke follows the pitch as it moves. Times are beats at bpm (60: one beat a second). Notes shorter than ms (80) are left out. It writes into the freqt at most every 2 seconds, each time one event of the history. Connect its outlet to the freqt's inlet.\"\n :usage \"transcribe [bpm] [notes|glide] [ms]\"\n :io [2 1]\n :size [124 30]\n :args [[:bpm 60] [:mode nil] [:min-ms 80]]\n\n ;; ── logic ──\n :receive (fn [{:keys [inlet value state args at]}]\n (let [state (or state {})]\n (cond\n ;; rec: a take from 0 (its first write clears the freqt); [:rec t] (msg rec 8): from beat t on,\n ;; keeping what is before t (punch-in)\n ;; ovr t (msg ovr 8): over what is there from beat t to where the take stops, keeping what comes after\n (or (#{:rec :ovr} value) (and (vector? value) (#{:rec :ovr} (first value))))\n (let [bpm (api/clamp (:bpm args) 10 400)\n ovr (or (= value :ovr) (and (vector? value) (= :ovr (first value))))\n from (or (when (vector? value) (let [x (second value)] (when (and (number? x) (>= x 0)) x)))\n (when ovr 0))]\n {:state (cond-> {:on true :t0 (- at (* (or from 0) (/ 60 bpm))) :bpm bpm :mode (mode-of args)\n :min-ms (api/clamp (:min-ms args) 0 2000) :done [] :count 0 :fresh true}\n from (assoc :from from)\n ovr (assoc :ovr true :cut-from from))\n :show {:on true :count 0}})\n (not (:on state)) {}\n (= value :stop)\n (let [t (beats state at)\n st (if (seq (:run state)) (settle state (or (:silent state) t) true) state)\n st (end-note st t)\n [st edit _] (flush st at true)]\n (cond-> {:state (assoc st :on false) :show {:on false :count (:count st)}}\n edit (assoc :out [[0 edit]])))\n (= value [:flush]) (result state (flush (settled (assoc state :pending false) (beats state at)) at false))\n ;; an onset (a bang, or its strength 0…1 from onset~'s second outlet): the note that sounds ends\n ;; here and a new one begins at its pitch, with that strength; in silence, the strength waits for\n ;; the next note to begin\n (and (= inlet 1) (= :notes (:mode state)))\n {:state (update state :marks (fnil conj []) [(beats state at) (when (number? value) (api/clamp value 0 1))])}\n (= inlet 1)\n (let [t (beats state at)\n state (if (number? value) (assoc state :acc [t (api/clamp value 0 1)]) state)]\n (if (:cur state)\n (let [hz (second (peek (get-in state [:cur :pts])))]\n (result state (flush (-> state (end-note t) (start-note t hz)) at false)))\n {:state state}))\n (and (= inlet 0) (number? value)) (result state (hear state value at))\n :else {})))\n\n ;; ── view ──\n :derive (fn [{:keys [shown]}]\n {:label (if (:on shown) (str \"● transcribing · \" (or (:count shown) 0)) (str \"transcribe · \" (or (:count shown) 0) \" notes\"))})\n :view [[:rect {:x 2 :y 2 :w 120 :h 26 :r 3 :stroke :grid :fill :bg}]\n [:text {:x 8 :y 19 :size 10 :fill :soft} :$label]]})\n"
60
+ "doc": "Writes what pitch~ hears into a freqt, as strokes. Hz on the left (from pitch~: 0 when it hears none), :rec to start from 0 (its first write clears the freqt) or :rec t (msg :rec 8) to start from beat t, keeping what is before it, or :ovr t (msg :ovr 8) to write over only what the take covers, keeping what comes after and :stop to end; a bang on the right (from onset~) begins a new note even on the same pitch, and a number there (onset~'s strength, 0…1) does too and gives the note that weight: its stroke is written heavier (as an event's :amp), and the quantizer reads it as the note's :vel, which helps find the beat and the bar (issue #106). In notes mode the contour of each voiced stretch is cut into notes where it settles on another pitch (a slide, a vibrato or an ornament that passes does not start notes; an onset on the right does), each a flat stroke at its median pitch (the freqt's tuning, or its score view, rounds it), written when the voice stops; in glide mode a stroke follows the pitch as it moves. Times are beats at bpm (60: one beat a second). Notes shorter than ms (80) are left out. It writes into the freqt at most every 2 seconds, each time one event of the history. Connect its outlet to the freqt's inlet.",
61
+ "source": ";; transcribe · what pitch~ hears, written as strokes of a freqt\n;; Part of @kairotic/analysis. Written against the Kairotic API (api/…). Pure logic: no audio of its own.\n(let [;; ── what it understands ──\n flush-every 2 ; seconds between two writes into the freqt (each write is one event of the history)\n max-points 64 ; points of a stroke in glide mode\n cents (fn [a b] (* 1200 (/ (js/Math.log (/ b a)) js/Math.LN2)))\n ;; the pitch a note holds longest: pitch~ only sends what changed, so a steady note is one point and\n ;; its attack many; each pitch weighs the time until the next one\n held (fn [pts t1]\n (let [ws (map (fn [[t f] [t2 _]] [f (max 0 (- t2 t))]) pts (concat (rest pts) [[t1 nil]]))\n total (reduce + (map second ws))\n sorted (sort-by first ws)]\n (if (pos? total)\n (loop [[[f w] & more] sorted acc 0]\n (let [acc (+ acc w)] (if (or (>= acc (/ total 2)) (empty? more)) f (recur more acc))))\n (second (last pts)))))\n mode-of (fn [args] (if (= :glide (:mode args)) :glide :notes))\n beats (fn [state at] (* (- at (:t0 state)) (/ (:bpm state) 60)))\n r3 (fn [x] (/ (js/Math.round (* x 1000)) 1000))\n ;; ── a note that ends: a stroke, if it lasted long enough ──\n stroke-of (fn [{:keys [pts]} t1 mode]\n (let [t0 (ffirst pts) hz (held pts t1)]\n (if (= mode :glide)\n (let [kept (reduce (fn [acc [t f]]\n (if (> (js/Math.abs (cents (second (peek acc)) f)) 10) (conj acc [t f]) acc))\n [(first pts)] (rest pts))\n kept (if (> (count kept) max-points)\n (vec (take-nth (js/Math.ceil (/ (count kept) max-points)) kept))\n kept)]\n (mapv (fn [[t f]] [(r3 t) (r3 f)]) (conj kept [t1 (second (peek pts))])))\n [[(r3 t0) (r3 hz)] [(r3 t1) (r3 hz)]])))\n end-note (fn [state t]\n (if-let [cur (:cur state)]\n (let [t0 (ffirst (:pts cur))\n long-enough (>= (/ (- t t0) (/ (:bpm state) 60)) (/ (:min-ms state) 1000))]\n (cond-> (dissoc state :cur)\n long-enough (update :done conj {:pts (stroke-of cur t (:mode state)) :acc (:acc cur)})\n long-enough (update :count inc)))\n state))\n ;; a note takes the strength of the onset that began it (a number on the right inlet, from onset~), if it\n ;; came at most 0.15 s before: onset~ often hears the attack before pitch~ hears the pitch\n start-note (fn [state t hz]\n (let [[at s] (:acc state) acc (when (and at (<= (- t at) (* 0.15 (/ (:bpm state) 60)))) s)]\n (cond-> (assoc state :cur (cond-> {:pts [[t hz]] :midi (api/hz->midi hz)} acc (assoc :acc acc)))\n acc (dissoc :acc))))\n ;; how a note with the strength of its attack is written: the weight of its stroke, as an event's :amp is\n ;; (0 … 1 → 0.4 … 2.8 times the usual); the quantizer reads it back as the note's :vel (issue #106)\n edit-of (fn [{:keys [pts acc]}] (if (number? acc) [:stroke pts {:w (/ (js/Math.round (* 100 (+ 0.4 (* 2.4 acc)))) 100)}] [:stroke pts]))\n ;; ── writing: the strokes that ended, as one [:batch …], at most every flush-every seconds ──\n flush (fn [state at force]\n (let [now (r3 (beats state at))]\n (cond\n ;; ovr at stop: what the take covered since the last write is overwritten, even by silence\n (and (empty? (:done state)) (not (and force (:ovr state) (> now (:cut-from state))))) [state nil nil]\n (or force (nil? (:sent state)) (>= (- at (:sent state)) flush-every))\n ;; the first write of a take clears the freqt (as part of the same event); a take over (ovr)\n ;; cuts what it covered since the last write, and keeps what comes after\n [(cond-> (assoc state :done [] :sent at :pending false :fresh false)\n (:ovr state) (assoc :cut-from now))\n [:batch (into (cond (:ovr state) [[:cut (:cut-from state) now]]\n (:fresh state) [(if (:from state) [:cut (:from state)] [:clear])]\n :else [])\n (mapv edit-of (:done state)))] nil]\n (:pending state) [state nil nil]\n :else [(assoc state :pending true) nil [(+ (:sent state) flush-every) 0 [:flush]]])))\n result (fn [state [st edit later]]\n (cond-> {:state st :show (select-keys st [:on :count])}\n edit (assoc :out [[0 edit]])\n later (assoc :later [later])))\n ;; ── notes mode: the contour of each voiced stretch, segmented (issue #106, measured on cante2midi) ──\n ;; What pitch~ sends while a voice sounds is kept as a run of points; when the voice stops (or the run is long)\n ;; the run is read as a contour every 10 ms, smoothed by a median of 100 ms (vibrato and jitter), and cut into\n ;; notes with hysteresis: a note goes on while the contour stays within a semitone of its pitch; a new one\n ;; begins only where the contour, away from it, has held still (within 1.1 semitones) for 70 ms, so a slide\n ;; or an ornament that passes does not start notes. An onset (inlet 1) cuts there too. Each note is written\n ;; at the median pitch of its frames. Silences of 30 ms at most are bridged.\n median (fn [xs] (let [v (vec (sort xs))] (nth v (quot (count v) 2))))\n segment (fn [run marks t-end k minlen]\n (let [dt (* 0.01 k) w 5 band 1 stab 1.1 hold-n 7\n ts (vec (range (ffirst run) t-end dt))\n ;; the contour on the grid: each frame, the last point at or before it (as MIDI)\n raw (loop [i 0 [p & more :as ps] run cur nil out (transient [])]\n (if (>= i (count ts))\n (persistent! out)\n (let [t (ts i)]\n (if (and p (<= (first p) (+ t 1e-9)))\n (recur i more (api/hz->midi (second p)) out)\n (recur (inc i) ps cur (conj! out cur))))))\n n (count raw)\n sm (mapv (fn [i] (median (subvec raw (max 0 (- i w)) (min n (+ i w 1))))) (range n))\n cut? (fn [i] (let [a (if (pos? i) (ts (dec i)) (- (ts 0) dt)) b (ts i)] (some (fn [[mt]] (and (> mt a) (<= mt b))) marks)))\n note (fn [s e ms] {:s s :e e :ms ms})\n notes (loop [i 0 cur nil pend [] out []]\n (if (>= i n)\n (if cur (conj out (assoc cur :e t-end)) out)\n (let [t (ts i) m (sm i)]\n (cond\n (nil? cur) (recur (inc i) (note t (+ t dt) [m]) [] out)\n ;; (not while the contour is away from the note and not settled yet: there the contour\n ;; decides; measured on cante2midi with onsets from its Bark bands)\n (and (empty? pend) (cut? i) (> t (:s cur))) (recur (inc i) (note t (+ t dt) [m]) [] (conj out (assoc cur :e t)))\n (<= (js/Math.abs (- m (median (take-last 24 (:ms cur))))) band)\n (recur (inc i) (assoc cur :e (+ t dt) :ms (into (conj (:ms cur) m) (map second pend))) [] out)\n :else\n (let [pend (conj pend [t m]) tail (take-last hold-n pend) tm (map second tail)]\n (if (and (>= (count pend) hold-n) (< (- (apply max tm) (apply min tm)) stab))\n ;; the new note begins where the contour left the old one; its pitch is that of where it settled\n (recur (inc i) (note (ffirst pend) (+ t dt) (vec tm)) [] (conj out (assoc cur :e (ffirst pend))))\n (recur (inc i) cur pend out)))))))]\n (->> notes\n (filter (fn [{:keys [s e]}] (>= (- e s) minlen)))\n (mapv (fn [{:keys [s e ms]}]\n (let [hz (api/midi->hz (median ms))\n ;; the strength of an onset heard at most 0.15 s before the note (or just after its start)\n acc (some (fn [[mt x]] (when (and (number? x) (<= (- s (* 0.15 k)) mt (+ s (* 0.05 k)))) x)) marks)]\n (cond-> {:pts [[(r3 s) (r3 hz)] [(r3 e) (r3 hz)]]} acc (assoc :acc acc))))))))\n ;; the run segmented into :done: all of it (the voice stopped at t-end), or all but its last note (a long run\n ;; still sounding: that note goes on in the run)\n settle (fn [state t-end all?]\n (let [run (:run state) k (/ (:bpm state) 60) minlen (* k (/ (:min-ms state) 1000))]\n (if (or (empty? run) (<= t-end (ffirst run)))\n (dissoc state :run :silent)\n (let [ns (segment run (:marks state) t-end k minlen)\n [keep-ns rest-from] (if (or all? (< (count ns) 2)) [ns nil] [(pop ns) (ffirst (:pts (peek ns)))])\n state (-> state (update :done into keep-ns) (update :count + (count keep-ns)))]\n (if rest-from\n (let [before (last (filter (fn [[t]] (<= t rest-from)) run))]\n (assoc state :run (into [[rest-from (second before)]] (filter (fn [[t]] (> t rest-from)) run))\n :marks (filterv (fn [[t]] (>= t (- rest-from (* 0.15 k)))) (:marks state))))\n (-> state (dissoc :run :silent) (assoc :marks (filterv (fn [[t]] (> t t-end)) (:marks state)))))))))\n ;; a silence that lasted more than 30 ms: the run ends where it began\n settled (fn [state t]\n (if (and (:silent state) (> (- t (:silent state)) (* 0.03 (/ (:bpm state) 60))))\n (settle state (:silent state) true)\n state))\n hear-notes (fn [state hz t]\n (let [k (/ (:bpm state) 60)\n state (settled state t)]\n (cond\n (not (pos? hz)) (if (and (seq (:run state)) (nil? (:silent state))) (assoc state :silent t) state)\n :else (let [state (-> state (dissoc :silent) (update :run (fnil conj []) [t hz]))]\n ;; a run of 8 s and more: what is settled is written, the last note goes on\n (if (> (- t (ffirst (:run state))) (* 8 k)) (settle state t false) state)))))\n ;; ── one value of pitch~ (Hz, 0 when it hears none) ──\n hear (fn [state hz at]\n (let [t (beats state at)\n cur (:cur state)\n state (cond\n (= :notes (:mode state)) (hear-notes state hz t)\n (not (pos? hz)) (end-note state t)\n (nil? cur) (start-note state t hz)\n :else (update-in state [:cur :pts] conj [t hz]))\n [st edit later] (flush state at false)]\n ;; the voice stopped: a look again in a little while, to write its notes even if nothing else arrives\n [st edit (or later (when (and (:silent st) (not (:pending st))) [(+ at 0.05) 0 [:flush]]))]))]\n {;; ── about ──\n :class \"transcribe\"\n :license \"MPL-2.0\"\n :doc \"Writes what pitch~ hears into a freqt, as strokes. Hz on the left (from pitch~: 0 when it hears none), :rec to start from 0 (its first write clears the freqt) or :rec t (msg :rec 8) to start from beat t, keeping what is before it, or :ovr t (msg :ovr 8) to write over only what the take covers, keeping what comes after and :stop to end; a bang on the right (from onset~) begins a new note even on the same pitch, and a number there (onset~'s strength, 0…1) does too and gives the note that weight: its stroke is written heavier (as an event's :amp), and the quantizer reads it as the note's :vel, which helps find the beat and the bar (issue #106). In notes mode the contour of each voiced stretch is cut into notes where it settles on another pitch (a slide, a vibrato or an ornament that passes does not start notes; an onset on the right does), each a flat stroke at its median pitch (the freqt's tuning, or its score view, rounds it), written when the voice stops; in glide mode a stroke follows the pitch as it moves. Times are beats at bpm (60: one beat a second). Notes shorter than ms (80) are left out. It writes into the freqt at most every 2 seconds, each time one event of the history. Connect its outlet to the freqt's inlet.\"\n :usage \"transcribe [bpm] [notes|glide] [ms]\"\n :io [2 1]\n :size [124 30]\n :args [[:bpm 60] [:mode nil] [:min-ms 80]]\n\n ;; ── logic ──\n :receive (fn [{:keys [inlet value state args at]}]\n (let [state (or state {})]\n (cond\n ;; rec: a take from 0 (its first write clears the freqt); [:rec t] (msg :rec 8): from beat t on,\n ;; keeping what is before t (punch-in)\n ;; ovr t (msg :ovr 8): over what is there from beat t to where the take stops, keeping what comes after\n (or (#{:rec :ovr} value) (and (vector? value) (#{:rec :ovr} (first value))))\n (let [bpm (api/clamp (:bpm args) 10 400)\n ovr (or (= value :ovr) (and (vector? value) (= :ovr (first value))))\n from (or (when (vector? value) (let [x (second value)] (when (and (number? x) (>= x 0)) x)))\n (when ovr 0))]\n {:state (cond-> {:on true :t0 (- at (* (or from 0) (/ 60 bpm))) :bpm bpm :mode (mode-of args)\n :min-ms (api/clamp (:min-ms args) 0 2000) :done [] :count 0 :fresh true}\n from (assoc :from from)\n ovr (assoc :ovr true :cut-from from))\n :show {:on true :count 0}})\n (not (:on state)) {}\n (= value :stop)\n (let [t (beats state at)\n st (if (seq (:run state)) (settle state (or (:silent state) t) true) state)\n st (end-note st t)\n [st edit _] (flush st at true)]\n (cond-> {:state (assoc st :on false) :show {:on false :count (:count st)}}\n edit (assoc :out [[0 edit]])))\n (= value [:flush]) (result state (flush (settled (assoc state :pending false) (beats state at)) at false))\n ;; an onset (a bang, or its strength 0…1 from onset~'s second outlet): the note that sounds ends\n ;; here and a new one begins at its pitch, with that strength; in silence, the strength waits for\n ;; the next note to begin\n (and (= inlet 1) (= :notes (:mode state)))\n {:state (update state :marks (fnil conj []) [(beats state at) (when (number? value) (api/clamp value 0 1))])}\n (= inlet 1)\n (let [t (beats state at)\n state (if (number? value) (assoc state :acc [t (api/clamp value 0 1)]) state)]\n (if (:cur state)\n (let [hz (second (peek (get-in state [:cur :pts])))]\n (result state (flush (-> state (end-note t) (start-note t hz)) at false)))\n {:state state}))\n (and (= inlet 0) (number? value)) (result state (hear state value at))\n :else {})))\n\n ;; ── view ──\n :derive (fn [{:keys [shown]}]\n {:label (if (:on shown) (str \"● transcribing · \" (or (:count shown) 0)) (str \"transcribe · \" (or (:count shown) 0) \" notes\"))})\n :view [[:rect {:x 2 :y 2 :w 120 :h 26 :r 3 :stroke :grid :fill :bg}]\n [:text {:x 8 :y 19 :size 10 :fill :soft} :$label]]})\n"
62
62
  }
63
63
  ]
64
64
  };
package/dist/monitor.mjs CHANGED
@@ -1,7 +1,7 @@
1
1
  // Generated by packages/build.mjs from src/monitor.cljs
2
2
  export default {
3
3
  "class": "monitor",
4
- "version": "0.7.2",
4
+ "version": "0.7.5",
5
5
  "author": "bonuoq",
6
6
  "license": "MPL-2.0",
7
7
  "doc": "Draws the messages that arrive over the last seconds (6 by default), one lane per inlet: a lane for each word written after the seconds, which names it (monitor 6 rms centroid chroma has 3 inlets), or one lane without words. A number is drawn as a line in the range it has had in the window, a list of numbers as a strip with one row per element (chroma, mfcc), anything else (a bang, a word) as a mark. Time is that of the messages: the right edge is the last one. Sends nothing.",
package/dist/onset.mjs CHANGED
@@ -1,7 +1,7 @@
1
1
  // Generated by packages/build.mjs from src/onset.cljs
2
2
  export default {
3
3
  "class": "onset~",
4
- "version": "0.7.2",
4
+ "version": "0.7.5",
5
5
  "author": "bonuoq",
6
6
  "license": "MPL-2.0",
7
7
  "doc": "The beginnings of sounds in the signal on its left — a note, a stroke, a syllable —: a bang on the left for each, and how strong it was (the growth of its spectrum, 0…1) on the second outlet; on the third, that growth for every frame (the onset strength envelope, for beat tracking: issue #106). threshold (3) is how far above the usual change of the last frames an onset must rise, while the energy does not fall (a sound cut at once is not an onset); ms (80) the least time between two. Every 512 samples (about 11 ms), on frames of 2048. An onset is reported with the frame it is first seen in: measured on sines, plucked tones and noise bursts that start at a known sample, 6 ms after it as a median (0.7 to 10.7 ms: one hop), and 11 ms for a new note over a held one. [:threshold x] and [:ms y] change them while it listens, and a [:segment …] from form~ makes it forget the usual change of the section before (so a quiet section after a loud one is not deaf). Into a counter, a metro that follows, a sample's rec, or the right inlet of transcribe (its strength, as the weight of the note).",
package/dist/pitch.mjs CHANGED
@@ -1,7 +1,7 @@
1
1
  // Generated by packages/build.mjs from src/pitch.cljs
2
2
  export default {
3
3
  "class": "pitch~",
4
- "version": "0.7.2",
4
+ "version": "0.7.5",
5
5
  "author": "bonuoq",
6
6
  "license": "MPL-2.0",
7
7
  "doc": "The pitch of the signal on its left, by YIN, every 1024 samples (about 21 ms): its frequency in Hz on the left, when it finds one between lo and hi (60 and 1500 by default), and 0 once when it no longer does (silence, noise, a chord); how clearly it is periodic (0…1) on the right. It sends only what changed. threshold (0.15) is how strict it is: lower finds fewer pitches, surer ones. Into ftom (Hz → MIDI pitch), a synth~, or a faust~.",
package/dist/realize.mjs CHANGED
@@ -1,9 +1,9 @@
1
1
  // Generated by packages/build.mjs from src/realize.cljs
2
2
  export default {
3
3
  "class": "realize~",
4
- "version": "0.7.2",
4
+ "version": "0.7.5",
5
5
  "author": "bonuoq",
6
6
  "license": "MPL-2.0",
7
7
  "doc": "Plays a faust~ from descriptors, as a score of them: loudness (dB, or a level 0…1 as features~ rms), brightness (Hz, the spectral centroid), noisiness (0…1, the spectral flatness) and pitch (Hz as pitch~ sends it, 0 for none; MIDI with `midi`). Its left outlet goes to the faust~: the faust~'s pitch or freq slider takes the pitch, its gain slider the loudness, and its other sliders are set to the settings of that faust~ that sound nearest to the brightness and noisiness written. It learns them first by playing steady settings of the faust~ offline and listening (300 by default), when the first values arrive or on learn; learn again after changing the faust~'s code. With features~ of the faust~'s sound on its last three inlets (rms, centroid, flatness) it listens to what it plays and asks for more where it falls short. On the right, how far the nearest setting is from what is written (above 1: the faust~ cannot sound like that).",
8
- "source": ";; realize~ · descriptors written → the sliders of a faust~ that sound like them (discussion #114, Study 37)\n;; Part of @kairotic/analysis. Written against the Kairotic API (api/…).\n;; What arrives is a frame of a score: a loudness, a brightness (the spectral centroid, Hz), a noisiness (the\n;; spectral flatness) and a pitch — what features~ and pitch~ hear, so another sound can drive it, or curves\n;; drawn on canvases read back (reader~ → snapshot~). What leaves is [:slider value] for the faust~ on its\n;; left outlet: its `pitch` (MIDI) or `freq` (Hz) slider takes the pitch, its `gain` (dB or a level) the\n;; loudness, and its other sliders, the timbre, are set to the blend of the settings of that faust~ that come\n;; nearest to the brightness and noisiness written (where a pitch is written, among the settings in which YIN\n;; would hear it). To know them, it learns the faust~ first (the task :learn: steady settings played offline\n;; and heard, @kairotic/faust-units' notation.mjs), when the first frame arrives or on `learn`.\n;; It can also listen to what it plays: features~ of the faust~'s sound on its last three inlets (loudness,\n;; brightness, noisiness heard). Then it asks for more where what is heard falls short of what is written,\n;; little by little (a correction that grows while the difference lasts, within limits), as the study's\n;; passes did offline.\n(let [meyda-url \"npm:meyda@5.6.3/dist/web/meyda.min.js\"\n origins [\"npm:@kairotic/faust-units@0.1.5/dist/notation.mjs\" \"kairotic/packages/faust-units/notation.mjs\"]\n silent -60 ; dB: below it nothing is written (a microphone in a room is often near -50)\n every 0.04 ; seconds: at most one new setting this often\n ;; listening: how much of each difference heard goes into the correction, and how far it may go\n rate 0.08\n leak 0.995 ; what it corrected fades unless it is still needed\n hold 0.25 ; seconds a sound has lasted before it is corrected (not its attack)\n limits {:loud 18 :lb 2 :nz 0.4}\n bank-size (fn [args] (js/Math.round (api/clamp (or (first (filter number? args)) 300) 40 2000)))\n midi-in? (fn [args] (some #{:midi} args))\n log2 (fn [x] (/ (js/Math.log x) js/Math.LN2))\n ;; a loudness as dB: a level (0 < v ≤ 1, as features~ rms) or dB already (≤ 0); 0 is silence\n db-of (fn [v] (cond (not (number? v)) nil (zero? v) -100 (and (pos? v) (<= v 1)) (api/db v) :else v))\n ;; the nearest settings of the bank to what is asked, blended: {:u [...] :loud0 dB :d distance}. The search\n ;; itself is the bank's query (notation.mjs: plain loops over typed arrays), asked for every frame it plays\n nearest (fn [bank {:keys [pitch lb nz voiced]}]\n (let [r ((:query bank) pitch lb nz (boolean voiced))]\n {:loud0 (aget r 0) :d (aget r 1) :u (vec (.slice r 2))}))\n ;; the values for the faust~, and only those that changed: [[0 [:cutoff 1800]] …]\n settle (fn [st at]\n (let [{:keys [bank]} st\n ;; a lane that never arrived keeps the middle of what the faust~ can do\n want (merge {:lb (first (:mid bank)) :nz (second (:mid bank))} (:want st))\n ;; what is asked of the bank: what is written, corrected by what was heard\n {cl :loud clb :lb cnz :nz} (:corr st)\n [lb-lo lb-hi] (or (:lbr bank) [0 20])\n ask (cond-> want\n (and cl (:loud want)) (update :loud + cl)\n clb (assoc :lb (api/clamp (+ (:lb want) clb) lb-lo lb-hi))\n cnz (assoc :nz (api/clamp (+ (:nz want) cnz) 0 1)))\n loud (:loud want)]\n (if (and (nil? loud) (nil? (:pitch want)) (nil? (get-in st [:want :lb])) (nil? (get-in st [:want :nz])))\n {:state (assoc st :t at)}\n (let [quiet? (and loud (< loud silent))\n {:keys [u loud0 d]} (if quiet? {:u (:u st) :loud0 0 :d (:reach st)} (nearest bank ask))\n ;; the timbre glides to where it goes (half the way every 40 ms), and keeps\n ;; gliding when nothing more arrives (a value written once, a canvas held)\n goal u\n u (if (and (:u st) (not quiet?)) (mapv (fn [a b] (+ a (* 0.5 (- b a)))) (:u st) u) u)\n moving? (and goal u (not quiet?) (some (fn [[a b]] (> (js/Math.abs (- a b)) 0.004)) (map vector u goal)))\n g (:gain bank) p (:pitch bank)\n gain-v (when (and g loud) (let [dB (if quiet? -100 (- (:loud ask) loud0))]\n (if (:db g) (api/clamp dB (:min g) (:max g))\n (api/clamp (js/Math.pow 10 (/ dB 20)) (:min g) (:max g)))))\n pitch-v (when (and p (:pitch want))\n (api/clamp (if (= \"hz\" (:kind p)) (api/midi->hz (:pitch want)) (:pitch want)) (:min p) (:max p)))\n vals (cond-> (if (and u (not quiet?))\n (into {} (map (fn [s x] [(:label s) (api/from-unit x (:min s) (:max s) (:log s))]) (:timbre bank) u))\n {})\n gain-v (assoc (:label g) gain-v)\n pitch-v (assoc (:label p) pitch-v))\n span (fn [label] (let [s (or (first (filter #(= label (:label %)) (:timbre bank))) (when (= label (:label g)) g) p)]\n (js/Math.abs (- (:max s) (:min s)))))\n changed (filter (fn [[k v]] (let [old (get (:sent st) k)]\n (or (nil? old) (> (js/Math.abs (- v old)) (* 0.002 (span k))))))\n vals)\n reach (when d (/ (js/Math.round (* 100 d)) 100))]\n {:state (-> st (assoc :t at :u (or u (:u st)) :reach reach :dirty (boolean moving?)\n :loud0 (if quiet? (:loud0 st) loud0)\n :tick (or (:tick st) (boolean moving?)))\n (update :sent merge (into {} changed)))\n :later (when (and moving? (not (:tick st))) [[(+ at every) 0 :realize-tick]])\n :out (cond-> (mapv (fn [[k v]] [0 [(keyword k) (api/nice v)]]) changed)\n (and reach (not= reach (:reach st))) (conj [1 reach]))\n :show {:status :ready :faust (:faust bank) :n (count (:entries bank)) :want want :reach reach\n :listening (some? (:corr st))}}))))\n ;; each request is numbered, so the answer to an older one is dropped\n learn (fn [st args]\n (let [k (inc (:k st 0))]\n {:state (assoc st :learning true :error nil :k k :corr nil)\n :show {:status :learning}\n :effects [[:learn {:n (bank-size args) :k k}]]}))]\n {;; ── about ──\n :class \"realize~\"\n :license \"MPL-2.0\"\n :doc \"Plays a faust~ from descriptors, as a score of them: loudness (dB, or a level 0…1 as features~ rms), brightness (Hz, the spectral centroid), noisiness (0…1, the spectral flatness) and pitch (Hz as pitch~ sends it, 0 for none; MIDI with `midi`). Its left outlet goes to the faust~: the faust~'s pitch or freq slider takes the pitch, its gain slider the loudness, and its other sliders are set to the settings of that faust~ that sound nearest to the brightness and noisiness written. It learns them first by playing steady settings of the faust~ offline and listening (300 by default), when the first values arrive or on learn; learn again after changing the faust~'s code. With features~ of the faust~'s sound on its last three inlets (rms, centroid, flatness) it listens to what it plays and asks for more where it falls short. On the right, how far the nearest setting is from what is written (above 1: the faust~ cannot sound like that).\"\n :usage \"realize~ [settings] [midi]\"\n :io [7 2]\n :size [236 46]\n\n ;; ── tasks: learning the faust~ it drives, outside its pure logic ──\n :tasks {:learn (fn [{:keys [n]} box]\n (let [f (first (filter (fn [w] (and (= \"faust~\" (:class w)) (zero? (:outlet w)))) (:wired box)))\n src (str (get-in f [:data :source]))\n ;; the compilation of its code, waited for (a faust~ just written is still compiling)\n compiled (fn compiled [tries]\n (let [r (api/compiled \"faust\" src)]\n (cond (= :ok (:status r)) (js/Promise.resolve r)\n (and (#{:compiling :empty} (:status r)) (pos? tries))\n (js/Promise. (fn [ok] (js/setTimeout (fn [] (ok nil)) 250)))\n :else (js/Promise.reject (js/Error. (str \"the faust~ does not compile: \" (or (:error r) \"no code\")))))))\n wait (fn wait [tries]\n (.then (compiled tries) (fn [r] (if r r (wait (dec tries))))))]\n (if-not f\n (js/Promise.reject (js/Error. \"connect its left outlet to a faust~: it learns the one it plays\"))\n (-> (wait 120)\n (.then (fn [r]\n (when (get-in r [:info :poly])\n (throw (js/Error. \"it plays a faust~ of one voice (this one declares nvoices)\")))\n (let [fac (.-factory (:value r))]\n (-> (js/Promise.all #js [(api/load-script meyda-url) (api/import-module origins)])\n (.then (fn [[_ m]]\n (.then (.learn m #js {:module (or (.-module fac) (.-code fac))\n :json (js/JSON.parse (.-json fac))\n :Meyda (.-Meyda js/window) :n n})\n (fn [b] [b (.index m b)]))))))))\n (.then (fn [[bjs query]]\n ;; the settings stay a JS array (the query reads them); the rest as data\n (let [es (.-entries bjs)\n col (fn [j] (map (fn [e] (aget e j)) (array-seq es)))\n avg (fn [j] (/ (reduce + (col j)) (.-length es)))]\n (-> (js->clj #js {:pitch (.-pitch bjs) :gain (.-gain bjs) :timbre (.-timbre bjs) :sd (.-sd bjs)}\n :keywordize-keys true)\n (assoc :query query :entries es :faust (name (:id f)) :mid [(avg 1) (avg 2)]\n :lbr [(reduce min (col 1)) (reduce max (col 1))])))))))))}\n\n ;; ── logic ──\n :receive (fn [{:keys [inlet value state args at]}]\n (let [st (or state {})]\n (cond\n ;; what learning found\n (= inlet :task)\n (cond\n (not= (:k (:arg value)) (:k st)) {}\n (:error value) {:state (assoc st :learning false :error (:error value))\n :show {:status :error :error (:error value)}}\n :else (let [st (assoc st :learning false :bank (:value value) :sent {} :u nil)]\n (settle st (or at (:t st) 0))))\n\n (#{:learn \"learn\" :bang} value) (learn st args)\n\n ;; a moment later: the last frame that arrived while it waited\n (= value :realize-tick) (if (and (:bank st) (:dirty st)) (settle (assoc st :tick false) at) {:state (assoc st :tick false)})\n\n ;; what is heard in what it plays (features~ of the faust~): the correction grows by a part of\n ;; the difference, while something is written and sounds\n (and (#{4 5 6} inlet) (number? value))\n (let [heard (case inlet 4 (db-of value) 5 (when (pos? value) (log2 value)) 6 value)\n k ({4 :loud 5 :lb 6 :nz} inlet)\n want (:want st)\n st (cond-> st (= k :loud) (assoc :heard-loud heard))\n sounding? (and (:bank st) (number? (:loud want)) (> (:loud want) silent)\n (> (or (:heard-loud st) -100) silent))]\n (if (and sounding? (number? heard) (number? (get want k)) (:since st) (>= (- at (:since st)) hold))\n ;; within what the faust~ can give (the bank's brightness, noisiness 0…1, the gain slider's\n ;; range): a correction that cannot be played does not grow, so it does not wind up\n (let [bank (:bank st) w (get want k) g (:gain bank) [lb-lo lb-hi] (or (:lbr bank) [0 20])\n [lo hi] (case k\n :lb [(- lb-lo w) (- lb-hi w)]\n :nz [(- w) (- 1 w)]\n :loud (if (and g (:db g) (:loud0 st))\n [(+ (:min g) (:loud0 st) (- w)) (+ (:max g) (:loud0 st) (- w))]\n [(- (limits k)) (limits k)]))\n c (-> (+ (* leak (get-in st [:corr k] 0)) (* rate (- w heard)))\n (api/clamp (max lo (- (limits k))) (min hi (limits k))))\n st (-> st (assoc-in [:corr k] c) (assoc :dirty true))]\n (if (or (nil? (:t st)) (>= (- at (:t st)) every))\n (settle st at)\n (if (:tick st) {:state st} {:state (assoc st :tick true) :later [[(+ (:t st) every) 0 :realize-tick]]})))\n {:state st}))\n\n (and (integer? inlet) (number? value))\n (let [midi (when (and (= inlet 3) (pos? value)) (if (midi-in? args) value (api/hz->midi value)))\n want (case inlet\n 0 (assoc (:want st) :loud (db-of value))\n 1 (if (pos? value) (assoc (:want st) :lb (log2 value)) (:want st))\n 2 (assoc (:want st) :nz (api/clamp value 0 1))\n 3 (if midi (assoc (:want st) :pitch midi :voiced true) (assoc (:want st) :voiced false))\n (:want st))\n ;; since when what is written sounds (its attack is not corrected)\n st (cond-> (assoc st :want want :dirty true)\n (= inlet 0) (assoc :since (if (> (or (:loud want) -100) silent) (or (:since st) at) nil)))]\n (cond\n (:bank st) (if (or (nil? (:t st)) (>= (- at (:t st)) every))\n (settle st at)\n (if (:tick st)\n {:state st}\n {:state (assoc st :tick true) :later [[(+ (:t st) every) 0 :realize-tick]]}))\n (or (:learning st) (:error st)) {:state st}\n :else (learn st args)))\n\n :else {})))\n\n ;; ── view: what it learnt, what is written now, and how far the faust~ is from it ──\n :draw (fn [{:keys [shown w h]}]\n (let [{:keys [status faust n want reach error]} shown\n line1 (case status\n :learning \"learning the faust~ it plays…\"\n :ready (str \"plays \" faust \" · \" n \" settings heard\" (when (:listening shown) \" · listening\"))\n :error (str \"✗ \" error)\n \"waits for a faust~ and a score\")\n fmt (fn [x d] (if (number? x) (.toFixed x d) \"–\"))\n line2 (when (= status :ready)\n (str (fmt (:loud want) 0) \" dB · \"\n (if (:lb want) (str (js/Math.round (js/Math.pow 2 (:lb want))) \" Hz\") \"–\") \" · \"\n (fmt (:nz want) 2)\n (when (and (:voiced want) (:pitch want)) (str \" · \" (api/midi-name (js/Math.round (:pitch want)))))))\n bw (- w 16) far (when (number? reach) (api/clamp (/ reach 2) 0 1))]\n (cond-> [[:rect {:x 2 :y 2 :w (- w 4) :h (- h 4) :r 4 :stroke :grid :fill :bg}]\n [:text {:x 8 :y 15 :size 9 :fill (if (= status :error) :kairos :ink)} line1]]\n line2 (conj [:text {:x 8 :y 28 :size 9 :fill :soft} line2])\n far (conj [:rect {:x 8 :y 35 :w bw :h 4 :r 2 :fill :grid :stroke nil}]\n [:rect {:x 8 :y 35 :w (max 2 (* bw far)) :h 4 :r 2 :fill (if (> reach 1) :kairos :accent) :stroke nil}]))))})\n"
8
+ "source": ";; realize~ · descriptors written → the sliders of a faust~ that sound like them (discussion #114, Study 37)\n;; Part of @kairotic/analysis. Written against the Kairotic API (api/…).\n;; What arrives is a frame of a score: a loudness, a brightness (the spectral centroid, Hz), a noisiness (the\n;; spectral flatness) and a pitch — what features~ and pitch~ hear, so another sound can drive it, or curves\n;; drawn on canvases read back (reader~ → snapshot~). What leaves is [:slider value] for the faust~ on its\n;; left outlet: its `pitch` (MIDI) or `freq` (Hz) slider takes the pitch, its `gain` (dB or a level) the\n;; loudness, and its other sliders, the timbre, are set to the blend of the settings of that faust~ that come\n;; nearest to the brightness and noisiness written (where a pitch is written, among the settings in which YIN\n;; would hear it). To know them, it learns the faust~ first (the task :learn: steady settings played offline\n;; and heard, @kairotic/faust-units' notation.mjs), when the first frame arrives or on `learn`.\n;; It can also listen to what it plays: features~ of the faust~'s sound on its last three inlets (loudness,\n;; brightness, noisiness heard). Then it asks for more where what is heard falls short of what is written,\n;; little by little (a correction that grows while the difference lasts, within limits), as the study's\n;; passes did offline.\n(let [meyda-url \"npm:meyda@5.6.3/dist/web/meyda.min.js\"\n origins [\"npm:@kairotic/faust-units@0.1.5/dist/notation.mjs\" \"kairotic/packages/faust-units/notation.mjs\"]\n silent -60 ; dB: below it nothing is written (a microphone in a room is often near -50)\n every 0.04 ; seconds: at most one new setting this often\n ;; listening: how much of each difference heard goes into the correction, and how far it may go\n rate 0.08\n leak 0.995 ; what it corrected fades unless it is still needed\n hold 0.25 ; seconds a sound has lasted before it is corrected (not its attack)\n limits {:loud 18 :lb 2 :nz 0.4}\n bank-size (fn [args] (js/Math.round (api/clamp (or (first (filter number? args)) 300) 40 2000)))\n midi-in? (fn [args] (some #{:midi} args))\n log2 (fn [x] (/ (js/Math.log x) js/Math.LN2))\n ;; a loudness as dB: a level (0 < v ≤ 1, as features~ rms) or dB already (≤ 0); 0 is silence\n db-of (fn [v] (cond (not (number? v)) nil (zero? v) -100 (and (pos? v) (<= v 1)) (api/db v) :else v))\n ;; the nearest settings of the bank to what is asked, blended: {:u [...] :loud0 dB :d distance}. The search\n ;; itself is the bank's query (notation.mjs: plain loops over typed arrays), asked for every frame it plays\n nearest (fn [bank {:keys [pitch lb nz voiced]}]\n (let [r ((:query bank) pitch lb nz (boolean voiced))]\n {:loud0 (aget r 0) :d (aget r 1) :u (vec (.slice r 2))}))\n ;; the values for the faust~, and only those that changed: [[0 [:cutoff 1800]] …]\n settle (fn [st at]\n (let [{:keys [bank]} st\n ;; a lane that never arrived keeps the middle of what the faust~ can do\n want (merge {:lb (first (:mid bank)) :nz (second (:mid bank))} (:want st))\n ;; what is asked of the bank: what is written, corrected by what was heard\n {cl :loud clb :lb cnz :nz} (:corr st)\n [lb-lo lb-hi] (or (:lbr bank) [0 20])\n ask (cond-> want\n (and cl (:loud want)) (update :loud + cl)\n clb (assoc :lb (api/clamp (+ (:lb want) clb) lb-lo lb-hi))\n cnz (assoc :nz (api/clamp (+ (:nz want) cnz) 0 1)))\n loud (:loud want)]\n (if (and (nil? loud) (nil? (:pitch want)) (nil? (get-in st [:want :lb])) (nil? (get-in st [:want :nz])))\n {:state (assoc st :t at)}\n (let [quiet? (and loud (< loud silent))\n {:keys [u loud0 d]} (if quiet? {:u (:u st) :loud0 0 :d (:reach st)} (nearest bank ask))\n ;; the timbre glides to where it goes (half the way every 40 ms), and keeps\n ;; gliding when nothing more arrives (a value written once, a canvas held)\n goal u\n u (if (and (:u st) (not quiet?)) (mapv (fn [a b] (+ a (* 0.5 (- b a)))) (:u st) u) u)\n moving? (and goal u (not quiet?) (some (fn [[a b]] (> (js/Math.abs (- a b)) 0.004)) (map vector u goal)))\n g (:gain bank) p (:pitch bank)\n gain-v (when (and g loud) (let [dB (if quiet? -100 (- (:loud ask) loud0))]\n (if (:db g) (api/clamp dB (:min g) (:max g))\n (api/clamp (js/Math.pow 10 (/ dB 20)) (:min g) (:max g)))))\n pitch-v (when (and p (:pitch want))\n (api/clamp (if (= \"hz\" (:kind p)) (api/midi->hz (:pitch want)) (:pitch want)) (:min p) (:max p)))\n vals (cond-> (if (and u (not quiet?))\n (into {} (map (fn [s x] [(:label s) (api/from-unit x (:min s) (:max s) (:log s))]) (:timbre bank) u))\n {})\n gain-v (assoc (:label g) gain-v)\n pitch-v (assoc (:label p) pitch-v))\n span (fn [label] (let [s (or (first (filter #(= label (:label %)) (:timbre bank))) (when (= label (:label g)) g) p)]\n (js/Math.abs (- (:max s) (:min s)))))\n changed (filter (fn [[k v]] (let [old (get (:sent st) k)]\n (or (nil? old) (> (js/Math.abs (- v old)) (* 0.002 (span k))))))\n vals)\n reach (when d (/ (js/Math.round (* 100 d)) 100))]\n {:state (-> st (assoc :t at :u (or u (:u st)) :reach reach :dirty (boolean moving?)\n :loud0 (if quiet? (:loud0 st) loud0)\n :tick (or (:tick st) (boolean moving?)))\n (update :sent merge (into {} changed)))\n :later (when (and moving? (not (:tick st))) [[(+ at every) 0 :realize-tick]])\n :out (cond-> (mapv (fn [[k v]] [0 [(keyword k) (api/nice v)]]) changed)\n (and reach (not= reach (:reach st))) (conj [1 reach]))\n :show {:status :ready :faust (:faust bank) :n (count (:entries bank)) :want want :reach reach\n :listening (some? (:corr st))}}))))\n ;; each request is numbered, so the answer to an older one is dropped\n learn (fn [st args]\n (let [k (inc (:k st 0))]\n {:state (assoc st :learning true :error nil :k k :corr nil)\n :show {:status :learning}\n :effects [[:learn {:n (bank-size args) :k k}]]}))]\n {;; ── about ──\n :class \"realize~\"\n :license \"MPL-2.0\"\n :doc \"Plays a faust~ from descriptors, as a score of them: loudness (dB, or a level 0…1 as features~ rms), brightness (Hz, the spectral centroid), noisiness (0…1, the spectral flatness) and pitch (Hz as pitch~ sends it, 0 for none; MIDI with `midi`). Its left outlet goes to the faust~: the faust~'s pitch or freq slider takes the pitch, its gain slider the loudness, and its other sliders are set to the settings of that faust~ that sound nearest to the brightness and noisiness written. It learns them first by playing steady settings of the faust~ offline and listening (300 by default), when the first values arrive or on learn; learn again after changing the faust~'s code. With features~ of the faust~'s sound on its last three inlets (rms, centroid, flatness) it listens to what it plays and asks for more where it falls short. On the right, how far the nearest setting is from what is written (above 1: the faust~ cannot sound like that).\"\n :usage \"realize~ [settings] [midi]\"\n :io [7 2]\n :size [236 46]\n\n ;; ── tasks: learning the faust~ it drives, outside its pure logic ──\n :tasks {:learn (fn [{:keys [n]} box]\n (let [f (first (filter (fn [w] (and (= \"faust~\" (:class w)) (zero? (:outlet w)))) (:wired box)))\n src (str (get-in f [:data :source]))\n ;; the compilation of its code, waited for (a faust~ just written is still compiling)\n compiled (fn compiled [tries]\n (let [r (api/compiled \"faust\" src)]\n (cond (= :ok (:status r)) (js/Promise.resolve r)\n (and (#{:compiling :empty} (:status r)) (pos? tries))\n (js/Promise. (fn [ok] (js/setTimeout (fn [] (ok nil)) 250)))\n :else (js/Promise.reject (js/Error. (str \"the faust~ does not compile: \" (or (:error r) \"no code\")))))))\n wait (fn wait [tries]\n (.then (compiled tries) (fn [r] (if r r (wait (dec tries))))))]\n (if-not f\n (js/Promise.reject (js/Error. \"connect its left outlet to a faust~: it learns the one it plays\"))\n (-> (wait 120)\n (.then (fn [r]\n (when (get-in r [:info :poly])\n (throw (js/Error. \"it plays a faust~ of one voice (this one declares nvoices)\")))\n (let [fac (.-factory (:value r))]\n (-> (js/Promise.all #js [(api/load-script meyda-url) (api/import-module origins)])\n (.then (fn [[_ m]]\n (.then (.learn m #js {:module (or (.-module fac) (.-code fac))\n :json (js/JSON.parse (.-json fac))\n :Meyda (.-Meyda js/window) :n n})\n (fn [b] [b (.index m b)]))))))))\n (.then (fn [[bjs query]]\n ;; the settings stay a JS array (the query reads them); the rest as data\n (let [es (.-entries bjs)\n col (fn [j] (map (fn [e] (aget e j)) (array-seq es)))\n avg (fn [j] (/ (reduce + (col j)) (.-length es)))]\n (-> (js->clj #js {:pitch (.-pitch bjs) :gain (.-gain bjs) :timbre (.-timbre bjs) :sd (.-sd bjs)}\n :keywordize-keys true)\n (assoc :query query :entries es :faust (name (:id f)) :mid [(avg 1) (avg 2)]\n :lbr [(reduce min (col 1)) (reduce max (col 1))])))))))))}\n\n ;; ── logic ──\n :receive (fn [{:keys [inlet value state args at]}]\n (let [st (or state {})]\n (cond\n ;; what learning found\n (= inlet :task)\n (cond\n (not= (:k (:arg value)) (:k st)) {}\n (:error value) {:state (assoc st :learning false :error (:error value))\n :show {:status :error :error (:error value)}}\n :else (let [st (assoc st :learning false :bank (:value value) :sent {} :u nil)]\n (settle st (or at (:t st) 0))))\n\n (#{:learn :bang} value) (learn st args)\n\n ;; a moment later: the last frame that arrived while it waited\n (= value :realize-tick) (if (and (:bank st) (:dirty st)) (settle (assoc st :tick false) at) {:state (assoc st :tick false)})\n\n ;; what is heard in what it plays (features~ of the faust~): the correction grows by a part of\n ;; the difference, while something is written and sounds\n (and (#{4 5 6} inlet) (number? value))\n (let [heard (case inlet 4 (db-of value) 5 (when (pos? value) (log2 value)) 6 value)\n k ({4 :loud 5 :lb 6 :nz} inlet)\n want (:want st)\n st (cond-> st (= k :loud) (assoc :heard-loud heard))\n sounding? (and (:bank st) (number? (:loud want)) (> (:loud want) silent)\n (> (or (:heard-loud st) -100) silent))]\n (if (and sounding? (number? heard) (number? (get want k)) (:since st) (>= (- at (:since st)) hold))\n ;; within what the faust~ can give (the bank's brightness, noisiness 0…1, the gain slider's\n ;; range): a correction that cannot be played does not grow, so it does not wind up\n (let [bank (:bank st) w (get want k) g (:gain bank) [lb-lo lb-hi] (or (:lbr bank) [0 20])\n [lo hi] (case k\n :lb [(- lb-lo w) (- lb-hi w)]\n :nz [(- w) (- 1 w)]\n :loud (if (and g (:db g) (:loud0 st))\n [(+ (:min g) (:loud0 st) (- w)) (+ (:max g) (:loud0 st) (- w))]\n [(- (limits k)) (limits k)]))\n c (-> (+ (* leak (get-in st [:corr k] 0)) (* rate (- w heard)))\n (api/clamp (max lo (- (limits k))) (min hi (limits k))))\n st (-> st (assoc-in [:corr k] c) (assoc :dirty true))]\n (if (or (nil? (:t st)) (>= (- at (:t st)) every))\n (settle st at)\n (if (:tick st) {:state st} {:state (assoc st :tick true) :later [[(+ (:t st) every) 0 :realize-tick]]})))\n {:state st}))\n\n (and (integer? inlet) (number? value))\n (let [midi (when (and (= inlet 3) (pos? value)) (if (midi-in? args) value (api/hz->midi value)))\n want (case inlet\n 0 (assoc (:want st) :loud (db-of value))\n 1 (if (pos? value) (assoc (:want st) :lb (log2 value)) (:want st))\n 2 (assoc (:want st) :nz (api/clamp value 0 1))\n 3 (if midi (assoc (:want st) :pitch midi :voiced true) (assoc (:want st) :voiced false))\n (:want st))\n ;; since when what is written sounds (its attack is not corrected)\n st (cond-> (assoc st :want want :dirty true)\n (= inlet 0) (assoc :since (if (> (or (:loud want) -100) silent) (or (:since st) at) nil)))]\n (cond\n (:bank st) (if (or (nil? (:t st)) (>= (- at (:t st)) every))\n (settle st at)\n (if (:tick st)\n {:state st}\n {:state (assoc st :tick true) :later [[(+ (:t st) every) 0 :realize-tick]]}))\n (or (:learning st) (:error st)) {:state st}\n :else (learn st args)))\n\n :else {})))\n\n ;; ── view: what it learnt, what is written now, and how far the faust~ is from it ──\n :draw (fn [{:keys [shown w h]}]\n (let [{:keys [status faust n want reach error]} shown\n line1 (case status\n :learning \"learning the faust~ it plays…\"\n :ready (str \"plays \" faust \" · \" n \" settings heard\" (when (:listening shown) \" · listening\"))\n :error (str \"✗ \" error)\n \"waits for a faust~ and a score\")\n fmt (fn [x d] (if (number? x) (.toFixed x d) \"–\"))\n line2 (when (= status :ready)\n (str (fmt (:loud want) 0) \" dB · \"\n (if (:lb want) (str (js/Math.round (js/Math.pow 2 (:lb want))) \" Hz\") \"–\") \" · \"\n (fmt (:nz want) 2)\n (when (and (:voiced want) (:pitch want)) (str \" · \" (api/midi-name (js/Math.round (:pitch want)))))))\n bw (- w 16) far (when (number? reach) (api/clamp (/ reach 2) 0 1))]\n (cond-> [[:rect {:x 2 :y 2 :w (- w 4) :h (- h 4) :r 4 :stroke :grid :fill :bg}]\n [:text {:x 8 :y 15 :size 9 :fill (if (= status :error) :kairos :ink)} line1]]\n line2 (conj [:text {:x 8 :y 28 :size 9 :fill :soft} line2])\n far (conj [:rect {:x 8 :y 35 :w bw :h 4 :r 2 :fill :grid :stroke nil}]\n [:rect {:x 8 :y 35 :w (max 2 (* bw far)) :h 4 :r 2 :fill (if (> reach 1) :kairos :accent) :stroke nil}]))))})\n"
9
9
  };
package/dist/spectrum.mjs CHANGED
@@ -1,7 +1,7 @@
1
1
  // Generated by packages/build.mjs from src/spectrum.cljs
2
2
  export default {
3
3
  "class": "spectrum~",
4
- "version": "0.7.2",
4
+ "version": "0.7.5",
5
5
  "author": "bonuoq",
6
6
  "license": "MPL-2.0",
7
7
  "doc": "The spectrum of the signal on its left, drawn in the patch about 30 times a second: frequency on a log scale from lo to hi Hz (40 and 16000 by default), level in dB from -90 to 0. Computed with Meyda.",
@@ -1,9 +1,9 @@
1
1
  // Generated by packages/build.mjs from src/transcribe.cljs
2
2
  export default {
3
3
  "class": "transcribe",
4
- "version": "0.7.2",
4
+ "version": "0.7.5",
5
5
  "author": "bonuoq",
6
6
  "license": "MPL-2.0",
7
- "doc": "Writes what pitch~ hears into a freqt, as strokes. Hz on the left (from pitch~: 0 when it hears none), rec to start from 0 (its first write clears the freqt) or rec t (msg rec 8) to start from beat t, keeping what is before it, or ovr t (msg ovr 8) to write over only what the take covers, keeping what comes after and stop to end; a bang on the right (from onset~) begins a new note even on the same pitch, and a number there (onset~'s strength, 0…1) does too and gives the note that weight: its stroke is written heavier (as an event's :amp), and the quantizer reads it as the note's :vel, which helps find the beat and the bar (issue #106). In notes mode the contour of each voiced stretch is cut into notes where it settles on another pitch (a slide, a vibrato or an ornament that passes does not start notes; an onset on the right does), each a flat stroke at its median pitch (the freqt's tuning, or its score view, rounds it), written when the voice stops; in glide mode a stroke follows the pitch as it moves. Times are beats at bpm (60: one beat a second). Notes shorter than ms (80) are left out. It writes into the freqt at most every 2 seconds, each time one event of the history. Connect its outlet to the freqt's inlet.",
8
- "source": ";; transcribe · what pitch~ hears, written as strokes of a freqt\n;; Part of @kairotic/analysis. Written against the Kairotic API (api/…). Pure logic: no audio of its own.\n(let [;; ── what it understands ──\n flush-every 2 ; seconds between two writes into the freqt (each write is one event of the history)\n max-points 64 ; points of a stroke in glide mode\n cents (fn [a b] (* 1200 (/ (js/Math.log (/ b a)) js/Math.LN2)))\n ;; the pitch a note holds longest: pitch~ only sends what changed, so a steady note is one point and\n ;; its attack many; each pitch weighs the time until the next one\n held (fn [pts t1]\n (let [ws (map (fn [[t f] [t2 _]] [f (max 0 (- t2 t))]) pts (concat (rest pts) [[t1 nil]]))\n total (reduce + (map second ws))\n sorted (sort-by first ws)]\n (if (pos? total)\n (loop [[[f w] & more] sorted acc 0]\n (let [acc (+ acc w)] (if (or (>= acc (/ total 2)) (empty? more)) f (recur more acc))))\n (second (last pts)))))\n mode-of (fn [args] (if (= :glide (:mode args)) :glide :notes))\n beats (fn [state at] (* (- at (:t0 state)) (/ (:bpm state) 60)))\n r3 (fn [x] (/ (js/Math.round (* x 1000)) 1000))\n ;; ── a note that ends: a stroke, if it lasted long enough ──\n stroke-of (fn [{:keys [pts]} t1 mode]\n (let [t0 (ffirst pts) hz (held pts t1)]\n (if (= mode :glide)\n (let [kept (reduce (fn [acc [t f]]\n (if (> (js/Math.abs (cents (second (peek acc)) f)) 10) (conj acc [t f]) acc))\n [(first pts)] (rest pts))\n kept (if (> (count kept) max-points)\n (vec (take-nth (js/Math.ceil (/ (count kept) max-points)) kept))\n kept)]\n (mapv (fn [[t f]] [(r3 t) (r3 f)]) (conj kept [t1 (second (peek pts))])))\n [[(r3 t0) (r3 hz)] [(r3 t1) (r3 hz)]])))\n end-note (fn [state t]\n (if-let [cur (:cur state)]\n (let [t0 (ffirst (:pts cur))\n long-enough (>= (/ (- t t0) (/ (:bpm state) 60)) (/ (:min-ms state) 1000))]\n (cond-> (dissoc state :cur)\n long-enough (update :done conj {:pts (stroke-of cur t (:mode state)) :acc (:acc cur)})\n long-enough (update :count inc)))\n state))\n ;; a note takes the strength of the onset that began it (a number on the right inlet, from onset~), if it\n ;; came at most 0.15 s before: onset~ often hears the attack before pitch~ hears the pitch\n start-note (fn [state t hz]\n (let [[at s] (:acc state) acc (when (and at (<= (- t at) (* 0.15 (/ (:bpm state) 60)))) s)]\n (cond-> (assoc state :cur (cond-> {:pts [[t hz]] :midi (api/hz->midi hz)} acc (assoc :acc acc)))\n acc (dissoc :acc))))\n ;; how a note with the strength of its attack is written: the weight of its stroke, as an event's :amp is\n ;; (0 … 1 → 0.4 … 2.8 times the usual); the quantizer reads it back as the note's :vel (issue #106)\n edit-of (fn [{:keys [pts acc]}] (if (number? acc) [:stroke pts {:w (/ (js/Math.round (* 100 (+ 0.4 (* 2.4 acc)))) 100)}] [:stroke pts]))\n ;; ── writing: the strokes that ended, as one [:batch …], at most every flush-every seconds ──\n flush (fn [state at force]\n (let [now (r3 (beats state at))]\n (cond\n ;; ovr at stop: what the take covered since the last write is overwritten, even by silence\n (and (empty? (:done state)) (not (and force (:ovr state) (> now (:cut-from state))))) [state nil nil]\n (or force (nil? (:sent state)) (>= (- at (:sent state)) flush-every))\n ;; the first write of a take clears the freqt (as part of the same event); a take over (ovr)\n ;; cuts what it covered since the last write, and keeps what comes after\n [(cond-> (assoc state :done [] :sent at :pending false :fresh false)\n (:ovr state) (assoc :cut-from now))\n [:batch (into (cond (:ovr state) [[:cut (:cut-from state) now]]\n (:fresh state) [(if (:from state) [:cut (:from state)] [:clear])]\n :else [])\n (mapv edit-of (:done state)))] nil]\n (:pending state) [state nil nil]\n :else [(assoc state :pending true) nil [(+ (:sent state) flush-every) 0 [:flush]]])))\n result (fn [state [st edit later]]\n (cond-> {:state st :show (select-keys st [:on :count])}\n edit (assoc :out [[0 edit]])\n later (assoc :later [later])))\n ;; ── notes mode: the contour of each voiced stretch, segmented (issue #106, measured on cante2midi) ──\n ;; What pitch~ sends while a voice sounds is kept as a run of points; when the voice stops (or the run is long)\n ;; the run is read as a contour every 10 ms, smoothed by a median of 100 ms (vibrato and jitter), and cut into\n ;; notes with hysteresis: a note goes on while the contour stays within a semitone of its pitch; a new one\n ;; begins only where the contour, away from it, has held still (within 1.1 semitones) for 70 ms, so a slide\n ;; or an ornament that passes does not start notes. An onset (inlet 1) cuts there too. Each note is written\n ;; at the median pitch of its frames. Silences of 30 ms at most are bridged.\n median (fn [xs] (let [v (vec (sort xs))] (nth v (quot (count v) 2))))\n segment (fn [run marks t-end k minlen]\n (let [dt (* 0.01 k) w 5 band 1 stab 1.1 hold-n 7\n ts (vec (range (ffirst run) t-end dt))\n ;; the contour on the grid: each frame, the last point at or before it (as MIDI)\n raw (loop [i 0 [p & more :as ps] run cur nil out (transient [])]\n (if (>= i (count ts))\n (persistent! out)\n (let [t (ts i)]\n (if (and p (<= (first p) (+ t 1e-9)))\n (recur i more (api/hz->midi (second p)) out)\n (recur (inc i) ps cur (conj! out cur))))))\n n (count raw)\n sm (mapv (fn [i] (median (subvec raw (max 0 (- i w)) (min n (+ i w 1))))) (range n))\n cut? (fn [i] (let [a (if (pos? i) (ts (dec i)) (- (ts 0) dt)) b (ts i)] (some (fn [[mt]] (and (> mt a) (<= mt b))) marks)))\n note (fn [s e ms] {:s s :e e :ms ms})\n notes (loop [i 0 cur nil pend [] out []]\n (if (>= i n)\n (if cur (conj out (assoc cur :e t-end)) out)\n (let [t (ts i) m (sm i)]\n (cond\n (nil? cur) (recur (inc i) (note t (+ t dt) [m]) [] out)\n ;; (not while the contour is away from the note and not settled yet: there the contour\n ;; decides; measured on cante2midi with onsets from its Bark bands)\n (and (empty? pend) (cut? i) (> t (:s cur))) (recur (inc i) (note t (+ t dt) [m]) [] (conj out (assoc cur :e t)))\n (<= (js/Math.abs (- m (median (take-last 24 (:ms cur))))) band)\n (recur (inc i) (assoc cur :e (+ t dt) :ms (into (conj (:ms cur) m) (map second pend))) [] out)\n :else\n (let [pend (conj pend [t m]) tail (take-last hold-n pend) tm (map second tail)]\n (if (and (>= (count pend) hold-n) (< (- (apply max tm) (apply min tm)) stab))\n ;; the new note begins where the contour left the old one; its pitch is that of where it settled\n (recur (inc i) (note (ffirst pend) (+ t dt) (vec tm)) [] (conj out (assoc cur :e (ffirst pend))))\n (recur (inc i) cur pend out)))))))]\n (->> notes\n (filter (fn [{:keys [s e]}] (>= (- e s) minlen)))\n (mapv (fn [{:keys [s e ms]}]\n (let [hz (api/midi->hz (median ms))\n ;; the strength of an onset heard at most 0.15 s before the note (or just after its start)\n acc (some (fn [[mt x]] (when (and (number? x) (<= (- s (* 0.15 k)) mt (+ s (* 0.05 k)))) x)) marks)]\n (cond-> {:pts [[(r3 s) (r3 hz)] [(r3 e) (r3 hz)]]} acc (assoc :acc acc))))))))\n ;; the run segmented into :done: all of it (the voice stopped at t-end), or all but its last note (a long run\n ;; still sounding: that note goes on in the run)\n settle (fn [state t-end all?]\n (let [run (:run state) k (/ (:bpm state) 60) minlen (* k (/ (:min-ms state) 1000))]\n (if (or (empty? run) (<= t-end (ffirst run)))\n (dissoc state :run :silent)\n (let [ns (segment run (:marks state) t-end k minlen)\n [keep-ns rest-from] (if (or all? (< (count ns) 2)) [ns nil] [(pop ns) (ffirst (:pts (peek ns)))])\n state (-> state (update :done into keep-ns) (update :count + (count keep-ns)))]\n (if rest-from\n (let [before (last (filter (fn [[t]] (<= t rest-from)) run))]\n (assoc state :run (into [[rest-from (second before)]] (filter (fn [[t]] (> t rest-from)) run))\n :marks (filterv (fn [[t]] (>= t (- rest-from (* 0.15 k)))) (:marks state))))\n (-> state (dissoc :run :silent) (assoc :marks (filterv (fn [[t]] (> t t-end)) (:marks state)))))))))\n ;; a silence that lasted more than 30 ms: the run ends where it began\n settled (fn [state t]\n (if (and (:silent state) (> (- t (:silent state)) (* 0.03 (/ (:bpm state) 60))))\n (settle state (:silent state) true)\n state))\n hear-notes (fn [state hz t]\n (let [k (/ (:bpm state) 60)\n state (settled state t)]\n (cond\n (not (pos? hz)) (if (and (seq (:run state)) (nil? (:silent state))) (assoc state :silent t) state)\n :else (let [state (-> state (dissoc :silent) (update :run (fnil conj []) [t hz]))]\n ;; a run of 8 s and more: what is settled is written, the last note goes on\n (if (> (- t (ffirst (:run state))) (* 8 k)) (settle state t false) state)))))\n ;; ── one value of pitch~ (Hz, 0 when it hears none) ──\n hear (fn [state hz at]\n (let [t (beats state at)\n cur (:cur state)\n state (cond\n (= :notes (:mode state)) (hear-notes state hz t)\n (not (pos? hz)) (end-note state t)\n (nil? cur) (start-note state t hz)\n :else (update-in state [:cur :pts] conj [t hz]))\n [st edit later] (flush state at false)]\n ;; the voice stopped: a look again in a little while, to write its notes even if nothing else arrives\n [st edit (or later (when (and (:silent st) (not (:pending st))) [(+ at 0.05) 0 [:flush]]))]))]\n {;; ── about ──\n :class \"transcribe\"\n :license \"MPL-2.0\"\n :doc \"Writes what pitch~ hears into a freqt, as strokes. Hz on the left (from pitch~: 0 when it hears none), rec to start from 0 (its first write clears the freqt) or rec t (msg rec 8) to start from beat t, keeping what is before it, or ovr t (msg ovr 8) to write over only what the take covers, keeping what comes after and stop to end; a bang on the right (from onset~) begins a new note even on the same pitch, and a number there (onset~'s strength, 0…1) does too and gives the note that weight: its stroke is written heavier (as an event's :amp), and the quantizer reads it as the note's :vel, which helps find the beat and the bar (issue #106). In notes mode the contour of each voiced stretch is cut into notes where it settles on another pitch (a slide, a vibrato or an ornament that passes does not start notes; an onset on the right does), each a flat stroke at its median pitch (the freqt's tuning, or its score view, rounds it), written when the voice stops; in glide mode a stroke follows the pitch as it moves. Times are beats at bpm (60: one beat a second). Notes shorter than ms (80) are left out. It writes into the freqt at most every 2 seconds, each time one event of the history. Connect its outlet to the freqt's inlet.\"\n :usage \"transcribe [bpm] [notes|glide] [ms]\"\n :io [2 1]\n :size [124 30]\n :args [[:bpm 60] [:mode nil] [:min-ms 80]]\n\n ;; ── logic ──\n :receive (fn [{:keys [inlet value state args at]}]\n (let [state (or state {})]\n (cond\n ;; rec: a take from 0 (its first write clears the freqt); [:rec t] (msg rec 8): from beat t on,\n ;; keeping what is before t (punch-in)\n ;; ovr t (msg ovr 8): over what is there from beat t to where the take stops, keeping what comes after\n (or (#{:rec :ovr} value) (and (vector? value) (#{:rec :ovr} (first value))))\n (let [bpm (api/clamp (:bpm args) 10 400)\n ovr (or (= value :ovr) (and (vector? value) (= :ovr (first value))))\n from (or (when (vector? value) (let [x (second value)] (when (and (number? x) (>= x 0)) x)))\n (when ovr 0))]\n {:state (cond-> {:on true :t0 (- at (* (or from 0) (/ 60 bpm))) :bpm bpm :mode (mode-of args)\n :min-ms (api/clamp (:min-ms args) 0 2000) :done [] :count 0 :fresh true}\n from (assoc :from from)\n ovr (assoc :ovr true :cut-from from))\n :show {:on true :count 0}})\n (not (:on state)) {}\n (= value :stop)\n (let [t (beats state at)\n st (if (seq (:run state)) (settle state (or (:silent state) t) true) state)\n st (end-note st t)\n [st edit _] (flush st at true)]\n (cond-> {:state (assoc st :on false) :show {:on false :count (:count st)}}\n edit (assoc :out [[0 edit]])))\n (= value [:flush]) (result state (flush (settled (assoc state :pending false) (beats state at)) at false))\n ;; an onset (a bang, or its strength 0…1 from onset~'s second outlet): the note that sounds ends\n ;; here and a new one begins at its pitch, with that strength; in silence, the strength waits for\n ;; the next note to begin\n (and (= inlet 1) (= :notes (:mode state)))\n {:state (update state :marks (fnil conj []) [(beats state at) (when (number? value) (api/clamp value 0 1))])}\n (= inlet 1)\n (let [t (beats state at)\n state (if (number? value) (assoc state :acc [t (api/clamp value 0 1)]) state)]\n (if (:cur state)\n (let [hz (second (peek (get-in state [:cur :pts])))]\n (result state (flush (-> state (end-note t) (start-note t hz)) at false)))\n {:state state}))\n (and (= inlet 0) (number? value)) (result state (hear state value at))\n :else {})))\n\n ;; ── view ──\n :derive (fn [{:keys [shown]}]\n {:label (if (:on shown) (str \"● transcribing · \" (or (:count shown) 0)) (str \"transcribe · \" (or (:count shown) 0) \" notes\"))})\n :view [[:rect {:x 2 :y 2 :w 120 :h 26 :r 3 :stroke :grid :fill :bg}]\n [:text {:x 8 :y 19 :size 10 :fill :soft} :$label]]})\n"
7
+ "doc": "Writes what pitch~ hears into a freqt, as strokes. Hz on the left (from pitch~: 0 when it hears none), :rec to start from 0 (its first write clears the freqt) or :rec t (msg :rec 8) to start from beat t, keeping what is before it, or :ovr t (msg :ovr 8) to write over only what the take covers, keeping what comes after and :stop to end; a bang on the right (from onset~) begins a new note even on the same pitch, and a number there (onset~'s strength, 0…1) does too and gives the note that weight: its stroke is written heavier (as an event's :amp), and the quantizer reads it as the note's :vel, which helps find the beat and the bar (issue #106). In notes mode the contour of each voiced stretch is cut into notes where it settles on another pitch (a slide, a vibrato or an ornament that passes does not start notes; an onset on the right does), each a flat stroke at its median pitch (the freqt's tuning, or its score view, rounds it), written when the voice stops; in glide mode a stroke follows the pitch as it moves. Times are beats at bpm (60: one beat a second). Notes shorter than ms (80) are left out. It writes into the freqt at most every 2 seconds, each time one event of the history. Connect its outlet to the freqt's inlet.",
8
+ "source": ";; transcribe · what pitch~ hears, written as strokes of a freqt\n;; Part of @kairotic/analysis. Written against the Kairotic API (api/…). Pure logic: no audio of its own.\n(let [;; ── what it understands ──\n flush-every 2 ; seconds between two writes into the freqt (each write is one event of the history)\n max-points 64 ; points of a stroke in glide mode\n cents (fn [a b] (* 1200 (/ (js/Math.log (/ b a)) js/Math.LN2)))\n ;; the pitch a note holds longest: pitch~ only sends what changed, so a steady note is one point and\n ;; its attack many; each pitch weighs the time until the next one\n held (fn [pts t1]\n (let [ws (map (fn [[t f] [t2 _]] [f (max 0 (- t2 t))]) pts (concat (rest pts) [[t1 nil]]))\n total (reduce + (map second ws))\n sorted (sort-by first ws)]\n (if (pos? total)\n (loop [[[f w] & more] sorted acc 0]\n (let [acc (+ acc w)] (if (or (>= acc (/ total 2)) (empty? more)) f (recur more acc))))\n (second (last pts)))))\n mode-of (fn [args] (if (= :glide (:mode args)) :glide :notes))\n beats (fn [state at] (* (- at (:t0 state)) (/ (:bpm state) 60)))\n r3 (fn [x] (/ (js/Math.round (* x 1000)) 1000))\n ;; ── a note that ends: a stroke, if it lasted long enough ──\n stroke-of (fn [{:keys [pts]} t1 mode]\n (let [t0 (ffirst pts) hz (held pts t1)]\n (if (= mode :glide)\n (let [kept (reduce (fn [acc [t f]]\n (if (> (js/Math.abs (cents (second (peek acc)) f)) 10) (conj acc [t f]) acc))\n [(first pts)] (rest pts))\n kept (if (> (count kept) max-points)\n (vec (take-nth (js/Math.ceil (/ (count kept) max-points)) kept))\n kept)]\n (mapv (fn [[t f]] [(r3 t) (r3 f)]) (conj kept [t1 (second (peek pts))])))\n [[(r3 t0) (r3 hz)] [(r3 t1) (r3 hz)]])))\n end-note (fn [state t]\n (if-let [cur (:cur state)]\n (let [t0 (ffirst (:pts cur))\n long-enough (>= (/ (- t t0) (/ (:bpm state) 60)) (/ (:min-ms state) 1000))]\n (cond-> (dissoc state :cur)\n long-enough (update :done conj {:pts (stroke-of cur t (:mode state)) :acc (:acc cur)})\n long-enough (update :count inc)))\n state))\n ;; a note takes the strength of the onset that began it (a number on the right inlet, from onset~), if it\n ;; came at most 0.15 s before: onset~ often hears the attack before pitch~ hears the pitch\n start-note (fn [state t hz]\n (let [[at s] (:acc state) acc (when (and at (<= (- t at) (* 0.15 (/ (:bpm state) 60)))) s)]\n (cond-> (assoc state :cur (cond-> {:pts [[t hz]] :midi (api/hz->midi hz)} acc (assoc :acc acc)))\n acc (dissoc :acc))))\n ;; how a note with the strength of its attack is written: the weight of its stroke, as an event's :amp is\n ;; (0 … 1 → 0.4 … 2.8 times the usual); the quantizer reads it back as the note's :vel (issue #106)\n edit-of (fn [{:keys [pts acc]}] (if (number? acc) [:stroke pts {:w (/ (js/Math.round (* 100 (+ 0.4 (* 2.4 acc)))) 100)}] [:stroke pts]))\n ;; ── writing: the strokes that ended, as one [:batch …], at most every flush-every seconds ──\n flush (fn [state at force]\n (let [now (r3 (beats state at))]\n (cond\n ;; ovr at stop: what the take covered since the last write is overwritten, even by silence\n (and (empty? (:done state)) (not (and force (:ovr state) (> now (:cut-from state))))) [state nil nil]\n (or force (nil? (:sent state)) (>= (- at (:sent state)) flush-every))\n ;; the first write of a take clears the freqt (as part of the same event); a take over (ovr)\n ;; cuts what it covered since the last write, and keeps what comes after\n [(cond-> (assoc state :done [] :sent at :pending false :fresh false)\n (:ovr state) (assoc :cut-from now))\n [:batch (into (cond (:ovr state) [[:cut (:cut-from state) now]]\n (:fresh state) [(if (:from state) [:cut (:from state)] [:clear])]\n :else [])\n (mapv edit-of (:done state)))] nil]\n (:pending state) [state nil nil]\n :else [(assoc state :pending true) nil [(+ (:sent state) flush-every) 0 [:flush]]])))\n result (fn [state [st edit later]]\n (cond-> {:state st :show (select-keys st [:on :count])}\n edit (assoc :out [[0 edit]])\n later (assoc :later [later])))\n ;; ── notes mode: the contour of each voiced stretch, segmented (issue #106, measured on cante2midi) ──\n ;; What pitch~ sends while a voice sounds is kept as a run of points; when the voice stops (or the run is long)\n ;; the run is read as a contour every 10 ms, smoothed by a median of 100 ms (vibrato and jitter), and cut into\n ;; notes with hysteresis: a note goes on while the contour stays within a semitone of its pitch; a new one\n ;; begins only where the contour, away from it, has held still (within 1.1 semitones) for 70 ms, so a slide\n ;; or an ornament that passes does not start notes. An onset (inlet 1) cuts there too. Each note is written\n ;; at the median pitch of its frames. Silences of 30 ms at most are bridged.\n median (fn [xs] (let [v (vec (sort xs))] (nth v (quot (count v) 2))))\n segment (fn [run marks t-end k minlen]\n (let [dt (* 0.01 k) w 5 band 1 stab 1.1 hold-n 7\n ts (vec (range (ffirst run) t-end dt))\n ;; the contour on the grid: each frame, the last point at or before it (as MIDI)\n raw (loop [i 0 [p & more :as ps] run cur nil out (transient [])]\n (if (>= i (count ts))\n (persistent! out)\n (let [t (ts i)]\n (if (and p (<= (first p) (+ t 1e-9)))\n (recur i more (api/hz->midi (second p)) out)\n (recur (inc i) ps cur (conj! out cur))))))\n n (count raw)\n sm (mapv (fn [i] (median (subvec raw (max 0 (- i w)) (min n (+ i w 1))))) (range n))\n cut? (fn [i] (let [a (if (pos? i) (ts (dec i)) (- (ts 0) dt)) b (ts i)] (some (fn [[mt]] (and (> mt a) (<= mt b))) marks)))\n note (fn [s e ms] {:s s :e e :ms ms})\n notes (loop [i 0 cur nil pend [] out []]\n (if (>= i n)\n (if cur (conj out (assoc cur :e t-end)) out)\n (let [t (ts i) m (sm i)]\n (cond\n (nil? cur) (recur (inc i) (note t (+ t dt) [m]) [] out)\n ;; (not while the contour is away from the note and not settled yet: there the contour\n ;; decides; measured on cante2midi with onsets from its Bark bands)\n (and (empty? pend) (cut? i) (> t (:s cur))) (recur (inc i) (note t (+ t dt) [m]) [] (conj out (assoc cur :e t)))\n (<= (js/Math.abs (- m (median (take-last 24 (:ms cur))))) band)\n (recur (inc i) (assoc cur :e (+ t dt) :ms (into (conj (:ms cur) m) (map second pend))) [] out)\n :else\n (let [pend (conj pend [t m]) tail (take-last hold-n pend) tm (map second tail)]\n (if (and (>= (count pend) hold-n) (< (- (apply max tm) (apply min tm)) stab))\n ;; the new note begins where the contour left the old one; its pitch is that of where it settled\n (recur (inc i) (note (ffirst pend) (+ t dt) (vec tm)) [] (conj out (assoc cur :e (ffirst pend))))\n (recur (inc i) cur pend out)))))))]\n (->> notes\n (filter (fn [{:keys [s e]}] (>= (- e s) minlen)))\n (mapv (fn [{:keys [s e ms]}]\n (let [hz (api/midi->hz (median ms))\n ;; the strength of an onset heard at most 0.15 s before the note (or just after its start)\n acc (some (fn [[mt x]] (when (and (number? x) (<= (- s (* 0.15 k)) mt (+ s (* 0.05 k)))) x)) marks)]\n (cond-> {:pts [[(r3 s) (r3 hz)] [(r3 e) (r3 hz)]]} acc (assoc :acc acc))))))))\n ;; the run segmented into :done: all of it (the voice stopped at t-end), or all but its last note (a long run\n ;; still sounding: that note goes on in the run)\n settle (fn [state t-end all?]\n (let [run (:run state) k (/ (:bpm state) 60) minlen (* k (/ (:min-ms state) 1000))]\n (if (or (empty? run) (<= t-end (ffirst run)))\n (dissoc state :run :silent)\n (let [ns (segment run (:marks state) t-end k minlen)\n [keep-ns rest-from] (if (or all? (< (count ns) 2)) [ns nil] [(pop ns) (ffirst (:pts (peek ns)))])\n state (-> state (update :done into keep-ns) (update :count + (count keep-ns)))]\n (if rest-from\n (let [before (last (filter (fn [[t]] (<= t rest-from)) run))]\n (assoc state :run (into [[rest-from (second before)]] (filter (fn [[t]] (> t rest-from)) run))\n :marks (filterv (fn [[t]] (>= t (- rest-from (* 0.15 k)))) (:marks state))))\n (-> state (dissoc :run :silent) (assoc :marks (filterv (fn [[t]] (> t t-end)) (:marks state)))))))))\n ;; a silence that lasted more than 30 ms: the run ends where it began\n settled (fn [state t]\n (if (and (:silent state) (> (- t (:silent state)) (* 0.03 (/ (:bpm state) 60))))\n (settle state (:silent state) true)\n state))\n hear-notes (fn [state hz t]\n (let [k (/ (:bpm state) 60)\n state (settled state t)]\n (cond\n (not (pos? hz)) (if (and (seq (:run state)) (nil? (:silent state))) (assoc state :silent t) state)\n :else (let [state (-> state (dissoc :silent) (update :run (fnil conj []) [t hz]))]\n ;; a run of 8 s and more: what is settled is written, the last note goes on\n (if (> (- t (ffirst (:run state))) (* 8 k)) (settle state t false) state)))))\n ;; ── one value of pitch~ (Hz, 0 when it hears none) ──\n hear (fn [state hz at]\n (let [t (beats state at)\n cur (:cur state)\n state (cond\n (= :notes (:mode state)) (hear-notes state hz t)\n (not (pos? hz)) (end-note state t)\n (nil? cur) (start-note state t hz)\n :else (update-in state [:cur :pts] conj [t hz]))\n [st edit later] (flush state at false)]\n ;; the voice stopped: a look again in a little while, to write its notes even if nothing else arrives\n [st edit (or later (when (and (:silent st) (not (:pending st))) [(+ at 0.05) 0 [:flush]]))]))]\n {;; ── about ──\n :class \"transcribe\"\n :license \"MPL-2.0\"\n :doc \"Writes what pitch~ hears into a freqt, as strokes. Hz on the left (from pitch~: 0 when it hears none), :rec to start from 0 (its first write clears the freqt) or :rec t (msg :rec 8) to start from beat t, keeping what is before it, or :ovr t (msg :ovr 8) to write over only what the take covers, keeping what comes after and :stop to end; a bang on the right (from onset~) begins a new note even on the same pitch, and a number there (onset~'s strength, 0…1) does too and gives the note that weight: its stroke is written heavier (as an event's :amp), and the quantizer reads it as the note's :vel, which helps find the beat and the bar (issue #106). In notes mode the contour of each voiced stretch is cut into notes where it settles on another pitch (a slide, a vibrato or an ornament that passes does not start notes; an onset on the right does), each a flat stroke at its median pitch (the freqt's tuning, or its score view, rounds it), written when the voice stops; in glide mode a stroke follows the pitch as it moves. Times are beats at bpm (60: one beat a second). Notes shorter than ms (80) are left out. It writes into the freqt at most every 2 seconds, each time one event of the history. Connect its outlet to the freqt's inlet.\"\n :usage \"transcribe [bpm] [notes|glide] [ms]\"\n :io [2 1]\n :size [124 30]\n :args [[:bpm 60] [:mode nil] [:min-ms 80]]\n\n ;; ── logic ──\n :receive (fn [{:keys [inlet value state args at]}]\n (let [state (or state {})]\n (cond\n ;; rec: a take from 0 (its first write clears the freqt); [:rec t] (msg :rec 8): from beat t on,\n ;; keeping what is before t (punch-in)\n ;; ovr t (msg :ovr 8): over what is there from beat t to where the take stops, keeping what comes after\n (or (#{:rec :ovr} value) (and (vector? value) (#{:rec :ovr} (first value))))\n (let [bpm (api/clamp (:bpm args) 10 400)\n ovr (or (= value :ovr) (and (vector? value) (= :ovr (first value))))\n from (or (when (vector? value) (let [x (second value)] (when (and (number? x) (>= x 0)) x)))\n (when ovr 0))]\n {:state (cond-> {:on true :t0 (- at (* (or from 0) (/ 60 bpm))) :bpm bpm :mode (mode-of args)\n :min-ms (api/clamp (:min-ms args) 0 2000) :done [] :count 0 :fresh true}\n from (assoc :from from)\n ovr (assoc :ovr true :cut-from from))\n :show {:on true :count 0}})\n (not (:on state)) {}\n (= value :stop)\n (let [t (beats state at)\n st (if (seq (:run state)) (settle state (or (:silent state) t) true) state)\n st (end-note st t)\n [st edit _] (flush st at true)]\n (cond-> {:state (assoc st :on false) :show {:on false :count (:count st)}}\n edit (assoc :out [[0 edit]])))\n (= value [:flush]) (result state (flush (settled (assoc state :pending false) (beats state at)) at false))\n ;; an onset (a bang, or its strength 0…1 from onset~'s second outlet): the note that sounds ends\n ;; here and a new one begins at its pitch, with that strength; in silence, the strength waits for\n ;; the next note to begin\n (and (= inlet 1) (= :notes (:mode state)))\n {:state (update state :marks (fnil conj []) [(beats state at) (when (number? value) (api/clamp value 0 1))])}\n (= inlet 1)\n (let [t (beats state at)\n state (if (number? value) (assoc state :acc [t (api/clamp value 0 1)]) state)]\n (if (:cur state)\n (let [hz (second (peek (get-in state [:cur :pts])))]\n (result state (flush (-> state (end-note t) (start-note t hz)) at false)))\n {:state state}))\n (and (= inlet 0) (number? value)) (result state (hear state value at))\n :else {})))\n\n ;; ── view ──\n :derive (fn [{:keys [shown]}]\n {:label (if (:on shown) (str \"● transcribing · \" (or (:count shown) 0)) (str \"transcribe · \" (or (:count shown) 0) \" notes\"))})\n :view [[:rect {:x 2 :y 2 :w 120 :h 26 :r 3 :stroke :grid :fill :bg}]\n [:text {:x 8 :y 19 :size 10 :fill :soft} :$label]]})\n"
9
9
  };
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@kairotic/analysis",
3
- "version": "0.7.2",
3
+ "version": "0.7.5",
4
4
  "description": "Kairotic analysis components: features~ (descriptors with Meyda), pitch~ (YIN), onset~, chroma~ spectrum~, transcribe (pitch~ into a freqt), form~ (self-similarity: segments and periods) and monitor (any stream of messages, drawn), from any signal or adc~ into messages and pictures",
5
5
  "author": "bonuoq",
6
6
  "license": "MPL-2.0",
package/src/form.cljs CHANGED
@@ -79,7 +79,7 @@
79
79
  {;; ── about ──
80
80
  :class "form~"
81
81
  :license "MPL-2.0"
82
- :doc "How alike the moments of what arrives are, over the last seconds (20): a self-similarity matrix, drawn, of one frame every step seconds (0.25) — the mean of the lists that arrived in it (a chroma~, a features~ mfcc), or, for a stream of numbers (onset~'s third outlet, an rms), the shape of its values. On the left the novelty of each frame (0…1: how much the music changes there, by a checkerboard of kernel seconds (2) on each side, so it comes kernel seconds late); a bang on the second outlet where a segment begins (a novelty peak high for the window, two kernels after the last at least), and, on the fourth, [:segment {:t :at :length :period}] for the segment it ends; on the third, the period that repeats most in the window, in seconds (the bar, the phrase), when it changes. [:seconds x], [:step x] and [:kernel x] change them, clear forgets. Into onset~ (a [:segment …] makes it forget the usual change of the section before: adaptive onsets), a message box with [:threshold x], or a print. Self-similarity after Foote (1999, 2000, 2001); the same functions for a whole take or a tree are api/ssm, api/novelty, api/boundaries and api/note-frames."
82
+ :doc "How alike the moments of what arrives are, over the last seconds (20): a self-similarity matrix, drawn, of one frame every step seconds (0.25) — the mean of the lists that arrived in it (a chroma~, a features~ mfcc), or, for a stream of numbers (onset~'s third outlet, an rms), the shape of its values. On the left the novelty of each frame (0…1: how much the music changes there, by a checkerboard of kernel seconds (2) on each side, so it comes kernel seconds late); a bang on the second outlet where a segment begins (a novelty peak high for the window, two kernels after the last at least), and, on the fourth, [:segment {:t :at :length :period}] for the segment it ends; on the third, the period that repeats most in the window, in seconds (the bar, the phrase), when it changes. [:seconds x], [:step x] and [:kernel x] change them, :clear forgets. Into onset~ (a [:segment …] makes it forget the usual change of the section before: adaptive onsets), a message box with [:threshold x], or a print. Self-similarity after Foote (1999, 2000, 2001); the same functions for a whole take or a tree are api/ssm, api/novelty, api/boundaries and api/note-frames."
83
83
  :usage "form~ [seconds] [step] [kernel]"
84
84
  :io [1 4]
85
85
  :size [220 128]
package/src/realize.cljs CHANGED
@@ -142,7 +142,7 @@
142
142
  :else (let [st (assoc st :learning false :bank (:value value) :sent {} :u nil)]
143
143
  (settle st (or at (:t st) 0))))
144
144
 
145
- (#{:learn "learn" :bang} value) (learn st args)
145
+ (#{:learn :bang} value) (learn st args)
146
146
 
147
147
  ;; a moment later: the last frame that arrived while it waited
148
148
  (= value :realize-tick) (if (and (:bank st) (:dirty st)) (settle (assoc st :tick false) at) {:state (assoc st :tick false)})
@@ -156,7 +156,7 @@
156
156
  {;; ── about ──
157
157
  :class "transcribe"
158
158
  :license "MPL-2.0"
159
- :doc "Writes what pitch~ hears into a freqt, as strokes. Hz on the left (from pitch~: 0 when it hears none), rec to start from 0 (its first write clears the freqt) or rec t (msg rec 8) to start from beat t, keeping what is before it, or ovr t (msg ovr 8) to write over only what the take covers, keeping what comes after and stop to end; a bang on the right (from onset~) begins a new note even on the same pitch, and a number there (onset~'s strength, 0…1) does too and gives the note that weight: its stroke is written heavier (as an event's :amp), and the quantizer reads it as the note's :vel, which helps find the beat and the bar (issue #106). In notes mode the contour of each voiced stretch is cut into notes where it settles on another pitch (a slide, a vibrato or an ornament that passes does not start notes; an onset on the right does), each a flat stroke at its median pitch (the freqt's tuning, or its score view, rounds it), written when the voice stops; in glide mode a stroke follows the pitch as it moves. Times are beats at bpm (60: one beat a second). Notes shorter than ms (80) are left out. It writes into the freqt at most every 2 seconds, each time one event of the history. Connect its outlet to the freqt's inlet."
159
+ :doc "Writes what pitch~ hears into a freqt, as strokes. Hz on the left (from pitch~: 0 when it hears none), :rec to start from 0 (its first write clears the freqt) or :rec t (msg :rec 8) to start from beat t, keeping what is before it, or :ovr t (msg :ovr 8) to write over only what the take covers, keeping what comes after and :stop to end; a bang on the right (from onset~) begins a new note even on the same pitch, and a number there (onset~'s strength, 0…1) does too and gives the note that weight: its stroke is written heavier (as an event's :amp), and the quantizer reads it as the note's :vel, which helps find the beat and the bar (issue #106). In notes mode the contour of each voiced stretch is cut into notes where it settles on another pitch (a slide, a vibrato or an ornament that passes does not start notes; an onset on the right does), each a flat stroke at its median pitch (the freqt's tuning, or its score view, rounds it), written when the voice stops; in glide mode a stroke follows the pitch as it moves. Times are beats at bpm (60: one beat a second). Notes shorter than ms (80) are left out. It writes into the freqt at most every 2 seconds, each time one event of the history. Connect its outlet to the freqt's inlet."
160
160
  :usage "transcribe [bpm] [notes|glide] [ms]"
161
161
  :io [2 1]
162
162
  :size [124 30]
@@ -166,9 +166,9 @@
166
166
  :receive (fn [{:keys [inlet value state args at]}]
167
167
  (let [state (or state {})]
168
168
  (cond
169
- ;; rec: a take from 0 (its first write clears the freqt); [:rec t] (msg rec 8): from beat t on,
169
+ ;; rec: a take from 0 (its first write clears the freqt); [:rec t] (msg :rec 8): from beat t on,
170
170
  ;; keeping what is before t (punch-in)
171
- ;; ovr t (msg ovr 8): over what is there from beat t to where the take stops, keeping what comes after
171
+ ;; ovr t (msg :ovr 8): over what is there from beat t to where the take stops, keeping what comes after
172
172
  (or (#{:rec :ovr} value) (and (vector? value) (#{:rec :ovr} (first value))))
173
173
  (let [bpm (api/clamp (:bpm args) 10 400)
174
174
  ovr (or (= value :ovr) (and (vector? value) (= :ovr (first value))))