@kairotic/analysis 0.2.1 → 0.3.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +13 -1
- package/dist/chroma.mjs +1 -1
- package/dist/features.mjs +1 -1
- package/dist/index.mjs +9 -3
- package/dist/monitor.mjs +9 -0
- package/dist/onset.mjs +1 -1
- package/dist/pitch.mjs +1 -1
- package/dist/spectrum.mjs +1 -1
- package/dist/transcribe.mjs +3 -3
- package/package.json +4 -3
- package/src/monitor.cljs +105 -0
- package/src/transcribe.cljs +23 -13
package/README.md
CHANGED
|
@@ -11,7 +11,8 @@ From any signal — an `adc~`, a `sample` played by a `reader~`, a `faust~` —
|
|
|
11
11
|
| `onset~ [threshold] [ms]` | a bang at the beginning of each sound (a note, a stroke, a syllable) and its strength |
|
|
12
12
|
| `chroma~ [ms]` | 12 values from C to B and the strongest pitch class as a word (`:a`, `:c#`…) |
|
|
13
13
|
| `spectrum~ [lo hi]` | the spectrum drawn in the patch, on a log scale of frequency |
|
|
14
|
-
| `transcribe [bpm] [notes\|glide] [ms]` | what `pitch~` hears (and `onset~` splits), written into a `freqt` as strokes: flat notes at the pitch they hold longest, or contours that follow the voice; times in beats at bpm. `rec` starts from 0, `stop` ends |
|
|
14
|
+
| `transcribe [bpm] [notes\|glide] [ms]` | what `pitch~` hears (and `onset~` splits), written into a `freqt` as strokes: flat notes at the pitch they hold longest, or contours that follow the voice; times in beats at bpm. `rec` starts from 0, `rec t` from beat t, `ovr t` over only what the take covers, `stop` ends |
|
|
15
|
+
| `monitor [seconds] [name …]` | whatever arrives, drawn over the last seconds (6), one lane per inlet: a lane for each name written (`monitor 6 rms centroid chroma` has 3 inlets), or one. A number is a line in the range it has had in the window, a list of numbers a strip with a row per element (chroma, mfcc), a bang or a word a mark. It sends nothing |
|
|
15
16
|
|
|
16
17
|
Each sends only what changed. `pitch~` runs YIN in its own AudioWorklet; the others take frames of 2048
|
|
17
18
|
samples through the Kairotic API (`api/tap`) and compute them with
|
|
@@ -25,6 +26,17 @@ samples through the Kairotic API (`api/tap`) and compute them with
|
|
|
25
26
|
(connect :b1 0 :b2 0) (connect :b2 0 :b3 0) (connect :b3 0 :b4 0)
|
|
26
27
|
```
|
|
27
28
|
|
|
29
|
+
To watch what a sound does over time, several descriptors side by side:
|
|
30
|
+
|
|
31
|
+
```clojure
|
|
32
|
+
(box :b1 "adc~ 1" 40 30)
|
|
33
|
+
(box :b2 "features~ 20 rms centroid" 40 90)
|
|
34
|
+
(box :b3 "chroma~" 260 90)
|
|
35
|
+
(box :b4 "monitor 6 rms centroid chroma" 40 160)
|
|
36
|
+
(connect :b1 0 :b2 0) (connect :b1 0 :b3 0)
|
|
37
|
+
(connect :b2 0 :b4 0) (connect :b2 1 :b4 1) (connect :b3 0 :b4 2)
|
|
38
|
+
```
|
|
39
|
+
|
|
28
40
|
An improvisation into notation: `adc~` → `pitch~` → `transcribe` → `freqt`, with `onset~` on the right
|
|
29
41
|
of `transcribe`; `rec`, play, `stop`; then `(view :bN :score)` on the freqt, or `(keep! …)` and `mt`.
|
|
30
42
|
|
package/dist/chroma.mjs
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
// Generated by packages/build.mjs from src/chroma.cljs
|
|
2
2
|
export default {
|
|
3
3
|
"class": "chroma~",
|
|
4
|
-
"version": "0.
|
|
4
|
+
"version": "0.3.0",
|
|
5
5
|
"author": "bonuoq",
|
|
6
6
|
"license": "MPL-2.0",
|
|
7
7
|
"doc": "How much of each pitch class the signal on its left has, every ms (40 by default): a list of 12 values from C to B (0…1, the strongest 1) on the left, and the strongest class as a word (:c, :c#, … :b) on the right when it changes. Silence sends nothing. Chords, keys and drones become data for mt and the patch. Computed with Meyda.",
|
package/dist/features.mjs
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
// Generated by packages/build.mjs from src/features.cljs
|
|
2
2
|
export default {
|
|
3
3
|
"class": "features~",
|
|
4
|
-
"version": "0.
|
|
4
|
+
"version": "0.3.0",
|
|
5
5
|
"author": "bonuoq",
|
|
6
6
|
"license": "MPL-2.0",
|
|
7
7
|
"doc": "Descriptors of the signal on its left, every ms (20 by default), one outlet for each word written after it, in order: rms, energy, zcr, loudness, centroid and spread (in Hz), flatness, rolloff (Hz), kurtosis, skewness, slope, crest, sharpness, perceptualSpread, mfcc (13 values) and chroma (12). Without words: rms centroid flatness. Each outlet sends only what changed. Computed with Meyda.",
|
package/dist/index.mjs
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
// Generated by packages/build.mjs
|
|
2
2
|
export default {
|
|
3
3
|
"package": "@kairotic/analysis",
|
|
4
|
-
"version": "0.
|
|
4
|
+
"version": "0.3.0",
|
|
5
5
|
"api": 1,
|
|
6
6
|
"author": "bonuoq",
|
|
7
7
|
"license": "MPL-2.0",
|
|
@@ -18,6 +18,12 @@ export default {
|
|
|
18
18
|
"doc": "Descriptors of the signal on its left, every ms (20 by default), one outlet for each word written after it, in order: rms, energy, zcr, loudness, centroid and spread (in Hz), flatness, rolloff (Hz), kurtosis, skewness, slope, crest, sharpness, perceptualSpread, mfcc (13 values) and chroma (12). Without words: rms centroid flatness. Each outlet sends only what changed. Computed with Meyda.",
|
|
19
19
|
"source": ";; features~ · descriptors of a sound, every few milliseconds\n;; Part of @kairotic/analysis. Written against the Kairotic API (api/…). The descriptors are Meyda's\n;; (MIT, meyda.js.org), loaded from jsDelivr the first time one is needed.\n(let [meyda-url \"npm:meyda@5.6.3/dist/web/meyda.min.js\"\n ;; every analysis box of the package uses frames of this size: Meyda is one object for the whole\n ;; page and keeps some filter banks for the size it saw first\n size 2048\n aliases {:centroid :spectralCentroid :flatness :spectralFlatness :rolloff :spectralRolloff\n :spread :spectralSpread :kurtosis :spectralKurtosis :skewness :spectralSkewness\n :slope :spectralSlope :crest :spectralCrest :sharpness :perceptualSharpness}\n known #{:rms :energy :zcr :loudness :mfcc :chroma :spectralCentroid :spectralFlatness :spectralRolloff\n :spectralSpread :spectralKurtosis :spectralSkewness :spectralSlope :spectralCrest\n :perceptualSharpness :perceptualSpread}\n feats (fn [args]\n (let [ws (->> args (filter keyword?) (map (fn [k] (get aliases k k))) (filter known) distinct vec)]\n (if (seq ws) ws [:rms :spectralCentroid :spectralFlatness])))\n hop-ms (fn [args] (api/clamp (or (first (filter number? args)) 20) 5 1000))\n ok? (fn [x] (and (number? x) (js/isFinite x)))\n ;; a value as it leaves the box: Hz where Meyda gives bins, the total of loudness, lists rounded\n value (fn [k v sr]\n (cond\n (#{:spectralCentroid :spectralSpread} k) (when (ok? v) (api/nice (* v (/ sr size))))\n (= k :loudness) (when v (let [t (.-total v)] (when (ok? t) (api/nice t))))\n (#{:mfcc :chroma} k) (when v (let [xs (vec (array-seq v))] (when (every? ok? xs) (mapv api/nice xs))))\n :else (when (ok? v) (api/nice v))))]\n {;; ── about ──\n :class \"features~\"\n :license \"MPL-2.0\"\n :doc \"Descriptors of the signal on its left, every ms (20 by default), one outlet for each word written after it, in order: rms, energy, zcr, loudness, centroid and spread (in Hz), flatness, rolloff (Hz), kurtosis, skewness, slope, crest, sharpness, perceptualSpread, mfcc (13 values) and chroma (12). Without words: rms centroid flatness. Each outlet sends only what changed. Computed with Meyda.\"\n :usage \"features~ [ms] [descriptor …]\"\n :io (fn [b] [1 (count (feats (:args b)))])\n :dynamic-io true\n\n ;; ── logic ──\n :receive (fn [{:keys [inlet value state args]}]\n (cond\n (and (= inlet :audio) (:error value)) {:effects [[:print (:error value)]]}\n (and (= inlet :audio) (map? value))\n (let [fs (feats args)\n outs (vec (keep-indexed (fn [i k] (let [v (get value k)]\n (when (and (some? v) (not= v (get state k))) [i v])))\n fs))]\n (if (seq outs) {:state (merge state (select-keys value fs)) :out outs} {}))\n :else {}))\n\n ;; ── audio ──\n :audio {:create (fn [ac b _]\n (let [in (.createGain ac)\n sr (.-sampleRate ac)\n fs (feats (:args b))\n names (clj->js (mapv name fs))\n node {:in in :M (atom nil) :tap (atom nil) :alive (atom true) :inlets [[in 0]] :outlets []}\n on-block (fn [data at & _]\n (when-let [M @(:M node)]\n (set! (.-bufferSize M) size)\n (set! (.-sampleRate M) sr)\n (let [r (.extract M names (first data))]\n (api/report! (:id b) (into {} (for [k fs] [k (value k (aget r (name k)) sr)])) at))))]\n (-> (api/load-script meyda-url)\n (.then (fn [_] (reset! (:M node) (.-Meyda js/window))))\n (.catch (fn [e] (api/report! (:id b) {:error (str \"features~: \" (.-message e))} nil))))\n (-> (api/tap ac {:mode :frames :size size :hop (js/Math.round (* sr (/ (hop-ms (:args b)) 1000)))} on-block)\n (.then (fn [w] (if @(:alive node) (do (.connect in w) (reset! (:tap node) w))\n (try (.disconnect w) (catch :default _ nil)))))\n (.catch (fn [e] (js/console.error \"features~\" e))))\n node))\n :dispose (fn [node]\n (reset! (:alive node) false)\n (try (.disconnect (:in node)) (catch :default _ nil)))}})\n"
|
|
20
20
|
},
|
|
21
|
+
{
|
|
22
|
+
"class": "monitor",
|
|
23
|
+
"license": "MPL-2.0",
|
|
24
|
+
"doc": "Draws the messages that arrive over the last seconds (6 by default), one lane per inlet: a lane for each word written after the seconds, which names it (monitor 6 rms centroid chroma has 3 inlets), or one lane without words. A number is drawn as a line in the range it has had in the window, a list of numbers as a strip with one row per element (chroma, mfcc), anything else (a bang, a word) as a mark. Time is that of the messages: the right edge is the last one. Sends nothing.",
|
|
25
|
+
"source": ";; monitor · the messages that arrive, drawn over the last seconds\n;; Part of @kairotic/analysis. Written against the Kairotic API (api/…).\n;; One lane per inlet. What a lane draws depends on what arrives: a number is a line in the range it has\n;; had in the window, a list of numbers is a strip of cells (one row per element: chroma, mfcc…), and\n;; anything else (a bang, a word) is a mark. Time is that of the messages: the right edge is the last one.\n(let [L 56 ; the column of names and values, on the left\n budget 360 ; cells for all the strips of a box (the patch draws at most 600 shapes)\n seconds (fn [args] (api/clamp (or (first (filter number? args)) 6) 0.5 120))\n names (fn [args] (vec (filter keyword? args)))\n lanes (fn [args] (let [ns (names args)] (if (seq ns) (count ns) 1)))\n kind (fn [v] (cond (number? v) :line\n (and (sequential? v) (seq v) (every? number? v)) :cells\n :else :mark))\n label (fn [v] (cond (number? v) (str (api/nice v))\n (sequential? v) (str \"[\" (count v) \"]\")\n (= v :bang) \"bang\"\n (keyword? v) (name v)\n :else (let [s (str v)] (if (> (count s) 10) (str (subs s 0 9) \"…\") s))))\n ;; keeps a value: a line keeps about 150 points in the window, a strip about 48, marks all\n ;; (64 at most); what left the window goes, but one point before it stays for the line to enter\n keep-in (fn [pts at v win]\n (let [k (kind v) [t-last v-last] (peek pts)\n step (case k :line (/ win 150) :cells (/ win 48) 0)\n pts (if (and t-last (= k (kind v-last)) (pos? step) (< (- at t-last) step))\n (conj (pop pts) [t-last v])\n (conj pts [at v]))\n pts (if (and (= k :mark) (> (count pts) 64)) (subvec pts (- (count pts) 64)) pts)\n old (count (take-while (fn [[t]] (< t (- at win))) pts))]\n (if (> old 1) (subvec pts (dec old)) pts)))]\n {;; ── about ──\n :class \"monitor\"\n :license \"MPL-2.0\"\n :doc \"Draws the messages that arrive over the last seconds (6 by default), one lane per inlet: a lane for each word written after the seconds, which names it (monitor 6 rms centroid chroma has 3 inlets), or one lane without words. A number is drawn as a line in the range it has had in the window, a list of numbers as a strip with one row per element (chroma, mfcc), anything else (a bang, a word) as a mark. Time is that of the messages: the right edge is the last one. Sends nothing.\"\n :usage \"monitor [seconds] [name …]\"\n :io (fn [b] [(lanes (:args b)) 0])\n :dynamic-io true\n :size (fn [b] [300 (+ 8 (* 44 (lanes (:args b))))])\n\n ;; ── logic ──\n ;; the state is what is drawn: {:t last instant :lanes {inlet [[at value] …]}}\n :receive (fn [{:keys [inlet value state args at]}]\n (if (and (integer? inlet) (some? value) (number? at))\n (let [win (seconds args)\n ;; the engine started again (its clock went back): start from nothing\n state (if (and (:t state) (< at (- (:t state) 1))) nil state)\n s (-> (or state {:lanes {}})\n (assoc :t (max at (or (:t state) at)))\n (update-in [:lanes inlet] (fn [pts] (keep-in (or pts []) at value win))))]\n {:state s :show s})\n {}))\n\n ;; ── view ──\n :draw (fn [{:keys [shown args w h]}]\n (let [win (seconds args) n (lanes args) ns (names args)\n lh (/ (- h 8) n) x0 L x1 (- w 6) pw (- x1 x0)\n t1 (or (:t shown) 0) t0 (- t1 win)\n tx (fn [at] (+ x0 (* pw (/ (- at t0) win))))\n strips (max 1 (count (filter (fn [i] (= :cells (kind (second (peek (get-in shown [:lanes i])))))) (range n))))]\n (into\n [[:rect {:x 2 :y 2 :w (- w 4) :h (- h 4) :r 4 :stroke :grid :fill :bg}]]\n (mapcat\n (fn [i]\n (let [pts (get-in shown [:lanes i] []) top (+ 4 (* i lh)) bottom (+ top lh -4)\n last-v (second (peek pts)) k (kind last-v)\n title (if-let [nm (get ns i)] (name nm) (str \"in \" i))]\n (concat\n [[:text {:x 8 :y (+ top 12) :size 9 :fill :ink} title]\n (when (some? last-v) [:text {:x 8 :y (+ top 24) :size 8 :fill :soft} (label last-v)])\n (when (pos? i) [:line {:x1 6 :y1 (- top 2) :x2 x1 :y2 (- top 2) :stroke :grid :weight 1}])]\n (case k\n :line\n (let [ps (filter (fn [[_ v]] (number? v)) pts)\n vs (map second ps)\n lo (if (seq vs) (apply min vs) 0) hi (if (seq vs) (apply max vs) 1)\n [lo hi] (if (< (- hi lo) 1e-9) [(- lo 0.5) (+ hi 0.5)] [lo hi])\n ty (fn [v] (- bottom (* (- bottom top 2) (/ (- v lo) (- hi lo)))))\n xys (mapv (fn [[at v]] [(api/clamp (tx at) x0 x1) (ty v)]) ps)]\n [[:text {:x 8 :y (+ top 35) :size 7 :fill :soft} (str (api/nice lo) \"…\" (api/nice hi))]\n (when (> (count xys) 1) [:path {:points xys :stroke :accent :weight 1.4}])\n (when-let [[x y] (peek xys)] [:circle {:cx x :cy y :r 2.4 :fill :accent :stroke nil}])])\n :cells\n (let [ps (filterv (fn [[at v]] (and (>= at t0) (= :cells (kind v)))) pts)\n rows (reduce max 1 (map (fn [[_ v]] (count v)) ps))\n cols (max 8 (min 48 (js/Math.floor (/ budget strips rows))))\n cw (/ pw cols) ch (/ (- bottom top) rows)\n ;; each column: the mean of what arrived in its slice of time (or the last value before)\n col-of (fn [at] (api/clamp (js/Math.floor (/ (- at t0) (/ win cols))) 0 (dec cols)))\n buckets (reduce (fn [m [at v]] (update m (col-of at) (fnil conj []) v)) {} ps)\n mean (fn [vs] (mapv (fn [j] (/ (reduce + (map (fn [v] (or (get v j) 0)) vs)) (count vs))) (range rows)))\n cells (into {} (for [[c vs] buckets] [c (mean vs)]))\n all (mapcat identity (vals cells))\n lo (if (seq all) (apply min all) 0) hi (if (seq all) (apply max all) 1)\n span (max 1e-9 (- hi lo))]\n (for [[c vs] cells [j v] (map-indexed vector vs)\n :let [u (/ (- v lo) span)] :when (> u 0.05)]\n [:rect {:x (+ x0 (* c cw)) :y (+ top (* (- rows j 1) ch)) :w (+ cw 0.4) :h (max 0.6 (- ch 0.6))\n :fill :accent :stroke nil :opacity (api/clamp u 0 1)}]))\n :mark\n (mapcat (fn [[at v]]\n (let [x (tx at)]\n (when (>= x x0)\n [[:line {:x1 x :y1 (+ top 2) :x2 x :y2 bottom :stroke :kairos :weight 1.4}]\n (when-not (= v :bang) [:text {:x (+ x 3) :y (+ top 12) :size 8 :fill :kairos} (label v)])])))\n pts)))))\n (range n)))))})\n"
|
|
26
|
+
},
|
|
21
27
|
{
|
|
22
28
|
"class": "onset~",
|
|
23
29
|
"license": "MPL-2.0",
|
|
@@ -39,8 +45,8 @@ export default {
|
|
|
39
45
|
{
|
|
40
46
|
"class": "transcribe",
|
|
41
47
|
"license": "MPL-2.0",
|
|
42
|
-
"doc": "Writes what pitch~ hears into a freqt, as strokes. Hz on the left (from pitch~: 0 when it hears none), rec to start from 0 (its first write clears the freqt) or rec t (msg rec 8) to start from beat t, keeping what is before it and stop to end; a bang on the right (from onset~) begins a new note even on the same pitch. In notes mode each note is a flat stroke at the pitch it holds longest (the freqt's tuning, or its score view, rounds it); in glide mode a stroke follows the pitch as it moves. Times are beats at bpm (60: one beat a second). Notes shorter than ms (60) are left out. It writes into the freqt at most every 2 seconds, each time one event of the history. Connect its outlet to the freqt's inlet.",
|
|
43
|
-
"source": ";; transcribe · what pitch~ hears, written as strokes of a freqt\n;; Part of @kairotic/analysis. Written against the Kairotic API (api/…). Pure logic: no audio of its own.\n(let [;; ── what it understands ──\n flush-every 2 ; seconds between two writes into the freqt (each write is one event of the history)\n max-points 64 ; points of a stroke in glide mode\n cents (fn [a b] (* 1200 (/ (js/Math.log (/ b a)) js/Math.LN2)))\n ;; the pitch a note holds longest: pitch~ only sends what changed, so a steady note is one point and\n ;; its attack many; each pitch weighs the time until the next one\n held (fn [pts t1]\n (let [ws (map (fn [[t f] [t2 _]] [f (max 0 (- t2 t))]) pts (concat (rest pts) [[t1 nil]]))\n total (reduce + (map second ws))\n sorted (sort-by first ws)]\n (if (pos? total)\n (loop [[[f w] & more] sorted acc 0]\n (let [acc (+ acc w)] (if (or (>= acc (/ total 2)) (empty? more)) f (recur more acc))))\n (second (last pts)))))\n mode-of (fn [args] (if (= :glide (:mode args)) :glide :notes))\n beats (fn [state at] (* (- at (:t0 state)) (/ (:bpm state) 60)))\n r3 (fn [x] (/ (js/Math.round (* x 1000)) 1000))\n ;; ── a note that ends: a stroke, if it lasted long enough ──\n stroke-of (fn [{:keys [pts]} t1 mode]\n (let [t0 (ffirst pts) hz (held pts t1)]\n (if (= mode :glide)\n (let [kept (reduce (fn [acc [t f]]\n (if (> (js/Math.abs (cents (second (peek acc)) f)) 10) (conj acc [t f]) acc))\n [(first pts)] (rest pts))\n kept (if (> (count kept) max-points)\n (vec (take-nth (js/Math.ceil (/ (count kept) max-points)) kept))\n kept)]\n (mapv (fn [[t f]] [(r3 t) (r3 f)]) (conj kept [t1 (second (peek pts))])))\n [[(r3 t0) (r3 hz)] [(r3 t1) (r3 hz)]])))\n end-note (fn [state t]\n (if-let [cur (:cur state)]\n (let [t0 (ffirst (:pts cur))\n long-enough (>= (/ (- t t0) (/ (:bpm state) 60)) (/ (:min-ms state) 1000))]\n (cond-> (dissoc state :cur)\n long-enough (update :done conj (stroke-of cur t (:mode state)))\n long-enough (update :count inc)))\n state))\n start-note (fn [state t hz] (assoc state :cur {:pts [[t hz]] :midi (api/hz->midi hz)}))\n ;; ── writing: the strokes that ended, as one [:batch …], at most every flush-every seconds ──\n flush (fn [state at force]\n (
|
|
48
|
+
"doc": "Writes what pitch~ hears into a freqt, as strokes. Hz on the left (from pitch~: 0 when it hears none), rec to start from 0 (its first write clears the freqt) or rec t (msg rec 8) to start from beat t, keeping what is before it, or ovr t (msg ovr 8) to write over only what the take covers, keeping what comes after and stop to end; a bang on the right (from onset~) begins a new note even on the same pitch. In notes mode each note is a flat stroke at the pitch it holds longest (the freqt's tuning, or its score view, rounds it); in glide mode a stroke follows the pitch as it moves. Times are beats at bpm (60: one beat a second). Notes shorter than ms (60) are left out. It writes into the freqt at most every 2 seconds, each time one event of the history. Connect its outlet to the freqt's inlet.",
|
|
49
|
+
"source": ";; transcribe · what pitch~ hears, written as strokes of a freqt\n;; Part of @kairotic/analysis. Written against the Kairotic API (api/…). Pure logic: no audio of its own.\n(let [;; ── what it understands ──\n flush-every 2 ; seconds between two writes into the freqt (each write is one event of the history)\n max-points 64 ; points of a stroke in glide mode\n cents (fn [a b] (* 1200 (/ (js/Math.log (/ b a)) js/Math.LN2)))\n ;; the pitch a note holds longest: pitch~ only sends what changed, so a steady note is one point and\n ;; its attack many; each pitch weighs the time until the next one\n held (fn [pts t1]\n (let [ws (map (fn [[t f] [t2 _]] [f (max 0 (- t2 t))]) pts (concat (rest pts) [[t1 nil]]))\n total (reduce + (map second ws))\n sorted (sort-by first ws)]\n (if (pos? total)\n (loop [[[f w] & more] sorted acc 0]\n (let [acc (+ acc w)] (if (or (>= acc (/ total 2)) (empty? more)) f (recur more acc))))\n (second (last pts)))))\n mode-of (fn [args] (if (= :glide (:mode args)) :glide :notes))\n beats (fn [state at] (* (- at (:t0 state)) (/ (:bpm state) 60)))\n r3 (fn [x] (/ (js/Math.round (* x 1000)) 1000))\n ;; ── a note that ends: a stroke, if it lasted long enough ──\n stroke-of (fn [{:keys [pts]} t1 mode]\n (let [t0 (ffirst pts) hz (held pts t1)]\n (if (= mode :glide)\n (let [kept (reduce (fn [acc [t f]]\n (if (> (js/Math.abs (cents (second (peek acc)) f)) 10) (conj acc [t f]) acc))\n [(first pts)] (rest pts))\n kept (if (> (count kept) max-points)\n (vec (take-nth (js/Math.ceil (/ (count kept) max-points)) kept))\n kept)]\n (mapv (fn [[t f]] [(r3 t) (r3 f)]) (conj kept [t1 (second (peek pts))])))\n [[(r3 t0) (r3 hz)] [(r3 t1) (r3 hz)]])))\n end-note (fn [state t]\n (if-let [cur (:cur state)]\n (let [t0 (ffirst (:pts cur))\n long-enough (>= (/ (- t t0) (/ (:bpm state) 60)) (/ (:min-ms state) 1000))]\n (cond-> (dissoc state :cur)\n long-enough (update :done conj (stroke-of cur t (:mode state)))\n long-enough (update :count inc)))\n state))\n start-note (fn [state t hz] (assoc state :cur {:pts [[t hz]] :midi (api/hz->midi hz)}))\n ;; ── writing: the strokes that ended, as one [:batch …], at most every flush-every seconds ──\n flush (fn [state at force]\n (let [now (r3 (beats state at))]\n (cond\n ;; ovr at stop: what the take covered since the last write is overwritten, even by silence\n (and (empty? (:done state)) (not (and force (:ovr state) (> now (:cut-from state))))) [state nil nil]\n (or force (nil? (:sent state)) (>= (- at (:sent state)) flush-every))\n ;; the first write of a take clears the freqt (as part of the same event); a take over (ovr)\n ;; cuts what it covered since the last write, and keeps what comes after\n [(cond-> (assoc state :done [] :sent at :pending false :fresh false)\n (:ovr state) (assoc :cut-from now))\n [:batch (into (cond (:ovr state) [[:cut (:cut-from state) now]]\n (:fresh state) [(if (:from state) [:cut (:from state)] [:clear])]\n :else [])\n (mapv (fn [pts] [:stroke pts]) (:done state)))] nil]\n (:pending state) [state nil nil]\n :else [(assoc state :pending true) nil [(+ (:sent state) flush-every) 0 [:flush]]])))\n result (fn [state [st edit later]]\n (cond-> {:state st :show (select-keys st [:on :count])}\n edit (assoc :out [[0 edit]])\n later (assoc :later [later])))\n ;; ── one value of pitch~ (Hz, 0 when it hears none) ──\n hear (fn [state hz at]\n (let [t (beats state at)\n cur (:cur state)\n state (cond\n (not (pos? hz)) (end-note state t)\n (nil? cur) (start-note state t hz)\n ;; notes: another note when the pitch moves more than 0.6 of a semitone away\n (and (= :notes (:mode state)) (> (js/Math.abs (- (api/hz->midi hz) (:midi cur))) 0.6))\n (-> state (end-note t) (start-note t hz))\n :else (update-in state [:cur :pts] conj [t hz]))]\n (flush state at false)))]\n {;; ── about ──\n :class \"transcribe\"\n :license \"MPL-2.0\"\n :doc \"Writes what pitch~ hears into a freqt, as strokes. Hz on the left (from pitch~: 0 when it hears none), rec to start from 0 (its first write clears the freqt) or rec t (msg rec 8) to start from beat t, keeping what is before it, or ovr t (msg ovr 8) to write over only what the take covers, keeping what comes after and stop to end; a bang on the right (from onset~) begins a new note even on the same pitch. In notes mode each note is a flat stroke at the pitch it holds longest (the freqt's tuning, or its score view, rounds it); in glide mode a stroke follows the pitch as it moves. Times are beats at bpm (60: one beat a second). Notes shorter than ms (60) are left out. It writes into the freqt at most every 2 seconds, each time one event of the history. Connect its outlet to the freqt's inlet.\"\n :usage \"transcribe [bpm] [notes|glide] [ms]\"\n :io [2 1]\n :size [124 30]\n :args [[:bpm 60] [:mode nil] [:min-ms 60]]\n\n ;; ── logic ──\n :receive (fn [{:keys [inlet value state args at]}]\n (let [state (or state {})]\n (cond\n ;; rec: a take from 0 (its first write clears the freqt); [:rec t] (msg rec 8): from beat t on,\n ;; keeping what is before t (punch-in)\n ;; ovr t (msg ovr 8): over what is there from beat t to where the take stops, keeping what comes after\n (or (#{:rec :ovr} value) (and (vector? value) (#{:rec :ovr} (first value))))\n (let [bpm (api/clamp (:bpm args) 10 400)\n ovr (or (= value :ovr) (and (vector? value) (= :ovr (first value))))\n from (or (when (vector? value) (let [x (second value)] (when (and (number? x) (>= x 0)) x)))\n (when ovr 0))]\n {:state (cond-> {:on true :t0 (- at (* (or from 0) (/ 60 bpm))) :bpm bpm :mode (mode-of args)\n :min-ms (api/clamp (:min-ms args) 0 2000) :done [] :count 0 :fresh true}\n from (assoc :from from)\n ovr (assoc :ovr true :cut-from from))\n :show {:on true :count 0}})\n (not (:on state)) {}\n (= value :stop)\n (let [st (end-note state (beats state at))\n [st edit _] (flush st at true)]\n (cond-> {:state (assoc st :on false) :show {:on false :count (:count st)}}\n edit (assoc :out [[0 edit]])))\n (= value [:flush]) (result state (flush (assoc state :pending false) at false))\n ;; an onset: the note that sounds ends here and a new one begins at its pitch\n (and (= inlet 1) (:cur state))\n (let [t (beats state at) hz (second (peek (get-in state [:cur :pts])))]\n (result state (flush (-> state (end-note t) (start-note t hz)) at false)))\n (and (= inlet 0) (number? value)) (result state (hear state value at))\n :else {})))\n\n ;; ── view ──\n :derive (fn [{:keys [shown]}]\n {:label (if (:on shown) (str \"● transcribing · \" (or (:count shown) 0)) (str \"transcribe · \" (or (:count shown) 0) \" notes\"))})\n :view [[:rect {:x 2 :y 2 :w 120 :h 26 :r 3 :stroke :grid :fill :bg}]\n [:text {:x 8 :y 19 :size 10 :fill :soft} :$label]]})\n"
|
|
44
50
|
}
|
|
45
51
|
]
|
|
46
52
|
};
|
package/dist/monitor.mjs
ADDED
|
@@ -0,0 +1,9 @@
|
|
|
1
|
+
// Generated by packages/build.mjs from src/monitor.cljs
|
|
2
|
+
export default {
|
|
3
|
+
"class": "monitor",
|
|
4
|
+
"version": "0.3.0",
|
|
5
|
+
"author": "bonuoq",
|
|
6
|
+
"license": "MPL-2.0",
|
|
7
|
+
"doc": "Draws the messages that arrive over the last seconds (6 by default), one lane per inlet: a lane for each word written after the seconds, which names it (monitor 6 rms centroid chroma has 3 inlets), or one lane without words. A number is drawn as a line in the range it has had in the window, a list of numbers as a strip with one row per element (chroma, mfcc), anything else (a bang, a word) as a mark. Time is that of the messages: the right edge is the last one. Sends nothing.",
|
|
8
|
+
"source": ";; monitor · the messages that arrive, drawn over the last seconds\n;; Part of @kairotic/analysis. Written against the Kairotic API (api/…).\n;; One lane per inlet. What a lane draws depends on what arrives: a number is a line in the range it has\n;; had in the window, a list of numbers is a strip of cells (one row per element: chroma, mfcc…), and\n;; anything else (a bang, a word) is a mark. Time is that of the messages: the right edge is the last one.\n(let [L 56 ; the column of names and values, on the left\n budget 360 ; cells for all the strips of a box (the patch draws at most 600 shapes)\n seconds (fn [args] (api/clamp (or (first (filter number? args)) 6) 0.5 120))\n names (fn [args] (vec (filter keyword? args)))\n lanes (fn [args] (let [ns (names args)] (if (seq ns) (count ns) 1)))\n kind (fn [v] (cond (number? v) :line\n (and (sequential? v) (seq v) (every? number? v)) :cells\n :else :mark))\n label (fn [v] (cond (number? v) (str (api/nice v))\n (sequential? v) (str \"[\" (count v) \"]\")\n (= v :bang) \"bang\"\n (keyword? v) (name v)\n :else (let [s (str v)] (if (> (count s) 10) (str (subs s 0 9) \"…\") s))))\n ;; keeps a value: a line keeps about 150 points in the window, a strip about 48, marks all\n ;; (64 at most); what left the window goes, but one point before it stays for the line to enter\n keep-in (fn [pts at v win]\n (let [k (kind v) [t-last v-last] (peek pts)\n step (case k :line (/ win 150) :cells (/ win 48) 0)\n pts (if (and t-last (= k (kind v-last)) (pos? step) (< (- at t-last) step))\n (conj (pop pts) [t-last v])\n (conj pts [at v]))\n pts (if (and (= k :mark) (> (count pts) 64)) (subvec pts (- (count pts) 64)) pts)\n old (count (take-while (fn [[t]] (< t (- at win))) pts))]\n (if (> old 1) (subvec pts (dec old)) pts)))]\n {;; ── about ──\n :class \"monitor\"\n :license \"MPL-2.0\"\n :doc \"Draws the messages that arrive over the last seconds (6 by default), one lane per inlet: a lane for each word written after the seconds, which names it (monitor 6 rms centroid chroma has 3 inlets), or one lane without words. A number is drawn as a line in the range it has had in the window, a list of numbers as a strip with one row per element (chroma, mfcc), anything else (a bang, a word) as a mark. Time is that of the messages: the right edge is the last one. Sends nothing.\"\n :usage \"monitor [seconds] [name …]\"\n :io (fn [b] [(lanes (:args b)) 0])\n :dynamic-io true\n :size (fn [b] [300 (+ 8 (* 44 (lanes (:args b))))])\n\n ;; ── logic ──\n ;; the state is what is drawn: {:t last instant :lanes {inlet [[at value] …]}}\n :receive (fn [{:keys [inlet value state args at]}]\n (if (and (integer? inlet) (some? value) (number? at))\n (let [win (seconds args)\n ;; the engine started again (its clock went back): start from nothing\n state (if (and (:t state) (< at (- (:t state) 1))) nil state)\n s (-> (or state {:lanes {}})\n (assoc :t (max at (or (:t state) at)))\n (update-in [:lanes inlet] (fn [pts] (keep-in (or pts []) at value win))))]\n {:state s :show s})\n {}))\n\n ;; ── view ──\n :draw (fn [{:keys [shown args w h]}]\n (let [win (seconds args) n (lanes args) ns (names args)\n lh (/ (- h 8) n) x0 L x1 (- w 6) pw (- x1 x0)\n t1 (or (:t shown) 0) t0 (- t1 win)\n tx (fn [at] (+ x0 (* pw (/ (- at t0) win))))\n strips (max 1 (count (filter (fn [i] (= :cells (kind (second (peek (get-in shown [:lanes i])))))) (range n))))]\n (into\n [[:rect {:x 2 :y 2 :w (- w 4) :h (- h 4) :r 4 :stroke :grid :fill :bg}]]\n (mapcat\n (fn [i]\n (let [pts (get-in shown [:lanes i] []) top (+ 4 (* i lh)) bottom (+ top lh -4)\n last-v (second (peek pts)) k (kind last-v)\n title (if-let [nm (get ns i)] (name nm) (str \"in \" i))]\n (concat\n [[:text {:x 8 :y (+ top 12) :size 9 :fill :ink} title]\n (when (some? last-v) [:text {:x 8 :y (+ top 24) :size 8 :fill :soft} (label last-v)])\n (when (pos? i) [:line {:x1 6 :y1 (- top 2) :x2 x1 :y2 (- top 2) :stroke :grid :weight 1}])]\n (case k\n :line\n (let [ps (filter (fn [[_ v]] (number? v)) pts)\n vs (map second ps)\n lo (if (seq vs) (apply min vs) 0) hi (if (seq vs) (apply max vs) 1)\n [lo hi] (if (< (- hi lo) 1e-9) [(- lo 0.5) (+ hi 0.5)] [lo hi])\n ty (fn [v] (- bottom (* (- bottom top 2) (/ (- v lo) (- hi lo)))))\n xys (mapv (fn [[at v]] [(api/clamp (tx at) x0 x1) (ty v)]) ps)]\n [[:text {:x 8 :y (+ top 35) :size 7 :fill :soft} (str (api/nice lo) \"…\" (api/nice hi))]\n (when (> (count xys) 1) [:path {:points xys :stroke :accent :weight 1.4}])\n (when-let [[x y] (peek xys)] [:circle {:cx x :cy y :r 2.4 :fill :accent :stroke nil}])])\n :cells\n (let [ps (filterv (fn [[at v]] (and (>= at t0) (= :cells (kind v)))) pts)\n rows (reduce max 1 (map (fn [[_ v]] (count v)) ps))\n cols (max 8 (min 48 (js/Math.floor (/ budget strips rows))))\n cw (/ pw cols) ch (/ (- bottom top) rows)\n ;; each column: the mean of what arrived in its slice of time (or the last value before)\n col-of (fn [at] (api/clamp (js/Math.floor (/ (- at t0) (/ win cols))) 0 (dec cols)))\n buckets (reduce (fn [m [at v]] (update m (col-of at) (fnil conj []) v)) {} ps)\n mean (fn [vs] (mapv (fn [j] (/ (reduce + (map (fn [v] (or (get v j) 0)) vs)) (count vs))) (range rows)))\n cells (into {} (for [[c vs] buckets] [c (mean vs)]))\n all (mapcat identity (vals cells))\n lo (if (seq all) (apply min all) 0) hi (if (seq all) (apply max all) 1)\n span (max 1e-9 (- hi lo))]\n (for [[c vs] cells [j v] (map-indexed vector vs)\n :let [u (/ (- v lo) span)] :when (> u 0.05)]\n [:rect {:x (+ x0 (* c cw)) :y (+ top (* (- rows j 1) ch)) :w (+ cw 0.4) :h (max 0.6 (- ch 0.6))\n :fill :accent :stroke nil :opacity (api/clamp u 0 1)}]))\n :mark\n (mapcat (fn [[at v]]\n (let [x (tx at)]\n (when (>= x x0)\n [[:line {:x1 x :y1 (+ top 2) :x2 x :y2 bottom :stroke :kairos :weight 1.4}]\n (when-not (= v :bang) [:text {:x (+ x 3) :y (+ top 12) :size 8 :fill :kairos} (label v)])])))\n pts)))))\n (range n)))))})\n"
|
|
9
|
+
};
|
package/dist/onset.mjs
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
// Generated by packages/build.mjs from src/onset.cljs
|
|
2
2
|
export default {
|
|
3
3
|
"class": "onset~",
|
|
4
|
-
"version": "0.
|
|
4
|
+
"version": "0.3.0",
|
|
5
5
|
"author": "bonuoq",
|
|
6
6
|
"license": "MPL-2.0",
|
|
7
7
|
"doc": "The beginnings of sounds in the signal on its left — a note, a stroke, a syllable —: a bang on the left for each, and how strong it was (the growth of its spectrum, 0…1) on the right. threshold (3) is how far above the usual change of the last frames an onset must rise; ms (80) the least time between two. Every 512 samples (about 11 ms), on frames of 2048. Into a counter, a metro that follows, a sample's rec.",
|
package/dist/pitch.mjs
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
// Generated by packages/build.mjs from src/pitch.cljs
|
|
2
2
|
export default {
|
|
3
3
|
"class": "pitch~",
|
|
4
|
-
"version": "0.
|
|
4
|
+
"version": "0.3.0",
|
|
5
5
|
"author": "bonuoq",
|
|
6
6
|
"license": "MPL-2.0",
|
|
7
7
|
"doc": "The pitch of the signal on its left, by YIN, every 1024 samples (about 21 ms): its frequency in Hz on the left, when it finds one between lo and hi (60 and 1500 by default), and 0 once when it no longer does (silence, noise, a chord); how clearly it is periodic (0…1) on the right. It sends only what changed. threshold (0.15) is how strict it is: lower finds fewer pitches, surer ones. Into ftom (Hz → MIDI pitch), a synth~, or a faust~.",
|
package/dist/spectrum.mjs
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
// Generated by packages/build.mjs from src/spectrum.cljs
|
|
2
2
|
export default {
|
|
3
3
|
"class": "spectrum~",
|
|
4
|
-
"version": "0.
|
|
4
|
+
"version": "0.3.0",
|
|
5
5
|
"author": "bonuoq",
|
|
6
6
|
"license": "MPL-2.0",
|
|
7
7
|
"doc": "The spectrum of the signal on its left, drawn in the patch about 30 times a second: frequency on a log scale from lo to hi Hz (40 and 16000 by default), level in dB from -90 to 0. Computed with Meyda.",
|
package/dist/transcribe.mjs
CHANGED
|
@@ -1,9 +1,9 @@
|
|
|
1
1
|
// Generated by packages/build.mjs from src/transcribe.cljs
|
|
2
2
|
export default {
|
|
3
3
|
"class": "transcribe",
|
|
4
|
-
"version": "0.
|
|
4
|
+
"version": "0.3.0",
|
|
5
5
|
"author": "bonuoq",
|
|
6
6
|
"license": "MPL-2.0",
|
|
7
|
-
"doc": "Writes what pitch~ hears into a freqt, as strokes. Hz on the left (from pitch~: 0 when it hears none), rec to start from 0 (its first write clears the freqt) or rec t (msg rec 8) to start from beat t, keeping what is before it and stop to end; a bang on the right (from onset~) begins a new note even on the same pitch. In notes mode each note is a flat stroke at the pitch it holds longest (the freqt's tuning, or its score view, rounds it); in glide mode a stroke follows the pitch as it moves. Times are beats at bpm (60: one beat a second). Notes shorter than ms (60) are left out. It writes into the freqt at most every 2 seconds, each time one event of the history. Connect its outlet to the freqt's inlet.",
|
|
8
|
-
"source": ";; transcribe · what pitch~ hears, written as strokes of a freqt\n;; Part of @kairotic/analysis. Written against the Kairotic API (api/…). Pure logic: no audio of its own.\n(let [;; ── what it understands ──\n flush-every 2 ; seconds between two writes into the freqt (each write is one event of the history)\n max-points 64 ; points of a stroke in glide mode\n cents (fn [a b] (* 1200 (/ (js/Math.log (/ b a)) js/Math.LN2)))\n ;; the pitch a note holds longest: pitch~ only sends what changed, so a steady note is one point and\n ;; its attack many; each pitch weighs the time until the next one\n held (fn [pts t1]\n (let [ws (map (fn [[t f] [t2 _]] [f (max 0 (- t2 t))]) pts (concat (rest pts) [[t1 nil]]))\n total (reduce + (map second ws))\n sorted (sort-by first ws)]\n (if (pos? total)\n (loop [[[f w] & more] sorted acc 0]\n (let [acc (+ acc w)] (if (or (>= acc (/ total 2)) (empty? more)) f (recur more acc))))\n (second (last pts)))))\n mode-of (fn [args] (if (= :glide (:mode args)) :glide :notes))\n beats (fn [state at] (* (- at (:t0 state)) (/ (:bpm state) 60)))\n r3 (fn [x] (/ (js/Math.round (* x 1000)) 1000))\n ;; ── a note that ends: a stroke, if it lasted long enough ──\n stroke-of (fn [{:keys [pts]} t1 mode]\n (let [t0 (ffirst pts) hz (held pts t1)]\n (if (= mode :glide)\n (let [kept (reduce (fn [acc [t f]]\n (if (> (js/Math.abs (cents (second (peek acc)) f)) 10) (conj acc [t f]) acc))\n [(first pts)] (rest pts))\n kept (if (> (count kept) max-points)\n (vec (take-nth (js/Math.ceil (/ (count kept) max-points)) kept))\n kept)]\n (mapv (fn [[t f]] [(r3 t) (r3 f)]) (conj kept [t1 (second (peek pts))])))\n [[(r3 t0) (r3 hz)] [(r3 t1) (r3 hz)]])))\n end-note (fn [state t]\n (if-let [cur (:cur state)]\n (let [t0 (ffirst (:pts cur))\n long-enough (>= (/ (- t t0) (/ (:bpm state) 60)) (/ (:min-ms state) 1000))]\n (cond-> (dissoc state :cur)\n long-enough (update :done conj (stroke-of cur t (:mode state)))\n long-enough (update :count inc)))\n state))\n start-note (fn [state t hz] (assoc state :cur {:pts [[t hz]] :midi (api/hz->midi hz)}))\n ;; ── writing: the strokes that ended, as one [:batch …], at most every flush-every seconds ──\n flush (fn [state at force]\n (
|
|
7
|
+
"doc": "Writes what pitch~ hears into a freqt, as strokes. Hz on the left (from pitch~: 0 when it hears none), rec to start from 0 (its first write clears the freqt) or rec t (msg rec 8) to start from beat t, keeping what is before it, or ovr t (msg ovr 8) to write over only what the take covers, keeping what comes after and stop to end; a bang on the right (from onset~) begins a new note even on the same pitch. In notes mode each note is a flat stroke at the pitch it holds longest (the freqt's tuning, or its score view, rounds it); in glide mode a stroke follows the pitch as it moves. Times are beats at bpm (60: one beat a second). Notes shorter than ms (60) are left out. It writes into the freqt at most every 2 seconds, each time one event of the history. Connect its outlet to the freqt's inlet.",
|
|
8
|
+
"source": ";; transcribe · what pitch~ hears, written as strokes of a freqt\n;; Part of @kairotic/analysis. Written against the Kairotic API (api/…). Pure logic: no audio of its own.\n(let [;; ── what it understands ──\n flush-every 2 ; seconds between two writes into the freqt (each write is one event of the history)\n max-points 64 ; points of a stroke in glide mode\n cents (fn [a b] (* 1200 (/ (js/Math.log (/ b a)) js/Math.LN2)))\n ;; the pitch a note holds longest: pitch~ only sends what changed, so a steady note is one point and\n ;; its attack many; each pitch weighs the time until the next one\n held (fn [pts t1]\n (let [ws (map (fn [[t f] [t2 _]] [f (max 0 (- t2 t))]) pts (concat (rest pts) [[t1 nil]]))\n total (reduce + (map second ws))\n sorted (sort-by first ws)]\n (if (pos? total)\n (loop [[[f w] & more] sorted acc 0]\n (let [acc (+ acc w)] (if (or (>= acc (/ total 2)) (empty? more)) f (recur more acc))))\n (second (last pts)))))\n mode-of (fn [args] (if (= :glide (:mode args)) :glide :notes))\n beats (fn [state at] (* (- at (:t0 state)) (/ (:bpm state) 60)))\n r3 (fn [x] (/ (js/Math.round (* x 1000)) 1000))\n ;; ── a note that ends: a stroke, if it lasted long enough ──\n stroke-of (fn [{:keys [pts]} t1 mode]\n (let [t0 (ffirst pts) hz (held pts t1)]\n (if (= mode :glide)\n (let [kept (reduce (fn [acc [t f]]\n (if (> (js/Math.abs (cents (second (peek acc)) f)) 10) (conj acc [t f]) acc))\n [(first pts)] (rest pts))\n kept (if (> (count kept) max-points)\n (vec (take-nth (js/Math.ceil (/ (count kept) max-points)) kept))\n kept)]\n (mapv (fn [[t f]] [(r3 t) (r3 f)]) (conj kept [t1 (second (peek pts))])))\n [[(r3 t0) (r3 hz)] [(r3 t1) (r3 hz)]])))\n end-note (fn [state t]\n (if-let [cur (:cur state)]\n (let [t0 (ffirst (:pts cur))\n long-enough (>= (/ (- t t0) (/ (:bpm state) 60)) (/ (:min-ms state) 1000))]\n (cond-> (dissoc state :cur)\n long-enough (update :done conj (stroke-of cur t (:mode state)))\n long-enough (update :count inc)))\n state))\n start-note (fn [state t hz] (assoc state :cur {:pts [[t hz]] :midi (api/hz->midi hz)}))\n ;; ── writing: the strokes that ended, as one [:batch …], at most every flush-every seconds ──\n flush (fn [state at force]\n (let [now (r3 (beats state at))]\n (cond\n ;; ovr at stop: what the take covered since the last write is overwritten, even by silence\n (and (empty? (:done state)) (not (and force (:ovr state) (> now (:cut-from state))))) [state nil nil]\n (or force (nil? (:sent state)) (>= (- at (:sent state)) flush-every))\n ;; the first write of a take clears the freqt (as part of the same event); a take over (ovr)\n ;; cuts what it covered since the last write, and keeps what comes after\n [(cond-> (assoc state :done [] :sent at :pending false :fresh false)\n (:ovr state) (assoc :cut-from now))\n [:batch (into (cond (:ovr state) [[:cut (:cut-from state) now]]\n (:fresh state) [(if (:from state) [:cut (:from state)] [:clear])]\n :else [])\n (mapv (fn [pts] [:stroke pts]) (:done state)))] nil]\n (:pending state) [state nil nil]\n :else [(assoc state :pending true) nil [(+ (:sent state) flush-every) 0 [:flush]]])))\n result (fn [state [st edit later]]\n (cond-> {:state st :show (select-keys st [:on :count])}\n edit (assoc :out [[0 edit]])\n later (assoc :later [later])))\n ;; ── one value of pitch~ (Hz, 0 when it hears none) ──\n hear (fn [state hz at]\n (let [t (beats state at)\n cur (:cur state)\n state (cond\n (not (pos? hz)) (end-note state t)\n (nil? cur) (start-note state t hz)\n ;; notes: another note when the pitch moves more than 0.6 of a semitone away\n (and (= :notes (:mode state)) (> (js/Math.abs (- (api/hz->midi hz) (:midi cur))) 0.6))\n (-> state (end-note t) (start-note t hz))\n :else (update-in state [:cur :pts] conj [t hz]))]\n (flush state at false)))]\n {;; ── about ──\n :class \"transcribe\"\n :license \"MPL-2.0\"\n :doc \"Writes what pitch~ hears into a freqt, as strokes. Hz on the left (from pitch~: 0 when it hears none), rec to start from 0 (its first write clears the freqt) or rec t (msg rec 8) to start from beat t, keeping what is before it, or ovr t (msg ovr 8) to write over only what the take covers, keeping what comes after and stop to end; a bang on the right (from onset~) begins a new note even on the same pitch. In notes mode each note is a flat stroke at the pitch it holds longest (the freqt's tuning, or its score view, rounds it); in glide mode a stroke follows the pitch as it moves. Times are beats at bpm (60: one beat a second). Notes shorter than ms (60) are left out. It writes into the freqt at most every 2 seconds, each time one event of the history. Connect its outlet to the freqt's inlet.\"\n :usage \"transcribe [bpm] [notes|glide] [ms]\"\n :io [2 1]\n :size [124 30]\n :args [[:bpm 60] [:mode nil] [:min-ms 60]]\n\n ;; ── logic ──\n :receive (fn [{:keys [inlet value state args at]}]\n (let [state (or state {})]\n (cond\n ;; rec: a take from 0 (its first write clears the freqt); [:rec t] (msg rec 8): from beat t on,\n ;; keeping what is before t (punch-in)\n ;; ovr t (msg ovr 8): over what is there from beat t to where the take stops, keeping what comes after\n (or (#{:rec :ovr} value) (and (vector? value) (#{:rec :ovr} (first value))))\n (let [bpm (api/clamp (:bpm args) 10 400)\n ovr (or (= value :ovr) (and (vector? value) (= :ovr (first value))))\n from (or (when (vector? value) (let [x (second value)] (when (and (number? x) (>= x 0)) x)))\n (when ovr 0))]\n {:state (cond-> {:on true :t0 (- at (* (or from 0) (/ 60 bpm))) :bpm bpm :mode (mode-of args)\n :min-ms (api/clamp (:min-ms args) 0 2000) :done [] :count 0 :fresh true}\n from (assoc :from from)\n ovr (assoc :ovr true :cut-from from))\n :show {:on true :count 0}})\n (not (:on state)) {}\n (= value :stop)\n (let [st (end-note state (beats state at))\n [st edit _] (flush st at true)]\n (cond-> {:state (assoc st :on false) :show {:on false :count (:count st)}}\n edit (assoc :out [[0 edit]])))\n (= value [:flush]) (result state (flush (assoc state :pending false) at false))\n ;; an onset: the note that sounds ends here and a new one begins at its pitch\n (and (= inlet 1) (:cur state))\n (let [t (beats state at) hz (second (peek (get-in state [:cur :pts])))]\n (result state (flush (-> state (end-note t) (start-note t hz)) at false)))\n (and (= inlet 0) (number? value)) (result state (hear state value at))\n :else {})))\n\n ;; ── view ──\n :derive (fn [{:keys [shown]}]\n {:label (if (:on shown) (str \"● transcribing · \" (or (:count shown) 0)) (str \"transcribe · \" (or (:count shown) 0) \" notes\"))})\n :view [[:rect {:x 2 :y 2 :w 120 :h 26 :r 3 :stroke :grid :fill :bg}]\n [:text {:x 8 :y 19 :size 10 :fill :soft} :$label]]})\n"
|
|
9
9
|
};
|
package/package.json
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@kairotic/analysis",
|
|
3
|
-
"version": "0.
|
|
4
|
-
"description": "Kairotic analysis components: features~ (descriptors with Meyda), pitch~ (YIN), onset~, chroma~ spectrum
|
|
3
|
+
"version": "0.3.0",
|
|
4
|
+
"description": "Kairotic analysis components: features~ (descriptors with Meyda), pitch~ (YIN), onset~, chroma~ spectrum~, transcribe (pitch~ into a freqt) and monitor (any stream of messages, drawn), from any signal or adc~ into messages and pictures",
|
|
5
5
|
"author": "bonuoq",
|
|
6
6
|
"license": "MPL-2.0",
|
|
7
7
|
"repository": {
|
|
@@ -36,6 +36,7 @@
|
|
|
36
36
|
"onset",
|
|
37
37
|
"chroma",
|
|
38
38
|
"meyda",
|
|
39
|
-
"transcription"
|
|
39
|
+
"transcription",
|
|
40
|
+
"monitor"
|
|
40
41
|
]
|
|
41
42
|
}
|
package/src/monitor.cljs
ADDED
|
@@ -0,0 +1,105 @@
|
|
|
1
|
+
;; monitor · the messages that arrive, drawn over the last seconds
|
|
2
|
+
;; Part of @kairotic/analysis. Written against the Kairotic API (api/…).
|
|
3
|
+
;; One lane per inlet. What a lane draws depends on what arrives: a number is a line in the range it has
|
|
4
|
+
;; had in the window, a list of numbers is a strip of cells (one row per element: chroma, mfcc…), and
|
|
5
|
+
;; anything else (a bang, a word) is a mark. Time is that of the messages: the right edge is the last one.
|
|
6
|
+
(let [L 56 ; the column of names and values, on the left
|
|
7
|
+
budget 360 ; cells for all the strips of a box (the patch draws at most 600 shapes)
|
|
8
|
+
seconds (fn [args] (api/clamp (or (first (filter number? args)) 6) 0.5 120))
|
|
9
|
+
names (fn [args] (vec (filter keyword? args)))
|
|
10
|
+
lanes (fn [args] (let [ns (names args)] (if (seq ns) (count ns) 1)))
|
|
11
|
+
kind (fn [v] (cond (number? v) :line
|
|
12
|
+
(and (sequential? v) (seq v) (every? number? v)) :cells
|
|
13
|
+
:else :mark))
|
|
14
|
+
label (fn [v] (cond (number? v) (str (api/nice v))
|
|
15
|
+
(sequential? v) (str "[" (count v) "]")
|
|
16
|
+
(= v :bang) "bang"
|
|
17
|
+
(keyword? v) (name v)
|
|
18
|
+
:else (let [s (str v)] (if (> (count s) 10) (str (subs s 0 9) "…") s))))
|
|
19
|
+
;; keeps a value: a line keeps about 150 points in the window, a strip about 48, marks all
|
|
20
|
+
;; (64 at most); what left the window goes, but one point before it stays for the line to enter
|
|
21
|
+
keep-in (fn [pts at v win]
|
|
22
|
+
(let [k (kind v) [t-last v-last] (peek pts)
|
|
23
|
+
step (case k :line (/ win 150) :cells (/ win 48) 0)
|
|
24
|
+
pts (if (and t-last (= k (kind v-last)) (pos? step) (< (- at t-last) step))
|
|
25
|
+
(conj (pop pts) [t-last v])
|
|
26
|
+
(conj pts [at v]))
|
|
27
|
+
pts (if (and (= k :mark) (> (count pts) 64)) (subvec pts (- (count pts) 64)) pts)
|
|
28
|
+
old (count (take-while (fn [[t]] (< t (- at win))) pts))]
|
|
29
|
+
(if (> old 1) (subvec pts (dec old)) pts)))]
|
|
30
|
+
{;; ── about ──
|
|
31
|
+
:class "monitor"
|
|
32
|
+
:license "MPL-2.0"
|
|
33
|
+
:doc "Draws the messages that arrive over the last seconds (6 by default), one lane per inlet: a lane for each word written after the seconds, which names it (monitor 6 rms centroid chroma has 3 inlets), or one lane without words. A number is drawn as a line in the range it has had in the window, a list of numbers as a strip with one row per element (chroma, mfcc), anything else (a bang, a word) as a mark. Time is that of the messages: the right edge is the last one. Sends nothing."
|
|
34
|
+
:usage "monitor [seconds] [name …]"
|
|
35
|
+
:io (fn [b] [(lanes (:args b)) 0])
|
|
36
|
+
:dynamic-io true
|
|
37
|
+
:size (fn [b] [300 (+ 8 (* 44 (lanes (:args b))))])
|
|
38
|
+
|
|
39
|
+
;; ── logic ──
|
|
40
|
+
;; the state is what is drawn: {:t last instant :lanes {inlet [[at value] …]}}
|
|
41
|
+
:receive (fn [{:keys [inlet value state args at]}]
|
|
42
|
+
(if (and (integer? inlet) (some? value) (number? at))
|
|
43
|
+
(let [win (seconds args)
|
|
44
|
+
;; the engine started again (its clock went back): start from nothing
|
|
45
|
+
state (if (and (:t state) (< at (- (:t state) 1))) nil state)
|
|
46
|
+
s (-> (or state {:lanes {}})
|
|
47
|
+
(assoc :t (max at (or (:t state) at)))
|
|
48
|
+
(update-in [:lanes inlet] (fn [pts] (keep-in (or pts []) at value win))))]
|
|
49
|
+
{:state s :show s})
|
|
50
|
+
{}))
|
|
51
|
+
|
|
52
|
+
;; ── view ──
|
|
53
|
+
:draw (fn [{:keys [shown args w h]}]
|
|
54
|
+
(let [win (seconds args) n (lanes args) ns (names args)
|
|
55
|
+
lh (/ (- h 8) n) x0 L x1 (- w 6) pw (- x1 x0)
|
|
56
|
+
t1 (or (:t shown) 0) t0 (- t1 win)
|
|
57
|
+
tx (fn [at] (+ x0 (* pw (/ (- at t0) win))))
|
|
58
|
+
strips (max 1 (count (filter (fn [i] (= :cells (kind (second (peek (get-in shown [:lanes i])))))) (range n))))]
|
|
59
|
+
(into
|
|
60
|
+
[[:rect {:x 2 :y 2 :w (- w 4) :h (- h 4) :r 4 :stroke :grid :fill :bg}]]
|
|
61
|
+
(mapcat
|
|
62
|
+
(fn [i]
|
|
63
|
+
(let [pts (get-in shown [:lanes i] []) top (+ 4 (* i lh)) bottom (+ top lh -4)
|
|
64
|
+
last-v (second (peek pts)) k (kind last-v)
|
|
65
|
+
title (if-let [nm (get ns i)] (name nm) (str "in " i))]
|
|
66
|
+
(concat
|
|
67
|
+
[[:text {:x 8 :y (+ top 12) :size 9 :fill :ink} title]
|
|
68
|
+
(when (some? last-v) [:text {:x 8 :y (+ top 24) :size 8 :fill :soft} (label last-v)])
|
|
69
|
+
(when (pos? i) [:line {:x1 6 :y1 (- top 2) :x2 x1 :y2 (- top 2) :stroke :grid :weight 1}])]
|
|
70
|
+
(case k
|
|
71
|
+
:line
|
|
72
|
+
(let [ps (filter (fn [[_ v]] (number? v)) pts)
|
|
73
|
+
vs (map second ps)
|
|
74
|
+
lo (if (seq vs) (apply min vs) 0) hi (if (seq vs) (apply max vs) 1)
|
|
75
|
+
[lo hi] (if (< (- hi lo) 1e-9) [(- lo 0.5) (+ hi 0.5)] [lo hi])
|
|
76
|
+
ty (fn [v] (- bottom (* (- bottom top 2) (/ (- v lo) (- hi lo)))))
|
|
77
|
+
xys (mapv (fn [[at v]] [(api/clamp (tx at) x0 x1) (ty v)]) ps)]
|
|
78
|
+
[[:text {:x 8 :y (+ top 35) :size 7 :fill :soft} (str (api/nice lo) "…" (api/nice hi))]
|
|
79
|
+
(when (> (count xys) 1) [:path {:points xys :stroke :accent :weight 1.4}])
|
|
80
|
+
(when-let [[x y] (peek xys)] [:circle {:cx x :cy y :r 2.4 :fill :accent :stroke nil}])])
|
|
81
|
+
:cells
|
|
82
|
+
(let [ps (filterv (fn [[at v]] (and (>= at t0) (= :cells (kind v)))) pts)
|
|
83
|
+
rows (reduce max 1 (map (fn [[_ v]] (count v)) ps))
|
|
84
|
+
cols (max 8 (min 48 (js/Math.floor (/ budget strips rows))))
|
|
85
|
+
cw (/ pw cols) ch (/ (- bottom top) rows)
|
|
86
|
+
;; each column: the mean of what arrived in its slice of time (or the last value before)
|
|
87
|
+
col-of (fn [at] (api/clamp (js/Math.floor (/ (- at t0) (/ win cols))) 0 (dec cols)))
|
|
88
|
+
buckets (reduce (fn [m [at v]] (update m (col-of at) (fnil conj []) v)) {} ps)
|
|
89
|
+
mean (fn [vs] (mapv (fn [j] (/ (reduce + (map (fn [v] (or (get v j) 0)) vs)) (count vs))) (range rows)))
|
|
90
|
+
cells (into {} (for [[c vs] buckets] [c (mean vs)]))
|
|
91
|
+
all (mapcat identity (vals cells))
|
|
92
|
+
lo (if (seq all) (apply min all) 0) hi (if (seq all) (apply max all) 1)
|
|
93
|
+
span (max 1e-9 (- hi lo))]
|
|
94
|
+
(for [[c vs] cells [j v] (map-indexed vector vs)
|
|
95
|
+
:let [u (/ (- v lo) span)] :when (> u 0.05)]
|
|
96
|
+
[:rect {:x (+ x0 (* c cw)) :y (+ top (* (- rows j 1) ch)) :w (+ cw 0.4) :h (max 0.6 (- ch 0.6))
|
|
97
|
+
:fill :accent :stroke nil :opacity (api/clamp u 0 1)}]))
|
|
98
|
+
:mark
|
|
99
|
+
(mapcat (fn [[at v]]
|
|
100
|
+
(let [x (tx at)]
|
|
101
|
+
(when (>= x x0)
|
|
102
|
+
[[:line {:x1 x :y1 (+ top 2) :x2 x :y2 bottom :stroke :kairos :weight 1.4}]
|
|
103
|
+
(when-not (= v :bang) [:text {:x (+ x 3) :y (+ top 12) :size 8 :fill :kairos} (label v)])])))
|
|
104
|
+
pts)))))
|
|
105
|
+
(range n)))))})
|
package/src/transcribe.cljs
CHANGED
|
@@ -40,15 +40,21 @@
|
|
|
40
40
|
start-note (fn [state t hz] (assoc state :cur {:pts [[t hz]] :midi (api/hz->midi hz)}))
|
|
41
41
|
;; ── writing: the strokes that ended, as one [:batch …], at most every flush-every seconds ──
|
|
42
42
|
flush (fn [state at force]
|
|
43
|
-
(
|
|
44
|
-
(
|
|
45
|
-
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
|
|
49
|
-
|
|
50
|
-
|
|
51
|
-
|
|
43
|
+
(let [now (r3 (beats state at))]
|
|
44
|
+
(cond
|
|
45
|
+
;; ovr at stop: what the take covered since the last write is overwritten, even by silence
|
|
46
|
+
(and (empty? (:done state)) (not (and force (:ovr state) (> now (:cut-from state))))) [state nil nil]
|
|
47
|
+
(or force (nil? (:sent state)) (>= (- at (:sent state)) flush-every))
|
|
48
|
+
;; the first write of a take clears the freqt (as part of the same event); a take over (ovr)
|
|
49
|
+
;; cuts what it covered since the last write, and keeps what comes after
|
|
50
|
+
[(cond-> (assoc state :done [] :sent at :pending false :fresh false)
|
|
51
|
+
(:ovr state) (assoc :cut-from now))
|
|
52
|
+
[:batch (into (cond (:ovr state) [[:cut (:cut-from state) now]]
|
|
53
|
+
(:fresh state) [(if (:from state) [:cut (:from state)] [:clear])]
|
|
54
|
+
:else [])
|
|
55
|
+
(mapv (fn [pts] [:stroke pts]) (:done state)))] nil]
|
|
56
|
+
(:pending state) [state nil nil]
|
|
57
|
+
:else [(assoc state :pending true) nil [(+ (:sent state) flush-every) 0 [:flush]]])))
|
|
52
58
|
result (fn [state [st edit later]]
|
|
53
59
|
(cond-> {:state st :show (select-keys st [:on :count])}
|
|
54
60
|
edit (assoc :out [[0 edit]])
|
|
@@ -68,7 +74,7 @@
|
|
|
68
74
|
{;; ── about ──
|
|
69
75
|
:class "transcribe"
|
|
70
76
|
:license "MPL-2.0"
|
|
71
|
-
:doc "Writes what pitch~ hears into a freqt, as strokes. Hz on the left (from pitch~: 0 when it hears none), rec to start from 0 (its first write clears the freqt) or rec t (msg rec 8) to start from beat t, keeping what is before it and stop to end; a bang on the right (from onset~) begins a new note even on the same pitch. In notes mode each note is a flat stroke at the pitch it holds longest (the freqt's tuning, or its score view, rounds it); in glide mode a stroke follows the pitch as it moves. Times are beats at bpm (60: one beat a second). Notes shorter than ms (60) are left out. It writes into the freqt at most every 2 seconds, each time one event of the history. Connect its outlet to the freqt's inlet."
|
|
77
|
+
:doc "Writes what pitch~ hears into a freqt, as strokes. Hz on the left (from pitch~: 0 when it hears none), rec to start from 0 (its first write clears the freqt) or rec t (msg rec 8) to start from beat t, keeping what is before it, or ovr t (msg ovr 8) to write over only what the take covers, keeping what comes after and stop to end; a bang on the right (from onset~) begins a new note even on the same pitch. In notes mode each note is a flat stroke at the pitch it holds longest (the freqt's tuning, or its score view, rounds it); in glide mode a stroke follows the pitch as it moves. Times are beats at bpm (60: one beat a second). Notes shorter than ms (60) are left out. It writes into the freqt at most every 2 seconds, each time one event of the history. Connect its outlet to the freqt's inlet."
|
|
72
78
|
:usage "transcribe [bpm] [notes|glide] [ms]"
|
|
73
79
|
:io [2 1]
|
|
74
80
|
:size [124 30]
|
|
@@ -80,12 +86,16 @@
|
|
|
80
86
|
(cond
|
|
81
87
|
;; rec: a take from 0 (its first write clears the freqt); [:rec t] (msg rec 8): from beat t on,
|
|
82
88
|
;; keeping what is before t (punch-in)
|
|
83
|
-
|
|
89
|
+
;; ovr t (msg ovr 8): over what is there from beat t to where the take stops, keeping what comes after
|
|
90
|
+
(or (#{:rec :ovr} value) (and (vector? value) (#{:rec :ovr} (first value))))
|
|
84
91
|
(let [bpm (api/clamp (:bpm args) 10 400)
|
|
85
|
-
|
|
92
|
+
ovr (or (= value :ovr) (and (vector? value) (= :ovr (first value))))
|
|
93
|
+
from (or (when (vector? value) (let [x (second value)] (when (and (number? x) (>= x 0)) x)))
|
|
94
|
+
(when ovr 0))]
|
|
86
95
|
{:state (cond-> {:on true :t0 (- at (* (or from 0) (/ 60 bpm))) :bpm bpm :mode (mode-of args)
|
|
87
96
|
:min-ms (api/clamp (:min-ms args) 0 2000) :done [] :count 0 :fresh true}
|
|
88
|
-
from (assoc :from from)
|
|
97
|
+
from (assoc :from from)
|
|
98
|
+
ovr (assoc :ovr true :cut-from from))
|
|
89
99
|
:show {:on true :count 0}})
|
|
90
100
|
(not (:on state)) {}
|
|
91
101
|
(= value :stop)
|