@plotdb/lotion 0.1.0 → 0.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +13 -0
- package/README.md +14 -26
- package/cli.js +21 -32
- package/package.json +1 -1
- package/prompt/{explainer.md → lotion.md} +7 -29
- package/audio/beats.py +0 -93
- package/audio/bgm.py +0 -181
- package/audio/common.py +0 -30
- package/audio/mix.py +0 -51
- package/audio/sfx.py +0 -93
- package/prompt/ui-loop.md +0 -497
package/CHANGELOG.md
CHANGED
|
@@ -1,3 +1,16 @@
|
|
|
1
|
+
## v0.2.0
|
|
2
|
+
|
|
3
|
+
- breaking:
|
|
4
|
+
- remove audio generators ( `audio/`, `lotion bgm | sfx | mix | beats` ); the commands now only print a notice.
|
|
5
|
+
they moved to the authoring toolkit `@plotdb/lotitor` ( not public yet )
|
|
6
|
+
- move `prompt/explainer.md` and `prompt/ui-loop.md` to `@plotdb/lotitor`
|
|
7
|
+
- features:
|
|
8
|
+
- `prompt/lotion.md`: how to write a lotion animation, independent of form and style
|
|
9
|
+
( split from `prompt/explainer.md` )
|
|
10
|
+
- tweaks:
|
|
11
|
+
- cli: when `playwright` is missing, print how to install it instead of a node stack trace
|
|
12
|
+
|
|
13
|
+
|
|
1
14
|
## v0.1.0
|
|
2
15
|
|
|
3
16
|
- features:
|
package/README.md
CHANGED
|
@@ -8,8 +8,11 @@ play, scrub, embed, or render them frame by frame into video.
|
|
|
8
8
|
- block player ( `dist/block/player.js` ): package an animation as a `@plotdb/block` and embed it anywhere.
|
|
9
9
|
- cli ( `lotion` ): render any page following the render protocol into png frames, contact sheets or mp4,
|
|
10
10
|
and bundle blocks.
|
|
11
|
-
-
|
|
12
|
-
|
|
11
|
+
- prompt ( `prompt/lotion.md` ): how to write a lotion animation correctly, for people and AI agents.
|
|
12
|
+
|
|
13
|
+
lotion is self-contained for writing, playing and rendering animations. Authoring aids ( guides for specific forms,
|
|
14
|
+
a style library, asset and audio generation ) are kept in a separate toolkit, `@plotdb/lotitor`, which is not public
|
|
15
|
+
yet; nothing in lotion depends on it.
|
|
13
16
|
|
|
14
17
|
|
|
15
18
|
## Install
|
|
@@ -125,7 +128,7 @@ targets under the root ( or use `:scope` ). A block sample is in `web/src/pug/bl
|
|
|
125
128
|
lotion video <src> <out.mp4> render to mp4
|
|
126
129
|
lotion cues <src> <out.json> dump window.cues()
|
|
127
130
|
lotion bundle <base-url> <block> <out> pack a block and its dependencies into one file
|
|
128
|
-
lotion bgm | sfx | mix | beats
|
|
131
|
+
lotion bgm | sfx | mix | beats removed, see Audio
|
|
129
132
|
|
|
130
133
|
`<src>` is an http(s) url, or a local html file / directory served by a built-in static server.
|
|
131
134
|
|
|
@@ -143,36 +146,21 @@ options:
|
|
|
143
146
|
|
|
144
147
|
## Audio
|
|
145
148
|
|
|
146
|
-
|
|
147
|
-
|
|
148
|
-
|
|
149
|
-
|
|
150
|
-
synthesize a track around a repeating bassline. bass presets: `pulse`, `octave`, `house`, `drive`, `walk`,
|
|
151
|
-
or 16 custom steps ( semitones from the chord root, `.` rest, `-` hold ). drum presets: `four`, `half`, `broken`,
|
|
152
|
-
`none`. `--loop` makes it loop seamlessly; `--grid grid.json` writes its beat grid.
|
|
153
|
-
- `lotion beats music.mp3 --start 64.11 --dur 30 --out grid.json`: estimate bpm and the beat grid
|
|
154
|
-
( `{bpm, offset, beats, downbeats}` ). with an off-beat bassline and syncopated kicks the grid may be half a beat
|
|
155
|
-
off; correct it with `--shift 0.5`.
|
|
156
|
-
- `lotion sfx sfx/`: synthesize ui sound effects ( click, tick, scrub, pop, toggle, success, notify, type, whoosh )
|
|
157
|
-
and measure each transient peak into `sfx/peaks.json`.
|
|
158
|
-
- `lotion mix cues.json out.wav --dur 30 --music bgm.wav --sfx sfx/ [--loop | --fade 2]`: mix music with sound
|
|
159
|
-
effects placed at cue times ( `[{t, sfx, gain}]`, e.g. from `lotion cues` ), aligning each effect's measured
|
|
160
|
-
peak rather than its file start.
|
|
161
|
-
|
|
162
|
-
A typical flow:
|
|
163
|
-
|
|
164
|
-
lotion bgm bgm.wav --bpm 120 --bars 16 --grid grid.json # time the animation on this grid
|
|
165
|
-
lotion sfx sfx/
|
|
149
|
+
The audio generators ( `bgm`, `sfx`, `mix`, `beats` ) are no longer part of lotion; `lotion bgm ...` and the like
|
|
150
|
+
only print a notice. Sound stays aligned through cues: the page's `cues()` lists `[{t, ...}]`, `lotion cues` exports
|
|
151
|
+
it, and any audio tool can place sounds at those times. Then add the track to the rendered video:
|
|
152
|
+
|
|
166
153
|
lotion cues page.html cues.json
|
|
167
|
-
lotion mix cues.json audio.wav --dur 32 --music bgm.wav --sfx sfx/ --fade 2
|
|
168
154
|
lotion video page.html silent.mp4
|
|
169
155
|
ffmpeg -i silent.mp4 -i audio.wav -c:v copy -c:a aac -shortest out.mp4
|
|
170
156
|
|
|
171
157
|
|
|
172
158
|
## Prompts
|
|
173
159
|
|
|
174
|
-
- `prompt/
|
|
175
|
-
|
|
160
|
+
- `prompt/lotion.md`: writing `seek(t)`, embedding into pages, block packaging, and common pitfalls.
|
|
161
|
+
|
|
162
|
+
Guides for specific forms ( explainers, UI loops ) and for choosing a style moved to the authoring toolkit
|
|
163
|
+
`@plotdb/lotitor` ( not public yet ). `prompt/lotion.md` covers everything needed to write a lotion animation.
|
|
176
164
|
|
|
177
165
|
|
|
178
166
|
## Development
|
package/cli.js
CHANGED
|
@@ -1,10 +1,10 @@
|
|
|
1
1
|
#!/usr/bin/env node
|
|
2
|
-
var fs, path, http, child_process, usage, parse, types, serve, open, shot, frames, sheet, video, cues, bundle,
|
|
2
|
+
var fs, path, http, child_process, usage, parse, types, serve, playwright, open, shot, frames, sheet, video, cues, bundle, moved, argv, ref$, pos, opt, cmd, src, out, rest, times, p, slice$ = [].slice;
|
|
3
3
|
fs = require('fs');
|
|
4
4
|
path = require('path');
|
|
5
5
|
http = require('http');
|
|
6
6
|
child_process = require('child_process');
|
|
7
|
-
usage = 'usage:\n lotion <frames|sheet|video|cues> <src> <out> [t...] [options]\n lotion bundle <base-url> <block-name> <out> [options]\n
|
|
7
|
+
usage = 'usage:\n lotion <frames|sheet|video|cues> <src> <out> [t...] [options]\n lotion bundle <base-url> <block-name> <out> [options]\n audio generators ( bgm | sfx | mix | beats ) were removed, see README > Audio\nsee README for options.';
|
|
8
8
|
parse = function(argv){
|
|
9
9
|
var ref$, pos, opt, i, a;
|
|
10
10
|
ref$ = [[], {}], pos = ref$[0], opt = ref$[1];
|
|
@@ -75,9 +75,22 @@ serve = function(src){
|
|
|
75
75
|
});
|
|
76
76
|
});
|
|
77
77
|
};
|
|
78
|
+
playwright = function(){
|
|
79
|
+
var e;
|
|
80
|
+
try {
|
|
81
|
+
return require('playwright');
|
|
82
|
+
} catch (e$) {
|
|
83
|
+
e = e$;
|
|
84
|
+
if (e.code !== 'MODULE_NOT_FOUND' || !/playwright/.test(e.message)) {
|
|
85
|
+
throw e;
|
|
86
|
+
}
|
|
87
|
+
console.error('lotion: this command needs playwright, which is not installed. install it with:\n\n npm i -D playwright\n npx playwright install chromium');
|
|
88
|
+
return process.exit(1);
|
|
89
|
+
}
|
|
90
|
+
};
|
|
78
91
|
open = function(src, opt){
|
|
79
92
|
var chromium, p;
|
|
80
|
-
chromium =
|
|
93
|
+
chromium = playwright().chromium;
|
|
81
94
|
p = /^https?:/.test(src)
|
|
82
95
|
? Promise.resolve({
|
|
83
96
|
url: src,
|
|
@@ -260,7 +273,7 @@ cues = function(src, out, opt){
|
|
|
260
273
|
};
|
|
261
274
|
bundle = function(base, name, out, opt){
|
|
262
275
|
var chromium, m;
|
|
263
|
-
chromium =
|
|
276
|
+
chromium = playwright().chromium;
|
|
264
277
|
base = base.replace(/\/$/, '');
|
|
265
278
|
m = function(f){
|
|
266
279
|
return require.resolve(f);
|
|
@@ -322,37 +335,13 @@ bundle = function(base, name, out, opt){
|
|
|
322
335
|
});
|
|
323
336
|
});
|
|
324
337
|
};
|
|
325
|
-
|
|
326
|
-
|
|
327
|
-
|
|
328
|
-
});
|
|
329
|
-
};
|
|
330
|
-
audio = function(cmd, args){
|
|
331
|
-
return new Promise(function(res, rej){
|
|
332
|
-
var d, f, py;
|
|
333
|
-
if (!(d = audioDir())) {
|
|
334
|
-
return rej(new Error("audio scripts not found"));
|
|
335
|
-
}
|
|
336
|
-
f = path.join(d, cmd + ".py");
|
|
337
|
-
py = child_process.spawn(process.env.PYTHON || 'python3', [f].concat(args), {
|
|
338
|
-
stdio: 'inherit'
|
|
339
|
-
});
|
|
340
|
-
py.on('error', rej);
|
|
341
|
-
return py.on('close', function(c){
|
|
342
|
-
if (c) {
|
|
343
|
-
return rej(new Error(cmd + ".py exited with " + c));
|
|
344
|
-
} else {
|
|
345
|
-
return res();
|
|
346
|
-
}
|
|
347
|
-
});
|
|
348
|
-
});
|
|
338
|
+
moved = function(cmd){
|
|
339
|
+
console.error("lotion " + cmd + ": audio generators are no longer part of lotion ( moved to @plotdb/lotitor, not public yet ).\nto align any audio with the animation, export cue times with `lotion cues <src> cues.json`. see README > Audio.");
|
|
340
|
+
return process.exit(1);
|
|
349
341
|
};
|
|
350
342
|
argv = process.argv.slice(2);
|
|
351
343
|
if ((ref$ = argv[0]) === 'bgm' || ref$ === 'sfx' || ref$ === 'mix' || ref$ === 'beats') {
|
|
352
|
-
|
|
353
|
-
console.error(e.message);
|
|
354
|
-
return process.exit(1);
|
|
355
|
-
});
|
|
344
|
+
moved(argv[0]);
|
|
356
345
|
} else {
|
|
357
346
|
ref$ = parse(argv), pos = ref$.pos, opt = ref$.opt;
|
|
358
347
|
cmd = pos[0], src = pos[1], out = pos[2], rest = slice$.call(pos, 3);
|
package/package.json
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"author":"zbryikt","name":"@plotdb/lotion","version":"0.
|
|
1
|
+
{"author":"zbryikt","name":"@plotdb/lotion","version":"0.2.0","description":"deterministic motion toolkit: animation as a pure function of time, with player, frame renderer and block packaging","license":"MIT","main":"index.min.js","browser":"index.min.js","bin":{"lotion":"./cli.js"},"homepage":"https://github.com/plotdb/lotion","repository":{"type":"git","url":"git+https://github.com/plotdb/lotion.git"},"scripts":{"start":"npx server -r web -o true"},"engines":{"node":">=14"},"keywords":["motion","animation","video","playwright","block"],"dependencies":{"@plotdb/block":"^5.7.3","@plotdb/csscope":"^5.0.4","@plotdb/rescope":"^5.1.0","@plotdb/semver":"^0.0.4","proxise":"^1.0.1"},"peerDependencies":{"playwright":">=1.40"},"peerDependenciesMeta":{"playwright":{"optional":true}},"devDependencies":{"@plotdb/guides":"github:plotdb/guides","@zbryikt/template":"^2.5.0","fedep":"^1.9.0","livescript":"^1.6.0","playwright":"^1.63.0","stylus":"^0.64.0","uglify-js":"^3.19.3"},"frontendDependencies":{"root":"web/static/assets/lib","modules":[]}}
|
|
@@ -1,31 +1,17 @@
|
|
|
1
|
-
#
|
|
1
|
+
# 寫 lotion 動畫
|
|
2
2
|
|
|
3
|
-
|
|
4
|
-
|
|
5
|
-
|
|
6
|
-
|
|
7
|
-
|
|
8
|
-
## 流程
|
|
9
|
-
|
|
10
|
-
1. 先寫故事線:一句話一個段落 ( chapter ),每段只推進一個概念。
|
|
11
|
-
概念之間若有缺口 ( 例如要證明「一定有贏家」之前,得先介紹「路徑強度」),主動補段並告知。
|
|
12
|
-
2. 準備資料:用真正的函式庫計算,畫面上的數字一律取自計算結果,旁白再依資料撰寫。
|
|
13
|
-
需要特定性質的資料 ( 例如出現循環、但最後能排出完整順序 ) 時,用程式搜尋,不要手湊。
|
|
14
|
-
3. 排時間軸:以段落起點加相對時間描述所有事件,方便整段平移或調整長度。
|
|
15
|
-
4. 實作 `seek(t)`,再用 `lotion sheet` 輸出各段代表畫面檢查版面,修正後再輸出影片。
|
|
3
|
+
所有 lotion 動畫都適用的寫法與陷阱,與動畫的形式、風格無關。
|
|
4
|
+
寫 lotion 動畫只需要這份文件。特定形式 ( 解說片、UI loop ) 的做法、風格庫、素材與音訊生成,
|
|
5
|
+
另有製作工具 `@plotdb/lotitor` ( 目前未公開 );沒有它不影響這裡的任何內容。
|
|
16
6
|
|
|
17
7
|
|
|
18
8
|
## 原則
|
|
19
9
|
|
|
20
10
|
- 畫面是時間的純函數:`seek(t)` 只依 `t` 計算,不保留跨影格狀態。
|
|
21
11
|
播放、拖曳、跳段與逐格輸出都只是用不同的 `t` 呼叫它。
|
|
22
|
-
-
|
|
23
|
-
|
|
24
|
-
-
|
|
25
|
-
旁白進出場用淡入淡出加輕微模糊,舊的出場與新的進場不可重疊在同一位置。
|
|
26
|
-
- 強調只用一種語彙:凸顯 = 加粗變深,其它變淡;錯誤 / 矛盾 = 紅色。避免同時有多種強調方式。
|
|
27
|
-
- 圖的配置要先避開交錯:例如四個節點的完全圖,把「輸給所有人」的節點放在三角形中央即可平面化。
|
|
28
|
-
- 標籤不要壓在連線上:名字預設放右側,遇到連線時個別調整,移動時以彈簧插值切換位置。
|
|
12
|
+
- `seek` 必須對任何 `t` 都成立,包含 0 與總長之後。
|
|
13
|
+
- 寫完先用 `lotion sheet` 輸出各段頭、尾、交界與轉場中間點的畫面檢查,再輸出影片。
|
|
14
|
+
- 音效時間點由頁面的 `cues()` 提供,與畫面共用同一組時間函數,`lotion cues` 可匯出。
|
|
29
15
|
|
|
30
16
|
|
|
31
17
|
## 嵌入網頁時
|
|
@@ -42,14 +28,6 @@
|
|
|
42
28
|
以 block 封裝時,interface 要帶上 `ready: player.ready`,render 協定才會等它就緒。
|
|
43
29
|
|
|
44
30
|
|
|
45
|
-
## 配樂與音效
|
|
46
|
-
|
|
47
|
-
- 先決定節奏再排時間軸:`lotion bgm --bpm 120 --grid grid.json` 產生配樂與節拍格線,段落起點與重要轉場對齊拍點。
|
|
48
|
-
使用現成音樂時以 `lotion beats` 分析,並確認第一拍是否偏了半拍。
|
|
49
|
-
- 音效以 cue 描述 ( `[{t, sfx, gain}]` ),由頁面的 `cues()` 提供,與畫面共用同一份時間軸;
|
|
50
|
-
混音時對齊的是音效量測到的 transient peak,而不是檔案開頭。
|
|
51
|
-
|
|
52
|
-
|
|
53
31
|
## 可攜的封裝
|
|
54
32
|
|
|
55
33
|
- 要放進其它網站時,把動畫包成 `@plotdb/block`:依賴 ( 包含 lotion ) 以 `pkg.dependencies` 宣告,
|
package/audio/beats.py
DELETED
|
@@ -1,93 +0,0 @@
|
|
|
1
|
-
# 節拍分析: 估計 BPM 與第一拍位置, 輸出節拍格線 ( 整理自 0926-motion 的 bpm / grid / refine ).
|
|
2
|
-
# python audio/beats.py <music> [--start s] [--dur s] [--min 70] [--max 180] [--out grid.json]
|
|
3
|
-
# - BPM: onset 包絡的自相關, 同時考慮 1 / 2 / 4 倍拍長, 再以相位一致性細調到 0.005 BPM
|
|
4
|
-
# - 第一拍: 在一個週期內掃描起點, 取節拍位置上包絡總和最大者 ( 以低頻為主 ); 並補償分析窗的延遲
|
|
5
|
-
# - 小節起點: 比較 4 個相位上低頻 ( kick ) 的強度, 取最強者
|
|
6
|
-
# - 限制: bassline 落在反拍、kick 又是切分時, 拍點可能偏半拍; 以 --shift 0.5 ( 單位: 拍 ) 校正
|
|
7
|
-
# - 輸出: {bpm, offset, beats, downbeats}; 時間皆相對於 --start
|
|
8
|
-
import numpy as np, argparse, json
|
|
9
|
-
from common import load
|
|
10
|
-
|
|
11
|
-
SR = 22050
|
|
12
|
-
HOP = 256
|
|
13
|
-
FPS = SR / HOP
|
|
14
|
-
|
|
15
|
-
def frames(x, n=1024):
|
|
16
|
-
m = (len(x) - n) // HOP
|
|
17
|
-
return np.lib.stride_tricks.as_strided(x, (m, n), (x.strides[0] * HOP, x.strides[0])) * np.hanning(n)
|
|
18
|
-
|
|
19
|
-
def onset_env(x):
|
|
20
|
-
S = np.log1p(100 * np.abs(np.fft.rfft(frames(x), axis=1)))
|
|
21
|
-
fl = np.maximum(0, np.diff(S, axis=0)).sum(1)
|
|
22
|
-
fl = fl - np.convolve(fl, np.ones(32) / 32, 'same')
|
|
23
|
-
return np.maximum(fl, 0)
|
|
24
|
-
|
|
25
|
-
def low_env(x, cut=150):
|
|
26
|
-
S = np.abs(np.fft.rfft(frames(x), axis=1))
|
|
27
|
-
f = np.fft.rfftfreq(1024, 1 / SR)
|
|
28
|
-
return np.maximum(0, np.diff(np.log1p(100 * S[:, f < cut]).sum(1)))
|
|
29
|
-
|
|
30
|
-
def coarse_tempo(env, lo, hi):
|
|
31
|
-
e = env - env.mean()
|
|
32
|
-
ac = np.correlate(e, e, 'full')[len(e) - 1:]
|
|
33
|
-
bpms = np.arange(lo, hi + 0.01, 0.1)
|
|
34
|
-
def score(b):
|
|
35
|
-
lag, s = 60 * FPS / b, 0
|
|
36
|
-
for k in (1, 2, 4):
|
|
37
|
-
L = lag * k; i = int(L); f = L - i
|
|
38
|
-
if i + 1 < len(ac): s += (ac[i] * (1 - f) + ac[i + 1] * f) / k
|
|
39
|
-
return s
|
|
40
|
-
sc = np.array([score(b) for b in bpms])
|
|
41
|
-
return bpms[sc.argmax()], sc.max() / ac[0]
|
|
42
|
-
|
|
43
|
-
def fine_tempo(env, b0):
|
|
44
|
-
t = np.arange(len(env)) / FPS
|
|
45
|
-
bs = np.arange(b0 - 1, b0 + 1, 0.005)
|
|
46
|
-
sc = [abs(sum(np.sum(env * np.exp(-2j * np.pi * (b / 60) * k * t)) for k in (1, 2, 4))) for b in bs]
|
|
47
|
-
return bs[int(np.argmax(sc))]
|
|
48
|
-
|
|
49
|
-
# 分析窗的延遲: diff 後第 i 個值反映的是第 i + 1 個窗, 窗的中心再晚半個窗長
|
|
50
|
-
LAG = (HOP + 512) / SR
|
|
51
|
-
|
|
52
|
-
def norm(v): return v / (v.max() + 1e-9)
|
|
53
|
-
|
|
54
|
-
def grid(x, bpm):
|
|
55
|
-
env = onset_env(x); low = low_env(x)
|
|
56
|
-
m = min(len(env), len(low))
|
|
57
|
-
# 拍點以低頻 ( kick / bass 起音 ) 為主, 避免被反拍的 hat 或 16 分音符拉偏
|
|
58
|
-
e = norm(low[:m]) * 2 + norm(env[:m])
|
|
59
|
-
t = np.arange(m) / FPS
|
|
60
|
-
period = 60 / bpm
|
|
61
|
-
idx = np.arange(m)
|
|
62
|
-
def score(o): return np.interp(np.arange(o, t[-1] - 0.1, period) * FPS, idx, e).sum()
|
|
63
|
-
cands = np.arange(0, period, 0.001)
|
|
64
|
-
off = cands[int(np.argmax([score(o) for o in cands]))]
|
|
65
|
-
beats = np.arange(off, t[-1] - 0.1, period)
|
|
66
|
-
lb = np.interp(beats * FPS, np.arange(len(low)), low)
|
|
67
|
-
phase = int(np.argmax([lb[i::4].mean() for i in range(4)]))
|
|
68
|
-
beats = beats + LAG
|
|
69
|
-
return beats, beats[phase::4]
|
|
70
|
-
|
|
71
|
-
if __name__ == '__main__':
|
|
72
|
-
ap = argparse.ArgumentParser(description='estimate bpm and beat grid of a music file')
|
|
73
|
-
ap.add_argument('music')
|
|
74
|
-
ap.add_argument('--start', type=float, default=0)
|
|
75
|
-
ap.add_argument('--dur', type=float, default=None)
|
|
76
|
-
ap.add_argument('--min', type=float, default=70)
|
|
77
|
-
ap.add_argument('--max', type=float, default=180)
|
|
78
|
-
ap.add_argument('--out', default=None, help='write grid json here')
|
|
79
|
-
ap.add_argument('--shift', type=float, default=0, help='shift the grid by this many beats')
|
|
80
|
-
a = ap.parse_args()
|
|
81
|
-
x = load(a.music, sr=SR, start=a.start, dur=a.dur)
|
|
82
|
-
b0, conf = coarse_tempo(onset_env(x), a.min, a.max)
|
|
83
|
-
bpm = fine_tempo(onset_env(x), b0)
|
|
84
|
-
beats, downbeats = grid(x, bpm)
|
|
85
|
-
if a.shift:
|
|
86
|
-
d = a.shift * 60 / bpm
|
|
87
|
-
beats, downbeats = beats + d, downbeats + d
|
|
88
|
-
print('bpm %.3f ( coarse %.1f, confidence %.2f ) first beat %.4fs first downbeat %.4fs beats %d'
|
|
89
|
-
% (bpm, b0, conf, beats[0], downbeats[0], len(beats)))
|
|
90
|
-
if a.out:
|
|
91
|
-
json.dump({'bpm': round(float(bpm), 3), 'offset': round(float(beats[0]), 4),
|
|
92
|
-
'beats': [round(float(v), 4) for v in beats], 'downbeats': [round(float(v), 4) for v in downbeats]},
|
|
93
|
-
open(a.out, 'w'), indent=1)
|
package/audio/bgm.py
DELETED
|
@@ -1,181 +0,0 @@
|
|
|
1
|
-
# 簡易背景音樂合成: 以重複的 bassline 為骨幹, 加上鼓組與和弦墊音. 只依賴 numpy.
|
|
2
|
-
# python audio/bgm.py <out.wav> [--bpm 120] [--bars 16] [--key A] [--scale minor] [--prog 1,6,3,7]
|
|
3
|
-
# [--bass pulse] [--drums four] [--pad] [--intro 2] [--outro 2] [--loop] [--grid grid.json]
|
|
4
|
-
#
|
|
5
|
-
# 設計:
|
|
6
|
-
# - 和弦進行以音階級數表示 ( --prog 1,6,3,7 ), 每小節一個和弦, 循環使用
|
|
7
|
-
# - bassline 是一小節 16 格的樣式 ( --bass ), 每格為相對於和弦根音的半音數, '.' 為休止; 每小節重複, 隨和弦移調
|
|
8
|
-
# - 鼓組樣式 ( --drums ): kick / hat / clap 各 16 格
|
|
9
|
-
# - 編排: 前 --intro 小節只有 bass 與 hat, 最後 --outro 小節逐漸收掉; --loop 時不做 intro / outro,
|
|
10
|
-
# 並把超出尾端的音尾繞回開頭, 可無縫循環
|
|
11
|
-
# - t = 0 即第一小節的第一拍, 方便與動畫時間軸對齊; --grid 會輸出節拍格線 ( 與 beats.py 格式相同 )
|
|
12
|
-
import numpy as np, argparse, json
|
|
13
|
-
from common import SR, write, normalize
|
|
14
|
-
|
|
15
|
-
NOTE = {'C': 0, 'C#': 1, 'Db': 1, 'D': 2, 'D#': 3, 'Eb': 3, 'E': 4, 'F': 5, 'F#': 6, 'Gb': 6,
|
|
16
|
-
'G': 7, 'G#': 8, 'Ab': 8, 'A': 9, 'A#': 10, 'Bb': 10, 'B': 11}
|
|
17
|
-
SCALE = {
|
|
18
|
-
'major': [0, 2, 4, 5, 7, 9, 11], 'minor': [0, 2, 3, 5, 7, 8, 10],
|
|
19
|
-
'dorian': [0, 2, 3, 5, 7, 9, 10], 'mixolydian': [0, 2, 4, 5, 7, 9, 10]
|
|
20
|
-
}
|
|
21
|
-
# bassline 樣式: 16 格, 數字為相對和弦根音的半音數, '.' 休止, '-' 延續前一音
|
|
22
|
-
BASS = {
|
|
23
|
-
'pulse': '0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0',
|
|
24
|
-
'octave': '0 . 12 . 0 . 12 . 0 . 12 . 0 . 12 .',
|
|
25
|
-
'house': '. . 0 . . . 0 . . . 0 . . . 0 12',
|
|
26
|
-
'drive': '0 . 0 12 . 0 7 . 0 . 0 12 . 7 5 .',
|
|
27
|
-
'walk': '0 - - - 7 - - - 12 - - - 7 - 5 -',
|
|
28
|
-
}
|
|
29
|
-
DRUMS = {
|
|
30
|
-
'four': {'kick': 'x...x...x...x...', 'hat': '..x...x...x...x.', 'clap': '....x.......x...'},
|
|
31
|
-
'half': {'kick': 'x.......x.......', 'hat': 'x.x.x.x.x.x.x.x.', 'clap': '........x.......'},
|
|
32
|
-
'broken': {'kick': 'x.....x...x.....', 'hat': '..x...x...x...xx', 'clap': '....x.......x..x'},
|
|
33
|
-
'none': {'kick': '................', 'hat': '................', 'clap': '................'},
|
|
34
|
-
}
|
|
35
|
-
|
|
36
|
-
rng = np.random.default_rng(3)
|
|
37
|
-
|
|
38
|
-
def hz(midi): return 440 * 2 ** ((midi - 69) / 12)
|
|
39
|
-
|
|
40
|
-
def env(n, a, d, sustain=0.0, r=0.0):
|
|
41
|
-
"""attack / 指數衰減至 sustain; 結尾 r 秒線性釋放"""
|
|
42
|
-
t = np.arange(n) / SR
|
|
43
|
-
e = (1 - np.exp(-t / max(a, 1e-4))) * (sustain + (1 - sustain) * np.exp(-t / d))
|
|
44
|
-
if r > 0:
|
|
45
|
-
k = min(n, int(r * SR)); e[n - k:] *= np.linspace(1, 0, k)
|
|
46
|
-
return e
|
|
47
|
-
|
|
48
|
-
def saw(f, n, detune=0.0):
|
|
49
|
-
t = np.arange(n) / SR
|
|
50
|
-
x = 2 * ((f * t) % 1) - 1
|
|
51
|
-
if detune: x = 0.5 * x + 0.5 * (2 * ((f * (1 + detune) * t + 0.37) % 1) - 1)
|
|
52
|
-
return x
|
|
53
|
-
|
|
54
|
-
def lowpass(x, cutoff, q=0.9):
|
|
55
|
-
"""時變截止頻率的 state variable filter. cutoff 可為陣列 ( 每個 sample 一個值 )"""
|
|
56
|
-
cutoff = np.broadcast_to(cutoff, x.shape)
|
|
57
|
-
f = 2 * np.sin(np.pi * np.clip(cutoff, 20, SR / 6) / SR)
|
|
58
|
-
low = band = 0.0
|
|
59
|
-
y = np.empty_like(x)
|
|
60
|
-
damp = 1 / q
|
|
61
|
-
for i in range(len(x)):
|
|
62
|
-
low += f[i] * band
|
|
63
|
-
high = x[i] - low - damp * band
|
|
64
|
-
band += f[i] * high
|
|
65
|
-
y[i] = low
|
|
66
|
-
return y
|
|
67
|
-
|
|
68
|
-
def bass_note(midi, dur, accent=1.0):
|
|
69
|
-
n = int(dur * SR)
|
|
70
|
-
x = saw(hz(midi), n, detune=0.004) + 0.5 * np.sin(2 * np.pi * hz(midi - 12) * np.arange(n) / SR)
|
|
71
|
-
cut = 180 + 1400 * accent * np.exp(-np.arange(n) / SR / 0.09)
|
|
72
|
-
return lowpass(x, cut, q=1.4) * env(n, 0.003, 0.25, 0.55, 0.02) * 0.5
|
|
73
|
-
|
|
74
|
-
def kick():
|
|
75
|
-
n = int(0.35 * SR); t = np.arange(n) / SR
|
|
76
|
-
f = 48 + 110 * np.exp(-t / 0.035)
|
|
77
|
-
return np.sin(2 * np.pi * np.cumsum(f) / SR) * env(n, 0.001, 0.16) * 0.95
|
|
78
|
-
|
|
79
|
-
def noise_hp(n, cut):
|
|
80
|
-
X = np.fft.rfft(rng.standard_normal(n)); fq = np.fft.rfftfreq(n, 1 / SR)
|
|
81
|
-
X[fq < cut] = 0
|
|
82
|
-
y = np.fft.irfft(X, n); return y / (np.abs(y).max() + 1e-9)
|
|
83
|
-
|
|
84
|
-
def hat(open_=False):
|
|
85
|
-
n = int((0.18 if open_ else 0.05) * SR)
|
|
86
|
-
return noise_hp(n, 7000) * env(n, 0.0005, 0.06 if open_ else 0.012) * 0.22
|
|
87
|
-
|
|
88
|
-
def clap():
|
|
89
|
-
n = int(0.22 * SR)
|
|
90
|
-
x = noise_hp(n, 1200) * env(n, 0.001, 0.06)
|
|
91
|
-
for d in (0.011, 0.022): # 拍手的多重起音
|
|
92
|
-
k = int(d * SR); x[k:] += noise_hp(n - k, 1200) * env(n - k, 0.001, 0.012) * 0.6
|
|
93
|
-
return x * 0.28
|
|
94
|
-
|
|
95
|
-
def pad_chord(midis, dur):
|
|
96
|
-
n = int(dur * SR)
|
|
97
|
-
x = sum(saw(hz(m), n, detune=0.006) for m in midis) / len(midis)
|
|
98
|
-
return lowpass(x, 900, q=0.7) * env(n, 0.35, 10, 0.9, 0.4) * 0.16
|
|
99
|
-
|
|
100
|
-
def parse_steps(s):
|
|
101
|
-
tok = s.split() if ' ' in s.strip() else list(s)
|
|
102
|
-
if len(tok) != 16: raise SystemExit('pattern must have 16 steps: %r' % s)
|
|
103
|
-
return tok
|
|
104
|
-
|
|
105
|
-
if __name__ == '__main__':
|
|
106
|
-
ap = argparse.ArgumentParser(description='synthesize a simple looping bassline track')
|
|
107
|
-
ap.add_argument('out')
|
|
108
|
-
ap.add_argument('--bpm', type=float, default=120)
|
|
109
|
-
ap.add_argument('--bars', type=int, default=16)
|
|
110
|
-
ap.add_argument('--key', default='A', help='tonic, e.g. A, C#, Eb')
|
|
111
|
-
ap.add_argument('--octave', type=int, default=2, help='octave of the bass tonic')
|
|
112
|
-
ap.add_argument('--scale', default='minor', choices=list(SCALE))
|
|
113
|
-
ap.add_argument('--prog', default='1,6,3,7', help='chord degrees, one per bar, cycled')
|
|
114
|
-
ap.add_argument('--bass', default='drive', help='preset (%s) or 16 custom steps' % ', '.join(BASS))
|
|
115
|
-
ap.add_argument('--drums', default='four', choices=list(DRUMS))
|
|
116
|
-
ap.add_argument('--pad', action='store_true', help='add sustained chord pad')
|
|
117
|
-
ap.add_argument('--intro', type=int, default=2, help='bars with bass and hat only')
|
|
118
|
-
ap.add_argument('--outro', type=int, default=2, help='bars fading out at the end')
|
|
119
|
-
ap.add_argument('--loop', action='store_true', help='seamless loop: no intro / outro, tails wrap around')
|
|
120
|
-
ap.add_argument('--swing', type=float, default=0, help='0 ~ 0.3, delays every second 16th')
|
|
121
|
-
ap.add_argument('--grid', default=None, help='write beat grid json here')
|
|
122
|
-
a = ap.parse_args()
|
|
123
|
-
|
|
124
|
-
scale = SCALE[a.scale]
|
|
125
|
-
tonic = 12 * (a.octave + 1) + NOTE[a.key]
|
|
126
|
-
prog = [int(v) for v in a.prog.split(',')]
|
|
127
|
-
bass = parse_steps(BASS.get(a.bass, a.bass))
|
|
128
|
-
drums = {k: parse_steps(v) for k, v in DRUMS[a.drums].items()}
|
|
129
|
-
intro, outro = (0, 0) if a.loop else (a.intro, a.outro)
|
|
130
|
-
|
|
131
|
-
step = 60 / a.bpm / 4
|
|
132
|
-
total = a.bars * 16 * step
|
|
133
|
-
n = int(round(total * SR))
|
|
134
|
-
out = np.zeros((n + int(SR * 2), 2), np.float32) # 多留 2 秒放音尾
|
|
135
|
-
|
|
136
|
-
def place(x, t, gain=1.0, pan=0.0):
|
|
137
|
-
s = int(round(t * SR)); e = s + len(x)
|
|
138
|
-
l, r = np.cos((pan + 1) * np.pi / 4), np.sin((pan + 1) * np.pi / 4)
|
|
139
|
-
out[s:e, 0] += x * gain * l * 1.414
|
|
140
|
-
out[s:e, 1] += x * gain * r * 1.414
|
|
141
|
-
|
|
142
|
-
def degree(d, shift=0):
|
|
143
|
-
i = d - 1 + shift
|
|
144
|
-
return scale[i % 7] + 12 * (i // 7)
|
|
145
|
-
|
|
146
|
-
K, C = kick(), clap()
|
|
147
|
-
for bar in range(a.bars):
|
|
148
|
-
root = tonic + degree(prog[bar % len(prog)])
|
|
149
|
-
t0 = bar * 16 * step
|
|
150
|
-
# 編排: intro 只有 bass + hat; outro 逐小節降低音量
|
|
151
|
-
level = 1.0 if bar < a.bars - outro else (a.bars - bar) / (outro + 1)
|
|
152
|
-
full = bar >= intro
|
|
153
|
-
if a.pad and full:
|
|
154
|
-
d = prog[bar % len(prog)]
|
|
155
|
-
chord = [tonic + 12 + degree(d), tonic + 12 + degree(d, 2), tonic + 12 + degree(d, 4)]
|
|
156
|
-
place(pad_chord(chord, 16 * step), t0, level)
|
|
157
|
-
for i in range(16):
|
|
158
|
-
t = t0 + i * step + (a.swing * step if i % 2 else 0)
|
|
159
|
-
s = bass[i]
|
|
160
|
-
if s not in ('.', '-'):
|
|
161
|
-
length = 1
|
|
162
|
-
while i + length < 16 and bass[i + length] == '-': length += 1
|
|
163
|
-
place(bass_note(root + int(s), length * step * 0.92, 1.0 if i % 4 == 0 else 0.6), t, level)
|
|
164
|
-
if full and drums['kick'][i] == 'x': place(K, t, level)
|
|
165
|
-
if full and drums['clap'][i] == 'x': place(C, t, level * 0.9, 0.15)
|
|
166
|
-
if drums['hat'][i] == 'x': place(hat(), t, level * (1 if full else 0.7), -0.25)
|
|
167
|
-
|
|
168
|
-
if a.loop:
|
|
169
|
-
tail = out[n:].copy()
|
|
170
|
-
out = out[:n]
|
|
171
|
-
k = min(len(tail), n)
|
|
172
|
-
out[:k] += tail[:k]
|
|
173
|
-
else:
|
|
174
|
-
tail = int(SR * 1.5)
|
|
175
|
-
out = out[:n + tail]
|
|
176
|
-
out[n:] *= np.linspace(1, 0, tail)[:, None]
|
|
177
|
-
write(a.out, normalize(out * 0.9, 0.95))
|
|
178
|
-
beats = [round(i * 4 * step, 4) for i in range(a.bars * 4)]
|
|
179
|
-
print('%s %.1f bpm %d bars %.2fs key %s %s prog %s' % (a.out, a.bpm, a.bars, total, a.key, a.scale, a.prog))
|
|
180
|
-
if a.grid:
|
|
181
|
-
json.dump({'bpm': a.bpm, 'offset': 0, 'beats': beats, 'downbeats': beats[::4]}, open(a.grid, 'w'), indent=1)
|
package/audio/common.py
DELETED
|
@@ -1,30 +0,0 @@
|
|
|
1
|
-
# 音訊工具共用: 讀檔 ( 經 ffmpeg ) 與寫 wav. 只依賴 numpy.
|
|
2
|
-
import numpy as np, subprocess, wave
|
|
3
|
-
|
|
4
|
-
SR = 48000
|
|
5
|
-
|
|
6
|
-
def load(path, sr=SR, start=0, dur=None, channels=1):
|
|
7
|
-
"""以 ffmpeg 解碼任意音訊檔, 回傳 float32 陣列 ( channels > 1 時為 (n, channels) )"""
|
|
8
|
-
cmd = ['ffmpeg', '-v', 'quiet', '-ss', str(start)] + (['-t', str(dur)] if dur else []) + \
|
|
9
|
-
['-i', path, '-ac', str(channels), '-ar', str(sr), '-f', 'f32le', '-']
|
|
10
|
-
x = np.frombuffer(subprocess.run(cmd, capture_output=True, check=True).stdout, dtype=np.float32)
|
|
11
|
-
return x.reshape(-1, channels).copy() if channels > 1 else x.copy()
|
|
12
|
-
|
|
13
|
-
def write(path, x, sr=SR):
|
|
14
|
-
"""寫入 16-bit wav. x 為 (n,) 或 (n, channels), 數值範圍 -1 ~ 1"""
|
|
15
|
-
x = np.clip(np.asarray(x, dtype=np.float32), -1, 1)
|
|
16
|
-
ch = 1 if x.ndim == 1 else x.shape[1]
|
|
17
|
-
with wave.open(path, 'wb') as w:
|
|
18
|
-
w.setnchannels(ch); w.setsampwidth(2); w.setframerate(sr)
|
|
19
|
-
w.writeframes((x * 32767).astype('<i2').tobytes())
|
|
20
|
-
|
|
21
|
-
def read_wav(path):
|
|
22
|
-
"""讀取 16-bit wav, 回傳 (資料, sr). 多聲道時為 (n, channels)"""
|
|
23
|
-
with wave.open(path) as w:
|
|
24
|
-
ch, sr = w.getnchannels(), w.getframerate()
|
|
25
|
-
x = np.frombuffer(w.readframes(w.getnframes()), '<i2').astype(np.float32) / 32767
|
|
26
|
-
return (x.reshape(-1, ch) if ch > 1 else x), sr
|
|
27
|
-
|
|
28
|
-
def normalize(x, peak=0.98):
|
|
29
|
-
m = np.abs(x).max()
|
|
30
|
-
return x * (peak / m) if m > peak else x
|
package/audio/mix.py
DELETED
|
@@ -1,51 +0,0 @@
|
|
|
1
|
-
# 混音: 背景音樂 + 依 cue 放置的音效 ( 整理自 0926-motion ).
|
|
2
|
-
# python audio/mix.py <cues.json> <out.wav> --dur s [--music f] [--start s] [--sfx dir] [--fade s] [--loop]
|
|
3
|
-
# - cues.json: [{t, sfx, gain?}, ...], 通常由 `lotion cues` 從頁面匯出
|
|
4
|
-
# - 音效以 sfx 目錄裡 peaks.json 量到的 transient peak 對齊 cue 時間, 而非檔案開頭 ( 見 sfx.py )
|
|
5
|
-
# - --music: 任意音訊檔 ( 例如 bgm.py 的輸出 ); --start 為音樂中對齊 t = 0 的位置
|
|
6
|
-
# - --loop: 超出尾端的音效回繞到開頭, 音訊可無縫循環; 否則尾端以 --fade 秒淡出
|
|
7
|
-
import numpy as np, json, os, argparse
|
|
8
|
-
from common import SR, load, read_wav, write, normalize
|
|
9
|
-
|
|
10
|
-
ap = argparse.ArgumentParser(description='mix background music and cue-aligned sound effects')
|
|
11
|
-
ap.add_argument('cues')
|
|
12
|
-
ap.add_argument('out')
|
|
13
|
-
ap.add_argument('--dur', type=float, required=True, help='length of the output in seconds')
|
|
14
|
-
ap.add_argument('--music', default=None)
|
|
15
|
-
ap.add_argument('--start', type=float, default=0, help='position in music aligned to t = 0')
|
|
16
|
-
ap.add_argument('--sfx', default='sfx', help='directory of sfx wav files and peaks.json')
|
|
17
|
-
ap.add_argument('--fade', type=float, default=0)
|
|
18
|
-
ap.add_argument('--loop', action='store_true')
|
|
19
|
-
ap.add_argument('--sfx-gain', type=float, default=0.55)
|
|
20
|
-
ap.add_argument('--music-gain', type=float, default=0.8)
|
|
21
|
-
a = ap.parse_args()
|
|
22
|
-
|
|
23
|
-
n = int(SR * a.dur)
|
|
24
|
-
mix = np.zeros((n, 2), np.float32)
|
|
25
|
-
if a.music:
|
|
26
|
-
music = load(a.music, start=a.start, dur=a.dur, channels=2)[:n]
|
|
27
|
-
mix[:len(music)] = music * a.music_gain
|
|
28
|
-
|
|
29
|
-
cues = json.load(open(a.cues))
|
|
30
|
-
peaks = json.load(open(os.path.join(a.sfx, 'peaks.json'))) if cues else {}
|
|
31
|
-
cache = {}
|
|
32
|
-
for c in cues:
|
|
33
|
-
name = c['sfx']
|
|
34
|
-
if name not in cache:
|
|
35
|
-
x, _ = read_wav(os.path.join(a.sfx, name + '.wav'))
|
|
36
|
-
cache[name] = x if x.ndim == 1 else x.mean(1)
|
|
37
|
-
x = cache[name] * c.get('gain', 1) * a.sfx_gain
|
|
38
|
-
s = int(round((c['t'] - peaks.get(name, 0)) * SR))
|
|
39
|
-
idx = s + np.arange(len(x))
|
|
40
|
-
if a.loop: idx %= n
|
|
41
|
-
else:
|
|
42
|
-
m = (idx >= 0) & (idx < n); idx, x = idx[m], x[m]
|
|
43
|
-
np.add.at(mix[:, 0], idx, x)
|
|
44
|
-
np.add.at(mix[:, 1], idx, x)
|
|
45
|
-
|
|
46
|
-
if a.fade > 0 and not a.loop:
|
|
47
|
-
f = int(SR * a.fade)
|
|
48
|
-
mix[n - f:] *= np.linspace(1, 0, f)[:, None]
|
|
49
|
-
pk = float(np.abs(mix).max())
|
|
50
|
-
write(a.out, normalize(mix))
|
|
51
|
-
print('%s cues %d peak %.3f' % (a.out, len(cues), pk))
|
package/audio/sfx.py
DELETED
|
@@ -1,93 +0,0 @@
|
|
|
1
|
-
# UI 音效合成 ( 整理自 0926-motion ). 輸出 <outdir>/*.wav, 並量測每個檔的 transient peak 寫入 <outdir>/peaks.json
|
|
2
|
-
# python audio/sfx.py <outdir> [--list]
|
|
3
|
-
# peak 定義: 3ms RMS 包絡第一次達到全域最大 80% 的位置
|
|
4
|
-
# - 多音符音效 ( chime / toggle ) 取第一個音的擊點, 而非較晚但較響的音
|
|
5
|
-
# - 混音時以 peak 而非檔案開頭對齊 cue 時間 ( 見 mix.py )
|
|
6
|
-
import numpy as np, json, os, argparse
|
|
7
|
-
from common import SR, write
|
|
8
|
-
|
|
9
|
-
rng = np.random.default_rng(7)
|
|
10
|
-
|
|
11
|
-
def t_(dur): return np.arange(int(SR * dur)) / SR
|
|
12
|
-
def env(t, a, d): return (1 - np.exp(-t / max(a, 1e-4))) * np.exp(-t / d)
|
|
13
|
-
def bandnoise(n, lo, hi):
|
|
14
|
-
x = rng.standard_normal(n)
|
|
15
|
-
X = np.fft.rfft(x); f = np.fft.rfftfreq(n, 1 / SR)
|
|
16
|
-
X[(f < lo) | (f > hi)] = 0
|
|
17
|
-
y = np.fft.irfft(X, n); return y / (np.abs(y).max() + 1e-9)
|
|
18
|
-
def sine(t, f, ph=0): return np.sin(2 * np.pi * f * t + ph)
|
|
19
|
-
def pad(x, pre=0.0, post=0.0): return np.concatenate([np.zeros(int(SR * pre)), x, np.zeros(int(SR * post))])
|
|
20
|
-
|
|
21
|
-
def click():
|
|
22
|
-
t = t_(0.06)
|
|
23
|
-
body = sine(t, 1850) * env(t, 0.0004, 0.006) * 0.7 + sine(t, 520) * env(t, 0.0008, 0.012) * 0.5
|
|
24
|
-
return body + bandnoise(len(t), 2500, 9000) * env(t, 0.0002, 0.003) * 0.5
|
|
25
|
-
|
|
26
|
-
def tick():
|
|
27
|
-
t = t_(0.03)
|
|
28
|
-
return sine(t, 3400) * env(t, 0.0003, 0.004) * 0.6 + bandnoise(len(t), 4000, 12000) * env(t, 0.0002, 0.0015) * 0.3
|
|
29
|
-
|
|
30
|
-
def scrub():
|
|
31
|
-
t = t_(0.015)
|
|
32
|
-
return bandnoise(len(t), 5000, 12000) * env(t, 0.0002, 0.0012) * 0.5
|
|
33
|
-
|
|
34
|
-
def pop():
|
|
35
|
-
t = t_(0.14)
|
|
36
|
-
f = 380 + 520 * np.exp(-t / 0.018)
|
|
37
|
-
return np.sin(2 * np.pi * np.cumsum(f) / SR) * env(t, 0.001, 0.035) * 0.9
|
|
38
|
-
|
|
39
|
-
def toggle():
|
|
40
|
-
a = click() * 0.6
|
|
41
|
-
t = t_(0.05)
|
|
42
|
-
b = sine(t, 2600) * env(t, 0.0004, 0.005) * 0.6 + sine(t, 780) * env(t, 0.0006, 0.01) * 0.4
|
|
43
|
-
out = np.zeros(int(SR * 0.12)); out[:len(a)] += a
|
|
44
|
-
o = int(SR * 0.028); out[o:o + len(b)] += b
|
|
45
|
-
return out
|
|
46
|
-
|
|
47
|
-
def chime(notes, gap, dur=0.5):
|
|
48
|
-
out = np.zeros(int(SR * (gap * len(notes) + dur)))
|
|
49
|
-
for i, f in enumerate(notes):
|
|
50
|
-
t = t_(dur)
|
|
51
|
-
x = (sine(t, f) + 0.25 * sine(t, f * 2.01) + 0.08 * sine(t, f * 3.02)) * env(t, 0.002, 0.16)
|
|
52
|
-
o = int(SR * gap * i); out[o:o + len(x)] += x * (0.8 if i == 0 else 0.65)
|
|
53
|
-
return out * 0.45
|
|
54
|
-
|
|
55
|
-
def success(): return chime([1318.5, 1975.5], 0.07)
|
|
56
|
-
def notify(): return chime([1567.98, 2093.0, 2637.0], 0.06, 0.45)
|
|
57
|
-
|
|
58
|
-
def typing():
|
|
59
|
-
t = t_(0.05)
|
|
60
|
-
return bandnoise(len(t), 1200, 6000) * env(t, 0.0003, 0.005) * 0.55 + sine(t, 210) * env(t, 0.001, 0.01) * 0.35
|
|
61
|
-
|
|
62
|
-
def whoosh():
|
|
63
|
-
t = t_(0.32)
|
|
64
|
-
n = bandnoise(len(t), 300, 4000)
|
|
65
|
-
e = np.sin(np.pi * np.clip(t / 0.32, 0, 1)) ** 2.5
|
|
66
|
-
return n * e * 0.18
|
|
67
|
-
|
|
68
|
-
SFX = dict(click=click, tick=tick, scrub=scrub, pop=pop, toggle=toggle, success=success, notify=notify,
|
|
69
|
-
type=typing, whoosh=whoosh)
|
|
70
|
-
|
|
71
|
-
def peak(x):
|
|
72
|
-
win = int(SR * 0.003)
|
|
73
|
-
e = np.sqrt(np.convolve(x * x, np.ones(win) / win, 'same'))
|
|
74
|
-
i = int(np.argmax(e >= 0.8 * e.max()))
|
|
75
|
-
# 往後找到該段的局部最大
|
|
76
|
-
while i + 1 < len(e) and e[i + 1] >= e[i]: i += 1
|
|
77
|
-
return i
|
|
78
|
-
|
|
79
|
-
if __name__ == '__main__':
|
|
80
|
-
ap = argparse.ArgumentParser(description='synthesize ui sound effects')
|
|
81
|
-
ap.add_argument('outdir')
|
|
82
|
-
ap.add_argument('--list', action='store_true', help='only list available effects')
|
|
83
|
-
a = ap.parse_args()
|
|
84
|
-
if a.list:
|
|
85
|
-
print(' '.join(SFX)); raise SystemExit
|
|
86
|
-
os.makedirs(a.outdir, exist_ok=True)
|
|
87
|
-
peaks = {}
|
|
88
|
-
for name, fn in SFX.items():
|
|
89
|
-
x = pad(fn(), 0.005, 0.02)
|
|
90
|
-
write(os.path.join(a.outdir, name + '.wav'), x)
|
|
91
|
-
peaks[name] = round(peak(x) / SR, 5)
|
|
92
|
-
json.dump(peaks, open(os.path.join(a.outdir, 'peaks.json'), 'w'), indent=1)
|
|
93
|
-
print(json.dumps(peaks, indent=1))
|
package/prompt/ui-loop.md
DELETED
|
@@ -1,497 +0,0 @@
|
|
|
1
|
-
You are a senior motion designer and creative frontend engineer.
|
|
2
|
-
|
|
3
|
-
Your task is to create a premium, Dribbble-level UI motion design entirely with code.
|
|
4
|
-
|
|
5
|
-
Do NOT start coding immediately.
|
|
6
|
-
|
|
7
|
-
PHASE 1 — ASK FOR INPUTS
|
|
8
|
-
|
|
9
|
-
First ask me for:
|
|
10
|
-
|
|
11
|
-
1. 8–12 UI states that the main shape should transform into.
|
|
12
|
-
Examples:
|
|
13
|
-
|
|
14
|
-
- Button
|
|
15
|
-
- Loader
|
|
16
|
-
- Success check
|
|
17
|
-
- Dynamic Island
|
|
18
|
-
- Music player
|
|
19
|
-
- Progress scrubber
|
|
20
|
-
- Volume slider
|
|
21
|
-
- Toggle
|
|
22
|
-
- Tabs
|
|
23
|
-
- Chart
|
|
24
|
-
- Command palette
|
|
25
|
-
- Toast
|
|
26
|
-
|
|
27
|
-
2. Visual palette:
|
|
28
|
-
|
|
29
|
-
- Pure black + white
|
|
30
|
-
- OR black + white + one accent color
|
|
31
|
-
|
|
32
|
-
3. A royalty-free music track around 120 BPM.
|
|
33
|
-
|
|
34
|
-
Prefer music that can legally be used commercially, such as tracks from Mixkit or similar royalty-free libraries.
|
|
35
|
-
|
|
36
|
-
After I answer, DO NOT code yet.
|
|
37
|
-
|
|
38
|
-
---
|
|
39
|
-
|
|
40
|
-
PHASE 2 — BUILD THE MOTION SCORE
|
|
41
|
-
|
|
42
|
-
Create a beat-by-beat animation plan.
|
|
43
|
-
|
|
44
|
-
Target:
|
|
45
|
-
|
|
46
|
-
120 BPM
|
|
47
|
-
7 bars
|
|
48
|
-
4 beats per bar
|
|
49
|
-
28 primary beats
|
|
50
|
-
|
|
51
|
-
Something meaningful should happen on nearly every beat.
|
|
52
|
-
|
|
53
|
-
Show the timeline before writing code.
|
|
54
|
-
|
|
55
|
-
For every beat, define:
|
|
56
|
-
|
|
57
|
-
- beat number
|
|
58
|
-
- timestamp
|
|
59
|
-
- current UI state
|
|
60
|
-
- transformation
|
|
61
|
-
- cursor action
|
|
62
|
-
- camera action
|
|
63
|
-
- sound effect
|
|
64
|
-
- spring / easing behavior
|
|
65
|
-
|
|
66
|
-
Example flow:
|
|
67
|
-
|
|
68
|
-
Button
|
|
69
|
-
→ Loader
|
|
70
|
-
→ Check
|
|
71
|
-
→ Dynamic Island
|
|
72
|
-
→ Music Player
|
|
73
|
-
→ Play/Pause morph
|
|
74
|
-
→ Progress scrub
|
|
75
|
-
→ Volume slider
|
|
76
|
-
→ Overscroll/stretch
|
|
77
|
-
→ Toggle
|
|
78
|
-
→ Liquid tabs
|
|
79
|
-
→ Chart
|
|
80
|
-
→ Tooltip
|
|
81
|
-
→ Command palette
|
|
82
|
-
→ Search typing
|
|
83
|
-
→ Enter
|
|
84
|
-
→ Toast
|
|
85
|
-
→ Original button
|
|
86
|
-
|
|
87
|
-
The final state must transition perfectly into the first frame.
|
|
88
|
-
|
|
89
|
-
---
|
|
90
|
-
|
|
91
|
-
VISUAL DIRECTION
|
|
92
|
-
|
|
93
|
-
Aim for premium contemporary product motion design.
|
|
94
|
-
|
|
95
|
-
Reference quality:
|
|
96
|
-
|
|
97
|
-
- high-end Dribbble motion
|
|
98
|
-
- Linear
|
|
99
|
-
- Arc
|
|
100
|
-
- Raycast
|
|
101
|
-
- Apple system UI
|
|
102
|
-
- Stripe
|
|
103
|
-
- modern Vercel-style interfaces
|
|
104
|
-
|
|
105
|
-
Canvas:
|
|
106
|
-
|
|
107
|
-
1440 × 1440 square.
|
|
108
|
-
|
|
109
|
-
Background:
|
|
110
|
-
|
|
111
|
-
Light warm gray.
|
|
112
|
-
|
|
113
|
-
UI:
|
|
114
|
-
|
|
115
|
-
Mostly black and white.
|
|
116
|
-
|
|
117
|
-
Optionally one accent color.
|
|
118
|
-
|
|
119
|
-
Typography:
|
|
120
|
-
|
|
121
|
-
Geist or another clean modern UI sans-serif.
|
|
122
|
-
|
|
123
|
-
Icons:
|
|
124
|
-
|
|
125
|
-
Consistent stroke weight.
|
|
126
|
-
|
|
127
|
-
No mismatched icon families.
|
|
128
|
-
|
|
129
|
-
---
|
|
130
|
-
|
|
131
|
-
CORE MOTION RULE
|
|
132
|
-
|
|
133
|
-
There is ONE primary shape throughout the entire animation.
|
|
134
|
-
|
|
135
|
-
Never cut between unrelated objects.
|
|
136
|
-
|
|
137
|
-
Every interface state must feel like the SAME physical object transforming.
|
|
138
|
-
|
|
139
|
-
Transformation properties may include:
|
|
140
|
-
|
|
141
|
-
- width
|
|
142
|
-
- height
|
|
143
|
-
- border radius
|
|
144
|
-
- position
|
|
145
|
-
- fill
|
|
146
|
-
- stroke
|
|
147
|
-
- internal layout
|
|
148
|
-
- content
|
|
149
|
-
- clipping mask
|
|
150
|
-
|
|
151
|
-
Content may change during a morph using a very short blur/fade transition.
|
|
152
|
-
|
|
153
|
-
The object itself must remain visually continuous.
|
|
154
|
-
|
|
155
|
-
---
|
|
156
|
-
|
|
157
|
-
INTERACTION
|
|
158
|
-
|
|
159
|
-
A visible cursor drives the animation.
|
|
160
|
-
|
|
161
|
-
The cursor should:
|
|
162
|
-
|
|
163
|
-
- move intentionally
|
|
164
|
-
- click real controls
|
|
165
|
-
- drag sliders
|
|
166
|
-
- hover charts
|
|
167
|
-
- type into inputs
|
|
168
|
-
- trigger state changes
|
|
169
|
-
|
|
170
|
-
Avoid arbitrary cursor movement.
|
|
171
|
-
|
|
172
|
-
Every cursor action must have a visible cause and effect.
|
|
173
|
-
|
|
174
|
-
Clicks should feel tactile.
|
|
175
|
-
|
|
176
|
-
Drags must use direct manipulation.
|
|
177
|
-
|
|
178
|
-
While dragging:
|
|
179
|
-
|
|
180
|
-
value = function(cursor position)
|
|
181
|
-
|
|
182
|
-
Do not fake the slider movement independently of the pointer.
|
|
183
|
-
|
|
184
|
-
When released, the element may continue using a spring from its release position.
|
|
185
|
-
|
|
186
|
-
---
|
|
187
|
-
|
|
188
|
-
SPRINGS
|
|
189
|
-
|
|
190
|
-
Use springs extensively.
|
|
191
|
-
|
|
192
|
-
Do NOT use generic CSS easing curves for the primary motion.
|
|
193
|
-
|
|
194
|
-
Springs must be implemented as closed-form mathematical step responses.
|
|
195
|
-
|
|
196
|
-
Avoid exaggerated bouncing.
|
|
197
|
-
|
|
198
|
-
Desired character:
|
|
199
|
-
|
|
200
|
-
fast
|
|
201
|
-
precise
|
|
202
|
-
slightly physical
|
|
203
|
-
tiny overshoot at most
|
|
204
|
-
|
|
205
|
-
If a property changes target multiple times, compute it as the sum of independent spring responses for each target change.
|
|
206
|
-
|
|
207
|
-
The animation must remain a pure function of time.
|
|
208
|
-
|
|
209
|
-
---
|
|
210
|
-
|
|
211
|
-
LIQUID MOTION
|
|
212
|
-
|
|
213
|
-
For elements such as:
|
|
214
|
-
|
|
215
|
-
- tabs
|
|
216
|
-
- toggles
|
|
217
|
-
- sliders
|
|
218
|
-
- pills
|
|
219
|
-
|
|
220
|
-
animate the leading and trailing edges independently.
|
|
221
|
-
|
|
222
|
-
Example:
|
|
223
|
-
|
|
224
|
-
When a tab indicator moves right:
|
|
225
|
-
|
|
226
|
-
leading edge spring → faster
|
|
227
|
-
|
|
228
|
-
trailing edge spring → slightly slower
|
|
229
|
-
|
|
230
|
-
This creates temporary stretching.
|
|
231
|
-
|
|
232
|
-
Then both edges converge.
|
|
233
|
-
|
|
234
|
-
Apply the same principle to the toggle knob.
|
|
235
|
-
|
|
236
|
-
The effect should feel elastic but restrained.
|
|
237
|
-
|
|
238
|
-
Never cartoonishly bouncy.
|
|
239
|
-
|
|
240
|
-
---
|
|
241
|
-
|
|
242
|
-
CAMERA
|
|
243
|
-
|
|
244
|
-
The virtual camera should subtly reframe each state.
|
|
245
|
-
|
|
246
|
-
Every important UI state should comfortably fill the composition.
|
|
247
|
-
|
|
248
|
-
Use:
|
|
249
|
-
|
|
250
|
-
- scale
|
|
251
|
-
- translation
|
|
252
|
-
|
|
253
|
-
Avoid unnecessary rotation.
|
|
254
|
-
|
|
255
|
-
Camera movement must also be driven entirely by time.
|
|
256
|
-
|
|
257
|
-
Avoid blurry text.
|
|
258
|
-
|
|
259
|
-
CRITICAL:
|
|
260
|
-
|
|
261
|
-
Do NOT apply "will-change" to elements that the camera scales.
|
|
262
|
-
|
|
263
|
-
It can cause browser text rasterization and make typography blurry.
|
|
264
|
-
|
|
265
|
-
---
|
|
266
|
-
|
|
267
|
-
TEXT TRANSITIONS
|
|
268
|
-
|
|
269
|
-
When content changes inside a morphing container:
|
|
270
|
-
|
|
271
|
-
old content must have a defined EXIT window.
|
|
272
|
-
|
|
273
|
-
new content must have a defined ENTER window.
|
|
274
|
-
|
|
275
|
-
Never allow both text states to occupy the same visual space unintentionally.
|
|
276
|
-
|
|
277
|
-
Suggested transition:
|
|
278
|
-
|
|
279
|
-
old text
|
|
280
|
-
→ fade + blur + slight movement
|
|
281
|
-
→ container morph
|
|
282
|
-
→ new text unblur + fade in
|
|
283
|
-
|
|
284
|
-
Keep transitions short.
|
|
285
|
-
|
|
286
|
-
---
|
|
287
|
-
|
|
288
|
-
AUDIO
|
|
289
|
-
|
|
290
|
-
Analyze the selected music using Python + NumPy.
|
|
291
|
-
|
|
292
|
-
Determine:
|
|
293
|
-
|
|
294
|
-
- BPM
|
|
295
|
-
- beat timestamps
|
|
296
|
-
- downbeats
|
|
297
|
-
- useful transients
|
|
298
|
-
|
|
299
|
-
Start the motion sequence on a strong downbeat.
|
|
300
|
-
|
|
301
|
-
Align important transitions to musical beats.
|
|
302
|
-
|
|
303
|
-
UI sound effects may include:
|
|
304
|
-
|
|
305
|
-
- click
|
|
306
|
-
- toggle
|
|
307
|
-
- pop
|
|
308
|
-
- scrub
|
|
309
|
-
- typing
|
|
310
|
-
- success
|
|
311
|
-
- notification
|
|
312
|
-
|
|
313
|
-
Measure each sound effect's actual transient peak.
|
|
314
|
-
|
|
315
|
-
Align the measured peak—not merely the start of the audio file—to the visual action.
|
|
316
|
-
|
|
317
|
-
---
|
|
318
|
-
|
|
319
|
-
IMPLEMENTATION ARCHITECTURE
|
|
320
|
-
|
|
321
|
-
Create ONE HTML file.
|
|
322
|
-
|
|
323
|
-
Resolution:
|
|
324
|
-
|
|
325
|
-
1440 × 1440
|
|
326
|
-
|
|
327
|
-
The entire animation must be controlled through:
|
|
328
|
-
|
|
329
|
-
seek(t)
|
|
330
|
-
|
|
331
|
-
where:
|
|
332
|
-
|
|
333
|
-
t = animation time in seconds
|
|
334
|
-
|
|
335
|
-
Every visual property must be derived from "t".
|
|
336
|
-
|
|
337
|
-
NO:
|
|
338
|
-
|
|
339
|
-
- CSS transitions
|
|
340
|
-
- setTimeout
|
|
341
|
-
- request-driven state machines
|
|
342
|
-
- persistent animation state
|
|
343
|
-
- frame-to-frame dependency
|
|
344
|
-
|
|
345
|
-
Given the same time value, "seek(t)" must always produce exactly the same frame.
|
|
346
|
-
|
|
347
|
-
This is essential for deterministic rendering.
|
|
348
|
-
|
|
349
|
-
---
|
|
350
|
-
|
|
351
|
-
RENDERING
|
|
352
|
-
|
|
353
|
-
Use Playwright to render frames.
|
|
354
|
-
|
|
355
|
-
Target final output:
|
|
356
|
-
|
|
357
|
-
60 FPS
|
|
358
|
-
|
|
359
|
-
For motion blur:
|
|
360
|
-
|
|
361
|
-
render 4 temporal subframes for every final frame.
|
|
362
|
-
|
|
363
|
-
Example:
|
|
364
|
-
|
|
365
|
-
frame t
|
|
366
|
-
|
|
367
|
-
samples:
|
|
368
|
-
|
|
369
|
-
t - 1.5Δ
|
|
370
|
-
t - 0.5Δ
|
|
371
|
-
t + 0.5Δ
|
|
372
|
-
t + 1.5Δ
|
|
373
|
-
|
|
374
|
-
Blend these using FFmpeg "tmix" or equivalent temporal averaging.
|
|
375
|
-
|
|
376
|
-
The motion blur must remain subtle.
|
|
377
|
-
|
|
378
|
-
UI must stay sharp enough to read.
|
|
379
|
-
|
|
380
|
-
---
|
|
381
|
-
|
|
382
|
-
PREVIEW VALIDATION
|
|
383
|
-
|
|
384
|
-
Before doing the full render:
|
|
385
|
-
|
|
386
|
-
render exactly ONE representative frame per beat.
|
|
387
|
-
|
|
388
|
-
Create a contact sheet or preview sequence.
|
|
389
|
-
|
|
390
|
-
Inspect:
|
|
391
|
-
|
|
392
|
-
- beat synchronization
|
|
393
|
-
- spacing
|
|
394
|
-
- typography
|
|
395
|
-
- clipping
|
|
396
|
-
- visual hierarchy
|
|
397
|
-
- cursor placement
|
|
398
|
-
- morph continuity
|
|
399
|
-
- camera framing
|
|
400
|
-
|
|
401
|
-
If anything is:
|
|
402
|
-
|
|
403
|
-
- off-grid
|
|
404
|
-
- cramped
|
|
405
|
-
- unreadable
|
|
406
|
-
- awkward
|
|
407
|
-
- visually discontinuous
|
|
408
|
-
|
|
409
|
-
fix it before the full render.
|
|
410
|
-
|
|
411
|
-
---
|
|
412
|
-
|
|
413
|
-
LOOP REQUIREMENT
|
|
414
|
-
|
|
415
|
-
The animation must loop perfectly.
|
|
416
|
-
|
|
417
|
-
The final frame must mathematically match the first frame.
|
|
418
|
-
|
|
419
|
-
Match:
|
|
420
|
-
|
|
421
|
-
- shape
|
|
422
|
-
- dimensions
|
|
423
|
-
- radius
|
|
424
|
-
- color
|
|
425
|
-
- internal content
|
|
426
|
-
- camera position
|
|
427
|
-
- camera scale
|
|
428
|
-
- cursor position
|
|
429
|
-
- cursor velocity
|
|
430
|
-
- animation velocity
|
|
431
|
-
|
|
432
|
-
Do not merely make the positions equal.
|
|
433
|
-
|
|
434
|
-
The velocity around the loop boundary must also feel continuous.
|
|
435
|
-
|
|
436
|
-
There should be no perceptible stutter when playback restarts.
|
|
437
|
-
|
|
438
|
-
---
|
|
439
|
-
|
|
440
|
-
BANNED
|
|
441
|
-
|
|
442
|
-
Do not use:
|
|
443
|
-
|
|
444
|
-
- exaggerated bouncy easing
|
|
445
|
-
- particle explosions
|
|
446
|
-
- random decorative particles
|
|
447
|
-
- glow-heavy interfaces
|
|
448
|
-
- gradients on UI chrome
|
|
449
|
-
- inconsistent icon strokes
|
|
450
|
-
- generic template animations
|
|
451
|
-
- excessive glassmorphism
|
|
452
|
-
- meaningless floating objects
|
|
453
|
-
- dead time
|
|
454
|
-
- unnecessary 3D transforms
|
|
455
|
-
- random camera movement
|
|
456
|
-
|
|
457
|
-
Every movement must communicate state, interaction, or rhythm.
|
|
458
|
-
|
|
459
|
-
---
|
|
460
|
-
|
|
461
|
-
DESIGN PRINCIPLE
|
|
462
|
-
|
|
463
|
-
The viewer should feel like they are watching ONE intelligent interface object continuously transform.
|
|
464
|
-
|
|
465
|
-
Not:
|
|
466
|
-
|
|
467
|
-
"a collection of UI animations."
|
|
468
|
-
|
|
469
|
-
But:
|
|
470
|
-
|
|
471
|
-
"one object performing a choreographed sequence."
|
|
472
|
-
|
|
473
|
-
Every transformation must answer:
|
|
474
|
-
|
|
475
|
-
Why did this object become the next thing?
|
|
476
|
-
|
|
477
|
-
The cursor, music, interaction, and physical motion should provide that answer.
|
|
478
|
-
|
|
479
|
-
---
|
|
480
|
-
|
|
481
|
-
EXECUTION ORDER
|
|
482
|
-
|
|
483
|
-
Follow this exact workflow:
|
|
484
|
-
|
|
485
|
-
1. Ask for my UI states, palette, and song.
|
|
486
|
-
2. Create the complete beat grid.
|
|
487
|
-
3. Show me the proposed state sequence.
|
|
488
|
-
4. Identify difficult morph transitions.
|
|
489
|
-
5. Design the spring / interaction strategy.
|
|
490
|
-
6. Only then write the HTML animation.
|
|
491
|
-
7. Render one frame per beat.
|
|
492
|
-
8. Visually inspect the preview.
|
|
493
|
-
9. Correct layout and timing problems.
|
|
494
|
-
10. Render the final 60 FPS video with motion blur.
|
|
495
|
-
11. Verify that the final frame loops seamlessly into the first.
|
|
496
|
-
|
|
497
|
-
Do not skip directly to implementation.
|