vizcore 0.1.0 → 1.0.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (126) hide show
  1. checksums.yaml +4 -4
  2. data/README.md +544 -9
  3. data/docs/.nojekyll +0 -0
  4. data/docs/assets/site.css +744 -0
  5. data/docs/assets/vizcore-demo.gif +0 -0
  6. data/docs/assets/vizcore-poster.png +0 -0
  7. data/docs/assets/vj-tunnel.js +159 -0
  8. data/docs/index.html +224 -0
  9. data/examples/README.md +59 -0
  10. data/examples/assets/README.md +19 -0
  11. data/examples/audio_inspector.rb +34 -0
  12. data/examples/club_intro_drop.rb +78 -0
  13. data/examples/kansai_rubykaigi_visual.rb +70 -0
  14. data/examples/live_coding_minimal.rb +22 -0
  15. data/examples/midi_controller_show.rb +78 -0
  16. data/examples/midi_scene_switch.rb +3 -1
  17. data/examples/parser_visualizer.rb +48 -0
  18. data/examples/readme_demo.rb +17 -0
  19. data/examples/rhythm_geometry.rb +34 -0
  20. data/examples/ruby_crystal_show.rb +35 -0
  21. data/examples/shader_playground.rb +18 -0
  22. data/examples/unyo_liquid.rb +59 -0
  23. data/examples/vj_ambient_chill_room.rb +124 -0
  24. data/examples/vj_dnb_jungle.rb +170 -0
  25. data/examples/vj_festival_mainstage.rb +245 -0
  26. data/examples/vj_festival_mainstage.yml +17 -0
  27. data/examples/vj_glitch_industrial.rb +164 -0
  28. data/examples/vj_hiphop_cipher.rb +167 -0
  29. data/examples/vj_jpop_idol_live.rb +210 -0
  30. data/examples/vj_synthwave_retro.rb +173 -0
  31. data/examples/vj_techno_warehouse.rb +195 -0
  32. data/frontend/index.html +468 -2
  33. data/frontend/src/audio-inspector.js +40 -0
  34. data/frontend/src/live-controls.js +131 -0
  35. data/frontend/src/main.js +792 -16
  36. data/frontend/src/midi-learn.js +194 -0
  37. data/frontend/src/performance-monitor.js +183 -0
  38. data/frontend/src/plugin-runtime.js +130 -0
  39. data/frontend/src/projector-mode.js +56 -0
  40. data/frontend/src/renderer/engine.js +148 -3
  41. data/frontend/src/renderer/layer-manager.js +428 -30
  42. data/frontend/src/renderer/shader-manager.js +26 -0
  43. data/frontend/src/runtime-control-preset.js +11 -0
  44. data/frontend/src/shader-error-overlay.js +29 -0
  45. data/frontend/src/shader-param-controls.js +93 -0
  46. data/frontend/src/shaders/builtins.js +380 -2
  47. data/frontend/src/shaders/post-effects.js +52 -0
  48. data/frontend/src/visual-regression.js +67 -0
  49. data/frontend/src/visual-settings-preset.js +103 -0
  50. data/frontend/src/visuals/geometry.js +268 -0
  51. data/frontend/src/visuals/image-renderer.js +291 -0
  52. data/frontend/src/visuals/particle-system.js +56 -10
  53. data/frontend/src/visuals/spectrogram-renderer.js +226 -0
  54. data/frontend/src/visuals/text-renderer.js +112 -11
  55. data/frontend/src/websocket-client.js +12 -1
  56. data/lib/vizcore/analysis/adaptive_normalizer.rb +70 -0
  57. data/lib/vizcore/analysis/beat_detector.rb +4 -2
  58. data/lib/vizcore/analysis/bpm_estimator.rb +8 -0
  59. data/lib/vizcore/analysis/feature_recorder.rb +159 -0
  60. data/lib/vizcore/analysis/feature_replay.rb +84 -0
  61. data/lib/vizcore/analysis/pipeline.rb +235 -11
  62. data/lib/vizcore/analysis/tap_tempo.rb +74 -0
  63. data/lib/vizcore/analysis.rb +4 -0
  64. data/lib/vizcore/audio/dummy_sine_input.rb +1 -1
  65. data/lib/vizcore/audio/fixture_input.rb +65 -0
  66. data/lib/vizcore/audio/input_manager.rb +4 -2
  67. data/lib/vizcore/audio/mic_input.rb +24 -8
  68. data/lib/vizcore/audio/portaudio_ffi.rb +106 -1
  69. data/lib/vizcore/audio.rb +1 -0
  70. data/lib/vizcore/cli/doctor.rb +159 -0
  71. data/lib/vizcore/cli/dsl_reference.rb +99 -0
  72. data/lib/vizcore/cli/layer_docs.rb +46 -0
  73. data/lib/vizcore/cli/scene_diagnostics.rb +23 -0
  74. data/lib/vizcore/cli/scene_inspector.rb +136 -0
  75. data/lib/vizcore/cli/scene_validator.rb +245 -0
  76. data/lib/vizcore/cli/shader_template.rb +68 -0
  77. data/lib/vizcore/cli/shader_uniform_docs.rb +54 -0
  78. data/lib/vizcore/cli.rb +689 -18
  79. data/lib/vizcore/config.rb +103 -2
  80. data/lib/vizcore/control_preset.rb +68 -0
  81. data/lib/vizcore/dsl/engine.rb +277 -5
  82. data/lib/vizcore/dsl/layer_builder.rb +491 -22
  83. data/lib/vizcore/dsl/layer_group_builder.rb +112 -0
  84. data/lib/vizcore/dsl/mapping_resolver.rb +132 -3
  85. data/lib/vizcore/dsl/mapping_transform_builder.rb +71 -0
  86. data/lib/vizcore/dsl/reaction_builder.rb +44 -0
  87. data/lib/vizcore/dsl/scene_builder.rb +61 -5
  88. data/lib/vizcore/dsl/shader_source_resolver.rb +67 -6
  89. data/lib/vizcore/dsl/style_builder.rb +68 -0
  90. data/lib/vizcore/dsl/timeline_builder.rb +138 -0
  91. data/lib/vizcore/dsl/transition_controller.rb +77 -0
  92. data/lib/vizcore/dsl.rb +5 -1
  93. data/lib/vizcore/layer_catalog.rb +273 -0
  94. data/lib/vizcore/project_manifest.rb +152 -0
  95. data/lib/vizcore/renderer/png_writer.rb +57 -0
  96. data/lib/vizcore/renderer/render_sequence.rb +153 -0
  97. data/lib/vizcore/renderer/scene_frame_source.rb +119 -0
  98. data/lib/vizcore/renderer/scene_serializer.rb +36 -3
  99. data/lib/vizcore/renderer/snapshot.rb +38 -0
  100. data/lib/vizcore/renderer/snapshot_renderer.rb +446 -0
  101. data/lib/vizcore/renderer.rb +5 -0
  102. data/lib/vizcore/server/frame_broadcaster.rb +91 -5
  103. data/lib/vizcore/server/gallery_app.rb +155 -0
  104. data/lib/vizcore/server/gallery_page.rb +100 -0
  105. data/lib/vizcore/server/gallery_runner.rb +48 -0
  106. data/lib/vizcore/server/rack_app.rb +203 -4
  107. data/lib/vizcore/server/runner.rb +370 -22
  108. data/lib/vizcore/server/scene_dependency_watcher.rb +79 -0
  109. data/lib/vizcore/server/websocket_handler.rb +60 -10
  110. data/lib/vizcore/server.rb +4 -0
  111. data/lib/vizcore/sync/osc_message.rb +103 -0
  112. data/lib/vizcore/sync/osc_receiver.rb +68 -0
  113. data/lib/vizcore/sync.rb +4 -0
  114. data/lib/vizcore/templates/midi_control_scene.rb +3 -1
  115. data/lib/vizcore/templates/plugin_layer.rb +20 -0
  116. data/lib/vizcore/templates/plugin_readme.md +23 -0
  117. data/lib/vizcore/templates/plugin_renderer.js +43 -0
  118. data/lib/vizcore/templates/plugin_scene.rb +14 -0
  119. data/lib/vizcore/templates/project_readme.md +7 -23
  120. data/lib/vizcore/templates/rubykaigi_scene.rb +30 -0
  121. data/lib/vizcore/version.rb +1 -1
  122. data/lib/vizcore.rb +27 -0
  123. data/scripts/browser_capture.mjs +75 -0
  124. data/sig/vizcore.rbs +362 -0
  125. metadata +83 -3
  126. data/docs/GETTING_STARTED.md +0 -105
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 824691790c88752d418cb0e2ca49a39a15e76926cc8025ac51403b3ba2797f1c
4
- data.tar.gz: bc3ed3cd8282e51dbc21e175b6f8aa76ea7d9c7a7d1ccdfb6e2fc5dfa5b42374
3
+ metadata.gz: 1e2df3ba6e61404ef98823f75264a532ae90e42f0f144a52d89990a08f1f0db9
4
+ data.tar.gz: f38c6843f675ba22c29877718cd518cc460d936181cd381d7d370474984c3181
5
5
  SHA512:
6
- metadata.gz: 4786e0f33292f0598c19c106d544f25775a721e76173bee09fd45f8917a8c148f10a30756c948753165937917f4e5e44c97295e12882243aba6605c07cd3746b
7
- data.tar.gz: 99cbff89caa54d6bbfedfd116754827eb3d09267d931839a4fae2dff65b3ae65ca5efeb8b43f5652c3f35e591aa4a50c37967f1942af66594ea5a3528167bb42
6
+ metadata.gz: cd54c8a93847f3c5d5aea68e9e4ef5a640d8020438f0e0ba479236f68a0343c3b855111b0928f36030950dc84edf64fb8de9496e6ea83c3ac19196bff4c94577
7
+ data.tar.gz: 74b780b71f824a854c8b00575a13cb8dcac7b4f5c4be5138018b0df2efe5b690ab43cf94c6b81cf85fd64b8ac947412e6b07261e690d5f58ce207e54ae6b941c
data/README.md CHANGED
@@ -18,7 +18,7 @@ bundle add vizcore
18
18
 
19
19
  macOS:
20
20
  ```bash
21
- brew install portaudio ffmpeg # ffmpeg only needed for MP3/FLAC input
21
+ brew install portaudio ffmpeg # ffmpeg is needed for MP3/FLAC input and MP4 render output
22
22
  brew install fftw # optional: faster FFT
23
23
  ```
24
24
 
@@ -31,12 +31,19 @@ sudo apt install -y libfftw3-dev # optional: faster FFT
31
31
  ## Quick Start
32
32
 
33
33
  ```bash
34
+ vizcore doctor
35
+ vizcore demo
34
36
  vizcore start examples/basic.rb
35
37
  ```
36
38
 
37
39
  Then open `http://127.0.0.1:4567`.
38
40
 
39
- For full setup, device listing, and troubleshooting, see [GETTING_STARTED.md](docs/GETTING_STARTED.md).
41
+ <p align="center">
42
+ <img src="docs/assets/vizcore-demo.gif" width="640" alt="Animated Vizcore demo where detected beats expand concentric rings" />
43
+ </p>
44
+
45
+ This short preview is generated from `examples/readme_demo.rb` with the bundled
46
+ demo audio: `beat_pulse -> ring radius`.
40
47
 
41
48
  ## Scene DSL
42
49
 
@@ -61,42 +68,460 @@ Vizcore.define do
61
68
  map frequency_band(:low) => :size
62
69
  end
63
70
 
71
+ layer :waveform do
72
+ type :waveform
73
+ source :audio
74
+ style :ribbon
75
+ map amplitude, to: :height, range: 0.2..0.7
76
+ end
77
+
78
+ layer :spectrogram do
79
+ type :spectrogram
80
+ scroll :vertical
81
+ map amplitude, to: :gain, range: 0.8..3.0
82
+ end
83
+
84
+ layer :mesh do
85
+ type :mesh
86
+ geometry :icosahedron
87
+ material :wireframe
88
+ map bass, to: :scale, range: 0.8..1.4
89
+ map high, to: :deform
90
+ end
91
+
92
+ layer :rings do
93
+ circle count: 8 do
94
+ radius 100
95
+ stroke 2
96
+ map bass, to: :radius, range: 40..180
97
+ end
98
+ end
99
+
64
100
  layer :title do
65
101
  type :text
66
- content "DROP"
102
+ content "DROP\nNOW"
103
+ font "Inter Black"
67
104
  font_size 96
105
+ letter_spacing 4
106
+ align :center
107
+ fill "#ffffff"
108
+ stroke width: 2, color: "#111111"
109
+ shadow color: "rgba(0, 0, 0, 0.45)", blur: 18
68
110
  map beat? => :flash
69
111
  end
112
+
113
+ layer :logo do
114
+ type :svg
115
+ file "assets/logo.svg"
116
+ scale 0.9
117
+ map bass, to: :scale, range: 0.8..1.15
118
+ end
119
+
120
+ layer :photo do
121
+ type :image
122
+ file "assets/noise.png"
123
+ fit :cover
124
+ blend :screen
125
+ map amplitude, to: :opacity, range: 0.25..0.85
126
+ end
127
+
128
+ layer :footage do
129
+ type :video
130
+ file "assets/loop.mp4"
131
+ fit :cover
132
+ map beat?, to: :invert
133
+ end
70
134
  end
71
135
 
72
136
  transition from: :intro, to: :drop do
73
- trigger { beat_count >= 64 }
137
+ on_bar 16
74
138
  effect :crossfade, duration: 1.4
75
139
  end
76
140
  end
77
141
  ```
78
142
 
143
+ Mapping options can exaggerate small analysis values, clamp them into useful visual ranges, and smooth changes over time:
144
+
145
+ ```ruby
146
+ layer :liquid do
147
+ shader :liquid_wobble
148
+ wobble 0.25
149
+ warp 0.45
150
+
151
+ map amplitude, to: :wobble, gain: 3.5, range: 0.12..1.4, curve: :sqrt
152
+ map frequency_band(:low), to: :warp, gain: 2.2, range: 0.25..2.4
153
+ map onset(:high), to: :spark, range: 0.0..1.0
154
+ map kick, to: :pulse, range: 0.0..1.0
155
+ map beat_pulse, to: :effect_intensity, range: 0.08..0.35
156
+ end
157
+ ```
158
+
159
+ Available transform options are `gain`, `range`, `min`, `max`, `curve`, `deadzone`, `attack`, and `release`. `curve` supports `:linear`, `:sqrt`, `:square`, and `:ease_out`. Existing mappings such as `map amplitude => :speed` continue to work.
160
+ Use block syntax when shaping a mapping reads better:
161
+
162
+ ```ruby
163
+ map amplitude, to: :scale do
164
+ gain 2.0
165
+ range 0.8..1.6
166
+ curve :ease_out
167
+ smooth attack: 0.02, release: 0.18
168
+ deadzone 0.05
169
+ end
170
+ ```
171
+
172
+ Block syntax is additive and writes the same transform metadata as keyword
173
+ syntax. `deadzone` suppresses tiny values before gain and curve are applied;
174
+ `curve` also supports `:ease_out`.
175
+
176
+ For tracks with very different levels, opt in to adaptive feature
177
+ normalization at the top of the scene file:
178
+
179
+ ```ruby
180
+ audio_normalize mode: :adaptive, window: 3.0, target: 0.85, floor: 0.05
181
+ ```
182
+
183
+ This keeps `amplitude` and FFT-driven mappings in a repeatable range without
184
+ changing the default analysis behavior.
185
+
186
+ When the detected tempo should not drift during a prepared file or live set,
187
+ lock BPM at the top of the scene file:
188
+
189
+ ```ruby
190
+ bpm 128
191
+ bpm_lock true
192
+ ```
193
+
194
+ When you want to set tempo by ear from the browser during a live set, opt in to
195
+ tap tempo and choose the keyboard key:
196
+
197
+ ```ruby
198
+ tap_tempo key: :space
199
+ ```
200
+
201
+ After two valid taps, Vizcore estimates BPM from recent intervals and applies it
202
+ as a locked BPM so beat-driven visuals stop drifting.
203
+
204
+ Browser keyboard shortcuts can also be declared in the scene file for operators
205
+ who do not have a MIDI controller:
206
+
207
+ ```ruby
208
+ key "d" do
209
+ switch_scene :drop
210
+ end
211
+
212
+ key "x" do
213
+ blackout
214
+ end
215
+ ```
216
+
217
+ For a more music-oriented style, `react_to` groups the same mappings by source:
218
+
219
+ ```ruby
220
+ layer :particles do
221
+ type :particle_field
222
+
223
+ react_to bass do
224
+ change :size, gain: 4.0, range: 2.0..8.0, curve: :sqrt
225
+ end
226
+
227
+ react_to beat do
228
+ trigger :burst
229
+ end
230
+ end
231
+ ```
232
+
233
+ `react_to` is additive syntax; it serializes to the same mapping model as `map`.
234
+
235
+ Transitions can use explicit trigger blocks, or beat/bar helpers when that reads
236
+ closer to the structure of a track:
237
+
238
+ ```ruby
239
+ transition from: :build, to: :drop do
240
+ on_bar 8
241
+ effect :flash, duration: 0.35
242
+ end
243
+ ```
244
+
245
+ For simple song structure, `section` defines scenes in order and creates
246
+ beat-counted transitions between adjacent sections:
247
+
248
+ ```ruby
249
+ section :intro, bars: 8 do
250
+ layer :pulse do
251
+ map amplitude => :scale
252
+ end
253
+ end
254
+
255
+ section :drop, bars: 16 do
256
+ layer :sparks do
257
+ map beat? => :burst
258
+ end
259
+ end
260
+ ```
261
+
262
+ For file-backed shows or rehearsed sets, `timeline` can mark existing scenes at
263
+ ordered beat or second positions and generate the transitions:
264
+
265
+ ```ruby
266
+ timeline do
267
+ at beats(0), scene: :intro
268
+ at bars(8), scene: :build
269
+ at bars(16), scene: :drop
270
+ end
271
+ ```
272
+
273
+ Layers can choose their compositing mode with `blend`. Supported modes are `:alpha` / `:normal`, `:add`, `:multiply`, `:screen`, and `:difference`:
274
+
275
+ ```ruby
276
+ layer :sparks do
277
+ type :particle_field
278
+ blend :screen
279
+ map treble, to: :sparkle
280
+ end
281
+ ```
282
+
283
+ Waveform layers draw the current audio features as browser line geometry. Use
284
+ `style :line`, `:mirror`, or `:ribbon`; `source :audio` documents that the layer
285
+ uses the active audio stream:
286
+
287
+ ```ruby
288
+ layer :scope do
289
+ type :waveform
290
+ source :audio
291
+ style :ribbon
292
+ map amplitude, to: :height, range: 0.2..0.7
293
+ end
294
+ ```
295
+
296
+ Video layers embed MP4/WebM/OGV assets as muted looping textures. They share
297
+ `fit`, `scale`, `rotation`, `blend`, and post-effect params with image layers:
298
+
299
+ ```ruby
300
+ layer :footage do
301
+ type :video
302
+ file "assets/loop.mp4"
303
+ fit :cover
304
+ map beat?, to: :invert
305
+ end
306
+ ```
307
+
308
+ Spectrogram layers render a short scrolling FFT heatmap. Use `scroll :vertical`
309
+ for a waterfall view or `scroll :horizontal` when time should move left to right:
310
+
311
+ ```ruby
312
+ layer :waterfall do
313
+ type :spectrogram
314
+ scroll :vertical
315
+ bins 96
316
+ history 128
317
+ map amplitude, to: :gain, range: 0.8..3.0
318
+ end
319
+ ```
320
+
321
+ Mesh layers render preset 3D wireframes without writing GLSL. Available
322
+ geometry presets are `:cube`, `:tetrahedron`, `:octahedron`, and
323
+ `:icosahedron`; use `material :wireframe`:
324
+
325
+ ```ruby
326
+ layer :mesh do
327
+ type :mesh
328
+ geometry :icosahedron
329
+ material :wireframe
330
+ map bass, to: :scale, range: 0.8..1.4
331
+ map high, to: :deform
332
+ end
333
+ ```
334
+
335
+ Shape layers provide declarative circles and lines for scenes that do not need
336
+ custom GLSL. Shape-local mappings target the primitive inside the block:
337
+
338
+ ```ruby
339
+ layer :rings do
340
+ circle count: 8 do
341
+ radius 100
342
+ stroke 2
343
+ map bass, to: :radius, range: 40..180
344
+ end
345
+
346
+ draw do
347
+ line x1: 0, y1: 360, x2: 1280, y2: 360
348
+ end
349
+ end
350
+ ```
351
+
352
+ Layers can also apply browser-side post effects. Supported effects are
353
+ `:bloom`, `:glitch`, `:chromatic`, `:feedback`, `:motion_blur`, and `:crt`:
354
+
355
+ ```ruby
356
+ layer :tunnel do
357
+ shader :bass_tunnel
358
+ effect :motion_blur
359
+ map bass, to: :effect_intensity, range: 0.1..0.7
360
+ end
361
+ ```
362
+
363
+ Reusable layer styles keep repeated visual params in one place:
364
+
365
+ ```ruby
366
+ style :neon do
367
+ palette "#00ffff", "#ff00aa", "#facc15"
368
+ glow_strength 0.45
369
+ blend :add
370
+ end
371
+
372
+ scene :drop do
373
+ layer :title do
374
+ type :text
375
+ use_style :neon
376
+ content "DROP"
377
+ end
378
+ end
379
+ ```
380
+
381
+ Themes provide scene-wide layer defaults:
382
+
383
+ ```ruby
384
+ theme :ruby_night do
385
+ palette "#e11d48", "#f59e0b", "#38bdf8"
386
+ glow_strength 0.5
387
+ blend :screen
388
+ end
389
+
390
+ scene :drop do
391
+ use_theme :ruby_night
392
+
393
+ layer :title do
394
+ type :text
395
+ content "DROP"
396
+ end
397
+ end
398
+ ```
399
+
400
+ Layer groups apply shared params to a subset of adjacent layers while keeping the
401
+ runtime payload as normal ordered layers:
402
+
403
+ ```ruby
404
+ scene :drop do
405
+ group :foreground do
406
+ use_style :neon
407
+ blend :add
408
+ opacity 0.9
409
+
410
+ layer(:particles) { type :particle_field }
411
+ layer(:title) { type :text }
412
+ end
413
+ end
414
+ ```
415
+
416
+ Scenes can inherit shared layers from an earlier scene:
417
+
418
+ ```ruby
419
+ scene :base do
420
+ layer(:background) { shader :neon_grid }
421
+ end
422
+
423
+ scene :drop, extends: :base do
424
+ layer(:particles) { type :particle_field }
425
+ end
426
+ ```
427
+
428
+ Frequency bands can be written with musical aliases when that reads better in a scene:
429
+
430
+ ```ruby
431
+ map bass, to: :size # same as frequency_band(:low)
432
+ map mid, to: :twist
433
+ map treble, to: :sparkle # same as frequency_band(:high)
434
+ map beat_confidence, to: :sync_strength
435
+ ```
436
+
79
437
  ### Custom GLSL Shaders
80
438
 
439
+ Built-in shader presets include `:gradient_pulse`, `:bass_tunnel`,
440
+ `:neon_grid`, `:kaleidoscope`, `:spectrum_rings`, `:liquid_wobble`,
441
+ `:audio_bars`, `:ruby_crystal`, `:starfield`, `:waveform_ribbon`,
442
+ `:unyo_geometry`, and `:glitch_flash`.
443
+
81
444
  ```ruby
82
445
  layer :wave_shader do
83
446
  type :shader
84
447
  glsl "shaders/custom_wave.frag"
448
+ param :intensity, default: 0.6, range: 0.0..2.0, step: 0.05
85
449
  map amplitude => :param_intensity
86
450
  map frequency_band(:low) => :param_bass
87
451
  map beat? => :param_flash
88
452
  end
89
453
  ```
90
454
 
455
+ Path-style shader declarations are also accepted, so `shader "shaders/liquid.frag", reload: true` is equivalent to `glsl "shaders/liquid.frag"` for custom fragment shaders. When hot reload is enabled, Vizcore watches referenced GLSL files and pushes updated shader source to connected browsers.
456
+
457
+ Use `vizcore shader new liquid` to create `shaders/liquid.frag` with a GLSL ES
458
+ starter template.
459
+
460
+ Custom fragment shaders must be GLSL ES 3.00. Run `vizcore shader-docs`
461
+ to print the generated uniform reference. Common uniforms include:
462
+
463
+ - `u_amplitude`
464
+ - `u_bass` / `u_mid` / `u_high`
465
+ - `u_beat`
466
+ - `u_beat_pulse`
467
+ - `u_onset` / `u_low_onset` / `u_mid_onset` / `u_high_onset`
468
+ - `u_kick` / `u_snare` / `u_hihat`
469
+ - `u_bpm`
470
+ - `u_fft[32]`
471
+ - `u_fft_size`
472
+ - `u_param_<name>`
473
+ - `u_global_<name>`
474
+
475
+ For backward compatibility, a DSL target like `:param_intensity` is also exposed as `u_param_intensity`.
476
+ Runtime globals set with `set :global_intensity, 0.75` are exposed as `u_global_intensity`.
477
+ Use `param :name, default:, range:, step:` to attach numeric metadata for shader
478
+ params that can be surfaced by tooling.
479
+ The browser HUD turns this metadata into per-layer shader parameter sliders, so
480
+ declared params can be adjusted during a live run without editing the scene file.
481
+
482
+ ### Plugin Layer Capabilities
483
+
484
+ Plugins can register layer capability metadata for validation, docs, and HUD
485
+ tooling without patching Vizcore itself:
486
+
487
+ ```ruby
488
+ # in a plugin file loaded by Ruby
489
+ Vizcore.register_layer_capability(
490
+ type: :laser_grid,
491
+ aliases: %i[laser_layer],
492
+ params: { beam_count: "Integer", intensity: "Float" },
493
+ mappable_params: %i[intensity opacity],
494
+ description: "Plugin laser renderer."
495
+ )
496
+ ```
497
+
498
+ Load a plugin with `Vizcore.plugin "vizcore-laser-grid"` before defining
499
+ scenes. A plugin-provided browser renderer or shader still needs to handle the
500
+ custom layer type at runtime; the capability API keeps Ruby validation and docs
501
+ aware of the extension.
502
+
503
+ `vizcore plugin new laser-grid` creates a small plugin scaffold with a Ruby
504
+ capability file, a browser renderer that registers with
505
+ `globalThis.VizcorePlugins`, and an example scene. The browser plugin API exposes
506
+ `apiVersion` and version 1 supports `registerLayerRenderer(type, renderer)` for
507
+ line renderers and `registerShaderRenderer(type, renderer)` for shader
508
+ renderers. Browser plugin renderers can return
509
+ `{ kind: "lines", points: [...], color: [r, g, b] }`; shader renderers can return
510
+ a GLSL fragment shader string or `{ kind: "shader", fragmentShader }`. Vizcore
511
+ composites the result using the layer's normal `opacity`, `blend`, and palette
512
+ behavior.
513
+
91
514
  ### MIDI Scene Switching
92
515
 
93
516
  ```ruby
94
517
  Vizcore.define do
518
+ set :global_intensity, 0.65
519
+
95
520
  midi :controller, device: :default
96
521
 
97
522
  scene :warmup do
98
523
  layer :grid do
99
- shader :neon_grid
524
+ shader :waveform_ribbon
100
525
  map frequency_band(:mid) => :intensity
101
526
  end
102
527
  end
@@ -114,8 +539,23 @@ end
114
539
  ## CLI
115
540
 
116
541
  ```bash
117
- vizcore start SCENE_FILE [--host 127.0.0.1] [--port 4567] [--audio-source mic|file|dummy] [--audio-file PATH]
118
- vizcore new PROJECT_NAME
542
+ vizcore start [SCENE_FILE] [--manifest vizcore.yml] [--host 127.0.0.1] [--port 4567] [--audio-source mic|file|dummy] [--audio-file PATH] [--audio-device INDEX_OR_NAME] [--feature-file features.json] [--control-preset controls.json] [--osc-port 9000] [--noise-gate RMS] [--bpm BPM --bpm-lock] [--reload|--no-reload] [--projector]
543
+ vizcore demo [--host 127.0.0.1] [--port 4567] [--control-preset controls.json] [--osc-port 9000] [--projector]
544
+ vizcore doctor
545
+ vizcore validate SCENE_FILE
546
+ vizcore inspect SCENE_FILE
547
+ vizcore snapshot SCENE_FILE [--audio-source dummy|file|mic] [--audio-file PATH] [--out screenshot.png]
548
+ vizcore render SCENE_FILE [--audio-source dummy|file|mic] [--audio-file PATH] [--out frames|movie.mp4] [--frames 60] [--fps 30]
549
+ vizcore capture SCENE_FILE [--audio-source dummy|file|mic] [--out browser-capture.png]
550
+ vizcore browser-capture http://127.0.0.1:4567/projector [--out browser-capture.png]
551
+ vizcore record-features AUDIO_FILE [--out features.json] [--frames 300] [--fps 30]
552
+ vizcore gallery [--host 127.0.0.1] [--port 4568]
553
+ vizcore layers
554
+ vizcore dsl-docs
555
+ vizcore shader new NAME [--out shaders/name.frag]
556
+ vizcore shader-docs
557
+ vizcore plugin new NAME [--out plugins/name]
558
+ vizcore new PROJECT_NAME [--template standard|minimal|shader|midi|live-set|rubykaigi]
119
559
  vizcore devices [audio|midi]
120
560
  ```
121
561
 
@@ -129,39 +569,134 @@ vizcore devices [audio|midi]
129
569
 
130
570
  ```bash
131
571
  # Microphone
132
- vizcore start scene.rb --audio-source mic
572
+ vizcore start examples/audio_inspector.rb --audio-source mic
573
+
574
+ # Specific microphone device
575
+ vizcore devices audio
576
+ vizcore start examples/audio_inspector.rb --audio-source mic --audio-device 5
577
+
578
+ # Lower this when the selected input is too quiet to move the visual
579
+ vizcore start examples/audio_inspector.rb --audio-source mic --audio-device 5 --noise-gate 0.001
133
580
 
134
581
  # WAV file
135
582
  vizcore start scene.rb --audio-source file --audio-file track.wav
136
583
 
137
584
  # MP3/FLAC (requires ffmpeg)
138
585
  vizcore start scene.rb --audio-source file --audio-file set.mp3
586
+
587
+ # Recorded analysis features
588
+ vizcore record-features track.wav --out features.json --frames 300 --fps 30
589
+ vizcore start scene.rb --feature-file features.json
139
590
  ```
140
591
 
141
592
  When using file source, the HUD exposes **Play Audio** / **Pause Audio** controls and shows BPM, Beat, and Beat Count.
142
593
 
594
+ The browser HUD also includes an Audio Inspector with amplitude, sub/low/mid/high meters, FFT preview bars, a performance monitor for FPS/frame/latency/drop/audio/shader/reconnect health, shader compile error overlay, emergency Blackout/Freeze controls, and Visual Gain, Bass Boost, Smoothing, Beat Hold, and Wobble controls for adapting visual response to different tracks and input levels. Reactivity controls can be saved, loaded, imported, and exported for repeatable HUD presets. When `--control-preset` or manifest `control_preset` is writable, the HUD also shows **Save Project** to write the current reactivity and MIDI Learn bindings back to that JSON file. MIDI Learn can bind Web MIDI note/CC/program messages to the current scene, Blackout/Freeze, or reactivity controls. Scene launcher entries can be selected with `1`-`9`, and scene-defined `key` mappings can switch scenes or toggle live controls. Use `--projector` or open `/projector` when the browser output should hide operator UI, and open `/control` for a separate operator panel.
595
+
596
+ Project manifests keep show startup repeatable:
597
+
598
+ ```yaml
599
+ # vizcore.yml
600
+ scene: scenes/show.rb
601
+ audio:
602
+ source: file
603
+ file: audio/set.wav
604
+ control_preset: controls/live.json
605
+ sync:
606
+ osc:
607
+ port: 9000
608
+ plugins:
609
+ - require: vizcore-laser-grid
610
+ frontend: frontend/laser-grid-renderer.js
611
+ plugin_assets:
612
+ - frontend/local-overlay.js
613
+ profiles:
614
+ rehearsal:
615
+ audio:
616
+ source: dummy
617
+ control_preset: controls/rehearsal.json
618
+ ```
619
+
620
+ Start it with `vizcore start --manifest vizcore.yml`, or
621
+ `vizcore start --manifest vizcore.yml --profile rehearsal` for profile-specific
622
+ overrides. Manifest plugin `frontend` entries and `plugin_assets` are served by
623
+ RackApp and loaded before `/src/main.js`. A control preset JSON can include
624
+ `visual_settings` and `midi_learn_bindings`; Vizcore sends it through `/runtime`
625
+ and the browser applies it to HUD reactivity and MIDI Learn state.
626
+ Set `--osc-port` or `sync.osc.port` to receive OSC controls:
627
+ `/vizcore/scene` with a string scene name, `/vizcore/tap`,
628
+ `/vizcore/bpm`, `/vizcore/bpm_unlock`, `/vizcore/global/<name>`,
629
+ `/vizcore/live/blackout`, `/vizcore/live/freeze`, and
630
+ `/vizcore/transport/play` or `/vizcore/transport/stop` for file transport.
631
+
632
+ `vizcore demo` starts a bundled scene with bundled audio, so it is the quickest way to verify a fresh installation.
633
+
634
+ `vizcore start scene.rb --reload` watches the scene file and pushes changes to connected browsers without restarting the server. Hot reload is enabled by default; use `--no-reload` when you want a fixed scene for a show.
635
+
636
+ Use `vizcore snapshot scene.rb --audio-source dummy --out screenshot.png` to create a software-rendered PNG preview for README, social cards, or quick visual checks without starting the browser.
637
+
638
+ Use `vizcore render scene.rb --audio-source file --audio-file track.wav --out frames --frames 120 --fps 30` to write a software-rendered PNG image sequence, or `--out movie.mp4` to encode the frames to MP4 with `ffmpeg`.
639
+
640
+ Use `vizcore browser-capture http://127.0.0.1:4567/projector --out browser.png`
641
+ when you need a PNG from an already-running browser/WebGL renderer. Use
642
+ `vizcore capture scene.rb --out browser.png` to start a temporary projector
643
+ server, capture it, and stop the server automatically. Both commands require
644
+ Playwright in the local Node environment.
645
+
646
+ Use `vizcore record-features track.wav --out features.json --frames 300 --fps 30` to capture the same audio analysis values as JSON for debugging mappings or comparing tracks. Replay the file with `vizcore start scene.rb --feature-file features.json` when you want deterministic visual behavior without live audio input.
647
+
143
648
  ## Requirements
144
649
 
145
650
  - Ruby `>= 3.2`
146
651
  - `portaudio` for microphone input
147
- - `ffmpeg` on `PATH` when using `.mp3` or `.flac` file input
652
+ - `ffmpeg` on `PATH` when using `.mp3` / `.flac` file input or `.mp4` render output
148
653
  - `fftw3` (optional) — Vizcore falls back to pure-Ruby FFT automatically when unavailable
149
654
 
150
655
  ## Examples
151
656
 
657
+ Run `vizcore gallery` to open a browser gallery of bundled examples with scene counts, layer counts, audio-source hints, and launch commands.
658
+
152
659
  | File | Description |
153
660
  |------|-------------|
154
661
  | `examples/basic.rb` | Single wireframe cube layer |
155
662
  | `examples/intro_drop.rb` | Beat-triggered scene transition |
156
663
  | `examples/file_audio_demo.rb` | File audio source walkthrough |
157
664
  | `examples/complex_audio_showcase.rb` | Dense multi-layer showcase |
665
+ | `examples/rhythm_geometry.rb` | Single large morphing geometric pattern scene with drum-reactive motion |
666
+ | `examples/ruby_crystal_show.rb` | Ruby-themed crystal, particles, and title visual |
667
+ | `examples/parser_visualizer.rb` | Parser-themed token, AST, and reduce visual sketch |
668
+ | `examples/live_coding_minimal.rb` | Tiny scene for live-coding demos |
669
+ | `examples/club_intro_drop.rb` | Intro, build, and drop flow for rhythmic file input |
670
+ | `examples/shader_playground.rb` | Focused shader scene with declared params |
671
+ | `examples/audio_inspector.rb` | Audio bars and blob for analysis visualization |
672
+ | `examples/readme_demo.rb` | Minimal beat pulse to ring radius demo |
158
673
  | `examples/midi_scene_switch.rb` | MIDI-driven scene switching |
674
+ | `examples/midi_controller_show.rb` | MIDI pads for scenes and knobs for global shader intensity/color |
675
+ | `examples/kansai_rubykaigi_visual.rb` | Event showcase with ruby crystal, water ripple, and Kyoto-inspired pattern |
159
676
  | `examples/custom_shader.rb` | Custom GLSL shader with audio mapping |
677
+ | `examples/unyo_liquid.rb` | Organic liquid wobble scene with FFT blob and particles |
678
+
679
+ ### VJ Set Examples
680
+
681
+ Ready-to-run scenes grouped by genre / mood. Each accepts live mic input or any
682
+ audio file, including the bundled `examples/assets/complex_demo_loop.wav`.
683
+
684
+ | File | Genre / Mood | BPM | Scenes | Notes |
685
+ |------|--------------|-----|--------|-------|
686
+ | `examples/vj_techno_warehouse.rb` | Techno / Warehouse | 128-140 | 4 | wireframe + particles |
687
+ | `examples/vj_dnb_jungle.rb` | Drum & Bass / Jungle | 170-180 | 3 | kick/snare/hihat split |
688
+ | `examples/vj_ambient_chill_room.rb` | Ambient / Chill | 60-90 | 2 | beatless, drone-friendly |
689
+ | `examples/vj_hiphop_cipher.rb` | HipHop / Cipher | 85-100 | 3 | text-forward |
690
+ | `examples/vj_jpop_idol_live.rb` | J-POP / Idol | 130-180 | 4 | color fields + tap tempo |
691
+ | `examples/vj_synthwave_retro.rb` | Synthwave / Retro | 100-120 | 3 | circle and line primitives |
692
+ | `examples/vj_glitch_industrial.rb` | Glitch / Industrial | n/a | 3 | feedback + difference blend |
693
+ | `examples/vj_festival_mainstage.rb` | EDM / Mainstage | 124-132 | 5 | uses `extends:` and `group` |
160
694
 
161
695
  ## Development
162
696
 
163
697
  ```bash
164
698
  bundle exec rspec
699
+ npm --prefix frontend test
165
700
  ```
166
701
 
167
702
 
data/docs/.nojekyll ADDED
File without changes