@algorisys/tinyfly 0.100.0 → 0.102.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (71) hide show
  1. package/README.md +6 -6
  2. package/bin/tinyfly.mjs +16 -6
  3. package/docs/character-appearance.md +26 -0
  4. package/docs/editor-guide.md +1 -1
  5. package/docs/examples.md +1 -1
  6. package/docs/getting-started.md +1 -1
  7. package/docs/gsap-compat.md +1 -1
  8. package/docs/llm-guide.md +22 -4
  9. package/docs/maps.md +2 -2
  10. package/docs/scene-3d.md +140 -9
  11. package/docs/teaching.md +1 -1
  12. package/docs/video-rendering.md +198 -1
  13. package/lib/addons/adapters.js +431 -332
  14. package/lib/addons/characters.js +3889 -3773
  15. package/lib/addons/load-scene-oWNsYYC1.js +372 -0
  16. package/lib/addons/scene-3d-webgl.js +31 -31
  17. package/lib/addons/scene-3d.js +669 -314
  18. package/lib/addons/shading-0lPgnXmD.js +96 -0
  19. package/lib/browser/tinyfly-scene-3d-webgl.iife.js +2 -2
  20. package/lib/browser/tinyfly-scene-3d.iife.js +2 -2
  21. package/lib/browser/tinyfly.iife.js +15 -15
  22. package/lib/browser/tinyfly.js +6232 -5723
  23. package/lib/browser/tinyfly.umd.js +15 -15
  24. package/lib/engine/tinyfly-engine.js +1563 -1170
  25. package/lib/engine/tinyfly-engine.umd.cjs +2 -2
  26. package/lib/headless/headless.js +189 -6
  27. package/lib/types/adapters/canvas/framing.d.ts +27 -0
  28. package/lib/types/adapters/canvas/impossible.d.ts +15 -0
  29. package/lib/types/adapters/canvas/index.d.ts +4 -0
  30. package/lib/types/adapters/canvas/nested-zoom.d.ts +40 -0
  31. package/lib/types/adapters/canvas/zoom-blur.d.ts +49 -0
  32. package/lib/types/characters/character-script-3d.d.ts +21 -1
  33. package/lib/types/characters/character.d.ts +23 -3
  34. package/lib/types/characters/head/face.d.ts +19 -2
  35. package/lib/types/characters/index.d.ts +2 -1
  36. package/lib/types/characters/staging-3d.d.ts +56 -0
  37. package/lib/types/engine/authoring/beat-loop.d.ts +46 -0
  38. package/lib/types/engine/authoring/curve.d.ts +26 -0
  39. package/lib/types/engine/authoring/framing.d.ts +48 -0
  40. package/lib/types/engine/authoring/index.d.ts +4 -0
  41. package/lib/types/engine/authoring/periodic.d.ts +89 -0
  42. package/lib/types/engine/path/goo.d.ts +103 -0
  43. package/lib/types/engine/path/impossible.d.ts +92 -0
  44. package/lib/types/engine/path/index.d.ts +6 -0
  45. package/lib/types/engine/path/similarity.d.ts +48 -0
  46. package/lib/types/headless/audio-mix.d.ts +40 -0
  47. package/lib/types/headless/ffmpeg-args.d.ts +2 -0
  48. package/lib/types/headless/frame-renderer.d.ts +7 -1
  49. package/lib/types/headless/index.d.ts +2 -0
  50. package/lib/types/headless/loop-audio.d.ts +21 -0
  51. package/lib/types/headless/node-audio.d.ts +39 -0
  52. package/lib/types/headless/node-video.d.ts +2 -0
  53. package/lib/types/headless/video-scene.d.ts +21 -0
  54. package/lib/types/scene-3d/camera.d.ts +11 -2
  55. package/lib/types/scene-3d/geometry/mesh.d.ts +2 -0
  56. package/lib/types/scene-3d/geometry/primitives.d.ts +7 -0
  57. package/lib/types/scene-3d/impossible-3d.d.ts +29 -0
  58. package/lib/types/scene-3d/index.d.ts +7 -3
  59. package/lib/types/scene-3d/lighting.d.ts +58 -0
  60. package/lib/types/scene-3d/resolve-scene.d.ts +18 -2
  61. package/lib/types/scene-3d/scene-3d-adapter.d.ts +4 -0
  62. package/lib/types/scene-3d/scene-types.d.ts +43 -6
  63. package/lib/types/scene-3d/shading.d.ts +2 -0
  64. package/lib/types/scene-3d/shots.d.ts +162 -0
  65. package/lib/types/scene-3d/software-renderer.d.ts +40 -0
  66. package/llms-full.txt +395 -23
  67. package/llms.txt +29 -29
  68. package/package.json +1 -1
  69. package/skills/tinyfly/SKILL.md +11 -1
  70. package/lib/addons/load-scene-CQvcFWGJ.js +0 -331
  71. package/lib/addons/shading-o5T_v9DT.js +0 -89
@@ -14,6 +14,7 @@ npx tinyfly video scene.mjs --stills stills/ # one PNG per caption line (els
14
14
  npx tinyfly video scene.mjs --stills stills/ --frames 0,79,88 # stills at chosen frames (or --times 0,2640 in ms)
15
15
  npx tinyfly video scene.mjs --loop-check # does it loop? (exit 1 when not)
16
16
  npx tinyfly video scene.mjs --scale 0.5 --fps 12 # quick preview
17
+ npx tinyfly video scene.mjs --codec prores # a ProRes 422 HQ .mov for editing elsewhere
17
18
  ```
18
19
 
19
20
  Runnable scenes are in [`examples/headless-video/`](../examples/headless-video/):
@@ -142,6 +143,38 @@ Scene modules may use top-level `await`, or default-export an async function.
142
143
  `assembleNarration(plan, clips, sampleRate)` and `encodeWav(samples, sampleRate)`
143
144
  are the pure steps underneath, for audio you generate yourself.
144
145
 
146
+ ### A sound mix
147
+
148
+ A film's sound is layers: a music bed, the narration, effects at the moments
149
+ things happen, with the music ducked under the voice. `renderMix` decodes
150
+ every file with ffmpeg, mixes them and writes one WAV for the scene's
151
+ `audio`:
152
+
153
+ ```js
154
+ import { renderMix } from '@algorisys/tinyfly/headless'
155
+
156
+ const audio = await renderMix(
157
+ [
158
+ { id: 'bed', file: 'music/bed.mp3', loop: true, gain: 0.6, fadeOut: 1500, duck: { under: 'voice', amount: 0.7 } },
159
+ { id: 'voice', file: 'build/narration.wav' },
160
+ { file: 'sfx/tap.wav', at: [1200, 2650, 4100], gain: 0.8 }, // a sound effect at three moments
161
+ { file: 'sfx/thud.wav', at: landings.map((t) => t + 40) }, // or at every landing a script reports
162
+ ],
163
+ { output: 'build/mix.wav', duration: END, baseDir: here }
164
+ )
165
+ export default { /* … */ audio }
166
+ ```
167
+
168
+ Each layer has `start` (ms) or `at` (times for a sound effect), `gain`,
169
+ `fadeIn` and `fadeOut`, `loop` for a bed shorter than the film, and `duck:
170
+ { under, amount, attack, release }` to make way for another layer while it
171
+ sounds. The whole mix is soft-limited so peaks never clip. `mixAudio(layers,
172
+ { sampleRate, duration, master })` is the pure step on samples. For cuts and
173
+ dances on the music's beat, `detectTempo(samples, sampleRate)` finds the
174
+ grid; `snapShotsToBeats(shots, grid)` moves each cut to the nearest beat,
175
+ and `danceTracks(target, { bpm: grid.bpm, start: nextBeat(grid, t) })`
176
+ dances to it.
177
+
145
178
  ## Characters
146
179
 
147
180
  `@algorisys/tinyfly/characters` has a poseable stick figure. A pose is a set of
@@ -557,6 +590,141 @@ The numbers are PSNR (higher is more alike; fast or trembling motion makes
557
590
  every step lower). From code: `checkLoop(scene)` returns the same report, and
558
591
  `psnr(a, b)` / `loopReport(frames)` work on any RGBA frames.
559
592
 
593
+ ### A scene that loops
594
+
595
+ Say so with `loop: true`: the duration is the period. `tinyfly video` then
596
+ runs the loop check before it encodes anything and stops (exit 1) if the loop
597
+ does not close; every frame carries `period`, which the periodic helpers and
598
+ 3D trails without a `period` of their own use.
599
+
600
+ ```js
601
+ import { wobble, loopNoise, loopEase } from '@algorisys/tinyfly'
602
+
603
+ export default {
604
+ width: 1280, height: 720, duration: 4000, loop: true,
605
+ draw(ctx, frame) {
606
+ const sway = wobble(frame.time, { period: frame.period, cycles: 2, amplitude: 12 }) // a sine, two whole cycles
607
+ const drift = loopNoise(frame.time, { period: frame.period, seed: 'tree' }) * 30 // smooth noise round a circle
608
+ const breathe = loopEase(frame.time, { period: frame.period, easing: 'ease-in-out' }) // 0 → 1 → 0, eased
609
+ },
610
+ }
611
+ ```
612
+
613
+ | Helper | Repeats exactly every `period` |
614
+ |---|---|
615
+ | `wobble(t, { period, cycles, amplitude, phase, shape })` | A sine (or `triangle`, `square`) with a whole number of cycles |
616
+ | `loopEase(t, { period, cycles, easing, phase })` | 0 → 1 → 0, the easing on the way up and mirrored down |
617
+ | `loopNoise(t, { period, seed, cycles, octaves, phase })` | Seeded value noise read round a circle, about −1..1: the end meets the start |
618
+ | `torusNoise(t, along, { period, periodAlong, seed, … })` | Noise periodic in time and along a length (ripples, grass, a flag) |
619
+ | `phaseOf(t, period, cycles)`, `pingPong(u)` | The building blocks: where in the cycle, and there-and-back |
620
+
621
+ Each is a pure function of time, so frames render in any order and the loop
622
+ check reports "closes", not "seamless".
623
+
624
+ ### Sound for a loop
625
+
626
+ A loop's sound must come round with its picture. `cutAudioToLoop(file, {
627
+ output, period, crossfade, fadeIn, fadeOut })` (Node) decodes a drone, a
628
+ riser or a beat, trims it to the period with its end blended into its start
629
+ (tiling it first if it is shorter) and writes the WAV to pass as the scene's
630
+ `audio`; `loopSamples` is the pure step on samples. For motion on the beat,
631
+ `loopBeats(period, grid)` cuts a tempo to the loop (the nearest whole number
632
+ of beats, the bpm nudged to fit exactly), `beatPulse(time, grid, { attack,
633
+ decay, barAccent })` spikes to 1 on every beat and decays, and
634
+ `beatPeriod(grid, beats)` is the length of so many beats. For motion that
635
+ follows the sound itself, `audioEnvelope(samples, sampleRate, { fps,
636
+ smoothing })` gives a level per frame (rising at once, letting go slowly,
637
+ loudest 1) and `envelopeAt(levels, time, fps, period)` reads it at any time,
638
+ wrapped for the loop:
639
+
640
+ ```js
641
+ import { decodeAudio, cutAudioToLoop } from '@algorisys/tinyfly/headless'
642
+ import { audioEnvelope, envelopeAt, loopBeats, beatPulse, detectTempo } from '@algorisys/tinyfly'
643
+
644
+ const PERIOD = 4000
645
+ const audio = await cutAudioToLoop('drone.wav', { output: 'build/drone-loop.wav', period: PERIOD, baseDir: here })
646
+ const samples = await decodeAudio(audio, { sampleRate: 48000 })
647
+ const levels = audioEnvelope(samples, 48000, { fps: 30 })
648
+ const { grid } = loopBeats(PERIOD, detectTempo(samples, 48000))
649
+ export default {
650
+ width: 1280, height: 720, duration: PERIOD, loop: true, audio,
651
+ draw(ctx, frame) {
652
+ const level = envelopeAt(levels, frame.time, 30, frame.period) // 0..1 with the sound
653
+ const pulse = beatPulse(frame.time, grid) // 1 on each beat, decaying
654
+ },
655
+ }
656
+ ```
657
+
658
+ ### Nested zooms (Droste)
659
+
660
+ A picture inside itself, the camera diving in: after one step every level is
661
+ where the next one was, so a video one step long loops. `similarityAbout(
662
+ scale, turn, centre)` is the step (a scale, a turn and a move as one value;
663
+ `powSimilarity(step, k)` is a fraction of it, `composeSimilarity` chains
664
+ them, `similarityFixedPoint` is the centre everything converges on), and
665
+ `nestedZoom` draws the levels:
666
+
667
+ ```js
668
+ import { similarityAbout } from '@algorisys/tinyfly'
669
+ import { nestedZoom } from '@algorisys/tinyfly/adapters'
670
+
671
+ const step = similarityAbout(0.5, 12, { x: 640, y: 360 })
672
+ export default {
673
+ width: 1280, height: 720, duration: 3000, loop: true, background: '#05030c',
674
+ draw(ctx, frame) {
675
+ nestedZoom(ctx, {
676
+ step,
677
+ phase: frame.time / frame.period, // 0..1: how far into one step
678
+ extent: 1600, // how far a level's drawing reaches at level 0, px
679
+ coversAbove: 1.1, // a level this big fills the frame: nothing behind it is drawn
680
+ draw(ctx, level, scale) { drawFrame(ctx, level) }, // in level-0 coordinates
681
+ })
682
+ },
683
+ }
684
+ ```
685
+
686
+ Levels smaller than `minSize` px (default 1) are skipped, and so is
687
+ everything behind a level at or above `coversAbove`, so a zoom of twenty
688
+ levels draws the five or six that show. `draw` gets the level number (0 is
689
+ the one at natural size at phase 0, 1 the one inside it) and its screen
690
+ scale, to keep line widths readable; colour or detail that should travel with
691
+ the zoom is a function of `level - phase`. `examples/headless-video/nested-zoom.mjs`
692
+ is a looping neon zoom built this way; `nestedZoomTransform(step, level,
693
+ phase)` gives a level's similarity for drawing it some other way.
694
+
695
+ A Droste *face*: `drawCharacterHead` draws a character's head big, and its
696
+ `face.inPupil` hook draws inside each pupil, clipped to it, so the head can
697
+ hold itself (see [Close-ups](character-appearance.md#close-ups)). Anything
698
+ that should change with a level's size on screen rather than with time (the
699
+ eyes opening as a head grows, a colour cooling with depth) is a `curve`:
700
+ `curve([{ at: 20, value: 0 }, { at: 300, value: 1, easing: 'ease-out' }])(radius)`
701
+ reads keyframes and easings at any input, holding beyond its ends.
702
+
703
+ ### Dives and impossible objects
704
+
705
+ Two post-effects for a dive, both deterministic. `applyZoomBlur(ctx, { centre,
706
+ strength, samples, blend })` streaks the finished frame out from a point (a
707
+ fixed set of scaled copies, averaged, or added with `blend: 'add'` for neon);
708
+ `drawDiveLines(ctx, { centre, count, inner, outer, length, phase, seed, color,
709
+ blend })` streams seeded lines past the camera, and `phase: phaseOf(time,
710
+ period)` moves them one line along per loop so they repeat exactly.
711
+
712
+ The impossible objects are built the way the sculptures are: real boxes that
713
+ line up from one viewpoint. `penroseTriangle({ size, thickness, centre, scale })`
714
+ and `penroseStairs({ steps, tread, rise, width, base, centre, scale })` give
715
+ boxes and their visible faces in drawing order for the isometric view (along
716
+ the (1, 1, 1) diagonal, z up); `drawIsometricFaces(ctx, faces, { top, left,
717
+ right, fill, outline })` fills them. The triangle's third beam ends one
718
+ diagonal from where the first began, so on screen the loop closes (the first
719
+ beam is drawn again over the third's end); the stairs climb all the way round
720
+ and the last step meets the first. `projectIsometric`, `isometricBoxFaces`
721
+ and `isometricFaces` are the building blocks for your own. In a 3D scene,
722
+ `penroseTriangle3D({ length, thickness, material })` returns the three beams
723
+ and an orthographic camera down the diagonal: from it the loop closes, from
724
+ anywhere else it is three beams (its third corner shows the beams' ends,
725
+ as a real sculpture's does; a `rotateY` track on a group round it breaks and
726
+ remakes the illusion).
727
+
560
728
  ## Bloom
561
729
 
562
730
  `bloom: true` (or options) adds a glow around everything bright, after each
@@ -633,6 +801,25 @@ beads where segments overlap and no seams between them. `ribbon(points)` and
633
801
  `line` and `trail` objects, sorted by depth among the meshes (see
634
802
  [3D Scenes](scene-3d.md#lines-and-trails)).
635
803
 
804
+ ## Satisfying matter
805
+
806
+ Goo as geometry, every shape a pure function of its inputs:
807
+
808
+ | Helper | Gives |
809
+ |---|---|
810
+ | `metaballContours(balls, { threshold, cell, smooth })` | The outlines of balls (`{ x, y, r }`) that merge through necks as they near and split as they part, as closed polygons; `metaballField` is the field itself |
811
+ | `dripPath({ x, y, length, width, progress })` | A drip under a surface: a neck thinning to a thread and a head swelling as `progress` runs 0 → 1 |
812
+ | `bubblePath({ x, y, r, grow, time, period, wobble })` | A bubble growing from nothing (with a little swell on the way) whose surface quivers with the time; with `period` the quiver repeats exactly |
813
+ | `bubblePop({ x, y, r, progress, pieces })` | The pop: arcs of the rim flying out and shrinking, gone at 1 |
814
+ | `squashStretch(amount, { keep })`, `scalePoints(points, scale, about)` | Stretch taller (or squash, negative) with the width giving way so the area (or the `volume`) stays; apply it about the point a blob lands on |
815
+
816
+ ```js
817
+ draw(ctx, frame) {
818
+ const balls = [{ x: 400, y: 300, r: 60 }, { x: 400 + wobble(frame.time, { period: frame.period, amplitude: 90 }), y: 300, r: 40 }]
819
+ for (const loop of metaballContours(balls, { cell: 3 })) fillPolygon(ctx, loop, '#b6ff3b')
820
+ }
821
+ ```
822
+
636
823
  ## Camera, overlay and drawing on twos
637
824
 
638
825
  - `camera: true` sees the scene through the tracks of the `Camera` target:
@@ -644,6 +831,12 @@ beads where segments overlap and no seams between them. `ribbon(points)` and
644
831
  stay put while the camera moves.
645
832
  - `drawingRate: 12` holds the timeline's poses for two frames of 24 fps ("on
646
833
  twos"), as hand-drawn animation is timed.
834
+ - `motionBlur: { samples: 4, shutter: 0.5 }` makes each frame the average of
835
+ four drawings spread over the shutter's open time (half the frame, a 180°
836
+ shutter): fast moves streak as a camera's do. Deterministic, `samples`
837
+ times the work.
838
+ - `supersample: 2` draws at twice the size and scales down: smoother edges
839
+ on thin lines and small text (four times the work).
647
840
 
648
841
  See [acting.md](acting.md) for camera shots, beat scripts and the rest of the
649
842
  cartoon acting tools.
@@ -668,6 +861,7 @@ await renderVideo(scene, {
668
861
  output: 'out.mp4',
669
862
  baseDir: '.', // relative audio and font paths resolve against this
670
863
  scale: 1, crf: 20, preset: 'medium',
864
+ codec: 'h264', // or 'prores' for a .mov
671
865
  onProgress: (done, total) => {},
672
866
  })
673
867
  await renderStills(scene, { dir: 'stills' })
@@ -724,5 +918,8 @@ room) was ported call for call to Canvas 2D and rendered at 1920×1080, 24 fps,
724
918
  rendering) and `ffmpeg`. Rendering defines `globalThis.Path2D` from the canvas
725
919
  package when it is missing, because path targets use it.
726
920
  - Output is H.264 MP4 (`yuv420p`, odd sizes padded to even), with AAC audio when
727
- `audio` is set; video stops at the shorter of picture and sound.
921
+ `audio` is set; video stops at the shorter of picture and sound. `codec:
922
+ 'prores'` (or `--codec prores`) writes ProRes 422 HQ (10-bit, PCM audio) in
923
+ a `.mov` for an editor; any size works, 1080p and 4K included; `renderStills`
924
+ gives a PNG sequence.
728
925
  - `image` targets need images loaded with the canvas package's `loadImage`.