framefields 2.1.3 → 2.2.0-beta.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +48 -618
- package/dist/audio/index.d.mts +4 -4
- package/dist/audio/index.mjs +10 -10
- package/dist/{audio-CfBpzxn9.d.mts → audio-1QquqDJs.d.mts} +3 -3
- package/dist/{audio-CfBpzxn9.d.mts.map → audio-1QquqDJs.d.mts.map} +1 -1
- package/dist/{audio-DqaHxtG3.mjs → audio-BYqbtdH8.mjs} +3 -3
- package/dist/{audio-DqaHxtG3.mjs.map → audio-BYqbtdH8.mjs.map} +1 -1
- package/dist/{dist-BgFYDCoL.mjs → dist-BLbXEC6V.mjs} +2 -2
- package/dist/{dist-BgFYDCoL.mjs.map → dist-BLbXEC6V.mjs.map} +1 -1
- package/dist/{dist-Cog5jyVI.mjs → dist-DQlYw4eG.mjs} +44 -4
- package/dist/dist-DQlYw4eG.mjs.map +1 -0
- package/dist/effects/index.d.mts +4 -4
- package/dist/effects/index.mjs +34 -34
- package/dist/{effects-CXrmO68D.mjs → effects-DgpPr38s.mjs} +2 -2
- package/dist/{effects-CXrmO68D.mjs.map → effects-DgpPr38s.mjs.map} +1 -1
- package/dist/fonts/index.mjs +2 -2
- package/dist/{fonts-BUVSUWXJ.mjs → fonts-DakFEiDw.mjs} +2 -2
- package/dist/{fonts-BUVSUWXJ.mjs.map → fonts-DakFEiDw.mjs.map} +1 -1
- package/dist/{gpu-Bj08zUc5.mjs → gpu-AbcLcr1f.mjs} +2 -2
- package/dist/{gpu-Bj08zUc5.mjs.map → gpu-AbcLcr1f.mjs.map} +1 -1
- package/dist/{index-Cpkzy5XR.d.mts → index--1jnkoBj.d.mts} +20 -2
- package/dist/index--1jnkoBj.d.mts.map +1 -0
- package/dist/{index-D__HdInU.d.mts → index-Bg8HIYAO.d.mts} +4 -4
- package/dist/{index-D__HdInU.d.mts.map → index-Bg8HIYAO.d.mts.map} +1 -1
- package/dist/{index-BzQc812z.d.mts → index-BqzHDFkU.d.mts} +3 -3
- package/dist/{index-BzQc812z.d.mts.map → index-BqzHDFkU.d.mts.map} +1 -1
- package/dist/{index-DpWia6iH.d.mts → index-CaZfxDXx.d.mts} +27 -3
- package/dist/index-CaZfxDXx.d.mts.map +1 -0
- package/dist/{index-Fciq8bcC.d.mts → index-D6_KcajH.d.mts} +2 -2
- package/dist/{index-Fciq8bcC.d.mts.map → index-D6_KcajH.d.mts.map} +1 -1
- package/dist/{index--0vVZDPM.d.mts → index-DgJIBC_N.d.mts} +3 -3
- package/dist/{index--0vVZDPM.d.mts.map → index-DgJIBC_N.d.mts.map} +1 -1
- package/dist/index.d.mts +17 -10
- package/dist/index.d.mts.map +1 -1
- package/dist/index.mjs +49 -49
- package/dist/index.mjs.map +1 -1
- package/dist/preview/index.d.mts +2 -2
- package/dist/preview/index.mjs +10 -10
- package/dist/{preview-BOqCRCgp.mjs → preview-RdaP0Rfv.mjs} +2 -2
- package/dist/{preview-BOqCRCgp.mjs.map → preview-RdaP0Rfv.mjs.map} +1 -1
- package/dist/preview-engine/audio.mjs +1 -1
- package/dist/preview-engine/{chunk-JJ5EF3EY.mjs → chunk-2W4BMOAZ.mjs} +1 -1
- package/dist/preview-engine/{chunk-AE2336RK.mjs → chunk-FFQ3K5FW.mjs} +1 -1
- package/dist/preview-engine/chunk-FJH5A2PV.mjs +1 -0
- package/dist/preview-engine/{chunk-5DOD2NVY.mjs → chunk-MOW55DX2.mjs} +1 -1
- package/dist/preview-engine/{chunk-USM33CF5.mjs → chunk-UKBUEM3G.mjs} +1 -1
- package/dist/preview-engine/chunk-VUFDQJ2B.mjs +1 -0
- package/dist/preview-engine/{chunk-ZIIFTWDH.mjs → chunk-XETMNJVV.mjs} +1 -1
- package/dist/preview-engine/effects.mjs +1 -1
- package/dist/preview-engine/framefields.mjs +21 -21
- package/dist/preview-engine/player.mjs +1 -1
- package/dist/preview-engine/{renderers-BH5MXUTT.mjs → renderers-OZYUGZG7.mjs} +1 -1
- package/dist/preview-engine/{renderers-2SVWAB42.mjs → renderers-UHYNEWQD.mjs} +1 -1
- package/dist/preview-engine/signals.mjs +1 -1
- package/dist/{program-D8JoVobr.mjs → program-Cc_2_YXn.mjs} +2 -2
- package/dist/{program-D8JoVobr.mjs.map → program-Cc_2_YXn.mjs.map} +1 -1
- package/dist/renderer/index.d.mts +3 -3
- package/dist/renderer/index.mjs +3 -3
- package/dist/{renderer-B9ProG0Z.mjs → renderer-2DjggBBS.mjs} +2 -2
- package/dist/{renderer-B9ProG0Z.mjs.map → renderer-2DjggBBS.mjs.map} +1 -1
- package/dist/{renderer-CBBhVrwO.mjs → renderer-4GYrnTfw.mjs} +2 -2
- package/dist/{renderer-CBBhVrwO.mjs.map → renderer-4GYrnTfw.mjs.map} +1 -1
- package/dist/renderer-9ETXaINw.mjs +5 -0
- package/dist/renderer-B3MVDN_e.mjs +5 -0
- package/dist/{renderer-CZ7-42SE.mjs → renderer-B4cnT3ce.mjs} +3 -3
- package/dist/{renderer-CZ7-42SE.mjs.map → renderer-B4cnT3ce.mjs.map} +1 -1
- package/dist/{renderer-0dD1VQsz.mjs → renderer-B5PFBOjv.mjs} +3 -3
- package/dist/{renderer-0dD1VQsz.mjs.map → renderer-B5PFBOjv.mjs.map} +1 -1
- package/dist/renderer-B6iUuWcx.mjs +6 -0
- package/dist/renderer-B8IdU-7S.mjs +5 -0
- package/dist/renderer-BEGJCxuL.mjs +5 -0
- package/dist/{renderer-BlTPu9QT.mjs → renderer-BEzUirF0.mjs} +2 -2
- package/dist/{renderer-BlTPu9QT.mjs.map → renderer-BEzUirF0.mjs.map} +1 -1
- package/dist/{renderer-ByGPFIu2.mjs → renderer-BKd2nRnO.mjs} +2 -2
- package/dist/{renderer-ByGPFIu2.mjs.map → renderer-BKd2nRnO.mjs.map} +1 -1
- package/dist/renderer-BLAkNFx7.mjs +5 -0
- package/dist/{renderer--2bwnzBi.mjs → renderer-BN0CzoG_.mjs} +2 -2
- package/dist/{renderer--2bwnzBi.mjs.map → renderer-BN0CzoG_.mjs.map} +1 -1
- package/dist/{renderer-aw8EsrS9.mjs → renderer-BNcsS03m.mjs} +2 -2
- package/dist/{renderer-aw8EsrS9.mjs.map → renderer-BNcsS03m.mjs.map} +1 -1
- package/dist/{renderer-Bs_f_App.d.mts → renderer-BO-7bYa5.d.mts} +2 -2
- package/dist/{renderer-Bs_f_App.d.mts.map → renderer-BO-7bYa5.d.mts.map} +1 -1
- package/dist/{renderer--JBbEfTc.mjs → renderer-BPMywsr4.mjs} +3 -3
- package/dist/{renderer--JBbEfTc.mjs.map → renderer-BPMywsr4.mjs.map} +1 -1
- package/dist/renderer-BVNhv2ft.mjs +5 -0
- package/dist/renderer-BaXTEIe0.mjs +5 -0
- package/dist/{renderer-BeuG4dyz.mjs → renderer-Bfrwo_It.mjs} +2 -2
- package/dist/{renderer-BeuG4dyz.mjs.map → renderer-Bfrwo_It.mjs.map} +1 -1
- package/dist/{renderer-DT1bySqu.mjs → renderer-BhxQzrHF.mjs} +2 -2
- package/dist/{renderer-DT1bySqu.mjs.map → renderer-BhxQzrHF.mjs.map} +1 -1
- package/dist/renderer-BjeA7s36.mjs +5 -0
- package/dist/renderer-BtGNxEay.mjs +5 -0
- package/dist/renderer-BvZg2RJG.mjs +5 -0
- package/dist/{renderer-BaYEXoS3.mjs → renderer-C2eBC0FB.mjs} +3 -3
- package/dist/{renderer-BaYEXoS3.mjs.map → renderer-C2eBC0FB.mjs.map} +1 -1
- package/dist/{renderer-B7jZuO6k.mjs → renderer-C4UqxwcT.mjs} +2 -2
- package/dist/{renderer-B7jZuO6k.mjs.map → renderer-C4UqxwcT.mjs.map} +1 -1
- package/dist/{renderer-Bm0W9qgc.mjs → renderer-C6u-XrH-.mjs} +3 -3
- package/dist/{renderer-Bm0W9qgc.mjs.map → renderer-C6u-XrH-.mjs.map} +1 -1
- package/dist/{renderer-BNyDJZzl.mjs → renderer-CB87_0vL.mjs} +3 -3
- package/dist/{renderer-BNyDJZzl.mjs.map → renderer-CB87_0vL.mjs.map} +1 -1
- package/dist/{renderer-FSCTtHrt.mjs → renderer-CD38wGmN.mjs} +2 -2
- package/dist/{renderer-FSCTtHrt.mjs.map → renderer-CD38wGmN.mjs.map} +1 -1
- package/dist/{renderer-ClIXn9ZU.mjs → renderer-CGZqCmf2.mjs} +2 -2
- package/dist/{renderer-ClIXn9ZU.mjs.map → renderer-CGZqCmf2.mjs.map} +1 -1
- package/dist/{renderer-CVD0oPsf.mjs → renderer-CHkqgIOL.mjs} +2 -2
- package/dist/{renderer-CVD0oPsf.mjs.map → renderer-CHkqgIOL.mjs.map} +1 -1
- package/dist/{renderer-Dj_K1f1T.mjs → renderer-CK0u3Psf.mjs} +3 -3
- package/dist/{renderer-Dj_K1f1T.mjs.map → renderer-CK0u3Psf.mjs.map} +1 -1
- package/dist/renderer-CLW6tUUn.mjs +5 -0
- package/dist/{renderer-BJ_GtjUk.mjs → renderer-CO4theVJ.mjs} +2 -2
- package/dist/{renderer-KcV0gCO5.mjs → renderer-CPW9o3Lc.mjs} +3 -3
- package/dist/{renderer-KcV0gCO5.mjs.map → renderer-CPW9o3Lc.mjs.map} +1 -1
- package/dist/renderer-CS-ojgJk.mjs +5 -0
- package/dist/{renderer-BwBc5xNu.mjs → renderer-CSfLMXN8.mjs} +3 -3
- package/dist/{renderer-BwBc5xNu.mjs.map → renderer-CSfLMXN8.mjs.map} +1 -1
- package/dist/{renderer-0fRv8mc5.mjs → renderer-CSoyJ8GH.mjs} +2 -2
- package/dist/{renderer-0fRv8mc5.mjs.map → renderer-CSoyJ8GH.mjs.map} +1 -1
- package/dist/renderer-CUQtKECw.mjs +5 -0
- package/dist/renderer-CV-l8wMq.mjs +5 -0
- package/dist/renderer-CV7_ALA5.mjs +5 -0
- package/dist/renderer-CVR6T6Lp.mjs +5 -0
- package/dist/{renderer-B-QXAPY0.mjs → renderer-CZ9gHx-4.mjs} +2 -2
- package/dist/{renderer-B-QXAPY0.mjs.map → renderer-CZ9gHx-4.mjs.map} +1 -1
- package/dist/{renderer-C-BjN5HM.mjs → renderer-CciB_sav.mjs} +3 -3
- package/dist/{renderer-C-BjN5HM.mjs.map → renderer-CciB_sav.mjs.map} +1 -1
- package/dist/{renderer-B6x_Mmpf.mjs → renderer-CfR0GeQL.mjs} +246 -62
- package/dist/renderer-CfR0GeQL.mjs.map +1 -0
- package/dist/{renderer-hIbXvzPu.mjs → renderer-ClWgdN2v.mjs} +2 -2
- package/dist/renderer-CtUF95UW.mjs +5 -0
- package/dist/{renderer-BJD6Ouon.mjs → renderer-Cv1wmjhA.mjs} +2 -2
- package/dist/{renderer-BJD6Ouon.mjs.map → renderer-Cv1wmjhA.mjs.map} +1 -1
- package/dist/{renderer-CJnIeI0m.mjs → renderer-Cxh7j423.mjs} +3 -3
- package/dist/{renderer-CJnIeI0m.mjs.map → renderer-Cxh7j423.mjs.map} +1 -1
- package/dist/{renderer-cjiONTY_.mjs → renderer-D0enmulV.mjs} +2 -2
- package/dist/{renderer-cjiONTY_.mjs.map → renderer-D0enmulV.mjs.map} +1 -1
- package/dist/renderer-D8ymQoTB.mjs +5 -0
- package/dist/renderer-D9x3UE1Y.mjs +5 -0
- package/dist/renderer-DAEXqa2Q.mjs +5 -0
- package/dist/renderer-DDhU9Goq.mjs +5 -0
- package/dist/{renderer-CgFs_3gW.mjs → renderer-DEQ09sg7.mjs} +2 -2
- package/dist/{renderer-CgFs_3gW.mjs.map → renderer-DEQ09sg7.mjs.map} +1 -1
- package/dist/{renderer-CATBdL9s.mjs → renderer-DRpIQFAK.mjs} +2 -2
- package/dist/{renderer-CATBdL9s.mjs.map → renderer-DRpIQFAK.mjs.map} +1 -1
- package/dist/{renderer-Dd9NUd_I.mjs → renderer-DT3QVVht.mjs} +2 -2
- package/dist/{renderer-Dd9NUd_I.mjs.map → renderer-DT3QVVht.mjs.map} +1 -1
- package/dist/{renderer-BqCx28Zb.mjs → renderer-DU3216cx.mjs} +3 -3
- package/dist/{renderer-BqCx28Zb.mjs.map → renderer-DU3216cx.mjs.map} +1 -1
- package/dist/{renderer-BXZo84vM.mjs → renderer-DU6i7shO.mjs} +2 -2
- package/dist/{renderer-BXZo84vM.mjs.map → renderer-DU6i7shO.mjs.map} +1 -1
- package/dist/renderer-DcCKath2.mjs +5 -0
- package/dist/{renderer-CmEvkdNL.mjs → renderer-DfSRmz7q.mjs} +3 -3
- package/dist/{renderer-CmEvkdNL.mjs.map → renderer-DfSRmz7q.mjs.map} +1 -1
- package/dist/renderer-DiBphLzn.mjs +5 -0
- package/dist/renderer-Diq9Z7gZ.mjs +5 -0
- package/dist/renderer-DjZiO7iA.mjs +5 -0
- package/dist/{renderer-Ba7Rc8DE.mjs → renderer-DkGODUgE.mjs} +3 -3
- package/dist/{renderer-Ba7Rc8DE.mjs.map → renderer-DkGODUgE.mjs.map} +1 -1
- package/dist/{renderer-DKmV8F6L.mjs → renderer-DlqDnh-8.mjs} +2 -2
- package/dist/{renderer-DKmV8F6L.mjs.map → renderer-DlqDnh-8.mjs.map} +1 -1
- package/dist/{renderer-CMOAtGdC.mjs → renderer-Dnm_SVy6.mjs} +3 -3
- package/dist/{renderer-CMOAtGdC.mjs.map → renderer-Dnm_SVy6.mjs.map} +1 -1
- package/dist/{renderer-Cun5J3z9.mjs → renderer-DppnY7RT.mjs} +3 -3
- package/dist/{renderer-Cun5J3z9.mjs.map → renderer-DppnY7RT.mjs.map} +1 -1
- package/dist/{renderer-CmpV4YNd.mjs → renderer-DqhZTIiC.mjs} +2 -2
- package/dist/{renderer-CmpV4YNd.mjs.map → renderer-DqhZTIiC.mjs.map} +1 -1
- package/dist/renderer-Dtj6o-74.mjs +5 -0
- package/dist/renderer-Duaam6bJ.mjs +5 -0
- package/dist/{renderer-CxE5__5p.mjs → renderer-DwCg5pXn.mjs} +2 -2
- package/dist/{renderer-CxE5__5p.mjs.map → renderer-DwCg5pXn.mjs.map} +1 -1
- package/dist/{renderer-B8Wjov5-.mjs → renderer-Dyh8c41v.mjs} +2 -2
- package/dist/renderer-FHqZe3K1.mjs +5 -0
- package/dist/renderer-FTKfOejk.mjs +5 -0
- package/dist/{renderer-BPDita2c.mjs → renderer-FgcNOe1R.mjs} +5 -5
- package/dist/{renderer-BPDita2c.mjs.map → renderer-FgcNOe1R.mjs.map} +1 -1
- package/dist/{renderer-58bVJfZx.mjs → renderer-L7UTxwZT.mjs} +2 -2
- package/dist/{renderer-58bVJfZx.mjs.map → renderer-L7UTxwZT.mjs.map} +1 -1
- package/dist/renderer-LmbkN82X.mjs +5 -0
- package/dist/{renderer-C-AzyBir.mjs → renderer-PbJk25JW.mjs} +3 -3
- package/dist/{renderer-C-AzyBir.mjs.map → renderer-PbJk25JW.mjs.map} +1 -1
- package/dist/renderer-SDspl6p-.mjs +5 -0
- package/dist/renderer-SWVUDypd.mjs +5 -0
- package/dist/renderer-WhpTBwDM.mjs +5 -0
- package/dist/renderer-WrpQEVdv.mjs +5 -0
- package/dist/{renderer-Br1OuXr_.mjs → renderer-dj3f8cMg.mjs} +3 -3
- package/dist/{renderer-Br1OuXr_.mjs.map → renderer-dj3f8cMg.mjs.map} +1 -1
- package/dist/{renderer-BcFdZx4j.mjs → renderer-dwTzrQ2G.mjs} +3 -3
- package/dist/{renderer-BcFdZx4j.mjs.map → renderer-dwTzrQ2G.mjs.map} +1 -1
- package/dist/{renderer-Cd5BLFnS.mjs → renderer-egZPy7vq.mjs} +2 -2
- package/dist/{renderer-Cd5BLFnS.mjs.map → renderer-egZPy7vq.mjs.map} +1 -1
- package/dist/{renderer-ChO8NNw9.mjs → renderer-gwV6hyYm.mjs} +2 -2
- package/dist/{renderer-ChO8NNw9.mjs.map → renderer-gwV6hyYm.mjs.map} +1 -1
- package/dist/{renderer-pNxZ_lV2.mjs → renderer-lXmtdrdI.mjs} +2 -2
- package/dist/{renderer-pNxZ_lV2.mjs.map → renderer-lXmtdrdI.mjs.map} +1 -1
- package/dist/{renderer-D_2Y_tdA.mjs → renderer-n2kYIGtb.mjs} +2 -2
- package/dist/{renderer-D_2Y_tdA.mjs.map → renderer-n2kYIGtb.mjs.map} +1 -1
- package/dist/renderer-r4ney0FZ.mjs +5 -0
- package/dist/{renderer-Bos8nRhT.mjs → renderer-sHDPBywo.mjs} +2 -2
- package/dist/{renderer-Bos8nRhT.mjs.map → renderer-sHDPBywo.mjs.map} +1 -1
- package/dist/{renderer-D9mf-msP.mjs → renderer-scbJx-Dx.mjs} +2 -2
- package/dist/{renderers-CP8CFWDs-ZKjdDfYg.mjs → renderers-CP8CFWDs-B8HIr3kj.mjs} +2 -2
- package/dist/{renderers-CP8CFWDs-ZKjdDfYg.mjs.map → renderers-CP8CFWDs-B8HIr3kj.mjs.map} +1 -1
- package/dist/signals/index.d.mts +5 -5
- package/dist/signals/index.mjs +4 -4
- package/package.json +55 -55
- package/dist/dist-Cog5jyVI.mjs.map +0 -1
- package/dist/index-Cpkzy5XR.d.mts.map +0 -1
- package/dist/index-DpWia6iH.d.mts.map +0 -1
- package/dist/preview-engine/chunk-CYY366IE.mjs +0 -1
- package/dist/preview-engine/chunk-NI577OMQ.mjs +0 -1
- package/dist/renderer-3MuT2oZO.mjs +0 -5
- package/dist/renderer-AX_fHrXM.mjs +0 -5
- package/dist/renderer-B2-LQreV.mjs +0 -5
- package/dist/renderer-B2QhkksW.mjs +0 -5
- package/dist/renderer-B5CoE2Cv.mjs +0 -5
- package/dist/renderer-B6Au6f82.mjs +0 -5
- package/dist/renderer-B6qAMLXh.mjs +0 -5
- package/dist/renderer-B6x_Mmpf.mjs.map +0 -1
- package/dist/renderer-BEfvyxL_.mjs +0 -5
- package/dist/renderer-BH4j4urG.mjs +0 -5
- package/dist/renderer-BH4zfzus.mjs +0 -5
- package/dist/renderer-BRjW_pgJ.mjs +0 -6
- package/dist/renderer-BVo6W_BO.mjs +0 -5
- package/dist/renderer-BbxBM7Vh.mjs +0 -5
- package/dist/renderer-C07CnYVd.mjs +0 -5
- package/dist/renderer-C2Z3vJCu.mjs +0 -5
- package/dist/renderer-C31qkFvm.mjs +0 -5
- package/dist/renderer-C3FV8cYm.mjs +0 -5
- package/dist/renderer-CQXszrAF.mjs +0 -5
- package/dist/renderer-CWK41ajp.mjs +0 -5
- package/dist/renderer-C_U225ub.mjs +0 -5
- package/dist/renderer-C_nPzKwZ.mjs +0 -5
- package/dist/renderer-Cd6I_dsC.mjs +0 -5
- package/dist/renderer-Ceb06HuN.mjs +0 -5
- package/dist/renderer-D3P5SLcx.mjs +0 -5
- package/dist/renderer-D9R8QXS7.mjs +0 -5
- package/dist/renderer-D9SJ4Fg9.mjs +0 -5
- package/dist/renderer-DGAypVNj.mjs +0 -5
- package/dist/renderer-DLAAfwg0.mjs +0 -5
- package/dist/renderer-DTkBPkBu.mjs +0 -5
- package/dist/renderer-DbaTMTuo.mjs +0 -5
- package/dist/renderer-DsxhQdCp.mjs +0 -5
- package/dist/renderer-Hf1pCqoN.mjs +0 -5
- package/dist/renderer-V0UuIniZ.mjs +0 -5
- package/dist/renderer-VCnMXWqE.mjs +0 -5
- package/dist/renderer-lifu3xIt.mjs +0 -5
- package/dist/renderer-mPDgLBi4.mjs +0 -5
package/README.md
CHANGED
|
@@ -4,30 +4,31 @@
|
|
|
4
4
|
|
|
5
5
|
<img src="https://raw.githubusercontent.com/gatewai-dev/framefields/main/assets/brand/banner.png" alt="framefields — code-first video, rendered natively on WebGPU" width="100%">
|
|
6
6
|
|
|
7
|
-
|
|
7
|
+
### Compositing, motion graphics, and 3D for code-first video — one npm package that AI agents drive with code.
|
|
8
8
|
|
|
9
|
-
[**framefields.dev**](https://framefields.dev/)
|
|
9
|
+
[**Documentation**](https://framefields.dev/docs)  ·  [**Website**](https://framefields.dev)  ·  [**Discord**](https://discord.gg/cbqMGGme5)  ·  [**Examples**](https://github.com/gatewai-dev/framefields/blob/main/examples)
|
|
10
10
|
|
|
11
11
|
[](https://www.npmjs.com/package/framefields)
|
|
12
|
+
[](https://framefields.dev/docs)
|
|
12
13
|
[](https://github.com/gatewai-dev/framefields/blob/main/LICENCE)
|
|
13
14
|
[](#)
|
|
14
15
|
[](https://discord.gg/cbqMGGme5)
|
|
15
16
|
[](https://www.youtube.com/@gatewai.studio)
|
|
16
17
|
[](https://nodejs.org)
|
|
17
18
|
[](https://www.w3.org/TR/webgpu/)
|
|
18
|
-
[](#
|
|
19
|
-
[](#6-on-device-vision--tracking)
|
|
19
|
+
[](#)
|
|
20
20
|
|
|
21
|
-
|
|
21
|
+
</div>
|
|
22
22
|
|
|
23
|
-
|
|
24
|
-
Renders directly on GPU hardware via Dawn / WebGPU / Metal / Vulkan in Node.js and modern WebGPU browsers.
|
|
23
|
+
Framefields is built for coding agents. It packs the capabilities usually split across three desktop creative suites—Photoshop-style compositing & VFX, After Effects-style typography & motion graphics, and Blender-style 3D scenes & cameras—into a single lightweight npm package.
|
|
25
24
|
|
|
26
|
-
|
|
25
|
+
- **Zero Headless-Browser Overhead** — No Chromium, no Puppeteer, no DOM reflows, and no screenshot pipeline. Renders directly on native GPU hardware via Dawn / WebGPU / Metal / Vulkan.
|
|
26
|
+
- **Deterministic & Frame-Accurate** — Discrete sample points, absolute frame clocks, and frame-accurate audio BeatGrids. No floating timers, no drift.
|
|
27
|
+
- **Agent-First Workflow** — Your coding agent writes TypeScript, inspects rendered frame grids, and exports hardware-encoded MP4s headlessly.
|
|
27
28
|
|
|
28
29
|
### Made with framefields
|
|
29
30
|
|
|
30
|
-
Every frame
|
|
31
|
+
Every frame is rendered by framefields directly from TypeScript in [`examples/`](https://github.com/gatewai-dev/framefields/blob/main/examples). Click a still to watch on YouTube:
|
|
31
32
|
|
|
32
33
|
<table>
|
|
33
34
|
<tr>
|
|
@@ -56,287 +57,40 @@ Every frame of these films is rendered by framefields from TypeScript in [`examp
|
|
|
56
57
|
</tr>
|
|
57
58
|
</table>
|
|
58
59
|
|
|
59
|
-
> [!NOTE]
|
|
60
|
-
> **Using an AI coding agent?** Install the framefields skills in one line.
|
|
61
|
-
>
|
|
62
|
-
> **Note:** Framefields plugin review in claude takes a bit long... Please use the alternative skills installation for now.
|
|
63
|
-
>
|
|
64
|
-
> **Claude Code**
|
|
65
|
-
> ```text
|
|
66
|
-
> /plugin install framefields
|
|
67
|
-
> ```
|
|
68
|
-
>
|
|
69
|
-
> **Any other agent** (Codex, Cursor, Hermes, Gemini CLI, Copilot, and more)
|
|
70
|
-
> ```bash
|
|
71
|
-
> npx skills add gatewai-dev/framefields
|
|
72
|
-
> ```
|
|
73
|
-
>
|
|
74
|
-
> See [Agent Skills & Plugins](#agent-skills--plugins) for details.
|
|
75
|
-
|
|
76
|
-
---
|
|
77
|
-
|
|
78
|
-
## Table of Contents
|
|
79
|
-
|
|
80
|
-
- [Why Framefields](#why-framefields)
|
|
81
|
-
- [Key Features & Capabilities](#key-features--capabilities)
|
|
82
|
-
- [1. Slug GPU Vector Typography & After Effects Animators](#1-slug-gpu-vector-typography--after-effects-animators)
|
|
83
|
-
- [2. Photoshop-Inspired WebGPU 2D VFX](#2-photoshop-inspired-webgpu-2d-vfx-50-shaders)
|
|
84
|
-
- [3. Unified 3D Scene Graph, Camera & Mesh Shading](#3-unified-3d-scene-graph-camera--mesh-shading)
|
|
85
|
-
- [4. Audio Layers, Procedural SFX & Reactive Signals](#4-audio-layers-procedural-sfx--reactive-signals)
|
|
86
|
-
- [5. Animated Charts](#5-animated-charts)
|
|
87
|
-
- [6. On-Device Vision & Tracking](#6-on-device-vision--tracking)
|
|
88
|
-
- [7. Headless Conformance & FrameGrid Testing](#7-headless-conformance--framegrid-testing)
|
|
89
|
-
- [8. Live Preview in the Browser](#8-live-preview-in-the-browser)
|
|
90
|
-
- [Monorepo Architecture](#monorepo-architecture)
|
|
91
|
-
- [Quickstart Guide](#quickstart-guide)
|
|
92
|
-
- [1. Basic Composition & Kinetic Auto-Layout](#1-basic-composition--kinetic-auto-layout)
|
|
93
|
-
- [2. Unified 3D Scene with Camera & 3D Model](#2-unified-3d-scene-with-camera--3d-model)
|
|
94
|
-
- [3. Audio Soundtrack, Procedural SFX & Reactive Signals](#3-audio-soundtrack-procedural-sfx--reactive-signals)
|
|
95
|
-
- [4. Chained WebGPU Post-Processing VFX](#4-chained-webgpu-post-processing-vfx)
|
|
96
|
-
- [5. Vision: Pin, Matte & Reframe](#5-vision-pin-matte--reframe)
|
|
97
|
-
- [6. Headless Video & FrameGrid Rendering](#6-headless-video--framegrid-rendering)
|
|
98
|
-
- [Engineering Doctrines & Best Practices](#engineering-doctrines--best-practices)
|
|
99
|
-
- [Agent Skills & Plugins](#agent-skills--plugins)
|
|
100
|
-
- [Reference Showcase Examples](#reference-showcase-examples)
|
|
101
|
-
- [Development & Building](#development--building)
|
|
102
|
-
- [Community](#community)
|
|
103
|
-
- [License](#license)
|
|
104
|
-
|
|
105
60
|
---
|
|
106
61
|
|
|
107
|
-
##
|
|
108
|
-
|
|
109
|
-
Modern automated video generation is usually constrained by the architectures of general-purpose web browsers: process overhead, non-deterministic DOM layout reflows, and slow screenshot capture. Framefields treats **video composition as software engineering**:
|
|
110
|
-
|
|
111
|
-
| | Pillar | What it means |
|
|
112
|
-
|---|---|---|
|
|
113
|
-
| 🚀 | **Zero Headless-Browser Overhead** | No Puppeteer, no Chromium IPC, no `page.screenshot()`. Framefields talks straight to native GPU devices via Dawn/WebGPU and hardware-encodes with `@napi-rs/webcodecs`. |
|
|
114
|
-
| 🎯 | **Deterministic Frame-Accurate Clock** | Absolute frame clocks, discrete sample points, and frame-accurate audio BeatGrids. No floating timers, no drift, no dropped frames. |
|
|
115
|
-
| 🔠 | **Analytic, Resolution-Independent Type** | The Slug algorithm evaluates glyph contours per-pixel in WGSL — no texture atlases, no scaling artifacts, razor-sharp from 10 px to 10,000 px. |
|
|
116
|
-
| 🎨 | **Photoshop-Inspired Tonal & Spatial VFX** | 50+ modular GPU shaders: Curves, Levels, Selective Color, 3D LUTs, Halftone, Film Grain, Unsharp Mask, Mesh Warp, and Screen-Space Relighting. |
|
|
117
|
-
| 🧊 | **Unified 3D & 2D Depth Compositing** | Nest 2D flex/box trees inside 3D homography planes, multiplane rigs, and meshes (OBJ, FBX, glTF/GLB, STL, PLY, VOX, 3DS, OFF), with GGX PBR, image-based light, shadows, bloom and SSAO. |
|
|
118
|
-
| 🔊 | **Built-in Procedural Audio DSP** | Multi-track soundtracks, deterministic procedural transition SFX (whoosh, impact, riser), and reactive signals that drive visuals from audio. |
|
|
119
|
-
| 👁️ | **On-Device Neural Vision** | Object tracking, instance segmentation, multi-person pose, and person mattes from Apache-2.0 ONNX models — feeding reactive signals without a round trip to disk. |
|
|
120
|
-
| ☁️ | **Cloud-Native & CI/CD Ready** | ~200–400 MB RAM per worker (vs. 2–4 GB for Chromium), ideal for serverless GPU render clusters (AWS G4/G5, Modal, RunPod, Kubernetes). |
|
|
121
|
-
|
|
122
|
-
---
|
|
62
|
+
## Agent Skills
|
|
123
63
|
|
|
124
|
-
|
|
125
|
-
|
|
126
|
-
### 1. Slug GPU Vector Typography & After Effects Animators
|
|
127
|
-
Traditional text relies on CPU rasterization or low-res SDF atlases that soften under 3D camera sweeps. Framefields integrates the **Slug algorithm** ([`SlugPipeline`](https://github.com/gatewai-dev/framefields/blob/main/packages/webgpu-renderers/src/slug/slug-pipeline.ts)):
|
|
128
|
-
- **Analytic GPU evaluation** — WGSL fragment shaders solve exact cubic/quadratic Béziers per-pixel. Glyphs stay sharp at 10 px or 10,000 px with zero CPU re-rasterization.
|
|
129
|
-
- **After Effects–style animators** — range selectors (`square`, `ramp_up`, `ramp_down`, `triangle`, `smooth`), `easeHigh`/`easeLow` curves, and seeded PRNG character shuffling ([`TextAnimator`](https://github.com/gatewai-dev/framefields/blob/main/packages/framefields/src/index.ts)).
|
|
130
|
-
- **Human typing cadence** — weighted punctuation delays (commas 3×, sentence ends 5.5×, newlines 7×) and trailing scramble resolution ([`TypewriterAnimator`](https://github.com/gatewai-dev/framefields/blob/main/packages/framefields/src/index.ts)).
|
|
131
|
-
- **3D volumetric formations** — map text onto cylindrical drums, logarithmic vortex spirals, and double-helix ribbons with surface-normal banking ([`evaluateVolumetricFormation`](https://github.com/gatewai-dev/framefields/blob/main/packages/framefields/src/index.ts)).
|
|
132
|
-
- **Dynamic leading & skew** — area-preserving unimodular shear and accordion line-leading anchored to baseline, center, or top.
|
|
133
|
-
|
|
134
|
-
### 2. Photoshop-Inspired WebGPU 2D VFX (50+ Shaders)
|
|
135
|
-
A comprehensive suite of professional image/video shader nodes in [`nodes/`](https://github.com/gatewai-dev/framefields/blob/main/nodes) and [`packages/webgpu-renderers`](https://github.com/gatewai-dev/framefields/blob/main/packages/webgpu-renderers):
|
|
136
|
-
- **Tonal grading** — Curves (RGB/R/G/B spline), Levels (black/white point, gamma, output), Shadows/Highlights, Selective Color (CMYK gamut isolation), 3D Cube LUT ([`ApplyLUT`](https://github.com/gatewai-dev/framefields/blob/main/packages/framefields/src/effects)).
|
|
137
|
-
- **Stylization & grain** — Film Grain (Gaussian emulsion with spatial seed variation), Halftone (mono/RGB/CMYK, adjustable dot shape & angle), Gradient Map, High Pass.
|
|
138
|
-
- **Optics & lens** — Bilateral Gaussian Blur, Unsharp Mask, Vignette, Refraction Caustics, PBR Glassmorphism with chromatic dispersion ([`PBRGlass`](https://github.com/gatewai-dev/framefields/blob/main/packages/framefields/src/effects)).
|
|
139
|
-
- **Distortion & warping** — Displacement Maps, Liquify, Mesh Warp, Corner Pin homography.
|
|
140
|
-
- **Gradients, mattes & transitions** — CSS `linear-` / `radial-` / `conic-gradient()` strings as any background or fill (blended in OKLab), track mattes (`node.withMatte(layer, "alpha" | "luma" …)`) for wipes, reveals and video-in-type, and [`Layer.sequence`](https://github.com/gatewai-dev/framefields/blob/main/packages/framefields/src/transitions.ts) with `Transition.push` / `wipe` / `iris` / `crossfade` / `zoom` / `whip` between scenes.
|
|
141
|
-
|
|
142
|
-
### 3. Unified 3D Scene Graph, Camera & Mesh Shading
|
|
143
|
-
- **Calibrated camera rig** — LookAt and Turntable cameras ([`Camera3D`](https://github.com/gatewai-dev/framefields/blob/main/packages/webgpu-renderers/src/math3d/camera3d.ts)) calibrated so `z = 0` matches 2D canvas pixel coordinates 1:1.
|
|
144
|
-
- **3D layout primitives** — [`Layer3D.cube`](https://github.com/gatewai-dev/framefields/blob/main/packages/framefields/src/shapes3d.ts), `carousel`, `prism`, `plane`, `grid` with unified depth-buffer testing, plus solid smooth-shaded meshes: `roundedBox` (with `taper`, for keycaps and device bodies), `sphere` and `cylinder`.
|
|
145
|
-
- **Zero-dependency model parsers** — OBJ, FBX, glTF/GLB, STL, PLY, VOX, 3DS, OFF.
|
|
146
|
-
- **Physically based look** — add [`Layer.environment("studio")`](https://github.com/gatewai-dev/framefields/blob/main/packages/webgpu-renderers/src/math3d/environment3d.ts) and the scene shades in linear light with GGX `roughness` / `metallic`, a procedural studio environment for image-based light and reflections, `emissive` surfaces, exposure, and Khronos PBR Neutral / AgX / ACES tone mapping. Scenes without one keep the classic Blinn-Phong look.
|
|
147
|
-
- **Shadows, bloom & fog** — soft PCF shadow maps from a `castShadow` directional or spot light, HDR bloom that only takes light brighter than white, and distance fog that fades into the background color. [`Lighting.studio()`](https://github.com/gatewai-dev/framefields/blob/main/packages/framefields/src/lighting.ts) / `soft` / `sunset` / `dramatic` rigs set all of it up in one call.
|
|
148
|
-
- **Skeletal animation & materials** — 128-bone Linear Blend Skinning; glTF base color, normal, metallic-roughness, occlusion and emissive maps; SSAO and optical DoF.
|
|
149
|
-
- **Physical motion blur** — 180° shutter motion blur with per-vertex velocity vectors packed into `rg16float` MRT buffers.
|
|
150
|
-
|
|
151
|
-
### 4. Audio Layers, Procedural SFX & Reactive Signals
|
|
152
|
-
- **Soundtrack layers** — `.audio` media nodes with frame-exact lifecycle control.
|
|
153
|
-
- **Procedural SFX** — deterministic CPU-synthesized whooshes, impacts, risers, downshifters, and glitches placed on the bar/beat grid (`renderSfx`, `mixSfxInto`, `softLimit`).
|
|
154
|
-
- **Multi-track mixing** — master tracks headlessly with `mixAudioTracks` and `encodeStereoWav`.
|
|
155
|
-
- **Reactive signals** — drive transforms, scale, borders, or shader uniforms from tempo signals (`Signal.builder`) or audio analysis.
|
|
156
|
-
|
|
157
|
-
### 5. Animated Charts
|
|
158
|
-
[`Layer.chart`](https://github.com/gatewai-dev/framefields/blob/main/packages/framefields/src/chart.ts) builds line, area, bar (grouped or stacked), scatter, candlestick, pie and donut charts. [d3](https://d3js.org) computes the scales, ticks and geometry; every bar, line, slice and label is an ordinary box, path or text node:
|
|
159
|
-
- Labels use the composition's registered fonts and the same GPU text renderer as the rest of the film.
|
|
160
|
-
- A built-in reveal draws lines on, grows bars from the baseline and staggers points and slices (`animate: { start, duration, stagger, ease }`, or `animate: false`).
|
|
161
|
-
- The chart is one box, so it positions, animates, grades and tilts into 3D like any other layer.
|
|
64
|
+
Teach Claude Code, Cursor, Codex, Copilot, or Hermes how to author, animate, and render compositions:
|
|
162
65
|
|
|
163
|
-
|
|
164
|
-
|
|
165
|
-
|
|
166
|
-
type: "bar",
|
|
167
|
-
width: 900,
|
|
168
|
-
height: 480,
|
|
169
|
-
categories: ["Q1", "Q2", "Q3", "Q4"],
|
|
170
|
-
series: [
|
|
171
|
-
{ name: "Revenue", data: [12, 19, 24, 31] },
|
|
172
|
-
{ name: "Costs", data: [8, 11, 13, 15] },
|
|
173
|
-
],
|
|
174
|
-
yAxis: { format: "$,.0f" },
|
|
175
|
-
animate: { start: 10, duration: 30 },
|
|
176
|
-
},
|
|
177
|
-
{ position: "absolute", x: 120, y: 200 },
|
|
178
|
-
);
|
|
179
|
-
```
|
|
180
|
-
|
|
181
|
-
### 6. Vector Animations & Lottie
|
|
182
|
-
[`Layer.lottie`](https://github.com/gatewai-dev/framefields/blob/main/packages/framefields/src/index.ts) plays Lottie JSON (`.json`) and dotLottie (`.lottie`) vector animations natively via ThorVG without browser overhead:
|
|
183
|
-
- **Headless vector decoding** — renders directly on WebGPU in Node.js via WASM/ThorVG.
|
|
184
|
-
- **Unified compositing** — positions with flex/box layout, animates with `LayerAnimation`, tilts into 3D scenes, masks (`.withMatte`), and applies GPU post-processing effects.
|
|
185
|
-
- **Configurable playback** — supports `fit` (`"contain"` | `"cover"` | `"fill"`), `loop`, `speed`, `autoplay`, and lifecycle frames.
|
|
186
|
-
- **Strict format boundary** — raster formats (`.png`, `.jpg`, `.webp`) use `Layer.image()`; passing `.json` or `.lottie` to `Layer.image()` throws/warns and is flagged by `comp.lint()`.
|
|
187
|
-
|
|
188
|
-
```typescript
|
|
189
|
-
comp.add(
|
|
190
|
-
Layer.lottie("assets/animations/badge.json", {
|
|
191
|
-
width: 400,
|
|
192
|
-
height: 400,
|
|
193
|
-
fit: "contain",
|
|
194
|
-
loop: true,
|
|
195
|
-
})
|
|
196
|
-
);
|
|
197
|
-
```
|
|
198
|
-
|
|
199
|
-
### 7. On-Device Vision & Tracking
|
|
200
|
-
[`@framefields/vision`](https://github.com/gatewai-dev/framefields/blob/main/packages/vision) runs ONNX models via `onnxruntime-node` (CPU) or `onnxruntime-web` (WebGPU) and wires every result into the same reactive signal surface the rest of Framefields consumes.
|
|
201
|
-
|
|
202
|
-
> [!TIP]
|
|
203
|
-
> **Lazy by construction.** `VisionRunner.create()`, `comp.withVision(...)` and `VisionNode.attach(...)` perform **zero I/O** — no downloads, no sessions, no file probes. A model is fetched the first time a task actually runs. To warm up ahead of time, call `await runner.preload(["detect", "pose"])` (or `await vision.ready()` on an attached node).
|
|
204
|
-
|
|
205
|
-
#### Tasks and models
|
|
206
|
-
|
|
207
|
-
Every model is **Apache-2.0**, pinned to an immutable Hugging Face revision, and verified by SHA-256 after download.
|
|
208
|
-
|
|
209
|
-
| Task | Option | Model | Output |
|
|
210
|
-
|---|---|---|---|
|
|
211
|
-
| **Detect** | `enableDetection` | RTMDet-Ins `t/s/m` (OpenMMLab) | COCO-80 boxes + scores, tracked over time |
|
|
212
|
-
| **Segment** | `enableSegmentation` | RTMDet-Ins (same forward pass as detect) | Soft per-instance masks, frame-aligned |
|
|
213
|
-
| **Pose** | `enablePose` | RTMO `t/s/m` (OpenMMLab) | 17 COCO keypoints + visibility per person |
|
|
214
|
-
| **Matte** | `enableMatte` | MediaPipe Selfie Segmenter (Google) | Fast person-vs-background alpha for portrait / webcam framing |
|
|
215
|
-
|
|
216
|
-
- **Variants** — `variant: "t" | "s" | "m"` (default `"s"`; ~24 / 43 / 116 MB for RTMDet-Ins). Tune `confidence` and a COCO `classes` filter per composition. On CPU, a 2K frame takes roughly 200–340 ms to detect + segment, ~120 ms for pose and ~20 ms for the matte with `"s"`.
|
|
217
|
-
- **One pass, two tasks** — detection and segmentation share a single RTMDet-Ins inference per frame.
|
|
218
|
-
- **Picking a matte** — the Selfie Segmenter is tuned for a person filling much of the frame: it misses distant figures and can report "person" on close-ups with nobody in them. For anything else, cut out with instance masks (`matteSource: "instance"`, the default).
|
|
219
|
-
- **Whole-subject cutouts** — `mask` / `matte` / `crop` modes merge every comparably sized instance that overlaps the main subject, so a flowing dress or a held instrument stays attached to the person, while a tunnel or window framing them does not.
|
|
220
|
-
- **One-frame delay** — vision reads each layer's previous rendered frame, so results trail the plate by one frame and frame 0 has none. Verify vision layers with the exported video or consecutive frames, not frame grids.
|
|
221
|
-
- **Model cache & mirrors** — models are cached atomically (temp + rename) in `$FRAMEFIELDS_MODELS_DIR` (default `~/.cache/framefields/models`). Point `baseUrl` or `FRAMEFIELDS_MODELS_BASE_URL` at your own mirror for air-gapped or CI renders.
|
|
222
|
-
|
|
223
|
-
#### GPU helpers
|
|
224
|
-
- **OpenPose-style skeleton textures** — rasterize COCO-17 keypoints into a VRAM conditioning texture ([`PoseSkeletonRenderer`](https://github.com/gatewai-dev/framefields/blob/main/packages/vision/src/gpu/pose-skeleton-renderer.ts)).
|
|
225
|
-
- **GPU segmentation texture pool** — reusable silhouette textures ([`SegmentationTexturePool`](https://github.com/gatewai-dev/framefields/blob/main/packages/vision/src/gpu/segmentation-texture-pool.ts)).
|
|
226
|
-
|
|
227
|
-
#### Temporal tracking & analysis
|
|
228
|
-
- **Multi-object tracker** ([`TemporalObjectTracker`](https://github.com/gatewai-dev/framefields/blob/main/packages/vision/src/tracking/temporal-object-tracker.ts)) assigns stable `trackId`s via IoU association, with configurable `minHits`, `positionSmoothing`, and velocity-based **coasting** for up to `maxMissedFrames` (default 15) so a transient miss holds the track instead of flashing.
|
|
229
|
-
- **Pose↔track matching** ([`pose-track-matcher`](https://github.com/gatewai-dev/framefields/blob/main/packages/vision/src/tracking/pose-track-matcher.ts)) binds keypoints to the right track by id, then by spatial IoU fallback.
|
|
230
|
-
- **One-shot sequence analysis** — `comp.analyzeVisionSequence(src, { tasks, categories })` decodes frames through the mediabunny pipeline, tracks them, and returns a **zod-serializable** report (per-track frame ranges, mean speed, sampled center paths, per-class presence/confidence, mean mask coverage, model download bytes/timing) ([`analyzeSequence`](https://github.com/gatewai-dev/framefields/blob/main/packages/vision/src/analysis/analyze-sequence.ts)).
|
|
231
|
-
|
|
232
|
-
#### Reactive vision signals
|
|
233
|
-
Every tracked entity is exposed as reactive `ProgrammaticSignal`s that animate layers and shader uniforms:
|
|
234
|
-
|
|
235
|
-
| Group | Highlights |
|
|
236
|
-
|---|---|
|
|
237
|
-
| `objects` | `get(trackId)`, `byCategory(cat, rank)`, `primary`, `count`, `hasCategory`, `detectedCategories` |
|
|
238
|
-
| `objects.*.bounds` | `x/y/width/height`, `screenX/screenY/screenWidth/screenHeight`, `aspectRatio`, `area` |
|
|
239
|
-
| `objects.*.anchors` | 9 anchors (corners, edges, center) ready for pinning |
|
|
240
|
-
| `objects.*.kinematics` | `vx`, `vy`, `speed`, `acceleration`, `headingRad/Deg` |
|
|
241
|
-
| `objects.*.pose` | All 17 COCO keypoints, plus `hasPose`, `wristSpeed`, `handRaised`, `bodyTiltAngle` |
|
|
242
|
-
| `masks` | `get(trackId)`, `subject`, `count`; per-mask `area`, `coverage`, `solidity`, `bboxFill` |
|
|
243
|
-
| `segmentation` | `subject`, `humanSilhouette`, `instanceMasks`, `matte.coverage`, GPU `stencilTexture` |
|
|
244
|
-
| `classes` | Per-class `count`, `maxConfidence`, `present`, `primary`, plus a detection `histogram` |
|
|
245
|
-
| Tensors | `poseLandmarksTensor [17,3]`, `objectsTensor [16,8]`, `masksTensor [16,2]`, `histogramTensor [80]` |
|
|
246
|
-
|
|
247
|
-
#### Spatial pinning
|
|
248
|
-
Project normalized landmarks to screen space with a configurable camera FOV, then bind any node to a track or landmark ([`SpatialLandmarkTransformer`](https://github.com/gatewai-dev/framefields/blob/main/packages/vision/src/spatial/camera-space-transformer.ts), [`spatial-pin`](https://github.com/gatewai-dev/framefields/blob/main/packages/vision/src/spatial/spatial-pin.ts)):
|
|
249
|
-
- `pinToObject(track, { anchor, offsetX/Y/Z, matchWidth, matchHeight, smoothFrames, hideWhenLost })`
|
|
250
|
-
- `pinToLandmark(coord, { offsetX/Y/Z })`
|
|
251
|
-
|
|
252
|
-
#### High-level composition helpers
|
|
253
|
-
- **Subject Sandwich** — `comp.addSubjectSandwich({ source, behind, feather, fit })` cuts the foreground subject out and places typography/graphics behind them.
|
|
254
|
-
- **Smart Reframing** — `comp.addSmartFraming({ source, target, targetAspect, damping, leadHeadroom })` auto-crops 16:9 → 9:16 while tracking `target`.
|
|
255
|
-
- **Subject Outline** — `comp.addSubjectOutline(vision.segmentation.subject, { source, color, width, blur })` strokes the segmented boundary as an audio-reactive contour glow.
|
|
256
|
-
- **Tracked Region Blur** — `layer.blurRegion(track, { strength })` blurs faces, plates, or any detected class.
|
|
257
|
-
- **Node modes** — `passthrough`, `mask`, `matte`, `crop`, `skeleton`, `boxes`, `tracking`; pick the cutout alpha with `matteSource: "instance" | "selfie"`, and optionally `keyBackground` to grow the subject into connected foreground.
|
|
258
|
-
|
|
259
|
-
#### Agent-first DX
|
|
260
|
-
- **Runtime config is zod-validated** and available from a **zod-only entry** (`@framefields/vision/schemas`) so the hot path stays zod-free. Unknown or removed options are rejected, not silently ignored.
|
|
261
|
-
- **`vision.summary(frame)`** returns a deterministic, serializable snapshot (objects, classes, masks) safe to call inside a frame hook.
|
|
262
|
-
- **Clear failures** — a model that is the wrong size, fails its checksum, or lacks an expected output raises an error naming the model and its source.
|
|
263
|
-
- **Browser entry** — `@framefields/vision/web` re-exports the engine plus `createWebGPUProvider()` / `hasWebGPU()`; `onnxruntime-web` is an optional lazy peer.
|
|
264
|
-
|
|
265
|
-
### 8. Headless Conformance & FrameGrid Testing
|
|
266
|
-
- **Pixel-sampling invariant assertions** — test compositions in Vitest with `skia-canvas` to verify shader math, font coverage, and Mean Squared Error (MSE) temporal deltas.
|
|
267
|
-
- **FrameGrid contact sheets** — `comp.renderFrameGrid(...)` outputs sequential-frame contact sheets for instant review of easing, kinetic type, and transitions.
|
|
268
|
-
|
|
269
|
-
### 9. Live Preview in the Browser
|
|
270
|
-
- **Runs your composition, not a video** — `startPreview({ entry, export })` serves a localhost WebGPU player that loads the composition's own module and renders its frames live in the browser. Nothing is streamed: the server only hands over the bundle, the project's assets, and the soundtrack mixed by the export engine.
|
|
271
|
-
- **Timeline, waveform & frame stepping** — play/pause, scrub, step frame by frame, and read resolution, FPS, duration, and audio status at a glance.
|
|
272
|
-
- **One stable URL per project** — the port is derived from the working directory, so re-running the preview replaces the running server and any open tab reloads into the new version by itself. Close the tab and the server shuts down about five seconds later. A reload comes back at the same moment, and `#t=12.5` links to one.
|
|
273
|
-
- **Notes pinned on the picture** — pause, click a spot (or drag an area) and write what should change. Each note keeps its moment, its spot and a picture of the frame with the spot marked, in `.framefields/preview-notes/`, and prints where the preview runs, so an agent can apply exactly what you meant. Tick notes off as you check them.
|
|
274
|
-
- **Tells you what happened** — the page's errors (a failed build, a frame that threw, a missing font) and why the server stopped are printed where the preview runs, so whoever started it, person or agent, sees them without opening devtools. Scripts can drive the page through `window.framefieldsPreview` (`seek`, `play`, `pause`, `state`).
|
|
275
|
-
- **Shown where you are** — `startPreview` serves the page and returns its URL instead of opening a browser, so an agent can show it in its own pane (Claude Code, Codex); pass `open: true` to open the system browser.
|
|
276
|
-
- **Plays in real time, even on a slow device** — Play draws the frame due on the soundtrack's clock and skips the ones it missed, so playback starts at once and the sound never waits. A device that can't keep up shows fewer frames, and the page shows the frame rate it reaches.
|
|
277
|
-
|
|
278
|
-
```typescript
|
|
279
|
-
import { startPreview } from "framefields";
|
|
280
|
-
|
|
281
|
-
const session = await startPreview(
|
|
282
|
-
{ entry: new URL("./film.ts", import.meta.url), export: "buildFilm" },
|
|
283
|
-
{ title: "framefields film" },
|
|
284
|
-
);
|
|
285
|
-
console.log(`Preview at ${session.url}`);
|
|
286
|
-
await session.closed; // serves until its tab closes or a newer preview takes over
|
|
66
|
+
**Claude Code**
|
|
67
|
+
```text
|
|
68
|
+
/plugin install framefields
|
|
287
69
|
```
|
|
288
70
|
|
|
289
|
-
|
|
290
|
-
|
|
291
|
-
|
|
292
|
-
|
|
293
|
-
## Monorepo Architecture
|
|
294
|
-
|
|
295
|
-
Managed with `pnpm` workspaces and `turbo`:
|
|
296
|
-
|
|
297
|
-
```
|
|
298
|
-
framefields/
|
|
299
|
-
├── packages/
|
|
300
|
-
│ ├── framefields/ # Unified SDK (Composition, Layer, LayerAnimation, Signal, effects)
|
|
301
|
-
│ ├── core/ # Core AST, Effect base class, VirtualMediaData, vision types
|
|
302
|
-
│ ├── compositions/ # Layout engine, Flex/Box AST compiler, timeline evaluator
|
|
303
|
-
│ ├── webgpu-renderers/ # WGSL shaders, Slug text engine, 3D renderer, camera, lights, materials
|
|
304
|
-
│ ├── tensor-webgpu/ # WebGPU compute pipelines (Canny, depth-to-normals, flow, deflicker, landmarks)
|
|
305
|
-
│ ├── vision/ # ONNX vision engine: detect, segment, pose, matte, tracking, signals
|
|
306
|
-
│ ├── renderer/ # Headless Node.js WebGPU renderer via Dawn, WebCodecs, skia-canvas
|
|
307
|
-
│ ├── renderers/ # Higher-level render orchestration
|
|
308
|
-
│ ├── media/ # Media decoding / encoding adapters
|
|
309
|
-
│ ├── node-sdk/ # Node renderer contracts and result schemas
|
|
310
|
-
│ ├── server-utils/ # Server infrastructure, storage, asset caches
|
|
311
|
-
│ └── client-utils/ # Shared browser utilities
|
|
312
|
-
├── nodes/ # 58+ specialized domain nodes (VFX, audio, layout, node-vision)
|
|
313
|
-
├── apps/
|
|
314
|
-
│ └── renderer-service/ # Production HTTP / gRPC rendering microservice container
|
|
315
|
-
├── examples/ # Reference compositions and films
|
|
316
|
-
├── plugins/framefields/ # Agent plugin: skills only (setup, compose, effects, render)
|
|
317
|
-
└── scripts/ # Build, release, and plugin validation tooling
|
|
71
|
+
**Cursor, Codex, Hermes, Copilot & others**
|
|
72
|
+
```bash
|
|
73
|
+
npx skills add gatewai-dev/framefields
|
|
318
74
|
```
|
|
319
75
|
|
|
320
76
|
---
|
|
321
77
|
|
|
322
|
-
##
|
|
323
|
-
|
|
324
|
-
### Installation
|
|
78
|
+
## Installation
|
|
325
79
|
|
|
326
80
|
```bash
|
|
327
81
|
pnpm add framefields
|
|
328
82
|
```
|
|
329
83
|
|
|
330
|
-
> **Requirements:** Node.js ≥ 22.
|
|
84
|
+
> **Requirements:** Node.js ≥ 22. Native GPU acceleration via WebGPU / Dawn (macOS Metal, Linux Vulkan, Windows D3D12).
|
|
331
85
|
|
|
332
86
|
---
|
|
333
87
|
|
|
334
|
-
|
|
88
|
+
## Quickstart
|
|
335
89
|
|
|
336
90
|
```typescript
|
|
337
91
|
import { Composition, Layer, LayerAnimation } from "framefields";
|
|
338
92
|
|
|
339
|
-
// 1.
|
|
93
|
+
// 1. Create a 1080p60 composition
|
|
340
94
|
const comp = new Composition({
|
|
341
95
|
width: 1920,
|
|
342
96
|
height: 1080,
|
|
@@ -346,21 +100,21 @@ const comp = new Composition({
|
|
|
346
100
|
fonts: ["assets/fonts/Inter.ttf", "assets/fonts/SpaceGrotesk.ttf"],
|
|
347
101
|
});
|
|
348
102
|
|
|
349
|
-
// 2.
|
|
350
|
-
const
|
|
103
|
+
// 2. Physical snap-overshoot entrance animation
|
|
104
|
+
const entrance = LayerAnimation.create()
|
|
351
105
|
.fadeIn(0, 20, "power2.out")
|
|
352
106
|
.fromTo("y", 60, 0, { start: 0, end: 35, ease: "back.out(1.5)" })
|
|
353
107
|
.fromTo("scale", 0.92, 1.0, { start: 0, end: 35, ease: "back.out(1.2)" });
|
|
354
108
|
|
|
355
|
-
// 3. Assemble a responsive flex-layout card
|
|
109
|
+
// 3. Assemble a responsive flex-layout card with Slug vector type
|
|
356
110
|
const heroCard = Layer.box({
|
|
357
|
-
width:
|
|
111
|
+
width: 760,
|
|
358
112
|
height: 380,
|
|
359
113
|
background: "#141721",
|
|
360
114
|
borderRadius: 24,
|
|
361
115
|
borderColor: "#262b3d",
|
|
362
116
|
borderWidth: 1.5,
|
|
363
|
-
padding:
|
|
117
|
+
padding: 40,
|
|
364
118
|
children: [
|
|
365
119
|
Layer.flex({
|
|
366
120
|
dir: "column",
|
|
@@ -372,13 +126,13 @@ const heroCard = Layer.box({
|
|
|
372
126
|
fill: "#6366f1",
|
|
373
127
|
letterSpacing: 2.0,
|
|
374
128
|
}),
|
|
375
|
-
Layer.text("
|
|
129
|
+
Layer.text("Code-First Video", {
|
|
376
130
|
fontSize: 48,
|
|
377
131
|
fontWeight: 700,
|
|
378
132
|
fill: "#f8fafc",
|
|
379
133
|
fontFamily: "SpaceGrotesk",
|
|
380
134
|
}),
|
|
381
|
-
Layer.text("Direct
|
|
135
|
+
Layer.text("Direct GPU rendering without headless browser overhead.", {
|
|
382
136
|
fontSize: 20,
|
|
383
137
|
fill: "#94a3b8",
|
|
384
138
|
lineHeight: 28,
|
|
@@ -386,364 +140,40 @@ const heroCard = Layer.box({
|
|
|
386
140
|
],
|
|
387
141
|
}),
|
|
388
142
|
],
|
|
389
|
-
}).animate(
|
|
143
|
+
}).animate(entrance);
|
|
390
144
|
|
|
391
145
|
comp.add(heroCard);
|
|
392
|
-
```
|
|
393
|
-
|
|
394
|
-
---
|
|
395
|
-
|
|
396
|
-
### 2. Unified 3D Scene with Camera & 3D Model
|
|
397
|
-
|
|
398
|
-
```typescript
|
|
399
|
-
import { Composition, Layer, Layer3D, CameraAnimation, Light } from "framefields";
|
|
400
|
-
|
|
401
|
-
const comp = new Composition({ width: 1920, height: 1080, fps: 60, durationFrames: 300 });
|
|
402
146
|
|
|
403
|
-
//
|
|
404
|
-
const cameraAnim = CameraAnimation.camera().orbit({
|
|
405
|
-
azimuth: { from: -30, to: 30 },
|
|
406
|
-
elevation: { from: 15, to: 15 },
|
|
407
|
-
radius: { to: 1200 },
|
|
408
|
-
start: 0,
|
|
409
|
-
end: 300,
|
|
410
|
-
});
|
|
411
|
-
|
|
412
|
-
comp.add(
|
|
413
|
-
Layer.camera({ x: 960, y: 540, z: -1000, targetX: 960, targetY: 540, targetZ: 0 }).animate(cameraAnim)
|
|
414
|
-
);
|
|
415
|
-
|
|
416
|
-
// 2. Studio lighting
|
|
417
|
-
comp.add(Light.ambient("#ffffff", 0.4));
|
|
418
|
-
comp.add(Light.directional({ color: "#e0e7ff", intensity: 1.2, x: 500, y: -800, z: -600 }));
|
|
419
|
-
|
|
420
|
-
// 3. 3D model with skeletal animation
|
|
421
|
-
comp.add(
|
|
422
|
-
Layer.glb("assets/models/character.glb", {
|
|
423
|
-
x: 960,
|
|
424
|
-
y: 640,
|
|
425
|
-
z: 0,
|
|
426
|
-
scale: 2.5,
|
|
427
|
-
material: "lit",
|
|
428
|
-
loop: true,
|
|
429
|
-
})
|
|
430
|
-
);
|
|
431
|
-
|
|
432
|
-
// 4. 3D prism layout carousel
|
|
433
|
-
comp.add(
|
|
434
|
-
Layer3D.carousel({
|
|
435
|
-
radius: 400,
|
|
436
|
-
items: [
|
|
437
|
-
Layer.box({ width: 280, height: 180, background: "#1e293b", borderRadius: 16 }),
|
|
438
|
-
Layer.box({ width: 280, height: 180, background: "#334155", borderRadius: 16 }),
|
|
439
|
-
Layer.box({ width: 280, height: 180, background: "#0f172a", borderRadius: 16 }),
|
|
440
|
-
],
|
|
441
|
-
})
|
|
442
|
-
);
|
|
443
|
-
```
|
|
444
|
-
|
|
445
|
-
---
|
|
446
|
-
|
|
447
|
-
### 3. Audio Soundtrack, Procedural SFX & Reactive Signals
|
|
448
|
-
|
|
449
|
-
```typescript
|
|
450
|
-
import { Composition, Layer, LayerAnimation, Signal, renderSfx, mixSfxInto, softLimit } from "framefields";
|
|
451
|
-
|
|
452
|
-
const comp = new Composition({ width: 1920, height: 1080, fps: 60 });
|
|
453
|
-
const totalFrames = 240;
|
|
454
|
-
|
|
455
|
-
// 1. Soundtrack layer
|
|
456
|
-
comp.addAudio(Layer.audio("assets/score.mp3", { volume: 0.9, durationFrames: totalFrames }));
|
|
457
|
-
|
|
458
|
-
// 2. Frame-accurate procedural SFX on the beat grid
|
|
459
|
-
const bed: [Float32Array, Float32Array] = [
|
|
460
|
-
new Float32Array(Math.ceil((totalFrames / 60) * 48000)),
|
|
461
|
-
new Float32Array(Math.ceil((totalFrames / 60) * 48000)),
|
|
462
|
-
];
|
|
463
|
-
mixSfxInto(bed, [
|
|
464
|
-
renderSfx({ type: "whoosh", atBar: 0.79, volume: 0.5 }, { sampleRate: 48000, secondsPerBar: 2.0, seed: 1 }),
|
|
465
|
-
renderSfx({ type: "impact", atBar: 1.0, volume: 0.8 }, { sampleRate: 48000, secondsPerBar: 2.0, seed: 2 }),
|
|
466
|
-
]);
|
|
467
|
-
softLimit(bed);
|
|
468
|
-
|
|
469
|
-
// 3. Tempo signal (120 BPM = 2 Hz)
|
|
470
|
-
const beatPulse = Signal.builder({ type: "sawtooth", frequency: 2, amplitude: 0.08, offset: 1.0 });
|
|
471
|
-
|
|
472
|
-
// 4. Bind it to visuals
|
|
473
|
-
const reactiveCard = Layer.box({ width: 400, height: 250, background: "#1c202e", borderRadius: 20 })
|
|
474
|
-
.animate(
|
|
475
|
-
LayerAnimation.create()
|
|
476
|
-
.signal("scale", beatPulse, { multiplier: 1.0, offset: 0.0 })
|
|
477
|
-
.fromTo("opacity", 0, 1, { start: 0, end: 15, ease: "power2.out" }),
|
|
478
|
-
);
|
|
479
|
-
|
|
480
|
-
comp.add(reactiveCard);
|
|
481
|
-
```
|
|
482
|
-
|
|
483
|
-
---
|
|
484
|
-
|
|
485
|
-
### 4. Chained WebGPU Post-Processing VFX
|
|
486
|
-
|
|
487
|
-
```typescript
|
|
488
|
-
import { Composition, FilmGrain, Vignette, ColorBalance } from "framefields";
|
|
489
|
-
|
|
490
|
-
const comp = new Composition({ width: 1920, height: 1080, fps: 60 });
|
|
491
|
-
|
|
492
|
-
// Whole-composition cinematic grade + film emulsion
|
|
493
|
-
comp.apply(new Vignette({ strength: 0.28, radius: 0.85 }));
|
|
494
|
-
comp.apply(new FilmGrain({ strength: 0.06, size: 1.5, animated: true }));
|
|
495
|
-
comp.apply(
|
|
496
|
-
new ColorBalance({
|
|
497
|
-
shadows: { cyanRed: 0, magentaGreen: 2, yellowBlue: 6 },
|
|
498
|
-
highlights: { cyanRed: 4, magentaGreen: 1, yellowBlue: -2 },
|
|
499
|
-
}),
|
|
500
|
-
);
|
|
501
|
-
```
|
|
502
|
-
|
|
503
|
-
---
|
|
504
|
-
|
|
505
|
-
### 5. Vision: Pin, Matte & Reframe
|
|
506
|
-
|
|
507
|
-
```typescript
|
|
508
|
-
import { Composition, Layer, Vignette } from "framefields";
|
|
509
|
-
|
|
510
|
-
const comp = new Composition({ width: 1920, height: 1080, fps: 30 });
|
|
511
|
-
|
|
512
|
-
// Run vision on the whole composition. Models download lazily on first use.
|
|
513
|
-
const vision = comp.withVision({
|
|
514
|
-
enableDetection: true,
|
|
515
|
-
enableSegmentation: true,
|
|
516
|
-
enablePose: true,
|
|
517
|
-
variant: "s",
|
|
518
|
-
confidence: 0.35,
|
|
519
|
-
});
|
|
520
|
-
|
|
521
|
-
// Pin a caption to the primary tracked subject (smoothing + auto-hide when lost)
|
|
522
|
-
comp.add(
|
|
523
|
-
Layer.text("SUBJECT 01", { fontSize: 40, fill: "#f8fafc" }).pinToObject(
|
|
524
|
-
vision.objects.primary,
|
|
525
|
-
{ anchor: "topCenter", offsetY: -48, smoothFrames: 5, hideWhenLost: true },
|
|
526
|
-
),
|
|
527
|
-
);
|
|
528
|
-
|
|
529
|
-
// Drive a shader uniform from a reactive signal — here, subject mask coverage
|
|
530
|
-
comp.add(
|
|
531
|
-
Layer.box({ width: 1920, height: 1080, background: "#000000" }).withEffect(
|
|
532
|
-
new Vignette({ strength: vision.segmentation.subject.coverage, radius: 0.9 }),
|
|
533
|
-
),
|
|
534
|
-
);
|
|
535
|
-
|
|
536
|
-
// Or use the one-liners for the common editorial moves:
|
|
537
|
-
// comp.addSubjectSandwich({ source: "assets/dancer.mp4", behind: [headline], feather: 4 });
|
|
538
|
-
// comp.addSmartFraming({ source: "assets/action.mp4", target: vision.objects.primary, targetAspect: 9 / 16 });
|
|
539
|
-
// comp.addSubjectOutline(vision.segmentation.subject, { source: "assets/character.mp4", color: "#FF5A1F", width: 6 });
|
|
540
|
-
|
|
541
|
-
// Inspect a source before authoring: one-shot, ffmpeg-free, zod-serializable report
|
|
542
|
-
const report = await comp.analyzeVisionSequence("assets/street.mp4", {
|
|
543
|
-
tasks: ["detect", "pose"],
|
|
544
|
-
categories: ["person"],
|
|
545
|
-
});
|
|
546
|
-
console.log(report.tracks.map((t) => `${t.category}#${t.trackId} ${t.frames.join("–")}`));
|
|
547
|
-
```
|
|
548
|
-
|
|
549
|
-
**Standalone runner (no composition):**
|
|
550
|
-
|
|
551
|
-
```typescript
|
|
552
|
-
import { VisionRunner } from "@framefields/vision";
|
|
553
|
-
|
|
554
|
-
const runner = VisionRunner.create({ variant: "s", confidence: 0.3 }); // zero I/O
|
|
555
|
-
const frame = { data: rgba, width: 1920, height: 1080 };
|
|
556
|
-
const boxes = await runner.detect(frame); // downloads RTMDet-Ins on first call
|
|
557
|
-
const { masks } = await runner.segment(frame); // same forward pass, no second inference
|
|
558
|
-
const { people } = await runner.pose(frame); // RTMO, COCO-17 keypoints
|
|
559
|
-
runner.close();
|
|
560
|
-
```
|
|
561
|
-
|
|
562
|
-
**In the browser (WebGPU EP):**
|
|
563
|
-
|
|
564
|
-
```typescript
|
|
565
|
-
import { VisionRunner, createWebGPUProvider, hasWebGPU } from "@framefields/vision/web";
|
|
566
|
-
|
|
567
|
-
if (hasWebGPU()) {
|
|
568
|
-
const runner = VisionRunner.create({ provider: createWebGPUProvider() });
|
|
569
|
-
}
|
|
570
|
-
```
|
|
571
|
-
|
|
572
|
-
---
|
|
573
|
-
|
|
574
|
-
### 6. Headless Video & FrameGrid Rendering
|
|
575
|
-
|
|
576
|
-
```typescript
|
|
577
|
-
import { buildMyComposition } from "./my-composition.js";
|
|
578
|
-
|
|
579
|
-
const comp = await buildMyComposition();
|
|
580
|
-
|
|
581
|
-
// 1. Single frame to a PNG buffer for visual inspection
|
|
582
|
-
const frameBuffer = await comp.renderFrame({ frame: 45 });
|
|
583
|
-
|
|
584
|
-
// 2. Contact-sheet grid of 12 sequential frames
|
|
585
|
-
const gridBuffer = await comp.renderFrameGrid({
|
|
586
|
-
startFrame: 0,
|
|
587
|
-
endFrame: 120,
|
|
588
|
-
stepFrames: 10,
|
|
589
|
-
cellWidth: 320,
|
|
590
|
-
showLabels: true,
|
|
591
|
-
});
|
|
592
|
-
|
|
593
|
-
// 3. Final hardware-encoded MP4 with mixed audio
|
|
147
|
+
// 4. Render to hardware-encoded MP4
|
|
594
148
|
const { filePath } = await comp.renderVideo({
|
|
595
|
-
outputPath: "output/
|
|
149
|
+
outputPath: "output/hero.mp4",
|
|
596
150
|
quality: "high",
|
|
597
|
-
concurrency: 4,
|
|
598
151
|
});
|
|
599
152
|
|
|
600
|
-
console.log(`
|
|
153
|
+
console.log(`Rendered to: ${filePath}`);
|
|
154
|
+
process.exit(0);
|
|
601
155
|
```
|
|
602
156
|
|
|
603
157
|
---
|
|
604
158
|
|
|
605
|
-
##
|
|
159
|
+
## Documentation
|
|
606
160
|
|
|
607
|
-
|
|
608
|
-
2. **WebGPU premultiplied-alpha invariant** — fragment shaders outputting premultiplied alpha (`color * opacity * alpha`) must use `srcFactor: "one"` in their blend state (`{ srcFactor: "one", dstFactor: "one-minus-src-alpha", operation: "add" }`). Never use `srcFactor: "src-alpha"` for premultiplied output — squaring alpha darkens fades into murky gray.
|
|
609
|
-
3. **Carrier match cuts** — carry a visual element (badge, card, cursor, container) across scene boundaries with continuous velocity and position to avoid jarring cuts.
|
|
610
|
-
4. **Physical easing vocabulary** — `back.out(1.4–1.7)` for snap-overshoot entrances, `spring` / `expo.out` for decelerating motion, `power2.in` for exits. Reserve `linear` for infinite spinners and time counters.
|
|
611
|
-
5. **Headless invariant verification** — verify shader transforms, glyph coverage, and temporal MSE deltas with `skia-canvas` pixel sampling in Vitest before shipping.
|
|
612
|
-
6. **Image vs. Lottie layers** — raster images (`.png`, `.jpg`, `.webp`) use `Layer.image()`. Vector animations (`.json`, `.lottie`) must use `Layer.lottie()` — passing `.json` or `.lottie` to `Layer.image()` throws/warns and is rejected by `comp.lint()` with `unsupported_image_format`.
|
|
613
|
-
|
|
614
|
-
---
|
|
161
|
+
Everything else—including full API references, interactive tutorials, and guides—is available at [**framefields.dev/docs**](https://framefields.dev/docs):
|
|
615
162
|
|
|
616
|
-
|
|
617
|
-
|
|
618
|
-
|
|
619
|
-
|
|
620
|
-
|
|
621
|
-
|
|
622
|
-
| `framefields` | Starting a project: install from npm, scaffold a composition and render script, first verified render |
|
|
623
|
-
| `framefields-compose` | Compositions, layer trees, layout, animation and easing, beat grids, film structure |
|
|
624
|
-
| `framefields-effects` | Effect classes, the unified section architecture, premultiplied-alpha invariants, vision conditioning |
|
|
625
|
-
| `framefields-render` | Headless rendering, FrameGrid inspection, pixel probes, MP4 delivery checks |
|
|
626
|
-
|
|
627
|
-
Once installed, skills load automatically when a task matches (e.g. *"add a film-grain pass to this scene"* or *"render a frame grid of intro.ts"*).
|
|
628
|
-
|
|
629
|
-
### What the plugin runs and sends
|
|
630
|
-
|
|
631
|
-
The plugin is instructions only. It bundles no executables, MCP servers, hooks, or package launchers, and it sends no data anywhere. The skills tell your agent to add the [`framefields`](https://www.npmjs.com/package/framefields) npm package to your project and how to use it. When that code uses on-device vision, the SDK downloads the pinned model weights from Hugging Face on first use (see [On-Device Vision](#6-on-device-vision--tracking)). Nothing else leaves your machine.
|
|
632
|
-
|
|
633
|
-
### Claude Code
|
|
634
|
-
|
|
635
|
-
```text
|
|
636
|
-
/plugin install framefields
|
|
637
|
-
```
|
|
638
|
-
|
|
639
|
-
Or from your shell:
|
|
640
|
-
|
|
641
|
-
```bash
|
|
642
|
-
claude plugin install framefields@claude-plugins-official
|
|
643
|
-
```
|
|
644
|
-
|
|
645
|
-
It installs from Anthropic's official marketplace, which Claude Code adds for you, so there is no marketplace step, and plugins from it update automatically. Afterwards, restart Claude Code or run `/reload-plugins`. `/plugin` commands need an interactive `claude` terminal; in the desktop app's Code tab, use the shell form or **+ > Plugins > Add plugin** and pick **Framefields**.
|
|
646
|
-
|
|
647
|
-
Add `--scope project` to the shell form to record the plugin in `.claude/settings.json` for the whole team.
|
|
648
|
-
|
|
649
|
-
**Enable it for everyone in your repo.** Commit this to `.claude/settings.json`; Claude Code prompts teammates to install it when they trust the folder:
|
|
650
|
-
|
|
651
|
-
```json
|
|
652
|
-
{
|
|
653
|
-
"enabledPlugins": {
|
|
654
|
-
"framefields@claude-plugins-official": true
|
|
655
|
-
}
|
|
656
|
-
}
|
|
657
|
-
```
|
|
658
|
-
|
|
659
|
-
**Straight from this repository** (tracks `main` instead of the directory release):
|
|
660
|
-
|
|
661
|
-
```text
|
|
662
|
-
/plugin marketplace add gatewai-dev/framefields
|
|
663
|
-
/plugin install framefields@framefields-plugins
|
|
664
|
-
```
|
|
665
|
-
|
|
666
|
-
### Codex, Cursor, Hermes, and other agents
|
|
667
|
-
|
|
668
|
-
The [`skills`](https://skills.sh) CLI installs the skills into any of 70+ agents, including Codex, Cursor, Hermes, Gemini CLI, GitHub Copilot, Windsurf, OpenCode, and Goose:
|
|
669
|
-
|
|
670
|
-
```bash
|
|
671
|
-
npx skills add gatewai-dev/framefields
|
|
672
|
-
```
|
|
673
|
-
|
|
674
|
-
It detects the agents on your machine and asks where to install. To choose them yourself, pass `-a` once per agent, add `-g` to install for your user instead of this project, and `-y` to skip the prompts:
|
|
675
|
-
|
|
676
|
-
```bash
|
|
677
|
-
npx skills add gatewai-dev/framefields -a codex -a cursor -a hermes-agent -g -y
|
|
678
|
-
```
|
|
679
|
-
|
|
680
|
-
Keep them current with `npx skills update`, and remove them with `npx skills remove`.
|
|
681
|
-
|
|
682
|
-
Or copy the folders by hand: put `plugins/framefields/skills/<name>/` into `.claude/skills/`, `.agents/skills/`, or `~/.agents/skills/`. VS Code / Copilot / Cursor / Kiro can load the portable [`plugin.json`](https://github.com/gatewai-dev/framefields/blob/main/plugins/framefields/plugin.json) through their plugin UI.
|
|
683
|
-
|
|
684
|
-
### Maintaining the plugin
|
|
685
|
-
|
|
686
|
-
The plugin lives in [`plugins/framefields/`](https://github.com/gatewai-dev/framefields/blob/main/plugins/framefields) so installs carry only the skills; users get the engine from npm. Two manifests there describe it: [`plugin.json`](https://github.com/gatewai-dev/framefields/blob/main/plugins/framefields/plugin.json) (portable [Agent Plugins 1.0](https://agent-plugins.org), which also carries the OpenAI listing metadata) and [`.claude-plugin/plugin.json`](https://github.com/gatewai-dev/framefields/blob/main/plugins/framefields/.claude-plugin/plugin.json). The marketplace catalog is [`.claude-plugin/marketplace.json`](https://github.com/gatewai-dev/framefields/blob/main/.claude-plugin/marketplace.json). The portable field set is closed — client-specific fields go in that client's manifest, not in `plugin.json`. The `version` in both follows the `framefields` package: `pnpm run version:packages` syncs it after `changeset version` (or run `pnpm run sync:plugin-version` on its own), since clients use it to decide when to update.
|
|
687
|
-
|
|
688
|
-
Inside this repository, Claude Code and other agents pick up skills through the symlinks in `.agents/skills/` and `.claude/skills/`. Skills live only under `plugins/framefields/skills/`; never copy them elsewhere. `pnpm run check:plugins` validates manifests, skill frontmatter, marketplace catalogs, symlinks, and the generated effects catalog. `pnpm run sync:effects-catalog` regenerates the `framefields-effects` catalog after any `Effect` class change.
|
|
689
|
-
|
|
690
|
-
---
|
|
691
|
-
|
|
692
|
-
## Reference Showcase Examples
|
|
693
|
-
|
|
694
|
-
The [`examples/`](https://github.com/gatewai-dev/framefields/blob/main/examples) directory holds production-grade reference compositions:
|
|
695
|
-
|
|
696
|
-
| Example | What it demonstrates |
|
|
697
|
-
| --- | --- |
|
|
698
|
-
| [`19_framefields_film`](https://github.com/gatewai-dev/framefields/blob/main/examples/19_framefields_film) | The 30-second master brand film — full pipeline, audio, VFX, 3D |
|
|
699
|
-
| [`21_full_circle`](https://github.com/gatewai-dev/framefields/blob/main/examples/21_full_circle) | Multi-scene narrative composition |
|
|
700
|
-
| [`22_framefields_launch`](https://github.com/gatewai-dev/framefields/blob/main/examples/22_framefields_launch) | Launch/product-motion composition |
|
|
701
|
-
| [`26_mac_keyboard_from_3d_primitives`](https://github.com/gatewai-dev/framefields/blob/main/examples/26_mac_keyboard_from_3d_primitives) | Mac keyboard product shot from 3D primitives — procedural keycap tapers, typing choreography, and light waves |
|
|
702
|
-
|
|
703
|
-
---
|
|
704
|
-
|
|
705
|
-
## Development & Building
|
|
706
|
-
|
|
707
|
-
Framefields uses `pnpm` (10+) and `turbo` for orchestration.
|
|
708
|
-
|
|
709
|
-
```bash
|
|
710
|
-
# Install
|
|
711
|
-
pnpm install
|
|
712
|
-
|
|
713
|
-
# Build all packages
|
|
714
|
-
pnpm build
|
|
715
|
-
|
|
716
|
-
# Run conformance tests
|
|
717
|
-
pnpm test
|
|
718
|
-
|
|
719
|
-
# Check the vision models end to end (downloads ~380 MB of weights once)
|
|
720
|
-
pnpm --filter @framefields/vision test:models
|
|
721
|
-
|
|
722
|
-
# Render a specific showcase example
|
|
723
|
-
pnpm --filter @framefields/example-21-full-circle render
|
|
724
|
-
|
|
725
|
-
# Render the master brand film
|
|
726
|
-
cd examples/19_framefields_film && pnpm render
|
|
727
|
-
```
|
|
728
|
-
|
|
729
|
-
### Docker Container for Production Rendering
|
|
730
|
-
|
|
731
|
-
An optimized [`Dockerfile.renderer`](https://github.com/gatewai-dev/framefields/blob/main/Dockerfile.renderer) deploys the renderer service into cloud GPU clusters:
|
|
732
|
-
|
|
733
|
-
```bash
|
|
734
|
-
docker build -t framefields-renderer -f Dockerfile.renderer .
|
|
735
|
-
```
|
|
163
|
+
- **[Typography & Animators](https://framefields.dev/docs)** — Slug analytic vector text, After Effects-style range animators, and typewriter cadence
|
|
164
|
+
- **[WebGPU 2D VFX](https://framefields.dev/docs)** — 50+ shaders: 3D LUTs, Curves, Film Grain, Glassmorphism, and Blur
|
|
165
|
+
- **[Unified 3D Scenes](https://framefields.dev/docs)** — LookAt cameras, PBR materials, studio lighting, shadows, and glTF/GLB models
|
|
166
|
+
- **[Audio & Procedural SFX](https://framefields.dev/docs)** — BeatGrids, procedural whooshes/impacts, and audio-reactive signals
|
|
167
|
+
- **[On-Device Vision](https://framefields.dev/docs)** — Apache-2.0 ONNX models for object tracking, pose estimation, and subject cutouts
|
|
168
|
+
- **[Live Browser Preview](https://framefields.dev/docs)** — Localhost WebGPU player with scrubbing, waveform timeline, and frame-stepping
|
|
736
169
|
|
|
737
170
|
---
|
|
738
171
|
|
|
739
|
-
## Community
|
|
740
|
-
|
|
741
|
-
- **Website:** [framefields.dev](https://framefields.dev/)
|
|
742
|
-
- **Discord:** ask questions, share renders, and follow development at [discord.gg/cbqMGGme5](https://discord.gg/cbqMGGme5).
|
|
743
|
-
- **YouTube:** watch films made with framefields on [@gatewai.studio](https://www.youtube.com/@gatewai.studio).
|
|
744
|
-
|
|
745
|
-
---
|
|
172
|
+
## Community & License
|
|
746
173
|
|
|
747
|
-
|
|
174
|
+
- **Documentation:** [framefields.dev/docs](https://framefields.dev/docs)
|
|
175
|
+
- **Website:** [framefields.dev](https://framefields.dev)
|
|
176
|
+
- **Discord:** [discord.gg/cbqMGGme5](https://discord.gg/cbqMGGme5)
|
|
177
|
+
- **YouTube:** [@gatewai.studio](https://www.youtube.com/@gatewai.studio)
|
|
748
178
|
|
|
749
|
-
|
|
179
|
+
Licensed under [Apache-2.0](https://github.com/gatewai-dev/framefields/blob/main/LICENCE).
|