@omnimedia/omnitool 1.1.0-46 → 1.1.0-47

package/README.md

@@ -1,120 +1,156 @@
 
 # 🚧 Work In Progress
 
-> **Note:** Omni Tools is under development. Expect breaking changes, evolving APIs, and experimental features.
+> **Note:** Omnitool is under development. Expect breaking changes, evolving APIs, and experimental features.
 
 ---
 
-# 🎬 Omnitool
+## ✅ What this library is
 
-> Code-first video editing toolkit behind [Omniclip](https://omniclip.app) — build timelines, render videos, and automate workflows.
+- API for building and rendering timelines in the browser
+- Uses WebCodecs, Web Workers, and the File System Access API for export
+- usage is currently JavaScript/TypeScript only, cli soon
 
----
+## 🚀 Install
 
-## 🧱 Modular by Design
+```bash
+npm i @omnimedia/omnitool
+```
 
-Omni Tools is a collection of composable utilities for working with Omniclip timelines — via code, JSON, or CLI.
+## 📦 Quick Start
 
-- ✅ Define timelines in JSON or TypeScript
-- ✅ Automate rendering with CLI tools
-- ✅ Ideal for scripting, CI/CD, and AI-generated workflows
+```ts
+import {Driver, Omni, Datafile} from "@omnimedia/omnitool"
+
+const driver = await Driver.setup()
+const omni = new Omni(driver)
+
+const {clip} = await omni.load({
+	clip: Datafile.make(file) // file is a File or Blob
+})
+
+const timeline = omni.timeline(o => {
+	const caption = o.text("Hello world", {
+		duration: 1500,
+		styles: {fill: "white", fontSize: 48}
+	})
+	const xfade = o.transition.crossfade(500)
+
+	return o.sequence(
+		o.stack(
+			o.video(clip, {start: 0, duration: 3000}),
+			caption
+		),
+		o.gap(400),
+		xfade,
+		o.video(clip, {start: 5000, duration: 2500}),
+		o.audio(clip, {start: 5000, duration: 2500})
+	)
+})
+```
 
----
+## 🧭 Spatial Transforms
 
-## 🚀 Quick Start
+```ts
+const timeline = omni.timeline(o => {
+	const move = o.spatial(o.transform({
+		position: [120, 40],
+		scale: [0.6, 0.6],
+		rotation: 0.2
+	}))
+
+	const title = o.text("Lower third", {
+		duration: 2000,
+		styles: {fill: "white", fontSize: 36}
+	})
+	o.set(title.id, {spatialId: move.id})
+
+	return o.stack(
+		o.video(clip, {duration: 4000}),
+		title
+	)
+})
+```
 
-### Install
+Worker URL notes:
+- `Driver.setup()` defaults to `/node_modules/@omnimedia/omnitool/x/driver/driver.worker.bundle.min.js`.
+- If you serve the worker from a different location, pass `workerUrl`:
 
-```bash
-npm i @omnimedia/omnitool
+```ts
+const workerUrl = new URL(
+	"/path/to/driver.worker.bundle.min.js",
+	window.location.href
+)
+const driver = await Driver.setup({workerUrl})
 ```
 
----
-
-## 📦 Example (Programmatic Timeline)
+## ▶️ Playback
 
 ```ts
-import { subtitle, crossfade, sequence, stack, video } from "@omni/tools"
-
-const watermark = subtitle("omniclip")
-const xfade = crossfade(500)
-
-const timeline = sequence(
-  video("opening-credits.mp4"),
-  xfade,
-  stack(
-    video("skateboarding.mp4"),
-    watermark
-  ),
-  xfade,
-  stack(
-    video("biking.mp4"),
-    watermark
-  )
-)
-	
+const player = await omni.playback(timeline)
+
+document.body.appendChild(player.canvas)
+player.play()
 ```
 
----
+Notes:
+- Call `player.update(timeline)` if you update the timeline.
+
+## 📤 Export
 
-## 🧩 Timeline Format (Omni Timeline Format)
+```ts
+await omni.render(timeline, framerate)
+```
+
+## 🧩 Timeline Format (TimelineFile)
 
-Every timeline is defined as a graph:
+All durations and timestamps are in milliseconds.
 
 ```json
 {
-  "format": "omni-timeline@1",
-  "root": "root-1",
-  "items": [
-    ["root-1", ["sequence", { "children": ["video-1", "stack-1"] }]],
-    ["video-1", ["video", { ... }]],
-    ["stack-1", ["stack", { "children": ["text-1", "audio-1"] }]],
-    ["text-1", ["text", { ... }]],
-    ["audio-1", ["audio", { ... }]]
-  ]
+	"format": "timeline",
+	"info": "https://omniclip.app/",
+	"version": 0,
+	"rootId": 123,
+	"items": [
+		{"id": 123, "kind": 0, "childrenIds": [456, 789]},
+		{"id": 456, "kind": 2, "mediaHash": "...", "start": 0, "duration": 3000},
+		{"id": 789, "kind": 4, "content": "Hello", "duration": 1500}
+	]
 }
 ```
 
-Each item is a `[id, item]` pair. Items can reference others by ID, forming a directed graph of reusable, composable blocks.
-
----
+Timeline items:
+- 0 `Sequence`
+- 1 `Stack`
+- 2 `Video`
+- 3 `Audio`
+- 4 `Text`
+- 5 `Gap`
+- 6 `Spatial`
+- 7 `Transition`
+- 8 `TextStyle`
 
-## 🖥 CLI Usage
+## 🗺️ Roadmap
+- Planned CLI commands (not available in this repo yet):
 
 ```bash
-# Build a timeline (manually or via AI)
+# build a reusable template from a timeline
 omnitool build-template promo.json
-
-# Validate structure
+# validate a timeline file
 omnitool validate promo.json
-
-# Render video
+# export a timeline to a video file
 omnitool export promo.json --output final.mp4
-
-# Batch render
+# batch export multiple timelines
 omnitool batch-export ./projects/* --output-dir ./exports
+# headless timeline viewer
+omnitool preview promo.json
+# auto-fit timeline elements
+omnitool optimize promo.json
+# prompt-to-timeline generation
+omnitool ai "make a 15s promo for tea"
 ```
 
----
-
-## ✅ Use Cases
-
-- Render videos from scripts, templates, or AI
-- Build and test timelines without opening the UI
-- Generate video pipelines from code or prompts
-
----
-
-## 📘 More to come
-
-- `omnitool preview` – headless timeline viewer
-- `omnitool optimize` – auto-fit timeline elements
-- `omnitool ai` – native prompt-to-timeline generation
-
----
-
-## 🧠 Learn More
-
-Omni Tools powers AI agents, programmatic editing, and upcoming new version of omniclip video editor.
+- smooth seeking
+- server-side, not just browsers
 
-→ [Visit Omniclip](https://omniclip.app)

package/package.json

@@ -1,6 +1,6 @@
 {
 	"name": "@omnimedia/omnitool",
-	"version": "1.1.0-46",
+	"version": "1.1.0-47",
 	"description": "open source video processing tools",
 	"license": "MIT",
 	"author": "Przemysław Gałęzki",

package/s/demo/demo.bundle.ts

@@ -38,7 +38,7 @@ async function startDemoImport(e: Event)
 
 		const {timeline, omni} = await TimelineSchemaTest(driver, file)
 
-		playbackTest(driver, timeline, omni)
+		playbackTest(timeline, omni)
 		exportTest(omni, timeline)
 	}
 	// const [fileHandle] = await window.showOpenFilePicker()

package/s/demo/routines/playback-test.ts

@@ -1,13 +1,12 @@
 
-import {Driver} from "../../driver/driver.js"
-import {O, Omni, TimelineFile, VideoPlayer} from "../../timeline/index.js"
+import {O, Omni, TimelineFile} from "../../timeline/index.js"
 
-export async function playbackTest(driver: Driver, timeline: TimelineFile, omni: Omni) {
+export async function playbackTest(timeline: TimelineFile, omni: Omni) {
 	const playButton = document.querySelector(".play") as HTMLButtonElement
 	const stopButton = document.querySelector(".stop") as HTMLButtonElement
 	const seekButton = document.querySelector(".seek") as HTMLButtonElement
 	const o = new O({project: timeline})
-	const player = await VideoPlayer.create(driver, timeline, (hash) => omni.resources.require(hash).url)
+	const player = await omni.playback(timeline)
 	document.body.appendChild(player.canvas)
 
 	playButton.addEventListener("click", () => player.play())

package/s/timeline/parts/renderers/export.ts

@@ -9,8 +9,7 @@ export class Export {
 	#sampler
 	constructor(
 		private driver: Driver,
-		private framerate = 30,
-		private resolveMedia: (hash: string) => DecoderSource = _hash => "/assets/temp/gl.mp4"
+		private resolveMedia: (hash: string) => DecoderSource
 	) {
 		this.#sampler = makeWebCodecsSampler(this.driver, this.resolveMedia)
 	}
@@ -21,7 +20,7 @@ export class Export {
 		return await buildWebCodecsNodeTree(rootItem, items, this.#sampler)
 	}
 
-	async render(timeline: TimelineFile) {
+	async render(timeline: TimelineFile, framerate: number) {
 		const root = await this.#build(timeline)
 
 		const videoStream = new TransformStream<VideoFrame, VideoFrame>()
@@ -50,10 +49,10 @@ export class Export {
 
 		const videoPromise = (async () => {
 			let i = 0
-			const dt = 1 / this.framerate
+			const dt = 1 / framerate
 
 			await fixedStep(
-				{fps: this.framerate, duration: root.duration ?? 0},
+				{fps: framerate, duration: root.duration ?? 0},
 				async t => {
 					const layers = await root.visuals?.sampleAt(t) ?? []
 					const composed = await this.driver.composite(layers)

package/s/timeline/sugar/omni.ts

@@ -7,14 +7,17 @@ import {Datafile} from "../utils/datafile.js"
 import {TimelineFile} from "../parts/basics.js"
 import {Export} from "../parts/renderers/export.js"
 import {ResourcePool} from "../parts/resource-pool.js"
-import {RenderConfig} from "../../driver/fns/schematic.js"
+import {VideoPlayer} from "../parts/renderers/playback.js"
 
 export class Omni {
 	resources = new ResourcePool()
 	#export: Export
 
 	constructor(private driver: Driver) {
-		this.#export = new Export(driver)
+		this.#export = new Export(
+			driver,
+			(hash) => this.resources.require(hash).url
+		)
 	}
 
 	load = async<S extends Record<string, Promise<Datafile>>>(spec: S) => {
@@ -38,8 +41,16 @@ export class Omni {
 		return o.state.project
 	}
 
-	render = async (timeline: TimelineFile) => {
-		await this.#export.render(timeline)
+	playback = async (timeline: TimelineFile) => {
+		return await VideoPlayer.create(
+			this.driver,
+			timeline,
+			(hash) => this.resources.require(hash).url
+		)
+	}
+
+	render = async (timeline: TimelineFile, framerate: number = 30) => {
+		await this.#export.render(timeline, framerate)
 	}
 }
 

package/x/demo/demo.bundle.js

@@ -30,7 +30,7 @@ async function startDemoImport(e) {
         run(transcode, file.name);
         // await transcriber.transcribe(file)
         const { timeline, omni } = await TimelineSchemaTest(driver, file);
-        playbackTest(driver, timeline, omni);
+        playbackTest(timeline, omni);
         exportTest(omni, timeline);
     }
     // const [fileHandle] = await window.showOpenFilePicker()

package/x/demo/demo.bundle.js.map

@@ -1 +1 @@
-{"version":3,"file":"demo.bundle.js","sourceRoot":"","sources":["../../s/demo/demo.bundle.ts"],"names":[],"mappings":"AACA,OAAO,EAAC,MAAM,EAAC,MAAM,qBAAqB,CAAA;AAC1C,OAAO,EAAC,UAAU,EAAC,MAAM,2BAA2B,CAAA;AACpD,OAAO,EAAC,YAAY,EAAC,MAAM,6BAA6B,CAAA;AACxD,OAAO,EAAC,YAAY,EAAC,MAAM,6BAA6B,CAAA;AACxD,OAAO,EAAC,kBAAkB,EAAE,MAAM,8BAA8B,CAAA;AAChE,OAAO,EAAC,aAAa,EAAC,MAAM,8BAA8B,CAAA;AAC1D,OAAO,EAAC,kBAAkB,EAAC,MAAM,8BAA8B,CAAA;AAE/D,MAAM,MAAM,GAAG,MAAM,MAAM,CAAC,KAAK,CAAC,EAAC,SAAS,EAAE,IAAI,GAAG,CAAC,uCAAuC,EAAE,MAAM,CAAC,IAAI,CAAC,GAAG,CAAC,EAAC,CAAC,CAAA;AACjH,MAAM,OAAO,GAAG,QAAQ,CAAC,aAAa,CAAC,UAAU,CAAE,CAAA;AAEnD,MAAM,WAAW,GAAG,QAAQ,CAAC,aAAa,CAAC,QAAQ,CAAC,CAAA;AACpD,MAAM,SAAS,GAAG,QAAQ,CAAC,aAAa,CAAC,aAAa,CAAqB,CAAA;AAE3E,WAAW,EAAE,gBAAgB,CAAC,OAAO,EAAE,cAAc,CAAC,CAAA;AACtD,SAAS,EAAE,gBAAgB,CAAC,OAAO,EAAE,eAAe,CAAC,CAAA;AAErD,YAAY,CAAC,MAAM,CAAC,CAAA;AACpB,oDAAoD;AAEpD,mBAAmB;AACnB,CAAC;IACA,MAAM,MAAM,CAAC,MAAM,CAAC,IAAI,CAAC,KAAK,EAAE,CAAA;IAChC,IAAI,MAAM,CAAC,OAAO,CAAC,KAAK,KAAK,CAAC;QAAE,OAAO,CAAC,GAAG,CAAC,gBAAgB,CAAC,CAAA;;QACxD,OAAO,CAAC,KAAK,CAAC,gCAAgC,CAAC,CAAA;AACrD,CAAC;AAED,oBAAoB;AACpB,KAAK,UAAU,eAAe,CAAC,CAAQ;IAEtC,MAAM,IAAI,GAAG,SAAS,CAAC,KAAK,EAAE,CAAC,CAAC,CAAC,CAAA;IACjC,IAAG,IAAI,EAAE,CAAC;QACT,MAAM,SAAS,GAAG,kBAAkB,CAAC,MAAM,EAAE,IAAI,CAAC,CAAA;QAClD,MAAM,aAAa,CAAC,IAAI,CAAC,CAAA;QACzB,GAAG,CAAC,SAAS,EAAE,IAAI,CAAC,IAAI,CAAC,CAAA;QACzB,qCAAqC;QAErC,MAAM,EAAC,QAAQ,EAAE,IAAI,EAAC,GAAG,MAAM,kBAAkB,CAAC,MAAM,EAAE,IAAI,CAAC,CAAA;QAE/D,YAAY,CAAC,MAAM,EAAE,QAAQ,EAAE,IAAI,CAAC,CAAA;QACpC,UAAU,CAAC,IAAI,EAAE,QAAQ,CAAC,CAAA;IAC3B,CAAC;IACD,yDAAyD;IACzD,0CAA0C;AAC3C,CAAC;AAED,KAAK,UAAU,cAAc;IAG5B,+BAA+B;IAC/B,MAAM,MAAM,GAAG;QACd,qBAAqB;KACrB,CAAA;IAED,gCAAgC;IAChC,KAAK,MAAM,GAAG,IAAI,MAAM,EAAE,CAAC;QAC1B,MAAM,SAAS,GAAG,kBAAkB,CAAC,MAAM,EAAE,qBAAqB,CAAC,CAAA;QACnE,GAAG,CAAC,SAAS,EAAE,GAAG,CAAC,CAAA;IACpB,CAAC;AACF,CAAC;AAED,KAAK,UAAU,GAAG,CAAC,SAAgD,EAAE,KAAa;IACjF,oBAAoB;IACpB,MAAM,GAAG,GAAG,QAAQ,CAAC,aAAa,CAAC,KAAK,CAAC,CAAA;IACzC,OAAO,CAAC,MAAM,CAAC,GAAG,CAAC,CAAA;IAEnB,kBAAkB;IAClB,MAAM,CAAC,GAAG,QAAQ,CAAC,aAAa,CAAC,GAAG,CAAC,CAAA;IACrC,CAAC,CAAC,WAAW,GAAG,KAAK,CAAA;IACrB,GAAG,CAAC,MAAM,CAAC,CAAC,CAAC,CAAA;IAEb,wBAAwB;IACxB,GAAG,CAAC,MAAM,CAAC,SAAS,CAAC,MAAM,CAAC,CAAA;IAE5B,eAAe;IACf,MAAM,SAAS,CAAC,GAAG,EAAE,CAAA;AACtB,CAAC"}
+{"version":3,"file":"demo.bundle.js","sourceRoot":"","sources":["../../s/demo/demo.bundle.ts"],"names":[],"mappings":"AACA,OAAO,EAAC,MAAM,EAAC,MAAM,qBAAqB,CAAA;AAC1C,OAAO,EAAC,UAAU,EAAC,MAAM,2BAA2B,CAAA;AACpD,OAAO,EAAC,YAAY,EAAC,MAAM,6BAA6B,CAAA;AACxD,OAAO,EAAC,YAAY,EAAC,MAAM,6BAA6B,CAAA;AACxD,OAAO,EAAC,kBAAkB,EAAE,MAAM,8BAA8B,CAAA;AAChE,OAAO,EAAC,aAAa,EAAC,MAAM,8BAA8B,CAAA;AAC1D,OAAO,EAAC,kBAAkB,EAAC,MAAM,8BAA8B,CAAA;AAE/D,MAAM,MAAM,GAAG,MAAM,MAAM,CAAC,KAAK,CAAC,EAAC,SAAS,EAAE,IAAI,GAAG,CAAC,uCAAuC,EAAE,MAAM,CAAC,IAAI,CAAC,GAAG,CAAC,EAAC,CAAC,CAAA;AACjH,MAAM,OAAO,GAAG,QAAQ,CAAC,aAAa,CAAC,UAAU,CAAE,CAAA;AAEnD,MAAM,WAAW,GAAG,QAAQ,CAAC,aAAa,CAAC,QAAQ,CAAC,CAAA;AACpD,MAAM,SAAS,GAAG,QAAQ,CAAC,aAAa,CAAC,aAAa,CAAqB,CAAA;AAE3E,WAAW,EAAE,gBAAgB,CAAC,OAAO,EAAE,cAAc,CAAC,CAAA;AACtD,SAAS,EAAE,gBAAgB,CAAC,OAAO,EAAE,eAAe,CAAC,CAAA;AAErD,YAAY,CAAC,MAAM,CAAC,CAAA;AACpB,oDAAoD;AAEpD,mBAAmB;AACnB,CAAC;IACA,MAAM,MAAM,CAAC,MAAM,CAAC,IAAI,CAAC,KAAK,EAAE,CAAA;IAChC,IAAI,MAAM,CAAC,OAAO,CAAC,KAAK,KAAK,CAAC;QAAE,OAAO,CAAC,GAAG,CAAC,gBAAgB,CAAC,CAAA;;QACxD,OAAO,CAAC,KAAK,CAAC,gCAAgC,CAAC,CAAA;AACrD,CAAC;AAED,oBAAoB;AACpB,KAAK,UAAU,eAAe,CAAC,CAAQ;IAEtC,MAAM,IAAI,GAAG,SAAS,CAAC,KAAK,EAAE,CAAC,CAAC,CAAC,CAAA;IACjC,IAAG,IAAI,EAAE,CAAC;QACT,MAAM,SAAS,GAAG,kBAAkB,CAAC,MAAM,EAAE,IAAI,CAAC,CAAA;QAClD,MAAM,aAAa,CAAC,IAAI,CAAC,CAAA;QACzB,GAAG,CAAC,SAAS,EAAE,IAAI,CAAC,IAAI,CAAC,CAAA;QACzB,qCAAqC;QAErC,MAAM,EAAC,QAAQ,EAAE,IAAI,EAAC,GAAG,MAAM,kBAAkB,CAAC,MAAM,EAAE,IAAI,CAAC,CAAA;QAE/D,YAAY,CAAC,QAAQ,EAAE,IAAI,CAAC,CAAA;QAC5B,UAAU,CAAC,IAAI,EAAE,QAAQ,CAAC,CAAA;IAC3B,CAAC;IACD,yDAAyD;IACzD,0CAA0C;AAC3C,CAAC;AAED,KAAK,UAAU,cAAc;IAG5B,+BAA+B;IAC/B,MAAM,MAAM,GAAG;QACd,qBAAqB;KACrB,CAAA;IAED,gCAAgC;IAChC,KAAK,MAAM,GAAG,IAAI,MAAM,EAAE,CAAC;QAC1B,MAAM,SAAS,GAAG,kBAAkB,CAAC,MAAM,EAAE,qBAAqB,CAAC,CAAA;QACnE,GAAG,CAAC,SAAS,EAAE,GAAG,CAAC,CAAA;IACpB,CAAC;AACF,CAAC;AAED,KAAK,UAAU,GAAG,CAAC,SAAgD,EAAE,KAAa;IACjF,oBAAoB;IACpB,MAAM,GAAG,GAAG,QAAQ,CAAC,aAAa,CAAC,KAAK,CAAC,CAAA;IACzC,OAAO,CAAC,MAAM,CAAC,GAAG,CAAC,CAAA;IAEnB,kBAAkB;IAClB,MAAM,CAAC,GAAG,QAAQ,CAAC,aAAa,CAAC,GAAG,CAAC,CAAA;IACrC,CAAC,CAAC,WAAW,GAAG,KAAK,CAAA;IACrB,GAAG,CAAC,MAAM,CAAC,CAAC,CAAC,CAAA;IAEb,wBAAwB;IACxB,GAAG,CAAC,MAAM,CAAC,SAAS,CAAC,MAAM,CAAC,CAAA;IAE5B,eAAe;IACf,MAAM,SAAS,CAAC,GAAG,EAAE,CAAA;AACtB,CAAC"}