@miketromba/ploof 0.1.5 → 0.3.0

package/README.md

@@ -9,7 +9,7 @@
   <img src="https://img.shields.io/badge/node-%3E%3D18-brightgreen" alt="node version" />
 </p>
 
-Ploof is a CLI for generating and editing creative assets with AI providers. It supports OpenAI image generation and editing today, plus the legacy OpenAI image variations endpoint when the authenticated project has access. The provider registry is designed for audio, video, and broader model marketplaces over time.
+Ploof is a CLI for generating and editing creative assets with AI providers. It supports OpenAI image, video, and audio generation/processing today, plus the legacy OpenAI image variations endpoint when the authenticated project has access. The provider registry is designed for broader model marketplaces over time.
 
 It is built for both developers and AI agents: predictable commands, parseable output, local auth profiles, YAML manifests, parallel execution, and a companion skill.
 
@@ -21,12 +21,18 @@ It is built for both developers and AI agents: predictable commands, parseable o
 | OpenAI image generation | Supported |
 | OpenAI image editing | Supported |
 | OpenAI image variations | Legacy endpoint; supported when available to the authenticated project |
+| OpenAI video generation | Supported |
+| OpenAI video editing/extensions | Supported |
+| OpenAI video downloads/library/characters | Supported |
+| OpenAI audio generation / TTS | Supported |
+| OpenAI audio transcription | Supported |
+| OpenAI audio translation | Supported |
 | Context images and masks | Supported |
+| Image, video, and audio input assets | Supported |
 | YAML/JSON batch manifests | Supported |
 | Dependency-aware parallel runs | Supported |
 | Agent instructions via `ploof learn` | Supported |
 | Additional providers | Planned |
-| Audio/video generation | Planned |
 
 ## Install
 
@@ -68,6 +74,24 @@ ploof image edit \
   --prompt "Replace the background with a clean marble countertop" \
   --out assets/edited.png
 
+# Generate and download a video
+ploof video generate \
+  --prompt "Wide tracking shot of a paper city at blue hour" \
+  --model sora-2 \
+  --size 1280x720 \
+  --seconds 4 \
+  --out assets/clip.mp4
+
+# Generate and transcribe speech
+ploof audio generate \
+  --text "Ploof can generate speech and process audio." \
+  --voice alloy \
+  --out assets/speech.mp3
+
+ploof audio transcribe \
+  --audio assets/speech.mp3 \
+  --out assets/transcript.json
+
 # Run a manifest
 ploof run assets.yaml --parallel 4
 ```
@@ -170,6 +194,120 @@ The plural alias also works:
 ploof image variations --image input.png --out variation.png
 ```
 
+## Video Generation
+
+OpenAI video generation uses the asynchronous Videos API. `ploof video generate` submits a job immediately. If you pass `--out` or `--download`, Ploof waits for completion and downloads the requested asset.
+
+```bash
+ploof video generate \
+  --provider openai \
+  --prompt "Wide tracking shot of a teal coupe on a desert highway" \
+  --model sora-2 \
+  --size 1280x720 \
+  --seconds 4 \
+  --out assets/clip.mp4 \
+  --output json
+```
+
+Useful generation flags:
+
+| Flag | Description |
+| --- | --- |
+| `--model <model>` | Video model, for example `sora-2` or `sora-2-pro` |
+| `--size <size>` | Output resolution, for example `1280x720` |
+| `--seconds <seconds>` | Clip or extension duration |
+| `--input-reference <path-or-url-or-file-id>` | Image reference for the first frame |
+| `--input-reference-file-id <id>` | OpenAI uploaded image file id |
+| `--input-reference-url <url>` | Image URL or data URL reference |
+| `--character <id>` | Reusable character id; repeat for multiple characters |
+| `--wait` | Poll until the job reaches a terminal status |
+| `--download` | Download after waiting |
+| `--variant <variant>` | `video`, `thumbnail`, or `spritesheet` |
+| `--poll-interval <seconds>` | Polling interval while waiting |
+| `--timeout <seconds>` | Maximum wait time |
+| `--param key=value` | Provider-specific pass-through parameter |
+| `--json '{...}'` | Provider-specific JSON object |
+
+If you omit `--model`, Ploof defaults OpenAI video generation to `sora-2`.
+
+## Video Editing And Library
+
+```bash
+ploof video edit \
+  --video-id video_abc123 \
+  --prompt "Shift the palette to teal and rust" \
+  --out assets/edit.mp4
+
+ploof video extend \
+  --video-id video_abc123 \
+  --prompt "Continue as the camera rises over the rooftops" \
+  --seconds 4 \
+  --out assets/extended.mp4
+
+ploof video download video_abc123 --variant thumbnail --out assets/thumb.webp
+ploof video status video_abc123 --output json
+ploof video list --limit 20 --output json
+ploof video delete video_abc123
+```
+
+OpenAI video edits accept either `--video-id <id>` for an existing completed OpenAI video or `--video <path>` for an uploaded source video when the authenticated project is eligible for that workflow. Extensions accept a source video id or upload, plus a prompt and `--seconds`.
+
+Reusable character commands:
+
+```bash
+ploof video character create --name Mossy --video character.mp4 --output json
+ploof video character get char_abc123 --output json
+```
+
+## Audio Generation And Processing
+
+OpenAI audio generation defaults to `gpt-4o-mini-tts`, `alloy`, and `mp3` when model, voice, and format are omitted.
+
+```bash
+ploof audio generate \
+  --provider openai \
+  --text "A concise product narration for the demo reel." \
+  --model gpt-4o-mini-tts \
+  --voice alloy \
+  --format mp3 \
+  --out assets/narration.mp3 \
+  --output json
+```
+
+Useful generation flags:
+
+| Flag | Description |
+| --- | --- |
+| `--model <model>` | TTS model, for example `gpt-4o-mini-tts` |
+| `--voice <voice>` | Built-in voice such as `alloy`, `coral`, `nova`, or `shimmer` |
+| `--voice-id <id>` | Custom voice id |
+| `--instructions <text>` | Voice/style instructions for supported models |
+| `--format <format>` | `mp3`, `opus`, `aac`, `flac`, `wav`, or `pcm` |
+| `--speed <number>` | Speech speed |
+| `--param key=value` | Provider-specific pass-through parameter |
+| `--json '{...}'` | Provider-specific JSON object |
+
+Transcription and translation:
+
+```bash
+ploof audio transcribe \
+  --audio assets/narration.mp3 \
+  --model gpt-4o-mini-transcribe \
+  --out assets/transcript.json \
+  --output json
+
+ploof audio translate \
+  --audio assets/spanish.mp3 \
+  --model whisper-1 \
+  --format text \
+  --out assets/translation.txt \
+  --output json
+```
+
+Transcription supports `--language`, `--prompt`, `--format`, `--temperature`, `--include`, `--timestamp-granularity`, `--chunking-strategy`, `--known-speaker-name`, and `--known-speaker-reference`. Translation supports `--prompt`, `--format`, and `--temperature`.
+
+Ploof writes complete static assets to disk. Streaming transport settings such as OpenAI `stream=true` for transcription or `stream_format=sse` for speech are rejected because they do not produce a finished asset file directly.
+
 ## Batch Manifests
 
 ```yaml
@@ -205,6 +343,39 @@ tasks:
       images:
         - task: base
     output: assets/variation.png
+
+  - id: clip
+    kind: video.generate
+    provider: openai
+    prompt: "Slow dolly shot through a miniature paper city"
+    params:
+      model: sora-2
+      size: 1280x720
+      seconds: "4"
+    wait: true
+    download: true
+    output: assets/clip.mp4
+
+  - id: narration
+    kind: audio.generate
+    provider: openai
+    text: "Short narration for the generated clip."
+    params:
+      model: gpt-4o-mini-tts
+      voice: alloy
+      response_format: mp3
+    output: assets/narration.mp3
+
+  - id: transcript
+    kind: audio.transcribe
+    provider: openai
+    needs: [narration]
+    inputs:
+      audio:
+        task: narration
+    params:
+      model: gpt-4o-mini-transcribe
+    output: assets/transcript.json
 ```
 
 Run it:
@@ -283,7 +454,7 @@ bun run build
 npm pack --dry-run
 ```
 
-The default test suite includes mocked OpenAI end-to-end tests. Those tests run real `ploof` CLI commands against a local mock OpenAI server and verify generated files, edit uploads, sidecar metadata, and dependency-aware manifests without spending API credits.
+The default test suite includes mocked OpenAI end-to-end tests. Those tests run real `ploof` CLI commands against a local mock OpenAI server and verify generated files, edit uploads, video job polling/downloads, audio generation/processing, sidecar metadata, and dependency-aware manifests without spending API credits.
 
 Live OpenAI tests are opt-in only:
 

package/SPEC.md

@@ -2,7 +2,7 @@
 
 ## Summary
 
-Ploof is an npm-published CLI for generating and editing assets through AI generation providers. It starts with OpenAI image generation and editing, but the architecture must support multiple authenticated providers, multiple asset modalities, provider-specific settings, and parallel execution across mixed jobs.
+Ploof is an npm-published CLI for generating, editing, and processing assets through AI generation providers. It starts with OpenAI image, video, and audio generation/processing, but the architecture must support multiple authenticated providers, multiple asset modalities, provider-specific settings, and parallel execution across mixed jobs.
 
 The product should feel like a small, sharp developer tool: easy to run manually, predictable in scripts, and optimized for AI agents.
 
@@ -87,6 +87,19 @@ Initial capabilities:
 - `image.generate`
 - `image.edit`
 - `image.variation`
+- `video.generate`
+- `video.edit`
+- `video.extend`
+- `video.remix`
+- `video.status`
+- `video.download`
+- `video.list`
+- `video.delete`
+- `video.character.create`
+- `video.character.get`
+- `audio.generate`
+- `audio.transcribe`
+- `audio.translate`
 
 Future providers should be added through the provider registry without changing the manifest model.
 
@@ -229,6 +242,144 @@ authenticated project has DALL-E 2 variation access; if OpenAI returns a 404,
 use `ploof image edit` for image-to-image workflows. `ploof image variations`
 is an alias.
 
+### Video Generation
+
+OpenAI video generation uses the asynchronous Videos API. `ploof video generate`
+submits a job; passing `--out` or `--download` makes Ploof poll until a terminal
+status and download the requested asset.
+
+```bash
+ploof video generate \
+  --provider openai \
+  --prompt "Wide tracking shot of a paper city at blue hour" \
+  --model sora-2 \
+  --size 1280x720 \
+  --seconds 4 \
+  --out assets/clip.mp4 \
+  --output json
+```
+
+First-class OpenAI video flags:
+
+- `--model <model>`
+- `--size <size>`
+- `--seconds <seconds>`
+- `--input-reference <path-or-url-or-file-id>`
+- `--input-reference-file-id <id>`
+- `--input-reference-url <url>`
+- `--character <id>`
+- `--wait`
+- `--download`
+- `--variant video|thumbnail|spritesheet`
+- `--poll-interval <seconds>`
+- `--timeout <seconds>`
+- `--param key=value`
+- `--json '{...}'`
+
+OpenAI video generation defaults to `sora-2` when no model is specified.
+
+### Video Editing And Library
+
+```bash
+ploof video edit \
+  --video-id video_abc123 \
+  --prompt "Shift the palette to teal and rust" \
+  --out assets/edit.mp4
+
+ploof video extend \
+  --video-id video_abc123 \
+  --prompt "Continue as the camera rises over the rooftops" \
+  --seconds 4 \
+  --out assets/extended.mp4
+
+ploof video download video_abc123 --variant thumbnail --out assets/thumb.webp
+ploof video status video_abc123 --output json
+ploof video list --limit 20 --output json
+ploof video delete video_abc123
+ploof video character create --name Mossy --video character.mp4 --output json
+ploof video character get char_abc123 --output json
+```
+
+Video edits accept either `--video-id <id>` for an existing completed OpenAI
+video or `--video <path>` for an uploaded source video when the authenticated
+project is eligible for that workflow. Extensions accept a source video id or
+upload, plus a prompt and `--seconds`. `video remix` is supported for the SDK's
+legacy remix endpoint, but new integrations should prefer `video edit`.
+
+### Audio Generation And Processing
+
+OpenAI audio generation uses the speech API and defaults to
+`gpt-4o-mini-tts`, `alloy`, and `mp3` when model, voice, and output format are
+omitted.
+
+```bash
+ploof audio generate \
+  --provider openai \
+  --text "Short narration for the generated asset." \
+  --model gpt-4o-mini-tts \
+  --voice alloy \
+  --format mp3 \
+  --out assets/narration.mp3 \
+  --output json
+```
+
+First-class OpenAI audio generation flags:
+
+- `--model <model>`
+- `--voice <voice>`
+- `--voice-id <id>`
+- `--instructions <text>`
+- `--format <format>` / `--response-format <format>`
+- `--speed <number>`
+- `--param key=value`
+- `--json '{...}'`
+
+Audio processing supports transcription and English translation:
+
+```bash
+ploof audio transcribe \
+  --audio assets/narration.mp3 \
+  --model gpt-4o-mini-transcribe \
+  --out assets/transcript.json \
+  --output json
+
+ploof audio translate \
+  --audio assets/spanish.mp3 \
+  --model whisper-1 \
+  --format text \
+  --out assets/translation.txt \
+  --output json
+```
+
+Transcription first-class flags:
+
+- `--model <model>`
+- `--language <code>`
+- `--prompt <prompt>`
+- `--format <format>` / `--response-format <format>`
+- `--temperature <number>`
+- `--include <value>`
+- `--timestamp-granularity word|segment`
+- `--chunking-strategy auto|{...}`
+- `--known-speaker-name <name>`
+- `--known-speaker-reference <data-url>`
+- `--param key=value`
+- `--json '{...}'`
+
+Translation first-class flags:
+
+- `--model <model>`
+- `--prompt <prompt>`
+- `--format <format>` / `--response-format <format>`
+- `--temperature <number>`
+- `--param key=value`
+- `--json '{...}'`
+
+Ploof is a static asset generation CLI. Audio commands request complete outputs
+and write them to disk. Streaming transport settings such as OpenAI
+`stream=true` for transcription or `stream_format=sse` for speech are rejected
+because they do not directly produce finished asset files.
+
 ### Batch Run
 
 ```bash
@@ -270,6 +421,39 @@ tasks:
       images:
         - task: base
     output: assets/variation.png
+
+  - id: clip
+    kind: video.generate
+    provider: openai
+    prompt: "Slow dolly shot through a miniature paper city"
+    params:
+      model: sora-2
+      size: 1280x720
+      seconds: "4"
+    wait: true
+    download: true
+    output: assets/clip.mp4
+
+  - id: narration
+    kind: audio.generate
+    provider: openai
+    text: "Short narration for the generated clip."
+    params:
+      model: gpt-4o-mini-tts
+      voice: alloy
+      response_format: mp3
+    output: assets/narration.mp3
+
+  - id: transcript
+    kind: audio.transcribe
+    provider: openai
+    needs: [narration]
+    inputs:
+      audio:
+        task: narration
+    params:
+      model: gpt-4o-mini-transcribe
+    output: assets/transcript.json
 ```
 
 ## Asset Input Model
@@ -297,6 +481,15 @@ OpenAI image editing maps:
 - `role=image` to image input files.
 - `role=mask` to mask file.
 
+OpenAI video generation/editing maps:
+
+- `role=reference` to `input_reference` for image-guided video generation.
+- `role=video` to source video uploads for eligible edit/extension workflows.
+
+OpenAI audio processing maps:
+
+- `role=audio` to the uploaded audio file for transcription or translation.
+
 Future providers can map roles such as `reference`, `style`, `init-image`, `audio`, or `video` differently.
 
 ## Provider Architecture
@@ -310,6 +503,19 @@ type Provider = {
   runImageGenerate(job, context): Promise<ProviderResult>
   runImageEdit(job, context): Promise<ProviderResult>
   runImageVariation(job, context): Promise<ProviderResult>
+  runVideoGenerate(job, context): Promise<ProviderResult>
+  runVideoEdit(job, context): Promise<ProviderResult>
+  runVideoExtend(job, context): Promise<ProviderResult>
+  runVideoRemix(job, context): Promise<ProviderResult>
+  runVideoStatus(job, context): Promise<ProviderResult>
+  runVideoDownload(job, context): Promise<ProviderResult>
+  runVideoList(job, context): Promise<ProviderResult>
+  runVideoDelete(job, context): Promise<ProviderResult>
+  runVideoCharacterCreate(job, context): Promise<ProviderResult>
+  runVideoCharacterGet(job, context): Promise<ProviderResult>
+  runAudioGenerate(job, context): Promise<ProviderResult>
+  runAudioTranscribe(job, context): Promise<ProviderResult>
+  runAudioTranslate(job, context): Promise<ProviderResult>
 }
 ```