@renoise/video-maker 0.1.3 → 0.2.1

package/skills/gemini-gen/SKILL.md

@@ -0,0 +1,236 @@
+---
+name: gemini-gen
+description: >
+  Visual understanding and multimodal analysis via Gemini 3.1 Pro.
+  Use when you need to understand, analyze, or extract information from images or videos:
+  analyze product photos, extract video scripts/dialogue, understand video content for replication,
+  compare visual assets, OCR/text extraction from images, describe scenes for prompt writing,
+  extract style/color/composition from reference footage.
+  Do NOT use for generating images or videos — use renoise-gen instead.
+allowed-tools: Bash, Read
+metadata:
+  author: renoise
+  version: 0.2.0
+  category: ai-foundation
+  tags: [vision, analysis, multimodal, gemini]
+---
+
+# Gemini Gen — Visual Understanding & Multimodal Analysis
+
+Gemini 3.1 Pro via Renoise gateway. Zero npm dependencies, native `fetch` only.
+
+## When to Use
+
+| Scenario                        | Example                                                                                            |
+| ------------------------------- | -------------------------------------------------------------------------------------------------- |
+| **Analyze product photos**      | Extract type, color, material, selling points, brand tone from product images                      |
+| **Understand video content**    | "What happens in this video?", summarize scenes, identify actions and objects                      |
+| **Extract scripts from video**  | Watch a reference video → output timestamped dialogue, scene descriptions, camera movements        |
+| **Replicate a video style**     | Analyze a reference clip → extract visual style, pacing, transitions, color grading for recreation |
+| **Compare visual assets**       | Side-by-side analysis of two product photos, before/after comparison                               |
+| **OCR / text extraction**       | Read text from screenshots, packaging, signage in images                                           |
+| **Describe scenes for prompts** | Look at a reference image → write a detailed prompt for `renoise-gen` to recreate the style        |
+| **Content review**              | Check if generated output matches the creative brief                                               |
+
+## When NOT to Use
+
+- **Generating images** → use `renoise-gen` with `nano-banana-2`
+- **Generating videos** → use `renoise-gen` with `renoise-2.0`
+- **Uploading large files** → use `file-upload` first, then pass the URL here
+
+## Authentication
+
+Use environment variable `RENOISE_API_KEY`. Get one at: https://www.renoise.ai
+
+## API Endpoint
+
+```
+POST https://renoise.ai/api/public/llm/proxy/v1beta/models/{model}:generateContent?key={RENOISE_API_KEY}
+```
+
+Default model: `gemini-3.1-pro`
+
+## Request Format
+
+```json
+{
+  "contents": [
+    {
+      "role": "user",
+      "parts": [{ "text": "your prompt here" }]
+    }
+  ],
+  "generationConfig": {
+    "temperature": 1.0,
+    "maxOutputTokens": 8192
+  }
+}
+```
+
+## Multimodal Input (images & videos)
+
+Gemini supports images and videos as inline base64 parts alongside text. Multiple files can be sent in the same request.
+
+### Image Input
+
+```json
+{
+  "contents": [
+    {
+      "parts": [
+        {
+          "inlineData": {
+            "mimeType": "image/jpeg",
+            "data": "<base64-encoded-data>"
+          },
+          "mediaResolution": { "level": "media_resolution_high" }
+        },
+        { "text": "Describe this image" }
+      ]
+    }
+  ]
+}
+```
+
+`mediaResolution` controls token allocation per image/frame:
+
+| Level                         | Image Tokens | Video Frame Tokens |
+| ----------------------------- | ------------ | ------------------ |
+| `media_resolution_low`        | 280          | 70                 |
+| `media_resolution_medium`     | 560          | 140                |
+| `media_resolution_high`       | 840          | 210                |
+| `media_resolution_ultra_high` | 1120         | 280                |
+
+Default: `media_resolution_medium`. Use `high` or `ultra_high` for detail-critical analysis (product photos, fine text). Use `low` for bulk/batch processing.
+
+### Video Input
+
+Same format — just use a video MIME type. Inline base64 has a **20MB limit**. For larger videos, use the dedicated file upload skill.
+
+```json
+{
+  "contents": [
+    {
+      "parts": [
+        {
+          "inlineData": {
+            "mimeType": "video/mp4",
+            "data": "<base64-encoded-data>"
+          },
+          "mediaResolution": { "level": "media_resolution_low" }
+        },
+        { "text": "Summarize what happens in this video" }
+      ]
+    }
+  ]
+}
+```
+
+> **Tip**: Use `media_resolution_low` for videos to reduce token consumption — video has many frames.
+
+### Multiple Files
+
+Send multiple images/videos in one request by adding more `inlineData` parts:
+
+```json
+{
+  "contents": [
+    {
+      "parts": [
+        {
+          "inlineData": { "mimeType": "image/jpeg", "data": "<base64-image-1>" }
+        },
+        {
+          "inlineData": { "mimeType": "image/png", "data": "<base64-image-2>" }
+        },
+        { "text": "Compare these two product photos" }
+      ]
+    }
+  ]
+}
+```
+
+### Large Files (> 20MB)
+
+Inline base64 has a **20MB limit**. For larger files, use the dedicated file upload skill to get a file URI, then reference it:
+
+```json
+{
+  "contents": [
+    {
+      "parts": [
+        {
+          "fileData": {
+            "mimeType": "video/mp4",
+            "fileUri": "<uploaded-file-uri>"
+          }
+        },
+        { "text": "Analyze this video" }
+      ]
+    }
+  ]
+}
+```
+
+## Supported MIME Types
+
+| Extension  | MIME Type       | Max Inline |
+| ---------- | --------------- | ---------- |
+| .jpg/.jpeg | image/jpeg      | 20MB       |
+| .png       | image/png       | 20MB       |
+| .webp      | image/webp      | 20MB       |
+| .gif       | image/gif       | 20MB       |
+| .mp4       | video/mp4       | 20MB       |
+| .mov       | video/quicktime | 20MB       |
+| .webm      | video/webm      | 20MB       |
+
+## Response Parsing
+
+```javascript
+const data = await response.json();
+const text = data.candidates[0].content.parts[0].text;
+```
+
+## Error Handling
+
+- 400: Bad request (check prompt format)
+- 403: Invalid API key
+- 429: Rate limited (wait and retry)
+- 500: Server error (retry with backoff)
+
+## CLI Script
+
+`${CLAUDE_SKILL_DIR}/scripts/gemini.mjs` — zero-dependency Node.js script, other skills can directly call it.
+
+```bash
+# Text only
+node ${CLAUDE_SKILL_DIR}/scripts/gemini.mjs "Explain quantum computing"
+
+# Analyze an image (high resolution for product detail)
+node ${CLAUDE_SKILL_DIR}/scripts/gemini.mjs --file photo.jpg --resolution high "Describe this product"
+
+# Analyze a video (low resolution to save tokens)
+node ${CLAUDE_SKILL_DIR}/scripts/gemini.mjs --file clip.mp4 --resolution low "Summarize this clip"
+
+# Multiple images
+node ${CLAUDE_SKILL_DIR}/scripts/gemini.mjs --file a.jpg --file b.jpg "Compare these two"
+
+# Uploaded file URI (from file upload skill, for files > 20MB)
+node ${CLAUDE_SKILL_DIR}/scripts/gemini.mjs --file-uri "<uri>" --file-mime video/mp4 "Analyze this video"
+
+# JSON output mode
+node ${CLAUDE_SKILL_DIR}/scripts/gemini.mjs --json "Return a JSON object with name and age"
+```
+
+### Options
+
+| Flag                   | Default          | Description                                          |
+| ---------------------- | ---------------- | ---------------------------------------------------- |
+| `--file <path>`        | —                | Attach local file (repeatable)                       |
+| `--file-uri <uri>`     | —                | Attach uploaded file by URI (requires `--file-mime`) |
+| `--file-mime <mime>`   | —                | MIME type for `--file-uri`                           |
+| `--resolution <level>` | `medium`         | `low` / `medium` / `high` / `ultra_high`             |
+| `--model <name>`       | `gemini-3.1-pro` | Model name                                           |
+| `--temperature <n>`    | `1.0`            | Temperature                                          |
+| `--max-tokens <n>`     | `8192`           | Max output tokens                                    |
+| `--json`               | off              | Request JSON response format                         |

package/skills/gemini-gen/scripts/gemini.mjs

@@ -0,0 +1,220 @@
+#!/usr/bin/env node
+
+/**
+ * Gemini API client via Renoise gateway.
+ * Zero npm dependencies — uses native fetch.
+ *
+ * Usage:
+ *   # Text only
+ *   node gemini.mjs "Explain quantum computing"
+ *
+ *   # With image(s)
+ *   node gemini.mjs --file photo.jpg "Describe this product"
+ *   node gemini.mjs --file a.jpg --file b.jpg "Compare these two"
+ *
+ *   # With video
+ *   node gemini.mjs --file clip.mp4 --resolution low "Summarize this clip"
+ *
+ *   # With uploaded file URI (from file upload skill)
+ *   node gemini.mjs --file-uri "https://...fileUri" --file-mime video/mp4 "Analyze this video"
+ *
+ *   # JSON output mode
+ *   node gemini.mjs --json "Return a JSON object with name and age"
+ *
+ * Options:
+ *   --file <path>         Attach a local file (image/video). Repeatable.
+ *   --file-uri <uri>      Attach an uploaded file by URI. Requires --file-mime.
+ *   --file-mime <mime>     MIME type for --file-uri.
+ *   --resolution <level>  Media resolution: low|medium|high|ultra_high (default: medium)
+ *   --model <name>        Model name (default: gemini-3.1-pro)
+ *   --temperature <n>     Temperature (default: 1.0)
+ *   --max-tokens <n>      Max output tokens (default: 8192)
+ *   --json                Request JSON response format
+ *
+ * Environment:
+ *   RENOISE_API_KEY       Required. Get one at https://www.renoise.ai
+ */
+
+import fs from "fs/promises";
+import path from "path";
+
+// --- Auth ---
+const RENOISE_API_KEY = process.env.RENOISE_API_KEY;
+if (!RENOISE_API_KEY) {
+  console.error(
+    "RENOISE_API_KEY not set. Get one at: https://www.renoise.ai"
+  );
+  process.exit(1);
+}
+
+// --- MIME detection ---
+const MIME_MAP = {
+  ".jpg": "image/jpeg",
+  ".jpeg": "image/jpeg",
+  ".png": "image/png",
+  ".webp": "image/webp",
+  ".gif": "image/gif",
+  ".mp4": "video/mp4",
+  ".mov": "video/quicktime",
+  ".webm": "video/webm",
+};
+
+function getMimeType(filePath) {
+  return (
+    MIME_MAP[path.extname(filePath).toLowerCase()] ?? "application/octet-stream"
+  );
+}
+
+// --- Resolution mapping ---
+const RESOLUTION_MAP = {
+  low: "media_resolution_low",
+  medium: "media_resolution_medium",
+  high: "media_resolution_high",
+  ultra_high: "media_resolution_ultra_high",
+};
+
+// --- Parse args ---
+function parseArgs(argv) {
+  const files = [];
+  let fileUri = null;
+  let fileMime = null;
+  let resolution = "medium";
+  let model = "gemini-3.1-pro";
+  let temperature = 1.0;
+  let maxTokens = 8192;
+  let json = false;
+  const textParts = [];
+
+  for (let i = 0; i < argv.length; i++) {
+    switch (argv[i]) {
+      case "--file":
+        files.push(argv[++i]);
+        break;
+      case "--file-uri":
+        fileUri = argv[++i];
+        break;
+      case "--file-mime":
+        fileMime = argv[++i];
+        break;
+      case "--resolution":
+        resolution = argv[++i];
+        break;
+      case "--model":
+        model = argv[++i];
+        break;
+      case "--temperature":
+        temperature = parseFloat(argv[++i]);
+        break;
+      case "--max-tokens":
+        maxTokens = parseInt(argv[++i], 10);
+        break;
+      case "--json":
+        json = true;
+        break;
+      default:
+        textParts.push(argv[i]);
+    }
+  }
+
+  return {
+    files,
+    fileUri,
+    fileMime,
+    resolution,
+    model,
+    temperature,
+    maxTokens,
+    json,
+    prompt: textParts.join(" "),
+  };
+}
+
+// --- Build parts ---
+async function buildParts(opts) {
+  const parts = [];
+  const resLevel = RESOLUTION_MAP[opts.resolution] ?? RESOLUTION_MAP.medium;
+
+  // Local files → inline base64
+  for (const filePath of opts.files) {
+    const data = await fs.readFile(filePath);
+    parts.push({
+      inlineData: {
+        mimeType: getMimeType(filePath),
+        data: data.toString("base64"),
+      },
+      mediaResolution: { level: resLevel },
+    });
+  }
+
+  // Uploaded file URI
+  if (opts.fileUri) {
+    parts.push({
+      fileData: {
+        mimeType: opts.fileMime ?? "application/octet-stream",
+        fileUri: opts.fileUri,
+      },
+    });
+  }
+
+  // Text prompt (always last)
+  if (opts.prompt) {
+    parts.push({ text: opts.prompt });
+  }
+
+  return parts;
+}
+
+// --- Main ---
+async function main() {
+  const opts = parseArgs(process.argv.slice(2));
+
+  if (!opts.prompt && opts.files.length === 0 && !opts.fileUri) {
+    console.error(
+      "Usage: node gemini.mjs [--file <path>] [--resolution low|medium|high|ultra_high] <prompt>"
+    );
+    process.exit(1);
+  }
+
+  const endpoint = `https://renoise.ai/api/public/llm/proxy/v1beta/models/${opts.model}:generateContent?key=${RENOISE_API_KEY}`;
+
+  const parts = await buildParts(opts);
+
+  const body = {
+    contents: [{ role: "user", parts }],
+    generationConfig: {
+      temperature: opts.temperature,
+      maxOutputTokens: opts.maxTokens,
+    },
+  };
+
+  if (opts.json) {
+    body.generationConfig.responseMimeType = "application/json";
+  }
+
+  const res = await fetch(endpoint, {
+    method: "POST",
+    headers: { "Content-Type": "application/json" },
+    body: JSON.stringify(body),
+  });
+
+  if (!res.ok) {
+    const errText = await res.text();
+    console.error(`Gemini API error ${res.status}: ${errText}`);
+    process.exit(1);
+  }
+
+  const data = await res.json();
+  const text = data.candidates?.[0]?.content?.parts?.[0]?.text;
+
+  if (!text) {
+    console.error("No text in response:", JSON.stringify(data, null, 2));
+    process.exit(1);
+  }
+
+  console.log(text);
+}
+
+main().catch((err) => {
+  console.error("ERROR:", err.message);
+  process.exit(1);
+});

package/skills/renoise-gen/SKILL.md

@@ -13,6 +13,8 @@ metadata:
 
 Generate AI videos and images through the Renoise platform.
 
+> **IMPORTANT**: The Renoise website is **https://www.renoise.ai** — NOT renoise.com. Always use `renoise.ai` when referencing the platform URL.
+
 ## Supported Models
 
 | Model | Type | Description |
@@ -96,7 +98,7 @@ See `${CLAUDE_SKILL_DIR}/references/video-capabilities.md` for details.
 
 CLI path: `${CLAUDE_SKILL_DIR}/renoise-cli.mjs` (Node.js 18+)
 
-API Key and base URL are configured via environment variables (`RENOISE_API_KEY`, `RENOISE_BASE_URL`).
+API Key and base URL are configured via environment variables (`RENOISE_API_KEY`, `RENOISE_BASE_URL`). Get your API key at https://www.renoise.ai (NOT renoise.com).
 
 ## CLI Commands
 

package/skills/renoise-gen/references/api-endpoints.md

@@ -2,50 +2,51 @@
 
 ## API Endpoints
 
-Base URL: `https://staging--ujgsvru36x4korjj10nq.edgespark.app`
+Base URL: `https://renoise.ai`
 API Prefix: `/api/public/v1`
 
 All endpoints require `X-API-Key: <api_key>` header.
 
 ### Credits
 
-| Method | Path | Description |
-|--------|------|-------------|
-| GET | `/api/public/v1/me` | User info + credit balance |
-| GET | `/api/public/v1/credit/estimate?model=X&duration=Y&hasVideoRef=0` | Cost estimate |
-| GET | `/api/public/v1/credit/history?limit=50&offset=0` | Credit transactions |
+| Method | Path                                                              | Description                |
+| ------ | ----------------------------------------------------------------- | -------------------------- |
+| GET    | `/api/public/v1/me`                                               | User info + credit balance |
+| GET    | `/api/public/v1/credit/estimate?model=X&duration=Y&hasVideoRef=0` | Cost estimate              |
+| GET    | `/api/public/v1/credit/history?limit=50&offset=0`                 | Credit transactions        |
 
 ### Tasks
 
-| Method | Path | Description |
-|--------|------|-------------|
-| POST | `/api/public/v1/tasks` | Create task |
-| GET | `/api/public/v1/tasks?status=X&tag=Y&limit=50&offset=0` | List tasks |
-| GET | `/api/public/v1/tasks/:id` | Task detail |
-| GET | `/api/public/v1/tasks/:id/result` | Task result (video/image/cover URLs) |
-| POST | `/api/public/v1/tasks/:id/cancel` | Cancel pending task |
-| PATCH | `/api/public/v1/tasks/:id/tags` | Update tags |
-| GET | `/api/public/v1/tags` | List all tags |
+| Method | Path                                                    | Description                          |
+| ------ | ------------------------------------------------------- | ------------------------------------ |
+| POST   | `/api/public/v1/tasks`                                  | Create task                          |
+| GET    | `/api/public/v1/tasks?status=X&tag=Y&limit=50&offset=0` | List tasks                           |
+| GET    | `/api/public/v1/tasks/:id`                              | Task detail                          |
+| GET    | `/api/public/v1/tasks/:id/result`                       | Task result (video/image/cover URLs) |
+| POST   | `/api/public/v1/tasks/:id/cancel`                       | Cancel pending task                  |
+| PATCH  | `/api/public/v1/tasks/:id/tags`                         | Update tags                          |
+| GET    | `/api/public/v1/tags`                                   | List all tags                        |
 
 ### Materials
 
-| Method | Path | Description |
-|--------|------|-------------|
-| POST | `/api/public/v1/materials/upload` | Upload material (multipart) |
-| GET | `/api/public/v1/materials?type=X&search=Y` | List materials with download URLs |
+| Method | Path                                       | Description                       |
+| ------ | ------------------------------------------ | --------------------------------- |
+| POST   | `/api/public/v1/materials/upload`          | Upload material (multipart)       |
+| GET    | `/api/public/v1/materials?type=X&search=Y` | List materials with download URLs |
 
 ### Characters
 
-| Method | Path | Description |
-|--------|------|-------------|
-| GET | `/api/public/v1/characters?category=X&usage_group=Y&search=Z&page=1&page_size=20` | List characters |
-| GET | `/api/public/v1/characters/:id` | Character detail |
+| Method | Path                                                                              | Description      |
+| ------ | --------------------------------------------------------------------------------- | ---------------- |
+| GET    | `/api/public/v1/characters?category=X&usage_group=Y&search=Z&page=1&page_size=20` | List characters  |
+| GET    | `/api/public/v1/characters/:id`                                                   | Character detail |
 
 ## Request/Response Formats
 
 ### POST /api/public/v1/tasks
 
 Request:
+
 ```json
 {
   "prompt": "string (required)",
@@ -62,6 +63,7 @@ Request:
 ```
 
 Response (201):
+
 ```json
 {
   "task": {
@@ -76,6 +78,7 @@ Response (201):
 ```
 
 Error (402 — insufficient credits):
+
 ```json
 {
   "error": "Insufficient credits",
@@ -87,6 +90,7 @@ Error (402 — insufficient credits):
 ### GET /api/public/v1/tasks/:id/result
 
 Response:
+
 ```json
 {
   "taskId": 1,
@@ -103,16 +107,16 @@ Response:
 
 ## Task Statuses
 
-| Status | Description |
-|--------|-------------|
-| `pending` | Waiting for assignment |
-| `assigning` | Being assigned to account |
-| `assigned` | Assigned, waiting to start |
-| `queued` | Queued on provider |
-| `running` | Generating |
-| `completed` | Done — result available |
-| `failed` | Failed — check error field |
-| `cancelled` | User cancelled |
+| Status      | Description                |
+| ----------- | -------------------------- |
+| `pending`   | Waiting for assignment     |
+| `assigning` | Being assigned to account  |
+| `assigned`  | Assigned, waiting to start |
+| `queued`    | Queued on provider         |
+| `running`   | Generating                 |
+| `completed` | Done — result available    |
+| `failed`    | Failed — check error field |
+| `cancelled` | User cancelled             |
 
 ## Models