vidpipe 1.3.7 → 1.3.9

package/README.md

@@ -9,16 +9,16 @@
    ╚═══╝  ╚═╝╚═════╝ ╚═╝     ╚═╝╚═╝     ╚══════╝
 ```
 
-**Your AI video editor — turn raw recordings into shorts, reels, captions, social posts, and blog posts. Record once, publish everywhere.**
+**Your AI video editor and content ideation engine — turn raw recordings into shorts, reels, captions, social posts, and blog posts. Ideate, record, edit, publish.**
 
-An agentic video editor that watches for new recordings and edits them into social-media-ready content — shorts, reels, captions, blog posts, and platform-tailored social posts — using [GitHub Copilot SDK](https://github.com/github/copilot-sdk) AI agents and OpenAI Whisper.
+An agentic video editor and content ideation platform that watches for new recordings and edits them into social-media-ready content — shorts, reels, captions, blog posts, and platform-tailored social posts — using [GitHub Copilot SDK](https://github.com/github/copilot-sdk) AI agents, OpenAI Whisper, and Google Gemini.
 
 [![CI](https://github.com/htekdev/vidpipe/actions/workflows/ci.yml/badge.svg)](https://github.com/htekdev/vidpipe/actions/workflows/ci.yml)
 [![npm version](https://img.shields.io/npm/v/vidpipe)](https://www.npmjs.com/package/vidpipe)
 [![Node.js 20+](https://img.shields.io/badge/node-20%2B-brightgreen)](https://nodejs.org/)
 [![License: ISC](https://img.shields.io/badge/license-ISC-blue)](./LICENSE)
 [![Docs](https://img.shields.io/badge/docs-vidpipe-a78bfa)](https://htekdev.github.io/vidpipe/)
-[![Last Updated](https://img.shields.io/badge/last_updated-February_2026-informational)](.)
+[![Last Updated](https://img.shields.io/badge/last_updated-March_2026-informational)](.)
 
 </div>
 
@@ -38,28 +38,32 @@ npm install -g vidpipe
 
 <table>
   <tr>
+    <td>💡 <b>Content Ideation (ID8)</b> — AI-generated, trend-backed video ideas</td>
     <td>🎙️ <b>Whisper Transcription</b> — Word-level timestamps</td>
-    <td>📐 <b>Split-Screen Layouts</b> — Portrait, square, and feed</td>
   </tr>
   <tr>
+    <td>📐 <b>Split-Screen Layouts</b> — Portrait, square, and feed</td>
     <td>🔇 <b>AI Silence Removal</b> — Context-aware, capped at 20%</td>
+  </tr>
+  <tr>
     <td>💬 <b>Karaoke Captions</b> — Word-by-word highlighting</td>
+    <td>✂️ <b>Short Clips</b> — Best 15–60s moments, hook-first ordering</td>
   </tr>
   <tr>
-    <td>✂️ <b>Short Clips</b> — Best 15–60s moments, multi-segment</td>
     <td>🎞️ <b>Medium Clips</b> — 1–3 min with crossfade transitions</td>
+    <td>📑 <b>Chapter Detection</b> — JSON, Markdown, YouTube, FFmeta</td>
   </tr>
   <tr>
-    <td>📑 <b>Chapter Detection</b> — JSON, Markdown, YouTube, FFmeta</td>
     <td>📱 <b>Social Posts</b> — TikTok, YouTube, Instagram, LinkedIn, X</td>
+    <td>📰 <b>Blog Post</b> — Dev.to style with web-sourced links</td>
   </tr>
   <tr>
-    <td>📰 <b>Blog Post</b> — Dev.to style with web-sourced links</td>
     <td>🎨 <b>Brand Voice</b> — Custom tone, hashtags via brand.json</td>
+    <td>🔍 <b>Face Detection</b> — ONNX-based webcam cropping</td>
   </tr>
   <tr>
-    <td>🔍 <b>Face Detection</b> — ONNX-based webcam cropping</td>
-    <td>🚀 <b>Auto-Publish</b> — Scheduled posting to TikTok, YouTube, Instagram, LinkedIn, X</td>
+    <td>🚀 <b>Auto-Publish</b> — Scheduled posting via Late API</td>
+    <td>👁️ <b>Gemini Vision</b> — AI video analysis and scene detection</td>
   </tr>
 </table>
 
@@ -92,6 +96,9 @@ vidpipe --watch-dir ~/Videos/Recordings
 # Generate a saved idea bank for future recordings
 vidpipe ideate --topics "GitHub Copilot, Azure, TypeScript" --count 4
 
+# Add a single idea with AI enrichment
+vidpipe ideate --add --topic "Building CI/CD with GitHub Actions"
+
 # Full example with options
 vidpipe \
   --watch-dir ~/Videos/Recordings \
@@ -118,28 +125,57 @@ vidpipe [options] [video-path]
 vidpipe init              # Interactive setup wizard
 vidpipe review            # Open post review web app
 vidpipe schedule          # View posting schedule
+vidpipe realign           # Realign scheduled posts to match schedule.json
 vidpipe ideate            # Generate or list saved content ideas
+vidpipe chat              # Interactive schedule management agent
+vidpipe doctor            # Check all prerequisites
 ```
 
+### Process Options
+
 | Option | Description |
 |--------|-------------|
-| `--doctor` | Check that all prerequisites (FFmpeg, API keys, etc.) are installed and configured |
 | `[video-path]` | Process a specific video file (implies `--once`) |
 | `--watch-dir <path>` | Folder to watch for new recordings |
 | `--output-dir <path>` | Output directory (default: `./recordings`) |
 | `--openai-key <key>` | OpenAI API key |
 | `--exa-key <key>` | Exa AI key for web search in social posts |
 | `--brand <path>` | Path to `brand.json` (default: `./brand.json`) |
+| `--ideas <ids>` | Comma-separated idea IDs to link to this video |
 | `--once` | Process next video and exit |
 | `--no-silence-removal` | Skip silence removal |
 | `--no-shorts` | Skip short clip extraction |
 | `--no-medium-clips` | Skip medium clip generation |
 | `--no-social` | Skip social media posts |
 | `--no-social-publish` | Skip social media queue-build stage |
-| `--late-api-key <key>` | Override Late API key |
 | `--no-captions` | Skip caption generation/burning |
 | `--no-git` | Skip git commit/push |
+| `--late-api-key <key>` | Override Late API key |
 | `-v, --verbose` | Debug-level logging |
+| `--doctor` | Check that all prerequisites are installed |
+
+### Ideate Options
+
+| Option | Description |
+|--------|-------------|
+| `--topics <topics>` | Comma-separated seed topics for trend research |
+| `--count <n>` | Number of ideas to generate (default: 5) |
+| `--list` | List existing ideas instead of generating |
+| `--status <status>` | Filter by status: `draft`, `ready`, `recorded`, `published` |
+| `--format <format>` | Output format: `table` (default) or `json` |
+| `--output <dir>` | Ideas directory (default: `./ideas`) |
+| `--brand <path>` | Brand config path (default: `./brand.json`) |
+| `--add` | Create a single idea (AI-enriched by default) |
+| `--topic <topic>` | Topic for the idea (required with `--add`) |
+| `--hook <hook>` | Opening hook (AI-generated if omitted) |
+| `--audience <audience>` | Target audience (default: `"developers"`) |
+| `--platforms <list>` | Comma-separated platforms: `youtube,tiktok,instagram,linkedin,x` |
+| `--key-takeaway <msg>` | Core message (AI-generated if omitted) |
+| `--talking-points <list>` | Comma-separated talking points |
+| `--tags <list>` | Comma-separated categorization tags |
+| `--publish-by <date>` | Publish-by date (default: 14 days from now) |
+| `--trend-context <text>` | Trend research context |
+| `--no-ai` | Skip AI research agent, use CLI values + defaults |
 
 ---
 
@@ -190,6 +226,69 @@ recordings/
 
 ---
 
+## 💡 Content Ideation (ID8)
+
+VidPipe includes a research-backed content ideation engine that generates video ideas before you record. Ideas are stored as GitHub Issues for full lifecycle tracking.
+
+```bash
+# Generate ideas backed by trend research
+vidpipe ideate --topics "GitHub Copilot, TypeScript" --count 4
+
+# List all saved ideas
+vidpipe ideate --list
+
+# Filter by status
+vidpipe ideate --list --status ready
+
+# JSON output for programmatic access (e.g., VidRecord integration)
+vidpipe ideate --list --format json
+
+# Link ideas to a recording
+vidpipe process video.mp4 --ideas 12,15
+```
+
+### Manual Idea Creation
+
+Add a single idea with AI enrichment or direct CLI values:
+
+```bash
+# AI-researched — full IdeationAgent with MCP research tools
+vidpipe ideate --add --topic "Building CI/CD with GitHub Actions"
+
+# Direct — skip AI, use CLI flags + defaults
+vidpipe ideate --add --topic "Quick Demo" --no-ai --hook "Ship it live" --audience "developers"
+
+# JSON output for programmatic consumers (e.g., VidRecord Electron app)
+vidpipe ideate --add --topic "My Topic" --format json
+```
+
+### How It Works
+
+The **IdeationAgent** uses MCP tools (Exa web search, YouTube, Perplexity) to research trending topics in your niche before generating ideas. Each idea includes:
+
+- **Topic & hook** — The angle that makes it compelling
+- **Audience & key takeaway** — Who it's for and what they'll learn
+- **Talking points** — Structured bullet points to guide your recording
+- **Publish-by date** — Based on timeliness (3–5 days for hot trends, months for evergreen)
+- **Trend context** — The research findings that back the idea
+
+### Idea Lifecycle
+
+```
+draft → ready → recorded → published
+```
+
+| Status | Meaning |
+|--------|---------|
+| `draft` | Generated by AI, awaiting your review |
+| `ready` | Approved — ready to record |
+| `recorded` | Linked to a video via `--ideas` flag |
+| `published` | Content from this idea has been published |
+
+Ideas automatically influence downstream content — when you link ideas to a recording with `--ideas`, the pipeline's agents (shorts, social posts, summaries, blog) reference your intended topic and hook for more focused output.
+
+---
+
 ## 📺 Review App
 
 VidPipe includes a built-in web app for reviewing, editing, and scheduling social media posts before publishing.
@@ -290,6 +389,8 @@ OUTPUT_DIR=/path/to/output
 # FFMPEG_PATH=/usr/local/bin/ffmpeg
 # FFPROBE_PATH=/usr/local/bin/ffprobe
 # LATE_API_KEY=sk_your_key_here   # Optional: Late API for social publishing
+# GITHUB_TOKEN=ghp_...            # Optional: GitHub token for ID8 idea storage
+# IDEAS_REPO=owner/repo           # Optional: GitHub repo for storing ideas as Issues
 ```
 
 Social media publishing is configured via `schedule.json` and the Late API. See [Social Publishing Guide](./docs/social-publishing.md) for details.
@@ -305,12 +406,29 @@ Social media publishing is configured via `schedule.json` and the Late API. See
 | [FFmpeg Setup](./docs/ffmpeg-setup.md) | Platform-specific install (Windows, macOS, Linux, ARM64) |
 | [Brand Customization](./docs/brand-customization.md) | Customize AI voice, vocabulary, hashtags, and content style |
 | [Social Publishing](./docs/social-publishing.md) | Review, schedule, and publish social posts via Late API |
+| [Architecture (L0–L7)](./docs/architecture/layers.md) | Layer hierarchy, import rules, and testing strategy |
+| [Platform Content Strategy](./docs/platform-content-strategy.md) | Research-backed recommendations per social platform |
+
+Full reference docs are available at [htekdev.github.io/vidpipe](https://htekdev.github.io/vidpipe/).
 
 ---
 
 ## 🏗️ Architecture
 
-Agentic architecture built on the [GitHub Copilot SDK](https://github.com/github/copilot-sdk) — each editing task is handled by a specialized AI agent:
+VidPipe uses a strict **L0–L7 layered architecture** where each layer can only import from specific lower layers. This enforces clean separation of concerns and makes every layer independently testable.
+
+```
+L7-app         CLI, servers, watchers          → L0, L1, L3, L6
+L6-pipeline    Stage orchestration             → L0, L1, L5
+L5-assets      Lazy-loaded asset + bridges     → L0, L1, L4
+L4-agents      LLM agents (BaseAgent)          → L0, L1, L3
+L3-services    Business logic + cost tracking  → L0, L1, L2
+L2-clients     External API/process wrappers   → L0, L1
+L1-infra       Infrastructure (config, logger) → L0
+L0-pure        Pure functions, zero I/O        → (nothing)
+```
+
+Each editing task is handled by a specialized AI agent built on the [GitHub Copilot SDK](https://github.com/github/copilot-sdk):
 
 ```mermaid
 graph TD
@@ -321,6 +439,7 @@ graph TD
     BP --> CA[ChapterAgent]
     BP --> SMA[SocialMediaAgent]
     BP --> BA[BlogAgent]
+    BP --> IA[IdeationAgent]
 
     SRA -->|tools| T1[detect_silence, decide_removals]
     SHA -->|tools| T2[plan_shorts]
@@ -329,11 +448,13 @@ graph TD
     SA -->|tools| T5[capture_frame, write_summary]
     SMA -->|tools| T6[search_links, create_posts]
     BA -->|tools| T7[search_web, write_blog]
+    IA -->|tools| T8[web_search, youtube_search, generate_ideas]
 
     style BP fill:#1e3a5f,stroke:#60a5fa,color:#fff
+    style IA fill:#5a4d27,stroke:#fbbf24,color:#fff
 ```
 
-Each agent communicates with the LLM through structured tool calls, ensuring reliable, parseable outputs.
+Each agent communicates with the LLM through structured tool calls, ensuring reliable, parseable outputs. See the [Architecture Guide](./docs/architecture/layers.md) for full details on layer rules and import enforcement.
 
 ---
 
@@ -344,23 +465,28 @@ Each agent communicates with the LLM through structured tool calls, ensuring rel
 | [TypeScript](https://www.typescriptlang.org/) | Language (ES2022, ESM) |
 | [GitHub Copilot SDK](https://github.com/github/copilot-sdk) | AI agent framework |
 | [OpenAI Whisper](https://platform.openai.com/docs/guides/speech-to-text) | Speech-to-text |
+| [Google Gemini](https://ai.google.dev/) | Vision-based video analysis |
 | [FFmpeg](https://ffmpeg.org/) | Video/audio processing |
 | [Sharp](https://sharp.pixelplumbing.com/) | Image analysis (webcam detection) |
+| [Octokit](https://github.com/octokit/octokit.js) | GitHub API (idea storage as Issues) |
 | [Commander.js](https://github.com/tj/commander.js) | CLI framework |
 | [Chokidar](https://github.com/paulmillr/chokidar) | File system watching |
 | [Winston](https://github.com/winstonjs/winston) | Logging |
-| [Exa AI](https://exa.ai/) | Web search for social posts and blog |
+| [Exa AI](https://exa.ai/) | Web search for social posts, blog, and ideation |
 
 ---
 
 ## 🗺️ Roadmap
 
 - [x] **Automated social posting** — Publish directly to platforms via Late API
+- [x] **Content ideation (ID8)** — AI-generated, trend-backed video ideas with lifecycle tracking
+- [x] **Gemini Vision integration** — AI-powered video analysis and scene detection
+- [x] **L0–L7 layered architecture** — Strict separation of concerns with import enforcement
+- [x] **GitHub agentic workflows** — Automated issue and PR triage via GitHub Actions
+- [x] **Hook-first clip ordering** — Most engaging moment plays first in shorts
 - [ ] **Multi-language support** — Transcription and summaries in multiple languages
 - [ ] **Custom templates** — User-defined Markdown & social post templates
-- [ ] **Web dashboard** — Browser UI for reviewing and editing outputs
 - [ ] **Batch processing** — Process an entire folder of existing videos
-- [ ] **Custom short criteria** — Configure what makes a "good" short for your content
 - [ ] **Thumbnail generation** — Auto-generate branded thumbnails for shorts
 
 ---

package/dist/cli.js

@@ -11557,12 +11557,13 @@ function isStringArray(value) {
 function hasField(source, field) {
   return Object.prototype.hasOwnProperty.call(source, field);
 }
-function normalizeCount(count) {
+function normalizeCount(count, allowSingle) {
+  const min = allowSingle ? 1 : MIN_IDEA_COUNT;
   if (typeof count !== "number" || Number.isNaN(count)) {
-    return MIN_IDEA_COUNT;
+    return min;
   }
   const rounded = Math.round(count);
-  return Math.min(MAX_IDEA_COUNT, Math.max(MIN_IDEA_COUNT, rounded));
+  return Math.min(MAX_IDEA_COUNT, Math.max(min, rounded));
 }
 function normalizeSeedTopics(seedTopics) {
   return (seedTopics ?? []).map((topic) => topic.trim()).filter((topic) => topic.length > 0);
@@ -12207,7 +12208,7 @@ var IdeationAgent = class extends BaseAgent {
 };
 async function generateIdeas(options = {}) {
   const seedTopics = normalizeSeedTopics(options.seedTopics);
-  const count = normalizeCount(options.count);
+  const count = normalizeCount(options.count, options.singleTopic);
   const config2 = getConfig();
   const previousBrandPath = config2.BRAND_PATH;
   if (options.brandPath) {
@@ -13161,11 +13162,30 @@ function generateIdeas3(...args) {
 }
 
 // src/L7-app/commands/ideate.ts
+init_types();
+var VALID_PLATFORMS = new Set(Object.values(Platform));
 async function runIdeate(options = {}) {
   initConfig();
+  if (options.add) {
+    await handleAdd(options);
+    return;
+  }
   if (options.list) {
     const ideas2 = await listIdeas();
     const filtered = options.status ? ideas2.filter((idea) => idea.status === options.status) : ideas2;
+    if (options.format === "json") {
+      const jsonIdeas = filtered.map((idea) => ({
+        issueNumber: idea.issueNumber,
+        id: idea.id,
+        topic: idea.topic,
+        hook: idea.hook,
+        audience: idea.audience,
+        platforms: idea.platforms,
+        status: idea.status
+      }));
+      console.log(JSON.stringify(jsonIdeas, null, 2));
+      return;
+    }
     if (filtered.length === 0) {
       console.log("No ideas found.");
       if (options.status) {
@@ -13188,12 +13208,14 @@ ${filtered.length} idea(s) total`);
   }
   const seedTopics = options.topics?.split(",").map((t) => t.trim()).filter(Boolean);
   const count = options.count ? parseInt(options.count, 10) : 5;
-  console.log("\n\u{1F9E0} Generating content ideas...\n");
-  if (seedTopics?.length) {
-    console.log(`Seed topics: ${seedTopics.join(", ")}`);
-  }
-  console.log(`Target count: ${count}
+  if (options.format !== "json") {
+    console.log("\n\u{1F9E0} Generating content ideas...\n");
+    if (seedTopics?.length) {
+      console.log(`Seed topics: ${seedTopics.join(", ")}`);
+    }
+    console.log(`Target count: ${count}
 `);
+  }
   const ideas = await generateIdeas3({
     seedTopics,
     count,
@@ -13201,7 +13223,24 @@ ${filtered.length} idea(s) total`);
     brandPath: options.brand
   });
   if (ideas.length === 0) {
-    console.log("No ideas were generated. Check your API key configuration.");
+    if (options.format === "json") {
+      console.log(JSON.stringify([], null, 2));
+    } else {
+      console.log("No ideas were generated. Check your API key configuration.");
+    }
+    return;
+  }
+  if (options.format === "json") {
+    const jsonIdeas = ideas.map((idea) => ({
+      issueNumber: idea.issueNumber,
+      id: idea.id,
+      topic: idea.topic,
+      hook: idea.hook,
+      audience: idea.audience,
+      platforms: idea.platforms,
+      status: idea.status
+    }));
+    console.log(JSON.stringify(jsonIdeas, null, 2));
     return;
   }
   console.log(`
@@ -13219,6 +13258,71 @@ ${filtered.length} idea(s) total`);
   console.log("Use `vidpipe ideate --list` to view all ideas.");
   console.log("Use `vidpipe process video.mp4 --ideas <issueNumber1>,<issueNumber2>` to link ideas to a recording.");
 }
+function parseCommaSeparated(value) {
+  if (!value) return [];
+  return value.split(",").map((s) => s.trim()).filter(Boolean);
+}
+function parsePlatforms(value) {
+  if (!value) return ["youtube" /* YouTube */];
+  const names = parseCommaSeparated(value);
+  const platforms = [];
+  for (const name of names) {
+    const lower = name.toLowerCase();
+    if (!VALID_PLATFORMS.has(lower)) {
+      throw new Error(`Invalid platform "${name}". Valid platforms: ${[...VALID_PLATFORMS].join(", ")}`);
+    }
+    platforms.push(lower);
+  }
+  return platforms.length > 0 ? platforms : ["youtube" /* YouTube */];
+}
+function defaultPublishBy() {
+  const date = /* @__PURE__ */ new Date();
+  date.setDate(date.getDate() + 14);
+  return date.toISOString().split("T")[0];
+}
+function buildDirectInput(options) {
+  const topic = options.topic;
+  const hook = options.hook ?? topic;
+  const audience = options.audience ?? "developers";
+  const platforms = parsePlatforms(options.platforms);
+  const keyTakeaway = options.keyTakeaway ?? hook;
+  const talkingPoints = parseCommaSeparated(options.talkingPoints);
+  const tags = parseCommaSeparated(options.tags);
+  const publishBy = options.publishBy ?? defaultPublishBy();
+  const trendContext = options.trendContext;
+  return { topic, hook, audience, keyTakeaway, talkingPoints, platforms, tags, publishBy, trendContext };
+}
+async function handleAdd(options) {
+  if (!options.topic) {
+    throw new Error("--topic is required when using --add");
+  }
+  const useAI = options.ai !== false;
+  if (useAI) {
+    const ideas = await generateIdeas3({
+      seedTopics: [options.topic],
+      count: 1,
+      singleTopic: true,
+      brandPath: options.brand
+    });
+    const idea = ideas[0];
+    if (!idea) {
+      throw new Error("IdeationAgent did not create an idea");
+    }
+    if (options.format === "json") {
+      console.log(JSON.stringify(idea, null, 2));
+    } else {
+      console.log(`Created idea #${idea.issueNumber}: "${idea.topic}"`);
+    }
+  } else {
+    const input = buildDirectInput(options);
+    const idea = await createIdea(input);
+    if (options.format === "json") {
+      console.log(JSON.stringify(idea, null, 2));
+    } else {
+      console.log(`Created idea #${idea.issueNumber}: "${idea.topic}"`);
+    }
+  }
+}
 
 // src/L1-infra/http/http.ts
 import { default as default8 } from "express";
@@ -13853,7 +13957,7 @@ program.command("chat").description("Interactive chat session with the schedule
 program.command("doctor").description("Check all prerequisites and dependencies").action(async () => {
   await runDoctor();
 });
-program.command("ideate").description("Generate AI-powered content ideas using trend research").option("--topics <topics>", "Comma-separated seed topics").option("--count <n>", "Number of ideas to generate (default: 5)", "5").option("--output <dir>", "Ideas directory (default: ./ideas)").option("--brand <path>", "Brand config path (default: ./brand.json)").option("--list", "List existing ideas instead of generating").option("--status <status>", "Filter by status when listing (draft|ready|recorded|published)").action(async (opts) => {
+program.command("ideate").description("Generate AI-powered content ideas using trend research").option("--topics <topics>", "Comma-separated seed topics").option("--count <n>", "Number of ideas to generate (default: 5)", "5").option("--output <dir>", "Ideas directory (default: ./ideas)").option("--brand <path>", "Brand config path (default: ./brand.json)").option("--list", "List existing ideas instead of generating").option("--status <status>", "Filter by status when listing (draft|ready|recorded|published)").option("--format <format>", "Output format: table (default) or json").option("--add", "Add a single idea (AI-researched by default, or --no-ai for direct)").option("--topic <topic>", "Idea topic/title (required with --add)").option("--hook <hook>", "Attention-grabbing hook (default: topic, --no-ai only)").option("--audience <audience>", "Target audience (default: developers, --no-ai only)").option("--platforms <platforms>", "Comma-separated platforms: tiktok,youtube,instagram,linkedin,x (--no-ai only)").option("--key-takeaway <takeaway>", "Core message the viewer should remember (--no-ai only)").option("--talking-points <points>", "Comma-separated talking points (--no-ai only)").option("--tags <tags>", "Comma-separated categorization tags (--no-ai only)").option("--publish-by <date>", "Publish deadline (ISO 8601 date, default: 14 days from now, --no-ai only)").option("--trend-context <context>", "Why this topic is timely (--no-ai only)").option("--no-ai", "Skip AI research agent \u2014 create directly from CLI flags + defaults").action(async (opts) => {
   initConfig();
   await runIdeate(opts);
   process.exit(0);