@twick/mcp-agent 0.14.15

package/README.md

@@ -0,0 +1,194 @@
+# MCP Twick Server
+
+An MCP (Model Context Protocol) server for Claude Desktop that generates video captions using Google Vertex AI (Gemini) and integrates with Twick Studio.
+
+## 📦 Installation
+
+### Quick Install (Recommended)
+
+```bash
+git clone <repository-url>
+cd twick-mcp-agent
+npm run install-claude
+```
+
+This will automatically install dependencies, build the project, and configure Claude Desktop.
+
+See [SHIPPING.md](./SHIPPING.md) for distribution options and release information.
+
+## Features
+
+- 🎬 **Video Transcription**: Transcribe videos from public URLs using Google Vertex AI
+- 🌍 **Multi-language Support**: Support for multiple languages and fonts
+- 📝 **Subtitle Generation**: Generate timed subtitle files in Twick Studio format
+- 🔗 **Twick Studio Integration**: Direct upload and link generation for Twick Studio
+- 📦 **Claude Desktop Extension**: Easy one-click installation
+
+## Prerequisites
+
+- Node.js 18+ and npm
+- Google Cloud Platform account with:
+  - Vertex AI API enabled
+  - Service account key file (`gcp-sa-key.json`)
+- Claude Desktop installed
+
+## Quick Installation (One-Click)
+
+```bash
+npm run install-claude
+```
+
+This will:
+1. Install all dependencies
+2. Build the TypeScript project
+3. Automatically merge the MCP server config into your Claude Desktop configuration
+
+## Manual Installation
+
+1. **Clone and install dependencies:**
+   ```bash
+   git clone <your-repo-url>
+   cd mcp-agent
+   npm install
+   ```
+
+2. **Build the project:**
+   ```bash
+   npm run build
+   ```
+
+3. **Configure Claude Desktop:**
+   - Open Claude Desktop settings
+   - Go to "MCP Servers" section
+   - Click "Import config from file"
+   - Select `claude_desktop_config.json` from this project
+   - Edit the environment variables with your values (see Configuration below)
+
+4. **Restart Claude Desktop**
+
+## Configuration
+
+Edit the `twick-mcp-agent` section in your Claude Desktop config file:
+
+**Windows:**
+```
+%APPDATA%\Claude\claude_desktop_config.json
+```
+
+**macOS:**
+```
+~/Library/Application Support/Claude/claude_desktop_config.json
+```
+
+**Linux:**
+```
+~/.config/Claude/claude_desktop_config.json
+```
+
+### Required Environment Variables
+
+- `GOOGLE_CLOUD_PROJECT`: Your GCP project ID
+- `GOOGLE_CLOUD_LOCATION`: GCP location (default: "global")
+- `GOOGLE_APPLICATION_CREDENTIALS`: Absolute path to your `gcp-sa-key.json` file
+
+### Optional Environment Variables
+
+- `GOOGLE_VERTEX_MODEL`: Vertex AI model name (default: "gemini-2.5-flash-lite")
+- `UPLOAD_API_URL`: Your upload API endpoint for uploading project files
+- `TWICK_STUDIO_URL`: Twick Studio URL with `$project` placeholder (e.g., `https://studio.example.com?project-file=$project`)
+
+### Example Configuration
+
+```json
+{
+  "mcpServers": {
+    "twick-mcp-agent": {
+      "command": "node",
+      "args": ["C:\\path\\to\\mcp-agent\\dist\\stdio-server.js"],
+      "env": {
+        "GOOGLE_CLOUD_PROJECT": "my-gcp-project",
+        "GOOGLE_CLOUD_LOCATION": "global",
+        "GOOGLE_APPLICATION_CREDENTIALS": "C:\\path\\to\\mcp-agent\\gcp-sa-key.json",
+        "GOOGLE_VERTEX_MODEL": "gemini-2.5-flash-lite",
+        "UPLOAD_API_URL": "https://api.example.com/upload",
+        "TWICK_STUDIO_URL": "https://studio.example.com?project-file=$project"
+      }
+    }
+  }
+}
+```
+
+## Usage
+
+Once installed, the `generate-subtitles` tool will be available in Claude Desktop. You can use it like:
+
+```
+Generate subtitless for this video: https://example.com/video.mp4
+```
+
+Or with specific language settings:
+
+```
+Generate subtitless for https://example.com/video.mp4 in Spanish with Spanish font
+```
+
+### Tool Parameters
+
+- `videoUrl` (required): Publicly accessible video URL
+- `language` (optional): Target language for transcription (default: "english")
+- `language_font` (optional): Font/script for subtitles (default: "english")
+
+## Project Structure
+
+```
+mcp-agent/
+├── dist/                 # Compiled JavaScript (generated)
+├── src/                  # TypeScript source files
+├── stdio-server.ts       # Main MCP server entry point
+├── transcriber.ts        # Video transcription logic
+├── utils.ts              # Utility functions
+├── claude_desktop_config.json  # Claude Desktop config template
+├── install.js            # Installation script
+├── package.json
+└── README.md
+```
+
+## Development
+
+```bash
+# Install dependencies
+npm install
+
+# Build TypeScript
+npm run build
+
+# Run in development mode
+npm run dev
+```
+
+## Troubleshooting
+
+### Server not appearing in Claude Desktop
+
+1. Check that the config file path is correct
+2. Verify all environment variables are set
+3. Check Claude Desktop logs for errors
+4. Ensure `dist/stdio-server.js` exists (run `npm run build`)
+
+### Transcription errors
+
+1. Verify your GCP credentials are valid
+2. Check that Vertex AI API is enabled in your GCP project
+3. Ensure the video URL is publicly accessible
+4. Check that you have sufficient GCP quota
+
+### Upload/Studio link issues
+
+- If `UPLOAD_API_URL` is not set, the tool will return the project as a downloadable JSON file
+- If `TWICK_STUDIO_URL` is not set, only the upload result will be returned
+- The `$project` placeholder in `TWICK_STUDIO_URL` will be automatically replaced with the encoded project URL
+
+## License
+
+ISC
+

package/claude_desktop_config.json

@@ -0,0 +1,18 @@
+{
+  "mcpServers": {
+    "twick-mcp-agent": {
+      "command": "node",
+      "args": ["dist/stdio-server.js"],
+      "env": {
+        "GOOGLE_CLOUD_PROJECT": "your-gcp-project-id",
+        "GOOGLE_CLOUD_LOCATION": "global",
+        "GOOGLE_APPLICATION_CREDENTIALS": "gcp-sa-key.json",
+        "GOOGLE_VERTEX_MODEL": "gemini-2.5-flash-lite",
+        "UPLOAD_API_URL": "https://your-upload-api.example.com/upload",
+        "TWICK_STUDIO_URL": "https://development.d1vtsw7m0lx01h.amplifyapp.com?project-file=$project"
+      }
+    }
+  }
+}
+
+

package/dist/stdio-server.d.ts

@@ -0,0 +1,2 @@
+export {};
+//# sourceMappingURL=stdio-server.d.ts.map

package/dist/stdio-server.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"stdio-server.d.ts","sourceRoot":"","sources":["../stdio-server.ts"],"names":[],"mappings":""}

package/dist/stdio-server.js

@@ -0,0 +1,94 @@
+import { Server } from "@modelcontextprotocol/sdk/server/index.js";
+import { StdioServerTransport } from "@modelcontextprotocol/sdk/server/stdio.js";
+import { CallToolRequestSchema, ListToolsRequestSchema, } from "@modelcontextprotocol/sdk/types.js";
+import { transcribeVideoUrl } from "./transcriber.js";
+import { processSubtitlesToProject, returnProjectAsFile, returnProjectAsTwickStudioLink, } from "./utils.js";
+import { fileURLToPath } from "node:url";
+import { dirname, resolve } from "node:path";
+import dotenv from "dotenv";
+// Get project root (go up from dist/ to project root)
+const __filename = fileURLToPath(import.meta.url);
+const __dirname = dirname(__filename);
+const PROJECT_ROOT = resolve(__dirname, "..");
+// Load .env.local from project root, not cwd
+dotenv.config({ path: resolve(PROJECT_ROOT, ".env.local") });
+// Initialize MCP server
+const server = new Server({
+    name: "twick-mcp-agent",
+    version: "1.0.0",
+}, {
+    capabilities: {
+        tools: {},
+    },
+});
+// Advertise tools so MCP clients can discover them
+server.setRequestHandler(ListToolsRequestSchema, async () => {
+    return {
+        tools: [
+            {
+                name: "generate-subtitles",
+                description: "Generate subtitles for an video URL using an external service",
+                inputSchema: {
+                    type: "object",
+                    properties: {
+                        videoUrl: {
+                            type: "string",
+                            description: "Publicly accessible media URL (video)",
+                        },
+                        language: {
+                            type: "string",
+                            description: "Language for transcription (default: english)",
+                        },
+                        language_font: {
+                            type: "string",
+                            description: "Language font (default: english)",
+                        },
+                    },
+                    required: ["videoUrl"],
+                },
+            },
+        ],
+    };
+});
+server.setRequestHandler(CallToolRequestSchema, async (request) => {
+    try {
+        if (request.params.name === "generate-subtitles") {
+            const { videoUrl, language, language_font } = request.params
+                .arguments;
+            if (!videoUrl || typeof videoUrl !== "string") {
+                throw new Error("Invalid or missing videoUrl");
+            }
+            const data = await transcribeVideoUrl({
+                videoUrl,
+                language: language ?? "english",
+                languageFont: language_font ?? "english",
+            });
+            const project = await processSubtitlesToProject({
+                subtitles: data.subtitles,
+                videoUrl: data.videoUrl,
+                duration: data.duration,
+                videoSize: { width: 720, height: 1280 },
+            });
+            if (process.env.UPLOAD_API_URL && process.env.TWICK_STUDIO_URL) {
+                return await returnProjectAsTwickStudioLink(project);
+            }
+            else {
+                return returnProjectAsFile(project);
+            }
+        }
+        else {
+            throw new Error(`Unknown tool: ${request.params.name}`);
+        }
+    }
+    catch (error) {
+        console.error(error);
+        throw error;
+    }
+});
+// Start the server with stdio transport
+async function main() {
+    const transport = new StdioServerTransport();
+    await server.connect(transport);
+}
+main().catch(console.error);
+//# sourceMappingURL=stdio-server.js.map

package/dist/stdio-server.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"stdio-server.js","sourceRoot":"","sources":["../stdio-server.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,MAAM,EAAE,MAAM,2CAA2C,CAAC;AACnE,OAAO,EAAE,oBAAoB,EAAE,MAAM,2CAA2C,CAAC;AACjF,OAAO,EACL,qBAAqB,EACrB,sBAAsB,GACvB,MAAM,oCAAoC,CAAC;AAC5C,OAAO,EAAE,kBAAkB,EAAE,MAAM,kBAAkB,CAAC;AACtD,OAAO,EACL,yBAAyB,EACzB,mBAAmB,EACnB,8BAA8B,GAC/B,MAAM,YAAY,CAAC;AACpB,OAAO,EAAE,aAAa,EAAE,MAAM,UAAU,CAAC;AACzC,OAAO,EAAE,OAAO,EAAE,OAAO,EAAE,MAAM,WAAW,CAAC;AAE7C,OAAO,MAAM,MAAM,QAAQ,CAAC;AAE5B,sDAAsD;AACtD,MAAM,UAAU,GAAG,aAAa,CAAC,MAAM,CAAC,IAAI,CAAC,GAAG,CAAC,CAAC;AAClD,MAAM,SAAS,GAAG,OAAO,CAAC,UAAU,CAAC,CAAC;AACtC,MAAM,YAAY,GAAG,OAAO,CAAC,SAAS,EAAE,IAAI,CAAC,CAAC;AAE9C,6CAA6C;AAC7C,MAAM,CAAC,MAAM,CAAC,EAAE,IAAI,EAAE,OAAO,CAAC,YAAY,EAAE,YAAY,CAAC,EAAE,CAAC,CAAC;AAE7D,wBAAwB;AACxB,MAAM,MAAM,GAAG,IAAI,MAAM,CACvB;IACE,IAAI,EAAE,iBAAiB;IACvB,OAAO,EAAE,OAAO;CACjB,EACD;IACE,YAAY,EAAE;QACZ,KAAK,EAAE,EAAE;KACV;CACF,CACF,CAAC;AAEF,mDAAmD;AACnD,MAAM,CAAC,iBAAiB,CAAC,sBAAsB,EAAE,KAAK,IAAI,EAAE;IAC1D,OAAO;QACL,KAAK,EAAE;YACL;gBACE,IAAI,EAAE,oBAAoB;gBAC1B,WAAW,EACT,+DAA+D;gBACjE,WAAW,EAAE;oBACX,IAAI,EAAE,QAAQ;oBACd,UAAU,EAAE;wBACV,QAAQ,EAAE;4BACR,IAAI,EAAE,QAAQ;4BACd,WAAW,EAAE,uCAAuC;yBACrD;wBACD,QAAQ,EAAE;4BACR,IAAI,EAAE,QAAQ;4BACd,WAAW,EAAE,+CAA+C;yBAC7D;wBACD,aAAa,EAAE;4BACb,IAAI,EAAE,QAAQ;4BACd,WAAW,EAAE,kCAAkC;yBAChD;qBACF;oBACD,QAAQ,EAAE,CAAC,UAAU,CAAC;iBACvB;aACF;SACF;KACF,CAAC;AACJ,CAAC,CAAC,CAAC;AAEH,MAAM,CAAC,iBAAiB,CAAC,qBAAqB,EAAE,KAAK,EAAE,OAAO,EAAE,EAAE;IAChE,IAAI,CAAC;QACH,IAAI,OAAO,CAAC,MAAM,CAAC,IAAI,KAAK,oBAAoB,EAAE,CAAC;YACjD,MAAM,EAAE,QAAQ,EAAE,QAAQ,EAAE,aAAa,EAAE,GAAG,OAAO,CAAC,MAAM;iBACzD,SAIF,CAAC;YAEF,IAAI,CAAC,QAAQ,IAAI,OAAO,QAAQ,KAAK,QAAQ,EAAE,CAAC;gBAC9C,MAAM,IAAI,KAAK,CAAC,6BAA6B,CAAC,CAAC;YACjD,CAAC;YAED,MAAM,IAAI,GAAG,MAAM,kBAAkB,CAAC;gBACpC,QAAQ;gBACR,QAAQ,EAAE,QAAQ,IAAI,SAAS;gBAC/B,YAAY,EAAE,aAAa,IAAI,SAAS;aACzC,CAAC,CAAC;YAEH,MAAM,OAAO,GAAG,MAAM,yBAAyB,CAAC;gBAC9C,SAAS,EAAE,IAAI,CAAC,SAAS;gBACzB,QAAQ,EAAE,IAAI,CAAC,QAAQ;gBACvB,QAAQ,EAAE,IAAI,CAAC,QAAQ;gBACvB,SAAS,EAAE,EAAE,KAAK,EAAE,GAAG,EAAE,MAAM,EAAE,IAAI,EAAE;aACxC,CAAC,CAAC;YAEH,IAAI,OAAO,CAAC,GAAG,CAAC,cAAc,IAAI,OAAO,CAAC,GAAG,CAAC,gBAAgB,EAAE,CAAC;gBAC/D,OAAO,MAAM,8BAA8B,CAAC,OAAO,CAAC,CAAC;YACvD,CAAC;iBAAM,CAAC;gBACN,OAAO,mBAAmB,CAAC,OAAO,CAAC,CAAC;YACtC,CAAC;QACH,CAAC;aAAM,CAAC;YACN,MAAM,IAAI,KAAK,CAAC,iBAAiB,OAAO,CAAC,MAAM,CAAC,IAAI,EAAE,CAAC,CAAC;QAC1D,CAAC;IACH,CAAC;IAAC,OAAO,KAAK,EAAE,CAAC;QACf,OAAO,CAAC,KAAK,CAAC,KAAK,CAAC,CAAC;QACrB,MAAM,KAAK,CAAC;IACd,CAAC;AACH,CAAC,CAAC,CAAC;AAEH,wCAAwC;AACxC,KAAK,UAAU,IAAI;IACjB,MAAM,SAAS,GAAG,IAAI,oBAAoB,EAAE,CAAC;IAC7C,MAAM,MAAM,CAAC,OAAO,CAAC,SAAS,CAAC,CAAC;AAClC,CAAC;AAED,IAAI,EAAE,CAAC,KAAK,CAAC,OAAO,CAAC,KAAK,CAAC,CAAC"}

package/dist/transcriber.d.ts

@@ -0,0 +1,21 @@
+/**
+ * Transcribe an audio URL to JSON subtitles using Google GenAI (Vertex AI),
+ * mirroring the Python implementation in `playground/vertex/transcript.py`.
+ *
+ * @param {Object} params
+ * @param {string} params.videoUrl - Publicly reachable video URL.
+ * @param {string} [params.language="english"] - Target transcription language (human-readable).
+ * @param {string} [params.languageFont="english"] - Target font/script for subtitles.
+ * @returns {Promise<{ subtitles: Array<{t: string, s: number, e: number}> }>} Subtitles array with text, start time, and end time.
+ * @throws {Error} When audioUrl is missing or downstream calls fail.
+ */
+export declare const transcribeVideoUrl: (params: {
+    videoUrl: string;
+    language?: string;
+    languageFont?: string;
+}) => Promise<{
+    subtitles: any[];
+    duration: number;
+    videoUrl: string;
+}>;
+//# sourceMappingURL=transcriber.d.ts.map

package/dist/transcriber.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"transcriber.d.ts","sourceRoot":"","sources":["../transcriber.ts"],"names":[],"mappings":"AAsMA;;;;;;;;;;GAUG;AACH,eAAO,MAAM,kBAAkB,GAAU,QAAQ;IAAE,QAAQ,EAAE,MAAM,CAAC;IAAC,QAAQ,CAAC,EAAE,MAAM,CAAC;IAAC,YAAY,CAAC,EAAE,MAAM,CAAA;CAAE;;;;EAwG9G,CAAC"}

package/dist/transcriber.js

@@ -0,0 +1,281 @@
+import { GoogleGenAI } from "@google/genai";
+import fs from "fs";
+import path, { join } from "path";
+import { mkdtemp, readFile, rm } from "fs/promises";
+import { tmpdir } from "os";
+import { execFile } from "child_process";
+import { promisify } from "util";
+import { Readable, pipeline } from "stream";
+import { fileURLToPath } from "node:url";
+import { dirname } from "node:path";
+// These packages provide prebuilt ffmpeg/ffprobe binaries. Types are not bundled,
+// so we import them as `any` to keep TypeScript satisfied.
+import ffmpeg from "@ffmpeg-installer/ffmpeg";
+import ffprobe from "@ffprobe-installer/ffprobe";
+const __filename = fileURLToPath(import.meta.url);
+const __dirname = dirname(__filename);
+// Get project root (go up from dist/ to project root)
+const PROJECT_ROOT = path.resolve(__dirname, "..");
+const execFileAsync = promisify(execFile);
+const pipelineAsync = promisify(pipeline);
+const ffmpegPath = ffmpeg.path;
+const ffprobePath = ffprobe.path;
+/**
+ * Read a required environment variable, optionally falling back to a default.
+ * Throws if neither value is available, making configuration errors obvious.
+ *
+ * @param {string} name - Environment variable to read.
+ * @param {string | undefined} defaultValue - Optional fallback value.
+ * @returns {string} The resolved value.
+ * @throws {Error} If no value is found.
+ */
+const ensureEnv = (name, defaultValue) => {
+    const value = process.env[name] ?? defaultValue;
+    if (!value) {
+        throw new Error(`Missing required environment variable: ${name}`);
+    }
+    return value;
+};
+const ensureCredentials = () => {
+    // Resolve gcp-sa-key.json relative to project root (not cwd, which may be Claude Desktop's directory)
+    // First try an explicit env var, then fall back to project root
+    const credentialsPath = process.env.GOOGLE_APPLICATION_CREDENTIALS
+        || process.env.GCP_SA_KEY_PATH
+        || path.join(PROJECT_ROOT, "gcp-sa-key.json");
+    if (!fs.existsSync(credentialsPath)) {
+        throw new Error(`Credentials file not found at ${credentialsPath}. Set GCP_SA_KEY_PATH environment variable or place gcp-sa-key.json in the project root (${PROJECT_ROOT})`);
+    }
+    process.env.GOOGLE_APPLICATION_CREDENTIALS = credentialsPath;
+};
+/**
+ * Initialize a Google GenAI client configured for Vertex AI.
+ * Ensures credentials, project, and location are available before instantiating.
+ *
+ * @returns {Promise<GoogleGenAI>} Configured GenAI client instance.
+ * @throws {Error} When required environment variables are missing.
+ */
+const createGenAIClient = async () => {
+    ensureCredentials();
+    const project = ensureEnv("GOOGLE_CLOUD_PROJECT");
+    const location = ensureEnv("GOOGLE_CLOUD_LOCATION", "global");
+    const client = new GoogleGenAI({
+        vertexai: true,
+        project: project,
+        location: location,
+    });
+    return client;
+};
+const extractAudioBufferFromVideo = async (videoUrl) => {
+    const videoResponse = await fetch(videoUrl);
+    if (!videoResponse.ok) {
+        throw new Error(`Failed to download video: ${videoResponse.status} ${videoResponse.statusText}`);
+    }
+    const tmpBase = await mkdtemp(join(tmpdir(), 'mcp-'));
+    const inputPath = join(tmpBase, 'input_video');
+    const outputPath = join(tmpBase, 'output_audio.mp3');
+    // Stream the video response directly to disk to avoid holding the full video in memory
+    if (!videoResponse.body) {
+        await rm(tmpBase, { recursive: true, force: true });
+        throw new Error("Video response has no body");
+    }
+    const videoStream = Readable.fromWeb(videoResponse.body);
+    const fileWriteStream = fs.createWriteStream(inputPath);
+    await pipelineAsync(videoStream, fileWriteStream);
+    // Get duration using bundled ffprobe
+    let duration = 0;
+    try {
+        const { stdout } = await execFileAsync(ffprobePath, [
+            '-v', 'error',
+            '-show_entries', 'format=duration',
+            '-of', 'default=noprint_wrappers=1:nokey=1',
+            inputPath
+        ]);
+        duration = parseFloat(stdout.toString().trim()) || 0;
+    }
+    catch (err) {
+        console.warn('Failed to get duration using ffprobe, duration will be 0');
+    }
+    try {
+        await execFileAsync(ffmpegPath, [
+            '-y',
+            '-i', inputPath,
+            '-vn',
+            '-acodec', 'libmp3lame',
+            '-q:a', '2',
+            outputPath
+        ]);
+    }
+    catch (err) {
+        await rm(tmpBase, { recursive: true, force: true });
+        const stderr = err?.stderr?.toString?.().trim?.() || "";
+        const msg = stderr || (err instanceof Error ? err.message : String(err));
+        throw new Error(`ffmpeg execution failed: ${msg}`);
+    }
+    const audioBuffer = await readFile(outputPath);
+    await rm(tmpBase, { recursive: true, force: true });
+    return { audioBuffer, duration };
+};
+/**
+ * Build the captioning prompt passed to the Gemini model.
+ *
+ * @param {string} language - Human-readable target language.
+ * @param {string} languageFont - Desired script/font name.
+ * @returns {string} Instruction prompt for the model.
+ */
+const buildPrompt = (duration, language, languageFont) => {
+    const durationMs = Math.round(duration * 1000);
+    return `You are a professional subtitle and transcription engine.
+
+## INPUT
+- Audio duration: ${durationMs} milliseconds
+- Target language: ${language}
+- Subtitle font script: ${languageFont}
+
+## OBJECTIVE
+Transcribe the audio into clear, readable subtitles.
+
+If the spoken audio is NOT in ${language}, translate it into ${language} before generating subtitles.
+
+## SUBTITLE SEGMENTATION RULES
+- Split speech into short, natural phrases.
+- Each subtitle phrase MUST contain a maximum of 4 words.
+- Do NOT split words across phrases.
+- Avoid breaking phrases mid-sentence unless required by timing constraints.
+
+## TIMING RULES (STRICT — MUST FOLLOW)
+- All timestamps are in **milliseconds**.
+- Each subtitle object MUST include:
+  - 's': start timestamp
+  - 'e': end timestamp
+- Duration of each phrase = 'e - s'
+- Minimum phrase duration: **100 ms**
+- 'e' MUST be greater than 's'
+- 'e' MUST be **less than or equal to ${durationMs}**
+- Subtitles MUST be sequential:
+  - 's' of the next phrase MUST be **greater than or equal to** the previous 'e'
+  - NO overlapping timestamps
+- Prefer aligning timestamps with natural speech pauses.
+
+## TEXT RULES
+- 't' MUST be written using ${languageFont} characters.
+- No emojis.
+- No punctuation-only subtitles.
+- Normalize casing according to the target language's writing system.
+- Remove filler sounds (e.g., “um”, “uh”) unless semantically important.
+
+## OUTPUT FORMAT (CRITICAL)
+Return ONLY a valid JSON array.
+- No markdown
+- No code blocks
+- No explanations
+- No additional text
+- Output MUST start with '[' and end with ']'
+
+## OUTPUT SCHEMA
+[
+  {
+    "t": "Subtitle text",
+    "s": 0,
+    "e": 1200
+  }
+]
+`.trim();
+};
+/**
+ * Transcribe an audio URL to JSON subtitles using Google GenAI (Vertex AI),
+ * mirroring the Python implementation in `playground/vertex/transcript.py`.
+ *
+ * @param {Object} params
+ * @param {string} params.videoUrl - Publicly reachable video URL.
+ * @param {string} [params.language="english"] - Target transcription language (human-readable).
+ * @param {string} [params.languageFont="english"] - Target font/script for subtitles.
+ * @returns {Promise<{ subtitles: Array<{t: string, s: number, e: number}> }>} Subtitles array with text, start time, and end time.
+ * @throws {Error} When audioUrl is missing or downstream calls fail.
+ */
+export const transcribeVideoUrl = async (params) => {
+    const { videoUrl, language = "english", languageFont = "english", } = params || {};
+    if (!videoUrl) {
+        throw new Error("Missing required parameter: videoUrl");
+    }
+    const { audioBuffer, duration } = await extractAudioBufferFromVideo(videoUrl);
+    if (!duration) {
+        throw new Error("Failed to get duration of video");
+    }
+    const prompt = buildPrompt(duration, language, languageFont);
+    const client = await createGenAIClient();
+    const modelName = process.env.GOOGLE_VERTEX_MODEL || "gemini-2.5-flash-lite";
+    const generationConfig = {
+        maxOutputTokens: 65535,
+        temperature: 1,
+        topP: 0.95,
+        thinkingConfig: {
+            thinkingBudget: 0,
+        },
+        safetySettings: [
+            {
+                category: "HARM_CATEGORY_HATE_SPEECH",
+                threshold: "OFF",
+            },
+            {
+                category: "HARM_CATEGORY_DANGEROUS_CONTENT",
+                threshold: "OFF",
+            },
+            {
+                category: "HARM_CATEGORY_SEXUALLY_EXPLICIT",
+                threshold: "OFF",
+            },
+            {
+                category: "HARM_CATEGORY_HARASSMENT",
+                threshold: "OFF",
+            },
+        ],
+    };
+    const req = {
+        model: modelName,
+        contents: [
+            {
+                role: "user",
+                parts: [
+                    {
+                        inlineData: {
+                            data: audioBuffer.toString("base64"),
+                            mimeType: "audio/mpeg",
+                        },
+                    },
+                    { text: prompt },
+                ],
+            },
+        ],
+        config: generationConfig,
+    };
+    //@ts-ignore
+    const response = await client.models.generateContent(req);
+    let textPart = response.text || "";
+    // Strip markdown code fences if present (```json ... ``` or ``` ... ```)
+    textPart = textPart
+        .replace(/^```json\s*/i, "") // Remove opening ```json
+        .replace(/^```\s*/i, "") // Remove opening ```
+        .replace(/\s*```$/i, "") // Remove closing ```
+        .trim();
+    let subtitles = [];
+    try {
+        // Try to find JSON array in the text (in case there's extra text)
+        const jsonMatch = textPart.match(/\[[\s\S]*\]/);
+        const jsonText = jsonMatch ? jsonMatch[0] : textPart;
+        subtitles = JSON.parse(jsonText);
+        if (!Array.isArray(subtitles)) {
+            throw new Error("Parsed subtitles are not an array");
+        }
+    }
+    catch (err) {
+        console.warn("Failed to parse model output as JSON subtitles, returning raw text", err);
+        console.warn("Raw response text:", textPart.substring(0, 500));
+        subtitles = [];
+    }
+    return {
+        subtitles,
+        duration,
+        videoUrl
+    };
+};
+//# sourceMappingURL=transcriber.js.map

package/dist/transcriber.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"transcriber.js","sourceRoot":"","sources":["../transcriber.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,WAAW,EAAE,MAAM,eAAe,CAAC;AAC5C,OAAO,EAAE,MAAM,IAAI,CAAC;AACpB,OAAO,IAAI,EAAE,EAAE,IAAI,EAAE,MAAM,MAAM,CAAC;AAClC,OAAO,EAAE,OAAO,EAAE,QAAQ,EAAE,EAAE,EAAE,MAAM,aAAa,CAAC;AACpD,OAAO,EAAE,MAAM,EAAE,MAAM,IAAI,CAAC;AAC5B,OAAO,EAAE,QAAQ,EAAE,MAAM,eAAe,CAAC;AACzC,OAAO,EAAE,SAAS,EAAE,MAAM,MAAM,CAAC;AACjC,OAAO,EAAE,QAAQ,EAAE,QAAQ,EAAE,MAAM,QAAQ,CAAC;AAC5C,OAAO,EAAE,aAAa,EAAE,MAAM,UAAU,CAAC;AACzC,OAAO,EAAE,OAAO,EAAE,MAAM,WAAW,CAAC;AACpC,kFAAkF;AAClF,2DAA2D;AAC3D,OAAO,MAAM,MAAM,0BAA0B,CAAC;AAC9C,OAAO,OAAO,MAAM,4BAA4B,CAAC;AAEjD,MAAM,UAAU,GAAG,aAAa,CAAC,MAAM,CAAC,IAAI,CAAC,GAAG,CAAC,CAAC;AAClD,MAAM,SAAS,GAAG,OAAO,CAAC,UAAU,CAAC,CAAC;AACtC,sDAAsD;AACtD,MAAM,YAAY,GAAG,IAAI,CAAC,OAAO,CAAC,SAAS,EAAE,IAAI,CAAC,CAAC;AAEnD,MAAM,aAAa,GAAG,SAAS,CAAC,QAAQ,CAAC,CAAC;AAC1C,MAAM,aAAa,GAAG,SAAS,CAAC,QAAQ,CAAC,CAAC;AAC1C,MAAM,UAAU,GAAG,MAAM,CAAC,IAAI,CAAC;AAC/B,MAAM,WAAW,GAAG,OAAO,CAAC,IAAI,CAAC;AAGjC;;;;;;;;GAQG;AACH,MAAM,SAAS,GAAG,CAAC,IAAY,EAAE,YAAqB,EAAE,EAAE;IACxD,MAAM,KAAK,GAAG,OAAO,CAAC,GAAG,CAAC,IAAI,CAAC,IAAI,YAAY,CAAC;IAChD,IAAI,CAAC,KAAK,EAAE,CAAC;QACX,MAAM,IAAI,KAAK,CAAC,0CAA0C,IAAI,EAAE,CAAC,CAAC;IACpE,CAAC;IACD,OAAO,KAAK,CAAC;AACf,CAAC,CAAC;AAGF,MAAM,iBAAiB,GAAG,GAAG,EAAE;IAC7B,sGAAsG;IACtG,gEAAgE;IAChE,MAAM,eAAe,GAAG,OAAO,CAAC,GAAG,CAAC,8BAA8B;WAC7D,OAAO,CAAC,GAAG,CAAC,eAAe;WAC3B,IAAI,CAAC,IAAI,CAAC,YAAY,EAAE,iBAAiB,CAAC,CAAC;IAEhD,IAAI,CAAC,EAAE,CAAC,UAAU,CAAC,eAAe,CAAC,EAAE,CAAC;QACpC,MAAM,IAAI,KAAK,CAAC,iCAAiC,eAAe,4FAA4F,YAAY,GAAG,CAAC,CAAC;IAC/K,CAAC;IAED,OAAO,CAAC,GAAG,CAAC,8BAA8B,GAAG,eAAe,CAAC;AAC/D,CAAC,CAAC;AAEF;;;;;;GAMG;AACH,MAAM,iBAAiB,GAAG,KAAK,IAAI,EAAE;IACnC,iBAAiB,EAAE,CAAC;IACpB,MAAM,OAAO,GAAG,SAAS,CAAC,sBAAsB,CAAC,CAAC;IAClD,MAAM,QAAQ,GAAG,SAAS,CAAC,uBAAuB,EAAE,QAAQ,CAAC,CAAC;IAC9D,MAAM,MAAM,GAAG,IAAI,WAAW,CAAC;QAC7B,QAAQ,EAAE,IAAI;QACd,OAAO,EAAE,OAAO;QAChB,QAAQ,EAAE,QAAQ;KACnB,CAAC,CAAC;IAEH,OAAO,MAAM,CAAC;AAChB,CAAC,CAAC;AAEF,MAAM,2BAA2B,GAAG,KAAK,EAAE,QAAgB,EAAE,EAAE;IAC7D,MAAM,aAAa,GAAG,MAAM,KAAK,CAAC,QAAQ,CAAC,CAAC;IAC5C,IAAI,CAAC,aAAa,CAAC,EAAE,EAAE,CAAC;QACtB,MAAM,IAAI,KAAK,CAAC,6BAA6B,aAAa,CAAC,MAAM,IAAI,aAAa,CAAC,UAAU,EAAE,CAAC,CAAC;IACnG,CAAC;IACD,MAAM,OAAO,GAAG,MAAM,OAAO,CAAC,IAAI,CAAC,MAAM,EAAE,EAAE,MAAM,CAAC,CAAC,CAAC;IACtD,MAAM,SAAS,GAAG,IAAI,CAAC,OAAO,EAAE,aAAa,CAAC,CAAC;IAC/C,MAAM,UAAU,GAAG,IAAI,CAAC,OAAO,EAAE,kBAAkB,CAAC,CAAC;IAErD,uFAAuF;IACvF,IAAI,CAAC,aAAa,CAAC,IAAI,EAAE,CAAC;QACxB,MAAM,EAAE,CAAC,OAAO,EAAE,EAAE,SAAS,EAAE,IAAI,EAAE,KAAK,EAAE,IAAI,EAAE,CAAC,CAAC;QACpD,MAAM,IAAI,KAAK,CAAC,4BAA4B,CAAC,CAAC;IAChD,CAAC;IACD,MAAM,WAAW,GAAG,QAAQ,CAAC,OAAO,CAAC,aAAa,CAAC,IAAI,CAAC,CAAC;IACzD,MAAM,eAAe,GAAG,EAAE,CAAC,iBAAiB,CAAC,SAAS,CAAC,CAAC;IACxD,MAAM,aAAa,CAAC,WAAW,EAAE,eAAe,CAAC,CAAC;IAElD,qCAAqC;IACrC,IAAI,QAAQ,GAAG,CAAC,CAAC;IACjB,IAAI,CAAC;QACH,MAAM,EAAE,MAAM,EAAE,GAAG,MAAM,aAAa,CAAC,WAAW,EAAE;YAClD,IAAI,EAAE,OAAO;YACb,eAAe,EAAE,iBAAiB;YAClC,KAAK,EAAE,oCAAoC;YAC3C,SAAS;SACV,CAAC,CAAC;QACH,QAAQ,GAAG,UAAU,CAAC,MAAM,CAAC,QAAQ,EAAE,CAAC,IAAI,EAAE,CAAC,IAAI,CAAC,CAAC;IACvD,CAAC;IAAC,OAAO,GAAG,EAAE,CAAC;QACb,OAAO,CAAC,IAAI,CAAC,0DAA0D,CAAC,CAAC;IAC3E,CAAC;IAED,IAAI,CAAC;QACH,MAAM,aAAa,CAAC,UAAU,EAAE;YAC9B,IAAI;YACJ,IAAI,EAAE,SAAS;YACf,KAAK;YACL,SAAS,EAAE,YAAY;YACvB,MAAM,EAAE,GAAG;YACX,UAAU;SACX,CAAC,CAAC;IACL,CAAC;IAAC,OAAO,GAAQ,EAAE,CAAC;QAClB,MAAM,EAAE,CAAC,OAAO,EAAE,EAAE,SAAS,EAAE,IAAI,EAAE,KAAK,EAAE,IAAI,EAAE,CAAC,CAAC;QACpD,MAAM,MAAM,GAAG,GAAG,EAAE,MAAM,EAAE,QAAQ,EAAE,EAAE,CAAC,IAAI,EAAE,EAAE,IAAI,EAAE,CAAC;QACxD,MAAM,GAAG,GAAG,MAAM,IAAI,CAAC,GAAG,YAAY,KAAK,CAAC,CAAC,CAAC,GAAG,CAAC,OAAO,CAAC,CAAC,CAAC,MAAM,CAAC,GAAG,CAAC,CAAC,CAAC;QACzE,MAAM,IAAI,KAAK,CAAC,4BAA4B,GAAG,EAAE,CAAC,CAAC;IACrD,CAAC;IAED,MAAM,WAAW,GAAG,MAAM,QAAQ,CAAC,UAAU,CAAC,CAAC;IAC/C,MAAM,EAAE,CAAC,OAAO,EAAE,EAAE,SAAS,EAAE,IAAI,EAAE,KAAK,EAAE,IAAI,EAAE,CAAC,CAAC;IACpD,OAAO,EAAE,WAAW,EAAE,QAAQ,EAAE,CAAC;AACnC,CAAC,CAAC;AAEF;;;;;;GAMG;AACH,MAAM,WAAW,GAAG,CAAC,QAAgB,EAAE,QAAgB,EAAE,YAAoB,EAAE,EAAE;IAC/E,MAAM,UAAU,GAAG,IAAI,CAAC,KAAK,CAAC,QAAQ,GAAG,IAAI,CAAC,CAAC;IAC/C,OAAO;;;oBAGW,UAAU;qBACT,QAAQ;0BACH,YAAY;;;;;gCAKN,QAAQ,uBAAuB,QAAQ;;;;;;;;;;;;;;;;wCAgB/B,UAAU;;;;;;;8BAOpB,YAAY;;;;;;;;;;;;;;;;;;;;;;CAsBzC,CAAC,IAAI,EAAE,CAAC;AACT,CAAC,CAAC;AAEF;;;;;;;;;;GAUG;AACH,MAAM,CAAC,MAAM,kBAAkB,GAAG,KAAK,EAAE,MAAsE,EAAE,EAAE;IACjH,MAAM,EACJ,QAAQ,EACR,QAAQ,GAAG,SAAS,EACpB,YAAY,GAAG,SAAS,GACzB,GAAG,MAAM,IAAI,EAAE,CAAC;IAEjB,IAAI,CAAC,QAAQ,EAAE,CAAC;QACd,MAAM,IAAI,KAAK,CAAC,sCAAsC,CAAC,CAAC;IAC1D,CAAC;IAED,MAAM,EAAE,WAAW,EAAE,QAAQ,EAAE,GAAG,MAAM,2BAA2B,CAAC,QAAQ,CAAC,CAAC;IAC9E,IAAI,CAAC,QAAQ,EAAE,CAAC;QACd,MAAM,IAAI,KAAK,CAAC,iCAAiC,CAAC,CAAC;IACrD,CAAC;IAED,MAAM,MAAM,GAAG,WAAW,CAAC,QAAQ,EAAE,QAAQ,EAAE,YAAY,CAAC,CAAC;IAE7D,MAAM,MAAM,GAAG,MAAM,iBAAiB,EAAE,CAAC;IACzC,MAAM,SAAS,GAAG,OAAO,CAAC,GAAG,CAAC,mBAAmB,IAAI,uBAAuB,CAAC;IAE7E,MAAM,gBAAgB,GAAG;QACvB,eAAe,EAAE,KAAK;QACtB,WAAW,EAAE,CAAC;QACd,IAAI,EAAE,IAAI;QACV,cAAc,EAAE;YACd,cAAc,EAAE,CAAC;SAClB;QACD,cAAc,EAAE;YACd;gBACE,QAAQ,EAAE,2BAA2B;gBACrC,SAAS,EAAE,KAAK;aACjB;YACD;gBACE,QAAQ,EAAE,iCAAiC;gBAC3C,SAAS,EAAE,KAAK;aACjB;YACD;gBACE,QAAQ,EAAE,iCAAiC;gBAC3C,SAAS,EAAE,KAAK;aACjB;YACD;gBACE,QAAQ,EAAE,0BAA0B;gBACpC,SAAS,EAAE,KAAK;aACjB;SACF;KACF,CAAC;IAEF,MAAM,GAAG,GAAG;QACV,KAAK,EAAE,SAAS;QAChB,QAAQ,EAAE;YACR;gBACE,IAAI,EAAE,MAAM;gBACZ,KAAK,EAAE;oBACL;wBACE,UAAU,EAAE;4BACV,IAAI,EAAE,WAAW,CAAC,QAAQ,CAAC,QAAQ,CAAC;4BACpC,QAAQ,EAAE,YAAY;yBACvB;qBACF;oBACD,EAAE,IAAI,EAAE,MAAM,EAAE;iBACjB;aACF;SACF;QACD,MAAM,EAAE,gBAAgB;KACzB,CAAC;IAEF,YAAY;IACZ,MAAM,QAAQ,GAAG,MAAM,MAAM,CAAC,MAAM,CAAC,eAAe,CAAC,GAAG,CAAC,CAAC;IAE1D,IAAI,QAAQ,GAAG,QAAQ,CAAC,IAAI,IAAI,EAAE,CAAC;IAEnC,yEAAyE;IACzE,QAAQ,GAAG,QAAQ;SAChB,OAAO,CAAC,cAAc,EAAE,EAAE,CAAC,CAAC,yBAAyB;SACrD,OAAO,CAAC,UAAU,EAAE,EAAE,CAAC,CAAC,qBAAqB;SAC7C,OAAO,CAAC,UAAU,EAAE,EAAE,CAAC,CAAC,qBAAqB;SAC7C,IAAI,EAAE,CAAC;IAGV,IAAI,SAAS,GAAG,EAAE,CAAC;IACnB,IAAI,CAAC;QACH,kEAAkE;QAClE,MAAM,SAAS,GAAG,QAAQ,CAAC,KAAK,CAAC,aAAa,CAAC,CAAC;QAChD,MAAM,QAAQ,GAAG,SAAS,CAAC,CAAC,CAAC,SAAS,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,QAAQ,CAAC;QAErD,SAAS,GAAG,IAAI,CAAC,KAAK,CAAC,QAAQ,CAAC,CAAC;QACjC,IAAI,CAAC,KAAK,CAAC,OAAO,CAAC,SAAS,CAAC,EAAE,CAAC;YAC9B,MAAM,IAAI,KAAK,CAAC,mCAAmC,CAAC,CAAC;QACvD,CAAC;IACH,CAAC;IAAC,OAAO,GAAG,EAAE,CAAC;QACb,OAAO,CAAC,IAAI,CACV,oEAAoE,EACpE,GAAG,CACJ,CAAC;QACF,OAAO,CAAC,IAAI,CAAC,oBAAoB,EAAE,QAAQ,CAAC,SAAS,CAAC,CAAC,EAAE,GAAG,CAAC,CAAC,CAAC;QAC/D,SAAS,GAAG,EAAE,CAAC;IACjB,CAAC;IAED,OAAO;QACL,SAAS;QACT,QAAQ;QACR,QAAQ;KACT,CAAC;AACJ,CAAC,CAAC"}

package/dist/utils.d.ts

@@ -0,0 +1,81 @@
+export declare const uploadDataToFile: (data: any) => Promise<unknown>;
+export declare const processSubtitles: (project: any) => Promise<unknown>;
+export declare const processSubtitlesToProject: ({ subtitles, videoUrl, duration, videoSize, }: {
+    subtitles: any;
+    videoUrl: string;
+    duration: number;
+    videoSize: {
+        width: number;
+        height: number;
+    };
+}) => Promise<{
+    properties: {
+        width: number;
+        height: number;
+    };
+    tracks: ({
+        id: string;
+        type: string;
+        elements: {
+            id: string;
+            type: string;
+            s: number;
+            e: number;
+            props: {
+                src: string;
+                width: number;
+                height: number;
+            };
+        }[];
+        props?: never;
+    } | {
+        id: string;
+        type: string;
+        props: {
+            capStyle: string;
+            font: {
+                size: number;
+                weight: number;
+                family: string;
+            };
+            colors: {
+                text: string;
+                highlight: string;
+                bgColor: string;
+            };
+            lineWidth: number;
+            stroke: string;
+            fontWeight: number;
+            shadowOffset: number[];
+            shadowColor: string;
+            x: number;
+            y: number;
+            applyToAll: boolean;
+        };
+        elements: any;
+    })[];
+    version: number;
+}>;
+export declare const getTwickStudioLink: (uploadResult: any) => string | undefined;
+export declare const returnProjectAsTwickStudioLink: (project: any) => Promise<{
+    content: {
+        type: string;
+        text: string;
+    }[];
+} | {
+    content: {
+        type: string;
+        text: string;
+    }[];
+    structuredContent: {
+        success: boolean;
+        result: string;
+    };
+}>;
+export declare const returnProjectAsFile: (project: any) => {
+    content: {
+        type: string;
+        text: string;
+    }[];
+};
+//# sourceMappingURL=utils.d.ts.map

package/dist/utils.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"utils.d.ts","sourceRoot":"","sources":["../utils.ts"],"names":[],"mappings":"AAAA,eAAO,MAAM,gBAAgB,GAAU,MAAM,GAAG,qBAoB/C,CAAC;AAEF,eAAO,MAAM,gBAAgB,GAAU,SAAS,GAAG,qBAclD,CAAC;AAEF,eAAO,MAAM,yBAAyB,GAAU,+CAK7C;IACD,SAAS,EAAE,GAAG,CAAC;IACf,QAAQ,EAAE,MAAM,CAAC;IACjB,QAAQ,EAAE,MAAM,CAAC;IACjB,SAAS,EAAE;QAAE,KAAK,EAAE,MAAM,CAAC;QAAC,MAAM,EAAE,MAAM,CAAA;KAAE,CAAC;CAC9C;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;EA2DA,CAAC;AAEF,eAAO,MAAM,kBAAkB,GAAI,cAAc,GAAG,uBAanD,CAAC;AAEF,eAAO,MAAM,8BAA8B,GAAU,SAAS,GAAG;;;;;;;;;;;;;;EAYhE,CAAC;AAEF,eAAO,MAAM,mBAAmB,GAAI,SAAS,GAAG;;;;;CAuB/C,CAAC"}

package/dist/utils.js

@@ -0,0 +1,142 @@
+export const uploadDataToFile = async (data) => {
+    const formData = new FormData();
+    const blob = new Blob([JSON.stringify(data, null, 2)], {
+        type: "application/json",
+    });
+    formData.append("file", blob, "data.json");
+    const res = await fetch(process.env.UPLOAD_API_URL ?? "", {
+        method: "POST",
+        body: formData,
+    });
+    if (!res.ok) {
+        throw new Error(`Failed to upload data to file: ${res.status} ${res.statusText}`);
+    }
+    return res.json();
+};
+export const processSubtitles = async (project) => {
+    try {
+        const uploadResult = await uploadDataToFile(project);
+        // Use stderr for logs so stdio JSON-RPC traffic on stdout is not corrupted.
+        console.error("Upload Result", uploadResult);
+        return uploadResult;
+    }
+    catch (err) {
+        console.error("Error processing subtitles", err);
+        return {
+            success: false,
+            error: "Error uploading project to file",
+            project: project,
+        };
+    }
+};
+export const processSubtitlesToProject = async ({ subtitles, videoUrl, duration, videoSize, }) => {
+    return {
+        properties: {
+            width: videoSize.width || 720,
+            height: videoSize.height || 1280,
+        },
+        tracks: [
+            {
+                id: "video",
+                type: "video",
+                elements: [
+                    {
+                        id: "video",
+                        type: "video",
+                        s: 0,
+                        e: duration,
+                        props: {
+                            src: videoUrl,
+                            width: videoSize.width || 720,
+                            height: videoSize.height || 1280,
+                        },
+                    },
+                ],
+            },
+            {
+                id: "subtitle",
+                type: "caption",
+                props: {
+                    capStyle: "highlight_bg",
+                    font: {
+                        size: 50,
+                        weight: 700,
+                        family: "Bangers",
+                    },
+                    colors: {
+                        text: "#ffffff",
+                        highlight: "#ff4081",
+                        bgColor: "#444444",
+                    },
+                    lineWidth: 0.35,
+                    stroke: "#000000",
+                    fontWeight: 700,
+                    shadowOffset: [-3, 3],
+                    shadowColor: "#000000",
+                    x: 0,
+                    y: 200,
+                    applyToAll: true,
+                },
+                elements: subtitles.map((subtitle, index) => ({
+                    id: `subtitle-${index}`,
+                    type: "caption",
+                    s: subtitle.s / 1000,
+                    e: subtitle.e / 1000,
+                    t: subtitle.t,
+                })),
+            },
+        ],
+        version: 1,
+    };
+};
+export const getTwickStudioLink = (uploadResult) => {
+    if (!uploadResult?.success)
+        return;
+    const fileUrl = uploadResult.result?.[0]?.url;
+    if (!fileUrl)
+        return;
+    const template = process.env.TWICK_STUDIO_URL;
+    if (!template)
+        return;
+    // If TWICK_STUDIO_URL contains a $project placeholder, substitute it
+    if (template.includes("$project")) {
+        return template.replace("$project", encodeURIComponent(fileUrl));
+    }
+};
+export const returnProjectAsTwickStudioLink = async (project) => {
+    const uploadResult = await uploadDataToFile(project);
+    const twickStudioLink = getTwickStudioLink(uploadResult);
+    if (twickStudioLink) {
+        return {
+            content: [
+                { type: "text", text: `Open link in browser: ${twickStudioLink}` },
+            ],
+            structuredContent: { success: true, result: twickStudioLink },
+        };
+    }
+    return returnProjectAsFile(project);
+};
+export const returnProjectAsFile = (project) => {
+    const data = Buffer.from(JSON.stringify(project, null, 2)).toString("base64");
+    const result = {
+        file: {
+            name: "project.json",
+            content: data,
+            encoding: "base64",
+            mimeType: "application/json",
+        },
+        metadata: {
+            size: data.length,
+            created: new Date().toISOString(),
+        },
+    };
+    return {
+        content: [
+            {
+                type: "text",
+                text: JSON.stringify(result, null, 2),
+            },
+        ],
+    };
+};
+//# sourceMappingURL=utils.js.map

package/dist/utils.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"utils.js","sourceRoot":"","sources":["../utils.ts"],"names":[],"mappings":"AAAA,MAAM,CAAC,MAAM,gBAAgB,GAAG,KAAK,EAAE,IAAS,EAAE,EAAE;IAClD,MAAM,QAAQ,GAAG,IAAI,QAAQ,EAAE,CAAC;IAChC,MAAM,IAAI,GAAG,IAAI,IAAI,CAAC,CAAC,IAAI,CAAC,SAAS,CAAC,IAAI,EAAE,IAAI,EAAE,CAAC,CAAC,CAAC,EAAE;QACrD,IAAI,EAAE,kBAAkB;KACzB,CAAC,CAAC;IAEH,QAAQ,CAAC,MAAM,CAAC,MAAM,EAAE,IAAI,EAAE,WAAW,CAAC,CAAC;IAE3C,MAAM,GAAG,GAAG,MAAM,KAAK,CAAC,OAAO,CAAC,GAAG,CAAC,cAAc,IAAI,EAAE,EAAE;QACxD,MAAM,EAAE,MAAM;QACd,IAAI,EAAE,QAAQ;KACf,CAAC,CAAC;IAEH,IAAI,CAAC,GAAG,CAAC,EAAE,EAAE,CAAC;QACZ,MAAM,IAAI,KAAK,CACb,kCAAkC,GAAG,CAAC,MAAM,IAAI,GAAG,CAAC,UAAU,EAAE,CACjE,CAAC;IACJ,CAAC;IAED,OAAO,GAAG,CAAC,IAAI,EAAE,CAAC;AACpB,CAAC,CAAC;AAEF,MAAM,CAAC,MAAM,gBAAgB,GAAG,KAAK,EAAE,OAAY,EAAE,EAAE;IACrD,IAAI,CAAC;QACH,MAAM,YAAY,GAAG,MAAM,gBAAgB,CAAC,OAAO,CAAC,CAAC;QACrD,4EAA4E;QAC5E,OAAO,CAAC,KAAK,CAAC,eAAe,EAAE,YAAY,CAAC,CAAC;QAC7C,OAAO,YAAY,CAAC;IACtB,CAAC;IAAC,OAAO,GAAG,EAAE,CAAC;QACb,OAAO,CAAC,KAAK,CAAC,4BAA4B,EAAE,GAAG,CAAC,CAAC;QACjD,OAAO;YACL,OAAO,EAAE,KAAK;YACd,KAAK,EAAE,iCAAiC;YACxC,OAAO,EAAE,OAAO;SACjB,CAAC;IACJ,CAAC;AACH,CAAC,CAAC;AAEF,MAAM,CAAC,MAAM,yBAAyB,GAAG,KAAK,EAAE,EAC9C,SAAS,EACT,QAAQ,EACR,QAAQ,EACR,SAAS,GAMV,EAAE,EAAE;IACH,OAAO;QACL,UAAU,EAAE;YACV,KAAK,EAAE,SAAS,CAAC,KAAK,IAAI,GAAG;YAC7B,MAAM,EAAE,SAAS,CAAC,MAAM,IAAI,IAAI;SACjC;QACD,MAAM,EAAE;YACN;gBACE,EAAE,EAAE,OAAO;gBACX,IAAI,EAAE,OAAO;gBACb,QAAQ,EAAE;oBACR;wBACE,EAAE,EAAE,OAAO;wBACX,IAAI,EAAE,OAAO;wBACb,CAAC,EAAE,CAAC;wBACJ,CAAC,EAAE,QAAQ;wBACX,KAAK,EAAE;4BACL,GAAG,EAAE,QAAQ;4BACb,KAAK,EAAE,SAAS,CAAC,KAAK,IAAI,GAAG;4BAC7B,MAAM,EAAE,SAAS,CAAC,MAAM,IAAI,IAAI;yBACjC;qBACF;iBACF;aACF;YACD;gBACE,EAAE,EAAE,UAAU;gBACd,IAAI,EAAE,SAAS;gBACf,KAAK,EAAE;oBACL,QAAQ,EAAE,cAAc;oBACxB,IAAI,EAAE;wBACJ,IAAI,EAAE,EAAE;wBACR,MAAM,EAAE,GAAG;wBACX,MAAM,EAAE,SAAS;qBAClB;oBACD,MAAM,EAAE;wBACN,IAAI,EAAE,SAAS;wBACf,SAAS,EAAE,SAAS;wBACpB,OAAO,EAAE,SAAS;qBACnB;oBACD,SAAS,EAAE,IAAI;oBACf,MAAM,EAAE,SAAS;oBACjB,UAAU,EAAE,GAAG;oBACf,YAAY,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC;oBACrB,WAAW,EAAE,SAAS;oBACtB,CAAC,EAAE,CAAC;oBACJ,CAAC,EAAE,GAAG;oBACN,UAAU,EAAE,IAAI;iBACjB;gBACD,QAAQ,EAAE,SAAS,CAAC,GAAG,CAAC,CAAC,QAAa,EAAE,KAAa,EAAE,EAAE,CAAC,CAAC;oBACzD,EAAE,EAAE,YAAY,KAAK,EAAE;oBACvB,IAAI,EAAE,SAAS;oBACf,CAAC,EAAE,QAAQ,CAAC,CAAC,GAAG,IAAI;oBACpB,CAAC,EAAE,QAAQ,CAAC,CAAC,GAAG,IAAI;oBACpB,CAAC,EAAE,QAAQ,CAAC,CAAC;iBACd,CAAC,CAAC;aACJ;SACF;QACD,OAAO,EAAE,CAAC;KACX,CAAC;AACJ,CAAC,CAAC;AAEF,MAAM,CAAC,MAAM,kBAAkB,GAAG,CAAC,YAAiB,EAAE,EAAE;IACtD,IAAI,CAAC,YAAY,EAAE,OAAO;QAAE,OAAO;IAEnC,MAAM,OAAO,GAAuB,YAAY,CAAC,MAAM,EAAE,CAAC,CAAC,CAAC,EAAE,GAAG,CAAC;IAClE,IAAI,CAAC,OAAO;QAAE,OAAO;IAErB,MAAM,QAAQ,GAAG,OAAO,CAAC,GAAG,CAAC,gBAAgB,CAAC;IAC9C,IAAI,CAAC,QAAQ;QAAE,OAAO;IAEtB,qEAAqE;IACrE,IAAI,QAAQ,CAAC,QAAQ,CAAC,UAAU,CAAC,EAAE,CAAC;QAClC,OAAO,QAAQ,CAAC,OAAO,CAAC,UAAU,EAAE,kBAAkB,CAAC,OAAO,CAAC,CAAC,CAAC;IACnE,CAAC;AACH,CAAC,CAAC;AAEF,MAAM,CAAC,MAAM,8BAA8B,GAAG,KAAK,EAAE,OAAY,EAAE,EAAE;IACnE,MAAM,YAAY,GAAG,MAAM,gBAAgB,CAAC,OAAO,CAAC,CAAC;IACrD,MAAM,eAAe,GAAG,kBAAkB,CAAC,YAAY,CAAC,CAAC;IACzD,IAAI,eAAe,EAAE,CAAC;QACpB,OAAO;YACL,OAAO,EAAE;gBACP,EAAE,IAAI,EAAE,MAAM,EAAE,IAAI,EAAE,yBAAyB,eAAe,EAAE,EAAE;aACnE;YACD,iBAAiB,EAAE,EAAE,OAAO,EAAE,IAAI,EAAE,MAAM,EAAE,eAAe,EAAE;SAC9D,CAAC;IACJ,CAAC;IACD,OAAO,mBAAmB,CAAC,OAAO,CAAC,CAAC;AACtC,CAAC,CAAC;AAEF,MAAM,CAAC,MAAM,mBAAmB,GAAG,CAAC,OAAY,EAAE,EAAE;IAClD,MAAM,IAAI,GAAG,MAAM,CAAC,IAAI,CAAC,IAAI,CAAC,SAAS,CAAC,OAAO,EAAE,IAAI,EAAE,CAAC,CAAC,CAAC,CAAC,QAAQ,CAAC,QAAQ,CAAC,CAAC;IAC9E,MAAM,MAAM,GAAG;QACb,IAAI,EAAE;YACJ,IAAI,EAAE,cAAc;YACpB,OAAO,EAAE,IAAI;YACb,QAAQ,EAAE,QAAQ;YAClB,QAAQ,EAAE,kBAAkB;SAC7B;QACD,QAAQ,EAAE;YACR,IAAI,EAAE,IAAI,CAAC,MAAM;YACjB,OAAO,EAAE,IAAI,IAAI,EAAE,CAAC,WAAW,EAAE;SAClC;KACF,CAAC;IAEF,OAAO;QACL,OAAO,EAAE;YACP;gBACE,IAAI,EAAE,MAAM;gBACZ,IAAI,EAAE,IAAI,CAAC,SAAS,CAAC,MAAM,EAAE,IAAI,EAAE,CAAC,CAAC;aACtC;SACF;KACF,CAAC;AACJ,CAAC,CAAC"}

package/install.js

@@ -0,0 +1,145 @@
+#!/usr/bin/env node
+
+/**
+ * Installation script for twick-mcp-agent Claude Desktop extension
+ * This script:
+ * 1. Installs npm dependencies
+ * 2. Builds the TypeScript project
+ * 3. Merges the MCP server config into Claude Desktop's config file
+ */
+
+import { execSync } from "child_process";
+import { readFileSync, writeFileSync, mkdirSync, existsSync } from "fs";
+import { join, dirname, resolve } from "path";
+import { fileURLToPath } from "url";
+import { homedir } from "os";
+
+const __filename = fileURLToPath(import.meta.url);
+const __dirname = dirname(__filename);
+const PROJECT_ROOT = resolve(__dirname);
+
+// Platform-specific Claude Desktop config paths
+function getClaudeConfigPath() {
+  const platform = process.platform;
+  if (platform === "win32") {
+    // Windows: %APPDATA%\Claude\claude_desktop_config.json
+    return join(process.env.APPDATA || "", "Claude", "claude_desktop_config.json");
+  } else if (platform === "darwin") {
+    // macOS: ~/Library/Application Support/Claude/claude_desktop_config.json
+    return join(homedir(), "Library", "Application Support", "Claude", "claude_desktop_config.json");
+  } else {
+    // Linux: ~/.config/Claude/claude_desktop_config.json
+    return join(homedir(), ".config", "Claude", "claude_desktop_config.json");
+  }
+}
+
+function mergeConfig(existingConfig, newServerConfig) {
+  const config = existingConfig || { mcpServers: {} };
+  
+  if (!config.mcpServers) {
+    config.mcpServers = {};
+  }
+
+  // Merge the new server config
+  Object.assign(config.mcpServers, newServerConfig.mcpServers);
+
+  return config;
+}
+
+function main() {
+  console.log("🚀 Installing twick-mcp-agent for Claude Desktop...\n");
+
+  try {
+    // Step 1: Install dependencies
+    console.log("📦 Installing npm dependencies...");
+    execSync("npm install", { cwd: PROJECT_ROOT, stdio: "inherit" });
+    console.log("✅ Dependencies installed\n");
+
+    // Step 2: Build the project
+    console.log("🔨 Building TypeScript project...");
+    execSync("npm run build", { cwd: PROJECT_ROOT, stdio: "inherit" });
+    console.log("✅ Build completed\n");
+
+    // Step 3: Read the MCP server config template
+    const configTemplatePath = join(PROJECT_ROOT, "claude_desktop_config.json");
+    if (!existsSync(configTemplatePath)) {
+      throw new Error(`Config template not found: ${configTemplatePath}`);
+    }
+
+    const configTemplate = JSON.parse(
+      readFileSync(configTemplatePath, "utf-8")
+    );
+
+    // Step 4: Update paths in config to be absolute
+    const serverConfig = { ...configTemplate };
+    const serverName = Object.keys(serverConfig.mcpServers)[0];
+    const serverEntry = serverConfig.mcpServers[serverName];
+    
+    // Make the args path absolute
+    if (serverEntry.args && serverEntry.args[0]) {
+      serverEntry.args[0] = join(PROJECT_ROOT, "dist", "stdio-server.js");
+    }
+
+    // Make GOOGLE_APPLICATION_CREDENTIALS path absolute if it's a relative path
+    if (serverEntry.env?.GOOGLE_APPLICATION_CREDENTIALS) {
+      const credsPath = serverEntry.env.GOOGLE_APPLICATION_CREDENTIALS;
+      if (!credsPath.startsWith("/") && !credsPath.match(/^[A-Z]:/)) {
+        // Relative path, make it absolute
+        serverEntry.env.GOOGLE_APPLICATION_CREDENTIALS = join(
+          PROJECT_ROOT,
+          credsPath
+        );
+      }
+    }
+
+    // Step 5: Merge into Claude Desktop config
+    const claudeConfigPath = getClaudeConfigPath();
+    console.log(`📝 Updating Claude Desktop config: ${claudeConfigPath}`);
+
+    let existingConfig = null;
+    if (existsSync(claudeConfigPath)) {
+      try {
+        existingConfig = JSON.parse(readFileSync(claudeConfigPath, "utf-8"));
+        console.log("✅ Found existing Claude Desktop config");
+      } catch (err) {
+        console.warn(`⚠️  Could not parse existing config: ${err.message}`);
+        console.log("📝 Creating new config file");
+      }
+    } else {
+      console.log("📝 Creating new Claude Desktop config file");
+      // Ensure directory exists
+      mkdirSync(dirname(claudeConfigPath), { recursive: true });
+    }
+
+    const mergedConfig = mergeConfig(existingConfig, serverConfig);
+    writeFileSync(
+      claudeConfigPath,
+      JSON.stringify(mergedConfig, null, 2) + "\n",
+      "utf-8"
+    );
+
+    console.log("✅ Configuration merged successfully\n");
+
+    // Step 6: Summary
+    console.log("🎉 Installation complete!\n");
+    console.log("📋 Next steps:");
+    console.log("   1. Edit your Claude Desktop config to set your environment variables:");
+    console.log(`      ${claudeConfigPath}`);
+    console.log("   2. Update the following in the 'twick-mcp-agent' section:");
+    console.log("      - GOOGLE_CLOUD_PROJECT: Your GCP project ID");
+    console.log("      - GOOGLE_APPLICATION_CREDENTIALS: Path to your GCP service account key");
+    console.log("      - UPLOAD_API_URL: Your upload API endpoint (optional)");
+    console.log("      - TWICK_STUDIO_URL: Your Twick Studio URL (optional)");
+    console.log("   3. Restart Claude Desktop to load the new MCP server");
+    console.log("\n✨ The 'generate-subtitles' tool will be available in Claude Desktop!\n");
+  } catch (error) {
+    console.error("\n❌ Installation failed:");
+    console.error(error.message);
+    if (error.stdout) console.error(error.stdout.toString());
+    if (error.stderr) console.error(error.stderr.toString());
+    process.exit(1);
+  }
+}
+
+main();
+

package/package.json

@@ -0,0 +1,61 @@
+{
+  "name": "@twick/mcp-agent",
+  "version": "0.14.15",
+  "description": "MCP server for Claude Desktop that generates video captions using Google Vertex AI and integrates with Twick Studio",
+  "main": "dist/stdio-server.js",
+  "type": "module",
+  "bin": {
+    "twick-mcp-agent": "./dist/stdio-server.js"
+  },
+  "files": [
+    "dist/**/*",
+    "claude_desktop_config.json",
+    "README.md",
+    "install.js"
+  ],
+  "publishConfig": {
+    "access": "public"
+  },
+  "scripts": {
+    "build": "tsc",
+    "start": "node dist/stdio-server.js",
+    "dev": "tsc && node dist/stdio-server.js",
+    "install-claude": "node install.js",
+    "prepublishOnly": "pnpm run build",
+    "test": "echo \"Error: no test specified\" && exit 1"
+  },
+  "keywords": [
+    "mcp",
+    "model-context-protocol",
+    "claude-desktop",
+    "video-transcription",
+    "captions",
+    "subtitle",
+    "google-vertex-ai",
+    "gemini",
+    "twick-studio"
+  ],
+  "author": "",
+  "license": "SEE LICENSE IN LICENSE.md",
+  "repository": {
+    "type": "git",
+    "url": ""
+  },
+  "engines": {
+    "node": ">=18.0.0"
+  },
+  "dependencies": {
+    "@modelcontextprotocol/sdk": "^1.20.0",
+    "@google/genai": "^1.0.0",
+    "@ffmpeg-installer/ffmpeg": "^1.1.0",
+    "@ffprobe-installer/ffprobe": "^2.0.0",
+    "dotenv": "^16.4.5",
+    "express": "^5.1.0",
+    "zod": "^3.25.76"
+  },
+  "devDependencies": {
+    "@types/express": "^5.0.0",
+    "@types/node": "^24.7.2",
+    "typescript": "^5.9.3"
+  }
+}