pi-media-models 0.1.3 → 0.1.5

package/README.md

@@ -10,13 +10,14 @@ This extension seamlessly bridges Pi's reasoning capabilities with top-tier AI m
 ## ✨ Features
 
 - **Unified Interface**: One request format (`provider`, `model`, `prompt`, `referenceImages`, etc.) maps automatically to the correct capability across providers.
+- **Config-First Authentication**: Manage all API keys and credentials directly in a single `media-models.json` configuration file — no cluttered environment variables needed.
 - **Auto-Download**: Output media (images, videos, audio) is automatically downloaded and saved to a local directory (`~/.pi/agent/media/outputs/`) using atomic `.part` renames. LLM context remains pristine and only receives local file paths.
-- **Smart Input Resolution**: Pass local paths (`C:/...`), file URIs (`file://`), standard URLs (`http(s)://`), or base64 (`data:...`). The router transparently handles multipart uploads, base64 encoding, or CDN pre-uploading (e.g., for `fal.ai`).
+- **Smart Input Resolution**: Pass local paths (`C:/...` or `/path/...`), file URIs (`file://`), standard URLs (`http(s)://`), or base64 (`data:...`). The router transparently handles multipart uploads, base64 encoding, or CDN pre-uploading (e.g., for `fal.ai`).
 - **Resilient Polling**: Advanced `MediaJob` processing handles asynchronous Long-Running Operations (LROs), 429 rate limits (respecting `Retry-After`), and timeouts. Supports remote job cancellation where supported by the provider.
 
 ## 📦 Installation
 
-This extension is built for the **Pi Coding Agent**. It includes both the executable extension logic and the LLM `SKILL.md` prompt context.
+This extension is built for the **Pi Coding Agent**. It bundles both the executable extension logic and the LLM `SKILL.md` prompt context.
 
 Install natively inside your Pi environment:
 
@@ -28,48 +29,91 @@ pi install npm:pi-media-models
 pi install git:github.com/luffysolution-svg/pi-media-models
 ```
 
-Once installed, simply type `/reload` in your active Pi session to apply the extension.
+Once installed, restart Pi or type `/reload` in your active session.
 
-## 🔌 Supported Providers
+## 🚀 Quick Usage
 
-| Provider | Adapter | Supported Capabilities | Required Env Var |
-|---|---|---|---|
-| **OpenRouter** | `OpenRouterAdapter` | Image Generation, Async Video, TTS | `OPENROUTER_API_KEY` |
-| **fal.ai** | `FalAdapter` | Images, Video, Audio, TTS, STT (Queue/CDN integration) | `FAL_KEY` |
-| **Google Gemini API** | `GoogleMediaAdapter` | Gemini/Imagen, Veo, Lyria, TTS, STT | `GEMINI_API_KEY` |
-| **Google Vertex AI** | `GoogleMediaAdapter` | ADC, Imagen/Gemini, Veo, Lyria, TTS, STT | ADC credentials |
-| **DashScope / 百炼** | `DashScopeAdapter` | Qwen/Wan Images, Wan Video, Fun-Music, TTS/STT | `DASHSCOPE_API_KEY` |
-| **QwenCloud** | `DashScopeAdapter` | Same as DashScope (International endpoints) | `DASHSCOPE_API_KEY` |
-| **OpenAI API** | `OpenAIAdapter` | Image Gen/Edit (DALL-E), TTS, STT (Whisper) | `OPENAI_API_KEY` |
-| **xAI (Grok Imagine)** | `XAIAdapter` | Image Gen/Edit, T2V/I2V, Reference-to-Video, Edit, Extend | `XAI_API_KEY` |
-| **Atlas API** | `AtlasAdapter` | Sync/Async Image Gen & Edit, Video LROs | `ATLAS_API_KEY` |
+After installation, the extension and its companion skill are active immediately. You can talk to Pi naturally — Pi knows how to query available models and route tasks to the appropriate tool.
+
+**Example Prompts:**
+
+```
+帮我用 fal.ai 画一张赛博朋克风格的雨夜街景，比例 16:9
+```
+
+```
+Use xAI to generate a 5-second video of ocean waves with audio
+```
+
+```
+使用 Atlas 编辑这张图片，把背景换成雪山：C:/assets/photo.jpg
+```
+
+```
+帮我查一下当前已配置好可用的多模态模型有哪些？
+```
+
+Pi 会自动调取对应工具、完成排队轮询与文件下载，并直接返回本地媒体文件的保存路径。
 
 ## ⚙️ Configuration
 
-API Keys can be provided as standard environment variables. Alternatively, you can configure them (along with advanced options) via JSON configuration.
+All API keys, custom endpoints, and output paths are configured directly in a single JSON file.
+
+**Configuration File Location:**
+- **Global (Recommended):** `~/.pi/agent/media-models.json`
+- **Project-Specific:** `<project_root>/.pi/media-models.json`
+
+### Example `media-models.json`
 
-**Configuration Path**: 
-`~/.pi/agent/media-models.json` (Global) or `<project_root>/.pi/media-models.json` (Project-specific).
+Create or edit `~/.pi/agent/media-models.json`:
 
 ```json
 {
-  "outputDir": "/path/to/custom/output/directory",
+  "outputDir": "~/.pi/agent/media/outputs",
   "providerOptions": {
+    "fal": {
+      "apiKey": "fal-xxxxxxxxxxxxxxxxxxxx"
+    },
+    "xai": {
+      "apiKey": "xai-xxxxxxxxxxxxxxxxxxxx"
+    },
+    "atlas": {
+      "apiKey": "sk-xxxxxxxxxxxxxxxxxxxx"
+    },
+    "dashscope": {
+      "apiKey": "sk-xxxxxxxxxxxxxxxxxxxx"
+    },
+    "openai": {
+      "apiKey": "sk-xxxxxxxxxxxxxxxxxxxx"
+    },
+    "gemini": {
+      "apiKey": "AIzaxxxxxxxxxxxxxxxxxxxx"
+    },
+    "openrouter": {
+      "apiKey": "sk-or-xxxxxxxxxxxxxxxxxxxx"
+    },
     "vertex": {
       "credentialsFile": "/path/to/vertex-service-account.json",
       "project": "my-gcp-project",
       "location": "us-central1"
-    },
-    "dashscope": {
-      "baseUrl": "https://<workspace_id>.cn-beijing.maas.aliyuncs.com"
     }
-  },
+  }
+}
+```
+
+> **Note**: You only need to fill in the providers you plan to use. Unused providers can simply be omitted.
+
+### Custom OpenAI-Compatible Providers
+
+You can connect any third-party or internal OpenAI-compatible media gateway by declaring it in `customProviders`:
+
+```json
+{
   "customProviders": [
     {
       "id": "my-custom-ai",
       "name": "Internal AI Gateway",
       "baseUrl": "https://api.internal.com/v1",
-      "apiKeyEnv": "MY_INTERNAL_KEY",
       "auth": "bearer",
       "models": [
         {
@@ -86,13 +130,25 @@ API Keys can be provided as standard environment variables. Alternatively, you c
 }
 ```
 
-*Note: Custom OpenAI-compatible endpoints require explicit capability mapping in the configuration, avoiding hallucinated unsupported paths from standard `/models` probing.*
+## 🔌 Supported Providers
+
+| Provider | Provider ID | Supported Capabilities | Config Option |
+|---|---|---|---|
+| **fal.ai** | `fal` | Images, Video, Audio, TTS, STT (Queue/CDN integration) | `providerOptions.fal.apiKey` |
+| **xAI (Grok Imagine)** | `xai` | Image Gen/Edit, T2V/I2V, Reference-to-Video, Video Extend | `providerOptions.xai.apiKey` |
+| **Atlas API** | `atlas` | Sync/Async Image Gen & Edit, Video LROs | `providerOptions.atlas.apiKey` |
+| **DashScope / 百炼** | `dashscope` | Qwen/Wan Images, Wan Video, Fun-Music, TTS/STT | `providerOptions.dashscope.apiKey` |
+| **QwenCloud** | `qwencloud` | Same as DashScope (International endpoints) | `providerOptions.qwencloud.apiKey` |
+| **OpenAI API** | `openai` | Image Gen/Edit (DALL-E), TTS, STT (Whisper) | `providerOptions.openai.apiKey` |
+| **Google Gemini API** | `gemini` | Gemini/Imagen, Veo, Lyria, TTS, STT | `providerOptions.gemini.apiKey` |
+| **Google Vertex AI** | `vertex` | ADC, Imagen/Gemini, Veo, Lyria, TTS, STT | `providerOptions.vertex.credentialsFile` |
+| **OpenRouter** | `openrouter` | Image Generation, Async Video, TTS | `providerOptions.openrouter.apiKey` |
 
 ## 🛠️ Exposed Tools
 
-The extension registers the following unified tools for the reasoning agent:
+The extension registers exactly 6 unified tools for the reasoning agent:
 
-1. `media_models`: Lists providers, configured models, and capabilities.
+1. `media_models`: Lists providers, configured models, and supported capabilities.
 2. `image_generate`: Generate images from text, image, or multiple reference inputs.
 3. `image_edit`: Edit existing images (supports masks and multiple references).
 4. `video_generate`: Generates, edits, or extends videos. Automatically maps inputs (`referenceImages`, `inputVideo`, `duration`, `generateAudio`, etc.) to the provider's exact capability.
@@ -102,7 +158,17 @@ The extension registers the following unified tools for the reasoning agent:
 ## 🔒 Security & Privacy
 
 - **No Key Logging**: API keys and Bearer tokens are redacted (`[REDACTED]`) from all error logs and HTTP outputs before being returned to the LLM.
-- **Stateless Configuration**: Configuration does not hardcode user secrets if initialized via environment variables.
+- **Local Downloads**: Media assets are fetched directly by your local client into your configured `outputDir` without third-party proxies.
+- **Git Ignored**: `media-models.json` is automatically ignored from git repositories to prevent accidental credential commits.
 
 ---
-*Built for production multimodal orchestration inside Pi.*
+
+## 🔄 Updating
+
+```bash
+pi update npm:pi-media-models
+```
+
+## 📄 License
+
+MIT

package/package.json

@@ -1,12 +1,18 @@
 {
   "name": "pi-media-models",
-  "version": "0.1.3",
+  "version": "0.1.5",
   "type": "module",
   "description": "Provider-neutral multimodal generation tools for Pi Coding Agent",
-  "keywords": ["pi-package"],
+  "keywords": [
+    "pi-package"
+  ],
   "pi": {
-    "extensions": ["./index.ts"],
-    "skills": ["./skills"]
+    "extensions": [
+      "./index.ts"
+    ],
+    "skills": [
+      "./skills"
+    ]
   },
   "scripts": {
     "typecheck": "tsc --noEmit",

package/skills/pi-media/SKILL.md

@@ -82,34 +82,28 @@ All generated media is automatically downloaded to `~/.pi/agent/media/outputs/`.
 
 ## Key Configuration
 
-If the user asks how to set up API keys, there are two methods:
-
-**Environment variables** (any shell or system env):
-
-| Provider | Variable |
-|---|---|
-| OpenAI | `OPENAI_API_KEY` |
-| fal.ai | `FAL_KEY` |
-| Gemini | `GEMINI_API_KEY` |
-| Vertex AI | `GOOGLE_APPLICATION_CREDENTIALS` (ADC) |
-| xAI | `XAI_API_KEY` |
-| DashScope / QwenCloud | `DASHSCOPE_API_KEY` |
-| OpenRouter | `OPENROUTER_API_KEY` |
-| Atlas | `ATLAS_API_KEY` |
-
-**Config file** (`~/.pi/agent/media-models.json` or `.pi/media-models.json` in the project root):
+All API keys and provider options are configured in `~/.pi/agent/media-models.json` (or `.pi/media-models.json` in the workspace root):
 
 ```json
 {
-  "outputDir": "/path/to/output",
+  "outputDir": "~/.pi/agent/media/outputs",
   "providerOptions": {
+    "fal": { "apiKey": "fal-..." },
     "xai": { "apiKey": "xai-..." },
+    "atlas": { "apiKey": "sk-..." },
+    "dashscope": { "apiKey": "sk-..." },
+    "openai": { "apiKey": "sk-..." },
+    "gemini": { "apiKey": "AIza..." },
+    "openrouter": { "apiKey": "sk-or-..." },
     "vertex": {
       "credentialsFile": "/path/to/service-account.json",
-      "project": "my-gcp-project"
+      "project": "my-gcp-project",
+      "location": "us-central1"
     }
   }
 }
 ```
 
-Keys in `providerOptions` take precedence over environment variables for that provider.
+Tell the user to edit `~/.pi/agent/media-models.json` directly to add or update API keys.
+
+*(Environment variables such as `FAL_KEY`, `OPENAI_API_KEY`, `XAI_API_KEY` are also checked as a fallback).*