pi-media-models 0.1.2 → 0.1.4

package/README.md

@@ -3,81 +3,193 @@
 [![npm version](https://img.shields.io/npm/v/pi-media-models.svg)](https://www.npmjs.com/package/pi-media-models)
 [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
 
-**Provider-neutral multimodal generation extension for Pi Coding Agent.**
+Provider-neutral multimodal generation extension for [Pi Coding Agent](https://pi.dev). Generate images, video, audio, and speech across 9 providers with a single unified interface.
 
-This extension seamlessly bridges Pi's reasoning capabilities with top-tier AI media generation platforms. It abstracts away complex multi-part uploads, background task polling, CDN hosting, and API differences, exposing exactly **6 unified tools** for the agent.
+---
+
+## Installation
+
+Run this once in any terminal:
+
+```bash
+pi install npm:pi-media-models
+```
+
+That's it. Pi downloads the package, registers the extension, and loads the skill guide automatically. No manual configuration needed to get started.
+
+If Pi is already running, type `/reload` in the chat to activate it in the current session.
+
+> **Alternative:** Install directly from source
+> ```bash
+> pi install git:github.com/luffysolution-svg/pi-media-models
+> ```
 
-## ✨ Features
+---
+
+## Quick Start
+
+After installation, open Pi and start chatting. No extra steps required if you have environment variables set for your chosen provider.
 
-- **Unified Interface**: One request format (`provider`, `model`, `prompt`, `referenceImages`, etc.) maps automatically to the correct capability across providers.
-- **Auto-Download**: Output media (images, videos, audio) is automatically downloaded and saved to a local directory (`~/.pi/agent/media/outputs/`) using atomic `.part` renames. LLM context remains pristine and only receives local file paths.
-- **Smart Input Resolution**: Pass local paths (`C:/...`), file URIs (`file://`), standard URLs (`http(s)://`), or base64 (`data:...`). The router transparently handles multipart uploads, base64 encoding, or CDN pre-uploading (e.g., for `fal.ai`).
-- **Resilient Polling**: Advanced `MediaJob` processing handles asynchronous Long-Running Operations (LROs), 429 rate limits (respecting `Retry-After`), and timeouts. Supports remote job cancellation where supported by the provider.
+**Example prompts:**
 
-## 📦 Installation
+```
+Generate an image of a misty mountain at sunrise using fal.ai
+```
 
-This extension is built for the **Pi Coding Agent**. It includes both the executable extension logic and the LLM `SKILL.md` prompt context.
+```
+Use xAI to create a 5-second video of ocean waves with audio
+```
 
-Install natively inside your Pi environment:
+```
+Convert this text to speech using OpenAI: "Hello, world"
+```
+
+```
+What image/video providers do I have configured?
+```
+
+Pi will automatically call the right tool, handle uploads and polling, and save the result to your local disk. It reports the saved file path when done.
+
+---
+
+## API Key Setup
+
+The extension reads keys from environment variables. Set whichever providers you want to use:
+
+| Provider | Environment Variable |
+|---|---|
+| fal.ai | `FAL_KEY` |
+| OpenAI | `OPENAI_API_KEY` |
+| xAI (Grok) | `XAI_API_KEY` |
+| Google Gemini | `GEMINI_API_KEY` |
+| DashScope / QwenCloud | `DASHSCOPE_API_KEY` |
+| OpenRouter | `OPENROUTER_API_KEY` |
+| Atlas | `ATLAS_API_KEY` |
+| Vertex AI | `GOOGLE_APPLICATION_CREDENTIALS` (ADC) |
+
+**macOS / Linux** — add to `~/.zshrc` or `~/.bashrc`:
 
 ```bash
-# Install via NPM (Recommended)
-pi install npm:pi-media-models
+export FAL_KEY="your-key"
+export XAI_API_KEY="your-key"
+```
+
+**Windows** — run in PowerShell (persists across reboots):
 
-# Or install directly from GitHub
-pi install git:github.com/luffysolution-svg/pi-media-models
+```powershell
+[System.Environment]::SetEnvironmentVariable("FAL_KEY", "your-key", "User")
+[System.Environment]::SetEnvironmentVariable("XAI_API_KEY", "your-key", "User")
 ```
 
-Once installed, simply type `/reload` in your active Pi session to apply the extension.
+Then restart Pi (or your terminal) for the variables to take effect.
 
-## 🔌 Supported Providers
+### Alternative: Config File
 
-| Provider | Adapter | Supported Capabilities | Required Env Var |
-|---|---|---|---|
-| **OpenRouter** | `OpenRouterAdapter` | Image Generation, Async Video, TTS | `OPENROUTER_API_KEY` |
-| **fal.ai** | `FalAdapter` | Images, Video, Audio, TTS, STT (Queue/CDN integration) | `FAL_KEY` |
-| **Google Gemini API** | `GoogleMediaAdapter` | Gemini/Imagen, Veo, Lyria, TTS, STT | `GEMINI_API_KEY` |
-| **Google Vertex AI** | `GoogleMediaAdapter` | ADC, Imagen/Gemini, Veo, Lyria, TTS, STT | ADC credentials |
-| **DashScope / 百炼** | `DashScopeAdapter` | Qwen/Wan Images, Wan Video, Fun-Music, TTS/STT | `DASHSCOPE_API_KEY` |
-| **QwenCloud** | `DashScopeAdapter` | Same as DashScope (International endpoints) | `DASHSCOPE_API_KEY` |
-| **OpenAI API** | `OpenAIAdapter` | Image Gen/Edit (DALL-E), TTS, STT (Whisper) | `OPENAI_API_KEY` |
-| **xAI (Grok Imagine)** | `XAIAdapter` | Image Gen/Edit, T2V/I2V, Reference-to-Video, Edit, Extend | `XAI_API_KEY` |
-| **Atlas API** | `AtlasAdapter` | Sync/Async Image Gen & Edit, Video LROs | `ATLAS_API_KEY` |
+If you prefer not to use environment variables, create the file `~/.pi/agent/media-models.json`:
 
-## ⚙️ Configuration
+```json
+{
+  "providerOptions": {
+    "fal": { "apiKey": "your-fal-key" },
+    "xai": { "apiKey": "xai-your-key" },
+    "openai": { "apiKey": "sk-your-key" }
+  }
+}
+```
 
-API Keys can be provided as standard environment variables. Alternatively, you can configure them (along with advanced options) via JSON configuration.
+Keys in the config file take precedence over environment variables for that provider.
 
-**Configuration Path**: 
-`~/.pi/agent/media-models.json` (Global) or `<project_root>/.pi/media-models.json` (Project-specific).
+**Vertex AI** requires a service account JSON file path instead of an API key:
 
 ```json
 {
-  "outputDir": "/path/to/custom/output/directory",
   "providerOptions": {
     "vertex": {
-      "credentialsFile": "/path/to/vertex-service-account.json",
+      "credentialsFile": "/path/to/service-account.json",
       "project": "my-gcp-project",
       "location": "us-central1"
-    },
-    "dashscope": {
-      "baseUrl": "https://<workspace_id>.cn-beijing.maas.aliyuncs.com"
     }
-  },
+  }
+}
+```
+
+---
+
+## Output Files
+
+All generated media is automatically downloaded and saved to:
+
+```
+~/.pi/agent/media/outputs/
+```
+
+To use a different directory, add `outputDir` to the config file:
+
+```json
+{
+  "outputDir": "/Users/alice/Pictures/ai-output"
+}
+```
+
+---
+
+## Supported Providers
+
+| Provider | Capabilities | Env Var |
+|---|---|---|
+| **fal.ai** | Images, Video, Audio, TTS, STT | `FAL_KEY` |
+| **xAI (Grok Imagine)** | Image gen/edit, T2V, I2V, video extend | `XAI_API_KEY` |
+| **OpenAI** | Image gen/edit (DALL-E), TTS, STT (Whisper) | `OPENAI_API_KEY` |
+| **Google Gemini** | Imagen, Gemini image, Veo video, Lyria audio, TTS | `GEMINI_API_KEY` |
+| **Google Vertex AI** | Same as Gemini, enterprise ADC auth | ADC JSON |
+| **DashScope / 百炼** | Qwen image, Wan video, music, TTS/STT | `DASHSCOPE_API_KEY` |
+| **QwenCloud** | Same as DashScope (international endpoints) | `DASHSCOPE_API_KEY` |
+| **OpenRouter** | Image gen, async video, TTS | `OPENROUTER_API_KEY` |
+| **Atlas** | Image gen/edit, video (sync + async) | `ATLAS_API_KEY` |
+
+To see which providers are active in your current setup, ask Pi:
+
+```
+What media providers do I have configured?
+```
+
+---
+
+## Available Tools
+
+The extension registers exactly 6 tools. Pi selects the correct one automatically based on your request.
+
+| Tool | Purpose |
+|---|---|
+| `media_models` | List providers, models, and capabilities |
+| `image_generate` | Generate images from text or reference images |
+| `image_edit` | Edit an existing image (inpainting, style, background) |
+| `video_generate` | Text-to-video, image-to-video, video extend/edit |
+| `audio_generate` | Generate music or raw audio |
+| `speech_generate` | TTS (text → audio file) or STT (audio → transcript) |
+
+---
+
+## Advanced: Custom OpenAI-Compatible Providers
+
+You can add any OpenAI-compatible API as a custom provider in the config file. Capabilities and endpoints must be declared explicitly — the extension does not probe `/models` to infer them.
+
+```json
+{
   "customProviders": [
     {
-      "id": "my-custom-ai",
+      "id": "my-gateway",
       "name": "Internal AI Gateway",
-      "baseUrl": "https://api.internal.com/v1",
+      "baseUrl": "https://api.internal.example.com/v1",
       "apiKeyEnv": "MY_INTERNAL_KEY",
       "auth": "bearer",
       "models": [
         {
-          "id": "internal-video-pro",
+          "id": "internal-image-v2",
           "vendor": "internal",
-          "capabilities": ["video.text_to_video"],
+          "capabilities": ["image.text_to_image"],
           "endpoints": {
-            "video.text_to_video": "/videos/generations"
+            "image.text_to_image": "/images/generations"
           }
         }
       ]
@@ -86,23 +198,24 @@ API Keys can be provided as standard environment variables. Alternatively, you c
 }
 ```
 
-*Note: Custom OpenAI-compatible endpoints require explicit capability mapping in the configuration, avoiding hallucinated unsupported paths from standard `/models` probing.*
+---
 
-## 🛠️ Exposed Tools
+## Security
 
-The extension registers the following unified tools for the reasoning agent:
+- API keys and Bearer tokens are automatically redacted (`[REDACTED]`) from all error messages and logs before being returned to the LLM context.
+- The config file (`media-models.json`) is listed in `.gitignore` and is never included in the npm package.
+- No telemetry. All requests go directly from your machine to the provider API.
 
-1. `media_models`: Lists providers, configured models, and capabilities.
-2. `image_generate`: Generate images from text, image, or multiple reference inputs.
-3. `image_edit`: Edit existing images (supports masks and multiple references).
-4. `video_generate`: Generates, edits, or extends videos. Automatically maps inputs (`referenceImages`, `inputVideo`, `duration`, `generateAudio`, etc.) to the provider's exact capability.
-5. `audio_generate`: Generate music or raw audio (separate from TTS).
-6. `speech_generate`: Handle TTS (Text-to-Speech) and STT (Speech-to-Text).
+---
 
-## 🔒 Security & Privacy
+## Updating
 
-- **No Key Logging**: API keys and Bearer tokens are redacted (`[REDACTED]`) from all error logs and HTTP outputs before being returned to the LLM.
-- **Stateless Configuration**: Configuration does not hardcode user secrets if initialized via environment variables.
+```bash
+pi update npm:pi-media-models
+```
 
 ---
-*Built for production multimodal orchestration inside Pi.*
+
+## License
+
+MIT

package/package.json

@@ -1,14 +1,17 @@
 {
   "name": "pi-media-models",
-  "version": "0.1.2",
+  "version": "0.1.4",
   "type": "module",
   "description": "Provider-neutral multimodal generation tools for Pi Coding Agent",
+  "keywords": [
+    "pi-package"
+  ],
   "pi": {
     "extensions": [
       "./index.ts"
     ],
     "skills": [
-      "./SKILL.md"
+      "./skills"
     ]
   },
   "scripts": {
@@ -21,6 +24,9 @@
     "google-auth-library": "^10.5.0",
     "typebox": "^1.0.62"
   },
+  "peerDependencies": {
+    "@earendil-works/pi-coding-agent": "*"
+  },
   "devDependencies": {
     "@earendil-works/pi-coding-agent": "0.84.4",
     "@types/node": "^24.10.0",
@@ -33,8 +39,8 @@
   },
   "files": [
     "src",
+    "skills",
     "index.ts",
-    "SKILL.md",
     "README.md"
   ]
 }