pi-media-models 0.1.1 → 0.1.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/SKILL.md +115 -1
- package/package.json +1 -1
package/SKILL.md
CHANGED
|
@@ -1 +1,115 @@
|
|
|
1
|
-
|
|
1
|
+
---
|
|
2
|
+
name: pi-media
|
|
3
|
+
description: Multimodal media generation for Pi — images, video, audio, and speech across OpenAI, Gemini, Vertex AI, xAI, Atlas, DashScope, fal.ai, and OpenRouter. Use whenever the user asks to generate, edit, or transform media content of any kind.
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# Pi Media Models
|
|
7
|
+
|
|
8
|
+
This skill activates the `pi-media-models` extension, which exposes six unified tools for multimodal generation across all configured providers. Read this skill before making any media tool call.
|
|
9
|
+
|
|
10
|
+
## Available Tools
|
|
11
|
+
|
|
12
|
+
| Tool | When to use |
|
|
13
|
+
|---|---|
|
|
14
|
+
| `media_models` | List available providers, models, and capabilities |
|
|
15
|
+
| `image_generate` | Generate images from text or reference images |
|
|
16
|
+
| `image_edit` | Edit an existing image (inpainting, style, background) |
|
|
17
|
+
| `video_generate` | Text-to-video, image-to-video, extend, or edit video |
|
|
18
|
+
| `audio_generate` | Generate music or raw audio (not TTS) |
|
|
19
|
+
| `speech_generate` | TTS (text → speech) or STT (audio → transcript) |
|
|
20
|
+
|
|
21
|
+
## Workflow
|
|
22
|
+
|
|
23
|
+
### 1. Discover what is configured
|
|
24
|
+
|
|
25
|
+
When the user has not specified a provider and model, call `media_models` first to see which providers have `configured: true` and which capabilities they support. Never invent model names.
|
|
26
|
+
|
|
27
|
+
```
|
|
28
|
+
media_models({ capability: "image.text_to_image" })
|
|
29
|
+
```
|
|
30
|
+
|
|
31
|
+
### 2. Call the right tool
|
|
32
|
+
|
|
33
|
+
Pick the tool that matches the request, then pass `provider`, `model`, and `prompt` at minimum. Add optional parameters as needed.
|
|
34
|
+
|
|
35
|
+
```
|
|
36
|
+
image_generate({
|
|
37
|
+
provider: "fal",
|
|
38
|
+
model: "fal-ai/flux/schnell",
|
|
39
|
+
prompt: "a sunset over misty mountains",
|
|
40
|
+
aspectRatio: "16:9"
|
|
41
|
+
})
|
|
42
|
+
```
|
|
43
|
+
|
|
44
|
+
### 3. Pass input files as-is
|
|
45
|
+
|
|
46
|
+
Do **not** read files yourself or convert them to base64. The extension handles all file resolution internally. Pass:
|
|
47
|
+
|
|
48
|
+
- Absolute local path: `/Users/alice/photo.jpg` or `C:\Users\alice\photo.jpg`
|
|
49
|
+
- File URI: `file:///Users/alice/photo.jpg`
|
|
50
|
+
- Remote URL: `https://example.com/image.png`
|
|
51
|
+
- Data URI: `data:image/png;base64,...`
|
|
52
|
+
|
|
53
|
+
```
|
|
54
|
+
image_edit({
|
|
55
|
+
provider: "openai",
|
|
56
|
+
model: "gpt-image-1",
|
|
57
|
+
prompt: "remove the background",
|
|
58
|
+
inputImage: "/Users/alice/photo.jpg"
|
|
59
|
+
})
|
|
60
|
+
```
|
|
61
|
+
|
|
62
|
+
### 4. Report the output path
|
|
63
|
+
|
|
64
|
+
All generated media is automatically downloaded to `~/.pi/agent/media/outputs/`. The tool returns local `path` values — show these to the user directly.
|
|
65
|
+
|
|
66
|
+
## Common Parameters
|
|
67
|
+
|
|
68
|
+
| Parameter | Description |
|
|
69
|
+
|---|---|
|
|
70
|
+
| `provider` | Provider id: `openai`, `gemini`, `vertex`, `xai`, `atlas`, `dashscope`, `qwencloud`, `fal`, `openrouter` |
|
|
71
|
+
| `model` | Exact model id as returned by `media_models` |
|
|
72
|
+
| `prompt` | Required. Describe the desired output |
|
|
73
|
+
| `aspectRatio` | `"16:9"`, `"9:16"`, `"1:1"`, `"4:3"`, etc. |
|
|
74
|
+
| `resolution` | `"1024x1024"`, `"720p"`, `"1080p"` |
|
|
75
|
+
| `duration` | Seconds (number). For video and audio |
|
|
76
|
+
| `generateAudio` | `true` to include audio track with video (xAI, Veo, Atlas) |
|
|
77
|
+
| `referenceImages` | Array of image inputs for style or subject reference |
|
|
78
|
+
| `inputImage` | Single input image for edit tasks |
|
|
79
|
+
| `inputVideo` | Input video for extend or edit tasks |
|
|
80
|
+
| `seed` | Integer seed for reproducibility |
|
|
81
|
+
| `providerOptions` | Provider-specific overrides (e.g. `{ "async": false }`) |
|
|
82
|
+
|
|
83
|
+
## Key Configuration
|
|
84
|
+
|
|
85
|
+
If the user asks how to set up API keys, there are two methods:
|
|
86
|
+
|
|
87
|
+
**Environment variables** (any shell or system env):
|
|
88
|
+
|
|
89
|
+
| Provider | Variable |
|
|
90
|
+
|---|---|
|
|
91
|
+
| OpenAI | `OPENAI_API_KEY` |
|
|
92
|
+
| fal.ai | `FAL_KEY` |
|
|
93
|
+
| Gemini | `GEMINI_API_KEY` |
|
|
94
|
+
| Vertex AI | `GOOGLE_APPLICATION_CREDENTIALS` (ADC) |
|
|
95
|
+
| xAI | `XAI_API_KEY` |
|
|
96
|
+
| DashScope / QwenCloud | `DASHSCOPE_API_KEY` |
|
|
97
|
+
| OpenRouter | `OPENROUTER_API_KEY` |
|
|
98
|
+
| Atlas | `ATLAS_API_KEY` |
|
|
99
|
+
|
|
100
|
+
**Config file** (`~/.pi/agent/media-models.json` or `.pi/media-models.json` in the project root):
|
|
101
|
+
|
|
102
|
+
```json
|
|
103
|
+
{
|
|
104
|
+
"outputDir": "/path/to/output",
|
|
105
|
+
"providerOptions": {
|
|
106
|
+
"xai": { "apiKey": "xai-..." },
|
|
107
|
+
"vertex": {
|
|
108
|
+
"credentialsFile": "/path/to/service-account.json",
|
|
109
|
+
"project": "my-gcp-project"
|
|
110
|
+
}
|
|
111
|
+
}
|
|
112
|
+
}
|
|
113
|
+
```
|
|
114
|
+
|
|
115
|
+
Keys in `providerOptions` take precedence over environment variables for that provider.
|