@imgly/plugin-ai-apps-web 0.1.9 → 0.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +34 -0
- package/README.md +422 -60
- package/dist/ActiveAssetSource.d.ts +11 -7
- package/dist/index.mjs +401 -38
- package/dist/index.mjs.map +4 -4
- package/dist/types.d.ts +11 -7
- package/package.json +7 -3
package/CHANGELOG.md
CHANGED
|
@@ -2,6 +2,40 @@
|
|
|
2
2
|
|
|
3
3
|
## [Unreleased]
|
|
4
4
|
|
|
5
|
+
## [0.2.0] - 2025-07-15
|
|
6
|
+
|
|
7
|
+
### Breaking Changes
|
|
8
|
+
|
|
9
|
+
- [all] **Provider Initialization**: `initProvider` is replaced with `initializeProviders` and `initializeProvider` with a different signature
|
|
10
|
+
- [all] **Quick Actions Structure**: `provider.input.quickctions.actions` replaced with `provider.input.quickActions.supported`
|
|
11
|
+
- [all] **History Asset Sources**: Combined history asset source is now not added to the default asset libraries anymore. Add the following sources to any library entry (default or not):
|
|
12
|
+
- Image Generation History: `ly.img.ai.image-generation.history`
|
|
13
|
+
- Video Generation History: `ly.img.ai.video-generation.history`
|
|
14
|
+
- Audio Generation History: `ly.img.ai.audio-generation.history`
|
|
15
|
+
- Sticker Generation History: `ly.img.ai.sticker-generation.history`
|
|
16
|
+
- [all] **ID Format Standardization**: All `ly.img.ai/` prefixed IDs changed to use dot notation `ly.img.ai.` for consistency
|
|
17
|
+
- Panel IDs: `ly.img.ai/apps` → `ly.img.ai.apps`, `ly.img.ai/image-generation` → `ly.img.ai.image-generation`
|
|
18
|
+
- Dock IDs: `ly.img.ai/apps.dock` → `ly.img.ai.apps.dock`
|
|
19
|
+
|
|
20
|
+
### New Features
|
|
21
|
+
|
|
22
|
+
- [all] **Multiple Providers Support**: All plugin packages now support arrays of providers with automatic selection UI
|
|
23
|
+
- [image-generation] **Recraft20b Provider**: Added new Recraft20b (v2) provider via fal.ai with support for icon styles including `broken_line`, `colored_outline`, `colored_shapes`, and more
|
|
24
|
+
- [sticker-generation] **New Sticker Generation Plugin**: Added `@imgly/plugin-ai-sticker-generation-web` plugin for AI-powered sticker generation with support for text-to-sticker generation using Recraft20b provider with icon styles
|
|
25
|
+
- [image-generation] **Flux Pro Kontext Provider**: Added Flux Pro Kontext provider for image editing with style transfer and artist style options
|
|
26
|
+
- [image-generation] **Flux Pro Kontext Max Provider**: Added Flux Pro Kontext Max provider for enhanced image editing capabilities
|
|
27
|
+
- [video-generation] **Veo3 Provider**: Added Veo3 text-to-video provider via fal.ai with support for 16:9, 9:16, and 1:1 aspect ratios and 8-second duration
|
|
28
|
+
- [video-generation] **Kling Video V2.1 Master Providers**: Added Kling Video V2.1 Master providers for both text-to-video and image-to-video generation with configurable duration (5s/10s) and aspect ratios
|
|
29
|
+
- [text-generation] **Model Selection**: OpenAI and Anthropic providers now support configurable model selection through the `model` parameter
|
|
30
|
+
|
|
31
|
+
### Deprecated Features
|
|
32
|
+
|
|
33
|
+
- [all] **Middleware Configuration**: The `middleware` property in provider configurations is deprecated. Use `middlewares` instead. The old property will continue to work for now.
|
|
34
|
+
|
|
35
|
+
## [0.1.10] - 2025-06-20
|
|
36
|
+
|
|
37
|
+
- [all] Fix issue with GPT provider when using text provider
|
|
38
|
+
|
|
5
39
|
## [0.1.9] - 2025-06-05
|
|
6
40
|
|
|
7
41
|
- [all] Add support for custom headers
|
package/README.md
CHANGED
|
@@ -6,13 +6,14 @@ A plugin for orchestrating all AI generation capabilities in CreativeEditor SDK.
|
|
|
6
6
|
|
|
7
7
|
The `@imgly/plugin-ai-apps-web` package provides a unified interface for accessing and organizing all AI generation features within the CreativeEditor SDK. It combines image, video, audio, and text generation capabilities into a single cohesive user experience.
|
|
8
8
|
|
|
9
|
-
Features
|
|
9
|
+
### Key Features
|
|
10
10
|
|
|
11
|
-
-
|
|
12
|
-
-
|
|
13
|
-
-
|
|
14
|
-
-
|
|
15
|
-
-
|
|
11
|
+
- **Unified AI Experience**: Centralized access to all AI generation capabilities
|
|
12
|
+
- **AI Dock Component**: Single entry point for all AI features with loading indicators
|
|
13
|
+
- **Smart Provider Management**: Automatically organizes providers based on editor mode
|
|
14
|
+
- **Integrated History**: Seamless integration with asset libraries for generated content
|
|
15
|
+
- **Mode-Aware Interface**: Different UI layouts for Design and Video modes
|
|
16
|
+
- **Cross-Plugin Support**: Works with all IMG.LY AI generation plugins
|
|
16
17
|
|
|
17
18
|
## Installation
|
|
18
19
|
|
|
@@ -20,7 +21,7 @@ Features include:
|
|
|
20
21
|
npm install @imgly/plugin-ai-apps-web
|
|
21
22
|
```
|
|
22
23
|
|
|
23
|
-
## Usage
|
|
24
|
+
## Basic Usage
|
|
24
25
|
|
|
25
26
|
To use the plugin, import it and configure it with your preferred providers:
|
|
26
27
|
|
|
@@ -33,50 +34,56 @@ import { AnthropicProvider } from '@imgly/plugin-ai-text-generation-web/anthropi
|
|
|
33
34
|
import FalAiImage from '@imgly/plugin-ai-image-generation-web/fal-ai';
|
|
34
35
|
import FalAiVideo from '@imgly/plugin-ai-video-generation-web/fal-ai';
|
|
35
36
|
import ElevenLabs from '@imgly/plugin-ai-audio-generation-web/elevenlabs';
|
|
37
|
+
import FalAiSticker from '@imgly/plugin-ai-sticker-generation-web/fal-ai';
|
|
36
38
|
|
|
37
39
|
// Initialize CreativeEditor SDK
|
|
38
40
|
CreativeEditorSDK.create(domElement, {
|
|
39
|
-
|
|
40
|
-
|
|
41
|
+
license: 'your-license-key'
|
|
42
|
+
// Other configuration options...
|
|
41
43
|
}).then(async (cesdk) => {
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
|
|
45
|
-
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
|
|
49
|
-
|
|
50
|
-
|
|
51
|
-
|
|
52
|
-
|
|
53
|
-
|
|
54
|
-
|
|
55
|
-
|
|
56
|
-
|
|
57
|
-
|
|
58
|
-
|
|
59
|
-
|
|
60
|
-
|
|
61
|
-
|
|
62
|
-
|
|
63
|
-
|
|
64
|
-
|
|
65
|
-
|
|
66
|
-
|
|
67
|
-
|
|
68
|
-
|
|
69
|
-
|
|
70
|
-
|
|
71
|
-
|
|
72
|
-
|
|
73
|
-
|
|
74
|
-
|
|
44
|
+
// Add the AI Apps plugin
|
|
45
|
+
cesdk.addPlugin(
|
|
46
|
+
AiApps({
|
|
47
|
+
providers: {
|
|
48
|
+
// Text generation
|
|
49
|
+
text2text: AnthropicProvider({
|
|
50
|
+
proxyUrl: 'http://your-proxy-server.com/api/proxy'
|
|
51
|
+
}),
|
|
52
|
+
|
|
53
|
+
// Image generation
|
|
54
|
+
text2image: FalAiImage.RecraftV3({
|
|
55
|
+
proxyUrl: 'http://your-proxy-server.com/api/proxy'
|
|
56
|
+
}),
|
|
57
|
+
image2image: FalAiImage.GeminiFlashEdit({
|
|
58
|
+
proxyUrl: 'http://your-proxy-server.com/api/proxy'
|
|
59
|
+
}),
|
|
60
|
+
|
|
61
|
+
// Video generation (used in video mode)
|
|
62
|
+
text2video: FalAiVideo.MinimaxVideo01Live({
|
|
63
|
+
proxyUrl: 'http://your-proxy-server.com/api/proxy'
|
|
64
|
+
}),
|
|
65
|
+
image2video: FalAiVideo.MinimaxVideo01LiveImageToVideo({
|
|
66
|
+
proxyUrl: 'http://your-proxy-server.com/api/proxy'
|
|
67
|
+
}),
|
|
68
|
+
|
|
69
|
+
// Audio generation (used in video mode)
|
|
70
|
+
text2speech: ElevenLabs.MonolingualV1({
|
|
71
|
+
proxyUrl: 'http://your-proxy-server.com/api/proxy'
|
|
72
|
+
}),
|
|
73
|
+
text2sound: ElevenLabs.SoundEffects({
|
|
74
|
+
proxyUrl: 'http://your-proxy-server.com/api/proxy'
|
|
75
|
+
}),
|
|
76
|
+
|
|
77
|
+
// Sticker generation
|
|
78
|
+
text2sticker: FalAiSticker.Recraft20b({
|
|
79
|
+
proxyUrl: 'http://your-proxy-server.com/api/proxy'
|
|
75
80
|
})
|
|
76
|
-
|
|
81
|
+
}
|
|
82
|
+
})
|
|
83
|
+
);
|
|
77
84
|
|
|
78
|
-
|
|
79
|
-
|
|
85
|
+
// Position the AI dock button in the dock order
|
|
86
|
+
cesdk.ui.setDockOrder(['ly.img.ai.apps.dock', ...cesdk.ui.getDockOrder()]);
|
|
80
87
|
});
|
|
81
88
|
```
|
|
82
89
|
|
|
@@ -102,13 +109,98 @@ The `providers` object can include the following provider functions:
|
|
|
102
109
|
| `image2video` | `Provider<'video'>` | Provider for image-to-video generation (video mode only) |
|
|
103
110
|
| `text2speech` | `Provider<'audio'>` | Provider for text-to-speech generation (video mode only) |
|
|
104
111
|
| `text2sound` | `Provider<'audio'>` | Provider for sound effects generation (video mode only) |
|
|
112
|
+
| `text2sticker` | `Provider<'sticker'>` | Provider for sticker generation |
|
|
113
|
+
|
|
114
|
+
### Provider Selection Strategy
|
|
115
|
+
|
|
116
|
+
The plugin intelligently selects which providers to use based on the current editor mode:
|
|
117
|
+
|
|
118
|
+
#### Design Mode
|
|
119
|
+
- **Uses**: `text2text`, `text2image`, `image2image`, `text2sticker`
|
|
120
|
+
- **Focus**: Image, text, and sticker generation for design workflows
|
|
121
|
+
- **UI**: Shows AI apps cards for different generation types
|
|
122
|
+
|
|
123
|
+
#### Video Mode
|
|
124
|
+
- **Uses**: All providers including `text2video`, `image2video`, `text2speech`, `text2sound`, `text2sticker`
|
|
125
|
+
- **Focus**: Comprehensive media generation for video production
|
|
126
|
+
- **UI**: Shows AI apps cards for different generation types
|
|
127
|
+
|
|
128
|
+
## Advanced Configuration
|
|
129
|
+
|
|
130
|
+
### Multiple Providers per Type
|
|
131
|
+
|
|
132
|
+
You can configure multiple providers for the same generation type by passing an array. When multiple providers are configured, a selection box will be rendered in the AI app interface allowing users to choose between different providers:
|
|
133
|
+
|
|
134
|
+
```typescript
|
|
135
|
+
import FalAiImage from '@imgly/plugin-ai-image-generation-web/fal-ai';
|
|
136
|
+
import OpenAiImage from '@imgly/plugin-ai-image-generation-web/open-ai';
|
|
137
|
+
|
|
138
|
+
cesdk.addPlugin(
|
|
139
|
+
AiApps({
|
|
140
|
+
providers: {
|
|
141
|
+
// Multiple image providers - selection box will be shown
|
|
142
|
+
text2image: [
|
|
143
|
+
FalAiImage.RecraftV3({
|
|
144
|
+
proxyUrl: 'http://your-proxy-server.com/api/proxy'
|
|
145
|
+
}),
|
|
146
|
+
FalAiImage.Recraft20b({
|
|
147
|
+
proxyUrl: 'http://your-proxy-server.com/api/proxy'
|
|
148
|
+
}),
|
|
149
|
+
OpenAiImage.GptImage1.Text2Image({
|
|
150
|
+
proxyUrl: 'http://your-proxy-server.com/api/proxy'
|
|
151
|
+
})
|
|
152
|
+
],
|
|
153
|
+
|
|
154
|
+
// Other providers...
|
|
155
|
+
}
|
|
156
|
+
})
|
|
157
|
+
);
|
|
158
|
+
```
|
|
159
|
+
|
|
160
|
+
### Custom Headers and Configuration
|
|
161
|
+
|
|
162
|
+
Pass custom headers and configuration to providers:
|
|
163
|
+
|
|
164
|
+
```typescript
|
|
165
|
+
cesdk.addPlugin(
|
|
166
|
+
AiApps({
|
|
167
|
+
providers: {
|
|
168
|
+
text2image: FalAiImage.RecraftV3({
|
|
169
|
+
proxyUrl: 'http://your-proxy-server.com/api/proxy',
|
|
170
|
+
headers: {
|
|
171
|
+
'x-client-version': '1.0.0',
|
|
172
|
+
'x-request-source': 'cesdk-plugin',
|
|
173
|
+
'x-user-id': 'user-123'
|
|
174
|
+
},
|
|
175
|
+
debug: true
|
|
176
|
+
})
|
|
177
|
+
},
|
|
178
|
+
debug: true
|
|
179
|
+
})
|
|
180
|
+
);
|
|
181
|
+
```
|
|
105
182
|
|
|
106
183
|
## UI Integration
|
|
107
184
|
|
|
108
|
-
The plugin adds the following UI components:
|
|
185
|
+
The plugin adds the following UI components to CreativeEditor SDK:
|
|
186
|
+
|
|
187
|
+
### AI Dock Button
|
|
188
|
+
|
|
189
|
+
The main entry point for all AI features, accessible from the dock:
|
|
190
|
+
|
|
191
|
+
- **ID**: `ly.img.ai.apps.dock`
|
|
192
|
+
- **Functionality**: Opens AI generation interface
|
|
193
|
+
- **Loading States**: Shows progress indicators during generation
|
|
194
|
+
- **Mode Awareness**: Adapts interface based on current editor mode
|
|
195
|
+
|
|
196
|
+
### AI Apps Menu
|
|
197
|
+
|
|
198
|
+
The plugin shows a card-based interface for different AI capabilities:
|
|
109
199
|
|
|
110
|
-
|
|
111
|
-
|
|
200
|
+
- **Generate Image**: Access to image generation providers
|
|
201
|
+
- **Generate Video**: Access to video generation providers
|
|
202
|
+
- **Generate Audio**: Access to audio generation providers
|
|
203
|
+
- **Edit Text**: Access to text generation providers
|
|
112
204
|
|
|
113
205
|
### Dock Integration
|
|
114
206
|
|
|
@@ -116,31 +208,301 @@ To position the AI dock button in your editor's dock, use the `setDockOrder` met
|
|
|
116
208
|
|
|
117
209
|
```typescript
|
|
118
210
|
// Add the AI dock component to the beginning of the dock
|
|
119
|
-
cesdk.ui.setDockOrder(['ly.img.ai
|
|
211
|
+
cesdk.ui.setDockOrder(['ly.img.ai.apps.dock', ...cesdk.ui.getDockOrder()]);
|
|
120
212
|
|
|
121
213
|
// Or add it at a specific position
|
|
122
214
|
const currentOrder = cesdk.ui.getDockOrder();
|
|
123
|
-
currentOrder.splice(2, 0, 'ly.img.ai
|
|
215
|
+
currentOrder.splice(2, 0, 'ly.img.ai.apps.dock');
|
|
124
216
|
cesdk.ui.setDockOrder(currentOrder);
|
|
125
217
|
```
|
|
126
218
|
|
|
127
219
|
## Asset History Integration
|
|
128
220
|
|
|
129
|
-
The plugin automatically integrates generated assets into the appropriate asset libraries:
|
|
221
|
+
The plugin automatically integrates generated assets into the appropriate asset libraries using the following history source IDs:
|
|
222
|
+
|
|
223
|
+
- **Image Generation History**: `ly.img.ai.image-generation.history`
|
|
224
|
+
- **Video Generation History**: `ly.img.ai.video-generation.history`
|
|
225
|
+
- **Audio Generation History**: `ly.img.ai.audio-generation.history`
|
|
226
|
+
- **Sticker Generation History**: `ly.img.ai.sticker-generation.history`
|
|
227
|
+
|
|
228
|
+
To add these history sources to existing asset library entries, use the following approach:
|
|
229
|
+
|
|
230
|
+
```typescript
|
|
231
|
+
// Add AI image history to the default image asset library
|
|
232
|
+
const imageEntry = cesdk.ui.getAssetLibraryEntry('ly.img.image');
|
|
233
|
+
if (imageEntry != null) {
|
|
234
|
+
cesdk.ui.updateAssetLibraryEntry('ly.img.image', {
|
|
235
|
+
sourceIds: [...imageEntry.sourceIds, 'ly.img.ai.image-generation.history']
|
|
236
|
+
});
|
|
237
|
+
}
|
|
238
|
+
|
|
239
|
+
// Add AI video history to the default video asset library
|
|
240
|
+
const videoEntry = cesdk.ui.getAssetLibraryEntry('ly.img.video');
|
|
241
|
+
if (videoEntry != null) {
|
|
242
|
+
cesdk.ui.updateAssetLibraryEntry('ly.img.video', {
|
|
243
|
+
sourceIds: [...videoEntry.sourceIds, 'ly.img.ai.video-generation.history']
|
|
244
|
+
});
|
|
245
|
+
}
|
|
246
|
+
|
|
247
|
+
// Add AI audio history to the default audio asset library
|
|
248
|
+
const audioEntry = cesdk.ui.getAssetLibraryEntry('ly.img.audio');
|
|
249
|
+
if (audioEntry != null) {
|
|
250
|
+
cesdk.ui.updateAssetLibraryEntry('ly.img.audio', {
|
|
251
|
+
sourceIds: [...audioEntry.sourceIds, 'ly.img.ai.audio-generation.history']
|
|
252
|
+
});
|
|
253
|
+
}
|
|
254
|
+
|
|
255
|
+
// Add AI sticker history to the default sticker asset library
|
|
256
|
+
const stickerEntry = instance.ui.getAssetLibraryEntry('ly.img.sticker');
|
|
257
|
+
if (stickerEntry != null) {
|
|
258
|
+
instance.ui.updateAssetLibraryEntry('ly.img.sticker', {
|
|
259
|
+
sourceIds: [...stickerEntry.sourceIds, 'ly.img.ai.sticker-generation.history']
|
|
260
|
+
});
|
|
261
|
+
}
|
|
262
|
+
```
|
|
263
|
+
|
|
264
|
+
This integration creates a seamless experience where users can easily find and reuse their AI-generated assets alongside other content.
|
|
265
|
+
|
|
266
|
+
## Quick Actions Integration
|
|
267
|
+
|
|
268
|
+
The plugin automatically integrates with the quick actions system, providing context-sensitive AI operations directly in the canvas menu. You need to specify the children with the quick action order:
|
|
269
|
+
|
|
270
|
+
```typescript
|
|
271
|
+
// Quick actions are automatically registered and will appear in canvas menus
|
|
272
|
+
cesdk.ui.setCanvasMenuOrder([
|
|
273
|
+
{
|
|
274
|
+
id: 'ly.img.ai.text.canvasMenu',
|
|
275
|
+
children: [
|
|
276
|
+
'ly.img.improve',
|
|
277
|
+
'ly.img.fix',
|
|
278
|
+
'ly.img.shorter',
|
|
279
|
+
'ly.img.longer',
|
|
280
|
+
'ly.img.separator',
|
|
281
|
+
'ly.img.changeTone',
|
|
282
|
+
'ly.img.translate',
|
|
283
|
+
'ly.img.separator',
|
|
284
|
+
'ly.img.changeTextTo'
|
|
285
|
+
]
|
|
286
|
+
},
|
|
287
|
+
{
|
|
288
|
+
id: 'ly.img.ai.image.canvasMenu',
|
|
289
|
+
children: [
|
|
290
|
+
'ly.img.styleTransfer',
|
|
291
|
+
'ly.img.artistTransfer',
|
|
292
|
+
'ly.img.separator',
|
|
293
|
+
'ly.img.editImage',
|
|
294
|
+
'ly.img.swapBackground',
|
|
295
|
+
'ly.img.createVariant',
|
|
296
|
+
'ly.img.combineImages',
|
|
297
|
+
'ly.img.separator',
|
|
298
|
+
'ly.img.remixPage',
|
|
299
|
+
'ly.img.separator',
|
|
300
|
+
'ly.img.createVideo'
|
|
301
|
+
]
|
|
302
|
+
},
|
|
303
|
+
...cesdk.ui.getCanvasMenuOrder()
|
|
304
|
+
]);
|
|
305
|
+
```
|
|
306
|
+
|
|
307
|
+
Quick actions provide:
|
|
308
|
+
- **Context-aware operations**: Work with currently selected blocks
|
|
309
|
+
- **One-click transformations**: Apply AI operations without opening panels
|
|
310
|
+
- **Cross-plugin functionality**: Actions from one plugin can work with providers from another
|
|
311
|
+
|
|
312
|
+
## Error Handling and Debugging
|
|
313
|
+
|
|
314
|
+
### Debug Mode
|
|
315
|
+
|
|
316
|
+
Enable debug mode to get detailed logging information:
|
|
317
|
+
|
|
318
|
+
```typescript
|
|
319
|
+
cesdk.addPlugin(
|
|
320
|
+
AiApps({
|
|
321
|
+
providers: {
|
|
322
|
+
text2image: FalAiImage.RecraftV3({
|
|
323
|
+
proxyUrl: 'http://your-proxy-server.com/api/proxy',
|
|
324
|
+
debug: true // Provider-level debugging
|
|
325
|
+
})
|
|
326
|
+
},
|
|
327
|
+
debug: true // Plugin-level debugging
|
|
328
|
+
})
|
|
329
|
+
);
|
|
330
|
+
```
|
|
331
|
+
|
|
332
|
+
### Common Issues and Solutions
|
|
333
|
+
|
|
334
|
+
#### Provider Not Loading
|
|
335
|
+
- **Check proxy URLs**: Ensure all proxy URLs are correctly configured and accessible
|
|
336
|
+
- **Verify licenses**: Make sure your CreativeEditor SDK license includes AI features
|
|
337
|
+
- **Check browser console**: Look for network errors or API issues
|
|
338
|
+
|
|
339
|
+
#### Quick Actions Not Appearing
|
|
340
|
+
- **Verify canvas menu order**: Ensure quick action menus are added to canvas menu order with proper children configuration
|
|
341
|
+
- **Check provider support**: Verify that providers declare support for the quick actions
|
|
342
|
+
- **Scope permissions**: Ensure blocks have the required scopes for quick actions
|
|
130
343
|
|
|
131
|
-
|
|
132
|
-
-
|
|
133
|
-
-
|
|
344
|
+
#### Generation Failures
|
|
345
|
+
- **API connectivity**: Check that your proxy endpoints are working
|
|
346
|
+
- **Rate limiting**: Verify you're not exceeding API rate limits
|
|
347
|
+
- **Input validation**: Ensure inputs meet provider requirements
|
|
134
348
|
|
|
135
|
-
|
|
349
|
+
## Plugin Architecture
|
|
350
|
+
|
|
351
|
+
### How It Works
|
|
352
|
+
|
|
353
|
+
The AI Apps plugin acts as an orchestrator that:
|
|
354
|
+
|
|
355
|
+
1. **Initializes Providers**: Sets up all configured AI providers
|
|
356
|
+
2. **Manages UI**: Creates appropriate interface based on editor mode
|
|
357
|
+
3. **Coordinates Actions**: Integrates quick actions across different plugins
|
|
358
|
+
4. **Handles Assets**: Manages generated content and asset library integration
|
|
359
|
+
|
|
360
|
+
### Provider Lifecycle
|
|
361
|
+
|
|
362
|
+
1. **Registration**: Providers are registered with the global ProviderRegistry
|
|
363
|
+
2. **Initialization**: Provider `initialize` methods are called
|
|
364
|
+
3. **UI Setup**: Panels and quick actions are registered
|
|
365
|
+
4. **Event Handling**: The plugin coordinates between providers and UI
|
|
366
|
+
|
|
367
|
+
## TypeScript Support
|
|
368
|
+
|
|
369
|
+
The plugin is fully typed with TypeScript, providing excellent development experience:
|
|
370
|
+
|
|
371
|
+
```typescript
|
|
372
|
+
import AiApps, { Providers } from '@imgly/plugin-ai-apps-web';
|
|
373
|
+
|
|
374
|
+
// Strongly typed provider configuration
|
|
375
|
+
const providers: Providers = {
|
|
376
|
+
text2image: FalAiImage.RecraftV3({
|
|
377
|
+
proxyUrl: 'http://your-proxy-server.com/api/proxy'
|
|
378
|
+
}),
|
|
379
|
+
// TypeScript will enforce correct provider types
|
|
380
|
+
};
|
|
381
|
+
|
|
382
|
+
cesdk.addPlugin(AiApps({ providers }));
|
|
383
|
+
```
|
|
136
384
|
|
|
137
385
|
## Related Packages
|
|
138
386
|
|
|
139
|
-
|
|
140
|
-
|
|
141
|
-
-
|
|
142
|
-
-
|
|
143
|
-
-
|
|
387
|
+
This plugin works with the following IMG.LY AI generation packages:
|
|
388
|
+
|
|
389
|
+
- **[@imgly/plugin-ai-generation-web](https://github.com/imgly/plugins/tree/main/packages/plugin-ai-generation-web)** - Core utilities for AI generation
|
|
390
|
+
- **[@imgly/plugin-ai-image-generation-web](https://github.com/imgly/plugins/tree/main/packages/plugin-ai-image-generation-web)** - AI image generation
|
|
391
|
+
- **[@imgly/plugin-ai-video-generation-web](https://github.com/imgly/plugins/tree/main/packages/plugin-ai-video-generation-web)** - AI video generation
|
|
392
|
+
- **[@imgly/plugin-ai-audio-generation-web](https://github.com/imgly/plugins/tree/main/packages/plugin-ai-audio-generation-web)** - AI audio generation
|
|
393
|
+
- **[@imgly/plugin-ai-text-generation-web](https://github.com/imgly/plugins/tree/main/packages/plugin-ai-text-generation-web)** - AI text generation
|
|
394
|
+
|
|
395
|
+
## Examples
|
|
396
|
+
|
|
397
|
+
### Simple Setup for Design Mode
|
|
398
|
+
|
|
399
|
+
```typescript
|
|
400
|
+
import CreativeEditorSDK from '@cesdk/cesdk-js';
|
|
401
|
+
import AiApps from '@imgly/plugin-ai-apps-web';
|
|
402
|
+
import FalAiImage from '@imgly/plugin-ai-image-generation-web/fal-ai';
|
|
403
|
+
|
|
404
|
+
CreativeEditorSDK.create(domElement, {
|
|
405
|
+
license: 'your-license-key'
|
|
406
|
+
}).then(async (cesdk) => {
|
|
407
|
+
cesdk.addPlugin(
|
|
408
|
+
AiApps({
|
|
409
|
+
providers: {
|
|
410
|
+
text2image: FalAiImage.RecraftV3({
|
|
411
|
+
proxyUrl: 'http://your-proxy-server.com/api/proxy'
|
|
412
|
+
})
|
|
413
|
+
}
|
|
414
|
+
})
|
|
415
|
+
);
|
|
416
|
+
|
|
417
|
+
cesdk.ui.setDockOrder(['ly.img.ai.apps.dock', ...cesdk.ui.getDockOrder()]);
|
|
418
|
+
});
|
|
419
|
+
```
|
|
420
|
+
|
|
421
|
+
### Complete Setup for Video Mode
|
|
422
|
+
|
|
423
|
+
```typescript
|
|
424
|
+
import CreativeEditorSDK from '@cesdk/cesdk-js';
|
|
425
|
+
import AiApps from '@imgly/plugin-ai-apps-web';
|
|
426
|
+
import FalAiImage from '@imgly/plugin-ai-image-generation-web/fal-ai';
|
|
427
|
+
import FalAiVideo from '@imgly/plugin-ai-video-generation-web/fal-ai';
|
|
428
|
+
import ElevenLabs from '@imgly/plugin-ai-audio-generation-web/elevenlabs';
|
|
429
|
+
import { AnthropicProvider } from '@imgly/plugin-ai-text-generation-web/anthropic';
|
|
430
|
+
|
|
431
|
+
CreativeEditorSDK.create(domElement, {
|
|
432
|
+
license: 'your-license-key',
|
|
433
|
+
ui: {
|
|
434
|
+
elements: {
|
|
435
|
+
panels: {
|
|
436
|
+
settings: true
|
|
437
|
+
}
|
|
438
|
+
}
|
|
439
|
+
}
|
|
440
|
+
}).then(async (cesdk) => {
|
|
441
|
+
cesdk.addPlugin(
|
|
442
|
+
AiApps({
|
|
443
|
+
providers: {
|
|
444
|
+
text2text: AnthropicProvider({
|
|
445
|
+
proxyUrl: 'http://your-proxy-server.com/api/proxy'
|
|
446
|
+
}),
|
|
447
|
+
text2image: FalAiImage.RecraftV3({
|
|
448
|
+
proxyUrl: 'http://your-proxy-server.com/api/proxy'
|
|
449
|
+
}),
|
|
450
|
+
image2image: FalAiImage.GeminiFlashEdit({
|
|
451
|
+
proxyUrl: 'http://your-proxy-server.com/api/proxy'
|
|
452
|
+
}),
|
|
453
|
+
text2video: FalAiVideo.MinimaxVideo01Live({
|
|
454
|
+
proxyUrl: 'http://your-proxy-server.com/api/proxy'
|
|
455
|
+
}),
|
|
456
|
+
image2video: FalAiVideo.MinimaxVideo01LiveImageToVideo({
|
|
457
|
+
proxyUrl: 'http://your-proxy-server.com/api/proxy'
|
|
458
|
+
}),
|
|
459
|
+
text2speech: ElevenLabs.MonolingualV1({
|
|
460
|
+
proxyUrl: 'http://your-proxy-server.com/api/proxy'
|
|
461
|
+
}),
|
|
462
|
+
text2sound: ElevenLabs.SoundEffects({
|
|
463
|
+
proxyUrl: 'http://your-proxy-server.com/api/proxy'
|
|
464
|
+
})
|
|
465
|
+
}
|
|
466
|
+
})
|
|
467
|
+
);
|
|
468
|
+
|
|
469
|
+
// Setup dock and canvas menus
|
|
470
|
+
cesdk.ui.setDockOrder(['ly.img.ai.apps.dock', ...cesdk.ui.getDockOrder()]);
|
|
471
|
+
cesdk.ui.setCanvasMenuOrder([
|
|
472
|
+
{
|
|
473
|
+
id: 'ly.img.ai.text.canvasMenu',
|
|
474
|
+
children: [
|
|
475
|
+
'ly.img.improve',
|
|
476
|
+
'ly.img.fix',
|
|
477
|
+
'ly.img.shorter',
|
|
478
|
+
'ly.img.longer',
|
|
479
|
+
'ly.img.separator',
|
|
480
|
+
'ly.img.changeTone',
|
|
481
|
+
'ly.img.translate',
|
|
482
|
+
'ly.img.separator',
|
|
483
|
+
'ly.img.changeTextTo'
|
|
484
|
+
]
|
|
485
|
+
},
|
|
486
|
+
{
|
|
487
|
+
id: 'ly.img.ai.image.canvasMenu',
|
|
488
|
+
children: [
|
|
489
|
+
'ly.img.styleTransfer',
|
|
490
|
+
'ly.img.artistTransfer',
|
|
491
|
+
'ly.img.separator',
|
|
492
|
+
'ly.img.editImage',
|
|
493
|
+
'ly.img.swapBackground',
|
|
494
|
+
'ly.img.createVariant',
|
|
495
|
+
'ly.img.combineImages',
|
|
496
|
+
'ly.img.separator',
|
|
497
|
+
'ly.img.remixPage',
|
|
498
|
+
'ly.img.separator',
|
|
499
|
+
'ly.img.createVideo'
|
|
500
|
+
]
|
|
501
|
+
},
|
|
502
|
+
...cesdk.ui.getCanvasMenuOrder()
|
|
503
|
+
]);
|
|
504
|
+
});
|
|
505
|
+
```
|
|
144
506
|
|
|
145
507
|
## License
|
|
146
508
|
|
|
@@ -1,14 +1,14 @@
|
|
|
1
1
|
import CreativeEditorSDK from '@cesdk/cesdk-js';
|
|
2
2
|
import type { AssetDefinition, AssetQueryData, AssetSource, AssetsQueryResult } from '@cesdk/engine';
|
|
3
3
|
/**
|
|
4
|
-
* A custom AssetSource implementation that manages assets from an array
|
|
4
|
+
* A custom AssetSource implementation that manages assets from an array or dynamic function
|
|
5
5
|
* and provides functionality to mark assets as active.
|
|
6
6
|
*/
|
|
7
7
|
export declare class CustomAssetSource implements AssetSource {
|
|
8
8
|
/** The unique id of the asset source */
|
|
9
9
|
id: string;
|
|
10
|
-
/** Array of assets
|
|
11
|
-
private
|
|
10
|
+
/** Array of assets or function that returns assets */
|
|
11
|
+
private assetsOrProvider;
|
|
12
12
|
/** Set of IDs for active assets */
|
|
13
13
|
private activeAssetIds;
|
|
14
14
|
private loaderDisposer;
|
|
@@ -17,9 +17,13 @@ export declare class CustomAssetSource implements AssetSource {
|
|
|
17
17
|
* Creates a new instance of CustomAssetSource
|
|
18
18
|
*
|
|
19
19
|
* @param id - The unique identifier for this asset source
|
|
20
|
-
* @param
|
|
20
|
+
* @param assetsOrProvider - Array of asset definitions or function that returns asset definitions
|
|
21
21
|
*/
|
|
22
|
-
constructor(id: string, cesdk: CreativeEditorSDK,
|
|
22
|
+
constructor(id: string, cesdk: CreativeEditorSDK, assetsOrProvider?: AssetDefinition[] | (() => AssetDefinition[]));
|
|
23
|
+
/**
|
|
24
|
+
* Get all current assets definitions used by this asset source
|
|
25
|
+
*/
|
|
26
|
+
getCurrentAssets(): AssetDefinition[];
|
|
23
27
|
/**
|
|
24
28
|
* Find assets based on the provided query data
|
|
25
29
|
* Supports pagination, searching, filtering, and active-first sorting
|
|
@@ -83,8 +87,8 @@ export declare class CustomAssetSource implements AssetSource {
|
|
|
83
87
|
* Helper function to create a CustomAssetSource instance
|
|
84
88
|
*
|
|
85
89
|
* @param id - The unique identifier for this asset source
|
|
86
|
-
* @param
|
|
90
|
+
* @param assetsOrProvider - Array of asset definitions or function that returns asset definitions
|
|
87
91
|
* @returns A new CustomAssetSource instance
|
|
88
92
|
*/
|
|
89
|
-
export declare function createCustomAssetSource(id: string, cesdk: CreativeEditorSDK,
|
|
93
|
+
export declare function createCustomAssetSource(id: string, cesdk: CreativeEditorSDK, assetsOrProvider?: AssetDefinition[] | (() => AssetDefinition[])): CustomAssetSource;
|
|
90
94
|
export default CustomAssetSource;
|