ima2-gen 2.0.0 → 2.0.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +150 -0
- package/README.md +12 -12
- package/bin/commands/backfillThumbs.js +24 -0
- package/bin/commands/edit.js +7 -6
- package/bin/commands/gen.js +13 -6
- package/bin/commands/multimode.js +5 -4
- package/bin/commands/node.js +4 -4
- package/bin/ima2.js +21 -11
- package/bin/lib/config-store.js +1 -1
- package/docs/API.md +184 -10
- package/docs/CLI.md +11 -4
- package/docs/FAQ.ko.md +16 -0
- package/docs/FAQ.md +30 -0
- package/docs/PROMPT_STUDIO.md +3 -1
- package/docs/README.ko.md +7 -3
- package/docs/migration/runtime-test-inventory.md +17 -1
- package/lib/agentImageVideoGen.js +261 -0
- package/lib/agentRuntime.js +11 -260
- package/lib/agentSettings.js +1 -1
- package/lib/agyImageAdapter.js +259 -0
- package/lib/capabilities.js +2 -1
- package/lib/configKeys.js +1 -1
- package/lib/errorClassify.js +8 -7
- package/lib/eventBus.js +71 -0
- package/lib/geminiApiImageAdapter.js +179 -0
- package/lib/generationErrors.js +3 -1
- package/lib/grokImageAdapter.js +74 -128
- package/lib/grokImageCore.js +153 -0
- package/lib/grokMultimodeAdapter.js +7 -4
- package/lib/grokRuntime.js +3 -0
- package/lib/grokSizeMapper.js +13 -1
- package/lib/grokVideoAdapter.js +14 -7
- package/lib/grokVideoCanvas.js +13 -0
- package/lib/grokVideoPlannerPrompt.js +53 -6
- package/lib/historyList.js +19 -2
- package/lib/imageModels.js +15 -0
- package/lib/imageThumb.js +38 -0
- package/lib/inflight.js +54 -17
- package/lib/multimodeHelpers.js +10 -0
- package/lib/nodeHelpers.js +59 -0
- package/lib/oauthProxy/prompts.js +30 -36
- package/lib/promptBuilder/systemPrompt.js +2 -5
- package/lib/promptSafetyPolicy.js +1 -5
- package/lib/providerOptions.js +36 -1
- package/lib/responsesFallback.js +53 -44
- package/lib/routeHelpers.js +44 -0
- package/lib/runtimeContext.js +27 -0
- package/lib/ssePublish.js +12 -0
- package/lib/storageMigration.js +1 -1
- package/lib/storyboardPrefix.js +28 -0
- package/lib/thumbBackfill.js +70 -0
- package/lib/vertexAuth.js +44 -0
- package/lib/videoThumb.js +60 -0
- package/package.json +7 -2
- package/routes/agy.js +44 -0
- package/routes/auth.js +242 -0
- package/routes/edit.js +48 -8
- package/routes/events.js +78 -0
- package/routes/generate.js +135 -135
- package/routes/history.js +13 -0
- package/routes/index.js +8 -0
- package/routes/keys.js +254 -0
- package/routes/multimode.js +138 -62
- package/routes/nodes.js +107 -129
- package/routes/quota.js +58 -7
- package/routes/video.js +107 -20
- package/server.js +123 -0
- package/skills/ima2/SKILL.md +98 -21
- package/ui/dist/.vite/manifest.json +12 -12
- package/ui/dist/assets/AgentWorkspace-Dth6YijN.js +3 -0
- package/ui/dist/assets/{CardNewsWorkspace-BN-ga1lG.js → CardNewsWorkspace-Dav3K5CT.js} +2 -2
- package/ui/dist/assets/{NodeCanvas-BbMa4IhI.js → NodeCanvas-C4ifFzB1.js} +2 -2
- package/ui/dist/assets/{PromptBuilderPanel-DRwBJRDQ.js → PromptBuilderPanel-CEcyU9PL.js} +1 -1
- package/ui/dist/assets/{PromptImportDialog-Dp85kHCq.js → PromptImportDialog-CgQ94Gth.js} +2 -2
- package/ui/dist/assets/{PromptImportDiscoverySection-BE8Q8MLD.js → PromptImportDiscoverySection-CuzyzbNI.js} +1 -1
- package/ui/dist/assets/{PromptImportFolderSection-PtH5x0sc.js → PromptImportFolderSection-DHLGlO6l.js} +1 -1
- package/ui/dist/assets/{PromptLibraryPanel-FnM9tHI9.js → PromptLibraryPanel-BOe18we8.js} +2 -2
- package/ui/dist/assets/SettingsWorkspace-Cdgnm4Wa.js +1 -0
- package/ui/dist/assets/index-C5PSahkr.js +1 -0
- package/ui/dist/assets/index-Dn2AhL6d.css +1 -0
- package/ui/dist/assets/index-Tjqx6wUV.js +23 -0
- package/ui/dist/index.html +2 -2
- package/ui/dist/assets/AgentWorkspace-C21zqdTZ.js +0 -3
- package/ui/dist/assets/SettingsWorkspace-MARPGyBL.js +0 -1
- package/ui/dist/assets/index-BAFI6htx.js +0 -42
- package/ui/dist/assets/index-BSXxr_Bt.js +0 -1
- package/ui/dist/assets/index-DS-ADE7U.css +0 -1
package/docs/API.md
CHANGED
|
@@ -10,11 +10,14 @@ http://localhost:3333
|
|
|
10
10
|
|
|
11
11
|
## Provider Policy
|
|
12
12
|
|
|
13
|
-
Image generation supports OAuth, API-key, and
|
|
13
|
+
Image generation supports OAuth, API-key, Grok, and Gemini (`agy` and `gemini-api`) providers.
|
|
14
14
|
|
|
15
15
|
- `provider: "oauth"` uses the local Codex OAuth proxy.
|
|
16
16
|
- `provider: "api"` uses the OpenAI Responses API with the hosted `image_generation` tool.
|
|
17
17
|
- `provider: "grok"` uses the bundled progrok xAI proxy. Classic, Node, and Agent generation run mandatory xAI Web Search through `/v1/responses`, then run a `grok-4.3` planner call with a forced local `generate_image` function, then ima2 executes xAI `/v1/images/generations`. If reference images, a Node parent image, or an Agent current image are attached, the final step switches to xAI `/v1/images/edits` so image-to-image context is preserved.
|
|
18
|
+
- `provider: "agy"` spawns the Antigravity CLI (`agy -p`) to generate images via Google Gemini's `default_api:generate_image` tool. Model is `nano-banana-2`. Output is fixed at 1024×1024 JPEG. Max 3 reference images (i2i). No web search, quality, size, or mask controls. Multimode returns a single image. Video is unsupported (`AGY_VIDEO_UNSUPPORTED`).
|
|
19
|
+
- `provider: "grok-api"` uses a direct xAI API key instead of the bundled progrok OAuth proxy. Same pipeline as `grok` (Web Search → planner → `/v1/images/generations`), same aspect ratio and resolution options. Requires an xAI API key configured via the web UI key management or `XAI_API_KEY` env var. Also supports video generation.
|
|
20
|
+
- `provider: "gemini-api"` calls the Google Generative Language API directly (or Vertex AI with a service account JSON). Supports models `nano-banana-2` (Gemini 3.1 Flash Image) and `nano-banana-pro` (Gemini 3 Pro Image). Supports variable aspect ratios (1:1 through 21:9) and four resolution tiers (512px, 1K, 2K, 4K); these are honored only on the direct API path — the Vertex AI endpoint (`aiplatform.googleapis.com`) rejects the `response_format` field and always returns a default 1K/1:1 image regardless of requested size. Auth: `GEMINI_API_KEY` env var, web UI key management (`/api/keys/gemini`), or a Vertex AI service account JSON (`VERTEX_SERVICE_ACCOUNT_JSON` or `/api/keys/vertex`). When both Vertex credentials and an API key are configured, Vertex takes priority. The chosen auth mode (`apikey` or `vertex`) persists to `~/.ima2/config.json` as `geminiAuthMode` and is restored on server startup. Per-model cost: `nano-banana-2` (Flash): 512=$0.001, 1K=$0.003, 2K=$0.004, 4K=$0.006; `nano-banana-pro`: 1K=$0.007, 2K=$0.007, 4K=$0.013. No web search or mask controls.
|
|
18
21
|
- API-key generation covers classic generate, edit, mask-guided edit, multimode, and node generation.
|
|
19
22
|
- If `provider: "api"` is requested without an API key, routes fail before upstream with `401` and `API_KEY_REQUIRED`.
|
|
20
23
|
- Grok generation maps `size` to xAI `aspect_ratio` and `resolution`; it does not send an OpenAI-style `size` field upstream. Grok edit uses xAI `/v1/images/edits`; Grok mask edit remains unsupported and returns `GROK_MASK_UNSUPPORTED`.
|
|
@@ -32,6 +35,16 @@ Generation section below for the full endpoint specification.
|
|
|
32
35
|
| `GET` | `/api/oauth/status` | OAuth proxy status and visible models |
|
|
33
36
|
| `GET` | `/api/grok/status` | Bundled progrok status and visible xAI image models |
|
|
34
37
|
| `GET` | `/api/billing` | Billing/status probe, including API key source when configured |
|
|
38
|
+
| `GET` | `/api/quota` | Provider quota: returns `{ codex, grok }`. Grok result includes `billing: { usedUsd, limitUsd }` and a `monthly` percent window drawn from the xAI billing API. |
|
|
39
|
+
|
|
40
|
+
## Account Switching
|
|
41
|
+
|
|
42
|
+
| Method | Path | Notes |
|
|
43
|
+
|---|---|---|
|
|
44
|
+
| `POST` | `/api/auth/switch` | Start a device-code OAuth flow. Body: `{ "provider": "grok" \| "codex" }`. Returns `{ sessionId, userCode, verificationUrl }`. |
|
|
45
|
+
| `GET` | `/api/auth/switch/:sessionId` | Poll switch-account session status. Returns `{ status }` where status is `pending`, `complete`, `error`, or `expired`. |
|
|
46
|
+
|
|
47
|
+
The Switch Account flow opens a browser verification URL. Once the user completes the device-code step, the server saves the new credentials (Grok: `~/.progrok/auth.json`; Codex: via `codex login --device-auth`) and the session transitions to `complete`. This endpoint is surfaced as a **Switch Account** button in the Settings QuotaCard for Grok and Codex providers.
|
|
35
48
|
|
|
36
49
|
## Storage
|
|
37
50
|
|
|
@@ -81,9 +94,81 @@ Storage `state` values:
|
|
|
81
94
|
| `GET` | `/api/inflight` | Active jobs only by default |
|
|
82
95
|
| `GET` | `/api/inflight?includeTerminal=1` | Includes recent terminal jobs for debugging |
|
|
83
96
|
| `DELETE` | `/api/inflight/:requestId` | Cancel or forget an active job |
|
|
97
|
+
| `GET` | `/api/events` | Persistent SSE multiplex channel for all async generation progress (see below) |
|
|
84
98
|
|
|
85
99
|
In-flight logs and responses use `requestId` for correlation. Logs should not include raw prompts, reference data URLs, generated base64, tokens, cookies, auth headers, or raw upstream bodies.
|
|
86
100
|
|
|
101
|
+
## Events (SSE Multiplexing)
|
|
102
|
+
|
|
103
|
+
### `GET /api/events` (SSE Multiplexing)
|
|
104
|
+
|
|
105
|
+
Single persistent Server-Sent Events channel that carries progress for all async generation jobs. The browser UI opens one `EventSource` here instead of holding a per-request SSE connection for each job, avoiding browser per-origin connection limits.
|
|
106
|
+
|
|
107
|
+
| Query | Notes |
|
|
108
|
+
|---|---|
|
|
109
|
+
| `lastEventId` | Optional. Reconnect cursor; also accepted via the `Last-Event-ID` request header |
|
|
110
|
+
|
|
111
|
+
**Response**: `text/event-stream` (persistent). Each frame uses standard SSE fields `id`, `event`, and `data` (JSON).
|
|
112
|
+
|
|
113
|
+
**Connection limits**: When active listeners reach 512, the server returns `503` with `SSE_CAPACITY` before opening the stream.
|
|
114
|
+
|
|
115
|
+
**Heartbeat**: Every 15 seconds the server writes a comment frame:
|
|
116
|
+
|
|
117
|
+
```text
|
|
118
|
+
: ping
|
|
119
|
+
```
|
|
120
|
+
|
|
121
|
+
**Replay**: On reconnect, the server replays events from an in-memory ring buffer (size 2000) for IDs newer than `lastEventId`. Large image payloads (>1000 characters) are omitted from replay with `_imageOmitted: true` in the `data` payload. If the requested ID is older than the oldest buffered event, the server emits a `replay-gap` event before live fan-out:
|
|
122
|
+
|
|
123
|
+
| Event | Data | Description |
|
|
124
|
+
|---|---|---|
|
|
125
|
+
| `replay-gap` | `{ lastEventId, oldestAvailableId }` | Client should reconcile inflight state (for example via `GET /api/inflight`) |
|
|
126
|
+
|
|
127
|
+
**Job routing**: Every `data` payload includes `jobId` (same value as the job's `requestId`). Event bodies also carry `requestId` where applicable. Clients filter events by matching `data.jobId` or `data.requestId` to the job they started.
|
|
128
|
+
|
|
129
|
+
**Event types** (fan-out to all connected clients):
|
|
130
|
+
|
|
131
|
+
| Event | Emitted by | Description |
|
|
132
|
+
|---|---|---|
|
|
133
|
+
| `phase` | node, multimode, video | Lifecycle phase change |
|
|
134
|
+
| `partial` | node, multimode | Progressive preview image (base64 data URL) |
|
|
135
|
+
| `image` | multimode | Final saved `GenerateItem` for one sequence image |
|
|
136
|
+
| `done` | node, multimode, video | Terminal success payload (route-specific shape) |
|
|
137
|
+
| `error` | all generation routes | Terminal failure |
|
|
138
|
+
| `submitted` | video | Job submitted to xAI |
|
|
139
|
+
| `progress` | video | Progress fraction 0.0–1.0 |
|
|
140
|
+
| `planning` | video | Video planner running |
|
|
141
|
+
|
|
142
|
+
Example SSE frame:
|
|
143
|
+
|
|
144
|
+
```text
|
|
145
|
+
id: 42
|
|
146
|
+
event: phase
|
|
147
|
+
data: {"requestId":"req_abc","jobId":"req_abc","phase":"streaming"}
|
|
148
|
+
```
|
|
149
|
+
|
|
150
|
+
### Async generation mode
|
|
151
|
+
|
|
152
|
+
`POST /api/node/generate`, `POST /api/generate/multimode`, and `POST /api/video/generate` support an async POST mode for clients that already hold `GET /api/events`:
|
|
153
|
+
|
|
154
|
+
```json
|
|
155
|
+
{
|
|
156
|
+
"async": true,
|
|
157
|
+
"requestId": "req_xxx",
|
|
158
|
+
"...": "other route fields"
|
|
159
|
+
}
|
|
160
|
+
```
|
|
161
|
+
|
|
162
|
+
| Outcome | HTTP | Body |
|
|
163
|
+
|---|---|---|
|
|
164
|
+
| Accepted | `202` | `{ "requestId": "req_xxx" }` |
|
|
165
|
+
| Duplicate active `requestId` | `409` | `REQUEST_ID_IN_USE` |
|
|
166
|
+
| More than 12 concurrent active jobs | `429` | `TOO_MANY_JOBS` with `Retry-After: 5` |
|
|
167
|
+
|
|
168
|
+
Progress events are published on `GET /api/events`. The POST response returns immediately; clients must not expect SSE on the POST connection when `async: true`.
|
|
169
|
+
|
|
170
|
+
CLI and legacy clients omit `async` and keep the original behavior: per-request SSE on the same POST response (`Accept: text/event-stream` where applicable). The server dual-emits in that mode — it writes SSE to the POST response and also publishes the same events on `GET /api/events`.
|
|
171
|
+
|
|
87
172
|
## Generation
|
|
88
173
|
|
|
89
174
|
### `POST /api/generate`
|
|
@@ -100,7 +185,8 @@ Text-to-image and reference-guided root generation.
|
|
|
100
185
|
"provider": "oauth",
|
|
101
186
|
"model": "gpt-5.4",
|
|
102
187
|
"references": [],
|
|
103
|
-
"requestId": "optional-client-id"
|
|
188
|
+
"requestId": "optional-client-id",
|
|
189
|
+
"storyboard": false
|
|
104
190
|
}
|
|
105
191
|
```
|
|
106
192
|
|
|
@@ -108,6 +194,9 @@ Supported quality values: `low`, `medium`, `high`.
|
|
|
108
194
|
|
|
109
195
|
Supported moderation values: `auto`, `low`.
|
|
110
196
|
|
|
197
|
+
When `storyboard` is `true`, the server prepends storyboard keyframe instructions so image
|
|
198
|
+
generations maintain character and scene continuity for multi-shot video production.
|
|
199
|
+
|
|
111
200
|
Recommended model: `gpt-5.4`. Current app default: `gpt-5.4-mini`. `gpt-5.5` is the strongest quality option when supported, but callers should expect higher quota pressure and possible Codex CLI/backend capability requirements.
|
|
112
201
|
|
|
113
202
|
When `provider` is `"grok"`, supported models are `grok-imagine-image` and
|
|
@@ -188,14 +277,46 @@ Body fields:
|
|
|
188
277
|
}
|
|
189
278
|
```
|
|
190
279
|
|
|
191
|
-
When `parentNodeId` is present, the server loads the stored parent node image and uses the edit path.
|
|
280
|
+
When `parentNodeId` is present, the server loads the stored parent node image and uses the edit path. Node-local references are allowed on both root and child/edit nodes; for child/edit nodes the parent image is sent first, then references, then the text prompt.
|
|
192
281
|
|
|
193
282
|
With `provider: "grok"`, Node Mode uses the same xAI search + `grok-4.3` planner + Images API pipeline as classic generation. A parent node image, `externalSrc`, or extra references are passed to the planner and then to xAI `/v1/images/edits`; otherwise the final call uses `/v1/images/generations`. Grok Node requests are capped at three total input images, counting the parent/current image plus references, and return `GROK_REF_TOO_MANY` before upstream when that limit is exceeded. `quality: "high"` promotes the final image model to `grok-imagine-image-quality`.
|
|
194
283
|
|
|
195
|
-
The route can stream Server-Sent Events when the client sends `Accept: text/event-stream`. Possible events include `phase`, `partial`, `done`, and `error`.
|
|
284
|
+
The route can stream Server-Sent Events when the client sends `Accept: text/event-stream`. Possible events include `phase`, `partial`, `done`, and `error`. Alternatively, send `{ "async": true, "requestId": "req_xxx" }` in the body to receive `202 { requestId }` immediately and follow progress on `GET /api/events` (see Events section).
|
|
196
285
|
|
|
197
286
|
Grok Node SSE responses do not include Responses API `partial` image events because the xAI Images API call is synchronous JSON. They still emit `phase` and `done`/`error` events so the Node UI can use the same in-flight lifecycle.
|
|
198
287
|
|
|
288
|
+
### `POST /api/generate/multimode` (SSE)
|
|
289
|
+
|
|
290
|
+
Multi-image sequence generation. SSE-only on the POST response unless async mode is used.
|
|
291
|
+
|
|
292
|
+
```json
|
|
293
|
+
{
|
|
294
|
+
"prompt": "a story in four panels",
|
|
295
|
+
"maxImages": 4,
|
|
296
|
+
"quality": "medium",
|
|
297
|
+
"size": "1024x1024",
|
|
298
|
+
"format": "png",
|
|
299
|
+
"moderation": "low",
|
|
300
|
+
"model": "gpt-5.4",
|
|
301
|
+
"provider": "oauth",
|
|
302
|
+
"references": [],
|
|
303
|
+
"requestId": "optional-client-id",
|
|
304
|
+
"async": false
|
|
305
|
+
}
|
|
306
|
+
```
|
|
307
|
+
|
|
308
|
+
Send `Accept: text/event-stream` for per-request SSE on the POST connection. Or set `"async": true` with a client `requestId` to get `202 { requestId }` and receive events on `GET /api/events`.
|
|
309
|
+
|
|
310
|
+
**SSE events**:
|
|
311
|
+
|
|
312
|
+
| Event | Data | Description |
|
|
313
|
+
|---|---|---|
|
|
314
|
+
| `phase` | `{ requestId, phase, sequenceId?, maxImages? }` | Lifecycle phase |
|
|
315
|
+
| `partial` | `{ requestId, image, index }` | Progressive preview |
|
|
316
|
+
| `image` | full `GenerateItem` | One saved sequence image |
|
|
317
|
+
| `done` | route-specific summary; may include `status: "partial"` after timeout if at least one image was saved | Sequence complete |
|
|
318
|
+
| `error` | `{ requestId, error, code?, status? }` | Generation failed |
|
|
319
|
+
|
|
199
320
|
### `GET /api/node/:nodeId`
|
|
200
321
|
|
|
201
322
|
Fetch stored node metadata and asset URL.
|
|
@@ -221,7 +342,7 @@ Server-side validation may return these reference codes:
|
|
|
221
342
|
|
|
222
343
|
### `POST /api/video/generate` (SSE)
|
|
223
344
|
|
|
224
|
-
Generate a video via the Grok video provider. Returns Server-Sent Events.
|
|
345
|
+
Generate a video via the Grok video provider. Returns Server-Sent Events on the POST connection, or accepts async mode (`{ "async": true, "requestId": "req_xxx" }`) for `202 { requestId }` with progress on `GET /api/events` (see Events section).
|
|
225
346
|
|
|
226
347
|
```json
|
|
227
348
|
{
|
|
@@ -256,7 +377,7 @@ Generate a video via the Grok video provider. Returns Server-Sent Events.
|
|
|
256
377
|
| Field | Type | Default | Notes |
|
|
257
378
|
|---|---|---|---|
|
|
258
379
|
| `prompt` | string | — | Required |
|
|
259
|
-
| `provider` | string | `"grok"` |
|
|
380
|
+
| `provider` | string | `"grok"` | `"grok"` or `"grok-api"` |
|
|
260
381
|
| `model` | string | `grok-imagine-video` | Video model |
|
|
261
382
|
| `duration` | integer | `5` | 1–15 seconds (clamped to 10 for reference-to-video) |
|
|
262
383
|
| `resolution` | string | `"480p"` | `480p` or `720p` |
|
|
@@ -267,13 +388,17 @@ Generate a video via the Grok video provider. Returns Server-Sent Events.
|
|
|
267
388
|
| `referenceFilenames` | string[] | — | Existing generated files for reference-to-video |
|
|
268
389
|
| `continueFromVideo` | string | — | Generated `.mp4` parent; server extracts its last frame and rebuilds lineage from sidecar |
|
|
269
390
|
| `continuityLineage` | object | — | Optional client hint; used only when `continueFromVideo` is absent |
|
|
391
|
+
| `plannerModel` | string | `grok-4.3` | Grok video planner model override (also via settings UI or `IMA2_GROK_PLANNER_MODEL`) |
|
|
392
|
+
| `storyboard` | boolean | `false` | Enable storyboard mode — maintains character/scene continuity across sequential clips |
|
|
270
393
|
|
|
271
394
|
Blank prompts return `PROMPT_REQUIRED` with a `guidance` string. The active
|
|
272
395
|
prompt should describe visual flow, motion flow, sound/music/no-music,
|
|
273
396
|
dialogue/no-dialogue, ending frame, and duration pacing. The video planner uses
|
|
274
397
|
the selected duration as the full clip runtime and expands short requests into a
|
|
275
398
|
production-level sequence with opening composition, connected motion/emotion
|
|
276
|
-
change, and a stable ending frame suitable for continuation.
|
|
399
|
+
change, and a stable ending frame suitable for continuation. For multi-character
|
|
400
|
+
scenes, the planner identifies speakers by visual appearance (clothing, physique,
|
|
401
|
+
position, props) rather than names, and attributes each dialogue line accordingly.
|
|
277
402
|
|
|
278
403
|
When `continueFromVideo` is present, the server treats the generated `.mp4`
|
|
279
404
|
sidecar as authoritative. Client `continuityLineage` cannot override it. The
|
|
@@ -313,7 +438,7 @@ Grok prompt surfaces used by video APIs:
|
|
|
313
438
|
|
|
314
439
|
| Surface | Model | Responsibility |
|
|
315
440
|
|---|---|---|
|
|
316
|
-
| Video planner | `grok-4.3` | Converts user prompt, search context, refs, and optional continuity lineage into the final English video prompt. It must structure core subject, action/motion, camera/composition, environment/style, dialogue/audio, ending-frame handoff, and constraints. |
|
|
441
|
+
| Video planner | `grok-4.3` (override via `plannerModel`) | Converts user prompt, search context, refs, and optional continuity lineage into the final English video prompt. It must structure core subject, action/motion, camera/composition, environment/style, dialogue/audio, ending-frame handoff, and constraints. Multi-character dialogue uses appearance-based speaker identification. |
|
|
317
442
|
| Video generation | xAI video model | Receives the planner prompt plus `sourceImage` or `referenceImages` when present. |
|
|
318
443
|
| Video analysis | `grok-4.3` | Reads first/last frame images from `/api/video/analyze` and returns recreation/continuation guidance. |
|
|
319
444
|
|
|
@@ -461,6 +586,47 @@ Style-sheet extraction can require an API key/openai client. Image generation al
|
|
|
461
586
|
| `GROK_RATE_LIMITED` | xAI returned a rate-limit response through progrok |
|
|
462
587
|
| `GROK_AUTH_FAILED` | progrok could not authenticate the xAI request |
|
|
463
588
|
| `GROK_SEARCH_TIMEOUT` / `GROK_PLANNER_TIMEOUT` / `GROK_IMAGE_TIMEOUT` | The Grok search, planner, or image API step exceeded its timeout budget |
|
|
589
|
+
| `AGY_GENERATION_FAILED` | Gemini (agy) image generation failed |
|
|
590
|
+
| `AGY_TIMEOUT` | Agy CLI process exceeded its 360-second timeout |
|
|
591
|
+
| `AGY_PROCESS_ERROR` | Agy CLI binary failed to start or crashed |
|
|
592
|
+
| `AGY_QUOTA_EXHAUSTED` | Gemini API quota exhausted (rate limit) |
|
|
593
|
+
| `AGY_PARSE_FAILED` | Could not parse artifact path from agy output |
|
|
594
|
+
| `AGY_ARTIFACT_NOT_FOUND` | Agy reported an artifact path that does not exist |
|
|
595
|
+
| `AGY_PATH_REJECTED` | Agy artifact path was outside allowed directories |
|
|
596
|
+
| `AGY_VIDEO_UNSUPPORTED` | Video generation is not supported by the Gemini (agy) provider |
|
|
597
|
+
| `AGY_MASK_UNSUPPORTED` | Mask-based editing is not supported by the Gemini (agy) provider |
|
|
598
|
+
| `AGY_REF_TOO_MANY` | Too many reference images for agy (max 3) |
|
|
599
|
+
| `GEMINI_API_KEY_MISSING` | Gemini API key or Vertex AI credentials not configured |
|
|
600
|
+
| `GEMINI_API_RATE_LIMITED` | Gemini API rate limited (429) |
|
|
601
|
+
| `GEMINI_API_BAD_REQUEST` | Gemini API bad request (400/403) |
|
|
602
|
+
| `GEMINI_API_SAFETY_BLOCKED` | Gemini API generation blocked by safety filter |
|
|
603
|
+
| `GEMINI_API_NO_IMAGE` | Gemini API returned no image in response |
|
|
604
|
+
| `VIDEO_PROVIDER_UNSUPPORTED` | Video generation requires provider `"grok"` or `"grok-api"` |
|
|
605
|
+
| `SSE_CAPACITY` | More than 512 concurrent `GET /api/events` listeners |
|
|
606
|
+
| `REQUEST_ID_IN_USE` | Async POST used a `requestId` that already has an active job |
|
|
607
|
+
| `TOO_MANY_JOBS` | More than 12 concurrent active generation jobs (`Retry-After: 5`) |
|
|
608
|
+
|
|
609
|
+
## Key Management
|
|
610
|
+
|
|
611
|
+
API key management endpoints for configuring provider credentials at runtime through the web UI or HTTP API.
|
|
612
|
+
|
|
613
|
+
| Endpoint | Method | Description |
|
|
614
|
+
|---|---|---|
|
|
615
|
+
| `/api/keys/status` | GET | Returns configured/valid/maskedKey status for all providers (openai, xai, gemini, vertex) |
|
|
616
|
+
| `/api/keys/:provider` | PUT | Save an API key. Body: `{ "apiKey": "..." }`. Validates key format and upstream before saving to config.json. Provider: `openai`, `xai`, or `gemini`. |
|
|
617
|
+
| `/api/keys/:provider` | DELETE | Remove a config-sourced API key. Env-sourced keys cannot be removed (`ENV_KEY_IMMUTABLE`). |
|
|
618
|
+
| `/api/keys/vertex` | PUT | Save a Vertex AI service account JSON. Body: `{ "serviceAccountJson": "..." }`. Validates JSON structure (`type: "service_account"`, `project_id` required). |
|
|
619
|
+
| `/api/keys/vertex` | DELETE | Remove a config-sourced Vertex AI service account. |
|
|
620
|
+
|
|
621
|
+
Keys saved via PUT are stored in `config.json` and hot-updated in the runtime context (no server restart required). Keys loaded from environment variables (`OPENAI_API_KEY`, `XAI_API_KEY`, `GEMINI_API_KEY`, `VERTEX_SERVICE_ACCOUNT_JSON`) take precedence and are immutable through the API.
|
|
622
|
+
|
|
623
|
+
## Thumbnail Backfill
|
|
624
|
+
|
|
625
|
+
| Endpoint | Method | Description |
|
|
626
|
+
|---|---|---|
|
|
627
|
+
| `/api/history/backfill-thumbnails` | POST | Generate missing `.thumb.jpg` thumbnails for all images and videos in the generated directory. Returns `{ ok, total, created, skipped, failed }`. Also available offline via `ima2 backfill-thumbs`. |
|
|
628
|
+
|
|
629
|
+
Thumbnails are also generated automatically on server startup for any media files that lack them.
|
|
464
630
|
|
|
465
631
|
## Endpoint → CLI Mapping
|
|
466
632
|
|
|
@@ -480,7 +646,7 @@ Most server routes under `/api/*` have a CLI wrapper. The exception is **Agent M
|
|
|
480
646
|
| `POST /api/node/generate` (SSE) / `GET /api/node/:id` | `ima2 node generate` / `ima2 node show` |
|
|
481
647
|
| `GET /api/history` | `ima2 ls` |
|
|
482
648
|
| `DELETE /api/history/:name` / `…/permanent` | `ima2 history rm [--permanent]` |
|
|
483
|
-
| `POST /api/history/restore` | `ima2 history restore --trash-id` |
|
|
649
|
+
| `POST /api/history/:filename/restore` | `ima2 history restore --trash-id` |
|
|
484
650
|
| `POST /api/history/favorite` | `ima2 history favorite` |
|
|
485
651
|
| `POST /api/history/import-local` | `ima2 history import` |
|
|
486
652
|
| `POST /api/metadata/read` | `ima2 metadata` / `ima2 show --metadata` |
|
|
@@ -495,17 +661,25 @@ Most server routes under `/api/*` have a CLI wrapper. The exception is **Agent M
|
|
|
495
661
|
| `…/api/cardnews/…` (gated on `features.cardNews`) | `ima2 cardnews …` |
|
|
496
662
|
| `POST /api/comfy/export-image` | `ima2 comfy export` |
|
|
497
663
|
| `GET /api/inflight` / `DELETE /api/inflight/:id` | `ima2 inflight ls` (alias `ps`) / `ima2 inflight rm` (alias `cancel`) |
|
|
664
|
+
| `GET /api/events` (SSE multiplex) | Web UI only (persistent `EventSource`; no CLI wrapper) |
|
|
498
665
|
| `GET /api/storage/status` / `POST /api/storage/open-generated-dir` | `ima2 storage status` / `ima2 storage open` |
|
|
499
666
|
| `GET /api/billing` / `GET /api/providers` / `GET /api/oauth/status` / `GET /api/grok/status` | `ima2 billing` / `ima2 providers` / `ima2 oauth status` / `ima2 grok status` |
|
|
667
|
+
| `GET /api/quota` | `ima2 billing` (includes Grok `usedUsd`/`limitUsd`) |
|
|
668
|
+
| `POST /api/auth/switch` / `GET /api/auth/switch/:sessionId` | Web UI only (Settings > QuotaCard > Switch Account) |
|
|
500
669
|
| `GET /api/health` | `ima2 ping` |
|
|
501
670
|
| `GET /api/capabilities` | `ima2 capabilities` |
|
|
671
|
+
| `GET /api/config/grok-planner` | — (Grok planner model query) |
|
|
672
|
+
| `PATCH /api/config/grok-planner` | — (Grok planner model update) |
|
|
673
|
+
| `GET /api/agy/status` | — (Antigravity CLI install status) |
|
|
674
|
+
| `POST /api/history/backfill-thumbnails` | `ima2 backfill-thumbs` |
|
|
675
|
+
| `GET /api/keys/status`, `PUT/DELETE /api/keys/:provider`, `PUT/DELETE /api/keys/vertex` | Web UI only (Settings > API Keys) |
|
|
502
676
|
| `GET/POST/PATCH/DELETE /api/agent/*` (sessions, turns, queue) | — (Agent Mode; web UI only, no CLI) |
|
|
503
677
|
| `POST /api/prompt-builder/chat` | `ima2 prompt build` |
|
|
504
678
|
|
|
505
679
|
Notes:
|
|
506
680
|
- `ima2 history favorite` and `ima2 annotate …` send `X-Ima2-Browser-Id: cli-<sha1prefix>` derived from the config dir, so CLI activity does not collide with browser sessions.
|
|
507
681
|
- `ima2 session graph save` performs a GET-then-PUT with `If-Match: "<version>"` to guard against `GRAPH_VERSION_CONFLICT`.
|
|
508
|
-
- `ima2 history import` and `ima2 canvas-versions save/update` send raw bytes with `Content-Type: image/<png|jpeg|webp>`; the SSE endpoints (`multimode`, `node generate`) use `Accept: text/event-stream`.
|
|
682
|
+
- `ima2 history import` and `ima2 canvas-versions save/update` send raw bytes with `Content-Type: image/<png|jpeg|webp>`; the SSE endpoints (`multimode`, `node generate`, `video`) use `Accept: text/event-stream`. The web UI instead uses `GET /api/events` plus `async: true` on POST routes.
|
|
509
683
|
- `ima2 cardnews …` checks `runtimeConfig.features.cardNews` before calling the gated endpoints; when disabled the CLI exits 2 with a clear message instead of producing a 404.
|
|
510
684
|
|
|
511
685
|
## CLI Discovery
|
package/docs/CLI.md
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# CLI Reference
|
|
2
2
|
|
|
3
|
-
Most server routes under `/api/*` have a CLI wrapper; Agent Mode (`/api/agent/*`) is web-UI-only and has no `ima2` subcommand. The prompt builder HTTP route (`POST /api/prompt-builder/chat`) is available through `ima2 prompt build`. The CLI is a thin shell over the local server, so most commands require a running `ima2 serve` (the few exceptions — `serve`, `setup`, `doctor`, `status`, `open`, `reset`, `config`, `grok`, `skill`, `capabilities`, and local `defaults` inspection — work without a live server).
|
|
3
|
+
Most server routes under `/api/*` have a CLI wrapper; Agent Mode (`/api/agent/*`) is web-UI-only and has no `ima2` subcommand. The prompt builder HTTP route (`POST /api/prompt-builder/chat`) is available through `ima2 prompt build`. The CLI is a thin shell over the local server, so most commands require a running `ima2 serve` (the few exceptions — `serve`, `setup`, `doctor`, `status`, `open`, `reset`, `config`, `grok`, `skill`, `capabilities`, `backfill-thumbs`, and local `defaults` inspection — work without a live server).
|
|
4
4
|
|
|
5
5
|
For a quick start, see the [main README](../README.md). For endpoint mapping, see [API.md](API.md).
|
|
6
6
|
|
|
@@ -16,6 +16,7 @@ For a quick start, see the [main README](../README.md). For endpoint mapping, se
|
|
|
16
16
|
| `ima2 open` | Open the web UI in a browser |
|
|
17
17
|
| `ima2 grok login/status/models/proxy` | Manage the bundled progrok runtime used by the Grok provider |
|
|
18
18
|
| `ima2 reset` | Remove saved config |
|
|
19
|
+
| `ima2 backfill-thumbs` | Generate missing gallery thumbnails for images and videos (offline, no running server needed) |
|
|
19
20
|
|
|
20
21
|
## Common flags
|
|
21
22
|
|
|
@@ -53,13 +54,15 @@ Agents should start from the packaged skill and capability commands instead of g
|
|
|
53
54
|
| `ima2 node generate` | Node-mode generate (SSE; supports `--no-stream`) |
|
|
54
55
|
| `ima2 node show <nodeId>` | Read node metadata |
|
|
55
56
|
|
|
56
|
-
Generation flags include `--provider <auto|oauth|api|grok>`, `--reasoning-effort {none\|low\|medium\|high\|xhigh}`, `--web-search` / `--no-web-search`, `--model`, `--mode`, `--moderation`, `--ref <file>` (repeatable, up to 5 where supported), `-q low|medium|high`, `-n <count>`, `-o <file>`.
|
|
57
|
+
Generation flags include `--provider <auto|oauth|api|grok|grok-api|agy|gemini-api>`, `--reasoning-effort {none\|low\|medium\|high\|xhigh}`, `--web-search` / `--no-web-search`, `--model`, `--mode`, `--moderation`, `--ref <file>` (repeatable, up to 5 where supported), `-q low|medium|high`, `-n <count>`, `-o <file>`.
|
|
57
58
|
|
|
58
59
|
Provider override semantics:
|
|
59
60
|
|
|
60
61
|
- `api` forces the API-key Responses path and requires a configured API key.
|
|
61
62
|
- `oauth` forces the local OAuth proxy path.
|
|
62
63
|
- `grok` uses the bundled progrok xAI proxy (`127.0.0.1:18645`). Classic generation first runs mandatory xAI Web Search through Responses API, then asks `grok-4.3` to call ima2's local `generate_image` tool, then ima2 executes xAI `/v1/images/generations`. If `--ref` images are attached, the final step uses xAI `/v1/images/edits` instead so image-to-image/reference context is preserved. Models: `grok-imagine-image`, `grok-imagine-image-quality`. Size is mapped to xAI `aspect_ratio` and `resolution`; the UI web-search toggle is OpenAI-provider-only because Grok search is always on in this path.
|
|
64
|
+
- `agy` spawns the Antigravity CLI to generate via Google Gemini (`nano-banana-2`). Fixed 1024×1024 JPEG output, max 3 refs. No web search, quality, size, or mask controls.
|
|
65
|
+
- `gemini-api` calls the Google Generative Language API directly. Models: `nano-banana-2` (Gemini 3.1 Flash Image) and `nano-banana-pro` (Gemini 3 Pro Image). Use `--model nano-banana-2` or `--model nano-banana-pro` to select. Supports `--size` for aspect ratio and resolution (512px–4K) on the direct API path; Vertex AI ignores aspect/size. Requires `GEMINI_API_KEY` or a Vertex AI service account (`VERTEX_SERVICE_ACCOUNT_JSON`). Switching from `agy` or `gemini-api` provider auto-selects the corresponding Gemini model; switching away resets to the GPT default.
|
|
63
66
|
- `auto` preserves route default behavior and currently resolves to GPT OAuth unless server routing changes.
|
|
64
67
|
|
|
65
68
|
`ima2 serve` starts the bundled Grok proxy automatically. No separate `progrok`
|
|
@@ -105,7 +108,7 @@ mockup`.
|
|
|
105
108
|
For dense or critical text, keep the text large and explicit. Exact placement,
|
|
106
109
|
small text, and pixel-perfect typography can still need iteration or post-editing.
|
|
107
110
|
|
|
108
|
-
Multimode-specific flags include `--max-images <1..8>`, `--ref <file>` (repeatable, max 5), `--mode <auto|direct>`, `--provider <auto|oauth|api|grok>`, and `--show-partial`. `ima2 edit --mask` remains intentionally deferred to #31 because current mask plumbing is guided edit rather than guaranteed true masked/inpaint semantics.
|
|
111
|
+
Multimode-specific flags include `--max-images <1..8>`, `--ref <file>` (repeatable, max 5), `--mode <auto|direct>`, `--provider <auto|oauth|api|grok|grok-api|agy|gemini-api>`, and `--show-partial`. `ima2 edit --mask` remains intentionally deferred to #31 because current mask plumbing is guided edit rather than guaranteed true masked/inpaint semantics.
|
|
109
112
|
|
|
110
113
|
## Video
|
|
111
114
|
|
|
@@ -126,6 +129,8 @@ Video generate flags:
|
|
|
126
129
|
| `--resolution <480p\|720p>` | Video resolution (default: 480p) |
|
|
127
130
|
| `--aspect-ratio <ratio\|auto>` | 1:1, 16:9, 9:16, 4:3, 3:4, 3:2, 2:3, auto (default: auto) |
|
|
128
131
|
| `--model <name>` | `grok-imagine-video` or `grok-imagine-video-1.5-preview` |
|
|
132
|
+
| `--planner-model <name>` | Grok planner override (default: `grok-4.3`; also in settings UI and `IMA2_GROK_PLANNER_MODEL`) |
|
|
133
|
+
| `--storyboard` | Enable storyboard mode — maintains character/scene continuity across sequential clips |
|
|
129
134
|
| `--ref <file>` | Attach source/reference image (repeatable, max 7) |
|
|
130
135
|
| `-o, --out <file>` | Output file path |
|
|
131
136
|
| `-d, --out-dir <dir>` | Output directory |
|
|
@@ -160,6 +165,8 @@ Video continue flags:
|
|
|
160
165
|
| `--aspect-ratio <ratio\|auto>` | New clip aspect ratio |
|
|
161
166
|
| `--model <name>` | Optional video generation model |
|
|
162
167
|
|
|
168
|
+
Video continue also accepts `--planner-model` and `--storyboard`.
|
|
169
|
+
|
|
163
170
|
Video mode is auto-detected from `--ref` count:
|
|
164
171
|
|
|
165
172
|
| Refs | Mode |
|
|
@@ -311,7 +318,7 @@ Card News requires the server to be started with `IMA2_CARD_NEWS=1` (or `feature
|
|
|
311
318
|
| `ima2 inflight rm <requestId>` | Force-remove a stuck job |
|
|
312
319
|
| `ima2 storage status` | Storage inspection (richer than `doctor`) |
|
|
313
320
|
| `ima2 storage open` | Open the generated dir in the OS file manager (POST) |
|
|
314
|
-
| `ima2 billing` | API usage / quota |
|
|
321
|
+
| `ima2 billing` | API usage / quota; Grok result includes `billing.usedUsd` / `billing.limitUsd` drawn from the xAI billing API |
|
|
315
322
|
| `ima2 providers` | Configured providers |
|
|
316
323
|
| `ima2 oauth status` | OAuth proxy state |
|
|
317
324
|
| `ima2 grok status` | Bundled progrok / xAI image-model probe state |
|
package/docs/FAQ.ko.md
CHANGED
|
@@ -323,6 +323,22 @@ export HTTPS_PROXY=http://127.0.0.1:7890
|
|
|
323
323
|
|
|
324
324
|
GPT OAuth는 OpenAI와 ChatGPT/Codex 관련 호스트 접근이 필요할 수 있습니다. 회사 방화벽, TLS 검사, VPN, 프록시가 흐름을 깨뜨릴 수 있습니다. 로그인 실패와 `failed to fetch`가 반복되면 다른 네트워크에서도 시도해 보세요.
|
|
325
325
|
|
|
326
|
+
## SSE 멀티플렉싱
|
|
327
|
+
|
|
328
|
+
### 왜 웹 UI가 단일 SSE 연결을 쓰나요?
|
|
329
|
+
|
|
330
|
+
브라우저는 같은 origin에 대해 동시 HTTP 연결 수를 제한합니다(보통 6개). 여러 이미지를 동시에 생성할 때 각 요청이 SSE 연결을 점유하면, multimode+node+video가 동시에 돌아갈 때 연결이 포화되어 갤러리 썸네일이 멈췄습니다.
|
|
331
|
+
|
|
332
|
+
이제 웹 UI는 `GET /api/events`로 하나의 SSE 연결만 열고, 모든 생성 진행 이벤트를 멀티플렉싱합니다. 생성 요청은 `async: true`로 보내면 즉시 `202 { requestId }` 응답을 받아 연결을 바로 해제합니다. CLI는 영향 없이 기존 per-request SSE를 그대로 사용합니다.
|
|
333
|
+
|
|
334
|
+
### SSE 연결이 끊기면 어떻게 되나요?
|
|
335
|
+
|
|
336
|
+
이벤트 채널 클라이언트가 지수 백오프로 자동 재연결합니다. 재연결 시 `Last-Event-ID`를 보내서 서버의 링 버퍼(최대 2000건)에서 놓친 이벤트를 재전송받습니다. 버퍼에서 이미 사라진 이벤트가 있으면 `replay-gap` 이벤트로 알려줍니다.
|
|
337
|
+
|
|
338
|
+
### 동시 작업 상한은 얼마인가요?
|
|
339
|
+
|
|
340
|
+
서버는 동시 생성 작업을 12건(`MAX_CONCURRENT_JOBS`)으로 제한합니다. 초과 요청은 `429`와 `Retry-After: 5`를 받습니다. SSE 엔드포인트 자체는 512개 동시 연결까지 지원합니다.
|
|
341
|
+
|
|
326
342
|
## CLI 점검 순서
|
|
327
343
|
|
|
328
344
|
아래 순서대로 확인해 보세요.
|
package/docs/FAQ.md
CHANGED
|
@@ -103,6 +103,20 @@ ima2 serve
|
|
|
103
103
|
|
|
104
104
|
If this happens on a company network, a firewall, VPN, proxy, or captive portal may also be blocking the OAuth flow.
|
|
105
105
|
|
|
106
|
+
### How do I use the Gemini providers?
|
|
107
|
+
|
|
108
|
+
Two Gemini providers are available:
|
|
109
|
+
|
|
110
|
+
- **`agy`** — uses the Antigravity CLI (`agy -p`) with no API key needed. Requires the `agy` binary to be installed and logged in. Model is `nano-banana-2`, output is fixed at 1024×1024.
|
|
111
|
+
|
|
112
|
+
- **`gemini-api`** — calls the Google Generative Language API directly. Add a `GEMINI_API_KEY` env var, or configure a key via Settings > API Keys. For Vertex AI, add a service account JSON via Settings or the `VERTEX_SERVICE_ACCOUNT_JSON` env var. When both an API key and Vertex credentials are present, Vertex takes priority. Use the auth-mode dropdown in Settings to switch between `apikey` and `vertex`; the choice is saved and restored automatically.
|
|
113
|
+
|
|
114
|
+
The `gemini-api` provider supports two models: `nano-banana-2` (Gemini 3.1 Flash Image) and `nano-banana-pro` (Gemini 3 Pro Image). The web UI shows aspect-ratio and resolution controls (512px–4K) for `gemini-api`; these are honored only on the direct Gemini API path and are ignored by Vertex AI.
|
|
115
|
+
|
|
116
|
+
### How do I re-authenticate Grok or Codex without restarting?
|
|
117
|
+
|
|
118
|
+
Use the **Switch Account** button in Settings > QuotaCard for the provider. This starts a device-code OAuth flow: a new browser tab opens the verification URL, you complete the login, and the server automatically picks up the new credentials. The Grok quota bar also shows `$used / $limit` (in USD) drawn from the xAI billing API.
|
|
119
|
+
|
|
106
120
|
## Models and quota
|
|
107
121
|
|
|
108
122
|
### Which model should I use?
|
|
@@ -334,6 +348,22 @@ Use the host and port from your proxy client. If `ima2-gen` still fails after th
|
|
|
334
348
|
|
|
335
349
|
GPT OAuth may require access to OpenAI and ChatGPT/Codex-related hosts. A corporate firewall, TLS inspection, VPN, or proxy can break the flow. Try a different network if login and `failed to fetch` errors keep repeating.
|
|
336
350
|
|
|
351
|
+
## SSE Multiplexing
|
|
352
|
+
|
|
353
|
+
### Why does the web UI use a single SSE connection?
|
|
354
|
+
|
|
355
|
+
Browsers limit the number of concurrent HTTP connections to the same origin (typically 6). When generating multiple images at once, each generation request used to hold a Server-Sent Events connection open. With multimode, node, and video running simultaneously, the browser would run out of connections and gallery thumbnails would hang.
|
|
356
|
+
|
|
357
|
+
The web UI now opens a single persistent `GET /api/events` SSE connection and all generation progress is multiplexed through it. Generation requests use `async: true` and receive an immediate `202 { requestId }` response, freeing the connection immediately. The CLI is unaffected — it still uses per-request SSE when `async` is not set.
|
|
358
|
+
|
|
359
|
+
### What happens if the SSE connection drops?
|
|
360
|
+
|
|
361
|
+
The event channel client reconnects automatically with exponential backoff. On reconnect, it sends `Last-Event-ID` so the server can replay missed events from its ring buffer (up to 2000 entries). If events have been evicted from the buffer, the server sends a `replay-gap` event so the client knows some updates may have been lost.
|
|
362
|
+
|
|
363
|
+
### What is the maximum number of concurrent jobs?
|
|
364
|
+
|
|
365
|
+
The server caps concurrent generation jobs at 12 (`MAX_CONCURRENT_JOBS`). Additional requests receive `429` with `Retry-After: 5`. The SSE endpoint itself caps at 512 simultaneous connections.
|
|
366
|
+
|
|
337
367
|
## CLI troubleshooting checklist
|
|
338
368
|
|
|
339
369
|
Run these in order:
|
package/docs/PROMPT_STUDIO.md
CHANGED
|
@@ -12,10 +12,12 @@ you want a reproducible way to report a workspace issue.
|
|
|
12
12
|
| Area | What it does | Notes |
|
|
13
13
|
|---|---|---|
|
|
14
14
|
| Composer | Holds the prompt for the next request. | Selecting an existing image is view-only. It should not overwrite the composer. |
|
|
15
|
+
| Storyboard | Maintains character and scene continuity across sequential frames. | Toggle in the composer. Works for image and video generation; image keyframes are composed for video production. |
|
|
15
16
|
| Multimode | Starts several separate image requests from the current prompt. | Each slot is a candidate output, not a collage panel or a guaranteed scene sequence. |
|
|
16
17
|
| 1:1 Direct | Sends the prompt through with less rewriting by the app. | Use it for exact wording, strict prompt experiments, or provider-side prompt syntax. |
|
|
17
18
|
| Model quick menu | Changes the image model and reasoning effort from the sidebar header. | The full Settings workspace remains the detailed configuration page. |
|
|
18
|
-
| Recent generations | Shows the visible Prompt Studio history domain. | Arrow keys move inside the same visible recent domain instead of hidden older rows. Video items render as video thumbnails. Drag any thumbnail to the composer to add it as a reference image. |
|
|
19
|
+
| Recent generations | Shows the visible Prompt Studio history domain. | Arrow keys move inside the same visible recent domain instead of hidden older rows. Video items render as video thumbnails. Drag any thumbnail to the composer to add it as a reference image. Video results expose First, Mid, and Last frame buttons to copy keyframes. |
|
|
20
|
+
| Video settings | Controls Grok video duration, resolution, aspect ratio, and planner model. | Default planner model is `grok-4.3`; override per request when needed. |
|
|
19
21
|
| Gallery | Browses saved local images, All/Favorites tabs, and folders. | Favorite toggles should preserve the gallery viewport you were browsing. |
|
|
20
22
|
| Prompt library | Imports saved prompt text into the composer intentionally. | Library insert/continue actions are explicit prompt imports; passive image selection is not. |
|
|
21
23
|
|
package/docs/README.ko.md
CHANGED
|
@@ -78,7 +78,7 @@ v1.1.22부터 Ctrl+C가 DB, 소켓, 자식 프로세스를 깨끗하게 정리
|
|
|
78
78
|
1. **GPT OAuth** — ChatGPT 계정으로 로그인 (무료, 이미지만)
|
|
79
79
|
2. **Grok OAuth** — xAI/Grok 계정으로 로그인 (이미지 + 영상)
|
|
80
80
|
3. **Both** — GPT + Grok 둘 다 (전체 기능)
|
|
81
|
-
4. **
|
|
81
|
+
4. **Web setup** — 웹 UI에서 전체 설정
|
|
82
82
|
|
|
83
83
|
영상 생성은 Grok OAuth(2번 또는 3번)가 필요합니다.
|
|
84
84
|
|
|
@@ -95,6 +95,10 @@ v1.1.22부터 Ctrl+C가 DB, 소켓, 자식 프로세스를 깨끗하게 정리
|
|
|
95
95
|
- **Mobile shell**: 작은 화면에서는 app bar, compose sheet, compact settings toggle로 조작합니다.
|
|
96
96
|
- **Observable jobs**: 진행 중인 작업과 최근 완료된 작업을 request ID로 추적합니다.
|
|
97
97
|
|
|
98
|
+
### SSE 멀티플렉싱
|
|
99
|
+
|
|
100
|
+
웹 UI는 단일 `GET /api/events` Server-Sent Events 연결로 모든 생성 진행 상황을 수신합니다. Multimode, node, video 요청은 비동기 POST(`202 { requestId }`)로 제출되고, 이벤트 버스를 통해 진행 이벤트가 멀티플렉싱됩니다. 기존 브라우저 6-연결 제한으로 인한 동시 생성 시 갤러리 hang 문제가 해결됩니다. `async: true`를 보내지 않는 CLI 클라이언트는 기존 per-request SSE 스트림을 그대로 사용할 수 있습니다.
|
|
101
|
+
|
|
98
102
|
## 이미지 생성 공급자
|
|
99
103
|
|
|
100
104
|
이미지 생성은 로컬 Codex/ChatGPT OAuth, OpenAI API key, 번들 Grok 공급자를 지원합니다.
|
|
@@ -232,8 +236,8 @@ environment variables > ~/.ima2/config.json > built-in defaults
|
|
|
232
236
|
| `IMA2_NO_GROK_PROXY` | — | `1`이면 progrok 자동 시작 비활성화 |
|
|
233
237
|
| `IMA2_GROK_PLANNER_MODEL` | `grok-4.3` | Grok 플래너 모델 (설정 UI 또는 `--planner-model` CLI 플래그로도 변경 가능) |
|
|
234
238
|
| `IMA2_GROK_IMAGE_MODEL_DEFAULT` | `grok-imagine-image` | 기본 Grok 이미지 모델 |
|
|
235
|
-
| `IMA2_LOG_LEVEL` | `
|
|
236
|
-
| `IMA2_INFLIGHT_TERMINAL_TTL_MS` | `
|
|
239
|
+
| `IMA2_LOG_LEVEL` | `info` | 일반 `serve`는 `info`, dev 모드는 `debug`. `debug`, `info`, `warn`, `error`, `silent` 지원 |
|
|
240
|
+
| `IMA2_INFLIGHT_TERMINAL_TTL_MS` | `300000` | 디버그용 최근 작업 보존 시간 (5분) |
|
|
237
241
|
| `OPENAI_API_KEY` | — | `provider: "api"` Responses 이미지 경로와 보조 기능용 API 키 |
|
|
238
242
|
|
|
239
243
|
### 로그 모드
|
|
@@ -4,7 +4,7 @@ Generated by `npm run test:inventory` (script: `scripts/classify-tests.mjs`).
|
|
|
4
4
|
|
|
5
5
|
_Tests considered "runtime-importing" if they import from `../lib/`, `../routes/`, `../bin/`, `../server`, or `../config`._
|
|
6
6
|
|
|
7
|
-
Total:
|
|
7
|
+
Total: 191 (runtime: 66, contract: 125)
|
|
8
8
|
|
|
9
9
|
## Runtime-importing tests
|
|
10
10
|
- `tests/agent-mode-auto-planner-contract.test.ts`
|
|
@@ -18,10 +18,14 @@ Total: 175 (runtime: 60, contract: 115)
|
|
|
18
18
|
- `tests/api-provider-parity.test.ts`
|
|
19
19
|
- `tests/billing-source.test.ts`
|
|
20
20
|
- `tests/card-news-contract.test.ts`
|
|
21
|
+
- `tests/card-news-template.test.ts`
|
|
22
|
+
- `tests/classic-generate-async.test.ts`
|
|
21
23
|
- `tests/cli-error-hints.test.ts`
|
|
22
24
|
- `tests/cli-lib.test.ts`
|
|
23
25
|
- `tests/comfy-bridge-contract.test.ts`
|
|
24
26
|
- `tests/error-classify.test.ts`
|
|
27
|
+
- `tests/event-bus.test.ts`
|
|
28
|
+
- `tests/events-channel-contract.test.ts`
|
|
25
29
|
- `tests/generate-route-validation-error.test.ts`
|
|
26
30
|
- `tests/generated-static-privacy.test.ts`
|
|
27
31
|
- `tests/generation-errors.test.ts`
|
|
@@ -36,6 +40,7 @@ Total: 175 (runtime: 60, contract: 115)
|
|
|
36
40
|
- `tests/image-metadata-route.test.ts`
|
|
37
41
|
- `tests/image-metadata-xmp.test.ts`
|
|
38
42
|
- `tests/image-model.test.ts`
|
|
43
|
+
- `tests/inflight-guard-contract.test.ts`
|
|
39
44
|
- `tests/inflight-persistence.test.ts`
|
|
40
45
|
- `tests/inflight.test.ts`
|
|
41
46
|
- `tests/local-import-contract.test.ts`
|
|
@@ -64,6 +69,7 @@ Total: 175 (runtime: 60, contract: 115)
|
|
|
64
69
|
- `tests/star-prompt.test.ts`
|
|
65
70
|
- `tests/storage-migration.test.ts`
|
|
66
71
|
- `tests/style-sheet.test.ts`
|
|
72
|
+
- `tests/thumb-backfill.test.ts`
|
|
67
73
|
- `tests/videoContinuity.test.ts`
|
|
68
74
|
- `tests/videoExtendedRoute.test.ts`
|
|
69
75
|
- `tests/videoRoute.test.ts`
|
|
@@ -74,6 +80,9 @@ Total: 175 (runtime: 60, contract: 115)
|
|
|
74
80
|
- `tests/agent-mode-right-sidebar-contract.test.js`
|
|
75
81
|
- `tests/agent-mode-tool-folding-contract.test.js`
|
|
76
82
|
- `tests/app-weight-splitting-contract.test.js`
|
|
83
|
+
- `tests/async-capacity-retry-behavior.test.ts`
|
|
84
|
+
- `tests/async-capacity-retry-contract.test.js`
|
|
85
|
+
- `tests/async-stream-subscribe-order.test.js`
|
|
77
86
|
- `tests/background-cleanup-brush-rasterize.test.js`
|
|
78
87
|
- `tests/background-cleanup-mask-compose.test.js`
|
|
79
88
|
- `tests/bin.test.js`
|
|
@@ -122,8 +131,12 @@ Total: 175 (runtime: 60, contract: 115)
|
|
|
122
131
|
- `tests/current-image-actions-readiness-contract.test.js`
|
|
123
132
|
- `tests/direct-mode-visual-contract.test.js`
|
|
124
133
|
- `tests/edit-mask-api-contract.test.js`
|
|
134
|
+
- `tests/frontend-connection-state-contract.test.js`
|
|
135
|
+
- `tests/frontend-sse-risk-contract.test.js`
|
|
136
|
+
- `tests/gallery-hang-regression-contract.test.ts`
|
|
125
137
|
- `tests/gallery-load-older-contract.test.js`
|
|
126
138
|
- `tests/gallery-navigation-ux-contract.test.js`
|
|
139
|
+
- `tests/gallery-selection-during-generation-contract.test.js`
|
|
127
140
|
- `tests/gallery-session-scope-contract.test.js`
|
|
128
141
|
- `tests/gallery-shortcuts-behavior.test.js`
|
|
129
142
|
- `tests/gallery-shortcuts-visible-domain-contract.test.js`
|
|
@@ -144,6 +157,7 @@ Total: 175 (runtime: 60, contract: 115)
|
|
|
144
157
|
- `tests/multimode-backend-contract.test.js`
|
|
145
158
|
- `tests/multimode-concurrent-store-contract.test.js`
|
|
146
159
|
- `tests/multimode-ui-contract.test.js`
|
|
160
|
+
- `tests/node-async-eventbus-contract.test.js`
|
|
147
161
|
- `tests/node-batch-contract.test.js`
|
|
148
162
|
- `tests/node-child-refs-contract.test.js`
|
|
149
163
|
- `tests/node-child-refs-payload.test.js`
|
|
@@ -155,6 +169,7 @@ Total: 175 (runtime: 60, contract: 115)
|
|
|
155
169
|
- `tests/node-layout-contract.test.js`
|
|
156
170
|
- `tests/node-pending-recovery-contract.test.js`
|
|
157
171
|
- `tests/node-regen-actions-contract.test.js`
|
|
172
|
+
- `tests/node-session-evaporation-contract.test.js`
|
|
158
173
|
- `tests/node-ui-contract.test.js`
|
|
159
174
|
- `tests/oauth-masked-edit-contract.test.js`
|
|
160
175
|
- `tests/oauth-proxy-edit-mask-contract.test.js`
|
|
@@ -182,5 +197,6 @@ Total: 175 (runtime: 60, contract: 115)
|
|
|
182
197
|
- `tests/toast-stack-contract.test.js`
|
|
183
198
|
- `tests/ui-error-code-contract.test.js`
|
|
184
199
|
- `tests/video-continuity-ui-contract.test.js`
|
|
200
|
+
- `tests/video-gallery-refresh-contract.test.ts`
|
|
185
201
|
- `tests/vite-dev-port-contract.test.js`
|
|
186
202
|
- `tests/web-search-toggle-contract.test.js`
|