@genex-ai/cli-demo 0.80.2-dev.213 → 0.84.0-dev.215
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/dist/index.js +801 -83
- package/package.json +1 -1
- package/templates/controllers/character/first-person.ts +54 -0
- package/templates/controllers/character/follow-camera.ts +35 -1
- package/templates/skills/genex-ai-character/SKILL.md +25 -5
- package/templates/skills/genex-ai-hud/SKILL.md +131 -36
- package/templates/skills/genex-ai-hud/references/masked-fill.md +19 -13
- package/templates/skills/genex-ai-hud/references/stage2-prompt-template.md +2 -2
- package/templates/skills/genex-ai-image/SKILL.md +40 -2
- package/templates/skills/genex-ai-menu/SKILL.md +35 -17
- package/templates/skills/genex-ai-model/SKILL.md +8 -0
- package/templates/skills/genex-ai-music/SKILL.md +142 -0
- package/templates/skills/genex-ai-skybox/SKILL.md +21 -5
- package/templates/skills/genex-ai-video/SKILL.md +13 -7
- package/templates/skills/genex-ai-voice/SKILL.md +151 -0
- package/templates/skills/genex-game-director/SKILL.md +128 -44
- package/templates/skills/genex-game-director/references/design-contract.md +7 -2
- package/templates/skills/genex-game-director/references/routing-map.md +4 -1
- package/templates/skills/genex-threejs-camera-direction/SKILL.md +4 -1
- package/templates/skills/genex-threejs-character-controller/SKILL.md +6 -1
- package/templates/skills/genex-threejs-character-controller/references/wiring.md +13 -2
- package/templates/skills/genex-threejs-creatures/SKILL.md +193 -0
- package/templates/skills/genex-threejs-game-ui/SKILL.md +5 -1
- package/templates/skills/genex-threejs-multiplayer/SKILL.md +6 -1
- package/templates/skills/genex-threejs-touch-controls/SKILL.md +7 -0
|
@@ -81,7 +81,10 @@ reason is the failure mode this section exists to kill:**
|
|
|
81
81
|
(cinematic, the example below) / CSS plate (styled FROM the brief —
|
|
82
82
|
gradients, borders, chamfers per the game-ui corner rules; default-gray
|
|
83
83
|
plates are a defect) / sprite chrome (from the HUD sheet or a dedicated
|
|
84
|
-
generation — Tier 2b, when the brief is ornate)
|
|
84
|
+
generation — Tier 2b, when the brief is ornate) / real glass
|
|
85
|
+
(`npx genex image "<frosted panel>" --glass` — menus love frosted panels,
|
|
86
|
+
and the solved RGBA carries TRUE translucency over the moving backdrop;
|
|
87
|
+
`$genex-ai-image` documents the lane). The per-genre lean lives
|
|
85
88
|
in the style capsules (`$genex-threejs-game-ui`'s
|
|
86
89
|
references/style-capsules.md) — a lean, not a rule.
|
|
87
90
|
|
|
@@ -136,7 +139,7 @@ cycle. Two stacked `<video>` elements with the same src crossfade at the
|
|
|
136
139
|
cycle end; any seam disappears deterministically, no regeneration lottery.
|
|
137
140
|
|
|
138
141
|
**Phone tiers get the poster, not the videos** (`$genex-threejs-adaptive-quality`):
|
|
139
|
-
two preloading
|
|
142
|
+
two preloading HD decoders while the 3D scene boots is a spike at exactly the
|
|
140
143
|
moment phones get killed for memory. On a phone tier, show the key-art poster
|
|
141
144
|
image (a captured frame of the clip works) and skip `seamlessLoop` entirely —
|
|
142
145
|
or defer ONE non-preloading video until after the first gameplay frame:
|
|
@@ -171,7 +174,10 @@ export function seamlessLoop(holder: HTMLElement, url: string, fade = 0.6): void
|
|
|
171
174
|
|
|
172
175
|
Pause both videos when the menu phase hides (they're decode work), and
|
|
173
176
|
resume the front one when it returns. Watch ONE full cycle in the browser
|
|
174
|
-
before calling the menu done — that's the seam check
|
|
177
|
+
before calling the menu done — that's the seam check, and once the video is
|
|
178
|
+
wired it is the menu's capture in the milestone smoke pass: one full loop
|
|
179
|
+
cycle watched AS RENDERED. A metadata probe (ffprobe) can't see a seam, a
|
|
180
|
+
panel covering the video, or a video that never plays — only watching can.
|
|
175
181
|
|
|
176
182
|
**Work async — the menu must never block the game.** The still is
|
|
177
183
|
`--edit`-anchored to the concept mockup, so it's style-dependent: generate it
|
|
@@ -184,6 +190,17 @@ the moment the user approves the concept (not before), enqueue the video with
|
|
|
184
190
|
(R2) — permanent, public, CORS-open; you load them straight from the printed
|
|
185
191
|
URLs, nothing is downloaded or committed.
|
|
186
192
|
|
|
193
|
+
**When the generated art lands, the upgrade is a RE-COMPOSITION — not a
|
|
194
|
+
backdrop swap.** DELETE the placeholder panel/card the moment the still or
|
|
195
|
+
video goes in: the still was generated with negative space precisely so the
|
|
196
|
+
buttons and logo sit DIRECTLY on the key art. A placeholder card left
|
|
197
|
+
floating over the finished video is the single most common bad menu — a
|
|
198
|
+
near-opaque panel (plus its darkening layers) hides the art you just paid
|
|
199
|
+
for, and the menu reads as the plain CSS draft it started as. Allowed over
|
|
200
|
+
the art: a thin, low-opacity scrim strictly behind the button rail for
|
|
201
|
+
legibility, and at most ONE subtle full-screen grade layer. NEVER a
|
|
202
|
+
card/panel covering the art.
|
|
203
|
+
|
|
187
204
|
**Two failed video attempts = ship the still. Hard stop.** Video is the one
|
|
188
205
|
generation that fails server-side with real frequency (render timeouts), and
|
|
189
206
|
every attempt costs minutes of waiting. One retry is fair — shorten the clip
|
|
@@ -350,7 +367,11 @@ not):
|
|
|
350
367
|
`npx genex image "the word 'EMBERFALL' as an ornate engraved game logo, <style brief>" --transparent`
|
|
351
368
|
— the wordmark alone on a transparent background, in the brief's display
|
|
352
369
|
register, no extra text or scenery. Clean with `--bg-mode glyph` if letter
|
|
353
|
-
counters get eaten, trim with `npx genex ui trim`.
|
|
370
|
+
counters get eaten, trim with `npx genex ui trim`. Then VERIFY it is
|
|
371
|
+
actually transparent — view the trimmed PNG over a bright test background;
|
|
372
|
+
background removal sometimes leaves an opaque disc or plate behind the
|
|
373
|
+
mark, and that plate ships as a hole punched in your key art. If any
|
|
374
|
+
background remains, re-run the clean. Short names (one or two
|
|
354
375
|
words) come out best. Wire it as the menu title AND the loader mark; the
|
|
355
376
|
DOM keeps an accessible text fallback (`aria-label` or visually-hidden
|
|
356
377
|
text).
|
|
@@ -370,14 +391,12 @@ not):
|
|
|
370
391
|
re-anchored the look) re-opens this rule once — re-edit the still
|
|
371
392
|
(`--edit` against the new concept mockup) and re-run the video from the
|
|
372
393
|
new still. Agent-initiated polish never does.
|
|
373
|
-
- **The menu
|
|
374
|
-
frame-conditioned
|
|
375
|
-
1080p
|
|
376
|
-
|
|
377
|
-
|
|
378
|
-
|
|
379
|
-
re-triggering the 422 and losing the video entirely. (Plain text-to-video,
|
|
380
|
-
without `--frame`, still defaults to 1080p.)
|
|
394
|
+
- **The menu video renders 1080p by default — leave it alone.** Every video
|
|
395
|
+
path, the frame-conditioned (`--frame`) menu route included, defaults to
|
|
396
|
+
1080p: the menu clip is full-screen key art, and 720p stretched across a
|
|
397
|
+
desktop reads soft. Don't pass a resolution flag at all; `--resolution 720p`
|
|
398
|
+
exists only as a deliberate cost opt-down for clips that are genuinely
|
|
399
|
+
incidental — never the main menu.
|
|
381
400
|
- **Pause/victory/defeat variants reuse the same video — as GRADES.** Same
|
|
382
401
|
`<video>` element or URL, different emotion via CSS `filter` on the
|
|
383
402
|
background: pause = a plain dark overlay (`rgba(0,0,0,0.55)`); defeat =
|
|
@@ -407,11 +426,10 @@ not):
|
|
|
407
426
|
genuine state change; expect a loop seam.
|
|
408
427
|
- `--duration <sec>` (video) — 4, 6, or 8 for frame-conditioned clips;
|
|
409
428
|
default 8.
|
|
410
|
-
- `--resolution <720p|1080p>` (video) —
|
|
411
|
-
|
|
412
|
-
|
|
413
|
-
(
|
|
414
|
-
resolution parameter).
|
|
429
|
+
- `--resolution <720p|1080p>` (video) — every path defaults to **1080p**, the
|
|
430
|
+
`--frame` menu route included. `720p` is the deliberate cost opt-down
|
|
431
|
+
(~half the credits) for incidental clips — not for the menu. Loop clips
|
|
432
|
+
(`--loop`) ignore it (that model has no resolution parameter).
|
|
415
433
|
- `--aspect 16:9 --quality high` (image) — the right settings for a menu frame.
|
|
416
434
|
- `--no-wait` — enqueue and return immediately with the generation id; pick
|
|
417
435
|
the result up later with `npx genex wait <id>` (safe to re-run — it attaches
|
|
@@ -14,6 +14,14 @@ Turn a text prompt into a real, game-ready **GLB** and drop it into the project.
|
|
|
14
14
|
- **Use `$genex-threejs-procedural-geometry`** for parametric/abstract shapes,
|
|
15
15
|
terrain, or anything you want to generate in code (infinite variations, no files).
|
|
16
16
|
|
|
17
|
+
**The output is a STATIC, unrigged mesh — no skeleton, no animation clips.**
|
|
18
|
+
A "wolf" or "guard" from this command can be posed and moved as one object,
|
|
19
|
+
but its limbs cannot animate. Anything that must walk, attack, or die on
|
|
20
|
+
screen routes to the creature lane instead: `npx genex creature` for
|
|
21
|
+
biped-shaped bodies (real rig + library clips), or this command plus the
|
|
22
|
+
procedural-motion recipes for everything else — `$genex-threejs-creatures`
|
|
23
|
+
owns that split. The player's own character is `$genex-ai-character`.
|
|
24
|
+
|
|
17
25
|
## Run
|
|
18
26
|
|
|
19
27
|
```bash
|
|
@@ -0,0 +1,142 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: genex-ai-music
|
|
3
|
+
description: Generate a real instrumental background-music track (mp3) from a text prompt with `npx genex music`, then loop it quietly under the game in Three.js. Use for the one looping gameplay track every game with generated audio should have (reused on the menu at lower volume), plus optional boss or menu-specific tracks — always instrumental, always behind Music/SFX volume sliders in settings.
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# Genex AI · Music
|
|
7
|
+
|
|
8
|
+
Turn a prompt into a real **mp3** instrumental track and loop it under the game.
|
|
9
|
+
One track transforms how finished a game feels — a silent game with polished art
|
|
10
|
+
still reads as a tech demo.
|
|
11
|
+
|
|
12
|
+
## When to use this vs. sfx
|
|
13
|
+
|
|
14
|
+
- **Use `npx genex music`** for the continuous bed: the gameplay loop, a boss
|
|
15
|
+
theme, a menu theme. Tracks are instrumental by design (the API enforces it —
|
|
16
|
+
vocals under gameplay read as karaoke).
|
|
17
|
+
- **Use `$genex-ai-sfx`** for event sounds — impacts, pickups, UI ticks. The
|
|
18
|
+
0.5–22 s sfx lane cannot produce a music bed; don't try.
|
|
19
|
+
|
|
20
|
+
**The default plan is ONE track:** a ~90 s loopable gameplay track, reused on
|
|
21
|
+
the menu (same URL, lower volume). Propose extra tracks (boss/menu-specific) as
|
|
22
|
+
their own Assets-table rows only when the game's shape earns them.
|
|
23
|
+
|
|
24
|
+
## Run
|
|
25
|
+
|
|
26
|
+
```bash
|
|
27
|
+
npx genex music "<prompt>" --no-wait
|
|
28
|
+
npx genex music "dark orchestral hunt, low strings and taiko, steady menace, seamless loop, consistent energy, no intro, no outro" --duration 120 --no-wait
|
|
29
|
+
```
|
|
30
|
+
|
|
31
|
+
`--duration <sec>` (10–300) sets the length; the default **90 s** is the sweet
|
|
32
|
+
spot for a loop. Music takes a minute-plus — enqueue with `--no-wait` alongside
|
|
33
|
+
your other generations and pick it up with `npx genex wait <id>`. Prints:
|
|
34
|
+
|
|
35
|
+
```
|
|
36
|
+
https://assets.genex.technology/generations/<id>/audio-music
|
|
37
|
+
```
|
|
38
|
+
|
|
39
|
+
The mp3 lives in Genex storage (R2) and loads straight from that URL — you don't
|
|
40
|
+
download it and nothing is committed to your repo. The URL is permanent (local
|
|
41
|
+
dev, published game, and remixes alike).
|
|
42
|
+
|
|
43
|
+
## Prompt recipe for loopable game music
|
|
44
|
+
|
|
45
|
+
Mood + instrumentation + energy + loop wording:
|
|
46
|
+
|
|
47
|
+
- Name the game's emotional register and 2–3 instruments ("brooding synthwave,
|
|
48
|
+
analog pads and arpeggio", "cozy acoustic folk, fingerpicked guitar and soft
|
|
49
|
+
strings").
|
|
50
|
+
- Always end with: **"seamless loop, consistent energy, no intro, no outro"** —
|
|
51
|
+
there is no API loop flag; consistent energy at both ends is what makes
|
|
52
|
+
`setLoop(true)` inaudible. A track with a big intro pops every cycle.
|
|
53
|
+
- Don't ask for vocals — the lane forces instrumental regardless.
|
|
54
|
+
|
|
55
|
+
## Wire it in Three.js
|
|
56
|
+
|
|
57
|
+
Music is **non-positional** (`THREE.Audio`, not `PositionalAudio`), looped,
|
|
58
|
+
and QUIET — the default volume is **0.30**: it sits under the sfx, never over
|
|
59
|
+
them. Browsers block autoplay, so start it on the first user gesture:
|
|
60
|
+
|
|
61
|
+
```ts
|
|
62
|
+
import * as THREE from "three";
|
|
63
|
+
|
|
64
|
+
const listener = new THREE.AudioListener();
|
|
65
|
+
camera.add(listener); // ONE listener serves music and every sfx
|
|
66
|
+
|
|
67
|
+
// the URL `npx genex music` printed (R2 sends CORS headers, cross-origin works):
|
|
68
|
+
const MUSIC_URL = "https://assets.genex.technology/generations/<id>/audio-music";
|
|
69
|
+
const music = new THREE.Audio(listener);
|
|
70
|
+
new THREE.AudioLoader().loadAsync(MUSIC_URL).then((buffer) => {
|
|
71
|
+
music.setBuffer(buffer);
|
|
72
|
+
music.setLoop(true);
|
|
73
|
+
music.setVolume(0.30); // the default — under the sfx, never over them
|
|
74
|
+
// autoplay policy: start on the first gesture (the menu's PLAY click is ideal)
|
|
75
|
+
const start = (): void => {
|
|
76
|
+
if (!music.isPlaying) music.play();
|
|
77
|
+
};
|
|
78
|
+
window.addEventListener("click", start, { once: true });
|
|
79
|
+
window.addEventListener("keydown", start, { once: true });
|
|
80
|
+
});
|
|
81
|
+
|
|
82
|
+
// Suspend when the tab hides (decode work + politeness), resume on return:
|
|
83
|
+
document.addEventListener("visibilitychange", () => {
|
|
84
|
+
const ctx = listener.context;
|
|
85
|
+
if (document.hidden) void ctx.suspend();
|
|
86
|
+
else void ctx.resume();
|
|
87
|
+
});
|
|
88
|
+
```
|
|
89
|
+
|
|
90
|
+
**Menu reuse:** the same `music` object at lower volume (`setVolume(0.18)`)
|
|
91
|
+
during the menu phase, restored to the setting's value on PLAY — zero extra
|
|
92
|
+
generations. If the loop point is audible (the track came back with an
|
|
93
|
+
intro/outro despite the prompt), crossfade the tail into the head with a second
|
|
94
|
+
`THREE.Audio` on the same buffer rather than regenerating.
|
|
95
|
+
|
|
96
|
+
## Volume sliders are NOT optional
|
|
97
|
+
|
|
98
|
+
A game with generated audio and no volume control is a failure mode. The
|
|
99
|
+
settings screen gets **two sliders — Music and SFX** (music default 0.30, sfx
|
|
100
|
+
~0.7), persisted (localStorage or the save slot) and applied live:
|
|
101
|
+
`music.setVolume(v)` for the music bed; keep every sfx's volume routed through
|
|
102
|
+
one shared multiplier. `$genex-threejs-game-ui` carries this as a hard settings
|
|
103
|
+
rule — wire the sliders in the same milestone as the track.
|
|
104
|
+
|
|
105
|
+
## Publish checklist
|
|
106
|
+
|
|
107
|
+
- Load it from the **URL** the command printed — absolute and permanent, so it
|
|
108
|
+
resolves the same in local dev, the published game, and remixes. Nothing to
|
|
109
|
+
commit.
|
|
110
|
+
- The track must be audible in the shipped game AND adjustable in settings —
|
|
111
|
+
an unwired paid track is flagged at preview/publish (the wiring audit).
|
|
112
|
+
|
|
113
|
+
## Options
|
|
114
|
+
|
|
115
|
+
- `--duration <sec>` — track length 10–300; default 90 (the loop sweet spot).
|
|
116
|
+
Longer costs more; a 90–120 s loop outlasts most sessions.
|
|
117
|
+
- `--no-wait` — enqueue and return immediately with the generation id; pick it
|
|
118
|
+
up later with `npx genex wait <id>` (safe to re-run — it attaches to the SAME
|
|
119
|
+
generation). The default for music: build while it renders.
|
|
120
|
+
- `--api-url <url>` — override the API base (local dev).
|
|
121
|
+
|
|
122
|
+
## Troubleshooting
|
|
123
|
+
|
|
124
|
+
- **"Not authorized"** — run `npx genex init` first (in the project — it resolves this project's own CLI) (it writes your `GENEX_TOKEN`).
|
|
125
|
+
- **"Out of credits" (`insufficient_credits`)** — the account has no credits left
|
|
126
|
+
for this music generation. Tell the user the facts the CLI printed: their
|
|
127
|
+
balance, this generation's cost, and when their credits refill. Then continue
|
|
128
|
+
the build without a music bed (the game must work silent anyway) and mark the
|
|
129
|
+
spot with `// TODO(genex): regenerate when credits refill` so the real track
|
|
130
|
+
is one command away later. Do not stop the session over this.
|
|
131
|
+
- **"Email not verified" (`email_verification_required`)** — generation credits
|
|
132
|
+
unlock after the account's email is verified. Give the user the verify link
|
|
133
|
+
the CLI printed, wait for them to confirm, then re-run the command.
|
|
134
|
+
- **No sound** — the `AudioContext` is suspended until a user gesture; the
|
|
135
|
+
first `music.play()` must come from a click/keydown. Confirm the camera has
|
|
136
|
+
an `AudioListener`.
|
|
137
|
+
- **The loop "pops" every cycle** — the track has an intro/outro (the prompt
|
|
138
|
+
missed the loop wording). One regeneration with "seamless loop, consistent
|
|
139
|
+
energy, no intro, no outro" is fair; after that, crossfade tail-into-head
|
|
140
|
+
with a second `THREE.Audio` on the same buffer instead of re-rolling.
|
|
141
|
+
- **Music drowns the game** — 0.30 is the ceiling for a bed, not the floor;
|
|
142
|
+
drop it further under dialogue-heavy or ambience-led scenes.
|
|
@@ -10,13 +10,26 @@ also lights it (image-based lighting).
|
|
|
10
10
|
|
|
11
11
|
## When to use this vs. a procedural sky
|
|
12
12
|
|
|
13
|
-
- **Use `npx genex skybox`** for a specific,
|
|
14
|
-
|
|
15
|
-
|
|
13
|
+
- **Use `npx genex skybox`** for a specific, describable ENVIRONMENT — "golden
|
|
14
|
+
hour over misty mountains", "rolling thunderstorm, towering cumulonimbus",
|
|
15
|
+
"alien purple nebula", "hazy dusk sky over a distant sea horizon". You get a
|
|
16
|
+
real image.
|
|
16
17
|
- **Use `$genex-threejs-atmosphere-aerial-perspective`** (or
|
|
17
18
|
`$genex-threejs-volumetric-clouds`) for a fully procedural, animated,
|
|
18
19
|
time-of-day sky generated in shaders.
|
|
19
20
|
|
|
21
|
+
**A skybox is environment only — never content.** Buildings, trees, ruins,
|
|
22
|
+
towers, or any object painted into the sky render at infinite distance: they
|
|
23
|
+
never get closer as the player moves, they sit at the wrong parallax against
|
|
24
|
+
the real 3D world, and they read as broken the moment the camera strafes.
|
|
25
|
+
Structures belong in the scene as geometry (`npx genex model`); the sky
|
|
26
|
+
carries atmosphere — light, weather, clouds, haze, stars. The CLI enforces
|
|
27
|
+
this: every skybox prompt gets an environment-only suffix appended (you'll
|
|
28
|
+
see `↳ environment-only guard applied` and the full final prompt is stored
|
|
29
|
+
with the generation). If you genuinely need content baked into the sky —
|
|
30
|
+
e.g. a space station panorama for a scene with no world geometry — pass
|
|
31
|
+
`--raw` to send your prompt exactly as written.
|
|
32
|
+
|
|
20
33
|
## Run
|
|
21
34
|
|
|
22
35
|
```bash
|
|
@@ -81,8 +94,11 @@ scene.background = texture; // keep the raw texture for the visible sky
|
|
|
81
94
|
|
|
82
95
|
## Options
|
|
83
96
|
|
|
84
|
-
- `--
|
|
85
|
-
|
|
97
|
+
- `--raw` — send the prompt exactly as written, skipping the appended
|
|
98
|
+
environment-only guard (for the rare sky that must carry content).
|
|
99
|
+
- `--no-wait` — enqueue and return immediately with the generation id; pick
|
|
100
|
+
the result up later with `npx genex wait <id>` (safe to re-run — it attaches
|
|
101
|
+
to the SAME generation).
|
|
86
102
|
- `--api-url <url>` — override the API base (local dev).
|
|
87
103
|
|
|
88
104
|
## Troubleshooting
|
|
@@ -37,11 +37,13 @@ The clip lives in Genex storage (R2) and loads straight from that URL — you do
|
|
|
37
37
|
download it and nothing is committed to your repo. The URL is permanent (local dev,
|
|
38
38
|
published game, and remixes alike).
|
|
39
39
|
|
|
40
|
-
> **Cost & length:** the default is a **5-second,
|
|
41
|
-
>
|
|
42
|
-
> genuinely needs to be longer (a cutscene)
|
|
43
|
-
>
|
|
44
|
-
>
|
|
40
|
+
> **Cost & length:** the default is a **5-second, 1080p** clip — the right default
|
|
41
|
+
> for anything the player looks at directly. Only pass `--duration` when the content
|
|
42
|
+
> genuinely needs to be longer (a cutscene), and `--resolution 720p` only when the
|
|
43
|
+
> clip is genuinely incidental (a small in-world screen seen from a distance): it
|
|
44
|
+
> halves the cost, but 720p stretched across the whole screen reads soft. mp4 has
|
|
45
|
+
> **no alpha channel**, so a video is always a full rectangle (there are no
|
|
46
|
+
> transparent video decals).
|
|
45
47
|
|
|
46
48
|
## Play it in Three.js
|
|
47
49
|
|
|
@@ -80,8 +82,9 @@ window.addEventListener("keydown", () => {
|
|
|
80
82
|
One `VideoTexture` can feed **many** surfaces — a bank of monitors, N sprays. Cost is
|
|
81
83
|
per distinct `<video>` element (media decode + one GPU upload per frame), **not** per
|
|
82
84
|
surface, so sharing one element for every screen is nearly free (all play in sync).
|
|
83
|
-
|
|
84
|
-
|
|
85
|
+
Pause the element (`video.pause()`) when no video surface is visible; on phones,
|
|
86
|
+
prefer the clip's source still as a poster and start the video only where it plays
|
|
87
|
+
smoothly.
|
|
85
88
|
|
|
86
89
|
## Video decals
|
|
87
90
|
|
|
@@ -132,6 +135,9 @@ See `$genex-threejs-multiplayer` for the `shared` channel rules and the room API
|
|
|
132
135
|
- `--loop` — a seamless loop (for screens, ambient backdrops, video decals).
|
|
133
136
|
- `--duration <sec>` — clip length 1–15; default 5. Only raise it when the content
|
|
134
137
|
genuinely needs more — longer clips cost more and take longer.
|
|
138
|
+
- `--resolution 720p|1080p` — output resolution (default **1080p**). `--resolution
|
|
139
|
+
720p` halves the cost — use it only when quality genuinely doesn't matter (a small
|
|
140
|
+
far-away in-world screen). `--loop` clips ignore it (model default).
|
|
135
141
|
- `--frame <url>` — one generated image as BOTH first and last frame — the
|
|
136
142
|
seamless-loop mode (motion must return to its start).
|
|
137
143
|
- `--first-frame <url>` / `--last-frame <url>` — two-frame motion between two
|
|
@@ -0,0 +1,151 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: genex-ai-voice
|
|
3
|
+
description: Generate a short spoken voice line (mp3) from text with `npx genex voice` — NPC barks, narrator beats, tutorial VO, announcer calls — picking a voice from the curated cast (`--voice narrator|heroine|gruff|elder|robot|imp`) or any raw ElevenLabs voice id (`--voice-id`). Billed per character, hard-capped at 1000 chars per line. Voice lines are content — keep them short, subtitled, and skippable.
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# Genex AI · Voice
|
|
7
|
+
|
|
8
|
+
Turn a line of text into a real spoken **mp3** and play it in the game. A
|
|
9
|
+
guard that actually says "Halt! Who goes there?", a narrator that opens the
|
|
10
|
+
run, a tutorial voice over the first jump — one or two spoken lines move a
|
|
11
|
+
game from "asset flip" to "authored" faster than almost any other audio spend.
|
|
12
|
+
|
|
13
|
+
## When to use this vs. music / sfx
|
|
14
|
+
|
|
15
|
+
- **Use `npx genex voice`** for SPEECH: NPC barks, narrator lines, tutorial
|
|
16
|
+
VO, announcer calls, boss taunts. The text you pass IS what gets spoken —
|
|
17
|
+
write the line, not a description of it.
|
|
18
|
+
- **Use `$genex-ai-music`** for the continuous instrumental bed and
|
|
19
|
+
**`$genex-ai-sfx`** for event sounds — impacts, pickups, UI ticks. A
|
|
20
|
+
scream, grunt, or monster roar is an SFX (describe it), not a voice line.
|
|
21
|
+
|
|
22
|
+
**Voice lines are content, not chrome: keep them SHORT, show a SUBTITLE for
|
|
23
|
+
every line, and let the player skip or interrupt.** A game that talks over
|
|
24
|
+
itself un-skippably reads worse than a silent one.
|
|
25
|
+
|
|
26
|
+
## Run
|
|
27
|
+
|
|
28
|
+
```bash
|
|
29
|
+
npx genex voice "Halt! Who goes there?" --voice gruff
|
|
30
|
+
npx genex voice "Level up! New ability unlocked." --voice robot --no-wait
|
|
31
|
+
```
|
|
32
|
+
|
|
33
|
+
Prints:
|
|
34
|
+
|
|
35
|
+
```
|
|
36
|
+
https://assets.genex.technology/generations/<id>/audio-voice
|
|
37
|
+
```
|
|
38
|
+
|
|
39
|
+
The mp3 lives in Genex storage (R2) and loads straight from that URL — you
|
|
40
|
+
don't download it and nothing is committed to your repo. The URL is permanent
|
|
41
|
+
(local dev, published game, and remixes alike).
|
|
42
|
+
|
|
43
|
+
## The cast
|
|
44
|
+
|
|
45
|
+
Six curated archetypes — pick by gameplay role, not by auditioning ids:
|
|
46
|
+
|
|
47
|
+
| `--voice` | Reads as | Use for |
|
|
48
|
+
| --- | --- | --- |
|
|
49
|
+
| `narrator` (default) | warm storyteller | intros, quest text, tutorials |
|
|
50
|
+
| `heroine` | confident young female lead | player character, guide |
|
|
51
|
+
| `gruff` | fierce warrior | guards, bosses, drill sergeants |
|
|
52
|
+
| `elder` | wise old mentor | sages, shopkeepers, lore |
|
|
53
|
+
| `robot` | calm neutral synthetic | AI companions, announcers, computers |
|
|
54
|
+
| `imp` | husky trickster | goblins, sidekicks, comic relief |
|
|
55
|
+
|
|
56
|
+
**Escape hatch:** `--voice-id <ElevenLabsVoiceId>` uses any raw ElevenLabs
|
|
57
|
+
voice id (overrides `--voice`) — for when the user has a specific voice in
|
|
58
|
+
mind. The cast covers the normal cases; don't browse ids speculatively.
|
|
59
|
+
|
|
60
|
+
Pick ONE voice per character and stay with it — a guard who changes voice
|
|
61
|
+
between barks breaks the character. The model is multilingual: text in the
|
|
62
|
+
game's language comes back spoken in that language.
|
|
63
|
+
|
|
64
|
+
## Cost honesty
|
|
65
|
+
|
|
66
|
+
Voice is billed **per character of the submitted text**, hard-capped at
|
|
67
|
+
**1000 characters** per line (longer text is clamped, and the clamp is what
|
|
68
|
+
bills). A one-sentence bark costs a fraction of a credit-priced generation —
|
|
69
|
+
but 30 speculative barks are 30 paid calls. Write the script first, generate
|
|
70
|
+
once per line, and reuse lines (the same "Halt!" serves every guard).
|
|
71
|
+
|
|
72
|
+
## Wire it in Three.js
|
|
73
|
+
|
|
74
|
+
Same machinery as sfx — one `AudioListener` on the camera serves everything.
|
|
75
|
+
Non-positional (`THREE.Audio`) for narrator/announcer voices; positional
|
|
76
|
+
(`THREE.PositionalAudio`) when a CHARACTER in the world speaks:
|
|
77
|
+
|
|
78
|
+
```ts
|
|
79
|
+
import * as THREE from "three";
|
|
80
|
+
|
|
81
|
+
const listener = new THREE.AudioListener();
|
|
82
|
+
camera.add(listener); // ONE listener serves music, sfx, and voice
|
|
83
|
+
|
|
84
|
+
const VOICE_URL = "https://assets.genex.technology/generations/<id>/audio-voice";
|
|
85
|
+
const line = new THREE.Audio(listener); // PositionalAudio(listener) at the speaker's mesh for world speech
|
|
86
|
+
const subtitles = document.getElementById("subtitles")!;
|
|
87
|
+
|
|
88
|
+
new THREE.AudioLoader().loadAsync(VOICE_URL).then((buffer) => {
|
|
89
|
+
line.setBuffer(buffer);
|
|
90
|
+
line.setVolume(0.9);
|
|
91
|
+
});
|
|
92
|
+
|
|
93
|
+
function sayLine(text: string): void {
|
|
94
|
+
if (line.isPlaying) line.stop(); // a new line interrupts the old — never overlap voices
|
|
95
|
+
line.play();
|
|
96
|
+
subtitles.textContent = text; // SUBTITLE every voice line, always
|
|
97
|
+
subtitles.classList.add("visible");
|
|
98
|
+
line.onEnded = () => subtitles.classList.remove("visible");
|
|
99
|
+
}
|
|
100
|
+
```
|
|
101
|
+
|
|
102
|
+
- **Subtitles are mandatory** — the same text you generated from, shown while
|
|
103
|
+
the line plays. Muted players and streamers still get the content.
|
|
104
|
+
- **Route voice volume through the SFX slider** (or its own Voice slider in a
|
|
105
|
+
dialogue-heavy game) — `$genex-threejs-game-ui`'s settings rule; an
|
|
106
|
+
uncontrollable voice is worse than none.
|
|
107
|
+
- **Duck the music** while a line plays (`music.setVolume(0.15)`, restore on
|
|
108
|
+
`onEnded`) so speech never fights the bed.
|
|
109
|
+
- Browsers block autoplay: no voice before the first user gesture (the menu's
|
|
110
|
+
PLAY click).
|
|
111
|
+
|
|
112
|
+
## Publish checklist
|
|
113
|
+
|
|
114
|
+
- Load from the **URL** the command printed — permanent, nothing to commit.
|
|
115
|
+
- Every generated line is WIRED and subtitled — an unwired paid line is
|
|
116
|
+
flagged at preview/publish (the wiring audit).
|
|
117
|
+
- Lines are skippable/interruptible and respect the volume sliders.
|
|
118
|
+
|
|
119
|
+
## Options
|
|
120
|
+
|
|
121
|
+
- `--voice <cast>` — `narrator|heroine|gruff|elder|robot|imp` (default
|
|
122
|
+
`narrator`).
|
|
123
|
+
- `--voice-id <id>` — any raw ElevenLabs voice id (overrides `--voice`).
|
|
124
|
+
- `--no-wait` — enqueue and return immediately with the generation id; pick
|
|
125
|
+
it up later with `npx genex wait <id>` (safe to re-run — it attaches to the
|
|
126
|
+
SAME generation).
|
|
127
|
+
- `--api-url <url>` — override the API base (local dev).
|
|
128
|
+
|
|
129
|
+
## Troubleshooting
|
|
130
|
+
|
|
131
|
+
- **"Not authorized"** — run `npx genex init` first (in the project — it resolves this project's own CLI) (it writes your `GENEX_TOKEN`).
|
|
132
|
+
- **"Out of credits" (`insufficient_credits`)** — the account has no credits
|
|
133
|
+
left for this voice generation. Tell the user the facts the CLI printed:
|
|
134
|
+
their balance, this generation's cost, and when their credits refill. Then
|
|
135
|
+
continue the build with the subtitle alone (the game must read fine silent
|
|
136
|
+
anyway) and mark the spot with `// TODO(genex): regenerate when credits refill`
|
|
137
|
+
so the real line is one command away later. Do not stop the session
|
|
138
|
+
over this.
|
|
139
|
+
- **"Email not verified" (`email_verification_required`)** — generation
|
|
140
|
+
credits unlock after the account's email is verified. Give the user the
|
|
141
|
+
verify link the CLI printed, wait for them to confirm, then re-run the
|
|
142
|
+
command.
|
|
143
|
+
- **The line got cut off** — text past 1000 characters is clamped (the CLI
|
|
144
|
+
warns). Split long copy into separate lines — narration beats work better
|
|
145
|
+
as short lines anyway.
|
|
146
|
+
- **Wrong language/accent** — the model speaks the language of the TEXT; for
|
|
147
|
+
an accent, pick a different cast voice (or a specific `--voice-id`) rather
|
|
148
|
+
than describing the accent in the text.
|
|
149
|
+
- **No sound** — the `AudioContext` is suspended until a user gesture; confirm
|
|
150
|
+
the camera has an `AudioListener` and the first play comes from a
|
|
151
|
+
click/keydown.
|