oda-engine 0.0.0-stage → 0.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +151 -2
- package/bin/nowhere.mjs +352 -0
- package/lib/account.mjs +57 -0
- package/lib/agentModules.mjs +57 -0
- package/lib/devServer.mjs +351 -0
- package/lib/headless.mjs +231 -0
- package/lib/headlessPreview.mjs +212 -0
- package/lib/keys.mjs +517 -0
- package/lib/make.mjs +855 -0
- package/lib/mcp.mjs +537 -0
- package/lib/open.mjs +22 -0
- package/lib/preview.mjs +87 -0
- package/lib/previewRpc.mjs +134 -0
- package/lib/project.mjs +359 -0
- package/lib/publishFlow.mjs +90 -0
- package/lib/resources.mjs +50 -0
- package/lib/rules.mjs +60 -0
- package/lib/scaffold.mjs +42 -0
- package/lib/setup.mjs +59 -0
- package/lib/tools.mjs +668 -0
- package/package.json +51 -4
- package/template/game.config.json +9 -0
- package/template/scene.json +4 -0
- package/template/src/main.ts +11 -0
- package/vendor/DOCS.md +447 -0
- package/vendor/agent/appRun.mjs +520 -0
- package/vendor/agent/assets/brief.mjs +242 -0
- package/vendor/agent/assets/collider.mjs +531 -0
- package/vendor/agent/assets/environments/index.mjs +12 -0
- package/vendor/agent/assets/environments/worldlabs.mjs +161 -0
- package/vendor/agent/assets/fetch.mjs +168 -0
- package/vendor/agent/assets/footprint.mjs +13 -0
- package/vendor/agent/assets/gltf.mjs +547 -0
- package/vendor/agent/assets/hdr.mjs +269 -0
- package/vendor/agent/assets/imageOps.mjs +171 -0
- package/vendor/agent/assets/images/codex.mjs +307 -0
- package/vendor/agent/assets/images/fal.mjs +75 -0
- package/vendor/agent/assets/images/gemini.mjs +84 -0
- package/vendor/agent/assets/images/index.mjs +25 -0
- package/vendor/agent/assets/images/openai.mjs +75 -0
- package/vendor/agent/assets/images/stability.mjs +71 -0
- package/vendor/agent/assets/images/views.mjs +31 -0
- package/vendor/agent/assets/import.mjs +1352 -0
- package/vendor/agent/assets/inspect.mjs +465 -0
- package/vendor/agent/assets/jobs.mjs +196 -0
- package/vendor/agent/assets/ktx2.mjs +290 -0
- package/vendor/agent/assets/ledger.mjs +466 -0
- package/vendor/agent/assets/net.mjs +254 -0
- package/vendor/agent/assets/prices.json +39 -0
- package/vendor/agent/assets/providers/fal.mjs +115 -0
- package/vendor/agent/assets/providers/index.mjs +22 -0
- package/vendor/agent/assets/providers/meshy.mjs +119 -0
- package/vendor/agent/assets/providers/tripo.mjs +188 -0
- package/vendor/agent/assets/sources/ambientcg.mjs +129 -0
- package/vendor/agent/assets/sources/index.mjs +23 -0
- package/vendor/agent/assets/sources/polyhaven.mjs +237 -0
- package/vendor/agent/assets/sources/polypizza.mjs +121 -0
- package/vendor/agent/assets/splat.mjs +631 -0
- package/vendor/agent/assets/targets.mjs +122 -0
- package/vendor/agent/assets/tile.mjs +261 -0
- package/vendor/agent/assets/unzip.mjs +180 -0
- package/vendor/agent/blender/cleanup.py +360 -0
- package/vendor/agent/blender/convert.py +167 -0
- package/vendor/agent/blender/find.mjs +192 -0
- package/vendor/agent/blender/nw_export.py +104 -0
- package/vendor/agent/blender/nw_rubric.py +426 -0
- package/vendor/agent/blender/run.mjs +193 -0
- package/vendor/agent/blender/runner.py +252 -0
- package/vendor/agent/blender/sheet.py +743 -0
- package/vendor/agent/depth.mjs +134 -0
- package/vendor/agent/director/board.mjs +451 -0
- package/vendor/agent/director/contracts.mjs +663 -0
- package/vendor/agent/director/director.mjs +2365 -0
- package/vendor/agent/director/estimate.mjs +200 -0
- package/vendor/agent/director/gate.mjs +802 -0
- package/vendor/agent/director/limits.json +16 -0
- package/vendor/agent/director/methods.mjs +457 -0
- package/vendor/agent/director/policy.json +33 -0
- package/vendor/agent/director/prompts.mjs +244 -0
- package/vendor/agent/director/reading.mjs +341 -0
- package/vendor/agent/director/router.mjs +184 -0
- package/vendor/agent/director/seams.mjs +243 -0
- package/vendor/agent/director/session.mjs +643 -0
- package/vendor/agent/director/store.mjs +119 -0
- package/vendor/agent/director/taskGate.mjs +110 -0
- package/vendor/agent/director/teamLedger.mjs +118 -0
- package/vendor/agent/director/triage.mjs +63 -0
- package/vendor/agent/downloads.cjs +363 -0
- package/vendor/agent/flow.mjs +78 -0
- package/vendor/agent/guard.mjs +613 -0
- package/vendor/agent/host.mjs +1736 -0
- package/vendor/agent/language.mjs +58 -0
- package/vendor/agent/ledger.mjs +360 -0
- package/vendor/agent/locale.cjs +186 -0
- package/vendor/agent/loop/capacity.mjs +74 -0
- package/vendor/agent/loop/compose.mjs +127 -0
- package/vendor/agent/loop/gate.mjs +205 -0
- package/vendor/agent/loop/judge.mjs +189 -0
- package/vendor/agent/loop/levels.mjs +273 -0
- package/vendor/agent/loop/state.mjs +358 -0
- package/vendor/agent/notes.cjs +278 -0
- package/vendor/agent/pace.mjs +258 -0
- package/vendor/agent/panels.mjs +431 -0
- package/vendor/agent/prompt.mjs +342 -0
- package/vendor/agent/reference/board.mjs +410 -0
- package/vendor/agent/reference/compose.mjs +364 -0
- package/vendor/agent/reference/decode.mjs +56 -0
- package/vendor/agent/reference/facts.mjs +1874 -0
- package/vendor/agent/reference/png.mjs +107 -0
- package/vendor/agent/reference/records.mjs +63 -0
- package/vendor/agent/reference/web.mjs +252 -0
- package/vendor/agent/roles/artist.mjs +69 -0
- package/vendor/agent/roles/builder.mjs +79 -0
- package/vendor/agent/roles/checks.mjs +223 -0
- package/vendor/agent/roles/common.mjs +188 -0
- package/vendor/agent/roles/critic.mjs +171 -0
- package/vendor/agent/roles/index.mjs +39 -0
- package/vendor/agent/roles/modeller.mjs +142 -0
- package/vendor/agent/roles/qa.mjs +58 -0
- package/vendor/agent/roles/rubrics.mjs +79 -0
- package/vendor/agent/roles/sound.mjs +69 -0
- package/vendor/agent/rounds.cjs +333 -0
- package/vendor/agent/shared.mjs +284 -0
- package/vendor/agent/stopGate.mjs +574 -0
- package/vendor/agent/subagents.mjs +129 -0
- package/vendor/agent/tools/assets.mjs +499 -0
- package/vendor/agent/tools/blender.mjs +754 -0
- package/vendor/agent/tools/cards.mjs +479 -0
- package/vendor/agent/tools/choices.mjs +479 -0
- package/vendor/agent/tools/compose.mjs +74 -0
- package/vendor/agent/tools/environment.mjs +488 -0
- package/vendor/agent/tools/eyes.mjs +733 -0
- package/vendor/agent/tools/generate.mjs +338 -0
- package/vendor/agent/tools/guide.mjs +46 -0
- package/vendor/agent/tools/harness.mjs +193 -0
- package/vendor/agent/tools/image.mjs +643 -0
- package/vendor/agent/tools/index.mjs +162 -0
- package/vendor/agent/tools/look.mjs +578 -0
- package/vendor/agent/tools/loop.mjs +858 -0
- package/vendor/agent/tools/others.mjs +123 -0
- package/vendor/agent/tools/pictures.mjs +535 -0
- package/vendor/agent/tools/reference.mjs +366 -0
- package/vendor/agent/transcripts.mjs +118 -0
- package/vendor/agent-loop.mjs +602 -0
- package/vendor/agent-plugin/.claude-plugin/plugin.json +5 -0
- package/vendor/agent-plugin/skills/art-direction/SKILL.md +82 -0
- package/vendor/agent-plugin/skills/asset-legality/SKILL.md +101 -0
- package/vendor/agent-plugin/skills/asset-sources/SKILL.md +111 -0
- package/vendor/agent-plugin/skills/blender-modeling/SKILL.md +97 -0
- package/vendor/agent-plugin/skills/look-dev/SKILL.md +104 -0
- package/vendor/agent-plugin/skills/playtesting/SKILL.md +74 -0
- package/vendor/agent-plugin/skills/reference-loop/SKILL.md +102 -0
- package/vendor/agent-plugin/skills/sound-design/SKILL.md +60 -0
- package/vendor/agent-plugin/skills/style/SKILL.md +35 -0
- package/vendor/agent-plugin/skills/tsl-r185/SKILL.md +107 -0
- package/vendor/auth.cjs +570 -0
- package/vendor/build.cjs +953 -0
- package/vendor/catalog.json +455 -0
- package/vendor/connectors.cjs +688 -0
- package/vendor/eyes.mjs +733 -0
- package/vendor/gameExports.json +1390 -0
- package/vendor/git.cjs +222 -0
- package/vendor/models.json +57 -0
- package/vendor/publish.cjs +1018 -0
- package/vendor/refBoard.cjs +301 -0
- package/vendor/schema/dist/assets.d.ts +335 -0
- package/vendor/schema/dist/assets.js +439 -0
- package/vendor/schema/dist/brief.d.ts +357 -0
- package/vendor/schema/dist/brief.js +302 -0
- package/vendor/schema/dist/common.d.ts +9 -0
- package/vendor/schema/dist/common.js +13 -0
- package/vendor/schema/dist/config.d.ts +406 -0
- package/vendor/schema/dist/config.js +355 -0
- package/vendor/schema/dist/contract.d.ts +599 -0
- package/vendor/schema/dist/contract.js +1146 -0
- package/vendor/schema/dist/index.d.ts +14 -0
- package/vendor/schema/dist/index.js +21 -0
- package/vendor/schema/dist/look.d.ts +463 -0
- package/vendor/schema/dist/look.js +681 -0
- package/vendor/schema/dist/lookStats.d.ts +230 -0
- package/vendor/schema/dist/lookStats.js +768 -0
- package/vendor/schema/dist/pilot.d.ts +32 -0
- package/vendor/schema/dist/pilot.js +44 -0
- package/vendor/schema/dist/publish.d.ts +214 -0
- package/vendor/schema/dist/publish.js +291 -0
- package/vendor/schema/dist/scene.d.ts +75 -0
- package/vendor/schema/dist/scene.js +120 -0
- package/vendor/schema/dist/skyLight.d.ts +141 -0
- package/vendor/schema/dist/skyLight.js +436 -0
- package/vendor/schema/dist/style.d.ts +131 -0
- package/vendor/schema/dist/style.js +155 -0
- package/vendor/schema/package.json +1 -0
- package/vendor/selectionPrompt.mjs +51 -0
- package/vendor/useTools.mjs +195 -0
- package/vendor/userMessage.mjs +26 -0
- package/web-dist/assets/GaussianSplatUtils-CB77cP9U.js +1 -0
- package/web-dist/assets/KTX2Loader-6OIWUF7s.js +2 -0
- package/web-dist/assets/SPZLoader-zYj0Alfv.js +1 -0
- package/web-dist/assets/gaussianSplat-CJTI1sEh.js +1 -0
- package/web-dist/assets/index-CcjyyQco.css +1 -0
- package/web-dist/assets/index-DMyJcoZJ.js +693 -0
- package/web-dist/assets/meshopt_decoder.module-BvwdjiD3.js +1 -0
- package/web-dist/assets/zstddec.module-CfFeQW7d.js +1 -0
- package/web-dist/index.html +34 -0
|
@@ -0,0 +1,102 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: reference-loop
|
|
3
|
+
description: The loop a new space or a large change runs: the kickoff card, the target pictures, the brief, rounds judged against the targets by a judge with no context, and the approval card. Load before the first step of a space.
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# The reference loop
|
|
7
|
+
|
|
8
|
+
A space is built against pictures of what it should look like, judged by someone who did not build it, for as long as the creator gave it. The bar is high on purpose: a first pass that "looks fine" scores 6. It is judged as the player draws it in real time, never as a video or an offline render, and every file still fits a planet (100 MB of files, 1 MB of code, 50 MB a file, the splat limits, KTX2, meshopt, the phone tiers).
|
|
9
|
+
|
|
10
|
+
Every step starts with one line to yourself: the bar (brief.json `bar.judge`), the time gone and the time left (its `budget.timeFloorMin` and `budget.timeCapMin`), the round you are on. Under a quality step there is no floor: the time cap is the time you have. At the bar after the least rounds the approval card may be shown; while a round still raises the score and time and rounds are left, run one more first.
|
|
11
|
+
|
|
12
|
+
## 1. The kickoff
|
|
13
|
+
|
|
14
|
+
`nw_ask_kickoff` comes before every other tool, once a space (a second one is refused; a plain cost card before it is too). It asks, in the screen language (the studio context names it):
|
|
15
|
+
- 0 to 4 questions whose answers change the space: for a place, its size, its hour, what it is for; for a game, the goal, how a visitor wins and loses, and how long one play should last. Options where you can name them, `free: true` where a short answer is better. Ask everything here: nothing after this card waits on the creator.
|
|
16
|
+
- how the environment is made: built in 3D (meshes and materials you make), a file of the creator's own (a model, or a splat with its collider), or generated with Marble (paid; never chosen for them, and its own cost card comes first).
|
|
17
|
+
- the speed level, each with its time and cost on this model:
|
|
18
|
+
|
|
19
|
+
| level | floor | cap | rounds | bar |
|
|
20
|
+
|---|---|---|---|---|
|
|
21
|
+
| quick | 0 min | 30 min | 2 | 8.0 |
|
|
22
|
+
| standard | 60 min | 3 h | 8 | 9.0 |
|
|
23
|
+
| deep | 4 h | 12 h | 24 | 9.2 |
|
|
24
|
+
|
|
25
|
+
Or the quality step the creator sent the request at (the composer's gauge, `oda make --quality`): the card then offers no level, sending approved the step's estimate, and a build runs at the step's model and effort.
|
|
26
|
+
|
|
27
|
+
| step | model | effort | bar | least rounds | round cap | minutes |
|
|
28
|
+
|---|---|---|---|---|---|---|
|
|
29
|
+
| 1 Quick | Sonnet 5.5 | medium | 8.0 | 1 | 2 | 30 |
|
|
30
|
+
| 2 Standard | Sonnet 5.5 | xhigh | 8.5 | 2 | 5 | 90 |
|
|
31
|
+
| 3 Careful | Opus 5.5 | xhigh | 9.0 | 3 | 10 | 240 |
|
|
32
|
+
| 4 High | Opus 5.5 | max | 9.1 | 5 | 16 | 360 |
|
|
33
|
+
| 5 Max | Fable 5.2 | max | 9.2 | 8 | 24 | 720 |
|
|
34
|
+
|
|
35
|
+
These are process values; they never say what a space looks like. The creator's answer approves the cost of the level picked. With no answer in 10 minutes it goes on at standard, built in 3D, and nothing is approved: ask one cost card, and if that goes unanswered too, change nothing and say the space waits for their approval. Marble goes only when the creator chose it here, and only on a cost card of its own after this one. The answer's level, times and bar go into brief.json (`budget.level`, `timeFloorMin`, `timeCapMin`, `roundCap`, `usdEstimate`, and `bar`); the tool writes them, or says what to write.
|
|
36
|
+
|
|
37
|
+
After the kickoff, any choice (which floor, which sky) is yours: pick, and say what you picked.
|
|
38
|
+
|
|
39
|
+
## 2. The style
|
|
40
|
+
|
|
41
|
+
style.json before any picture (nowhere:style has it in full). A photograph is one style, never the default. From the creator's pictures or a named style, analyse how they are drawn; otherwise research artists and styles, see the creator's other spaces (`nw_others`), draw three unlike directions (`image_generate` kind `"direction"`) and ask on the style card (`nw_ask_pictures` purpose `"style"`, allowed after the kickoff). Its rules govern shapes, materials and light; its look gives the drawing techniques. The targets are drawn from it, and the judge scores the drawing against them.
|
|
42
|
+
|
|
43
|
+
## 3. The targets
|
|
44
|
+
|
|
45
|
+
The targets are what the finished space looks like from each hero view. They come before the brief, because the brief is measured from them.
|
|
46
|
+
|
|
47
|
+
- Four or more photos from the creator: they are the targets. Find each one's camera with `nw_reference` (its horizon and camera guess) and put the hero view there.
|
|
48
|
+
- Otherwise plan 4 to 6 hero views first: H1 where the visitor arrives, then the views the request names (the fountain from the arcade, the lane from the far gate), each with a position at eye height unless the view says otherwise, a point it looks at and a fov. Then `image_generate` kind `"target"` with `heroes` (each id, what it shows, its camera) and one `prompt` for the place itself, written from the request and the kickoff's answers: what the place is, the hour, the weather, its materials and colours. The tool adds its own rules and leaves the slop list out. Its price is on the kickoff card.
|
|
49
|
+
- Look at every picture. One with lettering, a fisheye, a render's sheen or anything off the slop list is drawn again before the creator sees it.
|
|
50
|
+
- `nw_ask_targets` shows them with their palettes: the creator keeps, drops, writes notes, and accepts, sends them back to be drawn again (three times at most), or leaves it to you. A redraw keeps the cameras and answers the notes. With no answer in 10 minutes all of them stand.
|
|
51
|
+
- The kept ones join the reference board and brief.json's references as R ids with origin `target` (made) or `attached` (the creator's). Measure the board with `nw_reference` board true.
|
|
52
|
+
- A target is only a picture `image_generate` drew (listed in its folder's source.json) or a photo the creator attached. A still of the space, a capture or a web reference is refused (say it in the prompt): the rounds would be judged against themselves. `decide` records targets without a card only after the creator left them to you.
|
|
53
|
+
- No generator and fewer than four photos: `nw_ask_targets` with `none` and why (refused while a key or Codex is there). Rounds then take stills and look_compare's numbers but no judge scores them, so the bar cannot be reached; your reply and the approval card say so plainly.
|
|
54
|
+
|
|
55
|
+
## 4. The brief
|
|
56
|
+
|
|
57
|
+
brief.json as nowhere:art-direction has it, with the loop's parts: the targets as references; `heroViews` one per target, each `reference` citing its target and its `position`, `lookAt` and `fov` exactly the camera the target was drawn from (image_generate's record gives it), so every round's still matches its target's framing; `slop` for what this space also avoids; `budget` and `bar` from the kickoff. The palette, light and materials come from the targets' numbers. Nothing comes from a stock look.
|
|
58
|
+
|
|
59
|
+
## 5. Code first
|
|
60
|
+
|
|
61
|
+
Build the environment in code before reaching for files: terrain from a height function, buildings and walls from their plans, rocks, trees and plants grown and scattered with seeded noise and blue-noise spacing, water, the sky (a physical sky from the brief's sun), and surfaces as TSL node materials with their own detail at every distance. The SDK's procedural helpers (nw_docs) make the noise, the scatter, the height-field mesh, instancing with distance LODs and impostors for far trees and rocks; their values and shapes are this space's, with no default colour or material. Code costs no file weight, which leaves the 100 MB for what code cannot do.
|
|
62
|
+
|
|
63
|
+
Then only what code cannot bring to the target: a scanned surface or HDRI from Poly Haven or ambientCG, a model from a CC0 library or scripted in Blender, a generated model for a prop alone (delit), and sounds from a free library. Everything from outside comes in through asset_import.
|
|
64
|
+
|
|
65
|
+
Code spends frame time instead: keep each tier within its line (below), with instancing, LODs and fewer draw calls.
|
|
66
|
+
|
|
67
|
+
## 6. Rounds
|
|
68
|
+
|
|
69
|
+
A round, every time:
|
|
70
|
+
1. Make the changes you planned from the last round's top three gaps (and the creator's notes).
|
|
71
|
+
2. `nw_check` passes.
|
|
72
|
+
3. `nw_round` with one line on what changed. It checks again, waits for the space's files, takes a still at each hero view from its target's camera beside the target (target | now), runs look_compare and has the judge score it.
|
|
73
|
+
4. Read its answer: the score per view and overall, the history, the gaps (where, what, how much). The round's folder is `.nowhere/rounds/<n>/`; the score goes into brief.json's `scoreHistory`; the round shows in the chat without stopping the work.
|
|
74
|
+
5. Plan the next round on the top three gaps: the biggest visible difference first, a material, a light, a shape, the far distance.
|
|
75
|
+
|
|
76
|
+
A round changes the space; looking again at an unchanged one is not a round. Only a round that scored every hero view with a target counts toward the bar (`heroes` narrows one to plan, never to finish). `nw_round` refuses a brief whose hero views do not cite their targets from the targets' cameras. The round cap and the time cap come before the bar: past the time cap one last round runs; when either is reached, stop at the approval card and say plainly where it stands.
|
|
77
|
+
|
|
78
|
+
The creator may write while you work: their message arrives in your next tool result, sometimes with a picture they drew on (lines, circles, arrows, words, with the hero view when it is a round's still). The picture is kept under .nowhere/notes/: Read it to see the marks. Take it into the next round as its first gap.
|
|
79
|
+
|
|
80
|
+
## 7. The judge
|
|
81
|
+
|
|
82
|
+
It is a run of its own: it sees the targets, the stills from the same cameras, the rubric and the slop list, and nothing of this conversation, the brief or your intent. It scores each view 0 to 10 on composition and scale, materials, colour (greyed out or oversaturated), light and shadow, far detail, repetition, real-time artefacts (stair-stepped edges, flicker, an empty sky) and the slop list. With an OpenAI key connected a second judge scores too, and the lower score stands. Do not argue with a score: change the space.
|
|
83
|
+
|
|
84
|
+
## 8. The stop gate
|
|
85
|
+
|
|
86
|
+
The studio will not let a space end until: the kickoff was shown; the targets are settled; a round was judged on every hero view of the space as it is now; the time floor (if any) has gone by and the last score is at the bar after the least rounds, or the time cap or the round cap was reached (then the reply says plainly "below the bar, X of Y"); every tier's frame time, measured on the space as it is now, is within its line (high p95 16.7 ms headless, low 33 ms at the phone's pixel count; nw_check measures it at the spawn and every hero view); what it carries fits a planet; and the approval card was shown after the last round. The usual checks stand too: nw_check and a look after the last edit, brief.json valid, lookWarnings empty, no debug code. A run that stops again and again without judging a round is let go in the end; the reply then says plainly where it ended.
|
|
87
|
+
|
|
88
|
+
## 9. The approval
|
|
89
|
+
|
|
90
|
+
`nw_ask_approval` after the last round: the target | now pictures, the scores round by round against the bar, the weight of the files and the code, and the frame time by tier. Only the creator's Publish publishes (the studio's own flow, pre-flight and all); you never publish otherwise. "One more round" comes with a note: it is the first gap of that round, within the caps, then the card again.
|
|
91
|
+
|
|
92
|
+
## 10. Pace
|
|
93
|
+
|
|
94
|
+
On the creator's Claude plan the studio keeps to its five-hour window: when the work would use it up before it resets, the studio spaces your steps and says so. Take fewer, larger steps and fewer helpers then; never stop to wait.
|
|
95
|
+
|
|
96
|
+
## The slop list
|
|
97
|
+
|
|
98
|
+
Never in a space, whatever the request: decorative lettering (eyebrow labels, monospace capitals, a stray serif, 01/02/03 counters used as ornament); decoration for its own sake (glowing orbs, shockwave rings, an all-beige wash); colour gone grey, or oversaturated; the same thing copy-pasted in a visible repeat; bare single-colour surfaces; an empty sky; mush in the distance. In words: no em dashes. brief.json's `slop` adds to it. A game's score, timer and rule signs that the request needs are not slop: draw them inside the world in the space's style.
|
|
99
|
+
|
|
100
|
+
## When the creator asks what you made
|
|
101
|
+
|
|
102
|
+
Say it area by area, a line each (the gate, the arcade, the fountain, the far lane): what is there, what it is made of, where it is still weak. Short, so the creator finds the words for what to change.
|
|
@@ -0,0 +1,60 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: sound-design
|
|
3
|
+
description: Making an Oda game's sounds by Web Audio synthesis from the brief's sound mood, sources and events: the module with createAudio and renderOffline that the audio check measures, the Web Audio facts (envelopes, oscillators, noise, filters, panning, loops), and how each parameter is decided from the brief. Load before writing any sound.
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# Sound design by synthesis
|
|
7
|
+
|
|
8
|
+
Every sound of the game is made in code from the brief: `sound.mood`, `sound.sources` and `sound.events` (each with an id, when it plays and whether it loops). Synthesis is small, has no licence to track and can be made for exactly this game. There is no stock sound set to start from, and no table from moods to settings: each sound is designed for this brief, and two games with different briefs never sound alike.
|
|
9
|
+
|
|
10
|
+
## The module
|
|
11
|
+
|
|
12
|
+
One file, `src/audio/sound.ts`, that imports nothing but types, so it bundles on its own (the check bundles it alone):
|
|
13
|
+
|
|
14
|
+
```ts
|
|
15
|
+
export interface GameAudio {
|
|
16
|
+
readonly events: readonly string[]
|
|
17
|
+
play(event: string, opts?: { at?: [number, number, number]; gain?: number }): void
|
|
18
|
+
update(dt: number, listener?: { position: [number, number, number]; forward: [number, number, number] }): void
|
|
19
|
+
dispose(): void
|
|
20
|
+
}
|
|
21
|
+
export function createAudio(ctx: { input: { pressed(code: string): boolean } }): GameAudio
|
|
22
|
+
export function renderOffline(event: string, sampleRate?: number): Promise<Float32Array>
|
|
23
|
+
```
|
|
24
|
+
|
|
25
|
+
- One `AudioContext`, made lazily on the first `play` or `update`. Browsers start audio only after input: in `update`, the first time `ctx.input.pressed(...)` is true for any key, call `context.resume()`.
|
|
26
|
+
- Write each event as a function `(c: BaseAudioContext, out: AudioNode, t0: number) => number` that builds its graph from `t0` and returns its length in seconds. `play` calls it on the live context; `renderOffline` calls the same function on an `OfflineAudioContext(1, length * sampleRate, sampleRate)` and returns `startRendering()`'s first channel. The sound the check hears is then the sound the game plays.
|
|
27
|
+
- `dispose()` stops every loop, disconnects the master gain and closes the context.
|
|
28
|
+
- A master `GainNode` before the destination; each event's own gain into it. Keep the sum of what can play at once under full scale.
|
|
29
|
+
|
|
30
|
+
## Web Audio facts
|
|
31
|
+
|
|
32
|
+
- **Envelope**: an `AudioParam` is automated against the context's clock: `setValueAtTime(v, t)`, `linearRampToValueAtTime(v, t)`, `exponentialRampToValueAtTime(v, t)` (it can never reach or start from 0: ramp to a small value, then `setValueAtTime(0, end)`), `setTargetAtTime(v, t, timeConstant)`. A gain that jumps between samples clicks.
|
|
33
|
+
- **Oscillator** (`OscillatorNode`): `type` `sine`, `triangle`, `square`, `sawtooth` (or `setPeriodicWave`), `frequency` and `detune` (cents) as AudioParams, `start(t)` and `stop(t)`; a source plays once.
|
|
34
|
+
- **Noise**: an `AudioBuffer` filled with values from -1 to 1 (seed your own generator if the game needs repeatability) played by an `AudioBufferSourceNode`, with `loop`, `loopStart`, `loopEnd` and `playbackRate`.
|
|
35
|
+
- **Filters** (`BiquadFilterNode`): `type` `lowpass`, `highpass`, `bandpass`, `lowshelf`, `highshelf`, `peaking`, `notch`, `allpass`; `frequency`, `Q` and `gain` as AudioParams, so they sweep. `WaveShaperNode` bends the waveform (its `curve`, `oversample`).
|
|
36
|
+
- **Space**: `StereoPannerNode` (`pan` from -1 to 1), or `PannerNode` (`panningModel`, `distanceModel`, `refDistance`, `rolloffFactor`, and the listener's position and orientation on `context.listener`) for 3D; work out `at` against the listener passed to `update`.
|
|
37
|
+
- **Time and rooms**: `DelayNode` (`delayTime`; a gain under 1 fed back into it repeats and decays), `ConvolverNode` (an impulse response you synthesise into a buffer), `DynamicsCompressorNode`.
|
|
38
|
+
- **Modulation**: any node's output can drive an AudioParam (`osc.connect(gain.gain)`), so one oscillator can move another's pitch, a filter or a level.
|
|
39
|
+
|
|
40
|
+
## From the brief to parameters
|
|
41
|
+
|
|
42
|
+
Read the brief, not a genre, and decide each sound's parameters from what it says about that event: what makes the sound (its `sources`), how big and heavy that thing is (the brief's `forms.scale`), what it is made of (the brief's `materials`), the place it happens in and its atmosphere, the action that triggers it (the event's `when`), and the `mood`. For every event, write in a comment above its function which words of the brief set which numbers (pitch, waveform, envelope times, filter, level, space), and say the same in your report. Listen to the result through its render: check the numbers renderOffline gives against what the brief asks, and change what does not fit.
|
|
43
|
+
|
|
44
|
+
## Loops
|
|
45
|
+
|
|
46
|
+
An event with `loop: true` is an ambience that runs until `dispose()`. Make the loop seamless: a source with `loop = true` on a buffer whose last sample meets its first, or copies that crossfade into each other. `renderOffline` renders one cycle of at least 1 s: the check measures the step between its last and first sample (at most 0.02).
|
|
47
|
+
|
|
48
|
+
## What the audio check measures
|
|
49
|
+
|
|
50
|
+
For every event of the brief, `renderOffline(event)` in the preview:
|
|
51
|
+
- length 0.03 to 8 s (a loop at least 1 s), every sample a finite number;
|
|
52
|
+
- peak at most 0.89 (-1 dBFS); RMS at least 0.003 (not silent);
|
|
53
|
+
- the onset within 10 ms of the start (a sound that starts late reads as lag);
|
|
54
|
+
- a loop's seam step at most 0.02 (no click when it repeats).
|
|
55
|
+
|
|
56
|
+
A clipped event: lower its gain or soften the attack. A silent one: a gain left at 0, a node not connected to `out`, or a ramp to 0 before the sound. A click: a loop that does not meet its start, or a gain that jumps; ramp it.
|
|
57
|
+
|
|
58
|
+
## In the game
|
|
59
|
+
|
|
60
|
+
The builder imports `createAudio` through `src/audio/index.ts` (the seam), calls `audio.update(dt, listener)` each frame and `audio.play('<event id>')` where the brief's `when` says. The event ids are the brief's; never rename them.
|
|
@@ -0,0 +1,35 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: style
|
|
3
|
+
description: Deciding how a space is drawn before any picture of it, and writing style.json: analysing the creator's pictures or a named style for how they are drawn, or researching artists and styles and proposing three directions on the style card; then the rules and the look's drawing techniques. Load at the start of a new space.
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# The style
|
|
7
|
+
|
|
8
|
+
How the space is drawn comes before any picture of it, and goes in style.json (`nw_schema` kind `"style"`). The targets are drawn from its words (image_generate refuses without it), look_apply takes its drawing techniques, and the judge scores how each still is drawn against its target. A photograph is one style among the others, never the default. No space starts from another space's look: rounded toy shapes in pastel under soft light is a style too, to be chosen, not fallen into.
|
|
9
|
+
|
|
10
|
+
## Pictures or a named style
|
|
11
|
+
|
|
12
|
+
The creator attached pictures, or named an artist, a game, a film, a print or a movement. Analyse how it is drawn, not what it shows:
|
|
13
|
+
- shape language (rounded, faceted, spindly, chunky) and proportions;
|
|
14
|
+
- how much detail, and where it gathers;
|
|
15
|
+
- the line: none, outlines round forms, ink inside them too, a loose sketch, edges made by the paint;
|
|
16
|
+
- how light turns to shade: as light does, soft and simplified, flat steps, strokes, dots, paint;
|
|
17
|
+
- the surface: grain, brush, paper, flat colour, pixels;
|
|
18
|
+
- the palette, and how it is used.
|
|
19
|
+
|
|
20
|
+
Write it with `from` `"attached"` or `"named"`, each picture or work under `sources` with what this space takes from it.
|
|
21
|
+
|
|
22
|
+
## Neither: research and propose
|
|
23
|
+
|
|
24
|
+
1. With the web consent, search for concept artists, illustrators, printmakers, games, films and art movements whose way of drawing suits the request and its mood. Read about each and keep the page for `sources`. See their work with nw_find_pictures and keep the telling pictures with nw_keep_pictures: references to look at only, never a target or a generator's input.
|
|
25
|
+
2. `nw_others`: the creator's other spaces. Say in a line what they share, and write it in `avoid`.
|
|
26
|
+
3. Three directions that differ from each other in medium, line and shading, and from every one of those spaces. Each gets a name, its medium in a person's words and why it suits the request.
|
|
27
|
+
4. One sample each: `image_generate` kind `"direction"`, `out` its short name, the prompt the place where the visitor arrives drawn that way. Look at each: drawn wholly that way, unlike the others.
|
|
28
|
+
5. The style card: `nw_ask_pictures` with `purpose` `"style"`, one picture a direction, its label the name and its what the medium and why. It may be asked once a request, also after the kickoff.
|
|
29
|
+
6. Write the pick with `from` `"chosen"` and all three under `directions`. With no answer, choose the one least like the other spaces, `from` `"researched"`, and say which in the reply.
|
|
30
|
+
|
|
31
|
+
## The rules and the look
|
|
32
|
+
|
|
33
|
+
- `rules`: what the modelling, the materials and the light must do to draw that way, each checkable in a picture ("every silhouette reads as one dark line", "no texture finer than a hand", "shadows are one flat step toward the palette's violet").
|
|
34
|
+
- `look`: the engine's drawing techniques (nowhere:look-dev lists them: outline, bands, hatch, halftone, paint, paper, pixel), or null for a style the light and materials draw alone. look_apply copies them into game.config.json; nw_status says where the two differ.
|
|
35
|
+
- The style decides how things are made. A cel style wants a clear sun, few soft fills and flat or lightly textured materials; an ink style wants shapes with clean silhouettes and a pale paper ground; a painted style wants the palette's own light; a low-poly console style wants hard facets and small textures. Scanned materials under an HDRI read as a photograph whatever techniques sit on top.
|
|
@@ -0,0 +1,107 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: tsl-r185
|
|
3
|
+
description: The TSL facts for Oda games on three r186's WebGPURenderer: names that exist and do not, node material slots and their inputs, how uniforms and Fn work, what the studio reports when a material fails, and what a material costs. Load before writing any shader or custom material.
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# TSL on three r186
|
|
7
|
+
|
|
8
|
+
The studio and every planet draw with three r186's `WebGPURenderer` (WebGPU, or WebGL 2 where WebGPU is missing). Shaders are written in TSL, three's node language, in plain JavaScript; there is no GLSL anywhere. Every import and node name below exists in the installed r186 and was checked there. (The guide keeps the name tsl-r185 it has always been read by.)
|
|
9
|
+
|
|
10
|
+
This page is the API, not a set of looks: there are no ready-made materials in the SDK or here. Decide what each surface of this game needs from its brief (the colours, the roughness, how it moves, what glows), then write that shader for it from these parts, look at it with nw_view and change it until it reads as the brief says.
|
|
11
|
+
|
|
12
|
+
## Imports
|
|
13
|
+
|
|
14
|
+
```ts
|
|
15
|
+
import * as THREE from 'three/webgpu'
|
|
16
|
+
import { Fn, If, Loop, select, uniform, uniformArray, attribute, time, deltaTime, uv, vertexColor, instanceIndex, color, float, int, vec2, vec3, vec4, mix, step, smoothstep, clamp, saturate, min, max, abs, sign, floor, fract, mod, pow, sqrt, exp, sin, cos, atan, dot, cross, length, distance, normalize, reflect, oneMinus, hash, mx_noise_float, mx_fractal_noise_float, mx_worley_noise_float, triplanarTexture, texture, positionLocal, positionWorld, positionView, positionGeometry, normalLocal, normalWorld, normalView, normalWorldGeometry, positionViewDirection, cameraPosition, cameraViewMatrix, modelWorldMatrix, modelNormalMatrix, screenUV, viewportLinearDepth, PI, TWO_PI } from 'three/tsl'
|
|
17
|
+
```
|
|
18
|
+
|
|
19
|
+
- `three` and `three/webgpu` are the same namespace in an Oda game; take node materials from `three/webgpu` so the types know them. `three/tsl` holds the node functions.
|
|
20
|
+
- Plain materials (`MeshStandardMaterial`, `MeshBasicMaterial`) still work: the renderer turns them into node materials. A node material is needed only where a slot takes a node.
|
|
21
|
+
- These are the only modules a game gets from three: none of `three/addons` or `three/examples` is there. The build checks every name you import from `three`, `three/webgpu`, `three/tsl` and `@nowhere/sdk` against what r186 and the SDK really export, and fails on one that is missing, with the file, the line and what to use instead. A missing member read through a namespace (`THREE.ShaderChunk`, `TSL.atan2`) fails the build only where it certainly throws; anywhere else it is a warning in nw_check and nw_logs, so fix it all the same.
|
|
22
|
+
|
|
23
|
+
## Names r186 does not have
|
|
24
|
+
|
|
25
|
+
Older three.js, GLSL habits and other engines suggest these; none is exported by `three/tsl` in r186, and an import of one fails the build.
|
|
26
|
+
|
|
27
|
+
| Not in r186 | Use |
|
|
28
|
+
| --- | --- |
|
|
29
|
+
| `atan2(y, x)` | `atan(y, x)`: with two arguments it is atan2 (`mx_atan2` exists too, for MaterialX graphs) |
|
|
30
|
+
| `timerLocal()`, `timerGlobal()` | `time` (seconds, a node, not a function) |
|
|
31
|
+
| `timerDelta()` | `deltaTime` |
|
|
32
|
+
| `lerp(a, b, t)` | `mix(a, b, t)` |
|
|
33
|
+
| `tslFn(...)` | `Fn(...)` |
|
|
34
|
+
| `cond(c, a, b)` | `select(c, a, b)` |
|
|
35
|
+
| `loop(...)` | `Loop(...)` |
|
|
36
|
+
| `fmod`, `modInt`, `remainder` | `mod(a, b)` |
|
|
37
|
+
| `random`, `noise`, `perlin`, `simplex` | `hash(seed)` for a random number; `mx_noise_float`, `mx_fractal_noise_float`, `mx_worley_noise_float` for noise |
|
|
38
|
+
| `normalMatrix` | `modelNormalMatrix` |
|
|
39
|
+
| `append(node)` | `Stack(node)` (`.toStack()` as a method): r186 dropped the old name |
|
|
40
|
+
| `viewportResolution` | `screenSize`: r186 dropped the old name |
|
|
41
|
+
| `rangeFog(color, near, far)`, `densityFog(color, density)` | `fog(color, rangeFogFactor(near, far))`, `fog(color, densityFogFactor(density))` in a material: r186 dropped the old names. The planet's own fog is the look's (look_apply) |
|
|
42
|
+
| `PCFSoftShadowFilter` | `PCFShadowFilter`, or the SDK's `softShadowFilter` as a shadow's `filterNode`: r186 removed the soft one |
|
|
43
|
+
| `textureCubeUV` | `pmremTexture(texture, direction, roughness)` |
|
|
44
|
+
| `screen(base, blend)` | `blendScreen(base, blend)`: r186 dropped the old name |
|
|
45
|
+
| `bloom`, `gaussianBlur`, `rgbShift` | nothing in a game: these are addons, and the host owns post-processing (an `emissiveNode` above 1 blooms with the look's bloom, set with look_apply) |
|
|
46
|
+
|
|
47
|
+
They do exist, and are often missed: `atan` (one or two arguments), `mod`, `saturate`, `inverseSqrt`, `select`, `Loop`, `If`, `PI`, `TWO_PI`, `HALF_PI`, `EPSILON`, `deltaTime`, `screenUV`, `viewportUV`, `modelNormalMatrix`, `cameraNormalMatrix`, `uniformArray`, `instancedArray`, `storage`, `compute`, `positionGeometry`, `normalWorldGeometry`, `viewportLinearDepth`, `triplanarTexture`.
|
|
48
|
+
|
|
49
|
+
## Node materials and their slots
|
|
50
|
+
|
|
51
|
+
`MeshStandardNodeMaterial`, `MeshPhysicalNodeMaterial`, `MeshBasicNodeMaterial`, `MeshLambertNodeMaterial`, `MeshToonNodeMaterial`, `SpriteNodeMaterial`, `PointsNodeMaterial`, `LineBasicNodeMaterial`. Each takes the usual parameters (`name`, `color`, `roughness`, `metalness`, `map`, `side`, `transparent`) plus node slots:
|
|
52
|
+
|
|
53
|
+
- `colorNode`: the surface colour. It replaces `color`, `map` and vertex colours; multiply them in yourself.
|
|
54
|
+
- `emissiveNode`: light the surface gives off, added after lighting. Above 1 it blooms with the look's bloom.
|
|
55
|
+
- `roughnessNode`, `metalnessNode`, `opacityNode` (with `transparent: true`), `alphaTestNode`.
|
|
56
|
+
- `normalNode`: the normal in view space (from a world normal: `cameraViewMatrix.mul(vec4(n, 0)).xyz`). Inside it, `normalWorld` is the geometry's; use `normalWorldGeometry` to say so.
|
|
57
|
+
- `positionNode`: the vertex position in local space, moved before projection (normals do not follow). `positionGeometry` is the position before instancing.
|
|
58
|
+
- `MeshToonNodeMaterial` takes a `gradientMap` (a `DataTexture` with `NearestFilter`) for its bands; `MeshPhysicalNodeMaterial` adds `transmissionNode`, `iorNode`, `clearcoatNode`, `sheenNode`.
|
|
59
|
+
|
|
60
|
+
Inputs: `uv()`, `positionLocal`, `positionWorld`, `positionView`, `normalLocal`, `normalWorld`, `normalView`, `positionViewDirection` (towards the camera), `cameraPosition`, `time` (seconds), `vertexColor()` (glTF COLOR_0), `attribute('name')`, `instanceIndex` (in an `InstancedMesh`), `screenUV`. Maths as functions and as methods alike: `mix(a, b, t)` or `a.mix(b, t)`, `a.mul(b).add(0.5)`, `.clamp(0, 1)`, `.pow(3)`, `.x`, `.xz`, `.rgb`. A plain number where a node is wanted becomes a `float`. Noise and randomness: `mx_noise_float(p)` (about -1 to 1), `mx_fractal_noise_float(p, octaves, lacunarity, diminish)`, `mx_worley_noise_float(p)` (cell distance), `hash(seed)` (0 to 1).
|
|
61
|
+
|
|
62
|
+
## How the parts behave
|
|
63
|
+
|
|
64
|
+
A slot takes a node; nodes compose; a `uniform` is the one kind of value a game changes after the material is built:
|
|
65
|
+
|
|
66
|
+
```ts
|
|
67
|
+
import * as THREE from 'three/webgpu'
|
|
68
|
+
import { Fn, float, uniform, uv, vec3 } from 'three/tsl'
|
|
69
|
+
|
|
70
|
+
const m = new THREE.MeshStandardNodeMaterial({ name: 'example' })
|
|
71
|
+
const k = uniform(0.5) // set k.value in update(dt): the shader is not built again
|
|
72
|
+
const f = Fn(([p, r]) => p.length().sub(r)) // a node function: arguments come as one array
|
|
73
|
+
m.colorNode = vec3(f(uv().sub(0.5), float(0.25)).mul(k))
|
|
74
|
+
```
|
|
75
|
+
|
|
76
|
+
- Assigning a slot again, or a new node graph, compiles a new shader on the next frame that shows it: do it once, when the material is made. Change looks over time through `uniform`s and `time`.
|
|
77
|
+
- `If(cond, () => { ... })` and `Loop(n, ({ i }) => { ... })` work only inside an `Fn`; outside one, use `select(cond, a, b)`.
|
|
78
|
+
- `positionWorld` is final only in the fragment stage; inside `positionNode` read the world position as `modelWorldMatrix.mul(vec4(positionLocal, 1)).xyz`.
|
|
79
|
+
- Patterns from `positionWorld` or `positionLocal` need no UVs and line up across meshes; patterns from `uv()` follow the mesh's own unwrap.
|
|
80
|
+
- A colour hex in `color('#rrggbb')` is sRGB, turned linear as `THREE.Color` does.
|
|
81
|
+
|
|
82
|
+
## When three fails
|
|
83
|
+
|
|
84
|
+
three catches a material that fails and draws something else, so the studio watches for it and says it in nw_check, in nw_status (`materials`), in nw_logs (source `shader`) and on the creator's banner. It also builds every material once after the game mounts, hidden ones included, so a failure shows before the camera reaches it. Name your materials (`{ name: 'water' }`) so the finding says which one. What it reports:
|
|
85
|
+
|
|
86
|
+
- failed to build ... and is drawn flat black: a node graph threw while building. The finding names the material, its meshes (`root/ocean/sea`), the game line when the stack leads there (`at src/water.ts:42`) and three's words (`TSL: TypeError: ...`). Usually a value that is not a node where one is needed, a vector of the wrong size, or a name that is undefined.
|
|
87
|
+
- is not a material WebGPU can draw: a `ShaderMaterial` or `RawShaderMaterial`; it is drawn flat black. Rewrite it as a node material.
|
|
88
|
+
- sets onBeforeCompile / is a ShaderMaterial / customProgramCacheKey (warnings): WebGL habits that do nothing here. The build warns about the same in your source (`src/x.ts:L:C`), with GLSL strings and EffectComposer.
|
|
89
|
+
- The GPU refused material ... fragment line 88: the generated WGSL did not compile. The line is in three's shader, not your file: look at the nodes of that material for a type mismatch or a texture sampled in the wrong stage.
|
|
90
|
+
|
|
91
|
+
Fix each one, then nw_check again: a material error is not done.
|
|
92
|
+
|
|
93
|
+
## What does nothing here
|
|
94
|
+
|
|
95
|
+
- `material.onBeforeCompile`: never called by this renderer. Use the node slots.
|
|
96
|
+
- `ShaderMaterial` and `RawShaderMaterial`: GLSL is not compiled, and the mesh is drawn flat black; write a node material.
|
|
97
|
+
- `ShaderChunk`, `ShaderLib`, `#include <...>`, `gl_FragColor`: there is no GLSL source to patch.
|
|
98
|
+
- `EffectComposer` and the old post passes: the host owns post-processing (tone mapping, bloom, AO, grade). Set the planet's look with look_apply.
|
|
99
|
+
- Your own `requestAnimationFrame`: animate through `time` or uniforms set in `update(dt)`.
|
|
100
|
+
|
|
101
|
+
## What a material costs
|
|
102
|
+
|
|
103
|
+
- Each distinct node material is a shader to compile (the first frame that shows it stalls); meshes that share one material share its shader.
|
|
104
|
+
- An `InstancedMesh` is one draw call for all its copies; static geometry that never moves can be merged.
|
|
105
|
+
- Noise is evaluated per pixel per octave: its cost grows with the screen area it covers.
|
|
106
|
+
- Rebuilding a node graph every frame recompiles every frame; changing a uniform does not.
|
|
107
|
+
- After a change, look with nw_view (a close-up for the material, the player view for the cost) and check fps in nw_status.
|