@homie-rocks/studio 0.48.4 → 0.50.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +46 -0
- package/app-starters/gathering/README.md +4 -0
- package/app-starters/gathering/app.json +162 -0
- package/app-starters/gathering/index.html +3 -0
- package/app-starters/gathering/map/main.json +1 -0
- package/app-starters/gathering/src/move.ts +3 -0
- package/app-starters/gathering/src/rules.ts +21 -0
- package/app-starters/gathering/src/view.ts +74 -0
- package/app-starters/giving/README.md +5 -0
- package/app-starters/giving/app.json +162 -0
- package/app-starters/giving/index.html +3 -0
- package/app-starters/giving/map/main.json +1 -0
- package/app-starters/giving/src/move.ts +3 -0
- package/app-starters/giving/src/rules.ts +21 -0
- package/app-starters/giving/src/view.ts +74 -0
- package/app-starters/jobs/README.md +4 -0
- package/app-starters/jobs/app.json +162 -0
- package/app-starters/jobs/index.html +3 -0
- package/app-starters/jobs/map/main.json +1 -0
- package/app-starters/jobs/src/move.ts +3 -0
- package/app-starters/jobs/src/rules.ts +21 -0
- package/app-starters/jobs/src/view.ts +74 -0
- package/app-starters/quiz/README.md +4 -0
- package/app-starters/quiz/app.json +162 -0
- package/app-starters/quiz/index.html +3 -0
- package/app-starters/quiz/map/main.json +1 -0
- package/app-starters/quiz/src/move.ts +3 -0
- package/app-starters/quiz/src/rules.ts +21 -0
- package/app-starters/quiz/src/view.ts +74 -0
- package/app-starters/welcome/app.json +25 -2
- package/app-starters/welcome/index.html +1 -1
- package/app-starters/welcome/src/view.ts +20 -8
- package/bin/homie-studio.mjs +20 -16
- package/functions/FUNCTIONS.md +1 -1
- package/guides/app/SKILL.md +178 -0
- package/guides/art/SKILL.md +154 -0
- package/guides/art/references/RIGHTS.md +20 -0
- package/guides/art/scripts/art.mjs +308 -0
- package/guides/models/SKILL.md +198 -0
- package/guides/models/references/RIGHTS.md +46 -0
- package/guides/models/references/models.json +175 -0
- package/guides/models/scripts/models.mjs +483 -0
- package/guides/music/SKILL.md +197 -0
- package/guides/music/references/COMPOSITION.md +59 -0
- package/guides/music/references/GAME-AUDIO.md +50 -0
- package/guides/music/references/RIGHTS.md +43 -0
- package/guides/music/scripts/lib/audio.mjs +258 -0
- package/guides/music/scripts/lib/eleven.mjs +260 -0
- package/guides/music/scripts/lib/studio.mjs +147 -0
- package/guides/music/scripts/music.mjs +644 -0
- package/guides/parts/SKILL.md +158 -0
- package/guides/plan/SKILL.md +126 -0
- package/guides/plan/references/CODEX.md +105 -0
- package/guides/plan/references/INTERVIEW.md +86 -0
- package/guides/playtest/SKILL.md +205 -0
- package/guides/playtest/references/METHOD.md +124 -0
- package/guides/playtest/references/REVIEWER.md +78 -0
- package/guides/playtest/scripts/lib/exit.mjs +31 -0
- package/guides/playtest/scripts/lib/judge.mjs +502 -0
- package/guides/playtest/scripts/lib/pixels.mjs +128 -0
- package/guides/playtest/scripts/lib/preflight.mjs +69 -0
- package/guides/playtest/scripts/playtest.mjs +870 -0
- package/guides/shop/SKILL.md +153 -0
- package/guides/sound/SKILL.md +156 -0
- package/guides/sound/references/GAME-SOUND.md +100 -0
- package/guides/sound/references/SCORE.md +131 -0
- package/guides/sound/references/TRAPS.md +93 -0
- package/guides/sound/scripts/lib/measure.mjs +252 -0
- package/guides/sound/scripts/lib/score.mjs +415 -0
- package/guides/sound/scripts/lib/sfx.mjs +222 -0
- package/guides/sound/scripts/lib/synth.mjs +509 -0
- package/guides/sound/scripts/sound.mjs +520 -0
- package/guides/studio-setup/SKILL.md +419 -0
- package/guides/style/SKILL.md +105 -0
- package/lib/app-needs.mjs +63 -0
- package/lib/authoring-guides.mjs +5 -4
- package/lib/build.mjs +5 -4
- package/lib/check-site.mjs +6 -6
- package/lib/check.mjs +18 -5
- package/lib/codex.mjs +95 -18
- package/lib/decisions.mjs +9 -7
- package/lib/dev.mjs +4 -2
- package/lib/doctor.mjs +1 -1
- package/lib/handoff.mjs +13 -8
- package/lib/mcp-tools.mjs +19 -17
- package/lib/mcp.mjs +1 -1
- package/lib/parts.mjs +6 -6
- package/lib/scaffold.mjs +45 -11
- package/lib/shop-links.mjs +2 -2
- package/lib/shop.mjs +5 -3
- package/lib/studio.mjs +7 -6
- package/lib/template-history.json +84 -0
- package/package.json +4 -4
- package/shop/SHOP.md +4 -0
- package/shop/shop.ts +4 -4
- package/worker/app-format.mjs +1 -1
- package/worker/build-id.mjs +1 -1
- package/worker/functions-schema.mjs +15 -0
- package/worker/index.mjs +2 -1
- package/worker/parts.mjs +4 -4
- package/worker/shop-page.mjs +22 -16
- package/worker/shop-rules.mjs +4 -3
- package/worker/shop.mjs +6 -2
- package/worker/site.mjs +4 -2
- package/worker/version.mjs +1 -1
|
@@ -0,0 +1,153 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: shop
|
|
3
|
+
description: Sell things in a Homie studio's games with the studio's OWN Stripe - a supporter pack, cosmetics, a season pass, a one-time unlock, a tip - in real money, with Stripe Checkout, Stripe Tax or Stripe Managed Payments, refunds from the office, and optional studio policies with an open default; set up with Stripe's own agent tools (Stripe's MCP server and skills) with one browser approval and Payment Links by default, no human-handled key and no API key in the Worker. Use when someone asks to sell something, add a shop or store, take payments or donations, make money from a game, set up or connect Stripe, make the products in Stripe, asks "how are sales?", about tax or Managed Payments, to add a supporter badge, refund a player, or about chargebacks, referral shares or affiliate links between studios.
|
|
4
|
+
compatibility: The studio's own current toolkit with stripe_login or shop connect --renew, Wrangler, and the official Stripe CLI. Stripe's own agent plugin (MCP and skills) is optional for catalog work and analytics.
|
|
5
|
+
metadata:
|
|
6
|
+
providers: stripe
|
|
7
|
+
---
|
|
8
|
+
|
|
9
|
+
Customer AIs can buy through the studio's own `/mcp`. Read the toolkit's `tools/TOOLS.md`
|
|
10
|
+
and use the `tools` skill for `studio_catalogue`, cart quotes, standard agent payments,
|
|
11
|
+
private purchase claims, and human Stripe checkout approval. Priced studio tools sell
|
|
12
|
+
services through that same purchase engine; `shop connect` synchronizes their Payment Links.
|
|
13
|
+
Keep orders, grants and refunds in the existing shop. For order/refund follow-up code, use
|
|
14
|
+
Studio functions (`functions/FUNCTIONS.md`), with the event ID as the idempotency key.
|
|
15
|
+
|
|
16
|
+
|
|
17
|
+
# The shop
|
|
18
|
+
|
|
19
|
+
Check that the toolkit exposes `stripe_login`, or that `homie-studio --help` lists `shop connect` with `--renew` and `--manual`. An older toolkit's default is the paste page: upgrade before using this flow; do not send the owner to that old default.
|
|
20
|
+
The guide is `node_modules/@homie-rocks/studio/shop/SHOP.md`; the owner's plain words are `SELLING.md`.
|
|
21
|
+
|
|
22
|
+
**The model, said plainly to the person.** The studio sells with **its own Stripe account**. The studio is the
|
|
23
|
+
seller: its prices, its refunds, its disputes, its tax. Money goes straight from players to the studio's Stripe;
|
|
24
|
+
**homie.rocks never sees it, holds it or moves it, and Homie takes no cut.** The studio is responsible for the law where it sells and for Stripe's terms.
|
|
25
|
+
|
|
26
|
+
In a rules game, keep shop UI in the view. Do not turn a client character save, an
|
|
27
|
+
entity field or a command into proof of a purchase: grants still come from the shop's
|
|
28
|
+
verified server records. Room state is replicated to players; keep payment and private
|
|
29
|
+
customer data out of it. Read the game skill's RULES.md before changing game outcomes.
|
|
30
|
+
|
|
31
|
+
## Connect with Stripe's own browser approval
|
|
32
|
+
|
|
33
|
+
Use `stripe_login` (MCP) or `npx --no-install homie-studio shop connect`. Test mode and a
|
|
34
|
+
sandbox first. If the CLI is missing, run the one installation command the result gives:
|
|
35
|
+
`npm install -g @stripe/cli@latest`. The toolkit opens Stripe's page and completes its official
|
|
36
|
+
non-interactive login. Give the person the pairing code; they select their own account and approve.
|
|
37
|
+
Use `studio_job` for the background result. Do not call a raw CLI config command: it may print secrets.
|
|
38
|
+
|
|
39
|
+
The default is **keyless**: the toolkit uses the CLI's approved account to sync Products, Prices
|
|
40
|
+
(including custom amounts for tips), Payment Links and the webhook. It captures the signing secret
|
|
41
|
+
privately and installs it with the link configuration; it never exports the CLI credentials.
|
|
42
|
+
The Worker redirects buyers to Stripe and grants purchases from signed events. No desktop agent,
|
|
43
|
+
Homie service or API key is needed when someone pays. Run connect again after editing shop.json;
|
|
44
|
+
it updates changed objects and archives removed ones. Existing connections need no new approval.
|
|
45
|
+
|
|
46
|
+
Only when the person asks for combined carts, spending reservations, custom checkout expiry,
|
|
47
|
+
automatic recovery of lost webhooks or office refunds, explain one sentence:
|
|
48
|
+
“The fuller connection adds combined carts, spending limits, automatic recovery and refunds here.”
|
|
49
|
+
A pre-existing Worker key selects that path automatically. Never silently select `--manual` or ask
|
|
50
|
+
for a key: explain its remaining Dashboard step only if the person explicitly chooses that route.
|
|
51
|
+
Before relying on spending restrictions after switching, deactivate old Payment Links with the approved CLI and resolve in-flight checkouts; a credential change does not cancel them.
|
|
52
|
+
`shop connect` selects keyless again and removes an existing Worker API key after saving the links.
|
|
53
|
+
|
|
54
|
+
## Never
|
|
55
|
+
|
|
56
|
+
- Never ask a person for a Stripe key, a webhook secret or a password in chat. Never display a CLI
|
|
57
|
+
config, credential response, environment value or raw login-completion output. The toolkit captures
|
|
58
|
+
these privately. Keys belong to the studio; no Homie account or service takes payments.
|
|
59
|
+
- **Never make a webhook endpoint or an event destination through Stripe's MCP**: its signing secret
|
|
60
|
+
would enter the conversation. `shop connect` handles endpoint creation and secret storage privately.
|
|
61
|
+
- Never silently select `--manual`. Only when the owner chooses the fallback, run
|
|
62
|
+
`shop connect --manual` (or `--manual --live`) and give the local page link. The page explains the
|
|
63
|
+
remaining Dashboard restricted-key step and creates the webhook automatically. This fallback
|
|
64
|
+
requires copying a key and does not meet the one-approval target; say so plainly.
|
|
65
|
+
- Live mode only when the owner says go live. Account activation, tax, bank details and Stripe
|
|
66
|
+
approval links are the owner's. Never approve a Stripe confirmation link for them.
|
|
67
|
+
- The studio chooses its settings. Omitted policies are open; choose a preset only when requested.
|
|
68
|
+
|
|
69
|
+
## The owner's steps, for a creator with a fresh Stripe account
|
|
70
|
+
|
|
71
|
+
With an existing Stripe account and deployed studio, there is **one Stripe approval** for the CLI;
|
|
72
|
+
if that CLI profile is already connected, there are **zero new approvals**. An existing MCP-only
|
|
73
|
+
consent is not a CLI login and is not exported or borrowed. Account creation/activation, an administrator
|
|
74
|
+
enabling CLI access, and an initial Cloudflare sign-in are separate Stripe/hosting requirements.
|
|
75
|
+
|
|
76
|
+
| The owner | You |
|
|
77
|
+
|---|---|
|
|
78
|
+
| Approves Stripe's browser page, selecting their studio account and sandbox. | Create/check shop.json; run stripe_login / shop connect; follow studio_job to completion. |
|
|
79
|
+
| Nothing. | Sync all items and the webhook, install the signing secret and links privately, then verify a test purchase and grant. |
|
|
80
|
+
| Requests a refund in Stripe or asks their connected AI. | Use the official Stripe connection, honor any confirmation link, verify the signed refund event removes the item. |
|
|
81
|
+
| Says “go live” and completes Stripe activation if needed. | shop connect --live in the authorized account; verify only an authorized real purchase. |
|
|
82
|
+
|
|
83
|
+
`shop connect --renew` restarts the approval flow when needed. CLI OAuth refresh is the CLI's job;
|
|
84
|
+
there is no runtime API credential to expire. `shop catalog` remains the optional older catalog planner
|
|
85
|
+
for keyed shops; keyless shops use connect for the complete catalog, links and webhook sync.
|
|
86
|
+
|
|
87
|
+
## Optional Stripe agent tools
|
|
88
|
+
|
|
89
|
+
For catalog work, analytics and advice, `stripe agent setup` installs Stripe's own agent plugin and
|
|
90
|
+
skills; its OAuth MCP is `https://mcp.stripe.com`. This is optional and separate from Worker credentials.
|
|
91
|
+
Check `get_stripe_account_info` before writes; confirm the account and mode match the studio.
|
|
92
|
+
For CLI refunds or event resends, use the non-secret `profile` returned by connect with `stripe --project-name <profile>`; preserve its test/live context. Do not read credentials.
|
|
93
|
+
The tools include `stripe_api_read`, `stripe_api_write`, API discovery and `stripe_analytics`.
|
|
94
|
+
From **2026-10-31**, MCP rejects API keys without the Agent tag; OAuth remains supported. Agent keys
|
|
95
|
+
can put refunds behind Stripe approval. A returned confirmation link belongs to the owner; wait
|
|
96
|
+
for approval before retrying. Do not silently replace the payment runtime with MCP; a separate persistent MCP client would need its own production verification.
|
|
97
|
+
|
|
98
|
+
## Do
|
|
99
|
+
|
|
100
|
+
| The person says | Run | What happens |
|
|
101
|
+
|---|---|---|
|
|
102
|
+
| "Sell a supporter pack for $5" | `npx --no-install homie-studio shop init --supporter` (then `shop check`) | `shop.json` with a US$5 Supporter pack (a badge on their profile and beside their name in rooms, for a year; it changes nothing about play) and `SELLING.md`. Commit both. |
|
|
103
|
+
| "Sell a skin / a season pass / the full game" | edit `shop.json` `items` (`kind`: `cosmetic`, `pass`, `unlock`; `price` in Stripe currency units; `gives`: the keys the game reads), `shop check`, then `shop connect` again | Kinds are studio labels, without a fixed list. `"advantage": true` follows the studio policy for beginner and kids servers. Homie sets no price ceiling. |
|
|
104
|
+
| "Let people tip" | an item `{ "kind": "tip", "price": "choose", "min": 200, "max": 5000 }` | Pay what you want; amounts follow Stripe currency requirements, and max is optional and chosen by the studio. |
|
|
105
|
+
| "Make the products in Stripe" | `shop connect` | Idempotent Product, Price, Payment Link and webhook sync from shop.json; no secret enters tool output. |
|
|
106
|
+
| "Connect my Stripe" / "turn the shop on" | `stripe_login` or `shop connect` | Approve Stripe once if needed; the Worker receives links and the signing secret only. Follow the single next step returned. Live only when requested. |
|
|
107
|
+
| "Is tax set up?" | `stripe_api_read` `GET /v1/tax/settings`, and `GET /v1/tax/registrations` | Seller "stripe": Stripe Tax needs `status: active` (the business address in Settings, Tax), or checkouts fail; with no registrations it collects no tax anywhere: say so, the accountant decides where to register. Seller "stripe-managed": Stripe files the tax; the connect page's test checkout said whether Managed Payments is on. Change nothing yourself. |
|
|
108
|
+
| "Is the shop working?" | `npx --no-install homie-studio shop` | Open (test or live) or exactly what is missing, the last 30 days, the webhook address. |
|
|
109
|
+
| "How are sales?" / "Show me the sales" | `shop` and `shop orders` first (the studio's own books); with Stripe's MCP, read-only: `stripe_analytics`, or `stripe_api_read` on `/v1/balance`, `/v1/payouts`, `/v1/checkout/sessions` | Counts, money and payouts in a few lines; never a buyer's name, email or card. Never a write to answer a question. The owner's `/_studio/office/shop` links every Stripe page and gives the accountant a CSV. |
|
|
110
|
+
| "Refund that player" | The owner’s official Stripe connection, or link their Stripe Dashboard payment | Honor Stripe’s confirmation link; signed events update the books and revoke fully refunded items. The keyed way still offers office refunds. |
|
|
111
|
+
| "Someone charged back" | nothing to undo: the owner answers it in Stripe (the office links it) | While open nothing changes; lost: that one item goes; won: it stays. **The account is never locked or deleted over a dispute.** |
|
|
112
|
+
| "Show the item in the game" / "the supporter badge" | in the game: `createShop()` from `@homie-rocks/studio/shop`; `shop.has('skin:ember')`, `shop.on('change', …)`, `shop.open('ember-skin')` from a button the player pressed, `shop.used(key)` when equipped | The play shell answers for the signed-in player; a badge rides on their seat as `peer.badge` (the Worker sets it, never a hello). |
|
|
113
|
+
| "Pay studios that send us players" / "affiliate links" | `shop.json` `referrals` (rate, window, hold), `shop statements [--send]` | A `?via=<host>` link from another site (homie.rocks is one more referrer, on the same terms) counts for a new player's purchases; statements are signed with the studio's key; the referrer invoices the studio; the owner pays and marks it paid (an ASK from you). Nothing moves through Homie. |
|
|
114
|
+
|
|
115
|
+
## Your shop, your choices
|
|
116
|
+
|
|
117
|
+
The default is open: guests buy repeatedly without an age question, with any wording or kind, from any page.
|
|
118
|
+
`shop init` writes no policy. A released shop file without a preset now uses the open default.
|
|
119
|
+
Write `"policy": { "preset": "protective" }` to retain earlier account and age behavior.
|
|
120
|
+
`adults-only` is another optional bundle. Neither scans content. SHOP.md lists individual overrides.
|
|
121
|
+
|
|
122
|
+
The studio can set items, quantities, prices, tip ranges, entitlements, sale dates, currency, optional
|
|
123
|
+
`automaticTax`, `capPerPlayerMonth`, `refundDays`, referral terms and configurable flood protection.
|
|
124
|
+
No toolkit amount or duration ceiling applies. Stripe currency constraints and safe integer arithmetic apply.
|
|
125
|
+
A refund window includes used items unless `policy.refundUsedItems` is false. A player cannot refund a tip;
|
|
126
|
+
the owner can refund through Stripe; keyed shops also offer office refunds. Guest purchases stay owned when the buyer signs in later.
|
|
127
|
+
|
|
128
|
+
For keyed shops, carts: `shop.add(item, quantity, amount?)`, then `shop.checkout()`; `shop.buy(item)` buys directly.
|
|
129
|
+
Free and paid lines share one order and one Stripe Session. The studio chooses where to open the shop.
|
|
130
|
+
|
|
131
|
+
## The first sale (the acceptance)
|
|
132
|
+
|
|
133
|
+
In test mode: buy the supporter pack on a phone with Stripe's test card `4242 4242 4242 4242`, see "It's yours",
|
|
134
|
+
see the badge on the account page and beside the name in a room (from the next room the player joins), refund it
|
|
135
|
+
in Stripe (or through the connected AI), and see the badge go. With the protective preset, a kids server's room shows no shop and `/<game>/tv` shows only a code.
|
|
136
|
+
|
|
137
|
+
Keyless purchases buy one item type at a time, with quantity confirmed on Stripe. Tips choose the
|
|
138
|
+
amount on Stripe and always have quantity one. Free orders grant locally without Stripe.
|
|
139
|
+
Refund and dispute snapshots are stored by provider object ID and tolerate retries and reordering.
|
|
140
|
+
A missing webhook stays pending: ask Stripe to resend the event (Dashboard or official CLI
|
|
141
|
+
`stripe events resend <event> --webhook-endpoint <endpoint>`). Never grant based on the return URL.
|
|
142
|
+
Payment Links are public and reusable; do not promise that local sale dates, repeat-purchase or
|
|
143
|
+
eligibility checks revoke a previously opened link. Deactivation stops new sessions after re-sync;
|
|
144
|
+
already opened sessions may still complete. Strict pre-payment enforcement needs the keyed way.
|
|
145
|
+
Stripe always enables Adaptive Pricing on Payment Links; the buyer can use the original currency.
|
|
146
|
+
Keyless has no per-session expiry or provider-read reconciliation. The keyed path retains both,
|
|
147
|
+
carts, cap reservations, local refunds. Managed Payments is supported on Payment Links when Stripe enables it for the account. The studio chooses its needs.
|
|
148
|
+
|
|
149
|
+
For a cause, use `shop init --giving --currency <organisation currency>`: one-time and monthly suggested gifts, no badge or game access. Read `giving.receiptText` from the organisation; leave it empty until provided. Local /shop/ previews before Stripe. Monthly gifts use the existing keyless Payment Links connection and Stripe handles renewals.
|
|
150
|
+
|
|
151
|
+
## App payment references
|
|
152
|
+
|
|
153
|
+
Create the booking/ticket/gift intention first. `shop.buy('full-tune', 1, undefined, { reference: booking.id })` or `shop.checkout({ reference: booking.id })` sends the app's own reference through checkout. Existing boolean returns are unchanged: checkout opened is not paid. The verified durable `order.paid` event carries `data.order`, `data.reference` (null if omitted) and `data.app`. Reopened Payment Links retain the reference. A reference is buyer-supplied correlation, not authorization: verify the record, app, amount and currency, deduplicate by event/order ID, then fulfil it. Keep personal data out of references. Before Stripe is connected, local prices and unpaid requests can be demonstrated; actual payment and recurring collection remain unverified. Test events must be labelled simulations.
|
|
@@ -0,0 +1,156 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: sound
|
|
3
|
+
description: Make a game's sound effects and synthesized music on the creator's own computer, free, with no provider or account — a set of effects (jump, coin, hit, explosion, win, lose, UI) rendered from presets, a short synthesized theme or score from chords and patterns with stems and seamless section loops, a mix of any audio files and expressions — measured for loudness, clipping, late starts and what a phone speaker loses, then wired into the game with a small player that starts on the first touch and switches music on bar lines. Use when someone asks for sound effects, SFX, game audio, a chiptune or synth theme, background music without a paid service, music that changes with the game, or says a game is silent.
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# Sound for a studio's game
|
|
7
|
+
|
|
8
|
+
Everything here runs on this computer with Node 22 and ffmpeg: nothing is sent anywhere and nothing
|
|
9
|
+
costs money. Sung songs and produced music from a model are the `music` skill (ElevenLabs, paid);
|
|
10
|
+
this skill is effects and synthesized music. They share the studio's `music/` folder and manifest, so
|
|
11
|
+
a synthesized theme gets a song page just like a rendered one.
|
|
12
|
+
|
|
13
|
+
One script: `scripts/sound.mjs` in this skill's folder (Claude Code:
|
|
14
|
+
`node "${CLAUDE_PLUGIN_ROOT}/skills/sound/scripts/sound.mjs" <command>`), run from inside the studio
|
|
15
|
+
(the folder with `studio.json`). Every command takes `--json`.
|
|
16
|
+
|
|
17
|
+
```sh
|
|
18
|
+
node <sound.mjs> check # node, ffmpeg and its encoders, the studio and its games
|
|
19
|
+
node <sound.mjs> presets # every effect preset, instrument, groove and pattern, one line each
|
|
20
|
+
```
|
|
21
|
+
|
|
22
|
+
No ffmpeg: offer `brew install ffmpeg` (macOS) or the system package; the person approves.
|
|
23
|
+
Finish in this turn: an effect set takes seconds, a one-minute score under a minute.
|
|
24
|
+
|
|
25
|
+
## 1. Effects
|
|
26
|
+
|
|
27
|
+
```sh
|
|
28
|
+
node <sound.mjs> sfx <set> --kit arcade --for-game <id> # a kit: arcade, platformer, shooter, party, ui
|
|
29
|
+
node <sound.mjs> sfx <set> --only jump,coin,hit,win --style retro # just these, crunchy 8-bit
|
|
30
|
+
node <sound.mjs> sfx <set> --spec music/<set>/effects.json # your own list (below)
|
|
31
|
+
```
|
|
32
|
+
|
|
33
|
+
It writes `music/<set>/sfx/<name>-<n>.wav` and an Ogg copy of each, `music/<set>/sfx.json` (the
|
|
34
|
+
spec and every file's measured level), a reel to listen to (`<set>.mp3`) and its picture
|
|
35
|
+
(`<set>-reel.png`, a spectrogram over the waveform). **Open the picture and look**: a sound you
|
|
36
|
+
cannot hear is still a sound you can see. Then read the warnings; each one is something a person will
|
|
37
|
+
hear wrong, with the fix.
|
|
38
|
+
|
|
39
|
+
A spec is `{ "style": "clean", "variants": 3, "effects": [ { "name": "stomp", "preset": "land", "pitch": -3, "length": 1.2, "level": "big" } ] }`:
|
|
40
|
+
`preset` is any name from `presets` (the effect's name is used when it is one), `pitch` in
|
|
41
|
+
semitones, `length` a stretch, `seed` another take, `style` clean, retro or soft, `level` ui,
|
|
42
|
+
small, normal, big or a number in dB. Name effects after what happens in the game (`stomp`,
|
|
43
|
+
`gem`, `ko`), not after the preset.
|
|
44
|
+
|
|
45
|
+
- **Variants** (three by default) are the same recipe a touch higher or lower, so a sound heard fifty
|
|
46
|
+
times a round does not machine-gun. Signals heard once a round (win, lose, go, countdown) get one take.
|
|
47
|
+
- **Levels** are set by the loudest 50 ms of each effect into four classes (ui -20, small -16,
|
|
48
|
+
normal -12, big -9 dB), then held under -1 dBFS. The set sits together in a mix without balancing.
|
|
49
|
+
- **Every effect must start at once** (a hit that starts 30 ms late feels laggy) and **must reach a
|
|
50
|
+
phone speaker**: under about 300 Hz a phone plays almost nothing, so a thud needs a knock on top.
|
|
51
|
+
The warnings measure both.
|
|
52
|
+
|
|
53
|
+
## 2. A synthesized theme or score
|
|
54
|
+
|
|
55
|
+
Write `music/<slug>/score.json` (the full language, with a worked example: `references/SCORE.md`):
|
|
56
|
+
tempo, key, instruments (presets with a level, pan, echo in beats, reverb send, a duck under the
|
|
57
|
+
kick), sections in whole bars with chords and one pattern per instrument, an arrangement, and the
|
|
58
|
+
sections to loop. Then:
|
|
59
|
+
|
|
60
|
+
```sh
|
|
61
|
+
node <sound.mjs> score <slug> --spec music/<slug>/score.json --loops main,chase
|
|
62
|
+
```
|
|
63
|
+
|
|
64
|
+
- It renders the mix (-14 LUFS for a page; `--lufs -18` for a bed under a game's effects), the page
|
|
65
|
+
mp3, a 24-bit master kept off the site, one stem per instrument plus the room (they sum to the
|
|
66
|
+
mix), and **seamless loops** of the sections you name: each is rendered three times in a row and
|
|
67
|
+
the middle copy kept, so reverb and echoes that ring over the loop point are already in its head.
|
|
68
|
+
The report gives each loop's seam (1 or less is seamless) and its exact bar length. A loop over 1
|
|
69
|
+
is in `warnings` and the summary says NOT SEAMLESS (`loopsSeamless: false`): an empty warnings
|
|
70
|
+
list means the loops were checked and pass. The master's limiter is applied across the three
|
|
71
|
+
copies, so a loud loop keeps its seam; nothing is faded at the loop point and no bar is shortened.
|
|
72
|
+
- `plan.json` holds the bar grid, so the `video` skill cuts a trailer on this music's bar lines.
|
|
73
|
+
- Look at `<slug>.png` and read the numbers: loudness, true peak, and what it measures through a phone
|
|
74
|
+
speaker. Say them in your report, not adjectives.
|
|
75
|
+
- Write music a person would hum: a hook in the first bars, a clear melody over simple chords, a
|
|
76
|
+
change every 4 or 8 bars. A score for a game is sections of the same tempo and key (calm, chase,
|
|
77
|
+
final seconds) so the game can move between them on a bar line.
|
|
78
|
+
|
|
79
|
+
## 3. A mix of anything
|
|
80
|
+
|
|
81
|
+
```sh
|
|
82
|
+
node <sound.mjs> mix <slug> --spec music/<slug>/mix.json
|
|
83
|
+
```
|
|
84
|
+
|
|
85
|
+
Parts are audio files (a recording, a render, a loop), ffmpeg `aevalsrc` expressions (one, or two for
|
|
86
|
+
left and right), or effect presets, each placed (`at` seconds), levelled (`db`), panned, faded, repeated
|
|
87
|
+
and filtered with an ffmpeg `-af` chain. They are summed at exactly the levels written (no automatic
|
|
88
|
+
normalisation), limited, and brought to `master.lufs` if you give one. Expressions have traps that
|
|
89
|
+
fail silently: read `references/TRAPS.md` before writing one.
|
|
90
|
+
|
|
91
|
+
## 4. Into the game
|
|
92
|
+
|
|
93
|
+
```sh
|
|
94
|
+
node <sound.mjs> wire <set> <score slug> --game <id>
|
|
95
|
+
```
|
|
96
|
+
|
|
97
|
+
It copies the files into the game (`public/sound/`, or `sound/` for a static game), writes
|
|
98
|
+
`sound.json` (what exists) and `sound.js` (the player) and prints the two lines to add. Then, in the
|
|
99
|
+
game's code:
|
|
100
|
+
|
|
101
|
+
```js
|
|
102
|
+
import { createSound } from './sound.js';
|
|
103
|
+
const sound = createSound({ base: 'sound/' });
|
|
104
|
+
sound.music('<score slug>'); // once, at the top: it starts on the first touch
|
|
105
|
+
sound.play('coin'); // where the thing happens
|
|
106
|
+
sound.play('coin', { pitch: combo }); // a climbing combo; pan -1..1 for where it is
|
|
107
|
+
sound.section('chase'); // the next loop, exactly on the next bar line
|
|
108
|
+
sound.duck(0.4, 0.5); // music down for a big hit
|
|
109
|
+
```
|
|
110
|
+
|
|
111
|
+
- **Sound starts on the first touch or key, with no "tap for sound" screen**: the player resumes audio
|
|
112
|
+
inside the first gesture. A sound asked for before that is dropped, not queued.
|
|
113
|
+
- **Every browser plays its own sounds** for what it sees: call `play` where the each browser
|
|
114
|
+
draw the event (a pickup that shows up in a snapshot, a hit that lands on screen), not only in the
|
|
115
|
+
rules handlers. In a rules game emit a declared effect and play it from room.on in the view; the server has no speaker.
|
|
116
|
+
- Loops go out as Ogg with a WAV fallback (older Safari cannot decode Ogg): the player takes the first
|
|
117
|
+
one the browser decodes. `wire` prints the real download size; keep a game's sound under about 3 MB.
|
|
118
|
+
- Each variant of an effect, and each loop, is a list of files tried in order (`["jump.ogg", "jump.wav"]`);
|
|
119
|
+
a list of one file is fine.
|
|
120
|
+
- A trailer filmed frame by frame gets its sound from this player's own log of what it scheduled
|
|
121
|
+
(off unless a recorder asks; `references/GAME-SOUND.md`, "A log for a recorder"): sound that goes
|
|
122
|
+
through `sound.play` and `sound.music` is in the trailer, sound made any other way is not.
|
|
123
|
+
- Rules for game sound (every action answers, what repeats, music under effects, intensity changes on
|
|
124
|
+
bar lines, the live-loop traps): `references/GAME-SOUND.md`.
|
|
125
|
+
- For sound made live in the browser (a synth voice per player, pitch that follows speed, a beat
|
|
126
|
+
clock), the open engine package `@homie-rocks/audio` has the building blocks; install it exactly
|
|
127
|
+
pinned (`npm install --save-exact @homie-rocks/audio@0.1.0`).
|
|
128
|
+
|
|
129
|
+
## 5. Prove it, and publish it
|
|
130
|
+
|
|
131
|
+
Build, run the game, and let the `playtest` skill's sound row listen: it copies what the game really
|
|
132
|
+
sends to the speaker while it is played, and says how loud it is, whether it clips, what a phone loses,
|
|
133
|
+
whether actions answer with a sound, and whether the music ever started. Never say a game "has sound"
|
|
134
|
+
from the code alone.
|
|
135
|
+
|
|
136
|
+
```sh
|
|
137
|
+
node <sound.mjs> add <slug> --title "<Title>" --blurb "<one line>" [--for-game <id>] --publish
|
|
138
|
+
node <sound.mjs> publish <slug> # rebuild and redeploy the site; the page answers 200
|
|
139
|
+
```
|
|
140
|
+
|
|
141
|
+
An effect set gets a page with its reel and every effect to download; a score gets its player, its
|
|
142
|
+
loops and its stems. The credits line says they were synthesized from code (no samples, no model).
|
|
143
|
+
Commit `music/<slug>/` specs and the manifest; the audio files stay out of git (the studio's
|
|
144
|
+
`.gitignore`) and regenerate from the spec, byte for byte.
|
|
145
|
+
|
|
146
|
+
## Tell the person
|
|
147
|
+
|
|
148
|
+
The page or the game, what was made (how many effects, the theme's length and bars), the loudness
|
|
149
|
+
numbers, the warnings you fixed and any you did not, and that it cost nothing.
|
|
150
|
+
|
|
151
|
+
## Never
|
|
152
|
+
|
|
153
|
+
- Never ship a sound you have not measured and looked at.
|
|
154
|
+
- Never use a mp3 for an effect or a loop: encoders pad the start with silence (a late hit) and the
|
|
155
|
+
ends (a gap at the loop). WAV or Ogg.
|
|
156
|
+
- Never imitate a real song, artist or brand jingle.
|
|
@@ -0,0 +1,100 @@
|
|
|
1
|
+
# Sound in a game
|
|
2
|
+
|
|
3
|
+
What makes a web game sound finished, and the ways a correct-looking sound system goes quiet. Each
|
|
4
|
+
rule below was learned by a game that broke it.
|
|
5
|
+
|
|
6
|
+
## What the game owes a player
|
|
7
|
+
|
|
8
|
+
- **Every action answers with a sound**, on the frame it happens: a press that does something makes
|
|
9
|
+
a sound; a press that is refused makes a different one (`deny`). Movement itself does not need a
|
|
10
|
+
sound per press, but landing, bumping and stopping do.
|
|
11
|
+
- **Sound starts on the first touch**, with no "tap for sound" screen: browsers keep audio suspended
|
|
12
|
+
until a gesture, so resume inside the first `pointerdown`, `touchend` or `keydown` (the player's
|
|
13
|
+
`sound.js`, or the port toolkit, does it). On iOS a held stick is only a gesture when the finger
|
|
14
|
+
lifts, so the first tap or release is when sound begins; the silent switch mutes Web Audio unless
|
|
15
|
+
`navigator.audioSession.type = 'playback'`.
|
|
16
|
+
- **Music carries the round**: calm at the start, a change when it gets tense, something for the last
|
|
17
|
+
ten seconds, a stinger for the result. A round that is all one intensity has no shape; silence
|
|
18
|
+
followed by a hit is worth more than a louder loop.
|
|
19
|
+
- **It sounds good on a phone speaker and on a TV.** A phone plays almost nothing under 300 Hz: every
|
|
20
|
+
important sound needs energy between 500 Hz and 4 kHz (a knock on a thud, an upper harmonic on a
|
|
21
|
+
bass). Measure it: `sound.mjs analyze` gives the loss through a phone-speaker curve.
|
|
22
|
+
- **Nothing clips** when many things happen at once: sounds share a bus that ends in a gentle limiter,
|
|
23
|
+
and each sound is capped at a few voices (`sound.js` keeps four of a name and 24 in all).
|
|
24
|
+
|
|
25
|
+
## Levels that work
|
|
26
|
+
|
|
27
|
+
| what | level |
|
|
28
|
+
| --- | --- |
|
|
29
|
+
| music bed under play | -18 to -20 LUFS (`score --lufs -18`) |
|
|
30
|
+
| title or menu music | -14 LUFS |
|
|
31
|
+
| effects | the four classes: ui -20, small -16, normal -12, big -9 dB (loudest 50 ms) |
|
|
32
|
+
| a whole game while played | -16 to -12 LUFS measured off its output (the playtest's sound row) |
|
|
33
|
+
|
|
34
|
+
Duck the music for the moments that matter (`sound.duck(0.4, 0.5)` for a knockout or a line of
|
|
35
|
+
speech) instead of raising the effect.
|
|
36
|
+
|
|
37
|
+
## Repetition
|
|
38
|
+
|
|
39
|
+
- A sound heard many times a round needs **variants** (three takes a touch apart in pitch) and a
|
|
40
|
+
little random pitch on each play; the same sample fired twenty times in a row reads as a machine gun.
|
|
41
|
+
- Signals the player must learn (start, win, lose, a warning) stay **the same every time**.
|
|
42
|
+
- A long session should not loop one 8-bar clip for ten minutes: make two or three sections and move
|
|
43
|
+
between them with the game's state.
|
|
44
|
+
|
|
45
|
+
## Changing music with the game
|
|
46
|
+
|
|
47
|
+
- Every loop the same tempo and key; switch on the **next bar line**
|
|
48
|
+
(`next = start + ceil((now - start) / bar) * bar` on the audio clock), which `sound.section()` does.
|
|
49
|
+
- Use the audio clock (`AudioContext.currentTime`), not timers, to schedule anything musical. A timer
|
|
50
|
+
or a `requestAnimationFrame` pump skips beats it sees late, and a loaded machine makes it late: a
|
|
51
|
+
bar counter that rides a pump lost three quarters of its sections under load while the audio
|
|
52
|
+
itself played on. Work the bar out from the audio clock each time you need it.
|
|
53
|
+
- A live arrangement's position is `bar % total bars`: changing the total while it plays makes the
|
|
54
|
+
playhead jump. Change the content of a section, not the arrangement's length.
|
|
55
|
+
- A section that plays once as an intro and a lap that repeats are different lengths: compute the lap
|
|
56
|
+
from the repeating part only.
|
|
57
|
+
- A cue that names a sound that does not exist fails silently in most engines: check every name a
|
|
58
|
+
cue or a section uses against what was loaded, once, at start.
|
|
59
|
+
- A gain, a bus or a send changed after a sound was wired may never reach the running graph (a loader
|
|
60
|
+
that skips "already wired" sounds). Set levels before the first play, or rebuild the node.
|
|
61
|
+
- An `AudioContext` clock stalls while the machine is busy (a render, a capture), so a long set drifts
|
|
62
|
+
from wall time: correct its start offset, never its cues.
|
|
63
|
+
- A module served from cache after a change runs the old code: version its URL (`?v=2`) when you
|
|
64
|
+
change it on a live page.
|
|
65
|
+
|
|
66
|
+
## Events, not counts
|
|
67
|
+
|
|
68
|
+
When sounds follow a game's event list (hits, pickups), give each event a rising sequence number and
|
|
69
|
+
play every event newer than the last one played, in order. A list's length is not a signal: an
|
|
70
|
+
expired event replaced at the same length, or two events in one frame, are lost otherwise.
|
|
71
|
+
|
|
72
|
+
## Mixing effects into a running bed
|
|
73
|
+
|
|
74
|
+
Send one-shots (hits, chimes, slams) straight to the final bus, not through the music's filters: a hit
|
|
75
|
+
routed through a lowpass meant for the bed disappears into it. Bright noise belongs on the bright
|
|
76
|
+
one-shots only. Never raise the master to make hits stand out; lower the bed or duck it.
|
|
77
|
+
|
|
78
|
+
## Proving it
|
|
79
|
+
|
|
80
|
+
Scheduling a node is not proof that anything is audible. Three instruments, cheapest first:
|
|
81
|
+
|
|
82
|
+
1. `sound.mjs analyze <file>` on each asset: level, start, clipping, phone loss, and its picture.
|
|
83
|
+
2. The `playtest` skill's sound row: the game's real output copied off its graph while it is played;
|
|
84
|
+
loudness, clipping, gaps, whether actions answer, whether music ever started.
|
|
85
|
+
3. An offline render of the game's own audio graph (`OfflineAudioContext` in headless Chrome) with a
|
|
86
|
+
clock you can set: it finds NaN voices (a NaN anywhere silences the whole graph) and feedback loops
|
|
87
|
+
that double their gain, without playing anything out loud. `currentTime` stays 0 offline, so give
|
|
88
|
+
the game's audio a clock the render can drive.
|
|
89
|
+
|
|
90
|
+
## A log for a recorder
|
|
91
|
+
|
|
92
|
+
`sound.js` keeps a log of everything it schedules when a recorder asks for one, and only then: the
|
|
93
|
+
recorder sets `window.__homieSoundCapture = { events: [] }` before the page's scripts run, and each
|
|
94
|
+
`play`, each music loop, each stop, duck and fader move is pushed onto `events` with `performance.now()`
|
|
95
|
+
and what is needed to mix it again (the file's address, gain, playback rate, pan, loop, delay). The
|
|
96
|
+
head of `sound.js` lists the events. The `video` skill's `trailer` films the game frame by frame and
|
|
97
|
+
rebuilds its sound from that log and the game's own files. A game with audio code of its own can push
|
|
98
|
+
the same events, and only what is in the log is in the trailer's sound.
|
|
99
|
+
|
|
100
|
+
A visual check can never hear a game. Report numbers, and look at the spectrogram.
|
|
@@ -0,0 +1,131 @@
|
|
|
1
|
+
# The score language
|
|
2
|
+
|
|
3
|
+
A score is JSON: tempo and key, instruments, sections in whole bars, an arrangement. `sound.mjs
|
|
4
|
+
score` renders it the same way every time (every random choice is seeded), so the score IS the
|
|
5
|
+
music: commit it, and the audio regenerates byte for byte.
|
|
6
|
+
|
|
7
|
+
## A worked example (48 s, three sections, two loops)
|
|
8
|
+
|
|
9
|
+
```json
|
|
10
|
+
{
|
|
11
|
+
"title": "Night Market Theme",
|
|
12
|
+
"bpm": 128, "key": "A minor", "beatsPerBar": 4, "stepsPerBeat": 4, "swing": 0,
|
|
13
|
+
"instruments": {
|
|
14
|
+
"lead": { "preset": "pulse-lead", "db": -9, "pan": 0.1, "stepsPerBeat": 2, "echo": { "beats": 0.75, "feedback": 0.3, "mix": 0.2 }, "reverb": 0.15 },
|
|
15
|
+
"arp": { "preset": "pluck", "db": -15, "pan": -0.35, "reverb": 0.2 },
|
|
16
|
+
"bass": { "preset": "saw-bass", "db": -10, "duck": 0.35, "reverb": 0 },
|
|
17
|
+
"pad": { "preset": "pad", "db": -17, "duck": 0.5, "reverb": 0.3 },
|
|
18
|
+
"drums": { "preset": "kit", "db": -7, "reverb": 0.05 }
|
|
19
|
+
},
|
|
20
|
+
"sections": [
|
|
21
|
+
{ "name": "intro", "bars": 4, "chords": "Am F C G",
|
|
22
|
+
"play": { "pad": "chords", "arp": "arp:16", "drums": "hats" } },
|
|
23
|
+
{ "name": "main", "bars": 8, "chords": "Am F C G",
|
|
24
|
+
"play": {
|
|
25
|
+
"lead": "A4 . C5 . E5 - D5 C5 | B4 . G4 . B4*2 C5 D5 | E5 - . C5 A4 . C5 D5 | E5*3 . G5*2 E5*2 | F5 . E5 D5 C5 - A4 . | C5 . B4 . A4*2 G4*2 | A4 . C5 . E5 . A5 - | G5*4 E5*4",
|
|
26
|
+
"bass": "root8", "pad": "chords", "arp": "arp:16", "drums": "four" } },
|
|
27
|
+
{ "name": "chase", "bars": 4, "chords": "Dm E Am Am",
|
|
28
|
+
"play": {
|
|
29
|
+
"lead": { "notes": "D5 E5 F5 A5 | G#5*2 E5*2 | A5 E5 C5 A4 | E5*4", "stepsPerBeat": 1 },
|
|
30
|
+
"bass": "pulse", "arp": "arp-updown:16", "drums": "break+fill" } }
|
|
31
|
+
],
|
|
32
|
+
"arrangement": ["intro", "main", "chase", "main"],
|
|
33
|
+
"loops": ["main", "chase"],
|
|
34
|
+
"master": { "lufs": -14 }
|
|
35
|
+
}
|
|
36
|
+
```
|
|
37
|
+
|
|
38
|
+
## The top
|
|
39
|
+
|
|
40
|
+
| field | meaning |
|
|
41
|
+
| --- | --- |
|
|
42
|
+
| `bpm` | 40 to 240 |
|
|
43
|
+
| `beatsPerBar` | 4 by default (3 for a waltz) |
|
|
44
|
+
| `stepsPerBeat` | the grid of a notes pattern: 4 = sixteenth notes (default), 2 = eighths, 1 = quarters. An instrument or a single pattern can set its own |
|
|
45
|
+
| `swing` | 0 to 0.5: every second step comes that fraction of a step late (0.2 to 0.33 is a shuffle) |
|
|
46
|
+
| `arrangement` | section names in playing order; a section can repeat. Default: the sections in order |
|
|
47
|
+
| `loops` | sections to render as seamless loops for the game (`--loops` overrides; `all` for every one) |
|
|
48
|
+
| `master.lufs` | -14 for a page or a trailer, -18 to -20 for a bed under game effects (`--lufs` overrides) |
|
|
49
|
+
| `room` | the shared reverb's size, 0 to 1 (0.55 by default) |
|
|
50
|
+
|
|
51
|
+
## Instruments
|
|
52
|
+
|
|
53
|
+
`{ "preset", "db", "pan", "octave", "legato", "echo", "reverb", "duck", "stepsPerBeat", "tone" }`
|
|
54
|
+
|
|
55
|
+
- `db` the level (-10 by default). Start the lead around -9, drums -7, bass -10, pads -17, and move
|
|
56
|
+
one at a time while listening to the numbers (`analyze` a stem).
|
|
57
|
+
- `pan` -1 (left) to 1 (right). Keep the bass and kick in the middle.
|
|
58
|
+
- `octave` shifts every note of the instrument by octaves.
|
|
59
|
+
- `legato` how much of each step a note holds (0.92 by default; 0.5 is staccato).
|
|
60
|
+
- `echo` `{ beats, feedback, mix, pingpong }`: a tempo-synced echo (0.75 beats is the dotted eighth).
|
|
61
|
+
- `reverb` the send to the shared room, 0 to 1.
|
|
62
|
+
- `duck` 0 to 0.95: the instrument dips on every kick and comes back within half a beat (the pump
|
|
63
|
+
that makes a pad breathe with the drums). Only when the section has a kick.
|
|
64
|
+
- `tone` (drums) shifts the kit's pitch in semitones.
|
|
65
|
+
|
|
66
|
+
Presets (`sound.mjs presets` lists them with what each is for): leads `pulse-lead`, `square-lead`,
|
|
67
|
+
`saw-lead`, `soft-lead`, `bell`; keys `epiano`, `pluck`, `organ`; pads `pad`, `warm-pad`; basses
|
|
68
|
+
`saw-bass`, `square-bass`, `triangle-bass`, `sub-bass`; drums `kit`, `chip-kit`.
|
|
69
|
+
|
|
70
|
+
## Chords
|
|
71
|
+
|
|
72
|
+
`"chords": "Am F C G"` is one chord a bar, repeated to fill the section. With bars marked,
|
|
73
|
+
`"Am F | C G | Dm | E7"` puts two chords in a bar when you write two. Qualities: major (`C`),
|
|
74
|
+
`m`, `7`, `maj7`, `m7`, `m9`, `9`, `add9`, `6`, `m6`, `5`, `dim`, `dim7`, `m7b5`, `aug`, `sus2`,
|
|
75
|
+
`sus4`, `7sus4`; a slash puts another note in the bass (`C/E`). Sharps and flats: `F#m`, `Bb`.
|
|
76
|
+
|
|
77
|
+
## Patterns
|
|
78
|
+
|
|
79
|
+
A pattern is either generated from the chords, or written note by note.
|
|
80
|
+
|
|
81
|
+
**From the chords**: `root` (the root held for the chord), `root8` (eighth-note roots; `root8:16`
|
|
82
|
+
for sixteenths), `pulse` (sixteenth-note roots, a driving bass), `octaves` (root and its octave in
|
|
83
|
+
turn), `chords` (the chord held: a pad), `stabs` (short chords on the off-beats), `arp`, `arp-down`,
|
|
84
|
+
`arp-updown` (the chord's notes one at a time; `:16` sixteenths by default, `:8` eighths). The bass
|
|
85
|
+
plays low, pads in the middle, arpeggios an octave up; `octave` moves any of them.
|
|
86
|
+
|
|
87
|
+
**Notes**: one token per step of the pattern's grid.
|
|
88
|
+
|
|
89
|
+
| token | meaning |
|
|
90
|
+
| --- | --- |
|
|
91
|
+
| `A4`, `C#5`, `Bb3` | a note (A4 = 440 Hz; octaves change at C) |
|
|
92
|
+
| `.` | a rest |
|
|
93
|
+
| `-` | hold the previous note one more step |
|
|
94
|
+
| `A4*3`, `.*4` | a note or rest lasting 3 (or 4) steps |
|
|
95
|
+
| `[A3 C4 E4]` | a chord |
|
|
96
|
+
| `A4!` | an accent (louder); with a length: `A4!*2` |
|
|
97
|
+
| a vertical bar | a bar line: optional, but when you write them every bar must be full, which catches a miscounted bar |
|
|
98
|
+
|
|
99
|
+
A pattern shorter than its section repeats (a one-bar riff over eight bars). With the default grid a
|
|
100
|
+
bar of 4/4 is 16 tokens; for a melody in eighth notes give the instrument `"stepsPerBeat": 2` (8
|
|
101
|
+
tokens a bar), and a single section can differ with `{ "notes": "…", "stepsPerBeat": 1 }`.
|
|
102
|
+
|
|
103
|
+
**Drums**: a groove by name, `four` (four on the floor), `backbeat`, `half` (half-time), `break`,
|
|
104
|
+
`shuffle`, `hats`, `pulse` (kick only), `none`; `+fill` puts a snare and tom fill in the section's last
|
|
105
|
+
bar (`"break+fill"`). Or write the bar yourself, one line per drum, `x` a hit, `X` an accent, `.`
|
|
106
|
+
nothing, 16 cells a bar in 4/4 (the line repeats every bar):
|
|
107
|
+
|
|
108
|
+
```json
|
|
109
|
+
"drums": { "kick": "x.....x...x.....", "snare": "....x.......x...", "hat": "x.x.x.x.x.x.x.x.", "openhat": "..............x.", "fill": true }
|
|
110
|
+
```
|
|
111
|
+
|
|
112
|
+
Drums: `kick`, `snare`, `clap`, `hat`, `openhat`, `tom`, `rim`, `shaker`.
|
|
113
|
+
|
|
114
|
+
## What makes it good
|
|
115
|
+
|
|
116
|
+
- **A hook in the first bars.** The main melody should be singable: mostly steps, one leap, a note
|
|
117
|
+
held at the end of a phrase. Repeat it; change the last bar of the repeat.
|
|
118
|
+
- **Contrast between sections**: the calm section drops the drums or the bass; the chase doubles
|
|
119
|
+
the bass rhythm and adds the break; the final seconds go up a key or add the open hat.
|
|
120
|
+
- **Space for the game's effects.** A bed at -18 LUFS with the lead an octave away from the effects'
|
|
121
|
+
range; `duck` the pad and bass under the kick, and `sound.duck()` the music for big hits.
|
|
122
|
+
- **Loops in 4 or 8 bars**, every loop the same tempo and key, so the game can switch on a bar line.
|
|
123
|
+
- Listen with numbers: `analyze` each stem; a stem more than 12 dB under its neighbours is inaudible
|
|
124
|
+
in the mix, and a struck sound (a pluck, a bell) sits about 9 dB under a held one at the same peak,
|
|
125
|
+
so lengthen its decay instead of pushing its level.
|
|
126
|
+
|
|
127
|
+
## Errors that name the fix
|
|
128
|
+
|
|
129
|
+
A bar with the wrong number of steps, an unknown chord quality, a section that plays an instrument not
|
|
130
|
+
in `instruments`, an arrangement naming a missing section: each stops the render with a sentence
|
|
131
|
+
saying what and where. Fix that and render again; a render is free.
|
|
@@ -0,0 +1,93 @@
|
|
|
1
|
+
# Traps: ffmpeg, expressions and synthesis
|
|
2
|
+
|
|
3
|
+
Each of these produced silence, the wrong level or a click while every command exited 0.
|
|
4
|
+
|
|
5
|
+
## ffmpeg
|
|
6
|
+
|
|
7
|
+
- **Exit code 0 is not audio.** A graph that failed part-way still writes a file whose header promises
|
|
8
|
+
sound. Read it back: `ffprobe -v error -show_entries format=duration:stream=sample_rate,channels -of json <file>`,
|
|
9
|
+
and measure it (`sound.mjs analyze`). A duration of 0, or one far from what you asked for, is a
|
|
10
|
+
failed render with the right name.
|
|
11
|
+
- **`c=2` on a source makes one channel.** `aevalsrc` and friends want a layout name (`c=stereo`,
|
|
12
|
+
`c=mono`) or `2c`; a bare number is read as a channel mask. Ask for the count on the output
|
|
13
|
+
(`-ac 2 -ar 48000`) and the source cannot be wrong about it.
|
|
14
|
+
- **`amix` divides by the number of inputs** unless `normalize=0`: the levels you asked for never
|
|
15
|
+
arrive and nothing says so. (`sound.mjs mix` sums in JavaScript at the levels written.)
|
|
16
|
+
- **`adelay` takes one value per channel** (`adelay=500|500`); a single value delays the left channel only.
|
|
17
|
+
- **mp3 pads both ends with silence**: an effect starts late and a loop gaps at the wrap. Effects and
|
|
18
|
+
loops are WAV or Ogg; mp3 is for a page's player only.
|
|
19
|
+
- **`<audio loop>` gaps at the wrap** in every browser: loop a decoded buffer in Web Audio
|
|
20
|
+
(`source.loop = true`).
|
|
21
|
+
- There may be no `drawtext` in the user's ffmpeg build (it needs freetype): draw labels in a browser
|
|
22
|
+
or an image instead.
|
|
23
|
+
- `loudnorm` needs 400 ms blocks: integrated LUFS of a 0.2 s effect is meaningless. Level short
|
|
24
|
+
effects by their loudest 50 ms (what `sound.mjs` does).
|
|
25
|
+
- Encoders differ: many ffmpeg builds have no `libvorbis`; `libopus` in an `.ogg` is the fallback.
|
|
26
|
+
Older Safari cannot decode Ogg at all, so ship a WAV beside it.
|
|
27
|
+
|
|
28
|
+
## aevalsrc expressions
|
|
29
|
+
|
|
30
|
+
- **No comparison operators.** `t>=1` fails with a misleading "Missing ')'". Use `gte(t,1)`,
|
|
31
|
+
`lte()`, `gt()`, `lt()`, `between(t,a,b)`. There is no `sqr()`: `pow(x,2)`.
|
|
32
|
+
- **`0*inf` is NaN and a NaN silences the whole sum.** An envelope like `exp(-(t-a)*k)*gte(t,a)`
|
|
33
|
+
overflows before the gate: write `exp(-max(0,t-a)*k)`.
|
|
34
|
+
- **`random(x)` keeps its state in variable slot x** and overwrites it; put every `random` in one slot
|
|
35
|
+
(9) and keep `st()/ld()` state in 0..7. There is one random stream: to decorrelate a right channel,
|
|
36
|
+
advance it (`0*random(9)+…`). The first use of a slot must be its `st()`.
|
|
37
|
+
- Quotes, backslashes, semicolons and brackets belong to ffmpeg's graph parser, not the expression;
|
|
38
|
+
keep expressions to arithmetic and functions. Very long expressions (a few thousand characters)
|
|
39
|
+
are fragile: split the terms across parts, they sum to the same thing.
|
|
40
|
+
- `PI` may be missing: write `6.28318` for 2π.
|
|
41
|
+
- One expression renders mono; two (left, right) render true stereo.
|
|
42
|
+
|
|
43
|
+
## Pitch and phase
|
|
44
|
+
|
|
45
|
+
- **In `synth.mjs` a number is hertz, always.** `freq: 110` is 110 Hz and `slide.to: 45` is 45 Hz.
|
|
46
|
+
A note is a name (`freq: "A2"`) or a MIDI number said out loud (`midi: 45`, `slide.toMidi: 45`,
|
|
47
|
+
or `hzOfMidi(45)`). There is no guessing by size: a number of 127 or less used to be read as a
|
|
48
|
+
MIDI note, and every bass note is a frequency under 127 Hz, so a 110 Hz bass came out at 4.7 kHz.
|
|
49
|
+
- **A pitch bend is a change of frequency, integrated into phase.** `sin(2*PI*f(t)*t)` with a
|
|
50
|
+
changing `f` sweeps at twice the intended rate and runs backwards on the way down; the phase must be
|
|
51
|
+
the integral of the frequency (what `synth.mjs` does every sample). A scoop from below in closed
|
|
52
|
+
form is `+2π·df·τ·exp(-(t-a)/τ)` added to the phase.
|
|
53
|
+
- Vibrato depth scales with pitch: ±0.4% of the frequency is about ±7 cents at any note.
|
|
54
|
+
- A tremolo on one note is one voice whose envelope clock wraps (`mod(max(0,t-a)*rate,1)/rate`),
|
|
55
|
+
with the phase left running; restarting the phase each time clicks.
|
|
56
|
+
- An uneven pulse wave (duty other than 50%) has a DC offset: subtract `2·duty - 1`, or high-pass the
|
|
57
|
+
result, or it clicks at every start and stop.
|
|
58
|
+
|
|
59
|
+
## Loops and width
|
|
60
|
+
|
|
61
|
+
- **A held sound loops without a click only if every frequency in it, beating pairs included, is a
|
|
62
|
+
whole multiple of 1/loop-length** (0.125 Hz for an 8 s loop) and no channel starts at a phase
|
|
63
|
+
offset. Notes whose envelope closes before the loop point are exempt. Or render the section three
|
|
64
|
+
times and keep the middle (what `sound.mjs score` does).
|
|
65
|
+
- **Limit a loop before cutting it, not after.** A limiter run over the cut loop starts at full gain
|
|
66
|
+
and ends wherever the last bar left it: a step of gain at the wrap. `score` limits across the
|
|
67
|
+
three copies. A note, an echo or a reverb longer than the whole section still cannot repeat: the
|
|
68
|
+
seam goes over 1 and the report warns; make the section longer or the tail shorter.
|
|
69
|
+
- **A 1-3 ms delay between channels widens a transient but cancels a held tone** in mono (a phone
|
|
70
|
+
speaker folds to mono): correlation fell to -0.2 and the note lost 4 dB. Widen held sounds by
|
|
71
|
+
detuning or by panning different notes, never by a channel offset; offset only the attacks.
|
|
72
|
+
- Measure width with the stereo correlation and the mono loss (`analyze` gives both). Negative
|
|
73
|
+
correlation means parts cancel on a phone.
|
|
74
|
+
|
|
75
|
+
## Levels
|
|
76
|
+
|
|
77
|
+
- A struck sound (a pluck, a bell, a mallet) sits about 9 dB RMS under a held one at the same peak.
|
|
78
|
+
Lengthen its decay rather than raising its fader, then compare stems with their gains applied.
|
|
79
|
+
- A generated room tone or ambience can carry a steady hum (a narrow line in the spectrum): `analyze`
|
|
80
|
+
lists narrow tones; notch them or regenerate. Each one comes with `prominenceDb` (over its
|
|
81
|
+
neighbours, which a filter does not change: the neighbours fall with it), `levelDb` (how loud it
|
|
82
|
+
is, dBFS), `energyShare` and `presentShare` (the share of the file it is there). Only a tone that
|
|
83
|
+
is prominent, above -60 dBFS and there 60% of the time or more is called steady; a melody's brief
|
|
84
|
+
high partial is listed and not warned about. Read its level and share to judge it.
|
|
85
|
+
- Check every render at its head and tail as well as its peak: silence at the start of an effect is
|
|
86
|
+
latency; a tail cut mid-ring is a click.
|
|
87
|
+
|
|
88
|
+
## Provider renders (when a model made the audio)
|
|
89
|
+
|
|
90
|
+
- A paid render can land minutes after the command that asked for it timed out or failed. Before
|
|
91
|
+
asking again, list the output folder by time: the same request twice is twice the bill.
|
|
92
|
+
- Before cutting a straight loop out of a swung track, find the onsets in the rendered file: a hit a
|
|
93
|
+
few ms early is a musical flam, not an error, and triplet grids suit a shuffle.
|