@typetorch/framework 0.3.0 → 0.3.4
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +72 -18
- package/out/analytics/SCHEMA.md +24 -8
- package/out/analytics/engine.d.ts +8 -2
- package/out/analytics/engine.luau +31 -26
- package/out/analytics/protocol.d.ts +4 -1
- package/out/analytics/protocol.luau +6 -3
- package/out/analytics/schema.d.ts +32 -0
- package/out/analytics/schema.luau +10 -0
- package/out/analytics/server.d.ts +21 -0
- package/out/analytics/server.luau +263 -62
- package/out/analytics/settings.d.ts +10 -0
- package/out/analytics/settings.luau +85 -0
- package/out/analytics/sinks.d.ts +11 -4
- package/out/analytics/sinks.luau +35 -3
- package/out/devtools/ab.d.ts +6 -6
- package/out/devtools/ab.luau +5 -5
- package/out/devtools/admin-server.d.ts +18 -13
- package/out/devtools/admin-server.luau +55 -66
- package/out/devtools/admin-ui.d.ts +2 -2
- package/out/devtools/admin-ui.luau +794 -300
- package/out/devtools/assets-ui.d.ts +10 -0
- package/out/devtools/assets-ui.luau +51 -26
- package/out/devtools/build-actions.d.ts +102 -0
- package/out/devtools/build-actions.luau +272 -0
- package/out/devtools/claude.luau +111 -8
- package/out/devtools/client.luau +965 -327
- package/out/devtools/feeds.d.ts +53 -0
- package/out/devtools/feeds.luau +200 -0
- package/out/devtools/health.d.ts +22 -2
- package/out/devtools/health.luau +211 -33
- package/out/devtools/layout.d.ts +142 -0
- package/out/devtools/layout.luau +766 -0
- package/out/devtools/network-inspector.d.ts +2 -0
- package/out/devtools/network-inspector.luau +302 -167
- package/out/devtools/panes.d.ts +60 -0
- package/out/devtools/panes.luau +1388 -0
- package/out/devtools/roll-call.d.ts +2 -2
- package/out/devtools/roll-call.luau +4 -4
- package/out/devtools/server.luau +106 -14
- package/out/devtools/state-inspect.d.ts +87 -0
- package/out/devtools/state-inspect.luau +1030 -0
- package/out/devtools/state-ui.d.ts +44 -0
- package/out/devtools/state-ui.luau +698 -0
- package/out/devtools/state.d.ts +6 -0
- package/out/devtools/state.luau +62 -3
- package/out/devtools/widgets.d.ts +26 -0
- package/out/devtools/widgets.luau +136 -0
- package/out/kernel.d.ts +167 -13
- package/out/kernel.luau +34 -4
- package/out/typetorch.d.ts +6 -4
- package/out/typetorch.luau +9 -6
- package/out/version.d.ts +5 -0
- package/out/version.luau +11 -0
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -115,13 +115,42 @@ export class CoinService extends Module implements OnStart {
|
|
|
115
115
|
work the same way; see the transformer's README.
|
|
116
116
|
- **UI:** `observeElement(trove, tag, (instance, elementTrove) => ...)`, `isRealFrame`, `popIn` / `popOut` / `bump`
|
|
117
117
|
(UIScale, never Size tweens) and `PopupQueue` (one modal at a time).
|
|
118
|
-
- **Dev menu:** devs (Studio, project members, dev badge) get a DEV button, `Ctrl+Shift+D` and `/tt dev`: artifact
|
|
119
|
-
|
|
120
|
-
|
|
121
|
-
|
|
122
|
-
|
|
123
|
-
|
|
124
|
-
|
|
118
|
+
- **Dev menu:** devs (Studio, project members, dev badge) get a DEV button, `Ctrl+Shift+D` and `/tt dev`: artifact
|
|
119
|
+
(with how long the server and client generations have run, the server's uptime and the build's age), server status,
|
|
120
|
+
logs (server, own client, other players' clients), client and server dex, network stats, module state, a branch and
|
|
121
|
+
build picker (Server > Branch) and a Claude prompt. Prod-channel servers are read-only. The window can be dragged by
|
|
122
|
+
its header and resized from its corner (double-tap the header to reset).
|
|
123
|
+
- **Panes and windows (dev menu):** every page shows in a pane with a small header: tap the title to pick another
|
|
124
|
+
page, or use Split right, Split down, Open in new window and Close. Drag the line between two panes to resize them.
|
|
125
|
+
The sidebar opens pages in the focused pane (the highlighted one; tap a pane to focus it). Floating windows move by
|
|
126
|
+
their title bar and resize from their corner (double-tap the title bar to maximise); their title bar also has
|
|
127
|
+
Minimise, Dock (back into the main panel) and Close. Windows can be split too. The layout, the windows and each
|
|
128
|
+
pane's own settings (for example the open nodes of two Modules > State panes) survive swaps; a fresh join starts
|
|
129
|
+
with one pane. Claude, Server > Branch and the Manage pages open once (picking them again focuses their pane); every
|
|
130
|
+
other page can be open several times. Panes nobody sees (menu closed, window minimised) stop refreshing, and panes
|
|
131
|
+
on the same data share one request (Status, Overview, Assets, Network > Stats, server Logs, Packets); State panes
|
|
132
|
+
share one query budget. On phones and small screens panes only stack (two at most), and windows open maximised with
|
|
133
|
+
one pane each.
|
|
134
|
+
- **State explorer (Modules > State, Server | Client):** every live module of the running generation (services on the
|
|
135
|
+
server, controllers on the client) and the persist store. Tap a row to open it: the module's own fields (not its
|
|
136
|
+
methods), then tables, arrays, Maps and Sets, 100 entries per page (Prev / Next). Each value has a short preview:
|
|
137
|
+
strings (cut), numbers, booleans, nil, datatypes through `tostring`, an Instance as its full path, a Player as its
|
|
138
|
+
name, `function`, `thread`, buffer sizes. Nothing is ever called (raw reads only: no metamethods, no getters); a
|
|
139
|
+
value that is its own ancestor shows as a cycle. A key filter (open nodes stay listed), Refresh and Auto (every 2 s).
|
|
140
|
+
Server state comes from the op `state.inspect` (rate-limited, size-capped): devs on dev-channel servers, owners only
|
|
141
|
+
on prod-effective servers, since server state can hold player data.
|
|
142
|
+
- **Logs > Upload:** sends the logs shown (Server: this server's log ring; Client: your own client's; Others: the
|
|
143
|
+
picked player's client) to the dev PC paired in the Claude tab, which saves them under `<repo>/.typetorch/logs/`.
|
|
144
|
+
No Claude run, no prompt quota. Not paired: "Pair in the Claude tab first".
|
|
145
|
+
- **Switch a live server (kernel 0.3.4):** owners switch any server and load builds on it, public and prod ones too:
|
|
146
|
+
Server > Branch, "Switch" on a branch or "Load here" on a build, then the green Confirm. Everyone stays. These are
|
|
147
|
+
ordinary switches and pins: Reload and Rollback keep working, and going back to prod is Switch on the prod row (its
|
|
148
|
+
verified head). A public switch lasts for that server's lifetime and is never stored, so new servers still boot the
|
|
149
|
+
signed prod head. Others see "Join", which moves only them to a reserved server. Manage > Servers "Load a build..."
|
|
150
|
+
does the same for this server, the ticked servers or a share of one branch (A/B pins), with one status line under
|
|
151
|
+
the list.
|
|
152
|
+
- **Owners and devs:** two roles (no admins: an older kernel's or typetorch.json's "admin" counts as a dev). The Manage
|
|
153
|
+
group (Players, Servers, Bans) is for owners only.
|
|
125
154
|
- **Remote Claude** (dev-channel servers only): while `typetorch remote-claude` (`@typetorch/dev-server`) runs on a dev's machine, allowlisted
|
|
126
155
|
devs prompt Claude Code from the Claude tab. Each dev pairs once by pasting the pairing code printed by
|
|
127
156
|
typetorch-dev-server; the game server keeps the session URL and the tokens in memory and never sends them to clients.
|
|
@@ -155,13 +184,13 @@ this.trove.add(TypeTorch.onUpdatePending((update) => (hint.Visible = !update.can
|
|
|
155
184
|
const scores = TypeTorch.persist("scores.v1", () => new Map<number, number>());
|
|
156
185
|
|
|
157
186
|
// Dev and roles (server: the kernel decides; client: only the local player, cosmetic)
|
|
158
|
-
if (TypeTorch.
|
|
159
|
-
TypeTorch.onPlayerDevChanged((player, info) =>
|
|
187
|
+
if (TypeTorch.isOwner(player)) showOwnerPanel(player);
|
|
188
|
+
TypeTorch.onPlayerDevChanged((player, info) => setDevTools(player, info.dev));
|
|
160
189
|
|
|
161
190
|
// Server only
|
|
162
191
|
TypeTorch.status(); // uptime, players, memory, history: cheap
|
|
163
192
|
TypeTorch.branches(); TypeTorch.artifacts(); // registry reads, cached ~30 s: may yield, don't call per frame
|
|
164
|
-
TypeTorch.requestReload(player); //
|
|
193
|
+
TypeTorch.requestReload(player); // owners only, checked by the kernel
|
|
165
194
|
|
|
166
195
|
// Logs (the kernel's ring buffer)
|
|
167
196
|
TypeTorch.logs(0, 50);
|
|
@@ -279,13 +308,16 @@ const variant = analytics.experiment("onboarding", ["short", "long"]); // same a
|
|
|
279
308
|
and the next generation sends them. In edit mode (UI Labs) it is inert.
|
|
280
309
|
- **Server calls take the player first** (`track(player, ...)`); without one an event is server-only (no player id).
|
|
281
310
|
Client calls are always about the local player and are marked `src = "client"` (a client can lie). The server
|
|
282
|
-
checks their shape, size (props at most 4 KB) and rate.
|
|
311
|
+
checks their shape, size (props at most 4 KB) and rate. `purchase` and `currency` are server-only (revenue and the
|
|
312
|
+
economy are server-authoritative): on the client they warn once and send nothing, and the server refuses them from
|
|
313
|
+
clients.
|
|
283
314
|
- **Options** (all on by default): `sessions` (joins, leaves, device, join source, account age bucket, Premium,
|
|
284
315
|
country, friends in the server, first-ever vs returning, days since the last visit), `tech` (FPS, ping, memory, load
|
|
285
316
|
time, client and server errors, swaps and rollbacks, leaves within 60 s of a swap), `zones` (parts or models tagged
|
|
286
317
|
`TTZone`, named by a `Name` attribute or the instance name), `screens` (ScreenGuis in PlayerGui, GuiObjects tagged
|
|
287
|
-
`TTScreen`), `recording` (the first-ever session in detail), `fleet` (kernel 0.3.2 heartbeats and deploy reports)
|
|
288
|
-
`settings` (server only) replaces the
|
|
318
|
+
`TTScreen`), `recording` (the first-ever session in detail), `fleet` (kernel 0.3.2 heartbeats and deploy reports),
|
|
319
|
+
`identity` (one `{ pid, uid }` row per session to the dev's own server, below). `settings` (server only) replaces the
|
|
320
|
+
ConfigService key, for tests.
|
|
289
321
|
- **Every row** carries the time (server clock), a random player id (never the UserId), the session, the server
|
|
290
322
|
(JobId, type, place), the artifact (id, seq, branch, channel), the device, new vs returning, the player's state
|
|
291
323
|
(`zone:Lobby|screen:Shop|activity:round`) and experiment variants. The exact rows: `src/analytics/SCHEMA.md`.
|
|
@@ -297,16 +329,32 @@ const variant = analytics.experiment("onboarding", ["short", "long"]); // same a
|
|
|
297
329
|
about 10 times a second, every input (never while a TextBox or the chat has focus; text boxes only say which box was
|
|
298
330
|
used), buttons pressed and hovered, screens, prompts and deaths, from the join until 60 s after the first input.
|
|
299
331
|
Packed into small binary chunks (10-20 KB a player) and sent through the server.
|
|
300
|
-
- **Never collected:** chat or anything typed, usernames and display names (error texts have them replaced)
|
|
332
|
+
- **Never collected:** chat or anything typed, usernames and display names (error texts have them replaced). No event
|
|
333
|
+
row carries a UserId.
|
|
334
|
+
- **Identities** (option `identity`, default on): once the pid is known, one row `{ pid, uid, t }` (the UserId, nothing
|
|
335
|
+
else) per session goes to the dev's own server only, never into the events: duckdb games in the batch body, Basin
|
|
336
|
+
games to the fleet API (`TypeTorchFleet` url + `/v1/identity`, or the `identity` / `identityToken` settings). The
|
|
337
|
+
analytics server keeps pid -> UserId in a deletable table (support lookups, Right to Erasure).
|
|
301
338
|
- **Player ids:** DataStore `TypeTorchAnalytics`, key `p/<UserId>` -> `{ pid, first, last }`: one read per join, a
|
|
302
339
|
write on the first join and at leave. Deleting the key (Right to Erasure) leaves that player's rows anonymous.
|
|
303
340
|
- **Sending:** one queue on the server (10,000 events; over that the oldest are dropped and counted), a flush every
|
|
304
341
|
`flushSeconds` (15) or at 500 rows, at most ~10 HttpService requests a minute, retries with backoff, and a last
|
|
305
342
|
flush on shutdown (kernel 0.3.2). Delivery is at least once (a swap mid-request sends that batch again).
|
|
306
343
|
`analytics.stats()` (server) has the counters; `flush()` sends soon.
|
|
307
|
-
|
|
308
|
-
|
|
309
|
-
|
|
344
|
+
- **The cloud test sends nothing.** `typetorch test --cloud` boots the payload headless in a Luau Execution task where
|
|
345
|
+
HttpService works; there the engine (stub kernel `test = true`, or `workspace:GetAttribute("TypeTorchTest")`) collects
|
|
346
|
+
as usual but never uploads: no event rows, no identity rows, no HTTP request, so a prod deploy never puts a fake
|
|
347
|
+
server session into your analytics.
|
|
348
|
+
- **Bounds on what a client can cause:** at most 32 experiments per session (each is stamped on every later row), and
|
|
349
|
+
a client's `exp` messages count against its event budget (120 a minute, 5,000 a session). Clients can't send
|
|
350
|
+
`purchase` or `currency` rows; the analytics server's ingest also refuses client-marked ones (older engines) and its
|
|
351
|
+
revenue queries count `purchase` rows the server sent.
|
|
352
|
+
- **Friends in the server** (the join's `friends`): one `Players:GetFriendsAsync` per join (at most 10 pages, none
|
|
353
|
+
when the player is alone) intersected with the players there, not a web call per player.
|
|
354
|
+
|
|
355
|
+
**Settings** (server only, never sent to clients): the ConfigService key `TypeTorchAnalytics`, written with
|
|
356
|
+
`writeSettings()` from `@typetorch/analytics` (Open Cloud, universe:write) or in Creator Hub (Configs); no CLI command
|
|
357
|
+
writes it. Re-read live every few minutes:
|
|
310
358
|
|
|
311
359
|
```json
|
|
312
360
|
{ "backend": "basin", "events": "https://<stream-id>.ingest.cloudflare.com",
|
|
@@ -335,9 +383,15 @@ bun run build # rbxtsc --type package -> out/
|
|
|
335
383
|
kernel's and the template's `rokit.toml`). `scripts/test-generations.luau` takes a game's built payload:
|
|
336
384
|
`cd ../template && bun run payload && lune run ../framework/scripts/test-generations.luau build/payload.rbxm` boots
|
|
337
385
|
two generations in one VM and checks fresh registries, DI and generated guards, then runs the real `startServer`
|
|
338
|
-
with stub kernels (onStart failures reported to kernel 0.3.2, raised on older ones; onClose)
|
|
386
|
+
with stub kernels (onStart failures reported to kernel 0.3.2, raised on older ones; onClose), the health lines and
|
|
387
|
+
the Branch tab / "Load a build" rules (`devtools/build-actions.ts`).
|
|
339
388
|
To test framework changes before the template takes them, build the payload from a copy of the template whose
|
|
340
389
|
`node_modules/@typetorch/framework/out` is this repo's `out/`.
|
|
390
|
+
`scripts/test-state-inspect.luau` checks the state explorer's core (previews, raw reads only, cycles, paging,
|
|
391
|
+
Maps/Sets with Instance keys, the depth limit, filters, request validation) and that `src/version.ts` matches
|
|
392
|
+
package.json.
|
|
393
|
+
`scripts/test-layout.luau` checks the dev menu's pane layout core (`devtools/layout.ts`: splits and their limits,
|
|
394
|
+
closing, pop out and dock, singleton pages, ratios, focus, and cleaning a remembered layout).
|
|
341
395
|
`scripts/test-analytics.luau` checks the analytics engine's pure parts (experiment assignment, settings, the queue
|
|
342
396
|
and HTTP budget, tt-rec-1, the sink request bodies); `test-analytics-server.luau` and `test-analytics-client.luau`
|
|
343
397
|
run the compiled server and client cores against mocked services (sessions, intake, retries, a swap with a request
|
package/out/analytics/SCHEMA.md
CHANGED
|
@@ -31,7 +31,7 @@ Integers are written with every digit (never exponent notation).
|
|
|
31
31
|
| `state` | string | The player's state **after** this event: `zone:<z>\|screen:<s>\|activity:<a>`, empty parts left out (`""` when nothing is known). Server-only events: `activity:<server activity>` or `""` |
|
|
32
32
|
| `exp` | string | JSON object of the player's active experiments, `{"onboarding":"short"}`; `{}` when none |
|
|
33
33
|
| `sexp` | string | The server's experiment: the artifact id of a kernel A/B experiment pin, or `""` |
|
|
34
|
-
| `src` | string | `server`, or `client` (sent by a client: it can lie; the server checked shapes and rate) |
|
|
34
|
+
| `src` | string | `server`, or `client` (sent by a client: it can lie; the server checked shapes and rate). Never `client` on `purchase` or `currency` rows |
|
|
35
35
|
| `props` | string | JSON object text, at most 4096 bytes. Too large: `{"_trunc":<bytes>}`; not encodable: `{"_err":"encode"}`; a non-object value `x`: `{"value":x}` |
|
|
36
36
|
|
|
37
37
|
### recordings
|
|
@@ -57,7 +57,7 @@ First-ever-session detail, one row per packed chunk.
|
|
|
57
57
|
|
|
58
58
|
| kind | name | props |
|
|
59
59
|
|---|---|---|
|
|
60
|
-
| session | `join` | `from`: `direct` (Home, search, sorts: Roblox doesn't say), `teleport` (same game), `teleport_game` (another game), `referral` (invite/referral link), `share` (launch data), `follow` (followed someone); `ctx?` (GameJoinContext.JoinSource name); `tp?` (had teleport data); `party?` (players teleported along); `age` (account age: `<1d`, `1-7d`, `7-30d`, `30-90d`, `90-365d`, `1-3y`, `3y+`); `prem` (Premium); `country?` (ISO code); `friends` (friends in the server, at most
|
|
60
|
+
| session | `join` | `from`: `direct` (Home, search, sorts: Roblox doesn't say), `teleport` (same game), `teleport_game` (another game), `referral` (invite/referral link), `share` (launch data), `follow` (followed someone); `ctx?` (GameJoinContext.JoinSource name); `tp?` (had teleport data); `party?` (players teleported along); `age` (account age: `<1d`, `1-7d`, `7-30d`, `30-90d`, `90-365d`, `1-3y`, `3y+`); `prem` (Premium); `country?` (ISO code); `friends` (how many of the player's friends are in the server: one friend-list lookup per join, at most 10 pages, `0` when it fails); `ret` (days since the last visit, `-1` first visit or unknown); `late?` (the engine started over 60 s after the join). `t` is the join time |
|
|
61
61
|
| session | `device` (src client) | `input` (`kbm`, `touch`, `gamepad`, `vr`, `unknown`), `w`, `h` (viewport points), `touch`, `kb`, `mouse`, `pad`, `vr`. Once per session |
|
|
62
62
|
| session | `leave` | `secs` (session length), `why`: `left`, `shutdown`, `gone` (left while no generation ran the engine), `swap60?` (left within 60 s of a hot swap) |
|
|
63
63
|
| tech | `server` | Every `techEvery` s: `fps` (physics), `hb` (Heartbeat rate), `mem` (MB, 10s), `players`, `ping50?`, `ping90?` (ms), `q`, `qr` (queued events, recordings), `dropped`, `sent`, `fails`, `health?`, `errors?` (kernel 0.3.2 health) |
|
|
@@ -71,17 +71,21 @@ First-ever-session detail, one row per packed chunk.
|
|
|
71
71
|
| zone | `leave` | `zone`, `to?`, `secs` |
|
|
72
72
|
| state | `activity` | `to`, `from` (server or client src) |
|
|
73
73
|
| state | `screen` | `to`, `from` (`""` = none). Client: ScreenGuis in PlayerGui (Enabled), GuiObjects tagged `TTScreen` (Visible), `screen()` |
|
|
74
|
-
| experiment | `<experiment>` | `variant`, `variants` (count), `forced?`. Once per session when assigned or changed |
|
|
74
|
+
| experiment | `<experiment>` | `variant`, `variants` (count), `forced?`. Once per session when assigned or changed. At most 32 experiments per session: a 33rd `experiment()` returns the control (the first variant) and warns. A client's experiment calls count against its event budget (below) |
|
|
75
75
|
| recording_meta | `start` | `share` (the recorded share then) |
|
|
76
76
|
| recording_meta | `end` | `chunks`, `bytes`, `why`: `window` (60 s after the first input), `cap` (10 min, or the server's 256 KB / 80 chunks), `left`, `end` |
|
|
77
77
|
| fleet | `heartbeat` | The kernel's `fleetStatus()` (kernel 0.3.2+), every 60 ± 10 s and once at each generation start. pid `""` |
|
|
78
78
|
| fleet | `deploy_report` | One kernel deploy report `{s, b, a, j, r, e?, d?, t, g, k, p}` (r: `swapped`, `failed`, `rolled_back`, `skipped`, `booted`), once per (s, j, r) per server, across generations. pid `""` |
|
|
79
79
|
| custom | `<name>` | `track(name, props)`: the game's props |
|
|
80
80
|
| funnel | `<funnel>` | `step(funnel, index, name?)`: `i`, `step?` |
|
|
81
|
-
| purchase | `<kind>` | `purchase({product, robux, where?, kind?})`: name = `kind` (default `product`); `product`, `robux`, `where?` |
|
|
82
|
-
| currency | `<currency>` | `currency(name, delta, reason)`: `delta`, `reason` |
|
|
81
|
+
| purchase | `<kind>` | Server only. `purchase(player?, {product, robux, where?, kind?})`: name = `kind` (default `product`); `product`, `robux`, `where?` |
|
|
82
|
+
| currency | `<currency>` | Server only. `currency(player?, name, delta, reason)`: `delta`, `reason` |
|
|
83
83
|
|
|
84
|
-
Client-sent kinds are limited to `custom`, `funnel`, `
|
|
84
|
+
Client-sent kinds are limited to `custom`, `funnel`, `state`, `tech`, within a budget of 120 a minute and 5,000 a
|
|
85
|
+
session per player. `purchase` and `currency` are server-only (revenue and the economy are server-authoritative): on
|
|
86
|
+
the client `purchase()` and `currency()` warn once and send nothing, the server refuses those kinds from clients, and
|
|
87
|
+
the analytics server's ingest refuses such rows marked `src = client` (older engines let clients send them). Revenue
|
|
88
|
+
counts only server-sent `purchase` rows; rows from before `src` existed (no `src`) still count.
|
|
85
89
|
|
|
86
90
|
## tt-rec-1
|
|
87
91
|
|
|
@@ -150,6 +154,8 @@ ConfigService key `TypeTorchAnalytics` (server-only), a JSON object (or a JSON s
|
|
|
150
154
|
| `recordShare` | 1 | 0..1, share of new players recorded (stable per pid) |
|
|
151
155
|
| `techEvery` | 60 | 15..3600 seconds between tech samples |
|
|
152
156
|
| `experiments` | none | Per experiment: `active: false` (everyone gets the first variant, not stamped), `weights` (per variant, in the game's order), `variant` (force one) |
|
|
157
|
+
| `identity` | `TypeTorchFleet` url + `/v1/identity` | basin: where identity rows go (below); none and no readable `TypeTorchFleet`: not sent. duckdb: unused |
|
|
158
|
+
| `identityToken` | the `TypeTorchFleet` token | basin: the token for `identity`. Never logged |
|
|
153
159
|
|
|
154
160
|
Unknown fields are ignored. Settings errors are warned once (never with the token).
|
|
155
161
|
|
|
@@ -158,7 +164,13 @@ Unknown fields are ignored. Settings errors are warned once (never with the toke
|
|
|
158
164
|
- **basin:** `POST <events>` with a JSON array of event rows and `POST <recordings>` with a JSON array of recording
|
|
159
165
|
rows. `Content-Type: application/json`, `Authorization: Bearer <token>` when set, no compression.
|
|
160
166
|
- **duckdb:** `POST <events>` with `{"events":[...],"recordings":[...]}`, gzip (`Content-Encoding: gzip`, HttpService
|
|
161
|
-
`Compress`), `Content-Type: application/json`, `Authorization: Bearer <token>`.
|
|
167
|
+
`Compress`), `Content-Type: application/json`, `Authorization: Bearer <token>`. With identity rows waiting, the body
|
|
168
|
+
also has `"identities":[...]`.
|
|
169
|
+
- **Identities** (engine option `identity`, default on): once a player's pid is known, one row `{ "pid", "uid", "t" }`
|
|
170
|
+
(the UserId as a number, unix ms; nothing else about the player) per session, so the dev's own server can map pids
|
|
171
|
+
and UserIds (support, Right to Erasure) and delete the link. Never part of the events table. duckdb: in the batch
|
|
172
|
+
body; basin: `POST <identity>` with `{"identities":[...]}` and `Authorization: Bearer <identityToken>` (Basin rows
|
|
173
|
+
can't be deleted, so they never go to Basin). At most 200 per request; 500 wait at most.
|
|
162
174
|
- **Batches:** every `flushSeconds`, or at 500 queued events or 20 recordings. At most 500 event rows (fleet rows
|
|
163
175
|
first) and 50 recording rows per request, about 900 KB each. At most ~10 requests a minute per server (burst 3).
|
|
164
176
|
- **Answers:** 2xx sent; 413 the batch is halved and resent (a single row is dropped); 401/403/404 rows are kept and
|
|
@@ -166,6 +178,9 @@ Unknown fields are ignored. Settings errors are warned once (never with the toke
|
|
|
166
178
|
other 4xx drop the batch (counted as rejected).
|
|
167
179
|
- **Delivery is at least once:** a hot swap during a request sends its rows again. Readers may drop exact duplicate
|
|
168
180
|
rows.
|
|
181
|
+
- **The cloud test sends nothing:** inside `typetorch test --cloud` (the stub kernel's `test = true`, or the workspace
|
|
182
|
+
attribute `TypeTorchTest`) rows are collected as usual but dropped at each flush: no event, recording or identity
|
|
183
|
+
rows and no HTTP request, so a deploy never puts a fake server session into the analytics.
|
|
169
184
|
- **Queue:** 10,000 events (1,000 while no settings are known), 2,000 fleet rows, 300 recording chunks; over a cap the
|
|
170
185
|
oldest are dropped and counted (`tech/server` `dropped`). The queue lives in the kernel's persist store across hot
|
|
171
186
|
swaps and is flushed on shutdown (kernel 0.3.2 onClose; older kernels: one BindToClose relay).
|
|
@@ -174,7 +189,8 @@ Unknown fields are ignored. Settings errors are warned once (never with the toke
|
|
|
174
189
|
|
|
175
190
|
DataStore `TypeTorchAnalytics`, key `p/<UserId>` -> `{ "pid": "<32 hex>", "first": <unix s>, "last": <unix s> }`.
|
|
176
191
|
One read per join (with retries), a write on the first join (UpdateAsync, keeps a pid another server wrote first) and
|
|
177
|
-
one at leave (`last`). A Right to Erasure request deletes the key, which leaves the player's rows anonymous.
|
|
192
|
+
one at leave (`last`). A Right to Erasure request deletes the key, which leaves the player's rows anonymous. The
|
|
193
|
+
analytics server also keeps pid -> UserId from identity rows (above), which its erasure webhook uses and deletes.
|
|
178
194
|
|
|
179
195
|
## Suggested Basin stream schemas
|
|
180
196
|
|
|
@@ -15,10 +15,16 @@ export declare class AnalyticsEngine {
|
|
|
15
15
|
/** A funnel step, e.g. `step("onboarding", 3, "opened_shop")`. Indexes start at 1 by convention. */
|
|
16
16
|
step(funnel: string, index: number, name?: string): void;
|
|
17
17
|
step(player: Player, funnel: string, index: number, name?: string): void;
|
|
18
|
-
/**
|
|
18
|
+
/**
|
|
19
|
+
* A Robux purchase (log it once the receipt is granted). The event name is `kind` ("product" by default). Server
|
|
20
|
+
* only: revenue is server-authoritative, so on the client this warns once and sends nothing.
|
|
21
|
+
*/
|
|
19
22
|
purchase(purchase: AnalyticsPurchase): void;
|
|
20
23
|
purchase(player: Player, purchase: AnalyticsPurchase): void;
|
|
21
|
-
/**
|
|
24
|
+
/**
|
|
25
|
+
* Currency in (+) or out (-), e.g. `currency(player, "coins", 50, "round_reward")`. Server only: the economy is
|
|
26
|
+
* server-authoritative, so on the client this warns once and sends nothing.
|
|
27
|
+
*/
|
|
22
28
|
currency(name: string, delta: number, reason: string): void;
|
|
23
29
|
currency(player: Player, name: string, delta: number, reason: string): void;
|
|
24
30
|
/**
|
|
@@ -26,6 +26,17 @@ local function isPlayer(value)
|
|
|
26
26
|
end
|
|
27
27
|
return _condition
|
|
28
28
|
end
|
|
29
|
+
--* Server-only calls made on the client: warned once each per generation; the server refuses these kinds from clients.
|
|
30
|
+
local warnedServerOnly = {}
|
|
31
|
+
local function serverOnly(call)
|
|
32
|
+
local _call = call
|
|
33
|
+
if warnedServerOnly[_call] ~= nil then
|
|
34
|
+
return nil
|
|
35
|
+
end
|
|
36
|
+
local _call_1 = call
|
|
37
|
+
warnedServerOnly[_call_1] = true
|
|
38
|
+
warn("[src/analytics/engine.ts:31]", `[analytics] {call}() is server-only (revenue and currency are server-authoritative): the client sends nothing`)
|
|
39
|
+
end
|
|
29
40
|
local function currentServer(options)
|
|
30
41
|
if serverCore then
|
|
31
42
|
return serverCore
|
|
@@ -87,11 +98,11 @@ do
|
|
|
87
98
|
local existing = serverCore ~= nil
|
|
88
99
|
self.server = currentServer(options)
|
|
89
100
|
if existing and options.settings ~= nil then
|
|
90
|
-
warn("[src/analytics/engine.ts:
|
|
101
|
+
warn("[src/analytics/engine.ts:75]", "[analytics] an engine already runs: its settings stay")
|
|
91
102
|
end
|
|
92
103
|
else
|
|
93
104
|
if options.settings ~= nil then
|
|
94
|
-
warn("[src/analytics/engine.ts:
|
|
105
|
+
warn("[src/analytics/engine.ts:77]", "[analytics] settings are read on the server only; the client ignores them")
|
|
95
106
|
end
|
|
96
107
|
self.client = currentClient(options)
|
|
97
108
|
end
|
|
@@ -150,6 +161,10 @@ do
|
|
|
150
161
|
end
|
|
151
162
|
end
|
|
152
163
|
function AnalyticsEngine:purchase(first, second)
|
|
164
|
+
if self.client then
|
|
165
|
+
serverOnly("purchase")
|
|
166
|
+
return nil
|
|
167
|
+
end
|
|
153
168
|
local player = if isPlayer(first) then first else nil
|
|
154
169
|
local info = (if player then second else first)
|
|
155
170
|
local _condition = not info
|
|
@@ -162,7 +177,7 @@ do
|
|
|
162
177
|
end
|
|
163
178
|
end
|
|
164
179
|
if _condition then
|
|
165
|
-
warn("[src/analytics/engine.ts:
|
|
180
|
+
warn("[src/analytics/engine.ts:129]", "[analytics] purchase({ product, robux }) needs numbers")
|
|
166
181
|
return nil
|
|
167
182
|
end
|
|
168
183
|
local _kind = info.kind
|
|
@@ -180,37 +195,27 @@ do
|
|
|
180
195
|
if _result ~= nil then
|
|
181
196
|
_result:track(player, "purchase", kind, props)
|
|
182
197
|
end
|
|
183
|
-
local _result_1 = self.client
|
|
184
|
-
if _result_1 ~= nil then
|
|
185
|
-
_result_1:track("purchase", kind, props)
|
|
186
|
-
end
|
|
187
198
|
end
|
|
188
199
|
function AnalyticsEngine:currency(first, second, third, fourth)
|
|
200
|
+
if self.client then
|
|
201
|
+
serverOnly("currency")
|
|
202
|
+
return nil
|
|
203
|
+
end
|
|
189
204
|
if isPlayer(first) then
|
|
190
|
-
local props = {
|
|
191
|
-
delta = third,
|
|
192
|
-
reason = fourth,
|
|
193
|
-
}
|
|
194
205
|
local _result = self.server
|
|
195
206
|
if _result ~= nil then
|
|
196
|
-
_result:track(first, "currency", second,
|
|
197
|
-
|
|
198
|
-
|
|
199
|
-
|
|
200
|
-
_result_1:track("currency", second, props)
|
|
207
|
+
_result:track(first, "currency", second, {
|
|
208
|
+
delta = third,
|
|
209
|
+
reason = fourth,
|
|
210
|
+
})
|
|
201
211
|
end
|
|
202
212
|
else
|
|
203
|
-
local props = {
|
|
204
|
-
delta = second,
|
|
205
|
-
reason = third,
|
|
206
|
-
}
|
|
207
213
|
local _result = self.server
|
|
208
214
|
if _result ~= nil then
|
|
209
|
-
_result:track(nil, "currency", first,
|
|
210
|
-
|
|
211
|
-
|
|
212
|
-
|
|
213
|
-
_result_1:track("currency", first, props)
|
|
215
|
+
_result:track(nil, "currency", first, {
|
|
216
|
+
delta = second,
|
|
217
|
+
reason = third,
|
|
218
|
+
})
|
|
214
219
|
end
|
|
215
220
|
end
|
|
216
221
|
end
|
|
@@ -281,7 +286,7 @@ do
|
|
|
281
286
|
return self.client:experiment(first, variants)
|
|
282
287
|
end
|
|
283
288
|
if self.server then
|
|
284
|
-
warn("[src/analytics/engine.ts:
|
|
289
|
+
warn("[src/analytics/engine.ts:204]", `[analytics] experiment("{first}") on the server needs the player: experiment(player, name, variants)`)
|
|
285
290
|
end
|
|
286
291
|
local _condition = variants[1]
|
|
287
292
|
if _condition == nil then
|
|
@@ -52,7 +52,10 @@ export interface ServerHello {
|
|
|
52
52
|
/** Live experiment overrides by name. */
|
|
53
53
|
exps: Record<string, ExperimentOverride>;
|
|
54
54
|
}
|
|
55
|
-
/**
|
|
55
|
+
/**
|
|
56
|
+
* Kinds a client may send in "ev". Never `purchase` or `currency`: revenue and the economy are server-authoritative
|
|
57
|
+
* (a client could otherwise add fake payers and Robux), so the server refuses them from clients.
|
|
58
|
+
*/
|
|
56
59
|
export declare const CLIENT_KINDS: Set<string>;
|
|
57
60
|
/** A client's hello, checked field by field (it comes from an untrusted client). */
|
|
58
61
|
export declare function cleanHello(value: unknown): HelloInfo | undefined;
|
|
@@ -38,12 +38,15 @@ local INPUTS = {
|
|
|
38
38
|
["vr"] = true,
|
|
39
39
|
["unknown"] = true,
|
|
40
40
|
}
|
|
41
|
-
|
|
41
|
+
--[[
|
|
42
|
+
*
|
|
43
|
+
* Kinds a client may send in "ev". Never `purchase` or `currency`: revenue and the economy are server-authoritative
|
|
44
|
+
* (a client could otherwise add fake payers and Robux), so the server refuses them from clients.
|
|
45
|
+
|
|
46
|
+
]]
|
|
42
47
|
local CLIENT_KINDS = {
|
|
43
48
|
["custom"] = true,
|
|
44
49
|
["funnel"] = true,
|
|
45
|
-
["purchase"] = true,
|
|
46
|
-
["currency"] = true,
|
|
47
50
|
["state"] = true,
|
|
48
51
|
["tech"] = true,
|
|
49
52
|
}
|
|
@@ -7,6 +7,8 @@
|
|
|
7
7
|
export declare const SCHEMA_VERSION = 1;
|
|
8
8
|
/** The ConfigService key (server-only) that holds the sink settings. */
|
|
9
9
|
export declare const SETTINGS_KEY = "TypeTorchAnalytics";
|
|
10
|
+
/** The kernel's fleet API settings `{ url, token }` (server-only ConfigService key); identities can go there. */
|
|
11
|
+
export declare const FLEET_SETTINGS_KEY = "TypeTorchFleet";
|
|
10
12
|
/** The first-session recording codec (column `codec`). */
|
|
11
13
|
export declare const RECORDING_CODEC = "tt-rec-1";
|
|
12
14
|
/** Most bytes of the `props` JSON text. Larger props become `{"_trunc":<bytes>}`. */
|
|
@@ -68,6 +70,18 @@ export interface RecordingRow {
|
|
|
68
70
|
/** Position samples in the chunk. */
|
|
69
71
|
n: number;
|
|
70
72
|
}
|
|
73
|
+
/**
|
|
74
|
+
* Which UserId a pid belongs to, sent once per session when the pid is known. Only to the dev's own server (the
|
|
75
|
+
* analytics server, or the fleet API for Basin games), never into the events table: it lets the dev map pids and
|
|
76
|
+
* UserIds (support, Right to Erasure) and delete the link.
|
|
77
|
+
*/
|
|
78
|
+
export interface IdentityRow {
|
|
79
|
+
pid: string;
|
|
80
|
+
/** The player's UserId. */
|
|
81
|
+
uid: number;
|
|
82
|
+
/** Unix ms. */
|
|
83
|
+
t: number;
|
|
84
|
+
}
|
|
71
85
|
/** Live overrides of one per-player experiment (settings key `experiments`). */
|
|
72
86
|
export interface ExperimentOverride {
|
|
73
87
|
/** false: everyone gets the first variant (control) and the experiment isn't stamped on events. */
|
|
@@ -94,6 +108,14 @@ export interface AnalyticsSettings {
|
|
|
94
108
|
techEvery?: number;
|
|
95
109
|
/** Live experiment overrides by experiment name. */
|
|
96
110
|
experiments?: Record<string, ExperimentOverride>;
|
|
111
|
+
/**
|
|
112
|
+
* basin: where identity rows go (the fleet API's `POST /v1/identity` URL). Default: the `TypeTorchFleet` key's url
|
|
113
|
+
* + /v1/identity when the server can read it; without either, identities aren't sent. duckdb: unused (they go in
|
|
114
|
+
* the batch body).
|
|
115
|
+
*/
|
|
116
|
+
identity?: string;
|
|
117
|
+
/** basin: the token for `identity` (default: the `TypeTorchFleet` token). Never logged. */
|
|
118
|
+
identityToken?: string;
|
|
97
119
|
}
|
|
98
120
|
/** AnalyticsSettings with every default filled in. */
|
|
99
121
|
export interface ResolvedSettings {
|
|
@@ -105,6 +127,8 @@ export interface ResolvedSettings {
|
|
|
105
127
|
recordShare: number;
|
|
106
128
|
techEvery: number;
|
|
107
129
|
experiments: Map<string, ExperimentOverride>;
|
|
130
|
+
identity?: string;
|
|
131
|
+
identityToken?: string;
|
|
108
132
|
}
|
|
109
133
|
export type AnalyticsValue = string | number | boolean;
|
|
110
134
|
/** Event properties: a flat JSON object (nested tables work but are harder to query). */
|
|
@@ -140,6 +164,12 @@ export interface AnalyticsOptions {
|
|
|
140
164
|
* one row per deploy report, kind "fleet". They go before analytics events. Default true.
|
|
141
165
|
*/
|
|
142
166
|
fleet?: boolean;
|
|
167
|
+
/**
|
|
168
|
+
* Server: one identity row `{ pid, uid, t }` per session once the pid is known (UserId and pid, nothing else), so
|
|
169
|
+
* the dev's own server can map them. duckdb: in the batch body (`identities`); basin: to the fleet API (Basin rows
|
|
170
|
+
* can't be deleted). Never part of the events table. Default true.
|
|
171
|
+
*/
|
|
172
|
+
identity?: boolean;
|
|
143
173
|
}
|
|
144
174
|
/** Kernel 0.3.2+ `fleetStatus()`: the server's heartbeat (sent as the props of a "fleet" / "heartbeat" row). */
|
|
145
175
|
export type FleetStatus = {
|
|
@@ -159,6 +189,8 @@ export interface AnalyticsStats {
|
|
|
159
189
|
queuedRecordings: number;
|
|
160
190
|
/** Fleet rows waiting (they go first). */
|
|
161
191
|
queuedFleet: number;
|
|
192
|
+
/** Identity rows waiting (pid -> UserId, for the dev's own server). */
|
|
193
|
+
queuedIdentities: number;
|
|
162
194
|
/** Rows sent since the server started (across swaps). */
|
|
163
195
|
sent: number;
|
|
164
196
|
/** Rows dropped because the queue was full. */
|
|
@@ -10,12 +10,21 @@
|
|
|
10
10
|
local SCHEMA_VERSION = 1
|
|
11
11
|
--* The ConfigService key (server-only) that holds the sink settings.
|
|
12
12
|
local SETTINGS_KEY = "TypeTorchAnalytics"
|
|
13
|
+
--* The kernel's fleet API settings `{ url, token }` (server-only ConfigService key); identities can go there.
|
|
14
|
+
local FLEET_SETTINGS_KEY = "TypeTorchFleet"
|
|
13
15
|
--* The first-session recording codec (column `codec`).
|
|
14
16
|
local RECORDING_CODEC = "tt-rec-1"
|
|
15
17
|
--* Most bytes of the `props` JSON text. Larger props become `{"_trunc":<bytes>}`.
|
|
16
18
|
local PROPS_MAX_BYTES = 4096
|
|
17
19
|
--* One row of the events table. Flat; every column is always present.
|
|
18
20
|
--* One row of the recordings table: one packed chunk of a player's first-ever session.
|
|
21
|
+
--[[
|
|
22
|
+
*
|
|
23
|
+
* Which UserId a pid belongs to, sent once per session when the pid is known. Only to the dev's own server (the
|
|
24
|
+
* analytics server, or the fleet API for Basin games), never into the events table: it lets the dev map pids and
|
|
25
|
+
* UserIds (support, Right to Erasure) and delete the link.
|
|
26
|
+
|
|
27
|
+
]]
|
|
19
28
|
--* Live overrides of one per-player experiment (settings key `experiments`).
|
|
20
29
|
--* The sink settings: ConfigService key `TypeTorchAnalytics` (server-only), or `new AnalyticsEngine({ settings })`.
|
|
21
30
|
--* AnalyticsSettings with every default filled in.
|
|
@@ -27,6 +36,7 @@ local PROPS_MAX_BYTES = 4096
|
|
|
27
36
|
return {
|
|
28
37
|
SCHEMA_VERSION = SCHEMA_VERSION,
|
|
29
38
|
SETTINGS_KEY = SETTINGS_KEY,
|
|
39
|
+
FLEET_SETTINGS_KEY = FLEET_SETTINGS_KEY,
|
|
30
40
|
RECORDING_CODEC = RECORDING_CODEC,
|
|
31
41
|
PROPS_MAX_BYTES = PROPS_MAX_BYTES,
|
|
32
42
|
}
|
|
@@ -63,6 +63,8 @@ export declare class ServerAnalytics {
|
|
|
63
63
|
private settingsErrors;
|
|
64
64
|
/** The first settings read finished (found or not), so experiment overrides are as known as they get. */
|
|
65
65
|
private settingsTried;
|
|
66
|
+
/** The `TypeTorchFleet` value (Basin games send identities to the fleet API). */
|
|
67
|
+
private fleetSettings?;
|
|
66
68
|
private nextFlushAt;
|
|
67
69
|
private backoffUntil;
|
|
68
70
|
private batchRows;
|
|
@@ -72,6 +74,12 @@ export declare class ServerAnalytics {
|
|
|
72
74
|
private readonly identity;
|
|
73
75
|
private sexp;
|
|
74
76
|
private stopped;
|
|
77
|
+
/**
|
|
78
|
+
* `typetorch test --cloud`: the payload boots headless in a Luau Execution task, where HttpService works. The engine
|
|
79
|
+
* then collects as usual but sends nothing (no rows, no identity rows, no HTTP), or every prod deploy would put a
|
|
80
|
+
* fake server session into the owner's analytics. Set by the stub kernel's `test` or the TypeTorchTest attribute.
|
|
81
|
+
*/
|
|
82
|
+
private readonly testMode;
|
|
75
83
|
constructor(options: AnalyticsOptions, scope: GenerationScope, trove: Trove);
|
|
76
84
|
private serverExperiment;
|
|
77
85
|
private stateOf;
|
|
@@ -107,6 +115,12 @@ export declare class ServerAnalytics {
|
|
|
107
115
|
*/
|
|
108
116
|
private resolve;
|
|
109
117
|
private joinSource;
|
|
118
|
+
/**
|
|
119
|
+
* How many of the player's friends are in this server: one `GetFriendsAsync` per join (at most FRIEND_PAGES_MAX
|
|
120
|
+
* pages) intersected with the players here, instead of an IsFriendsWith call per player. Stops early once every
|
|
121
|
+
* other player was found; after each yield it stops with the count so far if the player left or the generation
|
|
122
|
+
* stopped. No call when the player is alone; a failed call counts what it read (0 at first).
|
|
123
|
+
*/
|
|
110
124
|
private friendsHere;
|
|
111
125
|
private emitJoin;
|
|
112
126
|
private onPlayer;
|
|
@@ -115,6 +129,8 @@ export declare class ServerAnalytics {
|
|
|
115
129
|
private recordingAllowed;
|
|
116
130
|
private sendHello;
|
|
117
131
|
private onHello;
|
|
132
|
+
/** One client-sent event (or an "exp" message, which may add a row): within the player's budget and the session cap. */
|
|
133
|
+
private takeClientEvent;
|
|
118
134
|
private onEvents;
|
|
119
135
|
private onChunk;
|
|
120
136
|
private startIntake;
|
|
@@ -128,6 +144,11 @@ export declare class ServerAnalytics {
|
|
|
128
144
|
private post;
|
|
129
145
|
/** Sends one batch per table. `final`: shutdown, ignore the budget. */
|
|
130
146
|
private flushOnce;
|
|
147
|
+
private identityList;
|
|
148
|
+
private queueIdentity;
|
|
149
|
+
private takeIdentities;
|
|
150
|
+
/** Puts identity rows back in front (a failed or unsent request), keeping the cap. */
|
|
151
|
+
private requeueIdentities;
|
|
131
152
|
private startFlushing;
|
|
132
153
|
/** game code's flush(): send now (within the budget), on its own thread. */
|
|
133
154
|
flushSoon(): void;
|