@seriousgaming/sdk-tools 1.11.0 → 1.12.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (192) hide show
  1. package/dist/agent/guidance.md +4 -0
  2. package/dist/{cli-F2D4GGUA.js → cli-2BKEIIA3.js} +66 -0
  3. package/dist/{cli-ZGCG6C22.js → cli-2QGTDHRX.js} +21 -21
  4. package/dist/{cli-SRXDLMXY.js → cli-2SDG3SZ7.js} +21 -21
  5. package/dist/{cli-DH35TZ2Q.js → cli-2VXP34R5.js} +2 -2
  6. package/dist/{cli-P3NUTOPK.js → cli-2W4B6PD5.js} +1 -1
  7. package/dist/{cli-WJVPSVF2.js → cli-2X7DKSXN.js} +24 -24
  8. package/dist/{cli-ATAJSIXH.js → cli-2ZP4N7HK.js} +19 -19
  9. package/dist/{cli-5O7XENQY.js → cli-3GJYZH4C.js} +5 -5
  10. package/dist/{cli-H6PP2MZT.js → cli-3GO3M2MX.js} +17 -13
  11. package/dist/{cli-HMDYG6PS.js → cli-3LDLUNME.js} +17 -17
  12. package/dist/{cli-BZOKZ7LP.js → cli-3WY6L2WP.js} +27 -27
  13. package/dist/{cli-REAM7LSC.js → cli-44MBABRD.js} +25 -25
  14. package/dist/{cli-5TTJWM2R.js → cli-4IO7MKLR.js} +20 -20
  15. package/dist/{cli-HNDFANQG.js → cli-4U2PY7W3.js} +1 -1
  16. package/dist/{cli-WQ7JBZNW.js → cli-4WI4LTBI.js} +2 -2
  17. package/dist/{cli-OTBCA54D.js → cli-55EIGGZK.js} +1 -1
  18. package/dist/{cli-FYPW25VD.js → cli-5AQV6FTL.js} +18 -18
  19. package/dist/{cli-RKPQLB7J.js → cli-5CW7BEU7.js} +3 -3
  20. package/dist/{cli-27TQVLVF.js → cli-5JUVHNZF.js} +12 -12
  21. package/dist/{cli-F3YJKP6D.js → cli-5KERL6HV.js} +9 -4
  22. package/dist/{cli-MSNSXVGF.js → cli-5KSA24NQ.js} +24 -24
  23. package/dist/{cli-5VZBFMCK.js → cli-5PIFO3EN.js} +8 -0
  24. package/dist/{cli-FT3X3D4P.js → cli-5QN2Z43X.js} +20 -20
  25. package/dist/cli-5XSPAA5X.js +69 -0
  26. package/dist/{cli-B2QAJ5RQ.js → cli-62KAYGXO.js} +20 -20
  27. package/dist/{cli-QPV3TYOM.js → cli-6IMV5ZZU.js} +26 -26
  28. package/dist/{cli-UULZSQBV.js → cli-6KWCIAN2.js} +764 -66
  29. package/dist/{cli-DQE7ITIR.js → cli-6NWETIX5.js} +20 -20
  30. package/dist/{cli-UVIURDZC.js → cli-7224T6HG.js} +20 -20
  31. package/dist/{cli-L7F4XKCB.js → cli-774W3MN5.js} +26 -26
  32. package/dist/cli-7BUD3VAD.js +16 -0
  33. package/dist/{cli-XNOAO6DY.js → cli-7KEP57S7.js} +12 -12
  34. package/dist/{cli-5UKWX3IU.js → cli-7OYFJMA6.js} +18 -18
  35. package/dist/{cli-WNHMJM3Z.js → cli-7ZNEDQ3G.js} +20 -20
  36. package/dist/{cli-5RRPUL6V.js → cli-AFK6OUID.js} +12 -12
  37. package/dist/{cli-6P6V2EHY.js → cli-AIYQFWXN.js} +25 -25
  38. package/dist/{cli-ERJNDBDH.js → cli-ARAAEWIQ.js} +2 -2
  39. package/dist/cli-AVQYV3N4.js +36 -0
  40. package/dist/{cli-TI2FOFPA.js → cli-B54Z6CXH.js} +16 -16
  41. package/dist/{cli-EEO5ZPAZ.js → cli-B5TC4MEL.js} +5 -5
  42. package/dist/{cli-5Z4D3F6Y.js → cli-BCXGOLU3.js} +3 -3
  43. package/dist/{cli-G2GLW5Z4.js → cli-BHAODVTZ.js} +1 -1
  44. package/dist/{cli-4F24FUW6.js → cli-BPS2JG74.js} +20 -20
  45. package/dist/{cli-W3RG2MRI.js → cli-BVO55SR2.js} +2 -2
  46. package/dist/{cli-R5SNTPND.js → cli-CFH3JZP5.js} +21 -21
  47. package/dist/{cli-Q7QSWNY2.js → cli-CTK2OIK6.js} +20 -20
  48. package/dist/{cli-ATWJCB2G.js → cli-CW5MV4VX.js} +4 -4
  49. package/dist/{cli-TCQJL2GY.js → cli-D5VOSJZ7.js} +1 -1
  50. package/dist/{cli-KIZTASLL.js → cli-DAEPMK7A.js} +5 -5
  51. package/dist/{cli-44CM2AH2.js → cli-DAHR7PKB.js} +1 -1
  52. package/dist/{cli-XW3JLRSA.js → cli-DWDSBTYL.js} +1 -1
  53. package/dist/cli-DYSDPGXW.js +95 -0
  54. package/dist/{cli-QEFYDXMH.js → cli-DZCLRZXJ.js} +1 -1
  55. package/dist/{cli-TEBV2OIW.js → cli-EG2VP7IV.js} +116 -27
  56. package/dist/{cli-77T4LJQF.js → cli-EJ47IE45.js} +24 -24
  57. package/dist/{cli-QDBYFITU.js → cli-EK4OSJHA.js} +2 -2
  58. package/dist/{cli-GO64NABP.js → cli-FCJHNXDA.js} +1 -1
  59. package/dist/{cli-ZNQBF4Z3.js → cli-FNESSE3L.js} +1 -1
  60. package/dist/{cli-D3S7NJZ3.js → cli-FTLH4WCE.js} +17 -17
  61. package/dist/{cli-JHLWBRXB.js → cli-FYV77YS2.js} +18 -18
  62. package/dist/cli-GMJX6Q4Q.js +36 -0
  63. package/dist/{cli-7JUNA74L.js → cli-GRBVNGDW.js} +1 -1
  64. package/dist/{cli-XKAIDYIE.js → cli-HAFCFY77.js} +33 -19
  65. package/dist/{cli-OXLOGXT3.js → cli-HCJLYI24.js} +3 -3
  66. package/dist/{cli-XOYUTNER.js → cli-HCXS5EXU.js} +25 -25
  67. package/dist/cli-HEUNNHDH.js +36 -0
  68. package/dist/{cli-G3TU6CP6.js → cli-HNUTIWDZ.js} +19 -19
  69. package/dist/{cli-JXQLHJO2.js → cli-HXT25ETJ.js} +244 -243
  70. package/dist/{cli-5EH4WTH4.js → cli-IGEL7UJQ.js} +5 -5
  71. package/dist/{cli-TLSTSZAC.js → cli-IO32YA5N.js} +26 -26
  72. package/dist/cli-IPTGZDBB.js +36 -0
  73. package/dist/{cli-OBLEROSC.js → cli-J25TYFKB.js} +19 -19
  74. package/dist/{cli-5ATO27Z4.js → cli-J7XYMEMX.js} +20 -20
  75. package/dist/{cli-X3TSTTYI.js → cli-JC5OJRID.js} +2 -2
  76. package/dist/{cli-25SJZI7N.js → cli-JD563OF4.js} +126 -27
  77. package/dist/{cli-VXLHOZSO.js → cli-JRJF2WHU.js} +20 -20
  78. package/dist/{cli-UQC46YSV.js → cli-JTFLIMIG.js} +3 -3
  79. package/dist/{cli-K4HSQWXC.js → cli-K2BSJ2PK.js} +2 -2
  80. package/dist/{cli-OM32UURE.js → cli-K4QTYMN2.js} +25 -25
  81. package/dist/{cli-BJFYGVJV.js → cli-KFJDD4C4.js} +20 -20
  82. package/dist/{cli-WW54THXN.js → cli-KVQ2JPVD.js} +1 -1
  83. package/dist/{cli-LHMRUJVN.js → cli-L6AHMW2Z.js} +1 -1
  84. package/dist/{cli-IKSLPAZN.js → cli-LCYUBCVY.js} +4 -4
  85. package/dist/{cli-HGCOEL5T.js → cli-LEWSCDH5.js} +4 -4
  86. package/dist/{cli-XJ7JESC2.js → cli-LI6G7GND.js} +27 -27
  87. package/dist/{cli-OMTHTWYJ.js → cli-LU2GCWIU.js} +21 -21
  88. package/dist/{cli-UEQMK4WX.js → cli-M3Z2KOEB.js} +1 -1
  89. package/dist/{cli-226ZJ72R.js → cli-M6E3E3I2.js} +170 -3
  90. package/dist/{cli-NKKCDO6K.js → cli-MCRPA4Z6.js} +20 -20
  91. package/dist/{cli-BQ36DV5C.js → cli-MI2DEDMA.js} +3 -3
  92. package/dist/{cli-6O7YCK2M.js → cli-MITZ6SE7.js} +3 -3
  93. package/dist/{cli-LPFSJ625.js → cli-MTIYD3BX.js} +26 -26
  94. package/dist/{cli-5M6KDNVX.js → cli-MTZUJGBK.js} +2 -2
  95. package/dist/{cli-GSOV26O5.js → cli-NCLMUXLC.js} +14 -14
  96. package/dist/{cli-X52TXCNF.js → cli-NLS3DKWH.js} +26 -26
  97. package/dist/{cli-AH6AMAIB.js → cli-NMDGS7IE.js} +1 -1
  98. package/dist/cli-O3DJO2CS.js +30 -0
  99. package/dist/{cli-BC67QHD5.js → cli-O47D5BFN.js} +2 -2
  100. package/dist/{cli-MDVC5ZNM.js → cli-OBRZ36VP.js} +14 -14
  101. package/dist/{cli-C32Q5E5I.js → cli-P6V3NSU4.js} +1 -1
  102. package/dist/{cli-RMD5WSUQ.js → cli-PAMQMTHK.js} +21 -21
  103. package/dist/{cli-WNBP3LOD.js → cli-PKTHOCT3.js} +26 -26
  104. package/dist/cli-QQ432JEN.js +36 -0
  105. package/dist/{cli-YYXAGJQR.js → cli-QVNDPTBU.js} +20 -20
  106. package/dist/{cli-KJA6GLZI.js → cli-R2DFD6ZZ.js} +15 -15
  107. package/dist/{cli-6BLVI2JM.js → cli-RMVJMOWU.js} +111 -7
  108. package/dist/{cli-X64BNBQH.js → cli-RVZH462G.js} +585 -8
  109. package/dist/{cli-6AMK3DYS.js → cli-SEEOQVUL.js} +12 -12
  110. package/dist/{cli-QTCTWYCM.js → cli-SO354HPI.js} +3 -3
  111. package/dist/{cli-RHL4B4H7.js → cli-T76HDH7F.js} +88 -4
  112. package/dist/{cli-ZJW6HQ6N.js → cli-TIS7DWRR.js} +7 -7
  113. package/dist/{cli-6JWDFCUV.js → cli-TY3B6XS7.js} +21 -21
  114. package/dist/{cli-4FJL3KQZ.js → cli-UKEZJFIT.js} +2 -2
  115. package/dist/cli-VQAMW4RF.js +95 -0
  116. package/dist/{cli-ATHATKI5.js → cli-W4LMRVZT.js} +78 -47
  117. package/dist/cli-W7I5KGIB.js +84 -0
  118. package/dist/{cli-S3NHXAJM.js → cli-WCFZNEK2.js} +18 -18
  119. package/dist/{cli-B3EFE7ZF.js → cli-WXGJFA4F.js} +1 -1
  120. package/dist/{cli-PNS4KWWC.js → cli-XCG33I77.js} +24 -24
  121. package/dist/{cli-DBMEH4Y5.js → cli-YHLSG5HB.js} +26 -26
  122. package/dist/cli-YNH5DTHY.js +36 -0
  123. package/dist/cli-YPMJWPDA.js +69 -0
  124. package/dist/{cli-PXQYY6KM.js → cli-YSXIR7EM.js} +0 -0
  125. package/dist/cli-ZPFWO4VW.js +85 -0
  126. package/dist/cli.js +53 -53
  127. package/dist/client.js +1951 -289
  128. package/dist/core/assessment-moves.d.ts +32 -0
  129. package/dist/core/assessment-reports.d.ts +38 -0
  130. package/dist/core/assessment-settings.d.ts +15 -0
  131. package/dist/core/benchmark-samples.d.ts +28 -0
  132. package/dist/core/session-host.d.ts +4 -0
  133. package/dist/core/session-roster.d.ts +6 -0
  134. package/dist/core/session-timers.d.ts +40 -0
  135. package/dist/core/session.d.ts +14 -1
  136. package/dist/core/tick-timer.d.ts +18 -0
  137. package/dist/dev/console/assets/{index-Ch6HHb2T.js → index-zd--5H1f.js} +26 -26
  138. package/dist/dev/console/index.html +1 -1
  139. package/dist/dev/shell.d.ts +21 -0
  140. package/dist/dev/view/commands.d.ts +3 -1
  141. package/dist/docs/api/index.md +1 -1
  142. package/dist/docs/api/sdk-commands.md +29 -0
  143. package/dist/docs/api/sdk-e2e.md +30 -0
  144. package/dist/docs/api/sdk-projection.md +5 -0
  145. package/dist/docs/api/sdk-tools-client.md +30 -0
  146. package/dist/docs/api/sdk-ui.md +7 -0
  147. package/dist/docs/api/sdk.md +834 -19
  148. package/dist/docs/changelog.md +34 -0
  149. package/dist/docs/cli-error-codes.md +5 -1
  150. package/dist/docs/cli-reference.md +76 -2
  151. package/dist/docs/concepts/index.md +2 -0
  152. package/dist/docs/concepts/terminology.md +12 -0
  153. package/dist/docs/guides/assessments.md +248 -0
  154. package/dist/docs/guides/benchmarks.md +241 -0
  155. package/dist/docs/guides/engine.md +8 -0
  156. package/dist/docs/guides/index.md +2 -0
  157. package/dist/docs/guides/presentation.md +11 -9
  158. package/dist/docs/guides/session-hooks.md +24 -22
  159. package/dist/docs/guides/tick-controller.md +22 -1
  160. package/dist/docs/guides/ui.md +24 -0
  161. package/dist/docs/guides/upgrading.md +21 -0
  162. package/dist/docs/page-index.md +16 -12
  163. package/dist/docs/recipes/create-assessment.md +124 -0
  164. package/dist/docs/recipes/create-benchmark.md +115 -0
  165. package/dist/docs/recipes/index.md +2 -0
  166. package/dist/execution/context-composition.d.ts +39 -4
  167. package/dist/execution/engine-state.d.ts +13 -0
  168. package/dist/execution/executor.d.ts +16 -0
  169. package/dist/execution/logic-result.d.ts +13 -2
  170. package/dist/help-examples.d.ts +12 -0
  171. package/dist/host/adapter.d.ts +24 -1
  172. package/dist/host/protocol.d.ts +115 -1
  173. package/dist/host.js +29 -1
  174. package/dist/playwright.js +1945 -283
  175. package/dist/runtime/index.d.ts +3 -1
  176. package/dist/runtime/manifest-version.d.ts +1 -1
  177. package/dist/runtime.js +298 -16
  178. package/dist/schema.json +189 -0
  179. package/dist/templating/resource-metadata.d.ts +1 -1
  180. package/dist/utils/error-codes.d.ts +9 -1
  181. package/dist/validation/assessment-validation.d.ts +34 -0
  182. package/dist/validation/benchmark-validation.d.ts +20 -0
  183. package/dist/validation/schema-validation.d.ts +4 -0
  184. package/package.json +3 -3
  185. package/dist/cli-27Z5T52R.js +0 -30
  186. package/dist/cli-3VLBFWHN.js +0 -36
  187. package/dist/cli-BINVO5R7.js +0 -36
  188. package/dist/cli-HLI5UEKX.js +0 -36
  189. package/dist/cli-OF36EA62.js +0 -36
  190. package/dist/cli-OVIJRBFI.js +0 -36
  191. package/dist/cli-UIYKPSIY.js +0 -36
  192. package/dist/cli-WZNIW6ZO.js +0 -16
@@ -0,0 +1,241 @@
1
+ # Benchmarks
2
+
3
+ A benchmark lets a debrief compare a team with everyone who has played the game, not just the other teams in the room.
4
+
5
+ This guide explains what a benchmark records and who sees it, then covers declaring one, writing it from the engine, and reading the pooled figures on a screen.
6
+
7
+ **Keywords:** benchmark, benchmarks, compare with other sessions, how you compare, debrief comparison, pooled results, percentile, median, getBenchmark, sample.json, tags, filters, requires benchmarks, BENCHMARK_UNWRITTEN
8
+
9
+ ## The model
10
+
11
+ A debrief can only compare a team with the other teams in its room. With one team there is nothing to compare against, and even a full room wants to know how it did against everyone who has played.
12
+
13
+ A benchmark is a small, named record of outcomes and decisions that the engine keeps for each team (or for the session). The platform pools those records anonymously across sessions of the same game, and a screen reads back statistics: a count, a median, percentiles, or how many chose each option.
14
+
15
+ - **The engine is the only writer.** It declares the benchmarks it writes, and returns writes under `benchmarks`, beside `capabilities`
16
+ - **Screens only read the pool.** The game UI, the waiting room UI, the session UI and the presentation call `getBenchmark`, and receive statistics, never another team's record
17
+ - **A team's own record stays in the session.** It is part of the team's state, so undo and reset rewind it, but no screen, hook, validator or endpoint ever receives it. A screen shows the team's own figure from its capabilities, as it always has, and puts the pooled figures beside it
18
+ - **The platform owns the pool.** Whose results are pooled (your organisation first, with a global pool only where it is enabled), which sessions are eligible, how many results a figure needs before it is shown, and how long records are kept are all the platform's decisions. A walkthrough is never recorded
19
+
20
+ ## Declaring a benchmark
21
+
22
+ A benchmark lives at `benchmarks/<name>/benchmark.sdk.json`, named by its directory, and is created from JSON:
23
+
24
+ ```bash
25
+ pnpm run sdk new benchmark '{
26
+ "definition": {
27
+ "identifier": "outcome",
28
+ "revision": 1,
29
+ "tags": ["difficulty"],
30
+ "fields": {
31
+ "NetProfit": { "type": "number", "description": "Profit over the whole game" },
32
+ "Round1Pricing": { "enum": ["premium", "match", "discount"] }
33
+ }
34
+ }
35
+ }'
36
+ ```
37
+
38
+ This writes the bundle and a `sample.json` beside it, which the dev server answers from (see [Previewing with sample.json](#previewing-with-samplejson)). The build leaves `sample.json` out: a benchmark ships no files.
39
+
40
+ | Field | What it holds |
41
+ | ---------- | ----------------------------------------------------------------------------------------------------------------------------------- |
42
+ | `revision` | A whole number, at least 1. Results are pooled per revision |
43
+ | `level` | `'team'` (the default): one record per team. `'session'`: one record for the whole session, written by an aggregate engine |
44
+ | `tags` | Up to five names a record can be grouped by, such as a difficulty or a market |
45
+ | `fields` | One to 20 fields. `{ "type": "number" }` is pooled as a distribution; `{ "enum": [...] }` (one to 20 options) as a count per option |
46
+
47
+ Field and tag names start with a letter and hold letters, digits and underscores. `tags` and `excluded` are reserved: every record has them, and a benchmark cannot declare a field with either name.
48
+
49
+ Then wire it in, in two places:
50
+
51
+ 1. **List it in the engine's `benchmarks`:**
52
+
53
+ ```bash
54
+ pnpm run sdk update engine --patch '[
55
+ { "op": "add", "path": "/definition/benchmarks", "value": ["outcome"] }
56
+ ]'
57
+ ```
58
+
59
+ The engine's generated `types.ts` then exports `Benchmarks`, with each record's fields typed, and `SessionBenchmarks` for session-level ones
60
+
61
+ 2. **List the `benchmarks` feature in `requires`** in `seriousgaming.sdk.json`, so a runtime that cannot record benchmarks refuses the package rather than the first round that writes one:
62
+
63
+ ```json
64
+ {
65
+ "requires": ["benchmarks"]
66
+ }
67
+ ```
68
+
69
+ ### Choosing what to record
70
+
71
+ Record the few figures a debrief will actually show: an outcome or two, and the decisions that explain them. A benchmark is not a log of the game. Keep fields to numbers and short option lists; there is no free text, so nothing a participant typed can leave the session.
72
+
73
+ There is no 'higher is better' flag. Whether a high figure is good is for the screen to say, in its own words.
74
+
75
+ ### When to change the revision
76
+
77
+ A pool carries on across releases of the game that leave the benchmark alone. Bump `revision` when a change makes earlier results no longer comparable, such as a new demand model that halves typical profits: the new revision starts a new pool, which fills as sessions on it are played.
78
+
79
+ ## Writing from the engine
80
+
81
+ The engine's `Init` and `Run` receive `benchmarks` beside `context`: the team's current records, keyed by identifier, holding what the engine last wrote. A benchmark not yet written is absent. They return writes under `benchmarks`:
82
+
83
+ ```typescript
84
+ import type { RunFunction } from '@seriousgaming/sdk';
85
+
86
+ import type { Benchmarks, Callbacks, Context, Settings } from './types';
87
+
88
+ export const Run: RunFunction<Context, Settings, Callbacks, never, Benchmarks> = async ({
89
+ context,
90
+ settings,
91
+ meta,
92
+ }) => {
93
+ const next = await playRound(context, settings);
94
+ return {
95
+ capabilities: next,
96
+ benchmarks: {
97
+ outcome: {
98
+ NetProfit: next.finance.TotalProfit,
99
+ tags: { difficulty: settings.setup.Difficulty },
100
+ ...(meta.tick === 1 ? { Round1Pricing: pricingStance(next.pricing.Price) } : {}),
101
+ },
102
+ },
103
+ };
104
+ };
105
+ ```
106
+
107
+ `Benchmarks` is the fifth type parameter of `RunFunction` and `InitializationFunction`, after the engine's private `state` (pass `never` when the engine keeps none). The compiler then checks every field name, every option and every tag.
108
+
109
+ A write is a patch, so a record can be filled in round by round:
110
+
111
+ - **Each field it carries replaces that field.** A field it leaves out keeps its value, which is why `Round1Pricing` above is written once and then left alone
112
+ - **`null` removes a field**, and `null` in place of the whole write removes the record. A record with nothing left in it is removed
113
+ - **`tags` replaces the record's tags whole**
114
+ - **Tag values are short.** One to 64 letters, digits, `:`, `.`, `_` or `-`: enough for a difficulty or a team count, too short to carry a name or a sentence out of the session
115
+
116
+ ### Aggregate engines
117
+
118
+ In an aggregate engine, each `teams` entry carries its team's `benchmarks`, and `shared.benchmarks` holds the session's own records. A team-level benchmark is written in a `teams` entry (or by a per-team `Init` or `Run`), and a session-level one only under `shared`. Type them with the last two parameters of `InitAggregateFunction` and `RunAggregateFunction`, after the two state parameters: `Benchmarks`, then `SessionBenchmarks`.
119
+
120
+ ### A bad write is dropped, not fatal
121
+
122
+ Every write is checked against its benchmark: each field declared, each number finite, each option and tag declared, `excluded` a boolean, and the benchmark written at its own level. A write that fails is dropped whole, the record keeps its previous value, and the round carries on. A failed round in front of a room costs far more than one missing figure.
123
+
124
+ The problem comes back as a warning: `sdk tick` prints it and adds it to `warnings` in its `--json` output, and a test session's `tick()` returns it in `warnings`. Only the shape of `benchmarks` itself is strict: an engine that declares no benchmarks and returns the key fails the call, as any unexpected key does.
125
+
126
+ ## Deciding when values count
127
+
128
+ At any moment, the pool holds each team's record as the session now holds it. A later write replaces the earlier value, an undo that takes a record back takes it back in the pool too, and a removed record leaves the pool. So the game, not the platform, decides when a team's figures count:
129
+
130
+ - **Count everyone, as they go.** Write every round, as the example above does. A team counts with whatever it last reached, even in a session that stops early
131
+ - **Count only finished teams.** Write the outcome on the last round only, or hold `excluded: true` until then and write `excluded: null` with the final figures
132
+ - **Leave a team out.** `excluded: true` withdraws the record from every pool while keeping it in the session, for a run under settings too unusual to compare
133
+
134
+ ## Tags and filters
135
+
136
+ Tags let a screen compare like with like: a team on a hard difficulty with others who played it hard. Declare the names in the bundle, and set a value for each on the record:
137
+
138
+ ```typescript
139
+ tags: { difficulty: settings.setup.Difficulty, teams: String(teamCount) }
140
+ ```
141
+
142
+ A screen never sees a team's own tags. It builds its filters from the settings it reads, which hold the same values, and never from the scenario's name, which differs between organisations and says nothing about how the game was set up.
143
+
144
+ ## Reading benchmarks on a screen
145
+
146
+ `getBenchmark(identifier, query?)` is on `useUserInterfaceCallbacks()` (the game UI and the waiting room UI), on `useSessionUserInterfaceCallbacks()` (the session UI), and on `usePresentation()`, where it works with or without a session. It is not gated by `uses`.
147
+
148
+ ```typescript
149
+ const stats = await getBenchmark('outcome', {
150
+ where: [{ difficulty: settings.setup.Difficulty }, {}],
151
+ });
152
+ ```
153
+
154
+ - **`where`** is a filter on tag values, or a list of filters tried in order, narrowest first. The first with enough results answers. Left out, it is `{}`: everyone
155
+ - **`matched`** in the answer is the filter that answered, so the screen can label what it shows: 'compared with other hard games', or 'compared with everyone'
156
+ - **`scope`** is `'organisation'` or `'global'`: whose results were pooled, for the screen to say so
157
+ - **`fields`** holds an entry per field with figures to show. A numeric field is `{ kind: 'number', count, mean?, min?, max?, p10?, p25?, p50?, p75?, p90? }`; a field with options is `{ kind: 'choice', count, options }`
158
+
159
+ The types are `BenchmarkStatistics`, `NumericBenchmarkStatistics` and `ChoiceBenchmarkStatistics`, from `@seriousgaming/sdk`. See the [API reference](../api/sdk.md#benchmarkstatistics).
160
+
161
+ ### Write for missing figures
162
+
163
+ The answer is often thinner than the sample. Every screen that reads a benchmark handles three cases:
164
+
165
+ - **`null`**: nothing to show. No results yet, too few for any filter, or a runtime that keeps no pool. Show the team's own figure on its own, or hide the comparison
166
+ - **A missing figure.** Only `count` is always present. The platform may hold back any figure until enough results back it, and `min` and `max` most of all, since each is one team's real result. An option chosen by too few may be absent from `options`. Check each figure before you show it
167
+ - **A missing field.** A field with nothing to show is absent from `fields`
168
+
169
+ ```tsx
170
+ const profit = stats?.fields.NetProfit;
171
+ const median = profit?.kind === 'number' ? profit.p50 : undefined;
172
+
173
+ return median === undefined ? (
174
+ <Typography>{t('compare.notYet')}</Typography>
175
+ ) : (
176
+ <Typography>{t('compare.median', { value: median })}</Typography>
177
+ );
178
+ ```
179
+
180
+ ### Results may be stale
181
+
182
+ The pool may lag behind, and may leave out the current session. Treat the figures as context for a debrief, never as a live leaderboard. The SDK's wrapper keeps an answer for a minute, so a second call with the same query in that time reuses it, and settles on `null` if the host does not answer within 10 seconds, so a debrief never hangs on it.
183
+
184
+ An identifier the package declares no benchmark for rejects, so a typo fails in your own call rather than looking like an empty pool.
185
+
186
+ ## Previewing with sample.json
187
+
188
+ The dev server has no other sessions to pool, so it answers `getBenchmark` from `benchmarks/<name>/sample.json`. `new benchmark` writes one with placeholder figures for every field:
189
+
190
+ ```json
191
+ {
192
+ "scope": "organisation",
193
+ "fields": {
194
+ "NetProfit": {
195
+ "kind": "number",
196
+ "count": 120,
197
+ "mean": 50,
198
+ "min": 0,
199
+ "max": 100,
200
+ "p10": 10,
201
+ "p25": 25,
202
+ "p50": 50,
203
+ "p75": 75,
204
+ "p90": 90
205
+ },
206
+ "Round1Pricing": {
207
+ "kind": "choice",
208
+ "count": 120,
209
+ "options": { "premium": 40, "match": 40, "discount": 40 }
210
+ }
211
+ }
212
+ }
213
+ ```
214
+
215
+ Replace the figures with ones that look like your game. The file is read on every call, so an edit shows on the next read without a restart. It answers every query alike, with `matched` set to the first filter tried.
216
+
217
+ Use it to rehearse the thin cases too: delete `min` and `max`, or a field, to see the screen without them, and delete the file to see the `null` case. A sample that is not valid JSON, or not the right shape, makes `getBenchmark` reject, naming the file.
218
+
219
+ ## Testing
220
+
221
+ - **The engine's writes.** In a unit test, assert on `result.benchmarks` from `Run`. To test a write that depends on what the engine wrote before, pass `benchmarks` beside the arguments `logicArgs.run` builds: `Run({ ...logicArgs.run({ context, settings }), benchmarks: { outcome: { NetProfit: 120 } } })`
222
+ - **A whole game.** In an engine test, `session.state().teams[0].benchmarks` holds each team's records, and `state().sharedBenchmarks` the session's own. Check `warnings` on what `tick()` returns, so a dropped write fails the test rather than going unnoticed
223
+ - **From the CLI.** `pnpm run --silent sdk view --full --json` shows the same `benchmarks` and `sharedBenchmarks`, and `sdk tick` prints a warning for each dropped write
224
+ - **A screen.** The render helpers in `@seriousgaming/sdk/ui/testing` supply a `getBenchmark` that answers `null`. Pass your own in `callbacks` to render the comparison: `renderUserInterface(<App />, { callbacks: { getBenchmark: async () => stats } })`
225
+
226
+ See [Engine testing](./engine-testing.md) and [Testing](./testing.md).
227
+
228
+ ## Rules the build checks
229
+
230
+ - **The `benchmarks` feature in `requires`** for a game that declares any benchmark (`FEATURE_UNDECLARED`)
231
+ - **At most 20 benchmarks** in a game
232
+ - **Every name in the engine's `benchmarks` is a benchmark the package declares** (`REFERENCE_UNKNOWN`)
233
+ - **A session-level benchmark needs an aggregate engine**, the only kind with a session result to write it under
234
+ - **A benchmark the engine does not list** can never hold a value, so the build warns on it (`BENCHMARK_UNWRITTEN`)
235
+
236
+ ## Next steps
237
+
238
+ - [Create a benchmark](../recipes/create-benchmark.md): the steps, start to finish
239
+ - [Engines](./engine.md): the result a write sits beside
240
+ - [Multi-team games](./multi-team-games.md): aggregate engines, which write session-level benchmarks
241
+ - [Building presentations](./presentation.md): a debrief deck that compares the room with everyone else
@@ -262,6 +262,14 @@ A step (`StepFunction`) receives `state` too, so a step can read it; only the ex
262
262
 
263
263
  Reach for a capability instead when anything else reads the value. `scope: 'private'` keeps a capability off every screen while leaving it readable by hooks and endpoints, so a value an endpoint reports on, such as last round's profit for a trend, stays a capability.
264
264
 
265
+ ### Recording benchmarks
266
+
267
+ An engine that lists [benchmarks](./benchmarks.md) in its bundle's `benchmarks` receives each team's records as `benchmarks`, beside `state`, and returns writes to them under `benchmarks`, beside `capabilities`. Type them with the generated `Benchmarks`, the fifth type parameter of `InitializationFunction` and `RunFunction`. A write that does not match its benchmark is dropped with a warning, and the round carries on. See [Writing from the engine](./benchmarks.md#writing-from-the-engine).
268
+
269
+ ### Scoring assessments
270
+
271
+ An engine that lists [assessments](./assessments.md) in its bundle's `assessments` receives each team's results as `assessments`, beside `state`, and returns writes to them under `assessments`, beside `capabilities`: one result for a team-level assessment, a result per participant id for a participant-level one. Type them with the generated `Assessments`, the sixth type parameter of `InitializationFunction` and `RunFunction`. A write that does not match its assessment is dropped with a warning, and the round carries on. See [Scoring from the engine](./assessments.md#scoring-from-the-engine).
272
+
265
273
  ## Ordering the simulation
266
274
 
267
275
  A game holds one engine, so there is no cross-engine ordering to resolve. Split the simulation into steps, one responsibility each, and order them by the order `Run` calls them:
@@ -43,6 +43,8 @@ New to the SDK? Start with [Designing a serious game](./designing-a-game.md), fo
43
43
 
44
44
  - **[Facilitator guides](./guide.md)**: markdown facilitation content shown on the facilitator's page
45
45
  - **[Building presentations](./presentation.md)**: the facilitator's slide deck, its deck library, resume and export
46
+ - **[Benchmarks](./benchmarks.md)**: outcomes the engine records, pooled across sessions so a debrief can compare a team with everyone who has played
47
+ - **[Assessments](./assessments.md)**: scores the engine gives each participant or team, placed in bands and judged a pass, for a debrief or a learning management system
46
48
  - **[File handling (assets and blobs)](./file-handling.md)**: uploading files from UIs, showing them, and reading them in logic
47
49
  - **[LLM integration guide](./llm-integration.md)**: generating text through the platform LLM API
48
50
  - **[Speech generation guide](./speech-generation.md)**: synthesising audio with `generateSpeech`
@@ -53,15 +53,17 @@ const { game, session, settings, theme, locale, getPastState, getAssetUrl } = us
53
53
  >();
54
54
  ```
55
55
 
56
- | Field | What it holds |
57
- | -------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
58
- | `game` | `name` in the current locale, `maxTicks` (or `null` when the engine or tick controller decides), `mode`, and `scenarioName` (or `null`). Present with or without a session |
59
- | `session` | `null` with no session. Otherwise its `id`, `name`, `completedTicks` (the most rounds any team has completed, 0 before the first), `data` (the shared store and every team, narrowed to `reads`), `teams`, and how the room joins: `participants`, `joinLinks`, `sessionJoinLink` and `joinPin` (see [Showing the room how to join](#showing-the-room-how-to-join)) |
60
- | `settings` | The settings resources the presentation declares, resolved for the scenario, and for the session when there is one |
61
- | `theme` | The host's theme, to map onto the deck's colours |
62
- | `locale` | The locale to render in |
63
- | `getPastState` | Reads past rounds, as the session UI does, with `teams` defaulting to every team. Rejects when there is no session |
64
- | `getAssetUrl` | Turns an `asset:`, `shared:` or `blob:` reference into a URL |
56
+ | Field | What it holds |
57
+ | --------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
58
+ | `game` | `name` in the current locale, `maxTicks` (or `null` when the engine or tick controller decides), `mode`, and `scenarioName` (or `null`). Present with or without a session |
59
+ | `session` | `null` with no session. Otherwise its `id`, `name`, `completedTicks` (the most rounds any team has completed, 0 before the first), `data` (the shared store and every team, narrowed to `reads`), `teams`, and how the room joins: `participants`, `joinLinks`, `sessionJoinLink` and `joinPin` (see [Showing the room how to join](#showing-the-room-how-to-join)) |
60
+ | `settings` | The settings resources the presentation declares, resolved for the scenario, and for the session when there is one |
61
+ | `theme` | The host's theme, to map onto the deck's colours |
62
+ | `locale` | The locale to render in |
63
+ | `getPastState` | Reads past rounds, as the session UI does, with `teams` defaulting to every team. Rejects when there is no session |
64
+ | `getBenchmark` | Reads a [benchmark](./benchmarks.md)'s statistics pooled across sessions, with or without a session. `null` when there is nothing to show |
65
+ | `getAssessment` | Reads an [assessment](./assessments.md)'s results for the whole session. Rejects when there is no session |
66
+ | `getAssetUrl` | Turns an `asset:`, `shared:` or `blob:` reference into a URL |
65
67
 
66
68
  A presentation sees every team: it is the facilitator's view of the room.
67
69
 
@@ -167,28 +167,30 @@ const { requestTick, pauseTimer, resumeTimer, undoTick, resetGame } =
167
167
  useSessionUserInterfaceCallbacks();
168
168
  ```
169
169
 
170
- | Callback | Signature | Description |
171
- | -------------------- | ---------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------ |
172
- | `requestTick` | `(teamId?: string) => Promise<void>` | Advance the game by one tick; omit `teamId` to advance all teams |
173
- | `pauseTimer` | `(teamId?: string) => Promise<void>` | Pause automatic ticking; omit `teamId` to pause all teams |
174
- | `resumeTimer` | `(teamId?: string) => Promise<void>` | Resume automatic ticking; omit `teamId` to resume all teams |
175
- | `undoTick` | `(teamId?: string) => Promise<void>` | Un-run the last tick, keeping the tick's decisions; omit `teamId` to undo all teams. Naming one is independent mode only |
176
- | `resetGame` | `(teamId?: string) => Promise<void>` | Re-initialise the game from scratch; omit `teamId` to reset all teams |
177
- | `getPastState` | `(query) => Promise<PastStateEntry[]>` | Read committed past rounds; `teams` defaults to every team, or name one team by id |
178
- | `extendTick` | `(byMs, teamId?) => Promise<void>` | Move the currently scheduled tick's deadline by `byMs` milliseconds; negative pulls it in |
179
- | `openHostOverlay` | `(overlay, target?) => Promise<void>` | Ask the host to open the guide or a print sheet |
180
- | `uploadBlob` | `(file: File) => Promise<string>` | Upload a file, returns a `blob:<id>` reference |
181
- | `getAssetUrl` | `(reference: string) => string` | Resolve an `asset:`, `shared:`, or `blob:` reference to a URL |
182
- | `readAsset` | `(reference: string) => Promise<{ data, mimeType, name }>` | Read bytes for an `asset:`, `shared:`, or `blob:` reference |
183
- | `endpoints` | `Record<string, (request) => Promise<Response>>` | Call a declared endpoint by identifier |
184
- | `setWaitingRoom` | `(open: boolean) => Promise<void>` | Open or close the session's waiting room; no `teamId` |
185
- | `resetWalkthrough` | `() => Promise<void>` | Throw the walkthrough away and restart it at tick 0; rejects while the session has no walkthrough |
186
- | `createTeam` | `(name?: string) => Promise<void>` | Add a team; the host assigns the index |
187
- | `renameTeam` | `(teamId, name) => Promise<void>` | Rename a team |
188
- | `renameParticipant` | `(participantId, name) => Promise<void>` | Rename a participant; the host resolves their team |
189
- | `moveParticipant` | `(participantId, teamId) => Promise<void>` | Reassign a participant to another team |
190
- | `deleteTeam` | `(teamId) => Promise<void>` | Delete an empty team; independent mode only, and never the last team |
191
- | `setThemePreference` | `(preference: ThemePreference) => void` | Set the host's theme preference |
170
+ | Callback | Signature | Description |
171
+ | -------------------- | -------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------ |
172
+ | `requestTick` | `(teamId?: string) => Promise<void>` | Advance the game by one tick; omit `teamId` to advance all teams |
173
+ | `pauseTimer` | `(teamId?: string) => Promise<void>` | Pause automatic ticking; omit `teamId` to pause all teams |
174
+ | `resumeTimer` | `(teamId?: string) => Promise<void>` | Resume automatic ticking; omit `teamId` to resume all teams |
175
+ | `undoTick` | `(teamId?: string) => Promise<void>` | Un-run the last tick, keeping the tick's decisions; omit `teamId` to undo all teams. Naming one is independent mode only |
176
+ | `resetGame` | `(teamId?: string) => Promise<void>` | Re-initialise the game from scratch; omit `teamId` to reset all teams |
177
+ | `getPastState` | `(query) => Promise<PastStateEntry[]>` | Read committed past rounds; `teams` defaults to every team, or name one team by id |
178
+ | `getBenchmark` | `(identifier, query?) => Promise<BenchmarkStatistics \| null>` | Read a [benchmark](./benchmarks.md)'s pooled statistics; `null` when there is nothing to show |
179
+ | `getAssessment` | `(identifier) => Promise<AssessmentReport>` | Read an [assessment](./assessments.md)'s results for every team and participant |
180
+ | `extendTick` | `(byMs, teamId?) => Promise<void>` | Move the currently scheduled tick's deadline by `byMs` milliseconds; negative pulls it in |
181
+ | `openHostOverlay` | `(overlay, target?) => Promise<void>` | Ask the host to open the guide or a print sheet |
182
+ | `uploadBlob` | `(file: File) => Promise<string>` | Upload a file, returns a `blob:<id>` reference |
183
+ | `getAssetUrl` | `(reference: string) => string` | Resolve an `asset:`, `shared:`, or `blob:` reference to a URL |
184
+ | `readAsset` | `(reference: string) => Promise<{ data, mimeType, name }>` | Read bytes for an `asset:`, `shared:`, or `blob:` reference |
185
+ | `endpoints` | `Record<string, (request) => Promise<Response>>` | Call a declared endpoint by identifier |
186
+ | `setWaitingRoom` | `(open: boolean) => Promise<void>` | Open or close the session's waiting room; no `teamId` |
187
+ | `resetWalkthrough` | `() => Promise<void>` | Throw the walkthrough away and restart it at tick 0; rejects while the session has no walkthrough |
188
+ | `createTeam` | `(name?: string) => Promise<void>` | Add a team; the host assigns the index |
189
+ | `renameTeam` | `(teamId, name) => Promise<void>` | Rename a team |
190
+ | `renameParticipant` | `(participantId, name) => Promise<void>` | Rename a participant; the host resolves their team |
191
+ | `moveParticipant` | `(participantId, teamId) => Promise<void>` | Reassign a participant to another team |
192
+ | `deleteTeam` | `(teamId) => Promise<void>` | Delete an empty team; independent mode only, and never the last team |
193
+ | `setThemePreference` | `(preference: ThemePreference) => void` | Set the host's theme preference |
192
194
 
193
195
  The promise-returning callbacks resolve and reject as described in [Controlling game flow](./session-game-flow.md#controlling-game-flow). The wrapper supplies every callback, so none needs an optional-chaining guard.
194
196
 
@@ -4,7 +4,7 @@ A tick controller makes ticks happen on their own, on a schedule your logic deci
4
4
 
5
5
  This guide explains the model, then is the reference for writing one.
6
6
 
7
- **Keywords:** tick controller, timer, automatic ticks, interval, schedule, timed rounds, nextTickAt, game completion, complete, enabled, pause, extend a tick, cron
7
+ **Keywords:** tick controller, timer, automatic ticks, interval, schedule, timed rounds, nextTickAt, game completion, complete, enabled, pause, extend a tick, cron, dormant, unattended session
8
8
 
9
9
  ## The model
10
10
 
@@ -415,6 +415,27 @@ pnpm run sdk timer resume --team 2 # one team, in an independent game
415
415
 
416
416
  The current schedule, a [`TimerState`](../api/sdk-ui.md#timerstate) with every duration and timestamp in milliseconds, is pushed to every game UI. UIs subscribe with `useUserInterfaceTimer` and render their own countdown or progress bar. See [Displaying the tick timer](./ui-ticks.md#displaying-the-tick-timer) in the user interfaces guide.
417
417
 
418
+ ## Unattended sessions
419
+
420
+ A session left running with nobody using it may be suspended by its host. Once nothing has used the session for at least 24 hours and at least 100 automatic ticks, the host may stop its automatic ticking. The session is then **dormant**. Both numbers are minimums you can rely on: a host may wait longer, or never suspend at all, but never suspends sooner.
421
+
422
+ Nobody has to do anything to bring a dormant session back. The next time somebody or something uses it, it wakes before that use is handled, and the timer carries on as it does after a pause: an interval schedule continues from the time it had left, and a `nextTickAt` schedule asks `GetTickController` again without firing the ticks it missed. Each of these always wakes a session:
423
+
424
+ - A surface or facilitator page opening
425
+ - A write from any surface or caller
426
+ - A facilitator action, such as a manual tick, a timer control or a roster change
427
+ - An endpoint call, from any caller
428
+
429
+ A host may count more, such as a surface that stays open: the dev server keeps a session awake while any page is open on it. Reading the session's state never wakes it. Waking never ends a facilitator's pause, so a timer paused before the session went dormant stays paused.
430
+
431
+ What this means for a long-running game:
432
+
433
+ - **A slow game is safe.** The tick count is there so a game that ticks daily or weekly is not suspended after one quiet day. It is counted across the session, so a game of ten independent teams reaches 100 after ten rounds each
434
+ - **Plan for the clock stopping.** A game that ticks every few minutes for weeks will stop ticking a day after everyone stops using it, and resume when they come back. If your simulation must run on whether or not anybody looks at it, have something call one of its endpoints, which counts as using it
435
+ - **Showing it.** While dormant, the [`TimerState`](../api/sdk-ui.md#timerstate) your UIs receive has `dormant: true` and `dormantAt`, and `nextTickAt` is null. A UI left open while its session went dormant can say why the countdown stopped. Opening a surface wakes the session first, so a surface opened later never sees it. On waking, the dormant time is added to `pausedMs`, so a progress bar built from it stays right
436
+
437
+ To see how your UIs show a dormant session, press **Make dormant** in the dev drawer's Session tab, or post `{ "type": "makeDormant" }` to `/api/command`. The session goes dormant at once, without waiting a day, and the next thing that uses it wakes it.
438
+
418
439
  ## Testing without waiting
419
440
 
420
441
  A one-minute timer takes a minute to test, and the 30-second floor means none takes less. Start the dev server on a fake clock instead, and move time yourself:
@@ -144,6 +144,30 @@ const { entries, error, loading, refetch } = usePastState<Context>(
144
144
 
145
145
  Pass `null` to read nothing yet. `refetch()` reads the same query again. It works on the game UI, the waiting room, the session UI and a presentation, the surfaces that have `getPastState`.
146
146
 
147
+ ### Comparing with other sessions
148
+
149
+ `useUserInterfaceCallbacks()` also exposes `getBenchmark(identifier, query?)`, which reads a [benchmark](./benchmarks.md): anonymous statistics for this game pooled across sessions, so a debrief can show where a team sits against everyone who has played. Pass a list of filters on tag values, narrowest first, built from the settings your UI reads:
150
+
151
+ ```typescript
152
+ const { getBenchmark } = useUserInterfaceCallbacks();
153
+ const stats = await getBenchmark('outcome', {
154
+ where: [{ difficulty: settings.setup.Difficulty }, {}],
155
+ });
156
+ ```
157
+
158
+ It resolves to `null` when there is nothing to show, and any figure but `count` may be missing, so render the team's own figure first and the comparison only where a figure is present. The waiting room, the session UI and a presentation have it too. See [Reading benchmarks on a screen](./benchmarks.md#reading-benchmarks-on-a-screen).
159
+
160
+ ### Showing a participant's results
161
+
162
+ `useUserInterfaceCallbacks()` also exposes `getAssessment(identifier)`, which reads an [assessment](./assessments.md): the scores the engine gave, each placed in a band and judged a pass with the thresholds in force for the session. A participant's screen receives only their own result and their team's, and an observer's the team's:
163
+
164
+ ```typescript
165
+ const { getAssessment } = useUserInterfaceCallbacks();
166
+ const mine = (await getAssessment('teamwork')).results[0];
167
+ ```
168
+
169
+ `mine` is absent until the engine has scored this participant. Read again after each round. See [Reading results on a screen](./assessments.md#reading-results-on-a-screen).
170
+
147
171
  ### Settings are what your UI declares
148
172
 
149
173
  `settings` holds exactly the settings resources listed in the UI's `settings`, and no others. A UI that needs one of the engine's values lists the same settings resource the engine does (`"settings": ["engine"]`) and reads `settings.engine.X`: there is no need to copy it into context at `Init`, and no copy to go stale.
@@ -715,6 +715,27 @@ Nothing here is required. A package made by `sdk new package` at 1.11 has all of
715
715
 
716
716
  A workspace of several games applies each step to every game. The copies stay the game's own, so keep any local edit a template does not have.
717
717
 
718
+ ### 1.12: add `getBenchmark` and `getAssessment` to a hand-built callbacks object
719
+
720
+ `UserInterfaceCallbacks`, `SessionUserInterfaceCallbacks` and the value `usePresentation()` returns gain `getBenchmark`, which reads a [benchmark](./benchmarks.md), and `getAssessment`, which reads an [assessment](./assessments.md). A game reading them through `useUserInterfaceCallbacks()`, `useSessionUserInterfaceCallbacks()` or `usePresentation()` needs nothing, and so does a test that renders through `renderUserInterface` or `renderSessionUserInterface`, whose `getBenchmark` answers `null` and whose `getAssessment` answers with no results. A callbacks object built by hand, or a `PresentationValue` passed to `PresentationContext.Provider`, stops compiling until it has both:
721
+
722
+ ```tsx
723
+ const callbacks = {
724
+ ...yourCallbacks,
725
+ getBenchmark: async () => null,
726
+ getAssessment: async (identifier: string) => ({
727
+ identifier,
728
+ level: 'participant' as const,
729
+ min: 0,
730
+ max: 100,
731
+ bands: [],
732
+ results: [],
733
+ }),
734
+ };
735
+ ```
736
+
737
+ Better still, render through the helpers, as [1.3](#13-let-the-render-helpers-build-your-callbacks) describes, so the next addition needs nothing either.
738
+
718
739
  ### Optional: import the SDK's agent guidance in `CLAUDE.md`
719
740
 
720
741
  A new package's `CLAUDE.md` no longer holds a copy of the SDK's guidance. Its first line imports the copy that ships with `@seriousgaming/sdk-tools`, so the guidance always matches the installed version, and the rest of the file is the package's own. An older package keeps the copy it was scaffolded with, which goes stale with every upgrade and, in a package that began as the starter game, describes the lemonade stand after a reskin.
@@ -25,8 +25,8 @@ This index lists pages; `--outline` lists a page's sections, each with its slug
25
25
  - [`changelog-0.x.md`](changelog-0.x.md): Every release before 1.0, from 0.0.1 to 0.6.0, kept as history.
26
26
  - [`changelog.md`](changelog.md): All notable changes to the Serious Gaming SDK are recorded here.
27
27
  - [`cli-and-dev-server.md`](cli-and-dev-server.md), about 15k characters: The dev server runs your game on your own machine, with every screen in the browser, and the CLI plays and inspects the same session from the terminal.
28
- - [`cli-error-codes.md`](cli-error-codes.md), about 33k characters: Every code a failed `seriousgaming-sdk` command can report.
29
- - [`cli-reference.md`](cli-reference.md), about 163k characters: Every `seriousgaming-sdk` command and flag, in one place.
28
+ - [`cli-error-codes.md`](cli-error-codes.md), about 34k characters: Every code a failed `seriousgaming-sdk` command can report.
29
+ - [`cli-reference.md`](cli-reference.md), about 168k characters: Every `seriousgaming-sdk` command and flag, in one place.
30
30
  - [`cli-simulation-testing.md`](cli-simulation-testing.md), about 11k characters: Play a game from the terminal: configure it, start a session, make decisions, advance ticks, and read the state back as JSON.
31
31
  - [`environment-variables.md`](environment-variables.md), about 10k characters: A single reference for every environment variable the dev server and CLI read.
32
32
  - [`getting-started.md`](getting-started.md), about 11k characters: Scaffold the starter game, play it in the dev server, change one thing, and see the change.
@@ -35,18 +35,20 @@ This index lists pages; `--outline` lists a page's sections, each with its slug
35
35
  ## Concepts
36
36
 
37
37
  - [`concepts/index.md`](concepts/index.md), about 9k characters: The model behind the SDK on one page, and how its parts fit together.
38
- - [`concepts/terminology.md`](concepts/terminology.md), about 26k characters: Definitions of the terms used across the SDK, the CLI, the dev server, and these docs.
38
+ - [`concepts/terminology.md`](concepts/terminology.md), about 27k characters: Definitions of the terms used across the SDK, the CLI, the dev server, and these docs.
39
39
 
40
40
  ## Guides
41
41
 
42
42
  - [`guides/index.md`](guides/index.md), about 5k characters: Each guide is the one page for a resource kind or a topic: it explains the model, then serves as the reference, with every field, every option, and the rules behind them.
43
43
  - [`guides/ai-facilitation.md`](guides/ai-facilitation.md), about 14k characters: A facilitator running your game may have an AI agent beside them: something they can ask 'how is Team B doing', 'why has their profit fallen', or 'set their unit price to 3.53'.
44
+ - [`guides/assessments.md`](guides/assessments.md), about 17k characters: An assessment is a score the engine gives each participant, or each team, placed in a band such as 'developing' or 'expert' and judged a pass or not.
45
+ - [`guides/benchmarks.md`](guides/benchmarks.md), about 15k characters: A benchmark lets a debrief compare a team with everyone who has played the game, not just the other teams in the room.
44
46
  - [`guides/capabilities.md`](guides/capabilities.md), about 20k characters: Capabilities hold your game's state: named groups of variables that the engine and screens read and write.
45
47
  - [`guides/designing-a-game.md`](guides/designing-a-game.md), about 13k characters: A serious game teaches by letting people act and then see what their actions caused.
46
48
  - [`guides/designing-screens.md`](guides/designing-screens.md), about 12k characters: What a good simulation screen looks like, whatever the brand: how to decide what a screen should be before you build it, and how to check it afterwards.
47
49
  - [`guides/endpoints.md`](guides/endpoints.md), about 29k characters: An endpoint is a named, server-side handler that a caller invokes by identifier: one of the game's own user interfaces, or a caller outside the game, such as a facilitator's AI agent or a platform API, when the endpoint allows it.
48
50
  - [`guides/engine-testing.md`](guides/engine-testing.md), about 30k characters: Test your whole game in-process: no dev server, no browser, and no LLM API key required, because service callbacks are stubbed by default.
49
- - [`guides/engine.md`](guides/engine.md), about 33k characters: The engine is the logic that moves your game forward: it sets up the board, then plays one round each tick.
51
+ - [`guides/engine.md`](guides/engine.md), about 34k characters: The engine is the logic that moves your game forward: it sets up the board, then plays one round each tick.
50
52
  - [`guides/file-handling.md`](guides/file-handling.md), about 28k characters: Ship files with your game or let participants upload them, then show them in any UI and read them in logic, all through one reference scheme.
51
53
  - [`guides/games.md`](guides/games.md), about 7k characters: Shape the game as a whole: how many rounds it runs, how teams form, which languages it speaks, and where participants wait.
52
54
  - [`guides/global-settings.md`](guides/global-settings.md), about 4k characters: The global settings file lets you store API keys and defaults once, in a single user-owned file, so every project picks them up.
@@ -61,7 +63,7 @@ This index lists pages; `--outline` lists a page's sections, each with its slug
61
63
  - [`guides/scenario-editor.md`](guides/scenario-editor.md), about 10k characters: The `scenario_editor` slot holds the screen people use to build new scenarios for a released game, without rebuilding it.
62
64
  - [`guides/scenarios.md`](guides/scenarios.md), about 31k characters: A scenario is a named set of settings overrides and static assets that makes a distinct flavour of a game: a different storyline, difficulty preset, or themed variant.
63
65
  - [`guides/session-game-flow.md`](guides/session-game-flow.md), about 20k characters: How a session UI drives the session: ticks, the timer, host overlays, undo and reset, the waiting room, the language, the roster, and the participant list.
64
- - [`guides/session-hooks.md`](guides/session-hooks.md), about 15k characters: Every hook a session UI reads its data and callbacks from, and the messages the host sends it.
66
+ - [`guides/session-hooks.md`](guides/session-hooks.md), about 16k characters: Every hook a session UI reads its data and callbacks from, and the messages the host sends it.
65
67
  - [`guides/session.md`](guides/session.md), about 23k characters: The session UI is the facilitator's view of the whole room: every team's position side by side, with the controls to run the session.
66
68
  - [`guides/settings.md`](guides/settings.md), about 34k characters: Settings are a game's read-only parameters: a price, a difficulty, a round length, a menu, an answer key.
67
69
  - [`guides/setup.md`](guides/setup.md), about 16k characters: The `setup` slot holds the small form a facilitator fills in when they create a session.
@@ -69,25 +71,27 @@ This index lists pages; `--outline` lists a page's sections, each with its slug
69
71
  - [`guides/starter-game.md`](guides/starter-game.md), about 18k characters: `new package` hands you Lemonade Stand, a complete game rather than an empty shell.
70
72
  - [`guides/testing.md`](guides/testing.md), about 34k characters: Test your engine's logic and your user interfaces, so a broken rule or a broken screen fails on your machine rather than in front of a room.
71
73
  - [`guides/theming-and-branding.md`](guides/theming-and-branding.md), about 11k characters: Dress every surface of your game in the host's colours, logo and fonts, so a game run by a white-label host looks like that host's product.
72
- - [`guides/tick-controller.md`](guides/tick-controller.md), about 30k characters: A tick controller makes ticks happen on their own, on a schedule your logic decides, and can say when the game is finished.
74
+ - [`guides/tick-controller.md`](guides/tick-controller.md), about 33k characters: A tick controller makes ticks happen on their own, on a schedule your logic decides, and can say when the game is finished.
73
75
  - [`guides/type-safety.md`](guides/type-safety.md), about 16k characters: The SDK generates TypeScript types from your schemas, so a misspelt variable fails when you build rather than in front of a room.
74
76
  - [`guides/ui-files-and-llms.md`](guides/ui-files-and-llms.md), about 13k characters: How a game UI links to static assets, uploads and shows files, reads an asset's contents, and reaches an LLM through an endpoint to build a chat.
75
77
  - [`guides/ui-participants.md`](guides/ui-participants.md), about 15k characters: How a game UI reads who is playing: the current participant, their team, observers and walkthroughs, and the controls that let a team join, name itself, and fix a participant's name.
76
78
  - [`guides/ui-styling.md`](guides/ui-styling.md), about 11k characters: How a game UI handles language, theme, and screen size, and how to customise the Vite build that serves and bundles it.
77
79
  - [`guides/ui-ticks.md`](guides/ui-ticks.md), about 15k characters: How a game UI ends a round and shows the time left: requesting a tick, awaiting a write or tick and handling a `MutationError`, and driving a countdown from the tick timer.
78
- - [`guides/ui.md`](guides/ui.md), about 17k characters: The game UI is what each participant sees and plays: where they read the situation, make decisions, and see what those decisions caused.
79
- - [`guides/upgrading.md`](guides/upgrading.md), about 71k characters: This guide takes a game package on 0.6 to 1.0, in 13 steps.
80
+ - [`guides/ui.md`](guides/ui.md), about 18k characters: The game UI is what each participant sees and plays: where they read the situation, make decisions, and see what those decisions caused.
81
+ - [`guides/upgrading.md`](guides/upgrading.md), about 72k characters: This guide takes a game package on 0.6 to 1.0, in 13 steps.
80
82
  - [`guides/validators.md`](guides/validators.md), about 25k characters: A validator is a read-only gate on what a participant does: a context write before it is applied, and a request for the next tick before it runs.
81
83
  - [`guides/waiting-room.md`](guides/waiting-room.md), about 14k characters: The waiting room is the holding screen participants see while a session is parked, and you can replace the host's stock panel with your own.
82
84
 
83
85
  ## Recipes
84
86
 
85
- - [`recipes/index.md`](recipes/index.md), about 7k characters: Each recipe is one task, start to finish: prerequisites, numbered steps, a way to check it worked, and fixes for what usually goes wrong.
87
+ - [`recipes/index.md`](recipes/index.md), about 8k characters: Each recipe is one task, start to finish: prerequisites, numbered steps, a way to check it worked, and fixes for what usually goes wrong.
86
88
  - [`recipes/act-on-the-session-from-an-endpoint.md`](recipes/act-on-the-session-from-an-endpoint.md), about 4k characters: Let one endpoint call do what a facilitator would otherwise do in several steps, such as posting a notice, closing the waiting room and starting the next round, under your game's own rules.
87
89
  - [`recipes/adapt-a-game-for-walkthroughs.md`](recipes/adapt-a-game-for-walkthroughs.md), about 6k characters: A walkthrough is a disposable second run of your game beside the session, for demonstrating it: one team, one participant, always from tick 0, and isolated from the session in both directions.
88
90
  - [`recipes/adapt-ui-for-observers.md`](recipes/adapt-ui-for-observers.md), about 4k characters: An observer is a facilitator watching one team's live game UI without joining as a team member.
89
91
  - [`recipes/call-endpoint-from-ui.md`](recipes/call-endpoint-from-ui.md), about 6k characters: Call a server-side endpoint from any of your game's screens and handle its response or its failure.
90
92
  - [`recipes/check-a-package-from-a-script.md`](recipes/check-a-package-from-a-script.md), about 5k characters: Validate a package and play a few ticks from CI or an AI agent, and turn each failure into the file and line to fix.
93
+ - [`recipes/create-assessment.md`](recipes/create-assessment.md), about 6k characters: Score each participant or each team from the engine, place the score in bands such as 'developing' and 'expert', and show the result on a screen.
94
+ - [`recipes/create-benchmark.md`](recipes/create-benchmark.md), about 6k characters: Record a team's outcomes from the engine, and show a debrief how they compare with everyone who has played the game.
91
95
  - [`recipes/create-capability.md`](recipes/create-capability.md), about 3k characters: Define a named slice of the game's state that the engine and screens read and write.
92
96
  - [`recipes/create-endpoint.md`](recipes/create-endpoint.md), about 7k characters: Add a server-side handler your screens, or callers outside the game, can invoke by name with a validated request and response.
93
97
  - [`recipes/create-engine.md`](recipes/create-engine.md), about 4k characters: Write the engine that moves the game on each tick: here, one that reads a price and writes the day's revenue.
@@ -139,12 +143,12 @@ This index lists pages; `--outline` lists a page's sections, each with its slug
139
143
  - [`api/index.md`](api/index.md), about 8k characters: Every type, function, hook and component your game imports, one page per entry point.
140
144
  - [`api/deck.md`](api/deck.md), about 8k characters: Every export of `@seriousgaming/deck`, with the declaration it is published with.
141
145
  - [`api/sdk-commands.md`](api/sdk-commands.md), about 7k characters: What a client may ask a session to do.
142
- - [`api/sdk-e2e.md`](api/sdk-e2e.md), about 11k characters: End-to-end testing helpers for Serious Gaming SDK consumers.
146
+ - [`api/sdk-e2e.md`](api/sdk-e2e.md), about 12k characters: End-to-end testing helpers for Serious Gaming SDK consumers.
143
147
  - [`api/sdk-projection.md`](api/sdk-projection.md), about 5k characters: The one documented shape of a session's observable state.
144
148
  - [`api/sdk-testing.md`](api/sdk-testing.md), about 7k characters: Helpers for unit-testing a game's logic without a runtime.
145
- - [`api/sdk-tools-client.md`](api/sdk-tools-client.md), about 15k characters: Programmatic API for testing a game in-process.
149
+ - [`api/sdk-tools-client.md`](api/sdk-tools-client.md), about 16k characters: Programmatic API for testing a game in-process.
146
150
  - [`api/sdk-tools-playwright.md`](api/sdk-tools-playwright.md), about 3k characters: Playwright for a game's browser tests: a dev server for each worker.
147
151
  - [`api/sdk-tools-vite.md`](api/sdk-tools-vite.md), under 1k characters: What a game's Vite and Vitest configs need from the SDK.
148
152
  - [`api/sdk-ui-testing.md`](api/sdk-ui-testing.md), about 8k characters: Every export of `@seriousgaming/sdk/ui/testing`, with the declaration it is published with.
149
153
  - [`api/sdk-ui.md`](api/sdk-ui.md), about 36k characters: Every export of `@seriousgaming/sdk/ui`, with the declaration it is published with.
150
- - [`api/sdk.md`](api/sdk.md), about 59k characters: Every export of `@seriousgaming/sdk`, with the declaration it is published with.
154
+ - [`api/sdk.md`](api/sdk.md), about 79k characters: Every export of `@seriousgaming/sdk`, with the declaration it is published with.