nccgs 3.0.0-rc.2 → 3.0.0-rc.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +5 -0
- package/README.md +3 -2
- package/cli/install.mjs +16 -0
- package/package.json +1 -1
- package/runtime/task/ledger.mjs +7 -2
- package/template/CLAUDE.md +1 -1
- package/template/skills/adopt/SKILL.md +63 -0
- package/unity-skills/implement-in-app-purchases/README.md +0 -233
- package/unity-skills/optimize-web/resources/WebOptimizer.cs +0 -21
- package/unity-skills/optimize-web/resources/toktx-examples.sh +0 -11
- package/unity-skills/setup-vivox-voice-chat/evals/.env.example +0 -6
- package/unity-skills/setup-vivox-voice-chat/evals/README.md +0 -101
- package/unity-skills/setup-vivox-voice-chat/evals/promptfooconfig.yaml +0 -32
- package/unity-skills/setup-vivox-voice-chat/evals/tests/init-and-login.yaml +0 -76
- package/unity-skills/setup-vivox-voice-chat/evals/tests/text-chat.yaml +0 -65
- package/unity-skills/setup-vivox-voice-chat/evals/tests/voice-channels.yaml +0 -74
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,10 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## 3.0.0-rc.3 — 2026-11-06
|
|
4
|
+
|
|
5
|
+
- `/adopt` skill for a project that already has code, scenes, tests and design documents: inventory (versions, asmdefs, tests, scenes, the team's own documents, existing Claude rules), `brief.md` and `systems.md` written from the project as it is, decisions recorded in `state.md`, and one `ADOPT` task (existing EditMode tests, console ground truth, Play Mode + one screenshot) that proves the loop runs there. Nothing in the game changes without the user agreeing to the item. `/unity-foundation` stays the empty-project version.
|
|
6
|
+
- Coming from NCCGS 2.x: `install`/`update` now recognise a 2.x install state and refuse with the exact steps (`npx nccgs@<2.x> uninstall`, then `npx nccgs@rc install`, then `/adopt`) instead of stacking two frameworks' hooks and skills. 2.x task records left in `.nccgs/tasks/` (schema 1/2) are kept as history and never listed, so they cannot block a session, a close or an uninstall.
|
|
7
|
+
|
|
3
8
|
## 3.0.0-rc.2 — 2026-11-05
|
|
4
9
|
|
|
5
10
|
- Feedback with consent and transport: `feedback.consent` in project.json is `ask` until the person decides (`auto` / `manual` / `off`); the first session where an issue shows up asks once through Claude. With `auto`, a bundle is POSTed to `feedback.endpoint` when diagnostics cross a threshold (any task crash or hook error, 3 failed Unity calls, 5 failed verifications), when a task closes partial/blocked/failed, or every `periodicDays`; with `manual`, NCCGS suggests `/feedback` at most once per 6 h. Every send is logged in `.nccgs/feedback/sent.jsonl`; a copy of each bundle stays local. `nccgs feedback --send | --consent <level>`. Receiver: `tools/feedback-worker` (Cloudflare Worker → GitHub issue).
|
package/README.md
CHANGED
|
@@ -7,7 +7,7 @@ brief → options for the user → contract → edit → run (tests, Play Mode,
|
|
|
7
7
|
→ look → fix or ask → independent review → close → one-page report with real token cost
|
|
8
8
|
```
|
|
9
9
|
|
|
10
|
-
**3.0.0-rc.
|
|
10
|
+
**3.0.0-rc.3.** Built and verified on Unity 6000.3 and 6000.6 with Unity CLI 1.0.0-beta.14 and com.unity.pipeline 0.9, Claude Code 2.1.3xx, Windows. The sample game (a 2D endless runner) went from an empty scene to a vertical slice in four tasks with this loop; its reports are the reference for what a closed task looks like.
|
|
11
11
|
|
|
12
12
|
## Requirements
|
|
13
13
|
|
|
@@ -22,7 +22,7 @@ npx nccgs doctor --project D:\Games\MyGame
|
|
|
22
22
|
npx nccgs run --project D:\Games\MyGame
|
|
23
23
|
```
|
|
24
24
|
|
|
25
|
-
`install` adds `CLAUDE.md` (kept if you have one), `.claude/skills/` (
|
|
25
|
+
`install` adds `CLAUDE.md` (kept if you have one), `.claude/skills/` (12 NCCGS skills + 35 official Unity Skills), `.claude/agents/nccgs-*.md`, `.claude/nccgs/` (runtime, protocols, Ponytail) and `.nccgs/` (project config, tasks, evidence, reports). It merges hooks into `.claude/settings.json` without touching your other settings. `update` replaces managed files you did not change and records the ones you did; `uninstall` removes managed files, keeps `.nccgs/` and backs up what it removes. On a project that still carries NCCGS 2.x, `install` stops and prints the path (`npx nccgs@2.1.1 uninstall --project .`, then `install`, then `/adopt`); 2.x records under `.nccgs/` stay as history. `run` launches Claude Code with the permission mode and MCP policy NCCGS expects (`acceptEdits` by default; `--autonomy full` for `bypassPermissions`).
|
|
26
26
|
|
|
27
27
|
## Daily use
|
|
28
28
|
|
|
@@ -35,6 +35,7 @@ Open the project in the Editor (screenshots and live tests need it; without it t
|
|
|
35
35
|
| `/status` | where are we | `.nccgs/project.html` and ten lines of status |
|
|
36
36
|
| `/brainstorm` → `/map-systems` → `/vertical-slice` | a new game or a multi-task feature | `brief.md`, `systems.md`, `slice.json`, then one `/work` per planned task |
|
|
37
37
|
| `/unity-foundation` | a fresh project, or tests cannot run | asmdefs, test assemblies, input, pipeline as a measured task |
|
|
38
|
+
| `/adopt` | a project that already has code, scenes, tests, documents | `brief.md` + `systems.md` from what exists, `foundation.md`, one `ADOPT` task proving the loop runs here |
|
|
38
39
|
| `/playtest` | does it feel right, tuning, difficulty | telemetry artifact, the playtester's judgement, your decision |
|
|
39
40
|
| `/art-bible`, `/asset-spec` | before and after art | visual rules; import settings checked against them |
|
|
40
41
|
| `/feedback` | something went wrong, or after a few days of use | a local bundle (`.nccgs/feedback/`) of what ran, failed and was blocked, to send for review |
|
package/cli/install.mjs
CHANGED
|
@@ -34,6 +34,7 @@ export function install(root, { update = false, force = false, dryRun = false, m
|
|
|
34
34
|
if (!isUnityProject(root)) throw new Error(`${root} is not a Unity project (needs Assets/ and ProjectSettings/ProjectVersion.txt)`);
|
|
35
35
|
const stateFile = at(root, "installState");
|
|
36
36
|
const previous = exists(stateFile) ? readJson(stateFile) : null;
|
|
37
|
+
if (previous && isLegacy(previous)) throw new Error(legacyMessage(previous));
|
|
37
38
|
if (update && !previous) throw new Error("NCCGS is not installed here; use `install`");
|
|
38
39
|
if (!update && previous && !force) throw new Error(`NCCGS ${previous.version} is already installed; use \`update\``);
|
|
39
40
|
const actions = []; // { action: add|replace|keep|override|skip, path }
|
|
@@ -81,6 +82,21 @@ export function install(root, { update = false, force = false, dryRun = false, m
|
|
|
81
82
|
return { version: PKG.version, dryRun, counts: count(actions), overrides, actions: dryRun ? actions.filter(a => a.action !== "keep") : undefined };
|
|
82
83
|
}
|
|
83
84
|
|
|
85
|
+
// NCCGS 2.x wrote the same state file with `frameworkVersion` and a `files` array. Its hooks live in
|
|
86
|
+
// .claude/nccgs/hooks/ (not runtime/hooks/), so a 3.x install on top would leave two frameworks running.
|
|
87
|
+
export const isLegacy = state => !!state && (typeof state.frameworkVersion === "string" || Array.isArray(state.files) || typeof state.version !== "string");
|
|
88
|
+
function legacyMessage(state) {
|
|
89
|
+
const v = state.frameworkVersion ?? "2.x";
|
|
90
|
+
return [
|
|
91
|
+
`NCCGS ${v} is installed here. 3.x does not upgrade a 2.x install in place (its hooks, 58 skills, agents, rules and .nccgs/project.yaml would stay active next to the new ones).`,
|
|
92
|
+
`Remove 2.x with its own package first, then install 3.x:`,
|
|
93
|
+
` npx nccgs@${/^\d/.test(v) ? v : "2"} uninstall --project . --dry-run`,
|
|
94
|
+
` npx nccgs@${/^\d/.test(v) ? v : "2"} uninstall --project .`,
|
|
95
|
+
` npx nccgs@rc install --project .`,
|
|
96
|
+
`2.x uninstall keeps .nccgs/ (state.md, tasks, reviews, evidence) and your CLAUDE.md; 3.x reads state.md and leaves the rest in place. Then run /adopt in Claude Code to bring the project under the 3.x loop.`
|
|
97
|
+
].join("\n");
|
|
98
|
+
}
|
|
99
|
+
|
|
84
100
|
function count(actions) { return actions.reduce((acc, a) => { acc[a.action] = (acc[a.action] ?? 0) + 1; return acc; }, {}); }
|
|
85
101
|
|
|
86
102
|
// Merge our hooks and deny rules into .claude/settings.json without touching anything else.
|
package/package.json
CHANGED
package/runtime/task/ledger.mjs
CHANGED
|
@@ -15,13 +15,18 @@ export const taskFile = (root, id) => path.join(at(root, "tasks"), `${id}.json`)
|
|
|
15
15
|
export function loadTask(root, id) {
|
|
16
16
|
const file = taskFile(root, id);
|
|
17
17
|
if (!exists(file)) throw new Error(`Task ${id} not found (${file})`);
|
|
18
|
-
|
|
18
|
+
const task = readJson(file);
|
|
19
|
+
if (!isOurs(task)) throw new Error(`Task ${id} is an NCCGS ${task.schemaVersion ?? "1"}.x record (${file}); 3.x reads only schema 3. It is kept as history and not opened or closed by this version.`);
|
|
20
|
+
return task;
|
|
19
21
|
}
|
|
22
|
+
// NCCGS 2.x kept its own records in the same folder (schema 1/2, `closed` flag). They stay as history
|
|
23
|
+
// and are never listed, so a leftover 2.x task cannot block a session or an uninstall.
|
|
24
|
+
const isOurs = task => task?.schemaVersion === 3 && typeof task.status === "string";
|
|
20
25
|
export function saveTask(root, task) { task.updatedAt = nowIso(); writeJson(taskFile(root, task.id), task); return task; }
|
|
21
26
|
export function listTasks(root) {
|
|
22
27
|
const dir = at(root, "tasks");
|
|
23
28
|
if (!exists(dir)) return [];
|
|
24
|
-
return fs.readdirSync(dir).filter(f => f.endsWith(".json")).map(f => readJson(path.join(dir, f))).sort((a, b) => (a.openedAt < b.openedAt ? 1 : -1));
|
|
29
|
+
return fs.readdirSync(dir).filter(f => f.endsWith(".json")).map(f => readJson(path.join(dir, f))).filter(isOurs).sort((a, b) => (a.openedAt < b.openedAt ? 1 : -1));
|
|
25
30
|
}
|
|
26
31
|
export function openTasks(root) { return listTasks(root).filter(t => t.status !== "closed"); }
|
|
27
32
|
|
package/template/CLAUDE.md
CHANGED
|
@@ -10,7 +10,7 @@ This Unity project runs under NCCGS 3: a task is done when the game has run and
|
|
|
10
10
|
5. `.claude/nccgs/protocols/*.md`, then the official Unity Skills in `.claude/skills/`, then your own inference.
|
|
11
11
|
|
|
12
12
|
## Skills
|
|
13
|
-
`/brainstorm` → `/map-systems` → `/vertical-slice` for a new game or a multi-task feature (brief, systems map + slice plan, then `/work` per planned task). `/unity-foundation` on a fresh project. `/playtest` for feel and tuning (telemetry + playtester judgement + user decision). `/art-bible` and `/asset-spec` before and after art. `/status` for where we are. `/feedback` when something went wrong or after a few days of use.
|
|
13
|
+
`/brainstorm` → `/map-systems` → `/vertical-slice` for a new game or a multi-task feature (brief, systems map + slice plan, then `/work` per planned task). `/unity-foundation` on a fresh project; `/adopt` on a project that already has code, scenes and tests (brief and systems map from what exists, one task that proves the loop runs). `/playtest` for feel and tuning (telemetry + playtester judgement + user decision). `/art-bible` and `/asset-spec` before and after art. `/status` for where we are. `/feedback` when something went wrong or after a few days of use.
|
|
14
14
|
|
|
15
15
|
## The loop (skill `/work`)
|
|
16
16
|
Brief → options for the user when the decision is subjective → `task open` → edit → `task verify` → **look at the screenshots and console** → fix or ask → `task review` (rigor ≥ production) → `task close` → report.
|
|
@@ -0,0 +1,63 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: adopt
|
|
3
|
+
description: Bring an existing Unity project (code, scenes, tests and design documents already there) under NCCGS - inventory what exists, write the brief and systems map from the project as it is, record the foundation, and close one task that proves the evidence loop runs on this project. Use right after `nccgs install` on a project that is not empty, or when NCCGS was added late.
|
|
4
|
+
allowed-tools: Read, Glob, Grep, Write, Edit, Bash, Agent, AskUserQuestion
|
|
5
|
+
---
|
|
6
|
+
|
|
7
|
+
# /adopt
|
|
8
|
+
|
|
9
|
+
Output: `.nccgs/design/brief.md`, `.nccgs/design/systems.md` and `.nccgs/design/foundation.md` written from the project **as it is** (not from an idea), and a closed task `ADOPT` whose evidence proves the loop runs here. Nothing in the game changes unless the user agrees to a specific item in step 4.
|
|
10
|
+
|
|
11
|
+
`/brainstorm` and `/map-systems` assume a new game; this skill is their replacement for a project that already has months of work in it. `/unity-foundation` is the smaller version of this for an empty project.
|
|
12
|
+
|
|
13
|
+
## 1. Inventory (read-only, no Unity calls)
|
|
14
|
+
|
|
15
|
+
Read, in this order, and write what you find to `.nccgs/design/foundation.md` under **Present / Missing / Decision needed**:
|
|
16
|
+
|
|
17
|
+
- `ProjectSettings/ProjectVersion.txt`, `Packages/manifest.json`: Editor version, render pipeline, Input System or legacy input, test framework, `com.unity.pipeline`.
|
|
18
|
+
- `Assets/` layout: top-level folders, every `.asmdef` (runtime and test), scenes in Build Settings (`ProjectSettings/EditorBuildSettings.asset`), the scene that loads first.
|
|
19
|
+
- Tests: `Assets/**/Tests/**`, EditMode and PlayMode assemblies, roughly how many tests, whether they compile against the current code (`unity test --mode EditMode` is run in step 5, not here).
|
|
20
|
+
- Design documents the team already keeps: `*.md` in the repo root and `Docs/`, `README`, a GDD, a Notion/Confluence export, issue templates. Name each with one line on what it is the source of truth for. These stay where they are; NCCGS points at them, it does not move them.
|
|
21
|
+
- Existing Claude material: the project's `CLAUDE.md` (`install` kept it), `.claude/rules/`, other `.claude/skills/` and `.claude/agents/` not from NCCGS, `.cursorrules`, `.github/copilot-instructions.md`. List anything that contradicts `.claude/nccgs/protocols/*.md` (for example a rule that says "done when tests pass" or a Unity MCP route).
|
|
22
|
+
- The game itself, from code: the MonoBehaviours on the first scene, the main loop (what starts a run, what ends it), the systems that own state (player, spawning, score, save, UI), where input is read (`InputAction` or polling), where tuning numbers live (ScriptableObjects, constants, inspector fields).
|
|
23
|
+
|
|
24
|
+
Keep the inventory to facts you read. Do not guess versions or test counts; write "not found" instead.
|
|
25
|
+
|
|
26
|
+
## 2. Brief and systems map from the existing project
|
|
27
|
+
|
|
28
|
+
Write `.nccgs/design/brief.md` with the `/brainstorm` headings (Fantasy, Core loop, Pillars, Vertical slice proves, Feel numbers, Out of scope, Open questions), but every line comes from what the project already does. "Vertical slice proves" becomes **What already works** (seen in code and scenes) and **What is unproven** (never had a test or a measurement). Feel numbers are the values found in ScriptableObjects, constants and inspector fields, with the file each one lives in; the "who decides" column is empty until the user fills it.
|
|
29
|
+
|
|
30
|
+
Write `.nccgs/design/systems.md` as `/map-systems` would: 3–8 systems, each with its responsibility, the state it owns, what it reads, its Unity surface and **the files that implement it today**. Mark the two riskiest couplings and any system whose logic is only in MonoBehaviours (no plain C# → no EditMode tests possible without a refactor; note it, do not refactor).
|
|
31
|
+
|
|
32
|
+
Show both files to the user with AskUserQuestion before going on; fix what they correct. Where the project's own documents disagree with the code, say so and ask which one is the truth — never rewrite their design documents from code without being asked.
|
|
33
|
+
|
|
34
|
+
## 3. Decisions the user owns
|
|
35
|
+
|
|
36
|
+
Ask only about real forks, one at a time:
|
|
37
|
+
|
|
38
|
+
- Which existing design document is the source of truth for the brief (`state.md` will link to it).
|
|
39
|
+
- Default `rigor` for this project (`production` is right for anything that ships; `prototype` for a jam).
|
|
40
|
+
- Rules in the existing `CLAUDE.md` or `.claude/rules/` that contradict NCCGS: keep theirs, drop theirs, or merge. Propose the merge text; the user decides. Never delete a rule silently.
|
|
41
|
+
- If input is polled or on the legacy Input Manager: note that `simulate_key`, `--shots` and playtests will not drive the game until it reads input through `InputAction`s. Offer it as a later `/work` task; do not do it now.
|
|
42
|
+
|
|
43
|
+
Write the decisions to `.nccgs/state.md` under a dated **Adopted** entry.
|
|
44
|
+
|
|
45
|
+
## 4. The ADOPT task
|
|
46
|
+
|
|
47
|
+
Open one task that proves the loop runs on this project, with the smallest possible edit set:
|
|
48
|
+
|
|
49
|
+
```
|
|
50
|
+
node .claude/nccgs/runtime/task/task.mjs open --id ADOPT --title "Adopt NCCGS on <project>" --rigor production --files "<test asmdefs and smoke test files only>" --criteria-json '[
|
|
51
|
+
{ "id": "editmode", "description": "the existing EditMode tests pass", "assertions": [{ "artifact": ".nccgs/evidence/measurements/ADOPT-tests.json", "pointer": "/failed", "op": "eq", "value": 0 }, { "artifact": ".nccgs/evidence/measurements/ADOPT-tests.json", "pointer": "/passed", "op": "gte", "value": 1 }] },
|
|
52
|
+
{ "id": "compiles", "description": "no compilation errors in the console", "assertions": [{ "artifact": ".nccgs/evidence/measurements/ADOPT-console.json", "pointer": "/groundTruth/compilationFailed", "op": "eq", "value": false }] },
|
|
53
|
+
{ "id": "observe", "description": "the first scene enters and exits Play Mode with no exception and one screenshot shows the game", "assertions": [{ "artifact": ".nccgs/evidence/measurements/ADOPT-observe.json", "pointer": "/exceptionCount", "op": "eq", "value": 0 }, { "artifact": ".nccgs/evidence/screens/ADOPT/01.png", "op": "image" }] }
|
|
54
|
+
]' --notes "How to try it: open the project, run the EditMode tests, press Play on <scene>."
|
|
55
|
+
```
|
|
56
|
+
|
|
57
|
+
If the project has no test assembly at all, add the `playmode` criterion from `/unity-foundation` and create only `Assets/Tests/EditMode/<Game>.Tests.EditMode.asmdef` plus one smoke test that references one real class of the game (so the test proves the runtime assembly is reachable). Nothing else is created; packages are not added without asking, and `com.unity.pipeline` is installed with `unity pipeline install --project-path .` only after the user agrees.
|
|
58
|
+
|
|
59
|
+
## 5. Run, look, review, close
|
|
60
|
+
|
|
61
|
+
`task verify --kind tests`, `--kind console`, `--kind observe --hold 5` as in `/work`; look at the screenshot and the console before anything else. A failing existing test is a finding for the user, not something to fix inside ADOPT: record it in `foundation.md` and in the task notes, and close `partial` if it blocks a criterion. Review and close as in `/work`; the report is the first one this project gets and the baseline every later report is compared with.
|
|
62
|
+
|
|
63
|
+
End with: the three design paths, the report path, the list of decisions still open, and `next:` either `/map-systems` (when the user wants a slice plan for the next milestone) or `/work` for the first real change.
|
|
@@ -1,233 +0,0 @@
|
|
|
1
|
-
# Unity In-App Purchases Skill
|
|
2
|
-
|
|
3
|
-
This skill helps you implement, configure, debug, and migrate Unity In-App Purchases (IAP) using `com.unity.purchasing` v5. It covers standard Apple App Store / Google Play billing, IAP D2C Capabilities (Direct-to-Customer — Stripe/Coda via Unity Cloud), and conversion from a wide range of third-party and native billing implementations.
|
|
4
|
-
|
|
5
|
-
---
|
|
6
|
-
|
|
7
|
-
## What This Skill Covers
|
|
8
|
-
|
|
9
|
-
| Path | When to use |
|
|
10
|
-
|---|---|
|
|
11
|
-
| **Add IAP to a new project** | No existing IAP — start from scratch with Unity IAP 5 |
|
|
12
|
-
| **Migrate v4 → v5** | Project uses `IStoreListener`, `UnityPurchasing.Initialize`, or `ConfigurationBuilder` |
|
|
13
|
-
| **Convert native Google BillingClient** | Project calls Android BillingClient via `AndroidJavaObject` / JNI bridge |
|
|
14
|
-
| **Convert native iOS StoreKit** | Project uses a custom ObjC/Swift StoreKit plugin bridged via `DllImport("__Internal")` |
|
|
15
|
-
| **Convert Essential Kit billing** | Project uses VoxelBusters Essential Kit for in-app purchases (without losing other features) |
|
|
16
|
-
| **Assess/Convert UniPay (FLOBUK)** | Project uses UniPay — determines whether migration is needed or possible |
|
|
17
|
-
| **Assess/Convert RevenueCat** | Project uses RevenueCat — evaluate/implement observer mode to work with Unity IAP 5 |
|
|
18
|
-
| **Assess/Convert Adapty** | Project uses Adapty — evaluate/implement observer mode to work with Unity IAP 5 |
|
|
19
|
-
| **Implement IAP D2C Capabilities** | Add third-party payment provider (Stripe or Coda) via Unity Cloud |
|
|
20
|
-
|
|
21
|
-
The skill always scans the project first and routes to the correct path automatically. You can also specify a path manually if you know exactly which one you need.
|
|
22
|
-
|
|
23
|
-
---
|
|
24
|
-
|
|
25
|
-
## Example Prompts
|
|
26
|
-
|
|
27
|
-
```
|
|
28
|
-
Add in-app purchases code to my game. I have a 100-coin pack and a Remove Ads unlock.
|
|
29
|
-
```
|
|
30
|
-
```
|
|
31
|
-
Migrate my existing IAP code in the project from Unity IAP v4 to v5.
|
|
32
|
-
```
|
|
33
|
-
```
|
|
34
|
-
My project uses AndroidJavaObject to call Google Play BillingClient. Convert it to Unity IAP.
|
|
35
|
-
```
|
|
36
|
-
```
|
|
37
|
-
I have a custom ObjC StoreKit plugin with SKPaymentQueue. Replace it with Unity IAP.
|
|
38
|
-
```
|
|
39
|
-
```
|
|
40
|
-
Disable Essential Kit billing and replace with Unity IAP 5. Keep other Essential Kit features working as before.
|
|
41
|
-
```
|
|
42
|
-
```
|
|
43
|
-
Can my RevenueCat project implement app store purchases through Unity's IAP v5?
|
|
44
|
-
```
|
|
45
|
-
```
|
|
46
|
-
My project uses Adapty. What features will we lose if we use Unity IAP instead? Give me a detailed report and do not make any changes now.
|
|
47
|
-
```
|
|
48
|
-
```
|
|
49
|
-
Add Stripe payment support via Unity IAP D2C Capabilities.
|
|
50
|
-
```
|
|
51
|
-
```
|
|
52
|
-
Add a web checkout option for my players using Unity's IAP D2C Capabilities with Coda.
|
|
53
|
-
```
|
|
54
|
-
|
|
55
|
-
---
|
|
56
|
-
|
|
57
|
-
## Path Details, Limitations, and Best Practices
|
|
58
|
-
|
|
59
|
-
### Add IAP to a New Project
|
|
60
|
-
|
|
61
|
-
**What it does:** Installs Unity IAP 5, creates an `IAPManager`, wires up the two-step purchase flow (pending → confirm), integrates with your existing save system, and connects to your shop UI.
|
|
62
|
-
|
|
63
|
-
**Limitations:**
|
|
64
|
-
- The skill cannot create App Store Connect or Google Play Console product entries — you must do that manually.
|
|
65
|
-
- If you use local Google Play receipt validation, you must run the Receipt Validation Obfuscator manually in the Unity Editor (**Services > In-App Purchasing > Receipt Validation Obfuscator**).
|
|
66
|
-
- Apple local receipt validation is a no-op under StoreKit 2. Use server-side JWS validation for Apple.
|
|
67
|
-
|
|
68
|
-
**Best practices:**
|
|
69
|
-
- Define all products in one authoritative catalog (a `ScriptableObject` or constants class) — not scattered across button click handlers.
|
|
70
|
-
- Always call `ConfirmPurchase` **after** saving the granted reward. An unconfirmed purchase re-delivers safely; a confirmed-but-unsaved one loses the reward permanently.
|
|
71
|
-
- Subscribe to all events **before** calling `Connect()` — pending purchases from a previous session may fire immediately on reconnect.
|
|
72
|
-
- Add a "Restore Purchases" button for any project with NonConsumable or Subscription products — required on iOS.
|
|
73
|
-
|
|
74
|
-
---
|
|
75
|
-
|
|
76
|
-
### Migrate v4 → v5
|
|
77
|
-
|
|
78
|
-
**What it does:** Replaces the listener-based `IStoreListener` / `UnityPurchasing.Initialize` / `ConfigurationBuilder` pattern with the event-driven `StoreController` pattern.
|
|
79
|
-
|
|
80
|
-
**Limitations:**
|
|
81
|
-
- `product.receipt` and `product.hasReceipt` are removed. The skill migrates ownership checks to `store.CheckEntitlement(product)` — verify your entitlement logic after migration.
|
|
82
|
-
- Apple local validation via `CrossPlatformValidator` still compiles but is a no-op under StoreKit 2. If your project validates Apple receipts locally, this validation silently stops working after migration.
|
|
83
|
-
- Developer payload (the third `Purchase()` argument) is removed. If your backend uses it, a backend change is required.
|
|
84
|
-
- `SubscriptionManager` is replaced — subscription info is now on `order.Info.PurchasedProductInfo`, not on `CartItem`.
|
|
85
|
-
|
|
86
|
-
**Best practices:**
|
|
87
|
-
- Migrate one system at a time: get initialization working first, then purchase flow, then restore.
|
|
88
|
-
- Run the migration in a branch. The skill uses `#if` guards where needed, but test thoroughly before removing the old code.
|
|
89
|
-
- After migration, recheck all `OnPurchaseConfirmed` handlers — the event now receives `Order` (base type) and you must pattern-match `ConfirmedOrder` vs `FailedOrder`.
|
|
90
|
-
|
|
91
|
-
---
|
|
92
|
-
|
|
93
|
-
### Convert Native Google BillingClient
|
|
94
|
-
|
|
95
|
-
**What it does:** Converts C# → JNI → BillingClient bridge code to Unity IAP 5, preserves the public billing API via a compatibility facade, and wraps old native code in `#if !USE_UNITY_IAP_V5` for easy rollback.
|
|
96
|
-
|
|
97
|
-
**Limitations:**
|
|
98
|
-
- **Multiple subscription base plans / offer tokens** (`basePlanId`, `offerToken`, `offerId`) are a hard blocker — Unity IAP does not expose offer-level selection. You must either simplify your Play Console subscription setup to one base plan per product, or stay on native billing.
|
|
99
|
-
- **Personalized price disclosure** (`setIsPersonalizedPrice`) and **alternative billing / external offers** are hard blockers.
|
|
100
|
-
- **Multi-quantity purchases** (quantity > 1 per transaction) are not supported in Unity IAP.
|
|
101
|
-
- A mixed native BillingClient + Unity IAP architecture is not supported — both compete for the same `PurchasesUpdatedListener` slot. The skill will not generate a mixed setup.
|
|
102
|
-
- The receipt format changes: the backend receives `order.Info.Receipt` (a JSON object containing `purchaseToken`, `orderId`, and `signature`) instead of raw fields. If your backend parses these fields individually, a backend update is required.
|
|
103
|
-
|
|
104
|
-
**Best practices:**
|
|
105
|
-
- Run the blocker scan and read the migration report before making any changes. Hard blockers require a decision before code is written.
|
|
106
|
-
- Keep the Gradle billing dependency in place until migration is validated — Unity IAP brings its own BillingClient, but removing the old dependency prematurely can break the build in unexpected ways.
|
|
107
|
-
- Use the `#if USE_UNITY_IAP_V5` / `#if !USE_UNITY_IAP_V5` guards throughout. The rollback path (remove the define) must work cleanly before you ship.
|
|
108
|
-
- Test on a physical Android device with a real sandbox account — the BillingClient behavior in the Unity Editor and on emulators is not representative of production.
|
|
109
|
-
|
|
110
|
-
---
|
|
111
|
-
|
|
112
|
-
### Convert Native iOS StoreKit
|
|
113
|
-
|
|
114
|
-
**What it does:** Converts a custom Objective-C or Swift StoreKit plugin (bridged via `DllImport("__Internal")` and `UnitySendMessage`) to Unity IAP 5, preserving the C# game-facing API via a compatibility facade.
|
|
115
|
-
|
|
116
|
-
**Limitations:**
|
|
117
|
-
- **Receipt format is a breaking backend change.** Unity IAP uses per-transaction JWS (StoreKit 2 style) rather than the SK1 base64 app receipt bundle. If your backend validates the receipt bundle against Apple's `/verifyReceipt` endpoint, it must migrate to Apple's App Store Server API or JWS validation before you can ship.
|
|
118
|
-
- **Promotional offer signing** (`SKPaymentDiscount` / SK2 signed offers) is a hard blocker — Unity IAP has no equivalent for server-signed promotional offers.
|
|
119
|
-
- **Win-back offers** (StoreKit 2) are not supported in Unity IAP 5.4.
|
|
120
|
-
- **`SKStorefront`** (region detection via storefront change observer) is not exposed in Unity IAP.
|
|
121
|
-
- **`SKReceiptRefreshRequest`** has no direct equivalent — `store.FetchPurchases()` covers the purchase re-delivery use case but not manual receipt refresh.
|
|
122
|
-
- ObjC/Swift plugin files cannot use C# `#if` defines. The skill guards only the C# call sites — the native files remain in `Assets/Plugins/iOS/` and compile unconditionally.
|
|
123
|
-
- A mixed native StoreKit + Unity IAP architecture is not supported — both register as `SKPaymentQueue` observers and only one reliably receives callbacks.
|
|
124
|
-
|
|
125
|
-
**Best practices:**
|
|
126
|
-
- Audit the backend receipt validation endpoint before starting. If it calls `/verifyReceipt`, plan the backend migration in parallel with the client migration.
|
|
127
|
-
- If the plugin intercepts App Store promotional purchases (`shouldAddStorePayment`), wire up `OnPromotionalPurchaseIntercepted` and `ContinuePromotionalPurchases()` before removing the native interceptor.
|
|
128
|
-
- Test Ask-to-Buy explicitly — the deferred → pending two-stage flow behaves differently than a native `PURCHASING` → `PURCHASED` transition.
|
|
129
|
-
- Validate the migration on TestFlight, not just in the Unity Editor or Simulator. StoreKit behavior differs between Editor sandbox, Simulator, and TestFlight builds.
|
|
130
|
-
|
|
131
|
-
---
|
|
132
|
-
|
|
133
|
-
### Convert Essential Kit Billing
|
|
134
|
-
|
|
135
|
-
**What it does:** Disables the Essential Kit Billing service flag in `EssentialKitSettings.asset`, removes the conflicting `com.android.billingclient` Gradle dependency, and implements Unity IAP 5 using the product catalog extracted from the Essential Kit settings.
|
|
136
|
-
|
|
137
|
-
**Limitations:**
|
|
138
|
-
- Essential Kit C# source files are **not deleted**. The Billing service is disabled via the settings flag — all EK code remains and compiles. Do not expect a clean removal.
|
|
139
|
-
- Only the Billing service is affected. All other Essential Kit services (Notification, GameServices, etc.) are left completely untouched.
|
|
140
|
-
- **Product IDs must not change.** The same IDs used in Essential Kit must be carried over to Unity IAP to preserve store history.
|
|
141
|
-
- If your project uses **server-side receipt validation**, the receipt format changes from `transaction.RawData` (Android) and `transaction.Receipt` (iOS JWS) to `order.Info.Receipt` and `order.Info.Apple?.jwsRepresentation`. Document the backend change required.
|
|
142
|
-
- Subscriptions are fully supported, but the restore path changes — EK's `OnRestorePurchasesComplete` maps to both `store.OnPurchasesFetched` and the `RestoreTransactions` callback in Unity IAP.
|
|
143
|
-
|
|
144
|
-
**Best practices:**
|
|
145
|
-
- Run `Assets > External Dependency Manager > Android Resolver > Force Resolve` after removing the Gradle billing dependency — do not skip this step.
|
|
146
|
-
- Verify the Essential Kit Billing service is visually disabled in **Window > Voxel Busters > Essential Kit > Open Settings → Services** after the settings file edit.
|
|
147
|
-
- Confirm all product IDs in App Store Connect and Play Console match the Unity IAP catalog exactly before testing.
|
|
148
|
-
|
|
149
|
-
---
|
|
150
|
-
|
|
151
|
-
### Assess UniPay (FLOBUK)
|
|
152
|
-
|
|
153
|
-
**What it does:** This path does **not** perform a conversion. It assesses whether migration is needed and routes to one of three outcomes: unsupported platform (stop), upgrade required (stop), or no action needed (UniPay already wraps Unity IAP 5).
|
|
154
|
-
|
|
155
|
-
**Limitations:**
|
|
156
|
-
- UniPay's Steam, Meta Quest, PayPal, and Facebook Instant Games integrations have **no Unity IAP equivalent**. If your project targets any of these platforms, there is no migration path — keep UniPay.
|
|
157
|
-
- If `com.unity.purchasing` is below v5.4, IAP D2C Capabilities (Stripe/Coda) are not available — an upgrade to the latest stable v5.4+ is needed before adding D2C support.
|
|
158
|
-
- The skill does not perform a "remove UniPay" conversion — that is a manual refactor scoped to what features you are replacing.
|
|
159
|
-
|
|
160
|
-
**Best practices:**
|
|
161
|
-
- If the assessment concludes "no action needed," there is no code to write. Work within UniPay's API for new products or purchase logic changes.
|
|
162
|
-
- If you want to remove UniPay entirely and use Unity IAP directly, ask the skill explicitly and describe which UniPay features you are replacing — it will assess feasibility and scope.
|
|
163
|
-
|
|
164
|
-
---
|
|
165
|
-
|
|
166
|
-
### Assess/Convert RevenueCat
|
|
167
|
-
|
|
168
|
-
**What it does:** Evaluates whether the project can switch to Unity IAP 5 for handling purchases. Produces one of three outcomes: already in observer mode (no action), blockers detected (report + two choices), or no blockers (two conversion paths: observer mode or full removal).
|
|
169
|
-
|
|
170
|
-
**Limitations:**
|
|
171
|
-
- **Amazon Appstore** is a hard blocker — Unity IAP 5 has removed Amazon support. RevenueCat observer mode also does not work reliably on Amazon builds. If your project targets Amazon, conversion is not viable.
|
|
172
|
-
- **RevenueCat Offerings / remote paywalls** have no Unity IAP equivalent — all products must be defined in code or a local catalog.
|
|
173
|
-
- **RevenueCat A/B testing (Experiments)** has no Unity IAP equivalent.
|
|
174
|
-
- **Cross-platform entitlement sync** (a user who buys on iOS retains access on Android) has no Unity IAP equivalent without a custom backend.
|
|
175
|
-
- **RevenueCat webhook events** (subscription renewals, cancellations, billing issues) are not delivered by Unity IAP — you would need to build your own subscription event infrastructure.
|
|
176
|
-
- In **observer mode**, `SyncPurchases()` must be called after every Unity IAP confirmed purchase. Missing this call means RevenueCat does not validate the receipt and `CustomerInfo` is not updated.
|
|
177
|
-
|
|
178
|
-
**Best practices:**
|
|
179
|
-
- Run the full feature check before deciding. RevenueCat's value often comes from features that are not obvious in the codebase (e.g., webhooks configured server-side).
|
|
180
|
-
- If the project has a marketing team managing paywall copy or pricing remotely via RevenueCat Offerings, a full removal will require significant UI work to replace that capability.
|
|
181
|
-
- Observer mode is the lower-risk path — it preserves RevenueCat's server-side validation and subscription lifecycle tracking while Unity IAP handles the native purchase flow.
|
|
182
|
-
|
|
183
|
-
---
|
|
184
|
-
|
|
185
|
-
### Assess/Convert Adapty
|
|
186
|
-
|
|
187
|
-
**What it does:** Evaluates whether the project can switch to Unity IAP 5 for app store purchase handling. Produces one of three outcomes: already in observer mode (no action), blockers detected (report + two choices), or no blockers (observer mode or full removal).
|
|
188
|
-
|
|
189
|
-
**Limitations:**
|
|
190
|
-
- **Adapty Paywall Builder** has no Unity IAP equivalent — and it is also **unavailable in Adapty's own Observer Mode**. Switching to observer mode loses Paywall Builder regardless of whether Unity IAP is involved.
|
|
191
|
-
- **Adapty A/B testing** has no Unity IAP equivalent. In observer mode, A/B testing is possible but requires significant manual instrumentation.
|
|
192
|
-
- **Cross-platform entitlement sync** has no Unity IAP equivalent without a custom backend.
|
|
193
|
-
- In **observer mode**, `Adapty.ReportTransaction()` must be called after every Unity IAP confirmed purchase. Missing this call means Adapty does not validate the receipt server-side.
|
|
194
|
-
|
|
195
|
-
**Best practices:**
|
|
196
|
-
- If your marketing team uses Adapty's Paywall Builder to update paywall layouts without app releases, note that this capability is lost in observer mode. Make sure all stakeholders understand this before proceeding.
|
|
197
|
-
- Full removal requires replacing any `AdaptyUI` / `PaywallView` shop UI with custom Unity UI before Adapty can be removed. Budget time for this work.
|
|
198
|
-
- Observer mode is the lower-risk path and preserves Adapty's webhook delivery and analytics integrations.
|
|
199
|
-
|
|
200
|
-
---
|
|
201
|
-
|
|
202
|
-
### Implement IAP D2C Capabilities
|
|
203
|
-
|
|
204
|
-
**What it does:** Adds Direct-to-Customer (D2C) third-party payment provider support (Stripe or Coda) via Unity Cloud, including remote catalog setup, deep link configuration, the built-in payment options picker UI, Apple/Google external purchase compliance tools, and entitlement delivery guidance.
|
|
205
|
-
|
|
206
|
-
**Limitations:**
|
|
207
|
-
- Requires **Unity IAP v5.4+**, **Unity Editor 2022.3+**, `com.unity.services.authentication` **v3.7.1+**, and `com.unity.services.core` **v1.18.0+**. All must be satisfied before any code is written.
|
|
208
|
-
- **Subscriptions are not supported** by IAP D2C Capabilities in v5.4. Only Consumable and NonConsumable products can be used.
|
|
209
|
-
- Requires a **Stripe or Coda account** connected in the Unity Cloud IAP dashboard, and Unity must enable D2C for your organization — contact your Unity Client Partner if it is not yet enabled.
|
|
210
|
-
- **Routing rules** must be configured in the Unity Dashboard before any player is offered a D2C payment option. Without a routing rule, no provider is offered even if a provider account is connected.
|
|
211
|
-
- **External web payments via Stripe/Coda are permitted in select regions only.** Apple and Google have their own program eligibility requirements. The developer is responsible for determining eligibility and meeting disclosure requirements — the skill does not perform compliance on your behalf.
|
|
212
|
-
- **Anonymous sign-in** must not be used as the authentication method. If a player's session token is lost (reinstall, app data clear), purchase history tied to an anonymous identity becomes unrecoverable.
|
|
213
|
-
- The **receipt format is different from standard IAP 5** — D2C purchases go through Unity Cloud, not the device's native store, so `order.Info.Apple` and `order.Info.Receipt` behave differently for D2C orders.
|
|
214
|
-
|
|
215
|
-
**Best practices:**
|
|
216
|
-
- Set up **routing rules** in the Unity Dashboard before testing. Without them, `GetEligiblePaymentProviders()` returns an empty list and the purchase UI never appears — this is a common "nothing happens" issue during initial integration.
|
|
217
|
-
- Use a **proxy HTML page** for the Success Redirect URL rather than a direct app-scheme URL. On some Android and iOS devices, direct app-scheme redirects from the payment provider domain are silently dropped. Host the proxy on a stable HTTPS domain you control.
|
|
218
|
-
- Use **`ShowPurchaseOption(catalogListingId)`** as the primary purchase entry point — it shows the built-in picker UI and handles provider selection automatically. Only fall back to `PurchaseProduct` directly when `GetEligiblePaymentProviders()` returns an empty `Providers` list.
|
|
219
|
-
- Configure a **Cancel Redirect URL** in the payment provider dashboard — without it, the checkout page has no "Back" button and players who change their mind are stuck in the browser.
|
|
220
|
-
- The SDK remembers the **last used payment provider per player per device** (provider memory). This is expected behavior, not a bug. Clear app data to reset it during testing.
|
|
221
|
-
- Deploy the **Deployment package** (`com.unity.services.deployment`) early — it is required to push `.ucat` product definitions to the Remote Catalog. Without it, the catalog cannot be deployed and `FetchRemoteCatalog()` returns no products.
|
|
222
|
-
|
|
223
|
-
---
|
|
224
|
-
|
|
225
|
-
## General Best Practices
|
|
226
|
-
|
|
227
|
-
- **Always let the skill scan first.** The pre-check (`pre-check.md`) detects third-party packages, native billing code, and existing Unity IAP versions before routing. Skipping it leads to incompatible changes.
|
|
228
|
-
- **Read the migration report before approving any code changes.** Every conversion path produces a report covering what will change, what blockers were found, and what manual steps remain. Review it before proceeding.
|
|
229
|
-
- **Never confirm a purchase before saving the reward.** An unconfirmed purchase re-delivers safely. A confirmed-but-unsaved one is gone permanently.
|
|
230
|
-
- **Subscribe to all failure events.** `OnProductsFetchFailed`, `OnPurchasesFetchFailed`, `OnStoreDisconnected` — not subscribing generates runtime warnings and leaves failures silently unhandled.
|
|
231
|
-
- **Subscribe to `OnPurchaseDeferred`.** Ask-to-Buy (iOS) and Google Play deferred purchases fire this event. Not subscribing silently drops them.
|
|
232
|
-
- **Use `#if USE_UNITY_IAP_V5` guards** for all migration work. The rollback path (remove the define from Player Settings) must work cleanly before shipping.
|
|
233
|
-
- **Test with real sandbox accounts on real devices.** Editor sandbox and emulators do not reproduce all edge cases — particularly pending purchases, deferred flows, and restore behavior.
|
|
@@ -1,21 +0,0 @@
|
|
|
1
|
-
using UnityEditor;
|
|
2
|
-
using UnityEditor.Build;
|
|
3
|
-
|
|
4
|
-
public class WebOptimizer
|
|
5
|
-
{
|
|
6
|
-
[MenuItem("Tools/Apply Web Release Settings")]
|
|
7
|
-
public static void Optimize()
|
|
8
|
-
{
|
|
9
|
-
var target = NamedBuildTarget.WebGL;
|
|
10
|
-
PlayerSettings.SetIl2CppCodeGeneration(target, Il2CppCodeGeneration.OptimizeSize);
|
|
11
|
-
PlayerSettings.SetManagedStrippingLevel(target, ManagedStrippingLevel.High);
|
|
12
|
-
PlayerSettings.stripUnusedMeshComponents = true;
|
|
13
|
-
PlayerSettings.WebGL.dataCaching = true;
|
|
14
|
-
PlayerSettings.WebGL.compressionFormat = WebGLCompressionFormat.Brotli;
|
|
15
|
-
PlayerSettings.WebGL.exceptionSupport = WebGLExceptionSupport.None;
|
|
16
|
-
PlayerSettings.WebGL.debugSymbolMode = WebGLDebugSymbolMode.Off;
|
|
17
|
-
PlayerSettings.WebGL.wasm2023 = true;
|
|
18
|
-
UnityEditor.WebGL.UserBuildSettings.codeOptimization =
|
|
19
|
-
UnityEditor.WebGL.WasmCodeOptimization.DiskSizeLTO;
|
|
20
|
-
}
|
|
21
|
-
}
|
|
@@ -1,11 +0,0 @@
|
|
|
1
|
-
# Albedo / diffuse (ETC1S, lossy, smallest)
|
|
2
|
-
toktx --bcmp --lower_left_maps_to_s0t0 output.ktx2 input.png
|
|
3
|
-
|
|
4
|
-
# Normal / metallic / detail (UASTC, high fidelity)
|
|
5
|
-
toktx --encode uastc --uastc_quality 2 --t2 --lower_left_maps_to_s0t0 output.ktx2 input.png
|
|
6
|
-
|
|
7
|
-
# Fix "ICC profile not found" errors
|
|
8
|
-
toktx --bcmp --assign_oetf srgb --lower_left_maps_to_s0t0 output.ktx2 input.png
|
|
9
|
-
|
|
10
|
-
# Linear data (normal maps, masks)
|
|
11
|
-
toktx --bcmp --assign_oetf linear --lower_left_maps_to_s0t0 output.ktx2 input.png
|
|
@@ -1,101 +0,0 @@
|
|
|
1
|
-
# Vivox Voice & Text Chat Skill Eval Suite
|
|
2
|
-
|
|
3
|
-
Evaluation suite for the `setup-vivox-voice-chat` skill, powered by [Promptfoo](https://www.promptfoo.dev/). Validates that the skill routes the model to the correct Vivox v16 APIs (no v4/legacy hallucinations) across init, channel join, and messaging.
|
|
4
|
-
|
|
5
|
-
## Prerequisites
|
|
6
|
-
|
|
7
|
-
- **Node.js** v18 or later
|
|
8
|
-
- A **LiteLLM API key** (from https://uai-litellm.internal.unity.com)
|
|
9
|
-
|
|
10
|
-
## Setup
|
|
11
|
-
|
|
12
|
-
### 1. Install Promptfoo
|
|
13
|
-
|
|
14
|
-
```bash
|
|
15
|
-
# Option A: install globally
|
|
16
|
-
npm install -g promptfoo
|
|
17
|
-
|
|
18
|
-
# Option B: use npx (no install needed)
|
|
19
|
-
npx promptfoo@latest eval
|
|
20
|
-
```
|
|
21
|
-
|
|
22
|
-
### 2. Configure your API key
|
|
23
|
-
|
|
24
|
-
```bash
|
|
25
|
-
cd evals/
|
|
26
|
-
cp .env.example .env
|
|
27
|
-
```
|
|
28
|
-
|
|
29
|
-
Open `.env` and set your personal LiteLLM key:
|
|
30
|
-
|
|
31
|
-
```
|
|
32
|
-
OPENAI_API_KEY=your-litellm-api-key-here
|
|
33
|
-
OPENAI_BASE_URL=https://uai-litellm.internal.unity.com
|
|
34
|
-
```
|
|
35
|
-
|
|
36
|
-
> **Important:** Never commit your `.env` file. It is already in `.gitignore`.
|
|
37
|
-
|
|
38
|
-
## Running the evals
|
|
39
|
-
|
|
40
|
-
All commands should be run from the `evals/` directory.
|
|
41
|
-
|
|
42
|
-
Use `-j 10` to run up to 10 eval requests concurrently.
|
|
43
|
-
|
|
44
|
-
### Run the full suite
|
|
45
|
-
|
|
46
|
-
```bash
|
|
47
|
-
promptfoo eval -j 10
|
|
48
|
-
```
|
|
49
|
-
|
|
50
|
-
### Run a specific test file
|
|
51
|
-
|
|
52
|
-
```bash
|
|
53
|
-
promptfoo eval --tests tests/init-and-login.yaml -j 10
|
|
54
|
-
promptfoo eval --tests tests/voice-channels.yaml -j 10
|
|
55
|
-
promptfoo eval --tests tests/text-chat.yaml -j 10
|
|
56
|
-
```
|
|
57
|
-
|
|
58
|
-
## Viewing results
|
|
59
|
-
|
|
60
|
-
### Terminal output
|
|
61
|
-
|
|
62
|
-
Results are printed to the terminal with pass/fail per assertion.
|
|
63
|
-
|
|
64
|
-
### Interactive web UI
|
|
65
|
-
|
|
66
|
-
```bash
|
|
67
|
-
promptfoo view
|
|
68
|
-
```
|
|
69
|
-
|
|
70
|
-
Opens a local UI (usually `http://localhost:15500`) for browsing results, filtering, and comparing runs.
|
|
71
|
-
|
|
72
|
-
## Assertions used
|
|
73
|
-
|
|
74
|
-
| Type | What it checks |
|
|
75
|
-
|---|---|
|
|
76
|
-
| `icontains` | Response contains a substring (case-insensitive), e.g. an exact Vivox API name |
|
|
77
|
-
| `not-icontains` | Response does NOT contain a substring (used to catch v4 legacy names like `Client.Instance`) |
|
|
78
|
-
| `llm-rubric` | An LLM judges whether the response meets a semantic requirement (e.g. correct init order) |
|
|
79
|
-
|
|
80
|
-
## Adding new tests
|
|
81
|
-
|
|
82
|
-
1. Create a new YAML file in `tests/`:
|
|
83
|
-
|
|
84
|
-
```yaml
|
|
85
|
-
- description: "Short description of what is being tested"
|
|
86
|
-
vars:
|
|
87
|
-
user_message: "The user request to test"
|
|
88
|
-
reference_content: "file://../references/your-reference.md" # optional
|
|
89
|
-
assert:
|
|
90
|
-
- type: icontains
|
|
91
|
-
value: "VivoxService.Instance.JoinGroupChannelAsync"
|
|
92
|
-
- type: not-icontains
|
|
93
|
-
value: "SendDirectedTextMessageAsync"
|
|
94
|
-
- type: llm-rubric
|
|
95
|
-
value: |
|
|
96
|
-
Describe the semantic requirement the response must meet.
|
|
97
|
-
```
|
|
98
|
-
|
|
99
|
-
2. Add the file to `promptfooconfig.yaml` under `tests:`.
|
|
100
|
-
|
|
101
|
-
3. Run it: `promptfoo eval --tests tests/your-new-test.yaml`.
|
|
@@ -1,32 +0,0 @@
|
|
|
1
|
-
description: "Vivox Voice & Text Chat Skill Eval Suite"
|
|
2
|
-
|
|
3
|
-
providers:
|
|
4
|
-
# Evaluated model: used by the test itself
|
|
5
|
-
- id: openai:chat:claude-sonnet-4-5
|
|
6
|
-
config:
|
|
7
|
-
max_tokens: 2048
|
|
8
|
-
|
|
9
|
-
prompts:
|
|
10
|
-
- id: eval-prompt
|
|
11
|
-
label: "Eval prompt"
|
|
12
|
-
raw: "{{skill_content}}\n\n{{reference_content}}\n\n{{custom_instructions}}\n\n## User Request\n\n{{user_message}}"
|
|
13
|
-
|
|
14
|
-
defaultTest:
|
|
15
|
-
options:
|
|
16
|
-
# Assertion judge: used by llm-rubric
|
|
17
|
-
provider: openai:chat:claude-sonnet-4-6
|
|
18
|
-
vars:
|
|
19
|
-
skill_content: file://../SKILL.md
|
|
20
|
-
reference_content: ""
|
|
21
|
-
custom_instructions: |
|
|
22
|
-
IMPORTANT:
|
|
23
|
-
- This is a planning eval. Do not emit MCP XML/tool-call tags.
|
|
24
|
-
- Refer to APIs with their exact names as documented in the provided skill and references. Do not invent or paraphrase symbol names.
|
|
25
|
-
- Do not mention tool names you will not call. If a step is inapplicable, explain the behavior without naming the omitted API.
|
|
26
|
-
- Always present the complete plan up front. If a step requires a user action, describe what you will do after it succeeds and what happens if it fails, in a single response.
|
|
27
|
-
- Answer only the step or phase requested by the user message. Do not include unrelated setup or migration content that was not asked for.
|
|
28
|
-
|
|
29
|
-
tests:
|
|
30
|
-
- file://tests/init-and-login.yaml
|
|
31
|
-
- file://tests/voice-channels.yaml
|
|
32
|
-
- file://tests/text-chat.yaml
|
|
@@ -1,76 +0,0 @@
|
|
|
1
|
-
# ============================================================
|
|
2
|
-
# WORKFLOW: Init + Login — correct order and re-init guard
|
|
3
|
-
# ============================================================
|
|
4
|
-
|
|
5
|
-
- description: "Cold-start init: UGS Core -> Auth -> Vivox Init -> Vivox Login, in order"
|
|
6
|
-
vars:
|
|
7
|
-
user_message: |
|
|
8
|
-
I'm adding Vivox to a fresh Unity project. Walk me through
|
|
9
|
-
the initialization from an empty MonoBehaviour Start method.
|
|
10
|
-
reference_content: file://../references/init-and-login.md
|
|
11
|
-
assert:
|
|
12
|
-
- type: icontains
|
|
13
|
-
value: "UnityServices.InitializeAsync"
|
|
14
|
-
- type: icontains
|
|
15
|
-
value: "AuthenticationService.Instance.SignInAnonymouslyAsync"
|
|
16
|
-
- type: icontains
|
|
17
|
-
value: "VivoxService.Instance.InitializeAsync"
|
|
18
|
-
- type: icontains
|
|
19
|
-
value: "VivoxService.Instance.LoginAsync"
|
|
20
|
-
- type: llm-rubric
|
|
21
|
-
value: |
|
|
22
|
-
The response MUST describe the four initialization calls in this
|
|
23
|
-
exact order:
|
|
24
|
-
1. UnityServices.InitializeAsync
|
|
25
|
-
2. AuthenticationService.Instance.SignInAnonymouslyAsync
|
|
26
|
-
3. VivoxService.Instance.InitializeAsync
|
|
27
|
-
4. VivoxService.Instance.LoginAsync
|
|
28
|
-
Any other ordering (e.g. Vivox init before UGS init, or Login
|
|
29
|
-
before Vivox init) is a failure.
|
|
30
|
-
The response MUST NOT use v4 legacy patterns (Client.Instance,
|
|
31
|
-
ILoginSession, AccountId, ChannelId) as callable code. It is
|
|
32
|
-
fine — even helpful — to mention those names in a "don't use
|
|
33
|
-
these" warning or migration note; a failure is only when the
|
|
34
|
-
code samples or step-by-step instructions actually invoke them.
|
|
35
|
-
|
|
36
|
-
- description: "Login with display name: LoginOptions.DisplayName"
|
|
37
|
-
vars:
|
|
38
|
-
user_message: |
|
|
39
|
-
After Vivox is initialized, sign the player in with the display name
|
|
40
|
-
"Sunbeam" and enable text-to-speech.
|
|
41
|
-
reference_content: file://../references/init-and-login.md
|
|
42
|
-
assert:
|
|
43
|
-
- type: icontains
|
|
44
|
-
value: "LoginOptions"
|
|
45
|
-
- type: icontains
|
|
46
|
-
value: "DisplayName"
|
|
47
|
-
- type: icontains
|
|
48
|
-
value: "EnableTTS"
|
|
49
|
-
- type: icontains
|
|
50
|
-
value: "VivoxService.Instance.LoginAsync"
|
|
51
|
-
- type: llm-rubric
|
|
52
|
-
value: |
|
|
53
|
-
The response must construct a LoginOptions with DisplayName set to
|
|
54
|
-
"Sunbeam" and EnableTTS set to true, then pass it to
|
|
55
|
-
VivoxService.Instance.LoginAsync. It must not USE the v4 AccountId
|
|
56
|
-
or ILoginSession types as callable code (mentioning them in a
|
|
57
|
-
"don't use" warning is acceptable — failure is only when the code
|
|
58
|
-
actually invokes them).
|
|
59
|
-
|
|
60
|
-
- description: "Double-init must warn about VxErrorAlreadyInitialized (5041)"
|
|
61
|
-
vars:
|
|
62
|
-
user_message: |
|
|
63
|
-
My Start method runs every time the main scene reloads and I'm
|
|
64
|
-
seeing Vivox errors. How do I stop it from re-initializing?
|
|
65
|
-
reference_content: file://../references/init-and-login.md
|
|
66
|
-
assert:
|
|
67
|
-
- type: icontains
|
|
68
|
-
value: "5041"
|
|
69
|
-
- type: llm-rubric
|
|
70
|
-
value: |
|
|
71
|
-
The response must identify the underlying issue as
|
|
72
|
-
VivoxService.Instance.InitializeAsync being called more than once,
|
|
73
|
-
cite the 5041 VxErrorAlreadyInitialized error, and propose a fix
|
|
74
|
-
such as guarding with IsInitialized or making the bootstrap
|
|
75
|
-
object DontDestroyOnLoad. The fix must NOT be to catch and
|
|
76
|
-
swallow the exception.
|
|
@@ -1,65 +0,0 @@
|
|
|
1
|
-
# ============================================================
|
|
2
|
-
# WORKFLOW: Text chat — channel messages, directed messages, history
|
|
3
|
-
# ============================================================
|
|
4
|
-
|
|
5
|
-
- description: "Send a channel message and receive channel messages"
|
|
6
|
-
vars:
|
|
7
|
-
user_message: |
|
|
8
|
-
Send the text "gg" into the "lobby" channel from a UI button, and
|
|
9
|
-
log every message received in that channel to the console.
|
|
10
|
-
reference_content: file://../references/text-chat.md
|
|
11
|
-
assert:
|
|
12
|
-
- type: icontains
|
|
13
|
-
value: "VivoxService.Instance.SendChannelTextMessageAsync"
|
|
14
|
-
- type: icontains
|
|
15
|
-
value: "VivoxService.Instance.ChannelMessageReceived"
|
|
16
|
-
- type: icontains
|
|
17
|
-
value: "VivoxMessage"
|
|
18
|
-
- type: llm-rubric
|
|
19
|
-
value: |
|
|
20
|
-
The response must call VivoxService.Instance.SendChannelTextMessageAsync
|
|
21
|
-
with channelName "lobby" and message "gg", AND subscribe to
|
|
22
|
-
VivoxService.Instance.ChannelMessageReceived with a handler that
|
|
23
|
-
takes a VivoxMessage and logs at least MessageText and
|
|
24
|
-
SenderDisplayName. It must NOT wire the send path to the
|
|
25
|
-
DirectedMessageReceived event.
|
|
26
|
-
|
|
27
|
-
- description: "Send a directed (direct) message — must use SendDirectTextMessageAsync, not SendDirected..."
|
|
28
|
-
vars:
|
|
29
|
-
user_message: |
|
|
30
|
-
Whisper "meet me at the north gate" to the player whose PlayerId
|
|
31
|
-
is "abc123".
|
|
32
|
-
reference_content: file://../references/text-chat.md
|
|
33
|
-
assert:
|
|
34
|
-
- type: icontains
|
|
35
|
-
value: "VivoxService.Instance.SendDirectTextMessageAsync"
|
|
36
|
-
- type: llm-rubric
|
|
37
|
-
value: |
|
|
38
|
-
The response must call VivoxService.Instance.SendDirectTextMessageAsync
|
|
39
|
-
with playerId "abc123" and the exact message
|
|
40
|
-
"meet me at the north gate". The method name must be
|
|
41
|
-
SendDirectTextMessageAsync (with "Direct", no "ed" before "Text").
|
|
42
|
-
SendDirectedTextMessageAsync does not exist in the SDK and MUST
|
|
43
|
-
NOT be USED as callable code — but mentioning it in a "common
|
|
44
|
-
hallucination, don't use this" warning is fine and even helpful.
|
|
45
|
-
The response may also mention subscribing to the
|
|
46
|
-
DirectedMessageReceived event on the recipient side.
|
|
47
|
-
|
|
48
|
-
- description: "Fetch the most recent channel chat history"
|
|
49
|
-
vars:
|
|
50
|
-
user_message: |
|
|
51
|
-
Fetch the last 25 messages from the "lobby" channel and print them
|
|
52
|
-
to the console in oldest-to-newest order.
|
|
53
|
-
reference_content: file://../references/text-chat.md
|
|
54
|
-
assert:
|
|
55
|
-
- type: icontains
|
|
56
|
-
value: "GetChannelTextMessageHistoryAsync"
|
|
57
|
-
- type: llm-rubric
|
|
58
|
-
value: |
|
|
59
|
-
The response must call
|
|
60
|
-
VivoxService.Instance.GetChannelTextMessageHistoryAsync with
|
|
61
|
-
channelName "lobby" and requestSize 25 (or equivalent). It must
|
|
62
|
-
note that the returned collection is newest-first and must
|
|
63
|
-
reverse the collection (or iterate in reverse) before printing to
|
|
64
|
-
achieve oldest-to-newest ordering. It should reference
|
|
65
|
-
VivoxMessage.SenderDisplayName and VivoxMessage.MessageText.
|
|
@@ -1,74 +0,0 @@
|
|
|
1
|
-
# ============================================================
|
|
2
|
-
# WORKFLOW: Voice channels — group, positional, join lifecycle
|
|
3
|
-
# ============================================================
|
|
4
|
-
|
|
5
|
-
- description: "Join a lobby group channel with voice and text"
|
|
6
|
-
vars:
|
|
7
|
-
user_message: |
|
|
8
|
-
The player is logged into Vivox. Have them join a non-positional
|
|
9
|
-
channel called "lobby" with both voice and text enabled.
|
|
10
|
-
reference_content: file://../references/voice-channels.md
|
|
11
|
-
assert:
|
|
12
|
-
- type: icontains
|
|
13
|
-
value: "VivoxService.Instance.JoinGroupChannelAsync"
|
|
14
|
-
- type: icontains
|
|
15
|
-
value: "ChatCapability.TextAndAudio"
|
|
16
|
-
- type: not-icontains
|
|
17
|
-
value: "JoinPositionalChannelAsync"
|
|
18
|
-
- type: not-icontains
|
|
19
|
-
value: "IChannelSession"
|
|
20
|
-
- type: llm-rubric
|
|
21
|
-
value: |
|
|
22
|
-
The response must call VivoxService.Instance.JoinGroupChannelAsync
|
|
23
|
-
with the channel name "lobby" and ChatCapability.TextAndAudio. It
|
|
24
|
-
must NOT use the positional or echo join methods, and must not
|
|
25
|
-
use any v4 IChannelSession API.
|
|
26
|
-
|
|
27
|
-
- description: "Join a 3D positional channel with Channel3DProperties"
|
|
28
|
-
vars:
|
|
29
|
-
user_message: |
|
|
30
|
-
Set up proximity voice: players near each other in the world should
|
|
31
|
-
hear each other, and voices fall off with distance. Name the
|
|
32
|
-
channel "world-proximity".
|
|
33
|
-
reference_content: file://../references/voice-channels.md
|
|
34
|
-
assert:
|
|
35
|
-
- type: icontains
|
|
36
|
-
value: "Channel3DProperties"
|
|
37
|
-
- type: icontains
|
|
38
|
-
value: "Set3DPosition"
|
|
39
|
-
- type: not-icontains
|
|
40
|
-
value: "JoinGroupChannelAsync"
|
|
41
|
-
- type: llm-rubric
|
|
42
|
-
value: |
|
|
43
|
-
This is a planning eval: judge the plan's correctness, not whether
|
|
44
|
-
it names APIs literally or includes runnable code. The plan must:
|
|
45
|
-
(a) identify positional (3D) channels as the mechanism — mentioning
|
|
46
|
-
"positional channel", "3D channel", or JoinPositionalChannelAsync
|
|
47
|
-
all count, since positional channels have exactly one join method;
|
|
48
|
-
(b) use the channel name "world-proximity";
|
|
49
|
-
(c) configure Channel3DProperties (naming at least the audible/
|
|
50
|
-
conversational distance concept, and ideally the fade model);
|
|
51
|
-
(d) state that each player's 3D position needs a per-frame update
|
|
52
|
-
via Set3DPosition (or an equivalent per-frame transform sync) so
|
|
53
|
-
distance attenuation actually works;
|
|
54
|
-
(e) make clear that awaiting the join call is not sufficient —
|
|
55
|
-
ChannelJoined must be subscribed to first for the join to be
|
|
56
|
-
observable.
|
|
57
|
-
|
|
58
|
-
- description: "Subscribe to ChannelJoined BEFORE calling JoinGroupChannelAsync"
|
|
59
|
-
vars:
|
|
60
|
-
user_message: |
|
|
61
|
-
When I call JoinGroupChannelAsync my UI never activates for the
|
|
62
|
-
newly joined channel. What am I doing wrong?
|
|
63
|
-
reference_content: file://../references/voice-channels.md
|
|
64
|
-
assert:
|
|
65
|
-
- type: icontains
|
|
66
|
-
value: "ChannelJoined"
|
|
67
|
-
- type: llm-rubric
|
|
68
|
-
value: |
|
|
69
|
-
The response must diagnose the problem as the ChannelJoined event
|
|
70
|
-
being subscribed AFTER the join call, and instruct the user to
|
|
71
|
-
subscribe to VivoxService.Instance.ChannelJoined BEFORE calling
|
|
72
|
-
JoinGroupChannelAsync. It must clearly state that awaiting the
|
|
73
|
-
JoinGroupChannelAsync call does NOT mean the join is complete —
|
|
74
|
-
the join completes when the ChannelJoined event fires.
|