@seriousgaming/sdk-tools 0.0.0-stage → 1.0.0-alpha.1

package/dist/docs/guides/llm-integration.md

@@ -0,0 +1,635 @@
+# LLM integration guide
+
+The Serious Gaming SDK generates text through the Serious Gaming platform's LLM pipeline, the same `generateText` path production game code runs. The engine can generate content dynamically using LLMs without configuring any provider credentials: all you need is a platform API key. This guide covers setup, usage, and best practices.
+
+## Table of contents
+
+- [Overview](#overview)
+- [Configuration](#configuration)
+- [LLM tiers](#llm-tiers)
+- [Response shape](#response-shape)
+- [Using LLMs in engines](#using-llms-in-engines)
+- [Tool use](#tool-use)
+- [Error handling](#error-handling)
+- [Declare the APIs a resource uses](#declare-the-apis-a-resource-uses)
+- [Using LLMs in user interfaces](#using-llms-in-user-interfaces)
+- [Best practices](#best-practices)
+- [Limitations](#limitations)
+
+## Overview
+
+LLM integration allows you to:
+
+- Generate reports based on game data
+- Create dynamic content during game execution
+- Process context and generate insights
+- Structure responses as JSON via JSON schemas
+
+### Key points
+
+- LLM calls are **async** and only work in async functions
+- Text generation runs through the Serious Gaming platform, authenticated with a platform API key
+- LLM client is automatically instantiated and passed as callbacks
+- Three tiers are available: `quick`, `standard`, and `expert`; the platform resolves each tier to a configured model
+- All errors have clear configuration instructions
+
+## Configuration
+
+The only required setting is a Serious Gaming platform API key. Issue one from your organisation's settings page on the platform; the key is shown exactly once at creation. The key acts as the account that created it, and that account needs access to the platform LLM API, which Business Simulations grants: ask your contact there to enable it. The same key is used by `package publish`.
+
+Configuration follows this priority (highest to lowest):
+
+1. **CLI flags** - Highest priority, override everything
+2. **Environment variables** - Override .env file
+3. **.env file** - Overrides global settings
+4. **Global settings file** - Lowest priority
+
+### Method 1: global settings (recommended)
+
+Set the key once and every project picks it up:
+
+```bash
+pnpm run sdk config set platform.apiKey sg_xxx
+```
+
+### Method 2: environment variables
+
+```bash
+export SERIOUSGAMING_API_KEY=sg_xxx
+export SERIOUSGAMING_API_HOST=https://app.businesssimulations.com  # optional
+
+pnpm run sdk dev
+```
+
+### Method 3: .env file
+
+Create `.env` in your project root:
+
+```bash
+SERIOUSGAMING_API_KEY=sg_xxx
+SERIOUSGAMING_API_HOST=https://app.businesssimulations.com   # optional
+```
+
+Then run:
+
+```bash
+pnpm run sdk dev
+```
+
+### Method 4: CLI flags
+
+```bash
+pnpm run sdk dev --api-key sg_xxx
+```
+
+Prefer the env var or global settings so the secret stays out of shell history.
+
+### Configuration options
+
+| Variable                 | Required | Default                               | Description                          |
+| ------------------------ | -------- | ------------------------------------- | ------------------------------------ |
+| `SERIOUSGAMING_API_KEY`  | Yes      | None                                  | Your Serious Gaming platform API key |
+| `SERIOUSGAMING_API_HOST` | No       | `https://app.businesssimulations.com` | Platform origin to call              |
+
+Which model each tier maps to is configured on the platform, not in your project. Raw model names are not accepted; calls select a model by `tier` only.
+
+## LLM tiers
+
+Each `generateText` call can specify a **tier** that controls which model is used. This lets you balance cost and capability per call rather than setting a single model for everything.
+
+### Available tiers
+
+| Tier       | When to use                                                                |
+| ---------- | -------------------------------------------------------------------------- |
+| `quick`    | Fast, low-cost tasks: classification, simple lookups, short labels         |
+| `standard` | Typical generation: summaries, reports, moderate reasoning                 |
+| `expert`   | Complex reasoning: deep analysis, multi-step evaluation, nuanced judgement |
+
+### Specifying a tier
+
+Pass `tier` in the options object. Omitting it defaults to `'quick'`.
+
+```typescript
+// quick (the default: omit tier for lowest cost)
+const label = await callbacks.generateText('Classify as: positive, neutral, or negative: ' + text);
+
+// standard
+const summary = await callbacks.generateText({
+    prompt: 'Summarise the market situation in one paragraph.',
+    tier: 'standard',
+});
+
+// expert
+const analysis = await callbacks.generateText({
+    prompt: 'Provide a deep strategic analysis of these business metrics...',
+    tier: 'expert',
+});
+```
+
+### Fallback behaviour
+
+Tier-to-model resolution happens on the platform. If a tier is not configured there, it falls back down the chain:
+
+```
+expert → standard → quick
+```
+
+This means game code can request any tier without worrying about platform configuration; the call always resolves to a model.
+
+## Response shape
+
+`generateText` returns a content-block array plus a `stopReason`:
+
+```typescript
+interface LLMResponse {
+    content: Array<
+        | { type: 'text'; text: string }
+        | { type: 'toolUse'; id: string; name: string; input: unknown }
+    >;
+    stopReason: 'endTurn' | 'toolUse' | 'maxTokens' | 'refusal';
+    usage?: { inputTokens: number; outputTokens: number };
+}
+```
+
+The blocks let one response carry text and one or more tool-use intents. The stop reason distinguishes normal completion (`'endTurn'`), tool use (`'toolUse'`), truncation (`'maxTokens'`), and refusal (`'refusal'`). When the provider reports token counts, `usage` carries the input and output token totals for the call.
+
+For the common case where you just want the assistant's text, use `getText`:
+
+```typescript
+import { getText } from '@seriousgaming/sdk';
+
+const response = await callbacks.generateText('Summarise this: ' + text);
+const summary = getText(response); // concatenates every text block
+```
+
+A `stopReason` of `'refusal'` indicates the provider declined to generate. Callers typically want a different branch (rewrite the prompt, escalate to a human) than a normal completion, so do not treat refusals as ordinary text.
+
+## Using LLMs in engines
+
+### Basic usage
+
+All engine functions are async and receive a `callbacks` parameter:
+
+```typescript
+import { RunFunction } from '@seriousgaming/sdk';
+import type { Context, Settings } from './types';
+
+import { getText } from '@seriousgaming/sdk';
+
+export const Run: RunFunction<Context, Settings> = async ({ context, settings, callbacks }) => {
+    // Simple text generation (quick tier by default)
+    const response = await callbacks.generateText('Your prompt here');
+
+    return {
+        capabilities: {
+            ...context,
+            report: {
+                ReportContent: getText(response),
+            },
+        },
+    };
+};
+```
+
+### Structured JSON responses
+
+Pass a `jsonSchema` to get predictable, structured output. The reply still arrives as content blocks, so walk it to text with `getText` before parsing:
+
+```typescript
+export const Run: RunFunction<Context, Settings> = async ({ context, callbacks }) => {
+    const response = await callbacks.generateText({
+        prompt: 'Analyze the business metrics and provide insights',
+        tier: 'expert',
+        jsonSchema: {
+            type: 'object',
+            properties: {
+                overallHealth: { type: 'string', enum: ['excellent', 'good', 'fair', 'poor'] },
+                topConcern: { type: 'string' },
+                recommendation: { type: 'string' },
+                riskLevel: { type: 'number', minimum: 0, maximum: 10 },
+            },
+            required: ['overallHealth', 'topConcern', 'recommendation', 'riskLevel'],
+        },
+    });
+
+    const analysis = JSON.parse(getText(response));
+
+    return {
+        capabilities: {
+            ...context,
+            analysis: {
+                OverallHealth: analysis.overallHealth,
+                TopConcern: analysis.topConcern,
+                Recommendation: analysis.recommendation,
+                RiskLevel: analysis.riskLevel,
+            },
+        },
+    };
+};
+```
+
+`JSON.parse` throws if the model returns something that is not valid JSON, so wrap it in the same `try` as the call when a failed parse should not fail the tick (see [Error handling](#error-handling)).
+
+### Data-driven report generation
+
+Extract context data and pass to LLM:
+
+```typescript
+export const Run: RunFunction<Context, Settings> = async ({ context, settings, callbacks }) => {
+    const production = context.production.Production;
+    const demand = context.demand.Demand;
+    const sales = context.sales.Sales;
+    const revenue = context.revenue.Revenue;
+    const costs = context.costs.Costs;
+    const profit = context.profit.Profit;
+
+    const prompt = `Generate a professional business summary based on:
+- Production: ${production} units
+- Demand: ${demand} units
+- Sales: ${sales} units
+- Revenue: $${revenue}
+- Costs: $${costs}
+- Profit: $${profit}
+
+Create a markdown report with:
+1. Executive summary
+2. Key metrics table
+3. Performance analysis
+4. Recommendations`;
+
+    const report = await callbacks.generateText({ prompt, tier: 'standard' });
+
+    return {
+        capabilities: {
+            ...context,
+            report: {
+                ReportContent: getText(report),
+            },
+        },
+    };
+};
+```
+
+### Mixed tiers in one engine
+
+Use the cheapest tier that does the job, and reserve `expert` for genuinely complex reasoning:
+
+```typescript
+export const Run: RunFunction<Context, Settings> = async ({ context, callbacks }) => {
+    // quick: simple classification
+    const sentiment = await callbacks.generateText(
+        `Classify as positive, neutral, or negative: "${context.feedback.Text}"`
+    );
+
+    // standard: prose summary
+    const summary = await callbacks.generateText({
+        prompt: `Summarise this production data in one paragraph: Production=${context.production.Production}`,
+        tier: 'standard',
+    });
+
+    // expert: complex strategic analysis
+    const strategy = await callbacks.generateText({
+        prompt: `Given these metrics, recommend a long-term pricing strategy with risk analysis:
+    Summary: ${getText(summary)}
+    Sentiment: ${getText(sentiment)}`,
+        tier: 'expert',
+    });
+
+    return {
+        capabilities: {
+            ...context,
+            report: {
+                Sentiment: getText(sentiment),
+                Summary: getText(summary),
+                Strategy: getText(strategy),
+            },
+        },
+    };
+};
+```
+
+## Tool use
+
+`generateText` accepts a `tools` array. When set, the model may emit `toolUse` content blocks instead of (or alongside) text. The SDK never executes anything itself: your code runs the tool and feeds the result back as a `toolResult` block on the next call.
+
+The loop is:
+
+1. Invoke `generateText({ messages, tools })`.
+2. If `stopReason === 'toolUse'`, run the tool(s) in your own code and build `toolResult` blocks.
+3. Invoke `generateText` again with the prior messages plus the assistant response plus a user message carrying the `toolResult` blocks.
+4. Repeat until `stopReason !== 'toolUse'`.
+
+Always cap iterations: the loop lives in your code and runaway loops are your responsibility.
+
+```typescript
+import { getText, LLMMessage, LLMTool } from '@seriousgaming/sdk';
+
+const MAX_ITERATIONS = 10;
+
+const tools: LLMTool[] = [
+    {
+        name: 'lookup_user',
+        description: 'Look up a user by ID',
+        inputSchema: {
+            type: 'object',
+            properties: { userId: { type: 'string' } },
+            required: ['userId'],
+        },
+    },
+];
+
+async function runLookup(name: string, input: unknown): Promise<string> {
+    // Your handler. Tool results are strings; serialise structured data as JSON.
+    if (name === 'lookup_user') {
+        const { userId } = input as { userId: string };
+        return JSON.stringify({ id: userId, name: 'Alex' });
+    }
+    return JSON.stringify({ error: `unknown tool: ${name}` });
+}
+
+const messages: LLMMessage[] = [{ role: 'user', content: 'Who is u-1?' }];
+
+for (let i = 0; i < MAX_ITERATIONS; i++) {
+    const response = await callbacks.generateText({ messages, tools });
+
+    // Append the assistant turn verbatim so the next call has full context.
+    messages.push({ role: 'assistant', content: response.content });
+
+    if (response.stopReason !== 'toolUse') {
+        const finalText = getText(response);
+        // …use finalText…
+        break;
+    }
+
+    const toolResults = await Promise.all(
+        response.content
+            .filter((b) => b.type === 'toolUse')
+            .map(async (b) => {
+                const result = await runLookup(b.name, b.input);
+                return {
+                    type: 'toolResult' as const,
+                    toolUseId: b.id,
+                    content: result,
+                };
+            })
+    );
+
+    messages.push({ role: 'user', content: toolResults });
+}
+```
+
+Tool names must match `^[a-zA-Z0-9_-]{1,64}$`. The dev server validates this at the boundary and throws an `LLMClientError` before making a request, so authoring mistakes fail fast.
+
+Tool handlers run in caller code. The SDK boundary is unchanged: declaring `generateText` in `uses` is enough; there is no separate `ApiUse` gate for tools.
+
+## Error handling
+
+### When LLM is not configured
+
+If no platform API key is configured, `generateText` throws: 'generateText needs a Serious Gaming platform API key, and none is configured.' A CLI command whose tick or endpoint call failed this way exits 5 with the code `LLM_NOT_CONFIGURED`, and its hint lists the ways to set a key (see [Configuration](#configuration)). Restart the dev server after setting one.
+
+A 401 response means the key is invalid or has been revoked. A 403 response means the key works but the account that created it does not have access to the platform LLM API. Ask Business Simulations to grant it.
+
+### Handling API errors
+
+```typescript
+export const Run: RunFunction<Context, Settings> = async ({ context, callbacks }) => {
+    try {
+        const response = await callbacks.generateText({ prompt: 'Your prompt', tier: 'standard' });
+        return {
+            capabilities: {
+                ...context,
+                report: {
+                    ReportContent: getText(response),
+                },
+            },
+        };
+    } catch (error) {
+        console.error('LLM generation failed:', error);
+        // Gracefully handle failure
+        return {
+            capabilities: {
+                ...context,
+                report: {
+                    ReportContent: 'Report generation failed',
+                },
+            },
+        };
+    }
+};
+```
+
+## Declare the APIs a resource uses
+
+`generateText` is one of four callbacks that reach a service outside the game. Every logic-bearing resource (the engine, a hook, the tick controller, an endpoint) lists the ones it calls in its bundle's `uses` array:
+
+```json
+{
+    "definition": {
+        "uses": ["generateText"]
+    }
+}
+```
+
+The declaration does three jobs at once:
+
+- **A safety boundary.** A resource that was not written to call a model cannot start doing so by accident or through a compromised dependency
+- **Operator visibility.** The built package's `seriousgaming.json` carries every bundle's `uses`, so an operator can forecast LLM cost, review what reaches a third-party service, or see that a game needs document parsing, without reading its source
+- **A guarantee.** A callback a resource declares is always there when it runs
+
+The mental model is a phone app's permissions: a static declaration, enforced at run time, visible without inspecting code.
+
+### The APIs it gates
+
+| API              | What it does                                                                                                           |
+| ---------------- | ---------------------------------------------------------------------------------------------------------------------- |
+| `generateText`   | LLM text generation, including [tool use](#tool-use)                                                                   |
+| `extractText`    | Document text extraction (currently PDF); see [File handling](./file-handling.md)                                      |
+| `generateSpeech` | Text-to-speech, returning an audio blob; see [Speech generation](./speech-generation.md)                               |
+| `writeBlob`      | Writes raw bytes to the blob store and returns a `blob:<id>` reference; most useful in an endpoint that returns a file |
+
+The list is closed and fixed by the SDK (see [`ApiUse`](../api/sdk.md#apiuse)). Every other callback is runtime plumbing and is never gated: logic always has `readAsset` and `getPastState`, and a UI's `getAssetUrl`, `uploadBlob`, `requestTick` and the rest are always there when the surface exposes them.
+
+### What happens when a resource calls one it did not declare
+
+The runtime wraps each resource's `callbacks`, and calling a gated callback the bundle did not declare throws an `UndeclaredApiUseError` naming the resource and the callback. Enforcement is the same in dev and production; there is no soft mode.
+
+The generated `types.ts` catches it earlier. Its `Callbacks` alias holds the declared callbacks, each required rather than optional, plus the ungated ones:
+
+```typescript
+// uses: ['generateText']
+export type Callbacks = LogicCallbacks<'generateText'>;
+
+// uses: [] (or omitted)
+export type Callbacks = LogicCallbacks;
+```
+
+Type `Init`, `Run`, a `HookFunction` or a `GetTickControllerFunction` with it, and an undeclared call fails at type-check. A declared one needs no `?.` or `!`: call `callbacks.generateText(...)` directly, never behind an `if (callbacks.generateText)` check.
+
+Only the resource that owns the `callbacks` argument declares. A helper function that calls `callbacks.generateText` has no bundle of its own, so the resource that calls the helper declares `'generateText'`.
+
+### A declared service is always present
+
+The other half of the declaration is a promise from the runtime. A runtime that cannot provide a service a package declares refuses the whole package when it loads it, naming the service. The dev server provides all four, so it refuses nothing on this ground.
+
+A service that is provided but fails when called is different: the package loads, and that call rejects with a typed error. A missing key is the usual case: `generateText` without `SERIOUSGAMING_API_KEY` rejects with `LLMClientError`, and `generateSpeech` without an ElevenLabs key with `TtsClientError` (see [Environment variables](../environment-variables.md)). Handle those as you would any failed call ([Error handling](#error-handling)).
+
+### User interfaces declare no `uses`
+
+A UI bundle has no `uses` field, and loading or building a package that gives one to any user interface fails, pointing at endpoints. A UI reaches these services by calling an [endpoint](./endpoints.md) whose own bundle declares them, as the next section shows.
+
+## Using LLMs in user interfaces
+
+User interfaces cannot call `generateText` directly. A UI reaches the LLM through an [endpoint](./endpoints.md) that wraps `generateText` server-side, and invokes it through `callbacks.endpoints['<endpoint-id>']`. Tier resolution happens on the platform and applies to endpoints exactly as it does to the engine.
+
+### The endpoint
+
+The endpoint bundle declares `"uses": ["generateText"]` and runs the LLM call server-side:
+
+```typescript
+// endpoints/chat/logic/logic.ts
+import { getText } from '@seriousgaming/sdk';
+import type { Handler } from '../types';
+
+export const Handle: Handler = async ({ request, callbacks }) => {
+    const response = await callbacks.generateText({
+        messages: request.messages,
+        systemPrompt:
+            request.systemPrompt ?? 'You are a helpful business advisor. Provide concise guidance.',
+        tier: 'standard',
+    });
+    return { reply: getText(response) };
+};
+```
+
+Declare the endpoint in the UI bundle's `endpoints` array. The endpoint validates the request against its `requestSchema` and the reply against its `responseSchema`, so both sides stay typed.
+
+Engines and endpoints receive the session locale as `meta.locale`, so either can ask the model to answer in the session's language on its own. When a UI needs its own localised system prompt, translate it in the UI and pass it to the endpoint in the request.
+
+The [user interfaces guide](./ui.md#using-llms-in-user-interfaces) carries the UI half in full: a chat component that persists its conversation in a capability, localised prompts, markdown rendering, and error handling. To build the endpoint, follow [Create an endpoint](../recipes/create-endpoint.md), and see [Call an endpoint from a UI](../recipes/call-endpoint-from-ui.md) for the call itself.
+
+## Best practices
+
+### 1. Default to `quick`, opt up deliberately
+
+Every call defaults to `quick` when no tier is specified. Escalate only when the task genuinely requires it, which keeps costs down: `standard` for a paragraph-length summary, `expert` sparingly, for tasks where reasoning quality visibly matters. See [Mixed tiers in one engine](#mixed-tiers-in-one-engine).
+
+### 2. Keep prompts specific
+
+❌ Bad:
+
+```typescript
+const response = await callbacks.generateText('Generate something');
+```
+
+✅ Good:
+
+```typescript
+const response = await callbacks.generateText(
+    `Generate a business report for production=${production} units, demand=${demand} units`
+);
+```
+
+### 3. Structure responses when possible
+
+Use a `jsonSchema` for predictable, structured output whenever your code reads the reply rather than displaying it. See [Structured JSON responses](#structured-json-responses).
+
+### 4. Extract context data clearly
+
+```typescript
+// Good: Extract all needed data upfront
+const { Production } = context.production;
+const { Demand } = context.demand;
+
+const prompt = `Production: ${Production}, Demand: ${Demand}`;
+```
+
+### 5. Localise prompts
+
+Every participant in a session shares one locale, and logic receives it as `meta.locale`. Ask for prose in that language rather than generating it in one language and translating later:
+
+```typescript
+export const Run: RunFunction<Context, Settings> = async ({ context, meta, callbacks }) => {
+    const response = await callbacks.generateText({
+        systemPrompt: `Write in the language with BCP 47 tag ${meta.locale}.`,
+        prompt: `Summarise this round: ${JSON.stringify(context.sales)}`,
+    });
+    // ...
+};
+```
+
+Fixed interface text still belongs in the UI's i18n resources.
+
+### 6. Put stable content in `systemPrompt`
+
+The platform's LLM pipeline applies prompt caching automatically: when the provider supports it, the stable prefix is stored on the first call and reused on subsequent calls within the cache window, cutting input cost and time to first token.
+
+For repeated calls (iterate-style flows, multi-turn assistants, batched analyses), keep your stable content (instructions, role description, reference data, JSON schema scaffolds) in `systemPrompt` and put only the per-call variation in `prompt` or the trailing `messages` entries.
+
+```typescript
+// Cache-friendly: long stable prefix in systemPrompt, short variation in prompt
+const response = await callbacks.generateText({
+    systemPrompt: STABLE_INSTRUCTIONS_AND_REFERENCE_DATA, // identical across turns
+    prompt: `Latest observation: ${observation}`,
+    tier: 'standard',
+});
+```
+
+Providers that don't support prompt caching silently ignore the annotation, so the same code stays fast everywhere.
+
+### 7. Cache or limit LLM calls
+
+Don't call LLM every tick for the same data:
+
+```typescript
+// Bad: Called every tick, expensive
+export const Run: RunFunction = async ({ context, callbacks }) => {
+  const report = await callbacks.generateText(...);
+  return { capabilities: { ...context, report } };
+};
+
+// Good: Call only when data changes
+export const Run: RunFunction = async ({ context, callbacks }) => {
+  const hasDataChanged = context.production.Production !==
+                         context['previousProduction'].Production;
+
+  if (hasDataChanged) {
+    const report = await callbacks.generateText({ prompt: '...', tier: 'standard' });
+    return { capabilities: { ...context, report } };
+  }
+
+  return { capabilities: context };
+};
+```
+
+## Limitations
+
+### Server-side only
+
+- LLM generation runs in the engine, hooks, tick controllers, or endpoints, never in a UI directly
+- Generated prose follows the session locale in `meta.locale`; fixed interface text is localised by UIs
+- A session has one locale, so each call generates in one language
+
+### API limitations
+
+- Subject to the platform's rate limits and the underlying provider's costs
+- Calls require a platform API key from an account with LLM access, which Business Simulations grants
+- No offline support (requires a connection to the platform)
+- Responses may vary slightly (non-deterministic)
+
+### Performance
+
+- LLM calls add latency to tick execution
+- `quick` calls are typically fastest; `expert` calls may take several seconds
+- Consider generating reports less frequently
+- Default to `quick` tier where output quality is not critical
+
+## See also
+
+- [The CLI and the dev server](../cli-and-dev-server.md) - what the dev server is and how to drive it
+- [Environment variables](../environment-variables.md) - Full env-var reference (LLM, TTS, publishing)
+- [User interfaces guide](./ui.md#using-llms-in-user-interfaces) - Building LLM-powered UIs through endpoints
+- [Call an endpoint from a UI](../recipes/call-endpoint-from-ui.md) - how a UI reaches LLM, document, and speech services
+- [Engine development](./engine.md) - Engine creation and patterns
+- [Service callbacks in tests](./engine-testing.md#service-callbacks-are-stubbed) - The offline `generateText` stub, and choosing its answer in a test
+- [Declare API uses](../recipes/declare-api-uses.md) - Step-by-step `uses` declaration