generalcontext 0.2.0 → 0.3.0

package/README.md

@@ -1,6 +1,8 @@
 # generalcontext
 
-Record your agent's model calls for General Context. Your agent never waits on us, and a failure on our side is logged and dropped, never thrown. No runtime dependencies. Supports OpenAI (`chat.completions`, `responses`) and Anthropic (`messages`), streamed or not.
+Record your AI agent's model calls for [General Context](https://generalcontextlabs.com/docs). We read your agent's production conversations and find where it goes wrong, and each failure you confirm becomes an RL and eval task built from the conversation. Your tasks together are your benchmark.
+
+Recording never gets in your agent's way: calls are sent in the background, a failure on our side is logged and dropped, never thrown, and there are no runtime dependencies.
 
 ## Install
 
@@ -10,110 +12,53 @@ npm install generalcontext
 
 Set `GENERALCONTEXT_API_KEY` to the key from your workspace's Connect page.
 
-## Initialize
+## Record
 
-Call once per process, before your agent makes model calls. Every OpenAI and Anthropic client is recorded, including ones created earlier.
+Call `initialize()` once per process, before your agent makes model calls. Pass the client modules you use: it is required whenever a bundler is involved.
 
 ```ts
 import OpenAI from 'openai';
 import Anthropic from '@anthropic-ai/sdk';
-import { initialize } from 'generalcontext';
+import { initialize, withSession, flush } from 'generalcontext';
 
 initialize({ appName: 'support-agent', instrumentModules: { openAI: OpenAI, anthropic: Anthropic } });
-```
-
-## Configuration
-
-| Option | Default | |
-| - | - | - |
-| `apiKey` | `GENERALCONTEXT_API_KEY` | |
-| `baseUrl` | `GENERALCONTEXT_BASE_URL`, else `https://generalcontextlabs.com` | The option wins over the env var. Must be `https://`, except `http://localhost`, `127.0.0.1` or `[::1]`. Other URLs are refused and nothing is sent. |
-| `appName` | none | Sent with every batch. |
-| `redact` | `false` | See [Redaction](#redaction). |
-| `beforeSend` | none | `(call) => call \| null`. Runs at send time on each call, before redaction. Return the call (edited or not) to send it, or `null` to drop it. A throw drops the call. |
-| `instrumentModules` | installed `openai` and `@anthropic-ai/sdk` | Pass these whenever a bundler is involved. |
-| `instrumentFetch` | `false` | SDK clients are always recorded. Set `true` to also record raw `fetch` POSTs to any `/chat/completions` URL. |
-| `disableBatch` | `false` | Send each call right away. |
-
-| Env var | |
-| - | - |
-| `GENERALCONTEXT_API_KEY` | API key, if `apiKey` is not passed. |
-| `GENERALCONTEXT_BASE_URL` | Collector URL, if `baseUrl` is not passed. |
-| `GENERALCONTEXT_REDACT_SALT` | Secret that keeps redaction placeholders stable across processes. |
-| `GENERALCONTEXT_DISABLED` | `1`, `true` or `yes` makes `initialize()` do nothing. |
-
-Requests never follow redirects. A call over 4 MB is dropped, and the queue holds at most 5000 calls or 64 MB. Pending calls are flushed when the process is about to exit, for at most 2 seconds.
-
-## Proxies
-
-Node's built-in `fetch` ignores `HTTPS_PROXY` by default. On Node 24+, set `NODE_USE_ENV_PROXY=1`. On older versions, install a global dispatcher (for example undici's `setGlobalDispatcher(new EnvHttpProxyAgent())`). If your proxy inspects TLS, point `NODE_EXTRA_CA_CERTS` at its CA certificate.
 
-## Next.js
+// Group every call in one conversation under the id you already use for it
+await withSession({ sessionId: conversationId, userId: customerId, channel: 'chat' }, () => handle(message));
 
-Initialize in `instrumentation.ts` at the project root, not in a route file:
-
-```ts
-export async function register() {
-  if (process.env.NEXT_RUNTIME === 'nodejs') {
-    const { initialize } = await import('generalcontext');
-    const { OpenAI } = await import('openai');
-    const anthropic = await import('@anthropic-ai/sdk');
-    initialize({ appName: 'my-agent', instrumentModules: { openAI: OpenAI, anthropic } });
-  }
-}
-```
-
-Keep the SDKs out of the bundle so routes use the instance we patched:
-
-```ts
-// next.config.ts
-export default { serverExternalPackages: ['openai', '@anthropic-ai/sdk'] };
+await flush(); // before a serverless function, job or script exits
 ```
 
-Use the Node.js runtime. The Edge runtime is not supported.
-
-## Sessions and tasks
-
-Tag calls with your own ids. Tags carry across `await`.
+If your app serves several companies of its own, pass `account` to `withSession` (the id or name you already use for that company) so conversations are grouped by account.
 
-```ts
-import { withSession, task } from 'generalcontext';
-
-await withSession({ sessionId: conversationId, userId: 'u_42', channel: 'chat' }, async () => {
-  await task('triage', () => client.chat.completions.create({ ... }));
-});
-```
-
-`withSession(conversationId, fn)` also works. `conversation({ id, ... }, fn)` is the older name.
-
-## Serverless and scripts
-
-Calls send in the background. Before a function or script exits:
-
-```ts
-await flush(); // never throws
-```
+| Provider | Package | `instrumentModules` key |
+| - | - | - |
+| OpenAI, including Azure OpenAI | `openai` | `openAI` |
+| Anthropic, including Bedrock, Vertex and beta | `@anthropic-ai/sdk` | `anthropic` |
+| Google Gemini | `@google/genai` | `googleGenAI` |
+| AWS Bedrock Converse | `@aws-sdk/client-bedrock-runtime` | `bedrockRuntime` |
+| Anything else | Send [OpenTelemetry](https://generalcontextlabs.com/docs/opentelemetry) | |
 
-## Outcomes and events
+**Voice (OpenAI Realtime).** `OpenAIRealtimeWS` (`openai/realtime/ws`) and `OpenAIRealtimeWebSocket`
+(`openai/realtime/websocket`) connections are recorded too. Create the connection inside `withSession(...)` and
+the whole call lands in that conversation. Each `response.done` is recorded as one `openai-realtime` call shaped
+like a Responses call: the instructions and tools, the conversation so far as text (caller turns from the input
+transcription, agent turns from the output transcript, tool calls and their results), and a `voice` field with
+`speech_stopped_at`, `first_audio_at`, `interrupted` and `truncated_at_ms`. Turn on input audio transcription in
+the session to record what the caller said. Audio is never recorded. Pass a connection to `wrap()` to record only
+that one.
 
-Both flush pending calls first, so the session exists before the outcome or event arrives. Both return `true` when accepted and never throw.
+Node 18 to 24, Node.js runtime only (not Edge). Supported client versions, Next.js, proxies, redaction, outcomes and events, and every option: [SDK reference](https://generalcontextlabs.com/docs/sdk).
 
-```ts
-await reportOutcome(conversationId, 'fail', 'customer reopened the ticket');
+## Command line
 
-await logEvent({ sessionId: conversationId, eventType: 'order.refunded', eventId: `refund-${refundId}`, properties: { amount: 42.5 } });
-await logEvent({ sessionId: conversationId, eventType: 'handoff.to_human', eventId: `handoff-${ticketId}` });
+```sh
+npx generalcontext verify                      # the key works, and when the last call arrived
+npx generalcontext verify --session conv-123   # that one conversation arrived
+npx generalcontext benchmark pull --out benchmark.json
 ```
 
-Pass a stable `eventId` so retries dedupe. `properties` are flat strings, numbers and booleans. `timestamp` defaults to now.
-
-## Redaction
-
-`initialize({ redact: true })` replaces emails, phone numbers and card or SSN-like numbers with placeholders such as `<email:3f9a1c>` before anything leaves your process. Set `GENERALCONTEXT_REDACT_SALT` to one secret in every process so the same value gets the same placeholder. Or pass your own `(text) => text`. Redaction also applies to `reportOutcome` and `logEvent`, so a redacted session id still matches its calls.
-
-## One client only
-
-`wrap(client, { redact })` records a single client instance without patching the SDK. Its `redact` applies to that client only. If `initialize()` already patched the SDK, the client is recorded with the `initialize()` settings.
+Running the benchmark is in the Python package. See the [command line docs](https://generalcontextlabs.com/docs/cli).
 
 ## License
 

package/dist/cli.d.ts

@@ -0,0 +1,2 @@
+#!/usr/bin/env node
+export {};

package/dist/cli.js

@@ -0,0 +1,175 @@
+#!/usr/bin/env node
+// Copyright 2026 General Context Labs, Inc.
+// SPDX-License-Identifier: Apache-2.0
+// generalcontext verify [--session ID] [--json]
+// generalcontext benchmark pull [--out FILE]
+// Exit 0 on success, 1 on a failed check, 2 on a usage or configuration error. No runtime dependencies.
+import { writeFileSync } from 'node:fs';
+const DEFAULT_BASE_URL = 'https://generalcontextlabs.com';
+const TIMEOUT_MS = 30_000;
+const USAGE = `Usage:
+  generalcontext verify [--session ID] [--json]
+  generalcontext benchmark pull [--out FILE]
+Uses GENERALCONTEXT_API_KEY and GENERALCONTEXT_BASE_URL.`;
+const PYTHON_HINT = 'Run the benchmark with Python: pip install generalcontext && generalcontext evals run ...';
+class Exit extends Error {
+    code;
+    constructor(code, message) {
+        super(message);
+        this.code = code;
+    }
+}
+const trimSlashes = (url) => { let end = url.length; while (end > 0 && url[end - 1] === '/')
+    end--; return url.slice(0, end); };
+// Same rule as the SDK: https, or plain http to this machine only.
+function secureUrl(url) {
+    try {
+        const u = new URL(url);
+        return u.protocol === 'https:' || (u.protocol === 'http:' && ['localhost', '127.0.0.1', '[::1]'].includes(u.hostname));
+    }
+    catch {
+        return false;
+    }
+}
+function settings() {
+    const key = (process.env.GENERALCONTEXT_API_KEY ?? '').trim();
+    if (!key)
+        throw new Exit(2, 'GENERALCONTEXT_API_KEY is not set. Create a key on the API keys page and export it.');
+    const base = trimSlashes((process.env.GENERALCONTEXT_BASE_URL ?? '').trim() || DEFAULT_BASE_URL);
+    if (!secureUrl(base))
+        throw new Exit(2, `GENERALCONTEXT_BASE_URL ${base} is not https. Use https, or http://localhost for local development.`);
+    return { key, base };
+}
+async function get(path) {
+    const { key, base } = settings();
+    let response;
+    try {
+        response = await fetch(`${base}/api/platform-collect${path}`, {
+            headers: { Authorization: `Bearer ${key}`, Accept: 'application/json' },
+            redirect: 'error', // never forward the key to another host
+            signal: AbortSignal.timeout(TIMEOUT_MS),
+        });
+    }
+    catch (error) {
+        throw new Exit(1, `Could not reach ${base}: ${error.message}`);
+    }
+    const text = await response.text();
+    let body = null;
+    try {
+        body = text ? JSON.parse(text) : null;
+    }
+    catch { /* not JSON */ }
+    return { status: response.status, body };
+}
+const errorText = (status, body) => typeof body?.error === 'string' ? body.error : `HTTP ${status}`;
+function failed(status, body) {
+    if (status === 401)
+        throw new Exit(1, 'The API key was not accepted. Check GENERALCONTEXT_API_KEY.');
+    throw new Exit(1, `Request failed: ${errorText(status, body)}`);
+}
+function parseFlags(args, known) {
+    const out = {};
+    for (let i = 0; i < args.length; i++) {
+        let arg = args[i], value;
+        const eq = arg.indexOf('=');
+        if (arg.startsWith('--') && eq > 0) {
+            value = arg.slice(eq + 1);
+            arg = arg.slice(0, eq);
+        }
+        const kind = known[arg];
+        if (!kind)
+            throw new Exit(2, `Unknown argument ${arg}.\n${USAGE}`);
+        if (kind === 'bool') {
+            out[arg] = true;
+            continue;
+        }
+        value ??= args[++i];
+        if (value === undefined || value === '')
+            throw new Exit(2, `${arg} needs a value.\n${USAGE}`);
+        out[arg] = value;
+    }
+    return out;
+}
+const scopeWords = { ingest: 'sends traffic', evals: 'runs the benchmark' };
+async function verify(args) {
+    const flags = parseFlags(args, { '--session': 'value', '--json': 'bool' });
+    const session = flags['--session'];
+    const { status, body } = await get(`/status${session ? `?session=${encodeURIComponent(session)}` : ''}`);
+    if (status !== 200 || !body)
+        failed(status, body);
+    const found = session ? Boolean(body.session?.found) : true;
+    if (flags['--json']) {
+        console.log(JSON.stringify(body, null, 2));
+        return found ? 0 : 1;
+    }
+    const scopes = Array.isArray(body.key?.scopes) ? body.key.scopes : [];
+    const label = body.key?.label ? ` "${body.key.label}"` : '';
+    console.log(`Workspace ${body.workspace}. Key${label} ${scopes.length ? scopes.map(s => scopeWords[s] ?? s).join(' and ') : 'has no scopes'}.`);
+    console.log(body.last_call_at ? `Last call arrived ${body.last_call_at} from ${body.last_source ?? 'unknown'}.` : 'No calls have arrived yet.');
+    if (session) {
+        const s = body.session ?? {};
+        if (!found) {
+            console.log(`Conversation ${session}: not found yet.`);
+            return 1;
+        }
+        const models = Array.isArray(s.models) ? s.models : [];
+        console.log(`Conversation ${session}: arrived, ${s.calls ?? 0} call${s.calls === 1 ? '' : 's'}${models.length ? ` (${models.join(', ')})` : ''}.`);
+    }
+    return 0;
+}
+async function benchmark(args) {
+    const [sub, ...rest] = args;
+    if (sub !== 'pull')
+        throw new Exit(2, USAGE);
+    const flags = parseFlags(rest, { '--out': 'value' });
+    const out = flags['--out'] ?? 'benchmark.json';
+    const { status, body } = await get('/benchmark');
+    if (status === 403)
+        throw new Exit(1, errorText(status, body) === `HTTP 403`
+            ? 'This key cannot download the benchmark. On API keys, create a key that can run the benchmark.' : errorText(status, body));
+    if (status !== 200 || !body)
+        failed(status, body);
+    try {
+        writeFileSync(out, JSON.stringify(body, null, 2) + '\n');
+    }
+    catch (error) {
+        throw new Exit(1, `Could not write ${out}: ${error.message}`);
+    }
+    const tasks = Array.isArray(body.tasks) ? body.tasks : [];
+    const splits = new Map();
+    for (const t of tasks) {
+        const s = typeof t?.split === 'string' && t.split ? t.split : 'unsplit';
+        splits.set(s, (splits.get(s) ?? 0) + 1);
+    }
+    const bySplit = [...splits].sort(([a], [b]) => (a < b ? -1 : a > b ? 1 : 0)).map(([s, n]) => `${n} ${s}`).join(', ');
+    console.log(`Wrote ${out}: ${tasks.length} task${tasks.length === 1 ? '' : 's'}${bySplit ? ` (${bySplit})` : ''}.`);
+    return 0;
+}
+async function main(argv) {
+    const [command, ...args] = argv;
+    try {
+        if (command === 'verify')
+            return await verify(args);
+        if (command === 'benchmark')
+            return await benchmark(args);
+        if (command === 'evals') {
+            console.error(PYTHON_HINT);
+            return 2;
+        }
+        if (command === '-h' || command === '--help' || command === 'help') {
+            console.log(USAGE);
+            return 0;
+        }
+        console.error(USAGE);
+        return 2;
+    }
+    catch (error) {
+        if (error instanceof Exit) {
+            (error.code === 0 ? console.log : console.error)(error.message);
+            return error.code;
+        }
+        console.error(`generalcontext: ${error?.message ?? error}`);
+        return 1;
+    }
+}
+main(process.argv.slice(2)).then(code => { process.exitCode = code; });

package/dist/index.d.ts

@@ -1,10 +1,11 @@
-export declare const VERSION = "0.2.0";
+export declare const VERSION = "0.3.0";
 type Metadata = Record<string, string | number | boolean>;
 export type Session = {
     sessionId: string;
     userId?: string;
     channel?: string;
     product?: string;
+    account?: string;
     metadata?: Metadata;
 };
 type Tags = {
@@ -12,9 +13,17 @@ type Tags = {
     user_id?: string;
     channel?: string;
     product?: string;
+    account?: string;
     metadata?: Metadata;
 };
-type Api = 'openai-chat' | 'openai-responses' | 'anthropic';
+type Api = 'openai-chat' | 'openai-responses' | 'anthropic' | 'gemini' | 'bedrock-converse' | 'openai-realtime';
+/** Voice timing on 'openai-realtime' calls. Timestamps are ISO strings, null when not seen. */
+export type VoiceTiming = {
+    speech_stopped_at: string | null;
+    first_audio_at: string | null;
+    interrupted: boolean;
+    truncated_at_ms?: number;
+};
 export type Call = {
     api: Api;
     conversation: Tags | null;
@@ -24,6 +33,7 @@ export type Call = {
     started_at: string;
     ended_at: string;
     error: string | null;
+    voice?: VoiceTiming;
 };
 type Scrub = (text: string) => string;
 type BeforeSend = (call: Call) => Call | null | undefined;
@@ -36,18 +46,29 @@ export type InitializeOptions = {
     redact?: boolean | Scrub;
     /** Runs at send time on each call, before redaction. Return the call to send, or null to drop it. */
     beforeSend?: BeforeSend;
-    /** SDK classes or modules to patch, e.g. { openAI: OpenAI, anthropic: Anthropic }. */
-    instrumentModules?: {
-        openAI?: unknown;
-        anthropic?: unknown;
-    };
+    /**
+     * SDK classes or modules to patch, e.g. { openAI: OpenAI, anthropic: Anthropic }. Also accepts
+     * googleGenAI (the '@google/genai' module) and bedrockRuntime (the '@aws-sdk/client-bedrock-runtime' module).
+     */
+    instrumentModules?: Modules;
     /** Also record raw fetch POSTs to any URL ending /chat/completions. Default false. */
     instrumentFetch?: boolean;
     /** Send each call right away instead of batching. */
     disableBatch?: boolean;
 };
+type Modules = {
+    openAI?: unknown;
+    anthropic?: unknown;
+    googleGenAI?: unknown;
+    bedrockRuntime?: unknown;
+};
 /** Configure recording once per process, before your agent makes model calls. */
 export declare function initialize(options?: InitializeOptions): void;
+/**
+ * Resolves once initialize() has patched every SDK build it found, including the ES module builds it loads
+ * asynchronously. Await it when a model call may run before the app's next await after initialize(). Never rejects.
+ */
+export declare function ready(): Promise<void>;
 /** Tag every call made inside `run` with one session. `conversation` is the same function. */
 export declare function withSession<T>(session: Session | string, run: () => T): T;
 /** Older name for withSession. Accepts { id } as well as { sessionId }. */
@@ -55,12 +76,34 @@ export declare function conversation<T>(tags: Session | {
     id: string;
     channel?: string;
     product?: string;
+    account?: string;
     metadata?: Metadata;
 } | string, run: () => T): T;
 /** Tag every call made inside `run` with a task name, such as one step of your agent loop. */
 export declare function task<T>(name: string, run: () => T): T;
-/** Wait until recorded calls are sent. Call before a serverless function or script exits. Never throws. */
-export declare function flush(): Promise<void>;
+/**
+ * The copy of a call that is recorded. Never changes `value`, and never touches what the caller sends to the provider.
+ * Agents resend the whole run on every call, so without this, later calls pass the per-call limit and are lost.
+ *
+ * - Inline binary becomes a placeholder that keeps the shape: a base64 data URL keeps its prefix
+ *   (`data:image/png;base64,[omitted N bytes sha256:ab12cd34ef56]`), and a base64 `data`, `result` or
+ *   `b64_json` field (Anthropic `source.data`, Gemini `inlineData.data`, OpenAI `input_audio.data`, Responses
+ *   `image_generation_call.result`) becomes `[omitted N bytes sha256:...]`. N is the length of the base64 text
+ *   and the hash is the first 12 hex digits of the sha256 of that text, so the same image gets the same placeholder
+ *   in every call. Media types stay. Binary is never recorded: it does not help review and it is a privacy risk.
+ * - Any string over 200,000 characters keeps its first 100,000 and last 20,000 characters around
+ *   `\n[... clipped N characters ...]\n`.
+ *
+ * Runs before beforeSend and redaction. A call still over the limit afterwards is dropped.
+ */
+export declare function compactForRecording(value: unknown, key?: string, depth?: number): unknown;
+export declare function maskSecrets(text: string): string;
+/**
+ * Wait until recorded calls are sent, for at most `timeoutMs` (default 10 s). Call before a serverless
+ * function or script exits. Resolves true when the queue was sent, false at the deadline. Never throws.
+ * Sending continues in the background after the deadline.
+ */
+export declare function flush(timeoutMs?: number): Promise<boolean>;
 /** Mark how a session ended. Returns true when the platform accepted it. Never throws. */
 export declare function reportOutcome(sessionId: string, outcome: 'success' | 'fail', note?: string): Promise<boolean>;
 export type AppEvent = {