nexus-ai-pro 2.18.0 → 2.19.0

package/API_STABILITY.md

@@ -41,6 +41,21 @@ Both are covered by this policy:
 Deep imports into `dist`, `dist-cjs`, or `src` are not supported. Import only from `nexus-ai-pro` or
 one of its explicit subpaths.
 
+## 2.19 stage (2.19.0)
+
+Everything 2.19 adds is stable from the start and additive, unless this section says otherwise.
+
+New and stable:
+- in the root: `CompletionRequest.logprobs`, `NexusResponse.logprobs`, `TokenLogprob`, and
+  `ModelCapabilities.logprobs`;
+- in `nexus-ai-pro/workflows`: `ClassifyRouteOptions.decisions`.
+
+New and experimental, until the live workflow passes against TypeSafe AI's Jev and a model that
+reports log probabilities: the typed decisions family, `nexus-ai-pro/decisions`,
+`nexus-ai-pro/decisions/typesafe`, and `nexus-ai-pro/decisions/mock`.
+
+Changed on purpose: `OperationFamily` and `RunKind` widen with `decision`.
+
 ## 2.18 stage (2.18.0)
 
 Everything 2.18 adds is stable from the start and additive, unless this section says otherwise.

package/CHANGELOG.md

@@ -4,6 +4,49 @@ Notable changes to this project are recorded here. The format follows [Keep a Ch
 
 ## [Unreleased]
 
+## [2.19.0] - 2026-10-24
+
+Typed decisions: a yes or no, a choice, or a score, answered with probabilities and a confidence read
+from them, from TypeSafe AI's Jev or from any model, acted on when sure and sent to a person when not.
+
+### Added
+
+- **Typed decisions.** `nexus-ai-pro/decisions` asks a `DecisionRequest`'s questions of one state: a
+  `YesNoQuestion`, a `ChoiceQuestion` of up to 255 options, or a `ScoreQuestion` of up to ten levels,
+  each answered in isolation with its probabilities and a confidence read the same way from any
+  provider. Answers are typed by their questions, so a choice's answer names only its options.
+  `DecisionManager` refuses a malformed question before it is sent, fails over, caches, traces each
+  decision as a run of kind `decision`, and runs it as an operation of the client, family
+  `decision`, budgeted at its model's price and on the tenant's key.
+- **TypeSafe AI's Jev.** `TypeSafeDecisionProvider`, on `nexus-ai-pro/decisions/typesafe`, speaks the
+  System One API: every question of a decision in one call, its refusals naming the field at fault,
+  priced at $42 per billion input tokens with output free.
+- **Decisions from any model.** `completionDecisions()` asks a completion model each question for a
+  one-token code, and reads its probabilities from that token's log probabilities where the provider
+  reports them, or counts them over samples where it does not, or refuses to.
+- **Gates and calibration.** `gate()` acts, sends to a person, or abstains by confidence; `calibrate()`
+  measures how answers' probabilities track how often they are right, and chooses the threshold that
+  holds a target accuracy on the application's own labelled answers.
+- **Decisions where the package decides.** `decisionRoute()` for a graph's branch, `decisionPermissions()`
+  for an agent's tool calls, `decisionGuard()` and `decisionModel()` as agent middleware,
+  `decisionEvaluator()` and `reviewUnsure()` for evaluation, and `classifyRoute()`'s `decisions`:
+  an uncertain answer goes to a person.
+- **Log probabilities.** `CompletionRequest.logprobs` asks for each answer token's log probability and
+  its alternatives, on `NexusResponse.logprobs` as `TokenLogprob` values, from OpenAI's Chat
+  Completions, the OpenAI-compatible adapters that report them, Gemini, and Ollama;
+  `ModelCapabilities.logprobs` declares a model that cannot.
+- `MockDecisionProvider`, on `nexus-ai-pro/decisions/mock`, and `runDecisionConformance()`.
+
+### Changed
+
+- `OperationFamily` and `RunKind` have `decision`; a `switch` over either that names every one needs
+  the new one.
+
+### Fixed
+
+- A test that waits forever fails at five minutes, and every child process the tests start
+  synchronously has a timeout of its own: the 2.18.0 publish's first attempt waited two hours on one.
+
 ## [2.18.0] - 2026-10-22
 
 Span events and links, media that knows its length, a tenant's threads and runs listed and paged
@@ -3179,7 +3222,8 @@ shared policy vocabulary is in place for a future release that revisits this.
 - URL fetching rejects unsafe private-network targets and limits response reads.
 - CLI and audit findings no longer disclose detected secret values.
 
-[Unreleased]: https://github.com/mkhitar-abrahamyan/nexus-ai/compare/v2.18.0...HEAD
+[Unreleased]: https://github.com/mkhitar-abrahamyan/nexus-ai/compare/v2.19.0...HEAD
+[2.19.0]: https://github.com/mkhitar-abrahamyan/nexus-ai/compare/v2.18.0...v2.19.0
 [2.18.0]: https://github.com/mkhitar-abrahamyan/nexus-ai/compare/v2.17.0...v2.18.0
 [2.17.0]: https://github.com/mkhitar-abrahamyan/nexus-ai/compare/v2.16.0...v2.17.0
 [2.16.0]: https://github.com/mkhitar-abrahamyan/nexus-ai/compare/v2.15.1...v2.16.0

package/README.md

@@ -156,6 +156,7 @@ from the doc comments. The guides live in the repository, so these links go to G
 | [Retrieval](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/retrieval.md) | Hybrid search, reranking, MMR, parent documents, and Redis, Pinecone, Weaviate, and Chroma stores |
 | [Embeddings and retrieval](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/embeddings.md) | Embeddings as a routed operation family, and RAG helpers |
 | [Media understanding](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/media.md) | Reading documents and images page by page, describing images, and answering questions about them, through any vision model or OCR |
+| [Typed decisions](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/decisions.md) | A yes or no, a choice, or a score, answered with probabilities and a calibrated confidence, from TypeSafe AI's Jev or any model, so a graph, an approval, or a guardrail acts when it is sure and asks a person when not |
 | [Video generation](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/video.md) | Videos from a prompt or a first frame, on Google Veo and Runway, as durable jobs streamed into an asset store and priced per second |
 | [Images (experimental)](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/images.md) | Generation and masked edits over three backends, input safety, moderation, asset stores |
 | [Voice](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/voice.md) | Transcription, speech, voice turns, and voice sessions |
@@ -330,11 +331,15 @@ Shipped so far:
   video is extended from its own job or edited on Gemini Omni Flash, Veo, and Runway;
 - 2.18 span events and links, as OpenTelemetry has them, media that knows its length from its own
   headers, a tenant's threads and runs listed and paged across all of its regions, MCP servers that
-  reconnect and change their tools within a session, and videos made after reference videos.
+  reconnect and change their tools within a session, and videos made after reference videos;
+- 2.19 typed decisions: a yes or no, a choice, or a score, answered with probabilities and a
+  confidence, from TypeSafe AI's Jev or from any model's log probabilities, gated by thresholds a
+  calibration chooses, so a graph's branch, an approval, or a guardrail acts when it is sure and asks
+  a person when it is not.
 
 Upgrading from 1.x: `npx nexus migrate src --write` moves your imports, and
 [MIGRATING.md](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/MIGRATING.md) covers the rest.
-Next, 2.19 reads a service's own API description into emulated tools, and lets an emulator hold
+Next, 2.20 reads a service's own API description into emulated tools, and lets an emulator hold
 what a whole evaluation did, so examples that build on each other are evaluated with nothing sent.
 Pinecone, images, the Gemini
 embedding batch tier, media understanding, video, Gemini Live, SIP, and both phone providers

package/dist/capabilities/negotiate.js

@@ -95,6 +95,11 @@ export function negotiateCompletionRequest(request, capabilities, options = {})
             target.seed = undefined;
         });
     }
+    if (request.logprobs !== undefined && capabilities.logprobs === false) {
+        refuse('logprobs', request.logprobs, 'the model does not report token log probabilities', (target) => {
+            target.logprobs = undefined;
+        });
+    }
     if (request.topK !== undefined && capabilities.topK === false) {
         refuse('topK', request.topK, 'the model does not support top-k sampling', (target) => {
             target.topK = undefined;

package/dist/decisions/completion.d.ts

@@ -0,0 +1,48 @@
+/**
+ * Decisions from any completion model: each question asked on its own, answered with a one-token code,
+ * and its probabilities read from the code token's log probabilities, or counted over samples where
+ * the model reports none.
+ */
+import type { CompletionRequest } from '../types/messages.js';
+import type { NexusResponse } from '../types/response.js';
+import { type DecisionProvider } from './core.js';
+/** Anything that completes a request, such as a `NexusAI` client. */
+export interface DecisionCompletionClient {
+    /** Runs one completion. */
+    complete(request: CompletionRequest): Promise<NexusResponse>;
+}
+/** How a completion model's probabilities are read. */
+export type CompletionDecisionMethod = 'auto' | 'logprobs' | 'sampling';
+/** Options for `completionDecisions()`. */
+export interface CompletionDecisionsOptions {
+    /** The model to ask, as the client names it. */
+    model: string;
+    /** The provider's name in results. Defaults to `model`. */
+    name?: string;
+    /**
+     * How probabilities are read: `logprobs`, from one call's token log probabilities; `sampling`, by
+     * counting the answers of several calls; or `auto`, the default: log probabilities, and sampling for
+     * a model that reports none or refuses to, which it then asks by sampling from the start.
+     */
+    method?: CompletionDecisionMethod;
+    /** Calls per question, when sampling. Defaults to 8. */
+    samples?: number;
+    /** The temperature samples are drawn at. Defaults to 1. */
+    temperature?: number;
+    /** Questions asked at once. Defaults to 8. */
+    concurrency?: number;
+    /** Instructions added to every question, such as what the application's terms mean. */
+    instructions?: string;
+}
+/**
+ * Any completion model as a decision provider, through a client's own `complete()`, so every call is
+ * routed, authorized, budgeted, traced, and priced as every completion is. Each question is asked on
+ * its own, at temperature 0, to answer with one code: a letter per option, Y or N, or a level's digit.
+ * Its probabilities are read from the log probabilities of that token's alternatives where the model
+ * reports them, the way TypeSafe AI's Jev reads its own; elsewhere the question is asked `samples`
+ * times, and each answer counted. Sampled probabilities come in steps of one sample, and cost as many
+ * calls.
+ */
+export declare function completionDecisions(client: DecisionCompletionClient, options: CompletionDecisionsOptions): DecisionProvider & {
+    readonly sampledModels: ReadonlySet<string>;
+};

package/dist/decisions/completion.js

@@ -0,0 +1,195 @@
+import { DecisionError, } from './core.js';
+const LOGPROB_OPTIONS = 20;
+export function completionDecisions(client, options) {
+    const method = options.method ?? 'auto';
+    const sampled = new Set();
+    const info = {
+        name: options.name ?? options.model,
+        capabilities: {
+            types: ['yes-no', 'choice', 'score'],
+            maxOptions: 255,
+            maxLevels: 10,
+            defaultModel: options.model,
+        },
+        chargedByItsCalls: true,
+    };
+    const ask = async (model, state, question, context, usage) => {
+        const codes = codesOf(question);
+        const prompt = promptOf(state, question, codes, options.instructions);
+        const request = (extra) => ({
+            model,
+            messages: [
+                {
+                    role: 'system',
+                    content: 'You answer one question about a state with exactly one of the codes given, and nothing else.',
+                },
+                { role: 'user', content: prompt },
+            ],
+            signal: context.signal,
+            ...(context.tenantId !== undefined ? { tenantId: context.tenantId } : {}),
+            ...extra,
+        });
+        const readable = codes.length <= LOGPROB_OPTIONS && codes.every((code) => code.length === 1);
+        if (method !== 'sampling' && readable && !sampled.has(model)) {
+            let response;
+            try {
+                response = await client.complete(request({ temperature: 0, maxTokens: 2, logprobs: { top: 20 } }));
+            }
+            catch (error) {
+                if (method === 'logprobs' || !refusedOption(error))
+                    throw error;
+            }
+            if (response) {
+                usage.add(response);
+                const weights = weightsFrom(response.logprobs, codes);
+                if (weights)
+                    return distributionOf(question, codes, weights);
+                if (method === 'logprobs') {
+                    throw new DecisionError(`"${model}" reported no log probabilities for the codes it was given`, 'DECISION_FAILED');
+                }
+            }
+            sampled.add(model);
+        }
+        if (!readable && method === 'logprobs') {
+            throw new DecisionError(`A question of ${codes.length} options cannot be read from one token's ${LOGPROB_OPTIONS} alternatives`, 'DECISION_UNSUPPORTED');
+        }
+        const count = Math.max(1, Math.floor(options.samples ?? 8));
+        const answers = await Promise.all(Array.from({ length: count }, () => client.complete(request({ temperature: options.temperature ?? 1, maxTokens: 4 })).then((response) => {
+            usage.add(response);
+            return codeIn(response.content, codes);
+        })));
+        const weights = codes.map((code) => answers.filter((answer) => answer === code).length);
+        if (weights.every((weight) => weight === 0)) {
+            throw new DecisionError(`"${model}" answered none of ${count} samples with a code it was given`, 'DECISION_FAILED');
+        }
+        return distributionOf(question, codes, weights);
+    };
+    return {
+        info,
+        sampledModels: sampled,
+        async decide(request, context) {
+            const totals = new Totals();
+            const entries = Object.entries(request.questions);
+            const distributions = {};
+            const limit = Math.max(1, Math.floor(options.concurrency ?? 8));
+            let next = 0;
+            const worker = async () => {
+                while (next < entries.length) {
+                    const [name, question] = entries[next++];
+                    distributions[name] = await ask(request.model, request.state, question, context, totals);
+                }
+            };
+            await Promise.all(Array.from({ length: Math.min(limit, entries.length) }, worker));
+            return {
+                model: request.model,
+                distributions,
+                usage: totals.usage(),
+                ...(totals.cost() ? { cost: totals.cost() } : {}),
+            };
+        },
+    };
+}
+class Totals {
+    inputTokens = 0;
+    outputTokens = 0;
+    calls = 0;
+    amount = 0;
+    priced = false;
+    add(response) {
+        this.calls += 1;
+        this.inputTokens += response.meta.usage?.inputTokens ?? 0;
+        this.outputTokens += response.meta.usage?.outputTokens ?? 0;
+        if (response.meta.cost) {
+            this.amount += response.meta.cost.amount;
+            this.priced = true;
+        }
+    }
+    usage() {
+        return { inputTokens: this.inputTokens, outputTokens: this.outputTokens, calls: this.calls };
+    }
+    cost() {
+        return this.priced ? { amount: this.amount, currency: 'USD', basis: 'estimated' } : undefined;
+    }
+}
+function codesOf(question) {
+    if (question.type === 'yes-no')
+        return ['Y', 'N'];
+    if (question.type === 'score')
+        return question.levels.map((_, level) => String(level));
+    return question.options.length <= 26
+        ? question.options.map((_, index) => String.fromCharCode(65 + index))
+        : question.options.map((_, index) => String(index + 1));
+}
+function promptOf(state, question, codes, instructions) {
+    const text = typeof state === 'string' ? state : JSON.stringify(state, null, 2);
+    const lines = [`State:\n${text}`, ''];
+    if (instructions)
+        lines.push(instructions, '');
+    if (question.type === 'yes-no') {
+        lines.push(`Question: ${question.question ?? 'Is the statement true of the state?'}`);
+        lines.push(`Y: yes${question.criteria?.yes ? `, ${question.criteria.yes}` : ''}`);
+        lines.push(`N: no${question.criteria?.no ? `, ${question.criteria.no}` : ''}`);
+    }
+    else if (question.type === 'choice') {
+        lines.push(`Question: ${question.question ?? 'Which option fits the state?'}`);
+        question.options.forEach((option, index) => {
+            lines.push(`${codes[index]}: ${option.name}${option.description ? `, ${option.description}` : ''}`);
+        });
+    }
+    else {
+        lines.push(`Question: ${question.question ?? 'Which level fits the state?'}`);
+        question.levels.forEach((level, index) => {
+            lines.push(`${codes[index]}: ${level}`);
+        });
+    }
+    lines.push('', `Answer with one code: ${codes.join(', ')}.`);
+    return lines.join('\n');
+}
+function codeOf(token) {
+    return token.replace(/^[\s"'`*(]+|[\s"'`*).:,]+$/g, '').toUpperCase();
+}
+function weightsFrom(logprobs, codes) {
+    const first = logprobs?.find((item) => item.token.trim() !== '');
+    if (!first)
+        return undefined;
+    const candidates = first.top?.length ? first.top : [{ token: first.token, logprob: first.logprob }];
+    const weights = codes.map(() => 0);
+    let found = false;
+    for (const candidate of candidates) {
+        const index = matchCode(candidate.token, codes);
+        if (index < 0 || !Number.isFinite(candidate.logprob))
+            continue;
+        weights[index] = weights[index] + Math.exp(candidate.logprob);
+        found = true;
+    }
+    return found ? weights : undefined;
+}
+function codeIn(content, codes) {
+    const head = content.trim().split(/[\s:,.)]+/)[0] ?? '';
+    return codes[matchCode(head, codes)];
+}
+function matchCode(token, codes) {
+    const code = codeOf(token);
+    const index = codes.indexOf(code);
+    if (index >= 0 || codes.length !== 2 || codes[0] !== 'Y' || codes[1] !== 'N')
+        return index;
+    if (code === 'YES' || code === 'TRUE')
+        return 0;
+    return code === 'NO' || code === 'FALSE' ? 1 : -1;
+}
+function distributionOf(question, codes, weights) {
+    const total = weights.reduce((sum, weight) => sum + weight, 0);
+    const share = (index) => (total > 0 ? (weights[index] ?? 0) / total : 1 / codes.length);
+    if (question.type === 'yes-no')
+        return { type: 'yes-no', yes: share(0) };
+    if (question.type === 'score')
+        return { type: 'score', probabilities: codes.map((_, index) => share(index)) };
+    return {
+        type: 'choice',
+        probabilities: Object.fromEntries(question.options.map((option, index) => [option.name, share(index)])),
+    };
+}
+function refusedOption(error) {
+    const status = error?.status;
+    return status === 400 || status === 422;
+}