@parcha/agentrun-jev 0.1.0-beta.13 → 0.1.0-beta.15

package/README.md

@@ -48,6 +48,21 @@ const runJudge = createJevRunner({
 });
 ```
 
+### Through OpenRouter
+
+Jev is also served by OpenRouter's System One path, on the same request shape. Choose it with `provider`:
+
+```ts
+import { createJevRunner, jevProviderFromEnv } from "@parcha/agentrun-jev";
+
+const runJudge = createJevRunner({ provider: "openrouter" });             // key: apiKey, else OPENROUTER_API_KEY
+const byEnv = createJevRunner({ provider: jevProviderFromEnv(process.env) }); // OpenRouter only when only OPENROUTER_API_KEY is set
+```
+
+`provider: "openrouter"` sends to `https://openrouter.ai/api/v1/systemone` (an explicit `baseURL` wins) and defaults the model to `~typesafe/jev-latest`; pin `typesafe/jev-1.14` when a result must be reproducible. It never falls back to `TYPESAFE_API_KEY`: with no OpenRouter key it is a configuration error. The default provider is `typesafe` and nothing changes for it. `provider` cannot be combined with `client`.
+
+Cost: when the response reports `usage.cost` (USD) it is `cost_usd`; otherwise a configured `pricing` estimate is used; otherwise `cost_usd` is `null`, never 0. A `usage` with a cost and no token counts is accepted and returns `usage: null`. Requests over about 32 KB of state run in the provider's slower long-context mode.
+
 The official `@typesafe-ai/sdk` resolves `TYPESAFE_API_KEY` and `TYPESAFE_BASE_URL`. Explicit `apiKey` and `baseURL` options take precedence. An API root ends before `/v1/systemone`. Set `fetch` for a custom transport, or supply a `client` with a `systemOne(request, options)` method. A custom client must honor the request signal and disable its own retries; it owns its credentials and cannot be combined with transport options.
 
 The adapter retries transient HTTP 408, 429, 5xx responses and connection failures with bounded exponential delays. SDK retries are disabled. Validation failures and other HTTP errors do not retry. The deadline and either the runner or per-call signal interrupt both requests and backoff. Cancellation bounds when the adapter returns; an uncooperative custom client can still continue its own work.
@@ -66,6 +81,36 @@ policy, or the submitted evidence. Unknown causes remain unspecified.
 
 Successful results retain `answers`, reported `model`, reported token `usage`, and a SHA-256 of the submitted request. `cost_usd` is `null` unless both usage and explicit `pricing: { inputUsdPerMillionTokens, outputUsdPerMillionTokens }` are available. Configured prices produce an estimate for the successful response; costs from unsuccessful or interrupted attempts are unknown.
 
+## Images and Box questions
+
+Jev 1.13 and 1.14 read still images, and Jev 1.14 can answer a Box question: where in an image something is. TypeSafe leaves both off unless a request says it carries one, and the runner says so for you: it sends `TypeSafe-Danger-Zone-Images: true` on every request that carries an image or a Box question and on no other.
+
+```ts
+import { createJevRunner, type JevJudge } from '@parcha/agentrun-jev';
+
+const judge: JevJudge = createJevRunner({ model: 'jev-1.14.0' });
+const { answers, usage, cost_usd } = await judge({
+  label: 'door', kind: 'judge',
+  state: 'The camera shows <photo>.',
+  questions: {
+    open: { type: 'noul', instructions: 'Is the door in <photo> open?' },
+    door: { type: 'box', instructions: 'the door in <photo>' },
+  },
+  assets: { '<photo>': { bytes: pngOrJpegFileBytes, mime: 'image/png' } },   // PNG or JPEG file bytes
+});
+// answers.door = { type: 'box', box: { top, bottom, left, right }, confidence } — edges are fractions of the image's height and width
+```
+
+- `assets` maps each reference, a string that appears literally in `state` or in a question's `instructions` or `criteria`, to the image. The runner builds the SDK's image from the bytes and sends it as a multipart file part. Each reference must appear inside a string value of the state, an instruction or a criterion (a question id or an option name does not count), or the request is refused.
+- A Box question needs `model: 'jev-1.14.0'` (or `'jev-preview'`); `jev-latest` is 1.13 and does not answer it. A Box on any other model, or a Box with no image, is refused with `JevError` code `invalid_request` before any request. `model` can also be named per call.
+- Before any request, the runner also refuses what Jev would reject: a file that is not PNG or JPEG (the bytes are checked against `mime`), an empty file, a file over 16 MiB, a reference nothing uses. It does not check pixel dimensions (each side 280 to 1568, total 78,400 to 2,458,624 pixels) or the 32 MiB request limit; Jev rejects those with a 400 or 413, which surfaces as a typed `http` error.
+- Images are billed as input tokens at the text price, and `usage.input_tokens` counts them. `cost_usd` is as for text: the provider's reported cost, else your `pricing` applied to the counted tokens, else `null`. `request_sha256` names each image by its own SHA-256, so the same request hashes the same and a different image does not.
+- A Box answer is always a box (there is no "not found"): ask a noul question in the same request to check presence. Edges and confidence are checked to lie in [0, 1] and the box to run forward (`top <= bottom`, `left <= right`); a confidence the provider omits is `null`.
+- `JevJudge`, `JevJudgeParams`, `JevJudgeResult` and `JevImage` are the types of this call. A package that asks Jev about images takes a `JevJudge` from its host and never imports the SDK.
+- Images and Box go to TypeSafe's own API only. The OpenRouter path is not verified for them, so a request that carries either is refused with `provider: "openrouter"`.
+- Request bodies are sent uncompressed; the SDK's default compresses them with zstd.
+- Box is a preview: Jev's page says its request format, answers and limits can change between releases.
+
 ## Offline tests
 
 Inject a client to exercise the real adapter without a key or network request. After installing the DSL and Jev packages, save this as `offline-jev.mjs` and run `node offline-jev.mjs`:

package/dist/index.d.ts

@@ -4,7 +4,32 @@ import type { WorkflowDeps, SystemOneResponseReason } from "@parcha/agentrun-dsl
 export interface JevClient {
     systemOne(request: SystemOneRequest, options: RequestOptions): PromiseLike<unknown>;
 }
+/** Where Jev is reached: TypeSafe's own API (default) or OpenRouter's System One path. */
+export type JevProvider = "typesafe" | "openrouter";
+/** The provider an environment's keys select: OpenRouter only when `OPENROUTER_API_KEY` is set and `TYPESAFE_API_KEY` is not. The runner itself never reads this. */
+export declare function jevProviderFromEnv(env?: Record<string, string | undefined>): JevProvider;
+/** A still image a request carries: PNG or JPEG file bytes and the type they are. The runner builds the SDK's image from it and sends it as a multipart file part. */
+export interface JevImage {
+    bytes: Uint8Array;
+    mime: "image/png" | "image/jpeg";
+}
+type RunJudge = NonNullable<WorkflowDeps["runJudge"]>;
+/** What one call to the runner takes: the interpreter's `runJudge` parameters, plus the images the request carries and an optional model for this call.
+ *  `assets` maps each reference, a string that appears literally in `state` or in a question's `instructions` or `criteria` (`"<photo>"`), to its image;
+ *  every reference must appear, and a Box question needs at least one image. */
+export type JevJudgeParams = Parameters<RunJudge>[0] & {
+    assets?: Record<string, JevImage>;
+    /** Names the model for this call, over the runner's `model`. A Box question needs `jev-1.14.0` or `jev-preview`; the default `jev-latest` is 1.13. */
+    model?: string;
+};
+/** The typed answers (choice, noul, score with their probabilities; box with its edges and confidence), the model that answered, its token usage, and
+ *  the cost: the provider's reported cost, else the host's configured price applied to the counted tokens, else null. */
+export type JevJudgeResult = Awaited<ReturnType<RunJudge>>;
+/** The judge `createJevRunner` returns, and the port a host passes to a package that asks Jev about images without importing the SDK. */
+export type JevJudge = (params: JevJudgeParams) => Promise<JevJudgeResult>;
 export interface JevOptions {
+    /** `typesafe` (default) or `openrouter`. OpenRouter sets the API root and the default model (`~typesafe/jev-latest`); its key is `apiKey` or `OPENROUTER_API_KEY`. Cannot be combined with `client`. */
+    provider?: JevProvider;
     apiKey?: string;
     /** API root, without /v1/systemone; useful for an explicitly configured gateway. */
     baseURL?: string;
@@ -20,7 +45,7 @@ export interface JevOptions {
     maxAttempts?: number;
     retryBaseMs?: number;
     retryMaxMs?: number;
-    /** Explicit estimate; unknown prices are never silently treated as free. */
+    /** Explicit estimate, used when the provider reports no cost; unknown prices are never silently treated as free. */
     pricing?: {
         inputUsdPerMillionTokens: number;
         outputUsdPerMillionTokens: number;
@@ -34,7 +59,7 @@ export type JevResponseDiagnostic = {
     reason: JevResponseReason;
 };
 export type JevRequestDiagnostic = {
-    reason: "invalid_state" | "state_too_large" | "invalid_questions";
+    reason: "invalid_state" | "state_too_large" | "invalid_questions" | "invalid_assets" | "unsupported_model" | "unsupported_provider";
     label?: string;
     stateBytes?: number;
     maxStateBytes?: number;
@@ -48,5 +73,5 @@ export declare class JevError extends Error {
     readonly responseDiagnostic?: JevResponseDiagnostic;
     constructor(code: JevErrorCode, message: string, attempts?: number, status?: number | undefined, requestDiagnostic?: JevRequestDiagnostic | undefined, responseDiagnostic?: JevResponseDiagnostic);
 }
-export declare function createJevRunner(options?: JevOptions): NonNullable<WorkflowDeps["runJudge"]>;
+export declare function createJevRunner(options?: JevOptions): JevJudge;
 export {};

package/dist/index.js

@@ -1,6 +1,37 @@
 import { createHash } from "node:crypto";
-import { APIConnectionError, APIError, TypeSafeClient } from "@typesafe-ai/sdk";
+import { APIConnectionError, APIError, Image, TypeSafeClient } from "@typesafe-ai/sdk";
 import { validateAnswers, SystemOneError, isSystemOneResponseReason } from "@parcha/agentrun-dsl";
+const keyVariable = { typesafe: "TYPESAFE_API_KEY", openrouter: "OPENROUTER_API_KEY" };
+const OPENROUTER_BASE_URL = "https://openrouter.ai/api";
+const OPENROUTER_DEFAULT_MODEL = "~typesafe/jev-latest";
+/** The provider an environment's keys select: OpenRouter only when `OPENROUTER_API_KEY` is set and `TYPESAFE_API_KEY` is not. The runner itself never reads this. */
+export function jevProviderFromEnv(env = process.env) {
+    const set = (name) => typeof env[name] === "string" && env[name].trim().length > 0;
+    return set("OPENROUTER_API_KEY") && !set("TYPESAFE_API_KEY") ? "openrouter" : "typesafe";
+}
+const IMAGE_HEADER = "TypeSafe-Danger-Zone-Images";
+const MAX_IMAGE_BYTES = 16 * 1024 * 1024;
+const BOX_MODEL = "jev-1.14.0";
+/** Models that answer a Box question: Jev 1.14 and later by name, and the preview alias. `jev-latest` is 1.13 and does not. */
+function answersBox(model) {
+    if (model === "jev-preview")
+        return true;
+    const version = /^jev-(\d+)\.(\d+)(?:\.\d+)?$/.exec(model);
+    return version !== null && (Number(version[1]) > 1 || (Number(version[1]) === 1 && Number(version[2]) >= 14));
+}
+/** Every string value of a JSON value, not its keys. */
+function* stringValues(value) {
+    if (typeof value === "string")
+        yield value;
+    else if (Array.isArray(value))
+        for (const each of value)
+            yield* stringValues(each);
+    else if (record(value))
+        for (const each of Object.values(value))
+            yield* stringValues(each);
+}
+const startsWith = (bytes, prefix) => prefix.every((byte, index) => bytes[index] === byte);
+const imageMagic = { "image/png": [0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a], "image/jpeg": [0xff, 0xd8, 0xff] };
 const adapterResponseReasons = ["response_shape", "token_usage", "model_identifier", "cost_range", "answer_validation", "max_tokens_exceeded"];
 export function isJevResponseReason(value) {
     return isSystemOneResponseReason(value) || typeof value === "string" && adapterResponseReasons.includes(value);
@@ -62,21 +93,29 @@ export function createJevRunner(options = {}) {
     configuration(timeoutMs !== null || options.client !== undefined, "The TypeSafe SDK requires a finite timeout. For timeoutMs: null, supply a cancellation-only host client.");
     configuration(Number.isInteger(maxAttempts) && maxAttempts >= 1 && maxAttempts <= 10, "maxAttempts must be an integer from 1 to 10.");
     configuration(milliseconds(retryBaseMs, true) && milliseconds(retryMaxMs, true) && retryBaseMs <= retryMaxMs, "Retry delays must be nonnegative integers; retryBaseMs cannot exceed retryMaxMs.");
-    configuration(!options.client || (options.apiKey === undefined && options.baseURL === undefined && options.fetch === undefined), "Supply either client or transport options, not both.");
+    configuration(!options.client || (options.apiKey === undefined && options.baseURL === undefined && options.fetch === undefined && options.provider === undefined), "Supply either client or transport options, not both.");
+    const provider = options.provider ?? "typesafe";
+    configuration(provider === "typesafe" || provider === "openrouter", "provider must be typesafe or openrouter.");
     const pricing = options.pricing ? { ...options.pricing } : undefined;
     if (pricing)
         configuration([pricing.inputUsdPerMillionTokens, pricing.outputUsdPerMillionTokens].every(n => Number.isFinite(n) && n >= 0), "Configured token prices must be finite nonnegative numbers.");
-    const model = options.model ?? "jev-latest";
+    const model = options.model ?? (provider === "openrouter" ? OPENROUTER_DEFAULT_MODEL : "jev-latest");
     configuration(typeof model === "string" && model.trim().length > 0, "model must be a nonempty string.");
     let client;
     try {
+        const key = options.apiKey ?? (provider === "openrouter" ? process.env.OPENROUTER_API_KEY : undefined);
+        // The SDK falls back to TYPESAFE_API_KEY when given no key: for OpenRouter that would send one provider's key to another.
+        if (provider === "openrouter" && (typeof key !== "string" || key.trim().length === 0))
+            throw new Error("no key");
         client = options.client ?? new TypeSafeClient({
-            apiKey: options.apiKey, baseURL: options.baseURL, fetch: options.fetch,
+            apiKey: key, baseURL: options.baseURL ?? (provider === "openrouter" ? OPENROUTER_BASE_URL : undefined), fetch: options.fetch,
             defaultModel: model, retry: { maxRetries: 0 }, timeout: timeoutMs ?? undefined, logLevel: "off",
+            // Bodies go as they always have: uncompressed JSON, which a host's own fetch, a gateway and OpenRouter all read.
+            compression: false,
         });
     }
     catch {
-        throw new JevError("configuration", "Could not configure Jev. Supply apiKey or set TYPESAFE_API_KEY, and check transport options.");
+        throw new JevError("configuration", `Could not configure Jev. Supply apiKey or set ${keyVariable[provider]}, and check transport options.`);
     }
     const runnerSignal = options.signal;
     return async (params) => {
@@ -112,8 +151,11 @@ export function createJevRunner(options = {}) {
             const stateBytes = Buffer.byteLength(serializedState, "utf8");
             if (maxStateBytes !== null && stateBytes > maxStateBytes)
                 throw invalid("state_too_large", `Jev assembled state is ${stateBytes} UTF-8 bytes; configured host limit is ${maxStateBytes} bytes. No request was sent.`, stateBytes);
+            const callModel = params.model ?? model;
+            if (typeof callModel !== "string" || callModel.trim().length === 0)
+                throw invalid("invalid_questions", "model must be a nonempty string; no request was sent.");
             try {
-                body = JSON.stringify({ state: JSON.parse(serializedState), model, questions: params.questions });
+                body = JSON.stringify({ state: JSON.parse(serializedState), model: callModel, questions: params.questions });
                 request = JSON.parse(body);
                 if (!record(request.questions) || Object.keys(request.questions).length === 0)
                     throw new Error();
@@ -121,13 +163,50 @@ export function createJevRunner(options = {}) {
             catch {
                 throw invalid("invalid_questions", "Jev requires a nonempty JSON question map; no request was sent.");
             }
+            // Images and Box questions: validated whole before any send; the header goes with exactly these requests.
+            const hasBox = Object.values(request.questions).some(question => record(question) && question.type === "box");
+            const given = params.assets === undefined || params.assets === null ? {} : params.assets;
+            if (!record(given))
+                throw invalid("invalid_assets", "assets must map each reference to an image; no request was sent.");
+            const references = Object.keys(given);
+            const carriesImage = references.length > 0;
+            if ((carriesImage || hasBox) && provider !== "typesafe" && options.client === undefined)
+                throw invalid("unsupported_provider", "Images and Box questions are sent to TypeSafe's own API only; the OpenRouter path is not verified for them. No request was sent.");
+            if (hasBox && !answersBox(callModel))
+                throw invalid("unsupported_model", `A Box question needs a model that answers it: ${BOX_MODEL} (or jev-preview); jev-latest is 1.13. No request was sent.`);
+            if (hasBox && !carriesImage)
+                throw invalid("invalid_assets", "A Box question locates something in an image, and the request carries none. No request was sent.");
+            let assets;
+            let assetDigests;
+            if (carriesImage) {
+                // A reference is replaced where it sits inside a string value of the state, an instruction or a criterion; keys (question ids, option names) are never replaced.
+                const texts = [...stringValues(JSON.parse(serializedState)), ...Object.values(request.questions).flatMap(question => record(question) ? [...stringValues(question.instructions), ...stringValues(question.criteria)] : [])];
+                assets = Object.create(null);
+                assetDigests = Object.create(null);
+                for (const reference of references) {
+                    const image = given[reference];
+                    const sound = typeof reference === "string" && reference.length > 0 && Buffer.byteLength(reference, "utf8") <= 1024 && !/["\\\u0000-\u001f]/.test(reference)
+                        && record(image) && image.bytes instanceof Uint8Array && (image.mime === "image/png" || image.mime === "image/jpeg")
+                        && image.bytes.byteLength > 0 && image.bytes.byteLength <= MAX_IMAGE_BYTES && startsWith(image.bytes, imageMagic[image.mime]);
+                    if (!sound)
+                        throw invalid("invalid_assets", "Each image is a PNG or JPEG file of at most 16 MiB, with bytes that are that type and a plain reference. No request was sent.");
+                    if (!texts.some(text => text.includes(reference)))
+                        throw invalid("invalid_assets", "Every image reference must appear in the state or a question. No request was sent.");
+                    assets[reference] = new Image(image.bytes);
+                    assetDigests[reference] = { sha256: createHash("sha256").update(image.bytes).digest("hex"), mime: image.mime, bytes: image.bytes.byteLength };
+                }
+                request = { ...request, assets };
+                // The hash names the images by their own digests, so it identifies the request that was sent without holding the bytes.
+                body = JSON.stringify({ ...JSON.parse(body), assets: assetDigests });
+            }
+            const headers = carriesImage || hasBox ? { [IMAGE_HEADER]: "true" } : undefined;
             for (attempt = 1;; attempt++) {
                 if (signal.aborted)
                     throw abortError();
                 let result;
                 try {
                     result = await abortable(client.systemOne(request, {
-                        signal, ...(timeoutMs === null ? {} : { timeout: Math.max(1, timeoutMs - (Date.now() - started)) }), retry: { maxRetries: 0 },
+                        signal, ...(timeoutMs === null ? {} : { timeout: Math.max(1, timeoutMs - (Date.now() - started)) }), retry: { maxRetries: 0 }, ...(headers ? { headers } : {}),
                     }), signal, abortError);
                 }
                 catch (error) {
@@ -163,14 +242,20 @@ export function createJevRunner(options = {}) {
                     throw new JevError("invalid_response", "Jev returned invalid answers or probability distributions.", attempt, undefined, undefined, { reason: error instanceof SystemOneError && isSystemOneResponseReason(error.responseReason) ? error.responseReason : "answer_validation" });
                 }
                 const usage = result.usage;
-                if (usage != null && (!record(usage) || !tokens(usage.input_tokens) || !tokens(usage.output_tokens))) {
+                // Usage carries token counts, a provider-reported cost (USD), or both; a half pair of counts or a bad cost is invalid.
+                const counted = record(usage) && tokens(usage.input_tokens) && tokens(usage.output_tokens);
+                const reported = record(usage) && usage.cost !== undefined && usage.cost !== null;
+                const reportedCost = reported && typeof usage.cost === "number" && Number.isFinite(usage.cost) && usage.cost >= 0 ? usage.cost : undefined;
+                const noCounts = record(usage) && usage.input_tokens === undefined && usage.output_tokens === undefined;
+                if (usage != null && (!record(usage) || (reported ? reportedCost === undefined || !(counted || noCounts) : !counted))) {
                     throw new JevError("invalid_response", "Jev returned invalid token usage.", attempt, undefined, undefined, { reason: "token_usage" });
                 }
                 if (result.model != null && (typeof result.model !== "string" || !result.model.trim())) {
                     throw new JevError("invalid_response", "Jev returned an invalid model identifier.", attempt, undefined, undefined, { reason: "model_identifier" });
                 }
-                const metering = record(usage) ? { input_tokens: usage.input_tokens, output_tokens: usage.output_tokens } : null;
-                const cost = pricing && metering ? (metering.input_tokens * pricing.inputUsdPerMillionTokens + metering.output_tokens * pricing.outputUsdPerMillionTokens) / 1_000_000 : null;
+                const metering = counted ? { input_tokens: usage.input_tokens, output_tokens: usage.output_tokens } : null;
+                // The provider's own cost wins; else the configured estimate; else unknown (null), never 0.
+                const cost = reportedCost !== undefined ? reportedCost : pricing && metering ? (metering.input_tokens * pricing.inputUsdPerMillionTokens + metering.output_tokens * pricing.outputUsdPerMillionTokens) / 1_000_000 : null;
                 if (cost !== null && !Number.isFinite(cost))
                     throw new JevError("invalid_response", "Jev cost estimate exceeded the numeric range.", attempt, undefined, undefined, { reason: "cost_range" });
                 return {

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@parcha/agentrun-jev",
-  "version": "0.1.0-beta.13",
+  "version": "0.1.0-beta.15",
   "description": "Optional Jev system one decisions for AgentRun workflows",
   "type": "module",
   "license": "Apache-2.0",
@@ -27,8 +27,8 @@
     "test": "node --test test/*.test.mjs"
   },
   "dependencies": {
-    "@parcha/agentrun-dsl": "0.1.0-beta.13",
-    "@typesafe-ai/sdk": "0.6.0"
+    "@parcha/agentrun-dsl": "0.1.0-beta.15",
+    "@typesafe-ai/sdk": "0.7.0"
   },
   "repository": {
     "type": "git",