converse-mcp-server 4.7.1 → 4.8.1

package/README.md

@@ -187,7 +187,7 @@ Cancel running asynchronous operations when needed.
 
 ### 4. Decide Tool
 
-Ask a decision model (OpenAI's `gpt-6-luna`, TypeSafe's Jev, or Cloudflare's Clef) typed questions about a state and get calibrated answers instead of text: a yes probability (`noul`), a chosen option with per-option probabilities (`choice`), or a rubric position with per-level probabilities (`score`). Batch many questions into one call; they are judged in parallel against the same state. Needs any of `OPENAI_API_KEY`, `TYPESAFE_API_KEY`, `CLOUDFLARE_ACCOUNT_ID` + `CLOUDFLARE_API_TOKEN`, or `OPENROUTER_API_KEY`; each model runs on its native host first and falls back to OpenRouter. `model: "auto"` tries `gpt-6-luna`, then `jev-latest`, then `clef`. Pass `images` to judge pictures with `gpt-6-luna` or Clef.
+Ask a decision model (OpenAI's `gpt-6-luna`, TypeSafe's Jev, or Cloudflare's Clef) typed questions about a state and get calibrated answers instead of text: a yes probability (`noul`), a chosen option with per-option probabilities (`choice`), a rubric position with per-level probabilities (`score`), or where a target is in an image (`box`, Jev 1.14). Batch many questions into one call; they are judged in parallel against the same state. Needs any of `OPENAI_API_KEY`, `TYPESAFE_API_KEY`, `CLOUDFLARE_ACCOUNT_ID` + `CLOUDFLARE_API_TOKEN`, or `OPENROUTER_API_KEY`; each model runs on its native host first and falls back to OpenRouter. `model: "auto"` tries `gpt-6-luna`, then `jev-latest`, then `clef`. Pass `images` to judge pictures with `gpt-6-luna`, Jev, or Clef.
 
 ```javascript
 {

package/docs/API.md

@@ -382,7 +382,7 @@ Only jobs in a `queued` or `running` state can be cancelled; already-completed,
       "additionalProperties": {
         "type": "object",
         "properties": {
-          "type": { "type": "string", "enum": ["noul", "choice", "score"] },
+          "type": { "type": "string", "enum": ["noul", "choice", "score", "box"] },
           "instructions": { "anyOf": [{ "type": "string" }, { "type": "object" }, { "type": "array" }] },
           "criteria": { "anyOf": [{ "type": "object" }, { "type": "array" }] }
         },
@@ -401,13 +401,19 @@ Only jobs in a `queued` or `running` state can be cancelled; already-completed,
 - **`state`**: the material to judge. An object with descriptively named fields works best; use an array for sequences such as chat messages. Optional when `files` is given.
 - **`questions`**: named questions, each judged in parallel and in isolation against the same state. The name is your own label and becomes the answer key. Batching many questions into one call adds almost no latency or cost.
 - **`files`**: text files added to the state as `{ "files": { "<path>": "<content>" } }`. When `state` is also given, it moves to `input`. Line ranges (`file.txt{10:50}`) are supported; image files belong in `images`.
-- **`images`**: PNG, JPEG or WebP images judged together with the state, as file paths or base64 data URLs; `state` is optional when images are given. Only `gpt-6-luna` (OpenAI) and `clef` / `clef-flash` (Cloudflare) take images, so `auto` skips other models and naming a text-only model is rejected before sending. OpenAI receives the state and images as one user message. Clef takes at most 4 images, and Cloudflare rejects a request whose size estimate (one token per 4 characters, base64 included) exceeds its 65,536-token window, so Clef images must total about 180 KB; `decide` checks this locally and suggests downscaling. OpenRouter routes are text only.
+- **`images`**: PNG, JPEG or WebP images judged together with the state, as file paths or base64 data URLs; `state` is optional when images are given. `gpt-6-luna` (OpenAI), Jev on TypeSafe, and `clef` / `clef-flash` (Cloudflare) take images; `auto` skips routes that cannot, and naming one is rejected before sending. OpenRouter routes are text only: OpenRouter accepts Jev image assets but drops them, and Jev answers as if no image were sent.
+  - OpenAI receives the state and images as one user message.
+  - TypeSafe receives each image as an asset with a UUID reference, and the state as `{ "input": <state>, "images": [<references>] }` (just `{ "images": [...] }` without a state), with the `TypeSafe-Danger-Zone-Images: true` header TypeSafe requires for images. Jev takes PNG and JPEG only, 256–1568 px per side and at least 78,400 px in total, up to 16 MiB each and 32 MiB per request, and never resizes; `decide` checks these locally, so a WebP image under `auto` moves on to Clef. TypeSafe ignores EXIF orientation, so rotate the pixels upright first. TypeSafe labels images "dreamer-ready": prototype quality whose format and accuracy may change between releases.
+  - Clef takes at most 4 images, and Cloudflare rejects a request whose size estimate (one token per 4 characters, base64 included) exceeds its 65,536-token window, so Clef images must total about 180 KB; `decide` checks this locally and suggests downscaling.
 
 | Type | `criteria` | Answer |
 |---|---|---|
 | `noul` | Optional `{ "true": "...", "false": "..." }` | `noul`: probability 0..1 of yes |
 | `choice` | Required `{ "<option>": "description" \| null }`, 2–255 options | `choice`, per-option `probabilities`, `confidence` |
 | `score` | Required ordered array of levels, lowest first, 2–10 levels; each a description or `{ "label": "...", "description": ... }` | `score` (probability-weighted level position), `legend`, per-level `probabilities`, `confidence` |
+| `box` | None; `instructions` names the target ("the coffee mug nearest the sink") | `box`: `{ top, bottom, left, right }` as fractions 0..1 of the image's height and width from the top-left corner, `confidence` |
+
+`box` needs an image and `jev-1.14` or `jev-preview` on TypeSafe; any other model, including `auto`, is rejected before sending. A box answer is always a box, even when the target is absent, so ask a `noul` about whether it is there in the same call. Each box question finds one target.
 
 `instructions` and criteria descriptions may be objects or arrays that bundle reference data with the question; refer to their fields by `` `name` `` in the text. Questions are validated before any request is sent.
 
@@ -422,7 +428,10 @@ Any answer from `gpt-6-luna` may be `{ "type": "refusal" }` instead of a value,
 | `gpt-6-luna` | `luna`, `gpt-6-luna-decisions` | `openai` | `openai/gpt-6-luna-decisions` |
 | `jev-latest` | `jev` | `typesafe` (`jev-latest`) | `~typesafe/jev-latest` |
 | `jev-1.13` | `jev-1.13.0` | `typesafe` (`jev-1.13.0`) | `typesafe/jev-1.13` |
+| `jev-1.14` | `jev-1.14.0` | `typesafe` (`jev-1.14.0`) | `typesafe/jev-1.14` |
 | `jev-preview` | | `typesafe` | not served |
+
+TypeSafe moves its aliases on each release: `jev-latest` is the newest stable release (1.13 as of 1.14's launch on 2026-10-14) and `jev-preview` the newest release of any kind (1.14). OpenRouter's `~typesafe/jev-latest` follows its own schedule and already serves 1.14, so a `jev-latest` fallback to OpenRouter can answer with a different version. The response's `model` names the version that answered.
 | `clef` | `@cf/cloudflare/clef` | `cloudflare` | `cloudflare/clef` |
 | `clef-flash` | `@cf/cloudflare/clef-flash` | `cloudflare` | `cloudflare/clef-flash` |
 
@@ -491,7 +500,9 @@ Decision · typesafe/jev-1.13-20260917 via OpenRouter · 394 input tokens · $0.
 
 ### Usage Guidance
 
-Decision models read questions literally and are weak at counting, arithmetic, date comparison, and multi-hop reasoning; do those in code and ask narrow, atomic questions. Split compound judgments into separate questions and combine them in code. Treat low `confidence` as a signal to escalate to a generative model or a human. Limits: Jev is text only and takes about 64k tokens per request and 32k for the state plus the longest single question; Clef has a 65,536-token context.
+Decision models read questions literally and are weak at counting, arithmetic, date comparison, and multi-hop reasoning; do those in code and ask narrow, atomic questions. Split compound judgments into separate questions and combine them in code. Treat low `confidence` as a signal to escalate to a generative model or a human. Limits: Jev takes about 64k tokens per request and 32k for the state plus the longest single question; Clef has a 65,536-token context.
+
+When TypeSafe rejects a request as longer than Jev's default context (`max_tokens_exceeded`), `decide` sends it once more with `TypeSafe-Danger-Zone-Long-Context: true`, which raises both limits to 262k tokens. TypeSafe documents long context as slower and less accurate, so it is never sent up front; an answer that needed it says `long context` in the summary line and carries `"long_context": true`. OpenRouter routes never retry this way; OpenRouter serves Jev with a 262k context and no header.
 
 ## Supported Models
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "converse-mcp-server",
-  "version": "4.7.1",
+  "version": "4.8.1",
   "description": "Converse MCP Server - Converse with other LLMs with chat and consensus tools",
   "type": "module",
   "main": "src/index.js",
@@ -95,8 +95,8 @@
   "dependencies": {
     "@anthropic-ai/claude-agent-sdk": "^0.3.296",
     "@anthropic-ai/sdk": "^0.133.0",
-    "@github/copilot-sdk": "^1.0.19",
-    "@google/genai": "^2.28.0",
+    "@github/copilot-sdk": "^1.0.20",
+    "@google/genai": "^2.29.0",
     "@lydell/node-pty": "1.2.0-beta.15",
     "@mistralai/mistralai": "^2.7.0",
     "@modelcontextprotocol/sdk": "^1.32.1",
@@ -111,10 +111,10 @@
     "vite": "^8.3.4"
   },
   "devDependencies": {
-    "@vitest/coverage-v8": "^5.0.3",
+    "@vitest/coverage-v8": "^5.0.4",
     "cross-env": "^10.1.0",
     "eslint": "^10.12.0",
     "rimraf": "^6.1.3",
-    "vitest": "^5.0.3"
+    "vitest": "^5.0.4"
   }
 }

package/src/decisionProviders/formats.js

@@ -8,10 +8,16 @@
  *
  * Every format returns `{ model, answers, usage, id }` from `parse`.
  * `request(model, state, questions, images)` receives images as
- * `{ mimeType, base64 }`; `images: true` marks formats that can send them.
+ * `{ mimeType, base64 }`; `images: true` marks formats that can send them, and
+ * `questionTypes` lists the question types the format can express.
  */
 
+import { randomUUID } from 'node:crypto';
 import { DecisionError } from './http.js';
+import { readImageSize } from './imageSize.js';
+
+/** The question types every decision model answers. */
+export const BASE_QUESTION_TYPES = ['noul', 'choice', 'score'];
 
 function asText(value) {
   return typeof value === 'string' ? value : JSON.stringify(value);
@@ -47,16 +53,81 @@ function toSystemOneQuestions(questions) {
 }
 
 /**
- * TypeSafe and OpenRouter serve System One natively. Jev is text only, and
- * OpenRouter's image routing for other decision models is unverified, so this
- * format sends no images.
+ * OpenRouter serves System One natively but drops TypeSafe `assets` without
+ * an error: Jev answers as if no image were sent, with confident-looking
+ * guesses. So this format sends text and the base question types only.
  */
 export const systemOneFormat = {
   images: false,
+  questionTypes: BASE_QUESTION_TYPES,
   request: (model, state, questions) => ({ model, state, questions: toSystemOneQuestions(questions) }),
   parse: (body) => body,
 };
 
+// TypeSafe rejects images outside these bounds rather than resizing them, and
+// its error names no limit, so they are checked before sending.
+const JEV_IMAGE_FORMATS = { 'image/png': 'png', 'image/jpeg': 'jpg' };
+const JEV_MIN_SIDE = 256;
+const JEV_MAX_SIDE = 1568;
+const JEV_MIN_PIXELS = 78_400;
+const JEV_MAX_IMAGE_BYTES = 16 * 1024 * 1024;
+const JEV_MAX_BODY_BYTES = 32 * 1024 * 1024;
+
+function jevLimitError(message) {
+  return new DecisionError(message, { terminal: true, notSent: true });
+}
+
+function checkJevImage(image, n) {
+  if (!JEV_IMAGE_FORMATS[image.mimeType]) {
+    throw jevLimitError(`Jev accepts PNG and JPEG images only; image ${n} is ${image.mimeType}.`);
+  }
+  const bytes = Buffer.from(image.base64, 'base64');
+  if (bytes.length > JEV_MAX_IMAGE_BYTES) {
+    throw jevLimitError(`Jev accepts images up to 16 MiB each; image ${n} is ${(bytes.length / 1048576).toFixed(1)} MiB.`);
+  }
+  const size = readImageSize(bytes);
+  if (!size) return;
+  if (size.animated) throw jevLimitError(`Jev accepts still images only; image ${n} is an animated PNG.`);
+  const { width, height } = size;
+  if (width < JEV_MIN_SIDE || height < JEV_MIN_SIDE || width > JEV_MAX_SIDE || height > JEV_MAX_SIDE || width * height < JEV_MIN_PIXELS) {
+    throw jevLimitError(
+      `Jev accepts images ${JEV_MIN_SIDE}–${JEV_MAX_SIDE} px per side and at least ${JEV_MIN_PIXELS} px in total; image ${n} is ${width}×${height}. ` +
+      `Resize it so the longest side is at most ${JEV_MAX_SIDE} px, or upscale or pad a small one.`,
+    );
+  }
+}
+
+/**
+ * TypeSafe's own System One endpoint. Images travel in a top-level `assets`
+ * map and appear wherever their reference string occurs; the references go
+ * into the state, so every question sees them. A reference is matched as
+ * plain text anywhere in the request, hence a UUID that cannot occur by
+ * accident.
+ */
+export const typesafeFormat = {
+  images: true,
+  questionTypes: [...BASE_QUESTION_TYPES, 'box'],
+  request(model, state, questions, images = []) {
+    const body = { model, state, questions: toSystemOneQuestions(questions) };
+    if (images.length === 0) return body;
+    images.forEach((image, i) => checkJevImage(image, i + 1));
+    const refs = images.map(() => `image-${randomUUID()}`);
+    const request = {
+      ...body,
+      state: state === undefined ? { images: refs } : { input: state, images: refs },
+      assets: Object.fromEntries(refs.map((ref, i) => [
+        ref,
+        { type: 'image', format: JEV_IMAGE_FORMATS[images[i].mimeType], data_base64: images[i].base64 },
+      ])),
+    };
+    if (Buffer.byteLength(JSON.stringify(request)) > JEV_MAX_BODY_BYTES) {
+      throw jevLimitError('Jev accepts requests up to 32 MiB including base64 image data; send fewer or smaller images.');
+    }
+    return request;
+  },
+  parse: (body) => body,
+};
+
 // Cloudflare rejects requests outside these bounds; checking first turns a
 // nested validation dump into a clear message.
 const CLEF_MAX_QUESTIONS = 64;
@@ -105,6 +176,7 @@ function checkClefImages(images, state, questions) {
  */
 export const cloudflareFormat = {
   images: true,
+  questionTypes: BASE_QUESTION_TYPES,
   request(model, state, questions, images = []) {
     const ids = Object.keys(questions);
     if (ids.length > CLEF_MAX_QUESTIONS) {
@@ -201,6 +273,7 @@ function fromOpenAIAnswer(answer) {
  */
 export const openaiFormat = {
   images: true,
+  questionTypes: BASE_QUESTION_TYPES,
   request(model, state, questions, images = []) {
     const input = images.length === 0
       ? asText(state)

package/src/decisionProviders/http.js

@@ -23,16 +23,18 @@ const MAX_RETRY_AFTER_MS = 30_000;
  * failing over (network, timeout, 408/429/5xx, auth); `terminal` marks request
  * faults that every host of the same model would reject the same way;
  * `notSent` marks a request refused by a local limit check before any network
- * call, so it is not reported as a failed request.
+ * call, so it is not reported as a failed request. `code` is the host's
+ * machine-readable error type, when it sends one.
  */
 export class DecisionError extends Error {
-  constructor(message, { status = null, retryable = false, terminal = false, notSent = false, requestId = null, retryAfterMs = null } = {}) {
+  constructor(message, { status = null, retryable = false, terminal = false, notSent = false, code = null, requestId = null, retryAfterMs = null } = {}) {
     super(message);
     this.name = 'DecisionError';
     this.status = status;
     this.retryable = retryable;
     this.terminal = terminal;
     this.notSent = notSent;
+    this.code = code;
     this.requestId = requestId;
     this.retryAfterMs = retryAfterMs;
   }
@@ -94,6 +96,11 @@ export function extractErrorMessage(body, rawText) {
     const type = body.detail.error_type ? `${body.detail.error_type}: ` : '';
     return `${type}${body.detail.message}`;
   }
+  if (typeof body?.detail?.error_type === 'string') {
+    return body.detail.error_type === 'max_tokens_exceeded'
+      ? 'max_tokens_exceeded: the request is longer than the model\'s context length'
+      : body.detail.error_type;
+  }
   const nested = body?.error?.message ?? body?.error ?? body?.detail ?? body?.message;
   if (typeof nested === 'string') {
     const inner = nested.match(/^HTTP \d+: (\{.*\})$/s);
@@ -221,6 +228,7 @@ async function sendOnce({ url, headers, body, parse, signal, timeoutMs }) {
       status,
       retryable: !firewall && (status === 408 || status === 429 || status >= 500 || status === 401 || status === 403),
       terminal: firewall || status === 400 || status === 422,
+      code: typeof parsed?.detail?.error_type === 'string' ? parsed.detail.error_type : null,
       requestId:
         response.headers.get('x-typesafe-request-id') ||
         response.headers.get('x-generation-id') ||

package/src/decisionProviders/imageSize.js

@@ -0,0 +1,63 @@
+/**
+ * Image header reading
+ *
+ * Reads pixel dimensions from PNG and JPEG headers so a host's size limits can
+ * be checked before upload, without an image library.
+ */
+
+const PNG_SIGNATURE = 0x89504e47;
+
+// JPEG start-of-frame markers carry the dimensions; C4 (DHT), C8 (JPG), and
+// CC (DAC) share the range but are not frames.
+function isStartOfFrame(marker) {
+  return marker >= 0xc0 && marker <= 0xcf && marker !== 0xc4 && marker !== 0xc8 && marker !== 0xcc;
+}
+
+// Markers that stand alone, with no length field: TEM, RST0–7, SOI.
+function isStandalone(marker) {
+  return marker === 0x01 || (marker >= 0xd0 && marker <= 0xd8);
+}
+
+function readPng(buffer) {
+  if (buffer.length < 24 || buffer.toString('ascii', 12, 16) !== 'IHDR') return null;
+  // An acTL chunk ahead of the first IDAT marks an animated PNG.
+  const actl = buffer.indexOf('acTL');
+  const idat = buffer.indexOf('IDAT');
+  return {
+    width: buffer.readUInt32BE(16),
+    height: buffer.readUInt32BE(20),
+    animated: actl !== -1 && (idat === -1 || actl < idat),
+  };
+}
+
+function readJpeg(buffer) {
+  let offset = 2;
+  while (offset + 9 <= buffer.length) {
+    if (buffer[offset] !== 0xff) return null;
+    const marker = buffer[offset + 1];
+    if (marker === 0xff) {
+      offset += 1;
+      continue;
+    }
+    if (isStandalone(marker)) {
+      offset += 2;
+      continue;
+    }
+    if (isStartOfFrame(marker)) {
+      return { width: buffer.readUInt16BE(offset + 7), height: buffer.readUInt16BE(offset + 5), animated: false };
+    }
+    offset += 2 + buffer.readUInt16BE(offset + 2);
+  }
+  return null;
+}
+
+/**
+ * @param {Buffer} buffer - Image file bytes
+ * @returns {{ width: number, height: number, animated: boolean }|null} null
+ *   when the bytes are not a PNG or JPEG this reader understands
+ */
+export function readImageSize(buffer) {
+  if (buffer.length >= 8 && buffer.readUInt32BE(0) === PNG_SIGNATURE) return readPng(buffer);
+  if (buffer.length >= 4 && buffer[0] === 0xff && buffer[1] === 0xd8) return readJpeg(buffer);
+  return null;
+}

package/src/decisionProviders/index.js

@@ -17,8 +17,22 @@
  */
 
 import { getCustomHeaders as getOpenRouterAttributionHeaders } from '../providers/openrouter.js';
-import { callDecisionApi } from './http.js';
-import { cloudflareFormat, openaiFormat, systemOneFormat } from './formats.js';
+import { callDecisionApi, DecisionError } from './http.js';
+import { BASE_QUESTION_TYPES, cloudflareFormat, openaiFormat, systemOneFormat, typesafeFormat } from './formats.js';
+
+// TypeSafe gates experimental features behind per-request opt-in headers.
+const TYPESAFE_IMAGES_HEADER = 'TypeSafe-Danger-Zone-Images';
+const TYPESAFE_LONG_CONTEXT_HEADER = 'TypeSafe-Danger-Zone-Long-Context';
+const TYPESAFE_LONG_CONTEXT_TOKENS = 262_144;
+
+function typesafeHeaders(config, { images = [], questions = {}, longContext = false } = {}) {
+  const usesImages = images.length > 0 || Object.values(questions).some((q) => q?.type === 'box');
+  return {
+    Authorization: `Bearer ${config.apiKeys.typesafe}`,
+    ...(usesImages ? { [TYPESAFE_IMAGES_HEADER]: 'true' } : {}),
+    ...(longContext ? { [TYPESAFE_LONG_CONTEXT_HEADER]: 'true' } : {}),
+  };
+}
 
 export const DECISION_PROVIDERS = {
   openai: {
@@ -37,9 +51,12 @@ export const DECISION_PROVIDERS = {
     label: 'TypeSafe',
     setup: 'set TYPESAFE_API_KEY',
     defaultModel: 'jev-latest',
-    format: systemOneFormat,
+    format: typesafeFormat,
     endpoint: () => 'https://api.typesafe.ai/v1/systemone',
-    headers: (config) => ({ Authorization: `Bearer ${config.apiKeys.typesafe}` }),
+    headers: typesafeHeaders,
+    // Long context is slower and less accurate, so it is only turned on for a
+    // request the default context rejects.
+    longContext: true,
     isAvailable: (config) => Boolean(config?.apiKeys?.typesafe),
     // TypeSafe accepts every published versioned ID, listed or not.
     passthrough: (name) => /^jev-\d+\.\d+(\.\d+)?$/i.test(name),
@@ -79,14 +96,17 @@ export const DECISION_PROVIDERS = {
 /** Failover order: each model's native host first, then OpenRouter. */
 export const DECISION_PROVIDER_PRIORITY = ['openai', 'typesafe', 'cloudflare', 'openrouter'];
 
+const WITH_BOX = [...BASE_QUESTION_TYPES, 'box'];
+
 /**
  * Decision models by canonical name. `hosts` maps each provider that serves
  * the model to that provider's own ID: OpenRouter only knows
- * `~typesafe/jev-latest` and `typesafe/jev-1.13` (it rejects `jev-1.13.0` and
- * `jev-preview`), so IDs are translated per host rather than by a blanket
- * prefix rewrite. Every host ID also routes to its model. `maxImages` is how
- * many images the model takes per call; a host also needs a format that can
- * send them.
+ * `~typesafe/jev-latest` and `typesafe/jev-1.13` / `typesafe/jev-1.14` (it
+ * rejects `jev-1.13.0` and `jev-preview`), so IDs are translated per host
+ * rather than by a blanket prefix rewrite. Every host ID also routes to its
+ * model. `maxImages` is how many images the model takes per call and
+ * `questionTypes` which question types it answers; a host also needs a format
+ * that can send them.
  */
 export const DECISION_MODELS = {
   'gpt-6-luna': {
@@ -94,31 +114,44 @@ export const DECISION_MODELS = {
     hosts: { openai: 'gpt-6-luna', openrouter: 'openai/gpt-6-luna-decisions' },
     // OpenAI documents no image count limit; the API enforces its own.
     maxImages: Infinity,
+    questionTypes: BASE_QUESTION_TYPES,
   },
+  // Jev's image limits are by size, not count; the format checks them.
   'jev-latest': {
     aliases: ['jev'],
     hosts: { typesafe: 'jev-latest', openrouter: '~typesafe/jev-latest' },
-    maxImages: 0,
+    maxImages: Infinity,
+    questionTypes: BASE_QUESTION_TYPES,
   },
   'jev-1.13': {
     aliases: [],
     hosts: { typesafe: 'jev-1.13.0', openrouter: 'typesafe/jev-1.13' },
-    maxImages: 0,
+    maxImages: Infinity,
+    questionTypes: BASE_QUESTION_TYPES,
+  },
+  'jev-1.14': {
+    aliases: [],
+    hosts: { typesafe: 'jev-1.14.0', openrouter: 'typesafe/jev-1.14' },
+    maxImages: Infinity,
+    questionTypes: WITH_BOX,
   },
   'jev-preview': {
     aliases: [],
     hosts: { typesafe: 'jev-preview' },
-    maxImages: 0,
+    maxImages: Infinity,
+    questionTypes: WITH_BOX,
   },
   clef: {
     aliases: ['@cf/cloudflare/clef'],
     hosts: { cloudflare: 'clef', openrouter: 'cloudflare/clef' },
     maxImages: 4,
+    questionTypes: BASE_QUESTION_TYPES,
   },
   'clef-flash': {
     aliases: ['@cf/cloudflare/clef-flash'],
     hosts: { cloudflare: 'clef-flash', openrouter: 'cloudflare/clef-flash' },
     maxImages: 4,
+    questionTypes: BASE_QUESTION_TYPES,
   },
 };
 
@@ -137,9 +170,10 @@ function findCanonicalModel(name) {
 /**
  * Resolve a name on one provider: a catalog model it hosts, then its
  * passthrough rule.
- * @returns {{ model: string, family: string, maxImages: number }|null} The ID
- *   to send to that provider, the model it names (hosts of one family share
- *   failures), and how many images this host can send that model
+ * @returns {{ model: string, family: string, maxImages: number, questionTypes: string[] }|null}
+ *   The ID to send to that provider, the model it names (hosts of one family
+ *   share failures), and how many images and which question types this host
+ *   can send that model
  */
 export function findDecisionModel(provider, name) {
   const trimmed = String(name).trim();
@@ -147,10 +181,17 @@ export function findDecisionModel(provider, name) {
   const canonical = findCanonicalModel(trimmed);
   const hosted = canonical ? DECISION_MODELS[canonical].hosts[provider.name] : undefined;
   if (hosted) {
-    const maxImages = provider.format.images ? DECISION_MODELS[canonical].maxImages : 0;
-    return { model: hosted, family: canonical, maxImages };
+    const entry = DECISION_MODELS[canonical];
+    return {
+      model: hosted,
+      family: canonical,
+      maxImages: provider.format.images ? entry.maxImages : 0,
+      questionTypes: entry.questionTypes.filter((type) => provider.format.questionTypes.includes(type)),
+    };
   }
-  return provider.passthrough(trimmed) ? { model: trimmed, family: trimmed, maxImages: 0 } : null;
+  return provider.passthrough(trimmed)
+    ? { model: trimmed, family: trimmed, maxImages: 0, questionTypes: BASE_QUESTION_TYPES }
+    : null;
 }
 
 function servedBy(providerName) {
@@ -246,7 +287,8 @@ export function resolveDecisionModel(spec, config) {
 
 /**
  * Ask one candidate: translate the System One request into the host's wire
- * format, send it, and map the answers back.
+ * format, send it, and map the answers back. A host with a long-context mode
+ * gets the request again in that mode when its default context rejects it.
  * @param {object} candidate - From resolveDecisionModel
  * @param {object} params
  * @param {string|object|Array} params.state
@@ -254,15 +296,32 @@ export function resolveDecisionModel(spec, config) {
  * @param {Array<{ mimeType: string, base64: string }>} [params.images]
  * @param {object} params.config - Server configuration
  * @param {AbortSignal} [params.signal]
- * @returns {Promise<{ model?: string, answers: object, usage?: object, id?: string }>}
+ * @returns {Promise<{ model?: string, answers: object, usage?: object, id?: string, longContext?: boolean }>}
  */
 export async function askDecisionModel(candidate, { state, questions, images = [], config, signal }) {
   const { provider, model } = candidate;
-  return callDecisionApi({
-    url: provider.endpoint(config, model),
-    headers: provider.headers(config),
-    body: provider.format.request(model, state, questions, images),
-    parse: provider.format.parse,
-    signal,
-  });
+  const body = provider.format.request(model, state, questions, images);
+  const send = (longContext) =>
+    callDecisionApi({
+      url: provider.endpoint(config, model),
+      headers: provider.headers(config, { images, questions, longContext }),
+      body,
+      parse: provider.format.parse,
+      signal,
+    });
+
+  try {
+    return await send(false);
+  } catch (error) {
+    if (!provider.longContext || error.code !== 'max_tokens_exceeded') throw error;
+  }
+  try {
+    return { ...(await send(true)), longContext: true };
+  } catch (error) {
+    if (error.code !== 'max_tokens_exceeded') throw error;
+    throw new DecisionError(
+      `${error.message} (over the ${Math.round(TYPESAFE_LONG_CONTEXT_TOKENS / 1000)}k-token long context too)`,
+      { status: error.status, terminal: true, code: error.code, requestId: error.requestId },
+    );
+  }
 }

package/src/tools/decide.js

@@ -17,7 +17,7 @@ import { createLogger } from '../utils/logger.js';
 
 const logger = createLogger('decide');
 
-const QUESTION_TYPES = ['noul', 'choice', 'score'];
+const QUESTION_TYPES = ['noul', 'choice', 'score', 'box'];
 const QUESTION_FIELDS = ['type', 'instructions', 'criteria'];
 const MAX_CHOICE_OPTIONS = 255;
 const MIN_SCORE_LEVELS = 2;
@@ -61,6 +61,10 @@ function validateQuestion(key, question) {
     return `${at}.instructions must be a non-empty string, object, or array.`;
   }
 
+  if (question.type === 'box') {
+    return criteria === undefined ? null : `${at} is a box question, which takes no criteria: describe the target in instructions.`;
+  }
+
   if (question.type === 'noul') {
     if (criteria === undefined) return null;
     const keys = isPlainObject(criteria) ? Object.keys(criteria).sort() : null;
@@ -177,7 +181,8 @@ async function loadFiles(files, contextProcessor, config) {
   return problems.length ? { error: problems.join('; ') } : { contents };
 }
 
-// The formats every image-capable host accepts (Clef takes no GIF or BMP).
+// The formats some image-capable host accepts (Clef takes no GIF or BMP; Jev
+// takes PNG and JPEG only, which its format checks).
 const IMAGE_MIME_TYPES = ['image/png', 'image/jpeg', 'image/webp'];
 const DATA_URL = /^data:([^;,]+);base64,([A-Za-z0-9+/=\s]+)$/i;
 
@@ -254,6 +259,11 @@ function summarizeAnswer(key, answer) {
     const label = (k) => (answer.legend?.[k] ? `${k} "${answer.legend[k]}"` : k);
     return `- ${key} (score): ${fixed(answer.score)}${range} · confidence ${fixed(answer.confidence)} · ${byProbability(answer.probabilities, label)}`;
   }
+  if (type === 'box') {
+    const box = answer.box || {};
+    const edges = ['left', 'top', 'right', 'bottom'].map((edge) => `${edge} ${fixed(box[edge])}`).join(', ');
+    return `- ${key} (box): ${edges} · confidence ${fixed(answer.confidence)}`;
+  }
   if (type === 'refusal') {
     return `- ${key} (refusal): the model declined to answer`;
   }
@@ -264,6 +274,7 @@ function formatResult(response, candidate, failures) {
   const usage = response.usage || {};
   const header = [
     `Decision · ${response.model || candidate.model} via ${candidate.provider.label}`,
+    response.longContext ? 'long context' : null,
     usage.input_tokens !== undefined ? `${usage.input_tokens} input tokens` : null,
     typeof usage.cost === 'number' ? `$${usage.cost.toFixed(6)}` : null,
   ].filter(Boolean).join(' · ');
@@ -277,6 +288,7 @@ function formatResult(response, candidate, failures) {
   const payload = {
     model: response.model ?? candidate.model,
     provider: candidate.providerName,
+    ...(response.longContext ? { long_context: true } : {}),
     answers: response.answers,
     usage: {
       input_tokens: usage.input_tokens ?? null,
@@ -316,14 +328,25 @@ export async function decideTool(args, dependencies) {
   if (stateError) return createToolError(stateError);
   const questionError = validateQuestions(questions);
   if (questionError) return createToolError(questionError);
+  const types = [...new Set(Object.values(questions).map((q) => q.type))];
+  if (types.includes('box') && images.length === 0) {
+    return createToolError('Box questions locate something in an image: pass the image in "images".');
+  }
 
   const route = resolveDecisionModel(model, config);
   if (route.status !== 'ok') return createToolError(route.error);
-  const candidates = route.candidates.filter((c) => c.maxImages >= images.length);
+  const answersAll = (c) => types.every((type) => c.questionTypes.includes(type));
+  if (!route.candidates.some(answersAll)) {
+    return createToolError(
+      `"${model}" cannot answer box questions on any configured host. ` +
+      'Box needs model "jev-1.14" or "jev-preview" on TypeSafe (TYPESAFE_API_KEY).',
+    );
+  }
+  const candidates = route.candidates.filter((c) => answersAll(c) && c.maxImages >= images.length);
   if (candidates.length === 0) {
     return createToolError(
       `"${model}" cannot take ${images.length} image(s) on any configured host. ` +
-      'Images work with gpt-6-luna (OpenAI) and clef / clef-flash (Cloudflare, up to 4); Jev and OpenRouter routes are text only.',
+      'Images work with gpt-6-luna (OpenAI), Jev (TypeSafe), and clef / clef-flash (Cloudflare, up to 4); OpenRouter routes Jev text only.',
     );
   }
 
@@ -366,14 +389,18 @@ export async function decideTool(args, dependencies) {
 decideTool.description =
   'DECIDE — ask a decision model (OpenAI gpt-6-luna, TypeSafe Jev, Cloudflare Clef) typed questions about a state and get calibrated answers, not text. ' +
   'Question types: "noul" (yes/no → probability 0..1), "choice" (pick one of 2–255 named options → choice, per-option probabilities, confidence), ' +
-  '"score" (ordered rubric of 2–10 levels → weighted position, per-level probabilities, confidence). ' +
+  '"score" (ordered rubric of 2–10 levels → weighted position, per-level probabilities, confidence), ' +
+  '"box" (where one target is in an image → left/top/right/bottom edges as 0..1 fractions of width/height, confidence; jev-1.14 / jev-preview only, needs an image, ' +
+  'always returns a box, so pair it with a noul asking whether the target is there). ' +
   'Batch independent questions over the same state into one call: they are judged in parallel and in isolation, so none sees another\'s answer; each extra question adds its own input tokens. ' +
   'Best for fast semantic judgments (classify, route, select, verify, rank). Ask one narrow, coherent judgment per question, with its full meaning in the question; ' +
   'split independently useful dimensions, but a bounded action choice or contextual interpretation is a valid single question. Do counting, arithmetic, and date comparison in code. ' +
   'confidence measures how concentrated the distribution is, not permission to act: take the top option to pick a best, and treat a noul near 0.5 as "yes and no equally likely". ' +
-  'No explanations are returned. Images (PNG/JPEG/WebP) work with gpt-6-luna and Clef (up to 4, ~180 KB total); Jev is text only, and "auto" skips models that cannot take the images. ' +
+  'No explanations are returned. Images work with gpt-6-luna (PNG/JPEG/WebP), Jev on TypeSafe (PNG/JPEG, 256–1568 px per side, upright pixels: EXIF rotation is ignored), ' +
+  'and Clef (PNG/JPEG/WebP, up to 4, ~180 KB total); "auto" skips models that cannot take the images. ' +
   'gpt-6-luna may answer a question with type "refusal" instead of a value: treat it as unanswered, never as a no. ' +
-  'Limits per model: Jev ~64k tokens per request (~32k for state plus the longest question); Clef ~64k tokens and at most 64 questions named with letters, digits, "_", ".", "-".';
+  'Limits per model: Jev ~64k tokens per request (~32k for state plus the longest question), and on TypeSafe a request over that is resent once in long-context mode ' +
+  '(262k, slower and less accurate; the result says "long context"); Clef ~64k tokens and at most 64 questions named with letters, digits, "_", ".", "-".';
 
 decideTool.inputSchema = {
   type: 'object',
@@ -387,9 +414,10 @@ decideTool.inputSchema = {
       type: 'object',
       description:
         'Named questions, all answered against the same state. The name is your own label and is returned as the answer key. ' +
-        'Each question: { "type": "noul"|"choice"|"score", "instructions": string|object|array, "criteria": ... }. ' +
+        'Each question: { "type": "noul"|"choice"|"score"|"box", "instructions": string|object|array, "criteria": ... }. ' +
         'criteria — noul: optional { "true": "...", "false": "..." }; choice: required { "<option>": "description" | null } (2–255 options); ' +
-        'score: required ordered array of levels, lowest first (2–10 levels); each level is a description, or { "label": "Short name", "description": "..." } to keep the legend short. ' +
+        'score: required ordered array of levels, lowest first (2–10 levels); each level is a description, or { "label": "Short name", "description": "..." } to keep the legend short; ' +
+        'box: none, the instructions name the target ("the coffee mug nearest the sink"). ' +
         'instructions may be an object bundling the question with reference data, referenced by `name` in the text. ' +
         'Example: { "team": { "type": "choice", "instructions": "Which team should handle this?", "criteria": { "billing": "Payments, refunds", "technical": "Bugs, outages" } } }',
       additionalProperties: {
@@ -407,7 +435,8 @@ decideTool.inputSchema = {
       type: 'string',
       description:
         'Decision model. "auto" (default): gpt-6-luna, then jev-latest, then clef, each on every configured host. ' +
-        'A model name ("gpt-6-luna"/"luna", "jev-latest"/"jev", "jev-1.13", "jev-preview", "clef", "clef-flash"): its native host first, falling back to OpenRouter. ' +
+        'A model name ("gpt-6-luna"/"luna", "jev-latest"/"jev", "jev-1.13", "jev-1.14", "jev-preview", "clef", "clef-flash"): its native host first, falling back to OpenRouter. ' +
+        'jev-latest is TypeSafe\'s stable release (1.13); jev-preview is its newest (1.14). ' +
         '"provider:model" (e.g. "openai:gpt-6-luna", "cloudflare:clef-flash", "openrouter:jev-latest"): that provider only. ' +
         'Providers: openai (OPENAI_API_KEY), typesafe (TYPESAFE_API_KEY), cloudflare (CLOUDFLARE_ACCOUNT_ID + CLOUDFLARE_API_TOKEN), openrouter (OPENROUTER_API_KEY).',
     },
@@ -422,7 +451,8 @@ decideTool.inputSchema = {
       items: { type: 'string' },
       description:
         'Images judged together with the state: file paths or base64 data URLs, PNG/JPEG/WebP. Refer to them in questions ("Does the image show ..."). ' +
-        'Supported by gpt-6-luna and clef/clef-flash (Clef: up to 4, about 180 KB in total); other models are skipped or rejected.',
+        'Supported by gpt-6-luna, Jev on TypeSafe (PNG/JPEG, 256–1568 px per side; sent as state { "input": <state>, "images": [...] }), ' +
+        'and clef/clef-flash (Clef: up to 4, about 180 KB in total); other routes are skipped or rejected.',
     },
   },
   required: ['questions'],