@modelprofile.com/flexharness 13.1.1 → 13.2.1

package/ts/readme.md

@@ -418,6 +418,7 @@ The tools use strict action-discriminated inputs:
 - `goal`: `get`, `set`, or `clear`.
 - `scratchpad`: `get`, `set`, `append`, or `clear`. Append concatenates the supplied content exactly.
 - `rename_session`: sets the active session title and returns the authoritative session.
+- `keep_cache_warm` (`cacheWarm: true`, root sessions only): keeps the session's prompt cache warm; see [Keeping the Prompt Cache Warm](#keeping-the-prompt-cache-warm).
 
 Every project action returns the authoritative revision and state; task mutations also return the affected task, and clear returns the removed tasks. Reads never save. A set, clear, append, update, idempotent create, or empty task clear that makes no state change returns the current revision without writing. Mutations load once, apply once, validate the complete next snapshot, and issue one compare-and-swap save at `revision + 1`. FlexHarness never retries or merges an external conflict.
 
@@ -729,6 +730,32 @@ The run takes a steer in at its next step boundary: after the tool results of it
 
 `steerPrompt()` resolves with `{ steerId, queueId, runId }` once the run accepted the steer and `prompt.steer.accepted` was emitted; `prompt.steer.applied` carries the `messageId` of the user message the steer became. Every accepted steer ends in exactly one of two ways: it is in the model context of later turns, or it is listed in `unappliedSteerIds`, in arrival order, on `IFlexPromptResult` and `run.finished`, and on the finished queue entry when that list is not empty. A steer is listed when the run did not take it in before it ended, because its last model step was already running or because it was stopped or failed, and also when the run itself does not stay in context. `keptInContext` on `run.finished` and on the finished queue entry says whether it does: it is `false` for a failed run and for a run stopped before any model output, and such a run lists every steer it accepted, including the ones it took in and reported with `prompt.steer.applied`. Send its prompt and its unapplied steers again to have them answered. The run stops taking steers atomically when its generation ends or `abort()`/`cancelPrompt()` stops it. A steer for a prompt without a running run, or whose run is ending, is refused with `FlexHarnessSteerRejectedError` reason `not-running`; the caller sends it as the next turn instead. Reusing a `steerId` within a run is refused with reason `duplicate`, an unknown `queueId` throws `FlexHarnessNotFoundError`, and child sessions reject this general API (their exact parent run uses `sendSubagentInput()` or `delegate_send` instead). A run holds at most `maxOutstandingPromptsPerSession` pending steers, and their bytes count against the session's outstanding prompt byte limit until the run takes them in or ends, so `startPrompt()` and `enqueuePrompt()` are refused with `FlexHarnessQueueFullError` while pending steers fill it. They are process-local like queue entries: a process restart ends the run and drops its pending steers, while steers it took in are durable parts of the turn.
 
+### Keeping the Prompt Cache Warm
+
+A provider's prompt cache expires when nothing reads it for its TTL, and the next turn then writes the whole prefix again. `keepCacheWarm()` keeps a root session's cache warm until a deadline:
+
+```typescript
+await harness.keepCacheWarm(scopeId, sessionId, { durationMs: 60 * 60 * 1000 }); // warm until 1 h from now
+await harness.keepCacheWarm(scopeId, sessionId, { durationMs: 0 }); // stop
+const state = await harness.getCacheWarm(scopeId, sessionId); // { sessionId, warmUntil?, nextWarmAt?, idleMs?, basis? }
+```
+
+Each call replaces the deadline with `durationMs` from now, capped at `cacheWarm.maxDurationMs` (default 4 h, at most 24 h); `0` stops it. While the deadline has not passed and the session is idle, the harness sends a cache-warm prompt once the session has had no model call for the cache TTL less one minute: 4 minutes for a 5-minute cache, 29 for a 30-minute one, 59 for a 1-hour one. The TTL comes from the request the provider actually sent last, its `basis` in `getCacheWarm()`:
+
+| `basis` | TTL | Sent request |
+| --- | --- | --- |
+| `anthropic-5m`, `anthropic-1h` | 5 min, 1 h | Anthropic: the shortest `cache_control` lifetime of the request |
+| `openai-30m` | 30 min | OpenAI's API, GPT-5.6 and later: `prompt_cache_options.ttl` is 30 minutes after the last write or reuse |
+| `openai-24h` | 30 min | OpenAI's API, an earlier model with `prompt_cache_retention: '24h'` (`cache: { retention: '24h' }`): OpenAI keeps the prefix typically around 30 minutes and up to 24 hours, without promising more |
+| `openai-in-memory` | 5 min | OpenAI's API, an earlier model with `in_memory` or no retention; or the ChatGPT backend of ChatGPT accounts, which takes no retention and documents none. In-memory entries last 5 to 10 minutes of inactivity |
+| `configured` | `cache.warmDurationMs` | any |
+
+With no basis (another provider or endpoint, or a custom transport that may change the request) the TTL is 5 minutes. No OpenAI retention promises a hit beyond about 30 minutes, so none makes warming unnecessary; extended retention only spaces warm turns further apart. Source: OpenAI's [prompt caching guide](https://developers.openai.com/api/docs/guides/prompt-caching), read 2026-10-11. A prompt, a steer or a running turn resets the wait, and no warm prompt is sent while a turn runs, a prompt is queued, an undo's redo history is open, a reversion is pending or a slash command runs. The warm prompt is the fixed text `FLEX_CACHE_WARM_PROMPT`, starting with `[cache-warm]`, which asks for the single word `ok`. Its turn uses the model and system of the session's last turn and offers the same tools, so it reads the cached prefix, but it refuses every tool call, takes at most two steps, caps each model call at `cacheWarm.maxOutputTokens` (default 1024) and takes no steer: `steerPrompt()` refuses one with reason `not-running`. Keeping a session warm costs a cache read of its prefix and a few new tokens per warm turn; the warm exchanges stay in the model context.
+
+Warm turns are marked so hosts can leave them out of what they show and of what they count as work: both messages, the session `activity`, the prompt queue entry, `run.started`, `run.finished` and `model.usage` carry `cacheWarm: true`. A warm turn changes neither the session's `updatedAt` nor its `preview`, and adds no undo step. Three events report the keeper: `cache.warm.changed` (`source` `host` or `agent`, `warmUntil` absent once stopped), `cache.warm.completed` with the warm turn's `usage`, whose `cacheReadTokens` and `cacheWriteTokens` are the provider's own counts and show whether it hit the cache (OpenAI's `input_tokens_details.cached_tokens` and `cache_write_tokens`, Anthropic's `cache_read_input_tokens` and `cache_creation_input_tokens`), its `idleForMs` since the session's previous model call and its `basis`, and `cache.warm.stopped` with `reason` `deadline`, `cleared`, `failed` (the warm turn failed; `error` set) or `unavailable` (the session or its scope went away).
+
+`builtInTools: { cacheWarm: true }` gives root sessions the `keep_cache_warm` tool, so the agent can keep its own cache warm: `{ duration: '1h' }` sets the deadline (amounts of `s`, `m` or `h`), `'off'` or `'0'` stops it. The keeper is process-local: it ends with `dispose()` and does not survive a restart. It warms only the loaded scope it was set in: retiring the scope or deleting the session stops it with `unavailable`, and it never loads a scope again. Tests pass `cacheWarm.clock` (`now`, `setTimeout`, `clearTimeout`) to drive it with a fake clock.
+
 ## Slash Commands
 
 `parseSlashCommand()` is the public strict pure parser. It accepts at most 768 KiB, returns `not-command` for input not starting with `/`, `malformed` for invalid slash syntax, and `parsed` with the exact input, lowercase command name, separator-stripped raw argument text, and OpenCode-compatible quoted tokenization. Command names match `[a-z][a-z0-9_-]{0,63}`. The public `isValidSlashCommandName(name)` and `isReservedSlashCommandName(name)` helpers let applications validate registration names against the same rules. Single and double quotes group tokens and are stripped; escapes are not interpreted.

package/ts/utils.harnessoptions.ts

@@ -12,6 +12,8 @@ import {
 import type {
   IFlexAgentSessionPolicy,
   IFlexAttachmentResolver,
+  IFlexCacheWarmClock,
+  IFlexCacheWarmOptions,
   IFlexCallbackLimits,
   IFlexHarnessOptions,
   IFlexHarnessStores,
@@ -46,8 +48,21 @@ export interface INormalizedProjectManagementTools {
 export interface INormalizedBuiltInTools {
   readonly renameSession: boolean;
   readonly projectManagement?: INormalizedProjectManagementTools;
+  readonly cacheWarm: boolean;
 }
 
+export interface INormalizedCacheWarmOptions {
+  readonly maxDurationMs: number;
+  readonly maxOutputTokens: number;
+  readonly clock: IFlexCacheWarmClock;
+}
+
+const defaultCacheWarmMaxDurationMs = 4 * 60 * 60 * 1000;
+// One call warms for at most a day; a host extends it by calling again.
+const maximumCacheWarmMaxDurationMs = 24 * 60 * 60 * 1000;
+const defaultCacheWarmMaxOutputTokens = 1024;
+const maximumCacheWarmMaxOutputTokens = 32_768;
+
 const DEFAULT_CALLBACK_LIMITS: Required<IFlexCallbackLimits> = {
   maxEvents: 10_000,
   maxOutputBytes: 1024 * 1024,
@@ -242,7 +257,7 @@ function normalizeCodeModeLimits(codeMode: unknown): Readonly<IFlexCodeModeLimit
 export function normalizeBuiltInTools(
   options: IFlexHarnessOptions<unknown>['builtInTools'],
 ): INormalizedBuiltInTools {
-  if (options === undefined) return Object.freeze({ renameSession: false });
+  if (options === undefined) return Object.freeze({ renameSession: false, cacheWarm: false });
   if (
     !options
     || typeof options !== 'object'
@@ -252,14 +267,19 @@ export function normalizeBuiltInTools(
     throw new FlexHarnessValidationError('builtInTools must be a plain object.');
   }
   const unsupported = Object.keys(options)
-    .find((key) => !['renameSession', 'projectManagement'].includes(key));
+    .find((key) => !['renameSession', 'projectManagement', 'cacheWarm'].includes(key));
   if (unsupported) throw new FlexHarnessValidationError(`builtInTools does not support "${unsupported}".`);
-  if (options.renameSession !== undefined && typeof options.renameSession !== 'boolean') {
-    throw new FlexHarnessValidationError('builtInTools.renameSession must be a boolean.');
+  for (const key of ['renameSession', 'cacheWarm'] as const) {
+    if (options[key] !== undefined && typeof options[key] !== 'boolean') {
+      throw new FlexHarnessValidationError(`builtInTools.${key} must be a boolean.`);
+    }
   }
   const projectManagement = options.projectManagement;
   if (projectManagement === undefined) {
-    return Object.freeze({ renameSession: options.renameSession ?? false });
+    return Object.freeze({
+      renameSession: options.renameSession ?? false,
+      cacheWarm: options.cacheWarm ?? false,
+    });
   }
   if (
     !projectManagement
@@ -283,6 +303,7 @@ export function normalizeBuiltInTools(
   }
   return Object.freeze({
     renameSession: options.renameSession ?? false,
+    cacheWarm: options.cacheWarm ?? false,
     projectManagement: Object.freeze({
       task: projectManagement.task ?? true,
       goal: projectManagement.goal ?? true,
@@ -374,6 +395,54 @@ export function normalizeSlashCommands<TScope>(
   return Object.freeze(normalized);
 }
 
+const systemCacheWarmClock: IFlexCacheWarmClock = Object.freeze({
+  now: () => Date.now(),
+  setTimeout: (callback: () => void, delayMs: number) => {
+    const handle = setTimeout(callback, delayMs);
+    // A warm period never holds the process open.
+    handle.unref?.();
+    return handle;
+  },
+  clearTimeout: (handle: unknown) => clearTimeout(handle as ReturnType<typeof setTimeout>),
+});
+
+export function normalizeCacheWarmOptions(
+  input: IFlexHarnessOptions<unknown>['cacheWarm'],
+): INormalizedCacheWarmOptions {
+  if (input !== undefined && !isPlainOptions(input as unknown)) {
+    throw new FlexHarnessValidationError('cacheWarm must be a plain object.');
+  }
+  const options: IFlexCacheWarmOptions = input ?? {};
+  const unsupported = Object.keys(options)
+    .find((key) => !['maxDurationMs', 'maxOutputTokens', 'clock'].includes(key));
+  if (unsupported) throw new FlexHarnessValidationError(`cacheWarm does not support "${unsupported}".`);
+  const clock: IFlexCacheWarmClock = options.clock ?? systemCacheWarmClock;
+  if (
+    !clock
+    || typeof clock !== 'object'
+    || typeof clock.now !== 'function'
+    || typeof clock.setTimeout !== 'function'
+    || typeof clock.clearTimeout !== 'function'
+  ) {
+    throw new FlexHarnessValidationError('cacheWarm.clock must provide now, setTimeout and clearTimeout.');
+  }
+  return Object.freeze({
+    maxDurationMs: resolveBoundedPositiveInteger(
+      options.maxDurationMs,
+      'cacheWarm.maxDurationMs',
+      defaultCacheWarmMaxDurationMs,
+      maximumCacheWarmMaxDurationMs,
+    ),
+    maxOutputTokens: resolveBoundedPositiveInteger(
+      options.maxOutputTokens,
+      'cacheWarm.maxOutputTokens',
+      defaultCacheWarmMaxOutputTokens,
+      maximumCacheWarmMaxOutputTokens,
+    ),
+    clock,
+  });
+}
+
 export function resolveBoundedPositiveInteger(
   value: number | undefined,
   name: string,

package/ts/utils.json.ts

@@ -566,7 +566,7 @@ function validateSession(value: unknown, path: string): IValidatedSessionIdentit
   const activity = requireRecord(session.activity, `${path}.activity`);
   requireOnlyKeys(
     activity,
-    ['runId', 'status', 'startedAt', 'completedAt', 'error', 'limit', 'rejectedPermissionId'],
+    ['runId', 'status', 'startedAt', 'completedAt', 'error', 'limit', 'rejectedPermissionId', 'cacheWarm'],
     `${path}.activity`,
   );
   if (!activityStatuses.includes(activity.status as (typeof activityStatuses)[number])) {
@@ -578,6 +578,11 @@ function validateSession(value: unknown, path: string): IValidatedSessionIdentit
   requireOptionalString(activity.error, `${path}.activity.error`);
   requireOptionalModelLimit(activity.limit, `${path}.activity.limit`);
   requireOptionalString(activity.rejectedPermissionId, `${path}.activity.rejectedPermissionId`);
+  // FlexHarness 13.2.0 stored a warm turn's activity with `cacheWarm`; session repair on load
+  // removes it and restores the latest real turn's activity.
+  if (activity.cacheWarm !== undefined && activity.cacheWarm !== true) {
+    throw new FlexHarnessStoreFormatError(`${path}.activity.cacheWarm must be true when present.`);
+  }
   const relationship = [
     session.parentSessionId,
     session.parentRunId,
@@ -776,6 +781,7 @@ function validateMessage(value: unknown, path: string): IValidatedMessageIdentit
       'limit',
       'rejectedPermissionId',
       'steerId',
+      'cacheWarm',
     ],
     path,
   );
@@ -799,6 +805,9 @@ function validateMessage(value: unknown, path: string): IValidatedMessageIdentit
       throw new FlexHarnessStoreFormatError(`${path}.steerId is only valid on a user message.`);
     }
   }
+  if (message.cacheWarm !== undefined && message.cacheWarm !== true) {
+    throw new FlexHarnessStoreFormatError(`${path}.cacheWarm must be true when present.`);
+  }
   if (message.model !== undefined) validateModel(message.model, `${path}.model`);
   if (message.usage !== undefined) validateUsage(message.usage, `${path}.usage`);
   if (!Array.isArray(message.parts)) {

package/ts/utils.sessionpreview.ts

@@ -1,13 +1,13 @@
 import { FLEX_PROJECT_MANAGEMENT_LIMITS, type IFlexMessage } from './interfaces.js';
 
 /**
- * What a session is about before anyone titles it: the text of its first prompt that has any, as
- * one line (every run of whitespace becomes one space), cut to the title bound at a character
- * boundary. Undefined while no prompt carries text.
+ * What a session is about before anyone titles it: the text of its first prompt that has any,
+ * cache-warm prompts aside, as one line (every run of whitespace becomes one space), cut to the
+ * title bound at a character boundary. Undefined while no prompt carries text.
  */
 export function sessionPreview(messagesArg: readonly IFlexMessage[]): string | undefined {
   for (const message of messagesArg) {
-    if (message.role !== 'user') continue;
+    if (message.role !== 'user' || message.cacheWarm) continue;
     const text = message.parts
       .map((part) => (part.type === 'text' ? part.text : ''))
       .join(' ')

package/ts_chat_harness/classes.transcript.ts

@@ -180,7 +180,7 @@ export class FlexHarnessTranscript {
   public view(): IFlexHarnessTranscriptView {
     return structuredClone({
       messages: this.flexMessages.flatMap((message, index) =>
-        flexMessageToHarnessRows(message, index, this.timings)),
+        message.cacheWarm ? [] : flexMessageToHarnessRows(message, index, this.timings)),
       permissions: [...this.permissionRows.values()],
       questions: [],
       status: this.status,
@@ -217,6 +217,11 @@ export class FlexHarnessTranscript {
     }
     if (event.scopeId !== this.scopeId || event.sessionId !== this.sessionId) return change;
     const timestamp = Date.parse(event.timestamp);
+    // A cache-warm turn's messages are held but not shown, so their events change no row.
+    const warmMessage = 'message' in event
+      ? event.message.cacheWarm === true
+      : 'messageId' in event && this.flexMessages.some((message) =>
+        message.messageId === event.messageId && message.cacheWarm);
     switch (event.type) {
       case 'session.created':
       case 'session.updated':
@@ -227,7 +232,7 @@ export class FlexHarnessTranscript {
         change.messages = change.permissions = change.status = change.usage = true;
         break;
       case 'run.started':
-        this.setStatus(change, { type: 'busy' });
+        if (!event.cacheWarm) this.setStatus(change, { type: 'busy' });
         break;
       case 'message.created':
       case 'message.updated':
@@ -280,13 +285,15 @@ export class FlexHarnessTranscript {
           this.usage = sumUsage(this.usage, flexUsageToHarness(event.message.usage));
           change.usage = true;
         }
+        if (event.cacheWarm) break;
         this.setStatus(change, event.status === 'failed'
           ? { type: 'error', ...(event.error ? { message: event.error.message } : {}) }
           : { type: 'idle' });
         break;
       case 'prompt.started':
       case 'prompt.running':
-        this.runningQueueId = event.queueId;
+        // A steer cannot join a cache-warm turn.
+        if (!event.entry.cacheWarm) this.runningQueueId = event.queueId;
         break;
       case 'prompt.finished':
         if (this.runningQueueId === event.queueId) this.runningQueueId = undefined;
@@ -297,6 +304,10 @@ export class FlexHarnessTranscript {
       default:
         break;
     }
+    if (warmMessage) {
+      change.messages = false;
+      change.deltas = [];
+    }
     return change;
   }
 

package/ts_chat_harness/readme.md

@@ -78,6 +78,7 @@ The host forwards every event of the harness instance, of every session, in orde
 - Permission rows stay in the transcript once answered, showing the decision. A request its run leaves unanswered, because the run was stopped or failed, is removed when the run finishes.
 - FlexHarness emits no event for project state. After a run, or after a project tool's row completes, read `getProjectState()` and pass it to `applyProjectState()` for the task list and the scratchpad.
 - `questions` is always empty: FlexHarness has no question requests.
+- A cache-warm turn (see the FlexHarness readme, Keeping the Prompt Cache Warm) has no rows and leaves the status and the session card as the last real turn left them and `activeQueueId` unset, so a steer never joins it. Its usage still counts in `usage`: it is spent.
 - Messages the session has not taken in yet (a queued prompt, a steer) carry no text in FlexHarness events. The host that sent them shows them as pending rows.
 - An attachment row needs a name, a media type and a size. The public projection keeps no payload and states these only when the prompt did, so an attachment without all three, such as a remote URL of unknown size, has no row.