omk-ai 0.95.0 → 0.95.2

package/README.md

@@ -9,6 +9,7 @@ Unified LLM API with automatic model discovery, provider configuration, token an
 - [Supported Providers](#supported-providers)
 - [Installation](#installation)
 - [Quick Start](#quick-start)
+- [Prompt Cache Boundaries](#prompt-cache-boundaries)
 - [Tools](#tools)
   - [Defining Tools](#defining-tools)
   - [Handling Tool Calls](#handling-tool-calls)
@@ -210,6 +211,29 @@ for (const block of response.content) {
 }
 ```
 
+## Prompt Cache Boundaries
+
+Use typed cache-boundary metadata when the start of a system prompt is stable but its suffix changes between requests:
+
+```typescript
+import { type Context, deriveContextPromptCacheKey } from 'omk-ai';
+
+const stablePrefix = 'You are a coding assistant.';
+const dynamicSuffix = '\nCurrent task: Review the changes.';
+const context: Context = {
+  systemPrompt: stablePrefix + dynamicSuffix,
+  systemPromptCacheBoundary: stablePrefix.length,
+  messages: []
+};
+
+// Optional diagnostic/affinity key. Keep the scope stable for the same cache domain.
+const cacheKey = deriveContextPromptCacheKey(context, 'openai/gpt-4o-mini');
+```
+
+`systemPromptCacheBoundary` is a UTF-16 offset, so JavaScript's `.length` produces the expected unit. `deriveContextPromptCacheKey()` hashes the stable prefix, scope, and canonical tool schemas; it excludes the dynamic suffix and session ID. When prompt caching is enabled, Anthropic marks the stable prefix block and supported OpenAI-family transports derive `prompt_cache_key` from the same metadata. Other providers may ignore it.
+
+Set `systemPromptCacheBoundaryBypass: true` when an extension or caller replaces the prompt with dynamic content. A bypassed, non-positive, non-integer, or out-of-range boundary suppresses explicit cache markers and content-derived affinity. It does not promise a cache hit: only provider-returned `usage.cacheRead` confirms one.
+
 ## Tools
 
 Tools enable LLMs to interact with external systems. This library uses TypeBox schemas for type-safe tool definitions with automatic validation using TypeBox's built-in validator and value conversion utilities. TypeBox schemas can be serialized and deserialized as plain JSON, making them ideal for distributed systems.
@@ -328,6 +352,7 @@ for await (const event of s) {
 ```
 
 **Important notes about partial tool arguments:**
+
 - During `toolcall_delta` events, `arguments` contains the best-effort parse of partial JSON
 - Fields may be missing or incomplete - always check for existence before use
 - String values may be truncated mid-word
@@ -377,7 +402,7 @@ for await (const event of s) {
 All streaming events emitted during assistant message generation:
 
 | Event Type | Description | Key Properties |
-|------------|-------------|----------------|
+| ------------ | ------------- | ---------------- |
 | `start` | Stream begins | `partial`: Initial assistant message structure |
 | `text_start` | Text block starts | `contentIndex`: Position in content array |
 | `text_delta` | Text chunk received | `delta`: New text, `contentIndex`: Position |
@@ -787,6 +812,7 @@ multiModel.unregister();
 ```
 
 Notes:
+
 - Responses are consumed from a queue in request start order.
 - If the queue is empty, the faux provider returns an assistant error message with `errorMessage: "No more faux responses queued"`.
 - Use `registration.setResponses([...])` to replace the remaining queue and `registration.appendResponses([...])` to add more responses.
@@ -801,6 +827,7 @@ Notes:
 ### Providers and Models
 
 A **provider** offers models through a specific API. For example:
+
 - **Anthropic** models use the `anthropic-messages` API
 - **Google** models use the `google-generative-ai` API
 - **OpenAI** models use the `openai-responses` API
@@ -1022,12 +1049,14 @@ const geminiResponse = await complete(gemini, context);
 ### Provider Compatibility
 
 All providers can handle messages from other providers, including:
+
 - Text content
 - Tool calls and tool results (including images in tool results)
 - Thinking/reasoning blocks (transformed to tagged text for cross-provider compatibility)
 - Aborted messages with partial content
 
 This enables flexible workflows where you can:
+
 - Start with a fast model for initial responses
 - Switch to a more capable model for complex reasoning
 - Use specialized models for specific tasks
@@ -1101,7 +1130,7 @@ const response = await complete(model, {
 In Node.js environments, you can set environment variables to avoid passing API keys:
 
 | Provider | Environment Variable(s) |
-|----------|------------------------|
+| ---------- | ------------------------ |
 | OpenAI | `OPENAI_API_KEY` |
 | Ant Ling | `ANT_LING_API_KEY` |
 | Azure OpenAI | `AZURE_OPENAI_API_KEY` + `AZURE_OPENAI_BASE_URL` (e.g. `https://{resource}.openai.azure.com`) or `AZURE_OPENAI_RESOURCE_NAME`. Supports `*.openai.azure.com` and `*.cognitiveservices.azure.com`; root endpoints auto-normalize to `/openai/v1`. Optional: `AZURE_OPENAI_API_VERSION` (default `v1`), `AZURE_OPENAI_DEPLOYMENT_NAME_MAP`. |

package/dist/index.d.ts

@@ -18,6 +18,7 @@ export type { MistralOptions } from "./providers/mistral.ts";
 export type { OpenAICodexResponsesOptions, OpenAICodexWebSocketDebugStats, } from "./providers/openai-codex-responses.ts";
 export type { OpenAICompletionsOptions } from "./providers/openai-completions.ts";
 export type { OpenAIResponsesOptions } from "./providers/openai-responses.ts";
+export { deriveContextPromptCacheKey } from "./providers/prompt-cache.ts";
 export * from "./providers/register-builtins.ts";
 export * from "./session-resources.ts";
 export * from "./stream.ts";

package/dist/index.d.ts.map

@@ -1 +1 @@
-{"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAAA,YAAY,EAAE,MAAM,EAAE,OAAO,EAAE,MAAM,SAAS,CAAC;AAC/C,OAAO,EAAE,IAAI,EAAE,MAAM,SAAS,CAAC;AAE/B,cAAc,mBAAmB,CAAC;AAClC,cAAc,mBAAmB,CAAC;AAClC,cAAc,mBAAmB,CAAC;AAClC,cAAc,aAAa,CAAC;AAC5B,cAAc,0BAA0B,CAAC;AACzC,cAAc,aAAa,CAAC;AAC5B,YAAY,EAAE,cAAc,EAAE,sBAAsB,EAAE,MAAM,+BAA+B,CAAC;AAC5F,YAAY,EAAE,eAAe,EAAE,gBAAgB,EAAE,wBAAwB,EAAE,MAAM,0BAA0B,CAAC;AAC5G,YAAY,EAAE,2BAA2B,EAAE,MAAM,uCAAuC,CAAC;AACzF,cAAc,qBAAqB,CAAC;AACpC,YAAY,EAAE,aAAa,EAAE,MAAM,uBAAuB,CAAC;AAC3D,YAAY,EAAE,mBAAmB,EAAE,MAAM,8BAA8B,CAAC;AACxE,YAAY,EAAE,mBAAmB,EAAE,MAAM,8BAA8B,CAAC;AACxE,cAAc,yCAAyC,CAAC;AACxD,YAAY,EAAE,cAAc,EAAE,MAAM,wBAAwB,CAAC;AAC7D,YAAY,EACX,2BAA2B,EAC3B,8BAA8B,GAC9B,MAAM,uCAAuC,CAAC;AAC/C,YAAY,EAAE,wBAAwB,EAAE,MAAM,mCAAmC,CAAC;AAClF,YAAY,EAAE,sBAAsB,EAAE,MAAM,iCAAiC,CAAC;AAC9E,cAAc,kCAAkC,CAAC;AACjD,cAAc,wBAAwB,CAAC;AACvC,cAAc,aAAa,CAAC;AAC5B,cAAc,YAAY,CAAC;AAC3B,cAAc,wBAAwB,CAAC;AACvC,cAAc,yBAAyB,CAAC;AACxC,cAAc,uBAAuB,CAAC;AACtC,YAAY,EACX,aAAa,EACb,gBAAgB,EAChB,mBAAmB,EACnB,mBAAmB,EACnB,WAAW,EACX,aAAa,EACb,eAAe,EACf,iBAAiB,EACjB,sBAAsB,EACtB,iBAAiB,EACjB,iBAAiB,GACjB,MAAM,wBAAwB,CAAC;AAChC,cAAc,qBAAqB,CAAC;AACpC,cAAc,kBAAkB,CAAC;AACjC,cAAc,4BAA4B,CAAC;AAC3C,cAAc,uBAAuB,CAAC","sourcesContent":["export type { Static, TSchema } from \"typebox\";\nexport { Type } from \"typebox\";\n\nexport * from \"./api-registry.ts\";\nexport * from \"./env-api-keys.ts\";\nexport * from \"./image-models.ts\";\nexport * from \"./images.ts\";\nexport * from \"./images-api-registry.ts\";\nexport * from \"./models.ts\";\nexport type { BedrockOptions, BedrockThinkingDisplay } from \"./providers/amazon-bedrock.ts\";\nexport type { AnthropicEffort, AnthropicOptions, AnthropicThinkingDisplay } from \"./providers/anthropic.ts\";\nexport type { AzureOpenAIResponsesOptions } from \"./providers/azure-openai-responses.ts\";\nexport * from \"./providers/faux.ts\";\nexport type { GoogleOptions } from \"./providers/google.ts\";\nexport type { GoogleThinkingLevel } from \"./providers/google-shared.ts\";\nexport type { GoogleVertexOptions } from \"./providers/google-vertex.ts\";\nexport * from \"./providers/images/register-builtins.ts\";\nexport type { MistralOptions } from \"./providers/mistral.ts\";\nexport type {\n\tOpenAICodexResponsesOptions,\n\tOpenAICodexWebSocketDebugStats,\n} from \"./providers/openai-codex-responses.ts\";\nexport type { OpenAICompletionsOptions } from \"./providers/openai-completions.ts\";\nexport type { OpenAIResponsesOptions } from \"./providers/openai-responses.ts\";\nexport * from \"./providers/register-builtins.ts\";\nexport * from \"./session-resources.ts\";\nexport * from \"./stream.ts\";\nexport * from \"./types.ts\";\nexport * from \"./utils/diagnostics.ts\";\nexport * from \"./utils/event-stream.ts\";\nexport * from \"./utils/json-parse.ts\";\nexport type {\n\tOAuthAuthInfo,\n\tOAuthCredentials,\n\tOAuthDeviceCodeInfo,\n\tOAuthLoginCallbacks,\n\tOAuthPrompt,\n\tOAuthProvider,\n\tOAuthProviderId,\n\tOAuthProviderInfo,\n\tOAuthProviderInterface,\n\tOAuthSelectOption,\n\tOAuthSelectPrompt,\n} from \"./utils/oauth/types.ts\";\nexport * from \"./utils/overflow.ts\";\nexport * from \"./utils/retry.ts\";\nexport * from \"./utils/typebox-helpers.ts\";\nexport * from \"./utils/validation.ts\";\n"]}
+{"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAAA,YAAY,EAAE,MAAM,EAAE,OAAO,EAAE,MAAM,SAAS,CAAC;AAC/C,OAAO,EAAE,IAAI,EAAE,MAAM,SAAS,CAAC;AAE/B,cAAc,mBAAmB,CAAC;AAClC,cAAc,mBAAmB,CAAC;AAClC,cAAc,mBAAmB,CAAC;AAClC,cAAc,aAAa,CAAC;AAC5B,cAAc,0BAA0B,CAAC;AACzC,cAAc,aAAa,CAAC;AAC5B,YAAY,EAAE,cAAc,EAAE,sBAAsB,EAAE,MAAM,+BAA+B,CAAC;AAC5F,YAAY,EAAE,eAAe,EAAE,gBAAgB,EAAE,wBAAwB,EAAE,MAAM,0BAA0B,CAAC;AAC5G,YAAY,EAAE,2BAA2B,EAAE,MAAM,uCAAuC,CAAC;AACzF,cAAc,qBAAqB,CAAC;AACpC,YAAY,EAAE,aAAa,EAAE,MAAM,uBAAuB,CAAC;AAC3D,YAAY,EAAE,mBAAmB,EAAE,MAAM,8BAA8B,CAAC;AACxE,YAAY,EAAE,mBAAmB,EAAE,MAAM,8BAA8B,CAAC;AACxE,cAAc,yCAAyC,CAAC;AACxD,YAAY,EAAE,cAAc,EAAE,MAAM,wBAAwB,CAAC;AAC7D,YAAY,EACX,2BAA2B,EAC3B,8BAA8B,GAC9B,MAAM,uCAAuC,CAAC;AAC/C,YAAY,EAAE,wBAAwB,EAAE,MAAM,mCAAmC,CAAC;AAClF,YAAY,EAAE,sBAAsB,EAAE,MAAM,iCAAiC,CAAC;AAC9E,OAAO,EAAE,2BAA2B,EAAE,MAAM,6BAA6B,CAAC;AAC1E,cAAc,kCAAkC,CAAC;AACjD,cAAc,wBAAwB,CAAC;AACvC,cAAc,aAAa,CAAC;AAC5B,cAAc,YAAY,CAAC;AAC3B,cAAc,wBAAwB,CAAC;AACvC,cAAc,yBAAyB,CAAC;AACxC,cAAc,uBAAuB,CAAC;AACtC,YAAY,EACX,aAAa,EACb,gBAAgB,EAChB,mBAAmB,EACnB,mBAAmB,EACnB,WAAW,EACX,aAAa,EACb,eAAe,EACf,iBAAiB,EACjB,sBAAsB,EACtB,iBAAiB,EACjB,iBAAiB,GACjB,MAAM,wBAAwB,CAAC;AAChC,cAAc,qBAAqB,CAAC;AACpC,cAAc,kBAAkB,CAAC;AACjC,cAAc,4BAA4B,CAAC;AAC3C,cAAc,uBAAuB,CAAC","sourcesContent":["export type { Static, TSchema } from \"typebox\";\nexport { Type } from \"typebox\";\n\nexport * from \"./api-registry.ts\";\nexport * from \"./env-api-keys.ts\";\nexport * from \"./image-models.ts\";\nexport * from \"./images.ts\";\nexport * from \"./images-api-registry.ts\";\nexport * from \"./models.ts\";\nexport type { BedrockOptions, BedrockThinkingDisplay } from \"./providers/amazon-bedrock.ts\";\nexport type { AnthropicEffort, AnthropicOptions, AnthropicThinkingDisplay } from \"./providers/anthropic.ts\";\nexport type { AzureOpenAIResponsesOptions } from \"./providers/azure-openai-responses.ts\";\nexport * from \"./providers/faux.ts\";\nexport type { GoogleOptions } from \"./providers/google.ts\";\nexport type { GoogleThinkingLevel } from \"./providers/google-shared.ts\";\nexport type { GoogleVertexOptions } from \"./providers/google-vertex.ts\";\nexport * from \"./providers/images/register-builtins.ts\";\nexport type { MistralOptions } from \"./providers/mistral.ts\";\nexport type {\n\tOpenAICodexResponsesOptions,\n\tOpenAICodexWebSocketDebugStats,\n} from \"./providers/openai-codex-responses.ts\";\nexport type { OpenAICompletionsOptions } from \"./providers/openai-completions.ts\";\nexport type { OpenAIResponsesOptions } from \"./providers/openai-responses.ts\";\nexport { deriveContextPromptCacheKey } from \"./providers/prompt-cache.ts\";\nexport * from \"./providers/register-builtins.ts\";\nexport * from \"./session-resources.ts\";\nexport * from \"./stream.ts\";\nexport * from \"./types.ts\";\nexport * from \"./utils/diagnostics.ts\";\nexport * from \"./utils/event-stream.ts\";\nexport * from \"./utils/json-parse.ts\";\nexport type {\n\tOAuthAuthInfo,\n\tOAuthCredentials,\n\tOAuthDeviceCodeInfo,\n\tOAuthLoginCallbacks,\n\tOAuthPrompt,\n\tOAuthProvider,\n\tOAuthProviderId,\n\tOAuthProviderInfo,\n\tOAuthProviderInterface,\n\tOAuthSelectOption,\n\tOAuthSelectPrompt,\n} from \"./utils/oauth/types.ts\";\nexport * from \"./utils/overflow.ts\";\nexport * from \"./utils/retry.ts\";\nexport * from \"./utils/typebox-helpers.ts\";\nexport * from \"./utils/validation.ts\";\n"]}

package/dist/index.js

@@ -7,6 +7,7 @@ export * from "./images-api-registry.js";
 export * from "./models.js";
 export * from "./providers/faux.js";
 export * from "./providers/images/register-builtins.js";
+export { deriveContextPromptCacheKey } from "./providers/prompt-cache.js";
 export * from "./providers/register-builtins.js";
 export * from "./session-resources.js";
 export * from "./stream.js";

package/dist/index.js.map

@@ -1 +1 @@
-{"version":3,"file":"index.js","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AACA,OAAO,EAAE,IAAI,EAAE,MAAM,SAAS,CAAC;AAE/B,cAAc,mBAAmB,CAAC;AAClC,cAAc,mBAAmB,CAAC;AAClC,cAAc,mBAAmB,CAAC;AAClC,cAAc,aAAa,CAAC;AAC5B,cAAc,0BAA0B,CAAC;AACzC,cAAc,aAAa,CAAC;AAI5B,cAAc,qBAAqB,CAAC;AAIpC,cAAc,yCAAyC,CAAC;AAQxD,cAAc,kCAAkC,CAAC;AACjD,cAAc,wBAAwB,CAAC;AACvC,cAAc,aAAa,CAAC;AAC5B,cAAc,YAAY,CAAC;AAC3B,cAAc,wBAAwB,CAAC;AACvC,cAAc,yBAAyB,CAAC;AACxC,cAAc,uBAAuB,CAAC;AActC,cAAc,qBAAqB,CAAC;AACpC,cAAc,kBAAkB,CAAC;AACjC,cAAc,4BAA4B,CAAC;AAC3C,cAAc,uBAAuB,CAAC","sourcesContent":["export type { Static, TSchema } from \"typebox\";\nexport { Type } from \"typebox\";\n\nexport * from \"./api-registry.ts\";\nexport * from \"./env-api-keys.ts\";\nexport * from \"./image-models.ts\";\nexport * from \"./images.ts\";\nexport * from \"./images-api-registry.ts\";\nexport * from \"./models.ts\";\nexport type { BedrockOptions, BedrockThinkingDisplay } from \"./providers/amazon-bedrock.ts\";\nexport type { AnthropicEffort, AnthropicOptions, AnthropicThinkingDisplay } from \"./providers/anthropic.ts\";\nexport type { AzureOpenAIResponsesOptions } from \"./providers/azure-openai-responses.ts\";\nexport * from \"./providers/faux.ts\";\nexport type { GoogleOptions } from \"./providers/google.ts\";\nexport type { GoogleThinkingLevel } from \"./providers/google-shared.ts\";\nexport type { GoogleVertexOptions } from \"./providers/google-vertex.ts\";\nexport * from \"./providers/images/register-builtins.ts\";\nexport type { MistralOptions } from \"./providers/mistral.ts\";\nexport type {\n\tOpenAICodexResponsesOptions,\n\tOpenAICodexWebSocketDebugStats,\n} from \"./providers/openai-codex-responses.ts\";\nexport type { OpenAICompletionsOptions } from \"./providers/openai-completions.ts\";\nexport type { OpenAIResponsesOptions } from \"./providers/openai-responses.ts\";\nexport * from \"./providers/register-builtins.ts\";\nexport * from \"./session-resources.ts\";\nexport * from \"./stream.ts\";\nexport * from \"./types.ts\";\nexport * from \"./utils/diagnostics.ts\";\nexport * from \"./utils/event-stream.ts\";\nexport * from \"./utils/json-parse.ts\";\nexport type {\n\tOAuthAuthInfo,\n\tOAuthCredentials,\n\tOAuthDeviceCodeInfo,\n\tOAuthLoginCallbacks,\n\tOAuthPrompt,\n\tOAuthProvider,\n\tOAuthProviderId,\n\tOAuthProviderInfo,\n\tOAuthProviderInterface,\n\tOAuthSelectOption,\n\tOAuthSelectPrompt,\n} from \"./utils/oauth/types.ts\";\nexport * from \"./utils/overflow.ts\";\nexport * from \"./utils/retry.ts\";\nexport * from \"./utils/typebox-helpers.ts\";\nexport * from \"./utils/validation.ts\";\n"]}
+{"version":3,"file":"index.js","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AACA,OAAO,EAAE,IAAI,EAAE,MAAM,SAAS,CAAC;AAE/B,cAAc,mBAAmB,CAAC;AAClC,cAAc,mBAAmB,CAAC;AAClC,cAAc,mBAAmB,CAAC;AAClC,cAAc,aAAa,CAAC;AAC5B,cAAc,0BAA0B,CAAC;AACzC,cAAc,aAAa,CAAC;AAI5B,cAAc,qBAAqB,CAAC;AAIpC,cAAc,yCAAyC,CAAC;AAQxD,OAAO,EAAE,2BAA2B,EAAE,MAAM,6BAA6B,CAAC;AAC1E,cAAc,kCAAkC,CAAC;AACjD,cAAc,wBAAwB,CAAC;AACvC,cAAc,aAAa,CAAC;AAC5B,cAAc,YAAY,CAAC;AAC3B,cAAc,wBAAwB,CAAC;AACvC,cAAc,yBAAyB,CAAC;AACxC,cAAc,uBAAuB,CAAC;AActC,cAAc,qBAAqB,CAAC;AACpC,cAAc,kBAAkB,CAAC;AACjC,cAAc,4BAA4B,CAAC;AAC3C,cAAc,uBAAuB,CAAC","sourcesContent":["export type { Static, TSchema } from \"typebox\";\nexport { Type } from \"typebox\";\n\nexport * from \"./api-registry.ts\";\nexport * from \"./env-api-keys.ts\";\nexport * from \"./image-models.ts\";\nexport * from \"./images.ts\";\nexport * from \"./images-api-registry.ts\";\nexport * from \"./models.ts\";\nexport type { BedrockOptions, BedrockThinkingDisplay } from \"./providers/amazon-bedrock.ts\";\nexport type { AnthropicEffort, AnthropicOptions, AnthropicThinkingDisplay } from \"./providers/anthropic.ts\";\nexport type { AzureOpenAIResponsesOptions } from \"./providers/azure-openai-responses.ts\";\nexport * from \"./providers/faux.ts\";\nexport type { GoogleOptions } from \"./providers/google.ts\";\nexport type { GoogleThinkingLevel } from \"./providers/google-shared.ts\";\nexport type { GoogleVertexOptions } from \"./providers/google-vertex.ts\";\nexport * from \"./providers/images/register-builtins.ts\";\nexport type { MistralOptions } from \"./providers/mistral.ts\";\nexport type {\n\tOpenAICodexResponsesOptions,\n\tOpenAICodexWebSocketDebugStats,\n} from \"./providers/openai-codex-responses.ts\";\nexport type { OpenAICompletionsOptions } from \"./providers/openai-completions.ts\";\nexport type { OpenAIResponsesOptions } from \"./providers/openai-responses.ts\";\nexport { deriveContextPromptCacheKey } from \"./providers/prompt-cache.ts\";\nexport * from \"./providers/register-builtins.ts\";\nexport * from \"./session-resources.ts\";\nexport * from \"./stream.ts\";\nexport * from \"./types.ts\";\nexport * from \"./utils/diagnostics.ts\";\nexport * from \"./utils/event-stream.ts\";\nexport * from \"./utils/json-parse.ts\";\nexport type {\n\tOAuthAuthInfo,\n\tOAuthCredentials,\n\tOAuthDeviceCodeInfo,\n\tOAuthLoginCallbacks,\n\tOAuthPrompt,\n\tOAuthProvider,\n\tOAuthProviderId,\n\tOAuthProviderInfo,\n\tOAuthProviderInterface,\n\tOAuthSelectOption,\n\tOAuthSelectPrompt,\n} from \"./utils/oauth/types.ts\";\nexport * from \"./utils/overflow.ts\";\nexport * from \"./utils/retry.ts\";\nexport * from \"./utils/typebox-helpers.ts\";\nexport * from \"./utils/validation.ts\";\n"]}