@usagetap/sdk 1.3.2 → 1.4.0

package/README.md

@@ -33,7 +33,7 @@ import { withMetering } from "@usagetap/sdk/openai";
 
 const openai = withMetering(new OpenAI(), "cust_123");
 const completion = await openai.responses.create({
-	model: "gpt-5.5-mini",
+	model: "gpt-5.6-luna",
 	input: "Draft a welcome email for our Pro plan",
 });
 
@@ -52,12 +52,9 @@ calls:
 import OpenAI from "openai";
 import { withCompression } from "@usagetap/sdk/openai";
 
-const openai = withCompression(new OpenAI(), {
-	// Defaults to 1,000. Use 0 to always attempt compression.
-	minContextTokens: 2_000,
-});
+const openai = withCompression(new OpenAI());
 const completion = await openai.responses.create({
-	model: "gpt-5.5-mini",
+	model: "gpt-5.6-luna",
 	input: longPrompt,
 });
 ```
@@ -66,12 +63,32 @@ The same `withMetering` and `withCompression` APIs are exported from
 `@usagetap/sdk/anthropic` and `@usagetap/sdk/openrouter`. Remove the wrapper or
 call `.unwrap()` to recover the original provider client.
 
-`withCompression` uses a fast token estimate over the combined request context
-and skips the compression step below 1,000 estimated tokens by default. This
-avoids an extra network round trip when likely savings are small. Override the
-cutoff with `minContextTokens`; set it to `0` to always attempt compression.
+`withCompression` compresses user messages only, uses a fast token estimate
+over the combined request context, and skips the compression step below 1,000
+estimated tokens by default. System instructions, tool content, and assistant
+messages remain unchanged. Override roles or the `minContextTokens` cutoff only
+when you need more control; set the cutoff to `0` to always attempt compression.
 The separate `minTokens` option remains a per-text-segment cutoff.
 
+When workload evidence calls for tuning, the wrapper also accepts the hosted
+Messages API controls directly:
+
+```ts
+const openai = withCompression(new OpenAI(), {
+	mode: "model_auto", // or "model_force" / "deterministic"
+	roles: {
+		user: { aggressiveness: 0.2 },
+		system: { aggressiveness: 0.1 },
+	},
+	latencyBudgetMs: 1_000,
+	compactEmptyUserMessages: false,
+	compactDuplicateUserTextParts: false,
+	failOpen: true,
+});
+```
+
+Omitting the options object remains the recommended starting point.
+
 Wrappers compose. Put metering outside compression so the metered operation
 includes compression and the provider call:
 
@@ -85,6 +102,126 @@ const openai = withMetering(
 Each `.unwrap()` removes one layer. Do not also set `promptCompression: true` on
 `withMetering` when using a separate `withCompression` layer.
 
+### UsageTap Gateway
+
+The core client can call the OpenAI-compatible UsageTap Gateway without a
+second SDK. Use a `utk-` key with `gateway:invoke`. Add
+`compression:invoke` if the same workflow also requests hosted Compression.
+Existing `gk-` and compatible `cmp-` keys continue to work:
+
+```ts
+import { UsageTap } from "@usagetap/sdk";
+
+const usageTap = new UsageTap();
+const completion = await usageTap.gateway.chat.completions.create({
+	model: "usagetap/standard",
+	customerId: "cust_123",
+	feature: "chat.reply",
+	messages: [{ role: "user", content: "Summarize this account." }],
+});
+
+console.log(completion.choices[0].message?.content);
+```
+
+The same resource exposes `models.list()` and the complete native batch
+lifecycle. Batch creation generates the required idempotency key unless one is
+provided:
+
+```ts
+const submitted = await usageTap.gateway.batches.create({
+	requests: reports.map((report) => ({
+		custom_id: report.id,
+		body: {
+			model: "usagetap/standard",
+			customerId: report.customerId,
+			messages: [{ role: "user", content: report.prompt }],
+		},
+	})),
+});
+
+const batch = await usageTap.gateway.batches.wait(submitted);
+if (batch.status !== "completed") {
+	throw new Error(`Batch ended with status: ${batch.status}`);
+}
+
+// Parses the Gateway's NDJSON result stream into typed objects.
+const results = await usageTap.gateway.batches.results(batch.id);
+```
+
+Use `gateway.batches.retrieve()`, `cancel()`, and `results()` when you want to
+manage polling yourself. Set `gatewayBaseUrl` or `USAGETAP_GATEWAY_URL` for a
+non-default deployment.
+
+### Context summarization
+
+Published context-summarization profiles are available through
+`usageTap.summarization`. A managed single summary can wait for completion in
+the initial request:
+
+```ts
+const summary = await usageTap.summarization.summaries.create({
+	profile: "weekly-account-summary-abcd5678",
+	wait: true,
+	context: {
+		id: "account-123",
+		type: "account_history",
+		content: accountHistory,
+	},
+});
+
+console.log(summary.result);
+```
+
+Batch submissions and polling use the same resource pattern as Gateway
+batches:
+
+```ts
+const submitted = await usageTap.summarization.batches.create({
+	profile: "weekly-account-summary-abcd5678",
+	items: accounts.map((account) => ({
+		id: account.id,
+		type: "account_history",
+		content: account.history,
+	})),
+});
+
+const batch = await usageTap.summarization.batches.wait(submitted);
+```
+
+Self-managed workflows can use `summarization.profiles.retrieve()` to load the
+published prompt and model settings, then
+`summarization.measurements.create()` to report source and summary token
+counts.
+
+### Runaway circuit breaker
+
+Set a local per-run call cap and pass the same `runId` on every model call in a
+workflow. Once the cap is reached, the SDK throws `USAGETAP_CIRCUIT_OPEN` before
+`call_begin` or the paid provider request can start:
+
+```ts
+import { UsageTapClient } from "@usagetap/sdk";
+
+const usageTap = new UsageTapClient({
+	circuitBreaker: { maxCallsPerRun: 20 },
+});
+const run = { customerId: "cust_123", runId: crypto.randomUUID() };
+
+try {
+	for (;;) {
+		const result = await usageTap.meter(run, async () => callModel());
+		if (result.done) break;
+	}
+} finally {
+	usageTap.resetRun(run);
+}
+```
+
+`canRunContinue(run)` returns the current decision for graceful partial-result
+handling. Idempotent retries do not consume another slot. The guard is
+process-local by design, so use a stable `runId` in each SDK process and keep
+account-level UsageTap limits enabled for distributed enforcement.
+
 For advanced entitlement control, `wrapOpenAI` exposes the full UsageTap context
 and applies entitlement-aware defaults when you omit `model`.
 
@@ -105,6 +242,25 @@ const ai = wrapOpenAI(openai, usageTap, {
 });
 ```
 
+### Optional end-user attribution
+
+`customerId` identifies the customer account. When that account has multiple
+users, add `customerUserId` to attribute the call to the responsible end user
+in Live activity. Prefer a stable, non-PII application user ID.
+`customerUserName` and `customerUserEmail` are optional display metadata. The
+SDK does not infer them, and existing integrations remain valid when these
+fields are omitted.
+
+```ts
+const context = {
+	customerId: currentCustomer.id,
+	customerUserId: currentUser.id,       // Optional, recommended when available
+	customerUserName: currentUser.name,   // Optional display metadata
+	customerUserEmail: currentUser.email, // Optional display metadata
+	feature: "chat.send",
+};
+```
+
 ### Prompt compression
 
 Prompt compression is an explicit step after `call_begin`. `beginCall` only starts the call and returns the `callId`; `promptCompress` compresses locally, records savings metadata against that call, and returns the compressed prompt for your vendor request. Raw prompt content is not sent to UsageTap.
@@ -114,6 +270,9 @@ import { protectPromptText } from "@usagetap/sdk";
 
 const begin = await usageTap.beginCall({
 	customerId: "cust_123",
+	customerUserId: currentUser.id,
+	customerUserName: currentUser.name,
+	customerUserEmail: currentUser.email,
 	feature: "chat.send",
 });
 
@@ -181,7 +340,7 @@ When using The Token Company, configure `tokenCompanyApiKey` on `UsageTapClient`
 
 For advanced custom flows, `compressPromptInput(input, options?)` returns compression results without recording telemetry, and `recordPromptCompression({ callId, promptCompression })` records precomputed savings metadata against a call.
 
-> **Heads up:** `UsageTapClient` always negotiates the canonical UsageTap media type by sending `Accept: application/vnd.usagetap.v1+json`. Every response now uses the `{ result, data, correlationId }` envelope exclusively and the begin payload includes `data.idempotency.key` (always matching `callId`), per-meter snapshots, and subscription metadata. Set `autoIdempotency: false` (or pass your own `idempotency`) to skip the SDK's auto-generated key and rely on the server's deterministic fallback when retriable semantics are acceptable.
+> **Heads up:** `UsageTapClient` always negotiates the canonical UsageTap media type by sending `Accept: application/vnd.usagetap.v1+json`. Every response uses the `{ result, data, correlationId }` envelope and the begin payload includes `data.idempotency.key` (matching `callId`), per-meter snapshots, and subscription metadata. Keep `autoIdempotency` enabled unless you provide a unique key yourself. The server fallback is deterministic, so identical inputs can replay an earlier call.
 
 ### Streaming helpers
 
@@ -207,7 +366,7 @@ export async function POST() {
 }
 ```
 
-`wrapOpenAI` inspects `begin.data.vendorHints.preferredModel`: premium entitlements resolve to `gpt5`, otherwise the wrapper falls back to `gpt5-mini`. Use the manual pattern shown earlier when you need to toggle reasoning effort or attach search tools based on the returned allowances.
+`wrapOpenAI` preserves the model already supplied by the application. It does not invent a model mapping or fallback. Use the manual `withUsage` pattern when BLOCK or DOWNGRADE must control provider invocation, model selection, reasoning effort, or search tools.
 
 ### Overriding usage context per request
 
@@ -218,7 +377,8 @@ await ai.chat.completions.create(
 	{ messages },
 	{
 		usageTap: {
-			customerId: currentUser.id,
+			customerId: currentCustomer.id,
+			customerUserId: currentUser.id, // Optional
 			feature: "chat.assist",
 			tags: ["beta"],
 			requested: { standard: true, premium: true, search: true, reasoningLevel: "HIGH" },
@@ -227,13 +387,13 @@ await ai.chat.completions.create(
 );
 ```
 
-The begin response for that call will promote premium plans to `gpt5`, fall back to `gpt5-mini` otherwise, and cap reasoning to the granted tier.
+The begin response returns the granted entitlements. The application must use those fields to select only a model and capabilities it has explicitly configured.
 
 For streaming calls created with `{ stream: true }`, UsageTap automatically calculates usage from the final OpenAI response (or falls back to estimates when available). The wrapped stream retains OpenAI-specific helpers like `finalChatCompletion()`.
 
 ### responses.create support
 
-The wrapper also instruments `openai.responses.create`, applying vendor hints (preferred models, token limits) and collecting usage data the same way as chat completions.
+The wrapper also instruments `openai.responses.create`, preserves the supplied model and request, and collects usage data the same way as chat completions.
 
 ### OpenRouter support
 
@@ -264,6 +424,7 @@ const ai = wrapOpenAI(openrouter, usageTap, {
 
 const completion = await ai.chat.completions.create(
 	{
+		model: "your-existing-openrouter-model",
 		messages: [{ role: "user", content: "Hello from OpenRouter!" }],
 	},
 	{
@@ -274,7 +435,7 @@ const completion = await ai.chat.completions.create(
 );
 ```
 
-`begin.data.models` will surface the OpenRouter-specific identifiers the customer can use (for example, `standard` ⇒ `gpt5-mini`, `premium` ⇒ `gpt5`). Since `wrapOpenAI` honors those hints, you can omit `model` and let UsageTap keep the request aligned with the active entitlement.
+`begin.data.models` may surface organization-configured model shortlists. Treat them as guidance; keep the application's existing model or select an explicitly approved fallback after checking `allowed`.
 
 ### Express middleware
 
@@ -319,7 +480,7 @@ app.post("/api/chat", async (req, res) => {
 });
 ```
 
-With that context in place, premium calls receive `gpt5` and everyone else falls back to `gpt5-mini`. To respect `allowed.reasoningLevel` or `allowed.search`, read the begin payload inside route handlers (see the manual `withUsage` example above) and shape the OpenAI request accordingly.
+The middleware meters the supplied provider request. To enforce model tier, `allowed.reasoningLevel`, or `allowed.search`, read the begin payload inside route handlers (see the manual `withUsage` example above) and shape the provider request accordingly.
 
 ### React hook for chat UIs
 
@@ -328,10 +489,11 @@ Build chat interfaces with automatic UsageTap tracking:
 ```tsx
 import { useChatWithUsage } from "@usagetap/sdk/react";
 
-function ChatComponent({ userId }) {
+function ChatComponent({ customerId, currentUser }) {
 	const { messages, input, setInput, handleSubmit, isLoading } = useChatWithUsage({
 		api: "/api/chat",
-		customerId: userId,
+		customerId,
+		customerUserId: currentUser.id, // Optional hint; validate on the server
 		feature: "chat.assistant",
 	});
 	
@@ -395,7 +557,7 @@ const completion = await openai.chat.completions.create({
 });
 ```
 
-`wrapFetch` detects OpenAI API endpoints, handles streaming and non-streaming responses, and automatically extracts usage data. Persist the `begin.data.allowed` blob wherever you store session context so every downstream `openai` call can resolve to `gpt5` (premium) or `gpt5-mini` (standard). You can override context per-request using special headers:
+`wrapFetch` detects OpenAI API endpoints, handles streaming and non-streaming responses, and automatically extracts usage data. It preserves the application's model. If limits must control provider selection, use an explicit begin decision before this layer. You can override metering context per request using special headers:
 
 ```ts
 await openai.chat.completions.create(
@@ -552,6 +714,7 @@ Custom meters allow you to track usage beyond standard LLM metrics—ideal for a
 ```ts
 const result = await usageTap.incrementCustomMeter({
 	customerId: "cust_123",
+	customerUserId: currentUser.id,
 	meterSlot: "CUSTOM1",  // or "CUSTOM2"
 	amount: 5,
 	feature: "agent_actions",
@@ -570,6 +733,8 @@ console.log("Blocked:", result.data.blocked);
 **Parameters:**
 
 - `customerId` (string, required): Customer identifier
+- `customerUserId` (string, optional): Stable identifier for the end user responsible for the event
+- `customerUserName` and `customerUserEmail` (string, optional): Display fields for live activity
 - `meterSlot` ("CUSTOM1" | "CUSTOM2", required): Which custom meter to increment
 - `amount` (number, required): Positive number to decrement from quota
 - `feature` (string, optional): Feature identifier for tracking
@@ -710,7 +875,7 @@ UsageTap responds exclusively with the canonical `{ result, data, correlationId
 - `meters` is a per-counter snapshot including remaining quotas, total limits, usage to date, and convenience ratios. `remainingRatios` mirrors the same information in a compact map for quick lookups.
 - `subscription` contains the active plan identity, versioning, and upcoming replenishment timestamps so you can render customer-facing UI without querying Dynamo yourself.
 - `models` surfaces per-organization vendor hints (e.g. standard vs. premium model shortlists).
-- `idempotency` reveals the actual key that was persisted (`callId` mirrors this value). When you omit `idempotency` in the request, the backend derives a deterministic hash from organization, customer, feature, and requested entitlements.
+- `idempotency` reveals the actual key that was persisted (`callId` mirrors this value). The SDK generates a unique key by default. With SDK auto-generation disabled, the backend derives a deterministic hash from organization, customer, feature, requested entitlements, call type, PAYG hold, and pricing mode; identical inputs can replay an earlier call.
 - `plan` and `balances` remain available alongside the core begin payload for backwards compatibility with earlier SDK versions.
 
 ### Example `call_end` success
@@ -767,7 +932,12 @@ UsageTap responds exclusively with the canonical `{ result, data, correlationId
 }
 ```
 
-Send `cachedInputTokens` when available so UsageTap can apply provider cache-read pricing correctly.
+Send `cachedInputTokens` and `cacheWriteInputTokens` when available so UsageTap
+can apply provider prompt-cache pricing correctly. `inputTokens` is always the
+total input count and includes both subsets. OpenAI's prompt-token total already
+includes cache reads. Anthropic reports ordinary input, cache reads, and cache
+writes separately, so the Anthropic wrapper adds the three counters for
+`inputTokens` while retaining both cache subsets.
 
 `metered` is derived from the raw Dynamo deltas. Additional meters (audio seconds, reasoning tokens, balances) will populate in later phases without breaking the contract.
 
@@ -798,7 +968,10 @@ This is useful when:
 
 ### Batch pricing
 
-Batch mode applies a **50% discount** to standard pricing rates. Set `batch: true` or `pricingMode: "batch"` on either `call_begin` or `call_end`.
+Batch mode applies a **50% discount** to standard pricing rates. UsageTap accepts
+the execution mode reported by your application; it does not attempt to infer
+or verify the vendor workflow. Prefer `pricingMode: "batch"`; `batch: true` is
+the compatibility form.
 
 When set on `call_begin`, the pricing mode carries through to `call_end` automatically. Setting it on `call_end` overrides the `call_begin` value.
 
@@ -813,7 +986,7 @@ const begin = await usageTap.beginCall({
 // Option 2: Set on call_end (overrides call_begin)
 await usageTap.endCall({
   callId: begin.data.callId,
-  modelUsed: "gpt-4o",
+  modelUsed: "gpt-5.6-sol",
   inputTokens: 100,
   responseTokens: 200,
   batch: true,
@@ -822,6 +995,28 @@ await usageTap.endCall({
 ```
 
 Both `batch` and `pricingMode` are echoed in the responses from `call_begin` and `call_end`.
+When both request fields are supplied, `pricingMode` is authoritative.
+
+OpenAI and Anthropic completion/message usage records provide token counts, but
+they do not provide a dependable per-response signal that proves the request
+received vendor batch pricing. The wrappers therefore never infer batch mode
+from `usage`. Set it explicitly in wrapper context when your surrounding
+workflow knows the request is a vendor batch:
+
+```ts
+const metered = withMetering(openai, {
+  customerId: "cust_123",
+  pricingMode: "batch",
+  usageTapClient: usageTap,
+});
+```
+
+The ordinary `wrapOpenAI` and `wrapAnthropic` create-method wrappers do not
+submit native vendor batch jobs. For OpenAI Batch, Anthropic Message Batches, or
+another asynchronous provider, open one UsageTap call per batch item, retain
+its `callId`, then call `endCall` with the usage returned for that item. The
+LLMAsAService `POST /v1/batches` integration performs this lifecycle
+automatically.
 
 ### Raw fetch integrations