@tollgateai/sdk 0.6.0 → 0.7.0

package/README.md

@@ -1,14 +1,60 @@
-# @tollgateai/sdk
+<p align="center">
+  <img src="https://tollgateai.vercel.app/logo.png" alt="Tollgate" width="120" />
+</p>
+
+<h1 align="center">@tollgateai/sdk</h1>
+
+<p align="center">
+  <strong>Real-time gross-margin observability for AI-powered products.</strong><br />
+  Track every LLM call's cost, attribute it to a customer, and know whether you're making money — before the invoice goes out.
+</p>
+
+<p align="center">
+  <a href="https://www.npmjs.com/package/@tollgateai/sdk"><img src="https://img.shields.io/npm/v/@tollgateai/sdk?color=blue&label=npm" alt="npm" /></a>
+  <a href="https://www.npmjs.com/package/@tollgateai/sdk"><img src="https://img.shields.io/npm/dm/@tollgateai/sdk?color=green" alt="downloads" /></a>
+  <img src="https://img.shields.io/badge/node-%3E%3D18-brightgreen" alt="node" />
+  <img src="https://img.shields.io/badge/dependencies-0-brightgreen" alt="zero deps" />
+  <img src="https://img.shields.io/badge/license-MIT-blue" alt="license" />
+</p>
+
+<p align="center">
+  <a href="https://tollgateai.vercel.app">Dashboard</a> &middot;
+  <a href="https://pypi.org/project/tollgateai/">Python SDK</a> &middot;
+  <a href="#quick-start">Quick Start</a> &middot;
+  <a href="#api-reference">API Reference</a>
+</p>
 
-> Real-time gross-margin observability for AI agents. Track every LLM call's cost, attribute it to a customer, and see whether you're making money — before the invoice goes out.
+---
 
-**v0.6.0** &middot; [npm](https://www.npmjs.com/package/@tollgateai/sdk) &middot; [Dashboard](https://tollgateai.vercel.app)
+## Why Tollgate?
 
----
+AI products bill customers on plans (per ticket, per seat, usage-based) but pay providers per token. **Tollgate joins the two in real time** — giving you per-customer, per-agent, per-run gross margin the moment each LLM call completes.
 
-## Why Tollgate
+- **2-line integration** — wrap your provider client once; every call is tracked automatically.
+- **Zero runtime dependencies** — ships as a single ESM + CJS bundle.
+- **Non-blocking** — usage reporting is fire-and-forget with automatic retries. Failures never break your LLM calls.
+- **Privacy-first** — no prompt content is ever transmitted. Only token counts, model identifiers, and metadata.
+- **Universal coverage** — Anthropic, OpenAI, Google Gemini, AWS Bedrock, and every OpenAI-compatible gateway.
 
-You sell an AI-powered product. Each customer interaction triggers LLM calls that cost you real money — input tokens, output tokens, reasoning tokens, audio tokens, cached tokens, web searches, tool calls. Tollgate captures that cost automatically from provider responses, joins it with the revenue your pricing model defines, and shows you per-customer, per-agent, per-run gross margin in real time.
+```
+┌─────────────┐    ┌──────────────┐    ┌───────────────┐
+│  Your App    │───▶│  LLM Provider │───▶│  Provider      │
+│  (SDK wrap)  │◀──│  (Anthropic,  │◀──│  Response       │
+│              │    │   OpenAI, …)  │    │  (tokens, id)  │
+└──────┬───────┘    └──────────────┘    └───────────────┘
+       │ POST /api/track (background, non-blocking)
+       ▼
+┌──────────────────────────────────────────────────────┐
+│  Tollgate Server                                      │
+│  ┌────────────┐  ┌──────────┐  ┌──────────────────┐  │
+│  │ Rate Card  │  │ Plan     │  │ Margin Rollups   │  │
+│  │ (1,500+    │  │ Revenue  │  │ (per customer,   │  │
+│  │  models)   │  │ Config   │  │  agent, run)     │  │
+│  └────────────┘  └──────────┘  └──────────────────┘  │
+└──────────────────────────────────────────────────────┘
+```
+
+---
 
 ## Installation
 
@@ -20,7 +66,9 @@ npm install @tollgateai/sdk
 pnpm add @tollgateai/sdk   # or yarn add @tollgateai/sdk
 ```
 
-Requires Node.js 18+. Zero runtime dependencies.
+**Requirements:** Node.js 18+ &middot; Zero runtime dependencies &middot; ESM and CommonJS supported
+
+---
 
 ## Quick Start
 
@@ -50,24 +98,30 @@ await tollgate.resolve({
 });
 ```
 
+---
+
 ## Provider Support
 
-| Provider | Wrapper | Streaming | What Gets Extracted |
+| Provider | Wrapper | Streaming | Extracted Fields |
 |---|---|---|---|
-| **Anthropic** | `wrapAnthropic` | Automatic | Tokens, thinking/reasoning, cache (read + write by TTL), web search requests, tool calls, latency |
-| **OpenAI** | `wrapOpenAI` | `stream_options: { include_usage: true }` | Tokens, reasoning, cached, audio in/out, text in/out, prediction tokens, service tier, tool calls, latency |
-| **Google Gemini** | `wrapGemini` | Automatic | Tokens, thinking, cached, audio/image/video per-modality, web search (grounding), tool calls, latency |
-| **OpenAI-compatible** | `wrapOpenAI` + `provider: 'openai_compatible'` | Same as OpenAI | Same as OpenAI |
-| **AWS Bedrock** | `wrapBedrock` | Automatic | Tokens, cache (read + write), tool calls, latency |
+| **Anthropic** | `wrapAnthropic` | Automatic | Input/output tokens, cache read/write, web search requests, tool calls, latency |
+| **OpenAI** | `wrapOpenAI` | `stream_options: { include_usage: true }` | Input/output tokens, reasoning, cached, audio in/out, text in/out, prediction tokens, service tier, tool calls, latency |
+| **Google Gemini** | `wrapGemini` | Automatic | Input/output tokens, thinking, cached, audio/image/video per-modality, web search (grounding), tool calls, latency |
+| **OpenAI-compatible** | `wrapOpenAI` + `provider: 'openai_compatible'` | Same as OpenAI | Same as OpenAI + gateway-reported cost (when available) |
+| **AWS Bedrock** | `wrapBedrock` | Automatic | Input/output tokens, cache read/write (per-TTL split), tool calls, latency |
+
+---
 
 ## Configuration
 
-| Environment Variable | Required | Default |
-|---|---|---|
-| `TOLLGATE_API_KEY` | Yes | — |
-| `TOLLGATE_BASE_URL` | No | `https://tollgateai.vercel.app` |
+### Environment Variables
 
-Or pass them directly:
+| Variable | Required | Default | Description |
+|---|---|---|---|
+| `TOLLGATE_API_KEY` | Yes | — | Your account API key (`tg_live_…`) |
+| `TOLLGATE_BASE_URL` | No | `https://tollgateai.vercel.app` | Self-hosted deployment URL |
+
+### Programmatic Configuration
 
 ```ts
 const tollgate = createTollgateClient({
@@ -82,7 +136,7 @@ const tollgate = createTollgateClient({
 
 ## Auto-Instrumentation
 
-Wrap your provider client once. Every `create` / `generateContent` call reports usage in the background — non-blocking, fire-and-forget. Failures go to `onError` (default: `console.warn`) and never break your LLM call.
+Wrap your provider client once. Every `create` / `generateContent` / `send` call reports usage in the background — non-blocking, fire-and-forget. Failures go to `onError` (default: `console.warn`) and never break your LLM call.
 
 ### Anthropic
 
@@ -137,7 +191,7 @@ const result = await model.generateContent('Explain quantum computing');
 
 ### OpenAI-Compatible Gateways
 
-Point the OpenAI SDK at any compatible endpoint and set `provider: 'openai_compatible'`:
+Works with any OpenAI-compatible endpoint — OpenRouter, Groq, Together, Nebius, Vercel AI Gateway, local vLLM, and more.
 
 ```ts
 import OpenAI from 'openai';
@@ -156,6 +210,8 @@ await groq.chat.completions.create({
 });
 ```
 
+> When a gateway returns cost inline (e.g. OpenRouter's `usage.cost`), the SDK captures it automatically as `providerCostCents`. The server uses it verbatim, bypassing the rate card. Gateways that don't return cost fall through to rate-card pricing. An explicit `providerCostCents` in the wrapper options always takes precedence.
+
 ### AWS Bedrock
 
 ```ts
@@ -194,40 +250,140 @@ for await (const chunk of stream) { /* render to UI */ }
 
 ---
 
-## What Gets Tracked
+## Tracked Fields
 
 Every auto-instrumented call captures these fields from the provider response:
 
 | Field | Providers | Description |
 |---|---|---|
-| `tokensIn` | All | Input tokens consumed |
-| `tokensOut` | All | Output tokens generated |
-| `reasoningTokens` | OpenAI, Anthropic, Gemini | Reasoning/thinking tokens (billed at reasoning rate) |
+| `tokensIn` | All | Input tokens (deduplicated — excludes cached/audio for OpenAI; excludes cached/audio/image/video for Gemini) |
+| `tokensOut` | All | Output tokens (deduplicated — excludes reasoning/audio for OpenAI; excludes audio/image for Gemini) |
+| `reasoningTokens` | OpenAI, Gemini | Reasoning/thinking tokens (billed at reasoning rate) |
 | `cachedTokens` | All | Prompt cache read tokens (reduced rate) |
-| `cacheWrite5mTokens` | Anthropic, Bedrock | 5-min TTL cache creation tokens |
-| `cacheWrite1hTokens` | Anthropic | 1-hour TTL cache creation tokens |
-| `audioTokensIn` | OpenAI | Audio input tokens (GPT-4o audio / Realtime) |
-| `audioTokensOut` | OpenAI, Gemini | Audio output tokens |
-| `imageTokensIn` | Gemini | Image/vision input tokens |
-| `imageTokensOut` | Gemini | Image generation output tokens |
+| `cacheWrite5mTokens` | Anthropic, Bedrock | Cache creation tokens (5-minute TTL) |
+| `cacheWrite1hTokens` | Bedrock | Cache creation tokens (1-hour TTL) |
+| `audioTokensIn` / `Out` | OpenAI, Gemini | Audio modality tokens (GPT-4o audio, Gemini multimodal) |
+| `imageTokensIn` / `Out` | Gemini | Image/vision input and generation output tokens |
 | `videoTokensIn` | Gemini | Video input tokens |
-| `textTokensIn` | OpenAI, Gemini | Text-only input tokens (modality split) |
-| `textTokensOut` | OpenAI, Gemini | Text-only output tokens |
+| `textTokensIn` / `Out` | OpenAI, Gemini | Text-only modality tokens |
 | `webSearchRequests` | Anthropic, Gemini | Web search requests (server tools / grounding) |
 | `acceptedPredictionTokens` | OpenAI | Predicted Outputs: accepted tokens |
-| `rejectedPredictionTokens` | OpenAI | Predicted Outputs: rejected tokens (waste) |
-| `serviceTier` | OpenAI | Service tier used (`default`, `flex`, `priority`) |
+| `rejectedPredictionTokens` | OpenAI | Predicted Outputs: rejected (waste) tokens |
+| `serviceTier` | OpenAI | Service tier (`default`, `flex`, `priority`) |
 | `latencyMs` | All | SDK-measured request duration in milliseconds |
 | `toolCalls` | All | Number of tool calls in the response |
+| `providerCostCents` | OpenAI-compatible | Gateway-reported cost — used verbatim, bypasses rate card |
 | `model` | All | Model identifier as reported by the provider |
 
-Cost is computed **server-side** from token counts and a rate card that auto-syncs daily from the LiteLLM registry (1,500+ models). Rate cards include per-token pricing for text, audio, image, video, cache, reasoning, and web search. Unknown models are priced at $0 and flagged in logs.
+Cost is computed **server-side** from token counts and a rate card that auto-syncs daily from the LiteLLM registry (1,500+ models). Rate cards include per-token pricing for every modality, cache tier, reasoning, and web search. Unknown models are priced at $0 and flagged in logs.
+
+---
+
+## Provider Field Coverage
+
+<details>
+<summary><strong>Anthropic</strong> — Messages API</summary>
+
+| Anthropic API Field | SDK Field | Notes |
+|---|---|---|
+| `usage.input_tokens` | `tokensIn` | Input tokens (excludes cached) |
+| `usage.output_tokens` | `tokensOut` | Output tokens (includes reasoning — billed at output rate) |
+| `usage.cache_read_input_tokens` | `cachedTokens` | Prompt cache read tokens |
+| `usage.cache_creation_input_tokens` | `cacheWrite5mTokens` | Prompt cache creation tokens |
+| `usage.server_tool_use.web_search_requests` | `webSearchRequests` | Web search server tool requests |
+| `response.content[]` (type `tool_use`) | `toolCalls` | Count of tool-use content blocks |
+| *(SDK-measured)* | `latencyMs` | Request duration |
+
+Anthropic bills reasoning tokens at the output rate. The SDK reports the full `output_tokens` count; the server-side rate card applies the matching output rate.
+
+In streaming mode, `message_start` carries input/cache counts and `message_delta` carries the output count. The SDK accumulates both automatically.
+
+</details>
+
+<details>
+<summary><strong>OpenAI</strong> — Chat Completions API</summary>
+
+| OpenAI API Field | SDK Field | Notes |
+|---|---|---|
+| `usage.prompt_tokens` | `tokensIn` | **Minus** cached and audio tokens to prevent double-billing |
+| `usage.completion_tokens` | `tokensOut` | **Minus** reasoning and audio tokens to prevent double-billing |
+| `usage.completion_tokens_details.reasoning_tokens` | `reasoningTokens` | Reasoning/thinking tokens |
+| `usage.prompt_tokens_details.cached_tokens` | `cachedTokens` | Prompt cache read tokens |
+| `usage.prompt_tokens_details.audio_tokens` | `audioTokensIn` | Audio input tokens |
+| `usage.completion_tokens_details.audio_tokens` | `audioTokensOut` | Audio output tokens |
+| `usage.prompt_tokens_details.text_tokens` | `textTokensIn` | Text modality input tokens |
+| `usage.completion_tokens_details.text_tokens` | `textTokensOut` | Text modality output tokens |
+| `usage.completion_tokens_details.accepted_prediction_tokens` | `acceptedPredictionTokens` | Predicted Outputs: accepted |
+| `usage.completion_tokens_details.rejected_prediction_tokens` | `rejectedPredictionTokens` | Predicted Outputs: rejected |
+| `service_tier` | `serviceTier` | Service tier used |
+| `choices[].message.tool_calls` | `toolCalls` | Tool call count |
+| *(SDK-measured)* | `latencyMs` | Request duration |
+
+OpenAI's `prompt_tokens` and `completion_tokens` are totals that include sub-category tokens. The SDK subtracts each sub-category so every token is costed at exactly one rate.
+
+</details>
+
+<details>
+<summary><strong>Google Gemini</strong> — Generative AI / Vertex AI</summary>
+
+| Google API Field | SDK Field | Notes |
+|---|---|---|
+| `usageMetadata.promptTokenCount` | `tokensIn` | **Minus** cached, audio, image, video to prevent double-billing |
+| `usageMetadata.candidatesTokenCount` | `tokensOut` | **Minus** audio and image output (thinking is already excluded by Google) |
+| `usageMetadata.thoughtsTokenCount` | `reasoningTokens` | Thinking/reasoning tokens (Gemini 2.x) |
+| `usageMetadata.cachedContentTokenCount` | `cachedTokens` | Prompt cache read tokens |
+| `promptTokensDetails[AUDIO]` | `audioTokensIn` | Audio input modality |
+| `candidatesTokensDetails[AUDIO]` | `audioTokensOut` | Audio output modality |
+| `promptTokensDetails[IMAGE]` | `imageTokensIn` | Image/vision input |
+| `candidatesTokensDetails[IMAGE]` | `imageTokensOut` | Image generation output |
+| `promptTokensDetails[VIDEO]` | `videoTokensIn` | Video input |
+| `promptTokensDetails[TEXT]` | `textTokensIn` | Text input |
+| `candidatesTokensDetails[TEXT]` | `textTokensOut` | Text output |
+| `candidates[].groundingMetadata.webSearchQueries` | `webSearchRequests` | Google Search grounding |
+| `candidates[].content.parts[].functionCall` | `toolCalls` | Function call count |
+| *(SDK-measured)* | `latencyMs` | Request duration |
+
+Google's `candidatesTokenCount` does **not** include `thoughtsTokenCount`, so reasoning tokens are not subtracted. However, it **does** include audio and image output tokens, so the SDK subtracts those to prevent double-billing.
+
+</details>
+
+<details>
+<summary><strong>AWS Bedrock</strong> — Converse API</summary>
+
+| Bedrock API Field | SDK Field | Notes |
+|---|---|---|
+| `usage.inputTokens` | `tokensIn` | Input tokens |
+| `usage.outputTokens` | `tokensOut` | Output tokens (includes reasoning — Bedrock does not split) |
+| `usage.cacheReadInputTokens` | `cachedTokens` | Prompt cache read tokens |
+| `usage.cacheDetails[ttl="5m"]` | `cacheWrite5mTokens` | Cache creation (5-minute TTL) |
+| `usage.cacheDetails[ttl="1h"]` | `cacheWrite1hTokens` | Cache creation (1-hour TTL, higher rate) |
+| `output.message.content[].toolUse` | `toolCalls` | Tool-use content block count |
+| *(SDK-measured)* | `latencyMs` | Request duration |
+
+Bedrock's `cacheDetails` array provides per-TTL breakdowns. The SDK splits these into `cacheWrite5mTokens` and `cacheWrite1hTokens`. When `cacheDetails` is absent, `cacheWriteInputTokens` falls back to the 5m bucket.
+
+In streaming mode (`ConverseStream`), the final `metadata` event carries usage totals. Tool calls are accumulated from `contentBlockStart` events.
+
+</details>
 
 ---
 
-## Outcome-Based Pricing
+## Pricing Models
 
-Under per-resolution pricing, only a **resolved** run earns revenue. An escalated or failed run earns $0 but its provider cost still counts.
+### Per-Call Revenue
+
+For simple per-call billing, pass `revenueUnitCents` in the wrapper options:
+
+```ts
+const anthropic = wrapAnthropic(new Anthropic(), tollgate, {
+  customerId: 'cust_acme',
+  revenueUnitCents: 50,  // $0.50 earned per LLM call
+});
+```
+
+### Outcome-Based Pricing
+
+Under per-resolution pricing, only a **resolved** run earns revenue. Escalated or failed runs earn $0, but provider costs still count against margin.
 
 ```ts
 const runId = 'ticket_8842';
@@ -246,24 +402,20 @@ await tollgate.resolve({
 });
 ```
 
-For simple per-call billing, pass `revenueUnitCents` in the wrap options and skip `resolve()`.
+### External Tool Costs
 
----
-
-## External Tool Costs
-
-Report costs from external services (image generation, code sandboxes, search APIs) alongside LLM calls:
+Report costs from non-LLM services (image generation, code sandboxes, search APIs) alongside LLM calls:
 
 ```ts
 await tollgate.track({
   customerId: 'cust_acme',
   runId: 'ticket_8842',
   provider: 'openai',
-  model: 'gpt-4o',
-  tokensIn: 500,
-  tokensOut: 200,
+  model: 'dall-e-3',
+  tokensIn: 0,
+  tokensOut: 0,
   externalCostCents: 4.0,     // $0.04 for the DALL-E call
-  idempotencyKey: 'ticket_8842#step_2',
+  idempotencyKey: 'ticket_8842#dalle',
 });
 ```
 
@@ -271,7 +423,7 @@ await tollgate.track({
 
 ## Customer & Plan Setup
 
-Create customers and assign plans before sending usage so plan-priced revenue is recognized from the first event. Idempotent.
+Create customers and assign plans before sending usage so plan-priced revenue is recognized from the first event. Idempotent — safe to call on every app boot.
 
 ```ts
 await tollgate.upsertCustomer({
@@ -287,6 +439,34 @@ await tollgate.upsertCustomer({
 
 ---
 
+## Error Handling
+
+The SDK separates **tracking errors** (non-fatal) from **client errors** (actionable):
+
+```ts
+// Tracking errors are swallowed by default (console.warn).
+// Override with onError to route to your observability stack:
+const anthropic = wrapAnthropic(new Anthropic(), tollgate, {
+  customerId: 'cust_acme',
+  onError: (err) => Sentry.captureException(err),
+});
+
+// Client errors (missing API key, invalid plan) throw TollgateError:
+import { TollgateError } from '@tollgateai/sdk';
+
+try {
+  await tollgate.upsertCustomer({ customerId: 'cust_acme' });
+} catch (err) {
+  if (err instanceof TollgateError) {
+    console.error(err.status, err.body);  // HTTP status + response body
+  }
+}
+```
+
+**Retry behavior:** The client retries on 5xx and 429 responses with exponential backoff (200ms, 400ms, ...). Deterministic 4xx errors (400, 401, 403, 404, 422) fail immediately.
+
+---
+
 ## API Reference
 
 ### Exports
@@ -294,7 +474,7 @@ await tollgate.upsertCustomer({
 ```ts
 // Client
 createTollgateClient(options?)   // -> TollgateClient
-TollgateError                    // Custom error with status & body
+TollgateError                    // Error with status & body
 
 // Auto-instrumentation wrappers
 wrapAnthropic(client, tollgate, options)       // -> instrumented Anthropic client
@@ -307,17 +487,33 @@ anthropicEventFrom(msg, options)               // -> TrackEventInput | null
 openAIEventFrom(completion, options)           // -> TrackEventInput | null
 bedrockEventFrom(usage, model, options)        // -> TrackEventInput | null
 geminiEventFrom(response, options)             // -> TrackEventInput | null
+
+// Types
+Provider         // 'anthropic' | 'openai' | 'openai_compatible' | 'bedrock' | 'google'
+RunOutcome       // 'resolved' | 'escalated' | 'failed'
+PricingModel     // 'per_unit' | 'per_resolution' | 'usage_based' | 'per_seat' | 'flat' | 'hybrid'
+TrackEventInput  // Full event payload type
 ```
 
-### TollgateClient
+### `TollgateClient`
 
 | Method | Description |
 |---|---|
-| `track(event)` | Report a single usage event. Idempotent on `idempotencyKey`. |
-| `resolve(input)` | Close a run with an outcome. Books revenue only when `outcome` is `'resolved'`. |
-| `upsertCustomer(input)` | Create or update a customer and optionally assign a plan. |
+| `track(event: TrackEventInput)` | Report a single usage event. Idempotent on `idempotencyKey`. Returns `{ status, eventId }`. |
+| `resolve(input: ResolveInput)` | Close a run with an outcome. Books revenue only when `outcome === 'resolved'`. |
+| `upsertCustomer(input: UpsertCustomerInput)` | Create or update a customer and optionally assign a plan. Returns `{ status, customerId, id, planId }`. |
+
+### `TollgateClientOptions`
+
+| Field | Type | Default | Description |
+|---|---|---|---|
+| `apiKey` | `string` | `TOLLGATE_API_KEY` env | Account API key |
+| `baseUrl` | `string` | `https://tollgateai.vercel.app` | Tollgate server URL |
+| `timeoutMs` | `number` | `10000` | Per-request timeout in milliseconds |
+| `maxRetries` | `number` | `2` | Retry attempts on 5xx / 429 / network errors |
+| `fetch` | `typeof fetch` | `globalThis.fetch` | Custom fetch implementation |
 
-### InstrumentOptions
+### `InstrumentOptions`
 
 | Field | Type | Required | Description |
 |---|---|---|---|
@@ -326,49 +522,53 @@ geminiEventFrom(response, options)             // -> TrackEventInput | null
 | `runId` | `string \| () => string` | No | Logical run ID (defaults to provider response ID) |
 | `provider` | `Provider` | No | Override the reported provider |
 | `revenueUnitCents` | `number \| (response) => number` | No | Revenue per call in cents |
-| `providerCostCents` | `number \| (response) => number` | No | Exact cost override (skips rate card) |
-| `onError` | `(err) => void` | No | Error handler for background tracking |
+| `providerCostCents` | `number \| (response) => number` | No | Exact cost override in cents (skips rate card) |
+| `onError` | `(err) => void` | No | Error handler for background tracking (default: `console.warn`) |
 
 ---
 
 ## How It Works
 
-1. **Proxy wrappers** intercept provider calls without modifying the request or response.
-2. After the provider responds, the wrapper extracts token counts (by modality), tool calls, service tier, and latency from the response.
-3. A `POST /api/track` fires **in the background** with automatic retries on transient failures.
-4. The server computes cost from tokens via rate cards (text, audio, image, video, cache, reasoning, web search), joins it with plan-configured revenue, and updates real-time margin rollups.
-5. Events are **idempotent** on `idempotencyKey` (auto-set to the provider response ID).
+1. **Proxy wrappers** intercept provider calls without modifying the request or response. Your code sees the exact same types and behavior as without the SDK.
+2. After the provider responds, the wrapper extracts token counts by modality, tool calls, service tier, and latency from the response object.
+3. A `POST /api/track` fires **in the background** with automatic retries on transient failures. Your application code continues immediately.
+4. The server computes cost from tokens via rate cards (per modality, cache tier, reasoning, and web search), joins it with plan-configured revenue, and updates real-time margin rollups.
+5. Events are **idempotent** — deduplication is based on `idempotencyKey` (auto-set to the provider response ID).
 
-## Privacy & Security
+## Security & Privacy
 
-- **No prompt content is ever sent.** Only token counts, model name, and metadata.
-- Events are deduplicated server-side.
-- Background tracking never throws into your application code.
+- **No prompt content is ever transmitted.** Only token counts, model identifiers, and metadata.
+- **Idempotent ingestion** — duplicate events are safely deduplicated server-side.
+- **Non-invasive** — background tracking never throws into your application code.
+- **Transport security** — all communication over HTTPS with Bearer token authentication.
 
 ---
 
-## What's New in v0.6.0
+## Changelog
+
+### v0.7.0
+
+- Google Gemini: fixed double-billing of multimodal input tokens — `tokensIn` now subtracts cached, audio, image, and video tokens
+- Aligned Anthropic extraction with actual API response fields
+- Simplified Anthropic streaming accumulation
+- Verified field coverage for all five providers
+
+### v0.6.0
 
-- **Fix: Anthropic thinking token extraction** — `output_tokens_details.thinking_tokens` is now extracted and costed at the reasoning rate instead of the output rate. Previously, thinking tokens from extended thinking (Sonnet 4.x, Opus 4.x) were invisible to cost computation.
-- **Fix: OpenAI double-counting** — `completion_tokens` includes reasoning and audio sub-totals; these are now subtracted from `tokensOut` so each token is costed at exactly one rate. Previously, reasoning tokens were billed at both the output rate and the reasoning rate.
-- **Fix: OpenAI input double-counting** — `prompt_tokens` includes cached and audio sub-totals; these are now subtracted from `tokensIn`. Previously, cached tokens were billed at both the full input rate and the cached rate.
-- **Fix: Multimodal-only events** — audio, image, video, and web search events now trigger rate-card lookup even when text token counts are zero.
-- `reasoningTokens` is now extracted from **all three** providers: OpenAI, Anthropic, and Gemini.
+- OpenAI: fixed double-counting of reasoning, audio, and cached tokens
+- Multimodal-only events now trigger rate-card lookup
+- Reasoning token extraction for OpenAI and Gemini
 
 ### v0.5.0
 
-- Google Gemini / Vertex AI support (`wrapGemini`) with full multimodal extraction
-- Audio token tracking (OpenAI GPT-4o audio / Realtime API)
-- Image & video token tracking (Gemini per-modality breakdowns)
-- Web search request tracking (Anthropic `server_tool_use`, Gemini grounding)
-- Latency measurement on all wrappers (SDK-measured `latencyMs`)
-- OpenAI Predicted Outputs (`acceptedPredictionTokens` / `rejectedPredictionTokens`)
-- Service tier tracking (OpenAI `flex` / `priority`, Anthropic `priority`)
-- Text modality split for accurate cost attribution in mixed-modal requests
-- Expanded rate card sync: audio, image, video, and web search rates from LiteLLM
+- Google Gemini / Vertex AI support with full multimodal extraction
+- Audio, image, video, and text modality token tracking
+- Web search request tracking (Anthropic, Gemini)
+- OpenAI Predicted Outputs and service tier tracking
+- Latency measurement on all wrappers
 
 ---
 
 ## License
 
-Licensed for use with Tollgate.
+MIT &mdash; see [LICENSE](LICENSE) for details.

package/dist/index.cjs

@@ -158,11 +158,7 @@ function anthropicEventFrom(msg, opts) {
   const usage = msg?.usage;
   if (!usage) return null;
   const runId = resolveRunId(opts, msg.id);
-  const fivem = usage.cache_creation?.ephemeral_5m_input_tokens;
-  const oneh = usage.cache_creation?.ephemeral_1h_input_tokens;
-  const hasSplit = fivem !== void 0 || oneh !== void 0;
   const toolCalls = Array.isArray(msg.content) ? msg.content.filter((b) => b.type === "tool_use").length : 0;
-  const thinkingTokens = usage.output_tokens_details?.thinking_tokens ?? 0;
   const event = {
     customerId: opts.customerId,
     agentId: opts.agentId,
@@ -170,11 +166,9 @@ function anthropicEventFrom(msg, opts) {
     provider: opts.provider ?? "anthropic",
     model: msg.model ?? "unknown",
     tokensIn: usage.input_tokens ?? 0,
-    tokensOut: (usage.output_tokens ?? 0) - thinkingTokens,
-    reasoningTokens: thinkingTokens,
+    tokensOut: usage.output_tokens ?? 0,
     cachedTokens: usage.cache_read_input_tokens ?? 0,
-    cacheWrite5mTokens: hasSplit ? fivem ?? 0 : usage.cache_creation_input_tokens ?? 0,
-    cacheWrite1hTokens: hasSplit ? oneh ?? 0 : 0,
+    cacheWrite5mTokens: usage.cache_creation_input_tokens ?? 0,
     webSearchRequests: usage.server_tool_use?.web_search_requests ?? 0,
     toolCalls,
     revenueUnitCents: resolveRevenue(opts, msg),
@@ -201,8 +195,7 @@ function wrapAnthropic(client, tollgate, opts) {
           } else if (ev.type === "message_delta" && ev.usage) {
             msg.usage = {
               ...msg.usage ?? {},
-              output_tokens: ev.usage.output_tokens,
-              output_tokens_details: ev.usage.output_tokens_details
+              output_tokens: ev.usage.output_tokens
             };
           } else if (ev.type === "content_block_start" && ev.content_block?.type === "tool_use") {
             toolUseBlocks.push(ev.content_block);
@@ -268,6 +261,10 @@ function openAIEventFrom(completion, opts) {
     revenueUnitCents: resolveRevenue(opts, completion),
     idempotencyKey: completion.id ?? `${runId}#${randomId()}`
   };
+  const gatewayCostUsd = usage.cost;
+  if (gatewayCostUsd != null && gatewayCostUsd > 0 && resolveCost(opts, completion) === void 0) {
+    event.providerCostCents = gatewayCostUsd * 100;
+  }
   return withCost(event, opts, completion);
 }
 function wrapOpenAI(client, tollgate, opts) {
@@ -333,6 +330,16 @@ function wrapOpenAI(client, tollgate, opts) {
 function bedrockEventFrom(usage, model, opts, response = void 0, toolCalls = 0) {
   if (!usage) return null;
   const runId = resolveRunId(opts, void 0);
+  let cacheWrite5m = 0;
+  let cacheWrite1h = 0;
+  if (usage.cacheDetails?.length) {
+    for (const d of usage.cacheDetails) {
+      if (d.ttl === "1h") cacheWrite1h += d.inputTokens ?? 0;
+      else cacheWrite5m += d.inputTokens ?? 0;
+    }
+  } else {
+    cacheWrite5m = usage.cacheWriteInputTokens ?? 0;
+  }
   const event = {
     customerId: opts.customerId,
     agentId: opts.agentId,
@@ -342,7 +349,8 @@ function bedrockEventFrom(usage, model, opts, response = void 0, toolCalls = 0)
     tokensIn: usage.inputTokens ?? 0,
     tokensOut: usage.outputTokens ?? 0,
     cachedTokens: usage.cacheReadInputTokens ?? 0,
-    cacheWrite5mTokens: usage.cacheWriteInputTokens ?? 0,
+    cacheWrite5mTokens: cacheWrite5m,
+    cacheWrite1hTokens: cacheWrite1h,
     toolCalls,
     revenueUnitCents: resolveRevenue(opts, response),
     idempotencyKey: `${runId}#${randomId()}`
@@ -409,21 +417,27 @@ function geminiEventFrom(response, opts) {
   }, 0);
   const promptDetails = usage.promptTokensDetails;
   const candidateDetails = usage.candidatesTokensDetails;
+  const cachedIn = usage.cachedContentTokenCount ?? 0;
+  const audioIn = modalityTokens(promptDetails, "AUDIO");
+  const imageIn = modalityTokens(promptDetails, "IMAGE");
+  const videoIn = modalityTokens(promptDetails, "VIDEO");
+  const audioOut = modalityTokens(candidateDetails, "AUDIO");
+  const imageOut = modalityTokens(candidateDetails, "IMAGE");
   const event = {
     customerId: opts.customerId,
     agentId: opts.agentId,
     runId,
     provider: opts.provider ?? "google",
     model: "unknown",
-    tokensIn: usage.promptTokenCount ?? 0,
-    tokensOut: usage.candidatesTokenCount ?? 0,
+    tokensIn: (usage.promptTokenCount ?? 0) - cachedIn - audioIn - imageIn - videoIn,
+    tokensOut: (usage.candidatesTokenCount ?? 0) - audioOut - imageOut,
     reasoningTokens: usage.thoughtsTokenCount ?? 0,
-    cachedTokens: usage.cachedContentTokenCount ?? 0,
-    audioTokensIn: modalityTokens(promptDetails, "AUDIO"),
-    audioTokensOut: modalityTokens(candidateDetails, "AUDIO"),
-    imageTokensIn: modalityTokens(promptDetails, "IMAGE"),
-    imageTokensOut: modalityTokens(candidateDetails, "IMAGE"),
-    videoTokensIn: modalityTokens(promptDetails, "VIDEO"),
+    cachedTokens: cachedIn,
+    audioTokensIn: audioIn,
+    audioTokensOut: audioOut,
+    imageTokensIn: imageIn,
+    imageTokensOut: imageOut,
+    videoTokensIn: videoIn,
     textTokensIn: modalityTokens(promptDetails, "TEXT"),
     textTokensOut: modalityTokens(candidateDetails, "TEXT"),
     webSearchRequests,