budget-agent 0.4.5 → 0.4.7

package/README.md

@@ -1,141 +1,74 @@
 # budget-agent
 
-Budget-aware enforcement layer for LLM agents. Track token, cost, and step usage in real time. Enforce limits before and after every LLM call. Works with any provider.
+Stop runaway LLM agents from burning your API credits. Set hard limits on cost, tokens, steps, and wall time. The SDK blocks each call before and after it hits your provider -- so you never overspend.
 
-```
+Works with **OpenAI**, **Anthropic**, **OpenRouter**, **Ollama**, **Together AI**, **Fireworks**, and any OpenAI-compatible API.
+
+## Install
+
+```bash
 npm install budget-agent
 ```
 
-## Quick start
-
-You bring your own API key and model. The SDK calls your provider.
+## Usage
 
 ```ts
-import { AgentBudget } from 'budget-agent';
+import { AgentBudget, BudgetError } from 'budget-agent';
 
 const agent = new AgentBudget({
   apiKey: process.env.OPENROUTER_API_KEY,
-  limits: { maxCostUSD: 0.05, maxSteps: 10 },
+  limits: {
+    maxCostUSD:   0.10,
+    maxSteps:     15,
+    maxTotalTokens: 50_000,
+    maxWallTimeMs: 30_000,
+  },
 });
 
 const response = await agent.step({
-  model: 'anthropic/claude-opus-4.8-fast',
+  model: 'anthropic/claude-sonnet-4-5',
   messages: [{ role: 'user', content: 'Hello' }],
 });
 
 console.log(agent.getUsage());
-// { steps: 1, totalCostUSD: 0.000015, totalInputTokens: 12, ... }
 ```
 
 ## How it works
 
-You provide the **model**, the **messages**, and your **API key**. The SDK:
-
-1. Checks budget before the call (pre-flight)
-2. Makes the API request to your provider
-3. Tracks tokens, cost, and duration
-4. Checks budget after the call (post-step)
-5. Emits events for streaming, warnings, and overages
-
-No provider is bundled. No model is defaulted. You bring everything.
-
-## Limits
-
-Budget guardrails that stop your agent before it spends too much:
+Every `step()` call runs two budget checks:
 
-```ts
-limits: {
-  maxCostUSD:     0.05,   // total USD before the agent aborts
-  maxSteps:       10,     // total LLM calls before abort
-  maxInputTokens: 50000,  // total input tokens sent to models
-  maxOutputTokens: 10000, // total output tokens received
-  maxTotalTokens:  60000, // input + output combined
-  maxWallTimeMs:   60000, // 60 seconds wall clock
-}
-```
-
-Every limit is optional. Omit what you don't want to enforce.
-
-### How enforcement works
-
-Each `step()` runs two checks:
-
-1. **Pre-flight** — before the API call. Estimates output cost (default 512 tokens) and catches over-budget calls before burning money.
-2. **Post-step** — after recording the real token/cost. If exceeded, the step is **rolled back** from the tracker so you can retry without a stale balance.
+1. **Before the API call** -- estimates cost and blocks if you'd go over budget.
+2. **After the API call** -- records actual tokens/cost and blocks if a limit was hit. The step rolls back so you can retry cleanly.
 
 ```ts
-const agent = new AgentBudget({
-  apiKey: key,
-  limits: { maxCostUSD: 0.01, maxSteps: 3 },
-});
-
 try {
   await agent.step({ model, messages });
 } catch (err) {
   if (err instanceof BudgetError) {
-    console.log(err.exceeded.reason); // 'cost' | 'steps' | 'wallTime' | ...
+    console.log(err.exceeded.reason);  // 'cost' | 'steps' | 'totalTokens' | 'wallTime'
+    console.log(err.exceeded.usage);   // full usage snapshot at cutoff
   }
 }
 ```
 
-### Custom callback instead of abort
-
-```ts
-const agent = new AgentBudget({
-  apiKey: key,
-  limits: { maxCostUSD: 0.01 },
-  onExceeded: (usage) => {
-    // Log, alert, switch models — never throws
-    console.log(`Over budget: $${usage.totalCostUSD}`);
-  },
-});
-```
+## Limits
 
-### Tune pre-flight estimation
+Every limit is optional. Set only what you need.
 
 ```ts
 limits: {
-  maxCostUSD: 0.05,
-  preflightCheck: false,              // skip pre-flight entirely
-  preflightOutputTokenEstimate: 2048, // safety buffer (default 512)
+  maxCostUSD:      0.05,   // total USD across all steps
+  maxSteps:        10,     // total LLM calls
+  maxInputTokens:  50000,  // input tokens only
+  maxOutputTokens: 10000,  // output tokens only
+  maxTotalTokens:  60000,  // input + output combined
+  maxWallTimeMs:   60000,  // wall clock time in ms
 }
 ```
 
-### Warning thresholds (non-blocking)
-
-```ts
-const agent = new AgentBudget({
-  limits: { maxCostUSD: 0.10 },
-  warningThreshold: 0.5, // fire 'budget:warning' at 50% consumption
-});
-
-agent.on('budget:warning', (e) => {
-  // { reason: 'cost', pctConsumed: 0.51, remaining: 0.049 }
-});
-```
-
-### Combine with adaptive routing
-
-```ts
-const agent = new AgentBudget({
-  apiKey: key,
-  limits: { maxCostUSD: 5.00 },
-  adaptiveRouting: {
-    fallbackChain: [
-      'anthropic/claude-opus-4.8-fast', // $15/M tokens — best model
-      'openai/gpt-4o',                  // $5/M tokens
-      'openrouter/free',                // $0 — emergency
-    ],
-    thresholds: [0.4, 0.75], // downgrade at 40% and 75% of budget consumed
-  },
-});
-```
-
-The router downgrades the model tier as the budget depletes. Each `step()` checks the current consumption against the thresholds and selects the appropriate model from the chain before the API call.
-
-## Bring your own executor
+## Custom executor (any provider)
 
-Use any LLM provider — OpenAI, Anthropic, Ollama, local models, or the OpenRouter Agent SDK:
+Use any LLM provider with a custom executor:
 
 ```ts
 import { AgentBudget } from 'budget-agent';
@@ -165,66 +98,25 @@ const agent = new AgentBudget({
     };
   },
 });
-
-const response = await agent.step({
-  model: 'anthropic/claude-opus-4.8-fast',
-  messages: [{ role: 'user', content: 'Hello' }],
-});
-```
-
-Or use raw fetch to any API:
-
-```ts
-const agent = new AgentBudget({
-  apiKey: 'none',
-  limits: { maxCostUSD: 0.05 },
-  executor: async (request) => {
-    const res = await fetch('http://localhost:11434/api/chat', {
-      method: 'POST',
-      body: JSON.stringify({ model: request.model, messages: request.messages }),
-    });
-    const data = await res.json();
-    return {
-      model: data.model,
-      usage: data.usage ?? { prompt_tokens: 0, completion_tokens: 0, total_tokens: 0 },
-      choices: data.messages?.map((m: any) => ({
-        message: { role: m.role, content: m.content },
-        finish_reason: 'stop',
-      })) ?? [],
-    };
-  },
-});
 ```
 
-## Built-in OpenRouter support
-
-By default, the SDK calls OpenRouter's API. Configure the endpoint and headers:
-
-```ts
-const agent = new AgentBudget({
-  apiKey: process.env.OPENROUTER_API_KEY,
-  baseUrl: 'https://openrouter.ai/api/v1',        // default — change for any OpenAI-compatible API
-  siteUrl: 'https://mysite.com',                   // OpenRouter attribution
-  appTitle: 'My App',                              // OpenRouter attribution
-  defaultHeaders: { 'X-Custom': 'value' },         // extra headers for every request
-  limits: { maxCostUSD: 0.10 },
-});
-```
-
-Works with any OpenAI-compatible endpoint: OpenRouter, OpenAI, Together AI, Fireworks, LocalAI, Ollama (with compat layer), etc.
+Works with OpenAI, Anthropic, Ollama, Together AI, Fireworks, LocalAI, or any OpenAI-compatible endpoint.
 
 ## Features
 
-- **Budget enforcement** — set limits on cost, tokens, steps, wall time. Checked pre-flight and post-step.
-- **Auto-compress** — truncate message history with an LLM summary when token count exceeds a threshold.
-- **Circuit breaker** — detect repetition or stagnation and halt the agent.
-- **Adaptive routing** — downgrade to cheaper models as budget depletes.
-- **Checkpoints** — save and resume agent state across restarts.
-- **Events** — subscribe to lifecycle events (`step:start`, `step:end`, `step:token`, `budget:exceeded`, etc.).
-- **Pricing cache** — model pricing fetched from OpenRouter with configurable TTL (or use `setModelPricing()` for any model).
-- **Rate-limit retry** — automatic 429 retry with exponential backoff (3 attempts).
-- **Streaming** — set `stream: true` and listen for `step:token` events.
-- **OpenTelemetry** — optional spans via `telemetry: { enabled: true }` (requires `@opentelemetry/api`).
+- **Cost limits** -- hard stop at a USD ceiling across all steps
+- **Token limits** -- cap input, output, or total tokens
+- **Step limits** -- max number of LLM calls
+- **Wall time limits** -- kill agents that run too long
+- **Pre-flight checks** -- estimate cost before spending money
+- **Rollback on exceed** -- step rolls back so retry stays clean
+- **Adaptive routing** -- auto-downgrade to cheaper models as budget depletes
+- **Circuit breaker** -- detect repetition or stagnation, halt the agent
+- **Auto-compress** -- truncate message history when tokens exceed threshold
+- **Checkpoints** -- save and resume agent state across restarts
+- **Streaming** -- set `stream: true`, listen for `step:token` events
+- **Rate-limit retry** -- automatic 429 retry with exponential backoff
+- **OpenTelemetry** -- optional tracing spans
 
 ## API
 
@@ -232,90 +124,56 @@ Works with any OpenAI-compatible endpoint: OpenRouter, OpenAI, Together AI, Fire
 
 | Option | Type | Default | Description |
 |--------|------|---------|-------------|
-| `apiKey` | `string` | — | Your provider API key |
-| `limits.*` | `object` | — | Budget limits (cost, tokens, steps, wall time) |
-| `executor` | `AgentExecutor` | — | Custom API executor (replaces built-in fetch) |
-| `baseUrl` | `string` | `https://openrouter.ai/api/v1` | API base URL for built-in fetch |
-| `defaultHeaders` | `object` | — | Extra HTTP headers for built-in fetch |
-| `autoCompress` | `object` | — | Auto-compress messages at token threshold |
-| `circuitBreaker` | `object` | — | Detect repetition/stagnation loops |
-| `adaptiveRouting` | `object` | — | Downgrade model tiers as budget depletes |
-| `checkpoint` | `object` | — | Persist and resume agent state |
-| `onExceeded` | `'abort' \| function` | `'abort'` | Strategy when budget exceeded |
-| `onEvent` | `function` | — | Global event listener |
-| `pricingCacheTTLMs` | `number` | `300_000` | Pricing cache TTL |
-| `siteUrl` | `string` | — | OpenRouter HTTP-Referer |
-| `appTitle` | `string` | — | OpenRouter X-OpenRouter-Title |
-| `telemetry` | `object` | — | Enable OpenTelemetry spans |
+| `apiKey` | `string` | required | Your provider API key |
+| `limits` | `object` | required | Budget limits (cost, tokens, steps, wall time) |
+| `executor` | `function` | -- | Custom API executor |
+| `baseUrl` | `string` | `https://openrouter.ai/api/v1` | API base URL |
+| `defaultHeaders` | `object` | -- | Extra HTTP headers |
+| `autoCompress` | `object` | -- | Auto-compress messages at token threshold |
+| `circuitBreaker` | `object` | -- | Detect repetition/stagnation |
+| `adaptiveRouting` | `object` | -- | Downgrade models as budget depletes |
+| `checkpoint` | `object` | -- | Persist and resume agent state |
+| `onExceeded` | `'abort' \| function` | `'abort'` | Strategy when limit hit |
+| `onEvent` | `function` | -- | Global event listener |
+| `warningThreshold` | `number` | `0.75` | Warning at this fraction of any limit |
+| `pricingCacheTTLMs` | `number` | `300000` | Pricing cache TTL |
+| `telemetry` | `object` | -- | Enable OpenTelemetry spans |
 
 ### `agent.step(request)`
 
-Make one LLM call. Checks limits before and after. Throws `BudgetError` if exceeded.
+One LLM call. Checks limits before and after. Throws `BudgetError` on exceed.
 
 ```ts
 const response = await agent.step({
-  model: 'anthropic/claude-opus-4.8-fast',            // any model slug
+  model: 'anthropic/claude-sonnet-4-5',
   messages: [{ role: 'user', content: 'Hi' }],
-  stream: true,                        // optional — emit step:token events
+  stream: true,
 });
 ```
 
-**Budget enforcement with rollback.** When a step exceeds budget, the step is recorded for circuit-breaker analysis, then rolled back before throwing. The tracker stays clean for retry. The actual spend is available in the `BudgetError`.
-
 ### `agent.getUsage()`
 
-Returns a snapshot of current usage:
-
-```ts
-{
-  steps: number;
-  totalInputTokens: number;
-  totalOutputTokens: number;
-  totalCostUSD: number;
-  elapsedMs: number;
-  stepHistory: StepUsage[];
-}
-```
-
-### `agent.summary()`
-
-Prints a formatted table to console and returns the same usage snapshot.
+Returns current usage: `steps`, `totalInputTokens`, `totalOutputTokens`, `totalCostUSD`, `elapsedMs`, `stepHistory`.
 
 ### `agent.reset()`
 
-Reset all usage counters.
-
-### `agent.compressMessages(messages, keepLastN?)`
-
-Manually compress a message array via LLM summary.
-
-### `agent.loadCheckpoint()` / `agent.clearCheckpoint()`
-
-Load or clear persisted checkpoint state.
+Reset all counters.
 
-### `AgentBudget.resume(options, checkpointPath?)`
+### `agent.refreshPricing()`
 
-Static factory. Creates a new agent pre-loaded with checkpoint state.
+Force re-fetch model prices from OpenRouter.
 
 ## Events
 
 ```ts
-agent.on('step:start', (event) => console.log('Step', event.stepIndex, 'started'));
-agent.on('step:token', (event) => process.stdout.write(event.token));
-agent.on('step:end', (event) => console.log('Step cost:', event.costUSD));
-agent.on('budget:exceeded', (event) => console.log('Limit hit:', event.exceeded.reason));
-agent.on('compress:triggered', (event) => console.log('Compressed:', event.messagesBefore, '→', event.messagesAfter));
-agent.on('model:downgraded', (event) => console.log('Downgraded to', event.to));
+agent.on('step:start', (e) => {});
+agent.on('step:token', (e) => {});
+agent.on('step:end', (e) => {});
+agent.on('budget:exceeded', (e) => {});
+agent.on('budget:warning', (e) => {});
+agent.on('model:downgraded', (e) => {});
 ```
 
-## Testing
-
-```
-npm test
-```
-
-Runs 10 real-API tests against OpenRouter with simulated pricing.
-
 ## License
 
 MIT

package/dist/index.js

@@ -519,9 +519,18 @@ export class AgentBudget {
         const response = request.stream === true
             ? await this._readStream(res, request.model, stepIndex, Date.now(), pricing)
             : (await res.json());
-        // OpenRouter may return HTTP 200 with an error inside choices[0].
-        // This happens when the provider rejects the request (insufficient
-        // credits, guardrail, provider outage, etc.).
+        // OpenRouter may return HTTP 200 with an error body.
+        // Check top-level error first (rate limit, auth, etc.).
+        const bodyAny = response;
+        if (bodyAny.error) {
+            const code = bodyAny.error.code ?? 500;
+            const msg = bodyAny.error.message ?? 'Unknown error';
+            if (code === 429) {
+                throw new RateLimitError(429, 0, msg);
+            }
+            throw new UpstreamError(code, msg);
+        }
+        // Also check choices[0].error (provider-level rejection).
         const choiceError = response.choices?.[0]?.error;
         if (choiceError) {
             throw new UpstreamError(choiceError.code, choiceError.message, choiceError.metadata);

package/package.json

@@ -1,7 +1,7 @@
 {
   "name": "budget-agent",
-  "version": "0.4.5",
-  "description": "Provider-agnostic budget enforcement SDK for LLM agents. Track token/cost/step usage, enforce limits, auto-compress, circuit-breaker, checkpoints, adaptive routing, and more.",
+  "version": "0.4.7",
+  "description": "Control LLM agent costs with real-time token, cost, and step tracking. Set budget limits, enforce spend caps, and prevent runaway agents. Works with OpenAI, Anthropic, OpenRouter, Ollama, and any provider.",
   "type": "module",
   "main": "./dist/index.js",
   "types": "./dist/index.d.ts",
@@ -18,19 +18,51 @@
   "engines": {
     "node": ">=18"
   },
+  "scripts": {
+    "build": "tsc",
+    "prepublishOnly": "npm run build",
+    "typecheck": "tsc --noEmit",
+    "test": "tsx test/run.ts",
+    "test:unit": "tsx test/run.ts --unit",
+    "test:integration": "tsx test/run.ts --integration",
+    "test:gauntlet": "tsx test-gauntlet.ts",
+    "test:legacy": "tsx test-integration.ts"
+  },
   "keywords": [
+    "llm",
     "agent",
     "budget",
-    "token",
-    "cost",
-    "llm",
+    "cost-control",
+    "token-limit",
+    "openrouter",
+    "openai",
+    "anthropic",
+    "llm-cost",
+    "agent-budget",
+    "spending-limit",
+    "rate-limit",
     "circuit-breaker",
     "checkpoint",
-    "rate-limit"
+    "token-tracker",
+    "cost-tracker",
+    "llm-agent",
+    "ai-agent",
+    "prompt-cost",
+    "usage-tracking",
+    "budget-enforcement",
+    "ollama",
+    "gpt-4",
+    "claude",
+    "llm-proxy"
   ],
   "license": "MIT",
   "repository": {
     "type": "git",
     "url": "https://github.com/duggal1/agent-budget.git"
+  },
+  "devDependencies": {
+    "dotenv": "^17.4.2",
+    "tsx": "^4.19.0",
+    "typescript": "^5.4.0"
   }
 }