@stigg/agent-sdk-js 0.3.1 → 0.3.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +73 -1367
- package/package.json +4 -3
package/README.md
CHANGED
|
@@ -1,1412 +1,118 @@
|
|
|
1
|
-
#
|
|
1
|
+
# Stigg Agent SDK
|
|
2
2
|
|
|
3
|
-
|
|
4
|
-
|
|
5
|
-
|
|
6
|
-
|
|
3
|
+
**Know what your agents spend — and stop them before they overspend.**
|
|
4
|
+
|
|
5
|
+
Check a customer's entitlements, credits or budget **before** a model call, meter what the call
|
|
6
|
+
actually used **after** it, and let [Stigg](https://stigg.io) handle the pricing, packaging and
|
|
7
|
+
billing behind it.
|
|
7
8
|
|
|
8
9
|
```bash
|
|
9
10
|
npm install @stigg/agent-sdk-js # or: yarn add @stigg/agent-sdk-js
|
|
10
11
|
```
|
|
11
12
|
|
|
12
13
|
Requires Node 22 or later. Ships CommonJS with type declarations, and a `./internal` subpath for
|
|
13
|
-
the engine seam.
|
|
14
|
-
|
|
15
|
-
Everything here runs **with no API keys and no Stigg account**: a built-in mock stands in for
|
|
16
|
-
both the LLM vendors and the Stigg API.
|
|
17
|
-
|
|
18
|
-
## What the SDK does
|
|
19
|
-
|
|
20
|
-
One core lifecycle, several bindings, one swappable engine.
|
|
21
|
-
|
|
22
|
-
```
|
|
23
|
-
construct → register? → wrap → [ check? → execute → ingest ] → deregister?
|
|
24
|
-
| | |
|
|
25
|
-
per env, optional: the boundary you declare; the client goes through
|
|
26
|
-
one SDK a write Stigg, and check and ingest happen inside it
|
|
27
|
-
about the
|
|
28
|
-
agent
|
|
29
|
-
```
|
|
30
|
-
|
|
31
|
-
**There is no run to open.** What governs a call — who it is for, what gates it, what it reports —
|
|
32
|
-
is a `stiggContext`, given once at `wrap` as the default for that client and again inside any
|
|
33
|
-
single call's own parameters to override it. One wrapped client therefore serves every customer
|
|
34
|
-
and every agent, which is what a multi-tenant server actually needs.
|
|
35
|
-
|
|
36
|
-
**No subscription required.** A customer's credit balance can be checked and drawn with no
|
|
37
|
-
subscription in place at all, so the shortest path from nothing to a working integration is a
|
|
38
|
-
credit currency and a `customerId` — no plan, no entitlement wiring, no catalog modelling first.
|
|
39
|
-
Model the pricing once the metering is real.
|
|
14
|
+
the engine seam. Twelve providers and gateways, **and not one of them is a dependency**.
|
|
40
15
|
|
|
41
|
-
|
|
42
|
-
request still goes out, and it is still billed.
|
|
16
|
+
## 🎯 Quick start
|
|
43
17
|
|
|
44
|
-
|
|
45
|
-
convenience for the next line, not a handle the rest depends on. Skip them and the agent has no
|
|
46
|
-
entity — calls still work, and checks and usage apply to the customer. That is deliberate:
|
|
47
|
-
**wrapping a client must never depend on governance being available**, and a governance-only
|
|
48
|
-
request (a budget, a parent) is the only thing that fails without it.
|
|
49
|
-
|
|
50
|
-
- **The agent is the entity; attribution is a dimension.** An entity upsert is idempotent and
|
|
51
|
-
cheap, but it is still a write — which is why provisioning is `register()` and never something a
|
|
52
|
-
gated call carries. Naming an `agentId` in a context attributes to that agent; it does not
|
|
53
|
-
create it.
|
|
54
|
-
- **A denial is a returned decision, not an exception** — it names which entity blocked and what
|
|
55
|
-
to do about it. Refusing is a commercial event (a top-up, an upgrade), not an incident. That is
|
|
56
|
-
`client.stigg.check()`. A **wrapped** call has nowhere to return one — the vendor's signature
|
|
57
|
-
owns the return type — so there it is raised as `AccessDeniedError`, and `stigg.denial(err)`
|
|
58
|
-
turns it back into the decision. A context that named no `entitlement` never gets there: it has
|
|
59
|
-
nothing to check against, so every call goes out and `check()` hands back an allow whose
|
|
60
|
-
`source` is `'unenforced'`.
|
|
61
|
-
- **Metering never breaks the call.** Ingest failures are swallowed into the `onError` hook;
|
|
62
|
-
enforcement failures are never swallowed. They are deliberately separate code paths.
|
|
18
|
+
Wrap the client you already have, and keep calling it the way you already do.
|
|
63
19
|
|
|
64
20
|
```ts
|
|
21
|
+
import Anthropic from '@anthropic-ai/sdk';
|
|
65
22
|
import { StiggAgentSDK } from '@stigg/agent-sdk-js';
|
|
66
23
|
|
|
67
|
-
|
|
68
|
-
// be left out entirely to read STIGG_SERVER_API_KEY, or replaced by a `baseUrl` that needs none -
|
|
69
|
-
// a local stand-in, or a self-hosted Stigg:
|
|
70
|
-
// new StiggAgentSDK({ apiKey, baseUrl: 'https://stigg.internal', ingest: 'sync' })
|
|
71
|
-
// A bare key is the shorthand for the common case.
|
|
72
|
-
const stigg = new StiggAgentSDK(apiKey);
|
|
73
|
-
|
|
74
|
-
// Provisioning the agent: a write, so it is its own call. A *bare* registration is remembered per
|
|
75
|
-
// process and skipped the second time; this one carries a parent and a budget, and a call that
|
|
76
|
-
// changes placement always writes - two differing instructions cannot be collapsed into one.
|
|
77
|
-
await stigg.register({
|
|
78
|
-
customerId: 'customer-acme',
|
|
79
|
-
agentId: 'agent-support-bot',
|
|
80
|
-
// The entity: what the agent IS. `register` is the one call that creates it, so it is the one
|
|
81
|
-
// call that needs the type.
|
|
82
|
-
entityTypeId: 'agent',
|
|
83
|
-
// The assignment: what it may SPEND, and where it sits. One object, because it is one write -
|
|
84
|
-
// over (entity x entitlement), which is why the entitlement is required inside it.
|
|
85
|
-
budget: {
|
|
86
|
-
entitlement: { currencyId: 'credits' },
|
|
87
|
-
parentId: 'team-eng',
|
|
88
|
-
limit: 5,
|
|
89
|
-
apply: 'on-create',
|
|
90
|
-
cadence: 'P1M',
|
|
91
|
-
},
|
|
92
|
-
});
|
|
93
|
-
|
|
94
|
-
// Governs the client and hands back a COPY whose calls are checked and metered. The client you
|
|
95
|
-
// passed keeps working ungoverned.
|
|
96
|
-
//
|
|
97
|
-
// Wrapped with no customer, this client serves every tenant the process handles, and each call
|
|
98
|
-
// names the one it is for. The types enforce that: `customerId` is **required** on every call
|
|
99
|
-
// through a client wrapped this way.
|
|
100
|
-
const govClient = stigg.wrap(new Anthropic(), {
|
|
101
|
-
stiggContext: {
|
|
102
|
-
// Naming an entitlement is the enforcement switch. Omit it and the calls meter without
|
|
103
|
-
// gating - every call still goes out, and nothing is ever refused.
|
|
104
|
-
entitlement: { currencyId: 'credits' },
|
|
105
|
-
// What the calls are attributed to. The attribution key is required with it - a guessed one
|
|
106
|
-
// resolves no entity, and the spend would land on the customer looking exactly like a call
|
|
107
|
-
// that was never governed. The parent needs no key: governance walks the chain up from here.
|
|
108
|
-
agentId: 'agent-support-bot',
|
|
109
|
-
attributionKey: 'agentId',
|
|
110
|
-
// Optional extra attribution, sent verbatim on every check and ingest.
|
|
111
|
-
// dimensions: { tenantId: 'acme-eu' },
|
|
112
|
-
},
|
|
113
|
-
});
|
|
114
|
-
|
|
115
|
-
// The call site is otherwise untouched; the context rides in the vendor's own parameter object
|
|
116
|
-
// and is lifted back out before the vendor sees it.
|
|
117
|
-
await govClient.messages.create({
|
|
118
|
-
model,
|
|
119
|
-
max_tokens: 1024,
|
|
120
|
-
messages,
|
|
121
|
-
stiggContext: { customerId: 'customer-acme' },
|
|
122
|
-
});
|
|
123
|
-
|
|
124
|
-
await govClient.messages.create({
|
|
125
|
-
model,
|
|
126
|
-
max_tokens: 1024,
|
|
127
|
-
messages,
|
|
128
|
-
stiggContext: { customerId: 'customer-globex' },
|
|
129
|
-
});
|
|
130
|
-
|
|
131
|
-
// Drains what is still in flight and reports what this client did.
|
|
132
|
-
await govClient.stigg.flush();
|
|
133
|
-
```
|
|
134
|
-
|
|
135
|
-
**Nothing in the setup path touches the network.** The constructor is synchronous and so is
|
|
136
|
-
`wrap` — resolving the engine is a probe that decides which transport serves which operation, and
|
|
137
|
-
both start that work rather than waiting on it. The first call that needs an engine awaits the
|
|
138
|
-
same probe; every call after it finds one already resolved. So there is no awaited setup step to
|
|
139
|
-
forget, and no call that silently runs ungoverned because setup had not finished.
|
|
140
|
-
|
|
141
|
-
The latency moves rather than disappearing: whoever makes the first call pays for the probe. If
|
|
142
|
-
you would rather pay at boot — and have a bad key fail there instead of inside a request —
|
|
143
|
-
`await stigg.init()` forces it early and hands back the resolution.
|
|
144
|
-
|
|
145
|
-
Construct one per environment and share it. A second one for the same environment is wasted
|
|
146
|
-
rather than wrong — and only partly a second client: `Stigg.initialize` keeps one global Node SDK
|
|
147
|
-
instance per process and hands it back when the API key matches, so the entitlement cache and its
|
|
148
|
-
socket are shared while the governance REST client is not. See
|
|
149
|
-
[what the cache promises](#what-the-cache-promises) for what that means for `close()`.
|
|
24
|
+
const stigg = new StiggAgentSDK(); // reads STIGG_SERVER_API_KEY
|
|
150
25
|
|
|
151
|
-
**A customer given at `wrap` is a default, not a scope.** One wrapped client can serve every
|
|
152
|
-
tenant, so naming a customer there means every call that does not override it bills *that* one —
|
|
153
|
-
correctly, silently, and forever if the client outlives the request. It is the right shape for a
|
|
154
|
-
single-tenant process and the wrong one for a server, which is why the example above leaves it
|
|
155
|
-
out and the types then require it per call.
|
|
156
|
-
|
|
157
|
-
```ts
|
|
158
|
-
// Single-tenant process: a default is fine, and calls need no context at all.
|
|
159
26
|
const client = stigg.wrap(new Anthropic(), {
|
|
160
27
|
stiggContext: { customerId: 'customer-acme', entitlement: { currencyId: 'credits' } },
|
|
161
28
|
});
|
|
29
|
+
|
|
30
|
+
// Checked before it runs, metered after it returns. Nothing else changes.
|
|
162
31
|
await client.messages.create({ model, max_tokens: 1024, messages });
|
|
163
32
|
```
|
|
164
33
|
|
|
165
|
-
`wrap()`
|
|
166
|
-
|
|
167
|
-
awaited: a failure there is logged, never thrown, and the first check falls back to asking the
|
|
168
|
-
server.
|
|
169
|
-
|
|
170
|
-
**`wrap` returns a copy.** The object you built is never mutated and keeps working ungoverned,
|
|
171
|
-
which is what makes the governed one safe to hand around. Wrapping an already-governed client
|
|
172
|
-
throws rather than stacking a second gate on it — there is nothing a second wrap could mean now
|
|
173
|
-
that the context travels with each call.
|
|
34
|
+
`wrap()` returns a **copy**. The client you built keeps working ungoverned, which is what makes the
|
|
35
|
+
governed one safe to hand around.
|
|
174
36
|
|
|
175
|
-
|
|
176
|
-
|
|
177
|
-
|
|
178
|
-
`wrap()` recognises the vendor from the client's own shape — there is nothing there the SDK
|
|
179
|
-
cannot see for itself — and throws, naming the vendors it knows, rather than handing back a
|
|
180
|
-
client that quietly meters nothing. Pass `recipe` to name an adapter for a vendor it cannot
|
|
181
|
-
identify.
|
|
182
|
-
|
|
183
|
-
**Gemini chats are wrapped in their own right.** `new GoogleGenAI()` builds its chat module over
|
|
184
|
-
the `models` module it holds — `new Chats(this.models, …)` — and every `Chat` keeps that
|
|
185
|
-
reference privately, so a chat created from a wrapped client still calls the original, unpatched
|
|
186
|
-
module. Wrapping the client governs `models.generateContent` and `models.generateContentStream`;
|
|
187
|
-
wrap the chat to govern a conversation:
|
|
37
|
+
When the customer is out of credits the call does not run — a refusal is a *commercial* event, so
|
|
38
|
+
offer a top-up rather than crashing:
|
|
188
39
|
|
|
189
40
|
```ts
|
|
190
|
-
|
|
191
|
-
|
|
192
|
-
|
|
193
|
-
|
|
194
|
-
|
|
195
|
-
|
|
196
|
-
|
|
197
|
-
|
|
198
|
-
tool and its SDK drives the tool loop for you — up to ten model calls for one
|
|
199
|
-
`generateContent` — and returns only the last response. Streamed, every round's chunks reach
|
|
200
|
-
this SDK and are summed; measured on the wire, 122 + 170 = 292 of 292. Not streamed, the earlier
|
|
201
|
-
rounds are discarded inside the vendor's SDK before it returns, so there is nothing left to read:
|
|
202
|
-
a two-round call billed at 315 reports 181. That turn is still metered for what it can see, is
|
|
203
|
-
tagged `gemini_afc_rounds`, and **raises** through `onError` with the shortfall named — it is
|
|
204
|
-
counted in `summary().ingestFailures` rather than left for you to notice. To meter every round,
|
|
205
|
-
stream the call, or set `automaticFunctionCalling: { disable: true }` and drive the loop
|
|
206
|
-
yourself, which makes each round a call this SDK gates and meters on its own.
|
|
207
|
-
|
|
208
|
-
Decisions describe themselves, so printing one takes no import: `decision.summary` is the log
|
|
209
|
-
line, `decision.message` is what to tell the user. `stigg.denial(err)` turns a caught error back
|
|
210
|
-
into a decision, or `undefined` when it was a real failure.
|
|
211
|
-
|
|
212
|
-
### What a call checks, and what it reports
|
|
213
|
-
|
|
214
|
-
They are different things, and keeping them apart is what lets Stigg do the pricing instead of
|
|
215
|
-
your application.
|
|
216
|
-
|
|
217
|
-
`entitlement` is what a call **checks** — a credit currency or a metered feature. Reporting goes
|
|
218
|
-
somewhere else entirely: to `usageMapper`, one report per metric, named and valued as the vendor
|
|
219
|
-
stated them. Stigg converts a credit-backed feature into the currency on its own side, which is
|
|
220
|
-
what the API was built for and why `usage.report` answers with a credit balance.
|
|
221
|
-
|
|
222
|
-
```
|
|
223
|
-
[governance] ingest input_tokens=12 cache_read_input_tokens=3 output_tokens=46 ... credits=0.94
|
|
224
|
-
```
|
|
225
|
-
|
|
226
|
-
`entitlement` is also **the enforcement switch**. Name one and every call through the wrapped
|
|
227
|
-
client is checked before it leaves. Leave it out and nothing is checked at all: no budget is
|
|
228
|
-
pre-fetched, no call can be refused, and the calls meter exactly as gated ones do. There is no
|
|
229
|
-
separate flag, because a context with nothing to check against cannot be enforced and a flag would
|
|
230
|
-
only be a second place for the two to disagree — so the client says which it is, once, at `wrap`:
|
|
231
|
-
|
|
232
|
-
```
|
|
233
|
-
[stigg] this client named no entitlement, so nothing is checked and no call can be refused - it
|
|
234
|
-
meters only. Pass `entitlement: { currencyId: 'credits' }` in `stiggContext`, at the wrap or on a
|
|
235
|
-
call, to gate it.
|
|
236
|
-
```
|
|
237
|
-
|
|
238
|
-
The default is the vendor's own vocabulary, so **each recipe is where you read what that vendor
|
|
239
|
-
meters** — `anthropicRecipe.metrics` is the list, typed as literals rather than as `string`.
|
|
240
|
-
|
|
241
|
-
#### The events this SDK reports, and the meters to put behind them
|
|
242
|
-
|
|
243
|
-
Reporting defaults to Stigg's event API (see the ingest table further down), and an event is
|
|
244
|
-
matched by **name**. A wrapped client names each call after the call — the name someone reading
|
|
245
|
-
that vendor's API would have guessed — so these are what your meters filter on:
|
|
246
|
-
|
|
247
|
-
| Binding | Event name | Fields a meter can aggregate |
|
|
248
|
-
| ------------------------- | ------------------------------------------------------------------- | ----------------------------------------------------------------------------------------- |
|
|
249
|
-
| Anthropic | `anthropic.messages.create` | `input_tokens`, `output_tokens`, `cache_read_input_tokens`, `cache_creation_input_tokens`, `total_tokens` |
|
|
250
|
-
| OpenAI (chat) | `openai.chat.completions.create` | `prompt_tokens`, `completion_tokens`, `cached_tokens`, `total_tokens` |
|
|
251
|
-
| OpenAI (responses) | `openai.responses.create` | `input_tokens`, `output_tokens`, `cached_tokens`, `total_tokens` |
|
|
252
|
-
| Groq | `groq.chat.completions.create` | the OpenAI chat set — one surface, because Groq serves no Responses endpoint |
|
|
253
|
-
| OpenRouter | `openrouter.chat.completions.create`, `openrouter.responses.create` | the OpenAI set, plus `cost` |
|
|
254
|
-
| OpenRouter — its own SDK | `openrouter.chat.send`, `openrouter.responses.send` | the same set — `@openrouter/sdk` camelCases its counters and the recipe reads them back under the wire's names, so one meter covers either client |
|
|
255
|
-
| LiteLLM | `litellm.chat.completions.create`, `litellm.responses.create` | the OpenAI set, plus `cost` |
|
|
256
|
-
| Mistral | `mistral.chat.complete`, `mistral.chat.stream`, `mistral.chat.parse`, `mistral.chat.parseStream`, `mistral.fim.complete`, `mistral.fim.stream`, `mistral.agents.complete`, `mistral.agents.stream` | `prompt_tokens`, `completion_tokens`, `total_tokens` — the **wire's** names, not the camelCase its SDK renames them to |
|
|
257
|
-
| Ollama | `ollama.chat`, `ollama.generate` | `prompt_eval_count`, `eval_count`, `total_tokens` — self-hosted, so these measure tokens you served yourself rather than tokens anyone billed you for |
|
|
258
|
-
| Bedrock | `bedrock.send` | `inputTokens`, `outputTokens`, `cacheReadInputTokens`, `cacheWriteInputTokens`, `totalTokens` — and no `cost`, because Bedrock states none |
|
|
259
|
-
| Databricks | `databricks.chat.completions.create`, `databricks.responses.create`, `databricks.query` | `prompt_tokens`, `completion_tokens`, `total_tokens`, `reasoning_tokens`, `cache_read_input_tokens`, `cache_creation_input_tokens` — and no `cost`, because Databricks states none on any response. `databricks.query` is the first-party client and reports the first three only; it camelCases the wire and is renamed back, so both clients report one vocabulary |
|
|
260
|
-
| Databricks — external models | `databricks-external.chat.completions.create`, `databricks-external.responses.create`, `databricks-external.query` | the same six. A separate vendor name because a different party is invoiced, and it is **opt-in**: the two routes are indistinguishable from the client, so naming it asserts something the SDK cannot check |
|
|
261
|
-
| Cloudflare | `cloudflare.chat.completions.create`, `cloudflare.responses.create`, `cloudflare.messages.create`, `cloudflare.ai.run` | the OpenAI set plus `cache_read_input_tokens`, `cache_creation_input_tokens` — the union across every wire it serves, and a call reports only the names its own route sent. No `cost` and no neuron count: neither Workers AI nor AI Gateway states a price on any response, so `cf_aig_log_id` rides as a dimension instead. A gateway **cache hit meters nothing**, because nobody was billed for it |
|
|
262
|
-
| TypeSafe | `typesafe.systemOne` | `input_tokens`, `output_tokens`, `total_tokens` |
|
|
263
|
-
| Gemini | `gemini.models.generateContent`, `gemini.models.generateContentStream` | `promptTokenCount`, `candidatesTokenCount`, `cachedContentTokenCount`, `thoughtsTokenCount`, `toolUsePromptTokenCount`, `totalTokenCount` |
|
|
264
|
-
| Gemini — a chat | `gemini.chats.sendMessage`, `gemini.chats.sendMessageStream` | the same five |
|
|
265
|
-
| A gateway, out of process | `<vendor>.proxied` | whatever the proxy declared its vendor to be |
|
|
266
|
-
|
|
267
|
-
Every metric rides the event under its own name, so **one event feeds as many features as you
|
|
268
|
-
configure meters for** — that is why a turn costs one request however many counters it reported.
|
|
269
|
-
The vendor's own usage block rides along too, under a `raw.` prefix and flattened to scalars, so a
|
|
270
|
-
meter can read a counter this SDK does not name yet: `raw.cache_creation_input_tokens`, or a
|
|
271
|
-
gateway's `raw.cost` before it was scaled into whole units. Your run's `dimensions` — `agentId`
|
|
272
|
-
and anything else you set — are on the same event, and win any key the vendor also used.
|
|
273
|
-
|
|
274
|
-
A meter to put behind one of them, in Stigg's own terms:
|
|
275
|
-
|
|
276
|
-
```jsonc
|
|
277
|
-
{
|
|
278
|
-
"aggregation": { "function": "SUM", "field": "output_tokens" },
|
|
279
|
-
"filters": [
|
|
280
|
-
{ "conditions": [{ "field": "eventName", "operation": "EQUALS", "value": "anthropic.messages.create" }] },
|
|
281
|
-
],
|
|
41
|
+
import { isAccessDenied } from '@stigg/agent-sdk-js';
|
|
42
|
+
|
|
43
|
+
try {
|
|
44
|
+
await client.messages.create({ model, max_tokens: 1024, messages });
|
|
45
|
+
} catch (error) {
|
|
46
|
+
if (isAccessDenied(error)) {
|
|
47
|
+
// Out of budget. Offer the upgrade; do not treat this as an outage.
|
|
48
|
+
}
|
|
282
49
|
}
|
|
283
50
|
```
|
|
284
51
|
|
|
285
|
-
|
|
286
|
-
|
|
287
|
-
|
|
288
|
-
|
|
289
|
-
> **None of this applies to a metric named after a credit currency.** That draws the wallet
|
|
290
|
-
> directly and needs no event name and no meter — see [the three ingest paths](#the-engine-sdk-first-rest-to-complete)
|
|
291
|
-
> below. If you are using `unitsPerTokens(currencyId, …)`, that is the path you are on, and the
|
|
292
|
-
> table above is about the other one.
|
|
293
|
-
|
|
294
|
-
#### Pricing them yourself
|
|
295
|
-
|
|
296
|
-
`weightedMetrics` turns a price-per-metric table into the mapper:
|
|
297
|
-
|
|
298
|
-
```ts
|
|
299
|
-
import { anthropicRecipe, weightedMetrics } from '@stigg/agent-sdk-js';
|
|
300
|
-
|
|
301
|
-
const govClient = stigg.wrap(new Anthropic(), {
|
|
302
|
-
stiggContext: {
|
|
303
|
-
customerId,
|
|
304
|
-
entitlement: { currencyId: 'data_credit' },
|
|
305
|
-
// A whole credit per input token, half a credit per output token, cache reads free.
|
|
306
|
-
usageMapper: weightedMetrics(anthropicRecipe, 'data_credit', {
|
|
307
|
-
input_tokens: 1,
|
|
308
|
-
output_tokens: 0.5,
|
|
309
|
-
}),
|
|
310
|
-
},
|
|
311
|
-
});
|
|
312
|
-
```
|
|
313
|
-
|
|
314
|
-
The table is keyed to that recipe's own names, so a typo does not compile. Naming a metric is
|
|
315
|
-
optional — an unpriced one is free, and a table that prices nothing meters nothing.
|
|
316
|
-
|
|
317
|
-
A weight is the price of **one unit** of that metric, so the example above is half a credit per
|
|
318
|
-
output token. Pass `{ per: 1000 }` when the numbers read better per thousand.
|
|
319
|
-
|
|
320
|
-
It also handles the two things a hand-written mapper has to get right: a turn that measured
|
|
321
|
-
nothing reports nothing rather than a zero, and a counter the response omitted is absent rather
|
|
322
|
-
than zero. For one number without weights, `unitsPerTokens(id, n)` is the short form.
|
|
323
|
-
|
|
324
|
-
Neither rounds. A credit currency takes a fractional amount, so a turn worth a fifth of a unit
|
|
325
|
-
costs a fifth of one — rounding it up would be this SDK's arithmetic on your price. **A feature is
|
|
326
|
-
the exception**: `POST /api/v1/usage` answers `400 Expected integer, received float`, so when the
|
|
327
|
-
target is a feature rather than a currency, pick a unit size that divides or round in your own
|
|
328
|
-
mapper.
|
|
329
|
-
|
|
330
|
-
> **An attributed run is the other exception, and the SDK rounds it up for you.** Per-entity usage
|
|
331
|
-
> is counted in whole units — governance validates its ingest as an integer and increments a Redis
|
|
332
|
-
> counter with `HINCRBY` — so a fractional amount is rejected there *while the wallet still moves*.
|
|
333
|
-
> The agent's budget would stop advancing with nothing in the response to say so.
|
|
334
|
-
>
|
|
335
|
-
> So a run naming an `agentId` rounds its wallet draw up, and logs that it did. A run with no
|
|
336
|
-
> entity attribution reports the fraction as measured. Denominate the currency finely enough that
|
|
337
|
-
> one unit is small, and the overcharge is noise; this goes away when governance counts fractions.
|
|
52
|
+
**No subscription required.** A credit balance can be checked and drawn with no plan in place, so
|
|
53
|
+
the shortest path from nothing to working metering is a currency and a `customerId` — no catalog
|
|
54
|
+
modelling first.
|
|
338
55
|
|
|
56
|
+
## 🧭 How it works
|
|
339
57
|
|
|
340
|
-
|
|
58
|
+
Three things happen around a call you already make:
|
|
341
59
|
|
|
342
|
-
|
|
343
|
-
|
|
344
|
-
|
|
60
|
+
| | | |
|
|
61
|
+
| --- | --- | --- |
|
|
62
|
+
| **1. Check** | before the call | synchronous, and a refusal is commercial rather than exceptional |
|
|
63
|
+
| **2. Execute** | your call, unchanged | the vendor's own client, own types, own streaming |
|
|
64
|
+
| **3. Ingest** | after the call | asynchronous and post-hoc; metering must never break a run |
|
|
345
65
|
|
|
346
|
-
|
|
347
|
-
|
|
348
|
-
|
|
349
|
-
const dataCredits = ({ metrics = {} }: { metrics?: AnthropicMetrics }): Metrics => {
|
|
350
|
-
const billable = (metrics.output_tokens ?? 0) * 5 + (metrics.input_tokens ?? 0);
|
|
351
|
-
return billable > 0 ? { data_credit: Math.ceil(billable / 100) } : {};
|
|
352
|
-
};
|
|
353
|
-
```
|
|
354
|
-
|
|
355
|
-
Every counter is optional, because a vendor reports cache reads only when there were some. Use
|
|
356
|
-
`AnthropicMetric` for one name rather than the table, and `MetricsOf<typeof someRecipe>` /
|
|
357
|
-
`MetricsFor<typeof someRecipe>` for a recipe you built yourself.
|
|
358
|
-
|
|
359
|
-
#### The overlap you have to know about
|
|
66
|
+
Metering and enforcement are deliberately not one component. One fails open, the other fails
|
|
67
|
+
closed, and conflating them is how an outage takes an application down.
|
|
360
68
|
|
|
361
|
-
|
|
362
|
-
metrics are reported **verbatim** — because a number published under a vendor's own field name
|
|
363
|
-
has to match that vendor's own dashboard.
|
|
69
|
+
## 🔌 What it governs
|
|
364
70
|
|
|
365
|
-
|
|
366
|
-
the vendor publishes one it is reported verbatim — `total_tokens`, or `totalTokenCount` for
|
|
367
|
-
Gemini. Anthropic and TypeSafe publish none, so theirs is summed from counters that are disjoint,
|
|
368
|
-
which is the only case where the SDK adds anything up. **Price the total or the breakdown, never
|
|
369
|
-
both.**
|
|
71
|
+
**Providers** — token counts, cache counters, streaming usage:
|
|
370
72
|
|
|
371
|
-
|
|
|
372
|
-
|
|
|
373
|
-
|
|
|
374
|
-
|
|
|
375
|
-
|
|
|
376
|
-
|
|
|
377
|
-
| Gemini | `promptTokenCount` | **Yes**, includes `cachedContentTokenCount` |
|
|
378
|
-
| Databricks | `prompt_tokens` | **Unverified** — Databricks documents neither direction. It spells the pair with Anthropic's names (`cache_read_input_tokens`), whose counters sit beside the input count, so this SDK does not subtract them. Its stated `total_tokens` is read rather than computed, so the total is right either way — price the total, or verify against your own endpoint before pricing the breakdown |
|
|
73
|
+
| | Provider | | Provider |
|
|
74
|
+
| --- | --- | --- | --- |
|
|
75
|
+
| <img src="https://raw.githubusercontent.com/stiggio/stigg-agent-sdk-ts/main/docs/logos/anthropic.svg" width="18" alt=""> | Anthropic | <img src="https://raw.githubusercontent.com/stiggio/stigg-agent-sdk-ts/main/docs/logos/mistral.svg" width="18" alt=""> | Mistral |
|
|
76
|
+
| <img src="https://raw.githubusercontent.com/stiggio/stigg-agent-sdk-ts/main/docs/logos/openai.svg" width="18" alt=""> | OpenAI | <img src="https://raw.githubusercontent.com/stiggio/stigg-agent-sdk-ts/main/docs/logos/ollama.svg" width="18" alt=""> | Ollama |
|
|
77
|
+
| <img src="https://raw.githubusercontent.com/stiggio/stigg-agent-sdk-ts/main/docs/logos/gemini.svg" width="18" alt=""> | Gemini | <img src="https://raw.githubusercontent.com/stiggio/stigg-agent-sdk-ts/main/docs/logos/typesafe.png" width="18" alt=""> | TypeSafe |
|
|
78
|
+
| <img src="https://raw.githubusercontent.com/stiggio/stigg-agent-sdk-ts/main/docs/logos/groq.svg" width="18" alt=""> | Groq | | |
|
|
379
79
|
|
|
380
|
-
|
|
381
|
-
`promptTokenCount` **and** `cachedContentTokenCount` double-bills the cached tokens, while
|
|
382
|
-
`thoughtsTokenCount` and `toolUsePromptTokenCount` sit *outside* the two headline counters —
|
|
383
|
-
leave them out of a table and a reasoning model's thinking tokens, usually the expensive half of
|
|
384
|
-
the answer, are billed at nothing. The four add back up to the vendor's own `totalTokenCount`,
|
|
385
|
-
which is the arithmetic to check a table against.
|
|
386
|
-
|
|
387
|
-
So if you price `prompt_tokens` **and** `cached_tokens` for OpenAI traffic, you bill the cached
|
|
388
|
-
ones twice. Price one or the other, or use `normalizedMetrics` — four disjoint counters under
|
|
389
|
-
SDK-owned names, comparable across every vendor:
|
|
390
|
-
|
|
391
|
-
```ts
|
|
392
|
-
import { normalizedMetrics } from '@stigg/agent-sdk-js';
|
|
393
|
-
stigg.wrap(client, { stiggContext: { customerId, usageMapper: normalizedMetrics } });
|
|
394
|
-
```
|
|
80
|
+
**Gateways** — what was actually *charged*, and which model actually served after routing:
|
|
395
81
|
|
|
396
|
-
|
|
397
|
-
|
|
398
|
-
|
|
399
|
-
|
|
400
|
-
|
|
401
|
-
A feature only drains a credit budget when your catalog maps it to that currency. Until it does,
|
|
402
|
-
every call succeeds, every ingest returns `ok`, and the budget never moves — so the call gates on a
|
|
403
|
-
number nothing spends. Stigg answers those reports with no credit balance, so the first one says
|
|
404
|
-
so:
|
|
405
|
-
|
|
406
|
-
```
|
|
407
|
-
[stigg] reported input_tokens but the response carried no credit balance - this feature is not
|
|
408
|
-
mapped to credits, so the budget this run checks against will never drain. Map it in Stigg, or
|
|
409
|
-
report to the currency directly.
|
|
410
|
-
```
|
|
411
|
-
|
|
412
|
-
### Bindings, by solution
|
|
413
|
-
|
|
414
|
-
Organised by where the customer's business logic sits, not by vendor. Each has a generic core;
|
|
415
|
-
vendor directories are thin recipes over it.
|
|
416
|
-
|
|
417
|
-
The word "gateway" below does two jobs, so the directories do not: one you **call** is an ordinary
|
|
418
|
-
client and lives in `bindings/client/gateways/`, beside `bindings/client/providers/` because both
|
|
419
|
-
are recipes over the same wrapper. One you **operate** is the opposite binding and lives in
|
|
420
|
-
`bindings/proxy/`.
|
|
421
|
-
|
|
422
|
-
| Binding | Reached as | Mechanism | Can refuse? |
|
|
423
|
-
| ------------------------------------------------ | ---------------------------------------------------- | ---------------------------------------------------------------------------------------- | ------------------------------------------ |
|
|
424
|
-
| Client | `stigg.wrap(client, { stiggContext })` | gates the recipe's methods on a copy, vendor matched by shape | Yes — when the context names an entitlement |
|
|
425
|
-
| Client — a gateway you **call** | `wrap(client, { recipe: openrouterRecipe() })` | same wrapper; the recipe bills `usage.cost` and attributes `response.model` | Yes, same as any client |
|
|
426
|
-
| Client — a gateway whose cost is in a **header** | `wrap(client, { recipe: litellmRecipe() })` | same wrapper; the recipe bills `x-litellm-response-cost` and attributes `response.model` | Yes, same as any client |
|
|
427
|
-
| Client — a gateway that states **no** price | `wrap(client, { recipe: databricksRecipe() })` | same wrapper; no `cost` metric at all, and the model that answered is attributed instead | Yes, same as any client |
|
|
428
|
-
| Gateway you **operate** (`bindings/proxy`) | `client.stigg.headers()` → `stigg.ingestFromHeaders(...)` | attribution travels as `x-stigg-*` headers; `vendor` names the shape it observed | **No — ingest only** |
|
|
429
|
-
|
|
430
|
-
A client run that names no `entitlement` lands in the same column as the gateway — metering
|
|
431
|
-
without a gate — except that it is in-process, and naming one turns it back on.
|
|
432
|
-
|
|
433
|
-
#### Providers that speak the OpenAI wire format
|
|
434
|
-
|
|
435
|
-
A large part of the ecosystem exposes an OpenAI-compatible API rather than one of its own:
|
|
436
|
-
OpenRouter, LiteLLM, Groq, Together, Fireworks, DeepSeek, xAI and Perplexity, plus self-hosted
|
|
437
|
-
servers like vLLM and Ollama. There is no separate client to install for any of them. You point
|
|
438
|
-
the OpenAI client at their base URL, and `wrap` matches on the **shape of the client**, not on who
|
|
439
|
-
is at the other end of it:
|
|
440
|
-
|
|
441
|
-
```ts
|
|
442
|
-
// Groq, Together, Ollama, vLLM - anything OpenAI-compatible
|
|
443
|
-
const client = stigg.wrap(
|
|
444
|
-
new OpenAI({ baseURL: 'https://api.groq.com/openai/v1', apiKey: process.env.GROQ_API_KEY }),
|
|
445
|
-
{ stiggContext: { customerId, entitlement: { currencyId: 'credits' } } },
|
|
446
|
-
);
|
|
447
|
-
```
|
|
448
|
-
|
|
449
|
-
So `OpenAI` in that line names the **wire format**, not the vendor being billed. What differs is
|
|
450
|
-
which recipe matches, and that decides what you can meter:
|
|
451
|
-
|
|
452
|
-
| Pointed at | Recipe matched | Events it emits | Reports |
|
|
453
|
-
| ------------------------------------ | ------------------ | --------------------------------- | --------------------------------- |
|
|
454
|
-
| `https://openrouter.ai/api/v1` | `openrouterRecipe` | `openrouter.*` (see below) | tokens, `cost`, model that served |
|
|
455
|
-
| the LiteLLM proxy URL you configured | `litellmRecipe` | `litellm.*` | tokens, `cost`, model that served |
|
|
456
|
-
| `https://api.groq.com/openai/v1` | `groqRecipe` | `groq.*` | tokens only |
|
|
457
|
-
| a Databricks workspace `/serving-endpoints` | `databricksRecipe` | `databricks.*` | tokens, model that answered |
|
|
458
|
-
| `https://api.cloudflare.com/client/v4/accounts/<id>/ai/v1` | `cloudflareRecipe` | `cloudflare.*` | tokens only |
|
|
459
|
-
| `https://gateway.ai.cloudflare.com/v1/<account>/<gateway>/<provider>` | `cloudflareRecipe` | `cloudflare.*` | tokens, model that served, log id |
|
|
460
|
-
| anything else OpenAI-compatible | `openAIRecipe` | `openai.*` | tokens only |
|
|
461
|
-
|
|
462
|
-
Each of those covers **both OpenAI surfaces**, and a surface is its own event: Chat Completions
|
|
463
|
-
reports `<vendor>.chat.completions.create` and the Responses API reports
|
|
464
|
-
`<vendor>.responses.create`. A meter filtering on one sees nothing from the other, so configure the
|
|
465
|
-
one your code actually calls - or both. The full list is in the event table earlier in this file.
|
|
466
|
-
|
|
467
|
-
OpenRouter and Groq are recognised by their **hostname** and LiteLLM by the base URL you hand its
|
|
468
|
-
recipe. For the two gateways that is what earns them a `cost` the token counters cannot recover;
|
|
469
|
-
for Groq, which publishes no cost, it earns a vendor name of its own. Everything else is OpenAI as
|
|
470
|
-
far as this SDK can tell, because that is all the client reveals.
|
|
471
|
-
|
|
472
|
-
> **One exception to that last row.** Given no base URL of its own, the LiteLLM recipe falls back to
|
|
473
|
-
> looking for `litellm` anywhere in the client's URL - a self-hosted proxy has no canonical hostname
|
|
474
|
-
> to compare against, the way OpenRouter's own hostname does. So an unrelated OpenAI-compatible
|
|
475
|
-
> endpoint that happens to contain the string, like `https://api.acme.com/litellm-compat/v1`, is
|
|
476
|
-
> claimed by it and reports under `litellm.*`. Pass `recipe: openAIRecipe` to settle it explicitly.
|
|
477
|
-
|
|
478
|
-
**The consequence worth planning for:** every provider in that last row reports under `openai.*`.
|
|
479
|
-
A raw-events meter cannot tell Together from Fireworks from OpenAI itself, and neither can a
|
|
480
|
-
per-provider budget. If you need them separated, give the call a dimension of your own —
|
|
481
|
-
|
|
482
|
-
```ts
|
|
483
|
-
stiggContext: { customerId, dimensions: { provider: 'groq' } }
|
|
484
|
-
```
|
|
485
|
-
|
|
486
|
-
— and filter the meter on it, or pass a recipe with its own `vendor` name, which changes the event
|
|
487
|
-
name to match.
|
|
488
|
-
|
|
489
|
-
> **Groq has its own recipe, and its own vendor name.** Groq is reached either through its own
|
|
490
|
-
> `groq-sdk`, which defaults to `https://api.groq.com`, or through an `OpenAI` client pointed at
|
|
491
|
-
> `https://api.groq.com/openai/v1`. Same hostname, so `groqRecipe` claims **both** and both report
|
|
492
|
-
> `groq.chat.completions.create`. One surface only: Groq serves no Responses endpoint, so there is
|
|
493
|
-
> no Groq event for that API and there never will be.
|
|
494
|
-
>
|
|
495
|
-
> On both routes its streamed chunks carry no top-level `usage` — the counters ride `x_groq.usage`
|
|
496
|
-
> on the final chunk — and that read is shared with `openAIRecipe` rather than copied, so a Groq
|
|
497
|
-
> deployment on some *other* base URL (`GROQ_BASE_URL` pointed at a proxy or a regional endpoint)
|
|
498
|
-
> is still metered correctly even though the hostname guard declines it. It reports `openai.*` in
|
|
499
|
-
> that case; name `recipe: groqRecipe` at `wrap` to attribute it to Groq explicitly.
|
|
500
|
-
>
|
|
501
|
-
> Note the same field means something else on a **buffered** response — `x_groq.usage` is hardware
|
|
502
|
-
> cache statistics (`dram_cached_tokens`, `sram_cached_tokens`) there, not tokens — so only the
|
|
503
|
-
> streamed reader consults it, and only when it names a token counter.
|
|
504
|
-
>
|
|
505
|
-
> Groq also reports a per-model `usage_breakdown` for its compound systems, where one request runs
|
|
506
|
-
> several models. That is **not** metered: a measurement carries one set of metrics and one
|
|
507
|
-
> `dimensions` object, so there is nowhere to put two models' counts without attributing both to
|
|
508
|
-
> one of them. The request's own `usage` total is reported, which is the honest number.
|
|
509
|
-
>
|
|
510
|
-
> ⚠️ **Migration.** Groq traffic used to report under `openai.chat.completions.create`, because
|
|
511
|
-
> shape was all that matched it. It now reports `groq.chat.completions.create`. **A Stigg
|
|
512
|
-
> raw-events meter configured against the `openai.*` name for Groq traffic will stop matching** —
|
|
513
|
-
> add `groq.chat.completions.create` to that meter, or pass `recipe: openAIRecipe` at `wrap` to
|
|
514
|
-
> keep the old name. Traffic to OpenAI itself is unaffected.
|
|
515
|
-
|
|
516
|
-
> **Mistral, both ways round.** Mistral ships its own client, and `mistralRecipe` governs it —
|
|
517
|
-
> `chat.complete`/`chat.stream`/`chat.parse`/`chat.parseStream`, `fim.*` and `agents.*`, eight
|
|
518
|
-
> surfaces over one usage block, reported as `mistral.*`. Point an `OpenAI` client at
|
|
519
|
-
> `https://api.mistral.ai/v1` instead and you get the row above: `openAIRecipe` matches it and it
|
|
520
|
-
> reports `openai.*`. Both report the same counter **names** — `prompt_tokens` and the rest — because
|
|
521
|
-
> the vendor SDK's camelCase is a rename of that same wire, and one account should not read as two
|
|
522
|
-
> vocabularies. Mistral is matched by **shape**, not hostname: it serves from `api.mistral.ai`,
|
|
523
|
-
> `api.eu.mistral.ai` and `api.us.mistral.ai`, and a per-request `serverURL` can override all three.
|
|
524
|
-
>
|
|
525
|
-
> Two Mistral call styles are **not** governed, and both are quiet about it. Every method also
|
|
526
|
-
> exists as a standalone function taking the client as an argument — `chatComplete(client, …)` from
|
|
527
|
-
> `@mistralai/mistralai/funcs/` — which never goes through the patched method, so such a call is
|
|
528
|
-
> neither gated nor metered. And a streamed response read through `getReader()` rather than
|
|
529
|
-
> `for await` bypasses the meter, because the stream is your object and cannot be swapped for a
|
|
530
|
-
> metered copy. Call the methods, and iterate the stream.
|
|
531
|
-
|
|
532
|
-
> **OpenRouter, both ways round.** OpenRouter's own `@openrouter/sdk` package is a different shape
|
|
533
|
-
> — its methods are `chat.send` and `responses.send`, and its counters come back camelCased — but
|
|
534
|
-
> `openrouterRecipe` declares those surfaces too, so `wrap` governs that client as well as an
|
|
535
|
-
> `OpenAI` one pointed at `https://openrouter.ai/api/v1`. Four event names rather than two
|
|
536
|
-
> (`openrouter.chat.send` beside `openrouter.chat.completions.create`), because a meter filtering
|
|
537
|
-
> on one must not catch the other. Both report the **wire's** counter names: the SDK's camelCase is
|
|
538
|
-
> a rename of the same block, and one account should not read as two vocabularies.
|
|
539
|
-
|
|
540
|
-
> **Databricks, as two recipes over one wire.** Databricks model serving is OpenAI-compatible, so
|
|
541
|
-
> an `OpenAI` client pointed at `https://<workspace>/serving-endpoints` is all the application
|
|
542
|
-
> holds. Two things make it worth a recipe rather than the `openAIRecipe` row above.
|
|
543
|
-
>
|
|
544
|
-
> The first is that **the request does not say what model is being used.** On the classic route the
|
|
545
|
-
> `model` argument is a serving *endpoint* name the customer chose — `prod-chat` is legal, and it
|
|
546
|
-
> may front DBRX, a Llama that Databricks hosts, or somebody's OpenAI account. The response's
|
|
547
|
-
> `model` field is what recovers that, and `databricksRecipe` puts it on the ingest as a dimension.
|
|
548
|
-
> Metered as `openai.*` the whole workspace is one undifferentiated stream.
|
|
549
|
-
>
|
|
550
|
-
> The second is **who gets invoiced**, and this one is a decision you have to make rather than one
|
|
551
|
-
> the SDK can make for you. Foundation Model API traffic (`databricks-claude-*`, `databricks-llama-*`,
|
|
552
|
-
> DBRX) is billed to your Databricks account in DBUs. An endpoint configured as an *external model*
|
|
553
|
-
> carries your own OpenAI or Anthropic key, and that provider invoices your account with them —
|
|
554
|
-
> Databricks only estimates the figure, into `system.ai_gateway.external_model_spend`, and says it
|
|
555
|
-
> is not liable for the difference. Same host, same path, same request, same response: **nothing on
|
|
556
|
-
> the client side distinguishes them.** So the split is an assertion, not a detection —
|
|
557
|
-
>
|
|
558
|
-
> ```ts
|
|
559
|
-
> stigg.wrap(client, { recipe: databricksExternalRecipe() });
|
|
560
|
-
> ```
|
|
561
|
-
>
|
|
562
|
-
> — and `databricksExternalRecipe` is deliberately not registered by default, because an assertion
|
|
563
|
-
> nobody made is not a default. Assert wrongly and nothing throws: DBU traffic lands on the meter
|
|
564
|
-
> you keep for upstream spend, or vice versa. If one workspace serves both kinds, wrap one client
|
|
565
|
-
> per kind.
|
|
566
|
-
>
|
|
567
|
-
> **Neither recipe reports a `cost`**, on either route — Databricks states no price on any
|
|
568
|
-
> response, in the body or in a header. Reconcile spend against its system tables out of band.
|
|
569
|
-
>
|
|
570
|
-
> Recognised by workspace host **and** serving path: the three public families
|
|
571
|
-
> (`cloud.databricks.com`, `azuredatabricks.net`, `gcp.databricks.com`) crossed with
|
|
572
|
-
> `/serving-endpoints` or `/ai-gateway/mlflow/v1`. The host alone is every Databricks API there is,
|
|
573
|
-
> most of which are not inference. A PrivateLink or custom-DNS workspace matches neither half —
|
|
574
|
-
> give the recipe its base URL: `databricksRecipe({ baseUrl })`.
|
|
575
|
-
>
|
|
576
|
-
> **Both clients, one recipe.** `@databricks/sdk-modelservingquery` is Databricks' own JS client,
|
|
577
|
-
> and its whole surface is one method — `client.query({ name, messages })`, reported as
|
|
578
|
-
> `databricks.query`. It is a second *surface* rather than a second vendor, because one Databricks
|
|
579
|
-
> account is billed whichever client the application imported; splitting it would split one bill
|
|
580
|
-
> across two meters. It is recognised automatically, so `wrap(client)` governs it with no recipe
|
|
581
|
-
> named.
|
|
582
|
-
>
|
|
583
|
-
> Three things are worth knowing before you choose it. It **camelCases the wire** — its generated
|
|
584
|
-
> parser turns `prompt_tokens` into `promptTokens` — so this SDK renames the counters back, and
|
|
585
|
-
> both clients report `prompt_tokens` for the same workspace. It reports **three counters, not
|
|
586
|
-
> six**: its usage type has no cache pair and no `reasoning_tokens`, which are absent rather than
|
|
587
|
-
> zero. And it **cannot stream**, and its request carries no `tools`, `response_format` or
|
|
588
|
-
> `stream_options` — so for real work the OpenAI-compatible route above is still the richer one,
|
|
589
|
-
> and it is what the Databricks docs show.
|
|
590
|
-
>
|
|
591
|
-
> One method serves chat, completions, embeddings and custom-model predictions, so the event name
|
|
592
|
-
> cannot say which ran. The response's own `object` field rides as a `response_object` dimension
|
|
593
|
-
> so a meter can tell them apart.
|
|
594
|
-
|
|
595
|
-
> **Cloudflare Workers AI is a provider, not a gateway.** Cloudflare runs these models on its own
|
|
596
|
-
> GPUs and invoices them to your Cloudflare account, which is the test this package applies. The
|
|
597
|
-
> `@cf/<author>/<model>` ids look like a reseller's catalogue and are not one — `<author>` is who
|
|
598
|
-
> *made* the weights, not who bills for running them, and `@cf/openai/gpt-oss-120b` is OpenAI's
|
|
599
|
-
> open-weights release at Cloudflare's own per-token rate rather than OpenAI API pricing. (`@hf/…`
|
|
600
|
-
> ids are Workers AI too, and about an eighth of the catalogue, so anything keying on the `@cf/`
|
|
601
|
-
> prefix alone is wrong.)
|
|
602
|
-
>
|
|
603
|
-
> Two clients reach it and both report under `cloudflare`, because one account is billed
|
|
604
|
-
> whichever is held: an `OpenAI` client pointed at
|
|
605
|
-
> `https://api.cloudflare.com/client/v4/accounts/<id>/ai/v1`, and the `cloudflare` package's own
|
|
606
|
-
> client, whose method is `ai.run(model, inputs)`. Two event names, so a meter can tell the wires
|
|
607
|
-
> apart.
|
|
608
|
-
>
|
|
609
|
-
> **It publishes no cost and no neuron count.** Cloudflare bills Workers AI in Neurons and puts a
|
|
610
|
-
> neuron figure on no response at all — not the binding's, not either REST route's — so converting
|
|
611
|
-
> tokens to Neurons needs Cloudflare's per-model rate table, which this SDK does not hold and will
|
|
612
|
-
> not guess at. Tokens are what the response states and tokens are what it reports.
|
|
613
|
-
>
|
|
614
|
-
> ⚠️ **A streamed `ai.run` is gated but not metered.** Asked for a stream, that method hands back
|
|
615
|
-
> the raw SSE byte stream rather than parsed chunks, so there is no object with a `usage` block to
|
|
616
|
-
> read — and the recipe attaches no reader rather than one that would find nothing and report a
|
|
617
|
-
> zero. The call is still checked before it leaves; the measurement afterwards is what is missing.
|
|
618
|
-
> Stream through the OpenAI-compatible route instead, where usage arrives on the final chunk, or
|
|
619
|
-
> call `ai.run` buffered.
|
|
620
|
-
>
|
|
621
|
-
> `ai.run` also carries **no model dimension**, because the model is that method's first argument
|
|
622
|
-
> and an extractor only ever sees the result. Nothing is lost that matters — Workers AI does not
|
|
623
|
-
> route, so what was asked for is what ran — but a meter that wants it can have it from
|
|
624
|
-
> `stiggContext: { dimensions: { model: '@cf/meta/llama-3.1-8b-instruct' } }`.
|
|
625
|
-
>
|
|
626
|
-
> One wrinkle about that hostname: `…/ai/v1` also serves **third-party** models (a bare
|
|
627
|
-
> `author/model` id like `openai/gpt-4.1`) routed through AI Gateway and paid out of Cloudflare
|
|
628
|
-
> credits under Unified Billing. Those land on this recipe, which is defensible rather than a
|
|
629
|
-
> mistake — Cloudflare is the payer either way — and the model is on the response, so a meter that
|
|
630
|
-
> wants them apart can split on it. AI Gateway proper lives on `gateway.ai.cloudflare.com` and has
|
|
631
|
-
> its own recipe.
|
|
632
|
-
|
|
633
|
-
Client recipes are data — method paths and usage extractors — so the factory wires the built-in
|
|
634
|
-
ones with no vendor package loaded. Nothing in `src/index.ts` pulls a vendor package in at
|
|
635
|
-
runtime, which is what keeps one import enough for every example.
|
|
636
|
-
|
|
637
|
-
"Gateway" covers two deployments that want opposite bindings, and the split is worth naming.
|
|
638
|
-
|
|
639
|
-
**A gateway you call** — OpenRouter as a service — returns its response into your process, and
|
|
640
|
-
that response carries two things no token counter can recover: the credits it actually charged
|
|
641
|
-
(`usage.cost`) and which model actually served after fallback routing (`response.model`). That is
|
|
642
|
-
an ordinary client, so it takes the ordinary client binding: `openrouterRecipe()` reads both, and
|
|
643
|
-
the call is gated before it ever leaves. It **is** registered by default and matched by base URL
|
|
644
|
-
ahead of `openAIRecipe`, because reading a richer response is not a decision anyone needs to opt
|
|
645
|
-
into — out of the box it reports the same token metrics the plain OpenAI recipe would, and adds
|
|
646
|
-
`cost`, the model that served and the gateway's own request id.
|
|
647
|
-
|
|
648
|
-
Tokens and money are different features, so both are reported and you price each in Stigg. When a
|
|
649
|
-
response carries no cost, the key is simply absent rather than substituted with something else.
|
|
650
|
-
`unitsPerCredit` (micro-credits by default) is the denomination of the `cost` feature.
|
|
651
|
-
|
|
652
|
-
**A gateway you call whose price is in a header** — LiteLLM Proxy — is the same binding with one
|
|
653
|
-
wrinkle worth knowing before you budget on it. LiteLLM prices every call itself, but writes the
|
|
654
|
-
number into `x-litellm-response-cost` rather than into the body, so `litellmRecipe()` reads the
|
|
655
|
-
response and not just the payload. Headers go on the wire before the model has produced a token,
|
|
656
|
-
so a **streamed** call cannot carry a cost at all — tokens survive a stream fine, a price does
|
|
657
|
-
not. So a non-streaming call reports its tokens **and** a `cost`, while a streamed one reports
|
|
658
|
-
the tokens alone — the same features either way, one of them simply missing where the protocol
|
|
659
|
-
cannot carry it.
|
|
660
|
-
That is the protocol rather than a limit of the binding, and the gateway example prints both
|
|
661
|
-
numbers side by side so the difference is not a surprise in an invoice.
|
|
662
|
-
|
|
663
|
-
**A gateway you operate** — a proxy fronting services that may not even be Node — is the one that
|
|
664
|
-
is not a binding in the full sense, and deliberately so: it is out-of-process and sees
|
|
665
|
-
**requests, not runs**. It cannot provision, cannot check against run-scoped state, and cannot
|
|
666
|
-
archive, and there is no client object out there to wrap. So the lifecycle splits across the
|
|
667
|
-
boundary — the application checks in-process, attribution travels as headers,
|
|
668
|
-
and the proxy only ingests. Governance never leaves the process.
|
|
669
|
-
|
|
670
|
-
A recipe reports a `Measurement`, not just token counts: `metrics` carries the vendor's own field
|
|
671
|
-
names and is what gets reported (a gateway puts its real spend in `metrics.cost` there, where a
|
|
672
|
-
token mapper would only lose it), `raw` rides an event's dimensions under a `raw.` prefix,
|
|
673
|
-
`dimensions` are merged over the context's own, and `idempotencyKey` lets the vendor's own request
|
|
674
|
-
id — stable across _its_ retries, which a local step counter cannot be — replace the default
|
|
675
|
-
`correlationId:step`. Returning
|
|
676
|
-
a bare `TokenUsage` still works and stays the common case.
|
|
677
|
-
|
|
678
|
-
### The engine: SDK first, REST to complete
|
|
679
|
-
|
|
680
|
-
Two clients, composed per operation, and the SDK tells you which one it picked:
|
|
681
|
-
|
|
682
|
-
```text
|
|
683
|
-
[governance] mode=governance check=rest report=node-sdk provisioning=rest (governance enabled)
|
|
684
|
-
```
|
|
685
|
-
|
|
686
|
-
| Operation | Client | Why |
|
|
687
|
-
| ------------------- | ------------------------------------- | ------------------------------------------------------------------------ |
|
|
688
|
-
| `check` | `@stigg/typescript` (REST, `v1-beta`) | only the REST surface takes `dimensions` and returns governance `chains` |
|
|
689
|
-
| `ingest` | `@stigg/node-server-sdk` | it has the in-memory cache, and usage reporting is the hot path |
|
|
690
|
-
| provision / archive | `@stigg/typescript` (REST, `v1-beta`) | entities and assignments exist only on the governance surface |
|
|
691
|
-
|
|
692
|
-
**Ingest has three paths, not two.** `ingest` picks between the first two — but **the name of the
|
|
693
|
-
metric decides first**, and that is the rule to read before the table:
|
|
694
|
-
|
|
695
|
-
> **If a metric is named after the credit currency the context checks, it draws the wallet
|
|
696
|
-
> directly, in both modes.** No event, no usage report, and no meter involved — `ingest` then only
|
|
697
|
-
> decides whether the call waits for the balance to come back.
|
|
698
|
-
|
|
699
|
-
That is the path `unitsPerTokens(currencyId, …)` puts you on, and it is why switching `async` and
|
|
700
|
-
`sync` can look like it changes nothing: both deduct credits, neither needs a metered feature, and
|
|
701
|
-
the only visible difference is whether `IngestResult.creditsConsumed` is filled in. Metrics named
|
|
702
|
-
after a **feature** take one of the other two columns.
|
|
703
|
-
|
|
704
|
-
| | A metric named after the currency | `ingest: 'async'` — the default | `ingest: 'sync'` |
|
|
705
|
-
| ---------------------- | --------------------------------------- | ----------------------------------- | ---------------------------------------------------- |
|
|
706
|
-
| Call | `consumeCredits` / `consumeCreditsAsync` | `events.report` / `reportEvent` | `usage.report` / `reportUsageBulk` |
|
|
707
|
-
| Latency and rate limit | follows the mode | low latency, much higher limit | higher latency, lower limit |
|
|
708
|
-
| The next check | includes it under `'sync'`; lags under `'async'` | lags while Stigg aggregates | already includes the spend |
|
|
709
|
-
| What you send | a currency id and an amount | the call, as one event | a `featureId` and a whole-number `value`, per metric |
|
|
710
|
-
| Meter type | **none — the wallet is drawn directly** | **Raw events**, configured in Stigg | calculated / incremental |
|
|
711
|
-
| What comes back | a balance under `'sync'`, nothing under `'async'` | an acknowledgement | a credit balance |
|
|
712
|
-
|
|
713
|
-
Async sends **one event for the whole turn**, every metric on its own dimension, because one event
|
|
714
|
-
feeds any number of meters — fanning out per metric buys nothing and costs a request each. Sync
|
|
715
|
-
sends one report per metric, batched into a single bulk call.
|
|
716
|
-
|
|
717
|
-
There is no event path from a raw event to a currency, which is why the first column exists at all:
|
|
718
|
-
a usage report takes only a `featureId`, so a currency has to be drawn rather than metered.
|
|
719
|
-
|
|
720
|
-
> **`'async'` does not settle the wallet before the next check.** `consumeCreditsAsync` is
|
|
721
|
-
> reconciled into the balance on Stigg's side, and nothing invalidates the entitlement cache in the
|
|
722
|
-
> meantime — so a check made immediately after a draw can still see the balance from before it, and
|
|
723
|
-
> approve a call the wallet could not afford. `'sync'` is what closes that window.
|
|
724
|
-
|
|
725
|
-
**The match is exact.** A metric draws the wallet only when its name *equals* the currency the
|
|
726
|
-
context checks. `unitsPerTokens('data_credit', …)` against `entitlement: { currencyId: 'credits' }`
|
|
727
|
-
is a feature report named `data_credit`, not a draw on `credits` — so it needs a meter like any
|
|
728
|
-
other feature, and silently meters nothing until one exists.
|
|
729
|
-
|
|
730
|
-
**A check asks for one unit by default, so the last call can overshoot.** The budget is allowed
|
|
731
|
-
below zero, and `'async'` adds a reconciliation window on top — calls made while Stigg is still
|
|
732
|
-
turning events into usage all see the same balance. Worst case is therefore one call's cost, plus
|
|
733
|
-
whatever lands in that window.
|
|
734
|
-
|
|
735
|
-
Narrow it per call with the recipe's own estimator, which asks for what the call could cost
|
|
736
|
-
instead of for one unit:
|
|
737
|
-
|
|
738
|
-
```ts
|
|
739
|
-
import { anthropicRecipe, estimateFromMaxTokens } from '@stigg/agent-sdk-js';
|
|
740
|
-
|
|
741
|
-
stigg.wrap(new Anthropic(), {
|
|
742
|
-
stiggContext: { customerId, entitlement: { currencyId: 'credits' } },
|
|
743
|
-
// `estimate` reads the call's own parameters - here `max_tokens` - and asks for that many
|
|
744
|
-
// units instead of one. The ingest that follows still corrects the balance.
|
|
745
|
-
recipe: {
|
|
746
|
-
...anthropicRecipe,
|
|
747
|
-
methods: anthropicRecipe.methods.map((method) => ({ ...method, estimate: estimateFromMaxTokens(1_000) })),
|
|
748
|
-
},
|
|
749
|
-
});
|
|
750
|
-
```
|
|
751
|
-
|
|
752
|
-
It is opt-in on purpose: an estimate that overshoots denies calls the budget could have afforded,
|
|
753
|
-
and the ingest that follows corrects the balance anyway. For a hard cap, pair it with
|
|
754
|
-
`ingest: 'sync'` — `'async'` trades exactness for throughput, so it is not what a hard cap is for.
|
|
755
|
-
|
|
756
|
-
**Which to reach for.** Credits when the unit is money-like, shared across features, pre-purchased
|
|
757
|
-
or topped up — one balance, many consumers. A metered feature when the limit is per-capability,
|
|
758
|
-
belongs to a plan entitlement, or resets on a billing cadence. Both when a plan grants a feature
|
|
759
|
-
allowance and a credit pool covers the overage; name the feature in `usageMapper` and let Stigg's
|
|
760
|
-
consumption mapping convert it, rather than pricing it in code.
|
|
761
|
-
|
|
762
|
-
**Governance is optional.** Not every account has it. When it is missing the factory falls back
|
|
763
|
-
to customer-level enforcement on the Node SDK alone — every example still runs, denials just name
|
|
764
|
-
the customer instead of an agent:
|
|
765
|
-
|
|
766
|
-
```text
|
|
767
|
-
[governance] mode=customer-level check=node-sdk report=node-sdk provisioning=unavailable
|
|
768
|
-
(403 GovernanceNotEnabled - customer-level fallback)
|
|
769
|
-
```
|
|
770
|
-
|
|
771
|
-
Nothing declares this. The client asks the account once at init and acts on the answer, so there
|
|
772
|
-
is no second source of truth to disagree with the first. Without governance, checks and usage
|
|
773
|
-
reporting still work at the customer level — and `register()` follows that split. A bare
|
|
774
|
-
registration warns and hands the id back: it asked for an identity, and a call needs none. A call
|
|
775
|
-
carrying a **budget or a parent** throws, because there is nowhere to write the assignment, and a
|
|
776
|
-
run that looked governed with a budget that was never written is the failure worth refusing.
|
|
777
|
-
|
|
778
|
-
### What the cache promises
|
|
779
|
-
|
|
780
|
-
**This package has no cache.** Checks are served by the Node server SDK's entitlement cache, which
|
|
781
|
-
is a far better one than a TTL over decisions could be:
|
|
782
|
-
|
|
783
|
-
- it writes the server's own measurement response back into cached usage, so a report that moved
|
|
784
|
-
the budget is visible to the very next check;
|
|
785
|
-
- it invalidates on realtime updates, so a plan change or a cancelled subscription lands without
|
|
786
|
-
waiting out a window;
|
|
787
|
-
- it applies usage-period resets at read time, rather than serving a number that expired;
|
|
788
|
-
- it falls back to configured entitlements when Stigg is unreachable;
|
|
789
|
-
- and it can be backed by **Redis**, so sibling processes share one view instead of each holding
|
|
790
|
-
their own.
|
|
791
|
-
|
|
792
|
-
All of that is configured through `cache`, which is the Node SDK's own configuration passed
|
|
793
|
-
straight through — `redis`, `memoryCacheMaxSizeBytes`, `entitlementsFallback`,
|
|
794
|
-
`creditEntitlementsFallback`, `realtimeUpdatesEnabled`, `enableEdge`, `offline`:
|
|
795
|
-
|
|
796
|
-
```ts
|
|
797
|
-
const stigg = new StiggAgentSDK(apiKey, {
|
|
798
|
-
cache: { redis: { host: 'redis', port: 6379, environmentPrefix: 'prod', ttl: -1 } },
|
|
799
|
-
});
|
|
800
|
-
```
|
|
82
|
+
| | Gateway | | Gateway |
|
|
83
|
+
| --- | --- | --- | --- |
|
|
84
|
+
| <img src="https://raw.githubusercontent.com/stiggio/stigg-agent-sdk-ts/main/docs/logos/bedrock.svg" width="18" alt=""> | Amazon Bedrock | <img src="https://raw.githubusercontent.com/stiggio/stigg-agent-sdk-ts/main/docs/logos/litellm.png" width="18" alt=""> | LiteLLM |
|
|
85
|
+
| <img src="https://raw.githubusercontent.com/stiggio/stigg-agent-sdk-ts/main/docs/logos/cloudflare.svg" width="18" alt=""> | Cloudflare Workers AI | <img src="https://raw.githubusercontent.com/stiggio/stigg-agent-sdk-ts/main/docs/logos/openrouter.svg" width="18" alt=""> | OpenRouter |
|
|
86
|
+
| <img src="https://raw.githubusercontent.com/stiggio/stigg-agent-sdk-ts/main/docs/logos/databricks.svg" width="18" alt=""> | Databricks | | |
|
|
801
87
|
|
|
802
|
-
|
|
803
|
-
|
|
804
|
-
is evaluated against the amount it actually asks for.
|
|
88
|
+
Each binding is written against the *shape a vendor returns*, not against its SDK — so installing
|
|
89
|
+
this package pulls in **no vendor packages at all**, and you install only the one you use.
|
|
805
90
|
|
|
806
|
-
|
|
91
|
+
`wrap()` recognises the vendor from the client's own shape. It throws, naming what it knows,
|
|
92
|
+
rather than handing back a client that quietly meters nothing.
|
|
807
93
|
|
|
808
|
-
|
|
94
|
+
## 🧩 Beyond a single call
|
|
809
95
|
|
|
810
|
-
|
|
811
|
-
const stigg = new StiggAgentSDK(apiKey, { client: appStiggClient });
|
|
812
|
-
```
|
|
813
|
-
|
|
814
|
-
This is about **ownership**, not deduplication. `Stigg.initialize` already keeps one global
|
|
815
|
-
instance per process and hands it back when the API key matches — so without `client` you may
|
|
816
|
-
well be handed the application's own, and nothing observable can tell that apart from a client
|
|
817
|
-
the SDK just built: the instance exposes neither its configuration nor its identity. Since
|
|
818
|
-
`close()` nulls that global and tears the runtime down, `stigg.close()` would then take the
|
|
819
|
-
application's cache and socket with it.
|
|
820
|
-
|
|
821
|
-
A client passed here is **borrowed and never closed**; one the SDK builds is owned and closed. It
|
|
822
|
-
replaces only the cached half — the governance REST client is still built from the API key,
|
|
823
|
-
because the Node SDK has no governance surface — and it is mutually exclusive with `cache`, which
|
|
824
|
-
configures a client the SDK is no longer building.
|
|
825
|
-
|
|
826
|
-
An earlier design cached decisions here for a `cacheTtlMs` (default 5s). It duplicated the
|
|
827
|
-
machinery above and degraded it: a cached allow outlived the report that spent the budget, could
|
|
828
|
-
not hear an invalidation, and did not carry the requested amount in its key — so an allow fetched
|
|
829
|
-
for one unit authorized a call asking for ten thousand. `cacheTtlMs` is now **ignored**, and
|
|
830
|
-
warns when set.
|
|
831
|
-
|
|
832
|
-
**The one path that caches nothing is an attributed governance check** — a check carrying
|
|
833
|
-
dimensions, which is how per-agent budgets are enforced. It resolves an entity, which only the
|
|
834
|
-
governance API can do, so it is a live call every time. That is why
|
|
835
|
-
`RestGovernanceEngine.capabilities.strictCheck` is `true`: there is no window in which two
|
|
836
|
-
processes can each approve the same budget. It costs a round trip per call, which is the right
|
|
837
|
-
trade in front of a multi-second model call and the wrong one to optimize away with a stale allow.
|
|
838
|
-
|
|
839
|
-
None of it applies to a context that names no `entitlement`: it warms nothing and checks nothing,
|
|
840
|
-
because it never asks.
|
|
841
|
-
|
|
842
|
-
#### How this SDK identifies itself
|
|
843
|
-
|
|
844
|
-
**There is no telemetry.** Nothing is emitted on a schedule, nothing is batched in the background,
|
|
845
|
-
and there is no second network path that could fail or leak. The SDK names itself on the requests
|
|
846
|
-
it was already making, and that is all:
|
|
847
|
-
|
|
848
|
-
| Half | What it sends |
|
|
96
|
+
| You want to | Use |
|
|
849
97
|
| --- | --- |
|
|
850
|
-
|
|
|
851
|
-
|
|
|
852
|
-
|
|
853
|
-
|
|
854
|
-
|
|
855
|
-
|
|
856
|
-
Two things it deliberately does **not** do. It does not send a distinct `source`: that header is a
|
|
857
|
-
server-side enum with no member for this SDK, the API groups on it, so a new member is a server
|
|
858
|
-
change that has to land first — the client name carries the distinction until then. And it does not
|
|
859
|
-
rename a client you passed in through `client`, which keeps whatever name your application gave it;
|
|
860
|
-
`Stigg.initialize` may also hand back a client your application built first, in which case the same
|
|
861
|
-
applies. Both cases are correct — the name belongs to whoever owns the client.
|
|
862
|
-
|
|
863
|
-
## 0. Try it with no setup at all
|
|
864
|
-
|
|
865
|
-
```bash
|
|
866
|
-
yarn workspace @stigg/agent-sdk-js run example-client-anthropic # or -openai, -gemini, -openrouter, -litellm
|
|
867
|
-
yarn workspace @stigg/agent-sdk-js run example-gateway # or example-gateway-litellm
|
|
868
|
-
```
|
|
869
|
-
|
|
870
|
-
The full list, with what each one demonstrates, is in [section 6](#6-run-the-examples).
|
|
871
|
-
|
|
872
|
-
The vendor mock speaks the real wire formats, so the vendor SDK parses a genuine response and
|
|
873
|
-
every field the examples read (`usage`, `model`, `cost`, tool calls) is populated. The Stigg mock
|
|
874
|
-
implements real entities, budgets and hierarchy, so a demo genuinely hits a denial and can name
|
|
875
|
-
which entity caused it. Only the prose and the storage are fake.
|
|
876
|
-
|
|
877
|
-
To force the mocks even when keys **are** present — useful when a key is valid but the account
|
|
878
|
-
has no credits — set `AGENT_SDK_OFFLINE=1` in `.env`.
|
|
879
|
-
|
|
880
|
-
## 1. Prerequisites
|
|
881
|
-
|
|
882
|
-
- **Node 22** — the version in the repo's `.nvmrc`. Check with `node --version`.
|
|
883
|
-
- **Yarn 4** via Corepack:
|
|
884
|
-
|
|
885
|
-
```bash
|
|
886
|
-
corepack enable
|
|
887
|
-
```
|
|
888
|
-
|
|
889
|
-
## 2. Get an Anthropic API key (optional)
|
|
890
|
-
|
|
891
|
-
Used by the client examples. Skip this to stay on the offline mock.
|
|
892
|
-
|
|
893
|
-
1. Go to **<https://console.anthropic.com>** and sign up or sign in.
|
|
894
|
-
2. Open **Billing** and add a payment method or buy credits. A brand-new org has no balance,
|
|
895
|
-
and requests fail with `400 credit balance is too low` until you do.
|
|
896
|
-
3. Open **Settings → API Keys** and click **Create Key**. Give it a name.
|
|
897
|
-
4. **Copy the key immediately** — it is shown only once. It starts with `sk-ant-`.
|
|
898
|
-
5. Put it in `.env` as `AGENT_SDK_ANTHROPIC_API_KEY` (step 5 below).
|
|
899
|
-
|
|
900
|
-
The examples default to the `claude-opus-5` model. Override with `AGENT_SDK_ANTHROPIC_MODEL` in `.env`.
|
|
901
|
-
|
|
902
|
-
## 3. Gateway access (optional)
|
|
903
|
-
|
|
904
|
-
Used by the gateway examples. Skip this entirely to stay on the offline mock, which speaks both
|
|
905
|
-
wire formats — including LiteLLM's header-borne cost, and its absence on a stream.
|
|
906
|
-
|
|
907
|
-
### OpenRouter
|
|
908
|
-
|
|
909
|
-
1. Go to **<https://openrouter.ai>** and sign in with Google, GitHub, or email.
|
|
910
|
-
2. Open **Settings → Credits** and add credits (a few dollars is plenty for these examples).
|
|
911
|
-
3. Go to **<https://openrouter.ai/keys>** and click **Create Key**.
|
|
912
|
-
4. Optionally set a **credit limit** on the key — a hard cap on what this key can ever spend.
|
|
913
|
-
Recommended while experimenting.
|
|
914
|
-
5. Copy the key (starts with `sk-or-`) and put it in `.env` as `AGENT_SDK_OPENROUTER_API_KEY`.
|
|
915
|
-
|
|
916
|
-
The example asks for `AGENT_SDK_OPENROUTER_MODEL` and falls back through
|
|
917
|
-
`AGENT_SDK_OPENROUTER_FALLBACK_MODELS`.
|
|
918
|
-
Both are model slugs — browse or verify them at **<https://openrouter.ai/models>**.
|
|
919
|
-
|
|
920
|
-
### LiteLLM
|
|
921
|
-
|
|
922
|
-
There is no signup: a LiteLLM proxy is something you host, so this is a URL rather than an account.
|
|
923
|
-
|
|
924
|
-
1. Start one — `pip install 'litellm[proxy]'`, then `litellm --model gpt-4o --port 4000`. It needs
|
|
925
|
-
the upstream key for whatever it fronts (`OPENAI_API_KEY`, and so on) in its own environment.
|
|
926
|
-
2. Put its URL in `.env` as `AGENT_SDK_LITELLM_BASE_URL`. The default is `http://localhost:4000` —
|
|
927
|
-
note a **local Stigg** uses port 4000 too, so move one of them if you run both.
|
|
928
|
-
3. Put a virtual key in `AGENT_SDK_LITELLM_API_KEY` if the proxy requires one (`--master_key`, or
|
|
929
|
-
a key minted through `/key/generate`). Leave it blank and the examples use the offline mock.
|
|
930
|
-
|
|
931
|
-
`AGENT_SDK_LITELLM_MODEL` is a **model group** from the proxy's own config, not a provider slug.
|
|
932
|
-
A group is free to resolve to a different deployment, and that difference is the routing drift
|
|
933
|
-
the examples flag.
|
|
934
|
-
|
|
935
|
-
## 4. Point at a Stigg environment (optional)
|
|
936
|
-
|
|
937
|
-
One key and one URL, pointed at whichever environment you mean. The key is environment-bound, so
|
|
938
|
-
it and the URL always move together — there is no separate profile name to keep in sync with them.
|
|
939
|
-
|
|
940
|
-
| Environment | `STIGG_BASE_URL` |
|
|
941
|
-
| ----------- | --------------------------------------------------------------- |
|
|
942
|
-
| production | `https://api.stigg.io` (the default when the variable is blank) |
|
|
943
|
-
| staging | `https://api-staging.stigg.io` |
|
|
944
|
-
| local | `http://localhost:4000` |
|
|
945
|
-
|
|
946
|
-
Get a server key for that environment from **app.stigg.io → Integrations → API keys** and put it
|
|
947
|
-
in `STIGG_API_KEY`.
|
|
948
|
-
|
|
949
|
-
**Leave `STIGG_API_KEY` blank and everything still runs** against a built-in Stigg mock, with no
|
|
950
|
-
account at all — the same rule the vendor keys follow. `AGENT_SDK_OFFLINE=1` forces the mocks even
|
|
951
|
-
when keys are present, which is what you want when a key is valid but the account has no credits.
|
|
952
|
-
|
|
953
|
-
Two things adjust themselves to the URL, so you do not configure them twice:
|
|
954
|
-
|
|
955
|
-
- The mock answers both transports — REST and `/graphql` — so the engine resolves exactly as it
|
|
956
|
-
does against a real account: governance checks and provisioning over REST, ingest and
|
|
957
|
-
unattributed checks over the Node SDK. The line the SDK prints at startup says which it picked.
|
|
958
|
-
- Against `localhost` the Node SDK's edge cache and realtime socket are switched off, because a
|
|
959
|
-
local API serves neither and the SDK maps a localhost base to edge port 3060.
|
|
960
|
-
|
|
961
|
-
### Seed the environment
|
|
962
|
-
|
|
963
|
-
```bash
|
|
964
|
-
yarn workspace @stigg/agent-sdk-js run example-setup # 5-credit agent budget
|
|
965
|
-
yarn workspace @stigg/agent-sdk-js run example-setup --agent-limit 6 # raise it
|
|
966
|
-
yarn workspace @stigg/agent-sdk-js run example-setup --no-governance # seed without entities
|
|
967
|
-
yarn workspace @stigg/agent-sdk-js run example-setup --archive # clean up, bottom-up
|
|
968
|
-
```
|
|
969
|
-
|
|
970
|
-
`--no-governance` describes this invocation only: it skips the entity and budget steps, so the
|
|
971
|
-
account is seeded without a tree. It is not a mode the examples inherit — offline, each one starts
|
|
972
|
-
its own mock, and that mock has governance enabled. To see customer-level enforcement end to end,
|
|
973
|
-
use an account that genuinely has no governance.
|
|
974
|
-
|
|
975
|
-
Idempotent per step, and safe to re-run — but not safe to run twice **at once**: the credit grant
|
|
976
|
-
reads the balance and then grants, with nothing between the two, so two concurrent invocations can
|
|
977
|
-
both decide the pool is short and both fund it. Run it once at a time. Steps are gated by what the
|
|
978
|
-
account actually has:
|
|
979
|
-
|
|
980
|
-
| Step | Needs governance? | What it does |
|
|
981
|
-
| -------------------------- | ----------------- | -------------------------------------------------------------------------------- |
|
|
982
|
-
| 1. customer | no | creates it if missing |
|
|
983
|
-
| 2. credit pool | no | grants credits when the balance is below `--pool-min`, guarded by a balance read |
|
|
984
|
-
| 3. entity types + entities | **yes** | `team` (attribution key `teamId`) and `agent` (`agentId`) |
|
|
985
|
-
| 4. assignments | **yes** | One agent per example, each = 5 per `P1M`, all parented to `team-research` |
|
|
986
|
-
|
|
987
|
-
Each example registers **its own agent** — `agent-support-bot-anthropic`, `-openai`, `-gemini`, `-openrouter`,
|
|
988
|
-
`-litellm`, and the base `agent-support-bot` for the gateway examples. Budgets attach to the agent, so
|
|
989
|
-
sharing one would mean running the OpenAI example spends what the Anthropic example had left. The
|
|
990
|
-
shared parent team keeps the ancestor rule demonstrable, and its budget is sized so one agent
|
|
991
|
-
exhausting itself does not block the others:
|
|
992
|
-
|
|
993
|
-
```text
|
|
994
|
-
team-research (team) usage=0/30
|
|
995
|
-
agent-support-bot usage=0/5
|
|
996
|
-
agent-support-bot-anthropic usage=0/5
|
|
997
|
-
agent-support-bot-openai usage=0/5
|
|
998
|
-
agent-support-bot-openrouter usage=0/5
|
|
999
|
-
agent-support-bot-litellm usage=0/5
|
|
1000
|
-
```
|
|
1001
|
-
|
|
1002
|
-
**The client examples provision their own agent** — the seed prepares the account (entity types,
|
|
1003
|
-
the parent team, a funded pool). That is the more realistic split: an admin sets up the account, an
|
|
1004
|
-
application provisions its own agents at runtime. `register()` carries it:
|
|
1005
|
-
|
|
1006
|
-
One caveat about the cap, because the two halves can overlap. `apply: 'on-create'` is exactly that
|
|
1007
|
-
— on create — so if the seed has already written an assignment for that agent, the application's
|
|
1008
|
-
cap is **not** applied and the seeded one stands. That is the point of the option (an admin who
|
|
1009
|
-
raises a budget out of band should not have it stamped back on every process start), but it does
|
|
1010
|
-
mean `--agent-limit` on the seed wins over the number below. Leave `apply` at its default
|
|
1011
|
-
(`'always'`) if the application is meant to own it.
|
|
1012
|
-
|
|
1013
|
-
```ts
|
|
1014
|
-
await stigg.register({
|
|
1015
|
-
customerId: 'customer-acme',
|
|
1016
|
-
agentId: 'agent-support-bot-anthropic',
|
|
1017
|
-
entityTypeId: 'agent',
|
|
1018
|
-
budget: {
|
|
1019
|
-
// A budget is written over (entity x entitlement), so the entitlement is required here: a
|
|
1020
|
-
// cap has nowhere to land without one.
|
|
1021
|
-
entitlement: { currencyId: 'credits' },
|
|
1022
|
-
parentId: 'team-research',
|
|
1023
|
-
limit: 5,
|
|
1024
|
-
cadence: 'P1M',
|
|
1025
|
-
},
|
|
1026
|
-
});
|
|
1027
|
-
```
|
|
1028
|
-
|
|
1029
|
-
A cap is **not** a field on the entity. An entity is just a node — id, type, display name,
|
|
1030
|
-
metadata — and a limit is an _assignment_ over `(entity × capability)`. So that one call is two
|
|
1031
|
-
idempotent upserts underneath: `PUT …/entities` then `PUT …/assignments`. It is also why
|
|
1032
|
-
`parentId` rides the assignment rather than the entity — and sits inside `budget` for the same
|
|
1033
|
-
reason: the hierarchy is per-capability, so one agent can sit under different parents for
|
|
1034
|
-
different currencies.
|
|
1035
|
-
|
|
1036
|
-
`budget.limit` is tri-state — a number caps it, `null` tracks without blocking (the observe-only
|
|
1037
|
-
rollout), and omitting it leaves whatever is already assigned untouched. Governance requires both
|
|
1038
|
-
it and `cadence` when it creates a budget, which is why both are passed above.
|
|
1039
|
-
|
|
1040
|
-
The pool is deliberately much larger than the agent budget, so a demo denial comes from the
|
|
1041
|
-
agent's own cap (`BudgetExceeded`) rather than an empty wallet (`InsufficientCredits`). Those are
|
|
1042
|
-
different products — an upsell versus an escalation to whoever set the budget — and the SDK keeps
|
|
1043
|
-
them distinct all the way to the message the user sees.
|
|
1044
|
-
|
|
1045
|
-
## 5. Configure and install
|
|
1046
|
-
|
|
1047
|
-
Two files, deliberately separate. Both are git-ignored, and each has a committed `.template`
|
|
1048
|
-
beside it to copy from.
|
|
1049
|
-
|
|
1050
|
-
**`.env`** (from `.env.template`) — credentials and SDK configuration. Everything in it is either a
|
|
1051
|
-
secret or a real setting you would also make in your own application.
|
|
1052
|
-
|
|
1053
|
-
**`examples/demo.env`** (from `examples/demo.env.template`) — the demo's fixtures: the tenant, the
|
|
1054
|
-
team, the agent. Nothing in it is configuration — a real application knows its tenant from the
|
|
1055
|
-
request it is serving, never from an env file.
|
|
1056
|
-
|
|
1057
|
-
Neither file is required **against the mocks**: every fixture default is compiled into the
|
|
1058
|
-
examples, so with no files and no keys at all they still run end to end. Those defaults are
|
|
1059
|
-
mock-only. Once `STIGG_API_KEY` is set the examples stop inventing ids for an environment they
|
|
1060
|
-
can actually write to — `AGENT_SDK_DEMO_CUSTOMER_ID` becomes required, and the rest degrade to a
|
|
1061
|
-
smaller demo (see the fixtures table below).
|
|
1062
|
-
|
|
1063
|
-
`.env` is loaded first and is never overwritten, so setting one of the `AGENT_SDK_DEMO_*` names
|
|
1064
|
-
there (or exporting it in your shell) overrides the fixture without editing `demo.env`.
|
|
1065
|
-
|
|
1066
|
-
### `.env` — credentials and endpoints
|
|
1067
|
-
|
|
1068
|
-
| Variable | Default | Purpose |
|
|
1069
|
-
| -------------------------------------- | ---------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------ |
|
|
1070
|
-
| `AGENT_SDK_ANTHROPIC_API_KEY` | — (falls back to `ANTHROPIC_API_KEY`) | Layers 1 and 3 |
|
|
1071
|
-
| `AGENT_SDK_ANTHROPIC_MODEL` | `claude-opus-5` | Claude model for layers 1 and 3 |
|
|
1072
|
-
| `AGENT_SDK_OPENAI_API_KEY` | — (falls back to `OPENAI_API_KEY`) | The OpenAI client example |
|
|
1073
|
-
| `AGENT_SDK_OPENAI_MODEL` | `gpt-5` | Model for the OpenAI client example |
|
|
1074
|
-
| `AGENT_SDK_GROQ_API_KEY` | — (falls back to `GROQ_API_KEY`) | The Groq client example. Blank ⇒ the offline mock |
|
|
1075
|
-
| `AGENT_SDK_GROQ_MODEL` | `llama-3.3-70b-versatile` | Model for the Groq client example |
|
|
1076
|
-
| `AGENT_SDK_MISTRAL_API_KEY` | — (falls back to `MISTRAL_API_KEY`) | The Mistral client example. Blank ⇒ the offline mock |
|
|
1077
|
-
| `AGENT_SDK_MISTRAL_MODEL` | `mistral-small-latest` | Model for the Mistral client example |
|
|
1078
|
-
| `AGENT_SDK_OLLAMA_HOST` | — (falls back to `OLLAMA_HOST`) | Ollama is self-hosted, so a **host** stands in for the key: blank ⇒ the offline mock, set ⇒ that server (default `http://127.0.0.1:11434`) |
|
|
1079
|
-
| `AGENT_SDK_OLLAMA_MODEL` | `llama3.3` | Model for the Ollama client example — must be pulled on that server first |
|
|
1080
|
-
| `AGENT_SDK_OPENROUTER_API_KEY` | — (falls back to `OPENROUTER_API_KEY`) | Layer 2 |
|
|
1081
|
-
| `AGENT_SDK_GEMINI_API_KEY` | — | Falls back to `GEMINI_API_KEY` / `GOOGLE_API_KEY`. Absent, the Gemini example runs on the built-in mock |
|
|
1082
|
-
| `AGENT_SDK_GEMINI_MODEL` | `gemini-3.8-flash` | Model for the Gemini client example |
|
|
1083
|
-
| `AGENT_SDK_OPENROUTER_MODEL` | `anthropic/claude-opus-5` | Primary model requested from the gateway |
|
|
1084
|
-
| `AGENT_SDK_OPENROUTER_FALLBACK_MODELS` | `anthropic/claude-sonnet-5,openai/gpt-5` | Comma-separated fallbacks tried in order |
|
|
1085
|
-
| `AGENT_SDK_BEDROCK_REGION` | `us-east-1` (falls back to `AWS_REGION`) | Region the Bedrock model is called in. There is no Bedrock key variable: AWS authenticates through its own credential chain, and the example asks that chain rather than looking for one |
|
|
1086
|
-
| `AGENT_SDK_BEDROCK_MODEL` | `anthropic.claude-sonnet-4-5-20250929-v1:0` | Bedrock model id. Names one model exactly — Bedrock does not route, so there is no fallback list |
|
|
1087
|
-
| `AGENT_SDK_LITELLM_API_KEY` | — (falls back to `LITELLM_API_KEY`) | Layer 2. Blank ⇒ the offline mock |
|
|
1088
|
-
| `AGENT_SDK_CLOUDFLARE_API_KEY` | none | Cloudflare API token with Workers AI read access (falls back to `CLOUDFLARE_API_TOKEN`) |
|
|
1089
|
-
| `AGENT_SDK_CLOUDFLARE_ACCOUNT_ID` | none | Required by every `ai.run` call — it is part of the URL path, not the body |
|
|
1090
|
-
| `AGENT_SDK_CLOUDFLARE_MODEL` | `@cf/meta/llama-3.1-8b-instruct` | A Workers AI model id. `@hf/…` ids are Workers AI too, so a matcher keyed on `@cf/` alone misses about an eighth of the catalogue |
|
|
1091
|
-
| `AGENT_SDK_DATABRICKS_API_KEY` | — (falls back to `DATABRICKS_TOKEN`) | A workspace token — a PAT, or an OAuth M2M token. Blank ⇒ the offline mock. Read by both Databricks examples |
|
|
1092
|
-
| `AGENT_SDK_DATABRICKS_BASE_URL` | none | Your workspace — customer-specific, so there is no default that could be right. `/serving-endpoints` is appended if you leave it off, and a PrivateLink workspace works because the recipe is handed this URL rather than recognising the host |
|
|
1093
|
-
| `AGENT_SDK_DATABRICKS_MODEL` | `databricks-claude-sonnet-4-6` | A serving **endpoint** name, not a model id — which is why the served model has to be read off the response. Read by both Databricks examples |
|
|
1094
|
-
| `AGENT_SDK_DATABRICKS_EXTERNAL` | unset | `1` asserts the endpoint carries your own upstream key, so that provider invoices it rather than Databricks. An assertion the SDK cannot check — it switches both Databricks examples to `databricks-external.*` |
|
|
1095
|
-
| `AGENT_SDK_LITELLM_BASE_URL` | `http://localhost:4000` | Your own proxy — self-hosted, so there is no public default |
|
|
1096
|
-
| `AGENT_SDK_LITELLM_MODEL` | `gpt-5` | A model **group** from the proxy's config, not a provider slug |
|
|
1097
|
-
| `STIGG_API_KEY` | — | **Server** key (`server-…`). A publishable `client-…` key fails every call with `IdentityForbidden`. Blank ⇒ the built-in Stigg mock |
|
|
1098
|
-
| `STIGG_BASE_URL` | `https://api.stigg.io` | Which environment that key belongs to |
|
|
1099
|
-
|
|
1100
|
-
### What the examples gate and meter against — in `examples/demo.env`
|
|
1101
|
-
|
|
1102
|
-
These moved out of `.env`, because they fail the test that separates the two files: **if you would
|
|
1103
|
-
change it to play with an example, it is a fixture.** A real application decides every one of them
|
|
1104
|
-
in code — what it bills against is part of its pricing, not of its deployment.
|
|
98
|
+
| Govern a client | `stigg.wrap(client, { stiggContext })` |
|
|
99
|
+
| Gate without a client to wrap | `stigg.check(context)` |
|
|
100
|
+
| Meter something you measured yourself | `stigg.report(context, metrics)` |
|
|
101
|
+
| Give an agent its own budget | `stigg.register({ customerId, agentId, budget })` |
|
|
102
|
+
| Meter a gateway you operate | `client.stigg.headers()` out, `stigg.ingestFromHeaders(...)` back |
|
|
1105
103
|
|
|
1106
|
-
|
|
1107
|
-
`
|
|
1108
|
-
environment, and `wrap(client, { stiggContext })` — overridable inside any call's own parameters —
|
|
1109
|
-
for what a call is governed by.
|
|
104
|
+
One wrapped client serves every customer and every agent: the context is an argument, given once at
|
|
105
|
+
`wrap` as a default and overridable inside any single call.
|
|
1110
106
|
|
|
1111
|
-
|
|
1112
|
-
| -------------------------------- | ----------------------- | ---------------------------------------------------------------------------------- |
|
|
1113
|
-
| `AGENT_SDK_DEMO_CURRENCY_ID` | `credits` *(mock only)* | `stiggContext.entitlement` — credit currency to budget against. The default is the examples' own and applies only offline; the SDK has none. Naming neither this nor a feature is legal and means the calls **meter without gating** — `entitlement` is itself the enforcement switch. `two-agents` and the gateway examples refuse to run that way, since a budget draining is their whole subject |
|
|
1114
|
-
| `AGENT_SDK_DEMO_FEATURE_ID` | — | `stiggContext.entitlement` — metered feature instead; exclusive with the above |
|
|
1115
|
-
| `AGENT_SDK_DEMO_EVENT_NAME` | — | `stiggContext.eventName` — overrides the name derived from the vendor call |
|
|
107
|
+
## 📚 Documentation
|
|
1116
108
|
|
|
1117
|
-
|
|
1118
|
-
|
|
1119
|
-
|
|
1120
|
-
|
|
1121
|
-
|
|
1122
|
-
|
|
1123
|
-
|
|
1124
|
-
| The variable is… | Against the mock | With a real `STIGG_API_KEY` |
|
|
1125
|
-
| --------------------- | ----------------------------- | ----------------------------------------------- |
|
|
1126
|
-
| **absent** | the default below | unset — the demo degrades, or says what it needs |
|
|
1127
|
-
| **set** | that value | that value |
|
|
1128
|
-
| **present but blank** | unset, deliberately | unset, deliberately |
|
|
1129
|
-
|
|
1130
|
-
The defaults are mock-only on purpose: against an environment you can write to, a guessed
|
|
1131
|
-
`customer-acme` is either someone else's tenant or one nobody asked for.
|
|
1132
|
-
|
|
1133
|
-
| Variable | Default *(mock only)* | Unset ⇒ |
|
|
1134
|
-
| ------------------------------------- | --------------------- | --------------------------------------------------------------------- |
|
|
1135
|
-
| `AGENT_SDK_DEMO_CUSTOMER_ID` | `customer-acme` | **Error.** Every governance call is scoped to a tenant, and the examples will not invent one |
|
|
1136
|
-
| `AGENT_SDK_DEMO_TEAM_ID` | `team-research` | The agents are **roots** — no ancestor budget binds them. `two-agents` needs one and says so |
|
|
1137
|
-
| `AGENT_SDK_DEMO_AGENT_ID` | `agent-support-bot` | **No agent at all**: checks and usage land on the customer's own pool. Blank it to see that path against the mock. Each client example appends its vendor (`…-anthropic`) |
|
|
1138
|
-
| `AGENT_SDK_DEMO_ENTITY_TYPE_ID` | `agent` *(mock only)* | `register`'s own `entityTypeId` — the entity type agents are provisioned as |
|
|
1139
|
-
| `AGENT_SDK_DEMO_ATTRIBUTION_KEY` | `agentId` *(mock only)* | `stiggContext.attributionKey` — the key that type identifies its agents under. **Both or neither**: the SDK defaults neither half, because a guessed key resolves no entity and the spend lands on the customer looking exactly like an ungoverned call |
|
|
1140
|
-
| `AGENT_SDK_OFFLINE` | empty | Set to `1` to force the mocks even when keys are present |
|
|
1141
|
-
| `AGENT_SDK_DEMO_MOCK_GOVERNANCE_FAIL` | empty | Mock only: make the runtime check return `503` |
|
|
1142
|
-
|
|
1143
|
-
Install dependencies from the **repo root**:
|
|
1144
|
-
|
|
1145
|
-
```bash
|
|
1146
|
-
corepack enable
|
|
1147
|
-
yarn install
|
|
1148
|
-
```
|
|
1149
|
-
|
|
1150
|
-
## 6. Run the examples
|
|
1151
|
-
|
|
1152
|
-
All commands work from anywhere in the repo, and all of them run with no keys at all — except
|
|
1153
|
-
`example-client-typesafe`, whose vendor the built-in mock does not speak. That one still needs no
|
|
1154
|
-
Stigg account.
|
|
1155
|
-
|
|
1156
|
-
| Command | Runs | What it shows |
|
|
1157
|
-
| --------------------------- | --------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------- |
|
|
1158
|
-
| `example-client-anthropic` | [`client/anthropic.ts`](./examples/client/anthropic.ts) | A governed chat. One `wrap()` gates and meters every turn |
|
|
1159
|
-
| `example-client-openai` | [`client/openai.ts`](./examples/client/openai.ts) | The same script, a different vendor — governance is not vendor-specific work |
|
|
1160
|
-
| `example-client-gemini` | [`client/gemini.ts`](./examples/client/gemini.ts) | The same script again, on a vendor that counts in its own names and streams running totals |
|
|
1161
|
-
| `example-client-groq` | [`client/groq.ts`](./examples/client/groq.ts) | Groq through the `openai` client: matched by hostname, so it reports `groq.*`, and its streamed counters come off `x_groq` |
|
|
1162
|
-
| `example-client-mistral` | [`client/mistral.ts`](./examples/client/mistral.ts) | A vendor whose SDK camelCases the wire — `chat.stream` is its own method, and the metered names are the wire's |
|
|
1163
|
-
| `example-client-ollama` | [`client/ollama.ts`](./examples/client/ollama.ts) | A self-hosted engine that bills nobody: a flat client, counters flat on the response, and capacity rather than spend |
|
|
1164
|
-
| `example-client-openrouter` | [`client/openrouter.ts`](./examples/client/openrouter.ts) | `openrouterRecipe()` billing the credits actually charged instead of a token count |
|
|
1165
|
-
| `example-client-litellm` | [`client/litellm.ts`](./examples/client/litellm.ts) | The same script against a self-hosted proxy. Streams, so it meters tokens — see the gateway example for why |
|
|
1166
|
-
| `example-client-databricks` | [`client/databricks.ts`](./examples/client/databricks.ts) | An `OpenAI` client pointed at a workspace `/serving-endpoints`. The request names an **endpoint**, not a model, so the served model on the way back is the thing to watch. `AGENT_SDK_DATABRICKS_EXTERNAL=1` asserts the external-model route |
|
|
1167
|
-
| `example-client-typesafe` | [`client/typesafe.ts`](./examples/client/typesafe.ts) | A non-chat client: typed questions, batched or one per call. **Needs a real TypeSafe key** |
|
|
1168
|
-
| `example-two-agents` | [`client/two-agents.ts`](./examples/client/two-agents.ts) | Two agents, one team: a sibling denied by its own cap, then the survivor denied by the team above it |
|
|
1169
|
-
| `example-gateway-openrouter`| [`gateway/openrouter.ts`](./examples/gateway/openrouter.ts) | The two gateway topologies: calling one gates, operating one only ingests. `example-gateway` runs this one |
|
|
1170
|
-
| `example-gateway-litellm` | [`gateway/litellm.ts`](./examples/gateway/litellm.ts) | The same two topologies when the cost is in a header: real spend, then tokens, from one request |
|
|
1171
|
-
| `example-gateway-bedrock` | [`gateway/bedrock.ts`](./examples/gateway/bedrock.ts) | A gateway that states no price: tokens are all there is to meter, and nothing proxies it |
|
|
1172
|
-
| `example-gateway-databricks` | [`gateway/databricks.ts`](./examples/gateway/databricks.ts) | Databricks' **own** client, `@databricks/sdk-modelservingquery` — one method, `client.query(…)`, governed by the same recipe as the row above and reported as `databricks.query`. Shows the camelCase counters renamed back to the wire's, and a budget draining until it refuses |
|
|
1173
|
-
| `example-gateway-cloudflare` | [`gateway/cloudflare.ts`](./examples/gateway/cloudflare.ts) | Cloudflare's own client (`ai.run`). The one example that does not stream — a streamed `ai.run` hands back raw SSE bytes, so there is no usage block to meter |
|
|
1174
|
-
| `example-setup` | [`shared/setup-governance.ts`](./examples/shared/setup-governance.ts) | Seeds a customer, a pool and per-entity budgets. Idempotent |
|
|
1175
|
-
|
|
1176
|
-
`example-client` is an alias for the Anthropic one. Prefix any of them with `AGENT_SDK_OFFLINE=1`
|
|
1177
|
-
to force the mocks even when keys are present.
|
|
1178
|
-
|
|
1179
|
-
### Layer 1 — client abstraction (interactive chat)
|
|
1180
|
-
|
|
1181
|
-
An interactive, multi-turn chat, governed end to end, against any of five vendors. Type `exit`,
|
|
1182
|
-
`quit`, an empty line, or press **Ctrl-D** / **Ctrl-C** to end it.
|
|
1183
|
-
|
|
1184
|
-
```bash
|
|
1185
|
-
yarn workspace @stigg/agent-sdk-js run example-client-anthropic # also: example-client
|
|
1186
|
-
yarn workspace @stigg/agent-sdk-js run example-client-openai
|
|
1187
|
-
yarn workspace @stigg/agent-sdk-js run example-client-gemini
|
|
1188
|
-
yarn workspace @stigg/agent-sdk-js run example-client-openrouter
|
|
1189
|
-
yarn workspace @stigg/agent-sdk-js run example-client-litellm
|
|
1190
|
-
```
|
|
1191
|
-
|
|
1192
|
-
All five run [the same script](./examples/client/core/chat.ts) — same governance wiring, same
|
|
1193
|
-
output, not one line of it vendor-specific. What differs is a helper per vendor
|
|
1194
|
-
([anthropic](./examples/client/anthropic.ts), [openai](./examples/client/openai.ts),
|
|
1195
|
-
[gemini](./examples/client/gemini.ts), [openrouter](./examples/client/openrouter.ts),
|
|
1196
|
-
[litellm](./examples/client/litellm.ts)) holding the four things that genuinely differ:
|
|
1197
|
-
which key to read, how to build the client, how to stream a turn, and where usage sits on the
|
|
1198
|
-
response. That is the same split the SDK makes internally, for the same reason.
|
|
1199
|
-
|
|
1200
|
-
All five report the names their own vendor used, which is why the ingest lines differ between
|
|
1201
|
-
them: Anthropic states `input_tokens`/`output_tokens`, OpenAI states `prompt_tokens`/
|
|
1202
|
-
`completion_tokens`, and Gemini states `promptTokenCount`/`candidatesTokenCount` — plus a
|
|
1203
|
-
`thoughtsTokenCount` that is billed as output and counted beside the candidates rather than
|
|
1204
|
-
inside them. Run `example-gateway-litellm` to watch a gateway add `cost` beside them.
|
|
1205
|
-
|
|
1206
|
-
```text
|
|
1207
|
-
[governance] stigg=in-process mock customer=customer-acme currency=credits
|
|
1208
|
-
[stigg] mode=governance check=rest (node-sdk when unattributed) report=node-sdk provisioning=rest (governance enabled)
|
|
1209
|
-
[stigg] agent agent-support-bot-anthropic registered under customer customer-acme
|
|
1210
|
-
|
|
1211
|
-
you> what is metering?
|
|
1212
|
-
[governance] check hasAccess=true remaining=5 source=rest
|
|
1213
|
-
assistant> Metering is the practice of recording how much of a service ...
|
|
1214
|
-
[usage:turn 1] input=12 output=46 cache_read=3 cache_creation=0
|
|
1215
|
-
[governance] ingest input_tokens=12 cache_read_input_tokens=3 output_tokens=46 key=9d63...:1 ok=true credits=0.94
|
|
1216
|
-
|
|
1217
|
-
...
|
|
1218
|
-
|
|
1219
|
-
you> one more
|
|
1220
|
-
[governance] check hasAccess=false remaining=0.5742 deniedBy=agent-support-bot-anthropic reason=BudgetExceeded remedy=raise-budget source=rest
|
|
1221
|
-
Budget exhausted for agent-support-bot-anthropic. Ask whoever set this budget to raise it, or wait for the next reset.
|
|
1222
|
-
Raise the budget in another shell, then just keep typing - the next message re-checks:
|
|
1223
|
-
yarn workspace @stigg/agent-sdk-js run example-setup --agent-limit 6
|
|
1224
|
-
```
|
|
1225
|
-
|
|
1226
|
-
The chat stays open after a denial. Run the suggested command in another shell and keep typing:
|
|
1227
|
-
because denials are never cached, the next message goes through. That is the whole point of
|
|
1228
|
-
making a refusal a commercial event rather than an error.
|
|
1229
|
-
|
|
1230
|
-
Every check reads `source=rest`, and `remaining` falls as the ingests land. This is an attributed
|
|
1231
|
-
check — the agent travels as a dimension, so it resolves an entity and only the governance API can
|
|
1232
|
-
answer it. Nothing is served from a decision cache, which is why the denial arrives on the first
|
|
1233
|
-
call that actually exceeds the budget rather than one window later.
|
|
1234
|
-
|
|
1235
|
-
### Layer 2 — gateways, in both directions
|
|
1236
|
-
|
|
1237
|
-
The same prompt through OpenRouter three times: wrapped with `openrouterRecipe()` (non-streaming,
|
|
1238
|
-
then streaming), then once more down the header path, so the two topologies sit side by side.
|
|
1239
|
-
|
|
1240
|
-
```bash
|
|
1241
|
-
yarn workspace @stigg/agent-sdk-js run example-gateway
|
|
1242
|
-
```
|
|
1243
|
-
|
|
1244
|
-
```text
|
|
1245
|
-
--- wrapped, non-streaming ---
|
|
1246
|
-
requested model : anthropic/claude-opus-5
|
|
1247
|
-
served model : anthropic/claude-sonnet-5 <- routing drift
|
|
1248
|
-
cost (credits) : 0.00108
|
|
1249
|
-
prompt tokens : 21 (cached: 7)
|
|
1250
|
-
completion tok. : 39 (reasoning: 4)
|
|
1251
|
-
[governance] hasAccess=true remaining=25000 source=rest
|
|
1252
|
-
[governance] ingest prompt_tokens=21 completion_tokens=39 cached_tokens=7 cost=1080 key=gen-mock... ok=true
|
|
1253
|
-
|
|
1254
|
-
--- header path, non-streaming ---
|
|
1255
|
-
...
|
|
1256
|
-
[governance] proxy ingest prompt_tokens=21 completion_tokens=39 cached_tokens=7 key=gen-mock... ok=true
|
|
1257
|
-
(ingest-only: no check on this path, and tokens not cost - the mapper cannot see usage.cost)
|
|
1258
|
-
```
|
|
1259
|
-
|
|
1260
|
-
The contrast is the whole point. Both report the same three token counters; only the wrapped call
|
|
1261
|
-
also reports **`cost=1080`** — 0.00108 credits in micro-credits, what the gateway really charged —
|
|
1262
|
-
attributed to the model that actually served. The header path carries a usage report rather than a
|
|
1263
|
-
price, so the one feature that says what the call cost in money is simply missing from it. Same
|
|
1264
|
-
request, same seams; the difference is a feature, not a denomination.
|
|
1265
|
-
|
|
1266
|
-
If the primary model is down, rate-limited, or refuses, OpenRouter serves a fallback slug instead
|
|
1267
|
-
and `served model` differs — flagged as `<- routing drift`, which silently changes COGS. The
|
|
1268
|
-
recipe puts that slug on the ingest as a `model` dimension, so the drift is visible in Stigg and
|
|
1269
|
-
not just in the log.
|
|
1270
|
-
|
|
1271
|
-
The `cost` feature is denominated by the recipe: `openrouterRecipe()` reports micro-credits by
|
|
1272
|
-
default, because OpenRouter charges fractional credits and Stigg ingests whole units. Set
|
|
1273
|
-
`unitsPerCredit` to match however the Stigg feature is defined. When a response carries no cost at
|
|
1274
|
-
all — BYOK, and a few upstreams — the `cost` key is simply absent; the token metrics were never
|
|
1275
|
-
conditional on it.
|
|
1276
|
-
|
|
1277
|
-
#### When the price is in a header
|
|
1278
|
-
|
|
1279
|
-
LiteLLM makes the same comparison and splits the wrapped call in two, because for this gateway
|
|
1280
|
-
_how_ you called it decides whether the price is knowable at all:
|
|
1281
|
-
|
|
1282
|
-
```bash
|
|
1283
|
-
yarn workspace @stigg/agent-sdk-js run example-gateway-litellm
|
|
1284
|
-
```
|
|
1285
|
-
|
|
1286
|
-
```text
|
|
1287
|
-
--- wrapped, non-streaming ---
|
|
1288
|
-
requested group : gpt-5
|
|
1289
|
-
served model : gpt-5-2025-08-07 <- routing drift
|
|
1290
|
-
cost (USD) : 0.001005
|
|
1291
|
-
[governance] ingest prompt_tokens=21 completion_tokens=36 cached_tokens=7 cost=1005 key=chatcmpl-... ok=true
|
|
1292
|
-
|
|
1293
|
-
--- wrapped, streaming ---
|
|
1294
|
-
cost (USD) : (not reported - the header precedes the body)
|
|
1295
|
-
[governance] ingest prompt_tokens=21 completion_tokens=36 cached_tokens=7 key=chatcmpl-... ok=true
|
|
1296
|
-
|
|
1297
|
-
--- header path, non-streaming ---
|
|
1298
|
-
[governance] proxy ingest prompt_tokens=21 completion_tokens=36 cached_tokens=7 key=chatcmpl-... ok=true
|
|
1299
|
-
(ingest-only: no check on this path, and tokens not cost - what crosses the boundary is a usage
|
|
1300
|
-
report, so the price the proxy computed has nowhere to go)
|
|
1301
|
-
```
|
|
1302
|
-
|
|
1303
|
-
Same proxy, same recipe, same prompt. The tokens are identical in all three; what differs is
|
|
1304
|
-
whether `cost` is there at all. The proxy knew the price every time — it computed it — but a
|
|
1305
|
-
header is flushed before generation starts, so the streamed call has nothing to read
|
|
1306
|
-
([BerriAI/litellm#12689](https://github.com/BerriAI/litellm/issues/12689)), and the ingest-only
|
|
1307
|
-
boundary carries a usage report rather than a price. Gating is unaffected throughout: both wrapped
|
|
1308
|
-
legs are refused before the request leaves, whatever they end up reporting.
|
|
1309
|
-
|
|
1310
|
-
So if you budget in spend against LiteLLM, price the `cost` feature and know that streamed turns
|
|
1311
|
-
will not contribute to it — the token features still will. `litellmRecipe()` reports micro-dollars
|
|
1312
|
-
by default; set `unitsPerDollar` to match however the Stigg feature is defined.
|
|
1313
|
-
|
|
1314
|
-
### Seeing the fail-closed path
|
|
1315
|
-
|
|
1316
|
-
The governance check is fail-closed server-side: it returns `503` rather than silently allowing.
|
|
1317
|
-
|
|
1318
|
-
```bash
|
|
1319
|
-
AGENT_SDK_OFFLINE=1 AGENT_SDK_DEMO_MOCK_GOVERNANCE_FAIL=1 \
|
|
1320
|
-
yarn workspace @stigg/agent-sdk-js run example-client-anthropic
|
|
1321
|
-
# [stigg] check failed: check: 503 governance cache unavailable
|
|
1322
|
-
# [governance] check hasAccess=false reason=EngineError remedy=retry source=policy
|
|
1323
|
-
|
|
1324
|
-
# The explicit opt-in - refuse the call when the check itself could not be made. An SDK option,
|
|
1325
|
-
# not an env var, and overridable per call through `stiggContext`:
|
|
1326
|
-
# new StiggAgentSDK({ apiKey, baseUrl, checkErrorAction: 'deny' })
|
|
1327
|
-
```
|
|
1328
|
-
|
|
1329
|
-
`allow` is the default: a soft limit, so an unreachable Stigg does not take the application down
|
|
1330
|
-
with it. `deny` mirrors the server's own fail-closed behaviour and is a decision worth making
|
|
1331
|
-
deliberately — "never break the customer's run" and "never spend past a budget" are not the same
|
|
1332
|
-
promise, and only you know which one this call is for.
|
|
1333
|
-
|
|
1334
|
-
## 7. Typecheck, lint and test
|
|
1335
|
-
|
|
1336
|
-
```bash
|
|
1337
|
-
yarn workspace @stigg/agent-sdk-js run build # typecheck, then emit dist/ - what gets published
|
|
1338
|
-
yarn workspace @stigg/agent-sdk-js run lint
|
|
1339
|
-
yarn workspace @stigg/agent-sdk-js run test # node:test via tsx; no network, literal fakes
|
|
1340
|
-
```
|
|
1341
|
-
|
|
1342
|
-
`build` is two passes over two configs, and the split is the point. `tsconfig.json` is the
|
|
1343
|
-
typechecker's view: it covers `src`, `examples`, `test` and `testing`, and emits nothing.
|
|
1344
|
-
`tsconfig.build.json` compiles the two entry points - `src/index.ts` and `src/internal.ts` - and
|
|
1345
|
-
whatever they import, so only `src/` reaches `dist/`. It clears `dist/` first, because a module
|
|
1346
|
-
that was renamed or deleted would otherwise sit there and ship.
|
|
1347
|
-
|
|
1348
|
-
Either pass resolves workspace dependencies to their built `.d.ts` (see `paths` in
|
|
1349
|
-
`tsconfig.json`) rather than to their source, which is compiled with different flags; Nx builds
|
|
1350
|
-
them first.
|
|
1351
|
-
|
|
1352
|
-
## 8. Troubleshooting
|
|
1353
|
-
|
|
1354
|
-
| Symptom | Cause and fix |
|
|
1355
|
-
| ------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
1356
|
-
| `403 GovernanceNotEnabled` | The account has no governance. Checks and usage still work; `register()` throws, because there is nowhere to create the agent entity. Contact Stigg to enable it. |
|
|
1357
|
-
| `No STIGG_API_KEY` | Set it to a server key for the environment in `STIGG_BASE_URL`, or leave it blank to run on the built-in mock. |
|
|
1358
|
-
| `503` and every turn denied | The governance cache is unavailable and `checkErrorAction` is `deny`. The default is `allow`, so this is a policy that was asked for; drop it to keep running through the outage, or relax a single call with `stiggContext: { checkErrorAction: 'allow' }`. |
|
|
1359
|
-
| `usage value must be a whole number to report against a feature` | `POST /api/v1/usage` takes integers. Report against a **credit currency** instead, which takes fractions, or raise the unit size in your `usageMapper` so the division comes out whole. |
|
|
1360
|
-
| `... is backed by a raw-events meter` | A `sync` run reported to a feature only a meter can fill. Report that run with `ingest: 'async'` (the default), or point it at a feature whose value your application computes. |
|
|
1361
|
-
| Ingest reports `ok=true` but usage stays 0 | Under `async` an acknowledgement is all the event API returns. Check a raw-events meter filters on the event name (see the event table above); against a **local** Stigg, also that the `ingest-usage-stream-processor` and `calculate-usage-stream-processor` lambdas are running. `ingest: 'sync'` answers with a balance and will say so instead. |
|
|
1362
|
-
| Still denied right after a top-up | Budgets reset per cadence window, so a spent agent stays spent. Re-run `example-setup --agent-limit N` with a higher limit, or use a new `AGENT_SDK_DEMO_AGENT_ID`. |
|
|
1363
|
-
| Denial names the customer, not the agent | Customer-level mode (no governance), or the context named no `agentId` and carried no attribution dimensions of its own. |
|
|
1364
|
-
| Denial names an agent you did not pass | A `dimensions` key resolves an entity that already exists in the account. The agent belongs in `stiggContext.agentId`; `dimensions` gates on whatever it names, registered here or not. |
|
|
1365
|
-
| `401` / `authentication_error` | The key is present but wrong. Clear it from `.env` to fall back to the offline mock, or fix it. |
|
|
1366
|
-
| `400 credit balance is too low` (Anthropic) | Add credits under Billing, or set `AGENT_SDK_OFFLINE=1` to run on the mock. |
|
|
1367
|
-
| `402` / insufficient credits (OpenRouter) | Add credits under Settings → Credits, or set `AGENT_SDK_OFFLINE=1` to run on the mock. |
|
|
1368
|
-
| `400` naming a model (OpenRouter) | A slug in `AGENT_SDK_OPENROUTER_MODEL` / `AGENT_SDK_OPENROUTER_FALLBACK_MODELS` does not exist. Check <https://openrouter.ai/models>. |
|
|
1369
|
-
| `ECONNREFUSED` on port 4000 (LiteLLM) | No proxy is running there. Start one (`litellm --model gpt-4o --port 4000`), fix `AGENT_SDK_LITELLM_BASE_URL`, or clear `AGENT_SDK_LITELLM_API_KEY` to fall back to the mock. |
|
|
1370
|
-
| `400` / unknown model group (LiteLLM) | `AGENT_SDK_LITELLM_MODEL` must name a group from the proxy's own config, not a provider slug. List them with `curl $AGENT_SDK_LITELLM_BASE_URL/v1/models`. |
|
|
1371
|
-
| LiteLLM turns bill `units=1`, not spend | Expected on a streamed call: `x-litellm-response-cost` is flushed before the body, so there is no cost to read. Call non-streaming to bill real spend. |
|
|
1372
|
-
| `Cannot find module` | Run `yarn install` from the repo root, not from this directory. |
|
|
1373
|
-
| Chat prints nothing and exits | Input is not a TTY (piped input hits end-of-file). Try `printf 'hi\nexit\n' \| yarn workspace @stigg/agent-sdk-js run example-client-anthropic`. |
|
|
1374
|
-
| A CLI hangs after the last line | The Node SDK keeps a background cache alive. Call `gov.flush({ close: true })`, as the examples do. |
|
|
1375
|
-
|
|
1376
|
-
## 9. Layout
|
|
1377
|
-
|
|
1378
|
-
```
|
|
1379
|
-
src/
|
|
1380
|
-
index.ts the whole public surface, one import
|
|
1381
|
-
stigg-agent-sdk.ts the composition root: the one file that knows both sides
|
|
1382
|
-
types.ts the vocabulary, and every call's argument list
|
|
1383
|
-
config.ts errors.ts logger.ts validated per-environment config, flat errors, structural logger
|
|
1384
|
-
decision-text.ts headers.ts how a decision reads; the cross-process attribution convention
|
|
1385
|
-
engine/ engine.ts decision.ts the four-operation seam; chains -> decision (pure)
|
|
1386
|
-
node-sdk-engine.ts the only file importing @stigg/node-server-sdk
|
|
1387
|
-
rest-governance-engine.ts the only file importing @stigg/typescript
|
|
1388
|
-
composite-engine.ts create-engine.ts routing per operation, and who picked what
|
|
1389
|
-
core/ run-handle.ts the handle every binding consumes
|
|
1390
|
-
bindings/ client/core/ the recipe contract, the registry, the wrapper
|
|
1391
|
-
client/providers/ client/gateways/ a model provider, or a gateway you call
|
|
1392
|
-
proxy/ a gateway you operate: headers in, ingest only
|
|
1393
|
-
shared/ usage shapes and metric tables both halves read
|
|
1394
|
-
|
|
1395
|
-
docs/ instances.md what exists at runtime, and what is shared
|
|
1396
|
-
|
|
1397
|
-
testing/ mock-stigg-server.ts the Stigg stand-in the examples and the tests share
|
|
1398
|
-
|
|
1399
|
-
examples/
|
|
1400
|
-
shared/ demo-env, the two mocks, setup-governance scaffolding every example uses
|
|
1401
|
-
client/ core/{chat,vendor} + anthropic|openai|gemini|openrouter|litellm|typesafe a helper per vendor
|
|
1402
|
-
two-agents two siblings, one team budget between them
|
|
1403
|
-
gateway/ core/{gateway,vendor,proxy} + openrouter|litellm calling one; operating one on a real socket
|
|
1404
|
-
```
|
|
109
|
+
- **[The full guide](docs/guide.md)** — every call, what it checks, what it reports, and a runnable
|
|
110
|
+
example per layer of the agent stack
|
|
111
|
+
- **[Examples](examples/)** — one script per provider and gateway, each runnable with **no API keys
|
|
112
|
+
and no Stigg account** against a built-in mock of both
|
|
113
|
+
- **[Adding a platform](.claude/skills/govern-new-platform/SKILL.md)** — provider or gateway, and
|
|
114
|
+
what every binding owes
|
|
1405
115
|
|
|
1406
|
-
|
|
1407
|
-
file is the shape the problem has, at both layers.
|
|
116
|
+
## 📄 License
|
|
1408
117
|
|
|
1409
|
-
|
|
1410
|
-
`stigg-agent-sdk.ts` knows both, which is what lets `wrap()` live on the SDK object
|
|
1411
|
-
without core learning a vendor's name. Adding a provider is a recipe file; adding an engine is a
|
|
1412
|
-
leaf file plus a branch in the factory.
|
|
118
|
+
MIT © Stigg Inc.
|