@tensorcost/sdk 1.3.3 → 1.5.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +226 -1
- package/README.md +35 -16
- package/dist/admit-client.d.ts +56 -0
- package/dist/admit-client.d.ts.map +1 -1
- package/dist/admit-client.js +93 -11
- package/dist/admit-client.js.map +1 -1
- package/dist/applied.d.ts +114 -0
- package/dist/applied.d.ts.map +1 -1
- package/dist/applied.js +40 -50
- package/dist/applied.js.map +1 -1
- package/dist/errors.d.ts +179 -24
- package/dist/errors.d.ts.map +1 -1
- package/dist/errors.js +165 -23
- package/dist/errors.js.map +1 -1
- package/dist/index.d.ts +2 -2
- package/dist/index.d.ts.map +1 -1
- package/dist/index.js +2 -2
- package/dist/index.js.map +1 -1
- package/dist/provider-errors.d.ts +15 -0
- package/dist/provider-errors.d.ts.map +1 -0
- package/dist/provider-errors.js +69 -0
- package/dist/provider-errors.js.map +1 -0
- package/dist/providers/anthropic.d.ts +54 -0
- package/dist/providers/anthropic.d.ts.map +1 -1
- package/dist/providers/anthropic.js +57 -14
- package/dist/providers/anthropic.js.map +1 -1
- package/dist/providers/bedrock.d.ts +4 -1
- package/dist/providers/bedrock.d.ts.map +1 -1
- package/dist/providers/bedrock.js +3 -2
- package/dist/providers/bedrock.js.map +1 -1
- package/dist/providers/detect.d.ts.map +1 -1
- package/dist/providers/detect.js +9 -6
- package/dist/providers/detect.js.map +1 -1
- package/dist/providers/openai.d.ts.map +1 -1
- package/dist/providers/openai.js +21 -44
- package/dist/providers/openai.js.map +1 -1
- package/dist/providers/vertex.d.ts +17 -0
- package/dist/providers/vertex.d.ts.map +1 -1
- package/dist/providers/vertex.js +5 -2
- package/dist/providers/vertex.js.map +1 -1
- package/dist/proxy-client.d.ts +157 -0
- package/dist/proxy-client.d.ts.map +1 -1
- package/dist/proxy-client.js +189 -175
- package/dist/proxy-client.js.map +1 -1
- package/dist/refusal.d.ts +43 -0
- package/dist/refusal.d.ts.map +1 -0
- package/dist/refusal.js +154 -0
- package/dist/refusal.js.map +1 -0
- package/dist/transport.d.ts.map +1 -1
- package/dist/transport.js +45 -13
- package/dist/transport.js.map +1 -1
- package/dist/types.d.ts +15 -1
- package/dist/types.d.ts.map +1 -1
- package/dist/version.d.ts +1 -1
- package/dist/version.js +1 -1
- package/dist/wrap-overhead.d.ts +43 -0
- package/dist/wrap-overhead.d.ts.map +1 -1
- package/dist/wrap-overhead.js +28 -1
- package/dist/wrap-overhead.js.map +1 -1
- package/dist/wrap.d.ts +25 -0
- package/dist/wrap.d.ts.map +1 -1
- package/dist/wrap.js +51 -8
- package/dist/wrap.js.map +1 -1
- package/package.json +1 -1
package/CHANGELOG.md
CHANGED
|
@@ -1,6 +1,231 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
-
##
|
|
3
|
+
## 1.5.0 — 2026-10-26
|
|
4
|
+
|
|
5
|
+
**1.4.0 could route around a hard spend cap when fail-open was on (the
|
|
6
|
+
default); upgrade.** When a money budget's hard cap refused a call (403
|
|
7
|
+
with `code: "BUDGET_HARD_CAP"`), 1.4.0 read the refusal as a provider error,
|
|
8
|
+
and with `failOpenEnabled` on it sent the call to the provider anyway. The
|
|
9
|
+
cap was enforced by the proxy and then skipped by the SDK.
|
|
10
|
+
|
|
11
|
+
This is a minor release, not a patch: it adds public error classes and
|
|
12
|
+
makes `wrap()` throw at startup without a tenant id. Read the upgrade notes.
|
|
13
|
+
|
|
14
|
+
### Upgrading from 1.4.x
|
|
15
|
+
|
|
16
|
+
- **`wrap()` requires a tenant id at `govern`, `steer` and `route`.** The
|
|
17
|
+
token exchange needs it, so without one the SDK never talked to TensorCost
|
|
18
|
+
at all: no `/v1/admit`, no proxy call, no observation, and every call went
|
|
19
|
+
straight to the provider while the README called the variable optional.
|
|
20
|
+
`wrap()` now throws `MissingConfigError` at those layers when neither
|
|
21
|
+
`tenantId` nor `TENSORCOST_TENANT_ID` is set, whatever `failOpenEnabled` says (1.4.0 skipped
|
|
22
|
+
TensorCost silently, or with `failOpenEnabled: false` in route threw
|
|
23
|
+
`TensorCostProxyError` on every call). Find the id in the TensorCost
|
|
24
|
+
console under **Developers › SDK setup**, in the `TENSORCOST_TENANT_ID`
|
|
25
|
+
line of the env block. `observe` without one keeps working and logs one
|
|
26
|
+
warning that nothing is being sent.
|
|
27
|
+
- **Unknown refusals are terminal.** A refusal code this version doesn't
|
|
28
|
+
know stops the call as `TensorCostRefusedError`; it no longer falls back
|
|
29
|
+
to the provider. Details below.
|
|
30
|
+
- **A revoked key fails closed.** A TensorCost key the token exchange,
|
|
31
|
+
`/v1/admit` or the proxy rejects is `TensorCostKeyRejectedError`, and the
|
|
32
|
+
call isn't sent. 1.4.0 let the call through to the provider when fail-open
|
|
33
|
+
was on (in steer/route with an openai@4 client it crashed instead, see
|
|
34
|
+
Fixed).
|
|
35
|
+
- **Route-mode embeddings, responses and legacy completions now go straight
|
|
36
|
+
to the provider.** Only `chat.completions` is proxied. 1.4.0 pointed the
|
|
37
|
+
whole client's `baseURL` at the proxy, so those calls went there too,
|
|
38
|
+
carrying the provider key as the bearer. Now they go to the provider and
|
|
39
|
+
are not proxied; legacy completions are observed, embeddings and
|
|
40
|
+
responses aren't wrapped at all.
|
|
41
|
+
|
|
42
|
+
### Changed: an unknown refusal is now terminal
|
|
43
|
+
|
|
44
|
+
1.4.0 decided whether a failed proxy call could go straight to the provider
|
|
45
|
+
with a list of refusals that must not. Any refusal missing from that list,
|
|
46
|
+
including ones added on the server later, fell back. 1.5.0 turns the rule
|
|
47
|
+
around: a call goes around TensorCost only when TensorCost is unavailable.
|
|
48
|
+
|
|
49
|
+
- Falls back (with `failOpenEnabled` on): network errors and connection
|
|
50
|
+
refused (`TensorCostNetworkError`), timeouts (`TensorCostTimeoutError`),
|
|
51
|
+
5xx from the proxy or an open circuit (`TensorCostProxyError`), and a 429
|
|
52
|
+
or 402 from the proxy with no refusal body (`TensorCostQuotaError`; the
|
|
53
|
+
429 is TensorCost's rate limit, the 402 the plan's monthly request
|
|
54
|
+
ceiling). The same outages on the token exchange count too.
|
|
55
|
+
- Never falls back: any 4xx carrying a TensorCost refusal, whatever the
|
|
56
|
+
code. A code this version doesn't know arrives as `TensorCostRefusedError`
|
|
57
|
+
with `code`, `reason` and the server's message intact. A refusal is never
|
|
58
|
+
retried, even on a 429.
|
|
59
|
+
- A provider's own 4xx relayed by the proxy (an OpenAI 400, say) doesn't
|
|
60
|
+
fall back either. It is thrown as the provider SDK's own error, built by
|
|
61
|
+
the client itself with `client.makeStatusError(...)`, the factory the
|
|
62
|
+
client uses for a direct call (`BadRequestError`, `AuthenticationError` and
|
|
63
|
+
the rest), with the original `TensorCostProviderError` on its `tensorcost`
|
|
64
|
+
property. The headers go in as a plain object first, which openai@4 reads,
|
|
65
|
+
then as a `Headers` instance, which current Stainless clients read with
|
|
66
|
+
`headers.get(...)`. Tested against openai@4.104.0 and @anthropic-ai/sdk
|
|
67
|
+
0.65.0, and against a stub of the openai v5/v6 `headers.get` contract;
|
|
68
|
+
openai v5/v6 itself wasn't run. A client that can't build one gets the
|
|
69
|
+
`TensorCostProviderError` (`status`, `body`, `headers`).
|
|
70
|
+
- A 4xx that isn't JSON (a firewall's HTML page, an empty body) is
|
|
71
|
+
`TensorCostGatewayRejectedError`: the request was turned away before it
|
|
72
|
+
reached TensorCost or the provider. Terminal.
|
|
73
|
+
- The proxy's own NestJS error body (`{ statusCode, message }`) is
|
|
74
|
+
TensorCost's, not the provider's: a 401 there is
|
|
75
|
+
`TensorCostKeyRejectedError`, anything else a `TensorCostRefusedError`
|
|
76
|
+
with the message.
|
|
77
|
+
|
|
78
|
+
A refusal is any 4xx whose body has an upper-case code at `error.code` or a
|
|
79
|
+
top-level `code` (`{"error": {"code": "SOME_CODE", ...}}` or
|
|
80
|
+
`{"code": "SOME_CODE", ...}`), or whose `x-tc-decision` header says
|
|
81
|
+
`action=refuse`. Lower-case codes (OpenAI's `invalid_api_key` and the like)
|
|
82
|
+
belong to the provider.
|
|
83
|
+
|
|
84
|
+
The govern layer follows the same rule: `/v1/admit` fails open only on an
|
|
85
|
+
outage, and only while `failOpenEnabled` is on. Before, admit failed open on
|
|
86
|
+
everything except a 403, whatever `failOpenEnabled` said.
|
|
87
|
+
|
|
88
|
+
### Added
|
|
89
|
+
|
|
90
|
+
**`TensorCostRefusedError`**, the base class of every refusal:
|
|
91
|
+
`TensorCostRunBudgetExceededError`, `TensorCostRunStoppedError`,
|
|
92
|
+
`TensorCostPeriodBudgetExceededError`, `TensorCostModelGovernanceDeniedError`,
|
|
93
|
+
`TensorCostGuardrailHardStopError`, `TensorCostComplianceDeniedError` and
|
|
94
|
+
`TensorCostComplianceTeamMismatchError` all extend it now, and all carry
|
|
95
|
+
`code`, `reason` and `detail` (the refusal object the server sent). Catch
|
|
96
|
+
`TensorCostRefusedError` to handle every refusal, current and future.
|
|
97
|
+
|
|
98
|
+
**`TensorCostBudgetHardCapError`** for a money budget's hard cap. It extends
|
|
99
|
+
`TensorCostPeriodBudgetExceededError`, so a `catch` written against 1.4.0 for
|
|
100
|
+
the current wire shape still matches. Fields: `budgetId`, `budgetName`,
|
|
101
|
+
`burnPct`, `scope`, `period`, `capCents`, `spentCents`, `reason`,
|
|
102
|
+
`budgetCode` (always `"BUDGET_HARD_CAP"`), `retryable` and
|
|
103
|
+
`retryAfterSeconds`. `retryable` is true when the budget's spend hasn't been
|
|
104
|
+
counted yet (`reason: "spend_not_known_yet"` or `retryable: true` on the
|
|
105
|
+
wire), usually for a few seconds after a budget is created or its scope
|
|
106
|
+
changes, and the message then says "the budget's spend hasn't been counted
|
|
107
|
+
yet; retry in 5 s". Otherwise it says "the budget's hard cap is reached". The
|
|
108
|
+
SDK doesn't retry for you. The shapes it accepts, on the chat path and on
|
|
109
|
+
`/v1/admit`:
|
|
110
|
+
|
|
111
|
+
```json
|
|
112
|
+
{ "error": { "code": "PERIOD_BUDGET_EXCEEDED", "budget_code": "BUDGET_HARD_CAP",
|
|
113
|
+
"budget_id": "b-1", "budget_name": "Prod AI", "burn_pct": 1.02,
|
|
114
|
+
"scope": "budget:b-1", "period": "monthly",
|
|
115
|
+
"cap_cents": 10000, "spent_cents": 10200, "retryable": false,
|
|
116
|
+
"message": "..." } }
|
|
117
|
+
|
|
118
|
+
{ "error": { "code": "PERIOD_BUDGET_EXCEEDED", "budget_code": "BUDGET_HARD_CAP",
|
|
119
|
+
"reason": "spend_not_known_yet", "retryable": true,
|
|
120
|
+
"retry_after_seconds": 5, "budget_id": "b-2", ... } }
|
|
121
|
+
+ header Retry-After: 5
|
|
122
|
+
|
|
123
|
+
{ "error": { "code": "BUDGET_HARD_CAP", "budget_id": "b1", "burn_pct": 0,
|
|
124
|
+
"reason": "spend_not_known_yet", "message": "..." } } (older servers)
|
|
125
|
+
```
|
|
126
|
+
|
|
127
|
+
`retryAfterSeconds` reads the `Retry-After` header (seconds or an HTTP date)
|
|
128
|
+
first, then `retry_after_seconds` or `retry_after` in the body. A
|
|
129
|
+
`PERIOD_BUDGET_EXCEEDED` without `budget_code` is still a plain
|
|
130
|
+
`TensorCostPeriodBudgetExceededError`.
|
|
131
|
+
|
|
132
|
+
**`TensorCostGatewayRejectedError`** (non-JSON 4xx, above) and
|
|
133
|
+
**`TensorCostKeyRejectedError`** (the token exchange, `/v1/admit` or the
|
|
134
|
+
proxy rejected the TensorCost key: "the TensorCost key was rejected; check
|
|
135
|
+
it's current"). Both are terminal. 1.4.0 threw a plain `Error` for a revoked
|
|
136
|
+
key, and with fail-open on, the call then went to the provider.
|
|
137
|
+
|
|
138
|
+
### Fixed
|
|
139
|
+
|
|
140
|
+
**steer/route with a real openai@4 client.** 1.4.0 rewrote the client's
|
|
141
|
+
`baseURL` to the proxy and replaced its `defaultHeaders()` method with a
|
|
142
|
+
plain object. Every call that reached the client's own request path died
|
|
143
|
+
with `TypeError: this.defaultHeaders is not a function`: every fail-open
|
|
144
|
+
fallback (proxy 5xx, 429, 402, network error, token-exchange outage), and
|
|
145
|
+
every call when a route-configured client ran below steer (published layer
|
|
146
|
+
govern or observe, TensorCost unreachable for more than 5 minutes). Had the
|
|
147
|
+
fallback run, it would have gone to the proxy URL, not the provider. 1.5.0
|
|
148
|
+
doesn't modify the client at all: the SDK sends the proxied call itself and
|
|
149
|
+
reads the provider URL and key from the client per call, and the fallback
|
|
150
|
+
goes to the client's own `baseURL`.
|
|
151
|
+
|
|
152
|
+
**Real Anthropic clients are detected as Anthropic.** The @anthropic-ai/sdk
|
|
153
|
+
client also has a legacy `completions.create`, so 1.4.0 detected it as an
|
|
154
|
+
OpenAI client and never wrapped `messages.create`: Claude calls were not
|
|
155
|
+
observed, admitted or stopped.
|
|
156
|
+
|
|
157
|
+
**Anthropic clients run in govern mode.** The proxy has no Anthropic Messages
|
|
158
|
+
route yet, so steer/route for an Anthropic client could only fail open around
|
|
159
|
+
TensorCost. `maxLayer: "steer"` or `"route"` (or `appliedMode: true`) on an
|
|
160
|
+
Anthropic client now runs as `"govern"`: every call is checked with
|
|
161
|
+
`/v1/admit` before it goes to Anthropic. One warning per process says so;
|
|
162
|
+
set `maxLayer: "govern"` to silence it.
|
|
163
|
+
|
|
164
|
+
**Bedrock and Vertex say govern isn't enforced.** There is no `/v1/admit`
|
|
165
|
+
check for them yet, so `maxLayer: "govern"` only observes. The SDK now logs
|
|
166
|
+
one warning per process saying so, and the README no longer lists govern as
|
|
167
|
+
supported for them.
|
|
168
|
+
|
|
169
|
+
**Govern-layer refusals are typed by their code.** 1.4.0 threw
|
|
170
|
+
`TensorCostModelGovernanceDeniedError` ("request refused at govern layer")
|
|
171
|
+
for every `/v1/admit` refusal except a run stop, budget caps included.
|
|
172
|
+
|
|
173
|
+
**Streaming proxy errors.** After retries ran out, a 5xx or 429 on the
|
|
174
|
+
streaming path was reported as `TensorCostProviderError`; it is
|
|
175
|
+
`TensorCostProxyError` / `TensorCostQuotaError` now.
|
|
176
|
+
|
|
177
|
+
### Not in 1.5.0: `application` and `tags` on observations
|
|
178
|
+
|
|
179
|
+
The server's observation endpoint reads `application` and `tags`, and the
|
|
180
|
+
SDK sends them only as `x-tc-application` / `x-tc-tags` on proxied calls.
|
|
181
|
+
Adding them to observations is built but switched off: byoc-v0.1.40
|
|
182
|
+
validates observations strictly and doesn't know the fields, so it would
|
|
183
|
+
reject every observation that carried them. They come in a later release, alongside the server release that accepts
|
|
184
|
+
them.
|
|
185
|
+
|
|
186
|
+
### Compatibility
|
|
187
|
+
|
|
188
|
+
- Server: no minimum version. 1.5.0 reads both the money-budget shape the
|
|
189
|
+
proxy sends since the change that added `budget_code` and the older
|
|
190
|
+
`code: "BUDGET_HARD_CAP"` shape. Nothing new is sent to the server, so
|
|
191
|
+
byoc-v0.1.40 keeps accepting every request and observation.
|
|
192
|
+
- `wrap()` at `govern`, `steer` or `route` without a tenant id throws
|
|
193
|
+
`MissingConfigError` (see above). Set `TENSORCOST_TENANT_ID`.
|
|
194
|
+
- `shouldFallBackToProvider()` returns false for `TensorCostProviderError`
|
|
195
|
+
and for errors that aren't `TensorCost*` outages (1.4.0 returned true).
|
|
196
|
+
- A relayed provider 4xx with an openai@4 client: in 1.4.0 the fail-open
|
|
197
|
+
retry crashed with the `defaultHeaders` TypeError above, so callers never
|
|
198
|
+
got `openai.BadRequestError` from it; with fail-open off they got
|
|
199
|
+
`TensorCostProxyError`. 1.5.0 throws the openai error type the client
|
|
200
|
+
builds, after one call.
|
|
201
|
+
- A TensorCost 4xx that isn't an outage (a rejected key, a refusal with an
|
|
202
|
+
unknown code, a firewall page) now stops the call instead of going around
|
|
203
|
+
it.
|
|
204
|
+
- `failOpenEnabled: false` now also fails the call when `/v1/admit` is down.
|
|
205
|
+
- `AdmitResult.refusal` is set on every refusal, not only `RUN_STOPPED`, and
|
|
206
|
+
can be a `TensorCostGatewayRejectedError` or `TensorCostKeyRejectedError`.
|
|
207
|
+
- Anthropic clients configured for steer/route make one `/v1/admit` call per
|
|
208
|
+
request instead of a proxy call.
|
|
209
|
+
- `wrap()` no longer sets `baseURL`, `defaultHeaders`, `x-tc-provider-url` or
|
|
210
|
+
`x-tc-provider-auth` on the client. If you read those back, read your own
|
|
211
|
+
config instead.
|
|
212
|
+
|
|
213
|
+
## 1.4.0 — 2026-10-25
|
|
214
|
+
|
|
215
|
+
### Added
|
|
216
|
+
|
|
217
|
+
**`TensorCostRunStoppedError`** — maps proxy `403` / `RUN_STOPPED` (an operator
|
|
218
|
+
stopped the agent run). Carries `workflowId`, `stoppedBy`, `stoppedAt` and
|
|
219
|
+
`reason`. Terminal: it never falls back to the provider, so a stopped run
|
|
220
|
+
stays stopped. Before this, a stop surfaced as a generic provider error and
|
|
221
|
+
applied mode's fail-open path would call the provider directly.
|
|
222
|
+
|
|
223
|
+
### Fixed
|
|
224
|
+
|
|
225
|
+
**Govern layer: a stopped run throws `TensorCostRunStoppedError`.** The proxy
|
|
226
|
+
now refuses a stopped run on `/v1/admit` too. The OpenAI and Anthropic
|
|
227
|
+
wrappers throw the stop with its detail instead of a generic "refused at
|
|
228
|
+
govern layer" governance error, and never call the provider.
|
|
4
229
|
|
|
5
230
|
## 1.3.0 — 2026-09-26
|
|
6
231
|
|
package/README.md
CHANGED
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
|
|
3
3
|
One-line wrapper for TensorCost — fire-and-forget LLM observability and optional inference control for OpenAI, Anthropic, AWS Bedrock, and Google Vertex AI clients in Node.js.
|
|
4
4
|
|
|
5
|
-
**Current version:** 1.
|
|
5
|
+
**Current version:** 1.5.0 — fixes 1.4.0 routing around a money budget's hard cap when fail-open was on (the default); upgrade. See [CHANGELOG](./CHANGELOG.md), [SDK compliance](https://github.com/vaadhlabs/tensorcost/blob/main/docs/features/sdk-compliance.md), and [Migrating from 0.x](#migrating-from-0x).
|
|
6
6
|
|
|
7
7
|
## Install
|
|
8
8
|
|
|
@@ -32,6 +32,7 @@ import { wrap } from "@tensorcost/sdk";
|
|
|
32
32
|
|
|
33
33
|
const client = wrap(new OpenAI({ apiKey: process.env.OPENAI_API_KEY }), {
|
|
34
34
|
apiKey: process.env.TENSORCOST_API_KEY,
|
|
35
|
+
tenantId: process.env.TENSORCOST_TENANT_ID,
|
|
35
36
|
});
|
|
36
37
|
|
|
37
38
|
const resp = await client.chat.completions.create({
|
|
@@ -48,6 +49,7 @@ import { wrap } from "@tensorcost/sdk";
|
|
|
48
49
|
|
|
49
50
|
const client = wrap(new Anthropic({ apiKey: process.env.ANTHROPIC_API_KEY }), {
|
|
50
51
|
apiKey: process.env.TENSORCOST_API_KEY,
|
|
52
|
+
tenantId: process.env.TENSORCOST_TENANT_ID,
|
|
51
53
|
});
|
|
52
54
|
|
|
53
55
|
await client.messages.create({
|
|
@@ -84,6 +86,7 @@ When routing is paused in the console, the published ceiling is capped at `gover
|
|
|
84
86
|
```ts
|
|
85
87
|
const client = wrap(new OpenAI({ apiKey: process.env.OPENAI_API_KEY }), {
|
|
86
88
|
apiKey: process.env.TENSORCOST_API_KEY,
|
|
89
|
+
tenantId: process.env.TENSORCOST_TENANT_ID,
|
|
87
90
|
maxLayer: "route",
|
|
88
91
|
proxyUrl: process.env.TENSORCOST_PROXY_URL, // required for steer | route
|
|
89
92
|
});
|
|
@@ -109,7 +112,7 @@ At `govern` and above, the SDK calls `POST /api/inference-proxy/v1/admit` with m
|
|
|
109
112
|
- **204** + `x-tc-decision: layer=govern; action=allow` → proceed to the provider
|
|
110
113
|
- **403** + `x-tc-decision: layer=govern; action=refuse; reason=…` → call blocked (budget, model governance, etc.)
|
|
111
114
|
|
|
112
|
-
Admit is **fail-open** on
|
|
115
|
+
Admit is **fail-open** on outages only (network error, timeout, 5xx, or a 429 with no refusal body), and only while `failOpenEnabled` is on: the provider call proceeds and an `admit_unavailable` observation is emitted. Any other non-2xx refuses the call with a typed `TensorCostRefusedError`. The admit HTTP call is capped at **500ms**.
|
|
113
116
|
|
|
114
117
|
### Compliance profiles (1.2.0+)
|
|
115
118
|
|
|
@@ -144,12 +147,14 @@ import { prewarm, wrap } from "@tensorcost/sdk";
|
|
|
144
147
|
|
|
145
148
|
await prewarm({
|
|
146
149
|
apiKey: process.env.TENSORCOST_API_KEY,
|
|
150
|
+
tenantId: process.env.TENSORCOST_TENANT_ID,
|
|
147
151
|
baseUrl: process.env.TENSORCOST_BASE_URL,
|
|
148
152
|
maxLayer: "route",
|
|
149
153
|
});
|
|
150
154
|
|
|
151
155
|
export const client = wrap(new OpenAI({ apiKey: process.env.OPENAI_API_KEY }), {
|
|
152
156
|
apiKey: process.env.TENSORCOST_API_KEY,
|
|
157
|
+
tenantId: process.env.TENSORCOST_TENANT_ID,
|
|
153
158
|
maxLayer: "route",
|
|
154
159
|
proxyUrl: process.env.TENSORCOST_PROXY_URL,
|
|
155
160
|
});
|
|
@@ -184,7 +189,7 @@ Recommended rollout: **`observe` → `govern` → `route`**, raising the console
|
|
|
184
189
|
| `TENSORCOST_API_KEY` | Long-lived SDK key from **Setup → Keys** | Yes |
|
|
185
190
|
| `TENSORCOST_BASE_URL` | Backend base URL (admit, sdk-layer, token exchange) | No (default `https://api.tensorcost.com`) |
|
|
186
191
|
| `TENSORCOST_PROXY_URL` | Inference-proxy base URL | Required when `maxLayer` is `steer` or `route` |
|
|
187
|
-
| `TENSORCOST_TENANT_ID` |
|
|
192
|
+
| `TENSORCOST_TENANT_ID` | Your tenant id, shown in the console under **Developers › SDK setup**. The token exchange needs it: without it the SDK never reaches TensorCost | **Yes** (`wrap()` throws `MissingConfigError` at `govern` and above without it; `observe` warns once and sends nothing) |
|
|
188
193
|
| `TENSORCOST_ENVIRONMENT` | Environment tag on every observation | No |
|
|
189
194
|
| `TENSORCOST_CONNECTION_ID` | Provider-connection id for environment resolution | No |
|
|
190
195
|
| `TENSORCOST_AGENT_ID` / `TENSORCOST_WORKFLOW_ID` | Agent-run attribution defaults | No |
|
|
@@ -240,13 +245,13 @@ const client = wrap(new OpenAI(...), {
|
|
|
240
245
|
|
|
241
246
|
| Option | Default | `TensorCostTimeoutError.kind` | On timeout (steer/route) |
|
|
242
247
|
|---|---|---|---|
|
|
243
|
-
| `headersTimeoutMs` | 2s | `"headers"` | Streaming
|
|
248
|
+
| `headersTimeoutMs` | 2s | `"headers"` | Streaming (`stream: true`) proxy calls; **not used for buffered JSON** |
|
|
244
249
|
| `timeoutMs` | 60s | `"total"` | TTFB + body for non-streaming proxy calls; fail-open when `failOpenEnabled: true` |
|
|
245
|
-
| `idleTimeoutMs` | 30s | `"idle"` |
|
|
250
|
+
| `idleTimeoutMs` | 30s | `"idle"` | Streaming proxy calls: no bytes for this long ends the stream |
|
|
246
251
|
|
|
247
252
|
For **non-streaming** steer/route, the SDK waits up to **`timeoutMs` for response headers** because the proxy may buffer the full upstream response before sending headers — using a 2s headers cap would fail-open and **double-bill** while the proxy still completes.
|
|
248
253
|
|
|
249
|
-
**Streaming (`stream: true`):**
|
|
254
|
+
**Streaming (`stream: true`):** OpenAI chat completions stream through the proxy at steer/route; the SDK reads the proxy's SSE and hands you the chunks. Before the stream starts, refusals and outages are handled like a buffered call (a refusal is terminal; an outage fails open when `failOpenEnabled` is on). Once bytes are flowing, an idle gap longer than `idleTimeoutMs` ends the stream with `TensorCostTimeoutError` (`"idle"`).
|
|
250
255
|
|
|
251
256
|
### Lifecycle hooks
|
|
252
257
|
|
|
@@ -283,6 +288,8 @@ import {
|
|
|
283
288
|
TensorCostProxyError,
|
|
284
289
|
TensorCostQuotaError,
|
|
285
290
|
TensorCostProviderError,
|
|
291
|
+
TensorCostRefusedError, // base of every refusal below; unknown codes arrive as this
|
|
292
|
+
TensorCostBudgetHardCapError,
|
|
286
293
|
TensorCostRunBudgetExceededError,
|
|
287
294
|
TensorCostPeriodBudgetExceededError,
|
|
288
295
|
TensorCostModelGovernanceDeniedError,
|
|
@@ -290,6 +297,10 @@ import {
|
|
|
290
297
|
} from "@tensorcost/sdk";
|
|
291
298
|
```
|
|
292
299
|
|
|
300
|
+
`TensorCostBudgetHardCapError` carries `budgetId`, `reason`, `retryable` (true for
|
|
301
|
+
`reason: "spend_not_known_yet"`, a budget whose spend hasn't been counted yet) and
|
|
302
|
+
`retryAfterSeconds`. The SDK doesn't retry it for you.
|
|
303
|
+
|
|
293
304
|
Govern refusals and proxy decisions are also reflected in the `x-tc-decision` header and attached to observations as `decision_action` / `decision_reason` metadata when present.
|
|
294
305
|
|
|
295
306
|
### Period budgets (1.3.0+)
|
|
@@ -314,6 +325,7 @@ try {
|
|
|
314
325
|
```
|
|
315
326
|
|
|
316
327
|
Per-run caps still raise `TensorCostRunBudgetExceededError` (`RUN_BUDGET_EXCEEDED`).
|
|
328
|
+
A run an operator has stopped raises `TensorCostRunStoppedError` (`RUN_STOPPED`) with `stoppedBy`, `stoppedAt` and `reason`; like the cap errors it never fails open to the provider.
|
|
317
329
|
|
|
318
330
|
## Data grants (`maxGrants`)
|
|
319
331
|
|
|
@@ -372,13 +384,16 @@ an agent link still attributes by `agentId` / `workflowId` alone.
|
|
|
372
384
|
|
|
373
385
|
## Fail-open guarantee
|
|
374
386
|
|
|
375
|
-
If TensorCost is slow
|
|
387
|
+
If TensorCost is slow or down, your customer call still succeeds. Internal errors are swallowed and logged via `console.warn` (or your lifecycle hook).
|
|
388
|
+
|
|
389
|
+
Since 1.5.0 "down" is an allow-list: a call goes around TensorCost only on a network error, a timeout, a 5xx, or a 429 or 402 without a refusal body, and only with `failOpenEnabled` on. Everything else is an answer and reaches your code:
|
|
376
390
|
|
|
377
391
|
- Missing `TENSORCOST_API_KEY` → `MissingConfigError` at `wrap()` time
|
|
392
|
+
- Missing `TENSORCOST_TENANT_ID` when `maxLayer` is `govern`, `steer` or `route` → `MissingConfigError` at `wrap()` time (1.5.0)
|
|
378
393
|
- Missing `proxyUrl` when `maxLayer` is `steer` or `route` → `TensorCostConfigError` at `wrap()` time
|
|
379
|
-
-
|
|
380
|
-
-
|
|
381
|
-
-
|
|
394
|
+
- Any TensorCost refusal (budget, hard cap, run stop, governance, guardrail, compliance, or a code this SDK version doesn't know) → a `TensorCostRefusedError` subclass, never retried, never fail-opened
|
|
395
|
+
- A provider's own 4xx relayed by the proxy → the provider SDK's own error (`openai.BadRequestError` and so on), built by your client; `TensorCostProviderError` with `status` and `body` when the client can't build one
|
|
396
|
+
- A 4xx that isn't JSON (a firewall page) → `TensorCostGatewayRejectedError`; a rejected TensorCost key → `TensorCostKeyRejectedError`
|
|
382
397
|
|
|
383
398
|
### Duplicate spend (honest)
|
|
384
399
|
|
|
@@ -430,17 +445,19 @@ Proxied requests include `x-tc-sdk-version`, `x-tc-sdk-capabilities`, and `x-tc-
|
|
|
430
445
|
|
|
431
446
|
| Provider | Package | Layers | Operations |
|
|
432
447
|
|---|---|---|---|
|
|
433
|
-
| OpenAI | `openai` ≥ 4.x | observe → route | chat,
|
|
434
|
-
| Anthropic | `@anthropic-ai/sdk` | observe
|
|
448
|
+
| OpenAI | `openai` ≥ 4.x | observe → route | `chat.completions` (observe, govern, steer/route, streaming); legacy `completions` (observed only) |
|
|
449
|
+
| Anthropic | `@anthropic-ai/sdk` | observe, govern (steer/route run as govern) | `messages` |
|
|
435
450
|
| Azure OpenAI | `openai` (`AzureOpenAI`) | observe → route | same as OpenAI (`provider: "openai"`) |
|
|
436
|
-
| AWS Bedrock | `@aws-sdk/client-bedrock-runtime` | observe
|
|
437
|
-
| Google Vertex | `@google/genai` (`vertexai: true`) | observe
|
|
451
|
+
| AWS Bedrock | `@aws-sdk/client-bedrock-runtime` | observe (govern accepted, not enforced yet) | InvokeModel, Converse (token placeholders) |
|
|
452
|
+
| Google Vertex | `@google/genai` (`vertexai: true`) | observe (govern accepted, not enforced yet) | generateContent (token placeholders) |
|
|
453
|
+
|
|
454
|
+
Nothing else on the client is wrapped: `embeddings`, `responses` and the rest go straight to the provider and are not observed.
|
|
438
455
|
|
|
439
|
-
**Steer/route streaming:**
|
|
456
|
+
**Steer/route streaming:** OpenAI `stream: true` chat completions go through the proxy's SSE (see Timeouts above). Anthropic streams go to Anthropic after the `/admit` check.
|
|
440
457
|
|
|
441
458
|
Full matrix: [`docs/demo/data-path-one-pager.md`](https://github.com/vaadhlabs/tensorcost/blob/main/docs/demo/data-path-one-pager.md).
|
|
442
459
|
|
|
443
|
-
**Bedrock and Vertex:** SigV4 / Vertex signing happens inside the AWS / Google SDK before the proxy can intercept — **`steer` and `route` are not supported**.
|
|
460
|
+
**Bedrock and Vertex:** SigV4 / Vertex signing happens inside the AWS / Google SDK before the proxy can intercept — **`steer` and `route` are not supported**. `govern` is accepted but there is no `/admit` check for Bedrock or Vertex yet, so calls are observed only and stops, caps and budgets are not enforced; the SDK warns once when you set it.
|
|
444
461
|
|
|
445
462
|
Bedrock peer dependency is optional:
|
|
446
463
|
|
|
@@ -450,6 +467,7 @@ import { wrap } from "@tensorcost/sdk";
|
|
|
450
467
|
|
|
451
468
|
const client = wrap(new BedrockRuntimeClient({ region: "us-east-1" }), {
|
|
452
469
|
apiKey: process.env.TENSORCOST_API_KEY,
|
|
470
|
+
tenantId: process.env.TENSORCOST_TENANT_ID,
|
|
453
471
|
});
|
|
454
472
|
```
|
|
455
473
|
|
|
@@ -463,6 +481,7 @@ import { wrapLangChain } from "@tensorcost/sdk";
|
|
|
463
481
|
|
|
464
482
|
const llm = wrapLangChain(new ChatOpenAI({ model: "gpt-4o-mini" }), {
|
|
465
483
|
apiKey: process.env.TENSORCOST_API_KEY,
|
|
484
|
+
tenantId: process.env.TENSORCOST_TENANT_ID,
|
|
466
485
|
maxLayer: "observe",
|
|
467
486
|
});
|
|
468
487
|
```
|
|
@@ -0,0 +1,56 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* Govern-layer admit — metadata-only POST /api/inference-proxy/v1/admit.
|
|
3
|
+
*
|
|
4
|
+
* Fail-open only on an outage: network error, timeout, 5xx, or a 429 / 402
|
|
5
|
+
* with no refusal body. The customer's call then proceeds to the provider and
|
|
6
|
+
* an `admit_unavailable` observation is emitted when transport is set.
|
|
7
|
+
*
|
|
8
|
+
* Every other non-2xx is an answer, and the result carries it as
|
|
9
|
+
* `refusal`, a TensorCostRefusedError subclass the wrapper throws as is:
|
|
10
|
+
* RUN_STOPPED with its who / when / why, BUDGET_HARD_CAP as the budget
|
|
11
|
+
* error, a code this SDK doesn't know as a plain TensorCostRefusedError.
|
|
12
|
+
* /admit only answers 403 when a gate refused, so a 403 with no readable
|
|
13
|
+
* code is still a refusal. A 401/400/404 means admission could not be
|
|
14
|
+
* checked for this call; that is not an outage either, so it refuses too.
|
|
15
|
+
*/
|
|
16
|
+
import type { ControlLayer } from "./layer.js";
|
|
17
|
+
import type { Operation, Provider } from "./types.js";
|
|
18
|
+
import type { ObservationTransport } from "./transport.js";
|
|
19
|
+
import { TensorCostGatewayRejectedError, TensorCostKeyRejectedError, TensorCostRefusedError } from "./errors.js";
|
|
20
|
+
export interface AdmitMetadata {
|
|
21
|
+
provider: Provider;
|
|
22
|
+
model: string;
|
|
23
|
+
operation: Operation;
|
|
24
|
+
correlation_id: string;
|
|
25
|
+
environment?: string | null;
|
|
26
|
+
connection_id?: string | null;
|
|
27
|
+
agent_id?: string;
|
|
28
|
+
workflow_id?: string;
|
|
29
|
+
customer?: string;
|
|
30
|
+
feature?: string;
|
|
31
|
+
application?: string | null;
|
|
32
|
+
tags?: string[];
|
|
33
|
+
team_id?: string | null;
|
|
34
|
+
}
|
|
35
|
+
export interface AdmitRequestOptions {
|
|
36
|
+
baseUrl: string;
|
|
37
|
+
bearerToken: string;
|
|
38
|
+
maxLayer: ControlLayer;
|
|
39
|
+
metadata: AdmitMetadata;
|
|
40
|
+
transport?: ObservationTransport;
|
|
41
|
+
failOpen?: boolean;
|
|
42
|
+
timeoutMs?: number;
|
|
43
|
+
fetchImpl?: typeof fetch;
|
|
44
|
+
}
|
|
45
|
+
export interface AdmitResult {
|
|
46
|
+
allowed: boolean;
|
|
47
|
+
decisionHeader: string | null;
|
|
48
|
+
/**
|
|
49
|
+
* Set whenever `allowed` is false: the refusal, typed by its code, or a
|
|
50
|
+
* TensorCostGatewayRejectedError when something in front of TensorCost
|
|
51
|
+
* answered with a non-JSON 4xx.
|
|
52
|
+
*/
|
|
53
|
+
refusal?: TensorCostRefusedError | TensorCostGatewayRejectedError | TensorCostKeyRejectedError;
|
|
54
|
+
}
|
|
55
|
+
export declare function admitRequest(opts: AdmitRequestOptions): Promise<AdmitResult>;
|
|
56
|
+
//# sourceMappingURL=admit-client.d.ts.map
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"admit-client.d.ts","sourceRoot":"","sources":["../src/admit-client.ts"],"names":[],"mappings":"AAAA
|
|
1
|
+
{"version":3,"file":"admit-client.d.ts","sourceRoot":"","sources":["../src/admit-client.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;;;;GAcG;AAEH,OAAO,KAAK,EAAE,YAAY,EAAE,MAAM,YAAY,CAAC;AAC/C,OAAO,KAAK,EAAE,SAAS,EAAE,QAAQ,EAAE,MAAM,YAAY,CAAC;AACtD,OAAO,KAAK,EAAE,oBAAoB,EAAE,MAAM,gBAAgB,CAAC;AAG3D,OAAO,EACL,8BAA8B,EAC9B,0BAA0B,EAI1B,sBAAsB,EAEvB,MAAM,aAAa,CAAC;AAKrB,MAAM,WAAW,aAAa;IAC5B,QAAQ,EAAE,QAAQ,CAAC;IACnB,KAAK,EAAE,MAAM,CAAC;IACd,SAAS,EAAE,SAAS,CAAC;IACrB,cAAc,EAAE,MAAM,CAAC;IACvB,WAAW,CAAC,EAAE,MAAM,GAAG,IAAI,CAAC;IAC5B,aAAa,CAAC,EAAE,MAAM,GAAG,IAAI,CAAC;IAC9B,QAAQ,CAAC,EAAE,MAAM,CAAC;IAClB,WAAW,CAAC,EAAE,MAAM,CAAC;IACrB,QAAQ,CAAC,EAAE,MAAM,CAAC;IAClB,OAAO,CAAC,EAAE,MAAM,CAAC;IACjB,WAAW,CAAC,EAAE,MAAM,GAAG,IAAI,CAAC;IAC5B,IAAI,CAAC,EAAE,MAAM,EAAE,CAAC;IAChB,OAAO,CAAC,EAAE,MAAM,GAAG,IAAI,CAAC;CACzB;AAED,MAAM,WAAW,mBAAmB;IAClC,OAAO,EAAE,MAAM,CAAC;IAChB,WAAW,EAAE,MAAM,CAAC;IACpB,QAAQ,EAAE,YAAY,CAAC;IACvB,QAAQ,EAAE,aAAa,CAAC;IACxB,SAAS,CAAC,EAAE,oBAAoB,CAAC;IACjC,QAAQ,CAAC,EAAE,OAAO,CAAC;IACnB,SAAS,CAAC,EAAE,MAAM,CAAC;IACnB,SAAS,CAAC,EAAE,OAAO,KAAK,CAAC;CAC1B;AAED,MAAM,WAAW,WAAW;IAC1B,OAAO,EAAE,OAAO,CAAC;IACjB,cAAc,EAAE,MAAM,GAAG,IAAI,CAAC;IAC9B;;;;OAIG;IACH,OAAO,CAAC,EAAE,sBAAsB,GAAG,8BAA8B,GAAG,0BAA0B,CAAC;CAChG;AAED,wBAAsB,YAAY,CAChC,IAAI,EAAE,mBAAmB,GACxB,OAAO,CAAC,WAAW,CAAC,CAwFtB"}
|
package/dist/admit-client.js
CHANGED
|
@@ -1,11 +1,24 @@
|
|
|
1
1
|
/**
|
|
2
2
|
* Govern-layer admit — metadata-only POST /api/inference-proxy/v1/admit.
|
|
3
3
|
*
|
|
4
|
-
* Fail-open on
|
|
5
|
-
*
|
|
4
|
+
* Fail-open only on an outage: network error, timeout, 5xx, or a 429 / 402
|
|
5
|
+
* with no refusal body. The customer's call then proceeds to the provider and
|
|
6
|
+
* an `admit_unavailable` observation is emitted when transport is set.
|
|
7
|
+
*
|
|
8
|
+
* Every other non-2xx is an answer, and the result carries it as
|
|
9
|
+
* `refusal`, a TensorCostRefusedError subclass the wrapper throws as is:
|
|
10
|
+
* RUN_STOPPED with its who / when / why, BUDGET_HARD_CAP as the budget
|
|
11
|
+
* error, a code this SDK doesn't know as a plain TensorCostRefusedError.
|
|
12
|
+
* /admit only answers 403 when a gate refused, so a 403 with no readable
|
|
13
|
+
* code is still a refusal. A 401/400/404 means admission could not be
|
|
14
|
+
* checked for this call; that is not an outage either, so it refuses too.
|
|
6
15
|
*/
|
|
7
16
|
import { ADMIT_PATH, ADMIT_TIMEOUT_MS } from "./layer.js";
|
|
8
17
|
import { SDK_VERSION, sdkCapabilityHeader } from "./version.js";
|
|
18
|
+
import { TensorCostGatewayRejectedError, TensorCostKeyRejectedError, TensorCostNetworkError, TensorCostProxyError, TensorCostQuotaError, TensorCostRefusedError, TensorCostTimeoutError, } from "./errors.js";
|
|
19
|
+
import { retryAfterMs } from "./retry.js";
|
|
20
|
+
import { decisionSaysRefuse, parseRefusalBody, refusalError } from "./refusal.js";
|
|
21
|
+
import { observationScope } from "./wrap-overhead.js";
|
|
9
22
|
export async function admitRequest(opts) {
|
|
10
23
|
const fetchImpl = opts.fetchImpl ?? globalThis.fetch.bind(globalThis);
|
|
11
24
|
const url = `${opts.baseUrl.replace(/\/+$/, "")}${ADMIT_PATH}`;
|
|
@@ -36,18 +49,38 @@ export async function admitRequest(opts) {
|
|
|
36
49
|
const controller = new AbortController();
|
|
37
50
|
const timer = setTimeout(() => controller.abort(), opts.timeoutMs ?? ADMIT_TIMEOUT_MS);
|
|
38
51
|
try {
|
|
39
|
-
|
|
40
|
-
|
|
41
|
-
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
|
|
52
|
+
let resp;
|
|
53
|
+
try {
|
|
54
|
+
resp = await fetchImpl(url, {
|
|
55
|
+
method: "POST",
|
|
56
|
+
headers,
|
|
57
|
+
body: JSON.stringify(body),
|
|
58
|
+
signal: controller.signal,
|
|
59
|
+
});
|
|
60
|
+
}
|
|
61
|
+
catch (err) {
|
|
62
|
+
const e = err;
|
|
63
|
+
throw e?.name === "AbortError" || e?.name === "TimeoutError"
|
|
64
|
+
? new TensorCostTimeoutError(`tensorcost: admit timed out after ${opts.timeoutMs ?? ADMIT_TIMEOUT_MS}ms`, "total", { cause: e })
|
|
65
|
+
: new TensorCostNetworkError(`tensorcost: network error reaching admit: ${e?.message ?? String(err)}`, { cause: e });
|
|
66
|
+
}
|
|
45
67
|
const decisionHeader = resp.headers.get("x-tc-decision");
|
|
46
|
-
|
|
47
|
-
|
|
68
|
+
const requestId = resp.headers.get("x-tc-request-id");
|
|
69
|
+
if (resp.status >= 400 && resp.status < 500) {
|
|
70
|
+
const refusal = await refusalFrom4xx(resp, decisionHeader);
|
|
71
|
+
if (refusal)
|
|
72
|
+
return { allowed: false, decisionHeader, refusal };
|
|
73
|
+
// Only a bare 429 or 402 gets here.
|
|
74
|
+
throw new TensorCostQuotaError(`admit POST failed status=${resp.status}`, retryAfterMs(resp) ?? null, {
|
|
75
|
+
status: resp.status,
|
|
76
|
+
requestId,
|
|
77
|
+
});
|
|
48
78
|
}
|
|
49
79
|
if (!resp.ok) {
|
|
50
|
-
throw new
|
|
80
|
+
throw new TensorCostProxyError(`admit POST failed status=${resp.status}`, {
|
|
81
|
+
status: resp.status,
|
|
82
|
+
requestId,
|
|
83
|
+
});
|
|
51
84
|
}
|
|
52
85
|
return { allowed: true, decisionHeader };
|
|
53
86
|
}
|
|
@@ -62,6 +95,54 @@ export async function admitRequest(opts) {
|
|
|
62
95
|
clearTimeout(timer);
|
|
63
96
|
}
|
|
64
97
|
}
|
|
98
|
+
/**
|
|
99
|
+
* The refusal in a 4xx from /admit, or undefined for the one 4xx that is
|
|
100
|
+
* an outage: a 429 (the tenant throttler) or 402 (plan ceiling) with no
|
|
101
|
+
* refusal body.
|
|
102
|
+
*/
|
|
103
|
+
async function refusalFrom4xx(resp, decisionHeader) {
|
|
104
|
+
const text = await resp.text().catch(() => "");
|
|
105
|
+
const parsed = parseRefusalBody(text);
|
|
106
|
+
if (parsed === null &&
|
|
107
|
+
(resp.status === 429 || resp.status === 402) &&
|
|
108
|
+
!decisionSaysRefuse(decisionHeader)) {
|
|
109
|
+
return undefined;
|
|
110
|
+
}
|
|
111
|
+
const ctx = {
|
|
112
|
+
status: resp.status,
|
|
113
|
+
requestId: resp.headers.get("x-tc-request-id"),
|
|
114
|
+
attempt: 1,
|
|
115
|
+
retryAfterHeader: resp.headers.get("retry-after"),
|
|
116
|
+
};
|
|
117
|
+
if (parsed !== null)
|
|
118
|
+
return refusalError(parsed, ctx);
|
|
119
|
+
let body;
|
|
120
|
+
try {
|
|
121
|
+
body = JSON.parse(text);
|
|
122
|
+
}
|
|
123
|
+
catch {
|
|
124
|
+
body = undefined;
|
|
125
|
+
}
|
|
126
|
+
if (body === undefined || body === null || typeof body !== "object") {
|
|
127
|
+
if (decisionSaysRefuse(decisionHeader))
|
|
128
|
+
return refusalError(null, ctx);
|
|
129
|
+
// HTML or nothing: something in front of TensorCost said no.
|
|
130
|
+
return new TensorCostGatewayRejectedError(resp.status, text ? text.slice(0, 500) : null, {
|
|
131
|
+
requestId: ctx.requestId,
|
|
132
|
+
});
|
|
133
|
+
}
|
|
134
|
+
if (resp.status === 403)
|
|
135
|
+
return refusalError(null, ctx);
|
|
136
|
+
if (resp.status === 401) {
|
|
137
|
+
// /admit rejected the TensorCost token.
|
|
138
|
+
return new TensorCostKeyRejectedError(401, { requestId: ctx.requestId });
|
|
139
|
+
}
|
|
140
|
+
const rec = body;
|
|
141
|
+
const message = typeof rec["message"] === "string" ? rec["message"] : null;
|
|
142
|
+
return new TensorCostRefusedError(`tensorcost: admission check returned ${resp.status}` +
|
|
143
|
+
(message ? ` (${message})` : "") +
|
|
144
|
+
"; the call was not sent", { code: null }, { status: resp.status, requestId: ctx.requestId, attempt: 1 });
|
|
145
|
+
}
|
|
65
146
|
function postAdmitFailureObservation(opts, err) {
|
|
66
147
|
if (!opts.transport)
|
|
67
148
|
return;
|
|
@@ -81,6 +162,7 @@ function postAdmitFailureObservation(opts, err) {
|
|
|
81
162
|
correlation_id: opts.metadata.correlation_id,
|
|
82
163
|
...(opts.metadata.environment ? { environment: opts.metadata.environment } : {}),
|
|
83
164
|
...(opts.metadata.connection_id ? { connection_id: opts.metadata.connection_id } : {}),
|
|
165
|
+
...observationScope(opts.metadata.application, opts.metadata.tags),
|
|
84
166
|
metadata: {
|
|
85
167
|
decision_action: "allow",
|
|
86
168
|
decision_reason: "admit_unavailable",
|
package/dist/admit-client.js.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"admit-client.js","sourceRoot":"","sources":["../src/admit-client.ts"],"names":[],"mappings":"AAAA
|
|
1
|
+
{"version":3,"file":"admit-client.js","sourceRoot":"","sources":["../src/admit-client.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;;;;GAcG;AAKH,OAAO,EAAE,UAAU,EAAE,gBAAgB,EAAE,MAAM,YAAY,CAAC;AAC1D,OAAO,EAAE,WAAW,EAAE,mBAAmB,EAAE,MAAM,cAAc,CAAC;AAChE,OAAO,EACL,8BAA8B,EAC9B,0BAA0B,EAC1B,sBAAsB,EACtB,oBAAoB,EACpB,oBAAoB,EACpB,sBAAsB,EACtB,sBAAsB,GACvB,MAAM,aAAa,CAAC;AACrB,OAAO,EAAE,YAAY,EAAE,MAAM,YAAY,CAAC;AAC1C,OAAO,EAAE,kBAAkB,EAAE,gBAAgB,EAAE,YAAY,EAAE,MAAM,cAAc,CAAC;AAClF,OAAO,EAAE,gBAAgB,EAAE,MAAM,oBAAoB,CAAC;AAwCtD,MAAM,CAAC,KAAK,UAAU,YAAY,CAChC,IAAyB;IAEzB,MAAM,SAAS,GAAG,IAAI,CAAC,SAAS,IAAI,UAAU,CAAC,KAAK,CAAC,IAAI,CAAC,UAAU,CAAC,CAAC;IACtE,MAAM,GAAG,GAAG,GAAG,IAAI,CAAC,OAAO,CAAC,OAAO,CAAC,MAAM,EAAE,EAAE,CAAC,GAAG,UAAU,EAAE,CAAC;IAC/D,MAAM,OAAO,GAA2B;QACtC,aAAa,EAAE,UAAU,IAAI,CAAC,WAAW,EAAE;QAC3C,cAAc,EAAE,kBAAkB;QAClC,kBAAkB,EAAE,WAAW;QAC/B,uBAAuB,EAAE,mBAAmB,EAAE;QAC9C,gBAAgB,EAAE,IAAI,CAAC,QAAQ;KAChC,CAAC;IAEF,MAAM,IAAI,GAAG;QACX,QAAQ,EAAE,IAAI,CAAC,QAAQ,CAAC,QAAQ;QAChC,KAAK,EAAE,IAAI,CAAC,QAAQ,CAAC,KAAK;QAC1B,SAAS,EAAE,IAAI,CAAC,QAAQ,CAAC,SAAS;QAClC,cAAc,EAAE,IAAI,CAAC,QAAQ,CAAC,cAAc;QAC5C,GAAG,CAAC,IAAI,CAAC,QAAQ,CAAC,WAAW,CAAC,CAAC,CAAC,EAAE,WAAW,EAAE,IAAI,CAAC,QAAQ,CAAC,WAAW,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC;QAChF,GAAG,CAAC,IAAI,CAAC,QAAQ,CAAC,aAAa,CAAC,CAAC,CAAC,EAAE,aAAa,EAAE,IAAI,CAAC,QAAQ,CAAC,aAAa,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC;QACtF,GAAG,CAAC,IAAI,CAAC,QAAQ,CAAC,QAAQ,CAAC,CAAC,CAAC,EAAE,QAAQ,EAAE,IAAI,CAAC,QAAQ,CAAC,QAAQ,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC;QACvE,GAAG,CAAC,IAAI,CAAC,QAAQ,CAAC,WAAW,CAAC,CAAC,CAAC,EAAE,WAAW,EAAE,IAAI,CAAC,QAAQ,CAAC,WAAW,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC;QAChF,GAAG,CAAC,IAAI,CAAC,QAAQ,CAAC,QAAQ,CAAC,CAAC,CAAC,EAAE,QAAQ,EAAE,IAAI,CAAC,QAAQ,CAAC,QAAQ,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC;QACvE,GAAG,CAAC,IAAI,CAAC,QAAQ,CAAC,OAAO,CAAC,CAAC,CAAC,EAAE,OAAO,EAAE,IAAI,CAAC,QAAQ,CAAC,OAAO,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC;QACpE,GAAG,CAAC,IAAI,CAAC,QAAQ,CAAC,WAAW,CAAC,CAAC,CAAC,EAAE,WAAW,EAAE,IAAI,CAAC,QAAQ,CAAC,WAAW,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC;QAChF,GAAG,CAAC,IAAI,CAAC,QAAQ,CAAC,IAAI,IAAI,IAAI,CAAC,QAAQ,CAAC,IAAI,CAAC,MAAM,GAAG,CAAC;YACrD,CAAC,CAAC,EAAE,IAAI,EAAE,IAAI,CAAC,QAAQ,CAAC,IAAI,CAAC,IAAI,CAAC,GAAG,CAAC,EAAE;YACxC,CAAC,CAAC,EAAE,CAAC;QACP,GAAG,CAAC,IAAI,CAAC,QAAQ,CAAC,OAAO,CAAC,CAAC,CAAC,EAAE,OAAO,EAAE,IAAI,CAAC,QAAQ,CAAC,OAAO,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC;KACrE,CAAC;IAEF,MAAM,UAAU,GAAG,IAAI,eAAe,EAAE,CAAC;IACzC,MAAM,KAAK,GAAG,UAAU,CACtB,GAAG,EAAE,CAAC,UAAU,CAAC,KAAK,EAAE,EACxB,IAAI,CAAC,SAAS,IAAI,gBAAgB,CACnC,CAAC;IAEF,IAAI,CAAC;QACH,IAAI,IAAc,CAAC;QACnB,IAAI,CAAC;YACH,IAAI,GAAG,MAAM,SAAS,CAAC,GAAG,EAAE;gBAC1B,MAAM,EAAE,MAAM;gBACd,OAAO;gBACP,IAAI,EAAE,IAAI,CAAC,SAAS,CAAC,IAAI,CAAC;gBAC1B,MAAM,EAAE,UAAU,CAAC,MAAM;aAC1B,CAAC,CAAC;QACL,CAAC;QAAC,OAAO,GAAG,EAAE,CAAC;YACb,MAAM,CAAC,GAAG,GAAY,CAAC;YACvB,MAAM,CAAC,EAAE,IAAI,KAAK,YAAY,IAAI,CAAC,EAAE,IAAI,KAAK,cAAc;gBAC1D,CAAC,CAAC,IAAI,sBAAsB,CACxB,qCAAqC,IAAI,CAAC,SAAS,IAAI,gBAAgB,IAAI,EAC3E,OAAO,EACP,EAAE,KAAK,EAAE,CAAC,EAAE,CACb;gBACH,CAAC,CAAC,IAAI,sBAAsB,CACxB,6CAA6C,CAAC,EAAE,OAAO,IAAI,MAAM,CAAC,GAAG,CAAC,EAAE,EACxE,EAAE,KAAK,EAAE,CAAC,EAAE,CACb,CAAC;QACR,CAAC;QAED,MAAM,cAAc,GAAG,IAAI,CAAC,OAAO,CAAC,GAAG,CAAC,eAAe,CAAC,CAAC;QACzD,MAAM,SAAS,GAAG,IAAI,CAAC,OAAO,CAAC,GAAG,CAAC,iBAAiB,CAAC,CAAC;QAEtD,IAAI,IAAI,CAAC,MAAM,IAAI,GAAG,IAAI,IAAI,CAAC,MAAM,GAAG,GAAG,EAAE,CAAC;YAC5C,MAAM,OAAO,GAAG,MAAM,cAAc,CAAC,IAAI,EAAE,cAAc,CAAC,CAAC;YAC3D,IAAI,OAAO;gBAAE,OAAO,EAAE,OAAO,EAAE,KAAK,EAAE,cAAc,EAAE,OAAO,EAAE,CAAC;YAChE,oCAAoC;YACpC,MAAM,IAAI,oBAAoB,CAAC,4BAA4B,IAAI,CAAC,MAAM,EAAE,EAAE,YAAY,CAAC,IAAI,CAAC,IAAI,IAAI,EAAE;gBACpG,MAAM,EAAE,IAAI,CAAC,MAAM;gBACnB,SAAS;aACV,CAAC,CAAC;QACL,CAAC;QAED,IAAI,CAAC,IAAI,CAAC,EAAE,EAAE,CAAC;YACb,MAAM,IAAI,oBAAoB,CAAC,4BAA4B,IAAI,CAAC,MAAM,EAAE,EAAE;gBACxE,MAAM,EAAE,IAAI,CAAC,MAAM;gBACnB,SAAS;aACV,CAAC,CAAC;QACL,CAAC;QAED,OAAO,EAAE,OAAO,EAAE,IAAI,EAAE,cAAc,EAAE,CAAC;IAC3C,CAAC;IAAC,OAAO,GAAG,EAAE,CAAC;QACb,IAAI,IAAI,CAAC,QAAQ,KAAK,KAAK,EAAE,CAAC;YAC5B,2BAA2B,CAAC,IAAI,EAAE,GAAY,CAAC,CAAC;YAChD,OAAO,EAAE,OAAO,EAAE,IAAI,EAAE,cAAc,EAAE,IAAI,EAAE,CAAC;QACjD,CAAC;QACD,MAAM,GAAG,CAAC;IACZ,CAAC;YAAS,CAAC;QACT,YAAY,CAAC,KAAK,CAAC,CAAC;IACtB,CAAC;AACH,CAAC;AAED;;;;GAIG;AACH,KAAK,UAAU,cAAc,CAC3B,IAAc,EACd,cAA6B;IAI7B,MAAM,IAAI,GAAG,MAAM,IAAI,CAAC,IAAI,EAAE,CAAC,KAAK,CAAC,GAAG,EAAE,CAAC,EAAE,CAAC,CAAC;IAC/C,MAAM,MAAM,GAAG,gBAAgB,CAAC,IAAI,CAAC,CAAC;IACtC,IACE,MAAM,KAAK,IAAI;QACf,CAAC,IAAI,CAAC,MAAM,KAAK,GAAG,IAAI,IAAI,CAAC,MAAM,KAAK,GAAG,CAAC;QAC5C,CAAC,kBAAkB,CAAC,cAAc,CAAC,EACnC,CAAC;QACD,OAAO,SAAS,CAAC;IACnB,CAAC;IACD,MAAM,GAAG,GAAG;QACV,MAAM,EAAE,IAAI,CAAC,MAAM;QACnB,SAAS,EAAE,IAAI,CAAC,OAAO,CAAC,GAAG,CAAC,iBAAiB,CAAC;QAC9C,OAAO,EAAE,CAAC;QACV,gBAAgB,EAAE,IAAI,CAAC,OAAO,CAAC,GAAG,CAAC,aAAa,CAAC;KAClD,CAAC;IACF,IAAI,MAAM,KAAK,IAAI;QAAE,OAAO,YAAY,CAAC,MAAM,EAAE,GAAG,CAAC,CAAC;IACtD,IAAI,IAAa,CAAC;IAClB,IAAI,CAAC;QACH,IAAI,GAAG,IAAI,CAAC,KAAK,CAAC,IAAI,CAAY,CAAC;IACrC,CAAC;IAAC,MAAM,CAAC;QACP,IAAI,GAAG,SAAS,CAAC;IACnB,CAAC;IACD,IAAI,IAAI,KAAK,SAAS,IAAI,IAAI,KAAK,IAAI,IAAI,OAAO,IAAI,KAAK,QAAQ,EAAE,CAAC;QACpE,IAAI,kBAAkB,CAAC,cAAc,CAAC;YAAE,OAAO,YAAY,CAAC,IAAI,EAAE,GAAG,CAAC,CAAC;QACvE,6DAA6D;QAC7D,OAAO,IAAI,8BAA8B,CAAC,IAAI,CAAC,MAAM,EAAE,IAAI,CAAC,CAAC,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC,EAAE,GAAG,CAAC,CAAC,CAAC,CAAC,IAAI,EAAE;YACvF,SAAS,EAAE,GAAG,CAAC,SAAS;SACzB,CAAC,CAAC;IACL,CAAC;IACD,IAAI,IAAI,CAAC,MAAM,KAAK,GAAG;QAAE,OAAO,YAAY,CAAC,IAAI,EAAE,GAAG,CAAC,CAAC;IACxD,IAAI,IAAI,CAAC,MAAM,KAAK,GAAG,EAAE,CAAC;QACxB,wCAAwC;QACxC,OAAO,IAAI,0BAA0B,CAAC,GAAG,EAAE,EAAE,SAAS,EAAE,GAAG,CAAC,SAAS,EAAE,CAAC,CAAC;IAC3E,CAAC;IACD,MAAM,GAAG,GAAG,IAA+B,CAAC;IAC5C,MAAM,OAAO,GAAG,OAAO,GAAG,CAAC,SAAS,CAAC,KAAK,QAAQ,CAAC,CAAC,CAAC,GAAG,CAAC,SAAS,CAAC,CAAC,CAAC,CAAC,IAAI,CAAC;IAC3E,OAAO,IAAI,sBAAsB,CAC/B,wCAAwC,IAAI,CAAC,MAAM,EAAE;QACnD,CAAC,OAAO,CAAC,CAAC,CAAC,KAAK,OAAO,GAAG,CAAC,CAAC,CAAC,EAAE,CAAC;QAChC,yBAAyB,EAC3B,EAAE,IAAI,EAAE,IAAI,EAAE,EACd,EAAE,MAAM,EAAE,IAAI,CAAC,MAAM,EAAE,SAAS,EAAE,GAAG,CAAC,SAAS,EAAE,OAAO,EAAE,CAAC,EAAE,CAC9D,CAAC;AACJ,CAAC;AAED,SAAS,2BAA2B,CAClC,IAAyB,EACzB,GAAU;IAEV,IAAI,CAAC,IAAI,CAAC,SAAS;QAAE,OAAO;IAC5B,IAAI,CAAC;QACH,IAAI,CAAC,SAAS,CAAC,IAAI,CAAC;YAClB,WAAW,EAAE,WAAW;YACxB,QAAQ,EAAE,IAAI,CAAC,QAAQ,CAAC,QAAQ;YAChC,KAAK,EAAE,IAAI,CAAC,QAAQ,CAAC,KAAK;YAC1B,SAAS,EAAE,IAAI,CAAC,QAAQ,CAAC,SAAS;YAClC,UAAU,EAAE,IAAI,IAAI,EAAE,CAAC,WAAW,EAAE;YACpC,WAAW,EAAE,IAAI,IAAI,EAAE,CAAC,WAAW,EAAE;YACrC,YAAY,EAAE,IAAI;YAClB,aAAa,EAAE,IAAI;YACnB,cAAc,EAAE,IAAI;YACpB,MAAM,EAAE,SAAS;YACjB,aAAa,EAAE,IAAI;YACnB,cAAc,EAAE,IAAI,CAAC,QAAQ,CAAC,cAAc;YAC5C,GAAG,CAAC,IAAI,CAAC,QAAQ,CAAC,WAAW,CAAC,CAAC,CAAC,EAAE,WAAW,EAAE,IAAI,CAAC,QAAQ,CAAC,WAAW,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC;YAChF,GAAG,CAAC,IAAI,CAAC,QAAQ,CAAC,aAAa,CAAC,CAAC,CAAC,EAAE,aAAa,EAAE,IAAI,CAAC,QAAQ,CAAC,aAAa,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC;YACtF,GAAG,gBAAgB,CAAC,IAAI,CAAC,QAAQ,CAAC,WAAW,EAAE,IAAI,CAAC,QAAQ,CAAC,IAAI,CAAC;YAClE,QAAQ,EAAE;gBACR,eAAe,EAAE,OAAO;gBACxB,eAAe,EAAE,mBAAmB;aACrC;SACF,CAAC,CAAC;IACL,CAAC;IAAC,MAAM,CAAC;QACP,yBAAyB;IAC3B,CAAC;AACH,CAAC"}
|