@evalguard/vercel-ai 1.0.0 → 1.0.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +51 -0
- package/README.md +48 -4
- package/dist/guardrail-client.d.ts +41 -0
- package/dist/guardrail-client.d.ts.map +1 -0
- package/dist/guardrail-client.js +88 -0
- package/dist/guardrail-client.js.map +1 -0
- package/dist/index.d.ts +40 -10
- package/dist/index.d.ts.map +1 -1
- package/dist/index.js +420 -88
- package/dist/index.js.map +1 -1
- package/dist/surface.d.ts +126 -0
- package/dist/surface.d.ts.map +1 -0
- package/dist/surface.js +264 -0
- package/dist/surface.js.map +1 -0
- package/dist/version.d.ts +1 -1
- package/dist/version.js +1 -1
- package/package.json +14 -14
package/CHANGELOG.md
ADDED
|
@@ -0,0 +1,51 @@
|
|
|
1
|
+
# @evalguard/vercel-ai
|
|
2
|
+
|
|
3
|
+
This package wraps the Vercel AI SDK and routes every `checkInput` / `scoreOutput`
|
|
4
|
+
call through one shared transport, `@evalguard/wrapper-core`'s
|
|
5
|
+
`fetchWithTimeout`. A defect in that transport is a defect here.
|
|
6
|
+
|
|
7
|
+
## 1.0.2 — unreleased
|
|
8
|
+
|
|
9
|
+
### Security — a redirect defeated the guardrail verdict
|
|
10
|
+
|
|
11
|
+
**Affects every published version up to and including `1.0.1`.**
|
|
12
|
+
|
|
13
|
+
`fetchWithTimeout` called `fetch` with no redirect posture, and WHATWG/undici
|
|
14
|
+
default to `redirect: "follow"` — so an omitted option is not neutral, it is
|
|
15
|
+
the vulnerable setting.
|
|
16
|
+
|
|
17
|
+
A `3xx` served on the API path rewrote the request to a bodyless `GET`
|
|
18
|
+
(`301`/`302`/`303`), which means the prompt or completion you asked EvalGuard
|
|
19
|
+
to screen **was never transmitted**. The redirect target's
|
|
20
|
+
`{"blocked": false}` came back as an authoritative clean verdict on text it had
|
|
21
|
+
never seen, and the wrapper — sitting on the hot path of your request handler —
|
|
22
|
+
allowed it through. On `307`/`308` the body survives instead, so the text under
|
|
23
|
+
inspection is re-POSTed verbatim to whatever host the redirect names.
|
|
24
|
+
|
|
25
|
+
The existing reply validation could not catch it. It checks the *shape* of a
|
|
26
|
+
verdict, and a redirected reply is perfectly well shaped: a well-formed verdict
|
|
27
|
+
about nothing. `assertSecureBaseUrl` validates hop 0 only.
|
|
28
|
+
|
|
29
|
+
**This needs no attacker.** An ordinary infrastructure redirect on the API path
|
|
30
|
+
— an nginx trailing-slash rule, a load balancer, a corporate proxy, a proxy in
|
|
31
|
+
front of a self-hosted deployment — silently converts every guardrail check
|
|
32
|
+
behind it into an ALLOW.
|
|
33
|
+
|
|
34
|
+
Fixed in `@evalguard/wrapper-core@1.3.0`, which sets `redirect: "error"` on
|
|
35
|
+
that transport so the call fails **closed**.
|
|
36
|
+
|
|
37
|
+
### How the fix reaches you
|
|
38
|
+
|
|
39
|
+
The published `@evalguard/vercel-ai@1.0.1` declares
|
|
40
|
+
`"@evalguard/wrapper-core": "^1.2.0"`, so a **fresh** resolve would pick up
|
|
41
|
+
`wrapper-core@1.3.0` on its own. A lockfile will not: it holds you on
|
|
42
|
+
`1.2.0`, which is what most installs are actually running. Upgrading this
|
|
43
|
+
package to `1.0.2` is the reliable fix.
|
|
44
|
+
|
|
45
|
+
Full advisory, including the other thirteen affected artifacts:
|
|
46
|
+
`docs/security/advisory-2026-08-10-verdict-redirect-bypass.md` (SEC-051).
|
|
47
|
+
|
|
48
|
+
### Earlier releases
|
|
49
|
+
|
|
50
|
+
Recorded in the monorepo git history and in
|
|
51
|
+
`packages/NPM_PUBLISH_CHECKLIST.md`; this file starts here.
|
package/README.md
CHANGED
|
@@ -161,10 +161,41 @@ withEvalguard(model, {
|
|
|
161
161
|
});
|
|
162
162
|
```
|
|
163
163
|
|
|
164
|
+
`withEvalguard` validates this object and **throws immediately** on a config it
|
|
165
|
+
cannot work with, rather than turning it into a per-request failure:
|
|
166
|
+
|
|
167
|
+
- `apiKey` must be a non-empty string (or a resolver function). An unset or
|
|
168
|
+
empty `EVALGUARD_API_KEY` used to be accepted, and then every guardrail check
|
|
169
|
+
failed — blocking every request under the default `blockOnViolation: true`,
|
|
170
|
+
and silently running **completely unguarded** under `blockOnViolation: false`.
|
|
171
|
+
- `baseUrl` must be `https://` (or `http://` on loopback, for local testing).
|
|
172
|
+
Any other scheme — `ftp:`, `file:`, `data:`, `javascript:` — is rejected;
|
|
173
|
+
previously only plaintext `http:` to a non-loopback host was.
|
|
174
|
+
- Unrecognised options are **warned about**, once per process. They have no
|
|
175
|
+
effect, so a typo (`blockOnviolation`) or an option that belongs to a
|
|
176
|
+
different wrapper (`evalOnResponse`, `onViolation`, `retry`, `circuit`,
|
|
177
|
+
`idempotency`) no longer passes in silence. Every documented boolean fails
|
|
178
|
+
safe when misspelled — `blockOnViolation`, `disableGuardrails` and
|
|
179
|
+
`disableLogging` all fall back to the guarded default — which is why this
|
|
180
|
+
warns rather than throws.
|
|
181
|
+
|
|
164
182
|
## Streaming works too
|
|
165
183
|
|
|
166
|
-
|
|
167
|
-
|
|
184
|
+
The rail runs on the **input**, before the provider is called — so a blocked
|
|
185
|
+
prompt throws `EvalguardBlockedError` and no stream is ever opened. Once the
|
|
186
|
+
stream is open, parts pass through verbatim with backpressure preserved, and
|
|
187
|
+
exactly one trace is emitted whichever way the stream ends: normally (assembled
|
|
188
|
+
text + token totals), on an upstream error (`streamStatus: "error"`, and the
|
|
189
|
+
error is surfaced to your consumer — never swallowed into a clean close), or on
|
|
190
|
+
a client cancel (`streamStatus: "cancelled"`, and the cancel is propagated
|
|
191
|
+
upstream so the provider connection is torn down).
|
|
192
|
+
|
|
193
|
+
> **The model's output is not scanned.** This wrapper runs the input rail only:
|
|
194
|
+
> generated text and stream chunks are returned to you verbatim. There is no
|
|
195
|
+
> `evalOnResponse` option here — that lives in `@evalguard/openai`,
|
|
196
|
+
> `@evalguard/anthropic` and `@evalguard/gemini`. Passing `evalOnResponse` (or
|
|
197
|
+
> any other unrecognised option) to `withEvalguard` logs a warning and has no
|
|
198
|
+
> effect.
|
|
168
199
|
|
|
169
200
|
```ts
|
|
170
201
|
import { streamText } from "ai";
|
|
@@ -181,12 +212,25 @@ for await (const chunk of textStream) {
|
|
|
181
212
|
> "guarantee" ("falls back to allow + don't log"). The wrappers went
|
|
182
213
|
> fail-closed on 2026-05-28; the docs were never updated.
|
|
183
214
|
|
|
184
|
-
If EvalGuard's API is unreachable, slow,
|
|
215
|
+
If EvalGuard's API is unreachable, slow, returns an error, or answers `200`
|
|
216
|
+
with a body that is not a firewall verdict (an empty body, `null`, non-JSON,
|
|
217
|
+
`{}`, or an unrelated payload injected by a proxy):
|
|
185
218
|
|
|
186
219
|
- `blockOnViolation: true` (**default**) — the wrapper **throws**
|
|
187
220
|
`EvalguardBlockedError` with a `guardrail_unavailable` violation. The
|
|
188
221
|
provider call is not made.
|
|
189
|
-
- `blockOnViolation: false` — the call proceeds and the outage is recorded
|
|
222
|
+
- `blockOnViolation: false` — the call proceeds, and the outage is recorded on
|
|
223
|
+
the emitted trace as a `guardrail_unavailable` violation in
|
|
224
|
+
`guardrailResult.violations`, so an unguarded call is distinguishable from a
|
|
225
|
+
guarded one in your dashboard.
|
|
226
|
+
|
|
227
|
+
> Corrected again 2026-08-07. The `blockOnViolation: false` half of that
|
|
228
|
+
> sentence had nothing behind it: the outage was swallowed and `guardrailResult`
|
|
229
|
+
> was left `undefined`, so the trace's metadata carried only
|
|
230
|
+
> provider/model/cost/costPricingSource/timestamp and nothing said the rail had
|
|
231
|
+
> not run. Now pinned by
|
|
232
|
+
> `src/__tests__/config-validation-and-rail-blindspots.test.ts` on both the
|
|
233
|
+
> `doGenerate` and `doStream` paths.
|
|
190
234
|
|
|
191
235
|
Trace-log errors are silently swallowed in both modes.
|
|
192
236
|
|
|
@@ -0,0 +1,41 @@
|
|
|
1
|
+
export interface GuardrailCheckResult {
|
|
2
|
+
allowed: boolean;
|
|
3
|
+
violations: GuardrailViolation[];
|
|
4
|
+
}
|
|
5
|
+
export interface GuardrailViolation {
|
|
6
|
+
type: string;
|
|
7
|
+
severity: "critical" | "high" | "medium" | "low" | "info";
|
|
8
|
+
message: string;
|
|
9
|
+
}
|
|
10
|
+
export interface TraceLogData {
|
|
11
|
+
model: string;
|
|
12
|
+
provider: string;
|
|
13
|
+
input: unknown;
|
|
14
|
+
output: unknown;
|
|
15
|
+
latencyMs: number;
|
|
16
|
+
tokenUsage: {
|
|
17
|
+
input: number;
|
|
18
|
+
output: number;
|
|
19
|
+
};
|
|
20
|
+
cost: number;
|
|
21
|
+
projectId?: string;
|
|
22
|
+
metadata?: Record<string, unknown>;
|
|
23
|
+
guardrailResult?: GuardrailCheckResult;
|
|
24
|
+
}
|
|
25
|
+
export declare class GuardrailClient {
|
|
26
|
+
private readonly apiKey;
|
|
27
|
+
private readonly baseUrl;
|
|
28
|
+
constructor(apiKey: string, baseUrl?: string);
|
|
29
|
+
/**
|
|
30
|
+
* Pre-request guardrail check. Returns { allowed: true, violations: [] }
|
|
31
|
+
* on any error so the customer's LLM call never fails because of us.
|
|
32
|
+
*/
|
|
33
|
+
checkInput(prompt: string, metadata?: Record<string, unknown>): Promise<GuardrailCheckResult>;
|
|
34
|
+
/**
|
|
35
|
+
* Post-request trace log. Fire-and-forget — errors silently swallowed.
|
|
36
|
+
*/
|
|
37
|
+
logTrace(data: TraceLogData): Promise<void>;
|
|
38
|
+
private headers;
|
|
39
|
+
private fetchWithTimeout;
|
|
40
|
+
}
|
|
41
|
+
//# sourceMappingURL=guardrail-client.d.ts.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"guardrail-client.d.ts","sourceRoot":"","sources":["../src/guardrail-client.ts"],"names":[],"mappings":"AAOA,MAAM,WAAW,oBAAoB;IACnC,OAAO,EAAE,OAAO,CAAC;IACjB,UAAU,EAAE,kBAAkB,EAAE,CAAC;CAClC;AAED,MAAM,WAAW,kBAAkB;IACjC,IAAI,EAAE,MAAM,CAAC;IACb,QAAQ,EAAE,UAAU,GAAG,MAAM,GAAG,QAAQ,GAAG,KAAK,GAAG,MAAM,CAAC;IAC1D,OAAO,EAAE,MAAM,CAAC;CACjB;AAED,MAAM,WAAW,YAAY;IAC3B,KAAK,EAAE,MAAM,CAAC;IACd,QAAQ,EAAE,MAAM,CAAC;IACjB,KAAK,EAAE,OAAO,CAAC;IACf,MAAM,EAAE,OAAO,CAAC;IAChB,SAAS,EAAE,MAAM,CAAC;IAClB,UAAU,EAAE;QAAE,KAAK,EAAE,MAAM,CAAC;QAAC,MAAM,EAAE,MAAM,CAAA;KAAE,CAAC;IAC9C,IAAI,EAAE,MAAM,CAAC;IACb,SAAS,CAAC,EAAE,MAAM,CAAC;IACnB,QAAQ,CAAC,EAAE,MAAM,CAAC,MAAM,EAAE,OAAO,CAAC,CAAC;IACnC,eAAe,CAAC,EAAE,oBAAoB,CAAC;CACxC;AAKD,qBAAa,eAAe;IAC1B,OAAO,CAAC,QAAQ,CAAC,MAAM,CAAS;IAChC,OAAO,CAAC,QAAQ,CAAC,OAAO,CAAS;gBAErB,MAAM,EAAE,MAAM,EAAE,OAAO,CAAC,EAAE,MAAM;IAK5C;;;OAGG;IACG,UAAU,CAAC,MAAM,EAAE,MAAM,EAAE,QAAQ,CAAC,EAAE,MAAM,CAAC,MAAM,EAAE,OAAO,CAAC,GAAG,OAAO,CAAC,oBAAoB,CAAC;IA4BnG;;OAEG;IACG,QAAQ,CAAC,IAAI,EAAE,YAAY,GAAG,OAAO,CAAC,IAAI,CAAC;IAwBjD,OAAO,CAAC,OAAO;IAQf,OAAO,CAAC,gBAAgB;CAKzB"}
|
|
@@ -0,0 +1,88 @@
|
|
|
1
|
+
// ── EvalGuard Guardrail Client (Vercel AI variant) ─────────────────────────
|
|
2
|
+
// Mirrors openai-wrapper / anthropic-wrapper guardrail client. Class
|
|
3
|
+
// itself is kept inline. Safety-critical escape-hatch logic now imports
|
|
4
|
+
// from wrapper-core — see langchain-wrapper/guardrail-client.ts header
|
|
5
|
+
// for the rationale.
|
|
6
|
+
import { isLegacyFailOpen, warnLegacyFailOpenOnce, parseFirewallCheckResult } from "@evalguard/wrapper-core";
|
|
7
|
+
const DEFAULT_BASE_URL = "https://evalguard.ai/api/v1";
|
|
8
|
+
const REQUEST_TIMEOUT_MS = 5_000;
|
|
9
|
+
export class GuardrailClient {
|
|
10
|
+
apiKey;
|
|
11
|
+
baseUrl;
|
|
12
|
+
constructor(apiKey, baseUrl) {
|
|
13
|
+
this.apiKey = apiKey;
|
|
14
|
+
this.baseUrl = (baseUrl ?? DEFAULT_BASE_URL).replace(/\/+$/, "");
|
|
15
|
+
}
|
|
16
|
+
/**
|
|
17
|
+
* Pre-request guardrail check. Returns { allowed: true, violations: [] }
|
|
18
|
+
* on any error so the customer's LLM call never fails because of us.
|
|
19
|
+
*/
|
|
20
|
+
async checkInput(prompt, metadata) {
|
|
21
|
+
// 2026-05-28: fail-CLOSED on outages. Escape-hatch env flag via
|
|
22
|
+
// shared wrapper-core helper (single source of truth).
|
|
23
|
+
try {
|
|
24
|
+
const res = await this.fetchWithTimeout(`${this.baseUrl}/firewall/check`, {
|
|
25
|
+
method: "POST",
|
|
26
|
+
headers: this.headers(),
|
|
27
|
+
body: JSON.stringify({ input: prompt, metadata }),
|
|
28
|
+
});
|
|
29
|
+
if (!res.ok) {
|
|
30
|
+
if (isLegacyFailOpen()) {
|
|
31
|
+
warnLegacyFailOpenOnce();
|
|
32
|
+
return { allowed: true, violations: [] };
|
|
33
|
+
}
|
|
34
|
+
throw new Error(`firewall/check returned ${res.status}`);
|
|
35
|
+
}
|
|
36
|
+
// Shared parser — reads the apiSuccess envelope data.blocked, NOT the
|
|
37
|
+
// non-existent body.action that silently ALLOWED every server block. P0 2026-05-30.
|
|
38
|
+
return parseFirewallCheckResult(await res.json());
|
|
39
|
+
}
|
|
40
|
+
catch (err) {
|
|
41
|
+
if (isLegacyFailOpen()) {
|
|
42
|
+
warnLegacyFailOpenOnce();
|
|
43
|
+
return { allowed: true, violations: [] };
|
|
44
|
+
}
|
|
45
|
+
throw err;
|
|
46
|
+
}
|
|
47
|
+
}
|
|
48
|
+
/**
|
|
49
|
+
* Post-request trace log. Fire-and-forget — errors silently swallowed.
|
|
50
|
+
*/
|
|
51
|
+
async logTrace(data) {
|
|
52
|
+
try {
|
|
53
|
+
await this.fetchWithTimeout(`${this.baseUrl}/traces`, {
|
|
54
|
+
method: "POST",
|
|
55
|
+
headers: this.headers(),
|
|
56
|
+
body: JSON.stringify({
|
|
57
|
+
projectId: data.projectId,
|
|
58
|
+
provider: data.provider,
|
|
59
|
+
model: data.model,
|
|
60
|
+
input: data.input,
|
|
61
|
+
output: data.output,
|
|
62
|
+
latencyMs: data.latencyMs,
|
|
63
|
+
tokenUsage: data.tokenUsage,
|
|
64
|
+
cost: data.cost,
|
|
65
|
+
metadata: data.metadata,
|
|
66
|
+
guardrailResult: data.guardrailResult,
|
|
67
|
+
timestamp: new Date().toISOString(),
|
|
68
|
+
}),
|
|
69
|
+
});
|
|
70
|
+
}
|
|
71
|
+
catch {
|
|
72
|
+
// logging failure must never break the user's call
|
|
73
|
+
}
|
|
74
|
+
}
|
|
75
|
+
headers() {
|
|
76
|
+
return {
|
|
77
|
+
"Content-Type": "application/json",
|
|
78
|
+
Authorization: `Bearer ${this.apiKey}`,
|
|
79
|
+
"User-Agent": "@evalguard/vercel-ai/1.0.0",
|
|
80
|
+
};
|
|
81
|
+
}
|
|
82
|
+
fetchWithTimeout(url, init) {
|
|
83
|
+
const controller = new AbortController();
|
|
84
|
+
const timer = setTimeout(() => controller.abort(), REQUEST_TIMEOUT_MS);
|
|
85
|
+
return fetch(url, { ...init, signal: controller.signal }).finally(() => clearTimeout(timer));
|
|
86
|
+
}
|
|
87
|
+
}
|
|
88
|
+
//# sourceMappingURL=guardrail-client.js.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"guardrail-client.js","sourceRoot":"","sources":["../src/guardrail-client.ts"],"names":[],"mappings":"AAAA,8EAA8E;AAC9E,qEAAqE;AACrE,wEAAwE;AACxE,uEAAuE;AACvE,qBAAqB;AACrB,OAAO,EAAE,gBAAgB,EAAE,sBAAsB,EAAE,wBAAwB,EAAE,MAAM,yBAAyB,CAAC;AA0B7G,MAAM,gBAAgB,GAAG,6BAA6B,CAAC;AACvD,MAAM,kBAAkB,GAAG,KAAK,CAAC;AAEjC,MAAM,OAAO,eAAe;IACT,MAAM,CAAS;IACf,OAAO,CAAS;IAEjC,YAAY,MAAc,EAAE,OAAgB;QAC1C,IAAI,CAAC,MAAM,GAAG,MAAM,CAAC;QACrB,IAAI,CAAC,OAAO,GAAG,CAAC,OAAO,IAAI,gBAAgB,CAAC,CAAC,OAAO,CAAC,MAAM,EAAE,EAAE,CAAC,CAAC;IACnE,CAAC;IAED;;;OAGG;IACH,KAAK,CAAC,UAAU,CAAC,MAAc,EAAE,QAAkC;QACjE,gEAAgE;QAChE,uDAAuD;QACvD,IAAI,CAAC;YACH,MAAM,GAAG,GAAG,MAAM,IAAI,CAAC,gBAAgB,CAAC,GAAG,IAAI,CAAC,OAAO,iBAAiB,EAAE;gBACxE,MAAM,EAAE,MAAM;gBACd,OAAO,EAAE,IAAI,CAAC,OAAO,EAAE;gBACvB,IAAI,EAAE,IAAI,CAAC,SAAS,CAAC,EAAE,KAAK,EAAE,MAAM,EAAE,QAAQ,EAAE,CAAC;aAClD,CAAC,CAAC;YACH,IAAI,CAAC,GAAG,CAAC,EAAE,EAAE,CAAC;gBACZ,IAAI,gBAAgB,EAAE,EAAE,CAAC;oBACvB,sBAAsB,EAAE,CAAC;oBACzB,OAAO,EAAE,OAAO,EAAE,IAAI,EAAE,UAAU,EAAE,EAAE,EAAE,CAAC;gBAC3C,CAAC;gBACD,MAAM,IAAI,KAAK,CAAC,2BAA2B,GAAG,CAAC,MAAM,EAAE,CAAC,CAAC;YAC3D,CAAC;YACD,sEAAsE;YACtE,oFAAoF;YACpF,OAAO,wBAAwB,CAAC,MAAM,GAAG,CAAC,IAAI,EAAE,CAAC,CAAC;QACpD,CAAC;QAAC,OAAO,GAAG,EAAE,CAAC;YACb,IAAI,gBAAgB,EAAE,EAAE,CAAC;gBACvB,sBAAsB,EAAE,CAAC;gBACzB,OAAO,EAAE,OAAO,EAAE,IAAI,EAAE,UAAU,EAAE,EAAE,EAAE,CAAC;YAC3C,CAAC;YACD,MAAM,GAAG,CAAC;QACZ,CAAC;IACH,CAAC;IAED;;OAEG;IACH,KAAK,CAAC,QAAQ,CAAC,IAAkB;QAC/B,IAAI,CAAC;YACH,MAAM,IAAI,CAAC,gBAAgB,CAAC,GAAG,IAAI,CAAC,OAAO,SAAS,EAAE;gBACpD,MAAM,EAAE,MAAM;gBACd,OAAO,EAAE,IAAI,CAAC,OAAO,EAAE;gBACvB,IAAI,EAAE,IAAI,CAAC,SAAS,CAAC;oBACnB,SAAS,EAAE,IAAI,CAAC,SAAS;oBACzB,QAAQ,EAAE,IAAI,CAAC,QAAQ;oBACvB,KAAK,EAAE,IAAI,CAAC,KAAK;oBACjB,KAAK,EAAE,IAAI,CAAC,KAAK;oBACjB,MAAM,EAAE,IAAI,CAAC,MAAM;oBACnB,SAAS,EAAE,IAAI,CAAC,SAAS;oBACzB,UAAU,EAAE,IAAI,CAAC,UAAU;oBAC3B,IAAI,EAAE,IAAI,CAAC,IAAI;oBACf,QAAQ,EAAE,IAAI,CAAC,QAAQ;oBACvB,eAAe,EAAE,IAAI,CAAC,eAAe;oBACrC,SAAS,EAAE,IAAI,IAAI,EAAE,CAAC,WAAW,EAAE;iBACpC,CAAC;aACH,CAAC,CAAC;QACL,CAAC;QAAC,MAAM,CAAC;YACP,mDAAmD;QACrD,CAAC;IACH,CAAC;IAEO,OAAO;QACb,OAAO;YACL,cAAc,EAAE,kBAAkB;YAClC,aAAa,EAAE,UAAU,IAAI,CAAC,MAAM,EAAE;YACtC,YAAY,EAAE,4BAA4B;SAC3C,CAAC;IACJ,CAAC;IAEO,gBAAgB,CAAC,GAAW,EAAE,IAAiB;QACrD,MAAM,UAAU,GAAG,IAAI,eAAe,EAAE,CAAC;QACzC,MAAM,KAAK,GAAG,UAAU,CAAC,GAAG,EAAE,CAAC,UAAU,CAAC,KAAK,EAAE,EAAE,kBAAkB,CAAC,CAAC;QACvE,OAAO,KAAK,CAAC,GAAG,EAAE,EAAE,GAAG,IAAI,EAAE,MAAM,EAAE,UAAU,CAAC,MAAM,EAAE,CAAC,CAAC,OAAO,CAAC,GAAG,EAAE,CAAC,YAAY,CAAC,KAAK,CAAC,CAAC,CAAC;IAC/F,CAAC;CACF"}
|
package/dist/index.d.ts
CHANGED
|
@@ -8,10 +8,17 @@
|
|
|
8
8
|
* EvalGuard project. Guardrail semantics are fail-CLOSED when
|
|
9
9
|
* blockOnViolation is on (the default): if the firewall is unreachable the
|
|
10
10
|
* call is BLOCKED with an EvalguardBlockedError, matching the langchain /
|
|
11
|
-
* llamaindex wrappers. In monitor-only mode (blockOnViolation:false)
|
|
12
|
-
*
|
|
11
|
+
* llamaindex wrappers. In monitor-only mode (blockOnViolation:false) the call
|
|
12
|
+
* proceeds and the outage is RECORDED on the trace as a `guardrail_unavailable`
|
|
13
|
+
* violation — it used to be silently swallowed, which made an unguarded call
|
|
14
|
+
* indistinguishable from a guarded one (audit 2026-08-07). Trace logging always
|
|
13
15
|
* fails open — a logging error never breaks or delays the user's call.
|
|
14
16
|
*
|
|
17
|
+
* SCOPE: this wrapper runs the INPUT rail only. Model output is streamed and
|
|
18
|
+
* returned verbatim and is never scanned — there is no `evalOnResponse` here,
|
|
19
|
+
* unlike @evalguard/openai, @evalguard/anthropic and @evalguard/gemini. Passing
|
|
20
|
+
* `evalOnResponse` to `withEvalguard` logs a warning and has no effect.
|
|
21
|
+
*
|
|
15
22
|
* Usage:
|
|
16
23
|
*
|
|
17
24
|
* import { openai } from "@ai-sdk/openai";
|
|
@@ -51,9 +58,26 @@
|
|
|
51
58
|
*
|
|
52
59
|
* Instead of importing `wrapLanguageModel` we re-implement the same "delegate
|
|
53
60
|
* to the model, override doGenerate/doStream" shape directly, as a PROXY over
|
|
54
|
-
* the original model.
|
|
55
|
-
*
|
|
56
|
-
*
|
|
61
|
+
* the original model.
|
|
62
|
+
*
|
|
63
|
+
* That proxy is DEFAULT-DENY as of 2026-08-08. Until then it forwarded
|
|
64
|
+
* everything except `doGenerate` / `doStream` straight to `Reflect.get` — a
|
|
65
|
+
* denylist of two guarded names — on the recorded argument that the AI SDK model
|
|
66
|
+
* interface is "a fixed, tiny surface". `@ai-sdk/provider@3.0.10` in fact
|
|
67
|
+
* declares FOUR call methods across SEVEN model specifications, and `doEmbed`
|
|
68
|
+
* (embedding models) and `doRerank` (reranking models) were measured delivering
|
|
69
|
+
* attack text to the provider on ZERO firewall calls, as were speech and
|
|
70
|
+
* transcription calls whose payload lives in `options.text` / `options.audio`
|
|
71
|
+
* rather than `options.prompt`. Every property is now guarded unless
|
|
72
|
+
* `./surface.ts` names it, using the same wrapper-core machinery as
|
|
73
|
+
* `@evalguard/anthropic` and `@evalguard/gemini`, so a method the AI SDK ships
|
|
74
|
+
* tomorrow is gated on the day it lands.
|
|
75
|
+
*
|
|
76
|
+
* Capability fields — most importantly `supportedUrls` — still pass through
|
|
77
|
+
* untouched; they are on the policy's justified passthrough set for exactly that
|
|
78
|
+
* reason, and `src/__tests__/model-spec-surface-matrix.test.ts` asserts a
|
|
79
|
+
* wrapped model's `supportedUrls` RegExps still work and cost zero firewall
|
|
80
|
+
* requests. Two earlier versions got this wrong:
|
|
57
81
|
* 1. hand-copying a fixed list of V1 props (`supportsUrl`,
|
|
58
82
|
* `supportsImageUrls`, …) that no longer exist on a V2 model, which
|
|
59
83
|
* silently STRIPPED `supportedUrls` (audit MEDIUM 2026-07-14);
|
|
@@ -211,6 +235,8 @@ export interface WithEvalguardConfig {
|
|
|
211
235
|
/** Free-form metadata attached to every trace. */
|
|
212
236
|
metadata?: Record<string, unknown>;
|
|
213
237
|
}
|
|
238
|
+
/** Internal — lets the test suite re-arm the once-per-process unknown-option warning. */
|
|
239
|
+
export declare function _resetConfigWarningsForTests(): void;
|
|
214
240
|
/**
|
|
215
241
|
* Error thrown when blockOnViolation=true and the input is rejected by
|
|
216
242
|
* EvalGuard's firewall. Customers can catch this distinctly from provider
|
|
@@ -241,11 +267,15 @@ export declare class EvalguardBlockedError extends Error {
|
|
|
241
267
|
* ai@6 + @ai-sdk/openai@3 and ai@7 + @ai-sdk/openai@4 from packed tarballs on
|
|
242
268
|
* 2026-08-02, and pinned by `src/__tests__/ai-sdk-assignability.test.ts`.
|
|
243
269
|
*
|
|
244
|
-
*
|
|
245
|
-
*
|
|
246
|
-
*
|
|
247
|
-
*
|
|
248
|
-
*
|
|
270
|
+
* `doGenerate` / `doStream` get the rich interception (stream teeing, token
|
|
271
|
+
* accounting, cost, trace logging). Every OTHER property is default-deny: a
|
|
272
|
+
* capability or identity field named on `./surface.ts`'s justified passthrough
|
|
273
|
+
* set is forwarded to the real instance verbatim — including prototype getters
|
|
274
|
+
* such as `supportedUrls` on a class-based provider model, so capability fields
|
|
275
|
+
* still compose with other middleware (caching, logging) the customer may
|
|
276
|
+
* already have — and anything else that is callable is put behind the firewall.
|
|
277
|
+
* See `./surface.ts` for why: `Reflect.get` for everything unlisted was measured
|
|
278
|
+
* shipping attack text to the provider on zero firewall calls.
|
|
249
279
|
*/
|
|
250
280
|
export declare function withEvalguard<TModel extends LanguageModelV2>(model: TModel, config: WithEvalguardConfig): TModel;
|
|
251
281
|
export type { GuardrailCheckResult, GuardrailViolation, TraceLogData } from "@evalguard/wrapper-core";
|
package/dist/index.d.ts.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAAA
|
|
1
|
+
{"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;GA8FG;AAEH,OAAO,EAQL,KAAK,oBAAoB,EAE1B,MAAM,yBAAyB,CAAC;AAsBjC;;;;;;;;;;;;;;GAcG;AACH,KAAK,uBAAuB,GACxB,MAAM,GACN;IAAE,KAAK,CAAC,EAAE,MAAM,GAAG,SAAS,CAAC;IAAC,CAAC,CAAC,EAAE,MAAM,GAAG,OAAO,CAAA;CAAE,GACpD,SAAS,CAAC;AAEd,UAAU,oBAAoB;IAE5B,WAAW,CAAC,EAAE,uBAAuB,CAAC;IACtC,YAAY,CAAC,EAAE,uBAAuB,CAAC;IAGvC,YAAY,CAAC,EAAE,MAAM,CAAC;IACtB,gBAAgB,CAAC,EAAE,MAAM,CAAC;CAC3B;AAED,UAAU,8BAA8B;IACtC,IAAI,EAAE,MAAM,CAAC;IACb,IAAI,EAAE,MAAM,CAAC;IACb,CAAC,CAAC,EAAE,MAAM,GAAG,OAAO,CAAC;CACtB;AAED,KAAK,0BAA0B,GAC3B,8BAA8B,GAC9B;IAAE,IAAI,EAAE,MAAM,CAAC;IAAC,CAAC,CAAC,EAAE,MAAM,GAAG,OAAO,CAAA;CAAE,CAAC;AAoB3C,UAAU,0BAA0B;IAClC,MAAM,EAAE,OAAO,CAAC;IAChB,CAAC,CAAC,EAAE,MAAM,GAAG,OAAO,CAAC;CACtB;AAED,UAAU,6BAA6B;IAErC,OAAO,CAAC,EAAE,0BAA0B,EAAE,CAAC;IAEvC,IAAI,CAAC,EAAE,MAAM,CAAC;IACd,SAAS,CAAC,EAAE,OAAO,CAAC;IACpB,KAAK,CAAC,EAAE,oBAAoB,CAAC;IAC7B,CAAC,CAAC,EAAE,MAAM,GAAG,OAAO,CAAC;CACtB;AAED;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;GAiCG;AACH,UAAU,2BAA2B;IACnC,MAAM,EAAE,cAAc,CAAC,OAAO,CAAC,CAAC;IAChC,CAAC,CAAC,EAAE,MAAM,GAAG,OAAO,CAAC;CACtB;AAED;;;;;;;;;GASG;AACH,MAAM,WAAW,eAAe;IAC9B;;;;;;;;OAQG;IACH,oBAAoB,EAAE,MAAM,CAAC;IAC7B,QAAQ,EAAE,MAAM,CAAC;IACjB,OAAO,EAAE,MAAM,CAAC;IAChB;;;;;OAKG;IACH,aAAa,CAAC,EACV,MAAM,CAAC,MAAM,EAAE,MAAM,EAAE,CAAC,GACxB,WAAW,CAAC,MAAM,CAAC,MAAM,EAAE,MAAM,EAAE,CAAC,CAAC,CAAC;IAC1C,UAAU,CAAC,OAAO,EAAE,0BAA0B,GAAG,WAAW,CAAC,6BAA6B,CAAC,CAAC;IAC5F,QAAQ,CAAC,OAAO,EAAE,0BAA0B,GAAG,WAAW,CAAC,2BAA2B,CAAC,CAAC;CASzF;AAED;;;;GAIG;AACH,MAAM,MAAM,eAAe,GAAG,eAAe,CAAC;AAE9C,MAAM,WAAW,mBAAmB;IAClC,mEAAmE;IACnE,MAAM,EAAE,MAAM,CAAC;IACf,sEAAsE;IACtE,SAAS,CAAC,EAAE,MAAM,CAAC;IACnB,gFAAgF;IAChF,OAAO,CAAC,EAAE,MAAM,CAAC;IACjB;;;OAGG;IACH,gBAAgB,CAAC,EAAE,OAAO,CAAC;IAC3B,qEAAqE;IACrE,cAAc,CAAC,EAAE,OAAO,CAAC;IACzB,kEAAkE;IAClE,iBAAiB,CAAC,EAAE,OAAO,CAAC;IAC5B,kDAAkD;IAClD,QAAQ,CAAC,EAAE,MAAM,CAAC,MAAM,EAAE,OAAO,CAAC,CAAC;CACpC;AA4HD,yFAAyF;AACzF,wBAAgB,4BAA4B,IAAI,IAAI,CAEnD;AAED;;;;GAIG;AACH,qBAAa,qBAAsB,SAAQ,KAAK;IAC9C,QAAQ,CAAC,UAAU,EAAE,oBAAoB,CAAC,YAAY,CAAC,CAAC;gBAC5C,OAAO,EAAE,MAAM,EAAE,UAAU,EAAE,oBAAoB,CAAC,YAAY,CAAC;CAK5E;AA0CD;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;GA8BG;AACH,wBAAgB,aAAa,CAAC,MAAM,SAAS,eAAe,EAC1D,KAAK,EAAE,MAAM,EACb,MAAM,EAAE,mBAAmB,GAC1B,MAAM,CAyUR;AAmID,YAAY,EAAE,oBAAoB,EAAE,kBAAkB,EAAE,YAAY,EAAE,MAAM,yBAAyB,CAAC;AAStG,OAAO,EAAE,YAAY,EAAE,oBAAoB,EAAE,aAAa,EAAE,MAAM,WAAW,CAAC;AAC9E,YAAY,EAAE,YAAY,EAAE,aAAa,EAAE,MAAM,WAAW,CAAC"}
|