@evalguard/vercel-ai 1.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +201 -0
- package/README.md +248 -0
- package/dist/cost.d.ts +4 -0
- package/dist/cost.d.ts.map +1 -0
- package/dist/cost.js +18 -0
- package/dist/cost.js.map +1 -0
- package/dist/index.d.ts +254 -0
- package/dist/index.d.ts.map +1 -0
- package/dist/index.js +395 -0
- package/dist/index.js.map +1 -0
- package/dist/version.d.ts +2 -0
- package/dist/version.d.ts.map +1 -0
- package/dist/version.js +5 -0
- package/dist/version.js.map +1 -0
- package/package.json +70 -0
- package/peer-matrix.json +50 -0
package/LICENSE
ADDED
|
@@ -0,0 +1,201 @@
|
|
|
1
|
+
Apache License
|
|
2
|
+
Version 2.0, January 2004
|
|
3
|
+
http://www.apache.org/licenses/
|
|
4
|
+
|
|
5
|
+
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
|
|
6
|
+
|
|
7
|
+
1. Definitions.
|
|
8
|
+
|
|
9
|
+
"License" shall mean the terms and conditions for use, reproduction,
|
|
10
|
+
and distribution as defined by Sections 1 through 9 of this document.
|
|
11
|
+
|
|
12
|
+
"Licensor" shall mean the copyright owner or entity authorized by
|
|
13
|
+
the copyright owner that is granting the License.
|
|
14
|
+
|
|
15
|
+
"Legal Entity" shall mean the union of the acting entity and all
|
|
16
|
+
other entities that control, are controlled by, or are under common
|
|
17
|
+
control with that entity. For the purposes of this definition,
|
|
18
|
+
"control" means (i) the power, direct or indirect, to cause the
|
|
19
|
+
direction or management of such entity, whether by contract or
|
|
20
|
+
otherwise, or (ii) ownership of fifty percent (50%) or more of the
|
|
21
|
+
outstanding shares, or (iii) beneficial ownership of such entity.
|
|
22
|
+
|
|
23
|
+
"You" (or "Your") shall mean an individual or Legal Entity
|
|
24
|
+
exercising permissions granted by this License.
|
|
25
|
+
|
|
26
|
+
"Source" form shall mean the preferred form for making modifications,
|
|
27
|
+
including but not limited to software source code, documentation
|
|
28
|
+
source, and configuration files.
|
|
29
|
+
|
|
30
|
+
"Object" form shall mean any form resulting from mechanical
|
|
31
|
+
transformation or translation of a Source form, including but
|
|
32
|
+
not limited to compiled object code, generated documentation,
|
|
33
|
+
and conversions to other media types.
|
|
34
|
+
|
|
35
|
+
"Work" shall mean the work of authorship, whether in Source or
|
|
36
|
+
Object form, made available under the License, as indicated by a
|
|
37
|
+
copyright notice that is included in or attached to the work
|
|
38
|
+
(an example is provided in the Appendix below).
|
|
39
|
+
|
|
40
|
+
"Derivative Works" shall mean any work, whether in Source or Object
|
|
41
|
+
form, that is based on (or derived from) the Work and for which the
|
|
42
|
+
editorial revisions, annotations, elaborations, or other modifications
|
|
43
|
+
represent, as a whole, an original work of authorship. For the purposes
|
|
44
|
+
of this License, Derivative Works shall not include works that remain
|
|
45
|
+
separable from, or merely link (or bind by name) to the interfaces of,
|
|
46
|
+
the Work and Derivative Works thereof.
|
|
47
|
+
|
|
48
|
+
"Contribution" shall mean any work of authorship, including
|
|
49
|
+
the original version of the Work and any modifications or additions
|
|
50
|
+
to that Work or Derivative Works thereof, that is intentionally
|
|
51
|
+
submitted to Licensor for inclusion in the Work by the copyright owner
|
|
52
|
+
or by an individual or Legal Entity authorized to submit on behalf of
|
|
53
|
+
the copyright owner. For the purposes of this definition, "submitted"
|
|
54
|
+
means any form of electronic, verbal, or written communication sent
|
|
55
|
+
to the Licensor or its representatives, including but not limited to
|
|
56
|
+
communication on electronic mailing lists, source code control systems,
|
|
57
|
+
and issue tracking systems that are managed by, or on behalf of, the
|
|
58
|
+
Licensor for the purpose of discussing and improving the Work, but
|
|
59
|
+
excluding communication that is conspicuously marked or otherwise
|
|
60
|
+
designated in writing by the copyright owner as "Not a Contribution."
|
|
61
|
+
|
|
62
|
+
"Contributor" shall mean Licensor and any individual or Legal Entity
|
|
63
|
+
on behalf of whom a Contribution has been received by Licensor and
|
|
64
|
+
subsequently incorporated within the Work.
|
|
65
|
+
|
|
66
|
+
2. Grant of Copyright License. Subject to the terms and conditions of
|
|
67
|
+
this License, each Contributor hereby grants to You a perpetual,
|
|
68
|
+
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
|
69
|
+
copyright license to reproduce, prepare Derivative Works of,
|
|
70
|
+
publicly display, publicly perform, sublicense, and distribute the
|
|
71
|
+
Work and such Derivative Works in Source or Object form.
|
|
72
|
+
|
|
73
|
+
3. Grant of Patent License. Subject to the terms and conditions of
|
|
74
|
+
this License, each Contributor hereby grants to You a perpetual,
|
|
75
|
+
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
|
76
|
+
(except as stated in this section) patent license to make, have made,
|
|
77
|
+
use, offer to sell, sell, import, and otherwise transfer the Work,
|
|
78
|
+
where such license applies only to those patent claims licensable
|
|
79
|
+
by such Contributor that are necessarily infringed by their
|
|
80
|
+
Contribution(s) alone or by combination of their Contribution(s)
|
|
81
|
+
with the Work to which such Contribution(s) was submitted. If You
|
|
82
|
+
institute patent litigation against any entity (including a
|
|
83
|
+
cross-claim or counterclaim in a lawsuit) alleging that the Work
|
|
84
|
+
or a Contribution incorporated within the Work constitutes direct
|
|
85
|
+
or contributory patent infringement, then any patent licenses
|
|
86
|
+
granted to You under this License for that Work shall terminate
|
|
87
|
+
as of the date such litigation is filed.
|
|
88
|
+
|
|
89
|
+
4. Redistribution. You may reproduce and distribute copies of the
|
|
90
|
+
Work or Derivative Works thereof in any medium, with or without
|
|
91
|
+
modifications, and in Source or Object form, provided that You
|
|
92
|
+
meet the following conditions:
|
|
93
|
+
|
|
94
|
+
(a) You must give any other recipients of the Work or
|
|
95
|
+
Derivative Works a copy of this License; and
|
|
96
|
+
|
|
97
|
+
(b) You must cause any modified files to carry prominent notices
|
|
98
|
+
stating that You changed the files; and
|
|
99
|
+
|
|
100
|
+
(c) You must retain, in the Source form of any Derivative Works
|
|
101
|
+
that You distribute, all copyright, patent, trademark, and
|
|
102
|
+
attribution notices from the Source form of the Work,
|
|
103
|
+
excluding those notices that do not pertain to any part of
|
|
104
|
+
the Derivative Works; and
|
|
105
|
+
|
|
106
|
+
(d) If the Work includes a "NOTICE" text file as part of its
|
|
107
|
+
distribution, then any Derivative Works that You distribute must
|
|
108
|
+
include a readable copy of the attribution notices contained
|
|
109
|
+
within such NOTICE file, excluding those notices that do not
|
|
110
|
+
pertain to any part of the Derivative Works, in at least one
|
|
111
|
+
of the following places: within a NOTICE text file distributed
|
|
112
|
+
as part of the Derivative Works; within the Source form or
|
|
113
|
+
documentation, if provided along with the Derivative Works; or,
|
|
114
|
+
within a display generated by the Derivative Works, if and
|
|
115
|
+
wherever such third-party notices normally appear. The contents
|
|
116
|
+
of the NOTICE file are for informational purposes only and
|
|
117
|
+
do not modify the License. You may add Your own attribution
|
|
118
|
+
notices within Derivative Works that You distribute, alongside
|
|
119
|
+
or as an addendum to the NOTICE text from the Work, provided
|
|
120
|
+
that such additional attribution notices cannot be construed
|
|
121
|
+
as modifying the License.
|
|
122
|
+
|
|
123
|
+
You may add Your own copyright statement to Your modifications and
|
|
124
|
+
may provide additional or different license terms and conditions
|
|
125
|
+
for use, reproduction, or distribution of Your modifications, or
|
|
126
|
+
for any such Derivative Works as a whole, provided Your use,
|
|
127
|
+
reproduction, and distribution of the Work otherwise complies with
|
|
128
|
+
the conditions stated in this License.
|
|
129
|
+
|
|
130
|
+
5. Submission of Contributions. Unless You explicitly state otherwise,
|
|
131
|
+
any Contribution intentionally submitted for inclusion in the Work
|
|
132
|
+
by You to the Licensor shall be under the terms and conditions of
|
|
133
|
+
this License, without any additional terms or conditions.
|
|
134
|
+
Notwithstanding the above, nothing herein shall supersede or modify
|
|
135
|
+
the terms of any separate license agreement you may have executed
|
|
136
|
+
with Licensor regarding such Contributions.
|
|
137
|
+
|
|
138
|
+
6. Trademarks. This License does not grant permission to use the trade
|
|
139
|
+
names, trademarks, service marks, or product names of the Licensor,
|
|
140
|
+
except as required for describing the origin of the Work and
|
|
141
|
+
reproducing the content of the NOTICE file.
|
|
142
|
+
|
|
143
|
+
7. Disclaimer of Warranty. Unless required by applicable law or
|
|
144
|
+
agreed to in writing, Licensor provides the Work (and each
|
|
145
|
+
Contributor provides its Contributions) on an "AS IS" BASIS,
|
|
146
|
+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
|
|
147
|
+
implied, including, without limitation, any warranties or conditions
|
|
148
|
+
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
|
|
149
|
+
PARTICULAR PURPOSE. You are solely responsible for determining the
|
|
150
|
+
appropriateness of using or redistributing the Work and assume any
|
|
151
|
+
risks associated with Your exercise of permissions under this License.
|
|
152
|
+
|
|
153
|
+
8. Limitation of Liability. In no event and under no legal theory,
|
|
154
|
+
whether in tort (including negligence), contract, or otherwise,
|
|
155
|
+
unless required by applicable law (such as deliberate and grossly
|
|
156
|
+
negligent acts) or agreed to in writing, shall any Contributor be
|
|
157
|
+
liable to You for damages, including any direct, indirect, special,
|
|
158
|
+
incidental, or consequential damages of any character arising as a
|
|
159
|
+
result of this License or out of the use or inability to use the
|
|
160
|
+
Work (including but not limited to damages for loss of goodwill,
|
|
161
|
+
work stoppage, computer failure or malfunction, or any and all
|
|
162
|
+
other commercial damages or losses), even if such Contributor
|
|
163
|
+
has been advised of the possibility of such damages.
|
|
164
|
+
|
|
165
|
+
9. Accepting Warranty or Support. While redistributing the Work or
|
|
166
|
+
Derivative Works thereof, You may choose to offer, and charge a
|
|
167
|
+
fee for, acceptance of support, warranty, indemnity, or other
|
|
168
|
+
liability obligations and/or rights consistent with this License.
|
|
169
|
+
However, in accepting such obligations, You may act only on Your
|
|
170
|
+
own behalf and on Your sole responsibility, not on behalf of any
|
|
171
|
+
other Contributor, and only if You agree to indemnify, defend,
|
|
172
|
+
and hold each Contributor harmless for any liability incurred by,
|
|
173
|
+
or claims asserted against, such Contributor by reason of your
|
|
174
|
+
accepting any such warranty or support.
|
|
175
|
+
|
|
176
|
+
END OF TERMS AND CONDITIONS
|
|
177
|
+
|
|
178
|
+
APPENDIX: How to apply the Apache License to your work.
|
|
179
|
+
|
|
180
|
+
To apply the Apache License to your work, attach the following
|
|
181
|
+
boilerplate notice, with the fields enclosed by brackets "[]"
|
|
182
|
+
replaced with your own identifying information. (Don't include
|
|
183
|
+
the brackets!) The text should be enclosed in the appropriate
|
|
184
|
+
comment syntax for the file format. We also recommend that a
|
|
185
|
+
file or class name and description of purpose be included on the
|
|
186
|
+
same "printed page" as the copyright notice for easier
|
|
187
|
+
identification within third-party archives.
|
|
188
|
+
|
|
189
|
+
Copyright 2024-2026 EvalGuard, Inc.
|
|
190
|
+
|
|
191
|
+
Licensed under the Apache License, Version 2.0 (the "License");
|
|
192
|
+
you may not use this file except in compliance with the License.
|
|
193
|
+
You may obtain a copy of the License at
|
|
194
|
+
|
|
195
|
+
http://www.apache.org/licenses/LICENSE-2.0
|
|
196
|
+
|
|
197
|
+
Unless required by applicable law or agreed to in writing, software
|
|
198
|
+
distributed under the License is distributed on an "AS IS" BASIS,
|
|
199
|
+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
|
200
|
+
See the License for the specific language governing permissions and
|
|
201
|
+
limitations under the License.
|
package/README.md
ADDED
|
@@ -0,0 +1,248 @@
|
|
|
1
|
+
# @evalguard/vercel-ai
|
|
2
|
+
|
|
3
|
+
Drop-in [Vercel AI SDK](https://sdk.vercel.ai) middleware that adds [EvalGuard](https://evalguard.ai)
|
|
4
|
+
guardrails (firewall input checks) and observability (trace logging, cost
|
|
5
|
+
tracking) to any language model from `@ai-sdk/openai`, `@ai-sdk/anthropic`,
|
|
6
|
+
`@ai-sdk/google`, `@ai-sdk/groq`, or any other provider that follows the AI SDK
|
|
7
|
+
model interface.
|
|
8
|
+
|
|
9
|
+
Verified against **`ai@5`, `ai@6` and `ai@7`** (model spec v2 / v3 / v4) — by a
|
|
10
|
+
check that runs before this package is published, not by assertion. For each
|
|
11
|
+
major, `scripts/verify-vercel-ai-peer-matrix.mjs` installs the packed tarball
|
|
12
|
+
into a clean consumer alongside that version of `ai` and its matching
|
|
13
|
+
`@ai-sdk/openai`, **type-checks the quickstart below verbatim** against it, and
|
|
14
|
+
**executes** `withEvalguard` end to end through that version's own
|
|
15
|
+
`generateText` / `streamText` (against a stub model, no network), asserting the
|
|
16
|
+
emitted trace carries numeric token counts.
|
|
17
|
+
|
|
18
|
+
The exact versions covered are in `peer-matrix.json`, which ships in this
|
|
19
|
+
tarball. Two steps in `.github/workflows/publish-wrappers.yml` run **before**
|
|
20
|
+
this package is published — `pnpm gate:publish-readiness` and
|
|
21
|
+
`pnpm gate:vercel-ai-peers` — and the first fails the release if the paragraph
|
|
22
|
+
above and `peer-matrix.json` disagree in *either* direction, or if no workflow
|
|
23
|
+
runs the second. So the sentence above cannot outlive the thing that makes it
|
|
24
|
+
true. (Naming both commands is deliberate: until 2026-08-03 this paragraph said
|
|
25
|
+
"a CI gate" and `gate:publish-readiness` appeared in no workflow at all, which
|
|
26
|
+
made the sentence describing the gate the very kind of unbacked claim the gate
|
|
27
|
+
exists to catch.)
|
|
28
|
+
|
|
29
|
+
The wrapper is a **Proxy** over the underlying model that intercepts only
|
|
30
|
+
`doGenerate` / `doStream`; every other property read is forwarded to the real
|
|
31
|
+
instance. That matters because real provider models are class instances whose
|
|
32
|
+
`supportedUrls` / `provider` / `modelId` are prototype getters — an object
|
|
33
|
+
spread (`{ ...model }`) copies own enumerable properties only and would drop
|
|
34
|
+
all of them. Capability fields therefore pass through untouched and URL/file
|
|
35
|
+
inputs the model handles natively are not stripped or re-downloaded.
|
|
36
|
+
|
|
37
|
+
Output text is read from the `content` array. Token counts are read from usage
|
|
38
|
+
in whichever shape the peer uses: a flat `inputTokens` number on `ai@5`, the
|
|
39
|
+
`{ total, … }` breakdown object on `ai@6` / `ai@7`, and the legacy v1
|
|
40
|
+
`promptTokens` / `completionTokens` as a back-compat fallback.
|
|
41
|
+
|
|
42
|
+
## Install
|
|
43
|
+
|
|
44
|
+
```bash
|
|
45
|
+
npm install @evalguard/vercel-ai ai @ai-sdk/openai
|
|
46
|
+
```
|
|
47
|
+
|
|
48
|
+
`ai` is a peer dependency (npm installs it for you, but naming it pins the
|
|
49
|
+
major you want). `@ai-sdk/openai` is the provider used by the quickstart below
|
|
50
|
+
— swap it for `@ai-sdk/anthropic`, `@ai-sdk/google`, or whichever provider you
|
|
51
|
+
actually use.
|
|
52
|
+
|
|
53
|
+
> ### ⚠️ ESM-only — requires `"type": "module"`
|
|
54
|
+
>
|
|
55
|
+
> `@evalguard/vercel-ai` ships **ES modules only** (`"type": "module"`, no CJS build). The
|
|
56
|
+
> quickstart below **will not type-check or run** in a default CommonJS
|
|
57
|
+
> TypeScript project — you get `TS1479` ("the referenced file is an ECMAScript
|
|
58
|
+
> module and cannot be imported with `require`") and, because the peer SDK's
|
|
59
|
+
> types then resolve under a different module mode, a confusing
|
|
60
|
+
> `TS2345 … Property '#private' … refers to a different member` on the very
|
|
61
|
+
> first call.
|
|
62
|
+
>
|
|
63
|
+
> **To use this package, your consuming project must be ESM:**
|
|
64
|
+
>
|
|
65
|
+
> ```jsonc
|
|
66
|
+
> // package.json
|
|
67
|
+
> { "type": "module" }
|
|
68
|
+
> ```
|
|
69
|
+
> ```jsonc
|
|
70
|
+
> // tsconfig.json
|
|
71
|
+
> { "compilerOptions": { "module": "node16", "moduleResolution": "node16" } }
|
|
72
|
+
> ```
|
|
73
|
+
>
|
|
74
|
+
> **Staying on CommonJS?** Load it with a dynamic `import()`. Unlike the
|
|
75
|
+
> SDK-shaped wrappers (`@evalguard/openai`, `@evalguard/anthropic`,
|
|
76
|
+
> `@evalguard/gemini`), `ai` and the `@ai-sdk/*` providers are dual-published,
|
|
77
|
+
> so those imports can stay static. Verified against `ai@5` +
|
|
78
|
+
> `@ai-sdk/openai@2`.
|
|
79
|
+
>
|
|
80
|
+
> The `await` must sit **inside an async function**: top-level `await` is an
|
|
81
|
+
> ESM-only feature, so a bare `await import(...)` at file scope in a CJS module
|
|
82
|
+
> is `TS1309: The current file is a CommonJS module and cannot use 'await' at
|
|
83
|
+
> the top level`.
|
|
84
|
+
>
|
|
85
|
+
> ```typescript
|
|
86
|
+
> // ✅ compiles under module/moduleResolution "node16", no "type": "module"
|
|
87
|
+
> import { openai } from "@ai-sdk/openai";
|
|
88
|
+
>
|
|
89
|
+
> async function main() {
|
|
90
|
+
> const { withEvalguard } = await import("@evalguard/vercel-ai");
|
|
91
|
+
> const model = withEvalguard(openai("gpt-4o-mini"), {
|
|
92
|
+
> apiKey: process.env.EVALGUARD_API_KEY!,
|
|
93
|
+
> projectId: "proj-123",
|
|
94
|
+
> });
|
|
95
|
+
>
|
|
96
|
+
> // …then pass `model` to generateText / streamText as the quickstart does.
|
|
97
|
+
> return model;
|
|
98
|
+
> }
|
|
99
|
+
>
|
|
100
|
+
> void main();
|
|
101
|
+
> ```
|
|
102
|
+
>
|
|
103
|
+
> The export is `withEvalguard` — lower-case `g`. This block said
|
|
104
|
+
> `withEvalGuard` until 2026-08-01, which does not exist and fails with
|
|
105
|
+
> `TS2339: Property 'withEvalGuard' does not exist`.
|
|
106
|
+
>
|
|
107
|
+
> If you swap in a provider package that is itself ESM-only, that import has to
|
|
108
|
+
> become dynamic as well — the rule is that **no ESM-only specifier may be
|
|
109
|
+
> imported statically from a CJS file**, not just this one.
|
|
110
|
+
>
|
|
111
|
+
> Node.js ≥ 22.12 can also `require()` an ESM module directly
|
|
112
|
+
> (`require(esm)`), but TypeScript still type-checks the import under CJS
|
|
113
|
+
> rules, so the dynamic-import form above is the supported path.
|
|
114
|
+
|
|
115
|
+
Peer requires `ai >= 5.0.0` (`ai@5`, `ai@6` and `ai@7` are all verified).
|
|
116
|
+
|
|
117
|
+
## Use
|
|
118
|
+
|
|
119
|
+
<!-- peer-matrix:quickstart -->
|
|
120
|
+
|
|
121
|
+
```ts
|
|
122
|
+
import { openai } from "@ai-sdk/openai";
|
|
123
|
+
import { generateText } from "ai";
|
|
124
|
+
import { withEvalguard } from "@evalguard/vercel-ai";
|
|
125
|
+
|
|
126
|
+
const model = withEvalguard(openai("gpt-4o-mini"), {
|
|
127
|
+
apiKey: process.env.EVALGUARD_API_KEY!,
|
|
128
|
+
projectId: "proj-123",
|
|
129
|
+
});
|
|
130
|
+
|
|
131
|
+
const { text } = await generateText({ model, prompt: "Say hi." });
|
|
132
|
+
```
|
|
133
|
+
|
|
134
|
+
<!-- /peer-matrix:quickstart -->
|
|
135
|
+
|
|
136
|
+
<sub>The two comments above are load-bearing:
|
|
137
|
+
`scripts/verify-vercel-ai-peer-matrix.mjs` extracts exactly what is between
|
|
138
|
+
them and type-checks that text against every peer major, so this block cannot
|
|
139
|
+
drift from what is actually verified. Removing them fails the gate.</sub>
|
|
140
|
+
|
|
141
|
+
That's it — every call now:
|
|
142
|
+
|
|
143
|
+
1. Runs your prompt through EvalGuard's 5-layer firewall (pattern, token,
|
|
144
|
+
semantic, output validation, allow-list precedence) before reaching
|
|
145
|
+
the LLM. Blocks on violation by default — set `blockOnViolation: false`
|
|
146
|
+
to log-only.
|
|
147
|
+
2. Records a trace with model, provider, input, output, latency, token
|
|
148
|
+
usage, and estimated cost to your EvalGuard project.
|
|
149
|
+
|
|
150
|
+
## Options
|
|
151
|
+
|
|
152
|
+
```ts
|
|
153
|
+
withEvalguard(model, {
|
|
154
|
+
apiKey: "...",
|
|
155
|
+
projectId: "proj-...", // optional — required to scope traces
|
|
156
|
+
baseUrl: "https://...", // optional — self-hosted EvalGuard
|
|
157
|
+
blockOnViolation: true, // default: true
|
|
158
|
+
disableGuardrails: false, // default: false
|
|
159
|
+
disableLogging: false, // default: false
|
|
160
|
+
metadata: { feature: "support-bot", env: "prod" },
|
|
161
|
+
});
|
|
162
|
+
```
|
|
163
|
+
|
|
164
|
+
## Streaming works too
|
|
165
|
+
|
|
166
|
+
Stream parts pass through verbatim. The trace is logged once on `finish`
|
|
167
|
+
with the assembled text and token totals.
|
|
168
|
+
|
|
169
|
+
```ts
|
|
170
|
+
import { streamText } from "ai";
|
|
171
|
+
|
|
172
|
+
const { textStream } = await streamText({ model, prompt: "..." });
|
|
173
|
+
for await (const chunk of textStream) {
|
|
174
|
+
process.stdout.write(chunk);
|
|
175
|
+
}
|
|
176
|
+
```
|
|
177
|
+
|
|
178
|
+
## Outage semantics — fail-CLOSED by default
|
|
179
|
+
|
|
180
|
+
> Corrected 2026-07-29. This section previously promised a fail-**open**
|
|
181
|
+
> "guarantee" ("falls back to allow + don't log"). The wrappers went
|
|
182
|
+
> fail-closed on 2026-05-28; the docs were never updated.
|
|
183
|
+
|
|
184
|
+
If EvalGuard's API is unreachable, slow, or returns an error:
|
|
185
|
+
|
|
186
|
+
- `blockOnViolation: true` (**default**) — the wrapper **throws**
|
|
187
|
+
`EvalguardBlockedError` with a `guardrail_unavailable` violation. The
|
|
188
|
+
provider call is not made.
|
|
189
|
+
- `blockOnViolation: false` — the call proceeds and the outage is recorded.
|
|
190
|
+
|
|
191
|
+
Trace-log errors are silently swallowed in both modes.
|
|
192
|
+
|
|
193
|
+
`EvalguardBlockedError` is also what a real firewall block raises, and it
|
|
194
|
+
is catchable distinctly:
|
|
195
|
+
|
|
196
|
+
```ts
|
|
197
|
+
import { EvalguardBlockedError } from "@evalguard/vercel-ai";
|
|
198
|
+
|
|
199
|
+
try {
|
|
200
|
+
const { text } = await generateText({ model, prompt: userInput });
|
|
201
|
+
} catch (err) {
|
|
202
|
+
if (err instanceof EvalguardBlockedError) {
|
|
203
|
+
return { error: "blocked", violations: err.violations };
|
|
204
|
+
}
|
|
205
|
+
throw err;
|
|
206
|
+
}
|
|
207
|
+
```
|
|
208
|
+
|
|
209
|
+
## Cost estimates: unpriced models are reported as unpriced
|
|
210
|
+
|
|
211
|
+
`estimateCost()` used to invent a price for any model outside a ~60-row table:
|
|
212
|
+
`estimateCost("gpt-5", 1000, 500)` and `estimateCost("totally-unknown-model",
|
|
213
|
+
1000, 500)` both returned `0.0105` from a blended $0.003/$0.015-per-1k
|
|
214
|
+
fallback — with no flag and no warning. A FinOps figure you cannot tell apart
|
|
215
|
+
from a real vendor price is worse than no figure at all.
|
|
216
|
+
|
|
217
|
+
Two things changed:
|
|
218
|
+
|
|
219
|
+
- **Coverage** — 2,200+ model ids now resolve to a real, sourced rate
|
|
220
|
+
(generated from EvalGuard's own pricing database, which is synced from the
|
|
221
|
+
LiteLLM catalogue). `gpt-5` is priced correctly.
|
|
222
|
+
- **Honesty** — a genuinely unknown model is now *visibly* unknown.
|
|
223
|
+
|
|
224
|
+
```typescript
|
|
225
|
+
import { estimateCostDetailed, isModelPriced } from "@evalguard/vercel-ai";
|
|
226
|
+
|
|
227
|
+
estimateCostDetailed("gpt-5", 1000, 500);
|
|
228
|
+
// { model: "gpt-5", costUsd: 0.00625, priced: true, pricingSource: "catalog" }
|
|
229
|
+
|
|
230
|
+
estimateCostDetailed("totally-unknown-model", 1000, 500);
|
|
231
|
+
// { model: "totally-unknown-model",
|
|
232
|
+
// costUsd: null, // <- never a fabricated number
|
|
233
|
+
// priced: false,
|
|
234
|
+
// pricingSource: "unpriced",
|
|
235
|
+
// blendedFallbackUsd: 0.0105 } // <- opt-in rough figure, clearly labelled
|
|
236
|
+
|
|
237
|
+
isModelPriced("totally-unknown-model"); // false
|
|
238
|
+
```
|
|
239
|
+
|
|
240
|
+
`estimateCost()` still returns a `number` for backwards compatibility, but it
|
|
241
|
+
now emits a one-time `console.warn` naming the unpriced model. Traces carry
|
|
242
|
+
`costPricingSource` alongside `cost`, and `cost` is `null` for an unpriced
|
|
243
|
+
model rather than a guess, so your EvalGuard dashboard shows "unpriced" instead
|
|
244
|
+
of a fake dollar amount.
|
|
245
|
+
|
|
246
|
+
## License
|
|
247
|
+
|
|
248
|
+
Apache-2.0
|
package/dist/cost.d.ts
ADDED
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"cost.d.ts","sourceRoot":"","sources":["../src/cost.ts"],"names":[],"mappings":"AAWA,OAAO,EAAE,YAAY,EAAE,MAAM,yBAAyB,CAAC;AAMvD,OAAO,EAAE,oBAAoB,EAAE,aAAa,EAAE,MAAM,yBAAyB,CAAC;AAC9E,YAAY,EAAE,YAAY,EAAE,aAAa,EAAE,MAAM,yBAAyB,CAAC"}
|
package/dist/cost.js
ADDED
|
@@ -0,0 +1,18 @@
|
|
|
1
|
+
// Cost estimation is shared across all EvalGuard wrappers — re-exported from
|
|
2
|
+
// @evalguard/wrapper-core so the model pricing table has a single source of
|
|
3
|
+
// truth (as anthropic-wrapper/src/cost.ts and openai-wrapper/src/cost.ts
|
|
4
|
+
// already do).
|
|
5
|
+
//
|
|
6
|
+
// This file used to carry its OWN partial price table that returned 0 for any
|
|
7
|
+
// model it didn't list. That silently contradicts wrapper-core's explicit
|
|
8
|
+
// policy — an unknown model falls back to a conservative $0.003/$0.015 per 1k
|
|
9
|
+
// because reporting 0 is worse than slightly-off: it MASKS the bill. A customer
|
|
10
|
+
// on a model missing from the local table (claude-sonnet-4-5, gpt-5, any new
|
|
11
|
+
// release) saw $0.00 spend in every trace while burning real provider budget.
|
|
12
|
+
export { estimateCost } from "@evalguard/wrapper-core";
|
|
13
|
+
// 2026-07-30 (audit finding 5): `estimateCostDetailed` is re-exported alongside
|
|
14
|
+
// it. The blended fallback above is no longer SILENT — an unpriced model now
|
|
15
|
+
// reports `{ costUsd: null, priced: false }` instead of a number a customer
|
|
16
|
+
// cannot tell apart from a real vendor price.
|
|
17
|
+
export { estimateCostDetailed, isModelPriced } from "@evalguard/wrapper-core";
|
|
18
|
+
//# sourceMappingURL=cost.js.map
|
package/dist/cost.js.map
ADDED
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"cost.js","sourceRoot":"","sources":["../src/cost.ts"],"names":[],"mappings":"AAAA,6EAA6E;AAC7E,4EAA4E;AAC5E,yEAAyE;AACzE,eAAe;AACf,EAAE;AACF,8EAA8E;AAC9E,0EAA0E;AAC1E,8EAA8E;AAC9E,gFAAgF;AAChF,6EAA6E;AAC7E,8EAA8E;AAC9E,OAAO,EAAE,YAAY,EAAE,MAAM,yBAAyB,CAAC;AAEvD,gFAAgF;AAChF,6EAA6E;AAC7E,4EAA4E;AAC5E,8CAA8C;AAC9C,OAAO,EAAE,oBAAoB,EAAE,aAAa,EAAE,MAAM,yBAAyB,CAAC"}
|
package/dist/index.d.ts
ADDED
|
@@ -0,0 +1,254 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* EvalGuard middleware for the Vercel AI SDK.
|
|
3
|
+
*
|
|
4
|
+
* Drop-in usage: wrap any language model from @ai-sdk/{openai,anthropic,
|
|
5
|
+
* google,...} — model spec v2 (`ai@5`), v3 (`ai@6`) or v4 (`ai@7`) —
|
|
6
|
+
* with `withEvalguard()` to get pre-call guardrails (firewall
|
|
7
|
+
* check) + post-call trace logging (latency, tokens, cost) sent to your
|
|
8
|
+
* EvalGuard project. Guardrail semantics are fail-CLOSED when
|
|
9
|
+
* blockOnViolation is on (the default): if the firewall is unreachable the
|
|
10
|
+
* call is BLOCKED with an EvalguardBlockedError, matching the langchain /
|
|
11
|
+
* llamaindex wrappers. In monitor-only mode (blockOnViolation:false) a
|
|
12
|
+
* guardrail outage is swallowed and the call proceeds. Trace logging always
|
|
13
|
+
* fails open — a logging error never breaks or delays the user's call.
|
|
14
|
+
*
|
|
15
|
+
* Usage:
|
|
16
|
+
*
|
|
17
|
+
* import { openai } from "@ai-sdk/openai";
|
|
18
|
+
* import { generateText } from "ai";
|
|
19
|
+
* import { withEvalguard } from "@evalguard/vercel-ai";
|
|
20
|
+
*
|
|
21
|
+
* const model = withEvalguard(openai("gpt-4o-mini"), {
|
|
22
|
+
* apiKey: process.env.EVALGUARD_API_KEY!,
|
|
23
|
+
* projectId: "proj-123",
|
|
24
|
+
* });
|
|
25
|
+
*
|
|
26
|
+
* const { text } = await generateText({ model, prompt: "hi" });
|
|
27
|
+
*
|
|
28
|
+
* The Vercel AI SDK's `wrapLanguageModel` is the documented composition hook
|
|
29
|
+
* for exactly this use case. We deliberately do NOT import it (nor `ai` itself)
|
|
30
|
+
* at RUNTIME: the only `dependencies` entry is `@evalguard/wrapper-core`, and
|
|
31
|
+
* `ai` is a PEER, so the version the customer already chose is the one that
|
|
32
|
+
* runs — we never resolve a second copy into their tree, and there is no
|
|
33
|
+
* import of `ai` to break if they are on a major we did not anticipate.
|
|
34
|
+
*
|
|
35
|
+
* That is NOT the same as "installing this package pulls nothing in".
|
|
36
|
+
* npm 7+ installs peerDependencies automatically, and it was measured doing
|
|
37
|
+
* exactly that on 2026-08-02:
|
|
38
|
+
*
|
|
39
|
+
* $ npm install ./evalguard-vercel-ai-1.0.0.tgz # clean `npm init -y`
|
|
40
|
+
* added 12 packages
|
|
41
|
+
* $ npm ls --all
|
|
42
|
+
* └─┬ @evalguard/vercel-ai@1.0.0
|
|
43
|
+
* └── ai@7.0.48
|
|
44
|
+
*
|
|
45
|
+
* The comment here previously claimed the opposite ("never pulls the AI SDK in
|
|
46
|
+
* as a transitive dependency", and before that "a tiny, zero-runtime-dependency
|
|
47
|
+
* install"), which also contradicted the README two paragraphs below it
|
|
48
|
+
* ("`ai` is a peer dependency (npm installs it for you…)"). This JSDoc block
|
|
49
|
+
* ships to consumers inside `dist/index.d.ts`, so it was a wrong claim in the
|
|
50
|
+
* published artifact, not just an internal note.
|
|
51
|
+
*
|
|
52
|
+
* Instead of importing `wrapLanguageModel` we re-implement the same "delegate
|
|
53
|
+
* to the model, override doGenerate/doStream" shape directly, as a PROXY over
|
|
54
|
+
* the original model. Everything except the two call methods is forwarded to the
|
|
55
|
+
* real instance, so capability fields — most importantly `supportedUrls` —
|
|
56
|
+
* pass through untouched. Two earlier versions got this wrong:
|
|
57
|
+
* 1. hand-copying a fixed list of V1 props (`supportsUrl`,
|
|
58
|
+
* `supportsImageUrls`, …) that no longer exist on a V2 model, which
|
|
59
|
+
* silently STRIPPED `supportedUrls` (audit MEDIUM 2026-07-14);
|
|
60
|
+
* 2. `{ ...model }` — an object spread copies OWN ENUMERABLE properties
|
|
61
|
+
* only. Every real provider model (`openai(...)`, `anthropic(...)`) is a
|
|
62
|
+
* CLASS INSTANCE whose `supportedUrls` / `provider` / `modelId` are
|
|
63
|
+
* prototype GETTERS, so the spread produced `{ doGenerate, doStream }`
|
|
64
|
+
* and nothing else — reintroducing exactly the `supportedUrls` loss the
|
|
65
|
+
* spread was introduced to fix, and dropping every provider method the
|
|
66
|
+
* SDK or downstream middleware might call (audit 2026-07-25).
|
|
67
|
+
* A Proxy is also the pattern the openai / anthropic / gemini wrappers in this
|
|
68
|
+
* monorepo already use.
|
|
69
|
+
*
|
|
70
|
+
* Spec reference: https://sdk.vercel.ai/docs/ai-sdk-core/middleware
|
|
71
|
+
*/
|
|
72
|
+
import { type GuardrailCheckResult } from "@evalguard/wrapper-core";
|
|
73
|
+
/**
|
|
74
|
+
* A single token count as the AI SDK has spelled it over time.
|
|
75
|
+
*
|
|
76
|
+
* - `number` — model spec v2 (`ai@5`): `usage.inputTokens` is the count itself.
|
|
77
|
+
* - `{ total, … }` — model spec v3 (`ai@6`) and v4 (`ai@7`): the flat count
|
|
78
|
+
* became a breakdown object (`{ total, noCache, cacheRead, cacheWrite }` on
|
|
79
|
+
* input, `{ total, text, reasoning }` on output).
|
|
80
|
+
*
|
|
81
|
+
* Reading `usage.inputTokens` directly on a v3/v4 model yields an OBJECT, which
|
|
82
|
+
* is why {@link readTokenCount} exists. Before 2026-08-02 that object flowed
|
|
83
|
+
* straight into `tokenUsage.input` and into the cost estimator, so every trace
|
|
84
|
+
* from an `ai@6` or `ai@7` consumer carried a non-numeric token field and a NaN
|
|
85
|
+
* cost — two of the three majors this package declares support for, including
|
|
86
|
+
* the one pinned as its own devDependency.
|
|
87
|
+
*/
|
|
88
|
+
type LanguageModelTokenCount = number | {
|
|
89
|
+
total?: number | undefined;
|
|
90
|
+
[k: string]: unknown;
|
|
91
|
+
} | undefined;
|
|
92
|
+
interface LanguageModelV2Usage {
|
|
93
|
+
inputTokens?: LanguageModelTokenCount;
|
|
94
|
+
outputTokens?: LanguageModelTokenCount;
|
|
95
|
+
promptTokens?: number;
|
|
96
|
+
completionTokens?: number;
|
|
97
|
+
}
|
|
98
|
+
interface LanguageModelV2TextContentPart {
|
|
99
|
+
type: "text";
|
|
100
|
+
text: string;
|
|
101
|
+
[k: string]: unknown;
|
|
102
|
+
}
|
|
103
|
+
type LanguageModelV2ContentPart = LanguageModelV2TextContentPart | {
|
|
104
|
+
type: string;
|
|
105
|
+
[k: string]: unknown;
|
|
106
|
+
};
|
|
107
|
+
interface LanguageModelV2CallOptions {
|
|
108
|
+
prompt: unknown;
|
|
109
|
+
[k: string]: unknown;
|
|
110
|
+
}
|
|
111
|
+
interface LanguageModelV2GenerateResult {
|
|
112
|
+
content?: LanguageModelV2ContentPart[];
|
|
113
|
+
text?: string;
|
|
114
|
+
toolCalls?: unknown;
|
|
115
|
+
usage?: LanguageModelV2Usage;
|
|
116
|
+
[k: string]: unknown;
|
|
117
|
+
}
|
|
118
|
+
/**
|
|
119
|
+
* `stream` is `ReadableStream<unknown>`, NOT
|
|
120
|
+
* `ReadableStream<LanguageModelV2StreamPart>`, on purpose.
|
|
121
|
+
*
|
|
122
|
+
* `ReadableStream<T>` is invariant in `T` (its reader's `read()` puts `T` in
|
|
123
|
+
* both an input and an output position), so a real provider's
|
|
124
|
+
* `ReadableStream<@ai-sdk/provider.LanguageModelV2StreamPart>` is NOT
|
|
125
|
+
* assignable to a stream of our locally-shimmed part union even though the
|
|
126
|
+
* part types themselves are structurally compatible. The consequence was that
|
|
127
|
+
* the README's own headline example —
|
|
128
|
+
* `withEvalguard(openai("gpt-4o-mini"), { … })` — did not type-check against
|
|
129
|
+
* `@ai-sdk/openai@2` / `ai@5` at all:
|
|
130
|
+
*
|
|
131
|
+
* error TS2345: Argument of type 'LanguageModelV2' is not assignable to
|
|
132
|
+
* parameter of type 'LanguageModelV2'. The types returned by
|
|
133
|
+
* 'doStream(...)' are incompatible … Types of property 'stream' are
|
|
134
|
+
* incompatible.
|
|
135
|
+
*
|
|
136
|
+
* It failed identically in ESM and CJS, so it was not the ESM-only hazard the
|
|
137
|
+
* README warns about — it was this shim. `unknown` removes the variance
|
|
138
|
+
* conflict; the `pull()` handler in `withEvalguard`'s `doStream` narrows each
|
|
139
|
+
* chunk back to {@link LanguageModelV2StreamPart} at the single read boundary,
|
|
140
|
+
* and it already tolerates an unrecognised `type` by passing the chunk through.
|
|
141
|
+
*
|
|
142
|
+
* 2026-08-02 correction: this comment used to end "Verified 2026-08-01",
|
|
143
|
+
* which read as "the README example now compiles". It did not. Fixing the
|
|
144
|
+
* ARGUMENT position left the RETURN position broken — `withEvalguard` still
|
|
145
|
+
* returned this file's shim, which is not assignable to `ai`'s `LanguageModel`,
|
|
146
|
+
* so `generateText({ model })` was TS2322 on ai@5/6/7 (and TS2345 again on
|
|
147
|
+
* ai@6/7 from the hard-coded `specificationVersion: "v2"`). Both are fixed;
|
|
148
|
+
* see {@link withEvalguard} and `src/__tests__/ai-sdk-assignability.test.ts`,
|
|
149
|
+
* which type-checks the wrapper against the real `ai` types so a future
|
|
150
|
+
* regression fails CI instead of only failing consumers.
|
|
151
|
+
*/
|
|
152
|
+
interface LanguageModelV2StreamResult {
|
|
153
|
+
stream: ReadableStream<unknown>;
|
|
154
|
+
[k: string]: unknown;
|
|
155
|
+
}
|
|
156
|
+
/**
|
|
157
|
+
* Structural shim for "a model the AI SDK will accept".
|
|
158
|
+
*
|
|
159
|
+
* This type is used ONLY as the constraint on {@link withEvalguard}'s `model`
|
|
160
|
+
* parameter — it is deliberately lenient (optional fields, index signatures)
|
|
161
|
+
* so a real provider model assigns cleanly. It is NOT assignable back to
|
|
162
|
+
* `ai`'s `LanguageModel`, which is why `withEvalguard` is generic and returns
|
|
163
|
+
* the caller's own model type rather than this shim (see the note on
|
|
164
|
+
* `withEvalguard`).
|
|
165
|
+
*/
|
|
166
|
+
export interface LanguageModelV2 {
|
|
167
|
+
/**
|
|
168
|
+
* `string`, not a `"v2"` literal. The peer range admits `ai@5` (model spec
|
|
169
|
+
* v2), `ai@6` (v3) and `ai@7` (v4), and the wrapper is spec-version agnostic:
|
|
170
|
+
* it is a Proxy that forwards everything and only intercepts
|
|
171
|
+
* `doGenerate` / `doStream`, both of which exist in every spec version.
|
|
172
|
+
* Pinning the literal to `"v2"` made `withEvalguard(openai("gpt-4o-mini"))`
|
|
173
|
+
* fail with `TS2345: Type '"v3"' is not assignable to type '"v2"'` on ai@6
|
|
174
|
+
* and ai@7 — a hard-coded literal that ages out with every AI SDK major.
|
|
175
|
+
*/
|
|
176
|
+
specificationVersion: string;
|
|
177
|
+
provider: string;
|
|
178
|
+
modelId: string;
|
|
179
|
+
/**
|
|
180
|
+
* V2 capability map — supported URL patterns by media type. Forwarded
|
|
181
|
+
* unchanged by the wrapper's Proxy (see `withEvalguard`) so URL and file
|
|
182
|
+
* inputs the model handles natively are NOT downloaded/dropped — including
|
|
183
|
+
* when the provider exposes it as a prototype getter on a class instance.
|
|
184
|
+
*/
|
|
185
|
+
supportedUrls?: Record<string, RegExp[]> | PromiseLike<Record<string, RegExp[]>>;
|
|
186
|
+
doGenerate(options: LanguageModelV2CallOptions): PromiseLike<LanguageModelV2GenerateResult>;
|
|
187
|
+
doStream(options: LanguageModelV2CallOptions): PromiseLike<LanguageModelV2StreamResult>;
|
|
188
|
+
}
|
|
189
|
+
/**
|
|
190
|
+
* @deprecated The Vercel AI SDK moved to LanguageModelV2 in v5+. Kept as an
|
|
191
|
+
* alias so pre-existing imports keep compiling; new code should use
|
|
192
|
+
* {@link LanguageModelV2}.
|
|
193
|
+
*/
|
|
194
|
+
export type LanguageModelV1 = LanguageModelV2;
|
|
195
|
+
export interface WithEvalguardConfig {
|
|
196
|
+
/** EvalGuard API key (https://evalguard.ai/dashboard/api-keys). */
|
|
197
|
+
apiKey: string;
|
|
198
|
+
/** Project ID — traces appear under this project. Optional in dev. */
|
|
199
|
+
projectId?: string;
|
|
200
|
+
/** Override the EvalGuard API base URL. Default: https://evalguard.ai/api/v1 */
|
|
201
|
+
baseUrl?: string;
|
|
202
|
+
/**
|
|
203
|
+
* Block calls that fail the guardrail check. Default: true.
|
|
204
|
+
* If false, violations are still logged but the call proceeds.
|
|
205
|
+
*/
|
|
206
|
+
blockOnViolation?: boolean;
|
|
207
|
+
/** Disable trace logging (still runs guardrails). Default: false. */
|
|
208
|
+
disableLogging?: boolean;
|
|
209
|
+
/** Disable guardrail input check (still logs). Default: false. */
|
|
210
|
+
disableGuardrails?: boolean;
|
|
211
|
+
/** Free-form metadata attached to every trace. */
|
|
212
|
+
metadata?: Record<string, unknown>;
|
|
213
|
+
}
|
|
214
|
+
/**
|
|
215
|
+
* Error thrown when blockOnViolation=true and the input is rejected by
|
|
216
|
+
* EvalGuard's firewall. Customers can catch this distinctly from provider
|
|
217
|
+
* errors.
|
|
218
|
+
*/
|
|
219
|
+
export declare class EvalguardBlockedError extends Error {
|
|
220
|
+
readonly violations: GuardrailCheckResult["violations"];
|
|
221
|
+
constructor(message: string, violations: GuardrailCheckResult["violations"]);
|
|
222
|
+
}
|
|
223
|
+
/**
|
|
224
|
+
* Wrap a Vercel AI SDK language model with EvalGuard middleware.
|
|
225
|
+
*
|
|
226
|
+
* Returns a Proxy over the original model, typed as **the caller's own model
|
|
227
|
+
* type** (`TModel`) rather than as our shim. That genericity is load-bearing,
|
|
228
|
+
* not stylistic: `ai`'s `LanguageModel` is a union of the REAL
|
|
229
|
+
* `@ai-sdk/provider` model specs, and {@link LanguageModelV2} above is a
|
|
230
|
+
* deliberately lenient structural shim that is not a member of that union
|
|
231
|
+
* (`supportedUrls` optional here vs required there; every `doGenerate` result
|
|
232
|
+
* field optional here vs required there). Returning the shim made the README's
|
|
233
|
+
* own headline example fail on every supported peer:
|
|
234
|
+
*
|
|
235
|
+
* error TS2322: Type 'LanguageModelV2' is not assignable to type
|
|
236
|
+
* 'LanguageModel'.
|
|
237
|
+
*
|
|
238
|
+
* at `generateText({ model, prompt })` — reproduced on ai@5, ai@6 and ai@7.
|
|
239
|
+
* Returning `TModel` hands `generateText` back exactly what `openai(...)` gave
|
|
240
|
+
* us, so it assigns by construction. Verified against ai@5 + @ai-sdk/openai@2,
|
|
241
|
+
* ai@6 + @ai-sdk/openai@3 and ai@7 + @ai-sdk/openai@4 from packed tarballs on
|
|
242
|
+
* 2026-08-02, and pinned by `src/__tests__/ai-sdk-assignability.test.ts`.
|
|
243
|
+
*
|
|
244
|
+
* Only `doGenerate` / `doStream` are intercepted; every other property read
|
|
245
|
+
* (including prototype getters such as `supportedUrls` on a class-based
|
|
246
|
+
* provider model) is forwarded to the real instance, so capability fields
|
|
247
|
+
* compose with other middleware (caching, logging) the customer may already
|
|
248
|
+
* have.
|
|
249
|
+
*/
|
|
250
|
+
export declare function withEvalguard<TModel extends LanguageModelV2>(model: TModel, config: WithEvalguardConfig): TModel;
|
|
251
|
+
export type { GuardrailCheckResult, GuardrailViolation, TraceLogData } from "@evalguard/wrapper-core";
|
|
252
|
+
export { estimateCost, estimateCostDetailed, isModelPriced } from "./cost.js";
|
|
253
|
+
export type { CostEstimate, PricingSource } from "./cost.js";
|
|
254
|
+
//# sourceMappingURL=index.d.ts.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;GAsEG;AAEH,OAAO,EAAuC,KAAK,oBAAoB,EAAE,MAAM,yBAAyB,CAAC;AAqBzG;;;;;;;;;;;;;;GAcG;AACH,KAAK,uBAAuB,GACxB,MAAM,GACN;IAAE,KAAK,CAAC,EAAE,MAAM,GAAG,SAAS,CAAC;IAAC,CAAC,CAAC,EAAE,MAAM,GAAG,OAAO,CAAA;CAAE,GACpD,SAAS,CAAC;AAEd,UAAU,oBAAoB;IAE5B,WAAW,CAAC,EAAE,uBAAuB,CAAC;IACtC,YAAY,CAAC,EAAE,uBAAuB,CAAC;IAGvC,YAAY,CAAC,EAAE,MAAM,CAAC;IACtB,gBAAgB,CAAC,EAAE,MAAM,CAAC;CAC3B;AAED,UAAU,8BAA8B;IACtC,IAAI,EAAE,MAAM,CAAC;IACb,IAAI,EAAE,MAAM,CAAC;IACb,CAAC,CAAC,EAAE,MAAM,GAAG,OAAO,CAAC;CACtB;AAED,KAAK,0BAA0B,GAC3B,8BAA8B,GAC9B;IAAE,IAAI,EAAE,MAAM,CAAC;IAAC,CAAC,CAAC,EAAE,MAAM,GAAG,OAAO,CAAA;CAAE,CAAC;AAoB3C,UAAU,0BAA0B;IAClC,MAAM,EAAE,OAAO,CAAC;IAChB,CAAC,CAAC,EAAE,MAAM,GAAG,OAAO,CAAC;CACtB;AAED,UAAU,6BAA6B;IAErC,OAAO,CAAC,EAAE,0BAA0B,EAAE,CAAC;IAEvC,IAAI,CAAC,EAAE,MAAM,CAAC;IACd,SAAS,CAAC,EAAE,OAAO,CAAC;IACpB,KAAK,CAAC,EAAE,oBAAoB,CAAC;IAC7B,CAAC,CAAC,EAAE,MAAM,GAAG,OAAO,CAAC;CACtB;AAED;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;GAiCG;AACH,UAAU,2BAA2B;IACnC,MAAM,EAAE,cAAc,CAAC,OAAO,CAAC,CAAC;IAChC,CAAC,CAAC,EAAE,MAAM,GAAG,OAAO,CAAC;CACtB;AAED;;;;;;;;;GASG;AACH,MAAM,WAAW,eAAe;IAC9B;;;;;;;;OAQG;IACH,oBAAoB,EAAE,MAAM,CAAC;IAC7B,QAAQ,EAAE,MAAM,CAAC;IACjB,OAAO,EAAE,MAAM,CAAC;IAChB;;;;;OAKG;IACH,aAAa,CAAC,EACV,MAAM,CAAC,MAAM,EAAE,MAAM,EAAE,CAAC,GACxB,WAAW,CAAC,MAAM,CAAC,MAAM,EAAE,MAAM,EAAE,CAAC,CAAC,CAAC;IAC1C,UAAU,CAAC,OAAO,EAAE,0BAA0B,GAAG,WAAW,CAAC,6BAA6B,CAAC,CAAC;IAC5F,QAAQ,CAAC,OAAO,EAAE,0BAA0B,GAAG,WAAW,CAAC,2BAA2B,CAAC,CAAC;CASzF;AAED;;;;GAIG;AACH,MAAM,MAAM,eAAe,GAAG,eAAe,CAAC;AAE9C,MAAM,WAAW,mBAAmB;IAClC,mEAAmE;IACnE,MAAM,EAAE,MAAM,CAAC;IACf,sEAAsE;IACtE,SAAS,CAAC,EAAE,MAAM,CAAC;IACnB,gFAAgF;IAChF,OAAO,CAAC,EAAE,MAAM,CAAC;IACjB;;;OAGG;IACH,gBAAgB,CAAC,EAAE,OAAO,CAAC;IAC3B,qEAAqE;IACrE,cAAc,CAAC,EAAE,OAAO,CAAC;IACzB,kEAAkE;IAClE,iBAAiB,CAAC,EAAE,OAAO,CAAC;IAC5B,kDAAkD;IAClD,QAAQ,CAAC,EAAE,MAAM,CAAC,MAAM,EAAE,OAAO,CAAC,CAAC;CACpC;AAED;;;;GAIG;AACH,qBAAa,qBAAsB,SAAQ,KAAK;IAC9C,QAAQ,CAAC,UAAU,EAAE,oBAAoB,CAAC,YAAY,CAAC,CAAC;gBAC5C,OAAO,EAAE,MAAM,EAAE,UAAU,EAAE,oBAAoB,CAAC,YAAY,CAAC;CAK5E;AA0CD;;;;;;;;;;;;;;;;;;;;;;;;;;GA0BG;AACH,wBAAgB,aAAa,CAAC,MAAM,SAAS,eAAe,EAC1D,KAAK,EAAE,MAAM,EACb,MAAM,EAAE,mBAAmB,GAC1B,MAAM,CAkPR;AA0BD,YAAY,EAAE,oBAAoB,EAAE,kBAAkB,EAAE,YAAY,EAAE,MAAM,yBAAyB,CAAC;AAStG,OAAO,EAAE,YAAY,EAAE,oBAAoB,EAAE,aAAa,EAAE,MAAM,WAAW,CAAC;AAC9E,YAAY,EAAE,YAAY,EAAE,aAAa,EAAE,MAAM,WAAW,CAAC"}
|
package/dist/index.js
ADDED
|
@@ -0,0 +1,395 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* EvalGuard middleware for the Vercel AI SDK.
|
|
3
|
+
*
|
|
4
|
+
* Drop-in usage: wrap any language model from @ai-sdk/{openai,anthropic,
|
|
5
|
+
* google,...} — model spec v2 (`ai@5`), v3 (`ai@6`) or v4 (`ai@7`) —
|
|
6
|
+
* with `withEvalguard()` to get pre-call guardrails (firewall
|
|
7
|
+
* check) + post-call trace logging (latency, tokens, cost) sent to your
|
|
8
|
+
* EvalGuard project. Guardrail semantics are fail-CLOSED when
|
|
9
|
+
* blockOnViolation is on (the default): if the firewall is unreachable the
|
|
10
|
+
* call is BLOCKED with an EvalguardBlockedError, matching the langchain /
|
|
11
|
+
* llamaindex wrappers. In monitor-only mode (blockOnViolation:false) a
|
|
12
|
+
* guardrail outage is swallowed and the call proceeds. Trace logging always
|
|
13
|
+
* fails open — a logging error never breaks or delays the user's call.
|
|
14
|
+
*
|
|
15
|
+
* Usage:
|
|
16
|
+
*
|
|
17
|
+
* import { openai } from "@ai-sdk/openai";
|
|
18
|
+
* import { generateText } from "ai";
|
|
19
|
+
* import { withEvalguard } from "@evalguard/vercel-ai";
|
|
20
|
+
*
|
|
21
|
+
* const model = withEvalguard(openai("gpt-4o-mini"), {
|
|
22
|
+
* apiKey: process.env.EVALGUARD_API_KEY!,
|
|
23
|
+
* projectId: "proj-123",
|
|
24
|
+
* });
|
|
25
|
+
*
|
|
26
|
+
* const { text } = await generateText({ model, prompt: "hi" });
|
|
27
|
+
*
|
|
28
|
+
* The Vercel AI SDK's `wrapLanguageModel` is the documented composition hook
|
|
29
|
+
* for exactly this use case. We deliberately do NOT import it (nor `ai` itself)
|
|
30
|
+
* at RUNTIME: the only `dependencies` entry is `@evalguard/wrapper-core`, and
|
|
31
|
+
* `ai` is a PEER, so the version the customer already chose is the one that
|
|
32
|
+
* runs — we never resolve a second copy into their tree, and there is no
|
|
33
|
+
* import of `ai` to break if they are on a major we did not anticipate.
|
|
34
|
+
*
|
|
35
|
+
* That is NOT the same as "installing this package pulls nothing in".
|
|
36
|
+
* npm 7+ installs peerDependencies automatically, and it was measured doing
|
|
37
|
+
* exactly that on 2026-08-02:
|
|
38
|
+
*
|
|
39
|
+
* $ npm install ./evalguard-vercel-ai-1.0.0.tgz # clean `npm init -y`
|
|
40
|
+
* added 12 packages
|
|
41
|
+
* $ npm ls --all
|
|
42
|
+
* └─┬ @evalguard/vercel-ai@1.0.0
|
|
43
|
+
* └── ai@7.0.48
|
|
44
|
+
*
|
|
45
|
+
* The comment here previously claimed the opposite ("never pulls the AI SDK in
|
|
46
|
+
* as a transitive dependency", and before that "a tiny, zero-runtime-dependency
|
|
47
|
+
* install"), which also contradicted the README two paragraphs below it
|
|
48
|
+
* ("`ai` is a peer dependency (npm installs it for you…)"). This JSDoc block
|
|
49
|
+
* ships to consumers inside `dist/index.d.ts`, so it was a wrong claim in the
|
|
50
|
+
* published artifact, not just an internal note.
|
|
51
|
+
*
|
|
52
|
+
* Instead of importing `wrapLanguageModel` we re-implement the same "delegate
|
|
53
|
+
* to the model, override doGenerate/doStream" shape directly, as a PROXY over
|
|
54
|
+
* the original model. Everything except the two call methods is forwarded to the
|
|
55
|
+
* real instance, so capability fields — most importantly `supportedUrls` —
|
|
56
|
+
* pass through untouched. Two earlier versions got this wrong:
|
|
57
|
+
* 1. hand-copying a fixed list of V1 props (`supportsUrl`,
|
|
58
|
+
* `supportsImageUrls`, …) that no longer exist on a V2 model, which
|
|
59
|
+
* silently STRIPPED `supportedUrls` (audit MEDIUM 2026-07-14);
|
|
60
|
+
* 2. `{ ...model }` — an object spread copies OWN ENUMERABLE properties
|
|
61
|
+
* only. Every real provider model (`openai(...)`, `anthropic(...)`) is a
|
|
62
|
+
* CLASS INSTANCE whose `supportedUrls` / `provider` / `modelId` are
|
|
63
|
+
* prototype GETTERS, so the spread produced `{ doGenerate, doStream }`
|
|
64
|
+
* and nothing else — reintroducing exactly the `supportedUrls` loss the
|
|
65
|
+
* spread was introduced to fix, and dropping every provider method the
|
|
66
|
+
* SDK or downstream middleware might call (audit 2026-07-25).
|
|
67
|
+
* A Proxy is also the pattern the openai / anthropic / gemini wrappers in this
|
|
68
|
+
* monorepo already use.
|
|
69
|
+
*
|
|
70
|
+
* Spec reference: https://sdk.vercel.ai/docs/ai-sdk-core/middleware
|
|
71
|
+
*/
|
|
72
|
+
import { GuardrailClient, collapsePromptText } from "@evalguard/wrapper-core";
|
|
73
|
+
import { estimateCostDetailed } from "./cost.js";
|
|
74
|
+
/**
|
|
75
|
+
* Error thrown when blockOnViolation=true and the input is rejected by
|
|
76
|
+
* EvalGuard's firewall. Customers can catch this distinctly from provider
|
|
77
|
+
* errors.
|
|
78
|
+
*/
|
|
79
|
+
export class EvalguardBlockedError extends Error {
|
|
80
|
+
violations;
|
|
81
|
+
constructor(message, violations) {
|
|
82
|
+
super(message);
|
|
83
|
+
this.name = "EvalguardBlockedError";
|
|
84
|
+
this.violations = violations;
|
|
85
|
+
}
|
|
86
|
+
}
|
|
87
|
+
/**
|
|
88
|
+
* Normalise one token count to a `number`, whatever spec version produced it.
|
|
89
|
+
* Returns `undefined` (not 0) when the count is absent or non-numeric, so the
|
|
90
|
+
* caller can fall through to the legacy v1 field names.
|
|
91
|
+
*/
|
|
92
|
+
function readTokenCount(count) {
|
|
93
|
+
if (typeof count === "number")
|
|
94
|
+
return Number.isFinite(count) ? count : undefined;
|
|
95
|
+
if (count && typeof count === "object" && typeof count.total === "number") {
|
|
96
|
+
return Number.isFinite(count.total) ? count.total : undefined;
|
|
97
|
+
}
|
|
98
|
+
return undefined;
|
|
99
|
+
}
|
|
100
|
+
/** Pull token counts off a v2/v3/v4 (or legacy v1) usage object. */
|
|
101
|
+
function readTokens(usage) {
|
|
102
|
+
return {
|
|
103
|
+
input: readTokenCount(usage?.inputTokens) ?? usage?.promptTokens ?? 0,
|
|
104
|
+
output: readTokenCount(usage?.outputTokens) ?? usage?.completionTokens ?? 0,
|
|
105
|
+
};
|
|
106
|
+
}
|
|
107
|
+
/**
|
|
108
|
+
* Extract the assistant's text from a v2 doGenerate result. V2 returns an
|
|
109
|
+
* ordered `content` array (text lives in `{ type: "text", text }` parts);
|
|
110
|
+
* we concatenate them. Falls back to a legacy top-level `text` so a v1-shaped
|
|
111
|
+
* result still logs its output instead of `null`.
|
|
112
|
+
*/
|
|
113
|
+
function extractOutputText(result) {
|
|
114
|
+
if (Array.isArray(result.content)) {
|
|
115
|
+
return result.content
|
|
116
|
+
.filter((p) => !!p && p.type === "text" && typeof p.text === "string")
|
|
117
|
+
.map((p) => p.text)
|
|
118
|
+
.join("");
|
|
119
|
+
}
|
|
120
|
+
return typeof result.text === "string" ? result.text : "";
|
|
121
|
+
}
|
|
122
|
+
/**
|
|
123
|
+
* Wrap a Vercel AI SDK language model with EvalGuard middleware.
|
|
124
|
+
*
|
|
125
|
+
* Returns a Proxy over the original model, typed as **the caller's own model
|
|
126
|
+
* type** (`TModel`) rather than as our shim. That genericity is load-bearing,
|
|
127
|
+
* not stylistic: `ai`'s `LanguageModel` is a union of the REAL
|
|
128
|
+
* `@ai-sdk/provider` model specs, and {@link LanguageModelV2} above is a
|
|
129
|
+
* deliberately lenient structural shim that is not a member of that union
|
|
130
|
+
* (`supportedUrls` optional here vs required there; every `doGenerate` result
|
|
131
|
+
* field optional here vs required there). Returning the shim made the README's
|
|
132
|
+
* own headline example fail on every supported peer:
|
|
133
|
+
*
|
|
134
|
+
* error TS2322: Type 'LanguageModelV2' is not assignable to type
|
|
135
|
+
* 'LanguageModel'.
|
|
136
|
+
*
|
|
137
|
+
* at `generateText({ model, prompt })` — reproduced on ai@5, ai@6 and ai@7.
|
|
138
|
+
* Returning `TModel` hands `generateText` back exactly what `openai(...)` gave
|
|
139
|
+
* us, so it assigns by construction. Verified against ai@5 + @ai-sdk/openai@2,
|
|
140
|
+
* ai@6 + @ai-sdk/openai@3 and ai@7 + @ai-sdk/openai@4 from packed tarballs on
|
|
141
|
+
* 2026-08-02, and pinned by `src/__tests__/ai-sdk-assignability.test.ts`.
|
|
142
|
+
*
|
|
143
|
+
* Only `doGenerate` / `doStream` are intercepted; every other property read
|
|
144
|
+
* (including prototype getters such as `supportedUrls` on a class-based
|
|
145
|
+
* provider model) is forwarded to the real instance, so capability fields
|
|
146
|
+
* compose with other middleware (caching, logging) the customer may already
|
|
147
|
+
* have.
|
|
148
|
+
*/
|
|
149
|
+
export function withEvalguard(model, config) {
|
|
150
|
+
const guardrail = new GuardrailClient({ apiKey: config.apiKey, baseUrl: config.baseUrl });
|
|
151
|
+
const blockOnViolation = config.blockOnViolation ?? true;
|
|
152
|
+
const enableLogging = !config.disableLogging;
|
|
153
|
+
const enableGuardrails = !config.disableGuardrails;
|
|
154
|
+
const overrides = {
|
|
155
|
+
async doGenerate(params) {
|
|
156
|
+
const t0 = performance.now();
|
|
157
|
+
let guardrailResult;
|
|
158
|
+
// Pre-call: extract a single text representation of the prompt for
|
|
159
|
+
// the firewall. Vercel's prompt shape is varied (chat / completion),
|
|
160
|
+
// so we collapse to a string the firewall can scan.
|
|
161
|
+
if (enableGuardrails) {
|
|
162
|
+
const promptText = collapsePrompt(params);
|
|
163
|
+
if (promptText) {
|
|
164
|
+
try {
|
|
165
|
+
guardrailResult = await guardrail.checkInput(promptText, {
|
|
166
|
+
provider: model.provider,
|
|
167
|
+
model: model.modelId,
|
|
168
|
+
...config.metadata,
|
|
169
|
+
});
|
|
170
|
+
if (!guardrailResult.allowed && blockOnViolation) {
|
|
171
|
+
throw new EvalguardBlockedError(`Request blocked by EvalGuard guardrails: ${guardrailResult.violations.map((v) => v.type).join(", ")}`, guardrailResult.violations);
|
|
172
|
+
}
|
|
173
|
+
}
|
|
174
|
+
catch (err) {
|
|
175
|
+
if (err instanceof EvalguardBlockedError)
|
|
176
|
+
throw err;
|
|
177
|
+
// wrapper-core.checkInput fails CLOSED on a guardrail outage
|
|
178
|
+
// (throws once retries/circuit give up). Honor blockOnViolation:
|
|
179
|
+
// an unreachable guardrail BLOCKS the call when it's on, and is
|
|
180
|
+
// swallowed (the call proceeds, monitor-only) when it's off.
|
|
181
|
+
// Mirrors the langchain / llamaindex wrappers.
|
|
182
|
+
if (blockOnViolation) {
|
|
183
|
+
throw new EvalguardBlockedError(`Request blocked: guardrail unavailable (${err instanceof Error ? err.message : "unknown"})`, [{
|
|
184
|
+
type: "guardrail_unavailable",
|
|
185
|
+
severity: "high",
|
|
186
|
+
message: err instanceof Error ? err.message : "guardrail check failed",
|
|
187
|
+
}]);
|
|
188
|
+
}
|
|
189
|
+
}
|
|
190
|
+
}
|
|
191
|
+
}
|
|
192
|
+
const result = await model.doGenerate(params);
|
|
193
|
+
const latencyMs = Math.round(performance.now() - t0);
|
|
194
|
+
// Post-call: log trace fire-and-forget so we never delay the user's
|
|
195
|
+
// response on a slow log POST.
|
|
196
|
+
if (enableLogging) {
|
|
197
|
+
const { input: inputTokens, output: outputTokens } = readTokens(result.usage);
|
|
198
|
+
const { costUsd: cost, pricingSource: costPricingSource } = estimateCostDetailed(model.modelId, inputTokens, outputTokens);
|
|
199
|
+
void guardrail.logTrace({
|
|
200
|
+
model: model.modelId,
|
|
201
|
+
provider: model.provider,
|
|
202
|
+
input: params.prompt,
|
|
203
|
+
// v2: text lives in the `content` array, not a top-level `text`.
|
|
204
|
+
output: extractOutputText(result) || result.toolCalls || null,
|
|
205
|
+
latencyMs,
|
|
206
|
+
tokenUsage: { input: inputTokens, output: outputTokens },
|
|
207
|
+
cost,
|
|
208
|
+
costPricingSource,
|
|
209
|
+
projectId: config.projectId,
|
|
210
|
+
metadata: config.metadata,
|
|
211
|
+
guardrailResult,
|
|
212
|
+
});
|
|
213
|
+
}
|
|
214
|
+
return result;
|
|
215
|
+
},
|
|
216
|
+
async doStream(params) {
|
|
217
|
+
const t0 = performance.now();
|
|
218
|
+
let guardrailResult;
|
|
219
|
+
if (enableGuardrails) {
|
|
220
|
+
const promptText = collapsePrompt(params);
|
|
221
|
+
if (promptText) {
|
|
222
|
+
try {
|
|
223
|
+
guardrailResult = await guardrail.checkInput(promptText, {
|
|
224
|
+
provider: model.provider,
|
|
225
|
+
model: model.modelId,
|
|
226
|
+
...config.metadata,
|
|
227
|
+
});
|
|
228
|
+
if (!guardrailResult.allowed && blockOnViolation) {
|
|
229
|
+
throw new EvalguardBlockedError(`Request blocked by EvalGuard guardrails: ${guardrailResult.violations.map((v) => v.type).join(", ")}`, guardrailResult.violations);
|
|
230
|
+
}
|
|
231
|
+
}
|
|
232
|
+
catch (err) {
|
|
233
|
+
if (err instanceof EvalguardBlockedError)
|
|
234
|
+
throw err;
|
|
235
|
+
// wrapper-core.checkInput fails CLOSED on a guardrail outage
|
|
236
|
+
// (throws once retries/circuit give up). Honor blockOnViolation:
|
|
237
|
+
// an unreachable guardrail BLOCKS the call when it's on, and is
|
|
238
|
+
// swallowed (the call proceeds, monitor-only) when it's off.
|
|
239
|
+
// Mirrors the langchain / llamaindex wrappers.
|
|
240
|
+
if (blockOnViolation) {
|
|
241
|
+
throw new EvalguardBlockedError(`Request blocked: guardrail unavailable (${err instanceof Error ? err.message : "unknown"})`, [{
|
|
242
|
+
type: "guardrail_unavailable",
|
|
243
|
+
severity: "high",
|
|
244
|
+
message: err instanceof Error ? err.message : "guardrail check failed",
|
|
245
|
+
}]);
|
|
246
|
+
}
|
|
247
|
+
}
|
|
248
|
+
}
|
|
249
|
+
}
|
|
250
|
+
const result = await model.doStream(params);
|
|
251
|
+
// Tee the stream — assemble the final text + token totals as parts
|
|
252
|
+
// arrive, then log a trace once `finish` lands. Pass-through preserves
|
|
253
|
+
// streaming semantics for the caller.
|
|
254
|
+
let assembledText = "";
|
|
255
|
+
let promptTokens = 0;
|
|
256
|
+
let completionTokens = 0;
|
|
257
|
+
let traceLogged = false;
|
|
258
|
+
const reader = result.stream.getReader();
|
|
259
|
+
/**
|
|
260
|
+
* 2026-07-29 (audit A284): emit the trace exactly once, whether the
|
|
261
|
+
* stream ran to completion, errored, or was CANCELLED by the client.
|
|
262
|
+
* Previously the trace was logged only on the clean-close path, so
|
|
263
|
+
* every aborted generation vanished from the dashboard — the exact
|
|
264
|
+
* calls a customer most wants to see (cost already incurred upstream).
|
|
265
|
+
*/
|
|
266
|
+
const finalizeTrace = (status) => {
|
|
267
|
+
if (traceLogged || !enableLogging)
|
|
268
|
+
return;
|
|
269
|
+
traceLogged = true;
|
|
270
|
+
const latencyMs = Math.round(performance.now() - t0);
|
|
271
|
+
const { costUsd: cost, pricingSource: costPricingSource } = estimateCostDetailed(model.modelId, promptTokens, completionTokens);
|
|
272
|
+
void guardrail.logTrace({
|
|
273
|
+
model: model.modelId,
|
|
274
|
+
provider: model.provider,
|
|
275
|
+
input: params.prompt,
|
|
276
|
+
output: assembledText,
|
|
277
|
+
latencyMs,
|
|
278
|
+
tokenUsage: { input: promptTokens, output: completionTokens },
|
|
279
|
+
cost,
|
|
280
|
+
costPricingSource,
|
|
281
|
+
projectId: config.projectId,
|
|
282
|
+
metadata: status ? { ...config.metadata, streamStatus: status } : config.metadata,
|
|
283
|
+
guardrailResult,
|
|
284
|
+
});
|
|
285
|
+
};
|
|
286
|
+
const wrappedStream = new ReadableStream({
|
|
287
|
+
// `pull` (not a fire-and-forget loop inside `start`) is what gives
|
|
288
|
+
// this stream backpressure: the runtime calls it only when the
|
|
289
|
+
// consumer has capacity, so a slow client no longer causes us to
|
|
290
|
+
// drain the whole upstream generation into memory.
|
|
291
|
+
async pull(controller) {
|
|
292
|
+
try {
|
|
293
|
+
const { done, value: raw } = await reader.read();
|
|
294
|
+
if (done) {
|
|
295
|
+
controller.close();
|
|
296
|
+
finalizeTrace();
|
|
297
|
+
return;
|
|
298
|
+
}
|
|
299
|
+
// The upstream stream is typed `unknown` (see
|
|
300
|
+
// LanguageModelV2StreamResult for why). This is the ONE read
|
|
301
|
+
// boundary, so narrow here and nowhere else. A chunk that is not
|
|
302
|
+
// an object, or carries an unrecognised `type`, falls through
|
|
303
|
+
// every branch and is enqueued verbatim — the wrapper must never
|
|
304
|
+
// drop or mangle a part it does not understand.
|
|
305
|
+
const value = raw;
|
|
306
|
+
const partType = raw !== null && typeof raw === "object" ? raw.type : undefined;
|
|
307
|
+
if (partType === "text-delta") {
|
|
308
|
+
const part = value;
|
|
309
|
+
// v2 emits `delta`; older v1 models emit `textDelta`.
|
|
310
|
+
assembledText += part.delta ?? part.textDelta ?? "";
|
|
311
|
+
}
|
|
312
|
+
if (partType === "finish") {
|
|
313
|
+
const finishPart = value;
|
|
314
|
+
const tokens = readTokens(finishPart.usage);
|
|
315
|
+
promptTokens = tokens.input;
|
|
316
|
+
completionTokens = tokens.output;
|
|
317
|
+
}
|
|
318
|
+
controller.enqueue(value);
|
|
319
|
+
}
|
|
320
|
+
catch (err) {
|
|
321
|
+
finalizeTrace("error");
|
|
322
|
+
controller.error(err);
|
|
323
|
+
}
|
|
324
|
+
},
|
|
325
|
+
// A284: without this, a consumer abort (user navigates away, the
|
|
326
|
+
// Next.js route's AbortSignal fires) never propagated upstream —
|
|
327
|
+
// the provider connection stayed open and kept billing, and the
|
|
328
|
+
// partial trace was dropped on the floor.
|
|
329
|
+
async cancel(reason) {
|
|
330
|
+
finalizeTrace("cancelled");
|
|
331
|
+
try {
|
|
332
|
+
await reader.cancel(reason);
|
|
333
|
+
}
|
|
334
|
+
catch {
|
|
335
|
+
/* upstream already torn down */
|
|
336
|
+
}
|
|
337
|
+
},
|
|
338
|
+
});
|
|
339
|
+
return { ...result, stream: wrappedStream };
|
|
340
|
+
},
|
|
341
|
+
};
|
|
342
|
+
return new Proxy(model, {
|
|
343
|
+
get(target, prop) {
|
|
344
|
+
if (prop === "doGenerate")
|
|
345
|
+
return overrides.doGenerate;
|
|
346
|
+
if (prop === "doStream")
|
|
347
|
+
return overrides.doStream;
|
|
348
|
+
// Proxy invariant: a non-configurable, non-writable own data property
|
|
349
|
+
// MUST be reported verbatim (returning a bound copy would throw).
|
|
350
|
+
const own = Object.getOwnPropertyDescriptor(target, prop);
|
|
351
|
+
if (own && own.configurable === false && own.writable === false) {
|
|
352
|
+
return own.value;
|
|
353
|
+
}
|
|
354
|
+
// Read with the TARGET as receiver, not the proxy, so a prototype getter
|
|
355
|
+
// that touches private state resolves against the real instance.
|
|
356
|
+
const value = Reflect.get(target, prop, target);
|
|
357
|
+
// Bind provider methods to the real model for the same reason — an
|
|
358
|
+
// unbound prototype method invoked as `wrapped.foo()` would get the
|
|
359
|
+
// Proxy as `this`.
|
|
360
|
+
return typeof value === "function" ? value.bind(target) : value;
|
|
361
|
+
},
|
|
362
|
+
});
|
|
363
|
+
}
|
|
364
|
+
/**
|
|
365
|
+
* Collapse Vercel's varied prompt shapes into a single string for the
|
|
366
|
+
* firewall scan.
|
|
367
|
+
*
|
|
368
|
+
* 2026-07-29 (audit A287). The previous implementation filtered to
|
|
369
|
+
* `role === "user"` on the rationale that "system messages + assistant
|
|
370
|
+
* replies are mostly app-controlled". That reasoning omits the two message
|
|
371
|
+
* kinds an attacker actually controls in an agent app:
|
|
372
|
+
* - `role: "tool"` messages carrying tool *results* (a poisoned or
|
|
373
|
+
* compromised tool returns injection text), and
|
|
374
|
+
* - retrieved RAG documents, which the AI SDK conventionally splices in
|
|
375
|
+
* as system/assistant context.
|
|
376
|
+
* Both reached the model without ever being scanned. It also silently
|
|
377
|
+
* `.slice(0, 8000)`'d the JSON fallback — a scanner that trims its own
|
|
378
|
+
* input reports clean on bytes it never examined.
|
|
379
|
+
*
|
|
380
|
+
* Now delegates to wrapper-core's `collapsePromptText`: every role, no
|
|
381
|
+
* truncation, and shared with the langchain / llamaindex wrappers so the
|
|
382
|
+
* three copies cannot drift apart again.
|
|
383
|
+
*/
|
|
384
|
+
function collapsePrompt(params) {
|
|
385
|
+
return collapsePromptText(params.prompt);
|
|
386
|
+
}
|
|
387
|
+
// ── Cost estimation (public surface) ────────────────────────────────────────
|
|
388
|
+
// The README's "Cost estimates" section tells the customer to
|
|
389
|
+
// `import { estimateCostDetailed, isModelPriced } from "@evalguard/vercel-ai"`.
|
|
390
|
+
// Until 2026-08-01 these lived only in `./cost.ts`, which is NOT an `exports`
|
|
391
|
+
// subpath, so that documented import resolved to nothing from the published
|
|
392
|
+
// tarball. The three SDK-shaped wrappers (openai / anthropic / gemini) already
|
|
393
|
+
// re-exported them; the three framework-shaped ones did not.
|
|
394
|
+
export { estimateCost, estimateCostDetailed, isModelPriced } from "./cost.js";
|
|
395
|
+
//# sourceMappingURL=index.js.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"index.js","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;GAsEG;AAEH,OAAO,EAAE,eAAe,EAAE,kBAAkB,EAA6B,MAAM,yBAAyB,CAAC;AACzG,OAAO,EAAE,oBAAoB,EAAE,MAAM,WAAW,CAAC;AA2MjD;;;;GAIG;AACH,MAAM,OAAO,qBAAsB,SAAQ,KAAK;IACrC,UAAU,CAAqC;IACxD,YAAY,OAAe,EAAE,UAA8C;QACzE,KAAK,CAAC,OAAO,CAAC,CAAC;QACf,IAAI,CAAC,IAAI,GAAG,uBAAuB,CAAC;QACpC,IAAI,CAAC,UAAU,GAAG,UAAU,CAAC;IAC/B,CAAC;CACF;AAED;;;;GAIG;AACH,SAAS,cAAc,CAAC,KAA8B;IACpD,IAAI,OAAO,KAAK,KAAK,QAAQ;QAAE,OAAO,MAAM,CAAC,QAAQ,CAAC,KAAK,CAAC,CAAC,CAAC,CAAC,KAAK,CAAC,CAAC,CAAC,SAAS,CAAC;IACjF,IAAI,KAAK,IAAI,OAAO,KAAK,KAAK,QAAQ,IAAI,OAAO,KAAK,CAAC,KAAK,KAAK,QAAQ,EAAE,CAAC;QAC1E,OAAO,MAAM,CAAC,QAAQ,CAAC,KAAK,CAAC,KAAK,CAAC,CAAC,CAAC,CAAC,KAAK,CAAC,KAAK,CAAC,CAAC,CAAC,SAAS,CAAC;IAChE,CAAC;IACD,OAAO,SAAS,CAAC;AACnB,CAAC;AAED,oEAAoE;AACpE,SAAS,UAAU,CAAC,KAAuC;IACzD,OAAO;QACL,KAAK,EAAE,cAAc,CAAC,KAAK,EAAE,WAAW,CAAC,IAAI,KAAK,EAAE,YAAY,IAAI,CAAC;QACrE,MAAM,EAAE,cAAc,CAAC,KAAK,EAAE,YAAY,CAAC,IAAI,KAAK,EAAE,gBAAgB,IAAI,CAAC;KAC5E,CAAC;AACJ,CAAC;AAED;;;;;GAKG;AACH,SAAS,iBAAiB,CAAC,MAAqC;IAC9D,IAAI,KAAK,CAAC,OAAO,CAAC,MAAM,CAAC,OAAO,CAAC,EAAE,CAAC;QAClC,OAAO,MAAM,CAAC,OAAO;aAClB,MAAM,CACL,CAAC,CAAC,EAAuC,EAAE,CACzC,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC,IAAI,KAAK,MAAM,IAAI,OAAQ,CAAoC,CAAC,IAAI,KAAK,QAAQ,CAC7F;aACA,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,IAAI,CAAC;aAClB,IAAI,CAAC,EAAE,CAAC,CAAC;IACd,CAAC;IACD,OAAO,OAAO,MAAM,CAAC,IAAI,KAAK,QAAQ,CAAC,CAAC,CAAC,MAAM,CAAC,IAAI,CAAC,CAAC,CAAC,EAAE,CAAC;AAC5D,CAAC;AAED;;;;;;;;;;;;;;;;;;;;;;;;;;GA0BG;AACH,MAAM,UAAU,aAAa,CAC3B,KAAa,EACb,MAA2B;IAE3B,MAAM,SAAS,GAAG,IAAI,eAAe,CAAC,EAAE,MAAM,EAAE,MAAM,CAAC,MAAM,EAAE,OAAO,EAAE,MAAM,CAAC,OAAO,EAAE,CAAC,CAAC;IAC1F,MAAM,gBAAgB,GAAG,MAAM,CAAC,gBAAgB,IAAI,IAAI,CAAC;IACzD,MAAM,aAAa,GAAG,CAAC,MAAM,CAAC,cAAc,CAAC;IAC7C,MAAM,gBAAgB,GAAG,CAAC,MAAM,CAAC,iBAAiB,CAAC;IAEnD,MAAM,SAAS,GAAG;QAChB,KAAK,CAAC,UAAU,CAAC,MAAkC;YACjD,MAAM,EAAE,GAAG,WAAW,CAAC,GAAG,EAAE,CAAC;YAC7B,IAAI,eAAiD,CAAC;YAEtD,mEAAmE;YACnE,qEAAqE;YACrE,oDAAoD;YACpD,IAAI,gBAAgB,EAAE,CAAC;gBACrB,MAAM,UAAU,GAAG,cAAc,CAAC,MAAM,CAAC,CAAC;gBAC1C,IAAI,UAAU,EAAE,CAAC;oBACf,IAAI,CAAC;wBACH,eAAe,GAAG,MAAM,SAAS,CAAC,UAAU,CAAC,UAAU,EAAE;4BACvD,QAAQ,EAAE,KAAK,CAAC,QAAQ;4BACxB,KAAK,EAAE,KAAK,CAAC,OAAO;4BACpB,GAAG,MAAM,CAAC,QAAQ;yBACnB,CAAC,CAAC;wBACH,IAAI,CAAC,eAAe,CAAC,OAAO,IAAI,gBAAgB,EAAE,CAAC;4BACjD,MAAM,IAAI,qBAAqB,CAC7B,4CAA4C,eAAe,CAAC,UAAU,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC,IAAI,CAAC,IAAI,CAAC,EAAE,EACtG,eAAe,CAAC,UAAU,CAC3B,CAAC;wBACJ,CAAC;oBACH,CAAC;oBAAC,OAAO,GAAG,EAAE,CAAC;wBACb,IAAI,GAAG,YAAY,qBAAqB;4BAAE,MAAM,GAAG,CAAC;wBACpD,6DAA6D;wBAC7D,iEAAiE;wBACjE,gEAAgE;wBAChE,6DAA6D;wBAC7D,+CAA+C;wBAC/C,IAAI,gBAAgB,EAAE,CAAC;4BACrB,MAAM,IAAI,qBAAqB,CAC7B,2CAA2C,GAAG,YAAY,KAAK,CAAC,CAAC,CAAC,GAAG,CAAC,OAAO,CAAC,CAAC,CAAC,SAAS,GAAG,EAC5F,CAAC;oCACC,IAAI,EAAE,uBAAuB;oCAC7B,QAAQ,EAAE,MAAM;oCAChB,OAAO,EAAE,GAAG,YAAY,KAAK,CAAC,CAAC,CAAC,GAAG,CAAC,OAAO,CAAC,CAAC,CAAC,wBAAwB;iCACvE,CAAC,CACH,CAAC;wBACJ,CAAC;oBACH,CAAC;gBACH,CAAC;YACH,CAAC;YAED,MAAM,MAAM,GAAG,MAAM,KAAK,CAAC,UAAU,CAAC,MAAM,CAAC,CAAC;YAC9C,MAAM,SAAS,GAAG,IAAI,CAAC,KAAK,CAAC,WAAW,CAAC,GAAG,EAAE,GAAG,EAAE,CAAC,CAAC;YAErD,oEAAoE;YACpE,+BAA+B;YAC/B,IAAI,aAAa,EAAE,CAAC;gBAClB,MAAM,EAAE,KAAK,EAAE,WAAW,EAAE,MAAM,EAAE,YAAY,EAAE,GAAG,UAAU,CAAC,MAAM,CAAC,KAAK,CAAC,CAAC;gBAC9E,MAAM,EAAE,OAAO,EAAE,IAAI,EAAE,aAAa,EAAE,iBAAiB,EAAE,GAAG,oBAAoB,CAC9E,KAAK,CAAC,OAAO,EACb,WAAW,EACX,YAAY,CACb,CAAC;gBACF,KAAK,SAAS,CAAC,QAAQ,CAAC;oBACtB,KAAK,EAAE,KAAK,CAAC,OAAO;oBACpB,QAAQ,EAAE,KAAK,CAAC,QAAQ;oBACxB,KAAK,EAAE,MAAM,CAAC,MAAM;oBACpB,iEAAiE;oBACjE,MAAM,EAAE,iBAAiB,CAAC,MAAM,CAAC,IAAI,MAAM,CAAC,SAAS,IAAI,IAAI;oBAC7D,SAAS;oBACT,UAAU,EAAE,EAAE,KAAK,EAAE,WAAW,EAAE,MAAM,EAAE,YAAY,EAAE;oBACxD,IAAI;oBACJ,iBAAiB;oBACjB,SAAS,EAAE,MAAM,CAAC,SAAS;oBAC3B,QAAQ,EAAE,MAAM,CAAC,QAAQ;oBACzB,eAAe;iBAChB,CAAC,CAAC;YACL,CAAC;YAED,OAAO,MAAM,CAAC;QAChB,CAAC;QAED,KAAK,CAAC,QAAQ,CAAC,MAAkC;YAC/C,MAAM,EAAE,GAAG,WAAW,CAAC,GAAG,EAAE,CAAC;YAC7B,IAAI,eAAiD,CAAC;YAEtD,IAAI,gBAAgB,EAAE,CAAC;gBACrB,MAAM,UAAU,GAAG,cAAc,CAAC,MAAM,CAAC,CAAC;gBAC1C,IAAI,UAAU,EAAE,CAAC;oBACf,IAAI,CAAC;wBACH,eAAe,GAAG,MAAM,SAAS,CAAC,UAAU,CAAC,UAAU,EAAE;4BACvD,QAAQ,EAAE,KAAK,CAAC,QAAQ;4BACxB,KAAK,EAAE,KAAK,CAAC,OAAO;4BACpB,GAAG,MAAM,CAAC,QAAQ;yBACnB,CAAC,CAAC;wBACH,IAAI,CAAC,eAAe,CAAC,OAAO,IAAI,gBAAgB,EAAE,CAAC;4BACjD,MAAM,IAAI,qBAAqB,CAC7B,4CAA4C,eAAe,CAAC,UAAU,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC,IAAI,CAAC,IAAI,CAAC,EAAE,EACtG,eAAe,CAAC,UAAU,CAC3B,CAAC;wBACJ,CAAC;oBACH,CAAC;oBAAC,OAAO,GAAG,EAAE,CAAC;wBACb,IAAI,GAAG,YAAY,qBAAqB;4BAAE,MAAM,GAAG,CAAC;wBACpD,6DAA6D;wBAC7D,iEAAiE;wBACjE,gEAAgE;wBAChE,6DAA6D;wBAC7D,+CAA+C;wBAC/C,IAAI,gBAAgB,EAAE,CAAC;4BACrB,MAAM,IAAI,qBAAqB,CAC7B,2CAA2C,GAAG,YAAY,KAAK,CAAC,CAAC,CAAC,GAAG,CAAC,OAAO,CAAC,CAAC,CAAC,SAAS,GAAG,EAC5F,CAAC;oCACC,IAAI,EAAE,uBAAuB;oCAC7B,QAAQ,EAAE,MAAM;oCAChB,OAAO,EAAE,GAAG,YAAY,KAAK,CAAC,CAAC,CAAC,GAAG,CAAC,OAAO,CAAC,CAAC,CAAC,wBAAwB;iCACvE,CAAC,CACH,CAAC;wBACJ,CAAC;oBACH,CAAC;gBACH,CAAC;YACH,CAAC;YAED,MAAM,MAAM,GAAG,MAAM,KAAK,CAAC,QAAQ,CAAC,MAAM,CAAC,CAAC;YAE5C,mEAAmE;YACnE,uEAAuE;YACvE,sCAAsC;YACtC,IAAI,aAAa,GAAG,EAAE,CAAC;YACvB,IAAI,YAAY,GAAG,CAAC,CAAC;YACrB,IAAI,gBAAgB,GAAG,CAAC,CAAC;YACzB,IAAI,WAAW,GAAG,KAAK,CAAC;YAExB,MAAM,MAAM,GAAG,MAAM,CAAC,MAAM,CAAC,SAAS,EAAE,CAAC;YAEzC;;;;;;eAMG;YACH,MAAM,aAAa,GAAG,CAAC,MAAe,EAAE,EAAE;gBACxC,IAAI,WAAW,IAAI,CAAC,aAAa;oBAAE,OAAO;gBAC1C,WAAW,GAAG,IAAI,CAAC;gBACnB,MAAM,SAAS,GAAG,IAAI,CAAC,KAAK,CAAC,WAAW,CAAC,GAAG,EAAE,GAAG,EAAE,CAAC,CAAC;gBACrD,MAAM,EAAE,OAAO,EAAE,IAAI,EAAE,aAAa,EAAE,iBAAiB,EAAE,GAAG,oBAAoB,CAC9E,KAAK,CAAC,OAAO,EACb,YAAY,EACZ,gBAAgB,CACjB,CAAC;gBACF,KAAK,SAAS,CAAC,QAAQ,CAAC;oBACtB,KAAK,EAAE,KAAK,CAAC,OAAO;oBACpB,QAAQ,EAAE,KAAK,CAAC,QAAQ;oBACxB,KAAK,EAAE,MAAM,CAAC,MAAM;oBACpB,MAAM,EAAE,aAAa;oBACrB,SAAS;oBACT,UAAU,EAAE,EAAE,KAAK,EAAE,YAAY,EAAE,MAAM,EAAE,gBAAgB,EAAE;oBAC7D,IAAI;oBACJ,iBAAiB;oBACjB,SAAS,EAAE,MAAM,CAAC,SAAS;oBAC3B,QAAQ,EAAE,MAAM,CAAC,CAAC,CAAC,EAAE,GAAG,MAAM,CAAC,QAAQ,EAAE,YAAY,EAAE,MAAM,EAAE,CAAC,CAAC,CAAC,MAAM,CAAC,QAAQ;oBACjF,eAAe;iBAChB,CAAC,CAAC;YACL,CAAC,CAAC;YAEF,MAAM,aAAa,GAAG,IAAI,cAAc,CAA4B;gBAClE,mEAAmE;gBACnE,+DAA+D;gBAC/D,iEAAiE;gBACjE,mDAAmD;gBACnD,KAAK,CAAC,IAAI,CAAC,UAAU;oBACnB,IAAI,CAAC;wBACH,MAAM,EAAE,IAAI,EAAE,KAAK,EAAE,GAAG,EAAE,GAAG,MAAM,MAAM,CAAC,IAAI,EAAE,CAAC;wBACjD,IAAI,IAAI,EAAE,CAAC;4BACT,UAAU,CAAC,KAAK,EAAE,CAAC;4BACnB,aAAa,EAAE,CAAC;4BAChB,OAAO;wBACT,CAAC;wBACD,8CAA8C;wBAC9C,6DAA6D;wBAC7D,iEAAiE;wBACjE,8DAA8D;wBAC9D,iEAAiE;wBACjE,gDAAgD;wBAChD,MAAM,KAAK,GAAG,GAAgC,CAAC;wBAC/C,MAAM,QAAQ,GACZ,GAAG,KAAK,IAAI,IAAI,OAAO,GAAG,KAAK,QAAQ,CAAC,CAAC,CAAE,GAA0B,CAAC,IAAI,CAAC,CAAC,CAAC,SAAS,CAAC;wBACzF,IAAI,QAAQ,KAAK,YAAY,EAAE,CAAC;4BAC9B,MAAM,IAAI,GAAG,KAA2C,CAAC;4BACzD,sDAAsD;4BACtD,aAAa,IAAI,IAAI,CAAC,KAAK,IAAI,IAAI,CAAC,SAAS,IAAI,EAAE,CAAC;wBACtD,CAAC;wBACD,IAAI,QAAQ,KAAK,QAAQ,EAAE,CAAC;4BAC1B,MAAM,UAAU,GAAG,KAAwC,CAAC;4BAC5D,MAAM,MAAM,GAAG,UAAU,CAAC,UAAU,CAAC,KAAK,CAAC,CAAC;4BAC5C,YAAY,GAAG,MAAM,CAAC,KAAK,CAAC;4BAC5B,gBAAgB,GAAG,MAAM,CAAC,MAAM,CAAC;wBACnC,CAAC;wBACD,UAAU,CAAC,OAAO,CAAC,KAAK,CAAC,CAAC;oBAC5B,CAAC;oBAAC,OAAO,GAAG,EAAE,CAAC;wBACb,aAAa,CAAC,OAAO,CAAC,CAAC;wBACvB,UAAU,CAAC,KAAK,CAAC,GAAG,CAAC,CAAC;oBACxB,CAAC;gBACH,CAAC;gBACD,iEAAiE;gBACjE,iEAAiE;gBACjE,gEAAgE;gBAChE,0CAA0C;gBAC1C,KAAK,CAAC,MAAM,CAAC,MAAM;oBACjB,aAAa,CAAC,WAAW,CAAC,CAAC;oBAC3B,IAAI,CAAC;wBACH,MAAM,MAAM,CAAC,MAAM,CAAC,MAAM,CAAC,CAAC;oBAC9B,CAAC;oBAAC,MAAM,CAAC;wBACP,gCAAgC;oBAClC,CAAC;gBACH,CAAC;aACF,CAAC,CAAC;YAEH,OAAO,EAAE,GAAG,MAAM,EAAE,MAAM,EAAE,aAAa,EAAE,CAAC;QAC9C,CAAC;KACF,CAAC;IAEF,OAAO,IAAI,KAAK,CAAC,KAAK,EAAE;QACtB,GAAG,CAAC,MAAM,EAAE,IAAI;YACd,IAAI,IAAI,KAAK,YAAY;gBAAE,OAAO,SAAS,CAAC,UAAU,CAAC;YACvD,IAAI,IAAI,KAAK,UAAU;gBAAE,OAAO,SAAS,CAAC,QAAQ,CAAC;YAEnD,sEAAsE;YACtE,kEAAkE;YAClE,MAAM,GAAG,GAAG,MAAM,CAAC,wBAAwB,CAAC,MAAM,EAAE,IAAI,CAAC,CAAC;YAC1D,IAAI,GAAG,IAAI,GAAG,CAAC,YAAY,KAAK,KAAK,IAAI,GAAG,CAAC,QAAQ,KAAK,KAAK,EAAE,CAAC;gBAChE,OAAO,GAAG,CAAC,KAAK,CAAC;YACnB,CAAC;YAED,yEAAyE;YACzE,iEAAiE;YACjE,MAAM,KAAK,GAAG,OAAO,CAAC,GAAG,CAAC,MAAM,EAAE,IAAI,EAAE,MAAM,CAAC,CAAC;YAChD,mEAAmE;YACnE,oEAAoE;YACpE,mBAAmB;YACnB,OAAO,OAAO,KAAK,KAAK,UAAU,CAAC,CAAC,CAAE,KAAsC,CAAC,IAAI,CAAC,MAAM,CAAC,CAAC,CAAC,CAAC,KAAK,CAAC;QACpG,CAAC;KACF,CAAW,CAAC;AACf,CAAC;AAED;;;;;;;;;;;;;;;;;;;GAmBG;AACH,SAAS,cAAc,CAAC,MAAkC;IACxD,OAAO,kBAAkB,CAAC,MAAM,CAAC,MAAiB,CAAC,CAAC;AACtD,CAAC;AAID,+EAA+E;AAC/E,8DAA8D;AAC9D,gFAAgF;AAChF,8EAA8E;AAC9E,4EAA4E;AAC5E,+EAA+E;AAC/E,6DAA6D;AAC7D,OAAO,EAAE,YAAY,EAAE,oBAAoB,EAAE,aAAa,EAAE,MAAM,WAAW,CAAC"}
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"version.d.ts","sourceRoot":"","sources":["../src/version.ts"],"names":[],"mappings":"AAGA,eAAO,MAAM,OAAO,UAAU,CAAC"}
|
package/dist/version.js
ADDED
|
@@ -0,0 +1,5 @@
|
|
|
1
|
+
// Single source of the wrapper version WITHIN src (so the User-Agent header
|
|
2
|
+
// can't drift between call sites). Kept in lockstep with package.json#version;
|
|
3
|
+
// a unit test (version.test.ts) fails CI if the two ever diverge. (#120)
|
|
4
|
+
export const VERSION = "1.0.0";
|
|
5
|
+
//# sourceMappingURL=version.js.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"version.js","sourceRoot":"","sources":["../src/version.ts"],"names":[],"mappings":"AAAA,4EAA4E;AAC5E,+EAA+E;AAC/E,yEAAyE;AACzE,MAAM,CAAC,MAAM,OAAO,GAAG,OAAO,CAAC"}
|
package/package.json
ADDED
|
@@ -0,0 +1,70 @@
|
|
|
1
|
+
{
|
|
2
|
+
"name": "@evalguard/vercel-ai",
|
|
3
|
+
"version": "1.0.0",
|
|
4
|
+
"description": "Drop-in Vercel AI SDK middleware with EvalGuard guardrails, trace logging & cost tracking",
|
|
5
|
+
"main": "./dist/index.js",
|
|
6
|
+
"exports": {
|
|
7
|
+
".": {
|
|
8
|
+
"types": "./dist/index.d.ts",
|
|
9
|
+
"default": "./dist/index.js"
|
|
10
|
+
},
|
|
11
|
+
"./package.json": "./package.json"
|
|
12
|
+
},
|
|
13
|
+
"files": [
|
|
14
|
+
"dist",
|
|
15
|
+
"README.md",
|
|
16
|
+
"LICENSE",
|
|
17
|
+
"peer-matrix.json"
|
|
18
|
+
],
|
|
19
|
+
"scripts": {
|
|
20
|
+
"build": "tsc",
|
|
21
|
+
"dev": "tsc --watch",
|
|
22
|
+
"type-check": "tsc --noEmit && tsc --noEmit -p tsconfig.test.json",
|
|
23
|
+
"test": "vitest run",
|
|
24
|
+
"lint": "eslint src --quiet",
|
|
25
|
+
"clean": "rm -rf dist .turbo",
|
|
26
|
+
"prepublishOnly": "pnpm run build"
|
|
27
|
+
},
|
|
28
|
+
"keywords": [
|
|
29
|
+
"vercel",
|
|
30
|
+
"ai-sdk",
|
|
31
|
+
"llm",
|
|
32
|
+
"guardrails",
|
|
33
|
+
"evalguard",
|
|
34
|
+
"evalguardai",
|
|
35
|
+
"ai",
|
|
36
|
+
"tracing",
|
|
37
|
+
"observability",
|
|
38
|
+
"middleware",
|
|
39
|
+
"language-model"
|
|
40
|
+
],
|
|
41
|
+
"author": "EvalGuard <support@evalguard.ai>",
|
|
42
|
+
"license": "Apache-2.0",
|
|
43
|
+
"homepage": "https://evalguard.ai",
|
|
44
|
+
"repository": {
|
|
45
|
+
"type": "git",
|
|
46
|
+
"url": "https://github.com/EvalGuardAi/evalguard"
|
|
47
|
+
},
|
|
48
|
+
"peerDependencies": {
|
|
49
|
+
"ai": ">=5.0.0"
|
|
50
|
+
},
|
|
51
|
+
"dependencies": {
|
|
52
|
+
"@evalguard/wrapper-core": "workspace:^"
|
|
53
|
+
},
|
|
54
|
+
"devDependencies": {
|
|
55
|
+
"@evalguard/config": "workspace:*",
|
|
56
|
+
"@types/node": "^26.0.0",
|
|
57
|
+
"@typescript-eslint/eslint-plugin": "^8.61.1",
|
|
58
|
+
"@typescript-eslint/parser": "^8.61.1",
|
|
59
|
+
"ai": "^6.0.208",
|
|
60
|
+
"eslint": "^10.5.0",
|
|
61
|
+
"typescript": "^6.0.3",
|
|
62
|
+
"vitest": "^4.1.9"
|
|
63
|
+
},
|
|
64
|
+
"type": "module",
|
|
65
|
+
"types": "./dist/index.d.ts",
|
|
66
|
+
"publishConfig": {
|
|
67
|
+
"access": "public",
|
|
68
|
+
"registry": "https://registry.npmjs.org/"
|
|
69
|
+
}
|
|
70
|
+
}
|
package/peer-matrix.json
ADDED
|
@@ -0,0 +1,50 @@
|
|
|
1
|
+
{
|
|
2
|
+
"$comment": [
|
|
3
|
+
"The AI SDK majors @evalguard/vercel-ai is verified against, and the SINGLE",
|
|
4
|
+
"source of truth for that claim. Three things read this file and they must",
|
|
5
|
+
"agree, which is the whole point:",
|
|
6
|
+
"",
|
|
7
|
+
" 1. scripts/verify-vercel-ai-peer-matrix.mjs — installs each entry into a",
|
|
8
|
+
" clean consumer, TYPE-CHECKS the README quickstart verbatim against it,",
|
|
9
|
+
" and EXECUTES the wrapper end-to-end through that major's own",
|
|
10
|
+
" generateText/streamText.",
|
|
11
|
+
" 2. .github/workflows/publish-wrappers.yml — runs that script before",
|
|
12
|
+
" @evalguard/vercel-ai is published, so 'on every release' is literal.",
|
|
13
|
+
" 3. scripts/publish-readiness-check.mjs — fails if README.md claims a peer",
|
|
14
|
+
" version this file does not list, if this file lists one the README does",
|
|
15
|
+
" not claim, or if no workflow actually runs the verifier.",
|
|
16
|
+
"",
|
|
17
|
+
"WHY: on 2026-08-02 the README said the quickstart was 'type-checked and",
|
|
18
|
+
"executed against ai@5, ai@6 and ai@7 on every release'. Nothing in the repo",
|
|
19
|
+
"did that — package.json had exactly one AI SDK devDependency (ai@^6) and the",
|
|
20
|
+
"publish workflow's only runtime step was `await import('@evalguard/vercel-ai')`.",
|
|
21
|
+
"A README claim with no executable behind it is the defect this file closes.",
|
|
22
|
+
"",
|
|
23
|
+
"ADDING A MAJOR: add the entry here, run `pnpm gate:vercel-ai-peers`, and only",
|
|
24
|
+
"then widen the README sentence. REMOVING one: narrow the README in the same",
|
|
25
|
+
"commit — the readiness gate fails on either half alone."
|
|
26
|
+
],
|
|
27
|
+
"peer": "ai",
|
|
28
|
+
"entries": [
|
|
29
|
+
{
|
|
30
|
+
"aiMajor": "5",
|
|
31
|
+
"specificationVersion": "v2",
|
|
32
|
+
"provider": "@ai-sdk/openai@2",
|
|
33
|
+
"mockExport": "MockLanguageModelV2",
|
|
34
|
+
"harnessDeps": ["msw", "vitest"],
|
|
35
|
+
"harnessDepsWhy": "ai@5's `ai/test` re-exports @ai-sdk/provider-utils/test, whose module graph imports `msw` and `vitest` at load time (ERR_MODULE_NOT_FOUND without them). ai@6/ai@7 dropped those imports. These are dependencies of the TEST HARNESS only — never installed by, or required for, @evalguard/vercel-ai itself."
|
|
36
|
+
},
|
|
37
|
+
{
|
|
38
|
+
"aiMajor": "6",
|
|
39
|
+
"specificationVersion": "v3",
|
|
40
|
+
"provider": "@ai-sdk/openai@3",
|
|
41
|
+
"mockExport": "MockLanguageModelV3"
|
|
42
|
+
},
|
|
43
|
+
{
|
|
44
|
+
"aiMajor": "7",
|
|
45
|
+
"specificationVersion": "v4",
|
|
46
|
+
"provider": "@ai-sdk/openai@4",
|
|
47
|
+
"mockExport": "MockLanguageModelV4"
|
|
48
|
+
}
|
|
49
|
+
]
|
|
50
|
+
}
|