modelpact-providers 1.1.0 → 1.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +161 -24
- package/dist/index.d.ts +4 -2
- package/dist/index.d.ts.map +1 -1
- package/dist/index.js +4 -2
- package/dist/index.js.map +1 -1
- package/dist/openai.d.ts +60 -0
- package/dist/openai.d.ts.map +1 -0
- package/dist/openai.js +488 -0
- package/dist/openai.js.map +1 -0
- package/package.json +13 -4
package/README.md
CHANGED
|
@@ -6,14 +6,15 @@
|
|
|
6
6
|
[](LICENSE)
|
|
7
7
|
|
|
8
8
|
**The transports for [modelpact](https://github.com/AvdienkoSergey/modelpact):
|
|
9
|
-
a daemon on your machine,
|
|
10
|
-
itself.
|
|
9
|
+
a daemon on your machine, any OpenAI-compatible server, the model inside
|
|
10
|
+
Chrome, and a model in the tab itself. Four backends, one contract, and that
|
|
11
|
+
contract's suite green on each.**
|
|
11
12
|
|
|
12
13
|
The engine holds the contract, the lifecycle and one backend with nothing
|
|
13
14
|
behind it. This package holds the ones with something behind them. They are
|
|
14
|
-
apart because they change for different reasons: a daemon's JSON, a
|
|
15
|
-
origin trial and a WebGPU runtime each move on their
|
|
16
|
-
them should move the contract.
|
|
15
|
+
apart because they change for different reasons: a daemon's JSON, a hosted
|
|
16
|
+
API's dialect, a browser's origin trial and a WebGPU runtime each move on their
|
|
17
|
+
own clock, and none of them should move the contract.
|
|
17
18
|
|
|
18
19
|
## Install
|
|
19
20
|
|
|
@@ -24,13 +25,14 @@ npm install modelpact-providers modelpact
|
|
|
24
25
|
`modelpact` is a peer dependency: this package is written against its contract
|
|
25
26
|
and carries no copy of it.
|
|
26
27
|
|
|
27
|
-
## The
|
|
28
|
+
## The four
|
|
28
29
|
|
|
29
|
-
| Provider | Reaches
|
|
30
|
-
| ----------------------- |
|
|
31
|
-
| `makeOllamaProvider` | a daemon over HTTP, usually local
|
|
32
|
-
| `
|
|
33
|
-
| `
|
|
30
|
+
| Provider | Reaches | Wants | Import from |
|
|
31
|
+
| ----------------------- | ----------------------------------------- | -------------------------------------------------- | ---------------------------- |
|
|
32
|
+
| `makeOllamaProvider` | a daemon over HTTP, usually local | Ollama on `127.0.0.1:11434` | `modelpact-providers` |
|
|
33
|
+
| `makeOpenAiProvider` | anything speaking the OpenAI HTTP dialect | a `baseUrl`; a key only where the server wants one | `modelpact-providers` |
|
|
34
|
+
| `makePromptApiProvider` | Chrome's built-in Gemini Nano | Chrome, and the weights downloaded once | `modelpact-providers` |
|
|
35
|
+
| `makeWebGpuProvider` | a model in the tab, on WebGPU | `@mlc-ai/web-llm`, and a GPU | `modelpact-providers/webgpu` |
|
|
34
36
|
|
|
35
37
|
```ts
|
|
36
38
|
import { makeOllamaProvider } from "modelpact-providers";
|
|
@@ -41,14 +43,15 @@ const opened = await access.open({ system: "Answer in one sentence." });
|
|
|
41
43
|
```
|
|
42
44
|
|
|
43
45
|
Everything after the provider line is the contract's, and identical across the
|
|
44
|
-
|
|
46
|
+
four — see [modelpact's README](https://github.com/AvdienkoSergey/modelpact#readme)
|
|
45
47
|
for what a session promises.
|
|
46
48
|
|
|
47
49
|
## Two entries, and the reason
|
|
48
50
|
|
|
49
|
-
`modelpact-providers` is the
|
|
50
|
-
for the daemon, a global for the browser's own
|
|
51
|
-
|
|
51
|
+
`modelpact-providers` is the three that cost a consumer nothing: `fetch` and
|
|
52
|
+
JSON for the daemon and for the OpenAI dialect, a global for the browser's own
|
|
53
|
+
model, no runtime dependency behind any of them.
|
|
54
|
+
`modelpact-providers/webgpu` is the fourth, because it carries
|
|
52
55
|
`@mlc-ai/web-llm` — an optional peer dependency, so an app on the daemon never
|
|
53
56
|
installs it. The split is by dependency, not by kind: the same reason
|
|
54
57
|
`modelpact/testing` is a separate entry for `vitest`.
|
|
@@ -62,6 +65,27 @@ carries the counts, a pull line carries `completed` and `total` per layer, and
|
|
|
62
65
|
an error is an HTTP status with a body. The daemon keeps nothing between
|
|
63
66
|
requests, so the session's record is resent whole every turn.
|
|
64
67
|
|
|
68
|
+
**An OpenAI-compatible server.** A dialect rather than a company:
|
|
69
|
+
`https://api.openai.com/v1` is the default and one instance of it, and the same
|
|
70
|
+
two endpoints answer on vLLM, llama.cpp, LM Studio, OpenRouter, Groq and
|
|
71
|
+
Ollama's own `/v1`. So `baseUrl` is the whole difference between them and
|
|
72
|
+
`apiKey` is optional — a model on your machine wants no key, and a type that
|
|
73
|
+
demanded one would be describing a service instead of a protocol. It is also
|
|
74
|
+
the only backend here that never answers `needs-download`: the weights are the
|
|
75
|
+
server's problem, and there is nothing this side could fetch.
|
|
76
|
+
|
|
77
|
+
Two things the wire does that the daemon next door does not, and both are
|
|
78
|
+
handled rather than assumed: a tool call arrives fragmented across frames,
|
|
79
|
+
keyed by `index`, with its arguments as a JSON string rather than an object;
|
|
80
|
+
and the counts arrive in a frame of their own after the last one carrying text,
|
|
81
|
+
which is what `stream_options: { include_usage: true }` asks for.
|
|
82
|
+
|
|
83
|
+
`contextWindow` is optional, and absent it `usage()` answers `unknown`. No
|
|
84
|
+
server in this dialect reports the window it loaded a model with, so a number
|
|
85
|
+
there is your declaration, not a discovery — and being a declaration is what
|
|
86
|
+
makes it a budget too: a transcript past it is an overflow, told by the counts
|
|
87
|
+
rather than by the server.
|
|
88
|
+
|
|
65
89
|
**Chrome's built-in model.** The one backend that keeps the conversation
|
|
66
90
|
itself: `LanguageModel` is a session object and `prompt()` appends to it, so
|
|
67
91
|
the record travels with the session rather than in the request. It fires its
|
|
@@ -80,14 +104,15 @@ was, and it needed nothing added to the contract.
|
|
|
80
104
|
|
|
81
105
|
## Tools
|
|
82
106
|
|
|
83
|
-
All
|
|
107
|
+
All four accept `ModelRequest.tools`, and each executes them the way its
|
|
84
108
|
transport can.
|
|
85
109
|
|
|
86
|
-
| Provider | How a call happens
|
|
87
|
-
| ---------- |
|
|
88
|
-
| Ollama | native `tool_calls`, answered under the `tool` role, in rounds bounded by `maxToolRounds`
|
|
89
|
-
|
|
|
90
|
-
|
|
|
110
|
+
| Provider | How a call happens |
|
|
111
|
+
| ---------- | --------------------------------------------------------------------------------------------------- |
|
|
112
|
+
| Ollama | native `tool_calls`, answered under the `tool` role, in rounds bounded by `maxToolRounds` |
|
|
113
|
+
| OpenAI | the same, tied by `tool_call_id` rather than by name, so two calls to one tool in a turn stay apart |
|
|
114
|
+
| Prompt API | handed to `create()`, and the browser calls `execute` itself |
|
|
115
|
+
| WebGPU | not yet: the request is refused at `access`, which is the contract's answer for a backend without |
|
|
91
116
|
|
|
92
117
|
Chrome 152 answers `available` to `availability()` with tools and then throws
|
|
93
118
|
`InvalidStateError` from `create()` — measured, and it arrives as a refusal at
|
|
@@ -124,19 +149,131 @@ Three tsconfigs, and each has one job:
|
|
|
124
149
|
| [`tsconfig.patched.json`](tsconfig.patched.json) | everything but WebGPU, with `skipLibCheck` off — the patch, still applying |
|
|
125
150
|
| [`tsconfig.surface.json`](tsconfig.surface.json) | the emitted declarations, as a consumer receives them |
|
|
126
151
|
|
|
152
|
+
[`demo/tsconfig.json`](demo/tsconfig.json) is the fourth, and the only one that
|
|
153
|
+
is an application rather than a check: `types: ["vite/client"]`, no ambient
|
|
154
|
+
Prompt API and no `@mlc-ai/web-llm`, which is the same claim made by code that
|
|
155
|
+
actually imports and calls the things. It is where the demo caught the
|
|
156
|
+
`MLCEngineInterface` leak the first time.
|
|
157
|
+
|
|
127
158
|
## Scripts
|
|
128
159
|
|
|
129
160
|
| Script | What it does |
|
|
130
161
|
| ----------------------- | ------------------------------------------------------ |
|
|
131
|
-
| `npm run typecheck` | both source configs
|
|
162
|
+
| `npm run typecheck` | both source configs, and the specs |
|
|
132
163
|
| `npm run lint` | ESLint, type-aware |
|
|
133
164
|
| `npm run format:check` | Prettier, check only |
|
|
134
165
|
| `npm test` | Vitest; the Ollama contract suite skips with no daemon |
|
|
166
|
+
| `npm run test:e2e` | Playwright; starts the demo itself |
|
|
135
167
|
| `npm run check:surface` | builds, then reads the declarations from outside |
|
|
168
|
+
| `npm run demo` | the demo on a dev server |
|
|
169
|
+
| `npm run demo:check` | the demo's own tsconfig — a consumer with no `@types` |
|
|
136
170
|
| `npm run build` | `dist/` — JS, declarations, maps |
|
|
137
171
|
|
|
138
|
-
The Ollama suite wants a daemon on `127.0.0.1:11434` holding `granite4:350m
|
|
139
|
-
|
|
172
|
+
The Ollama suite wants a daemon on `127.0.0.1:11434` holding `granite4:350m`,
|
|
173
|
+
and the OpenAI suite wants an OpenAI-compatible server — that same daemon's
|
|
174
|
+
`/v1`, by default, which is a genuine third-party implementation of the dialect
|
|
175
|
+
and already there. Without one, each skips loudly rather than passing quietly.
|
|
176
|
+
|
|
177
|
+
## The demo
|
|
178
|
+
|
|
179
|
+
[`demo/`](demo) is a chat with all four transports behind one picker, and
|
|
180
|
+
nothing else behind it: no mock, because the engine's demo has one and this
|
|
181
|
+
repository is about what happens when there is something on the other end. On a
|
|
182
|
+
machine with no daemon, no Gemini Nano and no GPU, every entry therefore answers
|
|
183
|
+
`unavailable` — honest, and dull. A daemon on `127.0.0.1:11434` holding
|
|
184
|
+
`granite4:350m` is what makes two of the four entries answer.
|
|
185
|
+
|
|
186
|
+
```sh
|
|
187
|
+
npm install
|
|
188
|
+
npm run demo
|
|
189
|
+
```
|
|
190
|
+
|
|
191
|
+
It takes `modelpact` from npm, at the version a stranger would get, and this
|
|
192
|
+
package from `file:..` through its own `exports` — so what runs in the page is
|
|
193
|
+
`dist/`, never `src/`. Both land on one copy of the engine because `demo` is a
|
|
194
|
+
workspace of this repository; the reason that matters, and what breaks without
|
|
195
|
+
it, is in [`demo/README.md`](demo/README.md).
|
|
196
|
+
|
|
197
|
+
Export `OPENAI_API_KEY` before `npm run demo` and the `openai` entry reaches
|
|
198
|
+
`api.openai.com` instead of the daemon, through a dev-server proxy that holds
|
|
199
|
+
the key in Node — the page is never told it, and a build never carries it. See
|
|
200
|
+
[`demo/README.md`](demo/README.md#pointing-it-at-hosted-openai).
|
|
201
|
+
|
|
202
|
+
It also type-checks itself with `types: ["vite/client"]` and nothing else,
|
|
203
|
+
which makes it the second half of the surface guard above: a declaration that
|
|
204
|
+
needs an ambient global fails in an app that never installed one.
|
|
205
|
+
|
|
206
|
+
## The browser suite
|
|
207
|
+
|
|
208
|
+
[`e2e/demo-e2e.spec.ts`](e2e/demo-e2e.spec.ts) drives the demo in Chromium.
|
|
209
|
+
|
|
210
|
+
```sh
|
|
211
|
+
npx playwright install chromium # once
|
|
212
|
+
npm run test:e2e # the demo server starts itself
|
|
213
|
+
```
|
|
214
|
+
|
|
215
|
+
Eleven specs. The vitest suites next to each backend check its logic against
|
|
216
|
+
the contract from node; these check what node cannot see:
|
|
217
|
+
|
|
218
|
+
- a page is not node. `fetch` held on its own throws `Illegal invocation` in
|
|
219
|
+
one and works in the other, and that bug is invisible to any vitest run;
|
|
220
|
+
- `LanguageModel` and `navigator.gpu` exist in a browser and nowhere else, so
|
|
221
|
+
the Prompt API and WebGPU backends have their first real `availability()`
|
|
222
|
+
call here;
|
|
223
|
+
- the emitted `.d.ts` files, a bundler, and React sit between the app and the
|
|
224
|
+
backend, which no unit test reproduces.
|
|
225
|
+
|
|
226
|
+
Seven of them want a daemon and skip without one — everything that needs a
|
|
227
|
+
session to actually open, which since the mock left the picker means every
|
|
228
|
+
promise the contract makes about a session. That is the cost of a demo about
|
|
229
|
+
transports, and it is paid where it belongs: CI installs a daemon for this job.
|
|
230
|
+
|
|
231
|
+
The other four assert a branch rather than a machine, and are green anywhere. A
|
|
232
|
+
runner with no Gemini Nano and no GPU still has `unavailable` to land on, and
|
|
233
|
+
landing on it is the claim. Both HTTP transports get their refusal answer on
|
|
234
|
+
every machine too, including a laptop with Ollama running, because that spec
|
|
235
|
+
refuses the connection itself rather than waiting for a machine without one.
|
|
236
|
+
|
|
237
|
+
## CI
|
|
238
|
+
|
|
239
|
+
[`.github/workflows/ci.yml`](.github/workflows/ci.yml) has three jobs, and each
|
|
240
|
+
can go red on its own:
|
|
241
|
+
|
|
242
|
+
| Job | Runs |
|
|
243
|
+
| -------- | ------------------------------------------------------------------------ |
|
|
244
|
+
| `check` | typecheck, lint, format, vitest, `check:surface` — one install, no model |
|
|
245
|
+
| `ollama` | the same vitest suites with a daemon answering |
|
|
246
|
+
| `e2e` | Playwright against the demo, with a daemon, in Chromium |
|
|
247
|
+
|
|
248
|
+
**Adding the browser suite to a fork or another pipeline** is four steps:
|
|
249
|
+
|
|
250
|
+
```yaml
|
|
251
|
+
- uses: actions/setup-node@v7
|
|
252
|
+
with:
|
|
253
|
+
node-version-file: .nvmrc
|
|
254
|
+
cache: npm
|
|
255
|
+
# `demo` is a workspace, so this installs it too — and puts one copy of
|
|
256
|
+
# `modelpact` where both it and this package find it.
|
|
257
|
+
- run: npm ci
|
|
258
|
+
- run: npx playwright install --with-deps chromium
|
|
259
|
+
- run: npm run test:e2e
|
|
260
|
+
```
|
|
261
|
+
|
|
262
|
+
`npm run test:e2e` starts the demo itself, and the demo's `predev` builds
|
|
263
|
+
`dist/` first, so there is no build step to add and no server to start.
|
|
264
|
+
|
|
265
|
+
Two options on top of that:
|
|
266
|
+
|
|
267
|
+
- **A daemon**, which is not really optional any more: half the specs skip
|
|
268
|
+
without one. Install Ollama, `ollama serve`, `ollama pull granite4:350m`, and
|
|
269
|
+
cache `~/.ollama/models` — the `e2e` job does exactly this, restore-only,
|
|
270
|
+
sharing the key the `ollama` job saves. Without it the suite still passes,
|
|
271
|
+
on the four specs that assert a branch rather than a session.
|
|
272
|
+
- **The report on failure**, which is `actions/upload-artifact` over
|
|
273
|
+
`playwright-report/`. Playwright writes it whether or not anyone collects it.
|
|
274
|
+
|
|
275
|
+
Nothing here needs a GPU or a Chrome with Gemini Nano. Those two backends
|
|
276
|
+
answer `unavailable` on a plain runner, and the specs assert the branch.
|
|
140
277
|
|
|
141
278
|
## Releases
|
|
142
279
|
|
package/dist/index.d.ts
CHANGED
|
@@ -1,12 +1,14 @@
|
|
|
1
1
|
/**
|
|
2
2
|
* The transports that cost a consumer nothing but this package: `fetch` and
|
|
3
|
-
* JSON for the daemon, a global for the browser's
|
|
4
|
-
* dependency behind
|
|
3
|
+
* JSON for the daemon and for the OpenAI dialect, a global for the browser's
|
|
4
|
+
* own model, no runtime dependency behind any of them. Import one and a
|
|
5
|
+
* bundler drops the rest.
|
|
5
6
|
*
|
|
6
7
|
* The WebGPU backend is `modelpact-providers/webgpu` instead, because it
|
|
7
8
|
* carries `@mlc-ai/web-llm` — a separate entry for a separate dependency, the
|
|
8
9
|
* way `modelpact/testing` is separate for `vitest`.
|
|
9
10
|
*/
|
|
10
11
|
export { makeOllamaProvider, type OllamaConfig } from "./ollama.js";
|
|
12
|
+
export { makeOpenAiProvider, type OpenAiConfig } from "./openai.js";
|
|
11
13
|
export { makePromptApiProvider } from "./prompt-api.js";
|
|
12
14
|
//# sourceMappingURL=index.d.ts.map
|
package/dist/index.d.ts.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAAA
|
|
1
|
+
{"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAAA;;;;;;;;;GASG;AAEH,OAAO,EAAE,kBAAkB,EAAE,KAAK,YAAY,EAAE,MAAM,aAAa,CAAC;AACpE,OAAO,EAAE,kBAAkB,EAAE,KAAK,YAAY,EAAE,MAAM,aAAa,CAAC;AACpE,OAAO,EAAE,qBAAqB,EAAE,MAAM,iBAAiB,CAAC"}
|
package/dist/index.js
CHANGED
|
@@ -1,12 +1,14 @@
|
|
|
1
1
|
/**
|
|
2
2
|
* The transports that cost a consumer nothing but this package: `fetch` and
|
|
3
|
-
* JSON for the daemon, a global for the browser's
|
|
4
|
-
* dependency behind
|
|
3
|
+
* JSON for the daemon and for the OpenAI dialect, a global for the browser's
|
|
4
|
+
* own model, no runtime dependency behind any of them. Import one and a
|
|
5
|
+
* bundler drops the rest.
|
|
5
6
|
*
|
|
6
7
|
* The WebGPU backend is `modelpact-providers/webgpu` instead, because it
|
|
7
8
|
* carries `@mlc-ai/web-llm` — a separate entry for a separate dependency, the
|
|
8
9
|
* way `modelpact/testing` is separate for `vitest`.
|
|
9
10
|
*/
|
|
10
11
|
export { makeOllamaProvider } from "./ollama.js";
|
|
12
|
+
export { makeOpenAiProvider } from "./openai.js";
|
|
11
13
|
export { makePromptApiProvider } from "./prompt-api.js";
|
|
12
14
|
//# sourceMappingURL=index.js.map
|
package/dist/index.js.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"index.js","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAAA
|
|
1
|
+
{"version":3,"file":"index.js","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAAA;;;;;;;;;GASG;AAEH,OAAO,EAAE,kBAAkB,EAAqB,MAAM,aAAa,CAAC;AACpE,OAAO,EAAE,kBAAkB,EAAqB,MAAM,aAAa,CAAC;AACpE,OAAO,EAAE,qBAAqB,EAAE,MAAM,iBAAiB,CAAC"}
|
package/dist/openai.d.ts
ADDED
|
@@ -0,0 +1,60 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* An OpenAI-compatible server, which is a dialect rather than a company.
|
|
3
|
+
*
|
|
4
|
+
* `https://api.openai.com/v1` is the default and one instance of it. The same
|
|
5
|
+
* two endpoints answer on vLLM, llama.cpp, LM Studio, OpenRouter, Groq and
|
|
6
|
+
* Ollama's own compatibility layer, so `baseUrl` is the whole difference
|
|
7
|
+
* between them and `apiKey` is optional — a model on this machine wants no key,
|
|
8
|
+
* and a type that demanded one would be describing a service instead of a
|
|
9
|
+
* protocol.
|
|
10
|
+
*
|
|
11
|
+
* Two endpoints are the whole backend: `/models` says what is served,
|
|
12
|
+
* `/chat/completions` generates. There is no third, because there is nothing to
|
|
13
|
+
* download — the weights are the server's problem, which is why this is the one
|
|
14
|
+
* backend here that never answers `needs-download`.
|
|
15
|
+
*
|
|
16
|
+
* Shapes were read off the wire, not off the docs: a stream is Server-Sent
|
|
17
|
+
* Events whose last frame carries the counts, a tool call arrives in fragments
|
|
18
|
+
* keyed by `index` with its arguments as a JSON string rather than an object,
|
|
19
|
+
* and an error is an HTTP status with `{"error": {"message": "…"}}` — or with
|
|
20
|
+
* `{"error": "…"}`, which is what several compatible servers send instead.
|
|
21
|
+
*/
|
|
22
|
+
import { type AiProvider } from "modelpact/backend";
|
|
23
|
+
export interface OpenAiConfig {
|
|
24
|
+
/** The id as `/models` lists it, such as `gpt-4o-mini` or `qwen2.5-coder`. */
|
|
25
|
+
readonly model: string;
|
|
26
|
+
/**
|
|
27
|
+
* Up to and including the version segment, because that is where servers
|
|
28
|
+
* disagree: OpenAI serves `/v1`, LM Studio serves `/v1`, and a proxy can
|
|
29
|
+
* serve neither. A trailing slash is trimmed rather than doubled into the
|
|
30
|
+
* path.
|
|
31
|
+
*/
|
|
32
|
+
readonly baseUrl?: string;
|
|
33
|
+
/**
|
|
34
|
+
* Absent, no `Authorization` header is sent at all — which is what a local
|
|
35
|
+
* server wants, and what an unauthenticated one refuses. Note where this
|
|
36
|
+
* ends up: a key in a browser bundle is a key handed to everyone who loads
|
|
37
|
+
* the page. Point `baseUrl` at your own server there and keep the key on it.
|
|
38
|
+
*/
|
|
39
|
+
readonly apiKey?: string;
|
|
40
|
+
/**
|
|
41
|
+
* The window to measure against. Optional and defaulted to nothing on
|
|
42
|
+
* purpose: no OpenAI-compatible server reports the window it loaded a model
|
|
43
|
+
* with, so a number here is the caller's declaration and not a discovery.
|
|
44
|
+
* Absent, `usage()` answers `unknown` rather than inventing a denominator.
|
|
45
|
+
*
|
|
46
|
+
* Declared, it is also a budget: a transcript past it is an overflow, told
|
|
47
|
+
* by the counts rather than by the server. See `#hasSpentTheWindow`.
|
|
48
|
+
*/
|
|
49
|
+
readonly contextWindow?: number;
|
|
50
|
+
/** For a proxy, an extra header, or a test with no server behind it. */
|
|
51
|
+
readonly fetch?: typeof globalThis.fetch;
|
|
52
|
+
/**
|
|
53
|
+
* How many times one turn may come back with tool calls before it is failed.
|
|
54
|
+
* Per turn, not per session: a model that keeps asking spends the window on
|
|
55
|
+
* its own questions and never answers.
|
|
56
|
+
*/
|
|
57
|
+
readonly maxToolRounds?: number;
|
|
58
|
+
}
|
|
59
|
+
export declare function makeOpenAiProvider(config: OpenAiConfig): AiProvider;
|
|
60
|
+
//# sourceMappingURL=openai.d.ts.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"openai.d.ts","sourceRoot":"","sources":["../src/openai.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;;;;;;;;;;GAoBG;AACH,OAAO,EAYL,KAAK,UAAU,EAUhB,MAAM,mBAAmB,CAAC;AAE3B,MAAM,WAAW,YAAY;IAC3B,8EAA8E;IAC9E,QAAQ,CAAC,KAAK,EAAE,MAAM,CAAC;IACvB;;;;;OAKG;IACH,QAAQ,CAAC,OAAO,CAAC,EAAE,MAAM,CAAC;IAC1B;;;;;OAKG;IACH,QAAQ,CAAC,MAAM,CAAC,EAAE,MAAM,CAAC;IACzB;;;;;;;;OAQG;IACH,QAAQ,CAAC,aAAa,CAAC,EAAE,MAAM,CAAC;IAChC,wEAAwE;IACxE,QAAQ,CAAC,KAAK,CAAC,EAAE,OAAO,UAAU,CAAC,KAAK,CAAC;IACzC;;;;OAIG;IACH,QAAQ,CAAC,aAAa,CAAC,EAAE,MAAM,CAAC;CACjC;AAgnBD,wBAAgB,kBAAkB,CAAC,MAAM,EAAE,YAAY,GAAG,UAAU,CASnE"}
|
package/dist/openai.js
ADDED
|
@@ -0,0 +1,488 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* An OpenAI-compatible server, which is a dialect rather than a company.
|
|
3
|
+
*
|
|
4
|
+
* `https://api.openai.com/v1` is the default and one instance of it. The same
|
|
5
|
+
* two endpoints answer on vLLM, llama.cpp, LM Studio, OpenRouter, Groq and
|
|
6
|
+
* Ollama's own compatibility layer, so `baseUrl` is the whole difference
|
|
7
|
+
* between them and `apiKey` is optional — a model on this machine wants no key,
|
|
8
|
+
* and a type that demanded one would be describing a service instead of a
|
|
9
|
+
* protocol.
|
|
10
|
+
*
|
|
11
|
+
* Two endpoints are the whole backend: `/models` says what is served,
|
|
12
|
+
* `/chat/completions` generates. There is no third, because there is nothing to
|
|
13
|
+
* download — the weights are the server's problem, which is why this is the one
|
|
14
|
+
* backend here that never answers `needs-download`.
|
|
15
|
+
*
|
|
16
|
+
* Shapes were read off the wire, not off the docs: a stream is Server-Sent
|
|
17
|
+
* Events whose last frame carries the counts, a tool call arrives in fragments
|
|
18
|
+
* keyed by `index` with its arguments as a JSON string rather than an object,
|
|
19
|
+
* and an error is an HTTP status with `{"error": {"message": "…"}}` — or with
|
|
20
|
+
* `{"error": "…"}`, which is what several compatible servers send instead.
|
|
21
|
+
*/
|
|
22
|
+
import { AiError, contextUsage, createProvider, err, findTool, ndjsonLines, ok, runTool, tokens, } from "modelpact/backend";
|
|
23
|
+
const DEFAULTS = {
|
|
24
|
+
baseUrl: "https://api.openai.com/v1",
|
|
25
|
+
maxToolRounds: 8,
|
|
26
|
+
};
|
|
27
|
+
const asRecord = (value) => {
|
|
28
|
+
const isObject = typeof value === "object" && value !== null;
|
|
29
|
+
return isObject && !Array.isArray(value)
|
|
30
|
+
? value
|
|
31
|
+
: null;
|
|
32
|
+
};
|
|
33
|
+
const asArray = (value) => Array.isArray(value) ? value : [];
|
|
34
|
+
const asString = (value) => typeof value === "string" ? value : null;
|
|
35
|
+
const asNumber = (value) => typeof value === "number" && Number.isFinite(value) ? value : null;
|
|
36
|
+
const parseJson = (text) => {
|
|
37
|
+
try {
|
|
38
|
+
return JSON.parse(text);
|
|
39
|
+
}
|
|
40
|
+
catch {
|
|
41
|
+
return null;
|
|
42
|
+
}
|
|
43
|
+
};
|
|
44
|
+
/**
|
|
45
|
+
* `baseUrl` without its trailing slashes, walked rather than matched.
|
|
46
|
+
*
|
|
47
|
+
* `/\/+$/` is a polynomial backtrack: on a long run of slashes the engine
|
|
48
|
+
* retries the greedy `\/+` from every position in it before `$` can fail. That
|
|
49
|
+
* is harmless on a URL somebody typed into a config and not harmless as a
|
|
50
|
+
* published API, where a consumer is free to build this string from something a
|
|
51
|
+
* user sent — which is what CodeQL means by uncontrolled, and it is right. An
|
|
52
|
+
* index walk is linear and says plainly what it does.
|
|
53
|
+
*/
|
|
54
|
+
const withoutTrailingSlashes = (url) => {
|
|
55
|
+
let end = url.length;
|
|
56
|
+
while (end > 0 && url[end - 1] === "/")
|
|
57
|
+
end -= 1;
|
|
58
|
+
return url.slice(0, end);
|
|
59
|
+
};
|
|
60
|
+
const toEndpoint = (config) => ({
|
|
61
|
+
baseUrl: withoutTrailingSlashes(config.baseUrl ?? DEFAULTS.baseUrl),
|
|
62
|
+
headers: config.apiKey === undefined
|
|
63
|
+
? {}
|
|
64
|
+
: { authorization: `Bearer ${config.apiKey}` },
|
|
65
|
+
// Bound, and not optional: in a browser `fetch` is a method of the window and
|
|
66
|
+
// throws `TypeError: Illegal invocation` once it is held on its own. Node
|
|
67
|
+
// does not care, so nothing but a page catches this.
|
|
68
|
+
call: config.fetch ?? globalThis.fetch.bind(globalThis),
|
|
69
|
+
});
|
|
70
|
+
const postTo = (endpoint, path, body, signal) => endpoint.call(`${endpoint.baseUrl}${path}`, {
|
|
71
|
+
method: "POST",
|
|
72
|
+
headers: { ...endpoint.headers, "content-type": "application/json" },
|
|
73
|
+
body: JSON.stringify(body),
|
|
74
|
+
...(signal === undefined ? {} : { signal }),
|
|
75
|
+
});
|
|
76
|
+
/** `{"error": {"message": "…"}}` is OpenAI's; `{"error": "…"}` is what several compatible servers send. */
|
|
77
|
+
const readErrorText = (body) => {
|
|
78
|
+
const errorValue = asRecord(body)?.error;
|
|
79
|
+
return asString(errorValue) ?? asString(asRecord(errorValue)?.message);
|
|
80
|
+
};
|
|
81
|
+
const failureFromResponse = async (response) => {
|
|
82
|
+
const text = await response.text().catch(() => "");
|
|
83
|
+
const detail = readErrorText(parseJson(text)) ?? `${response.status} from the server`;
|
|
84
|
+
// Three moves, three kinds. 400 is this side building the request badly; 401
|
|
85
|
+
// and 403 are the key, which no retry of the same request will fix; anything
|
|
86
|
+
// else — a 404 on a wrong `baseUrl`, a 429, a 500 — is the server's own
|
|
87
|
+
// trouble, and the detail is where it says which.
|
|
88
|
+
if (response.status === 400)
|
|
89
|
+
return { kind: "invalid-input", detail };
|
|
90
|
+
if (response.status === 401 || response.status === 403)
|
|
91
|
+
return { kind: "not-allowed" };
|
|
92
|
+
return { kind: "failed", detail };
|
|
93
|
+
};
|
|
94
|
+
const listModels = async (endpoint) => {
|
|
95
|
+
const response = await endpoint.call(`${endpoint.baseUrl}/models`, {
|
|
96
|
+
headers: endpoint.headers,
|
|
97
|
+
});
|
|
98
|
+
if (response.status === 401 || response.status === 403)
|
|
99
|
+
return { kind: "refused" };
|
|
100
|
+
if (!response.ok)
|
|
101
|
+
return { kind: "unlisted" };
|
|
102
|
+
const listed = asArray(asRecord(await response.json().catch(() => null))?.data);
|
|
103
|
+
const names = listed
|
|
104
|
+
.map((entry) => asString(asRecord(entry)?.id))
|
|
105
|
+
.filter((name) => name !== null);
|
|
106
|
+
return names.length === 0 ? { kind: "unlisted" } : { kind: "listed", names };
|
|
107
|
+
};
|
|
108
|
+
const getAvailability = async (config) => {
|
|
109
|
+
const endpoint = toEndpoint(config);
|
|
110
|
+
let listing;
|
|
111
|
+
try {
|
|
112
|
+
listing = await listModels(endpoint);
|
|
113
|
+
}
|
|
114
|
+
catch (cause) {
|
|
115
|
+
// Nothing answering is not a failed request, it is no server here.
|
|
116
|
+
return { kind: "unavailable", reason: { kind: "unsupported", cause } };
|
|
117
|
+
}
|
|
118
|
+
if (listing.kind === "refused")
|
|
119
|
+
return { kind: "unavailable", reason: { kind: "not-allowed" } };
|
|
120
|
+
// A listing that named models and did not name this one is the one case
|
|
121
|
+
// where the absence is evidence.
|
|
122
|
+
if (listing.kind === "listed" && !listing.names.includes(config.model))
|
|
123
|
+
return {
|
|
124
|
+
kind: "unavailable",
|
|
125
|
+
reason: { kind: "unsupported-config", languages: [] },
|
|
126
|
+
};
|
|
127
|
+
return { kind: "ready" };
|
|
128
|
+
};
|
|
129
|
+
const toWireTools = (tools) => tools.map((tool) => ({
|
|
130
|
+
type: "function",
|
|
131
|
+
function: {
|
|
132
|
+
name: tool.name,
|
|
133
|
+
description: tool.description,
|
|
134
|
+
parameters: tool.inputSchema,
|
|
135
|
+
},
|
|
136
|
+
}));
|
|
137
|
+
/**
|
|
138
|
+
* Server-Sent Events, as much of the protocol as a chat completion uses:
|
|
139
|
+
* frames separated by a blank line, one `data:` field each, and a last frame
|
|
140
|
+
* of `[DONE]`.
|
|
141
|
+
*
|
|
142
|
+
* `ndjsonLines` upstream does the cutting and drops the blank separators, so
|
|
143
|
+
* what arrives here is one field per call. Anything that is not `data:` is a
|
|
144
|
+
* field this protocol does not send — a `:` keep-alive comment from a proxy is
|
|
145
|
+
* the one seen in practice, and dropping it is the whole handling it needs.
|
|
146
|
+
*/
|
|
147
|
+
const serverSentData = () => new TransformStream({
|
|
148
|
+
transform: (line, controller) => {
|
|
149
|
+
if (!line.startsWith("data:"))
|
|
150
|
+
return;
|
|
151
|
+
const data = line.slice("data:".length).trim();
|
|
152
|
+
if (data === "" || data === "[DONE]")
|
|
153
|
+
return;
|
|
154
|
+
controller.enqueue(data);
|
|
155
|
+
},
|
|
156
|
+
});
|
|
157
|
+
/**
|
|
158
|
+
* A call arrives in pieces: the id and the name once, the arguments as
|
|
159
|
+
* fragments of one JSON string, every piece carrying the `index` it belongs to.
|
|
160
|
+
* Concatenating in arrival order without that key would interleave two calls
|
|
161
|
+
* made in the same turn into one unparseable string.
|
|
162
|
+
*/
|
|
163
|
+
const readToolCallDeltas = (delta, drafts) => {
|
|
164
|
+
for (const listed of asArray(delta?.tool_calls)) {
|
|
165
|
+
const listedCall = asRecord(listed);
|
|
166
|
+
// A server that omits `index` is sending one call at a time, so the count
|
|
167
|
+
// so far is the key it would have used.
|
|
168
|
+
const index = asNumber(listedCall?.index) ?? drafts.size;
|
|
169
|
+
const draft = drafts.get(index) ?? {
|
|
170
|
+
id: null,
|
|
171
|
+
name: null,
|
|
172
|
+
argumentParts: [],
|
|
173
|
+
};
|
|
174
|
+
drafts.set(index, draft);
|
|
175
|
+
draft.id ??= asString(listedCall?.id);
|
|
176
|
+
const calledFunction = asRecord(listedCall?.function);
|
|
177
|
+
draft.name ??= asString(calledFunction?.name);
|
|
178
|
+
const argumentPart = asString(calledFunction?.arguments);
|
|
179
|
+
if (argumentPart !== null)
|
|
180
|
+
draft.argumentParts.push(argumentPart);
|
|
181
|
+
}
|
|
182
|
+
};
|
|
183
|
+
/** In `index` order, because that is the order the model made them in; one with no name never happened. */
|
|
184
|
+
const toToolCalls = (drafts) => {
|
|
185
|
+
const indexes = [...drafts.keys()].sort((one, other) => one - other);
|
|
186
|
+
const calls = [];
|
|
187
|
+
for (const index of indexes) {
|
|
188
|
+
const draft = drafts.get(index);
|
|
189
|
+
if (draft?.name == null)
|
|
190
|
+
continue;
|
|
191
|
+
calls.push({
|
|
192
|
+
// Several compatible servers leave the id out. The next round needs one
|
|
193
|
+
// to tie each answer to its call, so an index-shaped one stands in.
|
|
194
|
+
id: draft.id ?? `call_${index}`,
|
|
195
|
+
type: "function",
|
|
196
|
+
function: { name: draft.name, arguments: draft.argumentParts.join("") },
|
|
197
|
+
});
|
|
198
|
+
}
|
|
199
|
+
return calls;
|
|
200
|
+
};
|
|
201
|
+
/** Whatever the model sent, read as an object; a fragmented string that never closed is no arguments at all. */
|
|
202
|
+
const toToolInput = (call) => asRecord(parseJson(call.function.arguments)) ?? {};
|
|
203
|
+
const readWhole = async (stream) => {
|
|
204
|
+
const reader = stream.getReader();
|
|
205
|
+
const parts = [];
|
|
206
|
+
try {
|
|
207
|
+
for (;;) {
|
|
208
|
+
const chunk = await reader.read();
|
|
209
|
+
if (chunk.done)
|
|
210
|
+
return ok(parts.join(""));
|
|
211
|
+
parts.push(chunk.value);
|
|
212
|
+
}
|
|
213
|
+
}
|
|
214
|
+
catch (error) {
|
|
215
|
+
return err(error instanceof AiError
|
|
216
|
+
? error.failure
|
|
217
|
+
: { kind: "failed", detail: "the chat stream broke", cause: error });
|
|
218
|
+
}
|
|
219
|
+
};
|
|
220
|
+
class OpenAiModel {
|
|
221
|
+
#endpoint;
|
|
222
|
+
#model;
|
|
223
|
+
#contextWindow;
|
|
224
|
+
#system;
|
|
225
|
+
#tools;
|
|
226
|
+
#maxToolRounds;
|
|
227
|
+
#reportOverflow;
|
|
228
|
+
/** The last turn's counts, which is what the context holds now rather than a running sum. */
|
|
229
|
+
#usedTokens = 0;
|
|
230
|
+
constructor(config, options) {
|
|
231
|
+
this.#endpoint = toEndpoint(config);
|
|
232
|
+
this.#model = config.model;
|
|
233
|
+
this.#contextWindow = config.contextWindow;
|
|
234
|
+
this.#system = options.session.system;
|
|
235
|
+
this.#tools = options.request.tools ?? [];
|
|
236
|
+
this.#maxToolRounds = config.maxToolRounds ?? DEFAULTS.maxToolRounds;
|
|
237
|
+
this.#reportOverflow = options.reportOverflow;
|
|
238
|
+
}
|
|
239
|
+
generateStream = async (input, request) => {
|
|
240
|
+
const conversation = this.#toConversation(input, request);
|
|
241
|
+
const responseResult = await this.#chat(conversation, request, true);
|
|
242
|
+
if (!responseResult.ok)
|
|
243
|
+
return responseResult;
|
|
244
|
+
const body = responseResult.value.body;
|
|
245
|
+
if (body === null)
|
|
246
|
+
return err({ kind: "failed", detail: "the chat sent no body" });
|
|
247
|
+
return ok(this.#streamRounds(body, conversation, request));
|
|
248
|
+
};
|
|
249
|
+
generateWhole = async (input, request) => {
|
|
250
|
+
// A turn with tools is rounds, and rounds are the streaming path read to
|
|
251
|
+
// its end; only a plain turn has a whole-answer call worth a second shape.
|
|
252
|
+
if (this.#tools.length > 0) {
|
|
253
|
+
const streamResult = await this.generateStream(input, request);
|
|
254
|
+
return streamResult.ok ? readWhole(streamResult.value) : streamResult;
|
|
255
|
+
}
|
|
256
|
+
const conversation = this.#toConversation(input, request);
|
|
257
|
+
const responseResult = await this.#chat(conversation, request, false);
|
|
258
|
+
if (!responseResult.ok)
|
|
259
|
+
return responseResult;
|
|
260
|
+
const parsedBody = asRecord(await responseResult.value.json().catch(() => null));
|
|
261
|
+
if (parsedBody === null)
|
|
262
|
+
return err({ kind: "failed", detail: "the chat sent no JSON" });
|
|
263
|
+
const choice = asRecord(asArray(parsedBody.choices)[0]);
|
|
264
|
+
const answerText = asString(asRecord(choice?.message)?.content);
|
|
265
|
+
if (answerText === null)
|
|
266
|
+
return err({ kind: "failed", detail: "the chat sent no message" });
|
|
267
|
+
this.#charge(asRecord(parsedBody.usage), asString(choice?.finish_reason));
|
|
268
|
+
return ok(answerText);
|
|
269
|
+
};
|
|
270
|
+
usage = () => {
|
|
271
|
+
if (this.#contextWindow === undefined)
|
|
272
|
+
return { kind: "unknown" };
|
|
273
|
+
const used = tokens(this.#usedTokens) ?? ZERO_TOKENS;
|
|
274
|
+
return contextUsage(used, this.#contextWindow);
|
|
275
|
+
};
|
|
276
|
+
/**
|
|
277
|
+
* Nothing to release. The connection is one `fetch` per turn and the server
|
|
278
|
+
* keeps nothing between them, so there is no session on the other end to
|
|
279
|
+
* close.
|
|
280
|
+
*/
|
|
281
|
+
dispose = () => undefined;
|
|
282
|
+
#toConversation(input, request) {
|
|
283
|
+
const askedMessage = { role: "user", content: input };
|
|
284
|
+
const conversation = [...request.history, askedMessage];
|
|
285
|
+
// A real role here, unlike the daemon next door: this dialect has one, and
|
|
286
|
+
// a server that prefers `developer` maps `system` onto it itself.
|
|
287
|
+
return this.#system === undefined
|
|
288
|
+
? conversation
|
|
289
|
+
: [{ role: "system", content: this.#system }, ...conversation];
|
|
290
|
+
}
|
|
291
|
+
async #chat(messages, request, stream) {
|
|
292
|
+
const body = {
|
|
293
|
+
model: this.#model,
|
|
294
|
+
messages,
|
|
295
|
+
stream,
|
|
296
|
+
// Only with `stream`: sent alongside `stream: false` it is a 400.
|
|
297
|
+
...(stream ? { stream_options: { include_usage: true } } : {}),
|
|
298
|
+
...(request.schema === undefined
|
|
299
|
+
? {}
|
|
300
|
+
: {
|
|
301
|
+
response_format: {
|
|
302
|
+
type: "json_schema",
|
|
303
|
+
json_schema: { name: "answer", schema: request.schema },
|
|
304
|
+
},
|
|
305
|
+
}),
|
|
306
|
+
...(this.#tools.length === 0 ? {} : { tools: toWireTools(this.#tools) }),
|
|
307
|
+
};
|
|
308
|
+
const response = await postTo(this.#endpoint, "/chat/completions", body, request.signal);
|
|
309
|
+
return response.ok
|
|
310
|
+
? ok(response)
|
|
311
|
+
: err(await failureFromResponse(response));
|
|
312
|
+
}
|
|
313
|
+
/**
|
|
314
|
+
* Rounds: an answer is read to its end, and where it ends in tool calls the
|
|
315
|
+
* calls are answered and the conversation sent again. Only text reaches the
|
|
316
|
+
* caller, so a turn that is all calls is silent until its last round.
|
|
317
|
+
* Bounded, because a model that keeps calling would spend the window on it.
|
|
318
|
+
*/
|
|
319
|
+
#streamRounds(firstBody, firstConversation, request) {
|
|
320
|
+
let conversation = firstConversation;
|
|
321
|
+
let currentRound = this.#openRound(firstBody);
|
|
322
|
+
let roundsTaken = 0;
|
|
323
|
+
const advance = async (controller) => {
|
|
324
|
+
for (;;) {
|
|
325
|
+
const chunk = await currentRound.reader.read();
|
|
326
|
+
if (!chunk.done) {
|
|
327
|
+
controller.enqueue(chunk.value);
|
|
328
|
+
return;
|
|
329
|
+
}
|
|
330
|
+
const calls = toToolCalls(currentRound.drafts);
|
|
331
|
+
if (calls.length === 0) {
|
|
332
|
+
controller.close();
|
|
333
|
+
return;
|
|
334
|
+
}
|
|
335
|
+
roundsTaken += 1;
|
|
336
|
+
if (roundsTaken > this.#maxToolRounds) {
|
|
337
|
+
throw new AiError({
|
|
338
|
+
kind: "failed",
|
|
339
|
+
detail: `the model called tools ${roundsTaken} times without answering`,
|
|
340
|
+
});
|
|
341
|
+
}
|
|
342
|
+
conversation = await this.#answerToolCalls(conversation, currentRound.contentParts.join(""), calls, request.signal);
|
|
343
|
+
const nextResult = await this.#chat(conversation, request, true);
|
|
344
|
+
if (!nextResult.ok)
|
|
345
|
+
throw new AiError(nextResult.error);
|
|
346
|
+
const nextBody = nextResult.value.body;
|
|
347
|
+
if (nextBody === null)
|
|
348
|
+
throw new AiError({
|
|
349
|
+
kind: "failed",
|
|
350
|
+
detail: "the chat sent no body",
|
|
351
|
+
});
|
|
352
|
+
currentRound = this.#openRound(nextBody);
|
|
353
|
+
}
|
|
354
|
+
};
|
|
355
|
+
return new ReadableStream({
|
|
356
|
+
pull: (controller) => advance(controller),
|
|
357
|
+
cancel: (reason) => currentRound.reader.cancel(reason),
|
|
358
|
+
});
|
|
359
|
+
}
|
|
360
|
+
/**
|
|
361
|
+
* Content out, calls and counts kept.
|
|
362
|
+
*
|
|
363
|
+
* `finish_reason` and the counts arrive in different frames — the reason on
|
|
364
|
+
* the last frame that has a choice, the counts on one after it whose
|
|
365
|
+
* `choices` is empty — so the reason is remembered and every frame from there
|
|
366
|
+
* on charges. `reportOverflow` is idempotent, which is what makes charging
|
|
367
|
+
* twice cost nothing; a server that ignored `stream_options` and sent no
|
|
368
|
+
* counts at all still reaches the `length` half of the check.
|
|
369
|
+
*/
|
|
370
|
+
#openRound(body) {
|
|
371
|
+
const contentParts = [];
|
|
372
|
+
const drafts = new Map();
|
|
373
|
+
let finishReason = null;
|
|
374
|
+
const readFrames = new TransformStream({
|
|
375
|
+
transform: (data, controller) => {
|
|
376
|
+
const frame = asRecord(parseJson(data));
|
|
377
|
+
if (frame === null)
|
|
378
|
+
return;
|
|
379
|
+
const errorText = readErrorText(frame);
|
|
380
|
+
if (errorText !== null)
|
|
381
|
+
throw new AiError({ kind: "failed", detail: errorText });
|
|
382
|
+
const choice = asRecord(asArray(frame.choices)[0]);
|
|
383
|
+
finishReason = asString(choice?.finish_reason) ?? finishReason;
|
|
384
|
+
const usage = asRecord(frame.usage);
|
|
385
|
+
if (usage !== null || finishReason !== null)
|
|
386
|
+
this.#charge(usage, finishReason);
|
|
387
|
+
const delta = asRecord(choice?.delta);
|
|
388
|
+
readToolCallDeltas(delta, drafts);
|
|
389
|
+
const text = asString(delta?.content) ?? "";
|
|
390
|
+
if (text === "")
|
|
391
|
+
return;
|
|
392
|
+
contentParts.push(text);
|
|
393
|
+
controller.enqueue(text);
|
|
394
|
+
},
|
|
395
|
+
});
|
|
396
|
+
const deltas = body
|
|
397
|
+
.pipeThrough(new TextDecoderStream())
|
|
398
|
+
.pipeThrough(ndjsonLines())
|
|
399
|
+
.pipeThrough(serverSentData())
|
|
400
|
+
.pipeThrough(readFrames);
|
|
401
|
+
return { reader: deltas.getReader(), contentParts, drafts };
|
|
402
|
+
}
|
|
403
|
+
/**
|
|
404
|
+
* The model's call goes back as its own turn, then each answer under the
|
|
405
|
+
* `tool` role tied to it by `tool_call_id` — the dialect matches answers to
|
|
406
|
+
* calls by id, not by name, so two calls to the same tool in one turn stay
|
|
407
|
+
* apart. A name the model made up is answered rather than failing the turn:
|
|
408
|
+
* the model can pick again, where a failed turn could not. A tool that throws
|
|
409
|
+
* cannot be answered for, and ends the turn.
|
|
410
|
+
*/
|
|
411
|
+
async #answerToolCalls(conversation, saidText, calls, signal) {
|
|
412
|
+
const answered = [
|
|
413
|
+
...conversation,
|
|
414
|
+
{ role: "assistant", content: saidText, tool_calls: calls },
|
|
415
|
+
];
|
|
416
|
+
for (const call of calls) {
|
|
417
|
+
const name = call.function.name;
|
|
418
|
+
const tool = findTool(this.#tools, name);
|
|
419
|
+
if (tool === undefined) {
|
|
420
|
+
answered.push({
|
|
421
|
+
role: "tool",
|
|
422
|
+
content: `there is no tool called "${name}"`,
|
|
423
|
+
tool_call_id: call.id,
|
|
424
|
+
});
|
|
425
|
+
continue;
|
|
426
|
+
}
|
|
427
|
+
const toolResult = await runTool(tool, toToolInput(call), signal);
|
|
428
|
+
if (!toolResult.ok)
|
|
429
|
+
throw new AiError(toolResult.error);
|
|
430
|
+
answered.push({
|
|
431
|
+
role: "tool",
|
|
432
|
+
content: toolResult.value,
|
|
433
|
+
tool_call_id: call.id,
|
|
434
|
+
});
|
|
435
|
+
}
|
|
436
|
+
return answered;
|
|
437
|
+
}
|
|
438
|
+
/**
|
|
439
|
+
* `prompt_tokens` is the whole prompt, history included, so the two counts
|
|
440
|
+
* together are what the window holds after this turn. Summing across turns
|
|
441
|
+
* would count the history once per turn. `total_tokens` is preferred where
|
|
442
|
+
* the server sends it, because a server that bills for reasoning tokens puts
|
|
443
|
+
* them there and in neither of the other two.
|
|
444
|
+
*/
|
|
445
|
+
#charge(usage, finishReason) {
|
|
446
|
+
if (usage !== null) {
|
|
447
|
+
const promptTokens = asNumber(usage.prompt_tokens) ?? 0;
|
|
448
|
+
const answerTokens = asNumber(usage.completion_tokens) ?? 0;
|
|
449
|
+
this.#usedTokens =
|
|
450
|
+
asNumber(usage.total_tokens) ?? promptTokens + answerTokens;
|
|
451
|
+
}
|
|
452
|
+
if (this.#hasSpentTheWindow(finishReason))
|
|
453
|
+
this.#reportOverflow();
|
|
454
|
+
}
|
|
455
|
+
/**
|
|
456
|
+
* Two ways to learn it, and they are independent.
|
|
457
|
+
*
|
|
458
|
+
* The server can say so: `max_tokens` is deliberately never sent, so
|
|
459
|
+
* `finish_reason: "length"` can only mean the model ran out of context. And
|
|
460
|
+
* the counts can say so against a `contextWindow` the caller declared, which
|
|
461
|
+
* is the only reading available for a server that stopped without saying why
|
|
462
|
+
* — and the only one at all for the many that never report a window.
|
|
463
|
+
*/
|
|
464
|
+
#hasSpentTheWindow(finishReason) {
|
|
465
|
+
if (finishReason === "length")
|
|
466
|
+
return true;
|
|
467
|
+
return (this.#contextWindow !== undefined &&
|
|
468
|
+
this.#usedTokens >= this.#contextWindow);
|
|
469
|
+
}
|
|
470
|
+
}
|
|
471
|
+
const ZERO_TOKENS = tokens(0);
|
|
472
|
+
/**
|
|
473
|
+
* Nothing to do but hand back a connection. There are no weights to fetch and
|
|
474
|
+
* no session to open on the far side — `availability` already asked the only
|
|
475
|
+
* question there was, and every turn is a request of its own.
|
|
476
|
+
*/
|
|
477
|
+
const connectOpenAi = (config, options) => Promise.resolve(ok(new OpenAiModel(config, options)));
|
|
478
|
+
export function makeOpenAiProvider(config) {
|
|
479
|
+
const backend = {
|
|
480
|
+
name: "openai",
|
|
481
|
+
modalities: ["text"],
|
|
482
|
+
tools: true,
|
|
483
|
+
availability: () => getAvailability(config),
|
|
484
|
+
connect: (options) => connectOpenAi(config, options),
|
|
485
|
+
};
|
|
486
|
+
return createProvider(backend);
|
|
487
|
+
}
|
|
488
|
+
//# sourceMappingURL=openai.js.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"openai.js","sourceRoot":"","sources":["../src/openai.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;;;;;;;;;;GAoBG;AACH,OAAO,EACL,OAAO,EACP,YAAY,EACZ,cAAc,EACd,GAAG,EACH,QAAQ,EACR,WAAW,EACX,EAAE,EACF,OAAO,EACP,MAAM,GAaP,MAAM,mBAAmB,CAAC;AAuC3B,MAAM,QAAQ,GAAG;IACf,OAAO,EAAE,2BAA2B;IACpC,aAAa,EAAE,CAAC;CACjB,CAAC;AAEF,MAAM,QAAQ,GAAG,CAAC,KAAc,EAAkC,EAAE;IAClE,MAAM,QAAQ,GAAG,OAAO,KAAK,KAAK,QAAQ,IAAI,KAAK,KAAK,IAAI,CAAC;IAC7D,OAAO,QAAQ,IAAI,CAAC,KAAK,CAAC,OAAO,CAAC,KAAK,CAAC;QACtC,CAAC,CAAE,KAAiC;QACpC,CAAC,CAAC,IAAI,CAAC;AACX,CAAC,CAAC;AAEF,MAAM,OAAO,GAAG,CAAC,KAAc,EAAsB,EAAE,CACrD,KAAK,CAAC,OAAO,CAAC,KAAK,CAAC,CAAC,CAAC,CAAE,KAA4B,CAAC,CAAC,CAAC,EAAE,CAAC;AAE5D,MAAM,QAAQ,GAAG,CAAC,KAAc,EAAiB,EAAE,CACjD,OAAO,KAAK,KAAK,QAAQ,CAAC,CAAC,CAAC,KAAK,CAAC,CAAC,CAAC,IAAI,CAAC;AAE3C,MAAM,QAAQ,GAAG,CAAC,KAAc,EAAiB,EAAE,CACjD,OAAO,KAAK,KAAK,QAAQ,IAAI,MAAM,CAAC,QAAQ,CAAC,KAAK,CAAC,CAAC,CAAC,CAAC,KAAK,CAAC,CAAC,CAAC,IAAI,CAAC;AAErE,MAAM,SAAS,GAAG,CAAC,IAAY,EAAW,EAAE;IAC1C,IAAI,CAAC;QACH,OAAO,IAAI,CAAC,KAAK,CAAC,IAAI,CAAC,CAAC;IAC1B,CAAC;IAAC,MAAM,CAAC;QACP,OAAO,IAAI,CAAC;IACd,CAAC;AACH,CAAC,CAAC;AAEF;;;;;;;;;GASG;AACH,MAAM,sBAAsB,GAAG,CAAC,GAAW,EAAU,EAAE;IACrD,IAAI,GAAG,GAAG,GAAG,CAAC,MAAM,CAAC;IACrB,OAAO,GAAG,GAAG,CAAC,IAAI,GAAG,CAAC,GAAG,GAAG,CAAC,CAAC,KAAK,GAAG;QAAE,GAAG,IAAI,CAAC,CAAC;IACjD,OAAO,GAAG,CAAC,KAAK,CAAC,CAAC,EAAE,GAAG,CAAC,CAAC;AAC3B,CAAC,CAAC;AAQF,MAAM,UAAU,GAAG,CAAC,MAAoB,EAAY,EAAE,CAAC,CAAC;IACtD,OAAO,EAAE,sBAAsB,CAAC,MAAM,CAAC,OAAO,IAAI,QAAQ,CAAC,OAAO,CAAC;IACnE,OAAO,EACL,MAAM,CAAC,MAAM,KAAK,SAAS;QACzB,CAAC,CAAC,EAAE;QACJ,CAAC,CAAC,EAAE,aAAa,EAAE,UAAU,MAAM,CAAC,MAAM,EAAE,EAAE;IAClD,8EAA8E;IAC9E,0EAA0E;IAC1E,qDAAqD;IACrD,IAAI,EAAE,MAAM,CAAC,KAAK,IAAI,UAAU,CAAC,KAAK,CAAC,IAAI,CAAC,UAAU,CAAC;CACxD,CAAC,CAAC;AAEH,MAAM,MAAM,GAAG,CACb,QAAkB,EAClB,IAAY,EACZ,IAAa,EACb,MAAoB,EACD,EAAE,CACrB,QAAQ,CAAC,IAAI,CAAC,GAAG,QAAQ,CAAC,OAAO,GAAG,IAAI,EAAE,EAAE;IAC1C,MAAM,EAAE,MAAM;IACd,OAAO,EAAE,EAAE,GAAG,QAAQ,CAAC,OAAO,EAAE,cAAc,EAAE,kBAAkB,EAAE;IACpE,IAAI,EAAE,IAAI,CAAC,SAAS,CAAC,IAAI,CAAC;IAC1B,GAAG,CAAC,MAAM,KAAK,SAAS,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,EAAE,MAAM,EAAE,CAAC;CAC5C,CAAC,CAAC;AAEL,2GAA2G;AAC3G,MAAM,aAAa,GAAG,CAAC,IAAa,EAAiB,EAAE;IACrD,MAAM,UAAU,GAAG,QAAQ,CAAC,IAAI,CAAC,EAAE,KAAK,CAAC;IACzC,OAAO,QAAQ,CAAC,UAAU,CAAC,IAAI,QAAQ,CAAC,QAAQ,CAAC,UAAU,CAAC,EAAE,OAAO,CAAC,CAAC;AACzE,CAAC,CAAC;AAEF,MAAM,mBAAmB,GAAG,KAAK,EAAE,QAAkB,EAAsB,EAAE;IAC3E,MAAM,IAAI,GAAG,MAAM,QAAQ,CAAC,IAAI,EAAE,CAAC,KAAK,CAAC,GAAG,EAAE,CAAC,EAAE,CAAC,CAAC;IACnD,MAAM,MAAM,GACV,aAAa,CAAC,SAAS,CAAC,IAAI,CAAC,CAAC,IAAI,GAAG,QAAQ,CAAC,MAAM,kBAAkB,CAAC;IACzE,6EAA6E;IAC7E,6EAA6E;IAC7E,wEAAwE;IACxE,kDAAkD;IAClD,IAAI,QAAQ,CAAC,MAAM,KAAK,GAAG;QAAE,OAAO,EAAE,IAAI,EAAE,eAAe,EAAE,MAAM,EAAE,CAAC;IACtE,IAAI,QAAQ,CAAC,MAAM,KAAK,GAAG,IAAI,QAAQ,CAAC,MAAM,KAAK,GAAG;QACpD,OAAO,EAAE,IAAI,EAAE,aAAa,EAAE,CAAC;IACjC,OAAO,EAAE,IAAI,EAAE,QAAQ,EAAE,MAAM,EAAE,CAAC;AACpC,CAAC,CAAC;AAgBF,MAAM,UAAU,GAAG,KAAK,EAAE,QAAkB,EAAoB,EAAE;IAChE,MAAM,QAAQ,GAAG,MAAM,QAAQ,CAAC,IAAI,CAAC,GAAG,QAAQ,CAAC,OAAO,SAAS,EAAE;QACjE,OAAO,EAAE,QAAQ,CAAC,OAAO;KAC1B,CAAC,CAAC;IACH,IAAI,QAAQ,CAAC,MAAM,KAAK,GAAG,IAAI,QAAQ,CAAC,MAAM,KAAK,GAAG;QACpD,OAAO,EAAE,IAAI,EAAE,SAAS,EAAE,CAAC;IAC7B,IAAI,CAAC,QAAQ,CAAC,EAAE;QAAE,OAAO,EAAE,IAAI,EAAE,UAAU,EAAE,CAAC;IAC9C,MAAM,MAAM,GAAG,OAAO,CACpB,QAAQ,CAAC,MAAM,QAAQ,CAAC,IAAI,EAAE,CAAC,KAAK,CAAC,GAAG,EAAE,CAAC,IAAI,CAAC,CAAC,EAAE,IAAI,CACxD,CAAC;IACF,MAAM,KAAK,GAAG,MAAM;SACjB,GAAG,CAAC,CAAC,KAAK,EAAE,EAAE,CAAC,QAAQ,CAAC,QAAQ,CAAC,KAAK,CAAC,EAAE,EAAE,CAAC,CAAC;SAC7C,MAAM,CAAC,CAAC,IAAI,EAAkB,EAAE,CAAC,IAAI,KAAK,IAAI,CAAC,CAAC;IACnD,OAAO,KAAK,CAAC,MAAM,KAAK,CAAC,CAAC,CAAC,CAAC,EAAE,IAAI,EAAE,UAAU,EAAE,CAAC,CAAC,CAAC,EAAE,IAAI,EAAE,QAAQ,EAAE,KAAK,EAAE,CAAC;AAC/E,CAAC,CAAC;AAEF,MAAM,eAAe,GAAG,KAAK,EAAE,MAAoB,EAAyB,EAAE;IAC5E,MAAM,QAAQ,GAAG,UAAU,CAAC,MAAM,CAAC,CAAC;IACpC,IAAI,OAAgB,CAAC;IACrB,IAAI,CAAC;QACH,OAAO,GAAG,MAAM,UAAU,CAAC,QAAQ,CAAC,CAAC;IACvC,CAAC;IAAC,OAAO,KAAK,EAAE,CAAC;QACf,mEAAmE;QACnE,OAAO,EAAE,IAAI,EAAE,aAAa,EAAE,MAAM,EAAE,EAAE,IAAI,EAAE,aAAa,EAAE,KAAK,EAAE,EAAE,CAAC;IACzE,CAAC;IACD,IAAI,OAAO,CAAC,IAAI,KAAK,SAAS;QAC5B,OAAO,EAAE,IAAI,EAAE,aAAa,EAAE,MAAM,EAAE,EAAE,IAAI,EAAE,aAAa,EAAE,EAAE,CAAC;IAClE,wEAAwE;IACxE,iCAAiC;IACjC,IAAI,OAAO,CAAC,IAAI,KAAK,QAAQ,IAAI,CAAC,OAAO,CAAC,KAAK,CAAC,QAAQ,CAAC,MAAM,CAAC,KAAK,CAAC;QACpE,OAAO;YACL,IAAI,EAAE,aAAa;YACnB,MAAM,EAAE,EAAE,IAAI,EAAE,oBAAoB,EAAE,SAAS,EAAE,EAAE,EAAE;SACtD,CAAC;IACJ,OAAO,EAAE,IAAI,EAAE,OAAO,EAAE,CAAC;AAC3B,CAAC,CAAC;AAkEF,MAAM,WAAW,GAAG,CAAC,KAAsB,EAAc,EAAE,CACzD,KAAK,CAAC,GAAG,CAAC,CAAC,IAAI,EAAE,EAAE,CAAC,CAAC;IACnB,IAAI,EAAE,UAAU;IAChB,QAAQ,EAAE;QACR,IAAI,EAAE,IAAI,CAAC,IAAI;QACf,WAAW,EAAE,IAAI,CAAC,WAAW;QAC7B,UAAU,EAAE,IAAI,CAAC,WAAW;KAC7B;CACF,CAAC,CAAC,CAAC;AAEN;;;;;;;;;GASG;AACH,MAAM,cAAc,GAAG,GAAoC,EAAE,CAC3D,IAAI,eAAe,CAAC;IAClB,SAAS,EAAE,CAAC,IAAI,EAAE,UAAU,EAAE,EAAE;QAC9B,IAAI,CAAC,IAAI,CAAC,UAAU,CAAC,OAAO,CAAC;YAAE,OAAO;QACtC,MAAM,IAAI,GAAG,IAAI,CAAC,KAAK,CAAC,OAAO,CAAC,MAAM,CAAC,CAAC,IAAI,EAAE,CAAC;QAC/C,IAAI,IAAI,KAAK,EAAE,IAAI,IAAI,KAAK,QAAQ;YAAE,OAAO;QAC7C,UAAU,CAAC,OAAO,CAAC,IAAI,CAAC,CAAC;IAC3B,CAAC;CACF,CAAC,CAAC;AAEL;;;;;GAKG;AACH,MAAM,kBAAkB,GAAG,CACzB,KAAqC,EACrC,MAAkC,EAC5B,EAAE;IACR,KAAK,MAAM,MAAM,IAAI,OAAO,CAAC,KAAK,EAAE,UAAU,CAAC,EAAE,CAAC;QAChD,MAAM,UAAU,GAAG,QAAQ,CAAC,MAAM,CAAC,CAAC;QACpC,0EAA0E;QAC1E,wCAAwC;QACxC,MAAM,KAAK,GAAG,QAAQ,CAAC,UAAU,EAAE,KAAK,CAAC,IAAI,MAAM,CAAC,IAAI,CAAC;QACzD,MAAM,KAAK,GAAG,MAAM,CAAC,GAAG,CAAC,KAAK,CAAC,IAAI;YACjC,EAAE,EAAE,IAAI;YACR,IAAI,EAAE,IAAI;YACV,aAAa,EAAE,EAAE;SAClB,CAAC;QACF,MAAM,CAAC,GAAG,CAAC,KAAK,EAAE,KAAK,CAAC,CAAC;QACzB,KAAK,CAAC,EAAE,KAAK,QAAQ,CAAC,UAAU,EAAE,EAAE,CAAC,CAAC;QACtC,MAAM,cAAc,GAAG,QAAQ,CAAC,UAAU,EAAE,QAAQ,CAAC,CAAC;QACtD,KAAK,CAAC,IAAI,KAAK,QAAQ,CAAC,cAAc,EAAE,IAAI,CAAC,CAAC;QAC9C,MAAM,YAAY,GAAG,QAAQ,CAAC,cAAc,EAAE,SAAS,CAAC,CAAC;QACzD,IAAI,YAAY,KAAK,IAAI;YAAE,KAAK,CAAC,aAAa,CAAC,IAAI,CAAC,YAAY,CAAC,CAAC;IACpE,CAAC;AACH,CAAC,CAAC;AAEF,2GAA2G;AAC3G,MAAM,WAAW,GAAG,CAAC,MAAkC,EAAkB,EAAE;IACzE,MAAM,OAAO,GAAG,CAAC,GAAG,MAAM,CAAC,IAAI,EAAE,CAAC,CAAC,IAAI,CAAC,CAAC,GAAG,EAAE,KAAK,EAAE,EAAE,CAAC,GAAG,GAAG,KAAK,CAAC,CAAC;IACrE,MAAM,KAAK,GAAmB,EAAE,CAAC;IACjC,KAAK,MAAM,KAAK,IAAI,OAAO,EAAE,CAAC;QAC5B,MAAM,KAAK,GAAG,MAAM,CAAC,GAAG,CAAC,KAAK,CAAC,CAAC;QAChC,IAAI,KAAK,EAAE,IAAI,IAAI,IAAI;YAAE,SAAS;QAClC,KAAK,CAAC,IAAI,CAAC;YACT,wEAAwE;YACxE,oEAAoE;YACpE,EAAE,EAAE,KAAK,CAAC,EAAE,IAAI,QAAQ,KAAK,EAAE;YAC/B,IAAI,EAAE,UAAU;YAChB,QAAQ,EAAE,EAAE,IAAI,EAAE,KAAK,CAAC,IAAI,EAAE,SAAS,EAAE,KAAK,CAAC,aAAa,CAAC,IAAI,CAAC,EAAE,CAAC,EAAE;SACxE,CAAC,CAAC;IACL,CAAC;IACD,OAAO,KAAK,CAAC;AACf,CAAC,CAAC;AAEF,gHAAgH;AAChH,MAAM,WAAW,GAAG,CAAC,IAAkB,EAA2B,EAAE,CAClE,QAAQ,CAAC,SAAS,CAAC,IAAI,CAAC,QAAQ,CAAC,SAAS,CAAC,CAAC,IAAI,EAAE,CAAC;AAErD,MAAM,SAAS,GAAG,KAAK,EACrB,MAA8B,EACM,EAAE;IACtC,MAAM,MAAM,GAAG,MAAM,CAAC,SAAS,EAAE,CAAC;IAClC,MAAM,KAAK,GAAa,EAAE,CAAC;IAC3B,IAAI,CAAC;QACH,SAAS,CAAC;YACR,MAAM,KAAK,GAAG,MAAM,MAAM,CAAC,IAAI,EAAE,CAAC;YAClC,IAAI,KAAK,CAAC,IAAI;gBAAE,OAAO,EAAE,CAAC,KAAK,CAAC,IAAI,CAAC,EAAE,CAAC,CAAC,CAAC;YAC1C,KAAK,CAAC,IAAI,CAAC,KAAK,CAAC,KAAK,CAAC,CAAC;QAC1B,CAAC;IACH,CAAC;IAAC,OAAO,KAAK,EAAE,CAAC;QACf,OAAO,GAAG,CACR,KAAK,YAAY,OAAO;YACtB,CAAC,CAAC,KAAK,CAAC,OAAO;YACf,CAAC,CAAC,EAAE,IAAI,EAAE,QAAQ,EAAE,MAAM,EAAE,uBAAuB,EAAE,KAAK,EAAE,KAAK,EAAE,CACtE,CAAC;IACJ,CAAC;AACH,CAAC,CAAC;AAEF,MAAM,WAAW;IACN,SAAS,CAAW;IACpB,MAAM,CAAS;IACf,cAAc,CAAqB;IACnC,OAAO,CAAqB;IAC5B,MAAM,CAAkB;IACxB,cAAc,CAAS;IACvB,eAAe,CAAa;IACrC,6FAA6F;IAC7F,WAAW,GAAG,CAAC,CAAC;IAEhB,YAAY,MAAoB,EAAE,OAAuB;QACvD,IAAI,CAAC,SAAS,GAAG,UAAU,CAAC,MAAM,CAAC,CAAC;QACpC,IAAI,CAAC,MAAM,GAAG,MAAM,CAAC,KAAK,CAAC;QAC3B,IAAI,CAAC,cAAc,GAAG,MAAM,CAAC,aAAa,CAAC;QAC3C,IAAI,CAAC,OAAO,GAAG,OAAO,CAAC,OAAO,CAAC,MAAM,CAAC;QACtC,IAAI,CAAC,MAAM,GAAG,OAAO,CAAC,OAAO,CAAC,KAAK,IAAI,EAAE,CAAC;QAC1C,IAAI,CAAC,cAAc,GAAG,MAAM,CAAC,aAAa,IAAI,QAAQ,CAAC,aAAa,CAAC;QACrE,IAAI,CAAC,eAAe,GAAG,OAAO,CAAC,cAAc,CAAC;IAChD,CAAC;IAEQ,cAAc,GAAG,KAAK,EAC7B,KAAa,EACb,OAAwB,EAC4B,EAAE;QACtD,MAAM,YAAY,GAAG,IAAI,CAAC,eAAe,CAAC,KAAK,EAAE,OAAO,CAAC,CAAC;QAC1D,MAAM,cAAc,GAAG,MAAM,IAAI,CAAC,KAAK,CAAC,YAAY,EAAE,OAAO,EAAE,IAAI,CAAC,CAAC;QACrE,IAAI,CAAC,cAAc,CAAC,EAAE;YAAE,OAAO,cAAc,CAAC;QAC9C,MAAM,IAAI,GAAG,cAAc,CAAC,KAAK,CAAC,IAAI,CAAC;QACvC,IAAI,IAAI,KAAK,IAAI;YACf,OAAO,GAAG,CAAC,EAAE,IAAI,EAAE,QAAQ,EAAE,MAAM,EAAE,uBAAuB,EAAE,CAAC,CAAC;QAClE,OAAO,EAAE,CAAC,IAAI,CAAC,aAAa,CAAC,IAAI,EAAE,YAAY,EAAE,OAAO,CAAC,CAAC,CAAC;IAC7D,CAAC,CAAC;IAEO,aAAa,GAAG,KAAK,EAC5B,KAAa,EACb,OAAwB,EACY,EAAE;QACtC,yEAAyE;QACzE,2EAA2E;QAC3E,IAAI,IAAI,CAAC,MAAM,CAAC,MAAM,GAAG,CAAC,EAAE,CAAC;YAC3B,MAAM,YAAY,GAAG,MAAM,IAAI,CAAC,cAAc,CAAC,KAAK,EAAE,OAAO,CAAC,CAAC;YAC/D,OAAO,YAAY,CAAC,EAAE,CAAC,CAAC,CAAC,SAAS,CAAC,YAAY,CAAC,KAAK,CAAC,CAAC,CAAC,CAAC,YAAY,CAAC;QACxE,CAAC;QACD,MAAM,YAAY,GAAG,IAAI,CAAC,eAAe,CAAC,KAAK,EAAE,OAAO,CAAC,CAAC;QAC1D,MAAM,cAAc,GAAG,MAAM,IAAI,CAAC,KAAK,CAAC,YAAY,EAAE,OAAO,EAAE,KAAK,CAAC,CAAC;QACtE,IAAI,CAAC,cAAc,CAAC,EAAE;YAAE,OAAO,cAAc,CAAC;QAC9C,MAAM,UAAU,GAAG,QAAQ,CACzB,MAAM,cAAc,CAAC,KAAK,CAAC,IAAI,EAAE,CAAC,KAAK,CAAC,GAAG,EAAE,CAAC,IAAI,CAAC,CACpD,CAAC;QACF,IAAI,UAAU,KAAK,IAAI;YACrB,OAAO,GAAG,CAAC,EAAE,IAAI,EAAE,QAAQ,EAAE,MAAM,EAAE,uBAAuB,EAAE,CAAC,CAAC;QAClE,MAAM,MAAM,GAAG,QAAQ,CAAC,OAAO,CAAC,UAAU,CAAC,OAAO,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC;QACxD,MAAM,UAAU,GAAG,QAAQ,CAAC,QAAQ,CAAC,MAAM,EAAE,OAAO,CAAC,EAAE,OAAO,CAAC,CAAC;QAChE,IAAI,UAAU,KAAK,IAAI;YACrB,OAAO,GAAG,CAAC,EAAE,IAAI,EAAE,QAAQ,EAAE,MAAM,EAAE,0BAA0B,EAAE,CAAC,CAAC;QACrE,IAAI,CAAC,OAAO,CAAC,QAAQ,CAAC,UAAU,CAAC,KAAK,CAAC,EAAE,QAAQ,CAAC,MAAM,EAAE,aAAa,CAAC,CAAC,CAAC;QAC1E,OAAO,EAAE,CAAC,UAAU,CAAC,CAAC;IACxB,CAAC,CAAC;IAEO,KAAK,GAAG,GAAiB,EAAE;QAClC,IAAI,IAAI,CAAC,cAAc,KAAK,SAAS;YAAE,OAAO,EAAE,IAAI,EAAE,SAAS,EAAE,CAAC;QAClE,MAAM,IAAI,GAAG,MAAM,CAAC,IAAI,CAAC,WAAW,CAAC,IAAI,WAAW,CAAC;QACrD,OAAO,YAAY,CAAC,IAAI,EAAE,IAAI,CAAC,cAAc,CAAC,CAAC;IACjD,CAAC,CAAC;IAEF;;;;OAIG;IACM,OAAO,GAAG,GAAS,EAAE,CAAC,SAAS,CAAC;IAEzC,eAAe,CAAC,KAAa,EAAE,OAAwB;QACrD,MAAM,YAAY,GAAc,EAAE,IAAI,EAAE,MAAM,EAAE,OAAO,EAAE,KAAK,EAAE,CAAC;QACjE,MAAM,YAAY,GAAkB,CAAC,GAAG,OAAO,CAAC,OAAO,EAAE,YAAY,CAAC,CAAC;QACvE,2EAA2E;QAC3E,kEAAkE;QAClE,OAAO,IAAI,CAAC,OAAO,KAAK,SAAS;YAC/B,CAAC,CAAC,YAAY;YACd,CAAC,CAAC,CAAC,EAAE,IAAI,EAAE,QAAQ,EAAE,OAAO,EAAE,IAAI,CAAC,OAAO,EAAE,EAAE,GAAG,YAAY,CAAC,CAAC;IACnE,CAAC;IAED,KAAK,CAAC,KAAK,CACT,QAAgC,EAChC,OAAwB,EACxB,MAAe;QAEf,MAAM,IAAI,GAAa;YACrB,KAAK,EAAE,IAAI,CAAC,MAAM;YAClB,QAAQ;YACR,MAAM;YACN,kEAAkE;YAClE,GAAG,CAAC,MAAM,CAAC,CAAC,CAAC,EAAE,cAAc,EAAE,EAAE,aAAa,EAAE,IAAI,EAAE,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC;YAC9D,GAAG,CAAC,OAAO,CAAC,MAAM,KAAK,SAAS;gBAC9B,CAAC,CAAC,EAAE;gBACJ,CAAC,CAAC;oBACE,eAAe,EAAE;wBACf,IAAI,EAAE,aAAa;wBACnB,WAAW,EAAE,EAAE,IAAI,EAAE,QAAQ,EAAE,MAAM,EAAE,OAAO,CAAC,MAAM,EAAE;qBACxD;iBACF,CAAC;YACN,GAAG,CAAC,IAAI,CAAC,MAAM,CAAC,MAAM,KAAK,CAAC,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,EAAE,KAAK,EAAE,WAAW,CAAC,IAAI,CAAC,MAAM,CAAC,EAAE,CAAC;SACzE,CAAC;QACF,MAAM,QAAQ,GAAG,MAAM,MAAM,CAC3B,IAAI,CAAC,SAAS,EACd,mBAAmB,EACnB,IAAI,EACJ,OAAO,CAAC,MAAM,CACf,CAAC;QACF,OAAO,QAAQ,CAAC,EAAE;YAChB,CAAC,CAAC,EAAE,CAAC,QAAQ,CAAC;YACd,CAAC,CAAC,GAAG,CAAC,MAAM,mBAAmB,CAAC,QAAQ,CAAC,CAAC,CAAC;IAC/C,CAAC;IAED;;;;;OAKG;IACH,aAAa,CACX,SAAwC,EACxC,iBAAyC,EACzC,OAAwB;QAExB,IAAI,YAAY,GAAG,iBAAiB,CAAC;QACrC,IAAI,YAAY,GAAG,IAAI,CAAC,UAAU,CAAC,SAAS,CAAC,CAAC;QAC9C,IAAI,WAAW,GAAG,CAAC,CAAC;QAEpB,MAAM,OAAO,GAAG,KAAK,EACnB,UAAmD,EACpC,EAAE;YACjB,SAAS,CAAC;gBACR,MAAM,KAAK,GAAG,MAAM,YAAY,CAAC,MAAM,CAAC,IAAI,EAAE,CAAC;gBAC/C,IAAI,CAAC,KAAK,CAAC,IAAI,EAAE,CAAC;oBAChB,UAAU,CAAC,OAAO,CAAC,KAAK,CAAC,KAAK,CAAC,CAAC;oBAChC,OAAO;gBACT,CAAC;gBACD,MAAM,KAAK,GAAG,WAAW,CAAC,YAAY,CAAC,MAAM,CAAC,CAAC;gBAC/C,IAAI,KAAK,CAAC,MAAM,KAAK,CAAC,EAAE,CAAC;oBACvB,UAAU,CAAC,KAAK,EAAE,CAAC;oBACnB,OAAO;gBACT,CAAC;gBACD,WAAW,IAAI,CAAC,CAAC;gBACjB,IAAI,WAAW,GAAG,IAAI,CAAC,cAAc,EAAE,CAAC;oBACtC,MAAM,IAAI,OAAO,CAAC;wBAChB,IAAI,EAAE,QAAQ;wBACd,MAAM,EAAE,0BAA0B,WAAW,0BAA0B;qBACxE,CAAC,CAAC;gBACL,CAAC;gBACD,YAAY,GAAG,MAAM,IAAI,CAAC,gBAAgB,CACxC,YAAY,EACZ,YAAY,CAAC,YAAY,CAAC,IAAI,CAAC,EAAE,CAAC,EAClC,KAAK,EACL,OAAO,CAAC,MAAM,CACf,CAAC;gBACF,MAAM,UAAU,GAAG,MAAM,IAAI,CAAC,KAAK,CAAC,YAAY,EAAE,OAAO,EAAE,IAAI,CAAC,CAAC;gBACjE,IAAI,CAAC,UAAU,CAAC,EAAE;oBAAE,MAAM,IAAI,OAAO,CAAC,UAAU,CAAC,KAAK,CAAC,CAAC;gBACxD,MAAM,QAAQ,GAAG,UAAU,CAAC,KAAK,CAAC,IAAI,CAAC;gBACvC,IAAI,QAAQ,KAAK,IAAI;oBACnB,MAAM,IAAI,OAAO,CAAC;wBAChB,IAAI,EAAE,QAAQ;wBACd,MAAM,EAAE,uBAAuB;qBAChC,CAAC,CAAC;gBACL,YAAY,GAAG,IAAI,CAAC,UAAU,CAAC,QAAQ,CAAC,CAAC;YAC3C,CAAC;QACH,CAAC,CAAC;QAEF,OAAO,IAAI,cAAc,CAAS;YAChC,IAAI,EAAE,CAAC,UAAU,EAAE,EAAE,CAAC,OAAO,CAAC,UAAU,CAAC;YACzC,MAAM,EAAE,CAAC,MAAM,EAAE,EAAE,CAAC,YAAY,CAAC,MAAM,CAAC,MAAM,CAAC,MAAM,CAAC;SACvD,CAAC,CAAC;IACL,CAAC;IAED;;;;;;;;;OASG;IACH,UAAU,CAAC,IAAmC;QAC5C,MAAM,YAAY,GAAa,EAAE,CAAC;QAClC,MAAM,MAAM,GAAG,IAAI,GAAG,EAAyB,CAAC;QAChD,IAAI,YAAY,GAAkB,IAAI,CAAC;QACvC,MAAM,UAAU,GAAG,IAAI,eAAe,CAAiB;YACrD,SAAS,EAAE,CAAC,IAAI,EAAE,UAAU,EAAE,EAAE;gBAC9B,MAAM,KAAK,GAAG,QAAQ,CAAC,SAAS,CAAC,IAAI,CAAC,CAAC,CAAC;gBACxC,IAAI,KAAK,KAAK,IAAI;oBAAE,OAAO;gBAC3B,MAAM,SAAS,GAAG,aAAa,CAAC,KAAK,CAAC,CAAC;gBACvC,IAAI,SAAS,KAAK,IAAI;oBACpB,MAAM,IAAI,OAAO,CAAC,EAAE,IAAI,EAAE,QAAQ,EAAE,MAAM,EAAE,SAAS,EAAE,CAAC,CAAC;gBAE3D,MAAM,MAAM,GAAG,QAAQ,CAAC,OAAO,CAAC,KAAK,CAAC,OAAO,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC;gBACnD,YAAY,GAAG,QAAQ,CAAC,MAAM,EAAE,aAAa,CAAC,IAAI,YAAY,CAAC;gBAC/D,MAAM,KAAK,GAAG,QAAQ,CAAC,KAAK,CAAC,KAAK,CAAC,CAAC;gBACpC,IAAI,KAAK,KAAK,IAAI,IAAI,YAAY,KAAK,IAAI;oBACzC,IAAI,CAAC,OAAO,CAAC,KAAK,EAAE,YAAY,CAAC,CAAC;gBAEpC,MAAM,KAAK,GAAG,QAAQ,CAAC,MAAM,EAAE,KAAK,CAAC,CAAC;gBACtC,kBAAkB,CAAC,KAAK,EAAE,MAAM,CAAC,CAAC;gBAClC,MAAM,IAAI,GAAG,QAAQ,CAAC,KAAK,EAAE,OAAO,CAAC,IAAI,EAAE,CAAC;gBAC5C,IAAI,IAAI,KAAK,EAAE;oBAAE,OAAO;gBACxB,YAAY,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;gBACxB,UAAU,CAAC,OAAO,CAAC,IAAI,CAAC,CAAC;YAC3B,CAAC;SACF,CAAC,CAAC;QACH,MAAM,MAAM,GAAG,IAAI;aAChB,WAAW,CAAC,IAAI,iBAAiB,EAAE,CAAC;aACpC,WAAW,CAAC,WAAW,EAAE,CAAC;aAC1B,WAAW,CAAC,cAAc,EAAE,CAAC;aAC7B,WAAW,CAAC,UAAU,CAAC,CAAC;QAC3B,OAAO,EAAE,MAAM,EAAE,MAAM,CAAC,SAAS,EAAE,EAAE,YAAY,EAAE,MAAM,EAAE,CAAC;IAC9D,CAAC;IAED;;;;;;;OAOG;IACH,KAAK,CAAC,gBAAgB,CACpB,YAAoC,EACpC,QAAgB,EAChB,KAA8B,EAC9B,MAAmB;QAEnB,MAAM,QAAQ,GAAkB;YAC9B,GAAG,YAAY;YACf,EAAE,IAAI,EAAE,WAAW,EAAE,OAAO,EAAE,QAAQ,EAAE,UAAU,EAAE,KAAK,EAAE;SAC5D,CAAC;QACF,KAAK,MAAM,IAAI,IAAI,KAAK,EAAE,CAAC;YACzB,MAAM,IAAI,GAAG,IAAI,CAAC,QAAQ,CAAC,IAAI,CAAC;YAChC,MAAM,IAAI,GAAG,QAAQ,CAAC,IAAI,CAAC,MAAM,EAAE,IAAI,CAAC,CAAC;YACzC,IAAI,IAAI,KAAK,SAAS,EAAE,CAAC;gBACvB,QAAQ,CAAC,IAAI,CAAC;oBACZ,IAAI,EAAE,MAAM;oBACZ,OAAO,EAAE,4BAA4B,IAAI,GAAG;oBAC5C,YAAY,EAAE,IAAI,CAAC,EAAE;iBACtB,CAAC,CAAC;gBACH,SAAS;YACX,CAAC;YACD,MAAM,UAAU,GAAG,MAAM,OAAO,CAAC,IAAI,EAAE,WAAW,CAAC,IAAI,CAAC,EAAE,MAAM,CAAC,CAAC;YAClE,IAAI,CAAC,UAAU,CAAC,EAAE;gBAAE,MAAM,IAAI,OAAO,CAAC,UAAU,CAAC,KAAK,CAAC,CAAC;YACxD,QAAQ,CAAC,IAAI,CAAC;gBACZ,IAAI,EAAE,MAAM;gBACZ,OAAO,EAAE,UAAU,CAAC,KAAK;gBACzB,YAAY,EAAE,IAAI,CAAC,EAAE;aACtB,CAAC,CAAC;QACL,CAAC;QACD,OAAO,QAAQ,CAAC;IAClB,CAAC;IAED;;;;;;OAMG;IACH,OAAO,CACL,KAAqC,EACrC,YAA2B;QAE3B,IAAI,KAAK,KAAK,IAAI,EAAE,CAAC;YACnB,MAAM,YAAY,GAAG,QAAQ,CAAC,KAAK,CAAC,aAAa,CAAC,IAAI,CAAC,CAAC;YACxD,MAAM,YAAY,GAAG,QAAQ,CAAC,KAAK,CAAC,iBAAiB,CAAC,IAAI,CAAC,CAAC;YAC5D,IAAI,CAAC,WAAW;gBACd,QAAQ,CAAC,KAAK,CAAC,YAAY,CAAC,IAAI,YAAY,GAAG,YAAY,CAAC;QAChE,CAAC;QACD,IAAI,IAAI,CAAC,kBAAkB,CAAC,YAAY,CAAC;YAAE,IAAI,CAAC,eAAe,EAAE,CAAC;IACpE,CAAC;IAED;;;;;;;;OAQG;IACH,kBAAkB,CAAC,YAA2B;QAC5C,IAAI,YAAY,KAAK,QAAQ;YAAE,OAAO,IAAI,CAAC;QAC3C,OAAO,CACL,IAAI,CAAC,cAAc,KAAK,SAAS;YACjC,IAAI,CAAC,WAAW,IAAI,IAAI,CAAC,cAAc,CACxC,CAAC;IACJ,CAAC;CACF;AAED,MAAM,WAAW,GAAG,MAAM,CAAC,CAAC,CAA2C,CAAC;AAExE;;;;GAIG;AACH,MAAM,aAAa,GAAG,CACpB,MAAoB,EACpB,OAAuB,EACsB,EAAE,CAC/C,OAAO,CAAC,OAAO,CAAC,EAAE,CAAC,IAAI,WAAW,CAAC,MAAM,EAAE,OAAO,CAAC,CAAC,CAAC,CAAC;AAExD,MAAM,UAAU,kBAAkB,CAAC,MAAoB;IACrD,MAAM,OAAO,GAAiB;QAC5B,IAAI,EAAE,QAAQ;QACd,UAAU,EAAE,CAAC,MAAM,CAAC;QACpB,KAAK,EAAE,IAAI;QACX,YAAY,EAAE,GAAG,EAAE,CAAC,eAAe,CAAC,MAAM,CAAC;QAC3C,OAAO,EAAE,CAAC,OAAO,EAAE,EAAE,CAAC,aAAa,CAAC,MAAM,EAAE,OAAO,CAAC;KACrD,CAAC;IACF,OAAO,cAAc,CAAC,OAAO,CAAC,CAAC;AACjC,CAAC"}
|
package/package.json
CHANGED
|
@@ -1,12 +1,14 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "modelpact-providers",
|
|
3
|
-
"version": "1.
|
|
4
|
-
"description": "Language-model backends written against the modelpact contract — one
|
|
3
|
+
"version": "1.2.0",
|
|
4
|
+
"description": "Language-model backends written against the modelpact contract — a daemon, the OpenAI dialect, the browser's own model, one in the tab, each green on the contract suite",
|
|
5
5
|
"keywords": [
|
|
6
6
|
"llm",
|
|
7
7
|
"modelpact",
|
|
8
8
|
"typescript",
|
|
9
9
|
"ollama",
|
|
10
|
+
"openai",
|
|
11
|
+
"openai-compatible",
|
|
10
12
|
"prompt-api",
|
|
11
13
|
"webgpu",
|
|
12
14
|
"on-device",
|
|
@@ -37,6 +39,9 @@
|
|
|
37
39
|
"files": [
|
|
38
40
|
"dist"
|
|
39
41
|
],
|
|
42
|
+
"workspaces": [
|
|
43
|
+
"demo"
|
|
44
|
+
],
|
|
40
45
|
"sideEffects": false,
|
|
41
46
|
"engines": {
|
|
42
47
|
"node": ">=22"
|
|
@@ -46,11 +51,14 @@
|
|
|
46
51
|
"build": "tsc -p tsconfig.build.json",
|
|
47
52
|
"test": "vitest run",
|
|
48
53
|
"test:watch": "vitest",
|
|
54
|
+
"test:e2e": "playwright test",
|
|
55
|
+
"demo": "npm --prefix demo run dev",
|
|
56
|
+
"demo:check": "npm --prefix demo run typecheck",
|
|
49
57
|
"lint": "eslint .",
|
|
50
58
|
"lint:fix": "eslint . --fix",
|
|
51
59
|
"format": "prettier --write .",
|
|
52
60
|
"format:check": "prettier --check .",
|
|
53
|
-
"typecheck": "tsc --noEmit && tsc -p tsconfig.patched.json",
|
|
61
|
+
"typecheck": "tsc --noEmit && tsc -p tsconfig.patched.json && tsc -p e2e --noEmit",
|
|
54
62
|
"check:surface": "npm run build && tsc -p tsconfig.surface.json",
|
|
55
63
|
"prepublishOnly": "npm run build"
|
|
56
64
|
},
|
|
@@ -60,12 +68,13 @@
|
|
|
60
68
|
},
|
|
61
69
|
"devDependencies": {
|
|
62
70
|
"@eslint/js": "^10.0.1",
|
|
71
|
+
"@playwright/test": "^1.62.1",
|
|
63
72
|
"@mlc-ai/web-llm": "^0.2.84",
|
|
64
73
|
"@types/dom-chromium-ai": "^0.0.17",
|
|
65
74
|
"@types/node": "^26.4.1",
|
|
66
75
|
"eslint": "^10.9.1",
|
|
67
76
|
"eslint-config-prettier": "^10.1.8",
|
|
68
|
-
"modelpact": "^2.2.
|
|
77
|
+
"modelpact": "^2.2.1",
|
|
69
78
|
"patch-package": "^8.0.1",
|
|
70
79
|
"prettier": "^3.9.6",
|
|
71
80
|
"typescript": "~6.0.3",
|