@gullabs/any-llm 0.11.1 → 0.16.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/NOTICE +6 -0
- package/README.md +28 -16
- package/dist/index.cjs +1 -1
- package/dist/index.cjs.map +1 -1
- package/dist/index.js +1 -1
- package/dist/index.js.map +1 -1
- package/package.json +20 -8
- package/skills/any-llm/SKILL.md +109 -18
package/NOTICE
ADDED
package/README.md
CHANGED
|
@@ -2,10 +2,11 @@
|
|
|
2
2
|
|
|
3
3
|
The default, batteries-included any-llm client package.
|
|
4
4
|
|
|
5
|
-
"Batteries-included" means concretely this: `@gullabs/any-llm`
|
|
6
|
-
|
|
7
|
-
|
|
8
|
-
client instead of three
|
|
5
|
+
"Batteries-included" means concretely this: `@gullabs/any-llm` depends on `@gullabs/google` (the
|
|
6
|
+
Gemini adapter) and `@google/genai` (the Gemini SDK), takes `@gullabs/core` (the engine) as an
|
|
7
|
+
exact-version peer dependency that your package manager installs for you, and re-exports the full
|
|
8
|
+
public API of core and google, so a single `pnpm add` gets you a working client instead of three
|
|
9
|
+
separate installs.
|
|
9
10
|
|
|
10
11
|
## Install
|
|
11
12
|
|
|
@@ -13,7 +14,18 @@ client instead of three separate installs.
|
|
|
13
14
|
pnpm add @gullabs/any-llm
|
|
14
15
|
```
|
|
15
16
|
|
|
16
|
-
This installs the
|
|
17
|
+
This installs the Gemini adapter and `@google/genai`, plus the `@gullabs/core` engine at the same
|
|
18
|
+
version. Core is an exact-version peer dependency, which npm 7+ and pnpm install automatically.
|
|
19
|
+
Package managers that do not install peers (pnpm with `autoInstallPeers: false`, yarn, npm with
|
|
20
|
+
`--legacy-peer-deps`) leave it out, and the import then fails with `Cannot find module '@gullabs/core'`.
|
|
21
|
+
There, add it yourself at the same version:
|
|
22
|
+
|
|
23
|
+
```bash
|
|
24
|
+
pnpm add @gullabs/any-llm @gullabs/core # or: yarn add ... / npm i ... --legacy-peer-deps
|
|
25
|
+
```
|
|
26
|
+
|
|
27
|
+
All `@gullabs/*` packages must be at one version; see
|
|
28
|
+
[Versioning](../../RELEASING.md#versioning-one-version-for-every-package).
|
|
17
29
|
|
|
18
30
|
## Usage
|
|
19
31
|
|
|
@@ -48,7 +60,7 @@ const summarize = defineCallSite({
|
|
|
48
60
|
// Auth is required per call — the library never reads environment variables itself.
|
|
49
61
|
const result = await client.runStructured(
|
|
50
62
|
summarize,
|
|
51
|
-
{ text:
|
|
63
|
+
{ text: 'The document to summarize.' },
|
|
52
64
|
{ auth: { apiKey: process.env.GEMINI_API_KEY! } },
|
|
53
65
|
)
|
|
54
66
|
|
|
@@ -81,16 +93,16 @@ This package re-exports the full public API of `@gullabs/core` and `@gullabs/goo
|
|
|
81
93
|
`geminiPricingSource`, `LlmError`, and every other named export from both packages. See their
|
|
82
94
|
READMEs for details:
|
|
83
95
|
|
|
84
|
-
| Export | What it is
|
|
85
|
-
| --------------------------- |
|
|
86
|
-
| `createClient(config)` | Wires ports into a `{ generate, runStructured }` client
|
|
87
|
-
| `composeProviders(plugins)` | Merges `ProviderPlugin`s into `ClientConfig` fields
|
|
88
|
-
| `defineCallSite(opts)` | Defines a typed, reusable prompt template bound to a model
|
|
89
|
-
| `googleProvider(opts?)` | The Gemini `ProviderPlugin` factory, from `@gullabs/google`
|
|
90
|
-
| `geminiAdapter(opts?)` | The Gemini `ProviderAdapter`, from `@gullabs/google`
|
|
91
|
-
| `geminiPricingSource()` | Built-in Gemini pricing snapshot, from `@gullabs/google`
|
|
92
|
-
| `LlmError` | Typed error class — always thrown on call failure
|
|
93
|
-
| `ANY_LLM_VERSION` | This package's version, sourced from `package.json`
|
|
96
|
+
| Export | What it is |
|
|
97
|
+
| --------------------------- | -------------------------------------------------------------------- |
|
|
98
|
+
| `createClient(config)` | Wires ports into a `{ generate, runStructured, countTokens }` client |
|
|
99
|
+
| `composeProviders(plugins)` | Merges `ProviderPlugin`s into `ClientConfig` fields |
|
|
100
|
+
| `defineCallSite(opts)` | Defines a typed, reusable prompt template bound to a model |
|
|
101
|
+
| `googleProvider(opts?)` | The Gemini `ProviderPlugin` factory, from `@gullabs/google` |
|
|
102
|
+
| `geminiAdapter(opts?)` | The Gemini `ProviderAdapter`, from `@gullabs/google` |
|
|
103
|
+
| `geminiPricingSource()` | Built-in Gemini pricing snapshot, from `@gullabs/google` |
|
|
104
|
+
| `LlmError` | Typed error class — always thrown on call failure |
|
|
105
|
+
| `ANY_LLM_VERSION` | This package's version, sourced from `package.json` |
|
|
94
106
|
|
|
95
107
|
Use `@gullabs/core` and `@gullabs/google` directly only when you want modular dependency control.
|
|
96
108
|
|
package/dist/index.cjs
CHANGED
package/dist/index.cjs.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"sources":["../package.json","../src/index.ts"],"names":[],"mappings":";;;;;;;;AAEE,IAAA,OAAA,GAAW,QAAA;;;ACcN,IAAM,eAAA,GAA0B","file":"index.cjs","sourcesContent":["{\n \"name\": \"@gullabs/any-llm\",\n \"version\": \"0.
|
|
1
|
+
{"version":3,"sources":["../package.json","../src/index.ts"],"names":[],"mappings":";;;;;;;;AAEE,IAAA,OAAA,GAAW,QAAA;;;ACcN,IAAM,eAAA,GAA0B","file":"index.cjs","sourcesContent":["{\n \"name\": \"@gullabs/any-llm\",\n \"version\": \"0.16.0\",\n \"description\": \"Batteries-included any-llm client for Gemini: engine, adapter, and Google SDK in one install.\",\n \"type\": \"module\",\n \"license\": \"Apache-2.0\",\n \"repository\": {\n \"type\": \"git\",\n \"url\": \"git+https://github.com/gul-labs/any-llm.git\",\n \"directory\": \"packages/any-llm\"\n },\n \"main\": \"./dist/index.cjs\",\n \"module\": \"./dist/index.js\",\n \"types\": \"./dist/index.d.ts\",\n \"exports\": {\n \".\": {\n \"import\": {\n \"types\": \"./dist/index.d.ts\",\n \"default\": \"./dist/index.js\"\n },\n \"require\": {\n \"types\": \"./dist/index.d.cts\",\n \"default\": \"./dist/index.cjs\"\n }\n },\n \"./package.json\": \"./package.json\"\n },\n \"files\": [\n \"dist\",\n \"skills\",\n \"NOTICE\"\n ],\n \"scripts\": {\n \"build\": \"tsup\"\n },\n \"dependencies\": {\n \"@google/genai\": \"^2.24.0\",\n \"@gullabs/google\": \"workspace:*\"\n },\n \"peerDependencies\": {\n \"@gullabs/core\": \"workspace:*\"\n },\n \"devDependencies\": {\n \"@gullabs/core\": \"workspace:*\"\n },\n \"engines\": {\n \"node\": \">=22.12.0\"\n },\n \"sideEffects\": false,\n \"keywords\": [\n \"llm\",\n \"gemini\",\n \"google-genai\",\n \"ai\",\n \"tokens\",\n \"cost\",\n \"usage\",\n \"observability\",\n \"typescript\"\n ],\n \"publishConfig\": {\n \"access\": \"public\"\n },\n \"homepage\": \"https://github.com/gul-labs/any-llm/tree/main/packages/any-llm#readme\",\n \"bugs\": \"https://github.com/gul-labs/any-llm/issues\"\n}\n","/**\n * @gullabs/any-llm — batteries-included public entrypoint.\n *\n * This package is the default client install path. It re-exports the core\n * engine and Gemini adapter while depending on the Gemini SDK for a one-package\n * setup.\n *\n * @module\n */\n\nexport * from '@gullabs/core'\nexport * from '@gullabs/google'\n\nimport { version } from '../package.json'\n\n/** Library version, sourced from package.json at build time. */\nexport const ANY_LLM_VERSION: string = version\n"]}
|
package/dist/index.js
CHANGED
package/dist/index.js.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"sources":["../package.json","../src/index.ts"],"names":[],"mappings":";;;;;;AAEE,IAAA,OAAA,GAAW,QAAA;;;ACcN,IAAM,eAAA,GAA0B","file":"index.js","sourcesContent":["{\n \"name\": \"@gullabs/any-llm\",\n \"version\": \"0.
|
|
1
|
+
{"version":3,"sources":["../package.json","../src/index.ts"],"names":[],"mappings":";;;;;;AAEE,IAAA,OAAA,GAAW,QAAA;;;ACcN,IAAM,eAAA,GAA0B","file":"index.js","sourcesContent":["{\n \"name\": \"@gullabs/any-llm\",\n \"version\": \"0.16.0\",\n \"description\": \"Batteries-included any-llm client for Gemini: engine, adapter, and Google SDK in one install.\",\n \"type\": \"module\",\n \"license\": \"Apache-2.0\",\n \"repository\": {\n \"type\": \"git\",\n \"url\": \"git+https://github.com/gul-labs/any-llm.git\",\n \"directory\": \"packages/any-llm\"\n },\n \"main\": \"./dist/index.cjs\",\n \"module\": \"./dist/index.js\",\n \"types\": \"./dist/index.d.ts\",\n \"exports\": {\n \".\": {\n \"import\": {\n \"types\": \"./dist/index.d.ts\",\n \"default\": \"./dist/index.js\"\n },\n \"require\": {\n \"types\": \"./dist/index.d.cts\",\n \"default\": \"./dist/index.cjs\"\n }\n },\n \"./package.json\": \"./package.json\"\n },\n \"files\": [\n \"dist\",\n \"skills\",\n \"NOTICE\"\n ],\n \"scripts\": {\n \"build\": \"tsup\"\n },\n \"dependencies\": {\n \"@google/genai\": \"^2.24.0\",\n \"@gullabs/google\": \"workspace:*\"\n },\n \"peerDependencies\": {\n \"@gullabs/core\": \"workspace:*\"\n },\n \"devDependencies\": {\n \"@gullabs/core\": \"workspace:*\"\n },\n \"engines\": {\n \"node\": \">=22.12.0\"\n },\n \"sideEffects\": false,\n \"keywords\": [\n \"llm\",\n \"gemini\",\n \"google-genai\",\n \"ai\",\n \"tokens\",\n \"cost\",\n \"usage\",\n \"observability\",\n \"typescript\"\n ],\n \"publishConfig\": {\n \"access\": \"public\"\n },\n \"homepage\": \"https://github.com/gul-labs/any-llm/tree/main/packages/any-llm#readme\",\n \"bugs\": \"https://github.com/gul-labs/any-llm/issues\"\n}\n","/**\n * @gullabs/any-llm — batteries-included public entrypoint.\n *\n * This package is the default client install path. It re-exports the core\n * engine and Gemini adapter while depending on the Gemini SDK for a one-package\n * setup.\n *\n * @module\n */\n\nexport * from '@gullabs/core'\nexport * from '@gullabs/google'\n\nimport { version } from '../package.json'\n\n/** Library version, sourced from package.json at build time. */\nexport const ANY_LLM_VERSION: string = version\n"]}
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@gullabs/any-llm",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.16.0",
|
|
4
4
|
"description": "Batteries-included any-llm client for Gemini: engine, adapter, and Google SDK in one install.",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"license": "Apache-2.0",
|
|
@@ -14,19 +14,31 @@
|
|
|
14
14
|
"types": "./dist/index.d.ts",
|
|
15
15
|
"exports": {
|
|
16
16
|
".": {
|
|
17
|
-
"
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
|
|
17
|
+
"import": {
|
|
18
|
+
"types": "./dist/index.d.ts",
|
|
19
|
+
"default": "./dist/index.js"
|
|
20
|
+
},
|
|
21
|
+
"require": {
|
|
22
|
+
"types": "./dist/index.d.cts",
|
|
23
|
+
"default": "./dist/index.cjs"
|
|
24
|
+
}
|
|
25
|
+
},
|
|
26
|
+
"./package.json": "./package.json"
|
|
21
27
|
},
|
|
22
28
|
"files": [
|
|
23
29
|
"dist",
|
|
24
|
-
"skills"
|
|
30
|
+
"skills",
|
|
31
|
+
"NOTICE"
|
|
25
32
|
],
|
|
26
33
|
"dependencies": {
|
|
27
34
|
"@google/genai": "^2.24.0",
|
|
28
|
-
"@gullabs/
|
|
29
|
-
|
|
35
|
+
"@gullabs/google": "0.16.0"
|
|
36
|
+
},
|
|
37
|
+
"peerDependencies": {
|
|
38
|
+
"@gullabs/core": "0.16.0"
|
|
39
|
+
},
|
|
40
|
+
"devDependencies": {
|
|
41
|
+
"@gullabs/core": "0.16.0"
|
|
30
42
|
},
|
|
31
43
|
"engines": {
|
|
32
44
|
"node": ">=22.12.0"
|
package/skills/any-llm/SKILL.md
CHANGED
|
@@ -46,7 +46,7 @@ explicit dependency control. Import names are identical either way.
|
|
|
46
46
|
`generate()` and `runStructured()` call requires `opts.auth = { apiKey: string }`
|
|
47
47
|
explicitly. `createClient()` itself takes no credentials.
|
|
48
48
|
|
|
49
|
-
```ts
|
|
49
|
+
```ts no-check
|
|
50
50
|
// WRONG — GenerateOptions.auth is a required field; this will not type-check, and if
|
|
51
51
|
// bypassed with `as any` it throws LlmError({ kind: 'invalid_auth' }) before any I/O.
|
|
52
52
|
const client = createClient({
|
|
@@ -83,6 +83,8 @@ import { createClient, composeProviders, googleProvider } from '@gullabs/any-llm
|
|
|
83
83
|
// (or: composeProviders from '@gullabs/core', googleProvider from '@gullabs/google',
|
|
84
84
|
// if using modular install)
|
|
85
85
|
|
|
86
|
+
declare const myResolvedGeminiKey: string // however your app resolves the key
|
|
87
|
+
|
|
86
88
|
const client = createClient({
|
|
87
89
|
...composeProviders([googleProvider()]),
|
|
88
90
|
})
|
|
@@ -113,11 +115,15 @@ throws `LlmError('bad_request')` when the pair is unregistered or the resolved a
|
|
|
113
115
|
doesn't implement token counting (`ProviderAdapter.countTokens` is optional).
|
|
114
116
|
|
|
115
117
|
```ts
|
|
118
|
+
import { createClient, composeProviders, googleProvider } from '@gullabs/any-llm'
|
|
119
|
+
|
|
120
|
+
const client = createClient({ ...composeProviders([googleProvider()]) })
|
|
121
|
+
declare const myResolvedGeminiKey: string
|
|
122
|
+
|
|
116
123
|
const count = await client.countTokens(
|
|
117
124
|
{
|
|
118
125
|
provider: 'google',
|
|
119
126
|
model: 'gemini-2.5-flash',
|
|
120
|
-
system: 'You are a concise summarizer.',
|
|
121
127
|
messages: [{ role: 'user', parts: [{ kind: 'text', text: 'Hello!' }] }],
|
|
122
128
|
},
|
|
123
129
|
{ auth: { apiKey: myResolvedGeminiKey } },
|
|
@@ -128,9 +134,15 @@ console.log(count.details) // optional per-category breakdown, e.g. { cached: 12
|
|
|
128
134
|
console.log(count.raw) // provider's raw token-count response, verbatim
|
|
129
135
|
```
|
|
130
136
|
|
|
131
|
-
`TokenCountRequest` is deliberately narrower than a generate request
|
|
132
|
-
`output`, no `providerOptions
|
|
133
|
-
`system`, and `
|
|
137
|
+
`TokenCountRequest` is deliberately narrower than a generate request: no `config`, no
|
|
138
|
+
`output`, no `providerOptions`. Token counting takes `provider`, `model`, `messages`, and
|
|
139
|
+
optionally `system` and `tools`. Google counts `messages`, `system` and `tools` (through the
|
|
140
|
+
REST `countTokens`); it does not count a response schema, thinking config or Search, so for a
|
|
141
|
+
generate call that sends those it is a floor. `accuracy` says how far to trust it: Google is
|
|
142
|
+
`'exact'` for the history it counts, and `'estimated'` for a Gemini 3 history with replayed
|
|
143
|
+
function calls (their thought signatures add up to about 110 prompt tokens each, depending on the model, that the count
|
|
144
|
+
cannot include). xAI counts text only (`'lower-bound'`) and rejects `tools` with `bad_request`.
|
|
145
|
+
To budget exactly, read `usage.inputTokens` from a real `generate()` result.
|
|
134
146
|
|
|
135
147
|
## Composing multiple providers — xAI Grok example
|
|
136
148
|
|
|
@@ -143,6 +155,8 @@ import { createClient, composeProviders } from '@gullabs/core'
|
|
|
143
155
|
import { googleProvider } from '@gullabs/google'
|
|
144
156
|
import { xaiProvider } from '@gullabs/xai'
|
|
145
157
|
|
|
158
|
+
declare const myResolvedXaiKey: string
|
|
159
|
+
|
|
146
160
|
const client = createClient({
|
|
147
161
|
...composeProviders([googleProvider(), xaiProvider()]),
|
|
148
162
|
})
|
|
@@ -200,9 +214,18 @@ plain text parts, or any `Part` sub-field this library can't losslessly represen
|
|
|
200
214
|
naming the offending field — nothing is ever silently dropped.
|
|
201
215
|
|
|
202
216
|
```ts
|
|
203
|
-
import {
|
|
217
|
+
import {
|
|
218
|
+
createClient,
|
|
219
|
+
composeProviders,
|
|
220
|
+
googleProvider,
|
|
221
|
+
geminiContentToMessages,
|
|
222
|
+
} from '@gullabs/any-llm'
|
|
204
223
|
import type { Content } from '@google/genai'
|
|
205
224
|
|
|
225
|
+
const client = createClient({ ...composeProviders([googleProvider()]) })
|
|
226
|
+
declare const myResolvedGeminiKey: string
|
|
227
|
+
declare const data: string // base64 image bytes
|
|
228
|
+
|
|
206
229
|
const contents: Content[] = [
|
|
207
230
|
{
|
|
208
231
|
role: 'user',
|
|
@@ -220,7 +243,12 @@ const { system, messages } = geminiContentToMessages({
|
|
|
220
243
|
})
|
|
221
244
|
|
|
222
245
|
const result = await client.generate(
|
|
223
|
-
{
|
|
246
|
+
{
|
|
247
|
+
provider: 'google',
|
|
248
|
+
model: 'gemini-2.5-pro',
|
|
249
|
+
...(system !== undefined ? { system } : {}),
|
|
250
|
+
messages,
|
|
251
|
+
},
|
|
224
252
|
{ auth: { apiKey: myResolvedGeminiKey } },
|
|
225
253
|
)
|
|
226
254
|
```
|
|
@@ -232,17 +260,26 @@ from `contents`.
|
|
|
232
260
|
|
|
233
261
|
Real hosts don't call `createClient()` at call sites — they own a factory module that
|
|
234
262
|
assembles the client once and hand call sites the built client. `@gullabs/testing`'s
|
|
235
|
-
fakes (`makeFakeGemini`, `FakeAdapter`, `RecordingSink`, `FakeClock`, `FakeIds`, ...) are
|
|
263
|
+
fakes (`makeFakeGemini`, `FakeAdapter`, `FakeClient`, `RecordingSink`, `FakeClock`, `FakeIds`, ...) are
|
|
236
264
|
designed to inject through that same host-owned factory unchanged, via injectable
|
|
237
265
|
override parameters with production defaults — not via `vi.mock()`. See
|
|
238
266
|
`packages/testing/README.md` § "Wiring fakes through a host-owned factory" for a
|
|
239
267
|
complete two-file (factory + vitest test) example, including the port-level
|
|
240
|
-
`FakeAdapter` variant for bypassing the Gemini SDK shape entirely.
|
|
268
|
+
`FakeAdapter` variant for bypassing the Gemini SDK shape entirely. Drive time with one
|
|
269
|
+
`FakeClock` passed as both `clock` and `scheduler` (timeouts, deadlines and retry back-off then
|
|
270
|
+
advance with it), build failures with `fakeHttpError` / `fakeProviderError` instead of
|
|
271
|
+
hand-made `{ status }` objects, and build results with `fakeLlmResult`. Host code that takes a
|
|
272
|
+
`Client` can be tested with `FakeClient` (`expectRequest` asserts the request it sent).
|
|
241
273
|
|
|
242
274
|
## `defineCallSite` — reusable prompt templates
|
|
243
275
|
|
|
244
276
|
```ts
|
|
245
277
|
import { defineCallSite } from '@gullabs/core'
|
|
278
|
+
import type { AuthMaterial, Client } from '@gullabs/core'
|
|
279
|
+
|
|
280
|
+
declare const client: Client
|
|
281
|
+
declare const auth: AuthMaterial
|
|
282
|
+
declare const text: string
|
|
246
283
|
|
|
247
284
|
const summarize = defineCallSite({
|
|
248
285
|
id: 'summarize-article', // persisted as callSiteId on every record
|
|
@@ -274,6 +311,10 @@ request is built — zero tokens spent:
|
|
|
274
311
|
|
|
275
312
|
```ts
|
|
276
313
|
import { defineCallSite, LlmError } from '@gullabs/core'
|
|
314
|
+
import type { AuthMaterial, Client } from '@gullabs/core'
|
|
315
|
+
|
|
316
|
+
declare const client: Client
|
|
317
|
+
declare const auth: AuthMaterial
|
|
277
318
|
|
|
278
319
|
const summarize = defineCallSite({
|
|
279
320
|
id: 'summarize-article',
|
|
@@ -301,8 +342,14 @@ valibot, ...) before interpolation runs, so a missing business field surfaces in
|
|
|
301
342
|
schema's vocabulary instead of as a downstream placeholder violation:
|
|
302
343
|
|
|
303
344
|
```ts
|
|
345
|
+
import { defineCallSite } from '@gullabs/core'
|
|
346
|
+
import type { AuthMaterial, Client } from '@gullabs/core'
|
|
304
347
|
import { z } from 'zod'
|
|
305
348
|
|
|
349
|
+
declare const client: Client
|
|
350
|
+
declare const auth: AuthMaterial
|
|
351
|
+
declare const diff: string
|
|
352
|
+
|
|
306
353
|
const reviewCallSite = defineCallSite({
|
|
307
354
|
id: 'code-review',
|
|
308
355
|
provider: 'google',
|
|
@@ -325,6 +372,15 @@ await client.runStructured(
|
|
|
325
372
|
path (callers who render their own prompt strings and never touch `CallSite`):
|
|
326
373
|
|
|
327
374
|
```ts
|
|
375
|
+
import type { AuthMaterial, Client } from '@gullabs/core'
|
|
376
|
+
import { z } from 'zod'
|
|
377
|
+
|
|
378
|
+
declare const client: Client
|
|
379
|
+
declare const auth: AuthMaterial
|
|
380
|
+
declare const renderedPrompt: string
|
|
381
|
+
declare const sourceContext: { article: string }
|
|
382
|
+
const myZodSchema = z.object({ article: z.string() })
|
|
383
|
+
|
|
328
384
|
const result = await client.generate(
|
|
329
385
|
{
|
|
330
386
|
provider: 'google',
|
|
@@ -382,7 +438,9 @@ Important distinctions:
|
|
|
382
438
|
|
|
383
439
|
- `descriptor.configSchema` is the runtime boundary for request config.
|
|
384
440
|
- `descriptor.configJsonSchema` is derived from that same schema for form generation.
|
|
385
|
-
- `request.output.jsonSchema` is
|
|
441
|
+
- `request.output.jsonSchema` is standard JSON Schema that constrains the model. The Google and xAI adapters
|
|
442
|
+
reject a keyword the provider would ignore (and malformed schemas) with `bad_request` before dispatch;
|
|
443
|
+
the library never validates the result.
|
|
386
444
|
- `providerOptions.google` is a typed provider-extension lane, not a caller-wins
|
|
387
445
|
override lane for `serviceTier`, sampling, reasoning, or response schema.
|
|
388
446
|
|
|
@@ -395,6 +453,8 @@ merging. `@gullabs/google` and `@gullabs/xai` are the two reference implementati
|
|
|
395
453
|
(`packages/google/src/types.ts`, `packages/xai/src/types.ts`):
|
|
396
454
|
|
|
397
455
|
```ts
|
|
456
|
+
import type { GoogleProviderOptions } from '@gullabs/google'
|
|
457
|
+
|
|
398
458
|
declare module '@gullabs/core' {
|
|
399
459
|
interface ProviderOptionsMap {
|
|
400
460
|
google?: GoogleProviderOptions
|
|
@@ -430,14 +490,20 @@ the type augmentation only makes the shape visible to the compiler.
|
|
|
430
490
|
## Structured output — auth + validation together
|
|
431
491
|
|
|
432
492
|
`request.output = { jsonSchema }` (or `callSite.jsonSchema`) is forwarded to the
|
|
433
|
-
provider as
|
|
434
|
-
|
|
435
|
-
|
|
493
|
+
provider as standard JSON Schema. The Google and xAI adapters reject, with
|
|
494
|
+
`bad_request` and the path, a keyword the provider would silently ignore (`const`,
|
|
495
|
+
`oneOf`, `allOf`, ...) and malformed schemas before dispatch; the schema constrains the
|
|
496
|
+
model but the library does not enforce it. The engine JSON-parses the response and sets
|
|
497
|
+
`outputParsed`; `result.output` is always `unknown`. **The caller owns shape
|
|
498
|
+
validation** — this library does not validate output shape itself.
|
|
436
499
|
|
|
437
500
|
```ts
|
|
438
501
|
import { createClient, composeProviders, googleProvider } from '@gullabs/any-llm'
|
|
439
502
|
import type { StandardSchemaV1 } from '@gullabs/core'
|
|
440
503
|
|
|
504
|
+
declare const myResolvedGeminiKey: string
|
|
505
|
+
declare const mySchema: StandardSchemaV1
|
|
506
|
+
|
|
441
507
|
const client = createClient({
|
|
442
508
|
...composeProviders([googleProvider()]),
|
|
443
509
|
})
|
|
@@ -494,13 +560,21 @@ discriminant (from `packages/core/src/errors.ts`):
|
|
|
494
560
|
|
|
495
561
|
```ts
|
|
496
562
|
import { LlmError } from '@gullabs/core'
|
|
563
|
+
import type { AuthMaterial, Client, LlmRequest } from '@gullabs/core'
|
|
564
|
+
|
|
565
|
+
declare const client: Client
|
|
566
|
+
declare const request: LlmRequest
|
|
567
|
+
declare const auth: AuthMaterial
|
|
568
|
+
declare function scheduleRetry(afterMs: number | undefined): void
|
|
569
|
+
declare function reportCredentialsError(error: LlmError): void
|
|
497
570
|
|
|
498
571
|
try {
|
|
499
572
|
const result = await client.generate(request, { auth })
|
|
573
|
+
console.log(result.text)
|
|
500
574
|
} catch (e) {
|
|
501
575
|
if (e instanceof LlmError) {
|
|
502
576
|
if (e.retryable) scheduleRetry(e.retryAfterMs)
|
|
503
|
-
else if (e.kind === 'invalid_auth')
|
|
577
|
+
else if (e.kind === 'invalid_auth') reportCredentialsError(e)
|
|
504
578
|
else throw e
|
|
505
579
|
} else {
|
|
506
580
|
throw e // never expected — the engine always throws LlmError
|
|
@@ -530,7 +604,7 @@ library is telling you the config is invalid, not transiently rejected.
|
|
|
530
604
|
|
|
531
605
|
## Reasoning / thinking budgets
|
|
532
606
|
|
|
533
|
-
```ts
|
|
607
|
+
```ts no-check
|
|
534
608
|
config: {
|
|
535
609
|
reasoning: { effort: 'medium', includeThoughts: true }
|
|
536
610
|
}
|
|
@@ -580,6 +654,11 @@ mirrors the selected model's explicit-caching minimum (1024 on Gemini 3.x;
|
|
|
580
654
|
|
|
581
655
|
```ts
|
|
582
656
|
import { GoogleCacheStore } from '@gullabs/google'
|
|
657
|
+
import type { Content, GoogleGenAI } from '@google/genai'
|
|
658
|
+
|
|
659
|
+
declare const myResolvedGeminiKey: string
|
|
660
|
+
declare const genaiClient: GoogleGenAI
|
|
661
|
+
declare const myGenaiContents: Content[]
|
|
583
662
|
|
|
584
663
|
const cacheStore = new GoogleCacheStore({
|
|
585
664
|
auth: { apiKey: myResolvedGeminiKey },
|
|
@@ -588,8 +667,15 @@ const cacheStore = new GoogleCacheStore({
|
|
|
588
667
|
// Receives genai-native Content[]/Content|string — NOT the library's
|
|
589
668
|
// Message[] shape; there is no automatic conversion. Hosts building from
|
|
590
669
|
// Message[] should call client.countTokens separately instead.
|
|
591
|
-
countTokens: async (
|
|
592
|
-
const result = await genaiClient.models.countTokens(
|
|
670
|
+
countTokens: async ({ model, contents, systemInstruction, tools }) => {
|
|
671
|
+
const result = await genaiClient.models.countTokens({
|
|
672
|
+
model,
|
|
673
|
+
contents: contents ?? [],
|
|
674
|
+
config: {
|
|
675
|
+
...(systemInstruction !== undefined ? { systemInstruction } : {}),
|
|
676
|
+
...(tools !== undefined ? { tools } : {}),
|
|
677
|
+
},
|
|
678
|
+
})
|
|
593
679
|
return result.totalTokens ?? 0
|
|
594
680
|
},
|
|
595
681
|
},
|
|
@@ -609,7 +695,12 @@ const handle = await cacheStore.create({
|
|
|
609
695
|
- Pre-send backpressure is a `RateLimiter` port (`ClientConfig.rateLimiter`); default
|
|
610
696
|
is a no-op. `@gullabs/core` ships a dependency-free `inMemoryRateLimiter`, and the
|
|
611
697
|
companion `@gullabs/quota` package provides shared/cross-process quota primitives —
|
|
612
|
-
see that package's README for setup.
|
|
698
|
+
see that package's README for setup. The limiter runs once per attempt (inside each
|
|
699
|
+
retry). Put `providerQuotaMiddleware` inside `retryMiddleware`; `createClient` rejects
|
|
700
|
+
the other order.
|
|
701
|
+
- A middleware cannot change `provider` or `model` (`bad_request`); route and fall back
|
|
702
|
+
in the host by making a new call. Correlate host retries with a shared `externalId`;
|
|
703
|
+
every attempt is its own billed row.
|
|
613
704
|
- Every call computes `result.cost` (micro-USD) via the configured `PricingSource`
|
|
614
705
|
(`geminiPricingSource()`), and, when `sink` is configured on `createClient`, persists
|
|
615
706
|
a full per-attempt record (usage, cost, warnings, error classification) — fail-open,
|