@byte-42/fantastico-sdk 0.0.0-stage → 0.21.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/LICENSE ADDED
@@ -0,0 +1,21 @@
1
+ MIT License
2
+
3
+ Copyright (c) 2026 Byte42 S.r.l.
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
package/README.md CHANGED
@@ -1,3 +1,470 @@
1
- # Temporary Holding Version
2
-
3
- This version is a temporary placeholder for this package. An operational version to replace this has been submitted for review and is awaiting a staged release.
1
+ # @byte-42/fantastico-sdk
2
+
3
+ > The FANTASTICO client: opaque tool orchestration, inference sent straight to the LLM provider,
4
+ > per-tenant budgets. It runs **inside your perimeter** (in your application's own process), not as a
5
+ > separate service — Byte42 never sees the prompt.
6
+
7
+ ## Installation
8
+
9
+ ```bash
10
+ npm install @byte-42/fantastico-sdk
11
+ ```
12
+
13
+ Published on the public npm registry: no `.npmrc`, no token. The package was previously
14
+ `@byte42/fantastico-sdk` on Byte42's private registry (up to 0.20.0); the API is unchanged, only
15
+ the import name differs — the `@byte42` scope on npmjs.org belongs to someone else.
16
+
17
+ Pure ESM package (`"type":"module"`). From a **CommonJS** consumer (e.g. TypeScript compiled with
18
+ `"module":"CommonJS"`) use a dynamic `import()` — never a top-level `require()`. `require()` does
19
+ work thanks to the `"default"` condition in the `exports` map (Node ≥22.12 loads it transparently
20
+ via require(esm)); on Node <22.12, or inside a bundler, you need dynamic `import()` or a CJS/ESM
21
+ dual build.
22
+
23
+ ## Quick start
24
+
25
+ ```typescript
26
+ import { Fantastico } from "@byte-42/fantastico-sdk";
27
+
28
+ const ai = new Fantastico({
29
+ apiKey: "sk-fant-prod-…", // project key: the only credential
30
+ baseUrl: "https://fantastico.cc/api",
31
+ });
32
+
33
+ const response = await ai.chat.completions.create({
34
+ messages: [{ role: "user", content: "What is the deadline for a short-form appeal?" }],
35
+ user: "studio-rossi", // end tenant → spend attributed and capped
36
+ });
37
+
38
+ console.log(response.choices[0].message.content);
39
+ // → Full answer, with statutory citations (MCP resolved internally)
40
+ ```
41
+
42
+ ## How it works
43
+
44
+ The SDK exposes **a single virtual LLM**. You know nothing about MCP, tools or services: you send
45
+ messages and you get answers. The orchestration is opaque:
46
+
47
+ ```
48
+ You → SDK ──► LLM provider (project runtime key, DIRECT call)
49
+ ↓ tool_use?
50
+ SDK runs the tool internally, against the tool's own endpoint
51
+ ↓ result
52
+ SDK calls the LLM provider again
53
+ ↓ stop
54
+ SDK → You (final text only + aggregated usage, with real USD cost)
55
+ ```
56
+
57
+ The FANTASTICO Cloud is involved only to mint the project runtime key (`POST /api/runtime/key`), which
58
+ also hands back one grant per enabled tool, and to reserve and settle tenant budgets
59
+ (`POST /api/runtime/budget`). Neither sits on the prompt's path — and neither does tool execution:
60
+ each grant carries the tool's own endpoint and a token issued for it, so the SDK calls the tool
61
+ directly.
62
+
63
+ ## Per-tenant budgets
64
+
65
+ Pass `user` (the standard OpenAI field) to attribute spend to a tenant. The SDK reserves budget
66
+ from the Cloud **before** the call — stopping with a 402 if the tenant or the project is out of
67
+ funds — and settles the real USD spend on the next round. At the end of a session,
68
+ `ai.flushBudget("studio-rossi")` clears the last outstanding amount.
69
+
70
+ Without `user`, only the project ceiling on the runtime key applies, and the LLM provider itself
71
+ enforces it.
72
+
73
+ ## Accounts
74
+
75
+ An account is the end tenant that spend is attributed to. Project and environment are **never
76
+ passed**: they are the ones belonging to the project key you built the client with.
77
+
78
+ ```typescript
79
+ await ai.accounts.create({ userKey: "studio-rossi", label: "Studio Rossi", spendLimit: 20 });
80
+ await ai.accounts.list();
81
+ await ai.accounts.get("studio-rossi");
82
+ await ai.accounts.update("studio-rossi", { spendLimit: 50 });
83
+ ```
84
+
85
+ ### Account keys — delegating to a perimeter you do not control
86
+
87
+ So far **you** operate the account, from this client, with the project key. That is the usual model, and
88
+ it is fine as long as the calling code runs in your own backend.
89
+
90
+ If the client runs somewhere you cannot reach — a plugin on a customer's server, a device, a
91
+ third-party installation — that project key cannot go there: it is worth the project's entire
92
+ reserve, and anyone tampering with it would burn every other account's credit. For that case there
93
+ is the **account key**: it is worth one account only, and the runtime key it mints is capped to
94
+ *that* account's credit.
95
+
96
+ ```typescript
97
+ const { key } = await ai.accounts.keys.issue("sito-1");
98
+ // ↑ the only time the value exists: at rest the Cloud keeps only its hash.
99
+ // Hand it over and forget it — a lost key is not recovered, it is rotated.
100
+
101
+ await ai.accounts.keys.list(); // which accounts have a credential out there
102
+ await ai.accounts.keys.rotate("sito-1"); // issues a new one; the old one dies immediately
103
+ await ai.accounts.keys.revoke("sito-1"); // the account stays, and becomes operable from here again
104
+ ```
105
+
106
+ **One live key per account**: if one already exists, `issue` answers `409 CONTO_KEY_EXISTS` — use
107
+ `rotate` to replace it. Two live credentials would make "revoke that site's key" ambiguous, and the
108
+ case that would motivate them (a site cloned to staging) is better served by rotating.
109
+
110
+ Issuing a key is **optional**: an account without one is not incomplete, it is an account you
111
+ operate yourself.
112
+
113
+ ## Options
114
+
115
+ ```typescript
116
+ new Fantastico({
117
+ // Required
118
+ apiKey: "sk-fant-prod-…", // project key, or account key `ck-fant-…`
119
+
120
+ // Optional
121
+ baseUrl: "https://fantastico.cc/api", // control plane, including /api
122
+ defaultModel: "anthropic/claude-sonnet-5",
123
+ activitySessionId: "…", // continue an activity session (rare)
124
+ reserveUsd: 0.1, // USD reserved per tenant each round
125
+ maxTurns: 8, // max internal orchestrator turns
126
+ budgetMs: 90_000, // orchestration timeout
127
+ mcpTimeoutMs: 30_000, // per-MCP-call timeout
128
+ retry: { attempts: 2 }, // transient provider errors; `false` disables
129
+ reasoning: { enabled: false }, // default for requests that don't carry their own
130
+ });
131
+ ```
132
+
133
+ ## The credential picks the surface
134
+
135
+ There is no option for this: **the key's prefix says it**, and saying it twice would mean being
136
+ able to say it two different ways.
137
+
138
+ | | `sk-fant-…` project key | `ck-fant-…` account key |
139
+ |---|---|---|
140
+ | Routes | `/api/runtime/*` | `/api/edge/*` |
141
+ | Reach | every account in the project | **one** account, the credential's own |
142
+ | Naming an account | yes (`user`, `userKey`) | **no**: there is no field for it |
143
+ | `ai.accounts.*` | yes | `SURFACE_UNSUPPORTED` |
144
+ | `ai.balance()` | `SURFACE_UNSUPPORTED` (use `accounts.get`) | yes |
145
+
146
+ `ai.surface` tells you which of the two you got. A string that is neither is an error **at
147
+ construction time**, not a 401 on every call.
148
+
149
+ ## The runtime key, when you need to see it
150
+
151
+ The client mints it by itself on the first call and keeps it in memory for its TTL — there is no
152
+ mode in which it does not. When you need to **hand it back** before it expires (every open key
153
+ holds a `maxOpenKeys` slot until it settles):
154
+
155
+ ```typescript
156
+ const key = await ai.runtimeKeys.ensure(); // mint now, if there isn't one already
157
+ console.log(key.recordId, key.limitUsd, key.expiresAt, key.activitySessionId);
158
+ // … use the client normally …
159
+ const live = ai.runtimeKeys.current(); // the one in hand RIGHT NOW: may not be the same one
160
+ if (live) await ai.runtimeKeys.revoke(live.recordId);
161
+ ```
162
+
163
+ Neither returns the secret: it lives inside the `AuthClient` and never crosses the public surface.
164
+ `ai.runtimeKeys.list()` lists the ones still open on the account — the answer to "what did I leave
165
+ behind" after a restart, when memory holds nothing any more.
166
+
167
+ Revoking the key currently in use also clears the client's in-memory copy, so the next call mints a
168
+ fresh one instead of presenting a credential the provider no longer knows.
169
+
170
+ ## Activity session
171
+
172
+ `ai.activitySessionId` is the id that groups this stretch of work's spend across the many runtime
173
+ keys that pass through it. **It is not a credential**: it authorises nothing, and it belongs in
174
+ logs and tickets, because the support query downstream is `WHERE activity_session_id = '…'`. It is
175
+ `null` before the first mint, and it can **change on its own**: the control plane silently replaces
176
+ one it no longer recognises, so re-read it rather than storing it once and for all.
177
+
178
+ ## Model
179
+
180
+ The provider is always OpenRouter: any model available there can be used, not just Claude. Two
181
+ levels:
182
+
183
+ - **Default**, at client construction — `defaultModel` (above). Omitted, it is
184
+ `anthropic/claude-sonnet-5`.
185
+ - **Per-call override** — the `model` field of `create()`/`createRaw()`:
186
+
187
+ ```typescript
188
+ await ai.chat.completions.create({
189
+ messages: [{ role: "user", content: "..." }],
190
+ model: "openai/gpt-4o", // applies to this call only
191
+ });
192
+ ```
193
+
194
+ Omitted, the client's `defaultModel` stands.
195
+
196
+ ## Usage in the response
197
+
198
+ ```typescript
199
+ response.usage = {
200
+ prompt_tokens: 1200, // aggregated across ALL internal turns
201
+ completion_tokens: 450,
202
+ total_tokens: 1650,
203
+ internal_turns: 3, // how many LLM turns it took
204
+ cache_creation_tokens: 0,
205
+ cache_read_tokens: 800,
206
+ reasoning_tokens: 0, // already INSIDE completion_tokens, do not add it (see Reasoning)
207
+ mcp_calls: 2, // how many MCP invocations
208
+ cost_usd: 0.0042, // real cost reported by the LLM provider (not estimated)
209
+ };
210
+ ```
211
+
212
+ ## Reasoning (thinking tokens)
213
+
214
+ Thinking-capable models can reason before answering. On OpenRouter those tokens are **billed as
215
+ output** and **count against the request's `max_tokens`** — so a model can spend the entire output
216
+ budget reasoning and return **empty or truncated content**, with HTTP 200 and usage metered as
217
+ usual.
218
+
219
+ > **Why this is a trap rather than an error.** Structured extraction reads empty content as a valid
220
+ > JSON `[]` — "nothing found": a plausible, metered, apparently successful, and wrong answer. A
221
+ > document anonymizer ships a document with the real names still in it.
222
+
223
+ The field goes to `create()` and `createRaw()`, in OpenRouter's own shape:
224
+
225
+ ```typescript
226
+ await ai.chat.completions.createRaw({
227
+ messages: [...],
228
+ reasoning: { enabled: false }, // or { effort: "low" } | { max_tokens: 2000 }
229
+ });
230
+ ```
231
+
232
+ | Field | What it does |
233
+ |---|---|
234
+ | `enabled` | turns reasoning on (without `effort`, OpenRouter uses "medium") |
235
+ | `effort` | `"none"`/`"minimal"`/`"low"`/`"medium"`/`"high"`/`"xhigh"`/`"max"` — **excludes** `max_tokens` |
236
+ | `max_tokens` | explicit reasoning budget (Anthropic/Gemini style) — **excludes** `effort` |
237
+ | `exclude` | reason, but do not return the reasoning |
238
+
239
+ **The SDK has no global default**: omitted everywhere, the field is not sent at all and the model's
240
+ and OpenRouter's own behaviour applies. On `create()` it applies to **every** turn of the
241
+ orchestration.
242
+
243
+ You can set it once per client, for requests that do not carry their own:
244
+
245
+ ```typescript
246
+ const ai = new Fantastico({ apiKey: "sk-fant-prod-…", reasoning: { enabled: false } });
247
+ ```
248
+
249
+ A per-request `reasoning` **replaces** the client default outright — no deep merge, so no field of
250
+ the default survives inside the request's object.
251
+
252
+ > ⚠️ Some models reason unconditionally. OpenRouter marks those `mandatory` and **rejects** an
253
+ > explicit disable rather than ignoring it, so a client-wide `{ enabled: false }` would make them
254
+ > unreachable through that client. Set it per request, or use a second client, if you mix them.
255
+
256
+ To tell whether it happened, two numbers in the response: `usage.reasoning_tokens` (what the
257
+ thinking cost — it is **inside** `completion_tokens`, so visible output is the difference) and
258
+ `choices[0].finish_reason`, which is `"length"` when the ceiling was hit. Both are available on
259
+ `create()` and on `createRaw()`.
260
+
261
+ And when the budget went entirely to thinking, you do not have to notice it yourself — see
262
+ [Empty completions](#empty-completions-are-errors-not-answers).
263
+
264
+ ## Empty completions are errors, not answers
265
+
266
+ When `finish_reason` is `"length"`, there are no tool calls, and the content is empty or
267
+ whitespace, both `create()` and `createRaw()` throw `EmptyCompletionError` instead of handing back
268
+ an empty string.
269
+
270
+ That combination is a failure wearing a 200: usage is billed, the response looks fine, and an empty
271
+ string parses as a valid `[]` — "nothing found". The error carries what tells you what to do:
272
+
273
+ ```typescript
274
+ import { EmptyCompletionError } from "@byte-42/fantastico-sdk";
275
+
276
+ try {
277
+ const res = await ai.chat.completions.createRaw({ messages });
278
+ } catch (err) {
279
+ if (err instanceof EmptyCompletionError) {
280
+ console.log(err.reasoningTokens, "of", err.completionTokens, "went to thinking");
281
+ console.log(err.maxTokens, err.model, err.message);
282
+ }
283
+ }
284
+ ```
285
+
286
+ `reasoningTokens > 0` means reasoning ate the answer: disable it or raise `max_tokens`.
287
+ `reasoningTokens === 0` means the answer simply did not fit: raise `max_tokens`. On `create()` the
288
+ numbers belong to the **failing turn**, not the whole orchestration.
289
+
290
+ It is never retried and never triggers a re-mint: a new key changes nothing, and retrying would
291
+ spend your money again on the same outcome.
292
+
293
+ **Not flagged**, because a false positive would fail a response you would have used: `"length"`
294
+ with non-empty content (truncated, but there — `finish_reason` already tells you), `"stop"` with
295
+ empty content (a model choosing to say nothing), and empty content with `tool_calls` (the normal
296
+ shape of a tool turn).
297
+
298
+ ## Your own tools: `createRaw()`
299
+
300
+ `create()` is opaque: it resolves only the MCPs enabled on the project, and does not accept tools
301
+ of yours. If you have your own tool-use loop (say an agent with application tools FANTASTICO knows
302
+ nothing about), use `createRaw()` — **a single call**, no orchestration, with `tool_calls` handed
303
+ back to you exactly as the model emits them:
304
+
305
+ ```typescript
306
+ const res = await ai.chat.completions.createRaw({
307
+ messages: [
308
+ { role: "system", content: "You are an assistant working on a case file." },
309
+ { role: "user", content: "Close the matter" },
310
+ ],
311
+ tools: [{ type: "function", function: { name: "closeMatter", description: "…", parameters: {} } }],
312
+ user: "studio-rossi", // same per-tenant budget as create()
313
+ });
314
+
315
+ if (res.choices[0].finish_reason === "tool_calls") {
316
+ // run the tool yourself, then call createRaw() again with the result as a
317
+ // { role: "tool", tool_call_id, content } message
318
+ }
319
+ ```
320
+
321
+ Runtime key and budget reservation are **the same** as `create()` — the only difference is that you
322
+ drive the turn, calling `createRaw()` at each step of your own loop.
323
+
324
+ To use FANTASTICO marketplace tools **as well** in the same loop (statutory search, for instance),
325
+ add them explicitly and route execution through `ai.tools`:
326
+
327
+ ```typescript
328
+ const fantasticoTools = await ai.tools.list(); // the project's tool definitions
329
+ const res = await ai.chat.completions.createRaw({
330
+ messages, user,
331
+ tools: [...myTools, ...fantasticoTools],
332
+ });
333
+ // in the loop: your tool_call → you run it; a FANTASTICO tool_call → ai.tools.execute(name, args)
334
+ ```
335
+
336
+ ### Prompt caching (`cache_control`)
337
+
338
+ `createRaw()` supports multi-part content with `cache_control: { type: "ephemeral" }` (Anthropic's
339
+ format — see [`ChatContentPart`](src/types.ts)). **Verified empirically**: without pinning routing
340
+ to Anthropic directly, the LLM provider may route to a sub-provider (e.g. Amazon Bedrock) that
341
+ ignores `cache_control` without error — you silently pay full price. That is why `createRaw()`
342
+ **pins** `provider: {order:["Anthropic"]}` automatically when it detects `cache_control` in the
343
+ messages (override with `pinProvider: false`/`true` if you need something else).
344
+
345
+ ```typescript
346
+ const res = await ai.chat.completions.createRaw({
347
+ messages: [
348
+ { role: "system", content: [{ type: "text", text: largeContext, cache_control: { type: "ephemeral" } }] },
349
+ { role: "user", content: "Summarise" },
350
+ ],
351
+ });
352
+ // The next turn with the SAME system block reads from cache (~90% cheaper on cached tokens —
353
+ // measured: $0.0229 → $0.0019 on a ~9k-token block).
354
+ ```
355
+
356
+ ## Speech (text-to-speech)
357
+
358
+ `ai.audio.speech.create()` is a drop-in for the OpenAI SDK's `audio.speech.create()`. It runs on
359
+ the **same account and the same runtime key** as every other call: per-tenant budget reservation
360
+ (`user`), re-mint on a rejected key, retries on transient errors. The spend lands on the key's
361
+ usage at the provider exactly like a chat turn does, so it is metered and settled on the account
362
+ without the client declaring anything.
363
+
364
+ ```typescript
365
+ const res = await ai.audio.speech.create({
366
+ model: "mistralai/voxtral-mini-tts-2603",
367
+ input: "Good morning, this is a generated voice.",
368
+ voice: "en_paul_neutral",
369
+ response_format: "mp3", // see below before setting it
370
+ user: "studio-rossi", // same per-tenant budget as create()
371
+ });
372
+ fs.writeFileSync("voice.mp3", res.audio); // Uint8Array
373
+ res.contentType; // "audio/mpeg" — or "audio/pcm;rate=24000;channels=1"
374
+ res.generationId; // the provider's generation id, for your logs
375
+ ```
376
+
377
+ **Pick the voice (and the price) from the catalogue.** TTS models are **not** in
378
+ `ai.models.list()`; they have their own list, with the voices each model accepts:
379
+
380
+ ```typescript
381
+ const models = await ai.audio.speech.models();
382
+ // [{ id, name, pricing: { prompt, completion }, supported_voices: ["Kore", "Puck", …], … }]
383
+ ```
384
+
385
+ How to read `pricing`: when `completion` is `"0"`, `prompt` is USD **per character** of input text
386
+ (nearly every model — verified: a 70-character sentence cost exactly 70 × `prompt`). When both are
387
+ non-zero the model bills per token, text in and audio out (Gemini TTS). When `prompt` is `"0"` it
388
+ bills per second of generated audio (Seed Audio). Some models declare no voices: they take the
389
+ voice from `input_references` instead.
390
+
391
+ **`response_format` is model-dependent and the catalogue does not say which one a model serves.**
392
+ HeyGen and Gemini TTS return only `pcm` (asking for `mp3` is a 400 that says so); Mistral and
393
+ ElevenLabs serve `mp3`. Left out, the field is **not sent** and the provider default (`pcm`)
394
+ applies. Always read `contentType` on the result rather than assuming.
395
+
396
+ There is no `usage.cost_usd` on the result, on purpose: the provider does not return it with the
397
+ audio, and the figure that matters is the one the ledger measures from the key — it shows up in the
398
+ account statement, grouped under the same activity session as the rest of the work.
399
+
400
+ ## OpenAI SDK compatibility
401
+
402
+ The interface is a drop-in for the OpenAI SDK:
403
+
404
+ ```typescript
405
+ // Before (OpenAI)
406
+ import OpenAI from "openai";
407
+ const ai = new OpenAI({ apiKey: "..." });
408
+
409
+ // After (FANTASTICO) — same interface
410
+ import { Fantastico } from "@byte-42/fantastico-sdk";
411
+ const ai = new Fantastico({ apiKey: "sk-fant-prod-..." });
412
+
413
+ // Your code does not change
414
+ const res = await ai.chat.completions.create({ messages: [...] });
415
+ ```
416
+
417
+ ## Errors
418
+
419
+ ```typescript
420
+ import { FantasticoError } from "@byte-42/fantastico-sdk";
421
+
422
+ try {
423
+ await ai.chat.completions.create({ messages: [...] });
424
+ } catch (err) {
425
+ if (err instanceof FantasticoError) {
426
+ console.log(err.code); // "ALLOWANCE_EXHAUSTED" | "PROVIDER_CONTEXT_LENGTH_EXCEEDED" | …
427
+ console.log(err.status); // 402, 400, 500, …
428
+ }
429
+ }
430
+ ```
431
+
432
+ On the account routes the code arrives **untouched from the control plane** instead of being
433
+ flattened into a generic one — it is the only thing that tells you what to do:
434
+
435
+ | `code` | | What to do |
436
+ |---|---|---|
437
+ | `CONTO_KEY_EXISTS` | 409 | the account already has a key → `rotate` instead of `issue` |
438
+ | `ALLOWANCE_EXHAUSTED` | 402 | the account is out of credit → top that account up |
439
+ | `PROJECT_FUNDS_EXHAUSTED` | 402 | the project reserve is empty → everything has stopped |
440
+ | `SCOPE_FORBIDDEN` | 403 | this project key lacks the powers for the operation |
441
+ | `ILLEGAL_MOVEMENT` | 422 | no such live account in this project → create it before issuing a key |
442
+
443
+ ```typescript
444
+ if (err.code === "CONTO_KEY_EXISTS") {
445
+ await ai.accounts.keys.rotate(userKey);
446
+ }
447
+ ```
448
+
449
+ Errors coming from the **LLM provider** carry a `PROVIDER_` prefix and the provider's own status,
450
+ so you can tell who spoke — 403, for one, is used both by the control plane (unverified account,
451
+ disabled account) and by the provider (moderation), and the remedies are opposite:
452
+
453
+ | `code` | | What to do |
454
+ |---|---|---|
455
+ | `PROVIDER_CONTEXT_LENGTH_EXCEEDED` | 400 | the conversation no longer fits → trim history |
456
+ | `PROVIDER_CONTENT_POLICY_VIOLATION` | 403 | the prompt was flagged → have the user rephrase |
457
+ | `PROVIDER_PERMISSION_DENIED` | 403 | no access to that model or a guardrail blocked it |
458
+ | `LLM_PROVIDER_ERROR` | 502 | two attempts failed — this is **not** a verdict about your funds |
459
+ | `EMPTY_COMPLETION` | 502 | ceiling hit with no content → see [Empty completions](#empty-completions-are-errors-not-answers) |
460
+ | `INTERNAL_ERROR` | 500 | an error the provider did not classify |
461
+
462
+ 429 and 503 are retried automatically (twice, exponential with jitter, honouring `Retry-After`); a
463
+ 502 or 504 is not, because the generation may have happened and the response been lost.
464
+
465
+ If you inject your own `LlmClient`, throw the exported `ProviderError` to take part in the automatic
466
+ remint on `401`/`402`; a plain `Error` is treated as an error and nothing more.
467
+
468
+ ## Licence
469
+
470
+ MIT — Byte42 S.r.l. See [LICENSE](LICENSE).
@@ -0,0 +1,72 @@
1
+ /**
2
+ * Cosa sta facendo un turno mentre lo fa. `create()`/`resume()` risolvono l'intero turno in una
3
+ * chiamata: senza questi eventi chi aspetta vede solo "sto ragionando" per tutta la durata, anche
4
+ * quando il modello sta leggendo dieci documenti. Con `onEvent` il chiamante li riceve man mano
5
+ * che accadono, e `writeActivityStream` li porta fino al browser, dove il widget
6
+ * (`@byte42/genioai-chat-widget`, `readActivityStream`) li mostra sopra al composer.
7
+ *
8
+ * Il formato di trasporto è NDJSON, una riga per oggetto: `{"event":…}` per ogni evento, poi
9
+ * esattamente una `{"result":…}` (l'esito di create/resume) oppure una `{"error":…}`. Il widget
10
+ * ne tiene una copia propria (non importa questo pacchetto, si builda isolato): le due forme
11
+ * vanno cambiate insieme.
12
+ */
13
+ export type ChatActivityEvent =
14
+ /** Il modello sta per essere chiamato (un turno del loop). */
15
+ {
16
+ type: "thinking";
17
+ turn: number;
18
+ }
19
+ /** Un tool comincia a girare. `label` viene da `LocalToolDefinition.activityLabel`, se c'è. */
20
+ | {
21
+ type: "tool_start";
22
+ id: string;
23
+ name: string;
24
+ label?: string;
25
+ }
26
+ /** Il tool è finito; `ok: false` quando il suo esito è un `{ error }`. */
27
+ | {
28
+ type: "tool_end";
29
+ id: string;
30
+ name: string;
31
+ ok: boolean;
32
+ durationMs: number;
33
+ };
34
+ export type ChatActivityListener = (event: ChatActivityEvent) => void;
35
+ /** Opzioni per turno di `create()`/`resume()` che non sono parte della richiesta al modello. */
36
+ export interface ChatActivityOptions {
37
+ /**
38
+ * Chiamato per ogni evento del turno, in ordine. Un'eccezione qui dentro è ignorata: chi
39
+ * guarda il turno non deve poterlo far fallire.
40
+ */
41
+ onEvent?: ChatActivityListener;
42
+ /**
43
+ * Tool locali **per questa chiamata**, in aggiunta a quelli del client (con lo stesso nome vince
44
+ * questo). Dalla 0.19.0.
45
+ *
46
+ * Servono quando un client è condiviso fra parti diverse di un'applicazione — in Fantastico
47
+ * Studio, fra agenti diversi dello stesso conto — e ognuna deve offrire al modello solo le
48
+ * proprie funzioni: registrarle sul client (`addTool`) le darebbe a tutti. Per `resume()` vanno
49
+ * ripassati gli stessi: lo stato di una pausa non porta funzioni, solo nomi.
50
+ */
51
+ localTools?: import("./local-tools.js").RegisteredTool[];
52
+ }
53
+ /** Il `Content-Type` della risposta scritta da `writeActivityStream`. */
54
+ export declare const ACTIVITY_STREAM_CONTENT_TYPE = "application/x-ndjson; charset=utf-8";
55
+ /**
56
+ * Esegue un turno inoltrando i suoi eventi come righe NDJSON a `write`, poi l'esito. Per un
57
+ * endpoint HTTP dell'host: si imposta `ACTIVITY_STREAM_CONTENT_TYPE`, si passa la `write` della
58
+ * risposta e si chiude la risposta quando la promessa si risolve.
59
+ *
60
+ * ```ts
61
+ * res.setHeader("Content-Type", ACTIVITY_STREAM_CONTENT_TYPE);
62
+ * await writeActivityStream((line) => res.write(line), (onEvent) =>
63
+ * ai.chat.completions.create({ messages }, { onEvent }));
64
+ * res.end();
65
+ * ```
66
+ *
67
+ * Un errore del turno diventa una riga `{"error":{message,code}}` (lo stato HTTP è già partito
68
+ * con la prima riga, non lo si può più cambiare) e la promessa si risolve con `undefined`.
69
+ */
70
+ export declare function writeActivityStream<T>(write: (line: string) => void, run: (onEvent: ChatActivityListener) => Promise<T>): Promise<T | undefined>;
71
+ /** Un tool è andato male se il suo esito (la stringa che torna al modello) è un `{ error }`. */
72
+ export declare function toolResultOk(result: string): boolean;
@@ -0,0 +1,22 @@
1
+ export const ACTIVITY_STREAM_CONTENT_TYPE = "application/x-ndjson; charset=utf-8";
2
+ export async function writeActivityStream(write, run) {
3
+ try {
4
+ const result = await run((event) => write(`${JSON.stringify({ event })}\n`));
5
+ write(`${JSON.stringify({ result })}\n`);
6
+ return result;
7
+ }
8
+ catch (err) {
9
+ const e = err;
10
+ write(`${JSON.stringify({ error: { message: e?.message ?? "Errore", code: e?.code } })}\n`);
11
+ return undefined;
12
+ }
13
+ }
14
+ export function toolResultOk(result) {
15
+ try {
16
+ const parsed = JSON.parse(result);
17
+ return !(parsed && typeof parsed === "object" && !Array.isArray(parsed) && "error" in parsed);
18
+ }
19
+ catch {
20
+ return true;
21
+ }
22
+ }