@jxsuite/ai 0.36.5 → 0.37.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +24 -105
- package/package.json +4 -4
- package/src/chat-state.ts +61 -6
- package/src/index.ts +1 -0
- package/src/streaming-client.ts +49 -3
package/README.md
CHANGED
|
@@ -1,25 +1,17 @@
|
|
|
1
1
|
# @jxsuite/ai
|
|
2
2
|
|
|
3
|
-
The provider-agnostic substrate Jx Studio's assistant is built on: a streaming LLM client
|
|
4
|
-
abstraction that normalizes provider SSE into six event shapes, a tool registry that speaks OpenAI
|
|
5
|
-
function-calling, and a reactive chat store. It has no Jx-domain dependencies (just a type-only
|
|
6
|
-
import of `ProblemDetails` from `@jxsuite/protocol`, plus `@vue/reactivity`), and its only platform
|
|
7
|
-
APIs are `fetch`, `TextDecoder` and `AbortSignal`, so the same modules run in the browser and in
|
|
8
|
-
Bun/Node.
|
|
3
|
+
The provider-agnostic substrate Jx Studio's assistant is built on: a streaming LLM client abstraction that normalizes provider SSE into seven event shapes, a tool registry that speaks OpenAI function-calling, and a reactive chat store. It has no Jx-domain dependencies (just a type-only import of `ProblemDetails` from `@jxsuite/protocol`, plus `@vue/reactivity`), and its only platform APIs are `fetch`, `TextDecoder` and `AbortSignal`, so the same modules run in the browser and in Bun/Node.
|
|
9
4
|
|
|
10
|
-
Governing spec: [`specs/ai.md`](../../specs/ai.md) §1-§2 (Status: Partial). User-facing
|
|
11
|
-
documentation for the Studio feature lives at [`docs/studio/ai.md`](../../docs/studio/ai.md).
|
|
5
|
+
Governing spec: [`specs/ai.md`](../../specs/ai.md) §1-§2 (Status: Partial). User-facing documentation for the Studio feature lives at [`docs/studio/ai.md`](../../docs/studio/ai.md).
|
|
12
6
|
|
|
13
7
|
| Entrypoint | Exports |
|
|
14
8
|
| ------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
15
9
|
| `@jxsuite/ai` | Barrel: the three factories, `createToolDefinition`, `createToolRegistry`, `createChatState`, `STREAM_EVENT_TYPES`, and the streaming types |
|
|
16
|
-
| `@jxsuite/ai/streaming-client` | `StreamingClient`, `StreamEvent` and its
|
|
10
|
+
| `@jxsuite/ai/streaming-client` | `StreamingClient`, `StreamEvent` and its seven members, the three factories and their option types, `STREAM_EVENT_TYPES` |
|
|
17
11
|
| `@jxsuite/ai/tools` | `createToolDefinition`, `createToolRegistry`, `toolSuccess`, `toolError`, and their types |
|
|
18
12
|
| `@jxsuite/ai/chat-state` | `createChatState`, `ChatStore`, `Message`, `ToolCallRecord`, `ChatState`, `MessageRole` |
|
|
19
13
|
|
|
20
|
-
`toolSuccess` / `toolError` are **not** re-exported by the barrel. Import them from
|
|
21
|
-
`@jxsuite/ai/tools`. Importing the subpaths instead of the root is also what makes the package
|
|
22
|
-
tree-shakeable.
|
|
14
|
+
`toolSuccess` / `toolError` are **not** re-exported by the barrel. Import them from `@jxsuite/ai/tools`. Importing the subpaths instead of the root is also what makes the package tree-shakeable.
|
|
23
15
|
|
|
24
16
|
## The streaming client
|
|
25
17
|
|
|
@@ -30,13 +22,7 @@ streamChat(messages: object[], tools: object[], systemPrompt: string, signal: Ab
|
|
|
30
22
|
: AsyncGenerator<StreamEvent>;
|
|
31
23
|
```
|
|
32
24
|
|
|
33
|
-
Every implementation yields the same discriminated union, whose type strings are also published as
|
|
34
|
-
`STREAM_EVENT_TYPES`: `delta {content}`, `tool_call_start {id, name}`, `tool_call_delta {id, args}`,
|
|
35
|
-
`tool_call_end {id}`, `done {stopReason}`, `error {message, code?, problem?}`. This union is the
|
|
36
|
-
contract a third-party Studio backend must satisfy for the `ai/chat` route (see
|
|
37
|
-
[`packages/protocol/README.md`](../protocol/README.md)); `@jxsuite/server` emits the same format
|
|
38
|
-
without depending on this package. `StreamEvent` and each member are exported, so an implementer can
|
|
39
|
-
import the type instead of reconstructing it.
|
|
25
|
+
Every implementation yields the same discriminated union, whose type strings are also published as `STREAM_EVENT_TYPES`: `delta {content}`, `tool_call_start {id, name}`, `tool_call_delta {id, args}`, `tool_call_end {id}`, `done {stopReason}`, `error {message, code?, problem?}`. This union is the contract a third-party Studio backend must satisfy for the `ai/chat` route (see [`packages/protocol/README.md`](../protocol/README.md)); `@jxsuite/server` emits the same format without depending on this package. `StreamEvent` and each member are exported, so an implementer can import the type instead of reconstructing it.
|
|
40
26
|
|
|
41
27
|
```ts
|
|
42
28
|
import { createProxyStreamingClient } from "@jxsuite/ai";
|
|
@@ -52,48 +38,19 @@ for await (const event of client.streamChat(messages, tools, systemPrompt, signa
|
|
|
52
38
|
}
|
|
53
39
|
```
|
|
54
40
|
|
|
55
|
-
- **`createOpenAIStreamingClient({ baseUrl, apiKey, model = "gpt-4o", temperature })`** POSTs to
|
|
56
|
-
|
|
57
|
-
|
|
58
|
-
prepended to your messages. A non-empty `tools` array also sets `tool_choice: "auto"` and
|
|
59
|
-
`parallel_tool_calls: true`; an empty one sends no `tools` key. `temperature` is forwarded **only
|
|
60
|
-
when defined**, because reasoning models (GPT-5.x, o-series) reject a custom temperature. That is a
|
|
61
|
-
provider behaviour recorded in the source comment, not covered by a test here. It does
|
|
62
|
-
its own SSE framing and reassembles fragmented tool-call argument JSON keyed by the provider's
|
|
63
|
-
`delta.tool_calls[].index`.
|
|
64
|
-
- **`createAnthropicStreamingClient({ baseUrl, apiKey })`** is a stub. It makes no network call and
|
|
65
|
-
yields exactly one `error` event with `code: "NOT_IMPLEMENTED"`. Only the OpenAI-compatible path
|
|
66
|
-
ships (`specs/ai.md` §2).
|
|
67
|
-
- **`createProxyStreamingClient({ chatUrl, model = "gpt-4o", apiKey, baseUrl })`** POSTs
|
|
68
|
-
`{ messages, tools, systemPrompt, model }` to an endpoint that already speaks the normalized
|
|
69
|
-
format (the dev/desktop server's `/__studio/ai/chat`). It handles no credentials of its own;
|
|
70
|
-
`apiKey` and `baseUrl` are passed through as the `X-Api-Key` and `X-Api-Base-URL` headers only
|
|
71
|
-
when truthy, and the proxy owns the provider key. It is a pass-through, **not** a translator: it
|
|
72
|
-
re-yields whatever parses out of each `data:` line and returns at the first `done` or `error`, so
|
|
73
|
-
frames after `done` are dropped.
|
|
41
|
+
- **`createOpenAIStreamingClient({ baseUrl, apiKey, model = "gpt-4o", temperature })`** POSTs to `${baseUrl}/chat/completions` with `Authorization: Bearer`, `stream: true`, `stream_options: { include_usage: true }`, and `{ role: "system", content: systemPrompt }` prepended to your messages. A non-empty `tools` array also sets `tool_choice: "auto"` and `parallel_tool_calls: true`; an empty one sends no `tools` key. `temperature` is forwarded **only when defined**, because reasoning models (GPT-5.x, o-series) reject a custom temperature. That is a provider behaviour recorded in the source comment, not covered by a test here. It does its own SSE framing and reassembles fragmented tool-call argument JSON keyed by the provider's `delta.tool_calls[].index`.
|
|
42
|
+
- **`createAnthropicStreamingClient({ baseUrl, apiKey })`** is a stub. It makes no network call and yields exactly one `error` event with `code: "NOT_IMPLEMENTED"`. Only the OpenAI-compatible path ships (`specs/ai.md` §2).
|
|
43
|
+
- **`createProxyStreamingClient({ chatUrl, model = "gpt-4o", apiKey, baseUrl })`** POSTs `{ messages, tools, systemPrompt, model }` to an endpoint that already speaks the normalized format (the dev/desktop server's `/__studio/ai/chat`). It handles no credentials of its own; `apiKey` and `baseUrl` are passed through as the `X-Api-Key` and `X-Api-Base-URL` headers only when truthy, and the proxy owns the provider key. It is a pass-through, **not** a translator: it re-yields whatever parses out of each `data:` line and returns at the first `done` or `error`, so frames after `done` are dropped.
|
|
74
44
|
|
|
75
45
|
Three behaviours are easy to get wrong when consuming the stream:
|
|
76
46
|
|
|
77
|
-
1. **Cancellation is a `done`, not an `error`.** An `AbortError` from `fetch` or mid-read becomes
|
|
78
|
-
|
|
79
|
-
|
|
80
|
-
`"length"`.
|
|
81
|
-
2. **The OpenAI client drains pending tool calls on every termination path.** `[DONE]`, a
|
|
82
|
-
`finish_reason` of `stop`, `length` or `tool_calls` (any other value is ignored and the stream
|
|
83
|
-
continues), and a body that simply closes all emit `tool_call_end` for anything still open, then
|
|
84
|
-
exactly one `done`. The proxy client tracks nothing of its own, so a body that closes without a
|
|
85
|
-
`done` frame ends its generator with no `done` at all.
|
|
86
|
-
3. **`error` frames may carry an RFC 9457 `problem` beside the human `message`,** because the
|
|
87
|
-
response has already begun with a 200 and the status can no longer change (a committed standards
|
|
88
|
-
row in `specs/ai.md` §5). The field is optional and no client in this package sets it.
|
|
89
|
-
`@jxsuite/server`'s proxy is what populates it, and `createProxyStreamingClient` passes it
|
|
90
|
-
through. Read `problem.type` for machine dispatch when it is present; keep showing `message`.
|
|
47
|
+
1. **Cancellation is a `done`, not an `error`.** An `AbortError` from `fetch` or mid-read becomes `{ type: "done", stopReason: "cancelled" }`. Treating abort as failure paints a user's Stop button as a crash. The OpenAI client's other stop reasons are `"stop"`, `"tool_calls"` and `"length"`.
|
|
48
|
+
2. **The OpenAI client drains pending tool calls on every termination path.** `[DONE]`, a `finish_reason` of `stop`, `length` or `tool_calls` (any other value is ignored and the stream continues), and a body that simply closes all emit `tool_call_end` for anything still open, then exactly one `done`. The proxy client tracks nothing of its own, so a body that closes without a `done` frame ends its generator with no `done` at all.
|
|
49
|
+
3. **`error` frames may carry an RFC 9457 `problem` beside the human `message`,** because the response has already begun with a 200 and the status can no longer change (a committed standards row in `specs/ai.md` §5). The field is optional and no client in this package sets it. `@jxsuite/server`'s proxy is what populates it, and `createProxyStreamingClient` passes it through. Read `problem.type` for machine dispatch when it is present; keep showing `message`.
|
|
91
50
|
|
|
92
51
|
## The tool registry
|
|
93
52
|
|
|
94
|
-
`createToolRegistry()` returns `{ register, list, listForLLM, validate, execute, getDefinition }`.
|
|
95
|
-
`listForLLM()` emits `{ type: "function", function: { name, description, parameters } }`, ready to
|
|
96
|
-
hand straight to `streamChat`'s `tools` argument.
|
|
53
|
+
`createToolRegistry()` returns `{ register, list, listForLLM, validate, execute, getDefinition }`. `listForLLM()` emits `{ type: "function", function: { name, description, parameters } }`, ready to hand straight to `streamChat`'s `tools` argument.
|
|
97
54
|
|
|
98
55
|
```ts
|
|
99
56
|
import { createToolDefinition, createToolRegistry, toolSuccess } from "@jxsuite/ai/tools";
|
|
@@ -113,65 +70,27 @@ registry.register(
|
|
|
113
70
|
);
|
|
114
71
|
```
|
|
115
72
|
|
|
116
|
-
- **`strict` and `llmStrict` are unrelated despite the names.** `strict` (default `true`) drives the
|
|
117
|
-
|
|
118
|
-
|
|
119
|
-
`additionalProperties: false` and every property in `required`, which Jx's own tools deliberately
|
|
120
|
-
violate, so GPT-5.x rejects such a request outright (again, committed rationale and no test).
|
|
121
|
-
- **`validate()` is a lightweight structural check, not a JSON Schema validator.** It reads only
|
|
122
|
-
`type`, `properties` and `required`: nested schemas, `enum`, `pattern`, `minimum` and every other
|
|
123
|
-
keyword are ignored. Unknown properties are ignored too (the model may send extra); a required key
|
|
124
|
-
that is present but `null`/`undefined` counts as missing; numeric strings coerce, so `"42"`
|
|
125
|
-
validates against `number`. `strict: false`, or a schema without `properties`, skips it entirely.
|
|
126
|
-
- **`execute()` never throws.** An unknown tool, a validation failure, and an exception inside the
|
|
127
|
-
tool all return `{ success: false, error }`, which is what lets an agent loop feed the failure
|
|
128
|
-
back to the model and self-correct. It calls `this.validate(...)`, so it is `this`-bound: do not
|
|
129
|
-
destructure it off the registry.
|
|
73
|
+
- **`strict` and `llmStrict` are unrelated despite the names.** `strict` (default `true`) drives the registry's own argument validation. `llmStrict` (default `false`) is the only one that reaches the wire, adding OpenAI's `strict: true` to the emitted function schema. OpenAI strict mode demands `additionalProperties: false` and every property in `required`, which Jx's own tools deliberately violate, so GPT-5.x rejects such a request outright (again, committed rationale and no test).
|
|
74
|
+
- **`validate()` is a lightweight structural check, not a JSON Schema validator.** It reads only `type`, `properties` and `required`: nested schemas, `enum`, `pattern`, `minimum` and every other keyword are ignored. Unknown properties are ignored too (the model may send extra); a required key that is present but `null`/`undefined` counts as missing; numeric strings coerce, so `"42"` validates against `number`. `strict: false`, or a schema without `properties`, skips it entirely.
|
|
75
|
+
- **`execute()` never throws.** An unknown tool, a validation failure, and an exception inside the tool all return `{ success: false, error }`, which is what lets an agent loop feed the failure back to the model and self-correct. It calls `this.validate(...)`, so it is `this`-bound: do not destructure it off the registry.
|
|
130
76
|
- Re-registering a name `console.warn`s and overwrites.
|
|
131
77
|
|
|
132
|
-
To vary the tool set per turn, wrap rather than mutate. Studio's `createGatedToolRegistry` returns
|
|
133
|
-
a `ToolRegistry` that filters `list()`/`listForLLM()` through per-tool predicates
|
|
134
|
-
(`packages/studio/src/services/gated-registry.ts`).
|
|
78
|
+
To vary the tool set per turn, wrap rather than mutate. Studio's `createGatedToolRegistry` returns a `ToolRegistry` that filters `list()`/`listForLLM()` through per-tool predicates (`packages/studio/src/services/gated-registry.ts`).
|
|
135
79
|
|
|
136
80
|
## The reactive chat store
|
|
137
81
|
|
|
138
|
-
`createChatState(options?: { model?: string })` returns a `@vue/reactivity` `reactive()` store with
|
|
139
|
-
|
|
140
|
-
|
|
141
|
-
`tokenCount` and `contextWarning`. The model falls back to `"gpt-4o"` via `||`, so an empty string
|
|
142
|
-
takes the default too. `status` is `"idle" | "streaming" | "error"`; `toMessagesArray()` serializes
|
|
143
|
-
back to OpenAI wire shape (assistant `tool_calls`, `tool` messages with `tool_call_id`, everything
|
|
144
|
-
else `{ role, content }`).
|
|
145
|
-
|
|
146
|
-
Two ordering rules inside `beginAssistantTurn` are load-bearing, and any reimplementation must keep
|
|
147
|
-
them: `status` is set to `"streaming"` **before** the placeholder message is pushed (the push itself
|
|
148
|
-
triggers effects, and a reader seeing `"idle"` renders the empty placeholder as a finished message),
|
|
149
|
-
and the pushed object is immediately re-read back through the proxy (`store.messages.at(-1)`),
|
|
150
|
-
because mutating the raw object would notify nothing. A regression test asserts that an `effect()`
|
|
151
|
-
reading `messages.at(-1).content` re-runs on every `appendDelta`.
|
|
82
|
+
`createChatState(options?: { model?: string })` returns a `@vue/reactivity` `reactive()` store with its mutators `Object.assign`ed onto the same proxy, so the returned value is both the state and the API. The store holds `messages`, `status`, `streamingContent`, `pendingToolCalls`, `error`, `model`, `tokenCount` and `contextWarning`. The model falls back to `"gpt-4o"` via `||`, so an empty string takes the default too. `status` is `"idle" | "streaming" | "error"`; `toMessagesArray()` serializes back to OpenAI wire shape (assistant `tool_calls`, `tool` messages with `tool_call_id`, everything else `{ role, content }`).
|
|
83
|
+
|
|
84
|
+
Two ordering rules inside `beginAssistantTurn` are load-bearing, and any reimplementation must keep them: `status` is set to `"streaming"` **before** the placeholder message is pushed (the push itself triggers effects, and a reader seeing `"idle"` renders the empty placeholder as a finished message), and the pushed object is immediately re-read back through the proxy (`store.messages.at(-1)`), because mutating the raw object would notify nothing. A regression test asserts that an `effect()` reading `messages.at(-1).content` re-runs on every `appendDelta`.
|
|
152
85
|
|
|
153
86
|
Further surprises:
|
|
154
87
|
|
|
155
|
-
- **`sendMessage(text)` produces two messages**, not one: the user message plus an empty assistant
|
|
156
|
-
|
|
157
|
-
|
|
158
|
-
|
|
159
|
-
- **`sendMessage` / `beginAssistantTurn` are no-ops while streaming**; `appendDelta` /
|
|
160
|
-
`appendToolCallStart` are no-ops while not streaming.
|
|
161
|
-
- **`setError` and `cancelStream` delete the partial streaming message**, because it may hold
|
|
162
|
-
incomplete `tool_calls` that would poison history on the next send. That deletion is why
|
|
163
|
-
`specs/ai.md` §3.2 forbids reporting an exhausted tool-call budget as an error: the error path
|
|
164
|
-
destroys the only account of the edits that did land.
|
|
165
|
-
- **`appendToolCallEnd(id)` is an intentional no-op.** Arguments are already fully accumulated; the
|
|
166
|
-
caller parses the JSON and executes the tool.
|
|
88
|
+
- **`sendMessage(text)` produces two messages**, not one: the user message plus an empty assistant placeholder, and status flips to `"streaming"`. There is no way to append a bare user message with it. `beginAssistantTurn()` exists separately so an agent loop can open the next round after pushing tool results, with no new user message.
|
|
89
|
+
- **`sendMessage` / `beginAssistantTurn` are no-ops while streaming**; `appendDelta` / `appendToolCallStart` are no-ops while not streaming.
|
|
90
|
+
- **`setError` and `cancelStream` delete the partial streaming message**, because it may hold incomplete `tool_calls` that would poison history on the next send. That deletion is why `specs/ai.md` §3.2 forbids reporting an exhausted tool-call budget as an error: the error path destroys the only account of the edits that did land.
|
|
91
|
+
- **`appendToolCallEnd(id)` is an intentional no-op.** Arguments are already fully accumulated; the caller parses the JSON and executes the tool.
|
|
167
92
|
- Message ids come from a module-level counter (`msg_${Date.now()}_${n}`), not a UUID.
|
|
168
93
|
|
|
169
94
|
## Versioning
|
|
170
95
|
|
|
171
|
-
Published to npm as `@jxsuite/ai`, following the monorepo's release train. Like every published
|
|
172
|
-
`@jxsuite` library, it ships TypeScript source: every `exports` subpath resolves to `./src/*.ts`, so
|
|
173
|
-
a consumer must be able to compile TypeScript out of `node_modules`. Within the monorepo,
|
|
174
|
-
`@jxsuite/studio` depends on it via `workspace:^` and is its only consumer. `@vue/reactivity` is
|
|
175
|
-
pinned to an **exact** version, matching `@jxsuite/studio`, `@jxsuite/runtime` and
|
|
176
|
-
`@jxsuite/compiler`: reactive proxies from one copy do not track in effects from another, and the
|
|
177
|
-
failure is silent.
|
|
96
|
+
Published to npm as `@jxsuite/ai`, following the monorepo's release train. Like every published `@jxsuite` library, it ships TypeScript source: every `exports` subpath resolves to `./src/*.ts`, so a consumer must be able to compile TypeScript out of `node_modules`. Within the monorepo, `@jxsuite/studio` depends on it via `workspace:^` and is its only consumer. `@vue/reactivity` is pinned to an **exact** version, matching `@jxsuite/studio`, `@jxsuite/runtime` and `@jxsuite/compiler`: reactive proxies from one copy do not track in effects from another, and the failure is silent.
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@jxsuite/ai",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.37.1",
|
|
4
4
|
"description": "AI infrastructure for Jx Suite — streaming LLM client abstraction, tool registry, and reactive chat state",
|
|
5
5
|
"license": "MIT",
|
|
6
6
|
"repository": {
|
|
@@ -28,10 +28,10 @@
|
|
|
28
28
|
"upgrade": "bun update --latest && bun install"
|
|
29
29
|
},
|
|
30
30
|
"dependencies": {
|
|
31
|
-
"@jxsuite/protocol": "^2.
|
|
32
|
-
"@vue/reactivity": "3.5.
|
|
31
|
+
"@jxsuite/protocol": "^2.3.0",
|
|
32
|
+
"@vue/reactivity": "3.5.42"
|
|
33
33
|
},
|
|
34
34
|
"devDependencies": {
|
|
35
|
-
"@happy-dom/global-registrator": "^20.
|
|
35
|
+
"@happy-dom/global-registrator": "^20.14.5"
|
|
36
36
|
}
|
|
37
37
|
}
|
package/src/chat-state.ts
CHANGED
|
@@ -21,6 +21,13 @@ export interface Message {
|
|
|
21
21
|
id: string;
|
|
22
22
|
role: MessageRole;
|
|
23
23
|
content: string;
|
|
24
|
+
/**
|
|
25
|
+
* The turn's chain-of-thought, when the provider streamed one (`reasoning_content` /
|
|
26
|
+
* `reasoning`). Kept because a thinking model's own history is not optional: DeepSeek's thinking
|
|
27
|
+
* mode REQUIRES every prior turn's `reasoning_content` back on any request that carries `tools`,
|
|
28
|
+
* which is every request the agent loop makes. See {@link toMessagesArray}.
|
|
29
|
+
*/
|
|
30
|
+
reasoningContent?: string;
|
|
24
31
|
toolCalls?: ToolCallRecord[];
|
|
25
32
|
toolCallId?: string;
|
|
26
33
|
timestamp: number;
|
|
@@ -67,6 +74,7 @@ function uid() {
|
|
|
67
74
|
* sendMessage: (text: string) => void;
|
|
68
75
|
* beginAssistantTurn: () => void;
|
|
69
76
|
* appendDelta: (content: string) => void;
|
|
77
|
+
* appendReasoning: (content: string) => void;
|
|
70
78
|
* appendToolCallStart: (id: string, name: string) => void;
|
|
71
79
|
* appendToolCallDelta: (id: string, args: string) => void;
|
|
72
80
|
* appendToolCallEnd: (id: string) => void;
|
|
@@ -183,6 +191,21 @@ export function createChatState(opts: { model?: string } = {}) {
|
|
|
183
191
|
_streamingMessage.content = store.streamingContent;
|
|
184
192
|
}
|
|
185
193
|
|
|
194
|
+
/**
|
|
195
|
+
* Append a reasoning delta to the streaming assistant message.
|
|
196
|
+
*
|
|
197
|
+
* Held on the message rather than in `streamingContent`: it is not part of the answer, but it IS
|
|
198
|
+
* part of the turn the provider will be shown again on the next round.
|
|
199
|
+
*
|
|
200
|
+
* @param {string} content
|
|
201
|
+
*/
|
|
202
|
+
function appendReasoning(content: string) {
|
|
203
|
+
if (store.status !== "streaming" || !_streamingMessage) {
|
|
204
|
+
return;
|
|
205
|
+
}
|
|
206
|
+
_streamingMessage.reasoningContent = (_streamingMessage.reasoningContent ?? "") + content;
|
|
207
|
+
}
|
|
208
|
+
|
|
186
209
|
/**
|
|
187
210
|
* Start tracking a tool call within the stream.
|
|
188
211
|
*
|
|
@@ -358,25 +381,56 @@ export function createChatState(opts: { model?: string } = {}) {
|
|
|
358
381
|
* Convert the current chat state to an array suitable for sending to an LLM API. Includes the
|
|
359
382
|
* system prompt as the first message and all conversation turns.
|
|
360
383
|
*
|
|
384
|
+
* Two things it does NOT do verbatim, both because the array is a transcript for the UI and this
|
|
385
|
+
* is the wire:
|
|
386
|
+
*
|
|
387
|
+
* 1. **An assistant turn carrying neither text nor tool calls is dropped.** The one that always
|
|
388
|
+
* exists is {@link beginAssistantTurn}'s placeholder — the message being generated by the very
|
|
389
|
+
* request this builds, so it is a turn that has not happened yet. It reached providers as a
|
|
390
|
+
* trailing `{"role":"assistant","content":""}`, which most read as an empty prefill and
|
|
391
|
+
* ignore; DeepSeek's thinking mode instead answers 400 `The reasoning_content in the thinking
|
|
392
|
+
* mode must be passed back to the API`, because a thinking-mode assistant turn owes it one.
|
|
393
|
+
* `persistChat` in Studio already skipped the same message for the same reason — it just
|
|
394
|
+
* skipped it on the way to storage, and nothing skipped it on the way to the provider.
|
|
395
|
+
* 2. **`reasoning_content` is replayed when the turn has one.** DeepSeek requires the reasoning of
|
|
396
|
+
* all previous turns back on any request carrying `tools` — which is every request the agent
|
|
397
|
+
* loop makes — and ignores it on requests that carry none, so echoing what the provider itself
|
|
398
|
+
* streamed is safe for providers that never send it (they get no field at all).
|
|
399
|
+
*
|
|
361
400
|
* @returns {object[]}
|
|
362
401
|
*/
|
|
363
402
|
function toMessagesArray() {
|
|
364
403
|
const out = [];
|
|
365
404
|
|
|
366
405
|
for (const msg of store.messages) {
|
|
367
|
-
if (msg.role === "assistant"
|
|
368
|
-
|
|
406
|
+
if (msg.role === "assistant") {
|
|
407
|
+
const calls = msg.toolCalls ?? [];
|
|
408
|
+
if (!msg.content && calls.length === 0) {
|
|
409
|
+
continue;
|
|
410
|
+
}
|
|
411
|
+
const entry: {
|
|
412
|
+
role: string;
|
|
413
|
+
content: string | null;
|
|
414
|
+
reasoning_content?: string;
|
|
415
|
+
tool_calls?: object[];
|
|
416
|
+
} = {
|
|
369
417
|
role: "assistant",
|
|
370
|
-
content: msg.content || null,
|
|
371
|
-
|
|
418
|
+
content: calls.length > 0 ? msg.content || null : msg.content,
|
|
419
|
+
};
|
|
420
|
+
if (msg.reasoningContent) {
|
|
421
|
+
entry.reasoning_content = msg.reasoningContent;
|
|
422
|
+
}
|
|
423
|
+
if (calls.length > 0) {
|
|
424
|
+
entry.tool_calls = calls.map((tc) => ({
|
|
372
425
|
id: tc.id,
|
|
373
426
|
type: "function",
|
|
374
427
|
function: {
|
|
375
428
|
name: tc.name,
|
|
376
429
|
arguments: tc.arguments,
|
|
377
430
|
},
|
|
378
|
-
}))
|
|
379
|
-
}
|
|
431
|
+
}));
|
|
432
|
+
}
|
|
433
|
+
out.push(entry);
|
|
380
434
|
} else if (msg.role === "tool") {
|
|
381
435
|
out.push({
|
|
382
436
|
role: "tool",
|
|
@@ -398,6 +452,7 @@ export function createChatState(opts: { model?: string } = {}) {
|
|
|
398
452
|
sendMessage,
|
|
399
453
|
beginAssistantTurn,
|
|
400
454
|
appendDelta,
|
|
455
|
+
appendReasoning,
|
|
401
456
|
appendToolCallStart,
|
|
402
457
|
appendToolCallDelta,
|
|
403
458
|
appendToolCallEnd,
|
package/src/index.ts
CHANGED
package/src/streaming-client.ts
CHANGED
|
@@ -5,7 +5,7 @@
|
|
|
5
5
|
* and concrete implementations for OpenAI and Anthropic. Designed upfront so switching
|
|
6
6
|
* providers is a new implementation, not a refactor.
|
|
7
7
|
*
|
|
8
|
-
* The union and its
|
|
8
|
+
* The union and its seven members are exported because they are the contract a third-party Studio
|
|
9
9
|
* backend implements for the `ai/chat` route, not an internal detail — see the docs page below.
|
|
10
10
|
*
|
|
11
11
|
* @license MIT
|
|
@@ -17,6 +17,7 @@ import type { ProblemDetails } from "@jxsuite/protocol";
|
|
|
17
17
|
|
|
18
18
|
export type StreamEvent =
|
|
19
19
|
| StreamDeltaEvent
|
|
20
|
+
| StreamReasoningEvent
|
|
20
21
|
| StreamToolCallStartEvent
|
|
21
22
|
| StreamToolCallDeltaEvent
|
|
22
23
|
| StreamToolCallEndEvent
|
|
@@ -28,6 +29,21 @@ export interface StreamDeltaEvent {
|
|
|
28
29
|
content: string;
|
|
29
30
|
}
|
|
30
31
|
|
|
32
|
+
/**
|
|
33
|
+
* A chain-of-thought fragment from a thinking model — OpenAI-compatible providers stream it as
|
|
34
|
+
* `reasoning_content` (DeepSeek, Volcengine) or `reasoning` (OpenRouter) beside `content`.
|
|
35
|
+
*
|
|
36
|
+
* It is a SEPARATE event rather than a `delta` because it is not part of the answer and must not be
|
|
37
|
+
* rendered as one — but it is part of the turn, and DeepSeek's thinking mode requires every prior
|
|
38
|
+
* turn's reasoning back on any request carrying `tools`. A client that drops these frames replays a
|
|
39
|
+
* history the provider rejects with 400 `The reasoning_content in the thinking mode must be passed
|
|
40
|
+
* back to the API`, so a backend implementing this route MUST forward them.
|
|
41
|
+
*/
|
|
42
|
+
export interface StreamReasoningEvent {
|
|
43
|
+
type: "reasoning";
|
|
44
|
+
content: string;
|
|
45
|
+
}
|
|
46
|
+
|
|
31
47
|
export interface StreamToolCallStartEvent {
|
|
32
48
|
type: "tool_call_start";
|
|
33
49
|
id: string;
|
|
@@ -105,7 +121,14 @@ interface OpenAIToolCallDelta {
|
|
|
105
121
|
interface OpenAIStreamChunk {
|
|
106
122
|
choices?: {
|
|
107
123
|
index?: number;
|
|
108
|
-
delta?: {
|
|
124
|
+
delta?: {
|
|
125
|
+
content?: string;
|
|
126
|
+
/** Thinking models' chain-of-thought: DeepSeek/Volcengine spell it this way… */
|
|
127
|
+
reasoning_content?: string;
|
|
128
|
+
/** …and OpenRouter this way. Both carry the same thing. */
|
|
129
|
+
reasoning?: string;
|
|
130
|
+
tool_calls?: OpenAIToolCallDelta[];
|
|
131
|
+
};
|
|
109
132
|
finish_reason?: string | null;
|
|
110
133
|
}[];
|
|
111
134
|
}
|
|
@@ -113,6 +136,14 @@ interface OpenAIStreamChunk {
|
|
|
113
136
|
/** Error response body — the proxy's flat `{ error: "..." }` or OpenAI's `{ error: { message } }`. */
|
|
114
137
|
interface ErrorResponseBody {
|
|
115
138
|
error?: string | { message?: string; code?: string; type?: string };
|
|
139
|
+
/**
|
|
140
|
+
* The proxy's machine code, alongside its human `error` — `cf_reconnect_required` and friends.
|
|
141
|
+
*
|
|
142
|
+
* A backend that knows WHY it refused says so here, and a client can only act on the reason it is
|
|
143
|
+
* given: the whole point of the code is that the Reconnect affordance appears without the user
|
|
144
|
+
* reading a sentence and deciding what it meant.
|
|
145
|
+
*/
|
|
146
|
+
code?: string;
|
|
116
147
|
}
|
|
117
148
|
|
|
118
149
|
// ─── Type definitions ────────────────────────────────────────────────────────
|
|
@@ -125,6 +156,7 @@ interface ErrorResponseBody {
|
|
|
125
156
|
*/
|
|
126
157
|
export const STREAM_EVENT_TYPES = {
|
|
127
158
|
DELTA: "delta",
|
|
159
|
+
REASONING: "reasoning",
|
|
128
160
|
TOOL_CALL_START: "tool_call_start",
|
|
129
161
|
TOOL_CALL_DELTA: "tool_call_delta",
|
|
130
162
|
TOOL_CALL_END: "tool_call_end",
|
|
@@ -314,6 +346,12 @@ export function createOpenAIStreamingClient({
|
|
|
314
346
|
yield { type: "delta", content: delta.content };
|
|
315
347
|
}
|
|
316
348
|
|
|
349
|
+
// Chain-of-thought, under either of the two names providers give it.
|
|
350
|
+
const reasoning = delta.reasoning_content ?? delta.reasoning;
|
|
351
|
+
if (typeof reasoning === "string" && reasoning) {
|
|
352
|
+
yield { type: "reasoning", content: reasoning };
|
|
353
|
+
}
|
|
354
|
+
|
|
317
355
|
// Tool calls
|
|
318
356
|
if (delta.tool_calls) {
|
|
319
357
|
for (const tc of delta.tool_calls) {
|
|
@@ -508,6 +546,11 @@ export function createProxyStreamingClient({
|
|
|
508
546
|
// Try to extract a clean message from JSON error bodies (e.g. the proxy's
|
|
509
547
|
// { error: "..." } shape or OpenAI's { error: { message: "..." } }).
|
|
510
548
|
let cleanMessage = errorBody || response.statusText;
|
|
549
|
+
/* The body's machine code beats the status. `code` was `String(response.status)`
|
|
550
|
+
unconditionally, so a proxy answering 401 `{ code: "cf_reconnect_required" }` reached the
|
|
551
|
+
client as `"401"` — indistinguishable from a bad BYOK key, and the one reading that would
|
|
552
|
+
have put a Reconnect button on screen was thrown away at the only place it arrived. */
|
|
553
|
+
let machineCode = "";
|
|
511
554
|
try {
|
|
512
555
|
const parsed = JSON.parse(errorBody) as ErrorResponseBody;
|
|
513
556
|
if (typeof parsed.error === "string") {
|
|
@@ -515,13 +558,16 @@ export function createProxyStreamingClient({
|
|
|
515
558
|
} else if (parsed.error?.message) {
|
|
516
559
|
cleanMessage = parsed.error.message;
|
|
517
560
|
}
|
|
561
|
+
if (typeof parsed.code === "string" && parsed.code) {
|
|
562
|
+
machineCode = parsed.code;
|
|
563
|
+
}
|
|
518
564
|
} catch {
|
|
519
565
|
/* Not JSON — use the raw body. */
|
|
520
566
|
}
|
|
521
567
|
yield {
|
|
522
568
|
type: "error",
|
|
523
569
|
message: cleanMessage,
|
|
524
|
-
code: String(response.status),
|
|
570
|
+
code: machineCode || String(response.status),
|
|
525
571
|
};
|
|
526
572
|
return;
|
|
527
573
|
}
|