llm-stream-assemble 1.0.0 → 1.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +211 -167
- package/dist/adapters/gemini.cjs +374 -0
- package/dist/adapters/gemini.cjs.map +1 -0
- package/dist/adapters/gemini.d.cts +9 -0
- package/dist/adapters/gemini.d.ts +9 -0
- package/dist/adapters/gemini.js +372 -0
- package/dist/adapters/gemini.js.map +1 -0
- package/dist/adapters/openai-chat.cjs +3 -0
- package/dist/adapters/openai-chat.cjs.map +1 -1
- package/dist/adapters/openai-chat.js +3 -0
- package/dist/adapters/openai-chat.js.map +1 -1
- package/dist/adapters/openai-compatible.cjs +18 -3
- package/dist/adapters/openai-compatible.cjs.map +1 -1
- package/dist/adapters/openai-compatible.d.cts +1 -1
- package/dist/adapters/openai-compatible.d.ts +1 -1
- package/dist/adapters/openai-compatible.js +18 -3
- package/dist/adapters/openai-compatible.js.map +1 -1
- package/dist/index.cjs +325 -3
- package/dist/index.cjs.map +1 -1
- package/dist/index.d.cts +1 -0
- package/dist/index.d.ts +1 -0
- package/dist/index.js +325 -4
- package/dist/index.js.map +1 -1
- package/package.json +19 -3
package/README.md
CHANGED
|
@@ -1,139 +1,81 @@
|
|
|
1
1
|
# llm-stream-assemble
|
|
2
2
|
|
|
3
|
-

|
|
4
4
|

|
|
5
5
|

|
|
6
|
-

|
|
7
7
|
[](https://github.com/01laky/llm-stream-assemble/actions/workflows/ci.yml)
|
|
8
|
-

|
|
9
9
|
|
|
10
|
-
|
|
10
|
+
**One typed event model for every LLM stream** — text, tool calls, reasoning, JSON, usage, refusals, errors, and non-streaming responses.
|
|
11
11
|
|
|
12
|
-
|
|
12
|
+
> A zero-dependency TypeScript layer for assembling **OpenAI**, **Anthropic**, **Google Gemini**, and **OpenAI-compatible** LLM streams into unified events — so you can stop hand-rolling provider parsers and keep one clean, typed event model across chat UIs, agents, proxies, and backends.
|
|
13
13
|
|
|
14
|
-
|
|
14
|
+
**Status:** Stable `1.2.0`. Five built-in adapters, twelve OpenAI-compatible host presets (including **Azure OpenAI**), transforms, replay helpers, and examples are production-ready. Pin semver ranges as usual and review [CHANGELOG.md](./CHANGELOG.md) before major upgrades.
|
|
15
15
|
|
|
16
|
-
|
|
16
|
+
---
|
|
17
|
+
|
|
18
|
+
## Contents
|
|
19
|
+
|
|
20
|
+
- [Why use this](#why-use-this)
|
|
21
|
+
- [Architecture](#architecture)
|
|
22
|
+
- [Providers at a glance](#providers-at-a-glance)
|
|
23
|
+
- [Install](#install)
|
|
24
|
+
- [Quickstart](#quickstart)
|
|
25
|
+
- [Documentation](#documentation)
|
|
26
|
+
- [Usage guides](#usage-guides)
|
|
27
|
+
- [Transforms & replay](#transforms--replay)
|
|
28
|
+
- [Examples & proxy safety](#examples--proxy-safety)
|
|
29
|
+
- [Non-goals](#non-goals)
|
|
30
|
+
- [Development](#development)
|
|
31
|
+
|
|
32
|
+
---
|
|
33
|
+
|
|
34
|
+
## Why use this
|
|
35
|
+
|
|
36
|
+
- **Zero runtime dependencies** — thin adapters + core assembly, no provider SDKs.
|
|
37
|
+
- **Stream and non-stream parity** — same `StreamEvent` union from SSE chunks or JSON bodies.
|
|
38
|
+
- **Provider presets, not forks** — Groq, Azure, Perplexity, xAI, and others reuse one compatible parser with dialect options.
|
|
39
|
+
- **Proxy-ready transforms** — `toSSE({ sanitizeErrors: true })`, `tapEvents`, `collectStream`, fixture replay.
|
|
40
|
+
|
|
41
|
+
---
|
|
42
|
+
|
|
43
|
+
## Architecture
|
|
17
44
|
|
|
18
45
|
Raw provider bytes enter through a **thin adapter**, get assembled into **typed events**, and leave through the same transform layer whether you stream live, replay fixtures, or proxy to a browser.
|
|
19
46
|
|
|
20
|
-
|
|
21
|
-
flowchart TB
|
|
22
|
-
subgraph PROVIDERS["LLM providers"]
|
|
23
|
-
direction LR
|
|
24
|
-
PC["OpenAI Chat"]
|
|
25
|
-
PA["Anthropic Messages"]
|
|
26
|
-
PO["OpenAI-compatible hosts"]
|
|
27
|
-
PR["OpenAI Responses"]
|
|
28
|
-
end
|
|
29
|
-
|
|
30
|
-
subgraph ADAPTERS["Adapters · parseChunk / parseResponse"]
|
|
31
|
-
direction LR
|
|
32
|
-
AC["openaiChatAdapter"]
|
|
33
|
-
AA["anthropicAdapter"]
|
|
34
|
-
AO["openaiCompatibleAdapter"]
|
|
35
|
-
AR["openaiResponsesAdapter"]
|
|
36
|
-
end
|
|
37
|
-
|
|
38
|
-
subgraph CORE["Core assembly · provider-agnostic"]
|
|
39
|
-
direction TB
|
|
40
|
-
STREAM(["ReadableStream / SSE"])
|
|
41
|
-
JSON(["JSON response body"])
|
|
42
|
-
PARSE["parseSSE · parsePartialJSON"]
|
|
43
|
-
RAW["RawChunk[]"]
|
|
44
|
-
ASM["EventAssembler"]
|
|
45
|
-
EVENTS(["StreamEvent stream"])
|
|
46
|
-
STREAM --> PARSE --> RAW --> ASM --> EVENTS
|
|
47
|
-
JSON --> RAW
|
|
48
|
-
end
|
|
49
|
-
|
|
50
|
-
subgraph TRANSFORMS["Transforms & helpers"]
|
|
51
|
-
direction LR
|
|
52
|
-
COL["collectStream"]
|
|
53
|
-
TAP["tapEvents"]
|
|
54
|
-
SSEOUT["toSSE"]
|
|
55
|
-
REPLAY["assembleFromFile"]
|
|
56
|
-
end
|
|
57
|
-
|
|
58
|
-
subgraph APPS["Your application"]
|
|
59
|
-
direction LR
|
|
60
|
-
UI["Chat UI"]
|
|
61
|
-
AGENT["Agents & tool routing"]
|
|
62
|
-
PROXY["Backend proxy"]
|
|
63
|
-
OBS["Logs & metrics"]
|
|
64
|
-
end
|
|
65
|
-
|
|
66
|
-
PC --> AC
|
|
67
|
-
PA --> AA
|
|
68
|
-
PO --> AO
|
|
69
|
-
PR --> AR
|
|
70
|
-
PARSE --> AC
|
|
71
|
-
PARSE --> AA
|
|
72
|
-
PARSE --> AO
|
|
73
|
-
PARSE --> AR
|
|
74
|
-
AC --> RAW
|
|
75
|
-
AA --> RAW
|
|
76
|
-
AO --> RAW
|
|
77
|
-
AR --> RAW
|
|
78
|
-
JSON --> AC
|
|
79
|
-
JSON --> AA
|
|
80
|
-
JSON --> AO
|
|
81
|
-
JSON --> AR
|
|
82
|
-
EVENTS --> COL
|
|
83
|
-
EVENTS --> TAP
|
|
84
|
-
EVENTS --> SSEOUT
|
|
85
|
-
EVENTS --> REPLAY
|
|
86
|
-
COL --> UI
|
|
87
|
-
COL --> AGENT
|
|
88
|
-
SSEOUT --> PROXY
|
|
89
|
-
TAP --> OBS
|
|
90
|
-
|
|
91
|
-
classDef provider fill:#0f172a,stroke:#38bdf8,stroke-width:2px,color:#e0f2fe
|
|
92
|
-
classDef adapter fill:#1e1b4b,stroke:#818cf8,stroke-width:2px,color:#ede9fe
|
|
93
|
-
classDef core fill:#042f2e,stroke:#2dd4bf,stroke-width:2px,color:#ccfbf1
|
|
94
|
-
classDef io fill:#134e4a,stroke:#5eead4,stroke-width:2px,color:#f0fdfa
|
|
95
|
-
classDef transform fill:#431407,stroke:#fb923c,stroke-width:2px,color:#ffedd5
|
|
96
|
-
classDef app fill:#172554,stroke:#60a5fa,stroke-width:2px,color:#dbeafe
|
|
97
|
-
|
|
98
|
-
class PC,PA,PO,PR provider
|
|
99
|
-
class AC,AA,AO,AR adapter
|
|
100
|
-
class PARSE,RAW,ASM core
|
|
101
|
-
class STREAM,JSON,EVENTS io
|
|
102
|
-
class COL,TAP,SSEOUT,REPLAY transform
|
|
103
|
-
class UI,AGENT,PROXY,OBS app
|
|
104
|
-
```
|
|
47
|
+

|
|
105
48
|
|
|
106
|
-
|
|
107
|
-
|
|
108
|
-
|
|
109
|
-
|
|
110
|
-
|
|
111
|
-
|
|
112
|
-
|
|
113
|
-
|
|
114
|
-
|
|
115
|
-
reasoning.delta
|
|
116
|
-
reasoning.done
|
|
117
|
-
Tools
|
|
118
|
-
tool_call.start
|
|
119
|
-
tool_call.args.delta
|
|
120
|
-
tool_call.done
|
|
121
|
-
Structured
|
|
122
|
-
json.delta
|
|
123
|
-
json.done
|
|
124
|
-
Control
|
|
125
|
-
message.start
|
|
126
|
-
metadata
|
|
127
|
-
usage
|
|
128
|
-
finish
|
|
129
|
-
error
|
|
130
|
-
Safety
|
|
131
|
-
refusal.delta
|
|
132
|
-
refusal.done
|
|
133
|
-
```
|
|
49
|
+
### Built-in adapters
|
|
50
|
+
|
|
51
|
+

|
|
52
|
+
|
|
53
|
+
### Unified event model
|
|
54
|
+
|
|
55
|
+
Every adapter maps provider-specific fragments into the same **`StreamEvent`** union:
|
|
56
|
+
|
|
57
|
+

|
|
134
58
|
|
|
135
59
|
**Design constraints:** adapters never accumulate cross-chunk state beyond id/index reconciliation; assembly, buffering, and `.done` emission live in core. No HTTP client, no tool execution, no UI — just the stream layer.
|
|
136
60
|
|
|
61
|
+
Diagram sources: [`docs/img/`](./docs/img/) (Mermaid `.mmd` + committed SVG). Regenerate with `pnpm diagrams:build`.
|
|
62
|
+
|
|
63
|
+
---
|
|
64
|
+
|
|
65
|
+
## Providers at a glance
|
|
66
|
+
|
|
67
|
+
| Adapter | Provider / API | Import |
|
|
68
|
+
| --------------------------------------- | ----------------------------------------------------------------------------------------------------------------------- | ------------------------------------------- |
|
|
69
|
+
| `openaiChatAdapter()` | OpenAI Chat Completions | `llm-stream-assemble` |
|
|
70
|
+
| `openaiCompatibleAdapter({ provider })` | Groq, DeepSeek, Mistral, Ollama, LM Studio, Together, Fireworks, OpenRouter, Perplexity, xAI, **Azure OpenAI**, generic | `llm-stream-assemble` |
|
|
71
|
+
| `anthropicAdapter()` | Anthropic Messages | `llm-stream-assemble` |
|
|
72
|
+
| `openaiResponsesAdapter()` | OpenAI Responses API | `llm-stream-assemble` |
|
|
73
|
+
| `geminiAdapter()` | Google AI Gemini | `llm-stream-assemble` or `/adapters/gemini` |
|
|
74
|
+
|
|
75
|
+
Full feature flags and quirks: [compatibility matrix](./docs/compatibility.md).
|
|
76
|
+
|
|
77
|
+
---
|
|
78
|
+
|
|
137
79
|
## Install
|
|
138
80
|
|
|
139
81
|
```bash
|
|
@@ -141,20 +83,38 @@ pnpm add llm-stream-assemble
|
|
|
141
83
|
# or npm install llm-stream-assemble
|
|
142
84
|
```
|
|
143
85
|
|
|
144
|
-
|
|
86
|
+
**Requirements:** Node.js 18+
|
|
87
|
+
|
|
88
|
+
---
|
|
145
89
|
|
|
146
|
-
|
|
90
|
+
## Quickstart
|
|
91
|
+
|
|
92
|
+
```ts
|
|
93
|
+
import { assembleStream, openaiChatAdapter } from "llm-stream-assemble";
|
|
94
|
+
|
|
95
|
+
for await (const event of assembleStream(response.body!, openaiChatAdapter())) {
|
|
96
|
+
if (event.type === "text.delta") process.stdout.write(event.text);
|
|
97
|
+
}
|
|
98
|
+
```
|
|
99
|
+
|
|
100
|
+
---
|
|
147
101
|
|
|
148
102
|
## Documentation
|
|
149
103
|
|
|
150
|
-
- [Product & technical proposal](./docs/proposal.md)
|
|
151
|
-
- [Post-1.0 provider roadmap (proposal)](./docs/post-1.0-provider-roadmap.md)
|
|
152
104
|
- [Provider compatibility matrix](./docs/compatibility.md)
|
|
153
105
|
- [Adapter author guide](./docs/adapter-guide.md)
|
|
106
|
+
- [Architecture diagrams](./docs/img/README.md)
|
|
107
|
+
- [Live smoke checklist (maintainers)](./docs/live-smoke.md)
|
|
108
|
+
- [Post-1.0 provider roadmap](./docs/post-1.0-provider-roadmap.md)
|
|
109
|
+
- [Product & technical proposal](./docs/proposal.md)
|
|
110
|
+
|
|
111
|
+
---
|
|
112
|
+
|
|
113
|
+
## Usage guides
|
|
154
114
|
|
|
155
|
-
|
|
115
|
+
### Core Usage
|
|
156
116
|
|
|
157
|
-
The core pipeline works with any adapter that emits `RawChunk[]`, including the built-in OpenAI Chat, OpenAI-compatible, Anthropic Messages,
|
|
117
|
+
The core pipeline works with any adapter that emits `RawChunk[]`, including the built-in OpenAI Chat, OpenAI-compatible, Anthropic Messages, OpenAI Responses, and Google Gemini adapters:
|
|
158
118
|
|
|
159
119
|
```ts
|
|
160
120
|
import { assembleFromPayloads, type StreamAdapter } from "llm-stream-assemble";
|
|
@@ -173,17 +133,7 @@ for await (const event of assembleFromPayloads(payloads, adapter)) {
|
|
|
173
133
|
|
|
174
134
|
Assembly buffers completed text, reasoning, JSON, and tool-call arguments so it can emit final `.done` events. Use `maxBufferBytes` to cap those buffers for untrusted or unusually large streams.
|
|
175
135
|
|
|
176
|
-
|
|
177
|
-
|
|
178
|
-
```ts
|
|
179
|
-
import { assembleStream, openaiChatAdapter } from "llm-stream-assemble";
|
|
180
|
-
|
|
181
|
-
for await (const event of assembleStream(response.body!, openaiChatAdapter())) {
|
|
182
|
-
if (event.type === "text.delta") process.stdout.write(event.text);
|
|
183
|
-
}
|
|
184
|
-
```
|
|
185
|
-
|
|
186
|
-
## OpenAI Chat Usage
|
|
136
|
+
### OpenAI Chat Usage
|
|
187
137
|
|
|
188
138
|
`openaiChatAdapter()` parses OpenAI Chat Completions payloads. Create one adapter instance per request/stream because it keeps minimal state for metadata and tool-call indexes.
|
|
189
139
|
|
|
@@ -211,7 +161,7 @@ for await (const event of assembleStream(response.body!, openaiChatAdapter())) {
|
|
|
211
161
|
|
|
212
162
|
Streaming usage requires `stream_options: { include_usage: true }` on the OpenAI request. JSON mode content is exposed by OpenAI as normal content deltas, so use `openaiChatAdapter({ jsonMode: true })` when you want content mapped to `json.*` events.
|
|
213
163
|
|
|
214
|
-
|
|
164
|
+
### OpenAI-Compatible Usage
|
|
215
165
|
|
|
216
166
|
`openaiCompatibleAdapter()` supports OpenAI-shaped Chat Completions APIs with best-effort provider presets. Create one adapter instance per request/stream.
|
|
217
167
|
|
|
@@ -229,15 +179,22 @@ for await (const event of assembleStream(response.body!, adapter)) {
|
|
|
229
179
|
|
|
230
180
|
Provider presets:
|
|
231
181
|
|
|
232
|
-
| Preset | Intended hosts | Notes
|
|
233
|
-
| ------------ | ----------------------------- |
|
|
234
|
-
| `generic` | Any OpenAI-shaped API | Loose defaults, best first try
|
|
235
|
-
| `openrouter` | OpenRouter | Mostly OpenAI-shaped; provider-specific metadata may appear
|
|
236
|
-
| `groq` | Groq OpenAI-compatible API | OpenAI-like; usage can vary by endpoint/model
|
|
237
|
-
| `
|
|
238
|
-
| `
|
|
239
|
-
| `
|
|
240
|
-
| `
|
|
182
|
+
| Preset | Intended hosts | Notes |
|
|
183
|
+
| ------------ | ----------------------------- | ------------------------------------------------------------------------------------------- |
|
|
184
|
+
| `generic` | Any OpenAI-shaped API | Loose defaults, best first try |
|
|
185
|
+
| `openrouter` | OpenRouter | Mostly OpenAI-shaped; provider-specific metadata may appear |
|
|
186
|
+
| `groq` | Groq OpenAI-compatible API | OpenAI-like; usage can vary by endpoint/model |
|
|
187
|
+
| `deepseek` | DeepSeek API | Maps `reasoning_content` to reasoning events on R1-style models |
|
|
188
|
+
| `mistral` | Mistral API | OpenAI-like; parallel tool calls supported |
|
|
189
|
+
| `ollama` | Ollama `/v1/chat/completions` | Local host, metadata may be sparse |
|
|
190
|
+
| `lmstudio` | LM Studio local server | Local host, metadata/usage may be sparse |
|
|
191
|
+
| `together` | Together AI | OpenAI-like; `reasoning` / `reasoning_delta` aliases |
|
|
192
|
+
| `fireworks` | Fireworks AI | OpenAI-like, usage/details may vary |
|
|
193
|
+
| `perplexity` | Perplexity API | Search-grounded answers; citations in `metadata.raw` |
|
|
194
|
+
| `xai` | xAI Grok API | OpenAI-compatible; `reasoning_content` mapped when present |
|
|
195
|
+
| `azure` | Azure OpenAI Chat Completions | Stricter preset; deployment URL + `api-key` auth; content filter metadata in `metadata.raw` |
|
|
196
|
+
|
|
197
|
+
Base URL examples: Groq `https://api.groq.com/openai/v1`, DeepSeek `https://api.deepseek.com`, Mistral `https://api.mistral.ai/v1`, Ollama `http://localhost:11434/v1`, LM Studio `http://localhost:1234/v1`, Together `https://api.together.xyz/v1`, Fireworks `https://api.fireworks.ai/inference/v1`, OpenRouter `https://openrouter.ai/api/v1`, Perplexity `https://api.perplexity.ai`, xAI `https://api.x.ai/v1`, Azure OpenAI `https://{resource}.openai.azure.com/openai/deployments/{deployment}/chat/completions?api-version={version}`.
|
|
241
198
|
|
|
242
199
|
Strict vs loose configuration:
|
|
243
200
|
|
|
@@ -262,7 +219,44 @@ Known limitations:
|
|
|
262
219
|
- Multi-choice terminal behavior is limited by the current core single terminal finish event.
|
|
263
220
|
- Missing tool ids are tolerated because core can synthesize stable ids by index.
|
|
264
221
|
|
|
265
|
-
|
|
222
|
+
### Azure OpenAI Usage
|
|
223
|
+
|
|
224
|
+
Azure OpenAI Chat Completions uses a deployment-scoped URL and **`api-key`** authentication instead of Bearer tokens. Use the **`azure`** preset — not `generic` — for stricter parsing aligned with OpenAI Chat semantics (`allowMissingMetadata: false`, `looseErrorShape: false`).
|
|
225
|
+
|
|
226
|
+
```ts
|
|
227
|
+
import { assembleStream, openaiCompatibleAdapter } from "llm-stream-assemble";
|
|
228
|
+
|
|
229
|
+
const resource = process.env.AZURE_OPENAI_RESOURCE!;
|
|
230
|
+
const deployment = process.env.AZURE_OPENAI_DEPLOYMENT!;
|
|
231
|
+
const apiVersion = process.env.AZURE_OPENAI_API_VERSION ?? "2024-10-21";
|
|
232
|
+
const url = `https://${resource}.openai.azure.com/openai/deployments/${deployment}/chat/completions?api-version=${apiVersion}`;
|
|
233
|
+
|
|
234
|
+
const response = await fetch(url, {
|
|
235
|
+
method: "POST",
|
|
236
|
+
headers: {
|
|
237
|
+
"api-key": process.env.AZURE_OPENAI_API_KEY!,
|
|
238
|
+
"Content-Type": "application/json",
|
|
239
|
+
},
|
|
240
|
+
body: JSON.stringify({
|
|
241
|
+
messages: [{ role: "user", content: "Hello" }],
|
|
242
|
+
stream: true,
|
|
243
|
+
stream_options: { include_usage: true },
|
|
244
|
+
}),
|
|
245
|
+
});
|
|
246
|
+
|
|
247
|
+
for await (const event of assembleStream(
|
|
248
|
+
response.body!,
|
|
249
|
+
openaiCompatibleAdapter({ provider: "azure" }),
|
|
250
|
+
)) {
|
|
251
|
+
if (event.type === "text.delta") process.stdout.write(event.text);
|
|
252
|
+
}
|
|
253
|
+
```
|
|
254
|
+
|
|
255
|
+
Use `openaiCompatibleAdapter({ provider: "azure", jsonMode: true })` when structured JSON output should map to `json.*` events. Content-filter blocks surface as `refusal.*` events with `finish_reason: content_filter`; filter result fields remain in `metadata.raw` for auditing. If an API gateway strips metadata from chunks, soften strict parsing server-side only with `allowMissingMetadata: true`.
|
|
256
|
+
|
|
257
|
+
See `examples/node-fetch/azure-openai.ts` for a URL builder helper and `examples/proxy-safety/README.md` for server-side proxy notes.
|
|
258
|
+
|
|
259
|
+
### Anthropic Messages Usage
|
|
266
260
|
|
|
267
261
|
`anthropicAdapter()` parses Anthropic Messages streaming events and non-streaming responses. Create one adapter instance per request/stream.
|
|
268
262
|
|
|
@@ -276,7 +270,7 @@ for await (const event of assembleStream(response.body!, anthropicAdapter())) {
|
|
|
276
270
|
|
|
277
271
|
Anthropic tool calls are emitted from `tool_use` content blocks. Fine-grained tool input streaming is supported through `input_json_delta`; partial input may be invalid JSON until the block ends, and core handles those partial previews best-effort. Thinking blocks map to `reasoning.*` events with `variant: "detail"`.
|
|
278
272
|
|
|
279
|
-
|
|
273
|
+
### OpenAI Responses Usage
|
|
280
274
|
|
|
281
275
|
`openaiResponsesAdapter()` parses OpenAI Responses API streaming events and non-streaming response objects. It focuses on output text and function call argument streams; Realtime, audio, and multimodal binary output are out of scope.
|
|
282
276
|
|
|
@@ -290,7 +284,44 @@ for await (const event of assembleStream(response.body!, openaiResponsesAdapter(
|
|
|
290
284
|
|
|
291
285
|
Use `openaiResponsesAdapter({ jsonMode: true })` to map output text to `json.*` events. Reasoning support is best-effort for string summary/detail fields. Create a new adapter instance per stream.
|
|
292
286
|
|
|
293
|
-
|
|
287
|
+
### Gemini Usage
|
|
288
|
+
|
|
289
|
+
`geminiAdapter()` parses Google AI Gemini `GenerateContentResponse` payloads from `streamGenerateContent?alt=sse` and non-streaming `generateContent`. Create one adapter instance per request/stream.
|
|
290
|
+
|
|
291
|
+
```ts
|
|
292
|
+
import { assembleStream, geminiAdapter } from "llm-stream-assemble";
|
|
293
|
+
|
|
294
|
+
const model = "gemini-2.5-flash";
|
|
295
|
+
const apiKey = process.env.GOOGLE_API_KEY!;
|
|
296
|
+
const url = `https://generativelanguage.googleapis.com/v1beta/models/${model}:streamGenerateContent?alt=sse&key=${encodeURIComponent(apiKey)}`;
|
|
297
|
+
|
|
298
|
+
const response = await fetch(url, {
|
|
299
|
+
method: "POST",
|
|
300
|
+
headers: { "Content-Type": "application/json" },
|
|
301
|
+
body: JSON.stringify({
|
|
302
|
+
contents: [{ role: "user", parts: [{ text: "Hello" }] }],
|
|
303
|
+
}),
|
|
304
|
+
});
|
|
305
|
+
|
|
306
|
+
for await (const event of assembleStream(response.body!, geminiAdapter())) {
|
|
307
|
+
if (event.type === "text.delta") process.stdout.write(event.text);
|
|
308
|
+
if (event.type === "tool_call.done") console.log(event.name, event.args);
|
|
309
|
+
}
|
|
310
|
+
```
|
|
311
|
+
|
|
312
|
+
Use `geminiAdapter({ jsonMode: true })` when structured JSON output should map to `json.*` instead of `text.*`. Thinking models may emit `thought` parts mapped to `reasoning.*` (best-effort). Gemini does not expose OpenAI-style `refusal.*` events — blocked prompts use `promptFeedback` or safety finish reasons instead.
|
|
313
|
+
|
|
314
|
+
Subpath import: `import { geminiAdapter } from "llm-stream-assemble/adapters/gemini"`.
|
|
315
|
+
|
|
316
|
+
Vertex AI and the Interactions API are out of scope for this adapter; see [compatibility matrix](./docs/compatibility.md).
|
|
317
|
+
|
|
318
|
+
---
|
|
319
|
+
|
|
320
|
+
## Transforms & replay
|
|
321
|
+
|
|
322
|
+

|
|
323
|
+
|
|
324
|
+
### Collecting a Stream
|
|
294
325
|
|
|
295
326
|
`collectStream()` materializes a full event stream into text, reasoning, refusals, JSON, tool calls, latest usage, and finish reason. It buffers full output in memory and aggregates multi-choice text in event order; it is not a per-choice collector and does not currently collect metadata.
|
|
296
327
|
|
|
@@ -301,7 +332,7 @@ const result = await collectStream(events);
|
|
|
301
332
|
console.log(result.text, result.toolCalls, result.finishReason);
|
|
302
333
|
```
|
|
303
334
|
|
|
304
|
-
|
|
335
|
+
### Tapping Events
|
|
305
336
|
|
|
306
337
|
`tapEvents()` lets you observe events for logging or metrics without changing the stream.
|
|
307
338
|
|
|
@@ -313,7 +344,7 @@ for await (const event of tapEvents(events, (event) => console.debug(event.type)
|
|
|
313
344
|
}
|
|
314
345
|
```
|
|
315
346
|
|
|
316
|
-
|
|
347
|
+
### Forwarding Unified SSE
|
|
317
348
|
|
|
318
349
|
`toSSE()` serializes unified `StreamEvent` objects as `data: <json>` SSE messages. It does not currently emit named SSE `event:` fields, and it emits unified event JSON rather than raw provider SSE.
|
|
319
350
|
|
|
@@ -327,7 +358,7 @@ return new Response(toSSE(events, { sanitizeErrors: true }), {
|
|
|
327
358
|
|
|
328
359
|
Use `sanitizeErrors: true` when forwarding events to browsers so raw provider internals are not exposed.
|
|
329
360
|
|
|
330
|
-
|
|
361
|
+
### Replaying Fixtures
|
|
331
362
|
|
|
332
363
|
`assembleFromFile()` is a Node/dev replay helper for local `.sse` and `.json` fixtures. It uses `node:fs/promises`, so avoid it in browser bundles; a dedicated browser/edge entry point can be added later if needed.
|
|
333
364
|
|
|
@@ -342,14 +373,21 @@ for await (const event of assembleFromFile(
|
|
|
342
373
|
}
|
|
343
374
|
```
|
|
344
375
|
|
|
345
|
-
|
|
376
|
+
---
|
|
377
|
+
|
|
378
|
+
## Examples & proxy safety
|
|
346
379
|
|
|
347
|
-
|
|
348
|
-
|
|
349
|
-
|
|
350
|
-
|
|
351
|
-
|
|
352
|
-
|
|
380
|
+
| Example | Description |
|
|
381
|
+
| ---------------------------------------------------------------------------------------- | --------------------------------------- |
|
|
382
|
+
| [`examples/node-fetch/openai-chat.ts`](./examples/node-fetch/openai-chat.ts) | OpenAI Chat Completions streaming |
|
|
383
|
+
| [`examples/node-fetch/openai-compatible.ts`](./examples/node-fetch/openai-compatible.ts) | OpenAI-compatible presets |
|
|
384
|
+
| [`examples/node-fetch/azure-openai.ts`](./examples/node-fetch/azure-openai.ts) | Azure OpenAI deployment URL + `api-key` |
|
|
385
|
+
| [`examples/node-fetch/perplexity.ts`](./examples/node-fetch/perplexity.ts) | Perplexity streaming |
|
|
386
|
+
| [`examples/node-fetch/xai.ts`](./examples/node-fetch/xai.ts) | xAI Grok streaming |
|
|
387
|
+
| [`examples/node-fetch/anthropic.ts`](./examples/node-fetch/anthropic.ts) | Anthropic Messages |
|
|
388
|
+
| [`examples/node-fetch/gemini.ts`](./examples/node-fetch/gemini.ts) | Google Gemini SSE |
|
|
389
|
+
| [`examples/node-fetch/replay-fixture.ts`](./examples/node-fetch/replay-fixture.ts) | Local fixture replay |
|
|
390
|
+
| [`examples/proxy-safety/`](./examples/proxy-safety/) | Proxy + browser client patterns |
|
|
353
391
|
|
|
354
392
|
Proxy safety:
|
|
355
393
|
|
|
@@ -358,6 +396,8 @@ Proxy safety:
|
|
|
358
396
|
- Never forward raw provider errors or upstream non-OK response bodies to browsers.
|
|
359
397
|
- CORS headers are application-specific and intentionally omitted from the Web-standard example.
|
|
360
398
|
|
|
399
|
+
---
|
|
400
|
+
|
|
361
401
|
## Non-goals
|
|
362
402
|
|
|
363
403
|
- No HTTP client, auth, retries, or provider SDK wrapper.
|
|
@@ -365,6 +405,8 @@ Proxy safety:
|
|
|
365
405
|
- No UI framework, React hooks, or browser components.
|
|
366
406
|
- No multimodal binary/audio/video parsing.
|
|
367
407
|
|
|
408
|
+
---
|
|
409
|
+
|
|
368
410
|
## Development
|
|
369
411
|
|
|
370
412
|
```bash
|
|
@@ -372,14 +414,16 @@ pnpm install
|
|
|
372
414
|
pnpm verify
|
|
373
415
|
```
|
|
374
416
|
|
|
375
|
-
|
|
417
|
+
| Command | Description |
|
|
418
|
+
| --------------------- | --------------------------------------------------- |
|
|
419
|
+
| `pnpm verify` | lint + typecheck + test + build |
|
|
420
|
+
| `pnpm verify:deps` | fail if runtime dependencies are added |
|
|
421
|
+
| `pnpm release:prep` | pre-tag checks (version, CHANGELOG, dist, npm pack) |
|
|
422
|
+
| `pnpm diagrams:build` | regenerate README SVGs from Mermaid sources |
|
|
423
|
+
| `pnpm test` | Vitest smoke tests |
|
|
424
|
+
| `pnpm build` | tsup → ESM + CJS + declarations |
|
|
376
425
|
|
|
377
|
-
|
|
378
|
-
| ------------------ | -------------------------------------- |
|
|
379
|
-
| `pnpm verify` | lint + typecheck + test + build |
|
|
380
|
-
| `pnpm verify:deps` | fail if runtime dependencies are added |
|
|
381
|
-
| `pnpm test` | Vitest smoke tests |
|
|
382
|
-
| `pnpm build` | tsup → ESM + CJS + declarations |
|
|
426
|
+
---
|
|
383
427
|
|
|
384
428
|
## Author
|
|
385
429
|
|