streamtrust 0.4.0 → 0.5.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -10,7 +10,7 @@ StreamTrust is a CI check for streaming applications such as AI chat. Its reusab
10
10
  - **Repeated runs.** Each scenario runs N times and the report shows every repetition, because cancellation and retry bugs are often intermittent.
11
11
  - **Your contract, written down.** An adapter implements your app's start, stream, cancel, retry and saved-state operations; StreamTrust invokes the adapter scenarios and writes JSON, JUnit, Markdown and HTML reports.
12
12
 
13
- **Status:** `0.4.0`, an early 0.x release for independent teams. APIs and report formats may change before 1.0. The CLI is dependency-free, ESM, and requires Node.js 22+. MIT licensed. Browser and database libraries belong to your adapter and are not installed by StreamTrust.
13
+ **Status:** `0.5.0`, an early 0.x release for independent teams. APIs and report formats may change before 1.0. The CLI is dependency-free, ESM, and requires Node.js 22+. MIT licensed. Browser and database libraries belong to your adapter and are not installed by StreamTrust.
14
14
 
15
15
  ## Watch the 66-second demo
16
16
 
@@ -28,7 +28,7 @@ npx streamtrust init
28
28
  npx streamtrust verify --adapter ./streamtrust/adapter.mjs
29
29
  ```
30
30
 
31
- For reproducible CI, pin `streamtrust@0.4.0` and commit your lockfile. The generated adapter initially exercises the bundled demonstration; map it to your application before using its results as application evidence.
31
+ For reproducible CI, pin `streamtrust@0.5.0` and commit your lockfile. The generated adapter initially exercises the bundled demonstration; map it to your application before using its results as application evidence.
32
32
 
33
33
  ## GitHub Action
34
34
 
@@ -70,6 +70,10 @@ A `vercel-ai` preset runs a minimal Next.js chat built on the AI SDK (`useChat`,
70
70
 
71
71
  Run `npx streamtrust init nextchat` to generate a Playwright adapter for a separate pinned NextChat checkout. It checks saved replies, Stop, regeneration and retry after a provider disconnect against the chat history stored in the browser. The local provider needs no model key. See the [NextChat setup and CI guide](examples/nextchat/README.md) and [integration workflow](.github/workflows/nextchat.yml).
72
72
 
73
+ ## Open WebUI integration
74
+
75
+ Generate the Open WebUI preset with `npx streamtrust init open-webui`. The adapter starts a disposable upstream container, drives chat through a browser and compares results with the saved-chat API. The [setup and CI guide](examples/open-webui/README.md) lists the contract for each report scenario.
76
+
73
77
  ## Run from source
74
78
 
75
79
  ```sh
@@ -88,7 +92,7 @@ Build the tarball with `npm pack` in this repository.
88
92
  Put the supplied tarball in your project's `vendor/` directory, then:
89
93
 
90
94
  ```sh
91
- npm install --save-dev ./vendor/streamtrust-0.4.0.tgz
95
+ npm install --save-dev ./vendor/streamtrust-0.5.0.tgz
92
96
  npx streamtrust init
93
97
  npx streamtrust verify --adapter ./streamtrust/adapter.mjs
94
98
  ```
@@ -184,7 +188,7 @@ Add StreamTrust as a test step in your existing CI. For GitHub Actions, follow t
184
188
  1. Install StreamTrust and create the starting adapter:
185
189
 
186
190
  ```sh
187
- npm install --save-dev --save-exact streamtrust@0.4.0
191
+ npm install --save-dev --save-exact streamtrust@0.5.0
188
192
  npx streamtrust init
189
193
  ```
190
194
 
@@ -0,0 +1,48 @@
1
+ # Open WebUI integration
2
+
3
+ The adapter drives a disposable Open WebUI v0.13.0 instance with Playwright and a local OpenAI-compatible provider. The upstream commit is `c38885a1a737da4968f157092f304f3b5481d0bf`. The official slim image index is pinned to `ghcr.io/open-webui/open-webui@sha256:283efb20756785229a388bcd1e56a052f5284ae8c861717eed5eaaa3bba6b20a` for amd64 and arm64.
4
+
5
+ ## Setup
6
+
7
+ Install Node 22+, Docker and Playwright. Start Docker before running the verifier. From a verification project, once the CLI release is available:
8
+
9
+ ```sh
10
+ npm install --save-dev --save-exact streamtrust@0.5.0 playwright@1.64.1
11
+ npx playwright install chromium
12
+ npx streamtrust init open-webui
13
+ npx streamtrust verify --adapter open-webui/adapter.mjs --output reports/open-webui --repetitions 3 --timeout-ms 900000
14
+ ```
15
+
16
+ Use `init <directory> --preset open-webui` to choose a directory. The adapter publishes the app on a randomly assigned loopback port, waits for health, signs into a runtime-generated admin account, and dismisses the first-launch release dialog. Each scenario uses a fresh browser context and a new chat. The container data directory lives in tmpfs and is removed on cleanup. The app connects to a mock on the Docker bridge gateway on native Linux or through Docker Desktop host routing on macOS. The provider requires a disposable bearer key.
17
+
18
+ ## Saved state and contracts
19
+
20
+ The adapter reads the authenticated saved-chat API (`GET /api/v1/chats/{id}`) backed by Open WebUI’s local database. The history contains the assistant records and parent IDs. Reload selects the persisted `current_message_id`, falling back to `chat.history.currentId` when the former is absent. The adapter compares the selected record and visible reply after reload.
21
+
22
+ | Report scenario | What a pass requires |
23
+ |---|---|
24
+ | `persistence.shared` | Shared evaluator: one matching complete, error-free answer, visible before and after reload, with no new generation on reload. The active saved record must match. |
25
+ | `cancellation.provider-abort-saved-partial` | App-specific: after native Stop, the mock sees its open HTTP response close before completion. The partial answer stays active and saved after reload, no late suffix appears, and generation count stays at one. Open WebUI can mark a stopped message done. |
26
+ | `regeneration.shared-active-sibling` | Shared evaluator plus active selection: the completed original is saved before Regenerate. The original content and ID survive, the replacement has a distinct ID and the same parent, and reload selects the replacement. |
27
+ | `error-retry.shared-saved-visible` | Shared evaluator plus active selection: a synthetic SSE provider error is saved and visible after reload before Regenerate. The replacement is active in storage and visible after a second reload. |
28
+
29
+ An unavailable required storage field produces UNSUPPORTED where the adapter cannot establish the contract. Runtime or selector errors produce harness errors. Shareable evidence includes booleans and counts. Only the synthetic text-error fixture and the pinned app’s single-user chat lifecycle are in scope.
30
+
31
+ ## Run in CI
32
+
33
+ Commit the generated adapter and consumer lockfile. On an Ubuntu Docker runner, after checkout, Node setup, `npm ci` and `npx playwright install --with-deps chromium`, use the [StreamTrust Action](https://github.com/santoshsibyala/streamtrust-action):
34
+
35
+ ```yaml
36
+ - uses: santoshsibyala/streamtrust-action@v1
37
+ with:
38
+ version: '0.5.0'
39
+ adapter: open-webui/adapter.mjs
40
+ repetitions: '3'
41
+ timeout-ms: '900000'
42
+ artifact-name: open-webui-safe-reports
43
+ fail-on: failures
44
+ ```
45
+
46
+ ## Upstream license
47
+
48
+ Open WebUI runs from a separate upstream image under the [Open WebUI License](https://github.com/open-webui/open-webui/blob/c38885a1a737da4968f157092f304f3b5481d0bf/LICENSE). It imposes branding, notice and endorsement conditions. This independent preset preserves the upstream interface and branding and ships only verification code.
@@ -0,0 +1,362 @@
1
+ import { chromium } from "playwright";
2
+ import { evaluateLifecycle } from "streamtrust/checks";
3
+ import { randomUUID } from "node:crypto";
4
+ import { execFileSync } from "node:child_process";
5
+ import { startProvider, until } from "./provider.mjs";
6
+
7
+ const image =
8
+ "ghcr.io/open-webui/open-webui@sha256:283efb20756785229a388bcd1e56a052f5284ae8c861717eed5eaaa3bba6b20a";
9
+ const revision = "c38885a1a737da4968f157092f304f3b5481d0bf";
10
+ const scenarios = [
11
+ "persistence.shared",
12
+ "cancellation.provider-abort-saved-partial",
13
+ "regeneration.shared-active-sibling",
14
+ "error-retry.shared-saved-visible",
15
+ ];
16
+ const docker = (args, env = {}) =>
17
+ execFileSync("docker", args, {
18
+ encoding: "utf8",
19
+ stdio: ["ignore", "pipe", "pipe"],
20
+ env: { ...process.env, ...env },
21
+ }).trim();
22
+
23
+ // The API gives current_message_id precedence over history.currentId on hydration.
24
+ async function saved(page) {
25
+ return page.evaluate(async () => {
26
+ const id = location.pathname.split("/").pop();
27
+ const response = await fetch("/api/v1/chats/" + id, {
28
+ headers: { authorization: "Bearer " + localStorage.token },
29
+ });
30
+ if (!response.ok) throw Error("Saved-chat API unavailable");
31
+ const chat = await response.json();
32
+ const history = chat.chat.history;
33
+ return {
34
+ active: chat.current_message_id || history.currentId,
35
+ answers: Object.values(history.messages).filter(
36
+ (m) => m.role === "assistant",
37
+ ),
38
+ };
39
+ });
40
+ }
41
+
42
+ export default async function create() {
43
+ const name = "streamtrust-webui-" + randomUUID();
44
+ const key = randomUUID(),
45
+ password = randomUUID(),
46
+ email = randomUUID() + "@example.com";
47
+ let browser,
48
+ provider,
49
+ started = false;
50
+ const close = async () => {
51
+ try {
52
+ await browser?.close();
53
+ } finally {
54
+ try {
55
+ if (started) docker(["rm", "-fv", name]);
56
+ } finally {
57
+ await provider?.close();
58
+ }
59
+ }
60
+ };
61
+ try {
62
+ // Linux containers reach the host on the default bridge gateway. Docker Desktop
63
+ // provides host.docker.internal for services listening on the host loopback.
64
+ const linux = process.platform === "linux";
65
+ const gateway = linux
66
+ ? docker([
67
+ "network",
68
+ "inspect",
69
+ "bridge",
70
+ "--format",
71
+ "{{(index .IPAM.Config 0).Gateway}}",
72
+ ])
73
+ : "127.0.0.1";
74
+ provider = await startProvider(gateway, key);
75
+ const settings = {
76
+ WEBUI_ADMIN_EMAIL: email,
77
+ WEBUI_ADMIN_PASSWORD: password,
78
+ WEBUI_SECRET_KEY: randomUUID(),
79
+ ENABLE_SIGNUP: "False",
80
+ OFFLINE_MODE: "True",
81
+ ENABLE_OLLAMA_API: "False",
82
+ ENABLE_OPENAI_API: "True",
83
+ OPENAI_API_BASE_URL: `http://${
84
+ linux ? gateway : "host.docker.internal"
85
+ }:${provider.port}/v1`,
86
+ OPENAI_API_KEY: key,
87
+ DEFAULT_MODELS: "gpt-4o-mini",
88
+ ENABLE_TITLE_GENERATION: "False",
89
+ ENABLE_TAGS_GENERATION: "False",
90
+ ENABLE_FOLLOW_UP_GENERATION: "False",
91
+ ENABLE_VERSION_UPDATE_CHECK: "False",
92
+ ENABLE_PERSISTENT_CONFIG: "False",
93
+ HF_HUB_OFFLINE: "1",
94
+ RAG_EMBEDDING_MODEL_AUTO_UPDATE: "False",
95
+ RAG_RERANKING_MODEL_AUTO_UPDATE: "False",
96
+ };
97
+ docker(
98
+ [
99
+ "run",
100
+ "-d",
101
+ "--rm",
102
+ "--name",
103
+ name,
104
+ "-p",
105
+ "127.0.0.1::8080",
106
+ "--tmpfs",
107
+ "/app/backend/data",
108
+ ...Object.keys(settings).flatMap((k) => ["-e", k]),
109
+ image,
110
+ ],
111
+ settings,
112
+ );
113
+ started = true;
114
+ const url = "http://" + docker(["port", name, "8080/tcp"]);
115
+ await until(
116
+ async () => {
117
+ try {
118
+ return (await fetch(url + "/health")).ok;
119
+ } catch {
120
+ return false;
121
+ }
122
+ },
123
+ "container health",
124
+ 180000,
125
+ );
126
+ const login = await fetch(url + "/api/v1/auths/signin", {
127
+ method: "POST",
128
+ headers: { "content-type": "application/json" },
129
+ body: JSON.stringify({ email, password }),
130
+ });
131
+ if (!login.ok) throw Error("Disposable admin sign-in failed");
132
+ const { token } = await login.json();
133
+ if (!token) throw Error("Disposable admin session unavailable");
134
+ browser = await chromium.launch({
135
+ channel: process.env.STREAMTRUST_BROWSER_CHANNEL,
136
+ });
137
+ return {
138
+ name: "Open WebUI",
139
+ scenarios,
140
+ close,
141
+ metadata: async () => ({
142
+ commit: revision,
143
+ image,
144
+ evidencePolicy: "summary-only-v1",
145
+ }),
146
+ async run(label) {
147
+ const scenario = label.split(".")[0];
148
+ const ctx = await browser.newContext({
149
+ locale: "en-US",
150
+ storageState: {
151
+ cookies: [],
152
+ origins: [
153
+ { origin: url, localStorage: [{ name: "token", value: token }] },
154
+ ],
155
+ },
156
+ });
157
+ let stage = "opening the local chat";
158
+ const start = provider.calls.length;
159
+ const calls = () => provider.calls.slice(start).filter((c) => c.send);
160
+ const check = (name, passed) => ({ name, passed });
161
+ try {
162
+ await ctx.route(/^https?:\/\//, (r) =>
163
+ new URL(r.request().url()).origin === url
164
+ ? r.continue()
165
+ : r.abort(),
166
+ );
167
+ const page = await ctx.newPage();
168
+ const text = () => page.locator("body").innerText();
169
+ const has = async (value) => (await text()).includes(value);
170
+ const untilVisible = (value) =>
171
+ until(() => has(value), "rendered answer");
172
+ const stop = page.getByRole("button", { name: "Stop", exact: true });
173
+ const dialog = page.getByRole("dialog", { name: /What.s New/i });
174
+ await page.addLocatorHandler(dialog, () =>
175
+ dialog.getByRole("button", { name: "Close", exact: true }).click(),
176
+ );
177
+ await page.goto(url);
178
+ await page
179
+ .getByText("gpt-4o-mini", { exact: true })
180
+ .first()
181
+ .waitFor();
182
+ await page.locator("#chat-input").fill("Synthetic lifecycle check");
183
+ await page
184
+ .getByRole("button", { name: "Send message", exact: true })
185
+ .click();
186
+ stage = "waiting for first provider call";
187
+ await until(() => calls().length === 1, "first generation");
188
+ const first = calls()[0];
189
+ first.send(first.prefix);
190
+ await untilVisible(first.prefix);
191
+ await page.waitForURL("**/c/*");
192
+ let target = first,
193
+ aborted = false,
194
+ failure,
195
+ originalId;
196
+ stage = "settling the first response";
197
+ if (scenario === "cancellation") {
198
+ await stop.click();
199
+ await stop.waitFor({ state: "hidden" });
200
+ // A closed response counts only while the provider is still holding it open.
201
+ const deadline = Date.now() + 5000;
202
+ while (!first.closed && Date.now() < deadline)
203
+ await new Promise((r) => setTimeout(r, 50));
204
+ aborted = first.closed && !first.sentFinish;
205
+ if (!first.closed) first.finish();
206
+ } else if (scenario === "error-retry") {
207
+ first.error();
208
+ await stop.waitFor({ state: "hidden" });
209
+ const stateBeforeRetry = await saved(page);
210
+ const failed = stateBeforeRetry.answers.find(
211
+ (m) =>
212
+ m.id === stateBeforeRetry.active &&
213
+ m.content?.includes(first.prefix) &&
214
+ JSON.stringify(m.error ?? "").includes(
215
+ "Synthetic test interruption",
216
+ ),
217
+ );
218
+ await page.reload();
219
+ await untilVisible(first.prefix);
220
+ await page
221
+ .getByText("Synthetic test interruption", { exact: false })
222
+ .last()
223
+ .waitFor({ timeout: 5000 })
224
+ .catch(() => {});
225
+ failure = {
226
+ saved: !!failed,
227
+ visibleAfterReload: await has("Synthetic test interruption"),
228
+ };
229
+ if (!failure.saved || !failure.visibleAfterReload)
230
+ return {
231
+ checks: [],
232
+ unsupported:
233
+ "Saved provider error and its visible marker could not both be established across reload before retry.",
234
+ evidence: failure,
235
+ };
236
+ } else {
237
+ first.finish();
238
+ await untilVisible(first.prefix + first.suffix);
239
+ }
240
+ if (scenario === "regeneration" || scenario === "error-retry") {
241
+ stage = "observing the original and regenerating";
242
+ if (scenario === "regeneration") {
243
+ await until(async () => {
244
+ const state = await saved(page);
245
+ originalId = state.answers.find(
246
+ (m) => m.content === first.prefix + first.suffix,
247
+ )?.id;
248
+ return originalId && state.active === originalId;
249
+ }, "saved original");
250
+ }
251
+ await page
252
+ .getByRole("button", { name: "Regenerate", exact: true })
253
+ .last()
254
+ .click();
255
+ const menu = page
256
+ .getByRole("menuitem", { name: /regenerate|try again/i })
257
+ .first();
258
+ if (await menu.isVisible()) await menu.click();
259
+ await until(() => calls().length === 2, "second generation");
260
+ target = calls()[1];
261
+ target.send(target.prefix);
262
+ target.finish();
263
+ }
264
+ stage = "checking saved state and reload";
265
+ const expected =
266
+ scenario === "cancellation"
267
+ ? target.prefix
268
+ : target.prefix + target.suffix;
269
+ await untilVisible(expected);
270
+ await until(
271
+ async () =>
272
+ (await saved(page)).answers.some((m) => m.content === expected),
273
+ "saved reply",
274
+ );
275
+ const beforeReload = await text(),
276
+ callsBeforeReload = calls().length;
277
+ await page.reload();
278
+ await untilVisible(target.prefix);
279
+ const reloaded = await text(),
280
+ state = await saved(page);
281
+ const active = state.answers.find((m) => m.id === state.active);
282
+ const activeMatches = active?.content === expected;
283
+ let checks;
284
+ if (scenario === "cancellation") {
285
+ checks = [
286
+ check("Provider observed abort before finish", aborted),
287
+ check(
288
+ "Active partial saved and visible after reload",
289
+ activeMatches && reloaded.includes(expected),
290
+ ),
291
+ check("Late suffix absent", !reloaded.includes(first.suffix)),
292
+ check(
293
+ "Exactly one generation across reload",
294
+ calls().length === 1,
295
+ ),
296
+ ];
297
+ } else {
298
+ if (state.answers.some((m) => typeof m.done !== "boolean"))
299
+ return {
300
+ checks: [],
301
+ unsupported:
302
+ "Saved assistant completion state unavailable for shared checks.",
303
+ evidence: {},
304
+ };
305
+ ({ checks } = evaluateLifecycle({
306
+ scenario,
307
+ expected,
308
+ original: first.prefix + first.suffix,
309
+ previousPrefix: first.prefix,
310
+ beforeReload,
311
+ reloaded,
312
+ callsBeforeReload,
313
+ callsAfterReload: calls().length,
314
+ failure,
315
+ messages: state.answers.map((m) => ({
316
+ id: m.id,
317
+ parentId: m.parentId,
318
+ text: m.content,
319
+ assistant: true,
320
+ unfinished: !m.done,
321
+ error: !!m.error,
322
+ })),
323
+ }));
324
+ checks.push(
325
+ check("Saved active ID selects expected answer", activeMatches),
326
+ );
327
+ if (scenario === "regeneration")
328
+ checks.push(
329
+ check(
330
+ "Original ID and content retained",
331
+ state.answers.some(
332
+ (m) =>
333
+ m.id === originalId &&
334
+ m.content === first.prefix + first.suffix,
335
+ ),
336
+ ),
337
+ );
338
+ }
339
+ return {
340
+ checks,
341
+ evidence: {
342
+ callsBeforeReload,
343
+ callsAfterReload: calls().length,
344
+ assistantCount: state.answers.length,
345
+ activeMatches,
346
+ },
347
+ };
348
+ } catch {
349
+ throw Error("Open WebUI observation failed while " + stage);
350
+ } finally {
351
+ for (const c of calls()) if (!c.closed) c.res.end();
352
+ await ctx.close();
353
+ }
354
+ },
355
+ };
356
+ } catch {
357
+ await close();
358
+ throw Error(
359
+ "Open WebUI setup failed. Check Docker, the pinned image, local provider routing, admin bootstrap and Chromium.",
360
+ );
361
+ }
362
+ }
@@ -0,0 +1,113 @@
1
+ import { createServer } from "node:http";
2
+ export async function startProvider(host, key) {
3
+ const calls = [];
4
+ const server = createServer(async (req, res) => {
5
+ if (req.headers.authorization !== "Bearer " + key) {
6
+ res.writeHead(401);
7
+ return res.end();
8
+ }
9
+ res.setHeader("Access-Control-Allow-Headers", "*");
10
+ if (req.method === "OPTIONS") return res.end();
11
+ if (req.url?.endsWith("/models")) {
12
+ res.setHeader("Content-Type", "application/json");
13
+ return res.end(
14
+ JSON.stringify({
15
+ object: "list",
16
+ data: [
17
+ { id: "gpt-4o-mini", object: "model", owned_by: "local-fixture" },
18
+ ],
19
+ }),
20
+ );
21
+ }
22
+ let raw = "";
23
+ for await (const b of req) raw += b;
24
+ const body = raw ? JSON.parse(raw) : {};
25
+ const id = `fixture-${calls.length + 1}`;
26
+ const call = {
27
+ id,
28
+ path: req.url,
29
+ model: body.model,
30
+ closed: false,
31
+ sentFinish: false,
32
+ res,
33
+ prefix: `Synthetic answer ${calls.length + 1}.`,
34
+ suffix: " Completed.",
35
+ };
36
+ calls.push(call);
37
+ res.on("close", () => {
38
+ call.closed = true;
39
+ });
40
+ if (!body.stream) {
41
+ res.setHeader("Content-Type", "application/json");
42
+ return res.end(
43
+ JSON.stringify({
44
+ id,
45
+ object: "chat.completion",
46
+ choices: [
47
+ {
48
+ index: 0,
49
+ message: { role: "assistant", content: "Synthetic title" },
50
+ finish_reason: "stop",
51
+ },
52
+ ],
53
+ usage: { prompt_tokens: 1, completion_tokens: 1, total_tokens: 2 },
54
+ }),
55
+ );
56
+ }
57
+ res.writeHead(200, {
58
+ "Content-Type": "text/event-stream",
59
+ "Cache-Control": "no-cache",
60
+ });
61
+ res.flushHeaders();
62
+ call.send = (text) =>
63
+ res.write(
64
+ `data: ${JSON.stringify({
65
+ id,
66
+ object: "chat.completion.chunk",
67
+ created: 1,
68
+ model: body.model,
69
+ choices: [
70
+ { index: 0, delta: { content: text }, finish_reason: null },
71
+ ],
72
+ })}\n\n`,
73
+ );
74
+ call.finish = () => {
75
+ call.sentFinish = true;
76
+ call.send(call.suffix);
77
+ res.write(
78
+ `data: ${JSON.stringify({
79
+ id,
80
+ object: "chat.completion.chunk",
81
+ created: 1,
82
+ model: body.model,
83
+ choices: [{ index: 0, delta: {}, finish_reason: "stop" }],
84
+ })}\n\ndata: [DONE]\n\n`,
85
+ );
86
+ res.end();
87
+ };
88
+ call.error = () =>
89
+ res.end(
90
+ 'data: {"error":{"message":"Synthetic test interruption"}}\n\ndata: [DONE]\n\n',
91
+ );
92
+ });
93
+ await new Promise((ready, reject) => {
94
+ server.once("error", reject);
95
+ server.listen(0, host, ready);
96
+ });
97
+ return {
98
+ port: server.address().port,
99
+ calls,
100
+ async close() {
101
+ server.closeAllConnections();
102
+ await new Promise((r) => server.close(r));
103
+ },
104
+ };
105
+ }
106
+ export async function until(fn, label, ms = 30000) {
107
+ const deadline = Date.now() + ms;
108
+ while (Date.now() < deadline) {
109
+ if (await fn()) return;
110
+ await new Promise((r) => setTimeout(r, 100));
111
+ }
112
+ throw new Error(`Adapter timeout: ${label}`);
113
+ }
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "streamtrust",
3
- "version": "0.4.0",
3
+ "version": "0.5.0",
4
4
  "description": "Adapter-driven CI verification for streaming application lifecycle guarantees",
5
5
  "type": "module",
6
6
  "license": "MIT",
@@ -39,7 +39,9 @@
39
39
  "README.md",
40
40
  "LICENSE",
41
41
  "examples/nextchat/*.mjs",
42
- "examples/nextchat/README.md"
42
+ "examples/nextchat/README.md",
43
+ "examples/open-webui/*.mjs",
44
+ "examples/open-webui/README.md"
43
45
  ],
44
46
  "scripts": {
45
47
  "test": "node --test test/*.test.mjs",
package/src/cli.js CHANGED
@@ -19,13 +19,15 @@ const PRESETS = {
19
19
  'reference-app/app/api/chat/route.js', 'reference-app/app/chat/[id]/chat.jsx',
20
20
  'reference-app/app/chat/[id]/page.jsx',
21
21
  ].map((name) => [`vercel-ai/${name}`, name]),
22
+ 'open-webui': ['adapter.mjs', 'provider.mjs', 'README.md'].map(
23
+ (name) => [`open-webui/${name}`, name]),
22
24
  nextchat: ['adapter.mjs', 'provider.mjs', 'README.md'].map(
23
25
  (name) => [`nextchat/${name}`, name]),
24
26
  };
25
27
  export function presets() { return Object.keys(PRESETS); }
26
28
  function presetFiles(preset) { return PRESETS[preset]; }
27
29
  const HELP = () =>
28
- `StreamTrust ${VERSION}\n\nUsage:\n streamtrust init [directory] [--preset <name>]\n streamtrust init nextchat\n streamtrust verify --adapter ./streamtrust/adapter.mjs [options]\n\nOptions:\n --output <directory> Reports (default: reports/streamtrust)\n --repetitions <1-1000> Repetitions per scenario (default: 3)\n --timeout-ms <integer> Total run deadline including setup/cleanup (default: 300000)\n --help Show help\n --version Show version\n\nPresets: ${presets().join(', ')} (default: http)\n\nExit codes: 0 all pass; 1 failed/unsupported checks; 2 setup, timeout or harness error.\nAdapters execute trusted local code. JavaScript ESM (.mjs) works without dependencies.\n`;
30
+ `StreamTrust ${VERSION}\n\nUsage:\n streamtrust init [directory] [--preset <name>]\n streamtrust init nextchat|open-webui\n streamtrust verify --adapter ./streamtrust/adapter.mjs [options]\n\nOptions:\n --output <directory> Reports (default: reports/streamtrust)\n --repetitions <1-1000> Repetitions per scenario (default: 3)\n --timeout-ms <integer> Total run deadline including setup/cleanup (default: 300000)\n --help Show help\n --version Show version\n\nPresets: ${presets().join(', ')} (default: http)\n\nExit codes: 0 all pass; 1 failed/unsupported checks; 2 setup, timeout or harness error.\nAdapters execute trusted local code. JavaScript ESM (.mjs) works without dependencies.\n`;
29
31
  export async function main(args) {
30
32
  let outputDir;
31
33
  try {
@@ -40,7 +42,7 @@ export async function main(args) {
40
42
  if (args[0] === 'init') {
41
43
  const initArgs = args.slice(1);
42
44
  const presetIndex = initArgs.indexOf('--preset');
43
- let preset = initArgs.length === 1 && initArgs[0] === 'nextchat' ? 'nextchat' : 'http';
45
+ let preset = initArgs.length === 1 && ['nextchat', 'open-webui'].includes(initArgs[0]) ? initArgs[0] : 'http';
44
46
  if (presetIndex !== -1) {
45
47
  preset = initArgs[presetIndex + 1];
46
48
  initArgs.splice(presetIndex, 2);