streamtrust 0.3.0 → 0.4.0

package/README.md

@@ -10,7 +10,7 @@ StreamTrust is a CI check for streaming applications such as AI chat. Its reusab
 - **Repeated runs.** Each scenario runs N times and the report shows every repetition, because cancellation and retry bugs are often intermittent.
 - **Your contract, written down.** An adapter implements your app's start, stream, cancel, retry and saved-state operations; StreamTrust invokes the adapter scenarios and writes JSON, JUnit, Markdown and HTML reports.
 
-**Status:** `0.3.0`, an early 0.x release for independent teams. APIs and report formats may change before 1.0. The CLI is dependency-free, ESM, and requires Node.js 22+. MIT licensed. Browser and database libraries belong to your adapter and are not installed by StreamTrust.
+**Status:** `0.4.0`, an early 0.x release for independent teams. APIs and report formats may change before 1.0. The CLI is dependency-free, ESM, and requires Node.js 22+. MIT licensed. Browser and database libraries belong to your adapter and are not installed by StreamTrust.
 
 ## Watch the 66-second demo
 
@@ -28,7 +28,31 @@ npx streamtrust init
 npx streamtrust verify --adapter ./streamtrust/adapter.mjs
 ```
 
-For reproducible CI, pin `streamtrust@0.3.0` and commit your lockfile. The generated adapter initially exercises the bundled demonstration; map it to your application before using its results as application evidence.
+For reproducible CI, pin `streamtrust@0.4.0` and commit your lockfile. The generated adapter initially exercises the bundled demonstration; map it to your application before using its results as application evidence.
+
+## GitHub Action
+
+Run StreamTrust through the [official composite GitHub Action](https://github.com/santoshsibyala/streamtrust-action), listed as [StreamTrust CI Verifier on GitHub Marketplace](https://github.com/marketplace/actions/streamtrust-ci-verifier):
+
+```yaml
+- uses: santoshsibyala/streamtrust-action@v1
+  with:
+    adapter: ./streamtrust/adapter.mjs
+    node-version: "22"
+    artifact-name: streamtrust-report
+```
+
+Check out your repository, install adapter dependencies, and start your test services first. The Action runs the pinned StreamTrust CLI, adds a job summary, and uploads diagnostic reports even when verification fails. By default, failed or unsupported checks and harness errors fail the job. See the [complete workflow, inputs, outputs, and failure policies](https://github.com/santoshsibyala/streamtrust-action/blob/main/README.md).
+
+Use `@v1.0.0` to pin this Action release; `@v1` follows compatible Action updates. The generated HTTP adapter initially tests a bundled demo: adapt it to your application before treating its results as application coverage. Adapters run trusted repository code; use disposable test data and keep secrets out of uploaded reports.
+
+### Published HTTP/SSE demo
+
+StreamTrust uses the published Action in its own CI. This [successful main-branch run](https://github.com/santoshsibyala/streamtrust/actions/runs/42171351265) used Action v1.0.0 and CLI 0.3.0 on Node 22: persistence, cancellation, and retry each passed three repetitions (9 PASS).
+
+[![Published HTTP/SSE demo: StreamTrust Action job summary showing 9 PASS, 0 FAIL, 0 UNSUPPORTED, and 0 HARNESS_ERROR](docs/images/published-action-http-sse-summary.png)](https://github.com/santoshsibyala/streamtrust/actions/runs/42171351265#summary-127739661923)
+
+This run checks the bundled HTTP/SSE teaching demo. The existing CLI, LibreChat, NextChat, and Vercel AI SDK workflows separately test the package built from the current checkout.
 
 ## LibreChat integration
 
@@ -40,7 +64,11 @@ A LibreChat preset now exercises persistence, native Stop, regeneration and erro
 
 A `vercel-ai` preset runs a minimal Next.js chat built on the AI SDK (`useChat`, `streamText`) in production mode with a synthetic OpenAI-compatible provider, so it needs no API key. It covers the same four scenarios through the browser and the app's saved state, and can be pointed at your own built AI SDK app after you adjust its selectors and saved-state read. It is not an official Vercel integration.
 
-**Available in StreamTrust 0.3.0.** Install from npm and generate it with `npx streamtrust init streamtrust-vercel-ai --preset vercel-ai`. See the [Vercel AI SDK quick start](examples/vercel-ai/README.md) for what it does and does not verify. It requires Node 22+ and Playwright in your verification project. An [integration workflow](.github/workflows/vercel-ai.yml) exercises the packed CLI against the reference app.
+**Available since StreamTrust 0.3.0.** Install from npm and generate it with `npx streamtrust init streamtrust-vercel-ai --preset vercel-ai`. See the [Vercel AI SDK quick start](examples/vercel-ai/README.md) for what it does and does not verify. It requires Node 22+ and Playwright in your verification project. An [integration workflow](.github/workflows/vercel-ai.yml) exercises the packed CLI against the reference app.
+
+## NextChat integration
+
+Run `npx streamtrust init nextchat` to generate a Playwright adapter for a separate pinned NextChat checkout. It checks saved replies, Stop, regeneration and retry after a provider disconnect against the chat history stored in the browser. The local provider needs no model key. See the [NextChat setup and CI guide](examples/nextchat/README.md) and [integration workflow](.github/workflows/nextchat.yml).
 
 ## Run from source
 
@@ -60,7 +88,7 @@ Build the tarball with `npm pack` in this repository.
 Put the supplied tarball in your project's `vendor/` directory, then:
 
 ```sh
-npm install --save-dev ./vendor/streamtrust-0.3.0.tgz
+npm install --save-dev ./vendor/streamtrust-0.4.0.tgz
 npx streamtrust init
 npx streamtrust verify --adapter ./streamtrust/adapter.mjs
 ```
@@ -156,7 +184,7 @@ Add StreamTrust as a test step in your existing CI. For GitHub Actions, follow t
 1. Install StreamTrust and create the starting adapter:
 
    ```sh
-   npm install --save-dev --save-exact streamtrust@0.3.0
+   npm install --save-dev --save-exact streamtrust@0.4.0
    npx streamtrust init
    ```
 

package/examples/librechat/README.md

@@ -4,11 +4,11 @@ This preset runs a **built local LibreChat checkout**, native browser UI and rea
 
 ## Installation
 
-Use Node.js 24 and npm 11. In a separate verification project, install StreamTrust 0.3.0 from npm:
+Use Node.js 24 and npm 11. In a separate verification project, install StreamTrust 0.4.0 from npm:
 
 ```sh
 npm init -y
-npm install --save-dev --save-exact streamtrust@0.3.0
+npm install --save-dev --save-exact streamtrust@0.4.0
 npm install --save-dev --save-exact playwright@1.64.1 mongodb@6.20.0 mongodb-memory-server@11.3.0
 npx playwright install chromium
 npx streamtrust init streamtrust-librechat --preset librechat
@@ -74,6 +74,31 @@ Use a fresh report directory for each job. Restrict workflow permissions to `con
 
 Exit 0 means all declared checks passed; 1 means failed or unsupported checks; 2 means setup, timeout, or harness error. Keep the exit status: do not use `continue-on-error`. Inspect reports before calling a failure an upstream defect.
 
+## Run in CI
+
+To use the published GitHub Action, keep the checkout, dependency installation, Playwright installation, and build steps in the CI example above. Replace only its verify step with:
+
+```yaml
+- name: Verify LibreChat with StreamTrust Action
+  uses: santoshsibyala/streamtrust-action@v1
+  with:
+    adapter: streamtrust-librechat/adapter.mjs
+    version: '0.4.0'
+    node-version: '24'
+    output: reports/librechat
+    repetitions: '3'
+    timeout-ms: '600000'
+    artifact-name: librechat-action-report
+    upload-report: 'false'
+    fail-on: failures
+  env:
+    STREAMTRUST_LIBRECHAT_ROOT: ${{ runner.temp }}/librechat
+```
+
+Keep the existing upload step: `upload-report: 'false'` disables the Action's directory upload so CI continues uploading only the four listed report files. The Action still adds a job summary and fails the job on failed or unsupported checks and harness errors. Use a fresh report directory for each invocation.
+
+`@v1` receives compatible Action updates; `@v1.0.0` pins the exact Action release. The CLI version is pinned separately by `version`. See the [Action README](https://github.com/santoshsibyala/streamtrust-action/blob/main/README.md) for setup, inputs, outputs, and failure policies. This uses the published CLI; retain direct CLI calls when testing a locally built, unreleased StreamTrust package.
+
 ## Evidence and limits
 
 `summary-only-v1` returns stable check identifiers, booleans, message counts and provider-call counts. Raw chat text, database records, user identifiers, network URLs and tokens remain in memory and are not returned as report evidence. The preset does not write screenshots, browser traces or application logs. Browser/database exceptions are replaced with fixed errors. App subprocess output is discarded. Upload only the four report files, not the entire working directory.

package/examples/nextchat/README.md

@@ -0,0 +1,51 @@
+# NextChat CI integration
+
+This preset drives a separate [NextChat checkout](https://github.com/ChatGPTNextWeb/NextChat/tree/1d3dc331512929a4b1d3e82354e72acce33b5c7a) with Playwright and a loopback OpenAI-compatible provider. NextChat is MIT licensed. Its source and dependencies stay in the checkout.
+
+## Setup
+
+In a verification project with Node 22 (the install command selects Yarn Classic 1.22.22):
+
+```sh
+npm init -y
+npm install --save-dev --save-exact streamtrust@0.4.0 playwright@1.64.1
+npx playwright install chromium
+npx streamtrust init nextchat
+export STREAMTRUST_NEXTCHAT_ROOT="$PWD/.cache/NextChat"
+git clone https://github.com/ChatGPTNextWeb/NextChat.git "$STREAMTRUST_NEXTCHAT_ROOT"
+git -C "$STREAMTRUST_NEXTCHAT_ROOT" checkout --detach 1d3dc331512929a4b1d3e82354e72acce33b5c7a
+(cd "$STREAMTRUST_NEXTCHAT_ROOT" && npx --yes --package=yarn@1.22.22 -- yarn install --frozen-lockfile --ignore-scripts)
+npx streamtrust verify --adapter nextchat/adapter.mjs --output reports/nextchat --repetitions 3 --timeout-ms 900000
+```
+
+Use a dedicated checkout with clean tracked files and no development dotenv files. The adapter starts NextChat in development mode on loopback, waits for HTTP readiness, and closes the app, provider and browser after verification. Requests use a synthetic key. Each repetition starts in a fresh browser context. To choose another output directory for the template, use `streamtrust init <directory> --preset nextchat`. To use installed Chrome, set `STREAMTRUST_BROWSER_CHANNEL=chrome`.
+
+## What it checks
+
+| Scenario | Expected outcome after reload |
+|---|---|
+| Persistence | One assistant answer matches the complete provider response, with one generation. |
+| Cancellation | Clicking Stop after the first chunk preserves only the partial answer. Late completion text stays absent even if the provider can finish. |
+| `regeneration.replace-turn` | Before Retry, the completed first answer is saved. After reload, the original assistant ID is absent and the sole assistant answer matches the second response. |
+| `error-retry.replace-turn-same-session` | After a mid-stream disconnect, the first partial answer is saved before Retry. The replacement survives reload and the original assistant ID is absent. |
+
+NextChat intentionally replaces the previous turn on Retry, so these NextChat-specific checks verify replacement in the active session; the shared evaluator checks branch preservation on regeneration and a saved error visible after reload before retry.
+
+The adapter reads the selected session in the persisted Zustand record `chat-next-web-store` (IndexedDB database `keyval-store`, object store `keyval`; NextChat also supports localStorage fallback). It compares assistant text from storage with the rendered page and counts streaming provider calls. This tests browser-local persistence in one session. The Stop check covers visible and saved output. It does not establish server-side cancellation. The retry check covers recovery in the same session, without requiring a durable error flag or sibling branches. The pre-Retry read checks partial text for error retry. Whether a failure flag survives a reload before Retry is outside this scenario. Per-run evidence contains only booleans. Message text and IDs stay in memory. A changed selector or unreadable storage can produce a harness error.
+
+## Run in CI
+
+After installing the locked consumer dependencies, installing Chromium, and preparing the pinned checkout above, export `STREAMTRUST_NEXTCHAT_ROOT` for the job. Commit the generated adapter and helper. With CLI 0.4.0 published, use the [StreamTrust Action](https://github.com/santoshsibyala/streamtrust-action):
+
+```yaml
+- name: Verify NextChat
+  uses: santoshsibyala/streamtrust-action@v1
+  with:
+    adapter: nextchat/adapter.mjs
+    version: '0.4.0'
+    node-version: '22'
+    repetitions: '3'
+    timeout-ms: '900000'
+    artifact-name: nextchat-safe-reports
+    fail-on: failures
+```

package/examples/nextchat/adapter.mjs

@@ -0,0 +1,316 @@
+import { chromium } from "playwright";
+import { spawn, execFileSync } from "node:child_process";
+import { createServer } from "node:net";
+import { existsSync } from "node:fs";
+import { resolve } from "node:path";
+
+import { startProvider, until } from "./provider.mjs";
+const commit = "1d3dc331512929a4b1d3e82354e72acce33b5c7a";
+
+async function availablePort() {
+  const socket = createServer();
+  await new Promise((ready, reject) => {
+    socket.once("error", reject);
+    socket.listen(0, "127.0.0.1", ready);
+  });
+  const port = socket.address().port;
+  await new Promise((done) => socket.close(done));
+  return port;
+}
+
+async function create() {
+  if (!process.env.STREAMTRUST_NEXTCHAT_ROOT)
+    throw Error(
+      "Set STREAMTRUST_NEXTCHAT_ROOT to the isolated NextChat checkout",
+    );
+  const root = resolve(process.env.STREAMTRUST_NEXTCHAT_ROOT);
+  const git = (...args) =>
+    execFileSync("git", args, { cwd: root, encoding: "utf8" }).trim();
+  if (
+    git("rev-parse", "HEAD") !== commit ||
+    git("status", "--porcelain", "--untracked-files=no")
+  )
+    throw Error("Expected clean pinned NextChat source");
+  for (const file of [
+    ".env",
+    ".env.local",
+    ".env.development",
+    ".env.development.local",
+  ]) {
+    if (existsSync(resolve(root, file)))
+      throw Error("Remove dotenv files from the isolated checkout");
+  }
+  const port = await availablePort();
+  const url = `http://127.0.0.1:${port}`;
+  const provider = await startProvider();
+  const app = spawn(
+    process.execPath,
+    [
+      resolve(root, "node_modules/next/dist/bin/next"),
+      "dev",
+      "-H",
+      "127.0.0.1",
+      "-p",
+      String(port),
+    ],
+    {
+      cwd: root,
+      env: {
+        PATH: process.env.PATH,
+        HOME: process.env.HOME,
+        TMPDIR: process.env.TMPDIR,
+        NEXT_TELEMETRY_DISABLED: "1",
+        BASE_URL: provider.url,
+        OPENAI_API_KEY: "local-synthetic-only",
+        DEFAULT_MODEL: "gpt-4o-mini",
+        CUSTOM_MODELS: "-all,+gpt-4o-mini",
+      },
+      stdio: "ignore",
+    },
+  );
+  let stopped = false;
+  const exited = new Promise((done) => {
+    const onExit = () => {
+      stopped = true;
+      done();
+    };
+    app.once("error", onExit);
+    app.once("exit", onExit);
+  });
+  let browser;
+  const close = async () => {
+    try {
+      await browser?.close();
+    } finally {
+      app.kill("SIGTERM");
+      try {
+        await provider.close();
+      } finally {
+        await exited;
+      }
+    }
+  };
+  try {
+    await until(
+      async () => {
+        if (stopped) throw Error("NextChat exited before readiness");
+        try {
+          return (await fetch(url)).ok;
+        } catch {
+          return false;
+        }
+      },
+      "NextChat readiness",
+      120000,
+    );
+    browser = await chromium.launch({
+      channel: process.env.STREAMTRUST_BROWSER_CHANNEL,
+    });
+  } catch (e) {
+    await close();
+    throw e;
+  }
+  return {
+    name: "NextChat local UI and IndexedDB",
+    metadata: async () => ({
+      evidencePolicy: "summary-only-v1",
+      provider: "local synthetic",
+      browser: browser.version(),
+      commit,
+    }),
+    scenarios: [
+      "persistence",
+      "cancellation",
+      "regeneration.replace-turn",
+      "error-retry.replace-turn-same-session",
+    ],
+    close,
+    async run(scenario, { repetition }) {
+      const context = await browser.newContext({ locale: "en-US" }),
+        page = await context.newPage();
+      page.setDefaultTimeout(10000);
+      await context.route(
+        /^https?:\/\/(?!127\.0\.0\.1[:/]|localhost[:/])/,
+        (r) => r.abort(),
+      );
+      const since = provider.calls.length;
+      const calls = () =>
+        provider.calls.slice(since).filter((x) => typeof x.send === "function");
+      const text = async () => page.locator("body").innerText();
+      const read = () =>
+        page.evaluate(async () => {
+          let db, raw;
+          try {
+            db = await new Promise((ok, fail) => {
+              let r = indexedDB.open("keyval-store");
+              r.onsuccess = () => ok(r.result);
+              r.onerror = () => fail(r.error);
+            });
+            raw = await new Promise((ok, fail) => {
+              let r = db
+                .transaction("keyval")
+                .objectStore("keyval")
+                .get("chat-next-web-store");
+              r.onsuccess = () => ok(r.result);
+              r.onerror = () => fail(r.error);
+            });
+          } catch {
+            // NextChat falls back to localStorage when idb-keyval is unavailable.
+          } finally {
+            db?.close();
+          }
+          const state = JSON.parse(
+            raw || localStorage.getItem("chat-next-web-store") || "{}",
+          ).state;
+          return state?.sessions[state.currentSessionIndex]?.messages || [];
+        });
+      const regenerate = scenario === "regeneration.replace-turn";
+      const retryError = scenario === "error-retry.replace-turn-same-session";
+      const retry = regenerate || retryError;
+      let original;
+      const evidence = {};
+      const checks = [];
+      const check = (name, passed) => checks.push({ name, passed: !!passed });
+      try {
+        await page.goto(url + "/#/chat");
+        const box = page.locator("textarea:visible").last();
+        await box.fill(`Synthetic lifecycle check ${repetition}`);
+        await box.press("Enter");
+        await until(() => calls().length > 0, "first local provider call");
+        let c = calls()[0];
+        c.send(c.prefix);
+        await until(
+          async () => (await text()).includes(c.prefix),
+          "first chunk displayed",
+        );
+        if (scenario === "cancellation") {
+          await page
+            .getByText("Stop", { exact: true })
+            .last()
+            .locator("..")
+            .click();
+          await until(
+            async () =>
+              !(await page
+                .getByText("Stop", { exact: true })
+                .last()
+                .isVisible()),
+            "stop control retired",
+            10000,
+          );
+          if (!c.closed) c.finish();
+        } else if (retryError) {
+          c.error();
+        } else {
+          c.finish();
+        }
+        if (retry) {
+          if (regenerate)
+            await until(
+              async () => (await text()).includes(c.prefix + c.suffix),
+              "first answer complete",
+            );
+          await page
+            .getByText("Stop", { exact: true })
+            .last()
+            .waitFor({ state: "hidden" });
+          await page
+            .locator('[class*="chat-message"]')
+            .filter({ hasText: c.prefix })
+            .last()
+            .hover();
+          const retryButton = page
+            .getByText("Retry", { exact: true })
+            .last()
+            .locator("..");
+          await retryButton.waitFor({ state: "visible" });
+          const assistants = (await read()).filter(
+            (m) => m.role === "assistant",
+          );
+          original = assistants[0];
+          evidence.originalSavedBeforeRetry =
+            assistants.length === 1 &&
+            typeof original.id === "string" &&
+            original.id.length > 0 &&
+            (regenerate
+              ? original.content === c.prefix + c.suffix &&
+                original.streaming === false
+              : typeof original.content === "string" &&
+                original.content.startsWith(c.prefix) &&
+                !original.content.includes(c.suffix));
+          check("originalSavedBeforeRetry", evidence.originalSavedBeforeRetry);
+          await retryButton.click();
+          await until(() => calls().length === 2, "second provider call");
+          c = calls()[1];
+          c.send(c.prefix);
+          c.finish();
+        }
+        const expected =
+          scenario === "cancellation" ? c.prefix : c.prefix + c.suffix;
+        await until(
+          async () => (await text()).includes(expected),
+          "answer visible",
+        );
+        await until(
+          async () =>
+            (await read()).some(
+              (m) =>
+                m.role === "assistant" &&
+                typeof m.content === "string" &&
+                m.content.includes(c.prefix),
+            ),
+          "IndexedDB contains displayed text",
+        );
+        await page.reload();
+        await page.locator("textarea:visible").last().waitFor();
+        await until(
+          async () => (await text()).includes(c.prefix),
+          "rehydrated answer",
+        );
+        const saved = (await read()).filter((m) => m.role === "assistant");
+        check(
+          "Saved answer equals expected text",
+          saved.length === 1 && saved[0].content === expected,
+        );
+        check(
+          "Reload shows expected answer",
+          (await text()).includes(expected),
+        );
+        check("No extra provider call", calls().length === (retry ? 2 : 1));
+        if (scenario === "cancellation")
+          check(
+            "No late completion displayed",
+            !(await text()).includes(c.suffix),
+          );
+        if (retry) {
+          evidence.originalAbsentAfterReload =
+            !!original?.id && saved.every((m) => m.id !== original.id);
+          check(
+            "originalAbsentAfterReload",
+            evidence.originalAbsentAfterReload,
+          );
+        }
+        return { checks, evidence };
+      } catch (error) {
+        if (process.env.STREAMTRUST_NEXTCHAT_DEBUG === "1")
+          console.error(error);
+        throw Error(
+          "NextChat scenario could not complete its UI or storage observation",
+        );
+      } finally {
+        for (const c of calls()) if (!c.closed) c.res.end();
+        await context.close();
+      }
+    },
+  };
+}
+
+export default async function createNextChatAdapter() {
+  try {
+    return await create();
+  } catch {
+    throw Error(
+      "NextChat setup failed. Check the pinned checkout, dotenv files, Yarn dependencies and Chromium installation.",
+    );
+  }
+}

package/examples/nextchat/provider.mjs

@@ -0,0 +1,107 @@
+import { createServer } from "node:http";
+export async function startProvider() {
+  const calls = [];
+  const server = createServer(async (req, res) => {
+    res.setHeader("Access-Control-Allow-Origin", "*");
+    res.setHeader("Access-Control-Allow-Headers", "*");
+    if (req.method === "OPTIONS") return res.end();
+    if (req.url?.endsWith("/models")) {
+      res.setHeader("Content-Type", "application/json");
+      return res.end(
+        JSON.stringify({
+          object: "list",
+          data: [
+            { id: "gpt-4o-mini", object: "model", owned_by: "local-fixture" },
+          ],
+        }),
+      );
+    }
+    let raw = "";
+    for await (const b of req) raw += b;
+    const body = raw ? JSON.parse(raw) : {};
+    const id = `fixture-${calls.length + 1}`;
+    const call = {
+      id,
+      path: req.url,
+      model: body.model,
+      closed: false,
+      sentFinish: false,
+      res,
+      prefix: `Synthetic answer ${calls.length + 1}.`,
+      suffix: " Completed.",
+    };
+    calls.push(call);
+    res.on("close", () => {
+      call.closed = true;
+    });
+    if (!body.stream) {
+      res.setHeader("Content-Type", "application/json");
+      return res.end(
+        JSON.stringify({
+          id,
+          object: "chat.completion",
+          choices: [
+            {
+              index: 0,
+              message: { role: "assistant", content: "Synthetic title" },
+              finish_reason: "stop",
+            },
+          ],
+          usage: { prompt_tokens: 1, completion_tokens: 1, total_tokens: 2 },
+        }),
+      );
+    }
+    res.writeHead(200, {
+      "Content-Type": "text/event-stream",
+      "Cache-Control": "no-cache",
+    });
+    res.flushHeaders();
+    call.send = (text) =>
+      res.write(
+        `data: ${JSON.stringify({
+          id,
+          object: "chat.completion.chunk",
+          created: 1,
+          model: body.model,
+          choices: [
+            { index: 0, delta: { content: text }, finish_reason: null },
+          ],
+        })}\n\n`,
+      );
+    call.finish = () => {
+      call.sentFinish = true;
+      call.send(call.suffix);
+      res.write(
+        `data: ${JSON.stringify({
+          id,
+          object: "chat.completion.chunk",
+          created: 1,
+          model: body.model,
+          choices: [{ index: 0, delta: {}, finish_reason: "stop" }],
+        })}\n\ndata: [DONE]\n\n`,
+      );
+      res.end();
+    };
+    call.error = () => res.destroy();
+  });
+  await new Promise((ready, reject) => {
+    server.once("error", reject);
+    server.listen(0, "127.0.0.1", ready);
+  });
+  return {
+    url: `http://127.0.0.1:${server.address().port}`,
+    calls,
+    async close() {
+      server.closeAllConnections();
+      await new Promise((r) => server.close(r));
+    },
+  };
+}
+export async function until(fn, label, ms = 30000) {
+  const deadline = Date.now() + ms;
+  while (Date.now() < deadline) {
+    if (await fn()) return;
+    await new Promise((r) => setTimeout(r, 100));
+  }
+  throw new Error(`Adapter timeout: ${label}`);
+}

package/examples/vercel-ai/README.md

@@ -4,11 +4,11 @@ This preset runs a **minimal Next.js chat built on the AI SDK** (`ai`, `@ai-sdk/
 
 ## Quick start (no API key)
 
-Use Node.js 22 or newer (AI SDK 7 requires it). This preset is available in StreamTrust 0.3.0. In a separate verification project:
+Use Node.js 22 or newer (AI SDK 7 requires it). This preset has been available since StreamTrust 0.3.0. In a separate verification project:
 
 ```sh
 npm init -y
-npm install --save-dev --save-exact streamtrust@0.3.0 playwright@1.64.1
+npm install --save-dev --save-exact streamtrust@0.4.0 playwright@1.64.1
 npx playwright install chromium
 npx streamtrust init streamtrust-vercel-ai --preset vercel-ai
 node streamtrust-vercel-ai/setup.mjs
@@ -93,6 +93,29 @@ Run the installation steps once, then commit your verification project package.j
 
 Exit 0 means all declared checks passed; 1 means failed or unsupported checks; 2 means setup, timeout or harness error. Keep the exit status: do not use `continue-on-error`.
 
+## Run in CI
+
+To use the published GitHub Action, keep the checkout, dependency installation, Playwright installation, and build steps in the CI example above. Replace only its verify step with:
+
+```yaml
+- name: Verify AI SDK app with StreamTrust Action
+  uses: santoshsibyala/streamtrust-action@v1
+  with:
+    adapter: streamtrust-vercel-ai/adapter.mjs
+    version: '0.4.0'
+    node-version: '22'
+    output: reports/vercel-ai
+    repetitions: '3'
+    timeout-ms: '600000'
+    artifact-name: vercel-ai-action-report
+    upload-report: 'false'
+    fail-on: failures
+```
+
+Keep the existing upload step: `upload-report: 'false'` disables the Action's directory upload so CI continues uploading only the four listed report files. The Action still adds a job summary and fails the job on failed or unsupported checks and harness errors. Use a fresh report directory for each invocation.
+
+`@v1` receives compatible Action updates; `@v1.0.0` pins the exact Action release. The CLI version is pinned separately by `version`. See the [Action README](https://github.com/santoshsibyala/streamtrust-action/blob/main/README.md) for setup, inputs, outputs, and failure policies. This uses the published CLI; retain direct CLI calls when testing a locally built, unreleased StreamTrust package.
+
 ## Evidence and limits
 
 Reports use `summary-only-v1`: stable check names, booleans, message counts and provider-call counts. Chat text, message ids, URLs and file paths stay in memory. Reports replace browser and app exceptions with fixed messages even in debug mode. Debug mode prints raw errors to the console; do not enable it in CI that retains logs. This is data minimization, not a sandbox: the adapter and the app under test are trusted code.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "streamtrust",
-  "version": "0.3.0",
+  "version": "0.4.0",
   "description": "Adapter-driven CI verification for streaming application lifecycle guarantees",
   "type": "module",
   "license": "MIT",
@@ -37,7 +37,9 @@
     "examples/vercel-ai/reference-app/app/**/*.js",
     "examples/vercel-ai/reference-app/app/**/*.jsx",
     "README.md",
-    "LICENSE"
+    "LICENSE",
+    "examples/nextchat/*.mjs",
+    "examples/nextchat/README.md"
   ],
   "scripts": {
     "test": "node --test test/*.test.mjs",
@@ -48,7 +50,14 @@
     "streaming",
     "verification",
     "testing",
-    "sse"
+    "sse",
+    "github-action",
+    "ai",
+    "llm",
+    "chat",
+    "playwright",
+    "librechat",
+    "vercel-ai-sdk"
   ],
   "repository": {
     "type": "git",

package/src/cli.js

@@ -19,11 +19,13 @@ const PRESETS = {
     'reference-app/app/api/chat/route.js', 'reference-app/app/chat/[id]/chat.jsx',
     'reference-app/app/chat/[id]/page.jsx',
   ].map((name) => [`vercel-ai/${name}`, name]),
+  nextchat: ['adapter.mjs', 'provider.mjs', 'README.md'].map(
+    (name) => [`nextchat/${name}`, name]),
 };
 export function presets() { return Object.keys(PRESETS); }
 function presetFiles(preset) { return PRESETS[preset]; }
 const HELP = () =>
-  `StreamTrust ${VERSION}\n\nUsage:\n  streamtrust init [directory] [--preset <name>]\n  streamtrust verify --adapter ./streamtrust/adapter.mjs [options]\n\nOptions:\n  --output <directory>    Reports (default: reports/streamtrust)\n  --repetitions <1-1000>   Repetitions per scenario (default: 3)\n  --timeout-ms <integer>  Total run deadline including setup/cleanup (default: 300000)\n  --help                 Show help\n  --version              Show version\n\nPresets: ${presets().join(', ')} (default: http)\n\nExit codes: 0 all pass; 1 failed/unsupported checks; 2 setup, timeout or harness error.\nAdapters execute trusted local code. JavaScript ESM (.mjs) works without dependencies.\n`;
+  `StreamTrust ${VERSION}\n\nUsage:\n  streamtrust init [directory] [--preset <name>]\n  streamtrust init nextchat\n  streamtrust verify --adapter ./streamtrust/adapter.mjs [options]\n\nOptions:\n  --output <directory>    Reports (default: reports/streamtrust)\n  --repetitions <1-1000>   Repetitions per scenario (default: 3)\n  --timeout-ms <integer>  Total run deadline including setup/cleanup (default: 300000)\n  --help                 Show help\n  --version              Show version\n\nPresets: ${presets().join(', ')} (default: http)\n\nExit codes: 0 all pass; 1 failed/unsupported checks; 2 setup, timeout or harness error.\nAdapters execute trusted local code. JavaScript ESM (.mjs) works without dependencies.\n`;
 export async function main(args) {
   let outputDir;
   try {
@@ -38,7 +40,7 @@ export async function main(args) {
     if (args[0] === 'init') {
       const initArgs = args.slice(1);
       const presetIndex = initArgs.indexOf('--preset');
-      let preset = 'http';
+      let preset = initArgs.length === 1 && initArgs[0] === 'nextchat' ? 'nextchat' : 'http';
       if (presetIndex !== -1) {
         preset = initArgs[presetIndex + 1];
         initArgs.splice(presetIndex, 2);