streamtrust 0.1.0 → 0.2.0

package/README.md

@@ -1,8 +1,20 @@
 # StreamTrust CLI
 
+[![npm version](https://img.shields.io/npm/v/streamtrust.svg)](https://www.npmjs.com/package/streamtrust)
+
+**[View StreamTrust on npm](https://www.npmjs.com/package/streamtrust)**
+
 Adapter-driven CI checks for streaming applications: compare observed output with authoritative state, exercise cancellation, and verify retry identity.
 
-**Status:** `0.1.0`, an early 0.x release for independent teams. APIs and report formats may change before 1.0. The CLI is dependency-free, ESM, and requires Node.js 22+. MIT licensed. Browser and database libraries belong to your adapter and are not installed by StreamTrust.
+**Status:** `0.2.0`, an early 0.x release for independent teams. APIs and report formats may change before 1.0. The CLI is dependency-free, ESM, and requires Node.js 22+. MIT licensed. Browser and database libraries belong to your adapter and are not installed by StreamTrust.
+
+## Watch the 66-second demo
+
+[![Open the narrated StreamTrust demo: persistence, cancellation, retry, and CI diagnostics](docs/media/streamtrust-demo-poster.png)](https://santoshsibyala.github.io/streamtrust/)
+
+**[Watch the narrated demo in your browser](https://santoshsibyala.github.io/streamtrust/)** · [Read the transcript](docs/media/streamtrust-demo-transcript.txt) · [Download subtitles](docs/media/streamtrust-demo-0.1.1-brian.srt) · [Silent version](https://raw.githubusercontent.com/santoshsibyala/streamtrust/main/docs/media/streamtrust-demo-0.1.1.mp4)
+
+A 66-second walkthrough narrated with ElevenLabs’ Brian voice (synthetic English narration), on-screen captions, and an optional English subtitle track: install StreamTrust, run the example checks, inspect the diagnostic report, and see a deliberate save failure return a nonzero CI exit code. Recorded with version 0.1.1. The bundled application and injected failure are demonstrations; connect an adapter to your own application to verify its behavior. The browser player includes playback controls, captions, and an MP4 download option.
 
 ## Install
 
@@ -12,7 +24,13 @@ npx streamtrust init
 npx streamtrust verify --adapter ./streamtrust/adapter.mjs
 ```
 
-For reproducible CI, pin `streamtrust@0.1.0` and commit your lockfile. The generated adapter initially exercises the bundled demonstration; map it to your application before using its results as application evidence.
+For reproducible CI, pin `streamtrust@0.2.0` and commit your lockfile. The generated adapter initially exercises the bundled demonstration; map it to your application before using its results as application evidence.
+
+## LibreChat integration
+
+A LibreChat preset now exercises persistence, native Stop, regeneration and error-retry through its browser UI and disposable MongoDB. It uses reusable `streamtrust/checks` assertions and summary-only evidence; no chat text, screenshots or application logs are included in its reports.
+
+**Available in StreamTrust 0.2.0.** Install from npm using the [LibreChat quick start and CI guide](examples/librechat/README.md). It requires Node 24 and optional Playwright/MongoDB dependencies. An [integration workflow](.github/workflows/librechat.yml) exercises the packed CLI against the pinned upstream version. The default `init` demonstration remains available.
 
 ## Run from source
 
@@ -32,7 +50,7 @@ Build the tarball with `npm pack` in this repository.
 Put the supplied tarball in your project's `vendor/` directory, then:
 
 ```sh
-npm install --save-dev ./vendor/streamtrust-0.1.0.tgz
+npm install --save-dev ./vendor/streamtrust-0.2.0.tgz
 npx streamtrust init
 npx streamtrust verify --adapter ./streamtrust/adapter.mjs
 ```
@@ -123,34 +141,82 @@ node --import tsx ./node_modules/streamtrust/bin/streamtrust.js verify \
 
 ## CI
 
-Install the pinned version and commit package.json and package-lock.json so CI can use `npm ci`. If using a locally built tarball instead, keep it in `vendor/` so the committed file dependency can be resolved. The example workflow assumes the application under test is already running; add its start/wait commands before verification. Omit the base URL only when intentionally testing the bundled demonstration.
+Add StreamTrust as a test step in your existing CI. For GitHub Actions, follow these steps:
+
+1. Install StreamTrust and create the starting adapter:
+
+   ```sh
+   npm install --save-dev --save-exact streamtrust@0.2.0
+   npx streamtrust init
+   ```
+
+2. Adapt `streamtrust/adapter.mjs` to your application's native start, stream, cancel, retry, and saved-state operations. **The generated adapter initially tests a bundled demo. Setting a base URL alone does not adapt it to arbitrary application APIs.** See [Connect your application](#connect-your-application) above.
+3. Commit the adapter, `package.json`, and `package-lock.json`. If the adapter imports browser or database libraries, add those dependencies and their required CI setup too.
+4. Save the workflow below as `.github/workflows/streamtrust.yml`, or copy its relevant steps into an existing job.
+
+**Replace `npm run start:test`, port `3000`, and `/health` with your application's test-server command and readiness endpoint.** These are placeholders, not commands or routes supplied by StreamTrust. Add any required build, database, migration, fixture, or browser-install steps before starting your app. Use a disposable test environment because the checks create and modify application state.
 
 ```yaml
 name: Streaming application verification
 on: [push, pull_request]
+permissions:
+  contents: read
 jobs:
   verify:
     runs-on: ubuntu-latest
     timeout-minutes: 10
     steps:
       - uses: actions/checkout@v4
+
       - uses: actions/setup-node@v4
         with:
           node-version: 22
           cache: npm
+
       - run: npm ci
-      # Start your test application and wait for readiness here.
-      - run: npx --no-install streamtrust verify --adapter ./streamtrust/adapter.mjs --output reports/streamtrust
+
+      # Add your application's build and test-service setup here.
+      - name: Start application and wait for readiness
+        shell: bash
+        run: |
+          npm run start:test > app.log 2>&1 &
+          app_pid=$!
+          for attempt in {1..30}; do
+            if ! kill -0 "$app_pid" 2>/dev/null; then
+              cat app.log
+              exit 1
+            fi
+            if curl --fail --silent --max-time 2 http://localhost:3000/health; then
+              exit 0
+            fi
+            sleep 2
+          done
+          cat app.log
+          echo "Application did not become ready in time."
+          exit 1
+
+      - name: Verify streaming behavior
+        run: |
+          npx --no-install streamtrust verify \
+            --adapter ./streamtrust/adapter.mjs \
+            --output ./reports/streamtrust
         env:
-          STREAMTRUST_BASE_URL: http://127.0.0.1:8080
-          STREAMTRUST_API_TOKEN: ${{ secrets.TEST_APP_TOKEN }}
-      - uses: actions/upload-artifact@v4
+          STREAMTRUST_BASE_URL: http://localhost:3000
+
+      - name: Save diagnostic reports
         if: always()
+        uses: actions/upload-artifact@v4
         with:
-          name: streamtrust-evidence
+          name: streamtrust-reports
           path: reports/streamtrust/
 ```
 
+If your application is already running in the job, omit the startup step and point your adapter at its test URL. For the bundled HTTP adapter's optional bearer authentication, add `STREAMTRUST_API_TOKEN: ${{ secrets.TEST_APP_TOKEN }}` to the verification step's `env`, and configure that secret in GitHub. Fork pull requests generally do not receive repository secrets; use a suitable unauthenticated test fixture or an explicitly trusted workflow for those runs. Never put credentials in the adapter or uploaded evidence.
+
+A failed assertion or harness error fails the verification step. The report-upload step runs even after a failure, so teams can inspect diagnostics when they were generated; startup failures may have no StreamTrust report. To block merging, configure the repository's branch protection or ruleset to require this workflow's `verify` check. StreamTrust does not configure merge rules itself.
+
+If you use a locally built tarball instead of npm, keep it in `vendor/` and commit the corresponding file dependency and lockfile so `npm ci` can resolve it.
+
 | Exit | Meaning |
 |---|---|
 | 0 | Every declared scenario and assertion passed. |
@@ -163,6 +229,6 @@ Use a unique output directory per concurrent job. The CLI takes an exclusive loc
 
 ## Scope and attribution
 
-StreamTrust checks what an adapter asserts. It does not prove exactly-once external effects, AI factuality, automatic recovery, production incident rates or adoption. The HTTP demo is synthetic. Separate development experiments have tested pinned Vercel Chatbot and LibreChat applications; their adapters and dependencies are not bundled here.
+StreamTrust checks what an adapter asserts. It does not prove exactly-once external effects, AI factuality, automatic recovery, production incident rates or adoption. The HTTP demo is synthetic. Separate development experiments have tested pinned Vercel Chatbot and LibreChat applications; the LibreChat adapter is now available as an npm preset with optional dependencies. The Vercel experiment remains separate.
 
 Local preparation: `npm pack` in this package directory runs the package tests and creates the tarball. The public repository runs tests and a clean tarball installation on Node.js 22 and 24. This release is published interactively; npm build provenance is not configured.

package/examples/librechat/README.md

@@ -0,0 +1,91 @@
+# LibreChat CI integration
+
+This preset runs a **built local LibreChat checkout**, native browser UI and real disposable MongoDB. A controlled OpenAI-compatible HTTP provider supplies synthetic text. No live model key or existing database is used. Four scenarios cover saved answers, Stop, regeneration branches, and retry after a provider disconnect. This is not an official LibreChat integration.
+
+## Installation
+
+Use Node.js 24 and npm 11. In a separate verification project, install StreamTrust 0.2.0 from npm:
+
+```sh
+npm init -y
+npm install --save-dev --save-exact streamtrust@0.2.0
+npm install --save-dev --save-exact playwright@1.64.1 mongodb@6.20.0 mongodb-memory-server@11.3.0
+npx playwright install chromium
+npx streamtrust init streamtrust-librechat --preset librechat
+```
+
+Use Node.js 24 and npm 11 for this preset. Commit the generated adapter and your package lock. The core CLI still has no runtime dependencies; browser/database dependencies belong to the verification project. Browser, MongoDB binary and LibreChat installation require network access and disk space. The first build takes several minutes; this is not a five-minute cold installation.
+
+## Run the supported upstream version
+
+```sh
+export STREAMTRUST_LIBRECHAT_ROOT="$PWD/.cache/librechat"
+node streamtrust-librechat/setup.mjs
+npx streamtrust verify --adapter streamtrust-librechat/adapter.mjs \
+  --output reports/librechat --repetitions 3 --timeout-ms 600000
+```
+
+The setup script clones and builds commit `a378657583567310c841d652b0f6b00c83e2bd6b`. It refuses to reset an existing or dirty checkout. The verifier starts its own loopback-only server, chooses an available port, creates a synthetic test user, and tears down its browser, provider and disposable database. The preset refuses checkouts containing `.env`, `.env.local` or `.env.production`. Do not place production configuration in this checkout.
+
+To test your team's committed changes, point `STREAMTRUST_LIBRECHAT_ROOT` at your **built, clean checkout** and set `STREAMTRUST_LIBRECHAT_COMMIT` to its full SHA. Skip setup if already built. This executes the actual application under test; it does not copy a test app into your repository. Compatibility with other commits is not guaranteed: changed selectors/routes/schema can cause HARNESS_ERROR rather than a product failure.
+
+Optional settings:
+
+| Variable | Purpose |
+|---|---|
+| `STREAMTRUST_LIBRECHAT_ROOT` | Required path to the built checkout |
+| `STREAMTRUST_LIBRECHAT_COMMIT` | Expected full SHA; defaults to the tested upstream commit |
+| `STREAMTRUST_LIBRECHAT_PORT` | Optional fixed loopback port; defaults to an ephemeral port |
+| `STREAMTRUST_BROWSER_CHANNEL` | Optional installed browser channel, such as `chrome`; default is Playwright Chromium |
+
+## CI
+
+In your verification project, install once and commit package.json/package-lock.json and the generated preset. Add these steps to a Node 24 GitHub Actions job:
+
+```yaml
+- uses: actions/checkout@v4
+- uses: actions/setup-node@v4
+  with:
+    node-version: '24'
+    cache: npm
+- run: npm ci
+- run: npx --no-install playwright install --with-deps chromium
+- name: Build isolated LibreChat checkout
+  run: node streamtrust-librechat/setup.mjs
+  env:
+    STREAMTRUST_LIBRECHAT_ROOT: ${{ runner.temp }}/librechat
+- name: Verify LibreChat
+  run: npx --no-install streamtrust verify --adapter streamtrust-librechat/adapter.mjs --output reports/librechat --repetitions 3 --timeout-ms 600000
+  env:
+    STREAMTRUST_LIBRECHAT_ROOT: ${{ runner.temp }}/librechat
+- uses: actions/upload-artifact@v4
+  if: always()
+  with:
+    name: librechat-verification
+    path: |
+      reports/librechat/report.json
+      reports/librechat/report.md
+      reports/librechat/junit.xml
+      reports/librechat/index.html
+    retention-days: 7
+```
+
+Use a fresh report directory for each job. Restrict workflow permissions to `contents: read`, set a 30-minute job timeout, and use normal pull-request workflows without production secrets. To verify your fork, replace the build step with checkout/build of the intended commit and pass its SHA to the verifier.
+
+Exit 0 means all declared checks passed; 1 means failed or unsupported checks; 2 means setup, timeout, or harness error. Keep the exit status: do not use `continue-on-error`. Inspect reports before calling a failure an upstream defect.
+
+## Evidence and limits
+
+`summary-only-v1` returns stable check identifiers, booleans, message counts and provider-call counts. Raw chat text, database records, user identifiers, network URLs and tokens remain in memory and are not returned as report evidence. The preset does not write screenshots, browser traces or application logs. Browser/database exceptions are replaced with fixed errors. App subprocess output is discarded. Upload only the four report files, not the entire working directory.
+
+This is data minimization, not a universal PII detector or sandbox. Other adapters, changes to this adapter, setup/build logs and third-party processes are outside that guarantee. The core verifier still runs trusted local code. The preset deliberately does not forward application environment variables, but LibreChat can load files in its checkout: use an isolated checkout with no production configuration.
+
+The model provider is synthetic. The test uses one server and an in-memory stream store: no Redis, real model behavior, database crash, worker kill, multi-replica or production incident-rate coverage. A single matching saved answer is not proof of exactly-once external side effects. Regeneration is a distinct sibling branch, not reuse of a request idempotency key.
+
+The exported `streamtrust/checks` evaluator can be reused by other adapters. Map observations to `LifecycleObservation`; the evaluator validates required types, computes checks, and returns summary-only evidence. Missing or malformed observations throw, becoming HARNESS_ERROR rather than a green result.
+
+## Validation
+
+Local validation used macOS arm64, Node 24.22.1, Playwright 1.64.1 Chromium 156.0.8078.4, MongoDB 8.2.6, and the pinned LibreChat commit. A clean tarball consumer passed all four scenarios with three repetitions (12 executions, 99 assertions). A checkout containing an environment file was rejected with exit 2. Success and error artifact checks found only the four intended files and no tested secret/chat-text markers. The package suite has 13 tests, including negative controls for shared checks and stale-screenshot exclusion.
+
+The [Linux GitHub Actions run](https://github.com/santoshsibyala/streamtrust/actions/runs/40451851475) also passed a cold clone/install/build, clean tarball installation, all 12 scenario executions, and upload of exactly the four report files. The verified implementation commit is `b1b2b27a19b512c6abce164502901d1716bf86a3`. Passing results do not establish an upstream defect or production reliability.

package/examples/librechat/adapter.mjs

@@ -0,0 +1,515 @@
+import { existsSync } from 'node:fs';
+import { evaluateLifecycle } from 'streamtrust/checks';
+import { createRequire } from 'node:module';
+import { spawn, execFileSync } from 'node:child_process';
+import { mkdtemp, rm } from 'node:fs/promises';
+import { tmpdir } from 'node:os';
+import { createServer } from 'node:net';
+import { readFile, writeFile } from 'node:fs/promises';
+import { resolve, dirname } from 'node:path';
+import { createHash, randomBytes } from 'node:crypto';
+
+import { startProvider } from './provider.mjs';
+
+const PIN = 'a378657583567310c841d652b0f6b00c83e2bd6b';
+export async function until(check, label, ms = 20000) {
+  const end = Date.now() + ms;
+  while (Date.now() < end) {
+    if (await check()) return;
+    await new Promise((r) => setTimeout(r, 100));
+  }
+  throw new Error(`Timeout: ${label}`);
+}
+let phase = 'preflight';
+async function createAdapter() {
+  if (!process.env.STREAMTRUST_LIBRECHAT_ROOT)
+    throw new Error(
+      'Set STREAMTRUST_LIBRECHAT_ROOT to a built LibreChat checkout',
+    );
+  if (Number(process.versions.node.split('.')[0]) < 24)
+    throw new Error('LibreChat preset requires Node.js 24+');
+  const root = resolve(process.env.STREAMTRUST_LIBRECHAT_ROOT);
+  if (
+    ['.env', '.env.local', '.env.production'].some((name) =>
+      existsSync(resolve(root, name)),
+    )
+  )
+    throw new Error('Use an isolated checkout without environment files');
+  const expectedCommit = process.env.STREAMTRUST_LIBRECHAT_COMMIT || PIN;
+  if (!/^[a-f0-9]{40}$/.test(expectedCommit))
+    throw new Error('Expected commit must be a full Git SHA');
+  const configPort = Number(process.env.STREAMTRUST_LIBRECHAT_PORT || 0);
+  if (!Number.isInteger(configPort) || configPort < 0 || configPort > 65535)
+    throw new Error('Invalid LibreChat port');
+  const port =
+    configPort ||
+    (await new Promise((ok, fail) => {
+      const server = createServer();
+      server.once('error', fail);
+      server.listen(0, '127.0.0.1', () => {
+        const value = server.address().port;
+        server.close(() => ok(value));
+      });
+    }));
+  const baseUrl = `http://127.0.0.1:${port}`;
+  const commit = execFileSync('git', ['rev-parse', 'HEAD'], {
+    cwd: root,
+    encoding: 'utf8',
+  }).trim();
+  const dirty = () =>
+    execFileSync('git', ['status', '--porcelain', '--untracked-files=no'], {
+      cwd: root,
+      encoding: 'utf8',
+    }).trim();
+  if (commit !== expectedCommit || dirty())
+    throw new Error('Expected clean pinned LibreChat checkout');
+  const require = createRequire(resolve(process.cwd(), 'package.json'));
+  const { chromium } = require('playwright');
+  const { MongoMemoryServer } = require('mongodb-memory-server');
+  const { MongoClient } = require('mongodb');
+  const cleanups = [];
+  let cleaned = false;
+  const cleanup = async () => {
+    if (cleaned) return;
+    cleaned = true;
+    let cleanupFailed = false;
+    for (const close of cleanups.reverse()) {
+      try {
+        await close();
+      } catch {
+        cleanupFailed = true;
+      }
+    }
+    if (cleanupFailed) throw new Error('LibreChat cleanup failed');
+  };
+  try {
+    phase = 'disposable database startup';
+    const mongo = await MongoMemoryServer.create({
+      instance: { ip: '127.0.0.1', dbName: 'streamtrust' },
+    });
+    cleanups.push(() => mongo.stop());
+    const client = new MongoClient(mongo.getUri());
+    cleanups.push(() => client.close());
+    await client.connect();
+    const db = client.db('streamtrust');
+    const provider = await startProvider();
+    cleanups.push(() => provider.close());
+    const temporary = await mkdtemp(
+      resolve(tmpdir(), 'streamtrust-librechat-'),
+    );
+    cleanups.unshift(() => rm(temporary, { recursive: true, force: true }));
+    const config = resolve(temporary, 'librechat.yaml');
+    await writeFile(
+      config,
+      `version: 1.3.11\ncache: true\nendpoints:\n  custom:\n    - name: StreamTrust Fixture\n      apiKey: fixture-not-a-real-key\n      baseURL: ${provider.url}\n      models:\n        default: [fixture-model]\n        fetch: false\n      titleConvo: false\n      modelDisplayLabel: Fixture\n`,
+    );
+    const runtime = process.execPath;
+    const env = {
+      // Do not forward provider keys, database URLs or application secrets.
+      ...Object.fromEntries(
+        ['PATH', 'HOME', 'TMPDIR', 'TEMP', 'SystemRoot']
+          .filter((k) => process.env[k])
+          .map((k) => [k, process.env[k]]),
+      ),
+      PATH: `${dirname(runtime)}:${process.env.PATH}`,
+      NODE_ENV: 'production',
+      HOST: '127.0.0.1',
+      PORT: String(port),
+      DOMAIN_CLIENT: baseUrl,
+      DOMAIN_SERVER: baseUrl,
+      MONGO_URI: mongo.getUri('streamtrust'),
+      CONFIG_PATH: config,
+      ENDPOINTS: 'custom',
+      OPENAI_API_KEY: '',
+      SEARCH: 'false',
+      NO_INDEX: 'true',
+      ALLOW_REGISTRATION: 'true',
+      ALLOW_SOCIAL_LOGIN: 'false',
+      ALLOW_EMAIL_LOGIN: 'true',
+      JWT_SECRET: randomBytes(32).toString('hex'),
+      JWT_REFRESH_SECRET: randomBytes(32).toString('hex'),
+      CREDS_KEY: randomBytes(32).toString('hex'),
+      CREDS_IV: randomBytes(16).toString('hex'),
+      SCHEDULES_SINGLE_PROCESS: 'true',
+      USE_REDIS: 'false',
+      USE_REDIS_STREAMS: 'false',
+      REGISTER_MAX: '100',
+      LOGIN_MAX: '100',
+    };
+    phase = 'application startup';
+    const app = spawn(runtime, ['api/server/index.js'], {
+      cwd: root,
+      env,
+      stdio: ['ignore', 'ignore', 'ignore'],
+    });
+    let appError = false;
+    app.on('error', () => {
+      appError = true;
+    });
+    cleanups.push(async () => {
+      if (!appError && app.exitCode === null && app.signalCode === null) {
+        const stopped = new Promise((r) => app.once('exit', () => r()));
+        app.kill('SIGTERM');
+        const timer = setTimeout(() => app.kill('SIGKILL'), 10000);
+        await stopped;
+        clearTimeout(timer);
+      }
+    });
+    phase = 'browser startup';
+    const browser = await chromium.launch(
+      !process.env.STREAMTRUST_BROWSER_CHANNEL
+        ? { headless: true }
+        : { channel: process.env.STREAMTRUST_BROWSER_CHANNEL, headless: true },
+    );
+    cleanups.push(() => browser.close());
+    phase = 'application readiness';
+    await until(
+      async () => {
+        if (appError || app.exitCode !== null)
+          throw new Error(
+            'LibreChat startup failed; check the build and port configuration',
+          );
+        try {
+          return (await fetch(baseUrl, { signal: AbortSignal.timeout(2000) }))
+            .ok;
+        } catch {
+          return false;
+        }
+      },
+      'app ready',
+      120000,
+    );
+    phase = 'synthetic user registration';
+    const email = `verifier-${Date.now()}@example.com`,
+      password = `Fixture-${randomBytes(12).toString('hex')}!`;
+    const registration = await browser.newContext();
+    const page = await registration.newPage();
+    await page.goto(`${baseUrl}/register`);
+    await page.getByLabel('Full name').fill('StreamTrust Verifier');
+    await page.getByLabel('Email', { exact: true }).fill(email);
+    await page.getByTestId('password').fill(password);
+    await page.getByTestId('confirm_password').fill(password);
+    await page.getByLabel('Submit registration').click();
+    await page.waitForURL('**/login**');
+    await page.getByLabel('Email', { exact: true }).fill(email);
+    await page.getByLabel('Password', { exact: true }).fill(password);
+    await page.getByTestId('login-button').click();
+    await page
+      .getByRole('textbox', { name: 'Message input' })
+      .waitFor({ timeout: 30000 });
+    await page.close();
+    async function messages(id) {
+      return db
+        .collection('messages')
+        .find(
+          { conversationId: id },
+          {
+            projection: {
+              _id: 0,
+              messageId: 1,
+              parentMessageId: 1,
+              conversationId: 1,
+              text: 1,
+              content: 1,
+              isCreatedByUser: 1,
+              error: 1,
+              unfinished: 1,
+            },
+          },
+        )
+        .toArray();
+    }
+    const textOf = (m) =>
+      m.text ||
+      (m.content ?? [])
+        .filter((p) => p.type === 'text')
+        .map((p) =>
+          typeof p.text === 'string' ? p.text : (p.text?.value ?? ''),
+        )
+        .join('');
+    const hasError = (m) =>
+      (m.content ?? []).some((part) => part.type === 'error');
+    const observed = async (check) => {
+      const deadline = Date.now() + 15000;
+      while (Date.now() < deadline) {
+        if (await check()) return true;
+        await new Promise((r) => setTimeout(r, 100));
+      }
+      return false;
+    };
+    const visible = (p) => p.getByTestId('messages-view').innerText();
+    async function waitGeneration(p, index) {
+      await until(
+        () => provider.generations.length > index,
+        'provider request',
+      );
+      const g = provider.generations[index];
+      await until(
+        async () => (await visible(p)).includes(g.prefix),
+        'first rendered chunk',
+      );
+      return g;
+    }
+    return {
+      name: 'LibreChat',
+      scenarios: ['persistence', 'cancellation', 'regeneration', 'error-retry'],
+      async metadata() {
+        return {
+          evidencePolicy: 'summary-only-v1',
+          checkVersion: 1,
+          mongodb: (await db.admin().command({ buildInfo: 1 })).version,
+          upstream: 'https://github.com/LibreChat-AI/LibreChat',
+          commit,
+          trackedSourceModified: !!dirty(),
+          node: execFileSync(runtime, ['--version'], {
+            encoding: 'utf8',
+          }).trim(),
+          browser: browser.version(),
+          lockSha256: createHash('sha256')
+            .update(await readFile(resolve(root, 'package-lock.json')))
+            .digest('hex'),
+          scope:
+            'Real native UI, authentication, application routes, generation jobs and MongoDB. Only external OpenAI-compatible provider responses are synthetic. Single server, in-memory stream store; no Redis, worker-kill, database-crash or multi-replica coverage. Regeneration is a branch, not overwriting history. No coordinator installed.',
+        };
+      },
+      async run(scenario, { repetition }) {
+        const context = registration;
+        const p = await context.newPage();
+        const network = [];
+
+        p.on('response', (r) => {
+          const path = new URL(r.url()).pathname;
+          if (path.startsWith('/api/') && /chat|abort|message/.test(path))
+            network.push({
+              path,
+              method: r.request().method(),
+              status: r.status(),
+            });
+        });
+        let first;
+        let second;
+        try {
+          await p.goto(`${baseUrl}/c/new`);
+          await p.getByRole('textbox', { name: 'Message input' }).waitFor();
+          const selector = p
+            .getByRole('button', { name: 'Select a model' })
+            .first();
+          if (
+            (await selector.count()) &&
+            !(await selector.innerText()).includes('fixture-model')
+          ) {
+            await selector.click();
+            await p
+              .getByRole('option', { name: 'StreamTrust Fixture', exact: true })
+              .click();
+            const model = p.getByRole('option', {
+              name: 'fixture-model',
+              exact: true,
+            });
+            if (await model.isVisible().catch(() => false)) await model.click();
+          }
+          const before = provider.generations.length;
+          await p
+            .getByRole('textbox', { name: 'Message input' })
+            .fill(
+              `StreamTrust ${scenario} ${repetition}. Write two sentences.`,
+            );
+          const admission = p.waitForResponse(
+            (r) =>
+              r.request().method() === 'POST' &&
+              new URL(r.url()).pathname.startsWith('/api/agents/chat') &&
+              !r.url().endsWith('/abort'),
+          );
+          await p
+            .getByRole('textbox', { name: 'Message input' })
+            .press('Enter');
+          first = await waitGeneration(p, before);
+          const accepted = await (await admission).json();
+          const id = accepted.conversationId;
+          if (!id || id === 'new')
+            throw new Error('Missing native admission conversation identity');
+          const checks = [];
+          let expected = first.prefix + first.suffix;
+          if (scenario === 'cancellation') {
+            await p
+              .getByRole('button', { name: 'Stop generating', exact: true })
+              .click();
+            await observed(() => first.closed);
+            expected = first.prefix;
+            checks.push({
+              name: 'Native server acknowledges Stop',
+              passed: await observed(() =>
+                network.some(
+                  (e) =>
+                    e.method === 'POST' &&
+                    e.path.endsWith('/abort') &&
+                    e.status === 200,
+                ),
+              ),
+            });
+            checks.push({
+              name: 'Stop aborts upstream HTTP response before completion',
+              passed: first.closed && !first.finished,
+            });
+            checks.push({
+              name: 'Late provider completion cannot be delivered',
+              passed: !provider.finish(first),
+            });
+          } else if (scenario === 'error-retry') {
+            provider.fail(first);
+          } else provider.finish(first);
+          await until(
+            async () =>
+              !(await p
+                .getByRole('button', { name: 'Stop generating', exact: true })
+                .count()),
+            'generation settles',
+          );
+
+          if (scenario === 'error-retry') {
+            const errorSaved = await observed(async () =>
+              (await messages(id)).some(hasError),
+            );
+
+            checks.push({
+              name: 'Provider failure retained as a native error part',
+              passed: errorSaved,
+            });
+            await p.reload();
+            await p.getByRole('textbox', { name: 'Message input' }).waitFor();
+            const errorVisible = await observed(async () =>
+              (await visible(p)).includes(
+                'model provider closed the connection',
+              ),
+            );
+            checks.push({
+              name: 'Provider error context survives reload before retry',
+              passed: errorVisible,
+            });
+          }
+          if (scenario === 'regeneration' || scenario === 'error-retry') {
+            const nextIndex = provider.generations.length;
+            await p
+              .getByRole('button', { name: 'Regenerate', exact: true })
+              .last()
+              .click();
+            second = await waitGeneration(p, nextIndex);
+            provider.finish(second);
+            expected = second.prefix + second.suffix;
+            await until(
+              async () =>
+                !(await p
+                  .getByRole('button', { name: 'Stop generating', exact: true })
+                  .count()),
+              'retry settles',
+            );
+          }
+          await observed(async () =>
+            (await messages(id)).some(
+              (m) => !m.isCreatedByUser && textOf(m).trim() === expected.trim(),
+            ),
+          );
+          const durable = await messages(id);
+          const beforeReload = await visible(p);
+          await p.reload();
+          await p.getByRole('textbox', { name: 'Message input' }).waitFor();
+          await observed(async () => (await visible(p)).includes(expected));
+          const reloaded = await visible(p);
+          return evaluateLifecycle({
+            scenario,
+            expected,
+            original: first.prefix + first.suffix,
+            previousPrefix: first.prefix,
+            beforeReload,
+            reloaded,
+            messages: durable
+              .filter((m) => !m.isCreatedByUser)
+              .map((m) => ({
+                id: m.messageId,
+                parentId: m.parentMessageId,
+                text: textOf(m),
+                assistant: !m.isCreatedByUser,
+                unfinished: m.unfinished,
+                error: !!m.error || hasError(m),
+              })),
+            callsBeforeReload: second ? second.id : first.id,
+            callsAfterReload: provider.generations.length,
+            cancellation:
+              scenario === 'cancellation'
+                ? {
+                    acknowledged: checks.find(
+                      (c) => c.name === 'Native server acknowledges Stop',
+                    ).passed,
+                    providerClosed: first.closed,
+                    providerFinished: first.finished,
+                    lateCompletionRejected: checks.find(
+                      (c) =>
+                        c.name ===
+                        'Late provider completion cannot be delivered',
+                    ).passed,
+                  }
+                : undefined,
+            failure:
+              scenario === 'error-retry'
+                ? {
+                    saved: checks.find(
+                      (c) =>
+                        c.name ===
+                        'Provider failure retained as a native error part',
+                    ).passed,
+                    visibleAfterReload: checks.find(
+                      (c) =>
+                        c.name ===
+                        'Provider error context survives reload before retry',
+                    ).passed,
+                  }
+                : undefined,
+          });
+        } catch {
+          // Browser/DB exceptions can contain prompts, selectors, URLs and credentials.
+          throw new Error(
+            'LibreChat scenario could not complete (details withheld by summary-only policy)',
+          );
+        } finally {
+          if (first && !first.closed) provider.finish(first);
+          if (second && !second.closed) provider.finish(second);
+          await p.close();
+        }
+      },
+      close: cleanup,
+    };
+  } catch (error) {
+    await cleanup();
+    throw error;
+  }
+}
+
+export default async function safeFactory(context) {
+  try {
+    phase = 'preflight';
+    const adapter = await createAdapter();
+    const run = adapter.run.bind(adapter);
+    adapter.run = async (...args) => {
+      try {
+        return await run(...args);
+      } catch {
+        throw new Error(
+          'LibreChat scenario could not complete (details withheld by summary-only policy)',
+        );
+      }
+    };
+    const metadata = adapter.metadata.bind(adapter);
+    adapter.metadata = async () => {
+      try {
+        return await metadata();
+      } catch {
+        throw new Error('LibreChat metadata unavailable (details withheld)');
+      }
+    };
+    return adapter;
+  } catch {
+    throw new Error(
+      `LibreChat setup failed during ${phase}. Check Node 24+, optional dependencies, built clean checkout without .env files, expected commit, and port. Raw details withheld.`,
+    );
+  }
+}

package/examples/librechat/provider.mjs

@@ -0,0 +1,79 @@
+import { createServer } from 'node:http';
+
+export async function startProvider() {
+  const generations = [];
+  const chunk = (g, delta, finish = null) =>
+    g.response.write(
+      `data: ${JSON.stringify({ id: `chatcmpl-fixture-${g.id}`, object: 'chat.completion.chunk', created: 1, model: 'fixture-model', choices: [{ index: 0, delta: { content: delta }, finish_reason: finish }] })}\n\n`,
+    );
+  const server = createServer(async (req, res) => {
+    let body = '';
+    for await (const part of req) body += part;
+    if (req.method !== 'POST') {
+      res.setHeader('content-type', 'application/json');
+      res.end(
+        JSON.stringify({ data: [{ id: 'fixture-model', object: 'model' }] }),
+      );
+      return;
+    }
+    const input = JSON.parse(body);
+    if (!input.stream) {
+      res.setHeader('content-type', 'application/json');
+      res.end(
+        JSON.stringify({
+          choices: [
+            {
+              message: { role: 'assistant', content: 'Fixture conversation' },
+              finish_reason: 'stop',
+            },
+          ],
+        }),
+      );
+      return;
+    }
+    const id = generations.length + 1;
+    const g = {
+      id,
+      prefix: `Response ${id} begins.`,
+      suffix: ` Response ${id} completes.`,
+      closed: false,
+      finished: false,
+      failed: false,
+      path: req.url,
+      response: res,
+    };
+    generations.push(g);
+    res.on('close', () => {
+      g.closed = true;
+    });
+    res.writeHead(200, {
+      'content-type': 'text/event-stream',
+      'cache-control': 'no-cache',
+    });
+    chunk(g, g.prefix);
+  });
+  await new Promise((r) => server.listen(0, '127.0.0.1', r));
+  return {
+    url: `http://127.0.0.1:${server.address().port}/v1`,
+    generations,
+    finish(g) {
+      if (g.closed) return false;
+      chunk(g, g.suffix);
+      chunk(g, '', 'stop');
+      g.finished = true;
+      g.response.end('data: [DONE]\n\n');
+      return true;
+    },
+    fail(g) {
+      g.failed = true;
+      g.response.destroy();
+    },
+    observations() {
+      return generations.map(({ response, ...g }) => g);
+    },
+    async close() {
+      server.closeAllConnections();
+      await new Promise((r) => server.close(() => r()));
+    },
+  };
+}

package/examples/librechat/setup.mjs

@@ -0,0 +1,42 @@
+import { execFileSync } from 'node:child_process';
+import { existsSync } from 'node:fs';
+import { resolve } from 'node:path';
+const root = process.env.STREAMTRUST_LIBRECHAT_ROOT;
+const pin =
+  process.env.STREAMTRUST_LIBRECHAT_COMMIT ||
+  'a378657583567310c841d652b0f6b00c83e2bd6b';
+if (
+  !root ||
+  Number(process.versions.node.split('.')[0]) < 24 ||
+  !/^[a-f0-9]{40}$/.test(pin)
+)
+  throw new Error(
+    'Set STREAMTRUST_LIBRECHAT_ROOT; use Node 24+ and a full commit SHA',
+  );
+const path = resolve(root);
+const run = (cmd, args, cwd) =>
+  execFileSync(cmd, args, { cwd, stdio: 'inherit' });
+if (!existsSync(path)) {
+  run('git', [
+    'clone',
+    '--no-checkout',
+    'https://github.com/LibreChat-AI/LibreChat.git',
+    path,
+  ]);
+  run('git', ['checkout', '--detach', pin], path);
+}
+if (
+  execFileSync('git', ['rev-parse', 'HEAD'], {
+    cwd: path,
+    encoding: 'utf8',
+  }).trim() !== pin ||
+  execFileSync('git', ['status', '--porcelain', '--untracked-files=no'], {
+    cwd: path,
+    encoding: 'utf8',
+  }).trim()
+)
+  throw new Error(
+    'Checkout must be clean and match the expected commit; setup never resets existing source',
+  );
+run('npm', ['ci', '--no-audit', '--no-fund'], path);
+run('npm', ['run', 'frontend'], path);

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "streamtrust",
-  "version": "0.1.0",
+  "version": "0.2.0",
   "description": "Adapter-driven CI verification for streaming application lifecycle guarantees",
   "type": "module",
   "license": "MIT",
@@ -14,6 +14,10 @@
     ".": {
       "types": "./src/index.d.ts",
       "import": "./src/index.js"
+    },
+    "./checks": {
+      "types": "./src/checks.d.ts",
+      "import": "./src/checks.js"
     }
   },
   "types": "src/index.d.ts",

package/src/checks.d.ts

@@ -0,0 +1,36 @@
+import type { ExternalObservation } from './index.js';
+export type LifecycleScenario =
+  | 'persistence'
+  | 'cancellation'
+  | 'regeneration'
+  | 'error-retry';
+export const LIFECYCLE_SCENARIOS: readonly LifecycleScenario[];
+export interface LifecycleObservation {
+  scenario: LifecycleScenario;
+  expected: string;
+  original: string;
+  previousPrefix: string;
+  beforeReload: string;
+  reloaded: string;
+  messages: Array<{
+    id: string;
+    parentId: string;
+    text: string;
+    assistant: boolean;
+    unfinished: boolean;
+    error: boolean;
+  }>;
+  callsBeforeReload: number;
+  callsAfterReload: number;
+  cancellation?: {
+    acknowledged: boolean;
+    providerClosed: boolean;
+    providerFinished: boolean;
+    lateCompletionRejected: boolean;
+  };
+  failure?: { saved: boolean; visibleAfterReload: boolean };
+}
+/** Evaluate in-memory observations; returns only fixed check names and allowlisted numeric evidence. */
+export function evaluateLifecycle(
+  observation: LifecycleObservation,
+): ExternalObservation;

package/src/checks.js

@@ -0,0 +1,105 @@
+// Raw observations remain in memory. Returned evidence is deliberately allowlisted.
+export const LIFECYCLE_SCENARIOS = Object.freeze([
+  'persistence',
+  'cancellation',
+  'regeneration',
+  'error-retry',
+]);
+export function evaluateLifecycle(o) {
+  const text = (v) => typeof v === 'string' && v.length > 0;
+  const bool = (v) => typeof v === 'boolean';
+  if (
+    !o ||
+    !LIFECYCLE_SCENARIOS.includes(o.scenario) ||
+    !text(o.expected) ||
+    !text(o.original) ||
+    !text(o.previousPrefix) ||
+    typeof o.beforeReload !== 'string' ||
+    typeof o.reloaded !== 'string' ||
+    !Array.isArray(o.messages) ||
+    o.messages.some(
+      (m) =>
+        !m ||
+        !text(m.id) ||
+        typeof m.parentId !== 'string' ||
+        typeof m.text !== 'string' ||
+        !bool(m.assistant) ||
+        !bool(m.unfinished) ||
+        !bool(m.error),
+    ) ||
+    !Number.isSafeInteger(o.callsBeforeReload) ||
+    o.callsBeforeReload < 1 ||
+    !Number.isSafeInteger(o.callsAfterReload) ||
+    o.callsAfterReload < 0
+  )
+    throw new Error('Invalid lifecycle observation');
+  const checks = [];
+  const check = (name, passed) => checks.push({ name, passed });
+  const matching = o.messages.filter(
+    (m) => m.assistant && m.text.trim() === o.expected.trim(),
+  );
+  check('saved.answer-present', matching.length > 0);
+  check('saved.single-matching-answer', matching.length === 1);
+  check('ui.answer-visible', o.beforeReload.includes(o.expected));
+  check(
+    'saved.lifecycle-state',
+    matching.length === 1 &&
+      (o.scenario === 'cancellation'
+        ? matching[0].unfinished === true
+        : matching[0].unfinished === false && !matching[0].error),
+  );
+  check('reload.answer-preserved', o.reloaded.includes(o.expected));
+  check('reload.no-new-generation', o.callsAfterReload === o.callsBeforeReload);
+  if (o.scenario === 'cancellation') {
+    const c = o.cancellation;
+    if (
+      !c ||
+      [
+        'acknowledged',
+        'providerClosed',
+        'providerFinished',
+        'lateCompletionRejected',
+      ].some((k) => !bool(c[k]))
+    )
+      throw new Error('Invalid cancellation observation');
+    check('cancel.server-acknowledged', c.acknowledged);
+    check('cancel.provider-aborted', c.providerClosed && !c.providerFinished);
+    check('cancel.late-completion-rejected', c.lateCompletionRejected);
+  }
+  if (o.scenario === 'regeneration') {
+    const original = o.messages.filter(
+      (m) => m.assistant && m.text === o.original,
+    );
+    check('retry.original-branch-preserved', original.length === 1);
+    check(
+      'retry.distinct-sibling',
+      original.length === 1 &&
+        matching.length === 1 &&
+        original[0].id !== matching[0].id &&
+        original[0].parentId === matching[0].parentId,
+    );
+  }
+  if (o.scenario === 'error-retry') {
+    const f = o.failure;
+    if (!f || !bool(f.saved) || !bool(f.visibleAfterReload))
+      throw new Error('Invalid failure observation');
+    check('error.saved-before-retry', f.saved);
+    check('error.visible-after-reload', f.visibleAfterReload);
+  }
+  if (['regeneration', 'error-retry'].includes(o.scenario))
+    check(
+      'retry.previous-answer-not-active',
+      !o.reloaded.includes(o.previousPrefix),
+    );
+  return {
+    checks,
+    evidence: {
+      policy: 'summary-only-v1',
+      checkVersion: 1,
+      savedMessageCount: o.messages.length,
+      matchingAnswerCount: matching.length,
+      providerCallsBeforeReload: o.callsBeforeReload,
+      providerCallsAfterReload: o.callsAfterReload,
+    },
+  };
+}

package/src/cli.js

@@ -5,7 +5,7 @@ import { fileURLToPath } from 'node:url';
 import { mkdir, copyFile, open, unlink, readFile } from 'node:fs/promises';
 import { constants } from 'node:fs';
 import { VERSION, failureReport } from './index.js';
-const HELP = `StreamTrust ${VERSION}\n\nUsage:\n  streamtrust init [directory]\n  streamtrust verify --adapter ./streamtrust/adapter.mjs [options]\n\nOptions:\n  --output <directory>    Reports (default: reports/streamtrust)\n  --repetitions <1-1000>   Repetitions per scenario (default: 3)\n  --timeout-ms <integer>  Total run deadline including setup/cleanup (default: 300000)\n  --help                 Show help\n  --version              Show version\n\nExit codes: 0 all pass; 1 failed/unsupported checks; 2 setup, timeout or harness error.\nAdapters execute trusted local code. JavaScript ESM (.mjs) works without dependencies.\n`;
+const HELP = `StreamTrust ${VERSION}\n\nUsage:\n  streamtrust init [directory] [--preset librechat]\n  streamtrust verify --adapter ./streamtrust/adapter.mjs [options]\n\nOptions:\n  --output <directory>    Reports (default: reports/streamtrust)\n  --repetitions <1-1000>   Repetitions per scenario (default: 3)\n  --timeout-ms <integer>  Total run deadline including setup/cleanup (default: 300000)\n  --help                 Show help\n  --version              Show version\n\nExit codes: 0 all pass; 1 failed/unsupported checks; 2 setup, timeout or harness error.\nAdapters execute trusted local code. JavaScript ESM (.mjs) works without dependencies.\n`;
 export async function main(args) {
   let outputDir;
   try {
@@ -18,17 +18,32 @@ export async function main(args) {
       return 0;
     }
     if (args[0] === 'init') {
-      if (args.length > 2 || args[1]?.startsWith('-'))
-        throw new Error('Usage: streamtrust init [directory]');
-      const dir = resolve(args[1] ?? 'streamtrust');
+      const initArgs = args.slice(1);
+      const presetIndex = initArgs.indexOf('--preset');
+      let preset = 'http';
+      if (presetIndex !== -1) {
+        preset = initArgs[presetIndex + 1];
+        initArgs.splice(presetIndex, 2);
+        if (preset !== 'librechat') throw new Error('Unknown init preset');
+      }
+      if (initArgs.length > 1 || initArgs[0]?.startsWith('-'))
+        throw new Error('Invalid init arguments');
+      const dir = resolve(initArgs[0] ?? 'streamtrust');
       await mkdir(dir, { recursive: true });
       // Exclusive copies preserve existing adapters. No generated config overrides user files.
       const created = [];
       try {
-        for (const [source, target] of [
-          ['http.adapter.mjs', 'adapter.mjs'],
-          ['demo-app.mjs', 'demo-app.mjs'],
-        ]) {
+        for (const [source, target] of preset === 'librechat'
+          ? [
+              ['librechat/adapter.mjs', 'adapter.mjs'],
+              ['librechat/provider.mjs', 'provider.mjs'],
+              ['librechat/setup.mjs', 'setup.mjs'],
+              ['librechat/README.md', 'README.md'],
+            ]
+          : [
+              ['http.adapter.mjs', 'adapter.mjs'],
+              ['demo-app.mjs', 'demo-app.mjs'],
+            ]) {
           const destination = resolve(dir, target);
           await copyFile(
             new URL(`../examples/${source}`, import.meta.url),

package/src/index.js

@@ -1,7 +1,10 @@
 import { mkdir, writeFile, rename, access } from 'node:fs/promises';
 import { resolve } from 'node:path';
 import { randomUUID } from 'node:crypto';
-export const VERSION = '0.1.0-preview.1';
+import { readFileSync } from 'node:fs';
+export const VERSION = JSON.parse(
+  readFileSync(new URL('../package.json', import.meta.url), 'utf8'),
+).version;
 const object = (value) =>
   value !== null && typeof value === 'object' && !Array.isArray(value);
 const nonempty = (value) =>
@@ -60,13 +63,15 @@ export async function writeReports(outputDir, report) {
   const rows = report.results;
   const screenshots = new Set();
   await Promise.all(
-    rows.map(async (row) => {
-      const path = `${row.scenario}-${row.repetition}.png`;
-      try {
-        await access(resolve(outputDir, path));
-        screenshots.add(path);
-      } catch {}
-    }),
+    (report.metadata?.evidencePolicy === 'summary-only-v1' ? [] : rows).map(
+      async (row) => {
+        const path = `${row.scenario}-${row.repetition}.png`;
+        try {
+          await access(resolve(outputDir, path));
+          screenshots.add(path);
+        } catch {}
+      },
+    ),
   );
   const suiteCases = report.suiteErrors
     .map(