streamtrust 0.0.0-stage → 0.1.0-preview.1

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 StreamTrust contributors
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -1,3 +1,168 @@
-# Temporary Holding Version
+# StreamTrust CLI — pilot release
 
-This version is a temporary placeholder for this package. An operational version to replace this has been submitted for review and is awaiting a staged release.
+Adapter-driven CI checks for streaming applications: compare observed output with authoritative state, exercise cancellation, and verify retry identity.
+
+**Status:** `0.1.0-preview.1`, an experimental preview for independent pilots. APIs and report formats may change before a stable release. The CLI is dependency-free, ESM, and requires Node.js 22+. MIT licensed. Browser and database libraries belong to your adapter and are not installed by StreamTrust.
+
+## Install the npm preview
+
+```sh
+npm install --save-dev streamtrust@preview
+npx streamtrust init
+npx streamtrust verify --adapter ./streamtrust/adapter.mjs
+```
+
+For reproducible CI, pin `streamtrust@0.1.0-preview.1` and commit your lockfile. The `preview` tag is intentionally separate from a stable release. The generated adapter initially exercises the bundled demonstration; map it to your application before using its results as application evidence.
+
+## Run from source
+
+```sh
+git clone https://github.com/santoshsibyala/streamtrust.git
+cd streamtrust
+node bin/streamtrust.js init demo
+node bin/streamtrust.js verify --adapter ./demo/adapter.mjs
+```
+
+Node.js 22+ is required. No dependency installation is needed to run this CLI or its tests. Run `npm test` to check the source, or `npm pack` to build an installable tarball. This repository contains the standalone CLI; the separate experimental application harnesses are not included.
+
+## Quick start: install a locally built tarball
+
+Build the tarball with `npm pack` in this repository.
+
+Put the supplied tarball in your project's `vendor/` directory, then:
+
+```sh
+npm install --save-dev ./vendor/streamtrust-0.1.0-preview.1.tgz
+npx streamtrust init
+npx streamtrust verify --adapter ./streamtrust/adapter.mjs
+```
+
+`init` creates `streamtrust/adapter.mjs` and `streamtrust/demo-app.mjs`. Existing files are never overwritten. Commit your adapted files and package-lock.json.
+
+Without `STREAMTRUST_BASE_URL`, the example starts an isolated local HTTP/SSE service with temporary file-backed state. It checks persistence, cancellation and retry, three times each. It requires no API key, browser, Redis or database installation. The fixture is removed at the end of the run.
+
+**The demonstration passing says nothing about your own application's reliability.** Replace the route mappings and assertions before using the result as application evidence.
+
+Reports appear in `reports/streamtrust/`:
+
+- `report.json`: versioned results, named assertions, evidence, metadata, run ID and suite errors.
+- `junit.xml`: test results for CI consumers.
+- `report.md` and `index.html`: human-readable diagnostics.
+
+## Connect your application
+
+The generated adapter is executable JavaScript. Map each scenario to your app's native operations and persistence contract. The bundled example uses this teaching protocol:
+
+| Operation | Example mapping |
+|---|---|
+| Start | `POST /runs` → `{id, parentId, status, content}` |
+| Observe stream | `GET /runs/:id/events` → SSE `text` and `terminal` JSON events |
+| Release model fixture | `POST /runs/:id/complete` |
+| Cancel | `POST /runs/:id/cancel` → `{accepted}` |
+| Retry | `POST /runs/:id/retry` → a new run linked to the old run |
+| Authoritative read | `GET /runs/:id` → saved state |
+
+The completion route is a test-fixture control, not an endpoint you must add to production. An adapter can instead control a deterministic provider, use Playwright to click the real Stop button, or read your database directly. Preserve your app's semantics: for example, regeneration may create a branch instead of replacing history. Do not invent commit receipts or cancellation guarantees your app does not expose.
+
+For an existing test service implementing this protocol:
+
+```sh
+STREAMTRUST_BASE_URL=http://127.0.0.1:8080 \
+  npx streamtrust verify --adapter ./streamtrust/adapter.mjs \
+  --output reports/streamtrust --repetitions 3 --timeout-ms 300000
+```
+
+The example optionally reads `STREAMTRUST_API_TOKEN` for bearer authentication, but does not put the token or authorization headers in reports. Run against a disposable test environment: these scenarios create, complete, cancel and retry operations. Adapters are trusted code and can access the same filesystem/network as your CI job. StreamTrust does not sandbox them or automatically redact arbitrary evidence. Return only the diagnostics you want uploaded as artifacts.
+
+## Adapter contract
+
+Export a default async factory. The CLI supplies `{outputDir, signal}`. Return:
+
+```js
+export default async function createAdapter({ signal }) {
+  return {
+    name: 'My application',
+    scenarios: ['persistence'],
+    async metadata() {
+      return { appRevision: process.env.APP_REVISION ?? 'unknown' };
+    },
+    async run(scenario, { repetition, outputDir, signal }) {
+      // Replace these calls with real observations in your adapter.
+      const displayed = await observeApplication(signal);
+      const saved = await readAuthoritativeState(signal);
+      return {
+        checks: [{
+          name: 'Displayed answer matches saved answer',
+          passed: displayed.text === saved.text,
+        }],
+        evidence: { displayed, saved },
+      };
+    },
+    async close() {
+      // Close browsers, connections and services owned by this adapter.
+    },
+  };
+}
+```
+
+The snippet illustrates the contract; `observeApplication` and `readAuthoritativeState` are your implementation. The generated HTTP adapter is the complete runnable example.
+
+Scenarios must be unique identifiers containing letters, digits, `_`, `-` or `.`. Assertions must have unique nonempty names and **boolean** `passed` values. Empty or malformed checks are harness errors. Unknown extra result fields cannot override the verdict. Evidence must be JSON-serializable. Unsupported capabilities return `{ checks: [], evidence: {}, unsupported: 'Reason' }`; they do not count as passes.
+
+The runner calls metadata before and after the suite, and calls close even when execution or validation fails. If your factory fails before returning an adapter, clean up resources in the factory's catch/finally. Use `signal` in network calls and release resources cooperatively. A failed assertion means the stated contract failed; it does not automatically identify a library defect.
+
+Types are exported from `streamtrust`: `ExternalAppAdapter`, `AdapterFactory`, `ExternalObservation`, `Check`, `Report`. The programmatic `runVerification(adapter, outputDir, repetitions)` returns a report; inspect `report.exitCode`. Only the CLI provides a separate watchdog process.
+
+JavaScript `.mjs` adapters work out of the box. For TypeScript adapters, install your own loader:
+
+```sh
+npm install --save-dev tsx
+node --import tsx ./node_modules/streamtrust/bin/streamtrust.js verify \
+  --adapter ./streamtrust/adapter.ts
+```
+
+## CI
+
+Install the pinned preview and commit package.json and package-lock.json so CI can use `npm ci`. If using a locally built tarball instead, keep it in `vendor/` so the committed file dependency can be resolved. The example workflow assumes the application under test is already running; add its start/wait commands before verification. Omit the base URL only when intentionally testing the bundled demonstration.
+
+```yaml
+name: Streaming application verification
+on: [push, pull_request]
+jobs:
+  verify:
+    runs-on: ubuntu-latest
+    timeout-minutes: 10
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-node@v4
+        with:
+          node-version: 22
+          cache: npm
+      - run: npm ci
+      # Start your test application and wait for readiness here.
+      - run: npx --no-install streamtrust verify --adapter ./streamtrust/adapter.mjs --output reports/streamtrust
+        env:
+          STREAMTRUST_BASE_URL: http://127.0.0.1:8080
+          STREAMTRUST_API_TOKEN: ${{ secrets.TEST_APP_TOKEN }}
+      - uses: actions/upload-artifact@v4
+        if: always()
+        with:
+          name: streamtrust-evidence
+          path: reports/streamtrust/
+```
+
+| Exit | Meaning |
+|---|---|
+| 0 | Every declared scenario and assertion passed. |
+| 1 | Failed assertions or explicitly unsupported scenarios. |
+| 2 | Invalid arguments/adapter, setup or cleanup failure, malformed observations, interrupted process or timeout. |
+
+Use a unique output directory per concurrent job. The CLI takes an exclusive lock and records a fresh run ID; it cannot reuse an old passing report after setup failure. Reports are overwritten on subsequent runs. If a process was forcibly killed, remove a stale `.streamtrust.lock` only after confirming that process has stopped. Invalid arguments and lock contention fail before a run starts and do not replace existing evidence.
+
+`--timeout-ms` bounds the entire invocation, including factory setup and cleanup (default five minutes), followed by up to one second of termination grace. On Linux/macOS the CLI terminates its worker process group, including ordinary child processes. Adapters that detach external services must arrange their own cleanup. Windows process-tree cleanup is not validated in this pilot. An outer CI timeout remains useful.
+
+## Scope and attribution
+
+StreamTrust checks what an adapter asserts. It does not prove exactly-once external effects, AI factuality, automatic recovery, production incident rates or adoption. The HTTP demo is synthetic. Separate development experiments have tested pinned Vercel Chatbot and LibreChat applications; their adapters and dependencies are not bundled here.
+
+Local preparation: `npm pack` in this package directory runs the package tests and creates the tarball. The public repository runs tests and a clean tarball installation on Node.js 22 and 24. This preview is prepared for interactive publication; npm build provenance is not configured.

package/bin/streamtrust.js

@@ -0,0 +1,3 @@
+#!/usr/bin/env node
+import { main } from '../src/cli.js';
+process.exitCode = await main(process.argv.slice(2));

package/examples/demo-app.mjs

@@ -0,0 +1,107 @@
+// Local teaching fixture, not a production implementation or evidence of customer impact.
+import { createServer } from 'node:http';
+import { mkdtemp, readFile, writeFile, rm } from 'node:fs/promises';
+import { tmpdir } from 'node:os';
+import { join } from 'node:path';
+import { randomUUID } from 'node:crypto';
+export async function startDemo() {
+  const directory = await mkdtemp(join(tmpdir(), 'streamtrust-demo-'));
+  const live = new Map();
+  const load = async (id) =>
+    JSON.parse(await readFile(join(directory, `${id}.json`), 'utf8'));
+  const save = (record) =>
+    writeFile(join(directory, `${record.id}.json`), JSON.stringify(record));
+  const first = 'The answer begins.';
+  const suffix = ' The answer is complete.';
+  const emit = (response, event) =>
+    response.write(`data: ${JSON.stringify(event)}\n\n`);
+  const create = async (parentId) => {
+    const record = {
+      id: randomUUID(),
+      parentId,
+      status: 'streaming',
+      content: first,
+    };
+    await save(record);
+    live.set(record.id, { record, response: null });
+    return record;
+  };
+  const server = createServer(async (req, res) => {
+    const url = new URL(req.url, 'http://localhost');
+    const parts = url.pathname.split('/').filter(Boolean);
+    const json = (value, status = 200) => {
+      res.writeHead(status, { 'content-type': 'application/json' });
+      res.end(JSON.stringify(value));
+    };
+    try {
+      if (req.method === 'POST' && url.pathname === '/runs') {
+        json(await create(null), 201);
+        return;
+      }
+      const id = parts[1];
+      if (parts[0] !== 'runs' || !/^[0-9a-f-]{36}$/.test(id ?? '')) {
+        json({ error: 'Not found' }, 404);
+        return;
+      }
+      const record = await load(id);
+      if (req.method === 'GET' && parts.length === 2) {
+        json(record);
+        return;
+      }
+      if (req.method === 'GET' && parts[2] === 'events') {
+        if (record.status !== 'streaming') {
+          json({ error: 'Already terminal' }, 409);
+          return;
+        }
+        res.writeHead(200, {
+          'content-type': 'text/event-stream',
+          'cache-control': 'no-cache',
+        });
+        live.get(id).response = res;
+        emit(res, { type: 'text', content: record.content });
+        return;
+      }
+      if (req.method === 'POST' && parts[2] === 'retry') {
+        json(await create(id), 201);
+        return;
+      }
+      if (req.method === 'POST' && ['complete', 'cancel'].includes(parts[2])) {
+        const entry = live.get(id);
+        if (record.status !== 'streaming') {
+          json({ accepted: false });
+          return;
+        }
+        record.status = parts[2] === 'cancel' ? 'cancelled' : 'completed';
+        if (record.status === 'completed') record.content += suffix;
+        // Explicit negative control for the package's own tests only.
+        if (process.env.STREAMTRUST_DEMO_FAULT !== 'drop-save')
+          await save(record);
+        if (entry?.response) {
+          emit(entry.response, {
+            type: 'terminal',
+            status: record.status,
+            content: record.content,
+          });
+          entry.response.end();
+        }
+        json({ accepted: true });
+        return;
+      }
+      json({ error: 'Not found' }, 404);
+    } catch (error) {
+      json({ error: String(error) }, 500);
+    }
+  });
+  await new Promise((ok, fail) => {
+    server.once('error', fail);
+    server.listen(0, '127.0.0.1', ok);
+  });
+  return {
+    baseUrl: `http://127.0.0.1:${server.address().port}`,
+    async close() {
+      server.closeAllConnections();
+      await new Promise((r) => server.close(r));
+      await rm(directory, { recursive: true, force: true });
+    },
+  };
+}

package/examples/http.adapter.mjs

@@ -0,0 +1,173 @@
+import { startDemo } from './demo-app.mjs';
+
+/**
+ * @returns {Promise<import('streamtrust').ExternalAppAdapter>}
+ * With no URL, starts the bundled local teaching fixture.
+ * For your app, replace these route mappings and assertions with its real contracts.
+ */
+export default async function createAdapter({ signal } = {}) {
+  const demo = process.env.STREAMTRUST_BASE_URL ? undefined : await startDemo();
+  const baseUrl = process.env.STREAMTRUST_BASE_URL || demo.baseUrl;
+  const headers = process.env.STREAMTRUST_API_TOKEN
+    ? { authorization: `Bearer ${process.env.STREAMTRUST_API_TOKEN}` }
+    : {};
+  const requestSignal = () =>
+    signal
+      ? AbortSignal.any([signal, AbortSignal.timeout(10000)])
+      : AbortSignal.timeout(10000);
+  async function json(path, method = 'GET') {
+    const response = await fetch(new URL(path, baseUrl), {
+      method,
+      headers,
+      signal: requestSignal(),
+    });
+    if (!response.ok)
+      throw new Error(`${method} ${path} returned ${response.status}`);
+    return response.json();
+  }
+  async function watch(id) {
+    const response = await fetch(new URL(`/runs/${id}/events`, baseUrl), {
+      headers,
+      signal: requestSignal(),
+    });
+    if (!response.ok || !response.body)
+      throw new Error('Expected SSE response');
+    const reader = response.body.getReader();
+    const decoder = new TextDecoder();
+    let buffer = '';
+    async function next() {
+      for (;;) {
+        const end = buffer.indexOf('\n\n');
+        if (end >= 0) {
+          const frame = buffer.slice(0, end);
+          buffer = buffer.slice(end + 2);
+          const data = frame
+            .split('\n')
+            .filter((l) => l.startsWith('data:'))
+            .map((l) => l.slice(5).trimStart())
+            .join('\n');
+          if (data) return JSON.parse(data);
+          continue;
+        }
+        const { done, value } = await reader.read();
+        if (done) throw new Error('Stream ended before the expected event');
+        buffer += decoder.decode(value, { stream: true });
+      }
+    }
+    return { next, close: () => reader.cancel() };
+  }
+  async function complete(id) {
+    const stream = await watch(id);
+    try {
+      const first = await stream.next();
+      await json(`/runs/${id}/complete`, 'POST');
+      const terminal = await stream.next();
+      return { first, terminal, durable: await json(`/runs/${id}`) };
+    } finally {
+      await stream.close();
+    }
+  }
+  return {
+    name: demo
+      ? 'Local HTTP/SSE demonstration'
+      : 'Your application HTTP adapter',
+    scenarios: ['persistence', 'cancellation', 'retry'],
+    async metadata() {
+      return {
+        mode: demo
+          ? 'synthetic local fixture'
+          : 'configured external test service',
+        contract: 'examples/http.adapter.mjs',
+        scope:
+          'HTTP/SSE observations and authoritative GET; no browser, database crash, billing, or production coverage',
+      };
+    },
+    async run(scenario) {
+      const initial = await json('/runs', 'POST');
+      if (scenario === 'cancellation') {
+        const stream = await watch(initial.id);
+        try {
+          const first = await stream.next();
+          const cancelled = await json(`/runs/${initial.id}/cancel`, 'POST');
+          const terminal = await stream.next();
+          const late = await json(`/runs/${initial.id}/complete`, 'POST');
+          const durable = await json(`/runs/${initial.id}`);
+          return {
+            checks: [
+              {
+                name: 'Server accepts cancellation',
+                passed: cancelled.accepted === true,
+              },
+              {
+                name: 'Terminal event and durable state agree on cancellation',
+                passed:
+                  terminal.status === 'cancelled' &&
+                  durable.status === 'cancelled',
+              },
+              {
+                name: 'Partial content remains authoritative',
+                passed: durable.content === first.content,
+              },
+              {
+                name: 'Late completion is rejected',
+                passed: late.accepted === false,
+              },
+            ],
+            evidence: { first, terminal, durable, late },
+          };
+        } finally {
+          await stream.close();
+        }
+      }
+      const original = await complete(initial.id);
+      if (scenario === 'persistence')
+        return {
+          checks: [
+            {
+              name: 'Terminal event indicates completed answer',
+              passed: original.terminal.status === 'completed',
+            },
+            {
+              name: 'Read after completion matches streamed result',
+              passed:
+                original.durable.status === 'completed' &&
+                original.durable.content === original.terminal.content,
+            },
+            {
+              name: 'Partial text is extended on completion',
+              passed:
+                original.terminal.content.startsWith(original.first.content) &&
+                original.terminal.content.length >
+                  original.first.content.length,
+            },
+          ],
+          evidence: original,
+        };
+      const retry = await json(`/runs/${initial.id}/retry`, 'POST');
+      const retried = await complete(retry.id);
+      const old = await json(`/runs/${initial.id}`);
+      return {
+        checks: [
+          {
+            name: 'Retry has a distinct identity linked to the original',
+            passed: retry.id !== initial.id && retry.parentId === initial.id,
+          },
+          {
+            name: 'Original result is unchanged',
+            passed: JSON.stringify(old) === JSON.stringify(original.durable),
+          },
+          {
+            name: 'Retried result is durably complete',
+            passed:
+              retried.durable.status === 'completed' &&
+              retried.durable.content === retried.terminal.content,
+          },
+        ],
+        evidence: { original: original.durable, retry: retried.durable },
+      };
+    },
+    async close() {
+      await demo?.close();
+    },
+  };
+}

package/package.json

@@ -1,6 +1,51 @@
 {
   "name": "streamtrust",
-  "version": "0.0.0-stage",
-  "stub": true,
-  "description": "Temporary package placeholder for staged publishing"
-}
+  "version": "0.1.0-preview.1",
+  "description": "Adapter-driven CI verification for streaming application lifecycle guarantees",
+  "type": "module",
+  "license": "MIT",
+  "engines": {
+    "node": ">=22"
+  },
+  "bin": {
+    "streamtrust": "bin/streamtrust.js"
+  },
+  "exports": {
+    ".": {
+      "types": "./src/index.d.ts",
+      "import": "./src/index.js"
+    }
+  },
+  "types": "src/index.d.ts",
+  "files": [
+    "bin/",
+    "src/",
+    "examples/",
+    "README.md",
+    "LICENSE"
+  ],
+  "scripts": {
+    "test": "node --test test/*.test.mjs",
+    "prepack": "npm test"
+  },
+  "keywords": [
+    "ci",
+    "streaming",
+    "verification",
+    "testing",
+    "sse"
+  ],
+  "repository": {
+    "type": "git",
+    "url": "git+https://github.com/santoshsibyala/streamtrust.git"
+  },
+  "homepage": "https://github.com/santoshsibyala/streamtrust#readme",
+  "bugs": {
+    "url": "https://github.com/santoshsibyala/streamtrust/issues"
+  },
+  "publishConfig": {
+    "access": "public",
+    "tag": "preview",
+    "registry": "https://registry.npmjs.org/"
+  }
+}

package/src/cli.d.ts

@@ -0,0 +1 @@
+export function main(args: string[]): Promise<number>;

package/src/cli.js

@@ -0,0 +1,178 @@
+import { fork } from 'node:child_process';
+import { randomUUID } from 'node:crypto';
+import { resolve } from 'node:path';
+import { fileURLToPath } from 'node:url';
+import { mkdir, copyFile, open, unlink, readFile } from 'node:fs/promises';
+import { constants } from 'node:fs';
+import { VERSION, failureReport } from './index.js';
+const HELP = `StreamTrust ${VERSION}\n\nUsage:\n  streamtrust init [directory]\n  streamtrust verify --adapter ./streamtrust/adapter.mjs [options]\n\nOptions:\n  --output <directory>    Reports (default: reports/streamtrust)\n  --repetitions <1-1000>   Repetitions per scenario (default: 3)\n  --timeout-ms <integer>  Total run deadline including setup/cleanup (default: 300000)\n  --help                 Show help\n  --version              Show version\n\nExit codes: 0 all pass; 1 failed/unsupported checks; 2 setup, timeout or harness error.\nAdapters execute trusted local code. JavaScript ESM (.mjs) works without dependencies.\n`;
+export async function main(args) {
+  let outputDir;
+  try {
+    if (args.length === 1 && ['--version', '-v'].includes(args[0])) {
+      console.log(VERSION);
+      return 0;
+    }
+    if (!args.length || args.includes('--help')) {
+      console.log(HELP);
+      return 0;
+    }
+    if (args[0] === 'init') {
+      if (args.length > 2 || args[1]?.startsWith('-'))
+        throw new Error('Usage: streamtrust init [directory]');
+      const dir = resolve(args[1] ?? 'streamtrust');
+      await mkdir(dir, { recursive: true });
+      // Exclusive copies preserve existing adapters. No generated config overrides user files.
+      const created = [];
+      try {
+        for (const [source, target] of [
+          ['http.adapter.mjs', 'adapter.mjs'],
+          ['demo-app.mjs', 'demo-app.mjs'],
+        ]) {
+          const destination = resolve(dir, target);
+          await copyFile(
+            new URL(`../examples/${source}`, import.meta.url),
+            destination,
+            constants.COPYFILE_EXCL,
+          );
+          created.push(destination);
+        }
+      } catch (error) {
+        await Promise.all(created.map((path) => unlink(path)));
+        throw error;
+      }
+      console.log(
+        `Created ${dir}\nRun: streamtrust verify --adapter ${JSON.stringify(resolve(dir, 'adapter.mjs'))}`,
+      );
+      return 0;
+    }
+    if (args[0] !== 'verify') throw new Error(`Unknown command: ${args[0]}`);
+    const values = new Map();
+    const known = new Set([
+      '--adapter',
+      '--output',
+      '--repetitions',
+      '--timeout-ms',
+    ]);
+    for (let i = 1; i < args.length; i += 2) {
+      const key = args[i],
+        value = args[i + 1];
+      if (
+        !known.has(key) ||
+        values.has(key) ||
+        !value ||
+        value.startsWith('--')
+      )
+        throw new Error(`Invalid or duplicate option: ${key}`);
+      values.set(key, value);
+    }
+    if (!values.has('--adapter')) throw new Error('--adapter is required');
+    const repetitions = Number(values.get('--repetitions') ?? 3),
+      timeout = Number(values.get('--timeout-ms') ?? 300000);
+    if (
+      !Number.isSafeInteger(repetitions) ||
+      repetitions < 1 ||
+      repetitions > 1000
+    )
+      throw new Error('--repetitions must be 1–1000');
+    if (!Number.isSafeInteger(timeout) || timeout < 100 || timeout > 86400000)
+      throw new Error('--timeout-ms must be 100–86400000');
+    outputDir = resolve(values.get('--output') ?? 'reports/streamtrust');
+    await mkdir(outputDir, { recursive: true });
+    const lock = resolve(outputDir, '.streamtrust.lock');
+    const handle = await open(lock, 'wx').catch((error) => {
+      throw new Error(
+        `Cannot lock output directory (${error.code}); use a unique directory per CI job. Remove a stale lock only after its process has stopped.`,
+      );
+    });
+    const runId = randomUUID();
+    await handle.writeFile(JSON.stringify({ pid: process.pid, runId }));
+    await handle.close();
+    try {
+      const config = {
+        adapter: resolve(values.get('--adapter')),
+        outputDir,
+        repetitions,
+        runId,
+      };
+      const child = fork(
+        fileURLToPath(new URL('./worker.js', import.meta.url)),
+        [JSON.stringify(config)],
+        {
+          execArgv: process.execArgv,
+          stdio: ['ignore', 'inherit', 'inherit', 'ipc'],
+          detached: process.platform !== 'win32',
+        },
+      );
+      const signalChild = (signal) => {
+        try {
+          if (process.platform !== 'win32') process.kill(-child.pid, signal);
+          else child.kill(signal);
+        } catch {}
+      };
+      let terminal, reason, timer, grace;
+      const interrupt = () => {
+        if (reason) return;
+        reason = 'Verifier interrupted';
+        signalChild('SIGTERM');
+        grace = setTimeout(() => signalChild('SIGKILL'), 1000);
+      };
+      process.once('SIGINT', interrupt);
+      process.once('SIGTERM', interrupt);
+      timer = setTimeout(() => {
+        reason = `Run deadline exceeded (${timeout}ms)`;
+        signalChild('SIGTERM');
+        grace = setTimeout(() => signalChild('SIGKILL'), 1000);
+      }, timeout);
+      let exited;
+      try {
+        child.on('message', (message) => {
+          if (message?.type === 'result' && message.runId === runId)
+            terminal = message;
+        });
+        exited = await new Promise((ok, fail) => {
+          child.once('error', fail);
+          child.once('exit', (code, signal) => ok({ code, signal }));
+        });
+      } finally {
+        clearTimeout(timer);
+        clearTimeout(grace);
+        process.removeListener('SIGINT', interrupt);
+        process.removeListener('SIGTERM', interrupt);
+        signalChild('SIGKILL');
+      }
+      if (
+        reason ||
+        !terminal ||
+        ![0, 1, 2].includes(terminal.exitCode) ||
+        exited.code !== terminal.exitCode
+      ) {
+        await failureReport(
+          outputDir,
+          runId,
+          reason ??
+            `Adapter process exited without a valid result (${JSON.stringify(exited)})`,
+        );
+        return 2;
+      }
+      const report = JSON.parse(
+        await readFile(resolve(outputDir, 'report.json'), 'utf8'),
+      );
+      if (report.runId !== runId || report.exitCode !== terminal.exitCode)
+        throw new Error('Report does not match current invocation');
+      console.log(
+        `StreamTrust: ${JSON.stringify(report.counts)}; exit ${report.exitCode}\nReports: ${outputDir}`,
+      );
+      return report.exitCode;
+    } catch (error) {
+      await failureReport(outputDir, runId, error);
+      console.error(String(error));
+      return 2;
+    } finally {
+      await unlink(lock);
+    }
+  } catch (error) {
+    console.error(`StreamTrust: ${error.message}`);
+    return 2;
+  }
+}

package/src/index.d.ts

@@ -0,0 +1,76 @@
+export const VERSION: string;
+export type Classification = 'PASS' | 'FAIL' | 'UNSUPPORTED' | 'HARNESS_ERROR';
+export interface Check {
+  name: string;
+  passed: boolean;
+  detail?: string;
+}
+export interface ExternalObservation {
+  checks: Check[];
+  evidence: unknown;
+  unsupported?: string;
+}
+export interface AdapterContext {
+  repetition: number;
+  outputDir: string;
+  signal?: AbortSignal;
+}
+export interface FactoryContext {
+  outputDir: string;
+  signal: AbortSignal;
+}
+export interface ExternalAppAdapter {
+  name: string;
+  scenarios: readonly string[];
+  metadata(): Promise<Record<string, unknown>>;
+  run(scenario: string, context: AdapterContext): Promise<ExternalObservation>;
+  close(): Promise<void>;
+}
+export type AdapterFactory = (
+  context: FactoryContext,
+) => Promise<ExternalAppAdapter>;
+export interface CaseResult {
+  scenario: string;
+  repetition: number;
+  classification: Classification;
+  durationMs: number;
+  checks: Check[];
+  evidence?: unknown;
+  error?: string;
+  unsupported?: string;
+}
+export interface Report {
+  schemaVersion: number;
+  version: string;
+  runId: string;
+  adapter: string;
+  generatedAt: string;
+  repetitions: number;
+  scenarios: string[];
+  metadata: unknown;
+  metadataAfter: unknown;
+  counts: Partial<Record<Classification, number>>;
+  results: CaseResult[];
+  suiteErrors: string[];
+  complete: boolean;
+  exitCode: number;
+}
+export function classifyExternal(result: unknown): Classification;
+export function validateAdapter(adapter: unknown): void;
+export function runVerification(
+  adapter: ExternalAppAdapter,
+  outputDir: string,
+  repetitions?: number,
+  options?: { signal?: AbortSignal; runId?: string },
+): Promise<Report>;
+export function verifyExternal(
+  adapter: ExternalAppAdapter,
+  outputDir: string,
+  repetitions?: number,
+): Promise<CaseResult[]>;
+export function writeReports(outputDir: string, report: Report): Promise<void>;
+export function failureReport(
+  outputDir: string,
+  runId: string,
+  error: unknown,
+): Promise<Report>;

package/src/index.js

@@ -0,0 +1,232 @@
+import { mkdir, writeFile, rename, access } from 'node:fs/promises';
+import { resolve } from 'node:path';
+import { randomUUID } from 'node:crypto';
+export const VERSION = '0.1.0-preview.1';
+const object = (value) =>
+  value !== null && typeof value === 'object' && !Array.isArray(value);
+const nonempty = (value) =>
+  typeof value === 'string' && value.trim().length > 0;
+export function classifyExternal(result) {
+  if (!object(result) || !Array.isArray(result.checks)) return 'HARNESS_ERROR';
+  if (result.unsupported !== undefined)
+    return nonempty(result.unsupported) && result.checks.length === 0
+      ? 'UNSUPPORTED'
+      : 'HARNESS_ERROR';
+  if (
+    !result.checks.length ||
+    result.checks.some(
+      (c) => !object(c) || !nonempty(c.name) || typeof c.passed !== 'boolean',
+    )
+  )
+    return 'HARNESS_ERROR';
+  if (new Set(result.checks.map((c) => c.name)).size !== result.checks.length)
+    return 'HARNESS_ERROR';
+  return result.checks.every((c) => c.passed) ? 'PASS' : 'FAIL';
+}
+export function validateAdapter(adapter) {
+  if (!object(adapter) || !nonempty(adapter.name))
+    throw new Error('Adapter must have a nonempty name');
+  if (
+    !Array.isArray(adapter.scenarios) ||
+    !adapter.scenarios.length ||
+    adapter.scenarios.some(
+      (s) => typeof s !== 'string' || !/^[a-zA-Z0-9][a-zA-Z0-9_.-]*$/.test(s),
+    )
+  )
+    throw new Error('Adapter scenarios must be nonempty safe identifiers');
+  if (new Set(adapter.scenarios).size !== adapter.scenarios.length)
+    throw new Error('Adapter scenario identifiers must be unique');
+  for (const method of ['metadata', 'run', 'close'])
+    if (typeof adapter[method] !== 'function')
+      throw new Error(`Adapter missing ${method}()`);
+}
+const escape = (value) =>
+  String(value)
+    .replace(/[\x00-\x08\x0b\x0c\x0e-\x1f]/g, '')
+    .replace(
+      /[&<>"']/g,
+      (c) =>
+        ({
+          '&': '&amp;',
+          '<': '&lt;',
+          '>': '&gt;',
+          '"': '&quot;',
+          "'": '&apos;',
+        })[c],
+    );
+const jsonSafe = (value) => JSON.parse(JSON.stringify(value ?? null));
+export async function writeReports(outputDir, report) {
+  await mkdir(outputDir, { recursive: true });
+  const rows = report.results;
+  const screenshots = new Set();
+  await Promise.all(
+    rows.map(async (row) => {
+      const path = `${row.scenario}-${row.repetition}.png`;
+      try {
+        await access(resolve(outputDir, path));
+        screenshots.add(path);
+      } catch {}
+    }),
+  );
+  const suiteCases = report.suiteErrors
+    .map(
+      (error, i) =>
+        `<testcase classname="streamtrust" name="harness-${i}"><error message="${escape(error)}"/></testcase>`,
+    )
+    .join('');
+  const cases = rows
+    .map(
+      (r) =>
+        `<testcase classname="${escape(report.adapter)}" name="${escape(r.scenario)}-${r.repetition}" time="${r.durationMs / 1000}">${r.classification === 'UNSUPPORTED' ? `<skipped message="${escape(r.unsupported)}"/>` : r.classification === 'HARNESS_ERROR' ? `<error message="${escape(r.error || 'Invalid adapter observation')}"/>` : r.classification === 'FAIL' ? `<failure message="Invariant check failed">${escape(JSON.stringify(r.checks))}</failure>` : ''}</testcase>`,
+    )
+    .join('');
+  const junit = `<?xml version="1.0" encoding="UTF-8"?><testsuite name="${escape(report.adapter)}" tests="${rows.length + report.suiteErrors.length}" failures="${report.counts.FAIL ?? 0}" errors="${(report.counts.HARNESS_ERROR ?? 0) + report.suiteErrors.length}" skipped="${report.counts.UNSUPPORTED ?? 0}">${cases}${suiteCases}</testsuite>`;
+  const md = `# ${report.adapter}: external application verification\n\nRun: ${report.runId} · StreamTrust ${VERSION} · exit ${report.exitCode}\n\n${JSON.stringify(report.counts)}\n\n| Scenario | Repetition | Result |\n|---|---:|---|\n${rows.map((r) => `| ${r.scenario} | ${r.repetition} | ${r.classification} |`).join('\n')}\n\n${report.suiteErrors.map((e) => `Harness error: ${e}`).join('\n')}\n\nPASS covers only the adapter's declared assertions. FAIL needs review before attribution to an upstream defect. UNSUPPORTED is not a pass. HARNESS_ERROR is inconclusive. See report.json for evidence and junit.xml for CI results.\n`;
+  const html = `<!doctype html><meta charset="utf-8"><title>StreamTrust report</title><style>body{max-width:1000px;margin:40px auto;padding:20px;font:16px system-ui}pre{white-space:pre-wrap;overflow-wrap:anywhere}details{padding:12px;border:1px solid #ddd;margin:12px 0}</style><h1>${escape(report.adapter)}</h1><pre>${escape(md)}</pre><a href="report.json">JSON evidence</a> · <a href="junit.xml">JUnit</a>${rows.map((r) => `<details><summary>${escape(r.scenario)} #${r.repetition}: ${r.classification}</summary><pre>${escape(JSON.stringify(r, null, 2))}</pre>${screenshots.has(`${r.scenario}-${r.repetition}.png`) ? `<img style="max-width:100%" alt="Scenario screenshot" src="${r.scenario}-${r.repetition}.png">` : ''}</details>`).join('')}`;
+  for (const [name, value] of Object.entries({
+    'report.json': JSON.stringify(report, null, 2),
+    'junit.xml': junit,
+    'report.md': md,
+    'index.html': html,
+  })) {
+    const tmp = resolve(outputDir, `.${name}.${report.runId}.tmp`);
+    await writeFile(tmp, value);
+    await rename(tmp, resolve(outputDir, name));
+  }
+}
+export async function failureReport(outputDir, runId, error) {
+  const report = {
+    schemaVersion: 1,
+    version: VERSION,
+    runId,
+    adapter: 'adapter initialization',
+    generatedAt: new Date().toISOString(),
+    repetitions: 0,
+    scenarios: [],
+    metadata: {},
+    metadataAfter: {},
+    counts: {},
+    results: [],
+    suiteErrors: [String(error)],
+    complete: false,
+    exitCode: 2,
+  };
+  await writeReports(outputDir, report);
+  return report;
+}
+export async function runVerification(
+  adapter,
+  outputDir,
+  repetitions = 3,
+  options = {},
+) {
+  const runId = options.runId ?? randomUUID();
+  const results = [];
+  const suiteErrors = [];
+  let metadata = {},
+    metadataAfter = {},
+    scenarios = [];
+  let name = 'invalid adapter';
+  try {
+    if (
+      !Number.isSafeInteger(repetitions) ||
+      repetitions < 1 ||
+      repetitions > 1000
+    )
+      throw new Error('repetitions must be an integer between 1 and 1000');
+    validateAdapter(adapter);
+    name = adapter.name;
+    scenarios = [...adapter.scenarios];
+    await mkdir(outputDir, { recursive: true });
+    metadata = jsonSafe(await adapter.metadata());
+    for (let repetition = 1; repetition <= repetitions; repetition++)
+      for (const scenario of scenarios) {
+        options.signal?.throwIfAborted();
+        const started = Date.now();
+        try {
+          const observation = jsonSafe(
+            await adapter.run(scenario, {
+              repetition,
+              outputDir: resolve(outputDir),
+              signal: options.signal,
+            }),
+          );
+          const classification = classifyExternal(observation);
+          // Copy only contract fields; adapter evidence cannot overwrite the verdict or identity.
+          const row = {
+            scenario,
+            repetition,
+            classification,
+            durationMs: Date.now() - started,
+            checks: observation?.checks ?? [],
+            evidence: observation?.evidence ?? null,
+          };
+          if (classification === 'HARNESS_ERROR')
+            row.error =
+              'Invalid adapter observation: require named boolean checks or explicit unsupported';
+          if (classification === 'UNSUPPORTED')
+            row.unsupported = observation.unsupported;
+          results.push(row);
+          console.log(`${classification} ${name} ${scenario} #${repetition}`);
+        } catch (error) {
+          results.push({
+            scenario,
+            repetition,
+            classification: 'HARNESS_ERROR',
+            durationMs: Date.now() - started,
+            error: String(error),
+            checks: [],
+          });
+          console.error(`HARNESS_ERROR ${scenario}: ${error}`);
+        }
+      }
+  } catch (error) {
+    suiteErrors.push(String(error));
+  } finally {
+    try {
+      if (typeof adapter?.metadata === 'function')
+        metadataAfter = jsonSafe(await adapter.metadata());
+    } catch (error) {
+      suiteErrors.push(`Final metadata: ${error}`);
+    }
+    try {
+      if (typeof adapter?.close === 'function') await adapter.close();
+    } catch (error) {
+      suiteErrors.push(`Cleanup: ${error}`);
+    }
+  }
+  const counts = results.reduce((a, r) => {
+    a[r.classification] = (a[r.classification] ?? 0) + 1;
+    return a;
+  }, {});
+  const complete =
+    scenarios.length > 0 && results.length === scenarios.length * repetitions;
+  const exitCode =
+    suiteErrors.length || !complete || counts.HARNESS_ERROR
+      ? 2
+      : counts.FAIL || counts.UNSUPPORTED
+        ? 1
+        : 0;
+  const report = {
+    schemaVersion: 1,
+    version: VERSION,
+    runId,
+    adapter: name,
+    generatedAt: new Date().toISOString(),
+    repetitions,
+    scenarios,
+    metadata,
+    metadataAfter,
+    counts,
+    results,
+    suiteErrors,
+    complete,
+    exitCode,
+  };
+  await writeReports(outputDir, report);
+  return report;
+}
+// Compatibility for the lab's existing programmatic callers.
+export async function verifyExternal(adapter, outputDir, repetitions = 3) {
+  return (await runVerification(adapter, outputDir, repetitions)).results;
+}

package/src/worker.js

@@ -0,0 +1,38 @@
+import { pathToFileURL } from 'node:url';
+import { runVerification, failureReport } from './index.js';
+const config = JSON.parse(process.argv[2]);
+const controller = new AbortController();
+process.once('SIGTERM', () =>
+  controller.abort(new Error('Verifier interrupted')),
+);
+process.once('SIGINT', () =>
+  controller.abort(new Error('Verifier interrupted')),
+);
+process.env.ST_REPORT_DIR = config.outputDir;
+let report;
+try {
+  const module = await import(pathToFileURL(config.adapter).href);
+  if (typeof module.default !== 'function')
+    throw new Error('Adapter must export a default async factory');
+  const adapter = await module.default({
+    outputDir: config.outputDir,
+    signal: controller.signal,
+  });
+  report = await runVerification(
+    adapter,
+    config.outputDir,
+    config.repetitions,
+    { signal: controller.signal, runId: config.runId },
+  );
+} catch (error) {
+  report = await failureReport(config.outputDir, config.runId, error);
+}
+if (process.send)
+  await new Promise((r) =>
+    process.send(
+      { type: 'result', runId: config.runId, exitCode: report.exitCode },
+      r,
+    ),
+  );
+// close() has finished; a forgotten adapter timer must not leave CI hanging.
+process.exit(report.exitCode);