@roku-sdk/device-test-client 0.0.0-stage → 2.19.0

package/docs/examples/log-driven-assertion.md

@@ -0,0 +1,99 @@
+# Example: log-driven assertion
+
+End-to-end causality pattern: snapshot a log position, trigger a
+device action, then `waitForLog` for the app-level ack. This is the
+canonical way to observe behaviour that *isn't* reflected in the DOM
+(analytics, network fires, side effects).
+
+## Prerequisites
+
+- App emits a deterministic log line when the observed action
+completes (e.g.
+`analytics:track event=tile_click id=tile-7`).
+- Log streaming has been installed — `installDeviceTestHooks` wires
+`ensureLogSession` automatically in `beforeEach`.
+
+## Test file
+
+```ts
+// tests/analytics.test.ts
+import {
+    installDeviceTestHooks
+} from "@roku-sdk/device-test-client";
+
+const ctx = installDeviceTestHooks();
+
+describe("tile click analytics", () => {
+    test("fires track event with correct id", async () => {
+        const client = ctx.getClient();
+
+        await client.buildAndDeploy("apps/grid", {
+            readyNode: { attr: { id: "grid" } }
+        });
+
+        // ─── Causality: snapshot → action → wait ───────────────────
+        const pos = client.logPosition();
+        await client.keypress("Select");
+
+        const line = await client.waitForLog(
+            /analytics:track event=(\w+) id=(\S+)/,
+            { since: pos, timeout: 3_000 }
+        );
+
+        const [, event, id] = line.message.match(
+            /analytics:track event=(\w+) id=(\S+)/
+        )!;
+        expect(event).toBe("tile_click");
+        expect(id).toMatch(/^tile-\d+$/);
+
+        // ─── Negative assertion: no double-fire within 500 ms ──────
+        await client.expectNoLog(/analytics:track event=tile_click/, {
+            since: pos,
+            window: 500,
+            // `pos` is before the keypress, so this tests "no OTHER
+            // tile_click event landed beyond the one we just matched".
+            // For strict no-double-fire, snapshot `pos2 = logPosition()`
+            // AFTER the first match and use that instead.
+        });
+    });
+});
+```
+
+## The causality pattern
+
+```text
+   T=0     client.logPosition()      ← snapshot
+   T=0+ε   client.keypress("Select") ← trigger
+   T=0+Δ   device emits log line     ← observation
+   T=Δ+n   waitForLog matches        ← assertion
+```
+
+The `since: pos` argument is what makes this reliable. Without it,
+`waitForLog` scans the buffer's minimum retained seq — which on a
+long-lived worker could match a line from a previous test.
+
+### Negative assertions
+
+`expectNoLog(pattern, { since, window })` runs the opposite: it
+throws `UnexpectedLogError` if *any* line in the window matches.
+Use it for "didn't accidentally navigate away", "didn't log an
+error", "didn't double-fire".
+
+`window` is optional — without it, `expectNoLog` evaluates
+immediately against already-buffered lines and returns.
+
+## Common pitfalls
+
+- **Don't snapshot AFTER the action.** The observed log line may
+arrive before the snapshot; subsequent `waitForLog` never
+matches.
+- **Use `{ since }`, not `{ timeout }` alone.** `timeout` is a
+watchdog; `since` is what scopes the window. Both are required
+for deterministic tests.
+- **Handle `LogStreamGapError`.** If the SSE stream dropped and
+reconnected during the window, the missing lines could have
+matched. Decide per-test whether to retry or surface.
+- **Don't leak buffer state between tests.** `installDeviceTestHooks`
+calls `disposeLogSession` in `afterEach` so each test starts with
+a fresh position. Custom setups must mirror that.
+

package/docs/examples/precondition-skips.md

@@ -0,0 +1,150 @@
+# Example: precondition skips
+
+Mix `itOnDevice` and `describeOnDevice` to gate tests on firmware,
+device class, locale, and video mode. Demonstrates the difference
+between "skip with reason" (registration-time wrappers) and "fail
+with reason" (runtime `requirePrecondition`).
+
+## Prerequisites
+
+- `createGlobalSetup` has run and written `test-results/device-env.json`
+  — preconditions read the cached `DeviceInfo` synchronously at
+  registration time.
+
+## Test file
+
+```ts
+// tests/precondition-matrix.test.ts
+import {
+    describeOnDevice,
+    itOnDevice,
+    requirePrecondition,
+    installDeviceTestHooks
+} from "@roku-sdk/device-test-client";
+
+const ctx = installDeviceTestHooks();
+
+// Whole suite gated on device class + video mode.
+describeOnDevice(
+    "4K playback",
+    () => {
+        itOnDevice(
+            "plays HEVC content",
+            async () => {
+                const client = ctx.getClient();
+                await client.buildAndDeploy("apps/player");
+                // ...
+            }
+        );
+
+        itOnDevice(
+            "uses HDR API (firmware 14+)",
+            async () => {
+                const client = ctx.getClient();
+                // ...
+            },
+            {
+                preconditions: { firmwareVersion: ">=14.0" }
+            }
+        );
+    },
+    {
+        preconditions: {
+            deviceClass: "high-end",
+            videoMode: "2160p"
+        }
+    }
+);
+
+describe("locale-specific UI", () => {
+    it("renders Spanish strings", async () => {
+        // Runtime form — fails the test if the device is not es-ES.
+        requirePrecondition({ locale: "es-ES" });
+
+        const client = ctx.getClient();
+        await client.buildAndDeploy("apps/i18n");
+        // ...
+    });
+});
+```
+
+## Walkthrough
+
+### `describeOnDevice` — whole-suite gate
+
+```ts
+describeOnDevice("suite name", body, { preconditions: {…} });
+```
+
+If the device doesn't satisfy every precondition, the suite
+registers as `describe.skip(...)` with the reason interpolated into
+the title. **All** `itOnDevice` and plain `it` inside the block
+skip with it.
+
+### `itOnDevice` — per-test gate
+
+```ts
+itOnDevice("test name", body, { preconditions: {…} });
+```
+
+Evaluates preconditions at registration time. If the device lacks
+the required fields, the test registers as `it.skip(...)`. If the
+device DOES satisfy them, the test runs and its callback executes
+with the usual `installDeviceTestHooks` plumbing.
+
+Inside a `describeOnDevice` block, `itOnDevice`'s preconditions
+**merge with** the enclosing block's — both must pass for the
+test to run.
+
+### `requirePrecondition` — runtime gate
+
+```ts
+it("something", async () => {
+    requirePrecondition({ locale: "es-ES" });
+    // ...
+});
+```
+
+Throws `PreconditionNotMetError` inside the test body. Jest reports
+the test as *failed*, not skipped. Use this only when the check
+genuinely needs to be runtime (e.g. depends on another test's side
+effect).
+
+## Precondition matching rules
+
+| Field | Matcher | Example |
+|-------|---------|---------|
+| `firmwareVersion` | semver range | `">=12.0"`, `"14.x"`, `"<15.0"` |
+| `modelFamily` | exact string | `"Roku Ultra"` |
+| `deviceClass` | derived tier | `"high-end"` \| `"mid-range"` \| `"low-end"` |
+| `videoMode` | prefix | `"2160p"` matches `"2160p60"`, `"2160p"` |
+| `locale` | exact string | `"en-US"` |
+| `country` | exact string | `"US"` |
+| `library` | **not implemented** — throws `PreconditionNotImplementedError` |
+
+`deviceClass` is derived from `modelName` via
+[`getDeviceClass`](../preconditions.md#devicecomputing-tiers).
+See the full tier table there.
+
+## Skip vs fail cheat sheet
+
+| Wrapper | Missing device info | Field mismatch | Not-implemented field |
+|---------|---------------------|----------------|-----------------------|
+| `itOnDevice` | skip + "missing device info" reason | skip + reason | skip + "not implemented" reason |
+| `describeOnDevice` | skip whole block | skip whole block | skip whole block |
+| `requirePrecondition` | throws `PreconditionNotMetError({reason:"missing-device-info"})` | throws `PreconditionNotMetError({reason:"mismatch"})` | throws `PreconditionNotImplementedError` |
+
+## Common pitfalls
+
+- **`device-env.json` must exist at registration time.** If a worker
+  starts before `globalSetup` writes the file, preconditions
+  degrade gracefully to "skip" with the `missing-device-info`
+  reason — tests don't run but no failures either. Hooks-based
+  setups never hit this.
+- **`library` is not yet implemented.** Any precondition
+  referencing it throws `PreconditionNotImplementedError`, which
+  the registration-time wrappers turn into `it.skip(...)` so the
+  test doesn't regress to red.
+- **Prefer `itOnDevice` over `requirePrecondition`.** Skip is the
+  correct CI signal for "device can't run this test"; fail is
+  reserved for "device should have been able but wasn't".

package/docs/examples/screenshot-regression.md

@@ -0,0 +1,103 @@
+# Example: screenshot regression
+
+Stable-capture + baseline-diff flow with a mask for the live clock
+widget. Demonstrates how to gate screenshot assertions on frame
+stability and how to regenerate baselines without accidentally
+green-lighting real regressions.
+
+## Prerequisites
+
+- Baseline PNG committed at
+  `baselines/hd/home/initial.png` (or `fhd/` if your device runs
+  1080p). See [baselines.md](../baselines.md) for the layout rules.
+- `setBaselineRoot(...)` called during Jest's `setupFilesAfterEnv`.
+
+## Test file
+
+```ts
+// tests/screenshot-home.test.ts
+import {
+    installDeviceTestHooks,
+    expectScreenshot,
+    type MaskRegion
+} from "@roku-sdk/device-test-client";
+
+const ctx = installDeviceTestHooks();
+
+const CLOCK_MASK: MaskRegion = {
+    // top-right HH:MM, coords in "hd" space
+    x: 1120,
+    y: 20,
+    width: 140,
+    height: 40
+};
+
+describe("home screen regression", () => {
+    test("matches baseline on boot", async () => {
+        const client = ctx.getClient();
+
+        await client.buildAndDeploy("apps/home", {
+            readyNode: { attr: { id: "home-screen" } }
+        });
+
+        const png = await client.captureStableScreenshot({
+            timeout: 8_000,
+            interval: 500,
+            mask: [CLOCK_MASK]
+        });
+
+        await expectScreenshot(png).toMatchBaseline("home/initial.png", {
+            threshold: 0.005,
+            mask: [CLOCK_MASK]
+        });
+    });
+});
+```
+
+## Walkthrough
+
+1. `captureStableScreenshot` loops `captureScreenshot` until two
+   consecutive PNGs are pixel-identical outside the mask, or
+   `timeout` elapses (then throws `ScreenshotNotStableError`). The
+   mask is applied inside the stability check too — otherwise the
+   ticking clock would keep the loop running forever.
+2. `expectScreenshot(png).toMatchBaseline(path, options)`:
+   - Resolves `path` against `baselineRoot` + the active client's
+     resolution subdir (`hd/` or `fhd/`).
+   - On missing baseline + `UPDATE_BASELINES=1`: writes the PNG
+     atomically and returns. Without the env var: throws
+     `ScreenshotBaselineMissingError`.
+   - On dimension mismatch: throws
+     `ScreenshotDimensionsMismatchError` (no diff written).
+   - On pixel mismatch over `threshold`: writes
+     `test-results/diffs/<flattened>.diff.png` and throws
+     `ScreenshotMismatchError` with `context.diffPath`.
+3. `threshold: 0.005` = up to 0.5 % of unmasked pixels may differ
+   before the assertion fails. Tune per scene — splash screens can
+   be near-zero; menu screens with subtle gradients may need 0.01
+   or more.
+
+## Regenerating the baseline
+
+When an intentional UI change makes the old baseline wrong:
+
+```bash
+UPDATE_BASELINES=1 pnpm jest tests/screenshot-home.test.ts
+```
+
+The env var is **blocked in CI** (both `CI` and `UPDATE_BASELINES`
+truthy simultaneously raises an error). Commit the regenerated PNG
+only after a visual review.
+
+## Common pitfalls
+
+- **Masks are in baseline-canvas coords, not device coords.** The
+  mask is applied before diffing, so the region must align with the
+  baseline image pixels. If the baseline is HD and the device is
+  FHD, capture a fresh FHD baseline — do not scale masks by hand.
+- **Don't raise the threshold to mute a real regression.** If the
+  diff PNG shows a genuine change, commit a new baseline instead of
+  loosening the threshold.
+- **Avoid capturing mid-transition.** Without
+  `captureStableScreenshot`, a single-frame capture will flake.
+- **Never commit `test-results/diffs/`.** Add it to `.gitignore`.

package/docs/examples/smoke-test.md

@@ -0,0 +1,69 @@
+# Example: smoke test
+
+The shortest possible end-to-end test: deploy the app, confirm the
+launch screen rendered, send a keypress, confirm a DOM transition.
+Use this as a template for the first test in a fresh repo.
+
+## Prerequisites
+
+- Jest config wired per [getting-started.md](../getting-started.md).
+- A Roku device reachable on the LAN with `rk server` running.
+- A sideloadable app at `apps/smoke` (or substitute your own path).
+
+## Test file
+
+```ts
+// tests/smoke.test.ts
+import {
+    findNode,
+    installDeviceTestHooks
+} from "@roku-sdk/device-test-client";
+
+const ctx = installDeviceTestHooks();
+
+describe("smoke", () => {
+    test("launches and responds to input", async () => {
+        const client = ctx.getClient();
+
+        await client.buildAndDeploy("apps/smoke", {
+            readyNode: { attr: { id: "home-screen" } }
+        });
+
+        const tree = await client.queryAppUi();
+        const playButton = findNode(tree, {
+            tag: "Button",
+            attr: { id: "play" }
+        });
+        expect(playButton.attrs.text).toBe("Play");
+
+        await client.keypress("Select");
+
+        await client.waitForNode(
+            { attr: { id: "player-screen" } },
+            { timeout: 5_000 }
+        );
+    });
+});
+```
+
+## What the test exercises
+
+1. `installDeviceTestHooks()` wires `beforeAll` / `beforeEach` /
+   `afterEach` hooks that resolve the device IP, build the
+   `RkClient`, reset the log session per test, and collect failure
+   artifacts on uncaught errors.
+2. `buildAndDeploy` sideloads the app and waits until `readyNode`
+   appears in the SceneGraph tree. Any crash or post-deploy system
+   dialog during readiness polling raises `DeployFailedError` /
+   `AppCrashedError`, which `installDeviceTestHooks` turns into a
+   full failure bundle (screenshot + dom + logs).
+3. `queryAppUi` fetches the current tree; `findNode` throws
+   `NodeNotFoundError` if the predicate doesn't match.
+4. `keypress("Select")` is ECP; subsequent `waitForNode` polls
+   `queryAppUi` on the default 200 ms cadence.
+
+## Why this is a template, not a framework
+
+No retries, no conditional logic, no preconditions — just four
+actions and two assertions. Add preconditions, screenshot checks,
+or log gates by layering them in. The other examples show how.

package/docs/getting-started.md

@@ -0,0 +1,195 @@
+# Getting started
+
+This page walks you from an empty workspace to a passing integration test
+running against a real Roku device. If you already have a Roku sideload
+toolchain (via `rk`) you can skip straight to
+[Step 3 — author your first test](#step-3--author-your-first-test).
+
+## Prerequisites
+
+- **Node.js 24+** — the SDK-wide supported runtime. The package needs a
+global `fetch`; runtimes without one fail log-session construction with
+"fetch implementation required".
+- **Jest 29 or 30** as the test runner.
+- A Roku device on your local network with **developer mode enabled** and
+a known **developer password**.
+- The `**rk` CLI** (from the `[@roku-sdk/tooling](../../tooling)` package)
+configured via `rk device add <ip>` so `.roku/roku-dev-config.json`
+exists next to your tests **OR** the env-var pair
+`RK_DEVICE_IP` + `RK_DEVICE_PASSWORD` set (intended for CI — see
+[Configuration → Device credentials](configuration.md#device-credentials)).
+When both env vars are set the `.roku/` walk is skipped entirely. With
+only `RK_DEVICE_IP` set, the password is taken from the `.roku/` entry
+with that IP.
+- `**rk server` running** on the host if you plan to use log assertions.
+The daemon tails the Roku telnet log port (8085) and exposes an HTTP /
+SSE front door that the SDK consumes.
+
+Verify the device is reachable:
+
+```bash
+rk device list          # the active device should have a green dot
+rk device ping          # smoke-checks /query/device-info
+```
+
+## Step 1 — install
+
+```bash
+pnpm add -D @roku-sdk/device-test-client jest ts-jest @types/jest
+# or npm/yarn equivalents
+```
+
+## Step 2 — wire Jest
+
+Drop these three files next to your tests. The exact paths are examples;
+adjust them in `jest.config.ts` if you prefer a different layout.
+
+### `jest.config.ts`
+
+```ts
+import type { Config } from "jest";
+
+const config: Config = {
+    preset: "ts-jest",
+    testEnvironment: "@roku-sdk/device-test-client/environment",
+    testMatch: ["<rootDir>/tests/**/*.test.ts"],
+    globalSetup: "<rootDir>/tests/support/global-setup.ts",
+    globalTeardown: "<rootDir>/tests/support/global-teardown.ts",
+    setupFilesAfterEach: ["<rootDir>/tests/support/setup.ts"],
+    maxWorkers: 1,
+    testTimeout: 60_000
+};
+
+export default config;
+```
+
+The custom `testEnvironment` is what captures per-failure bundles (see
+[Jest integration](jest-integration.md)). `maxWorkers: 1` is required —
+tests share a single device.
+
+### `tests/support/global-setup.ts`
+
+```ts
+import { createGlobalSetup } from "@roku-sdk/device-test-client";
+import { resolve } from "node:path";
+
+export default createGlobalSetup({
+    testResultsDir: resolve(__dirname, "../../test-results")
+});
+```
+
+`createGlobalSetup` resolves the active device, queries `DeviceInfo`,
+writes `test-results/device-env.json`, and boots a fresh `rk server` log
+session. See
+`[createGlobalSetup](jest-integration.md#creategloballsetup)` for the full
+contract.
+
+### `tests/support/global-teardown.ts`
+
+```ts
+import { createGlobalTeardown } from "@roku-sdk/device-test-client";
+
+export default createGlobalTeardown({ exitToHome: true });
+```
+
+### `tests/support/setup.ts`
+
+```ts
+import {
+    createDefaultClient,
+    installDeviceTestHooks
+} from "@roku-sdk/device-test-client";
+import { resolve } from "node:path";
+
+const client = createDefaultClient();
+
+installDeviceTestHooks({
+    client,
+    baselineRoot: resolve(__dirname, "../baselines"),
+    testResultsDir: resolve(__dirname, "../../test-results"),
+    diffRoot: resolve(__dirname, "../../test-results/diffs"),
+    deviceEnvFile: resolve(
+        __dirname,
+        "../../test-results/device-env.json"
+    )
+});
+
+export { client };
+```
+
+`installDeviceTestHooks` is the **single** wiring entry point in v2.0.0.
+It:
+
+- registers the `client` so `expectTree` / `expectScreenshot` /
+`itOnDevice` / `describeOnDevice` see the active device without extra
+setup calls,
+- anchors the **baselines root** (used by `expectTree` /
+`expectScreenshot`),
+- anchors the **screenshot diff root** (PNGs for failed
+`expectScreenshot` runs),
+- anchors the **device-env snapshot path** that
+`readCachedDeviceInfoSync` (via registration-time preconditions)
+reads,
+- starts the log SSE session in `beforeAll`,
+- aborts every subsequent test in a worker if the device flagged itself
+unhealthy (fail-fast cascade),
+- flushes per-failure bundles in the custom test environment.
+
+> **Migrated from v1.x?** The separate `setActiveClient(...)` /
+> `setBaselineRoot(...)` / `setScreenshotDiffRoot(...)` /
+> `setDeviceEnvFile(...)` calls you may have in an older `setup.ts`
+> were removed from the barrel in v2.0.0 — pass the same values as
+> `installDeviceTestHooks` options above. See the `installDeviceTestHooks` options above for the supported setup,
+> and [Versioning](../README.md#versioning) for the public API contract.
+
+## Step 3 — author your first test
+
+```ts
+// tests/smoke.test.ts
+import { client } from "./support/setup";
+
+test("app boots to the home scene", async () => {
+    await client.buildAndDeploy("apps/buttons", {
+        readyNode: { attr: { id: "home-title" } }
+    });
+
+    await client.keypress("Select");
+
+    const firstButton = await client.waitForNode(
+        { focused: true, attr: { id: "first-button" } },
+        { timeout: 3_000 }
+    );
+
+    expect(firstButton.attrs.text).toBe("Play");
+});
+```
+
+Run the suite:
+
+```bash
+pnpm jest
+```
+
+On success you should see:
+
+```
+[device-test-client] device: Roku Ultra (4800X), firmware 14.1.0,
+                     videoMode 1080p60, locale en_US
+PASS tests/smoke.test.ts
+```
+
+On failure, look under `test-results/failures/<sanitized-test-name>/` for
+the screenshot, DOM tree, recent logs, and error bundle — the reviewer
+gets everything they need to diagnose without re-running.
+
+## Where to next?
+
+- `[RkClient` reference](rkclient.md) — every method, every option.
+- [DOM assertions](dom-assertions.md) — `findNode`, `waitForNode`,
+`expectTree`.
+- [Screenshot assertions](screenshot-assertions.md) — `expectScreenshot`
+with masks and thresholds.
+- [Preconditions](preconditions.md) — skip tests cleanly on unsupported
+hardware.
+- [Examples](examples/) — end-to-end walkthroughs.
+