@junoflow/testkit 0.3.0 → 0.9.0

package/README.md

@@ -1,7 +1,8 @@
 # @junoflow/testkit
 
-Test harness for [Juno](https://github.com/boivie/juno) integrations. It runs
-your handlers with the same `ctx` the sandbox builds, but with a different
+Test harnesses for [Juno](https://github.com/boivie/juno) integrations
+(`harness`), flows (`flowHarness`) and apps (`appHarness`). Each runs your
+handlers with the same `ctx` the sandbox builds, but with a different
 implementation for tests.
 
 ```bash
@@ -190,3 +191,221 @@ hook reports through ctx and returns nothing" is what production says too.
   with one empty, so a handler never sees one — a test that skipped
   `h.credentials(...)` would be green for a state production forbids, with
   `undefined` sitting where the generated types promise a `string`.
+
+## `flowHarness`
+
+The flow half. It builds the `ctx` the flow sandbox builds from the flow's own
+manifest, runs one trigger handler against it, and answers every outward call
+from what the test supplies. A flow working copy's `npm test` runs it under
+vitest, and `juno flow init` writes a starter `test/flow.test.ts` for a version
+that stores no tests.
+
+```ts
+import { beforeEach, expect, it } from 'vitest';
+import { flowHarness } from '@junoflow/testkit';
+import manifest from '../manifest.json';
+import flow from '../src/flow';
+
+const h = flowHarness({ manifest, flow });
+beforeEach(() => h.reset());
+
+it('greets when someone is home', async () => {
+  h.now('2026-10-04T18:42:00').timezone('Europe/Stockholm');
+  h.states({ houseMode: 'home' });
+  h.targets.answer('kitchenSpeaker.ttsCloudSay', undefined);
+
+  const status = await h.trigger('goodMorning', {
+    event: {
+      entity_id: 'input_boolean.good_morning',
+      new_state: { state: 'on' },
+    },
+  });
+
+  expect(status).toEqual({ kind: 'ok', text: 'greeted' });
+  expect(h.targets.calls()).toEqual([
+    {
+      target: 'kitchenSpeaker',
+      action: 'ttsCloudSay',
+      params: { message: 'Good morning' },
+    },
+  ]);
+});
+```
+
+`flowHarness({ manifest, flow, now?, timezone?, solar? })` returns:
+
+| Member                                         | Does                                                                                        |
+| ---------------------------------------------- | ------------------------------------------------------------------------------------------- |
+| `h.trigger(name, { event })`                   | run a declared trigger; resolves to the handler's return value                              |
+| `h.scenario(s)`                                | set the world from one JSON object (below)                                                  |
+| `h.now(iso)` / `h.timezone(tz)` / `h.solar(v)` | what `ctx.now()` and `ctx.solar()` answer; the timezone is `UTC` unless set                 |
+| `h.states(values)` / `h.state(name)`           | seed a `read` or `read_write` binding; read its current value                               |
+| `h.stateCalls()`                               | `set`, `claim`, `release`, `signal`, `retract`, as `{binding, verb, value?, opts?}`         |
+| `h.targets.answer(key, a)` / `.calls()`        | answer `binding.method` with a value or a function of the params; the calls made            |
+| `h.controls.answer(key, a)` / `.calls()`       | answer `binding.verb`; the commands made                                                    |
+| `h.llm(id, answer)`                            | answer `step.llm(id, …)`; `id#N` answers only that occurrence                               |
+| `h.steps()`                                    | every `step.run` and `step.llm`, as `{id, output}`; repeats are `id`, `id#1`, `id#2`        |
+| `h.findings()`                                 | `{slug, key, severity, text, standing}`, collapsed by `(slug, key)` as the host does        |
+| `h.credentials(values)`                        | set `ctx.credentials` by declared id                                                        |
+| `h.logs()`                                     | `[{ level, message, data }]`                                                                |
+| `h.conformance()`                              | triggers declared vs implemented, both ways; `[]` when they agree                           |
+| `h.reset()`                                    | clear recordings, answers, seeds and scoped state; keep clock, timezone, solar, credentials |
+
+`h.trigger(name, { event })` sets `event.trigger` and spreads the payload you
+give onto the event, as the sandbox does. An array (a `batch` run) stays under
+`event.payload`. Nothing else is filled in, and the trigger's filter and
+modifiers are not evaluated. `h.now(iso)` without an offset is wall time in
+`h.timezone()`; a wall time the timezone skips or repeats is refused, so give
+one near a DST change an offset.
+
+### Scenarios
+
+`h.scenario(s)` applies a whole world at once, through the setters above and
+their checks:
+
+```ts
+import fixture from './fixtures/porch-42.json';
+
+it('replays run 42', async () => {
+  h.scenario(fixture);
+  const status = await h.trigger(fixture.trigger);
+  expect(status).toEqual(fixture.expect.status);
+  expect(h.steps()).toEqual(fixture.expect.steps);
+  expect(h.targets.calls()).toEqual(fixture.expect.calls);
+});
+```
+
+Its keys are `now`, `timezone`, `solar`, `states`, `credentials`, `targets`
+(`"binding.method": answer`), `controls` (`"binding.verb": answer`), `llm`
+(`"stepId": answer`) and `event`, which an `h.trigger(name)` with no `{ event }`
+runs with; an explicit event, even `null`, wins. `trigger`, `expect` and
+`flow_version` are carried and not applied. A key left out sets nothing; an
+unknown key fails. The `Scenario` type is exported.
+
+Two answer forms are reserved, in a scenario and in `h.targets.answer`,
+`h.controls.answer` and `h.llm` alike:
+
+- `{ "$replay": [a, b, …] }` answers calls in order; one too many fails with
+  `porchLight.getState answered 2 calls in the scenario; this was call 3`;
+- `{ "$error": "message" }` makes the call throw `new Error("message")`, which,
+  unlike a refusal, the handler may catch. A `$replay` item may be an `$error`.
+
+`juno flow fixture --run <seq>` writes a scenario from a recorded run, with
+`expect` filled in from what the run did; see the flow testing docs ("Replaying
+a run").
+
+### What fails
+
+Strict where a permissive double would pass a broken flow. Each of these fails
+the test, naming what was asked for and how to supply it:
+
+- a trigger, state binding, target, target method, control, finding or
+  credential the manifest does not declare, in the handler or in the `h.*` call
+  that seeds or answers it;
+- a state verb the binding was not granted (`set` on a `read` binding, `get` on
+  a slot), and `get` on a binding the test did not seed;
+- `ctx.sleep(…)` and `ctx.waitForEvent(…)`:
+  `a handler cannot suspend in a test: sleep("5m")`;
+- `ctx.now()`, `ctx.solar()`, a target call, a control command or a `step.llm`
+  the test did not answer, e.g.
+  `target call kitchenSpeaker.ttsCloudSay was not answered — h.targets.answer('kitchenSpeaker.ttsCloudSay', result)`.
+  An action with no result is answered with `undefined`, explicitly.
+
+A refusal the handler catches still fails the test. Answers and state values
+cross a JSON round trip, as on the wire.
+
+Two things are not modelled: `claim` always resolves `{ status: 'winning' }`,
+because the layer stack is the server's, and scoped state cannot be seeded from
+a test.
+
+## `appHarness`
+
+The app half. It builds the `ctx` the app sandbox builds from the app's own
+manifest and calls one handler with it, as the page's
+`juno.handlers.<name>(input)` does. An app working copy's `npm test` runs it
+under vitest, and `juno app init` writes a starter `test/app.test.ts` for a
+version that stores no tests.
+
+```ts
+import { beforeEach, expect, it } from 'vitest';
+import { appHarness } from '@junoflow/testkit';
+import manifest from '../manifest.json';
+import app from '../src/app';
+
+const h = appHarness({ manifest, app });
+beforeEach(() => h.reset());
+
+it('rates a meal', async () => {
+  h.now('2026-10-04T18:42:00').timezone('Europe/Stockholm');
+  expect(await h.call('rate', { meal: 'tacos', stars: 5 })).toEqual({
+    saved: true,
+  });
+  expect(h.events()).toEqual([
+    { event: 'rated', payload: { meal: 'tacos', stars: 5 } },
+  ]);
+});
+
+it('a read-only handler cannot claim', async () => {
+  await expect(h.call('peek', {})).rejects.toThrow(
+    'read-only handler "peek" cannot claim states'
+  );
+});
+```
+
+`appHarness({ manifest, app, now?, timezone?, solar? })` returns:
+
+| Member                                         | Does                                                                             |
+| ---------------------------------------------- | -------------------------------------------------------------------------------- |
+| `h.call(name, input?)`                         | call a declared handler; resolves to its result after a JSON round trip          |
+| `h.events()`                                   | every emit that passed, as `{event, payload}`, in emit order                     |
+| `h.scenario(s)`                                | set the world from one JSON object, as `flowHarness`'s                           |
+| `h.now(iso)` / `h.timezone(tz)` / `h.solar(v)` | what `ctx.now()` and `ctx.solar()` answer; the timezone is `UTC` unless set      |
+| `h.states(values)`                             | seed a `read` binding                                                            |
+| `h.stateCalls()`                               | `claim` and `release`, as `{binding, verb, value?, opts?}`                       |
+| `h.targets.answer(key, a)` / `.calls()`        | answer `binding.method` with a value or a function of the params; the calls made |
+| `h.controls.answer(key, a)` / `.calls()`       | answer `binding.verb`; the commands made                                         |
+| `h.credentials(values)`                        | set `ctx.credentials` by declared id                                             |
+| `h.logs()`                                     | `[{ level, message, data }]`                                                     |
+| `h.conformance()`                              | handlers declared vs implemented, both ways; `[]` when they agree                |
+| `h.reset()`                                    | clear recordings, answers and seeds; keep clock, timezone, solar, credentials    |
+
+`h.scenario(s)` takes the flow scenario's keys except `llm` and `flow_version`,
+which it refuses, and ignores `trigger`, `event` and `expect`, so one scenario
+can serve both. `$replay` and `$error` answer targets and controls as they do in
+a flow test.
+
+The handler's `ctx` is the app sandbox's: `log`, `now`, `solar`, `signal`,
+`states`, `controls`, `targets`, `credentials` and `events`. `ctx.signal` is
+never aborted. The binding handles, answers and clock are the ones `flowHarness`
+uses, with the same strictness.
+
+### What fails
+
+Everything `flowHarness` refuses for a binding, an answer or the clock, plus
+what the host refuses on a handler call, in the host's words:
+
+- a handler the manifest does not declare:
+  `app "meal-probe" has no handler "nope" (have: peek, rate)`;
+- an event the manifest does not declare:
+  `event "x" not declared in app "meal-probe" (have: rated)`, or
+  `(it declares no events)`;
+- a payload that fails the event's `payload` schema:
+  `event "rated" payload invalid:` followed by one line per problem;
+- in a handler declared `read_only: true`, a `claim`, a `release`, a control
+  verb or an emit: `read-only handler "peek" cannot claim states`,
+  `… cannot release states`, `… cannot command controls`,
+  `… cannot emit events`;
+- in a `read_only` handler, a call on a `homeassistant` or `mqtt` target:
+  `read-only handler "peek" cannot invoke target "kitchenSpeaker": a built-in homeassistant target is never read-only`.
+
+The input must be JSON-serialisable, and an omitted or `null` input arrives as
+`{}`. The result comes back as the page receives it: a `Date` becomes a string
+and an `undefined` field disappears. Neither is checked against a schema,
+because the host checks neither: a handler's input and output are its
+TypeScript.
+
+One host check is not made. A `read_only` handler may call an integration action
+only when the integration's manifest declares that action `read_only`, and the
+harness does not have that manifest. It allows every integration action from a
+`read_only` handler; the host refuses one that is not read-only, so try the
+handler against a draft with `npm run check`.

package/package.json

@@ -1,7 +1,7 @@
 {
   "name": "@junoflow/testkit",
-  "version": "0.3.0",
-  "description": "Test harness for Juno integrations",
+  "version": "0.9.0",
+  "description": "Test harnesses for Juno integrations, flows and apps",
   "license": "MIT",
   "repository": {
     "type": "git",

package/src/app.js

@@ -0,0 +1,274 @@
+import { fail } from './assert.js';
+import { createBindings, createWorld, jsonCopy } from './bindings.js';
+import { applyScenario } from './scenario.js';
+import { compile, describeErrors } from './schema.js';
+
+/** The only key an app's default export carries (BaseApp in app-api.d.ts). */
+const APP_KEYS = ['handlers'];
+
+/**
+ * appHarness({ manifest, app }) builds the ctx the app shim builds
+ * (sdk/src/app-entrypoint.ts), in process, and calls one handler with it.
+ *
+ * It refuses what the host refuses on a handler call, in the host's words:
+ * the invoker's undeclared handler and event checks and the event's payload
+ * schema (internal/apps/invoker.go), the read_only refusals
+ * (internal/executor/inprocess.go) and the broker's refusal of a built-in
+ * target (internal/broker/broker.go). It checks nothing the host does not —
+ * a handler's input and output are its TypeScript, never a schema.
+ *
+ * And, like flowHarness, it is strict where production asks the world: the
+ * clock, the sun, a target call and a control command fail unless the test
+ * answered them.
+ */
+export function appHarness({ manifest, app, now, timezone, solar } = {}) {
+  if (!manifest || typeof manifest !== 'object') {
+    fail(
+      "appHarness({ manifest, app }): manifest is required — import manifest from '../manifest.json'"
+    );
+  }
+  if (!app || typeof app !== 'object') {
+    fail(
+      "appHarness({ manifest, app }): app is required — import app from '../src/app'"
+    );
+  }
+  const handlers = manifest.handlers ?? {};
+  const events = manifest.events ?? {};
+  const appID = manifest.id ?? '';
+
+  const world = createWorld('appHarness', () => h);
+  const bindings = createBindings(manifest, () => h, {
+    statesRoot: 'states',
+    scoped: false,
+  });
+
+  // What handlers did. Cleared by reset().
+  const rec = { logs: [], events: [] };
+
+  const h = {
+    /**
+     * Calls a declared handler as the page's juno.handlers.<name>(input)
+     * does, and resolves to what the page would receive: the result after a
+     * JSON round trip.
+     */
+    async call(name, input) {
+      if (!Object.hasOwn(handlers, name)) {
+        // invoker.go, Invoker.Call: a name off the URL the manifest does not
+        // declare.
+        fail(
+          `app ${JSON.stringify(appID)} has no handler ${JSON.stringify(name)} (have: ${sorted(handlers)})`
+        );
+      }
+      const fns = app.handlers;
+      if (typeof fns !== 'object' || fns === null) {
+        fail('the app\'s default export must have a "handlers" object');
+      }
+      const handler = fns[name];
+      if (typeof handler !== 'function') {
+        // app-entrypoint.ts's refusal, for a handler declared and not
+        // implemented.
+        fail(
+          `no handler ${JSON.stringify(name)} (have: ${Object.keys(fns).join(', ') || '(none)'})`
+        );
+      }
+
+      // The body is JSON, and an empty one is {}; the shim also reads a null
+      // input as {}.
+      const arrived =
+        jsonCopy(input, `h.call(${JSON.stringify(name)}, …) — the input`) ?? {};
+
+      const run = { failures: [] };
+      const ctx = buildContext(run, name, handlers[name]?.read_only === true);
+
+      let result;
+      try {
+        result = await handler(ctx, arrived);
+      } catch (err) {
+        // A refusal the handler caught and turned into something else is
+        // still the cause; report it rather than whatever it became.
+        throw run.failures[0] ?? err;
+      }
+      // A handler that swallowed a refusal must not turn it into a green test.
+      if (run.failures.length) throw run.failures[0];
+
+      // app-entrypoint.ts: undefined crosses as null; anything JSON.stringify
+      // cannot render is the handler's error.
+      let json;
+      try {
+        json = JSON.stringify(result ?? null);
+      } catch (err) {
+        throw new Error(
+          `handler return value is not JSON-serializable: ${err}`
+        );
+      }
+      return JSON.parse(json);
+    },
+
+    /** Every ctx.events.<e>.emit that passed, in order. */
+    events: () => rec.events.map((e) => ({ ...e })),
+    /** Everything handlers logged, with its level. */
+    logs: () => rec.logs.slice(),
+    /** Manifest-vs-module problems; empty when they agree. */
+    conformance: () => appConformance(manifest, app),
+
+    ...world.h,
+
+    /**
+     * Applies a scenario through the setters: now, timezone, solar, states,
+     * controls, targets and credentials. Ignores trigger, event and expect;
+     * fails on any other key, llm included — an app has no step.llm.
+     */
+    scenario(s) {
+      return applyScenario(h, s, { kind: 'app' });
+    },
+
+    states: bindings.h.states,
+    stateCalls: bindings.h.stateCalls,
+    targets: bindings.h.targets,
+    controls: bindings.h.controls,
+    credentials: bindings.h.credentials,
+
+    /**
+     * Clears recordings, answers and seeded states; keeps the clock,
+     * timezone, solar and credentials.
+     */
+    reset() {
+      rec.logs.length = 0;
+      rec.events.length = 0;
+      bindings.reset();
+      return h;
+    },
+  };
+
+  world.init({ now, timezone, solar });
+
+  /** One call's ctx, exactly the members app-entrypoint.ts builds. */
+  function buildContext(run, handler, readOnly) {
+    const refuse = (message) => {
+      try {
+        fail(message);
+      } catch (err) {
+        run.failures.push(err);
+        throw err;
+      }
+    };
+    const b = bindings.ctx(refuse, { readOnly, handler });
+
+    const log = (message, data) =>
+      rec.logs.push({ level: 'debug', message, data });
+    log.warn = (message, data) =>
+      rec.logs.push({ level: 'warn', message, data });
+    log.error = (message, data) =>
+      rec.logs.push({ level: 'error', message, data });
+
+    return {
+      log,
+      ...world.ctx(refuse),
+      // The host aborts on timeout or shutdown; a test has neither.
+      signal: new AbortController().signal,
+      states: b.states,
+      controls: b.controls,
+      targets: b.targets,
+      credentials: b.credentials,
+      events: eventsProxy(refuse, handler, readOnly),
+    };
+  }
+
+  /**
+   * ctx.events, as buildEvents builds it: a handle for any name, checked when
+   * it emits. The checks are the host's, in its order: read_only
+   * (inprocess.go's refuseReadOnly), then Invoker.publish's declared event and
+   * payload schema.
+   */
+  function eventsProxy(refuse, handler, readOnly) {
+    return new Proxy(
+      {},
+      {
+        get(_own, event) {
+          if (typeof event !== 'string' || event === 'then') return undefined;
+          return {
+            async emit(payload) {
+              if (readOnly) {
+                refuse(
+                  `read-only handler ${JSON.stringify(handler)} cannot emit events`
+                );
+              }
+              const q = JSON.stringify(event);
+              const def = events[event];
+              if (!Object.hasOwn(events, event)) {
+                refuse(
+                  Object.keys(events).length === 0
+                    ? `event ${q} not declared in app ${JSON.stringify(appID)} (it declares no events)`
+                    : `event ${q} not declared in app ${JSON.stringify(appID)} (have: ${sorted(events)})`
+                );
+              }
+              // What the bus carries: the payload after the wire's JSON, an
+              // omitted one as null.
+              let sent;
+              try {
+                sent = JSON.parse(JSON.stringify({ payload })).payload ?? null;
+              } catch (err) {
+                refuse(`event ${q} payload invalid: ${err}`);
+              }
+              if (def?.payload !== undefined) {
+                const validate = compile(def.payload);
+                if (!validate(sent)) {
+                  refuse(
+                    `event ${q} payload invalid:\n${describeErrors(validate.errors)}`
+                  );
+                }
+              }
+              rec.events.push({ event, payload: sent });
+            },
+          };
+        },
+      }
+    );
+  }
+
+  return h;
+}
+
+/**
+ * Checks an app's manifest and module against each other and returns a list
+ * of problems, empty when they agree — flowConformance's pairing, for
+ * handlers. A handler declared and not implemented fails the page's first
+ * call to it; one implemented and not declared is never reachable.
+ */
+export function appConformance(manifest, app) {
+  const problems = [];
+
+  for (const key of Object.keys(app ?? {})) {
+    if (!APP_KEYS.includes(key)) {
+      problems.push(
+        `default export has an unknown key "${key}" (expected ${APP_KEYS.join(', ')})`
+      );
+    }
+  }
+  if (typeof app?.handlers !== 'object' || app.handlers === null) {
+    problems.push('default export has no "handlers" object');
+  }
+
+  const declared = Object.keys(manifest?.handlers ?? {});
+  for (const name of declared) {
+    if (typeof app?.handlers?.[name] !== 'function') {
+      problems.push(
+        `handler "${name}" is declared in manifest.json but not implemented`
+      );
+    }
+  }
+  for (const name of Object.keys(app?.handlers ?? {})) {
+    if (!declared.includes(name)) {
+      problems.push(
+        `handler "${name}" is implemented but not declared in manifest.json — the page can never call it`
+      );
+    }
+  }
+
+  return problems;
+}
+
+/** A manifest map's names as the invoker lists them: sorted, comma-joined. */
+function sorted(obj) {
+  return Object.keys(obj).sort().join(', ');
+}