@toggld/node 0.2.0 → 0.4.0

package/README.md

@@ -16,11 +16,14 @@ Two evaluation modes:
   evaluation" over the OFREP endpoints).
 - **Local** (`mode: "local"`) — recommended for long-lived servers. Flag
   definitions sync in the background (a snapshot fetch plus a live WebSocket
-  stream, with polling fallback) and every evaluation runs **in-process** on
-  the bundled engine: microsecond latency, zero server requests per
-  evaluation, and changes pushed to your process within milliseconds. Requires
+  stream where supported, with polling fallback) and every evaluation runs **in-process** on
+  the bundled engine without a server request per evaluation. Workers currently
+  uses polling for definition updates. Cache freshness, polling intervals, and
+  network conditions affect when changed definitions arrive. Requires
   a secret SDK key (`ffk_…`) — public browser tokens cannot read rulesets. See
-  [ADR-0012](../../docs/adr/0012-definition-sync-and-local-evaluation.md).
+  [ADR-0012](../../docs/adr/0012-definition-sync-and-local-evaluation.md). Local
+  mode can also read its snapshot from a **file** instead of the network — see
+  [Local flag files](#local-flag-files-offline) below.
 
 ## Install
 
@@ -89,16 +92,22 @@ scope.
 | `sync.transport` | `"ws" \| "poll"`      | `"ws"`           | Live stream (with poll safety net) vs polling only.              |
 | `sync.pollIntervalMs` | `number`         | `30000`          | Snapshot poll cadence; cold start retries faster.                |
 | `sync.readyTimeoutMs` | `number`         | `5000`           | How long `create` waits for the first snapshot.                  |
+| `sync.file`      | `string \| boolean`   | —                | Read the snapshot from a local file (offline); `true` ⇒ `./toggld.flags.json`. |
 | `defaultContext` | `EvaluationContext`   | `{}`             | Merged under per-call context (per-call attributes win).         |
 | `timeoutMs`      | `number`              | `5000`           | Per-request timeout.                                             |
 | `fetch`          | `typeof fetch`        | `globalThis.fetch` | Custom fetch (tests, proxies, instrumentation).               |
 
-In local mode `create` resolves once the first snapshot has synced (or after
-`sync.readyTimeoutMs`); until a snapshot arrives, evaluations return your
-defaults with `errorCode: "PROVIDER_NOT_READY"` and sync keeps retrying.
-`client.ready()` awaits sync explicitly. Updates arrive as WebSocket pushes
-within milliseconds of a management change; on runtimes where the stream is
-unavailable the client polls instead, automatically.
+In local mode, `create` waits for the first snapshot or `sync.readyTimeoutMs`.
+Until a snapshot arrives, evaluations return your defaults with
+`errorCode: "PROVIDER_NOT_READY"`, and sync keeps retrying. `client.ready()` waits
+for the first snapshot without a timeout.
+
+Changes on the connected Node instance normally arrive through WebSocket pushes.
+The SDK also polls every 30 seconds, even while the stream stays healthy.
+Polling recovers writes from other instances and announcements that the bounded
+server buffer drops. With healthy requests, the conservative bound is
+`sync.pollIntervalMs + 2 * timeoutMs`, plus scheduling delays, or 40 seconds by default.
+The [Node sync guide](../../docs/node-sync.md) defines the assumptions and deployment requirements.
 
 ### How fast is local mode?
 
@@ -110,6 +119,46 @@ sync OFF (remote OFREP)        ~11,600 evals/s   p50 2.4 ms
 sync ON  (local engine)     ~1,090,000 evals/s   p50 27 µs   (0.6 µs single-caller)
 ```
 
+## Local flag files (offline)
+
+For local development, CI, or offline runs, a client can evaluate against a
+**file** instead of syncing over the network — same engine, same results, no
+flag API required. Generate the file with the bundled CLI:
+
+```bash
+# Fetch the environment's current flags into ./toggld.flags.json.
+# Prompts for the key if you omit --sdk-key (or set TOGGLD_SDK_KEY).
+npx @toggld/node pull --sdk-key ffk_… --api https://your-flag-api.example.com
+```
+
+Then point a client at it — no `sdkKey` or `endpoint` needed:
+
+```ts
+const client = await ToggldClient.create({
+  mode: "local",
+  sync: { file: "./toggld.flags.json" }, // or `file: true` for that default path
+})
+```
+
+The file is a `FlagSnapshot` (the exact wire snapshot the SDK syncs) wrapped in a
+small provenance envelope, so it is human-readable, diffable, and editable —
+flip an `enabled`, nudge a rollout, commit a fixture. **Edits are picked up
+live** (the file is watched); no restart.
+
+You can also redirect *any* client to a file with **no code change** by setting
+`TOGGLD_FLAGS_FILE` — it overrides the network source even under the default
+remote mode:
+
+```bash
+TOGGLD_FLAGS_FILE=./toggld.flags.json node server.js
+```
+
+`pull` needs a secret key (`ffk_…`); it is one-shot and idempotent (an unchanged
+version is not rewritten — pass `--force` to override). See
+[ADR-0013](../../docs/adr/0013-local-flag-files-and-pull-cli.md). Remote-segment
+membership (ADR-0006) is not yet available offline; a file-backed client warns
+once and treats a remote segment's non-rule-matched subjects as non-members.
+
 ## Error semantics
 
 Evaluation never throws. On a network failure, timeout, unknown flag, server
@@ -117,3 +166,53 @@ error, or a value whose type doesn't match the requested getter, the call
 resolves to your **default value**. The `*Details` methods carry `reason:
 "ERROR"` and an OpenFeature `errorCode` (`FLAG_NOT_FOUND`, `TYPE_MISMATCH`,
 `TARGETING_KEY_MISSING`, `PARSE_ERROR`, `GENERAL`, …) describing what happened.
+
+
+## Evaluation telemetry
+
+In network-synced local mode, the SDK collects evaluation counts automatically.
+It groups counts by flag, variant, result reason, and UTC minute. The SDK sends
+batches to `POST /v1/telemetry` every 30 seconds and during `close()`.
+Evaluation calls do not wait for these requests. Both SDK APIs use this reporter.
+
+The SDK reports engine results for known flags, including engine errors.
+It excludes unknown flags, calls before readiness, and errors from typed getters.
+Each bulk evaluation counts each flag once. The SDK sends no user identities,
+evaluation context, values, or metadata. Remote mode relies on server collection.
+File-based mode sends no telemetry, even when credentials are present.
+
+```ts
+const client = await ToggldClient.create({
+  endpoint,
+  sdkKey,
+  mode: "local",
+  telemetry: { flushIntervalMs: 30_000 },
+})
+
+await client.getBooleanValue("checkout", false, { targetingKey: "user-123" })
+await client.flushTelemetry()
+await client.close()
+```
+
+Set `telemetry: false` to disable reporting. Use `flushTelemetry()` to send counts
+before the next interval. Effect users can yield `client.flushTelemetry` and close
+the layer scope to flush remaining counts. A flush failure leaves the batch
+queued for the next attempt. Retries keep the same batch ID to prevent duplicate
+counts after a lost response.
+
+The SDK holds at most 10,000 active groups and 10,000 groups awaiting retry.
+If the active buffer fills, the SDK drops new groups. Each request contains at
+most 1,000 groups. Each count stops at 1,000,000,000. Keys and variants longer
+than 256 characters do not produce telemetry. Process crashes can lose buffered
+counts. Closing the client limits its final telemetry attempt to `timeoutMs`,
+which defaults to five seconds. Counts remain approximate.
+
+
+The integration suite can use Postgres for the complete SDK-to-storage path.
+It creates tenants and flags, so use a disposable database. From the repository
+root, set `DATABASE_URL` to that database and run the command below. The test
+server applies migrations before it starts.
+
+```bash
+TOGGLD_TEST_POSTGRES=1 DATABASE_URL=postgres://postgres:postgres@localhost:55439/toggld pnpm --filter @toggld/node exec vitest run test/telemetry.test.ts
+```

package/dist/cli.d.mts

@@ -0,0 +1 @@
+export {}

package/dist/cli.mjs

@@ -0,0 +1,235 @@
+#!/usr/bin/env node
+import { t as FlagSnapshot } from "./src-B_M-Oix8.mjs";
+import { Effect, Schema } from "effect";
+import { readFile, writeFile } from "node:fs/promises";
+// src/internal/cli.ts
+/**
+* The `toggld` CLI (ADR-0013). Today it has one command:
+*
+*   toggld pull [--sdk-key <key>] [--api <url>] [--out <path>] [--env-name <name>] [--force]
+*
+* `pull` does exactly what a local-mode SDK does on bootstrap — `GET
+* /sync/v1/flags` with the SDK key — then writes the validated `FlagSnapshot` to
+* disk as a local flag file (the wire snapshot plus a small provenance
+* envelope). Point a file-backed client at it for offline local evaluation.
+*
+* This module is side-effect-free on import: {@link runPull} performs the fetch
+* and write, and {@link runMain} parses argv and prints. The executable entry
+* (`../cli.ts`) is a one-line wrapper. Keeping the logic here lets tests call it
+* directly instead of spawning a process.
+*/
+const SCHEMA_URL = "https://toggld.dev/schema/local-flags.v1.json";
+const DEFAULT_OUT = "./toggld.flags.json";
+const DEFAULT_ENDPOINT = "http://localhost:3000";
+/** A user-facing failure: printed as a clean message, never a stack trace. */
+var CliError = class extends Error {
+	/** An optional follow-up line with remediation guidance. */
+	hint;
+	constructor(message, hint) {
+		super(message);
+		this.name = "CliError";
+		this.hint = hint;
+	}
+};
+const decodeSnapshot = Schema.decodeUnknownEffect(FlagSnapshot);
+const normalizeEndpoint = (endpoint) => endpoint.replace(/\/+$/, "");
+/** The version already on disk at `out`, if it is a readable local flag file. */
+const existingVersion = async (out) => {
+	try {
+		const parsed = JSON.parse(await readFile(out, "utf8"));
+		return typeof parsed.version === "string" ? parsed.version : void 0;
+	} catch {
+		return;
+	}
+};
+/**
+* Fetch the environment's current snapshot for `sdkKey` and write it to `out` as
+* a local flag file. Idempotent: an unchanged version is a no-op unless `force`.
+*/
+const runPull = async (options) => {
+	const fetchImpl = options.fetch ?? globalThis.fetch;
+	const now = options.now ?? (() => (/* @__PURE__ */ new Date()).toISOString());
+	const endpoint = normalizeEndpoint(options.endpoint);
+	const url = `${endpoint}/sync/v1/flags`;
+	let response;
+	try {
+		response = await fetchImpl(url, { headers: {
+			authorization: `Bearer ${options.sdkKey}`,
+			accept: "application/json"
+		} });
+	} catch (error) {
+		throw new CliError(`Could not reach the flag API at ${endpoint}: ${describe(error)}`, "Check --api / TOGGLD_FLAG_API_URL and that the server is running.");
+	}
+	if (response.status === 401) throw new CliError("The flag API rejected the SDK key (401 Unauthorized).", "Check the key.");
+	if (response.status === 403) throw new CliError("This key cannot read flag definitions (403 Forbidden).", "`pull` needs a secret SDK key (ffk_…); public browser tokens (ffp_…) can't read definitions.");
+	if (!response.ok) throw new CliError(`The flag API returned ${response.status} ${response.statusText}.`);
+	const body = await response.json().catch(() => {
+		throw new CliError("The flag API response was not valid JSON.");
+	});
+	const snapshot = await Effect.runPromise(decodeSnapshot(body)).catch(() => {
+		throw new CliError("The flag API response did not match the expected snapshot format.");
+	});
+	if (!options.force && await existingVersion(options.out) === snapshot.version) return {
+		out: options.out,
+		version: snapshot.version,
+		flagCount: snapshot.flags.length,
+		segmentCount: snapshot.segments?.length ?? 0,
+		unchanged: true
+	};
+	const envelope = {
+		$schema: SCHEMA_URL,
+		toggld: {
+			...options.envName ? { environment: options.envName } : {},
+			apiBase: endpoint,
+			generatedAt: now(),
+			generatedBy: "toggld pull"
+		},
+		version: snapshot.version,
+		flags: snapshot.flags,
+		...snapshot.segments ? { segments: snapshot.segments } : {}
+	};
+	await writeFile(options.out, `${JSON.stringify(envelope, null, 2)}\n`, "utf8");
+	return {
+		out: options.out,
+		version: snapshot.version,
+		flagCount: snapshot.flags.length,
+		segmentCount: snapshot.segments?.length ?? 0,
+		unchanged: false
+	};
+};
+const describe = (e) => typeof e === "object" && e !== null && "message" in e && typeof e.message === "string" ? e.message : String(e);
+const USAGE = `toggld — Toggld feature-flag CLI
+
+Usage:
+  toggld pull [options]      Fetch an environment's flags into a local file
+
+Options for "pull":
+  -k, --sdk-key <key>   Secret SDK key (ffk_…). Falls back to TOGGLD_SDK_KEY,
+                        then prompts. Required.
+  -a, --api <url>       Flag API base URL. Falls back to TOGGLD_FLAG_API_URL,
+                        then ${DEFAULT_ENDPOINT}.
+  -o, --out <path>      Output file. Default ${DEFAULT_OUT}.
+      --env-name <name> Label recorded in the file's provenance envelope.
+  -f, --force           Rewrite even if the file already has this version.
+  -h, --help            Show this help.
+
+Point a local-mode client at the file with TOGGLD_FLAGS_FILE or
+{ sync: { file: "<path>" } }.`;
+const BOOLEAN_FLAGS = /* @__PURE__ */ new Set([
+	"force",
+	"f",
+	"help",
+	"h"
+]);
+const ALIASES = {
+	k: "sdk-key",
+	a: "api",
+	o: "out",
+	f: "force",
+	h: "help"
+};
+const parseArgs = (argv) => {
+	const flags = {};
+	let command;
+	for (let i = 0; i < argv.length; i++) {
+		const arg = argv[i];
+		if (!arg.startsWith("-")) {
+			command ??= arg;
+			continue;
+		}
+		const eq = arg.indexOf("=");
+		const rawName = (eq === -1 ? arg : arg.slice(0, eq)).replace(/^-+/, "");
+		const name = ALIASES[rawName] ?? rawName;
+		const canonical = ALIASES[rawName] ?? rawName;
+		if (BOOLEAN_FLAGS.has(rawName) || BOOLEAN_FLAGS.has(canonical)) {
+			flags[name] = true;
+			continue;
+		}
+		if (eq !== -1) flags[name] = arg.slice(eq + 1);
+		else flags[name] = argv[++i] ?? "";
+	}
+	return {
+		command,
+		flags
+	};
+};
+/** Read a secret without echoing it, from an interactive terminal. */
+const promptSecret = async (label) => {
+	if (!process.stdin.isTTY) throw new CliError("No SDK key provided.", "Pass --sdk-key, set TOGGLD_SDK_KEY, or run in an interactive terminal.");
+	const { createInterface } = await import("node:readline");
+	const rl = createInterface({
+		input: process.stdin,
+		output: process.stdout,
+		terminal: true
+	});
+	const internals = rl;
+	const write = internals.output.write.bind(internals.output);
+	let muted = false;
+	internals._writeToOutput = (s) => {
+		if (!muted || s.includes(label)) write(s);
+	};
+	try {
+		const answer = await new Promise((resolve) => {
+			rl.question(label, resolve);
+			muted = true;
+		});
+		write("\n");
+		return answer.trim();
+	} finally {
+		rl.close();
+	}
+};
+/**
+* Parse argv, run the requested command, and return a process exit code. Prints
+* results to stdout and {@link CliError}s to stderr; unexpected errors rethrow.
+*/
+const runMain = async (argv) => {
+	const { command, flags } = parseArgs(argv);
+	if (flags["help"] || command === void 0 || command === "help") {
+		process.stdout.write(`${USAGE}\n`);
+		return command === void 0 && !flags["help"] ? 1 : 0;
+	}
+	if (command !== "pull") {
+		process.stderr.write(`Unknown command "${command}".\n\n${USAGE}\n`);
+		return 1;
+	}
+	try {
+		const endpoint = asString(flags["api"]) ?? process.env.TOGGLD_FLAG_API_URL ?? DEFAULT_ENDPOINT;
+		const out = asString(flags["out"]) ?? DEFAULT_OUT;
+		const envName = asString(flags["env-name"]);
+		const sdkKey = asString(flags["sdk-key"]) ?? (process.env.TOGGLD_SDK_KEY || void 0) ?? await promptSecret("Toggld SDK key: ");
+		if (!sdkKey) throw new CliError("No SDK key provided.");
+		const result = await runPull({
+			sdkKey,
+			endpoint,
+			out,
+			force: Boolean(flags["force"]),
+			...envName ? { envName } : {}
+		});
+		if (result.unchanged) process.stdout.write(`✓ ${result.out} already up to date (version ${short(result.version)}).\n`);
+		else {
+			const segs = result.segmentCount > 0 ? `, ${result.segmentCount} segments` : "";
+			process.stdout.write(`✓ Wrote ${result.out} — ${result.flagCount} flags${segs} (version ${short(result.version)}).\n`);
+		}
+		return 0;
+	} catch (error) {
+		if (error instanceof CliError) {
+			process.stderr.write(`✗ ${error.message}\n`);
+			if (error.hint) process.stderr.write(`  ${error.hint}\n`);
+			return 1;
+		}
+		throw error;
+	}
+};
+const asString = (v) => typeof v === "string" && v.length > 0 ? v : void 0;
+const short = (version) => version.length > 12 ? `${version.slice(0, 12)}…` : version;
+// src/cli.ts
+/**
+* The `toggld` executable. All logic lives in `./internal/cli.ts` (so tests can
+* call it without spawning a process); this is just the entry that runs it.
+*/
+runMain(process.argv.slice(2)).then((code) => process.exit(code), (error) => {
+	console.error(error);
+	process.exit(1);
+});
+export {};