@configdirector/cli 0.21.0 → 0.22.0

package/CHANGELOG.md

@@ -4,6 +4,12 @@ All notable changes to `@configdirector/cli` are documented here. The format fol
 
 ## [Unreleased]
 
+## [0.22.0] - 2026-11-11
+
+### Added
+
+- `configdirector configs runs` shows how long a run in progress needs, under each rollout's primary metric results once its variations can be compared. For a real lift of 2%, 5%, 10%, 20% and 50% on the primary metric, in its better direction, it gives the exposures so far against those needed before a variation with that lift has a good chance of reading likely better, and about how many days remain at the pace of new exposures since the time it names. A lift the rollout has enough exposures for says so, more than 365 days reads as more than a year, and when no new contexts were exposed since that time, it says the days cannot be estimated and gives the exposures alone. It also says that a row's lift is one a variation might really have, not the lift measured so far, and that whatever the estimate says, a run should cover at least one full week. The estimate doesn't show while the control's numbers give nothing to estimate from, as when none of its contexts has sent the event yet. `--json` prints the Admin API's response as received, as before.
+
 ## [0.21.0] - 2026-11-10
 
 ### Added

package/README.md

@@ -342,7 +342,15 @@ Running since 2026-11-03 12:00 UTC
     810 contexts exposed, 3201 evaluations. The split matches the configured shares.
     Primary metric: checkout_completed, conversion, higher is better
       control (Start free trial)              2.9%   control
-      treatment-a (Try it free for 14 days)   4.5%   +55% (+12% to +98%)   >99% chance to beat control   likely better
+      treatment-a (Try it free for 14 days)   3.5%   +19% (-43% to +154%)   69% chance to beat control   no clear difference
+    How long to run, at the pace since 2026-11-04 00:00 UTC: about 98 new exposures a day
+      For a real lift of +2%    810 of about 2,082,762 exposures   more than a year remaining
+      For a real lift of +5%    810 of about 338,034 exposures     more than a year remaining
+      For a real lift of +10%   810 of about 86,501 exposures      more than a year remaining
+      For a real lift of +20%   810 of about 22,617 exposures      about 222 days remaining
+      For a real lift of +50%   810 of about 4,089 exposures       about 34 days remaining
+    A row's lift is one a variation might really have over the control, not the lift measured so far: the smaller it is, the longer the run.
+    Whatever the estimate says, a run should cover at least one full week, so every day of the week is in the result.
     Guardrail: support_ticket_opened, count per context, lower is better
       control (Start free trial)              0.12   control
       treatment-a (Try it free for 14 days)   0.13   +8% (-10% to +29%)   22% chance to beat control   no clear difference
@@ -350,7 +358,7 @@ Running since 2026-11-03 12:00 UTC
   Results can be up to 25 minutes behind.
 ```
 
-Each kind a metric reports gets a line with the metric's role, its event name, the kind and its direction, then one row per variation, the control first: its value, and for every other variation the lift against the control with the range it is likely in, the chance to beat control and the verdict, `likely better`, `likely worse` or `no clear difference`. With three or more variations, each row also shows the variation's chance to be best. Until every variation has enough exposures, the rollout says the run needs more exposures and shows the values alone. An ended run's last line says when its final results were taken, once they are in. A run also shows its hypothesis and the note it ended with, and lists the events sent with `track` during the run that no metric measures.
+Each kind a metric reports gets a line with the metric's role, its event name, the kind and its direction, then one row per variation, the control first: its value, and for every other variation the lift against the control with the range it is likely in, the chance to beat control and the verdict, `likely better`, `likely worse` or `no clear difference`. With three or more variations, each row also shows the variation's chance to be best. Until every variation has enough exposures, the rollout says the run needs more exposures and shows the values alone. While a run is going, `How long to run` follows the primary metric's results: for a real lift of 2%, 5%, 10%, 20% and 50%, in the metric's better direction, the exposures so far against those needed before a variation with that lift has a good chance of reading likely better, and about how many days remain at the pace of new exposures since the time it names. An ended run's last line says when its final results were taken, once they are in. A run also shows its hypothesis and the note it ended with, and lists the events sent with `track` during the run that no metric measures.
 
 - `configs ship` serves one variation to everyone in the environment and ends the run. `--variation` takes its name or its ID. The default targeting rule value becomes the variation's value, every percentage rollout is removed, including one a conditional rule serves, and a conditional rule serving a single value stays.
 - `configs stop` ends the run without choosing a variation: every percentage rollout is removed, including one a conditional rule serves, and the default targeting rule value and any conditional rule serving a single value stay.

package/dist/configdirector.mjs

@@ -14,7 +14,7 @@ import { promisify } from "node:util";
 import { randomUUID } from "node:crypto";
 import { text } from "node:stream/consumers";
 //#region package.json
-var version = "0.21.0";
+var version = "0.22.0";
 //#endregion
 //#region src/CliError.ts
 var CliError = class extends Error {
@@ -146,11 +146,22 @@ const VariationResultSchema = v.object({
 	chanceToBeBest: v.optional(v.number()),
 	verdict: v.optional(v.string())
 });
+const RunLengthEstimateSchema = v.object({
+	analyzedExposures: v.number(),
+	exposuresPerDay: v.number(),
+	paceMeasuredSince: TimestampSchema,
+	lifts: v.array(v.object({
+		lift: v.number(),
+		exposuresNeeded: v.number(),
+		daysRemaining: v.optional(v.number())
+	}))
+});
 const MetricKindResultSchema = v.object({
 	kind: v.string(),
 	rollouts: v.array(v.object({
 		ruleId: v.string(),
-		variations: v.array(VariationResultSchema)
+		variations: v.array(VariationResultSchema),
+		runLengthEstimate: v.optional(RunLengthEstimateSchema)
 	}))
 });
 const MetricResultSchema = v.object({
@@ -1676,9 +1687,53 @@ const resultRow = (variation, table) => {
 	}
 };
 const capitalized = (text) => `${text.charAt(0).toUpperCase()}${text.slice(1)}`;
+const DAYS_IN_A_YEAR = 365;
+const formatCount = (count) => count.toLocaleString("en-US");
+const liftRowFormOf = (daysRemaining) => {
+	if (daysRemaining === void 0) return { form: "no-pace" };
+	if (daysRemaining === 0) return { form: "enough-exposures" };
+	return daysRemaining > DAYS_IN_A_YEAR ? { form: "more-than-a-year" } : {
+		form: "days-remaining",
+		days: daysRemaining
+	};
+};
+const daysRemainingCells = (liftRow) => {
+	switch (liftRow.form) {
+		case "days-remaining": return [`about ${formatCount(liftRow.days)} ${plural(liftRow.days, "day", "days")} remaining`];
+		case "more-than-a-year": return ["more than a year remaining"];
+		case "enough-exposures": return ["enough exposures"];
+		case "no-pace": return [];
+	}
+};
+const exposurePaceOf = (estimate) => estimate.exposuresPerDay > 0 ? {
+	form: "new-exposures",
+	exposuresPerDay: Math.round(estimate.exposuresPerDay)
+} : { form: "no-new-exposures" };
+const paceLine = (estimate) => {
+	const since = utcMinute(estimate.paceMeasuredSince);
+	const pace = exposurePaceOf(estimate);
+	switch (pace.form) {
+		case "new-exposures": return `How long to run, at the pace since ${since}: about ${formatCount(pace.exposuresPerDay)} ${plural(pace.exposuresPerDay, "new exposure", "new exposures")} a day`;
+		case "no-new-exposures": return `How long to run: no new contexts were exposed since ${since}, so the days cannot be estimated.`;
+	}
+};
+const runLengthEstimateLines = (estimate) => {
+	if (estimate === void 0) return [];
+	return [
+		paceLine(estimate),
+		...alignedColumns(estimate.lifts.map((liftEstimate) => [
+			`For a real lift of ${formatLift(liftEstimate.lift)}`,
+			`${formatCount(estimate.analyzedExposures)} of about ${formatCount(liftEstimate.exposuresNeeded)} exposures`,
+			...daysRemainingCells(liftRowFormOf(liftEstimate.daysRemaining))
+		])).map((line) => `  ${line}`),
+		"A row's lift is one a variation might really have over the control, not the lift measured so far: the smaller it is, the longer the run.",
+		"Whatever the estimate says, a run should cover at least one full week, so every day of the week is in the result."
+	];
+};
 const kindResultLines = (role, metric, kindResult, rollout, variations) => {
-	const variationResults = kindResult.rollouts.find((candidate) => candidate.ruleId === rollout.ruleId)?.variations;
-	if (variationResults === void 0) return [];
+	const kindResultOfRollout = kindResult.rollouts.find((candidate) => candidate.ruleId === rollout.ruleId);
+	if (kindResultOfRollout === void 0) return [];
+	const variationResults = kindResultOfRollout.variations;
 	const table = {
 		kind: kindResult.kind,
 		controlVariationId: rollout.controlVariationId,
@@ -1686,7 +1741,11 @@ const kindResultLines = (role, metric, kindResult, rollout, variations) => {
 		variations,
 		showsChanceToBeBest: variationResults.some((variation) => variation.chanceToBeBest !== void 0)
 	};
-	return [`${capitalized(roleWords(role))}: ${metric.eventName}, ${kindWords(kindResult.kind)}, ${directionWords(metric.direction)}`, ...alignedColumns(variationResults.map((variation) => resultRow(variation, table))).map((line) => `  ${line}`.trimEnd())];
+	return [
+		`${capitalized(roleWords(role))}: ${metric.eventName}, ${kindWords(kindResult.kind)}, ${directionWords(metric.direction)}`,
+		...alignedColumns(variationResults.map((variation) => resultRow(variation, table))).map((line) => `  ${line}`.trimEnd()),
+		...runLengthEstimateLines(kindResultOfRollout.runLengthEstimate)
+	];
 };
 const crossoverLines = (crossovers) => {
 	if (crossovers === 0) return [];

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@configdirector/cli",
-  "version": "0.21.0",
+  "version": "0.22.0",
   "description": "Command line tool for ConfigDirector, the managed configuration and feature flag service.",
   "license": "MIT",
   "author": "ConfigDirector",