@configdirector/cli 0.19.0 → 0.21.0

package/CHANGELOG.md

@@ -4,6 +4,29 @@ All notable changes to `@configdirector/cli` are documented here. The format fol
 
 ## [Unreleased]
 
+## [0.21.0] - 2026-11-10
+
+### Added
+
+- `configdirector metrics list` lists the project's metrics, each an event name your application sends with `track`, with the kinds it reports, its direction and how many experiments use it, and `configdirector metrics get <event name>` shows one metric with each experiment that uses it, its role there, and the environments where a run is in progress. `configdirector metrics create --event-name <event name> --kind <kind> --direction <direction>`, with `--kind` repeatable and an optional `--description`, creates a metric; `configdirector metrics update <event name>` changes its kinds, direction or description; and `configdirector metrics delete <event name>` deletes it and detaches it from its experiments. They exit with 2, listing the experiments, when ConfigDirector refuses to remove a kind an experiment's primary metric decides on, or to delete a metric an experiment with a run in progress uses. `--json` prints the Admin API's response as received, `metrics get --json` the metric as the Admin API listed it for its event name, and `metrics delete --json` `{ "deletedEventName": ... }`.
+- `configdirector configs runs` shows the results of the metrics an experiment measures under each rollout: per metric kind, each variation's value against the control, with the lift and the range it is likely in, the chance to beat control, the chance to be best with three or more variations, and the verdict, likely better, likely worse or no clear difference. It says when a rollout needs more exposures before the variations can be compared and how many contexts were exposed to more than one variation, that results can be up to 25 minutes behind on a running run, and when an ended run's final results were taken. Each run also shows its hypothesis and the note it ended with, and lists the events sent with `track` during the run that no metric measures. `--json` prints the Admin API's response as received, as before.
+- `configdirector configs ship` and `configs stop` take `--note <text>`, up to 1,000 characters on why, which the ended run keeps.
+- `configdirector configs create` takes `--hypothesis <text>`, what an experiment is expected to do, which each of its runs keeps as it was when the run started.
+
+### Changed
+
+- `configdirector configs copy --help` lists the hypothesis among what a copy gets.
+
+### Fixed
+
+- `configdirector configs runs` describes the evaluations it leaves out of the exposures as coming from anonymous contexts: in a server SDK, a context without an identifier skips the split and is served the default targeting rule value, and any other anonymous context sees its share of the split without being stored. It said every such evaluation was served the default targeting rule value.
+
+## [0.20.0] - 2026-11-06
+
+### Changed
+
+- `configdirector evaluate` says when a percentage rollout was skipped because the context has no identifier. The rule's line ends with `rollout skipped: the context has no identifier`, where it showed `not matched` with no reason, both for a percentage rule and for a conditional rule that serves a percentage rollout once its conditions matched. The rules after it show `not evaluated` and the context gets the default targeting rule value, as before. `--json` prints the Admin API's response as received, as before.
+
 ## [0.19.0] - 2026-11-04
 
 ### Added

package/README.md

@@ -1,6 +1,6 @@
 # ConfigDirector CLI
 
-`configdirector` is the command line tool for [ConfigDirector](https://www.configdirector.com), the managed configuration and feature flag service. Commands and output may change before 1.0. This release signs in to one or more organizations, links a repository to a project, lists where the repository's code reads config keys, checks those keys against the project's configs, shows the value a context gets for a config and why, reads a project's environments, configs, targeting rules and segments, creates and copies configs, replaces a config's targeting rules in an environment, lists an experiment's runs and ships or stops them, archives configs, creates, updates and deletes segments, uploads the repository's code references to the project, and lists and removes the project's scanned repositories.
+`configdirector` is the command line tool for [ConfigDirector](https://www.configdirector.com), the managed configuration and feature flag service. Commands and output may change before 1.0. This release signs in to one or more organizations, links a repository to a project, lists where the repository's code reads config keys, checks those keys against the project's configs, shows the value a context gets for a config and why, reads a project's environments, configs, targeting rules, segments and metrics, creates and copies configs, replaces a config's targeting rules in an environment, lists an experiment's runs with their results and ships or stops them, archives configs, creates, updates and deletes segments and metrics, uploads the repository's code references to the project, and lists and removes the project's scanned repositories.
 
 The [CLI page of the ConfigDirector docs](https://docs.configdirector.com/integrations/cli) covers the API token, CI setup for GitHub Actions and GitLab, and what each problem `validate` reports means for your build.
 
@@ -159,7 +159,7 @@ true, served by rule 2
 Default targeting rule: false
 ```
 
-A rule that did not match names the first condition that failed and the context's value for it, as in `plan is one of pro, team, but it is "free"`, or the bucket a percentage rollout put the context in. A rule after the one that served the value is not evaluated, and a rule that could not be evaluated is shown as errored.
+A rule that did not match names the first condition that failed and the context's value for it, as in `plan is one of pro, team, but it is "free"`, or the bucket a percentage rollout put the context in. A context without an identifier gets no bucket: the first percentage rollout it reaches shows `rollout skipped: the context has no identifier`, and the context gets the default targeting rule value. A rule after the one that served the value, or after a skipped rollout, is not evaluated, and a rule that could not be evaluated is shown as errored.
 
 `--environment` takes the environment's slug and is required. Give the context one of three ways:
 
@@ -181,6 +181,8 @@ configdirector configs get <key>
 configdirector configs activity <key>
 configdirector segments list
 configdirector segments get <key>
+configdirector metrics list
+configdirector metrics get <event name>
 ```
 
 `projects list` lists your organization's projects with their slugs and names, whichever project is selected. The other commands read the project from `--project`, `CONFIGDIRECTOR_PROJECT` or `configdirector.json`, as `scan` does:
@@ -190,6 +192,7 @@ configdirector segments get <key>
 - `configs get <key>` shows the config's settings, including its type's constraints, then each environment's targeting rules in order, written as `evaluate` writes them, with the default targeting rule value last. An old key works, and the output then starts with a line naming the current key.
 - `configs activity <key>` shows when SDKs last evaluated the config, one line per environment, SDK key kind, SDK name and SDK version, with the time in UTC.
 - `segments list` lists the segments with how many configs use each, and `segments get <key>` shows a segment's condition groups and its environment overrides.
+- `metrics list` lists the metrics with their kinds, their direction and how many experiments use each, and `metrics get <event name>` shows a metric's kinds and direction, and each experiment that uses it, with its role and the environments of its runs in progress.
 
 ```txt
 new-checkout
@@ -213,11 +216,11 @@ production (live)
 
 On an experiment, each value comes after the name of its variation, as in `40% get control (Start free trial), 60% get treatment-a (Try it free for 14 days)`. `evaluate` names the variation it served the same way, as in `treatment-a (Try it free for 14 days), served by rule 2`, and `configs create` names the variation of the default targeting rule value it starts with.
 
-`projects list`, `environments list`, `configs list`, `configs activity` and `segments list` print a header row that names their columns above their lines, such as `KEY   TYPE   ROLE   LIFETIME   STATE` for `configs list`, and `--no-headers` leaves it out. An empty list prints only a sentence, such as `No configs found in web-app.`
+`projects list`, `environments list`, `configs list`, `configs activity`, `segments list` and `metrics list` print a header row that names their columns above their lines, such as `KEY   TYPE   ROLE   LIFETIME   STATE` for `configs list`, and `--no-headers` leaves it out. An empty list prints only a sentence, such as `No configs found in web-app.`
 
-With `--json`, each prints the Admin API's response as received, except `configs list --state`, which prints only the configs in that state. A key no config or segment has exits with 2.
+With `--json`, each prints the Admin API's response as received, except `configs list --state`, which prints only the configs in that state, and `metrics get`, which prints the metric as the Admin API listed it for its event name. A key no config or segment has, or an event name no metric has, exits with 2.
 
-`projects list` needs `projects:read`. The others need `projects:read` to find the project, and `environments list` needs `environments:read`, `configs list`, `configs get` and `configs activity` need `configs:read`, plus `tags:read` for `--tag`, and `segments list` and `segments get` need `segments:read`. Without `segments:read`, `configs get` shows a segment condition's segment ID instead of its name; without `environments:read`, `segments get` names an override's environment by its ID instead of its slug.
+`projects list` needs `projects:read`. The others need `projects:read` to find the project, and `environments list` needs `environments:read`, `configs list`, `configs get` and `configs activity` need `configs:read`, plus `tags:read` for `--tag`, `segments list` and `segments get` need `segments:read`, and `metrics list` and `metrics get` need `metrics:read`. Without `segments:read`, `configs get` shows a segment condition's segment ID instead of its name; without `environments:read`, `segments get` names an override's environment by its ID instead of its slug, and `metrics get` a run's environment.
 
 ## Create a config
 
@@ -234,7 +237,7 @@ Default targeting rule in every environment: false
 
 - `--key`, `--type` (`boolean`, `string`, `integer`, `float`, `timespan`, `enum`, `url` or `json`), `--role` (`config`, `flag`, `kill-switch` or `experiment`) and `--default-value` are required. `--default-value` is the default targeting rule value every environment starts with, not the in-code default value an application passes to the SDK.
 - `--lifetime` (`temporary` or `permanent`) is required for the `config` role. The role decides the lifetime of the others, so you can leave it out: a feature flag and an experiment are always temporary, and a kill switch is always permanent. A lifetime the role does not have is a usage error.
-- `--description <text>`; `--client-only` or `--server-only`, without which client and server SDKs both receive the config; and `--variation <name=value>`, repeatable, for an experiment's variations.
+- `--description <text>`; `--client-only` or `--server-only`, without which client and server SDKs both receive the config; `--variation <name=value>`, repeatable, for an experiment's variations; and `--hypothesis <text>`, what an experiment is expected to do, which each of its runs keeps as it was when the run started.
 - `--tag <name>`, repeatable, names a tag the project already has, and `--new-tag <name>`, repeatable, creates a tag with that name in the project's tag catalog. The config gets the tags of both, at most 20 in all. Every `--tag` name is looked up before any tag is created, so a misspelled `--tag` creates nothing; a name no tag has, a `--new-tag` name a tag already has, a `--new-tag` name given twice and a name given with both options exit with 2. When a later step fails, such as a key that is taken, the tags already created stay in the catalog, and the command names them so you can pass them with `--tag` next time.
 - The values of `--default-value` and `--variation` are read in the config's type: `true` or `false`, a number for an integer, float or timespan, the text as written for a string, enum or URL, and JSON for a JSON config, with `@file` reading it from a file. A value that does not fit the type is a usage error.
 - Constraints, each accepted only by its types: `--min` and `--max` (a value may equal them) for an integer, float or timespan; `--exclusive-min` and `--exclusive-max` for an integer or float; `--unit` (`milliseconds`, `seconds` or `minutes`) and `--allow-fractions` for a timespan, which needs `--unit`, `--min` and `--max`; `--allowed-value`, repeatable, and `--enum-value-type string|number` (default `string`) for an enum; `--https-only` for a URL, without which any HTTP or HTTPS URL is accepted; and `--json-schema <file>` for a JSON config. An option of another type is a usage error naming the types it belongs to. A timespan without `--unit`, `--min` or `--max` is a usage error too.
@@ -326,22 +329,33 @@ configdirector configs ship checkout-cta-copy --environment production --variati
 configdirector configs stop checkout-cta-copy --environment production
 ```
 
-A run is a period during which an experiment's targeting rules in one environment split traffic between its variations. `configs runs` lists the experiment's runs in an environment, newest first, with how many distinct contexts each variation reached, called exposures, against the share configured when the run started:
+A run is a period during which an experiment's targeting rules in one environment split traffic between its variations. `configs runs` lists the experiment's runs in an environment, newest first, with how many distinct contexts each variation reached, called exposures, against the share configured when the run started, and the results of the metrics the experiment measures:
 
 ```txt
 Runs of checkout-cta-copy in production (live), newest first:
 
 Running since 2026-11-03 12:00 UTC
+  Hypothesis: Urgency copy raises checkout completion by 5%
   Rollout: everyone not caught by a rule above
     control (Start free trial)              412 exposures   configured 50%
     treatment-a (Try it free for 14 days)   398 exposures   configured 50%
     810 contexts exposed, 3201 evaluations. The split matches the configured shares.
+    Primary metric: checkout_completed, conversion, higher is better
+      control (Start free trial)              2.9%   control
+      treatment-a (Try it free for 14 days)   4.5%   +55% (+12% to +98%)   >99% chance to beat control   likely better
+    Guardrail: support_ticket_opened, count per context, lower is better
+      control (Start free trial)              0.12   control
+      treatment-a (Try it free for 14 days)   0.13   +8% (-10% to +29%)   22% chance to beat control   no clear difference
+  Events with no metric: add_to_cart (340 contexts)
+  Results can be up to 25 minutes behind.
 ```
 
+Each kind a metric reports gets a line with the metric's role, its event name, the kind and its direction, then one row per variation, the control first: its value, and for every other variation the lift against the control with the range it is likely in, the chance to beat control and the verdict, `likely better`, `likely worse` or `no clear difference`. With three or more variations, each row also shows the variation's chance to be best. Until every variation has enough exposures, the rollout says the run needs more exposures and shows the values alone. An ended run's last line says when its final results were taken, once they are in. A run also shows its hypothesis and the note it ended with, and lists the events sent with `track` during the run that no metric measures.
+
 - `configs ship` serves one variation to everyone in the environment and ends the run. `--variation` takes its name or its ID. The default targeting rule value becomes the variation's value, every percentage rollout is removed, including one a conditional rule serves, and a conditional rule serving a single value stays.
 - `configs stop` ends the run without choosing a variation: every percentage rollout is removed, including one a conditional rule serves, and the default targeting rule value and any conditional rule serving a single value stay.
 
-Both print the rules they saved, ask for no confirmation and treat a live environment like any other, so check the environment before you run them.
+Both take `--note <text>`, up to 1,000 characters on why, which the ended run keeps. Both print the rules they saved, ask for no confirmation and treat a live environment like any other, so check the environment before you run them.
 
 `configs ship` exits with 2 when `--variation` matches none of the experiment's variations, or several with the same name, and lists them with their IDs, and when the config has no variations, since only an experiment has them. `configs stop` exits with 2 when the config is not an experiment or the environment holds no percentage rollout. All three exit with 2 when no config has the key or no environment has the slug. With `--json`, they print the Admin API's response as received.
 
@@ -400,6 +414,24 @@ configdirector segments delete beta-testers
 
 `segments create` needs `projects:read` and `segments:create`, `segments update` needs `projects:read`, `segments:read` and `segments:update`, and `segments delete` needs `projects:read` and `segments:delete`. Without `environments:read`, they name environments by ID instead of by slug. The change shows in the audit log with the source CLI.
 
+## Change metrics
+
+```bash
+configdirector metrics create --event-name checkout_completed --kind conversion --kind value-per-context --direction higher-is-better
+configdirector metrics update checkout_completed --direction lower-is-better
+configdirector metrics delete checkout_completed
+```
+
+A metric is what an experiment measures: an event name the application sends with the SDK's `track` call, the numbers it reports and whether a higher or a lower number is better. A project has one metric per event name, and the commands name a metric by its event name. Attaching metrics to an experiment happens on its Settings tab in the dashboard, or through the Admin API or the MCP server.
+
+- `metrics create` needs `--event-name`, exactly as the application passes it to `track`; `--kind`, repeatable: `conversion`, `count-per-context` or `value-per-context`; and `--direction`: `higher-is-better` or `lower-is-better`. `--description <text>` is optional. It prints `Created metric checkout_completed in web-app.`, then the metric as `metrics get` shows it.
+- `metrics update <event name>` changes what its options name: `--kind`, repeated, replaces the kinds, and `--direction` and `--description` replace theirs; `--description ""` removes the description. A metric keeps its event name. Removing a kind an experiment's primary metric decides on is refused, and the command exits with 2 listing those experiments.
+- `metrics delete <event name>` deletes the metric and detaches it from the experiments that use it, and prints `Deleted metric checkout_completed from web-app.`; `--json` prints `{ "deletedEventName": "checkout_completed" }` with the event name as given. While an experiment that uses the metric has a run in progress, it is not deleted: the command exits with 2 and lists each experiment with the environments of its runs.
+
+None of them asks for confirmation. An event name no metric has exits with 2 for `update` and `delete`, and so does a metric ConfigDirector refuses, such as a second metric for an event name. With `--json`, `create` and `update` print the metric as the Admin API returned it.
+
+`metrics create` needs `projects:read` and `metrics:create`, `metrics update` needs `projects:read`, `metrics:read` and `metrics:update`, and `metrics delete` needs `projects:read`, `metrics:read` and `metrics:delete`. Without `environments:read`, `metrics update` and `metrics delete` name environments by ID instead of by slug. The change shows in the audit log with the source CLI.
+
 ## Archive a config
 
 ```bash

package/dist/configdirector.mjs

@@ -1,7 +1,7 @@
 #!/usr/bin/env node
 import { setDefaultAutoSelectFamilyAttemptTimeout } from "node:net";
 import { homedir } from "node:os";
-import { Command, CommanderError, Option } from "commander";
+import { Command, CommanderError, InvalidArgumentError, Option } from "commander";
 import { setTimeout } from "node:timers/promises";
 import * as v from "valibot";
 import { basename, dirname, extname, isAbsolute, join, relative, resolve, sep } from "node:path";
@@ -14,7 +14,7 @@ import { promisify } from "node:util";
 import { randomUUID } from "node:crypto";
 import { text } from "node:stream/consumers";
 //#region package.json
-var version = "0.19.0";
+var version = "0.21.0";
 //#endregion
 //#region src/CliError.ts
 var CliError = class extends Error {
@@ -134,11 +134,50 @@ const RolloutExposuresSchema = v.object({
 		exposures: v.number()
 	}))
 });
+const VariationResultSchema = v.object({
+	variationId: v.string(),
+	value: v.number(),
+	lift: v.optional(v.number()),
+	interval: v.optional(v.object({
+		low: v.number(),
+		high: v.number()
+	})),
+	chanceToBeatControl: v.optional(v.number()),
+	chanceToBeBest: v.optional(v.number()),
+	verdict: v.optional(v.string())
+});
+const MetricKindResultSchema = v.object({
+	kind: v.string(),
+	rollouts: v.array(v.object({
+		ruleId: v.string(),
+		variations: v.array(VariationResultSchema)
+	}))
+});
+const MetricResultSchema = v.object({
+	eventName: v.string(),
+	direction: v.string(),
+	kinds: v.array(MetricKindResultSchema)
+});
+const RolloutResultsSchema = v.object({
+	ruleId: v.string(),
+	controlVariationId: v.string(),
+	state: v.string(),
+	crossovers: v.number()
+});
+const ResultsSchema = v.object({
+	asOf: v.optional(TimestampSchema),
+	rollouts: v.array(RolloutResultsSchema),
+	primary: v.array(MetricResultSchema),
+	secondary: v.array(MetricResultSchema),
+	guardrail: v.array(MetricResultSchema)
+});
 const ExperimentRunSchema = v.object({
 	startedAt: TimestampSchema,
 	endedAt: v.optional(TimestampSchema),
 	outcome: v.optional(v.string()),
 	shippedVariationId: v.optional(v.string()),
+	hypothesis: v.optional(v.string()),
+	note: v.optional(v.string()),
 	rollouts: v.array(v.object({
 		ruleId: v.string(),
 		conditions: v.optional(v.string())
@@ -147,10 +186,48 @@ const ExperimentRunSchema = v.object({
 		rollouts: v.array(RolloutExposuresSchema),
 		evaluationsWithoutIdentifier: v.number(),
 		evaluationsFromSdksNotReportingExposures: v.number()
-	})
+	}),
+	trackedEvents: v.object({ events: v.array(v.object({
+		name: v.string(),
+		contexts: v.number()
+	})) }),
+	results: ResultsSchema
 });
 const ExperimentRunsSchema = v.array(ExperimentRunSchema);
 //#endregion
+//#region src/admin-api/metrics.ts
+const OpenRunsSchema = v.array(v.object({ environmentId: v.string() }));
+const MetricUsageSchema = v.object({
+	configKey: v.string(),
+	role: v.string(),
+	primaryKind: v.nullable(v.string()),
+	openRuns: OpenRunsSchema
+});
+const MetricSchema = v.object({
+	id: v.string(),
+	eventName: v.string(),
+	description: v.nullable(v.string()),
+	direction: v.string(),
+	kinds: v.array(v.string()),
+	usages: v.array(MetricUsageSchema)
+});
+const DecidingExperimentSchema = v.object({
+	configKey: v.string(),
+	primaryKind: v.string()
+});
+const MetricKindInUseBodySchema = v.object({
+	code: v.literal("METRIC_KIND_IN_USE"),
+	usages: v.array(DecidingExperimentSchema)
+});
+const ExperimentWithOpenRunsSchema = v.object({
+	configKey: v.string(),
+	openRuns: OpenRunsSchema
+});
+const MetricInUseBodySchema = v.object({
+	code: v.literal("METRIC_IN_USE"),
+	usages: v.array(ExperimentWithOpenRunsSchema)
+});
+//#endregion
 //#region src/admin-api/config-creation.ts
 const CreatedConfigSchema = v.object({
 	key: v.string(),
@@ -399,7 +476,6 @@ const ConditionExplanationSchema = v.union([
 	SegmentConditionExplanationSchema
 ]);
 const BucketSchema = v.object({
-	defaultedToPredefinedBucket: v.boolean(),
 	assignedPercentage: v.number(),
 	shares: v.array(v.object({
 		percentageId: v.string(),
@@ -417,7 +493,8 @@ const RuleExplanationSchema = v.object({
 		"errored"
 	]),
 	conditions: v.array(ConditionExplanationSchema),
-	bucket: v.optional(BucketSchema)
+	bucket: v.optional(BucketSchema),
+	rolloutSkippedNoIdentifier: v.boolean()
 });
 const ConfigEvaluationSchema = v.object({
 	key: v.string(),
@@ -547,6 +624,28 @@ const segmentDeletionOutcomeOf = (received) => {
 	if (!received.response.ok) throw failureFor(received.url, received.response, received.body);
 	return { outcome: "deleted" };
 };
+const isMetricKindInUse = (received) => received.response.status === 409 && v.is(MetricKindInUseBodySchema, received.body);
+const metricUpdateOutcomeOf = (received) => {
+	if (isMissingResource(received)) return { outcome: "not-found" };
+	if (isMetricKindInUse(received)) return {
+		outcome: "kind-in-use",
+		decidingExperiments: received.body.usages
+	};
+	return {
+		outcome: "updated",
+		metric: parsedResponseOf(received, MetricSchema)
+	};
+};
+const isMetricInUse = (received) => received.response.status === 409 && v.is(MetricInUseBodySchema, received.body);
+const metricDeletionOutcomeOf = (received) => {
+	if (isMissingResource(received)) return { outcome: "not-found" };
+	if (isMetricInUse(received)) return {
+		outcome: "in-use",
+		experimentsWithOpenRuns: received.body.usages
+	};
+	if (!received.response.ok) throw failureFor(received.url, received.response, received.body);
+	return { outcome: "deleted" };
+};
 const parsedResponseOf = ({ url, response, body }, schema) => {
 	if (!response.ok) throw failureFor(url, response, body);
 	const parsedBody = v.safeParse(schema, body);
@@ -604,6 +703,33 @@ var AdminApiClient = class {
 	async deleteSegment(projectId, segmentKey) {
 		return segmentDeletionOutcomeOf(await this.send(`/v1/projects/${encodeURIComponent(projectId)}/segments/${encodeURIComponent(segmentKey)}`, { method: "DELETE" }));
 	}
+	listMetrics(projectId) {
+		return this.get(`/v1/projects/${encodeURIComponent(projectId)}/metrics`, v.array(MetricSchema));
+	}
+	async findMetricByEventName(projectId, eventName) {
+		const { parsedBody: metrics, responseBody } = await this.get(`/v1/projects/${encodeURIComponent(projectId)}/metrics?${new URLSearchParams({ eventName })}`, v.array(MetricSchema));
+		const index = metrics.findIndex((metric) => metric.eventName === eventName);
+		const metric = metrics[index];
+		return metric === void 0 ? void 0 : {
+			parsedBody: metric,
+			responseBody: v.parse(v.array(v.unknown()), responseBody)[index]
+		};
+	}
+	createMetric(projectId, request) {
+		return this.requestJson(`/v1/projects/${encodeURIComponent(projectId)}/metrics`, MetricSchema, {
+			method: "POST",
+			body: request
+		});
+	}
+	async updateMetric(projectId, metricId, request) {
+		return metricUpdateOutcomeOf(await this.send(`/v1/projects/${encodeURIComponent(projectId)}/metrics/${encodeURIComponent(metricId)}`, {
+			method: "PATCH",
+			body: request
+		}));
+	}
+	async deleteMetric(projectId, metricId) {
+		return metricDeletionOutcomeOf(await this.send(`/v1/projects/${encodeURIComponent(projectId)}/metrics/${encodeURIComponent(metricId)}`, { method: "DELETE" }));
+	}
 	async findTagByName(projectId, name) {
 		return (await this.getIfFound(`/v1/projects/${encodeURIComponent(projectId)}/tags/by-name?${new URLSearchParams({ name })}`, TagSchema))?.parsedBody;
 	}
@@ -628,16 +754,16 @@ var AdminApiClient = class {
 	listRuns(projectId, configKey, environmentId) {
 		return this.getIfFound(`/v1/projects/${encodeURIComponent(projectId)}/configs/${encodeURIComponent(configKey)}/environments/${encodeURIComponent(environmentId)}/runs`, ExperimentRunsSchema);
 	}
-	shipVariation(projectId, configKey, environmentId, variationId) {
+	shipVariation(projectId, configKey, environmentId, request) {
 		return this.requestIfFound(`/v1/projects/${encodeURIComponent(projectId)}/configs/${encodeURIComponent(configKey)}/environments/${encodeURIComponent(environmentId)}/ship`, ConfigWithTargetingRulesSchema, {
 			method: "POST",
-			body: { variationId }
+			body: request
 		});
 	}
-	stopRun(projectId, configKey, environmentId) {
+	stopRun(projectId, configKey, environmentId, request) {
 		return this.requestIfFound(`/v1/projects/${encodeURIComponent(projectId)}/configs/${encodeURIComponent(configKey)}/environments/${encodeURIComponent(environmentId)}/stop`, ConfigWithTargetingRulesSchema, {
 			method: "POST",
-			body: {}
+			body: request
 		});
 	}
 	async evaluateConfig(projectId, configKey, request) {
@@ -1065,6 +1191,11 @@ const describeCondition = (condition, segments) => {
 };
 const valueWithVariationName = (value, variationName) => variationName ? `${variationName} (${String(value)})` : String(value);
 const describeVariation = (variation) => valueWithVariationName(variation.value, variation.name);
+const describeVariationById = (variationId, variations) => {
+	const variation = variations.find((candidate) => candidate.id === variationId);
+	return variation && describeVariation(variation);
+};
+const describeVariationOrNoLongerListed = (variationId, variations) => describeVariationById(variationId, variations) ?? "a variation no longer listed";
 const describeValue = (value, variationId, variations) => valueWithVariationName(value, variations.find((variation) => variation.id === variationId)?.name);
 const describePercentages = (percentages, variations) => percentages.map((percentage) => `${formatPercentage(percentage.percentage)}% get ${describeValue(percentage.value, percentage.variationId, variations)}`).join(", ");
 const describeRule = (rule, segments, variations) => {
@@ -1444,6 +1575,7 @@ const newConfigOf = async (options, workingDirectory) => {
 		typeOptions: await CONFIG_TYPE_RULES[options.type].typeOptionsOf(options, workingDirectory),
 		...availabilityOf(options),
 		variations: await variationsOf(options, workingDirectory),
+		hypothesis: options.hypothesis,
 		defaultValue: await typedValueOf(options.type, {
 			argumentLabel: "--default-value",
 			text: options.defaultValue,
@@ -1452,6 +1584,170 @@ const newConfigOf = async (options, workingDirectory) => {
 	};
 };
 //#endregion
+//#region src/metrics/metric-words.ts
+const METRIC_KINDS = [
+	"conversion",
+	"count-per-context",
+	"value-per-context"
+];
+const METRIC_DIRECTIONS = ["higher-is-better", "lower-is-better"];
+const METRIC_ROLES = [
+	"primary",
+	"secondary",
+	"guardrail"
+];
+const isOneOf = (values, candidate) => values.includes(candidate);
+const isMetricKind = (kind) => isOneOf(METRIC_KINDS, kind);
+const KIND_WORDS = {
+	conversion: "conversion",
+	"count-per-context": "count per context",
+	"value-per-context": "value per context"
+};
+const DIRECTION_WORDS = {
+	"higher-is-better": "higher is better",
+	"lower-is-better": "lower is better"
+};
+const ROLE_WORDS = {
+	primary: "primary metric",
+	secondary: "secondary metric",
+	guardrail: "guardrail"
+};
+const kindWords = (kind) => isMetricKind(kind) ? KIND_WORDS[kind] : kind;
+const directionWords = (direction) => isOneOf(METRIC_DIRECTIONS, direction) ? DIRECTION_WORDS[direction] : direction;
+const roleWords = (role) => isOneOf(METRIC_ROLES, role) ? ROLE_WORDS[role] : role;
+//#endregion
+//#region src/configs/run-results-text.ts
+const VERDICT_WORDS = {
+	"likely-better": "likely better",
+	"likely-worse": "likely worse",
+	"no-clear-difference": "no clear difference"
+};
+const formatAverage = (value) => value.toLocaleString("en-US", {
+	minimumFractionDigits: 2,
+	maximumFractionDigits: 2
+});
+const VALUE_FORMATS = {
+	conversion: (value) => `${(value * 100).toFixed(1)}%`,
+	"count-per-context": formatAverage,
+	"value-per-context": formatAverage
+};
+const formatResultValue = (kind, value) => isMetricKind(kind) ? VALUE_FORMATS[kind](value) : formatAverage(value);
+const formatLift = (lift) => {
+	const wholePercent = Math.round(lift * 100);
+	if (wholePercent > 0) return `+${wholePercent.toLocaleString("en-US")}%`;
+	if (wholePercent < 0) return `${wholePercent.toLocaleString("en-US")}%`;
+	return "0%";
+};
+const formatChance = (chance) => {
+	const wholePercent = Math.round(chance * 100);
+	if (wholePercent < 1) return "<1%";
+	if (wholePercent > 99) return ">99%";
+	return `${wholePercent}%`;
+};
+const liftCell = (variation) => {
+	if (variation.lift === void 0) return "-";
+	if (variation.interval === void 0) return formatLift(variation.lift);
+	return `${formatLift(variation.lift)} (${formatLift(variation.interval.low)} to ${formatLift(variation.interval.high)})`;
+};
+const chanceCell = (chance, description) => chance === void 0 ? "" : `${formatChance(chance)} ${description}`;
+const comparisonOf = (rollout) => rollout.state === "needs-more-exposures" ? "needs-more-exposures" : "variations-compared";
+const resultRowFormOf = (variation, table) => {
+	if (variation.variationId === table.controlVariationId) return "control";
+	return table.comparison === "needs-more-exposures" ? "value-alone" : "compared";
+};
+const resultRow = (variation, table) => {
+	const nameAndValue = [describeVariationOrNoLongerListed(variation.variationId, table.variations), formatResultValue(table.kind, variation.value)];
+	const chanceToBeBestCells = table.showsChanceToBeBest ? [chanceCell(variation.chanceToBeBest, "chance to be best")] : [];
+	switch (resultRowFormOf(variation, table)) {
+		case "control": return [
+			...nameAndValue,
+			"control",
+			"",
+			...chanceToBeBestCells
+		];
+		case "value-alone": return nameAndValue;
+		case "compared": return [
+			...nameAndValue,
+			liftCell(variation),
+			chanceCell(variation.chanceToBeatControl, "chance to beat control"),
+			...chanceToBeBestCells,
+			variation.verdict === void 0 ? "" : VERDICT_WORDS[variation.verdict] ?? variation.verdict
+		];
+	}
+};
+const capitalized = (text) => `${text.charAt(0).toUpperCase()}${text.slice(1)}`;
+const kindResultLines = (role, metric, kindResult, rollout, variations) => {
+	const variationResults = kindResult.rollouts.find((candidate) => candidate.ruleId === rollout.ruleId)?.variations;
+	if (variationResults === void 0) return [];
+	const table = {
+		kind: kindResult.kind,
+		controlVariationId: rollout.controlVariationId,
+		comparison: comparisonOf(rollout),
+		variations,
+		showsChanceToBeBest: variationResults.some((variation) => variation.chanceToBeBest !== void 0)
+	};
+	return [`${capitalized(roleWords(role))}: ${metric.eventName}, ${kindWords(kindResult.kind)}, ${directionWords(metric.direction)}`, ...alignedColumns(variationResults.map((variation) => resultRow(variation, table))).map((line) => `  ${line}`.trimEnd())];
+};
+const crossoverLines = (crossovers) => {
+	if (crossovers === 0) return [];
+	return [crossovers === 1 ? "1 context was exposed to more than one variation and is left out of every metric." : `${crossovers} contexts were exposed to more than one variation and are left out of every metric.`];
+};
+const comparisonLines = (comparison) => {
+	switch (comparison) {
+		case "variations-compared": return [];
+		case "needs-more-exposures": return ["The run needs more exposures before the variations can be compared."];
+	}
+};
+const runResultsOf = (run) => METRIC_ROLES.some((role) => run.results[role].length > 0) ? {
+	form: "measured",
+	results: run.results
+} : { form: "no-metrics" };
+const rolloutResultLines = (runResults, ruleId, variations) => {
+	switch (runResults.form) {
+		case "no-metrics": return [];
+		case "measured": {
+			const { results } = runResults;
+			const rollout = results.rollouts.find((candidate) => candidate.ruleId === ruleId);
+			if (rollout === void 0) return [];
+			return [
+				...comparisonLines(comparisonOf(rollout)),
+				...crossoverLines(rollout.crossovers),
+				...METRIC_ROLES.flatMap((role) => results[role].flatMap((metric) => metric.kinds.flatMap((kindResult) => kindResultLines(role, metric, kindResult, rollout, variations))))
+			];
+		}
+	}
+};
+const measuredEventNamesOf = (runResults) => {
+	switch (runResults.form) {
+		case "no-metrics": return /* @__PURE__ */ new Set();
+		case "measured": return new Set(METRIC_ROLES.flatMap((role) => runResults.results[role].map((metric) => metric.eventName)));
+	}
+};
+const eventsWithoutAMetricLines = (run, runResults) => {
+	const measuredEventNames = measuredEventNamesOf(runResults);
+	const events = run.trackedEvents.events.filter((event) => !measuredEventNames.has(event.name));
+	if (events.length === 0) return [];
+	return [`Events with no metric: ${events.map((event) => `${event.name} (${event.contexts} ${plural(event.contexts, "context", "contexts")})`).join(", ")}`];
+};
+const decideResultsFreshness = (run, runResults) => {
+	if (runResults.form === "no-metrics") return { form: "no-results" };
+	if (run.endedAt === void 0) return { form: "running" };
+	if (runResults.results.asOf === void 0) return { form: "ended-before-final-results" };
+	return {
+		form: "final",
+		asOf: runResults.results.asOf
+	};
+};
+const resultsFreshnessLines = (run, runResults) => {
+	const freshness = decideResultsFreshness(run, runResults);
+	switch (freshness.form) {
+		case "no-results": return [];
+		case "running": return ["Results can be up to 25 minutes behind."];
+		case "final": return [`Final results, as of ${utcMinute(freshness.asOf)}, when the run ended.`];
+		case "ended-before-final-results": return [];
+	}
+};
+//#endregion
 //#region src/configs/run-text.ts
 const OUTCOME_LABELS = {
 	shipped: "Shipped",
@@ -1463,36 +1759,40 @@ const VERDICT_SENTENCES = {
 	mismatch: "The split does not match the configured shares. This usually means some contexts evaluate without a stable identifier, or a rule above the split catches part of the traffic. Results from this run are not reliable.",
 	"needs-more-exposures": "The run needs more exposures before the split can be checked against the configured shares."
 };
-const variationDescription = (variationId, variations) => {
-	const variation = variations.find((candidate) => candidate.id === variationId);
-	return variation && describeVariation(variation);
-};
 const runHeadline = (run, variations) => {
 	if (run.outcome === void 0 || run.endedAt === void 0) return `Running since ${utcMinute(run.startedAt)}`;
 	const outcomeLabel = OUTCOME_LABELS[run.outcome] ?? run.outcome;
-	const shippedVariation = run.shippedVariationId && variationDescription(run.shippedVariationId, variations);
+	const shippedVariation = run.shippedVariationId && describeVariationById(run.shippedVariationId, variations);
 	return `${shippedVariation ? `${outcomeLabel} ${shippedVariation}` : outcomeLabel}, ${utcMinute(run.startedAt)} to ${utcMinute(run.endedAt)}`;
 };
 const rolloutAudience = (run, rollout) => run.rollouts.find((candidate) => candidate.ruleId === rollout.ruleId)?.conditions ?? "everyone not caught by a rule above";
 const rolloutSummary = (rollout) => `${rollout.exposures} ${plural(rollout.exposures, "context exposed", "contexts exposed")}, ${rollout.evaluations} ${plural(rollout.evaluations, "evaluation", "evaluations")}. ` + (VERDICT_SENTENCES[rollout.verdict] ?? rollout.verdict);
-const rolloutLines = (run, rollout, variations) => [
+const rolloutLines = (run, runResults, rollout, variations) => [
 	`Rollout: ${rolloutAudience(run, rollout)}`,
 	...alignedColumns(rollout.variations.map((variation) => [
-		variationDescription(variation.variationId, variations) ?? "a variation no longer listed",
+		describeVariationOrNoLongerListed(variation.variationId, variations),
 		`${variation.exposures} ${plural(variation.exposures, "exposure", "exposures")}`,
 		`configured ${formatPercentage(variation.percentage)}%`
 	])).map((line) => `  ${line}`),
-	`  ${rolloutSummary(rollout)}`
+	`  ${rolloutSummary(rollout)}`,
+	...rolloutResultLines(runResults, rollout.ruleId, variations).map((line) => `  ${line}`)
 ];
 const uncountedEvaluationsLines = (evaluations, one, several) => {
 	if (evaluations === 0) return [];
 	return [evaluations === 1 ? `1 evaluation ${one}` : `${evaluations} evaluations ${several}`];
 };
-const runLines = (run, variations) => [runHeadline(run, variations), ...[
-	...run.exposures.rollouts.flatMap((rollout) => rolloutLines(run, rollout, variations)),
-	...uncountedEvaluationsLines(run.exposures.evaluationsFromSdksNotReportingExposures, "came from an SDK version that does not report exposures yet. It is not counted above; update that SDK to count it.", "came from SDK versions that do not report exposures yet. They are not counted above; update those SDKs to count them."),
-	...uncountedEvaluationsLines(run.exposures.evaluationsWithoutIdentifier, "came from a context without an identifier and was served the default targeting rule value. It skips the split and is not an exposure.", "came from contexts without an identifier and were served the default targeting rule value. They skip the split and are not exposures.")
-].map((line) => `  ${line}`)];
+const runLines = (run, variations) => {
+	const runResults = runResultsOf(run);
+	return [runHeadline(run, variations), ...[
+		...run.hypothesis === void 0 ? [] : [`Hypothesis: ${run.hypothesis}`],
+		...run.note === void 0 ? [] : [`Ended with note: ${run.note}`],
+		...run.exposures.rollouts.flatMap((rollout) => rolloutLines(run, runResults, rollout, variations)),
+		...uncountedEvaluationsLines(run.exposures.evaluationsFromSdksNotReportingExposures, "came from an SDK version that does not report exposures yet. It is not counted above; update that SDK to count it.", "came from SDK versions that do not report exposures yet. They are not counted above; update those SDKs to count them."),
+		...uncountedEvaluationsLines(run.exposures.evaluationsWithoutIdentifier, "came from an anonymous context, so it is not an exposure. In a server SDK, a context without an identifier skips the split and is served the default targeting rule value; any other anonymous context sees its share of the split without being stored.", "came from anonymous contexts, so they are not exposures. In a server SDK, a context without an identifier skips the split and is served the default targeting rule value; any other anonymous context sees its share of the split without being stored."),
+		...eventsWithoutAMetricLines(run, runResults),
+		...resultsFreshnessLines(run, runResults)
+	].map((line) => `  ${line}`)];
+};
 const runsText = (config, requestedKey, environment, runs) => {
 	const environmentName = environmentLabel(environment);
 	return [
@@ -1530,6 +1830,10 @@ const findEnvironmentBySlug = async (client, project, slug) => {
 	if (environment === void 0) throw new CliError(`No environment in ${project.slug} has the slug "${slug}". ${environmentSlugsSentence(environments)}`);
 	return environment;
 };
+const environmentSlugsById = async (client, project) => {
+	const environments = await ifPermitted(client.listEnvironments(project.id));
+	return new Map((environments?.parsedBody ?? []).map((environment) => [environment.id, environment.slug]));
+};
 //#endregion
 //#region src/project-selection.ts
 const requireProjectSlug = (context) => {
@@ -1804,13 +2108,17 @@ const showConfigRuns = async (context, [configKey], options) => {
 		exitCode: ExitCode.success
 	};
 };
+const runEndRequestOf = (options) => options.note === void 0 ? {} : { note: options.note };
 const shipVariation = async (context, [configKey], options) => {
 	const { client, project } = await connectToLinkedProject(context);
 	const environment = await findEnvironmentBySlug(client, project, options.environment);
 	const receivedConfig = await client.getConfig(project.id, configKey);
 	if (receivedConfig === void 0) throw new CliError(`No config in ${project.slug} has the key "${configKey}".`);
 	const variation = findVariationToShip(receivedConfig.parsedBody, options.variation);
-	const savedConfig = await client.shipVariation(project.id, configKey, environment.id, variation.id);
+	const savedConfig = await client.shipVariation(project.id, configKey, environment.id, {
+		variationId: variation.id,
+		...runEndRequestOf(options)
+	});
 	if (savedConfig === void 0) throw new CliError(`No config in ${project.slug} has the key "${configKey}".`);
 	const segments = await segmentNamesFor(client, project, savedConfig.parsedBody);
 	return {
@@ -1827,7 +2135,7 @@ const shipVariation = async (context, [configKey], options) => {
 const stopRun = async (context, [configKey], options) => {
 	const { client, project } = await connectToLinkedProject(context);
 	const environment = await findEnvironmentBySlug(client, project, options.environment);
-	const savedConfig = await client.stopRun(project.id, configKey, environment.id);
+	const savedConfig = await client.stopRun(project.id, configKey, environment.id, runEndRequestOf(options));
 	if (savedConfig === void 0) throw new CliError(`No config in ${project.slug} has the key "${configKey}".`);
 	const segments = await segmentNamesFor(client, project, savedConfig.parsedBody);
 	return {
@@ -2009,8 +2317,7 @@ const bucketPlacement = (bucket) => {
 	if (selectedShare !== void 0) return `${assigned} is in ${describeShareRange(selectedShare.from, selectedShare.to)}`;
 	return `${assigned} is outside ${describeShareRange(0, bucket.shares.at(-1)?.to ?? 0)}`;
 };
-const describeBucket = (bucket) => bucket.defaultedToPredefinedBucket ? `no identifier, so ${bucketPlacement(bucket)}` : bucketPlacement(bucket);
-const withBucket = (outcome, bucket) => bucket === void 0 ? outcome : `${outcome}: ${describeBucket(bucket)}`;
+const withBucket = (outcome, bucket) => bucket === void 0 ? outcome : `${outcome}: ${bucketPlacement(bucket)}`;
 const conditionsOf = (rule) => rule?.type === "conditional" ? rule.conditions : [];
 const segmentMismatchEvidence = (explanation) => {
 	if (!explanation.segmentFound) return "the segment was not found";
@@ -2032,17 +2339,19 @@ const describeNotMatched = (explanation, conditions, segments) => {
 	const failingCondition = failingConditionDescription(explanation, conditions, segments);
 	return failingCondition === void 0 ? withBucket("not matched", explanation.bucket) : `not matched: ${failingCondition}`;
 };
+const evaluatorRuleOutcomeOf = (explanation) => explanation.rolloutSkippedNoIdentifier ? "rollout-skipped-no-identifier" : explanation.outcome;
 const RULE_OUTCOME_DESCRIPTIONS = {
 	matched: (explanation) => withBucket("matched", explanation.bucket),
 	"not-matched": describeNotMatched,
 	"not-evaluated": () => "not evaluated",
-	errored: () => "errored"
+	errored: () => "errored",
+	"rollout-skipped-no-identifier": () => "rollout skipped: the context has no identifier"
 };
 const ruleLines = (evaluation) => {
 	const rulesById = new Map(evaluation.targetingRules.rules.map((rule) => [rule.id, rule]));
 	return alignedColumns(evaluation.rules.map((explanation, index) => {
 		const rule = rulesById.get(explanation.ruleId);
-		return [`${index + 1}. ${rule === void 0 ? explanation.ruleId : describeRule(rule, evaluation.segments, evaluation.variations)}`, RULE_OUTCOME_DESCRIPTIONS[explanation.outcome](explanation, conditionsOf(rule), evaluation.segments)];
+		return [`${index + 1}. ${rule === void 0 ? explanation.ruleId : describeRule(rule, evaluation.segments, evaluation.variations)}`, RULE_OUTCOME_DESCRIPTIONS[evaluatorRuleOutcomeOf(explanation)](explanation, conditionsOf(rule), evaluation.segments)];
 	}));
 };
 const evaluationText = (evaluation, requested) => [
@@ -2279,6 +2588,134 @@ const logout = async (context) => {
 	};
 };
 //#endregion
+//#region src/metrics/metric-changes.ts
+const metricChangesOf = (options) => {
+	const changes = {
+		...options.kind === void 0 ? {} : { kinds: options.kind },
+		...options.direction === void 0 ? {} : { direction: options.direction },
+		...options.description === void 0 ? {} : { description: options.description }
+	};
+	if (Object.keys(changes).length === 0) throw new CliError("Say what to change with --kind, --direction or --description.");
+	return changes;
+};
+//#endregion
+//#region src/metrics/metric-text.ts
+const openRunEnvironmentNames = (openRuns, environmentSlugs) => openRuns.map((run) => environmentSlugs.get(run.environmentId) ?? run.environmentId).join(", ");
+const usageLine$1 = (usage, environmentSlugs) => {
+	const primaryKind = usage.primaryKind === null ? "" : ` on ${kindWords(usage.primaryKind)}`;
+	const runningIn = usage.openRuns.length === 0 ? "" : `, running in ${openRunEnvironmentNames(usage.openRuns, environmentSlugs)}`;
+	return `  ${usage.configKey}: ${roleWords(usage.role)}${primaryKind}${runningIn}`;
+};
+const usageLines = (metric, environmentSlugs) => metric.usages.length === 0 ? ["No experiment uses it."] : ["Used by", ...metric.usages.map((usage) => usageLine$1(usage, environmentSlugs))];
+const metricText = (metric, environmentSlugs) => [
+	metric.eventName,
+	...metric.description === null ? [] : [`  Description: ${metric.description}`],
+	`  Kinds: ${metric.kinds.map(kindWords).join(", ")}`,
+	`  Direction: ${directionWords(metric.direction)}`,
+	"",
+	...usageLines(metric, environmentSlugs),
+	""
+].join("\n");
+//#endregion
+//#region src/metrics/new-metric.ts
+const newMetricOf = (options) => ({
+	eventName: options.eventName,
+	kinds: options.kind,
+	direction: options.direction,
+	...options.description === void 0 ? {} : { description: options.description }
+});
+//#endregion
+//#region src/commands/metrics.ts
+const usageDescription$1 = (metric) => `used by ${metric.usages.length} ${plural(metric.usages.length, "experiment", "experiments")}`;
+const METRIC_HEADER_ROW = [
+	"EVENT NAME",
+	"KINDS",
+	"DIRECTION",
+	"USED BY"
+];
+const metricLines = (projectSlug, metrics, options) => metrics.length === 0 ? [`No metrics found in ${projectSlug}.`] : alignedListing(METRIC_HEADER_ROW, metrics.map((metric) => [
+	metric.eventName,
+	metric.kinds.map(kindWords).join(", "),
+	directionWords(metric.direction),
+	usageDescription$1(metric)
+]), options);
+const listMetrics = async (context, _commandArguments, options) => {
+	const { client, project } = await connectToLinkedProject(context);
+	const { parsedBody: metrics, responseBody } = await client.listMetrics(project.id);
+	return {
+		json: responseBody,
+		text: [...metricLines(project.slug, metrics, options), ""].join("\n"),
+		exitCode: ExitCode.success
+	};
+};
+const openRunEnvironmentSlugsOf = async (client, project, metric) => metric.usages.some((usage) => usage.openRuns.length > 0) ? environmentSlugsById(client, project) : /* @__PURE__ */ new Map();
+const noMetricWithEventName = (project, eventName) => new CliError(`No metric in ${project.slug} has the event name "${eventName}".`);
+const getMetric = async (context, [eventName]) => {
+	const { client, project } = await connectToLinkedProject(context);
+	const receivedMetric = await client.findMetricByEventName(project.id, eventName);
+	if (receivedMetric === void 0) throw noMetricWithEventName(project, eventName);
+	const { parsedBody: metric, responseBody } = receivedMetric;
+	return {
+		json: responseBody,
+		text: metricText(metric, await openRunEnvironmentSlugsOf(client, project, metric)),
+		exitCode: ExitCode.success
+	};
+};
+const createMetric = async (context, _commandArguments, options) => {
+	const { client, project } = await connectToLinkedProject(context);
+	const { parsedBody: metric, responseBody } = await client.createMetric(project.id, newMetricOf(options));
+	return {
+		json: responseBody,
+		text: [`Created metric ${metric.eventName} in ${project.slug}.`, metricText(metric, /* @__PURE__ */ new Map())].join("\n"),
+		exitCode: ExitCode.success
+	};
+};
+const kindInUseError = (eventName, experiments) => new CliError([
+	`${experiments.length} ${plural(experiments.length, "experiment decides", "experiments decide")} on a kind this change removes, so ${eventName} was not updated:`,
+	...experiments.map((experiment) => `  ${experiment.configKey} decides on ${kindWords(experiment.primaryKind)}`),
+	`Give ${plural(experiments.length, "it", "each of them")} another primary kind first, on the experiment's Settings tab in the dashboard.`
+].join("\n"));
+const updateMetric = async (context, [eventName], options) => {
+	const changes = metricChangesOf(options);
+	const { client, project } = await connectToLinkedProject(context);
+	const currentMetric = await client.findMetricByEventName(project.id, eventName);
+	if (currentMetric === void 0) throw noMetricWithEventName(project, eventName);
+	const update = await client.updateMetric(project.id, currentMetric.parsedBody.id, changes);
+	switch (update.outcome) {
+		case "not-found": throw noMetricWithEventName(project, eventName);
+		case "kind-in-use": throw kindInUseError(eventName, update.decidingExperiments);
+		case "updated": {
+			const { parsedBody: metric, responseBody } = update.metric;
+			return {
+				json: responseBody,
+				text: [`Updated metric ${eventName} in ${project.slug}.`, metricText(metric, await openRunEnvironmentSlugsOf(client, project, metric))].join("\n"),
+				exitCode: ExitCode.success
+			};
+		}
+	}
+};
+const experimentWithOpenRunsLine = (experiment, environmentSlugs) => `  ${experiment.configKey}, running in ${openRunEnvironmentNames(experiment.openRuns, environmentSlugs)}`;
+const metricInUseError = (eventName, experiments, environmentSlugs) => new CliError([
+	`${experiments.length} ${plural(experiments.length, "experiment", "experiments")} with a run in progress ${plural(experiments.length, "uses", "use")} ${eventName}, so it was not deleted:`,
+	...experiments.map((experiment) => experimentWithOpenRunsLine(experiment, environmentSlugs)),
+	"Ship or stop each run first, with configdirector configs ship or configs stop, or detach the metric on the experiment's Settings tab in the dashboard."
+].join("\n"));
+const deleteMetric = async (context, [eventName]) => {
+	const { client, project } = await connectToLinkedProject(context);
+	const currentMetric = await client.findMetricByEventName(project.id, eventName);
+	if (currentMetric === void 0) throw noMetricWithEventName(project, eventName);
+	const deletion = await client.deleteMetric(project.id, currentMetric.parsedBody.id);
+	switch (deletion.outcome) {
+		case "deleted": return {
+			json: { deletedEventName: eventName },
+			text: [`Deleted metric ${eventName} from ${project.slug}.`, ""].join("\n"),
+			exitCode: ExitCode.success
+		};
+		case "not-found": throw noMetricWithEventName(project, eventName);
+		case "in-use": throw metricInUseError(eventName, deletion.experimentsWithOpenRuns, await environmentSlugsById(client, project));
+	}
+};
+//#endregion
 //#region src/commands/projects.ts
 const PROJECT_HEADER_ROW = ["SLUG", "NAME"];
 const projectLines = (projects, options) => projects.length === 0 ? ["No projects found."] : alignedListing(PROJECT_HEADER_ROW, projects.map((project) => [project.slug, project.name]), options);
@@ -5033,10 +5470,6 @@ const listSegments = async (context, _commandArguments, options) => {
 		exitCode: ExitCode.success
 	};
 };
-const environmentSlugsById = async (client, project) => {
-	const environments = await ifPermitted(client.listEnvironments(project.id));
-	return new Map((environments?.parsedBody ?? []).map((environment) => [environment.id, environment.slug]));
-};
 const overrideEnvironmentSlugsOf = async (client, project, segment) => Object.keys(segment.overrides).length === 0 ? /* @__PURE__ */ new Map() : environmentSlugsById(client, project);
 const noSegmentWithKey = (project, segmentKey) => new CliError(`No segment in ${project.slug} has the key "${segmentKey}".`);
 const getSegment = async (context, [segmentKey]) => {
@@ -5384,6 +5817,10 @@ const withoutTrailingSlashes = (url) => {
 	return trimmedUrl;
 };
 const repeatedOption = (value, values) => [...values ?? [], value];
+const repeatableChoiceOption = (flags, description, choices) => new Option(flags, description).choices(choices).argParser((value, values) => {
+	if (!choices.includes(value)) throw new InvalidArgumentError(`Allowed choices are ${choices.join(", ")}.`);
+	return repeatedOption(value, values);
+});
 const runCommand = async (invocation, command, actionArguments) => {
 	const parsedCommand = actionArguments.at(-1);
 	const options = parsedCommand.optsWithGlobals();
@@ -5428,11 +5865,11 @@ const run = async (invocation) => {
 	configsCommand.command("list").description("list the project's configs with their type, role, lifetime and lifecycle state").option("--tag <name>", "list only the configs that have this tag; repeatable, for configs with any of them", repeatedOption).addOption(new Option("--untagged", "list only the configs without tags").conflicts("tag")).addOption(new Option("--state <state>", "list only the configs in this lifecycle state").choices(LIFECYCLE_STATES)).option("--no-headers", NO_HEADERS_DESCRIPTION).action(commandAction(listConfigs));
 	configsCommand.command("get").argument("<key>", "the config's key, or an old key of it").description("show a config's settings and its targeting rules in every environment").action(commandAction(getConfig));
 	configsCommand.command("activity").argument("<key>", "the config's key, or an old key of it").description("show when SDKs last evaluated a config, per environment, SDK key kind, SDK name and SDK version").option("--no-headers", NO_HEADERS_DESCRIPTION).action(commandAction(showConfigActivity));
-	configsCommand.command("create").description("create a config from options that mirror the Admin API's fields").requiredOption("--key <key>", "the config's key (required)").option("--description <text>", "a description shown with the config in the dashboard").addOption(new Option("--type <type>", "the type of the config's values (required)").choices(CONFIG_TYPES).makeOptionMandatory()).addOption(new Option("--role <role>", "what the config is for (required)").choices(CONFIG_ROLES).makeOptionMandatory()).addOption(new Option("--lifetime <lifetime>", "whether the config is expected to be removed; required with --role config, while a feature flag and an experiment are always temporary and a kill switch always permanent").choices(CONFIG_LIFETIMES)).requiredOption("--default-value <value>", "the default targeting rule value every environment starts with, in the type given by --type; not the in-code default value an application passes to the SDK (required)").option("--client-only", "serve the config to client SDKs only (default: client and server SDKs)").addOption(new Option("--server-only", "serve the config to server SDKs only (default: client and server SDKs)").conflicts("clientOnly")).option("--tag <name>", "assign the project's tag with this name; repeatable", repeatedOption).option("--new-tag <name>", "create a tag with this name in the project's tag catalog and assign it; repeatable", repeatedOption).option("--variation <name=value>", "a variation of an experiment, its value in the type given by --type; repeatable", repeatedOption).addOption(new Option("--min <number>", "the smallest value allowed, included").helpGroup(typeConstraintHelpGroup("min"))).addOption(new Option("--max <number>", "the largest value allowed, included").helpGroup(typeConstraintHelpGroup("max"))).addOption(new Option("--exclusive-min <number>", "values must be greater than this number").conflicts("min").helpGroup(typeConstraintHelpGroup("exclusiveMin"))).addOption(new Option("--exclusive-max <number>", "values must be less than this number").conflicts("max").helpGroup(typeConstraintHelpGroup("exclusiveMax"))).addOption(new Option("--unit <unit>", "the unit of the number the SDKs receive (required)").choices(TIMESPAN_UNITS).helpGroup(typeConstraintHelpGroup("unit"))).addOption(new Option("--allow-fractions", "allow values with a fraction of the unit").helpGroup(typeConstraintHelpGroup("allowFractions"))).addOption(new Option("--allowed-value <value>", "a value the config can take; repeatable").argParser(repeatedOption).helpGroup(typeConstraintHelpGroup("allowedValue"))).addOption(new Option("--enum-value-type <type>", "whether the allowed values are strings or numbers (default: string)").choices(ENUM_VALUE_TYPES).helpGroup(typeConstraintHelpGroup("enumValueType"))).addOption(new Option("--https-only", "allow only HTTPS URLs").helpGroup(typeConstraintHelpGroup("httpsOnly"))).addOption(new Option("--json-schema <file>", "a file holding the JSON Schema every value must match").helpGroup(typeConstraintHelpGroup("jsonSchema"))).action(commandAction(createConfig));
-	configsCommand.command("copy").argument("<key>", "the key of the config to copy, or an old key of it").description("copy a config under a new key, in its project or in another project of the organization, and print the plan").requiredOption("--key <key>", "the copy's key (required)").option("--to <project slug>", "slug of the project to copy the config into, another project of the organization (default: the config's project)").option("--environment <destination slug>=<source slug>", "the source environment whose targeting rules an environment of the destination project starts with, needed for each destination environment with no source environment of its slug; repeatable", repeatedOption).option("--dry-run", "print the plan without creating anything").addHelpText("after", "\nThe copy gets the config's role, lifetime, type, constraints, variations, SDK availability, description and tags, and every environment's targeting rules and default targeting rule value. It starts as a new config, with no old keys, evaluations or code references, and shares nothing with the config it came from. An archived config cannot be copied; restore it first. In another project, environments are matched by slug, tags by name and segments by key: a source environment with no environment of its slug there is skipped, and an environment there with no source environment of its slug needs --environment. A targeting rule that uses a segment the other project doesn't have is dropped. A tag it doesn't have is created when the API token can create tags, and skipped otherwise. The plan names every skipped environment, dropped rule and tag; --dry-run shows it without creating anything.").action(commandAction(copyConfig));
-	configsCommand.command("runs").argument("<key>", "the experiment's key, or an old key of it").description("list an experiment's runs in an environment with their exposures, newest first").requiredOption("--environment <slug>", "slug of the environment to list the runs of (required)").action(commandAction(showConfigRuns));
-	configsCommand.command("ship").argument("<key>", "the experiment's key, or an old key of it").description("serve one variation of an experiment to everyone in an environment, ending its run").requiredOption("--environment <slug>", "slug of the environment to ship the variation in (required)").requiredOption("--variation <variation>", "the name or id of the variation to ship (required)").addHelpText("after", "\nIn one targeting rules save, the default targeting rule value becomes the variation's value, every percentage rollout is removed, including one a conditional rule serves, and a conditional rule serving a single value stays. It asks for no confirmation and treats a live environment like any other.").action(commandAction(shipVariation));
-	configsCommand.command("stop").argument("<key>", "the experiment's key, or an old key of it").description("end an experiment's run in an environment without choosing a variation").requiredOption("--environment <slug>", "slug of the environment to stop the run in (required)").addHelpText("after", "\nIn one targeting rules save, every percentage rollout is removed, including one a conditional rule serves, the default targeting rule value stays, and a conditional rule serving a single value stays. It asks for no confirmation and treats a live environment like any other. When the config is not an experiment, or no percentage rollout is saved in the environment, there is no run to stop, and the command exits with 2.").action(commandAction(stopRun));
+	configsCommand.command("create").description("create a config from options that mirror the Admin API's fields").requiredOption("--key <key>", "the config's key (required)").option("--description <text>", "a description shown with the config in the dashboard").addOption(new Option("--type <type>", "the type of the config's values (required)").choices(CONFIG_TYPES).makeOptionMandatory()).addOption(new Option("--role <role>", "what the config is for (required)").choices(CONFIG_ROLES).makeOptionMandatory()).addOption(new Option("--lifetime <lifetime>", "whether the config is expected to be removed; required with --role config, while a feature flag and an experiment are always temporary and a kill switch always permanent").choices(CONFIG_LIFETIMES)).requiredOption("--default-value <value>", "the default targeting rule value every environment starts with, in the type given by --type; not the in-code default value an application passes to the SDK (required)").option("--client-only", "serve the config to client SDKs only (default: client and server SDKs)").addOption(new Option("--server-only", "serve the config to server SDKs only (default: client and server SDKs)").conflicts("clientOnly")).option("--tag <name>", "assign the project's tag with this name; repeatable", repeatedOption).option("--new-tag <name>", "create a tag with this name in the project's tag catalog and assign it; repeatable", repeatedOption).option("--variation <name=value>", "a variation of an experiment, its value in the type given by --type; repeatable", repeatedOption).option("--hypothesis <text>", "what an experiment is expected to do; each run keeps the hypothesis it started with").addOption(new Option("--min <number>", "the smallest value allowed, included").helpGroup(typeConstraintHelpGroup("min"))).addOption(new Option("--max <number>", "the largest value allowed, included").helpGroup(typeConstraintHelpGroup("max"))).addOption(new Option("--exclusive-min <number>", "values must be greater than this number").conflicts("min").helpGroup(typeConstraintHelpGroup("exclusiveMin"))).addOption(new Option("--exclusive-max <number>", "values must be less than this number").conflicts("max").helpGroup(typeConstraintHelpGroup("exclusiveMax"))).addOption(new Option("--unit <unit>", "the unit of the number the SDKs receive (required)").choices(TIMESPAN_UNITS).helpGroup(typeConstraintHelpGroup("unit"))).addOption(new Option("--allow-fractions", "allow values with a fraction of the unit").helpGroup(typeConstraintHelpGroup("allowFractions"))).addOption(new Option("--allowed-value <value>", "a value the config can take; repeatable").argParser(repeatedOption).helpGroup(typeConstraintHelpGroup("allowedValue"))).addOption(new Option("--enum-value-type <type>", "whether the allowed values are strings or numbers (default: string)").choices(ENUM_VALUE_TYPES).helpGroup(typeConstraintHelpGroup("enumValueType"))).addOption(new Option("--https-only", "allow only HTTPS URLs").helpGroup(typeConstraintHelpGroup("httpsOnly"))).addOption(new Option("--json-schema <file>", "a file holding the JSON Schema every value must match").helpGroup(typeConstraintHelpGroup("jsonSchema"))).action(commandAction(createConfig));
+	configsCommand.command("copy").argument("<key>", "the key of the config to copy, or an old key of it").description("copy a config under a new key, in its project or in another project of the organization, and print the plan").requiredOption("--key <key>", "the copy's key (required)").option("--to <project slug>", "slug of the project to copy the config into, another project of the organization (default: the config's project)").option("--environment <destination slug>=<source slug>", "the source environment whose targeting rules an environment of the destination project starts with, needed for each destination environment with no source environment of its slug; repeatable", repeatedOption).option("--dry-run", "print the plan without creating anything").addHelpText("after", "\nThe copy gets the config's role, lifetime, type, constraints, variations, SDK availability, description, hypothesis and tags, and every environment's targeting rules and default targeting rule value. It starts as a new config, with no old keys, evaluations or code references, and shares nothing with the config it came from. An archived config cannot be copied; restore it first. In another project, environments are matched by slug, tags by name and segments by key: a source environment with no environment of its slug there is skipped, and an environment there with no source environment of its slug needs --environment. A targeting rule that uses a segment the other project doesn't have is dropped. A tag it doesn't have is created when the API token can create tags, and skipped otherwise. The plan names every skipped environment, dropped rule and tag; --dry-run shows it without creating anything.").action(commandAction(copyConfig));
+	configsCommand.command("runs").argument("<key>", "the experiment's key, or an old key of it").description("list an experiment's runs in an environment with their exposures and results, newest first").requiredOption("--environment <slug>", "slug of the environment to list the runs of (required)").action(commandAction(showConfigRuns));
+	configsCommand.command("ship").argument("<key>", "the experiment's key, or an old key of it").description("serve one variation of an experiment to everyone in an environment, ending its run").requiredOption("--environment <slug>", "slug of the environment to ship the variation in (required)").requiredOption("--variation <variation>", "the name or id of the variation to ship (required)").option("--note <text>", "why the variation is shipped, kept on the ended run").addHelpText("after", "\nIn one targeting rules save, the default targeting rule value becomes the variation's value, every percentage rollout is removed, including one a conditional rule serves, and a conditional rule serving a single value stays. It asks for no confirmation and treats a live environment like any other.").action(commandAction(shipVariation));
+	configsCommand.command("stop").argument("<key>", "the experiment's key, or an old key of it").description("end an experiment's run in an environment without choosing a variation").requiredOption("--environment <slug>", "slug of the environment to stop the run in (required)").option("--note <text>", "why the run is stopped, kept on the ended run").addHelpText("after", "\nIn one targeting rules save, every percentage rollout is removed, including one a conditional rule serves, the default targeting rule value stays, and a conditional rule serving a single value stays. It asks for no confirmation and treats a live environment like any other. When the config is not an experiment, or no percentage rollout is saved in the environment, there is no run to stop, and the command exits with 2.").action(commandAction(stopRun));
 	configsCommand.command("archive").argument("<key>", "the config's key, or an old key of it").description("archive a config, so SDKs stop receiving it; the dashboard can restore it for 30 days").addHelpText("after", "\nThe config is archived with no confirmation and no check of whether the code still reads its key, so run configdirector scan first. Applications that still read it get their in-code default value.").action(commandAction(archiveConfig));
 	const segmentsCommand = program.command("segments").description("list, show, create, update and delete a project's segments");
 	segmentsCommand.command("list").description("list the project's segments with how many configs use each").option("--no-headers", NO_HEADERS_DESCRIPTION).action(commandAction(listSegments));
@@ -5440,6 +5877,12 @@ const run = async (invocation) => {
 	segmentsCommand.command("create").description("create a segment from a JSON file").requiredOption("--segment <file>", SEGMENT_FILE_DESCRIPTION).addHelpText("after", "\nThe file holds {\"key\": ..., \"name\": ..., \"groups\": [...], \"overrides\": {...}}, like the output of segments get --json, so you can edit that output and send it. groups lists the condition groups, and a context is in the segment when every condition of any one group matches. overrides is optional: it maps an environment's id, which environments list --json shows, to the condition groups used there instead. A condition without an id gets a new one.").action(commandAction(createSegment));
 	segmentsCommand.command("update").argument("<key>", "the segment's key").description("replace a segment's key, name and condition groups with a JSON file, and its environment overrides when the file has them").requiredOption("--segment <file>", SEGMENT_FILE_DESCRIPTION).addHelpText("after", "\nThe file takes the same form as for segments create, and the segment keeps its id whatever id the file holds. Its key, name and condition groups are replaced, so a different key renames it. Without overrides in the file, the environment overrides stay as they are; with \"overrides\": {} they are all removed, and a map leaves exactly those environments overridden. A condition without an id gets a new one.").action(commandAction(updateSegment));
 	segmentsCommand.command("delete").argument("<key>", "the segment's key").description("delete a segment and its environment overrides").addHelpText("after", "\nThe segment is deleted with no confirmation. While the targeting rules of a config use it, the deletion is refused and the configs are listed with their environments.").action(commandAction(deleteSegment));
+	const metricsCommand = program.command("metrics").description("list, show, create, update and delete the metrics a project's experiments measure");
+	metricsCommand.command("list").description("list the project's metrics with their kinds, direction and how many experiments use each").option("--no-headers", NO_HEADERS_DESCRIPTION).action(commandAction(listMetrics));
+	metricsCommand.command("get").argument("<event name>", "the metric's event name, as the application passes it to track").description("show a metric's kinds and direction, and the experiments that use it").action(commandAction(getMetric));
+	metricsCommand.command("create").description("create the project's metric for an event name the application passes to track").requiredOption("--event-name <event name>", "the event name, exactly as the application passes it to track (required)").addOption(repeatableChoiceOption("--kind <kind>", "a number the metric reports: conversion, the share of exposed contexts that sent the event; count-per-context, how many times each sent it on average; value-per-context, the value sent with it per context on average; repeatable (required)", METRIC_KINDS).makeOptionMandatory()).addOption(new Option("--direction <direction>", "whether a higher or a lower number is better (required)").choices(METRIC_DIRECTIONS).makeOptionMandatory()).option("--description <text>", "what the event means").action(commandAction(createMetric));
+	metricsCommand.command("update").argument("<event name>", "the metric's event name").description("change a metric's kinds, direction or description, leaving the others as they are").addOption(repeatableChoiceOption("--kind <kind>", "a number the metric reports; repeatable, and the kinds given replace the metric's", METRIC_KINDS)).addOption(new Option("--direction <direction>", "whether a higher or a lower number is better").choices(METRIC_DIRECTIONS)).option("--description <text>", "what the event means; --description \"\" removes it").addHelpText("after", "\nA metric keeps its event name: for another event name, create another metric. Removing a kind that an experiment's primary metric decides on is refused, and the experiments are listed.").action(commandAction(updateMetric));
+	metricsCommand.command("delete").argument("<event name>", "the metric's event name").description("delete a metric and detach it from the experiments that use it").addHelpText("after", "\nThe metric is deleted with no confirmation. While an experiment that uses it has a run in progress, the deletion is refused and the experiments are listed with the environments of their runs. Events with its name keep being counted, so a metric created again for the name and attached again shows their history.").action(commandAction(deleteMetric));
 	const targetingRulesCommand = program.command("targeting-rules").description("replace a config's targeting rules in one environment and show the format they take");
 	targetingRulesCommand.command("set").argument("<key>", "the config's key, or an old key of it").description("replace a config's targeting rules in one environment with the rules of a JSON file").requiredOption("--environment <slug>", "slug of the environment whose targeting rules to replace (required)").requiredOption("--rules <file>", "a JSON file holding defaultValue and rules as the Admin API takes them, or - to read it from stdin (required)").addHelpText("after", "\nThe file holds {\"defaultValue\": ..., \"rules\": [...]}, like an entry of targets in configs get --json, so you can edit that output and write it back. Every rule of the environment is replaced. A rule, condition or percentage without an id gets a new one. configdirector targeting-rules reference lists the condition kinds, attributes and operators.").action(commandAction(setTargetingRules));
 	targetingRulesCommand.command("reference").description("show the format targeting rules take: the condition kinds, the attributes a condition can test and the operators").action(commandAction(showTargetingRulesReference));

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@configdirector/cli",
-  "version": "0.19.0",
+  "version": "0.21.0",
   "description": "Command line tool for ConfigDirector, the managed configuration and feature flag service.",
   "license": "MIT",
   "author": "ConfigDirector",