@configdirector/cli 0.21.0 → 0.23.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +20 -0
- package/README.md +51 -10
- package/dist/configdirector.mjs +284 -105
- package/package.json +1 -1
package/CHANGELOG.md
CHANGED
|
@@ -4,6 +4,26 @@ All notable changes to `@configdirector/cli` are documented here. The format fol
|
|
|
4
4
|
|
|
5
5
|
## [Unreleased]
|
|
6
6
|
|
|
7
|
+
## [0.23.0] - 2026-11-12
|
|
8
|
+
|
|
9
|
+
### Added
|
|
10
|
+
|
|
11
|
+
- `configdirector metrics create` takes `--filter <file>`, or `--filter -` for stdin, a JSON file holding the conditions on an event's details that the event must match to count for the metric, such as only the checkouts on the Team plan, in the shape `metrics get --json` prints a metric's `filter` in. It also takes `--value-field <pointer>`, the field of the event's details whose number value per context sums in place of the event's value, with `--kind value-per-context`. A metric keeps both for its whole life, and a run counts it from the moment it is attached to the experiment.
|
|
12
|
+
- `configdirector metrics get`, `metrics update` and `metrics delete` take `--id <metric id>` in place of the event name, to name one of several metrics with the same event name.
|
|
13
|
+
|
|
14
|
+
### Changed
|
|
15
|
+
|
|
16
|
+
- `configdirector metrics get`, `metrics update` and `metrics delete` change nothing when several metrics have the event name you give: they list the metrics with their IDs and exit with 2. Earlier versions act on the first metric ConfigDirector lists for the event name, so once a project has several metrics on one event name, their `metrics update` and `metrics delete` can change or delete another metric than the one you mean. Upgrade every CLI that changes metrics before you create a second metric on an event name.
|
|
17
|
+
- `configdirector metrics list`, `metrics get`, `metrics create`, `metrics update`, `metrics delete` and `configs runs` name a metric by its event name, then its filter's conditions, then its value field, as in `checkout_completed · /plan is one of [team] · value from /items`. The first column of `metrics list` is headed `METRIC`.
|
|
18
|
+
- `configdirector configs runs` says since when a run counted a metric with a filter or a value field that was attached to the experiment while the run was going, under each of the metric's kinds, and that events reported before then are not in its numbers. For such a metric that was not attached while the run was open, it says the run did not count it and shows no numbers. `--json` prints the Admin API's response as received, as before.
|
|
19
|
+
- `configdirector metrics delete --json` prints the deleted metric's `deletedMetricId` beside its `deletedEventName`.
|
|
20
|
+
|
|
21
|
+
## [0.22.0] - 2026-11-11
|
|
22
|
+
|
|
23
|
+
### Added
|
|
24
|
+
|
|
25
|
+
- `configdirector configs runs` shows how long a run in progress needs, under each rollout's primary metric results once its variations can be compared. For a real lift of 2%, 5%, 10%, 20% and 50% on the primary metric, in its better direction, it gives the exposures so far against those needed before a variation with that lift has a good chance of reading likely better, and about how many days remain at the pace of new exposures since the time it names. A lift the rollout has enough exposures for says so, more than 365 days reads as more than a year, and when no new contexts were exposed since that time, it says the days cannot be estimated and gives the exposures alone. It also says that a row's lift is one a variation might really have, not the lift measured so far, and that whatever the estimate says, a run should cover at least one full week. The estimate doesn't show while the control's numbers give nothing to estimate from, as when none of its contexts has sent the event yet. `--json` prints the Admin API's response as received, as before.
|
|
26
|
+
|
|
7
27
|
## [0.21.0] - 2026-11-10
|
|
8
28
|
|
|
9
29
|
### Added
|
package/README.md
CHANGED
|
@@ -183,6 +183,7 @@ configdirector segments list
|
|
|
183
183
|
configdirector segments get <key>
|
|
184
184
|
configdirector metrics list
|
|
185
185
|
configdirector metrics get <event name>
|
|
186
|
+
configdirector metrics get --id <metric id>
|
|
186
187
|
```
|
|
187
188
|
|
|
188
189
|
`projects list` lists your organization's projects with their slugs and names, whichever project is selected. The other commands read the project from `--project`, `CONFIGDIRECTOR_PROJECT` or `configdirector.json`, as `scan` does:
|
|
@@ -192,7 +193,7 @@ configdirector metrics get <event name>
|
|
|
192
193
|
- `configs get <key>` shows the config's settings, including its type's constraints, then each environment's targeting rules in order, written as `evaluate` writes them, with the default targeting rule value last. An old key works, and the output then starts with a line naming the current key.
|
|
193
194
|
- `configs activity <key>` shows when SDKs last evaluated the config, one line per environment, SDK key kind, SDK name and SDK version, with the time in UTC.
|
|
194
195
|
- `segments list` lists the segments with how many configs use each, and `segments get <key>` shows a segment's condition groups and its environment overrides.
|
|
195
|
-
- `metrics list` lists the metrics with their kinds, their direction and how many experiments use each, and `metrics get <event name>` shows a metric's kinds and direction, and each experiment that uses it, with its role and the environments of its runs in progress.
|
|
196
|
+
- `metrics list` lists the metrics with their kinds, their direction and how many experiments use each, and `metrics get <event name>` shows a metric's kinds and direction, and each experiment that uses it, with its role and the environments of its runs in progress. Both name a metric by its event name, then its filter's conditions, then its value field, as in `checkout_completed · /plan is one of [team] · value from /items`. When several metrics have the event name, `metrics get` lists them with their IDs and exits with 2, and `metrics get --id <metric id>` shows one of them.
|
|
196
197
|
|
|
197
198
|
```txt
|
|
198
199
|
new-checkout
|
|
@@ -218,7 +219,7 @@ On an experiment, each value comes after the name of its variation, as in `40% g
|
|
|
218
219
|
|
|
219
220
|
`projects list`, `environments list`, `configs list`, `configs activity`, `segments list` and `metrics list` print a header row that names their columns above their lines, such as `KEY TYPE ROLE LIFETIME STATE` for `configs list`, and `--no-headers` leaves it out. An empty list prints only a sentence, such as `No configs found in web-app.`
|
|
220
221
|
|
|
221
|
-
With `--json`, each prints the Admin API's response as received, except `configs list --state`, which prints only the configs in that state, and `metrics get`, which prints the metric as the Admin API listed it for its event name. A key no config or segment has, or an event name no metric has, exits with 2.
|
|
222
|
+
With `--json`, each prints the Admin API's response as received, except `configs list --state`, which prints only the configs in that state, and `metrics get`, which prints the metric as the Admin API listed it for its event name, or returned it for its ID. A key no config or segment has, or an event name or ID no metric has, exits with 2.
|
|
222
223
|
|
|
223
224
|
`projects list` needs `projects:read`. The others need `projects:read` to find the project, and `environments list` needs `environments:read`, `configs list`, `configs get` and `configs activity` need `configs:read`, plus `tags:read` for `--tag`, `segments list` and `segments get` need `segments:read`, and `metrics list` and `metrics get` need `metrics:read`. Without `segments:read`, `configs get` shows a segment condition's segment ID instead of its name; without `environments:read`, `segments get` names an override's environment by its ID instead of its slug, and `metrics get` a run's environment.
|
|
224
225
|
|
|
@@ -342,7 +343,15 @@ Running since 2026-11-03 12:00 UTC
|
|
|
342
343
|
810 contexts exposed, 3201 evaluations. The split matches the configured shares.
|
|
343
344
|
Primary metric: checkout_completed, conversion, higher is better
|
|
344
345
|
control (Start free trial) 2.9% control
|
|
345
|
-
treatment-a (Try it free for 14 days)
|
|
346
|
+
treatment-a (Try it free for 14 days) 3.5% +19% (-43% to +154%) 69% chance to beat control no clear difference
|
|
347
|
+
How long to run, at the pace since 2026-11-04 00:00 UTC: about 98 new exposures a day
|
|
348
|
+
For a real lift of +2% 810 of about 2,082,762 exposures more than a year remaining
|
|
349
|
+
For a real lift of +5% 810 of about 338,034 exposures more than a year remaining
|
|
350
|
+
For a real lift of +10% 810 of about 86,501 exposures more than a year remaining
|
|
351
|
+
For a real lift of +20% 810 of about 22,617 exposures about 222 days remaining
|
|
352
|
+
For a real lift of +50% 810 of about 4,089 exposures about 34 days remaining
|
|
353
|
+
A row's lift is one a variation might really have over the control, not the lift measured so far: the smaller it is, the longer the run.
|
|
354
|
+
Whatever the estimate says, a run should cover at least one full week, so every day of the week is in the result.
|
|
346
355
|
Guardrail: support_ticket_opened, count per context, lower is better
|
|
347
356
|
control (Start free trial) 0.12 control
|
|
348
357
|
treatment-a (Try it free for 14 days) 0.13 +8% (-10% to +29%) 22% chance to beat control no clear difference
|
|
@@ -350,7 +359,18 @@ Running since 2026-11-03 12:00 UTC
|
|
|
350
359
|
Results can be up to 25 minutes behind.
|
|
351
360
|
```
|
|
352
361
|
|
|
353
|
-
Each kind a metric reports gets a line with the metric's role, its
|
|
362
|
+
Each kind a metric reports gets a line with the metric's role, its name, the kind and its direction, then one row per variation, the control first: its value, and for every other variation the lift against the control with the range it is likely in, the chance to beat control and the verdict, `likely better`, `likely worse` or `no clear difference`. With three or more variations, each row also shows the variation's chance to be best. Until every variation has enough exposures, the rollout says the run needs more exposures and shows the values alone. While a run is going, `How long to run` follows the primary metric's results: for a real lift of 2%, 5%, 10%, 20% and 50%, in the metric's better direction, the exposures so far against those needed before a variation with that lift has a good chance of reading likely better, and about how many days remain at the pace of new exposures since the time it names. An ended run's last line says when its final results were taken, once they are in. A run also shows its hypothesis and the note it ended with, and lists the events sent with `track` during the run that no metric measures.
|
|
363
|
+
|
|
364
|
+
A run counts a metric with a filter or a value field from the moment it is attached to the experiment. When that was during the run, a line under each of the metric's kinds says since when, and that events reported before then are not in its numbers. When the metric was not attached while the run was open, the run shows the metric's line and says it did not count it, with no numbers:
|
|
365
|
+
|
|
366
|
+
```txt
|
|
367
|
+
Secondary metric: checkout_completed · /plan is one of [team], conversion, higher is better
|
|
368
|
+
Counted since 2026-11-05 09:30 UTC, when this metric was attached to the experiment. Events reported before then are not in these numbers.
|
|
369
|
+
control (Start free trial) 2.1% control
|
|
370
|
+
treatment-a (Try it free for 14 days) 3.0% +43% (-5% to +110%) 91% chance to beat control no clear difference
|
|
371
|
+
Guardrail: checkout_completed · value from /refunded, lower is better
|
|
372
|
+
This run did not count this metric: a metric with a filter or a value field is counted from the moment it is attached to an experiment, and this one was not attached while the run was open.
|
|
373
|
+
```
|
|
354
374
|
|
|
355
375
|
- `configs ship` serves one variation to everyone in the environment and ends the run. `--variation` takes its name or its ID. The default targeting rule value becomes the variation's value, every percentage rollout is removed, including one a conditional rule serves, and a conditional rule serving a single value stays.
|
|
356
376
|
- `configs stop` ends the run without choosing a variation: every percentage rollout is removed, including one a conditional rule serves, and the default targeting rule value and any conditional rule serving a single value stay.
|
|
@@ -418,17 +438,38 @@ configdirector segments delete beta-testers
|
|
|
418
438
|
|
|
419
439
|
```bash
|
|
420
440
|
configdirector metrics create --event-name checkout_completed --kind conversion --kind value-per-context --direction higher-is-better
|
|
441
|
+
configdirector metrics create --event-name checkout_completed --filter team-plan.json --value-field /items --kind value-per-context --direction higher-is-better
|
|
421
442
|
configdirector metrics update checkout_completed --direction lower-is-better
|
|
422
|
-
configdirector metrics delete
|
|
443
|
+
configdirector metrics delete --id 4f5a6b7c-8d9e-4f0a-9b1c-3d4e5f6a7b8c
|
|
423
444
|
```
|
|
424
445
|
|
|
425
|
-
A metric is what an experiment measures:
|
|
446
|
+
A metric is what an experiment measures: the events with one name the application sends with the SDK's `track` call, all of them or only those whose details match the metric's filter, the numbers it reports and whether a higher or a lower number is better. A project can have several metrics on one event name, told apart by their filter and value field, and the commands name a metric by its event name, then its filter's conditions, then its value field, as in `checkout_completed · /plan is one of [team] · value from /items`. Attaching metrics to an experiment happens on its Settings tab in the dashboard, or through the Admin API or the MCP server.
|
|
426
447
|
|
|
427
448
|
- `metrics create` needs `--event-name`, exactly as the application passes it to `track`; `--kind`, repeatable: `conversion`, `count-per-context` or `value-per-context`; and `--direction`: `higher-is-better` or `lower-is-better`. `--description <text>` is optional. It prints `Created metric checkout_completed in web-app.`, then the metric as `metrics get` shows it.
|
|
428
|
-
- `metrics
|
|
429
|
-
|
|
449
|
+
- `metrics create --filter <file>`, or `--filter -` for stdin, counts only the events whose details match every condition in the file, 1 to 5 of them, in the shape `metrics get --json` prints a metric's `filter` in:
|
|
450
|
+
|
|
451
|
+
```json
|
|
452
|
+
[{ "field": "/plan", "targetType": "text", "operator": "is one of", "targetValues": ["team"] }]
|
|
453
|
+
```
|
|
454
|
+
|
|
455
|
+
`field` is a JSON pointer into the details the application sends with the event. A `text` condition takes `is one of` or `is NOT one of`; a `number` condition takes `=`, `!=`, `>`, `>=`, `<` or `<=`, with its number as text, such as `"2"`; and a `boolean` condition takes `equals`, with `"true"` or `"false"`. A condition means what it means in a targeting rule, so a field an event lacks reads as empty text, and `is NOT one of` and `!=` count an event without it.
|
|
456
|
+
|
|
457
|
+
- `metrics create --value-field <pointer>` sums the number at that field of the event's details for value per context, in place of the event's value, and needs `--kind value-per-context`.
|
|
458
|
+
- A metric keeps its filter and value field for its whole life: for others, create another metric. A run counts a metric with a filter or a value field from the moment it is attached to the experiment.
|
|
459
|
+
- `metrics update <event name>` changes what its options name: `--kind`, repeated, replaces the kinds, and `--direction` and `--description` replace theirs; `--description ""` removes the description. A metric keeps its event name, filter and value field. Removing a kind an experiment's primary metric decides on is refused, and the command exits with 2 listing those experiments.
|
|
460
|
+
- `metrics delete <event name>` deletes the metric and detaches it from the experiments that use it, and prints `Deleted metric checkout_completed from web-app.`; `--json` prints `{ "deletedMetricId": "1c2d3e4f-5a6b-4c7d-8e9f-0a1b2c3d4e5f", "deletedEventName": "checkout_completed" }`. While an experiment that uses the metric has a run in progress, it is not deleted: the command exits with 2 and lists each experiment with the environments of its runs.
|
|
461
|
+
|
|
462
|
+
`metrics update` and `metrics delete` take the event name while only one metric has it. When several metrics have it, they change nothing: they list the metrics with their IDs and exit with 2, and `--id <metric id>` names one of them:
|
|
463
|
+
|
|
464
|
+
```txt
|
|
465
|
+
Error: 2 metrics in web-app have the event name "checkout_completed". Name one with --id:
|
|
466
|
+
1c2d3e4f-5a6b-4c7d-8e9f-0a1b2c3d4e5f checkout_completed
|
|
467
|
+
4f5a6b7c-8d9e-4f0a-9b1c-3d4e5f6a7b8c checkout_completed · /plan is one of [team]
|
|
468
|
+
```
|
|
469
|
+
|
|
470
|
+
Versions of the CLI before 0.23.0 act on the first metric ConfigDirector lists for an event name, so once a project has several metrics on one event name, their `metrics update` and `metrics delete` can change or delete another metric than the one you mean. Upgrade every CLI that changes metrics before you create a second metric on an event name.
|
|
430
471
|
|
|
431
|
-
None of them asks for confirmation. An event name no metric has exits with 2 for `update` and `delete`, and so does a metric ConfigDirector refuses, such as
|
|
472
|
+
None of them asks for confirmation. An event name or ID no metric has exits with 2 for `update` and `delete`, and so does a metric ConfigDirector refuses, such as one with the same event name, filter and value field as another. With `--json`, `create` and `update` print the metric as the Admin API returned it.
|
|
432
473
|
|
|
433
474
|
`metrics create` needs `projects:read` and `metrics:create`, `metrics update` needs `projects:read`, `metrics:read` and `metrics:update`, and `metrics delete` needs `projects:read`, `metrics:read` and `metrics:delete`. Without `environments:read`, `metrics update` and `metrics delete` name environments by ID instead of by slug. The change shows in the audit log with the source CLI.
|
|
434
475
|
|
|
@@ -458,7 +499,7 @@ It asks for no confirmation and does not check whether the code still reads the
|
|
|
458
499
|
| `--project <slug>` | `CONFIGDIRECTOR_PROJECT` | Project slug. Without either, every command that reads a project uses the `project` in `configdirector.json`. |
|
|
459
500
|
| `--json` | | Print JSON on stdout. Errors always go to stderr. |
|
|
460
501
|
|
|
461
|
-
The command exits with 0 when it succeeds, 1 when `validate` found a problem or `configs create` found a config that already has the key, and 2 when it could not run: a usage mistake, a rejected token, a suspended service, an unreachable API, or something the command names that does not exist, such as a config key, an environment, a saved context, a segment key, a tag or a scanned repository. `segments delete` also exits with 2 when the targeting rules of a config use the segment. A CI job can therefore tell a wrong config key from an outage.
|
|
502
|
+
The command exits with 0 when it succeeds, 1 when `validate` found a problem or `configs create` found a config that already has the key, and 2 when it could not run: a usage mistake, a rejected token, a suspended service, an unreachable API, or something the command names that does not exist, such as a config key, an environment, a saved context, a segment key, a tag or a scanned repository. `segments delete` also exits with 2 when the targeting rules of a config use the segment. `metrics get`, `metrics update` and `metrics delete` also exit with 2 when several metrics have the event name they are given. A CI job can therefore tell a wrong config key from an outage.
|
|
462
503
|
|
|
463
504
|
## License
|
|
464
505
|
|
package/dist/configdirector.mjs
CHANGED
|
@@ -7,14 +7,14 @@ import * as v from "valibot";
|
|
|
7
7
|
import { basename, dirname, extname, isAbsolute, join, relative, resolve, sep } from "node:path";
|
|
8
8
|
import { mkdir, open, readFile, readdir, rm, stat, writeFile } from "node:fs/promises";
|
|
9
9
|
import { createInterface } from "node:readline";
|
|
10
|
+
import { randomUUID } from "node:crypto";
|
|
11
|
+
import { text } from "node:stream/consumers";
|
|
10
12
|
import ignore from "ignore";
|
|
11
13
|
import picomatch from "picomatch";
|
|
12
14
|
import { execFile } from "node:child_process";
|
|
13
15
|
import { promisify } from "node:util";
|
|
14
|
-
import { randomUUID } from "node:crypto";
|
|
15
|
-
import { text } from "node:stream/consumers";
|
|
16
16
|
//#region package.json
|
|
17
|
-
var version = "0.
|
|
17
|
+
var version = "0.23.0";
|
|
18
18
|
//#endregion
|
|
19
19
|
//#region src/CliError.ts
|
|
20
20
|
var CliError = class extends Error {
|
|
@@ -121,6 +121,48 @@ const CopyPlanSchema = v.object({
|
|
|
121
121
|
});
|
|
122
122
|
const CopyPlanResponseSchema = v.object({ plan: CopyPlanSchema });
|
|
123
123
|
//#endregion
|
|
124
|
+
//#region src/admin-api/metrics.ts
|
|
125
|
+
const OpenRunsSchema = v.array(v.object({ environmentId: v.string() }));
|
|
126
|
+
const MetricUsageSchema = v.object({
|
|
127
|
+
configKey: v.string(),
|
|
128
|
+
role: v.string(),
|
|
129
|
+
primaryKind: v.nullable(v.string()),
|
|
130
|
+
openRuns: OpenRunsSchema
|
|
131
|
+
});
|
|
132
|
+
const MetricFilterConditionSchema = v.object({
|
|
133
|
+
field: v.string(),
|
|
134
|
+
targetType: v.string(),
|
|
135
|
+
operator: v.string(),
|
|
136
|
+
targetValues: v.array(v.string())
|
|
137
|
+
});
|
|
138
|
+
const MetricFilterSchema = v.nullable(v.array(MetricFilterConditionSchema));
|
|
139
|
+
const MetricSchema = v.object({
|
|
140
|
+
id: v.string(),
|
|
141
|
+
eventName: v.string(),
|
|
142
|
+
filter: MetricFilterSchema,
|
|
143
|
+
valueField: v.nullable(v.string()),
|
|
144
|
+
description: v.nullable(v.string()),
|
|
145
|
+
direction: v.string(),
|
|
146
|
+
kinds: v.array(v.string()),
|
|
147
|
+
usages: v.array(MetricUsageSchema)
|
|
148
|
+
});
|
|
149
|
+
const DecidingExperimentSchema = v.object({
|
|
150
|
+
configKey: v.string(),
|
|
151
|
+
primaryKind: v.string()
|
|
152
|
+
});
|
|
153
|
+
const MetricKindInUseBodySchema = v.object({
|
|
154
|
+
code: v.literal("METRIC_KIND_IN_USE"),
|
|
155
|
+
usages: v.array(DecidingExperimentSchema)
|
|
156
|
+
});
|
|
157
|
+
const ExperimentWithOpenRunsSchema = v.object({
|
|
158
|
+
configKey: v.string(),
|
|
159
|
+
openRuns: OpenRunsSchema
|
|
160
|
+
});
|
|
161
|
+
const MetricInUseBodySchema = v.object({
|
|
162
|
+
code: v.literal("METRIC_IN_USE"),
|
|
163
|
+
usages: v.array(ExperimentWithOpenRunsSchema)
|
|
164
|
+
});
|
|
165
|
+
//#endregion
|
|
124
166
|
//#region src/admin-api/experiment-runs.ts
|
|
125
167
|
const TimestampSchema = v.pipe(v.string(), v.isoTimestamp());
|
|
126
168
|
const RolloutExposuresSchema = v.object({
|
|
@@ -146,15 +188,37 @@ const VariationResultSchema = v.object({
|
|
|
146
188
|
chanceToBeBest: v.optional(v.number()),
|
|
147
189
|
verdict: v.optional(v.string())
|
|
148
190
|
});
|
|
191
|
+
const RunLengthEstimateSchema = v.object({
|
|
192
|
+
analyzedExposures: v.number(),
|
|
193
|
+
exposuresPerDay: v.number(),
|
|
194
|
+
paceMeasuredSince: TimestampSchema,
|
|
195
|
+
lifts: v.array(v.object({
|
|
196
|
+
lift: v.number(),
|
|
197
|
+
exposuresNeeded: v.number(),
|
|
198
|
+
daysRemaining: v.optional(v.number())
|
|
199
|
+
}))
|
|
200
|
+
});
|
|
149
201
|
const MetricKindResultSchema = v.object({
|
|
150
202
|
kind: v.string(),
|
|
151
203
|
rollouts: v.array(v.object({
|
|
152
204
|
ruleId: v.string(),
|
|
153
|
-
variations: v.array(VariationResultSchema)
|
|
205
|
+
variations: v.array(VariationResultSchema),
|
|
206
|
+
runLengthEstimate: v.optional(RunLengthEstimateSchema)
|
|
154
207
|
}))
|
|
155
208
|
});
|
|
209
|
+
const CountingSchema = v.variant("kind", [
|
|
210
|
+
v.object({ kind: v.literal("whole-run") }),
|
|
211
|
+
v.object({
|
|
212
|
+
kind: v.literal("since"),
|
|
213
|
+
since: TimestampSchema
|
|
214
|
+
}),
|
|
215
|
+
v.object({ kind: v.literal("not-counted") })
|
|
216
|
+
]);
|
|
156
217
|
const MetricResultSchema = v.object({
|
|
157
218
|
eventName: v.string(),
|
|
219
|
+
filter: MetricFilterSchema,
|
|
220
|
+
valueField: v.nullable(v.string()),
|
|
221
|
+
counting: CountingSchema,
|
|
158
222
|
direction: v.string(),
|
|
159
223
|
kinds: v.array(MetricKindResultSchema)
|
|
160
224
|
});
|
|
@@ -195,39 +259,6 @@ const ExperimentRunSchema = v.object({
|
|
|
195
259
|
});
|
|
196
260
|
const ExperimentRunsSchema = v.array(ExperimentRunSchema);
|
|
197
261
|
//#endregion
|
|
198
|
-
//#region src/admin-api/metrics.ts
|
|
199
|
-
const OpenRunsSchema = v.array(v.object({ environmentId: v.string() }));
|
|
200
|
-
const MetricUsageSchema = v.object({
|
|
201
|
-
configKey: v.string(),
|
|
202
|
-
role: v.string(),
|
|
203
|
-
primaryKind: v.nullable(v.string()),
|
|
204
|
-
openRuns: OpenRunsSchema
|
|
205
|
-
});
|
|
206
|
-
const MetricSchema = v.object({
|
|
207
|
-
id: v.string(),
|
|
208
|
-
eventName: v.string(),
|
|
209
|
-
description: v.nullable(v.string()),
|
|
210
|
-
direction: v.string(),
|
|
211
|
-
kinds: v.array(v.string()),
|
|
212
|
-
usages: v.array(MetricUsageSchema)
|
|
213
|
-
});
|
|
214
|
-
const DecidingExperimentSchema = v.object({
|
|
215
|
-
configKey: v.string(),
|
|
216
|
-
primaryKind: v.string()
|
|
217
|
-
});
|
|
218
|
-
const MetricKindInUseBodySchema = v.object({
|
|
219
|
-
code: v.literal("METRIC_KIND_IN_USE"),
|
|
220
|
-
usages: v.array(DecidingExperimentSchema)
|
|
221
|
-
});
|
|
222
|
-
const ExperimentWithOpenRunsSchema = v.object({
|
|
223
|
-
configKey: v.string(),
|
|
224
|
-
openRuns: OpenRunsSchema
|
|
225
|
-
});
|
|
226
|
-
const MetricInUseBodySchema = v.object({
|
|
227
|
-
code: v.literal("METRIC_IN_USE"),
|
|
228
|
-
usages: v.array(ExperimentWithOpenRunsSchema)
|
|
229
|
-
});
|
|
230
|
-
//#endregion
|
|
231
262
|
//#region src/admin-api/config-creation.ts
|
|
232
263
|
const CreatedConfigSchema = v.object({
|
|
233
264
|
key: v.string(),
|
|
@@ -706,14 +737,16 @@ var AdminApiClient = class {
|
|
|
706
737
|
listMetrics(projectId) {
|
|
707
738
|
return this.get(`/v1/projects/${encodeURIComponent(projectId)}/metrics`, v.array(MetricSchema));
|
|
708
739
|
}
|
|
709
|
-
async
|
|
740
|
+
async listMetricsWithEventName(projectId, eventName) {
|
|
710
741
|
const { parsedBody: metrics, responseBody } = await this.get(`/v1/projects/${encodeURIComponent(projectId)}/metrics?${new URLSearchParams({ eventName })}`, v.array(MetricSchema));
|
|
711
|
-
const
|
|
712
|
-
|
|
713
|
-
return metric === void 0 ? void 0 : {
|
|
742
|
+
const receivedMetrics = v.parse(v.array(v.unknown()), responseBody);
|
|
743
|
+
return metrics.flatMap((metric, index) => metric.eventName === eventName ? [{
|
|
714
744
|
parsedBody: metric,
|
|
715
|
-
responseBody:
|
|
716
|
-
};
|
|
745
|
+
responseBody: receivedMetrics[index]
|
|
746
|
+
}] : []);
|
|
747
|
+
}
|
|
748
|
+
getMetric(projectId, metricId) {
|
|
749
|
+
return this.getIfFound(`/v1/projects/${encodeURIComponent(projectId)}/metrics/${encodeURIComponent(metricId)}`, MetricSchema);
|
|
717
750
|
}
|
|
718
751
|
createMetric(projectId, request) {
|
|
719
752
|
return this.requestJson(`/v1/projects/${encodeURIComponent(projectId)}/metrics`, MetricSchema, {
|
|
@@ -1616,6 +1649,33 @@ const kindWords = (kind) => isMetricKind(kind) ? KIND_WORDS[kind] : kind;
|
|
|
1616
1649
|
const directionWords = (direction) => isOneOf(METRIC_DIRECTIONS, direction) ? DIRECTION_WORDS[direction] : direction;
|
|
1617
1650
|
const roleWords = (role) => isOneOf(METRIC_ROLES, role) ? ROLE_WORDS[role] : role;
|
|
1618
1651
|
//#endregion
|
|
1652
|
+
//#region src/metrics/metric-label.ts
|
|
1653
|
+
const FILTER_TARGET_TYPES = [
|
|
1654
|
+
"text",
|
|
1655
|
+
"number",
|
|
1656
|
+
"boolean"
|
|
1657
|
+
];
|
|
1658
|
+
const comparisonWithTargetValueList = (condition) => `${condition.operator} [${condition.targetValues.join(", ")}]`;
|
|
1659
|
+
const COMPARISON_WORDS = {
|
|
1660
|
+
text: comparisonWithTargetValueList,
|
|
1661
|
+
number: (condition) => `${condition.operator} ${condition.targetValues[0] ?? ""}`,
|
|
1662
|
+
boolean: (condition) => `is ${condition.targetValues[0] ?? ""}`
|
|
1663
|
+
};
|
|
1664
|
+
const conditionWords = (condition) => {
|
|
1665
|
+
const comparison = isOneOf(FILTER_TARGET_TYPES, condition.targetType) ? COMPARISON_WORDS[condition.targetType](condition) : comparisonWithTargetValueList(condition);
|
|
1666
|
+
return `${condition.field} ${comparison}`;
|
|
1667
|
+
};
|
|
1668
|
+
const metricLabel = (metric) => {
|
|
1669
|
+
const conditions = (metric.filter ?? []).map(conditionWords);
|
|
1670
|
+
const filterWords = conditions.length === 0 ? [] : [conditions.join(" and ")];
|
|
1671
|
+
const valueFieldWords = metric.valueField === null ? [] : [`value from ${metric.valueField}`];
|
|
1672
|
+
return [
|
|
1673
|
+
metric.eventName,
|
|
1674
|
+
...filterWords,
|
|
1675
|
+
...valueFieldWords
|
|
1676
|
+
].join(" · ");
|
|
1677
|
+
};
|
|
1678
|
+
//#endregion
|
|
1619
1679
|
//#region src/configs/run-results-text.ts
|
|
1620
1680
|
const VERDICT_WORDS = {
|
|
1621
1681
|
"likely-better": "likely better",
|
|
@@ -1676,9 +1736,54 @@ const resultRow = (variation, table) => {
|
|
|
1676
1736
|
}
|
|
1677
1737
|
};
|
|
1678
1738
|
const capitalized = (text) => `${text.charAt(0).toUpperCase()}${text.slice(1)}`;
|
|
1679
|
-
const
|
|
1680
|
-
|
|
1681
|
-
|
|
1739
|
+
const DAYS_IN_A_YEAR = 365;
|
|
1740
|
+
const formatCount = (count) => count.toLocaleString("en-US");
|
|
1741
|
+
const liftRowFormOf = (daysRemaining) => {
|
|
1742
|
+
if (daysRemaining === void 0) return { form: "no-pace" };
|
|
1743
|
+
if (daysRemaining === 0) return { form: "enough-exposures" };
|
|
1744
|
+
return daysRemaining > DAYS_IN_A_YEAR ? { form: "more-than-a-year" } : {
|
|
1745
|
+
form: "days-remaining",
|
|
1746
|
+
days: daysRemaining
|
|
1747
|
+
};
|
|
1748
|
+
};
|
|
1749
|
+
const daysRemainingCells = (liftRow) => {
|
|
1750
|
+
switch (liftRow.form) {
|
|
1751
|
+
case "days-remaining": return [`about ${formatCount(liftRow.days)} ${plural(liftRow.days, "day", "days")} remaining`];
|
|
1752
|
+
case "more-than-a-year": return ["more than a year remaining"];
|
|
1753
|
+
case "enough-exposures": return ["enough exposures"];
|
|
1754
|
+
case "no-pace": return [];
|
|
1755
|
+
}
|
|
1756
|
+
};
|
|
1757
|
+
const exposurePaceOf = (estimate) => estimate.exposuresPerDay > 0 ? {
|
|
1758
|
+
form: "new-exposures",
|
|
1759
|
+
exposuresPerDay: Math.round(estimate.exposuresPerDay)
|
|
1760
|
+
} : { form: "no-new-exposures" };
|
|
1761
|
+
const paceLine = (estimate) => {
|
|
1762
|
+
const since = utcMinute(estimate.paceMeasuredSince);
|
|
1763
|
+
const pace = exposurePaceOf(estimate);
|
|
1764
|
+
switch (pace.form) {
|
|
1765
|
+
case "new-exposures": return `How long to run, at the pace since ${since}: about ${formatCount(pace.exposuresPerDay)} ${plural(pace.exposuresPerDay, "new exposure", "new exposures")} a day`;
|
|
1766
|
+
case "no-new-exposures": return `How long to run: no new contexts were exposed since ${since}, so the days cannot be estimated.`;
|
|
1767
|
+
}
|
|
1768
|
+
};
|
|
1769
|
+
const runLengthEstimateLines = (estimate) => {
|
|
1770
|
+
if (estimate === void 0) return [];
|
|
1771
|
+
return [
|
|
1772
|
+
paceLine(estimate),
|
|
1773
|
+
...alignedColumns(estimate.lifts.map((liftEstimate) => [
|
|
1774
|
+
`For a real lift of ${formatLift(liftEstimate.lift)}`,
|
|
1775
|
+
`${formatCount(estimate.analyzedExposures)} of about ${formatCount(liftEstimate.exposuresNeeded)} exposures`,
|
|
1776
|
+
...daysRemainingCells(liftRowFormOf(liftEstimate.daysRemaining))
|
|
1777
|
+
])).map((line) => ` ${line}`),
|
|
1778
|
+
"A row's lift is one a variation might really have over the control, not the lift measured so far: the smaller it is, the longer the run.",
|
|
1779
|
+
"Whatever the estimate says, a run should cover at least one full week, so every day of the week is in the result."
|
|
1780
|
+
];
|
|
1781
|
+
};
|
|
1782
|
+
const roleAndMetric = (role, metric) => `${capitalized(roleWords(role))}: ${metricLabel(metric)}`;
|
|
1783
|
+
const kindResultLines = (role, metric, kindResult, rollout, variations, countingLines) => {
|
|
1784
|
+
const kindResultOfRollout = kindResult.rollouts.find((candidate) => candidate.ruleId === rollout.ruleId);
|
|
1785
|
+
if (kindResultOfRollout === void 0) return [];
|
|
1786
|
+
const variationResults = kindResultOfRollout.variations;
|
|
1682
1787
|
const table = {
|
|
1683
1788
|
kind: kindResult.kind,
|
|
1684
1789
|
controlVariationId: rollout.controlVariationId,
|
|
@@ -1686,7 +1791,23 @@ const kindResultLines = (role, metric, kindResult, rollout, variations) => {
|
|
|
1686
1791
|
variations,
|
|
1687
1792
|
showsChanceToBeBest: variationResults.some((variation) => variation.chanceToBeBest !== void 0)
|
|
1688
1793
|
};
|
|
1689
|
-
return [
|
|
1794
|
+
return [
|
|
1795
|
+
`${roleAndMetric(role, metric)}, ${kindWords(kindResult.kind)}, ${directionWords(metric.direction)}`,
|
|
1796
|
+
...countingLines,
|
|
1797
|
+
...alignedColumns(variationResults.map((variation) => resultRow(variation, table))).map((line) => ` ${line}`.trimEnd()),
|
|
1798
|
+
...runLengthEstimateLines(kindResultOfRollout.runLengthEstimate)
|
|
1799
|
+
];
|
|
1800
|
+
};
|
|
1801
|
+
const NOT_COUNTED_LINE = " This run did not count this metric: a metric with a filter or a value field is counted from the moment it is attached to an experiment, and this one was not attached while the run was open.";
|
|
1802
|
+
const countedSinceLine = (since) => ` Counted since ${utcMinute(since)}, when this metric was attached to the experiment. Events reported before then are not in these numbers.`;
|
|
1803
|
+
const metricResultLines = (role, metric, rollout, variations) => {
|
|
1804
|
+
const kindResultsWith = (countingLines) => metric.kinds.flatMap((kindResult) => kindResultLines(role, metric, kindResult, rollout, variations, countingLines));
|
|
1805
|
+
const { counting } = metric;
|
|
1806
|
+
switch (counting.kind) {
|
|
1807
|
+
case "whole-run": return kindResultsWith([]);
|
|
1808
|
+
case "since": return kindResultsWith([countedSinceLine(counting.since)]);
|
|
1809
|
+
case "not-counted": return [`${roleAndMetric(role, metric)}, ${directionWords(metric.direction)}`, NOT_COUNTED_LINE];
|
|
1810
|
+
}
|
|
1690
1811
|
};
|
|
1691
1812
|
const crossoverLines = (crossovers) => {
|
|
1692
1813
|
if (crossovers === 0) return [];
|
|
@@ -1712,7 +1833,7 @@ const rolloutResultLines = (runResults, ruleId, variations) => {
|
|
|
1712
1833
|
return [
|
|
1713
1834
|
...comparisonLines(comparisonOf(rollout)),
|
|
1714
1835
|
...crossoverLines(rollout.crossovers),
|
|
1715
|
-
...METRIC_ROLES.flatMap((role) => results[role].flatMap((metric) =>
|
|
1836
|
+
...METRIC_ROLES.flatMap((role) => results[role].flatMap((metric) => metricResultLines(role, metric, rollout, variations)))
|
|
1716
1837
|
];
|
|
1717
1838
|
}
|
|
1718
1839
|
}
|
|
@@ -2599,6 +2720,43 @@ const metricChangesOf = (options) => {
|
|
|
2599
2720
|
return changes;
|
|
2600
2721
|
};
|
|
2601
2722
|
//#endregion
|
|
2723
|
+
//#region src/metrics/metric-selection.ts
|
|
2724
|
+
const metricSelectionOf = (eventName, options) => {
|
|
2725
|
+
if (eventName !== void 0 && options.id !== void 0) throw new CliError("Name the metric by its event name or by its id with --id, not both.");
|
|
2726
|
+
if (eventName !== void 0) return {
|
|
2727
|
+
by: "event-name",
|
|
2728
|
+
eventName
|
|
2729
|
+
};
|
|
2730
|
+
if (options.id !== void 0) return {
|
|
2731
|
+
by: "id",
|
|
2732
|
+
metricId: options.id
|
|
2733
|
+
};
|
|
2734
|
+
throw new CliError("Name the metric by its event name, or by its id with --id.");
|
|
2735
|
+
};
|
|
2736
|
+
const noSelectedMetric = (project, selection) => {
|
|
2737
|
+
switch (selection.by) {
|
|
2738
|
+
case "event-name": return new CliError(`No metric in ${project.slug} has the event name "${selection.eventName}".`);
|
|
2739
|
+
case "id": return new CliError(`No metric in ${project.slug} has the id "${selection.metricId}".`);
|
|
2740
|
+
}
|
|
2741
|
+
};
|
|
2742
|
+
const severalMetricsWithEventName = (project, eventName, metrics) => new CliError([`${metrics.length} metrics in ${project.slug} have the event name "${eventName}". Name one with --id:`, ...alignedColumns(metrics.map((metric) => [metric.id, metricLabel(metric)])).map((line) => ` ${line}`)].join("\n"));
|
|
2743
|
+
const onlyMetricWithEventName = (project, selection, metricsWithEventName) => {
|
|
2744
|
+
const [onlyMetric, ...otherMetrics] = metricsWithEventName;
|
|
2745
|
+
if (onlyMetric === void 0) throw noSelectedMetric(project, selection);
|
|
2746
|
+
if (otherMetrics.length > 0) throw severalMetricsWithEventName(project, selection.eventName, metricsWithEventName.map((metric) => metric.parsedBody));
|
|
2747
|
+
return onlyMetric;
|
|
2748
|
+
};
|
|
2749
|
+
const findSelectedMetric = async (client, project, selection) => {
|
|
2750
|
+
switch (selection.by) {
|
|
2751
|
+
case "event-name": return onlyMetricWithEventName(project, selection, await client.listMetricsWithEventName(project.id, selection.eventName));
|
|
2752
|
+
case "id": {
|
|
2753
|
+
const metric = await client.getMetric(project.id, selection.metricId);
|
|
2754
|
+
if (metric === void 0) throw noSelectedMetric(project, selection);
|
|
2755
|
+
return metric;
|
|
2756
|
+
}
|
|
2757
|
+
}
|
|
2758
|
+
};
|
|
2759
|
+
//#endregion
|
|
2602
2760
|
//#region src/metrics/metric-text.ts
|
|
2603
2761
|
const openRunEnvironmentNames = (openRuns, environmentSlugs) => openRuns.map((run) => environmentSlugs.get(run.environmentId) ?? run.environmentId).join(", ");
|
|
2604
2762
|
const usageLine$1 = (usage, environmentSlugs) => {
|
|
@@ -2608,7 +2766,7 @@ const usageLine$1 = (usage, environmentSlugs) => {
|
|
|
2608
2766
|
};
|
|
2609
2767
|
const usageLines = (metric, environmentSlugs) => metric.usages.length === 0 ? ["No experiment uses it."] : ["Used by", ...metric.usages.map((usage) => usageLine$1(usage, environmentSlugs))];
|
|
2610
2768
|
const metricText = (metric, environmentSlugs) => [
|
|
2611
|
-
metric
|
|
2769
|
+
metricLabel(metric),
|
|
2612
2770
|
...metric.description === null ? [] : [` Description: ${metric.description}`],
|
|
2613
2771
|
` Kinds: ${metric.kinds.map(kindWords).join(", ")}`,
|
|
2614
2772
|
` Direction: ${directionWords(metric.direction)}`,
|
|
@@ -2617,9 +2775,49 @@ const metricText = (metric, environmentSlugs) => [
|
|
|
2617
2775
|
""
|
|
2618
2776
|
].join("\n");
|
|
2619
2777
|
//#endregion
|
|
2778
|
+
//#region src/request-file.ts
|
|
2779
|
+
const STANDARD_INPUT_ARGUMENT = "-";
|
|
2780
|
+
const readRequestSource = async (argumentLabel, argument, invocation) => {
|
|
2781
|
+
if (argument === STANDARD_INPUT_ARGUMENT) return {
|
|
2782
|
+
origin: `${argumentLabel} reads stdin`,
|
|
2783
|
+
contents: await text(invocation.stdin)
|
|
2784
|
+
};
|
|
2785
|
+
const { file, contents } = await readOptionFile(argumentLabel, argument, invocation.workingDirectory);
|
|
2786
|
+
return {
|
|
2787
|
+
origin: `${argumentLabel} names ${file}`,
|
|
2788
|
+
contents
|
|
2789
|
+
};
|
|
2790
|
+
};
|
|
2791
|
+
const readRequestJson = async (argumentLabel, argument, invocation) => {
|
|
2792
|
+
const { origin, contents } = await readRequestSource(argumentLabel, argument, invocation);
|
|
2793
|
+
return {
|
|
2794
|
+
origin,
|
|
2795
|
+
request: parseJson(contents)
|
|
2796
|
+
};
|
|
2797
|
+
};
|
|
2798
|
+
const readRequestFile = async (argumentLabel, argument, invocation, expectedFormatSentence) => {
|
|
2799
|
+
const { origin, request } = await readRequestJson(argumentLabel, argument, invocation);
|
|
2800
|
+
if (!isJsonObject(request)) throw new CliError(`${origin}, which does not hold a JSON object. ${expectedFormatSentence}`);
|
|
2801
|
+
return request;
|
|
2802
|
+
};
|
|
2803
|
+
const readRequestListFile = async (argumentLabel, argument, invocation, expectedFormatSentence) => {
|
|
2804
|
+
const { origin, request } = await readRequestJson(argumentLabel, argument, invocation);
|
|
2805
|
+
if (!Array.isArray(request)) throw new CliError(`${origin}, which does not hold a JSON array. ${expectedFormatSentence}`);
|
|
2806
|
+
return request;
|
|
2807
|
+
};
|
|
2808
|
+
const withId = (entry) => entry.id === void 0 ? {
|
|
2809
|
+
id: randomUUID(),
|
|
2810
|
+
...entry
|
|
2811
|
+
} : entry;
|
|
2812
|
+
//#endregion
|
|
2813
|
+
//#region src/metrics/filter-file.ts
|
|
2814
|
+
const readFilterFile = (filterArgument, invocation) => readRequestListFile("--filter", filterArgument, invocation, "It takes the conditions an event's details must match, such as [{\"field\": \"/plan\", \"targetType\": \"text\", \"operator\": \"is one of\", \"targetValues\": [\"team\"]}], like the filter that metrics get --json prints.");
|
|
2815
|
+
//#endregion
|
|
2620
2816
|
//#region src/metrics/new-metric.ts
|
|
2621
|
-
const newMetricOf = (options) => ({
|
|
2817
|
+
const newMetricOf = async (options, invocation) => ({
|
|
2622
2818
|
eventName: options.eventName,
|
|
2819
|
+
...options.filter === void 0 ? {} : { filter: await readFilterFile(options.filter, invocation) },
|
|
2820
|
+
...options.valueField === void 0 ? {} : { valueField: options.valueField },
|
|
2623
2821
|
kinds: options.kind,
|
|
2624
2822
|
direction: options.direction,
|
|
2625
2823
|
...options.description === void 0 ? {} : { description: options.description }
|
|
@@ -2628,13 +2826,13 @@ const newMetricOf = (options) => ({
|
|
|
2628
2826
|
//#region src/commands/metrics.ts
|
|
2629
2827
|
const usageDescription$1 = (metric) => `used by ${metric.usages.length} ${plural(metric.usages.length, "experiment", "experiments")}`;
|
|
2630
2828
|
const METRIC_HEADER_ROW = [
|
|
2631
|
-
"
|
|
2829
|
+
"METRIC",
|
|
2632
2830
|
"KINDS",
|
|
2633
2831
|
"DIRECTION",
|
|
2634
2832
|
"USED BY"
|
|
2635
2833
|
];
|
|
2636
2834
|
const metricLines = (projectSlug, metrics, options) => metrics.length === 0 ? [`No metrics found in ${projectSlug}.`] : alignedListing(METRIC_HEADER_ROW, metrics.map((metric) => [
|
|
2637
|
-
metric
|
|
2835
|
+
metricLabel(metric),
|
|
2638
2836
|
metric.kinds.map(kindWords).join(", "),
|
|
2639
2837
|
directionWords(metric.direction),
|
|
2640
2838
|
usageDescription$1(metric)
|
|
@@ -2649,12 +2847,10 @@ const listMetrics = async (context, _commandArguments, options) => {
|
|
|
2649
2847
|
};
|
|
2650
2848
|
};
|
|
2651
2849
|
const openRunEnvironmentSlugsOf = async (client, project, metric) => metric.usages.some((usage) => usage.openRuns.length > 0) ? environmentSlugsById(client, project) : /* @__PURE__ */ new Map();
|
|
2652
|
-
const
|
|
2653
|
-
const
|
|
2850
|
+
const getMetric = async (context, [eventName], options) => {
|
|
2851
|
+
const selection = metricSelectionOf(eventName, options);
|
|
2654
2852
|
const { client, project } = await connectToLinkedProject(context);
|
|
2655
|
-
const
|
|
2656
|
-
if (receivedMetric === void 0) throw noMetricWithEventName(project, eventName);
|
|
2657
|
-
const { parsedBody: metric, responseBody } = receivedMetric;
|
|
2853
|
+
const { parsedBody: metric, responseBody } = await findSelectedMetric(client, project, selection);
|
|
2658
2854
|
return {
|
|
2659
2855
|
json: responseBody,
|
|
2660
2856
|
text: metricText(metric, await openRunEnvironmentSlugsOf(client, project, metric)),
|
|
@@ -2662,57 +2858,61 @@ const getMetric = async (context, [eventName]) => {
|
|
|
2662
2858
|
};
|
|
2663
2859
|
};
|
|
2664
2860
|
const createMetric = async (context, _commandArguments, options) => {
|
|
2861
|
+
const newMetric = await newMetricOf(options, context.invocation);
|
|
2665
2862
|
const { client, project } = await connectToLinkedProject(context);
|
|
2666
|
-
const { parsedBody: metric, responseBody } = await client.createMetric(project.id,
|
|
2863
|
+
const { parsedBody: metric, responseBody } = await client.createMetric(project.id, newMetric);
|
|
2667
2864
|
return {
|
|
2668
2865
|
json: responseBody,
|
|
2669
|
-
text: [`Created metric ${metric
|
|
2866
|
+
text: [`Created metric ${metricLabel(metric)} in ${project.slug}.`, metricText(metric, /* @__PURE__ */ new Map())].join("\n"),
|
|
2670
2867
|
exitCode: ExitCode.success
|
|
2671
2868
|
};
|
|
2672
2869
|
};
|
|
2673
|
-
const kindInUseError = (
|
|
2674
|
-
`${experiments.length} ${plural(experiments.length, "experiment decides", "experiments decide")} on a kind this change removes, so ${
|
|
2870
|
+
const kindInUseError = (metric, experiments) => new CliError([
|
|
2871
|
+
`${experiments.length} ${plural(experiments.length, "experiment decides", "experiments decide")} on a kind this change removes, so ${metricLabel(metric)} was not updated:`,
|
|
2675
2872
|
...experiments.map((experiment) => ` ${experiment.configKey} decides on ${kindWords(experiment.primaryKind)}`),
|
|
2676
2873
|
`Give ${plural(experiments.length, "it", "each of them")} another primary kind first, on the experiment's Settings tab in the dashboard.`
|
|
2677
2874
|
].join("\n"));
|
|
2678
2875
|
const updateMetric = async (context, [eventName], options) => {
|
|
2876
|
+
const selection = metricSelectionOf(eventName, options);
|
|
2679
2877
|
const changes = metricChangesOf(options);
|
|
2680
2878
|
const { client, project } = await connectToLinkedProject(context);
|
|
2681
|
-
const currentMetric = await client
|
|
2682
|
-
|
|
2683
|
-
const update = await client.updateMetric(project.id, currentMetric.parsedBody.id, changes);
|
|
2879
|
+
const { parsedBody: currentMetric } = await findSelectedMetric(client, project, selection);
|
|
2880
|
+
const update = await client.updateMetric(project.id, currentMetric.id, changes);
|
|
2684
2881
|
switch (update.outcome) {
|
|
2685
|
-
case "not-found": throw
|
|
2686
|
-
case "kind-in-use": throw kindInUseError(
|
|
2882
|
+
case "not-found": throw noSelectedMetric(project, selection);
|
|
2883
|
+
case "kind-in-use": throw kindInUseError(currentMetric, update.decidingExperiments);
|
|
2687
2884
|
case "updated": {
|
|
2688
2885
|
const { parsedBody: metric, responseBody } = update.metric;
|
|
2689
2886
|
return {
|
|
2690
2887
|
json: responseBody,
|
|
2691
|
-
text: [`Updated metric ${
|
|
2888
|
+
text: [`Updated metric ${metricLabel(metric)} in ${project.slug}.`, metricText(metric, await openRunEnvironmentSlugsOf(client, project, metric))].join("\n"),
|
|
2692
2889
|
exitCode: ExitCode.success
|
|
2693
2890
|
};
|
|
2694
2891
|
}
|
|
2695
2892
|
}
|
|
2696
2893
|
};
|
|
2697
2894
|
const experimentWithOpenRunsLine = (experiment, environmentSlugs) => ` ${experiment.configKey}, running in ${openRunEnvironmentNames(experiment.openRuns, environmentSlugs)}`;
|
|
2698
|
-
const metricInUseError = (
|
|
2699
|
-
`${experiments.length} ${plural(experiments.length, "experiment", "experiments")} with a run in progress ${plural(experiments.length, "uses", "use")} ${
|
|
2895
|
+
const metricInUseError = (metric, experiments, environmentSlugs) => new CliError([
|
|
2896
|
+
`${experiments.length} ${plural(experiments.length, "experiment", "experiments")} with a run in progress ${plural(experiments.length, "uses", "use")} ${metricLabel(metric)}, so it was not deleted:`,
|
|
2700
2897
|
...experiments.map((experiment) => experimentWithOpenRunsLine(experiment, environmentSlugs)),
|
|
2701
2898
|
"Ship or stop each run first, with configdirector configs ship or configs stop, or detach the metric on the experiment's Settings tab in the dashboard."
|
|
2702
2899
|
].join("\n"));
|
|
2703
|
-
const deleteMetric = async (context, [eventName]) => {
|
|
2900
|
+
const deleteMetric = async (context, [eventName], options) => {
|
|
2901
|
+
const selection = metricSelectionOf(eventName, options);
|
|
2704
2902
|
const { client, project } = await connectToLinkedProject(context);
|
|
2705
|
-
const
|
|
2706
|
-
|
|
2707
|
-
const deletion = await client.deleteMetric(project.id, currentMetric.parsedBody.id);
|
|
2903
|
+
const { parsedBody: metric } = await findSelectedMetric(client, project, selection);
|
|
2904
|
+
const deletion = await client.deleteMetric(project.id, metric.id);
|
|
2708
2905
|
switch (deletion.outcome) {
|
|
2709
2906
|
case "deleted": return {
|
|
2710
|
-
json: {
|
|
2711
|
-
|
|
2907
|
+
json: {
|
|
2908
|
+
deletedMetricId: metric.id,
|
|
2909
|
+
deletedEventName: metric.eventName
|
|
2910
|
+
},
|
|
2911
|
+
text: [`Deleted metric ${metricLabel(metric)} from ${project.slug}.`, ""].join("\n"),
|
|
2712
2912
|
exitCode: ExitCode.success
|
|
2713
2913
|
};
|
|
2714
|
-
case "not-found": throw
|
|
2715
|
-
case "in-use": throw metricInUseError(
|
|
2914
|
+
case "not-found": throw noSelectedMetric(project, selection);
|
|
2915
|
+
case "in-use": throw metricInUseError(metric, deletion.experimentsWithOpenRuns, await environmentSlugsById(client, project));
|
|
2716
2916
|
}
|
|
2717
2917
|
};
|
|
2718
2918
|
//#endregion
|
|
@@ -5399,30 +5599,6 @@ const scan = async (context, [path], options) => {
|
|
|
5399
5599
|
};
|
|
5400
5600
|
};
|
|
5401
5601
|
//#endregion
|
|
5402
|
-
//#region src/request-file.ts
|
|
5403
|
-
const STANDARD_INPUT_ARGUMENT = "-";
|
|
5404
|
-
const readRequestSource = async (argumentLabel, argument, invocation) => {
|
|
5405
|
-
if (argument === STANDARD_INPUT_ARGUMENT) return {
|
|
5406
|
-
origin: `${argumentLabel} reads stdin`,
|
|
5407
|
-
contents: await text(invocation.stdin)
|
|
5408
|
-
};
|
|
5409
|
-
const { file, contents } = await readOptionFile(argumentLabel, argument, invocation.workingDirectory);
|
|
5410
|
-
return {
|
|
5411
|
-
origin: `${argumentLabel} names ${file}`,
|
|
5412
|
-
contents
|
|
5413
|
-
};
|
|
5414
|
-
};
|
|
5415
|
-
const readRequestFile = async (argumentLabel, argument, invocation, expectedFormatSentence) => {
|
|
5416
|
-
const { origin, contents } = await readRequestSource(argumentLabel, argument, invocation);
|
|
5417
|
-
const request = parseJson(contents);
|
|
5418
|
-
if (!isJsonObject(request)) throw new CliError(`${origin}, which does not hold a JSON object. ${expectedFormatSentence}`);
|
|
5419
|
-
return request;
|
|
5420
|
-
};
|
|
5421
|
-
const withId = (entry) => entry.id === void 0 ? {
|
|
5422
|
-
id: randomUUID(),
|
|
5423
|
-
...entry
|
|
5424
|
-
} : entry;
|
|
5425
|
-
//#endregion
|
|
5426
5602
|
//#region src/segments/segment-file.ts
|
|
5427
5603
|
const readSegmentFile = (segmentArgument, invocation) => readRequestFile("--segment", segmentArgument, invocation, "It takes {\"key\": ..., \"name\": ..., \"groups\": [...]}, like the output of segments get --json.");
|
|
5428
5604
|
const conditionWithId = (condition) => isJsonObject(condition) ? withId(condition) : condition;
|
|
@@ -5810,6 +5986,9 @@ const validate = async (context, [path]) => {
|
|
|
5810
5986
|
//#region src/run.ts
|
|
5811
5987
|
const DEFAULT_BASE_URL = "https://api.configdirector.com";
|
|
5812
5988
|
const NO_HEADERS_DESCRIPTION = "leave out the header row";
|
|
5989
|
+
const METRIC_EVENT_NAME_DESCRIPTION = "the metric's event name, as the application passes it to track, while only one metric has it";
|
|
5990
|
+
const METRIC_ID_DESCRIPTION = "the metric's id, which names one of several metrics with the same event name, in place of the event name";
|
|
5991
|
+
const SEVERAL_METRICS_HELP = "When several metrics have the event name, the command lists them with their ids and exits with 2 without changing anything. Name one with --id.";
|
|
5813
5992
|
const SEGMENT_FILE_DESCRIPTION = "a JSON file holding the segment as the Admin API takes it, or - to read it from stdin (required)";
|
|
5814
5993
|
const withoutTrailingSlashes = (url) => {
|
|
5815
5994
|
let trimmedUrl = url;
|
|
@@ -5879,10 +6058,10 @@ const run = async (invocation) => {
|
|
|
5879
6058
|
segmentsCommand.command("delete").argument("<key>", "the segment's key").description("delete a segment and its environment overrides").addHelpText("after", "\nThe segment is deleted with no confirmation. While the targeting rules of a config use it, the deletion is refused and the configs are listed with their environments.").action(commandAction(deleteSegment));
|
|
5880
6059
|
const metricsCommand = program.command("metrics").description("list, show, create, update and delete the metrics a project's experiments measure");
|
|
5881
6060
|
metricsCommand.command("list").description("list the project's metrics with their kinds, direction and how many experiments use each").option("--no-headers", NO_HEADERS_DESCRIPTION).action(commandAction(listMetrics));
|
|
5882
|
-
metricsCommand.command("get").argument("
|
|
5883
|
-
metricsCommand.command("create").description("create
|
|
5884
|
-
metricsCommand.command("update").argument("
|
|
5885
|
-
metricsCommand.command("delete").argument("
|
|
6061
|
+
metricsCommand.command("get").argument("[event name]", METRIC_EVENT_NAME_DESCRIPTION).option("--id <metric id>", METRIC_ID_DESCRIPTION).description("show a metric's kinds and direction, and the experiments that use it").addHelpText("after", `\n${SEVERAL_METRICS_HELP}`).action(commandAction(getMetric));
|
|
6062
|
+
metricsCommand.command("create").description("create a metric for an event name the application passes to track").requiredOption("--event-name <event name>", "the event name, exactly as the application passes it to track (required)").option("--filter <file>", "a JSON file holding the conditions an event's details must match for the event to count, or - to read them from stdin").option("--value-field <pointer>", "a JSON pointer, such as /items, to the field of the event's details holding the number that value per context sums in place of the event's value").addOption(repeatableChoiceOption("--kind <kind>", "a number the metric reports: conversion, the share of exposed contexts that sent the event; count-per-context, how many times each sent it on average; value-per-context, the value sent with it per context on average; repeatable (required)", METRIC_KINDS).makeOptionMandatory()).addOption(new Option("--direction <direction>", "whether a higher or a lower number is better (required)").choices(METRIC_DIRECTIONS).makeOptionMandatory()).option("--description <text>", "what the event means").addHelpText("after", "\nThe --filter file holds a list of 1 to 5 conditions, all of which an event must match to count, like the filter that metrics get --json prints. For the checkouts on the Team plan: [{\"field\": \"/plan\", \"targetType\": \"text\", \"operator\": \"is one of\", \"targetValues\": [\"team\"]}]. field is a JSON pointer into the details the application sends with the event. A text condition takes is one of or is NOT one of; a number condition =, !=, >, >=, < or <=, with its number as text, such as \"2\"; and a boolean condition equals, with \"true\" or \"false\". A field an event lacks reads as empty text, so is NOT one of and != count an event without it.\n\n--value-field /items sums the number each event's details hold at /items for value-per-context, in place of the event's value, and needs --kind value-per-context.\n\nA metric keeps its filter and value field for its whole life: for others, create another metric. A metric with a filter or a value field is counted from the moment it is attached to an experiment.").action(commandAction(createMetric));
|
|
6063
|
+
metricsCommand.command("update").argument("[event name]", METRIC_EVENT_NAME_DESCRIPTION).option("--id <metric id>", METRIC_ID_DESCRIPTION).description("change a metric's kinds, direction or description, leaving the others as they are").addOption(repeatableChoiceOption("--kind <kind>", "a number the metric reports; repeatable, and the kinds given replace the metric's", METRIC_KINDS)).addOption(new Option("--direction <direction>", "whether a higher or a lower number is better").choices(METRIC_DIRECTIONS)).option("--description <text>", "what the event means; --description \"\" removes it").addHelpText("after", "\nA metric keeps its event name, filter and value field: for others, create another metric. Removing a kind that an experiment's primary metric decides on is refused, and the experiments are listed.\n\n" + SEVERAL_METRICS_HELP).action(commandAction(updateMetric));
|
|
6064
|
+
metricsCommand.command("delete").argument("[event name]", METRIC_EVENT_NAME_DESCRIPTION).option("--id <metric id>", METRIC_ID_DESCRIPTION).description("delete a metric and detach it from the experiments that use it").addHelpText("after", "\nThe metric is deleted with no confirmation. While an experiment that uses it has a run in progress, the deletion is refused and the experiments are listed with the environments of their runs. Events with its name keep being counted, so a metric created again for the name without a filter or a value field and attached again shows their history, while one with a filter or a value field is counted from the moment it is attached.\n\n" + SEVERAL_METRICS_HELP).action(commandAction(deleteMetric));
|
|
5886
6065
|
const targetingRulesCommand = program.command("targeting-rules").description("replace a config's targeting rules in one environment and show the format they take");
|
|
5887
6066
|
targetingRulesCommand.command("set").argument("<key>", "the config's key, or an old key of it").description("replace a config's targeting rules in one environment with the rules of a JSON file").requiredOption("--environment <slug>", "slug of the environment whose targeting rules to replace (required)").requiredOption("--rules <file>", "a JSON file holding defaultValue and rules as the Admin API takes them, or - to read it from stdin (required)").addHelpText("after", "\nThe file holds {\"defaultValue\": ..., \"rules\": [...]}, like an entry of targets in configs get --json, so you can edit that output and write it back. Every rule of the environment is replaced. A rule, condition or percentage without an id gets a new one. configdirector targeting-rules reference lists the condition kinds, attributes and operators.").action(commandAction(setTargetingRules));
|
|
5888
6067
|
targetingRulesCommand.command("reference").description("show the format targeting rules take: the condition kinds, the attributes a condition can test and the operators").action(commandAction(showTargetingRulesReference));
|
package/package.json
CHANGED