@thinkingai/ae-cli 6.0.48 → 6.1.0-project-role-upsert.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +4 -1
- package/README.zh.md +4 -1
- package/dist/{capability-J4FU4JZW.js → capability-AXFQW5WM.js} +13 -10
- package/dist/{chunk-6KBBAPPP.js → chunk-JDPEVVHR.js} +4 -3
- package/dist/index.js +23 -13
- package/dist/{metadata-3J5ZAVMF.js → metadata-SOALPEA7.js} +3 -3
- package/dist/{personal-semantic-preference-32AAJ37M.js → personal-semantic-preference-LOF33N5X.js} +2 -2
- package/dist/project-semantic-RV6NA5LN.js +1113 -0
- package/dist/{te-analysis-WHYXHRJD.js → te-analysis-TV2YZ4IC.js} +23 -7
- package/dist/{te-engage-DHKNJ5BO.js → te-engage-EUVMTUDJ.js} +63 -5
- package/dist/te-experiment-ESEYE2SK.js +1059 -0
- package/package.json +2 -1
- package/skills/ae-analysis/SKILL.md +17 -1
- package/skills/ae-analysis/references/command_index.md +1 -1
- package/skills/ae-analysis/references/personal_semantic_preference_add.md +1 -1
- package/skills/ae-analysis/references/personal_semantic_preference_get.md +3 -1
- package/skills/ae-analysis/references/personal_semantic_preference_list.md +3 -1
- package/skills/ae-analysis/references/personal_semantic_preference_update.md +1 -1
- package/skills/ae-analysis/references/project_role_upsert.md +32 -2
- package/skills/ae-engage/SKILL.md +2 -0
- package/skills/ae-engage/references/save-flow.md +37 -1
- package/skills/ae-experiment/SKILL.md +117 -0
- package/skills/ae-experiment/references/batch_delete_experiment.md +11 -0
- package/skills/ae-experiment/references/batch_delete_feature.md +11 -0
- package/skills/ae-experiment/references/batch_delete_traffic_layer.md +11 -0
- package/skills/ae-experiment/references/cancel_experiment_query_by_request_id.md +11 -0
- package/skills/ae-experiment/references/check_experiment_ready.md +17 -0
- package/skills/ae-experiment/references/check_experiment_traffic_layer_conflict.md +37 -0
- package/skills/ae-experiment/references/delete_metric.md +13 -0
- package/skills/ae-experiment/references/manage_experiment.md +17 -0
- package/skills/ae-experiment/references/manage_feature_status.md +11 -0
- package/skills/ae-experiment/references/manage_feature_whitelist.md +66 -0
- package/skills/ae-experiment/references/manage_guardrail_metrics.md +26 -0
- package/skills/ae-experiment/references/query_bucket_list.md +12 -0
- package/skills/ae-experiment/references/query_experiment_detail.md +19 -0
- package/skills/ae-experiment/references/query_experiment_list.md +22 -0
- package/skills/ae-experiment/references/query_experiment_list_archived.md +18 -0
- package/skills/ae-experiment/references/query_experiment_metric_trend.md +12 -0
- package/skills/ae-experiment/references/query_experiment_operation_log.md +17 -0
- package/skills/ae-experiment/references/query_experiment_report_summary.md +12 -0
- package/skills/ae-experiment/references/query_experiment_sample_size_report.md +13 -0
- package/skills/ae-experiment/references/query_feature_detail.md +14 -0
- package/skills/ae-experiment/references/query_feature_list.md +12 -0
- package/skills/ae-experiment/references/query_feature_operation_log.md +19 -0
- package/skills/ae-experiment/references/query_feature_version_list.md +20 -0
- package/skills/ae-experiment/references/query_metric_detail.md +18 -0
- package/skills/ae-experiment/references/query_metric_list.md +15 -0
- package/skills/ae-experiment/references/query_traffic_layer_detail.md +13 -0
- package/skills/ae-experiment/references/query_traffic_layer_list.md +12 -0
- package/skills/ae-experiment/references/save_build_guide.md +39 -0
- package/skills/ae-experiment/references/save_experiment.md +210 -0
- package/skills/ae-experiment/references/save_feature.md +13 -0
- package/skills/ae-experiment/references/save_metric.md +91 -0
- package/skills/ae-experiment/references/save_submit_experiment.md +16 -0
- package/skills/ae-experiment/references/save_traffic_layer.md +13 -0
- package/skills/ae-experiment/references/save_validate.md +33 -0
- package/skills/ae-experiment/references/update_experiment_group.md +20 -0
- package/skills/ae-experiment-design/SKILL.md +149 -0
- package/skills/ae-experiment-design/agents/openai.yaml +4 -0
- package/skills/ae-experiment-design/references/client-experiment-sdk.md +147 -0
- package/skills/ae-experiment-design/references/experiment-creation.md +108 -0
- package/skills/ae-experiment-design/references/experiment-sdk-contract.md +100 -0
- package/skills/ae-experiment-design/references/exposure-contract.md +91 -0
- package/skills/ae-experiment-design/references/hybrid-experiment-sdk.md +74 -0
- package/skills/ae-experiment-design/references/metric-readiness.md +143 -0
- package/skills/ae-experiment-design/references/platform-operations.md +105 -0
- package/skills/ae-experiment-design/references/sdk-index.md +76 -0
- package/skills/ae-experiment-design/references/sdk-integration.md +114 -0
- package/skills/ae-experiment-design/references/sdk-troubleshooting.md +139 -0
- package/skills/ae-experiment-design/references/server-experiment-sdk.md +78 -0
- package/skills/ae-experiment-design/scripts/calculate_experiment_plan.py +450 -0
- package/skills/ae-experiment-insight/SKILL.md +149 -0
- package/skills/ae-experiment-insight/agents/openai.yaml +4 -0
- package/skills/ae-experiment-insight/references/decision-framework.md +69 -0
- package/skills/ae-experiment-insight/references/diagnostic-playbook.md +225 -0
- package/skills/ae-experiment-insight/references/platform-operations.md +82 -0
- package/skills/ae-experiment-insight/scripts/analyze_experiment.py +478 -0
- package/skills/ae-project-semantic/SKILL.md +193 -0
- package/skills/ae-project-semantic/references/query-routing-v5.md +165 -0
- package/skills/ae-project-semantic/references/recommendation-quality.md +68 -0
- package/dist/{chunk-HPLNYQPM.js → chunk-RSG4ONOI.js} +3 -3
- package/dist/{sync-2GTAKZYX.js → sync-YV3E66IF.js} +3 -3
|
@@ -0,0 +1,22 @@
|
|
|
1
|
+
# experiment experiment list
|
|
2
|
+
|
|
3
|
+
Query experiments in a project with optional filters from the existing list API.
|
|
4
|
+
|
|
5
|
+
```bash
|
|
6
|
+
ae-cli experiment experiment list --project-id <id>
|
|
7
|
+
ae-cli experiment experiment list --project-id <id> --status archive
|
|
8
|
+
ae-cli experiment experiment list --project-id <id> --traffic-layer-id <layer-id> --group-id <group-id> --query-name checkout
|
|
9
|
+
```
|
|
10
|
+
|
|
11
|
+
Flags:
|
|
12
|
+
- `--project-id`, `-p`: Project ID.
|
|
13
|
+
- `--status`: Optional status filter (`draft`, `testing`, `pending`, `running`, `paused`, `ended`, `archive`). For the dedicated archived list command, prefer `experiment experiment list-archived`.
|
|
14
|
+
- `--traffic-layer-id`: Optional traffic layer ID filter.
|
|
15
|
+
- `--group-id`: Optional business group ID filter.
|
|
16
|
+
- `--query-name`: Optional fuzzy match against experiment name (`exp_name LIKE`). Does not match `exp_id`.
|
|
17
|
+
|
|
18
|
+
Response shape: `data.items` contains the experiments, `data.total` contains the count, and item keys are recursively snake_case.
|
|
19
|
+
|
|
20
|
+
Targeting follows the same semantic projection as detail:
|
|
21
|
+
`targeting.definition_request` plus `targeting.definition_status`; internal
|
|
22
|
+
`target_config` is not returned.
|
|
@@ -0,0 +1,18 @@
|
|
|
1
|
+
# experiment experiment list-archived
|
|
2
|
+
|
|
3
|
+
Query archived experiments in a project.
|
|
4
|
+
|
|
5
|
+
This command reuses the existing experiment list API and hardcodes `status=archive`.
|
|
6
|
+
It only accepts `--project-id`; it does not expose additional filters.
|
|
7
|
+
|
|
8
|
+
```bash
|
|
9
|
+
ae-cli experiment experiment list-archived --project-id <id>
|
|
10
|
+
```
|
|
11
|
+
|
|
12
|
+
Flags:
|
|
13
|
+
- `--project-id`, `-p`: Project ID.
|
|
14
|
+
|
|
15
|
+
Response shape: `data.items` contains the archived experiments, `data.total` contains the count, and item keys are recursively snake_case.
|
|
16
|
+
|
|
17
|
+
Targeting is returned through `targeting.definition_request` and
|
|
18
|
+
`targeting.definition_status`; internal `target_config` is not returned.
|
|
@@ -0,0 +1,12 @@
|
|
|
1
|
+
# experiment report metric-trend
|
|
2
|
+
|
|
3
|
+
Query experiment metric trend report.
|
|
4
|
+
|
|
5
|
+
```bash
|
|
6
|
+
ae-cli experiment report metric-trend --project-id 1 --exp-id exp_123 --metric-id metric_1 --start-time 2026-07-01 --end-time 2026-07-07
|
|
7
|
+
```
|
|
8
|
+
|
|
9
|
+
Required flags: `--project-id`, `--exp-id`, `--metric-id`, `--start-time`, `--end-time`.
|
|
10
|
+
Optional: `--request-id`, `--force-refresh`.
|
|
11
|
+
|
|
12
|
+
Response shape: `data.report`, with recursively snake_case keys. Preserve `request_id` for cancellation via `capability run experiment.query.cancel`.
|
|
@@ -0,0 +1,17 @@
|
|
|
1
|
+
# experiment operation-log query
|
|
2
|
+
|
|
3
|
+
Query full operation logs for one experiment.
|
|
4
|
+
|
|
5
|
+
Returns the complete Atlas experiment operation-log payload, including `remark`,
|
|
6
|
+
`operation_content`, and structured `changes` when present. No field projection.
|
|
7
|
+
|
|
8
|
+
```bash
|
|
9
|
+
ae-cli experiment operation-log query --project-id <id> --exp-id <exp-id>
|
|
10
|
+
```
|
|
11
|
+
|
|
12
|
+
Flags:
|
|
13
|
+
- `--project-id`, `-p`: Project ID.
|
|
14
|
+
- `--exp-id`: Experiment ID.
|
|
15
|
+
|
|
16
|
+
Response shape: `data.items` contains the full log records, `data.total` contains
|
|
17
|
+
the count, and item keys are recursively snake_case.
|
|
@@ -0,0 +1,12 @@
|
|
|
1
|
+
# experiment report summary
|
|
2
|
+
|
|
3
|
+
Query experiment report summary.
|
|
4
|
+
|
|
5
|
+
```bash
|
|
6
|
+
ae-cli experiment report summary --project-id 1 --exp-id exp_123
|
|
7
|
+
ae-cli experiment report summary --project-id 1 --exp-id exp_123 --force-refresh true
|
|
8
|
+
```
|
|
9
|
+
|
|
10
|
+
Required flags: `--project-id`, `--exp-id`. Optional: `--force-refresh`.
|
|
11
|
+
|
|
12
|
+
Response shape: `data.report`, with recursively snake_case keys.
|
|
@@ -0,0 +1,13 @@
|
|
|
1
|
+
# experiment report sample-size
|
|
2
|
+
|
|
3
|
+
Query experiment sample-size report.
|
|
4
|
+
|
|
5
|
+
```bash
|
|
6
|
+
ae-cli experiment report sample-size --project-id 1 --exp-id exp_123 --start-time 2026-07-01 --end-time 2026-07-07
|
|
7
|
+
ae-cli experiment report sample-size --project-id 1 --exp-id exp_123 --start-time 2026-07-01 --end-time 2026-07-07 --by-hour true
|
|
8
|
+
```
|
|
9
|
+
|
|
10
|
+
Required flags: `--project-id`, `--exp-id`, `--start-time`, `--end-time`.
|
|
11
|
+
Optional: `--request-id`, `--force-refresh`, `--by-hour`.
|
|
12
|
+
|
|
13
|
+
Response shape: `data.report`, with recursively snake_case keys. Preserve `request_id` for cancellation via `capability run experiment.query.cancel`.
|
|
@@ -0,0 +1,14 @@
|
|
|
1
|
+
# experiment feature get
|
|
2
|
+
|
|
3
|
+
Query Feature detail by key and optional version.
|
|
4
|
+
|
|
5
|
+
```bash
|
|
6
|
+
ae-cli experiment feature get --project-id <id> --feature-key <key> [--version <versionId>]
|
|
7
|
+
```
|
|
8
|
+
|
|
9
|
+
Flags:
|
|
10
|
+
- `--project-id`, `-p`: Project ID.
|
|
11
|
+
- `--feature-key`: Feature key.
|
|
12
|
+
- `--version`: Optional Feature version ID.
|
|
13
|
+
|
|
14
|
+
Response shape: the Feature is in `data.item`, with recursively snake_case keys such as `feature_key` and `target_platform`.
|
|
@@ -0,0 +1,12 @@
|
|
|
1
|
+
# experiment feature list
|
|
2
|
+
|
|
3
|
+
Query Features in a project.
|
|
4
|
+
|
|
5
|
+
```bash
|
|
6
|
+
ae-cli experiment feature list --project-id <id>
|
|
7
|
+
```
|
|
8
|
+
|
|
9
|
+
Flags:
|
|
10
|
+
- `--project-id`, `-p`: Project ID.
|
|
11
|
+
|
|
12
|
+
Response shape: `data.items` contains Features, `data.total` contains the count, and item keys are recursively snake_case.
|
|
@@ -0,0 +1,19 @@
|
|
|
1
|
+
# experiment feature operation-log query
|
|
2
|
+
|
|
3
|
+
Query Feature operation logs for one Feature key.
|
|
4
|
+
|
|
5
|
+
Reuses Hermes `GET /v1/atlas/feature/operationLogs` and returns the generic Atlas
|
|
6
|
+
operation-log payload (`operation_type`, `operation_type_name`, `operator_name`,
|
|
7
|
+
`operation_content`, `operate_time`). This is separate from
|
|
8
|
+
`experiment operation-log query` and does not include experiment-side `changes`.
|
|
9
|
+
|
|
10
|
+
```bash
|
|
11
|
+
ae-cli experiment feature operation-log query --project-id <id> --feature-key <feature-key>
|
|
12
|
+
```
|
|
13
|
+
|
|
14
|
+
Flags:
|
|
15
|
+
- `--project-id`, `-p`: Project ID.
|
|
16
|
+
- `--feature-key`: Feature key.
|
|
17
|
+
|
|
18
|
+
Response shape: `data.items` contains the full log records, `data.total` contains
|
|
19
|
+
the count, and item keys are recursively snake_case.
|
|
@@ -0,0 +1,20 @@
|
|
|
1
|
+
# experiment feature version-list
|
|
2
|
+
|
|
3
|
+
List Feature version history for one Feature key.
|
|
4
|
+
|
|
5
|
+
Reuse the existing Hermes `GET /v1/atlas/feature/versions` API and return the full
|
|
6
|
+
version records, including `version_snapshot`.
|
|
7
|
+
|
|
8
|
+
Use with `experiment feature get --version <featureVersionId>` to inspect one version.
|
|
9
|
+
|
|
10
|
+
```bash
|
|
11
|
+
ae-cli experiment feature version-list --project-id <id> --feature-key <feature-key>
|
|
12
|
+
ae-cli experiment feature get --project-id <id> --feature-key <feature-key> --version <featureVersionId>
|
|
13
|
+
```
|
|
14
|
+
|
|
15
|
+
Flags:
|
|
16
|
+
- `--project-id`, `-p`: Project ID.
|
|
17
|
+
- `--feature-key`: Feature key.
|
|
18
|
+
|
|
19
|
+
Response shape: `data.items` contains version history records, `data.total` contains
|
|
20
|
+
the count, and item keys are recursively snake_case.
|
|
@@ -0,0 +1,18 @@
|
|
|
1
|
+
# experiment metric get
|
|
2
|
+
|
|
3
|
+
Query metric detail.
|
|
4
|
+
|
|
5
|
+
```bash
|
|
6
|
+
ae-cli experiment metric get --project-id <id> --metric-id <metricId>
|
|
7
|
+
```
|
|
8
|
+
|
|
9
|
+
Flags:
|
|
10
|
+
- `--project-id`, `-p`: Project ID.
|
|
11
|
+
- `--metric-id`: Metric ID.
|
|
12
|
+
|
|
13
|
+
Response shape: the metric is in `data.item`, with recursively snake_case keys.
|
|
14
|
+
|
|
15
|
+
Reversible QP is returned as `metric_definition` with
|
|
16
|
+
`metric_definition_status=AVAILABLE`. Missing QP is `NOT_APPLICABLE`; unsupported
|
|
17
|
+
historical QP is `UNAVAILABLE` with `metric_definition_unavailable_reason`.
|
|
18
|
+
Internal `metric_config` and `calc_type` are never returned.
|
|
@@ -0,0 +1,15 @@
|
|
|
1
|
+
# experiment metric list
|
|
2
|
+
|
|
3
|
+
Query metrics in a project.
|
|
4
|
+
|
|
5
|
+
```bash
|
|
6
|
+
ae-cli experiment metric list --project-id <id>
|
|
7
|
+
```
|
|
8
|
+
|
|
9
|
+
Flags:
|
|
10
|
+
- `--project-id`, `-p`: Project ID.
|
|
11
|
+
|
|
12
|
+
Response shape: `data.items` contains metrics, `data.total` contains the count, and item keys are recursively snake_case.
|
|
13
|
+
|
|
14
|
+
Each item uses `metric_definition` and `metric_definition_status`. Internal
|
|
15
|
+
`metric_config` and `calc_type` are never returned.
|
|
@@ -0,0 +1,13 @@
|
|
|
1
|
+
# experiment traffic-layer get
|
|
2
|
+
|
|
3
|
+
Query traffic layer detail.
|
|
4
|
+
|
|
5
|
+
```bash
|
|
6
|
+
ae-cli experiment traffic-layer get --project-id <id> --layer-id <layerId>
|
|
7
|
+
```
|
|
8
|
+
|
|
9
|
+
Flags:
|
|
10
|
+
- `--project-id`, `-p`: Project ID.
|
|
11
|
+
- `--layer-id`: Traffic layer ID.
|
|
12
|
+
|
|
13
|
+
Response shape: the traffic layer is in `data.item`, with recursively snake_case keys.
|
|
@@ -0,0 +1,12 @@
|
|
|
1
|
+
# experiment traffic-layer list
|
|
2
|
+
|
|
3
|
+
Query traffic layers in a project.
|
|
4
|
+
|
|
5
|
+
```bash
|
|
6
|
+
ae-cli experiment traffic-layer list --project-id <id>
|
|
7
|
+
```
|
|
8
|
+
|
|
9
|
+
Flags:
|
|
10
|
+
- `--project-id`, `-p`: Project ID.
|
|
11
|
+
|
|
12
|
+
Response shape: `data.items` contains traffic layers, `data.total` contains the count, and item keys are recursively snake_case.
|
|
@@ -0,0 +1,39 @@
|
|
|
1
|
+
# experiment save build-guide
|
|
2
|
+
|
|
3
|
+
Build a read-only save guide for feature, traffic layer, experiment, or metric saves.
|
|
4
|
+
|
|
5
|
+
```bash
|
|
6
|
+
ae-cli experiment save build-guide --project-id <id> --operation-mode save_experiment
|
|
7
|
+
```
|
|
8
|
+
|
|
9
|
+
Flags:
|
|
10
|
+
- `--project-id`, `-p`: Project ID.
|
|
11
|
+
- `--operation-mode`: `save_feature`, `save_traffic_layer`, `save_experiment`, or `save_metric`.
|
|
12
|
+
|
|
13
|
+
## CRITICAL — do not copy `example_args.req` keys verbatim
|
|
14
|
+
|
|
15
|
+
Guide responses are recursively snake_cased for display. That wrongly rewrites nested DTO
|
|
16
|
+
placeholders inside `data.guide.example_args.req` (for example `expName` → `exp_name`,
|
|
17
|
+
`metricId` → `metric_id`).
|
|
18
|
+
|
|
19
|
+
Those snake_case keys are **not** valid `--req` fields for the final save commands.
|
|
20
|
+
|
|
21
|
+
Authoritative field names:
|
|
22
|
+
|
|
23
|
+
| Mode | Inspect / final save | Nested `--req` style |
|
|
24
|
+
| --- | --- | --- |
|
|
25
|
+
| `save_experiment` | `experiment.experiment.save` | camelCase (`expName`, `trafficLayerId`, …) |
|
|
26
|
+
| `save_metric` | `experiment.metric.save` | camelCase (`metricId`, `metricDefinition`, …) |
|
|
27
|
+
| `save_feature` | `experiment.feature.save` | camelCase (`featureKey`, `targetPlatform`, …) |
|
|
28
|
+
| `save_traffic_layer` | `experiment.traffic-layer.save` | camelCase (`bucketId`, `layerName`, …) |
|
|
29
|
+
|
|
30
|
+
Before composing `--req`:
|
|
31
|
+
|
|
32
|
+
1. Read `data.guide.required_fields` / `mode_rules` for **what** is required (names there may
|
|
33
|
+
already be camelCase).
|
|
34
|
+
2. Treat `example_args.req` as structure-only placeholders; **rename keys to camelCase**.
|
|
35
|
+
3. Prefer `ae-cli capability inspect <final-save-capability-id>` `input_schema.properties.req`
|
|
36
|
+
when unsure.
|
|
37
|
+
4. Then run `experiment save validate` and the final `experiment … save`.
|
|
38
|
+
|
|
39
|
+
Response shape: `data.guide`.
|
|
@@ -0,0 +1,210 @@
|
|
|
1
|
+
# experiment experiment save
|
|
2
|
+
|
|
3
|
+
Progressively create or patch an experiment draft.
|
|
4
|
+
|
|
5
|
+
```bash
|
|
6
|
+
ae-cli experiment experiment save --project-id <id> --req '<json>'
|
|
7
|
+
```
|
|
8
|
+
|
|
9
|
+
Flags:
|
|
10
|
+
- `--project-id`, `-p`: Project ID.
|
|
11
|
+
- `--req`: Experiment save request JSON object.
|
|
12
|
+
|
|
13
|
+
## Contract
|
|
14
|
+
|
|
15
|
+
- Call shape is top-level `projectId` plus a non-empty `req` object. In CLI form, `--project-id` supplies the top-level project ID.
|
|
16
|
+
- Capability input uses outer `project_id`; nested `req` fields remain native camelCase. Hermes overwrites `req.projectId` with the outer project ID.
|
|
17
|
+
- Never call this with `req=null`, `req={}`, or a modify payload containing only `expId`; those are terminal validation failures.
|
|
18
|
+
- On `valid=false`, do not retry the same payload. Fix the reported field(s) or ask the user for missing input.
|
|
19
|
+
- **Do not** paste `save build-guide` / `save validate` `example_args.req` keys into `--req`.
|
|
20
|
+
Those responses snake_case nested placeholders (`exp_name`). Final save requires camelCase
|
|
21
|
+
(`expName`, `expSupposition`, `trafficLayerId`, `featureKeyList`, `metrics[].metricId`, …).
|
|
22
|
+
If unsure, run `ae-cli capability inspect experiment.experiment.save` and copy
|
|
23
|
+
`input_schema.properties.req` field names. `save validate` may return `valid: true` for
|
|
24
|
+
snake_case `req` and still fail here with `unknown field …`.
|
|
25
|
+
|
|
26
|
+
Response shape is `data.result`. Object keys inside the result are recursively snake_case. A create commonly returns the new experiment ID as `data.result`; a patch commonly returns `true`.
|
|
27
|
+
|
|
28
|
+
## Payload Type Contract
|
|
29
|
+
|
|
30
|
+
Nested payload fields must match the server DTO types, not just the visible CLI dry-run body shape.
|
|
31
|
+
|
|
32
|
+
| Field | Correct shape | Incorrect shape |
|
|
33
|
+
| --- | --- | --- |
|
|
34
|
+
| `featureKeyList` | `["payment_color"]` | `[{"featureKey":"payment_color"}]` |
|
|
35
|
+
| `allocation` | integer (e.g. `20`) | `20.5`, `33.33` |
|
|
36
|
+
| `groups[].allocation` | integer; all groups sum to `100` (e.g. `34+33+33`) | `33.33`, or sum `99`/`101` |
|
|
37
|
+
| `groups[].expGroupValue` | `"[\"green\"]"` | `["green"]` |
|
|
38
|
+
| `metrics` | `[{"metricId":"payment","metricRole":"primary"}]` | `["payment"]` |
|
|
39
|
+
|
|
40
|
+
- `featureKeyList` is an array of Feature key strings, not Feature objects. For feature experiments, pass one Feature key string.
|
|
41
|
+
- `allocation` (experiment-level) and `groups[].allocation` are validated at the CLI interface: **integers only, decimals rejected** (for example `33.33` fails before save).
|
|
42
|
+
- `groups[].allocation` rule: each value must be an **integer**, and the **sum of all group allocations must equal `100` exactly** (for example `34 + 33 + 33 = 100`, not `33 + 33 + 33 = 99` and not decimal thirds).
|
|
43
|
+
- `groups[].expGroupValue` is a JSON-encoded string of a string array. Pass a string whose content is a JSON array, not a native JSON array.
|
|
44
|
+
- `metrics` is an array of metric binding objects. Each object contains `metricId` and `metricRole`.
|
|
45
|
+
- Every `metricId` must come from `experiment metric list` for the same project. If save returns `error_code: METRIC_NOT_FOUND`, create the metric with `experiment metric save`, verify it, and then retry the experiment save.
|
|
46
|
+
|
|
47
|
+
## Modes
|
|
48
|
+
|
|
49
|
+
### Create Draft
|
|
50
|
+
|
|
51
|
+
Blank or missing `req.expId` creates a draft.
|
|
52
|
+
|
|
53
|
+
Required:
|
|
54
|
+
- `expName`
|
|
55
|
+
|
|
56
|
+
Optional while drafting:
|
|
57
|
+
- `expType` (blank defaults to `feature`)
|
|
58
|
+
- `groupId`
|
|
59
|
+
- `trafficLayerId`
|
|
60
|
+
- `allocation`
|
|
61
|
+
- `expSalt`
|
|
62
|
+
- `expSupposition`
|
|
63
|
+
- `expDesc`
|
|
64
|
+
- `isExpSharing`
|
|
65
|
+
- `bucketId`
|
|
66
|
+
- `expCycle`
|
|
67
|
+
- `groups`
|
|
68
|
+
- `featureKeyList`
|
|
69
|
+
- `metrics`
|
|
70
|
+
- `targeting`
|
|
71
|
+
|
|
72
|
+
Success returns the created `expId`.
|
|
73
|
+
|
|
74
|
+
```bash
|
|
75
|
+
ae-cli experiment experiment save --project-id 1 --req '{"expName":"Homepage CTA draft"}'
|
|
76
|
+
```
|
|
77
|
+
|
|
78
|
+
### Patch Draft
|
|
79
|
+
|
|
80
|
+
Non-blank `req.expId` patches an existing draft by merging the request with current experiment detail.
|
|
81
|
+
|
|
82
|
+
Required:
|
|
83
|
+
- `expId`
|
|
84
|
+
- At least one changed field besides `expId`
|
|
85
|
+
|
|
86
|
+
Merge behavior:
|
|
87
|
+
- Omitted fields keep their current values.
|
|
88
|
+
- Non-null scalar/object fields replace current values.
|
|
89
|
+
- Non-empty `groups`, `featureKeyList`, `metrics`, and non-null `targeting` replace saved values.
|
|
90
|
+
- Empty lists are not a reliable clear operation. Do not use empty arrays to clear bindings unless the backend contract is confirmed separately.
|
|
91
|
+
|
|
92
|
+
Success returns `true`; continue using the input `expId`.
|
|
93
|
+
|
|
94
|
+
```bash
|
|
95
|
+
ae-cli experiment experiment save --project-id 1 --req '{"expId":"exp_123","allocation":20}'
|
|
96
|
+
```
|
|
97
|
+
|
|
98
|
+
## Field Notes
|
|
99
|
+
|
|
100
|
+
- `isExpSharing=1` uses a shared traffic layer from `experiment traffic-layer list`; pick a `trafficLayerId` whose available idle traffic can cover `allocation`.
|
|
101
|
+
- `isExpSharing=0` uses exclusive traffic with `bucketId`; select the bucket with `experiment bucket list`.
|
|
102
|
+
- `allocation` (experiment-level) is the experiment's share of the traffic layer. It is optional during draft construction, but before readiness it must be greater than `0` and no more than `100`. **Integer only** (for example `20`); decimals such as `20.5` are rejected at the CLI interface.
|
|
103
|
+
- `groups[].allocation` (group-level): **integer only; sum must equal `100` exactly**. Example for 3 groups: `34 + 33 + 33 = 100`. Decimals such as `33.33` are rejected at the CLI interface before save.
|
|
104
|
+
- `expCycle.cycleType` can be `day` or `sample`; when `cycleType=day`, `dayNum` is `1..90`. Blank cycle defaults to day + 30.
|
|
105
|
+
- Before readiness, `groups` must be non-empty, contain exactly one control group (`isControl=1`), group allocations must sum to exactly `100`, and each `expGroupValue` must be a non-empty JSON string array.
|
|
106
|
+
- Feature experiments must bind `featureKeyList` before readiness. `featureKeyList` contains Feature key strings, not Feature objects. The MCP currently supports one `featureKey` for feature experiments.
|
|
107
|
+
- Metrics use `metricRole=primary|secondary|guardrail`; the internal `observation` role is currently unavailable for saves. Before readiness at least one primary metric is required.
|
|
108
|
+
- `targeting` replaces the saved targeting object when provided.
|
|
109
|
+
- Custom audience QP must be supplied as the semantic object
|
|
110
|
+
`targeting.definitionRequest`. Do not submit the internal `targeting.targetConfig`
|
|
111
|
+
JSON or any `Cxx`/`Axxx` code. Resolve event and property names with Analysis
|
|
112
|
+
metadata first.
|
|
113
|
+
- Avoid redundant adjacent `compound` groups with the same `relation`. Hermes
|
|
114
|
+
safely flattens those groups when an Atlas experiment is saved or submitted so
|
|
115
|
+
the generated cluster QP stays within the backend layer limit. Different
|
|
116
|
+
relations are always preserved.
|
|
117
|
+
|
|
118
|
+
Semantic audience example:
|
|
119
|
+
|
|
120
|
+
```json
|
|
121
|
+
{
|
|
122
|
+
"targeting": {
|
|
123
|
+
"definitionRequest": {
|
|
124
|
+
"type": "condition",
|
|
125
|
+
"conditions": {
|
|
126
|
+
"relation": "and",
|
|
127
|
+
"items": [
|
|
128
|
+
{"type": "user", "field": "country", "operator": "eq", "values": ["CN"]},
|
|
129
|
+
{
|
|
130
|
+
"type": "event",
|
|
131
|
+
"event": "purchase",
|
|
132
|
+
"operator": "gte",
|
|
133
|
+
"value": 2,
|
|
134
|
+
"aggregation": "sum",
|
|
135
|
+
"property": "amount",
|
|
136
|
+
"time_range": {"mode": "recent", "unit": "day", "value": 7}
|
|
137
|
+
}
|
|
138
|
+
]
|
|
139
|
+
}
|
|
140
|
+
}
|
|
141
|
+
}
|
|
142
|
+
}
|
|
143
|
+
```
|
|
144
|
+
|
|
145
|
+
## Payload Examples
|
|
146
|
+
|
|
147
|
+
Patch a Feature binding. `featureKeyList` is a string array:
|
|
148
|
+
|
|
149
|
+
```bash
|
|
150
|
+
ae-cli experiment experiment save --project-id 2 --req '{"expId":"e024","featureKeyList":["payment_color"]}'
|
|
151
|
+
```
|
|
152
|
+
|
|
153
|
+
Patch groups. `expGroupValue` is a JSON-encoded string, not a native array. `groups[].allocation` must be integers that sum to exactly `100`:
|
|
154
|
+
|
|
155
|
+
```bash
|
|
156
|
+
ae-cli experiment experiment save --project-id 2 --req '{"expId":"e024","groups":[{"expGroupName":"control","isControl":1,"allocation":50,"expGroupValue":"[\"green\"]"},{"expGroupName":"variant","isControl":0,"allocation":50,"expGroupValue":"[\"red\"]"}]}'
|
|
157
|
+
```
|
|
158
|
+
|
|
159
|
+
Three-group even split example:
|
|
160
|
+
|
|
161
|
+
```bash
|
|
162
|
+
ae-cli experiment experiment save --project-id 2 --req '{"expId":"e024","groups":[{"expGroupName":"control","isControl":1,"allocation":34,"expGroupValue":"[\"A\"]"},{"expGroupName":"variant_b","isControl":0,"allocation":33,"expGroupValue":"[\"B\"]"},{"expGroupName":"variant_c","isControl":0,"allocation":33,"expGroupValue":"[\"C\"]"}]}'
|
|
163
|
+
```
|
|
164
|
+
|
|
165
|
+
Patch metrics. `metrics` is an object array:
|
|
166
|
+
|
|
167
|
+
```bash
|
|
168
|
+
ae-cli experiment experiment save --project-id 2 --req '{"expId":"e024","metrics":[{"metricId":"payment","metricRole":"primary"}]}'
|
|
169
|
+
```
|
|
170
|
+
|
|
171
|
+
## Dry-run Limitation
|
|
172
|
+
|
|
173
|
+
`--dry-run` calls the Capability Gateway preview endpoint. It validates the outer input contract but does not guarantee that every nested DTO business rule will pass execution.
|
|
174
|
+
|
|
175
|
+
If execution reports a DTO type mismatch after a clean dry-run, inspect nested field types first:
|
|
176
|
+
|
|
177
|
+
- `featureKeyList` must be a string array.
|
|
178
|
+
- `groups[].allocation` must be an integer, and all group allocations must sum to exactly `100`.
|
|
179
|
+
- `groups[].expGroupValue` must be a JSON-encoded string.
|
|
180
|
+
- `metrics` must be an object array with `metricId` and `metricRole`.
|
|
181
|
+
|
|
182
|
+
## Verified Related CLI Commands
|
|
183
|
+
|
|
184
|
+
Use only these confirmed ae-cli commands when composing or checking a save payload:
|
|
185
|
+
|
|
186
|
+
- `experiment experiment get`: optional; use only when patching an existing `expId` and you need to inspect saved fields.
|
|
187
|
+
- `experiment bucket list`: find `bucketId`.
|
|
188
|
+
- `experiment traffic-layer list`: find shared `trafficLayerId`.
|
|
189
|
+
- `experiment feature list`: find `featureKey`.
|
|
190
|
+
- `experiment metric list`: find `metricId`.
|
|
191
|
+
- `experiment traffic-layer save`: create a missing traffic layer.
|
|
192
|
+
- `experiment feature save`: create a missing Feature.
|
|
193
|
+
- `experiment metric save`: create a missing metric.
|
|
194
|
+
- `experiment experiment ready-check`: verify completeness before moving to `pending`, `testing`, or `running`.
|
|
195
|
+
|
|
196
|
+
## Completion
|
|
197
|
+
|
|
198
|
+
After a successful save, output the experiment detail link in this form, substituting the returned or input `expId` and actual project ID:
|
|
199
|
+
|
|
200
|
+
```markdown
|
|
201
|
+
[View experiment](/#/atlas/experiment/detail?expId=<expId>¤tProjectId=<projectId>)
|
|
202
|
+
```
|
|
203
|
+
|
|
204
|
+
After every successful save, call:
|
|
205
|
+
|
|
206
|
+
```bash
|
|
207
|
+
ae-cli experiment experiment ready-check --project-id <id> --exp-id <expId>
|
|
208
|
+
```
|
|
209
|
+
|
|
210
|
+
Use readiness failures to decide the next patch. Do not report the experiment save work as complete until readiness has been checked.
|
|
@@ -0,0 +1,13 @@
|
|
|
1
|
+
# experiment feature save
|
|
2
|
+
|
|
3
|
+
Create or update a Feature.
|
|
4
|
+
|
|
5
|
+
```bash
|
|
6
|
+
ae-cli experiment feature save --project-id <id> --req '<json>'
|
|
7
|
+
```
|
|
8
|
+
|
|
9
|
+
Flags:
|
|
10
|
+
- `--project-id`, `-p`: Project ID.
|
|
11
|
+
- `--req`: Feature save request JSON object.
|
|
12
|
+
|
|
13
|
+
Create mode requires `featureKey`, `type`, and `targetPlatform`. Modify mode uses `id` or `update=true`.
|
|
@@ -0,0 +1,91 @@
|
|
|
1
|
+
# experiment metric save
|
|
2
|
+
|
|
3
|
+
Create or update a metric.
|
|
4
|
+
|
|
5
|
+
```bash
|
|
6
|
+
ae-cli experiment metric save --project-id <id> --req '<json>'
|
|
7
|
+
```
|
|
8
|
+
|
|
9
|
+
Flags:
|
|
10
|
+
- `--project-id`, `-p`: Project ID.
|
|
11
|
+
- `--req`: Metric save request JSON object (**camelCase** keys such as `metricId`, `metricDefinition`).
|
|
12
|
+
|
|
13
|
+
Create mode requires metric identifiers and metric configuration. Modify mode uses `update=true`.
|
|
14
|
+
|
|
15
|
+
Do not paste `save build-guide` / `save validate` `example_args.req` keys (`metric_id`, …) into
|
|
16
|
+
`--req`. Those responses snake_case nested placeholders; final save requires camelCase.
|
|
17
|
+
See [`save_build_guide.md`](save_build_guide.md) / [`save_validate.md`](save_validate.md).
|
|
18
|
+
|
|
19
|
+
## Semantic Metric Contract
|
|
20
|
+
|
|
21
|
+
- Supply metric QP as `req.metricDefinition`.
|
|
22
|
+
- Never submit internal `metricConfig`, `calcType`, or an `Axxx` calculation code.
|
|
23
|
+
- Experiment metrics currently accept event definitions only.
|
|
24
|
+
- Event-property comparisons belong in `metricDefinition.filters`. Each item uses
|
|
25
|
+
`field`, `operator`, and `values`; do not place `operator` or `value` directly on
|
|
26
|
+
`metricDefinition`.
|
|
27
|
+
- Supported aggregations:
|
|
28
|
+
- Preset event metrics (omit `property`): `total_count`, `user_count`, `active_days`
|
|
29
|
+
- Property-based (**`property` REQUIRED**): `sum`, `avg`, `max`, `distinct_count`
|
|
30
|
+
- `avg_per_user` (`A105`) is not supported for experiment metric creation.
|
|
31
|
+
|
|
32
|
+
### Property rule (mandatory)
|
|
33
|
+
|
|
34
|
+
When `aggregation` is `sum`, `avg`, `max`, or `distinct_count`:
|
|
35
|
+
|
|
36
|
+
1. Resolve the event's available properties with Analysis metadata first
|
|
37
|
+
(`ae-cli analysis-meta property list` / `get` for that event).
|
|
38
|
+
2. Pass one concrete available property name in `metricDefinition.property`.
|
|
39
|
+
3. Do **not** invent placeholder metrics without a property (for example
|
|
40
|
+
`metric_avg_no_property_*`). Hermes rejects property aggregations with a blank
|
|
41
|
+
or missing `property`.
|
|
42
|
+
|
|
43
|
+
When `aggregation` is `total_count`, `user_count`, or `active_days`, omit `property`.
|
|
44
|
+
|
|
45
|
+
### Preset event metric example (no property)
|
|
46
|
+
|
|
47
|
+
```bash
|
|
48
|
+
ae-cli experiment metric save --project-id 1 --req '{"metricId":"payment_users","metricName":"Payment users","createType":"event","goalDirection":"up","metricDesc":"Users who triggered payment","metricDefinition":{"type":"event","event":"payment","aggregation":"user_count"}}'
|
|
49
|
+
```
|
|
50
|
+
|
|
51
|
+
### Correct example (avg with property)
|
|
52
|
+
|
|
53
|
+
```bash
|
|
54
|
+
ae-cli experiment metric save --project-id 1 --req '{
|
|
55
|
+
"metricId":"startup_style_payment_avg_20260729",
|
|
56
|
+
"metricName":"Startup style payment avg",
|
|
57
|
+
"createType":"event",
|
|
58
|
+
"goalDirection":"up",
|
|
59
|
+
"metricDesc":"Average payment amount",
|
|
60
|
+
"metricDefinition":{
|
|
61
|
+
"type":"event",
|
|
62
|
+
"event":"payment",
|
|
63
|
+
"aggregation":"avg",
|
|
64
|
+
"property":"amount",
|
|
65
|
+
"filters":{
|
|
66
|
+
"relation":"and",
|
|
67
|
+
"items":[{"field":"amount","operator":"gt","values":[10]}]
|
|
68
|
+
}
|
|
69
|
+
}
|
|
70
|
+
}'
|
|
71
|
+
```
|
|
72
|
+
|
|
73
|
+
### Wrong example (avg without property — rejected)
|
|
74
|
+
|
|
75
|
+
```json
|
|
76
|
+
{
|
|
77
|
+
"metricId": "metric_avg_no_property_actual_20260729_144230",
|
|
78
|
+
"metricDefinition": {
|
|
79
|
+
"type": "event",
|
|
80
|
+
"event": "payment",
|
|
81
|
+
"aggregation": "avg"
|
|
82
|
+
}
|
|
83
|
+
}
|
|
84
|
+
```
|
|
85
|
+
|
|
86
|
+
Filters use the semantic `filters` tree with `relation=and|or` and items containing
|
|
87
|
+
`field`, `operator`, and `values`. For the frontend example `gold > 10`, use:
|
|
88
|
+
|
|
89
|
+
```json
|
|
90
|
+
{"filters":{"relation":"and","items":[{"field":"gold","operator":"gt","values":[10]}]}}
|
|
91
|
+
```
|
|
@@ -0,0 +1,16 @@
|
|
|
1
|
+
# capability run experiment.experiment.save-submit
|
|
2
|
+
|
|
3
|
+
Create or update a complete experiment and submit it in one call.
|
|
4
|
+
|
|
5
|
+
```bash
|
|
6
|
+
ae-cli capability run experiment.experiment.save-submit --input '{"project_id":1,"req":{...}}'
|
|
7
|
+
```
|
|
8
|
+
|
|
9
|
+
Input fields: `project_id` and `req`. The outer input is snake_case; fields inside `req` keep the native camelCase DTO shape.
|
|
10
|
+
|
|
11
|
+
Custom audiences use semantic `req.targeting.definitionRequest`. Never pass the
|
|
12
|
+
internal `targetConfig` field or hand-write `Cxx`/`Axxx` codes.
|
|
13
|
+
|
|
14
|
+
Response shape: `data.result`, with object keys recursively converted to snake_case.
|
|
15
|
+
|
|
16
|
+
Prefer `experiment experiment save` for progressive draft construction.
|
|
@@ -0,0 +1,13 @@
|
|
|
1
|
+
# experiment traffic-layer save
|
|
2
|
+
|
|
3
|
+
Create or update a traffic layer.
|
|
4
|
+
|
|
5
|
+
```bash
|
|
6
|
+
ae-cli experiment traffic-layer save --project-id <id> --req '<json>'
|
|
7
|
+
```
|
|
8
|
+
|
|
9
|
+
Flags:
|
|
10
|
+
- `--project-id`, `-p`: Project ID.
|
|
11
|
+
- `--req`: Traffic layer save request JSON object.
|
|
12
|
+
|
|
13
|
+
Create mode requires `bucketId`, `layerName`, and `layerType` in the request. Modify mode requires `layerId`.
|
|
@@ -0,0 +1,33 @@
|
|
|
1
|
+
# experiment save validate
|
|
2
|
+
|
|
3
|
+
Dry-run helper for save payloads. Does **not** persist data.
|
|
4
|
+
|
|
5
|
+
```bash
|
|
6
|
+
ae-cli experiment save validate --project-id <id> --operation-mode save_experiment --req '{"expName":"Demo"}'
|
|
7
|
+
```
|
|
8
|
+
|
|
9
|
+
Flags:
|
|
10
|
+
- `--project-id`, `-p`: Project ID.
|
|
11
|
+
- `--operation-mode`: `save_feature`, `save_traffic_layer`, `save_experiment`, or `save_metric`.
|
|
12
|
+
- `--req`: Candidate save request object. **Must use native camelCase DTO keys.**
|
|
13
|
+
|
|
14
|
+
## CRITICAL — `valid: true` is not a save schema pass
|
|
15
|
+
|
|
16
|
+
`experiment.save.validate` checks operation-mode / outer helper contract. It does **not**
|
|
17
|
+
fully enforce the final save capability `input_schema`.
|
|
18
|
+
|
|
19
|
+
Consequences:
|
|
20
|
+
|
|
21
|
+
- `--req '{"exp_name":"Demo"}'` can return `valid: true`.
|
|
22
|
+
- The same payload fails on `experiment experiment save` with `unknown field exp_name`.
|
|
23
|
+
|
|
24
|
+
Rules:
|
|
25
|
+
|
|
26
|
+
1. Never submit snake_case DTO keys in `--req` (`exp_name`, `metric_id`, `feature_key`, …).
|
|
27
|
+
2. Do not trust `data.validation.example_args.req` key casing; rebuild with camelCase.
|
|
28
|
+
3. After `valid: true`, still use camelCase and prefer
|
|
29
|
+
`ae-cli … save --dry-run` / `capability inspect` before a real write.
|
|
30
|
+
4. On final save `INVALID_CAPABILITY_INPUT` / `unknown field`, fix casing — do not widen
|
|
31
|
+
the payload with more snake_case keys.
|
|
32
|
+
|
|
33
|
+
Response shape: `data.validation`.
|