@roarkanalytics/cli 0.37.1 → 0.39.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (11) hide show
  1. package/CHANGELOG.md +14 -0
  2. package/README.md +30 -30
  3. package/commands.js +14 -239
  4. package/completions.js +4 -4
  5. package/man/roark-call-create.1 +1 -1
  6. package/man/roark-simulation-plan-create.1 +8 -73
  7. package/man/roark-simulation-plan-update.1 +2 -2
  8. package/npm-shrinkwrap.json +2 -2
  9. package/package.json +1 -1
  10. package/version.d.ts +1 -1
  11. package/version.js +1 -1
package/CHANGELOG.md CHANGED
@@ -1,5 +1,19 @@
1
1
  # Changelog
2
2
 
3
+ ## [0.39.0](https://github.com/roarkhq/cli-roark-analytics/compare/v0.38.0...v0.39.0) (2026-10-08)
4
+
5
+
6
+ ### Features
7
+
8
+ * **cli:** cli update ([#97](https://github.com/roarkhq/cli-roark-analytics/issues/97)) ([456dab8](https://github.com/roarkhq/cli-roark-analytics/commit/456dab84a9afc4cfd1a0c179bbb9b4f64cf66fc5))
9
+
10
+ ## [0.38.0](https://github.com/roarkhq/cli-roark-analytics/compare/v0.37.1...v0.38.0) (2026-10-07)
11
+
12
+
13
+ ### Features
14
+
15
+ * **cli:** cli update ([#95](https://github.com/roarkhq/cli-roark-analytics/issues/95)) ([f89088e](https://github.com/roarkhq/cli-roark-analytics/commit/f89088eafbadb8985cf42f9feb7cff1c5b76bc93))
16
+
3
17
  ## [0.37.1](https://github.com/roarkhq/cli-roark-analytics/compare/v0.37.0...v0.37.1) (2026-10-06)
4
18
 
5
19
 
package/README.md CHANGED
@@ -246,36 +246,36 @@ roark completion fish | source
246
246
 
247
247
  ### simulation
248
248
 
249
- | Command | Description |
250
- | ------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------- |
251
- | `roark simulation environment create --name <value> --background-noise <value>` | Create an environment |
252
- | `roark simulation environment delete <environment-id>` | Delete an environment |
253
- | `roark simulation environment get <environment-id>` | Get environment by ID |
254
- | `roark simulation environment list` | List environments |
255
- | `roark simulation environment update <environment-id>` | Update an environment |
256
- | `roark simulation job get <job-id>` | Get simulation by ID |
257
- | `roark simulation job lookup --roark-phone-number <value>` | Lookup by phone number |
258
- | `roark simulation job tool-mock list <job-id>` | List mocked tool invocations |
259
- | `roark simulation persona create --name <value> --language <value> --accent <value> --gender <value>` | Create a new persona |
260
- | `roark simulation persona get <persona-id>` | Get persona by ID |
261
- | `roark simulation persona list` | List personas |
262
- | `roark simulation persona update <persona-id>` | Update a persona |
263
- | `roark simulation plan create --name <value> --direction <value> --max-simulation-duration-seconds <value> --agent-endpoints <value>` | Create a run plan |
264
- | `roark simulation plan delete <plan-id>` | Delete a run plan |
265
- | `roark simulation plan get <plan-id>` | Get run plan by ID |
266
- | `roark simulation plan job cancel <job-id>` | Cancel a simulation plan job |
267
- | `roark simulation plan job get <job-id>` | Get simulation plan job |
268
- | `roark simulation plan job list` | List simulation plan jobs |
269
- | `roark simulation plan job start <plan-id>` | Run a simulation plan |
270
- | `roark simulation plan list` | List run plans |
271
- | `roark simulation plan update <plan-id>` | Update a run plan |
272
- | `roark simulation run --data '{ ... }'` | Run a simulation |
273
- | `roark simulation template list` | List simulation templates |
274
- | `roark simulation tool mock --simulation-job-id <value> --tool-name <value>` | Mock a tool call during a test call |
275
- | `roark simulation tool-fixture create --tool-name <value>` | Set a tool fixture |
276
- | `roark simulation tool-fixture delete <fixture-id>` | Delete a tool fixture |
277
- | `roark simulation tool-fixture list` | List tool fixtures |
278
- | `roark simulation tool-fixture update <fixture-id>` | Update a tool fixture |
249
+ | Command | Description |
250
+ | ----------------------------------------------------------------------------------------------------- | ----------------------------------- |
251
+ | `roark simulation environment create --name <value> --background-noise <value>` | Create an environment |
252
+ | `roark simulation environment delete <environment-id>` | Delete an environment |
253
+ | `roark simulation environment get <environment-id>` | Get environment by ID |
254
+ | `roark simulation environment list` | List environments |
255
+ | `roark simulation environment update <environment-id>` | Update an environment |
256
+ | `roark simulation job get <job-id>` | Get simulation by ID |
257
+ | `roark simulation job lookup --roark-phone-number <value>` | Lookup by phone number |
258
+ | `roark simulation job tool-mock list <job-id>` | List mocked tool invocations |
259
+ | `roark simulation persona create --name <value> --language <value> --accent <value> --gender <value>` | Create a new persona |
260
+ | `roark simulation persona get <persona-id>` | Get persona by ID |
261
+ | `roark simulation persona list` | List personas |
262
+ | `roark simulation persona update <persona-id>` | Update a persona |
263
+ | `roark simulation plan create --data '{ ... }'` | Create a run plan |
264
+ | `roark simulation plan delete <plan-id>` | Delete a run plan |
265
+ | `roark simulation plan get <plan-id>` | Get run plan by ID |
266
+ | `roark simulation plan job cancel <job-id>` | Cancel a simulation plan job |
267
+ | `roark simulation plan job get <job-id>` | Get simulation plan job |
268
+ | `roark simulation plan job list` | List simulation plan jobs |
269
+ | `roark simulation plan job start <plan-id>` | Run a simulation plan |
270
+ | `roark simulation plan list` | List run plans |
271
+ | `roark simulation plan update <plan-id>` | Update a run plan |
272
+ | `roark simulation run --data '{ ... }'` | Run a simulation |
273
+ | `roark simulation template list` | List simulation templates |
274
+ | `roark simulation tool mock --simulation-job-id <value> --tool-name <value>` | Mock a tool call during a test call |
275
+ | `roark simulation tool-fixture create --tool-name <value>` | Set a tool fixture |
276
+ | `roark simulation tool-fixture delete <fixture-id>` | Delete a tool fixture |
277
+ | `roark simulation tool-fixture list` | List tool fixtures |
278
+ | `roark simulation tool-fixture update <fixture-id>` | Update a tool fixture |
279
279
 
280
280
  ### webhook
281
281
 
package/commands.js CHANGED
@@ -1397,6 +1397,7 @@ exports.COMMANDS = [
1397
1397
  'AGENT_DID_NOT_SPEAK',
1398
1398
  'AGENT_STOPPED_SPEAKING',
1399
1399
  'AGENT_ENDED_CALL',
1400
+ 'AGENT_SAID_END_CALL_PHRASE',
1400
1401
  'AGENT_TRANSFERRED_CALL',
1401
1402
  'AGENT_BUSY',
1402
1403
  'AGENT_ERROR',
@@ -4800,248 +4801,22 @@ exports.COMMANDS = [
4800
4801
  httpMethod: 'post',
4801
4802
  httpPath: '/v1/simulation/plan',
4802
4803
  summary: 'Create a run plan',
4803
- description: 'Creates a new simulation run plan. To run a simulation, use POST /v1/simulation/run instead: it starts a run from a plan or from an inline configuration, and takes runtime variables. Create a plan here when you want a reusable, named one to run later.',
4804
+ description: 'Creates a new simulation run plan. To run a simulation, use POST /v1/simulation/run instead: it starts a run from a plan or from an inline configuration, and takes runtime variables. Create a plan here when you want a reusable, named one to run later. Send `template` instead of a full configuration to save one of the built-in templates as a plan. It takes the same fields as the template variant of POST /v1/simulation/run, builds the same plan, and never starts it. To compare one property, attach the flow once and send `comparisonProperty` with the `comparisonValues` to run: the plan attaches the flow once per value.',
4804
4805
  positionals: [],
4805
- flags: [
4806
- {
4807
- name: 'name',
4808
- path: ['name'],
4809
- location: 'body',
4810
- required: true,
4811
- description: 'Name of the run plan',
4812
- valueKind: 'string',
4813
- repeatable: false,
4814
- },
4815
- {
4816
- name: 'description',
4817
- path: ['description'],
4818
- location: 'body',
4819
- required: false,
4820
- description: 'Description of the run plan',
4821
- valueKind: 'string',
4822
- repeatable: false,
4823
- },
4824
- {
4825
- name: 'direction',
4826
- path: ['direction'],
4827
- location: 'body',
4828
- required: true,
4829
- description: 'Direction of the simulation (INBOUND or OUTBOUND)',
4830
- valueKind: 'string',
4831
- enumValues: ['INBOUND', 'OUTBOUND'],
4832
- repeatable: false,
4833
- },
4834
- {
4835
- name: 'iteration-count',
4836
- path: ['iterationCount'],
4837
- location: 'body',
4838
- required: false,
4839
- description: 'Number of iterations to run for each test case (1-10000)',
4840
- valueKind: 'integer',
4841
- repeatable: false,
4842
- },
4843
- {
4844
- name: 'max-concurrent-jobs',
4845
- path: ['maxConcurrentJobs'],
4846
- location: 'body',
4847
- required: false,
4848
- description: 'Maximum number of concurrent simulation jobs',
4849
- valueKind: 'integer',
4850
- repeatable: false,
4851
- },
4852
- {
4853
- name: 'max-simulation-duration-seconds',
4854
- path: ['maxSimulationDurationSeconds'],
4855
- location: 'body',
4856
- required: true,
4857
- description: 'Maximum duration in seconds for each simulation',
4858
- valueKind: 'integer',
4859
- repeatable: false,
4860
- },
4861
- {
4862
- name: 'silence-timeout-seconds',
4863
- path: ['silenceTimeoutSeconds'],
4864
- location: 'body',
4865
- required: false,
4866
- description: 'Timeout in seconds for silence detection',
4867
- valueKind: 'integer',
4868
- repeatable: false,
4869
- },
4870
- {
4871
- name: 'max-no-response-retries',
4872
- path: ['maxNoResponseRetries'],
4873
- location: 'body',
4874
- required: false,
4875
- description: 'How many more times to run a test case when the agent under test never responds: it never speaks on a call or never replies in a chat (0-10). 0 turns retries off. Failed checks and failures on Roark’s side are never retried. Each retry is a separate attempt, billed like any other, so a plan retrying N times can place up to N + 1 calls per test case. Every silent attempt stays on the run with its own call; the run settles once each test case has a final attempt, and the agent never spoke verdict is judged on each test case’s last attempt.',
4876
- valueKind: 'integer',
4877
- repeatable: false,
4878
- },
4879
- {
4880
- name: 'no-response-retry-backoff-seconds',
4881
- path: ['noResponseRetryBackoffSeconds'],
4882
- location: 'body',
4883
- required: false,
4884
- description: 'Seconds a retry waits before it dials (30-600). Only used when `maxNoResponseRetries` is above 0.',
4885
- valueKind: 'integer',
4886
- repeatable: false,
4887
- },
4888
- {
4889
- name: 'end-call-phrases',
4890
- path: ['endCallPhrases'],
4891
- location: 'body',
4892
- required: false,
4893
- description: 'Phrases that trigger end of call. Empty array disables the feature.',
4894
- valueKind: 'array',
4895
- repeatable: true,
4896
- },
4897
- {
4898
- name: 'end-call-reasons',
4899
- path: ['endCallReasons'],
4900
- location: 'body',
4901
- required: false,
4902
- description: 'Semantic conditions that trigger end of call. The LLM evaluates the conversation against these conditions. Empty array disables the feature.',
4903
- valueKind: 'array',
4904
- repeatable: true,
4905
- },
4906
- {
4907
- name: 'execution-mode',
4908
- path: ['executionMode'],
4909
- location: 'body',
4910
- required: false,
4911
- description: 'Execution mode (PARALLEL or SEQUENTIAL)',
4912
- valueKind: 'string',
4913
- enumValues: ['PARALLEL', 'SEQUENTIAL_SAME_RUN_PLAN', 'SEQUENTIAL_PROJECT'],
4914
- repeatable: false,
4915
- },
4916
- {
4917
- name: 'scenarios',
4918
- path: ['scenarios'],
4919
- location: 'body',
4920
- required: false,
4921
- description: 'Deprecated: use `flows` instead. Scenarios to include in this run plan. The same scenario ID can appear multiple times with different variables.',
4922
- valueKind: 'array',
4923
- repeatable: false,
4924
- },
4925
- {
4926
- name: 'flows',
4927
- path: ['flows'],
4928
- location: 'body',
4929
- required: false,
4930
- description: 'Customer flows to include in this run plan. The same flow can appear more than once with a different persona override, different variables, or different `overrides`: attaching it once per value of one property is how you compare that property without a template.',
4931
- valueKind: 'array',
4932
- repeatable: false,
4933
- },
4934
- {
4935
- name: 'personas',
4936
- path: ['personas'],
4937
- location: 'body',
4938
- required: false,
4939
- description: 'Personas to include in this run plan. Required with `scenarios`; ignored with `flows`, where each variant carries its own persona.',
4940
- valueKind: 'array',
4941
- repeatable: false,
4942
- },
4943
- {
4944
- name: 'agent-endpoints',
4945
- path: ['agentEndpoints'],
4946
- location: 'body',
4947
- required: true,
4948
- description: 'Agent endpoints to include in this run plan',
4949
- valueKind: 'array',
4950
- repeatable: false,
4951
- },
4952
- {
4953
- name: 'metrics',
4954
- path: ['metrics'],
4955
- location: 'body',
4956
- required: false,
4957
- description: 'Metric definitions to include in this run plan. Reference each by `id` (UUID) or `slug`. Optional when the attached `flows` carry the grading: metrics a flow declares itself (with `includeFlowMetrics`), or the Agent Expectations and Keypad Entry metrics a run adds for flows with expectations or expected keypad entries (with `includeAutomaticMetrics`). A plan with nothing to grade is rejected with a 400.',
4958
- valueKind: 'array',
4959
- repeatable: false,
4960
- },
4961
- {
4962
- name: 'include-flow-metrics',
4963
- path: ['includeFlowMetrics'],
4964
- location: 'body',
4965
- required: false,
4966
- description: "Also collect each attached flow's own metrics, on top of the `metrics` named here. Default true, which is what you want when you brought your own flows and their graders. Set false for a run whose metric list is meant to be exhaustive: a template like Load Testing or Voicemail deliberately grades a narrow set, and inheriting every flow metric on top multiplies analysis cost across the volume without adding signal. GET /v1/simulation/template returns the value each template expects.",
4967
- valueKind: 'boolean',
4968
- repeatable: false,
4969
- },
4970
- {
4971
- name: 'include-automatic-metrics',
4972
- path: ['includeAutomaticMetrics'],
4973
- location: 'body',
4974
- required: false,
4975
- description: 'Let the run add metrics by itself off the attached flows, on top of the `metrics` named here. Two attach this way today: Agent Expectations wherever an attached flow has agent expectations written on it, and Keypad Entry wherever one has steps where the agent is expected to press keys. Both grade something authored on the flow that nothing else measures, which is why it is on by default. Set false when the `metrics` list is meant to be exhaustive: a plan testing only whether the caller can complete the flow may not want the agent graded on its expectations as well. False also pins the plan against any automatic metric Roark adds later.',
4976
- valueKind: 'boolean',
4977
- repeatable: false,
4978
- },
4979
- {
4980
- name: 'comparison-property',
4981
- path: ['comparisonProperty'],
4982
- location: 'body',
4983
- required: false,
4984
- description: 'The property this run plan investigates: the one thing its arms differ by. Set it and the run report compares the arms on that property, so a run answers "what did background noise cost" rather than just "what did each arm score". Every value is a field already recorded on each call, so the report can label an arm `CRYING_BABY` rather than repeating a flow variant\'s title. Omit it and the report still compares when it can: it detects which property varies across the arms. Setting it is what tells the written summary what you were trying to find out, which detection cannot infer.',
4985
- valueKind: 'string',
4986
- enumValues: [
4987
- 'ACCENT',
4988
- 'AGE',
4989
- 'BACKGROUND_NOISE',
4990
- 'BACKGROUND_NOISE_VOLUME',
4991
- 'BASE_EMOTION',
4992
- 'CONFIRMATION_STYLE',
4993
- 'GENDER',
4994
- 'INTENT_CLARITY',
4995
- 'LANGUAGE',
4996
- 'INTERRUPTION',
4997
- 'MEMORY_RELIABILITY',
4998
- 'RESPONSE_TIMING',
4999
- 'SPEECH_CLARITY',
5000
- 'SPEECH_PACE',
5001
- ],
5002
- repeatable: false,
5003
- },
5004
- {
5005
- name: 'comparison-baseline',
5006
- path: ['comparisonBaseline'],
5007
- location: 'body',
5008
- required: false,
5009
- description: 'The reference value of `comparisonProperty`, for example `NONE` for `BACKGROUND_NOISE` or `NORMAL` for `SPEECH_PACE`: shown first in the results. Must be a value that property can take. Whether a value did significantly worse does not depend on it: that is decided against every other value combined (see `sweepAttribution`). Stored rather than assumed, so the report can say "compared against US accent" instead of implying Roark decided which value is normal. Most properties have an obvious baseline and the dashboard prefills it; `GENDER` has none, so choose the one you are testing against.',
5010
- valueKind: 'string',
5011
- repeatable: false,
5012
- },
5013
- {
5014
- name: 'comparison-values',
5015
- path: ['comparisonValues'],
5016
- location: 'body',
5017
- required: false,
5018
- description: 'Which values of `comparisonProperty` to run. This is what the plan costs: the flow is attached once per value, so ten values is ten times the calls of one. Omit it to run every value the property has, which for `ACCENT` is more than twenty. Send a subset to narrow the sweep, for example three accents you actually serve. A `comparisonBaseline` outside this set is rejected, because it would anchor every difference to an arm the run never made. Not stored as a field: the arms are the values. Reading the plan back returns them as its flow attachments.',
5019
- valueKind: 'array',
5020
- repeatable: true,
5021
- },
4806
+ flags: [],
4807
+ hasParams: true,
4808
+ paramsAllOptional: false,
4809
+ bodyOpaque: true,
4810
+ bodyVariants: [
5022
4811
  {
5023
- name: 'enrich-with-live-conversation',
5024
- path: ['enrichWithLiveConversation'],
5025
- location: 'body',
5026
- required: false,
5027
- description: "Merge the customer's own recording of the real call into each simulation, so metrics can be scored against the live leg as well as the simulated one. This is the API equivalent of the dashboard's live-enrichment toggle. With this on, the run provisions a phone number and holds each call open for up to 15 minutes waiting for a matching call to be posted to POST /v1/call. A call matches on the provisioned number (`roarkPhoneNumber` on the job) with a start time inside the simulation window. If nothing arrives, the simulation still completes and any `LIVE`-sourced metric produces no value. Required by any metric whose `requiresLiveConversation` is true: without it that metric is silently skipped.",
5028
- valueKind: 'boolean',
5029
- repeatable: false,
4812
+ name: 'CreateRunPlanFromConfig',
4813
+ required: ['name', 'direction', 'maxSimulationDurationSeconds', 'agentEndpoints'],
5030
4814
  },
5031
4815
  {
5032
- name: 'auto-run',
5033
- path: ['autoRun'],
5034
- location: 'body',
5035
- required: false,
5036
- description: 'Deprecated: use POST /v1/simulation/run, which starts a run and accepts runtime `variables` as well. This flag runs the plan with only the values pinned on it.',
5037
- valueKind: 'boolean',
5038
- repeatable: false,
4816
+ name: 'CreateRunPlanFromTemplate',
4817
+ required: ['template', 'agentEndpoints', 'direction'],
5039
4818
  },
5040
4819
  ],
5041
- hasParams: true,
5042
- paramsAllOptional: false,
5043
- bodyOpaque: false,
5044
- bodyVariants: [],
5045
4820
  acceptsBody: true,
5046
4821
  requiresAuth: true,
5047
4822
  },
@@ -5522,7 +5297,7 @@ exports.COMMANDS = [
5522
5297
  path: ['comparisonProperty'],
5523
5298
  location: 'body',
5524
5299
  required: false,
5525
- description: 'The property this plan investigates. Send `null` to clear the comparison; omit the field to leave it unchanged. See `POST /v1/simulation/plan`. The pair moves together. Sending `comparisonProperty` also sets `comparisonBaseline` to whatever this request carries, or to `null` if it carries none, because a baseline is a value of one specific property and keeping the old one would store a pair that is not valid.',
5300
+ description: "The property this plan investigates. Send `null` to clear the comparison; omit the field to leave it unchanged. See `POST /v1/simulation/plan`. The pair moves together. Sending `comparisonProperty` without `comparisonBaseline` keeps the stored baseline when the property is unchanged and the baseline is still one of the values being run. Otherwise it becomes the new property's norm, or `null` when that norm is not being run either, because a baseline is a value of one specific property.",
5526
5301
  valueKind: 'string',
5527
5302
  enumValues: [
5528
5303
  'ACCENT',
@@ -5556,9 +5331,9 @@ exports.COMMANDS = [
5556
5331
  path: ['comparisonValues'],
5557
5332
  location: 'body',
5558
5333
  required: false,
5559
- description: 'Which values of `comparisonProperty` to run. See `POST /v1/simulation/plan`. Omitting it keeps the arms the plan already has, so an edit that only renames the plan never widens a sweep you deliberately narrowed, and never multiplies what it costs.',
5334
+ description: 'The arms to run. See `POST /v1/simulation/plan`. Omitting it keeps the arms the plan already has, pins included, so an edit that only renames the plan never widens a sweep you deliberately narrowed, and never multiplies what it costs. Send it with `comparisonProperty` and `flows`, which the arms are rebuilt from.',
5560
5335
  valueKind: 'array',
5561
- repeatable: true,
5336
+ repeatable: false,
5562
5337
  },
5563
5338
  ],
5564
5339
  hasParams: true,