orizu 0.3.7 → 0.4.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -12,8 +12,9 @@
12
12
  1. Verify auth first: `orizu whoami`.
13
13
  2. Prefer explicit flags over prompts.
14
14
  3. Use interactive fallback only in TTY sessions.
15
- 4. Validate command output before proceeding.
16
- 5. On failure, fix flags/identifiers and rerun.
15
+ 4. Prefer `--json` for machine-readable output — every command supports it (prefix `orizu --json <cmd>` or trailing flag). Long-running commands emit the JSON summary as the final stdout line.
16
+ 5. Validate command output before proceeding.
17
+ 6. On failure, fix flags/identifiers and rerun.
17
18
 
18
19
  ## Command Matrix
19
20
 
@@ -33,6 +34,25 @@ Behavior:
33
34
  - PATs can also be revoked from the Personal Tokens page in Orizu.
34
35
  - PAT authorization follows the owning user's current team/project roles, so demotion or removal takes effect without rotating the token.
35
36
 
37
+ ### Agent Setup
38
+
39
+ ```bash
40
+ orizu capabilities --json
41
+ orizu skills path [--skill-md] [--json]
42
+ orizu skills status [--json]
43
+ orizu skills update [--dry-run] [--json]
44
+ orizu install-skill [--agent <claude|codex>]... [--scope global|project] [--mode auto|link|copy] [--yes] [--dry-run] [--json]
45
+ orizu setup [--agent <claude|codex>]... [--skip-login] [--no-install] [--workspace|--no-workspace] [--no-handoff] [--yes] [--dry-run] [--no-input] [--json]
46
+ orizu setup prompt [--json]
47
+ ```
48
+
49
+ Behavior:
50
+ - `capabilities --json` returns the full machine-readable command manifest, including `globalOptions`.
51
+ - `skills path --json` returns where this skill's source lives (`root`, `skillMd`, `source`, `cliVersion`, `skillHash`) so you can read it without installing, and verify an installed copy matches the CLI.
52
+ - `skills status` reports each install target as missing/current/stale/broken-link; `skills update` refreshes stale copies and repairs broken symlinks.
53
+ - `install-skill` selects targets by agent name; `--scope global` (default) installs user-level, `--scope project` into the repo. Project-scope installs always copy; global installs symlink when the CLI path is stable. Advanced target IDs remain available via `--target`.
54
+ - `orizu setup` is the guided onboarding flow (login → agent integration → `.orizu/` workspace → handoff prompt). `orizu setup prompt` prints the repo-adoption prompt for handing to a coding agent.
55
+
36
56
  ### Teams
37
57
 
38
58
  ```bash
@@ -166,23 +186,28 @@ orizu tasks create \
166
186
  Behavior:
167
187
  - task creation creates a draft by default and returns a task URL to test manually before assigning
168
188
  - use `--publish --assignees <userIdOrEmail1,userIdOrEmail2>` only when you intentionally want to create and ship immediately
189
+ - use `--assignment-file <path>` instead of `--assignees` when specific rows should go to specific labellers
169
190
  - after manually approving a draft, run `orizu tasks publish --task <taskId> --assignees <userId1,userId2>`
191
+ - or run `orizu tasks publish --task <taskId> --assignment-file <path>` to publish the exact row map
170
192
  - task creation resolves and stores the app's pinned current `version_id` at create time
171
193
  - downstream consumers (exports, judges, optimization) should trust the task's pinned `version_id`, not the app's current pointer
172
194
  - dataset compatibility is validated against that pinned app version before the task is inserted
173
195
  - malformed JSON and mixed-type assignee arrays fail with deterministic `400` responses
174
196
  - assignment fanout enforces unique `(assignee, row)` pairs; `--labels-per-item` cannot exceed the number of unique assignees, and the backend shortfalls instead of duplicating
197
+ - explicit assignment manifests are JSONL with `rowId` plus `assignee` or `assignees`; row IDs are canonical dataset row `id` values, assignees may be emails or user IDs, and V1 publish requires whole-dataset uniform coverage
175
198
 
176
199
  Publish:
177
200
 
178
201
  ```bash
179
202
  orizu tasks publish --task <taskId> --assignees <userId1,userId2>
203
+ orizu tasks publish --task <taskId> --assignment-file ./assignments.jsonl
180
204
  ```
181
205
 
182
206
  Assign:
183
207
 
184
208
  ```bash
185
209
  orizu tasks assign --task <taskId> --assignees <userId1,userId2>
210
+ orizu tasks assign --task <taskId> --assignment-file ./assignments.jsonl --replace-existing
186
211
  ```
187
212
 
188
213
  Status:
@@ -232,13 +257,14 @@ For prompts, judges, runners, run submission, optimizer artifacts, live event lo
232
257
  Optimization trace commands:
233
258
 
234
259
  ```bash
235
- orizu optimizations run-gepa ... [--num-threads auto|N] [--log-dir logs] [--no-local-log]
260
+ orizu optimizations run-gepa ... [--num-threads auto|N] [--reflection-retry-attempts 3] [--reflection-http-timeout-seconds 180] [--log-dir logs] [--no-local-log]
236
261
  orizu optimizations export <optimization-run-id> --out ./optimization.json
237
262
  orizu optimizations export <optimization-run-id> --json
238
263
  ```
239
264
 
240
265
  Behavior:
241
- - `run-gepa` defaults `--num-threads` to `auto`, resolving a conservative row-evaluation parallelism cap from mini-batch size, validation-set size, CPU count, memory estimate, file-descriptor limit, and an 8-thread hard cap.
266
+ - `run-gepa` defaults `--num-threads` to `auto`, resolving a row-evaluation parallelism cap from mini-batch size, validation-set size, 2x CPU count, memory estimate, file-descriptor limit, and a 64-thread default ceiling. Set `ORIZU_GEPA_AUTO_THREADS_MAX` or use `--num-threads <n>` only when the runner/provider capacity is known.
267
+ - `run-gepa` retries transient reflection-provider failures by default. If retries are exhausted, it logs `reflection_failed`, counts that reflection proposal against budget, and continues with the next iteration.
242
268
  - `run-gepa` writes a complete local trace under `logs/<optimization_run_id>` by default.
243
269
  - The local trace is the best artifact for coding-agent analysis because it includes full rows, outputs, scores, feedback, scorer responses, reflection prompts, reflection responses, candidate text, and `result.json`.
244
270
  - `optimizations export` writes a portable JSON artifact from server data when the local log is unavailable or the run happened elsewhere.
@@ -280,6 +306,8 @@ orizu tasks create \
280
306
 
281
307
  # Open the returned task URL and test the draft manually before assigning.
282
308
  orizu tasks publish --task <taskId> --assignees <userId1,userId2>
309
+ # Or publish a deterministic row map:
310
+ orizu tasks publish --task <taskId> --assignment-file ./assignments.jsonl
283
311
 
284
312
  orizu tasks status --task <taskId>
285
313
  orizu tasks export --task <taskId> --format csv --out ./support-round1.csv
@@ -300,7 +328,8 @@ Use these shortcuts only in TTY environments where prompts can run.
300
328
 
301
329
  - `tasks create` creates a draft by default, does not require `--assignees`, and pins the app's current version when the task is created.
302
330
  - `tasks create --publish --assignees <...>` intentionally creates and ships immediately.
303
- - `tasks assign` and `tasks publish` expect user IDs, not emails.
331
+ - `tasks create|publish|assign --assignment-file <path>` is mutually exclusive with `--assignees` and accepts emails or user IDs in the JSONL manifest.
332
+ - `tasks assign --assignees` and `tasks publish --assignees` expect user IDs, not emails.
304
333
  - Assignment queue reads are assignee-self-only; use task status/export as the operator summary path.
305
334
  - Assignment completion payloads are validated against the pinned app-version `output_json_schema`.
306
335
  - `datasets delete-rows` requires `--row-ids`.
@@ -130,9 +130,9 @@ Set an explicit expectation before the run: for example, "if best kappa does not
130
130
 
131
131
  ### Budget Versus Iteration Cap
132
132
 
133
- Default budget presets are DSPy-style metric-call budgets. `light`, `medium`, and `heavy` map to DSPy's `auto="light"`, `auto="medium"`, and `auto="heavy"` scales; `auto` is treated as the balanced `medium` preset. The final metric-call limit scales with validation-set size, rather than using fixed tiny constants. `--max-metric-calls` overrides this directly, while `--max-full-evals` is converted to a metric-call budget over `train + validation` rows.
133
+ Default budget presets are DSPy-style metric-call budgets. `light`, `medium`, and `heavy` map to DSPy's `auto="light"`, `auto="medium"`, and `auto="heavy"` scales; `auto` is treated as the balanced `medium` preset and is the default when no budget control is provided. The final metric-call limit scales with validation-set size, rather than using fixed tiny constants. `--budget`, `--max-metric-calls`, `--max-full-evals`, and `--max-iterations` are mutually exclusive CLI budget controls; choose only one.
134
134
 
135
- Budget is checked only between iterations. Once an iteration starts, `run-gepa` completes the parent minibatch, child minibatch, and full validation eval for an accepted child even if that overshoots the nominal budget. If a run ends as `budget_exhausted`, it pauses and does not auto-promote. If auto-promotion matters, either increase budget to fit the iteration target or lower `--max-iterations` to fit the budget.
135
+ Metric-call and full-eval budgets are checked only between iterations. Once an iteration starts, `run-gepa` completes the parent minibatch, child minibatch, and full validation eval for an accepted child even if that overshoots the nominal budget. If a run ends as `budget_exhausted`, it pauses and does not auto-promote. If auto-promotion matters, choose a single budget control large enough to cover the intended search.
136
136
 
137
137
  ### Reflection Signal Quality
138
138
 
@@ -62,6 +62,7 @@ Reflection output contract:
62
62
  - The default reflection prompt asks for only the complete updated prompt body. Do not ask the model to wrap the candidate in markdown fences or tags; real prompts often contain those characters.
63
63
  - Put provider-native reasoning controls in `--reflection-provider-settings <json|@file>`, not in the prompt body. For OpenAI reasoning models, use a shape such as `{"reasoning":{"effort":"medium","summary":"auto"}}`. For Anthropic Claude models with thinking controls, use a shape such as `{"thinking":{"type":"adaptive","display":"omitted"},"output_config":{"effort":"medium"}}`.
64
64
  - Reflection max-token limits are explicit. `--reflection-max-tokens <n>` maps to Anthropic `max_tokens` and OpenAI `max_output_tokens`; Anthropic native Messages reflection requires it, while OpenAI may omit it when no cap is desired.
65
+ - Reflection HTTP calls retry transient failures by default (`--reflection-retry-attempts 3`, `--reflection-http-timeout-seconds 180`). Exhausted retryable failures emit `reflection_failed`, consume one candidate-proposal budget slot, and move to the next iteration instead of killing the run.
65
66
 
66
67
  Full command syntax and event contracts: `prompt-control-plane.md`.
67
68
 
@@ -332,9 +332,21 @@ List:
332
332
 
333
333
  ```bash
334
334
  orizu --local prompts list --project <team>/<project>
335
+ orizu --local prompts list --project <team>/<project> --status archived
336
+ orizu --local prompts list --project <team>/<project> --status all
335
337
  orizu --local judges list --project <team>/<project>
336
338
  ```
337
339
 
340
+ Prompt and judge lists show active artifacts by default. Use `--status archived`
341
+ or `--status all` when you need archived artifacts.
342
+
343
+ Archive or restore a prompt:
344
+
345
+ ```bash
346
+ orizu --local prompts archive <prompt-id-or-name> --project <team>/<project>
347
+ orizu --local prompts restore <prompt-id-or-name> --project <team>/<project>
348
+ ```
349
+
338
350
  List prompt comment threads for the latest version, or a specific label/version:
339
351
 
340
352
  ```bash
@@ -607,20 +619,19 @@ orizu --local optimizations run-gepa \
607
619
  --train-split train \
608
620
  --val-split validation \
609
621
  --log-dir ./logs \
610
- --budget light \
611
- --max-iterations 3 \
612
622
  --minibatch-size 3 \
613
623
  --num-threads auto
614
624
  ```
615
625
 
616
626
  Useful GEPA flags:
617
627
 
618
- - `--budget auto|light|medium|heavy`, `--max-metric-calls <n>`, `--max-full-evals <n>`.
628
+ - Budget controls are mutually exclusive: choose at most one of `--budget auto|light|medium|heavy`, `--max-metric-calls <n>`, `--max-full-evals <n>`, or `--max-iterations <n>`. With none provided, `run-gepa` defaults to `--budget auto`, the balanced medium preset.
619
629
  - `--minibatch-size <n>` defaults to 3.
620
- - `--num-threads auto|N` defaults to `auto`; auto caps row-evaluation concurrency from mini-batch size, validation-set size, CPU count, memory estimate, file-descriptor limit, and an 8-thread hard cap.
630
+ - `--num-threads auto|N` defaults to `auto`; auto caps row-evaluation concurrency from mini-batch size, validation-set size, 2x CPU count, memory estimate, file-descriptor limit, and a 64-thread default ceiling. Set `ORIZU_GEPA_AUTO_THREADS_MAX` or use `--num-threads <n>` only when the runner/provider capacity is known.
621
631
  - `--candidate-selection-strategy pareto|current_best|epsilon_greedy`; default is `pareto`.
622
632
  - `--reflection-model <provider/model>`, `--reflection-temperature <n>`, `--reflection-prompt-template <text|@file>`.
623
633
  - `--reflection-max-tokens <n>` is explicit provider config, not a global default. It maps to Anthropic `max_tokens` and OpenAI `max_output_tokens`; Anthropic native Messages reflection requires it, while OpenAI can omit it unless the user wants a cap.
634
+ - `--reflection-retry-attempts` and `--reflection-http-timeout-seconds` tune transient reflection-provider retries. Exhausted retryable failures log `reflection_failed`, count against candidate-proposal budget, and continue with the next iteration.
624
635
  - `--reflection-provider-settings <json|@file>` passes provider-native reflection settings separately from the prompt text. Anthropic example: `{"thinking":{"type":"adaptive","display":"omitted"},"output_config":{"effort":"medium"}}`. OpenAI example: `{"reasoning":{"effort":"medium","summary":"auto"}}`.
625
636
  - `--disable-evaluation-cache` turns off candidate/row/scorer cache reuse.
626
637
  - `--auto-promote --promotion-label <label>` promotes the best candidate at the end.
@@ -718,7 +729,8 @@ Rules:
718
729
  - `eventId` must be stable and unique within one optimization run.
719
730
  - `eventLayer` is `core`, `extension`, or `system`.
720
731
  - Generic core events: `run_started`, `iteration_started`, `candidate_proposed`, `candidate_scored`, `candidate_recommended`, `iteration_completed`, `run_completed`, `run_failed`.
721
- - GEPA extension events: `seed_val_set_started`, `seed_val_set_completed`, `parent_minibatch_started`, `parent_minibatch_completed`, `reflection_started`, `reflection_completed`, `child_candidate_created`, `child_minibatch_started`, `child_minibatch_completed`, `acceptance_decision_made`, `merge_started`, `merge_completed`.
732
+ - GEPA extension events: `seed_val_set_started`, `seed_val_set_completed`, `parent_minibatch_started`, `parent_minibatch_completed`, `reflection_started`, `reflection_completed`, `child_candidate_created`, `child_minibatch_started`, `child_minibatch_completed`, `acceptance_decision_made`, `optimization_progress`, `merge_started`, `merge_completed`.
733
+ - Emit `optimization_progress` after each completed iteration with `percent`, `metric_calls_used`, `metric_call_budget`, and `metric_calls_remaining`. Cap displayed `percent` at `100` and floor `metric_calls_remaining` at `0`; a started iteration may spend past the nominal metric-call budget before the optimizer pauses.
722
734
  - Do not send per-LM-call telemetry to this endpoint in Phase 0. Put aggregate call/token/cost stats in iteration or run payloads.
723
735
 
724
736
  Shell fallback: