create-orcha 0.3.0 → 0.4.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "create-orcha",
3
- "version": "0.3.0",
3
+ "version": "0.4.0",
4
4
  "description": "Create a complete OrchaJS project with example agents and a local playground.",
5
5
  "type": "module",
6
6
  "license": "MIT",
@@ -7,14 +7,14 @@ files under `.orcha/`.
7
7
  ## Commands
8
8
 
9
9
  - `orcha init` creates the initial Orcha files without overwriting files.
10
- - `orcha dev` validates the registry and watches `orcha/**` for changes.
10
+ - `orcha dev` validates agent source and watches `orcha/**` for changes.
11
11
  - `orcha playground` opens the built-in local UI and hot reloads agent source.
12
12
  - `orcha run <agent> --input "…"` executes one registered agent.
13
13
  - `orcha run <agent> --input-file request.json` accepts structured input.
14
14
  - `orcha run <agent> --session <id> --input "…"` continues a session.
15
15
  - `orcha run <agent> --session <id> --tool-results results.json` submits
16
16
  pending client-action results.
17
- - `orcha test` runs every registered agent test.
17
+ - `orcha test` runs every enabled discovered agent test.
18
18
  - `orcha test <agent>` or `orcha test <agent>/<case>` narrows the run.
19
19
  - `orcha build` creates the production Orcha bundle without calling models.
20
20
  - Add `--json` to `run` and `test` for machine-readable output.
@@ -35,17 +35,16 @@ orcha.init({
35
35
  anthropic: process.env.ANTHROPIC_API_KEY ?? "",
36
36
  openai: process.env.OPENAI_API_KEY ?? "",
37
37
  },
38
- actions: { runtime: "native" },
39
38
  agents: {
40
39
  supportBot: "./supportBot",
41
40
  },
42
41
  });
43
42
  ```
44
43
 
45
- Only registered folders are compiled. Agent keys become runtime properties
46
- such as `orcha.supportBot`. Use `actions.runtime: "sandbox"` for isolated
47
- local action execution or `"native"` when the application intentionally
48
- allows action modules to execute in its Node.js process.
44
+ Only registered agents are compiled. Agent keys become runtime properties
45
+ such as `orcha.supportBot`. Local actions use the native Node.js runtime by
46
+ default. Set `actions.runtime: "sandbox"` only when isolated execution is
47
+ required.
49
48
 
50
49
  `orcha.init()` fields:
51
50
 
@@ -53,8 +52,8 @@ allows action modules to execute in its Node.js process.
53
52
  name.
54
53
  - `agents` (required): runtime property names mapped to folders relative to
55
54
  `orcha/`. At least one agent is required.
56
- - `actions` (required only when a registered agent has local actions):
57
- selects the local execution runtime and its environment/sandbox settings.
55
+ - `actions` (optional): overrides the native local-action runtime or configures
56
+ its environment and sandbox settings.
58
57
  - `storage.strategy` (optional): currently only `"node-jsonl"`.
59
58
  - `storage.directory` (optional): session directory relative to project
60
59
  root; defaults to `.orcha/sessions`.
@@ -363,7 +362,6 @@ actions/
363
362
  {
364
363
  "name": "lookup_account",
365
364
  "description": "Look up one account.",
366
- "execution": "local",
367
365
  "parameters": {
368
366
  "type": "object",
369
367
  "properties": {
@@ -402,8 +400,10 @@ Action `index.json` fields:
402
400
  - `name` (required): model-facing tool name, 1–64 letters, numbers,
403
401
  underscores, or hyphens. `load_skill` is reserved.
404
402
  - `description` (required): tells the model when and why to call the action.
405
- - `execution` (required): `"local"` executes `index.js`; `"client"`
406
- pauses the run and delegates execution to the application.
403
+ - `enabled` (optional): defaults to `true`. Set to `false` to keep a WIP
404
+ action inactive without deleting its folder.
405
+ - `execution` (optional): defaults to `"local"`, which executes `index.js`.
406
+ Set to `"client"` to pause and delegate execution to the application.
407
407
  - `parameters` (required): JSON Schema for model-generated arguments.
408
408
  - `outputSchema` (optional): JSON Schema validated against local or submitted
409
409
  client output before the model receives it.
@@ -432,7 +432,7 @@ Global local-action configuration:
432
432
 
433
433
  ```ts
434
434
  actions: {
435
- runtime: "sandbox", // required when any registered action is local
435
+ runtime: "sandbox", // optional; native is the default
436
436
  env: {
437
437
  BILLING_API_TOKEN: process.env.BILLING_API_TOKEN,
438
438
  },
@@ -450,18 +450,9 @@ allowlisted `env`, guarded `fetch`, and prefixed `log`.
450
450
 
451
451
  ## Skills
452
452
 
453
- Skills are lazy-loaded procedural instructions. Register only intended skills:
454
-
455
- ```js
456
- // skills/index.js
457
- import { defineSkills } from "orchajs/skills";
458
-
459
- export default defineSkills({
460
- incidentTriage: "./incidentTriage",
461
- });
462
- ```
463
-
464
- Each skill folder contains `index.json` metadata and `instructions.md`.
453
+ Skills are lazy-loaded procedural instructions. Direct child folders under
454
+ `skills/` are discovered automatically. Each folder contains `index.json`
455
+ metadata and `instructions.md`.
465
456
  The model receives a compact catalog and can call the internal `load_skill`
466
457
  tool. Loaded instructions remain active for the durable session. Lifecycle
467
458
  events are `skill.requested`, `skill.loaded`, and `skill.failed`.
@@ -471,28 +462,23 @@ Skill `index.json` fields:
471
462
  - `name` (required): model-facing name, 1–64 letters, numbers, underscores,
472
463
  or hyphens; unique within the agent.
473
464
  - `description` (required): compact catalog description shown before loading.
465
+ - `enabled` (optional): defaults to `true`. Set to `false` to keep a WIP
466
+ skill inactive.
474
467
  - `triggers` (optional): non-empty array of non-empty situations describing
475
468
  when the model should load the skill.
476
469
 
477
- `instructions.md` is required and cannot be empty. The key in
478
- `skills/index.js` is only a registration label; `index.json.name` is the
479
- name used by the model and durable events. Unregistered folders are ignored.
470
+ `instructions.md` is required and cannot be empty for enabled skills.
471
+ `index.json.name` is the name used by the model and durable events.
480
472
 
481
473
  ## Tests
482
474
 
483
- Register tests in `tests/index.js`:
484
-
485
- ```js
486
- import { defineTests } from "orchajs/testing";
487
-
488
- export default defineTests({
489
- activeAccount: "./activeAccount",
490
- });
491
- ```
492
-
493
- Each case's `index.json` defines `input`, mocked responses for every action,
494
- and `expect`. Tests run the real compiled agent and provider but never execute
495
- real actions. The mocked action set must exactly match the compiled action set.
475
+ Direct child folders under `tests/` are discovered automatically. Each
476
+ case's `index.json` defines `input`, optional mocked action responses, and
477
+ `expect`. Tests run the real compiled agent and provider. Actions listed
478
+ under `actions` are mocked; unlisted local actions execute live. Orcha reports
479
+ live action use in the terminal, test report, and session trace.
480
+ Client actions must always be mocked because no application client is attached
481
+ to the test runner.
496
482
 
497
483
  ```json
498
484
  {
@@ -523,12 +509,21 @@ through action-argument assertions.
523
509
 
524
510
  Test `index.json` fields:
525
511
 
512
+ - `name` (optional): friendly display name. The folder name remains the CLI
513
+ selector.
526
514
  - `description` (optional): human-readable purpose.
527
- - `input.content` (required): string or multimodal content array.
515
+ - `enabled` (optional): defaults to `true`. Set to `false` to keep a WIP test
516
+ inactive.
517
+ - `input` (required): an inline agent input object or a path to a JSON file
518
+ containing that object. Relative paths resolve from the test case directory.
519
+ - `input.content` (required for inline input): string or multimodal content
520
+ array.
528
521
  - `input.variables` (optional): string map available to the session.
529
522
  - `input.metadata` (optional): string, number, boolean, or null values.
530
- - `actions` (required): exactly one key for every compiled action, including
531
- local actions. Every `responses` array is consumed in call order.
523
+ - `actions` (optional): mocked actions keyed by compiled action name. Unlisted
524
+ local actions execute live and may cause side effects. Unknown action names
525
+ are rejected, and unlisted client actions are rejected. Every `responses`
526
+ array is consumed in call order.
532
527
  - `responses[].output` (required): mocked action result.
533
528
  - `responses[].isError` (optional): marks the mocked result as an error.
534
529
  - `expect.status` (optional): `"completed"` or `"failed"`; defaults to
@@ -540,15 +535,13 @@ Test `index.json` fields:
540
535
  - `expect.actions` (optional): ordered expected calls. Each may assert
541
536
  `arguments.equals` or `arguments.partial`.
542
537
 
543
- The registration key in `tests/index.js` is the test selector used by
544
- `orcha test agent/testName`; its value resolves to the case folder.
538
+ The test folder name is the selector used by `orcha test agent/testName`.
545
539
 
546
540
  ## Evaluations
547
541
 
548
- Evaluations are asynchronous LLM judges registered in
549
- `evaluations/index.js` with `defineEvaluations` from
550
- `orchajs/evaluations`. Each folder's `index.json` defines its provider,
551
- model, metrics, and thresholds:
542
+ Evaluations are asynchronous LLM judges discovered from direct child folders
543
+ under `evaluations/`. Each folder's `index.json` defines its provider, model,
544
+ metrics, and thresholds:
552
545
 
553
546
  ```json
554
547
  {
@@ -581,8 +574,8 @@ Evaluation `index.json` fields:
581
574
  - `description` (optional): short human-facing summary of the evaluator.
582
575
  - `instructions` (optional): detailed prompt supplied to the judge. When
583
576
  omitted, `description` is used for backward compatibility.
584
- - `enabled` (optional): defaults to `true`. Disabled evaluations are
585
- compiled but do not run.
577
+ - `enabled` (optional): defaults to `true`. Disabled evaluations are ignored
578
+ by compilation and runtime.
586
579
  - `provider` and `model` (required): independently select the judge. The
587
580
  provider must also exist in `orcha.init().providers`.
588
581
  - `maxTokens` (optional): positive integer; defaults to 2000 for judges.
@@ -917,8 +910,19 @@ type TestCompleted = SessionEvent<{
917
910
  durationMs: number;
918
911
  error?: { message: string };
919
912
  }>;
913
+ warnings?: Array<{
914
+ code: "live_actions";
915
+ message: string;
916
+ actions: string[];
917
+ }>;
920
918
  error?: ErrorData;
921
919
  }>; // type "test.completed", no run
920
+
921
+ type TestWarning = SessionEvent<{
922
+ code: "live_actions";
923
+ message: string;
924
+ actions: string[];
925
+ }>; // type "test.warning", no run
922
926
  ```
923
927
 
924
928
  Typical event order:
@@ -955,8 +959,9 @@ number. A conversational `resume(...content)` starts a new run number.
955
959
  1. `orcha/index.ts` calls `orcha.init()` with explicit agent paths.
956
960
  2. The compiler reads each registered agent's `index.json` and
957
961
  `instructions.md`.
958
- 3. Every directory under `actions/` is compiled. Skills, tests, and
959
- evaluations are included only through their local `index.js` registry.
962
+ 3. Direct child folders under `actions/`, `skills/`, `tests/`, and
963
+ `evaluations/` are discovered automatically. Items with `"enabled": false`
964
+ remain inactive.
960
965
  4. Local action source is bundled and SHA-256 hashed. Production bundles keep
961
966
  only provider adapters required by agents and enabled evaluations.
962
967
  5. Invalid paths, duplicate model-facing names, missing files, unsupported
@@ -1037,9 +1042,11 @@ needed for faithful continuation but are omitted from evaluation transcripts.
1037
1042
 
1038
1043
  ## Change rules
1039
1044
 
1040
- - Register every new agent, skill, test, and evaluation explicitly.
1045
+ - Register every new agent and subagent explicitly. Agent-owned actions, skills,
1046
+ tests, and evaluations are auto-discovered; use `"enabled": false` for WIP.
1041
1047
  - Keep runtime behavior provider-neutral.
1042
- - Do not call real actions from tests.
1048
+ - Mock actions when tests must avoid side effects; document intentional live
1049
+ action coverage.
1043
1050
  - Do not commit `.orcha/`; it contains generated output and session data.
1044
1051
  - Run `orcha dev` after filesystem changes and `orcha test` when behavior
1045
1052
  changes.
@@ -4,9 +4,6 @@ orcha.init({
4
4
  providers: {
5
5
  openai: process.env.OPENAI_API_KEY ?? "",
6
6
  },
7
- actions: {
8
- runtime: "native",
9
- },
10
7
  agents: {
11
8
  approvalAgent: "./approvalAgent",
12
9
  assistant: {
@@ -1,7 +1,6 @@
1
1
  {
2
2
  "name": "lookup_order",
3
3
  "description": "Look up the latest confirmed state of an order.",
4
- "execution": "local",
5
4
  "parameters": {
6
5
  "type": "object",
7
6
  "properties": {
@@ -1,5 +0,0 @@
1
- import { defineEvaluations } from "orchajs/evaluations";
2
-
3
- export default defineEvaluations({
4
- approvalQuality: "./approvalQuality",
5
- });
@@ -1,5 +0,0 @@
1
- import { defineSkills } from "orchajs/skills";
2
-
3
- export default defineSkills({
4
- purchaseReview: "./purchaseReview",
5
- });
@@ -1,5 +0,0 @@
1
- import { defineTests } from "orchajs/testing";
2
-
3
- export default defineTests({
4
- approvedPurchase: "./approvedPurchase",
5
- });
@@ -1,5 +0,0 @@
1
- import { defineEvaluations } from "orchajs/evaluations";
2
-
3
- export default defineEvaluations({
4
- responseQuality: "./responseQuality",
5
- });
@@ -1,5 +0,0 @@
1
- import { defineSkills } from "orchajs/skills";
2
-
3
- export default defineSkills({
4
- technicalExplanation: "./technicalExplanation",
5
- });
@@ -1,5 +0,0 @@
1
- import { defineTests } from "orchajs/testing";
2
-
3
- export default defineTests({
4
- explainsIdempotency: "./explainsIdempotency",
5
- });
@@ -1,5 +0,0 @@
1
- import { defineEvaluations } from "orchajs/evaluations";
2
-
3
- export default defineEvaluations({
4
- supportQuality: "./supportQuality",
5
- });
@@ -1,5 +0,0 @@
1
- import { defineSkills } from "orchajs/skills";
2
-
3
- export default defineSkills({
4
- delayedOrderHandling: "./delayedOrderHandling",
5
- });
@@ -1,5 +0,0 @@
1
- import { defineTests } from "orchajs/testing";
2
-
3
- export default defineTests({
4
- delayedOrder: "./delayedOrder",
5
- });