create-orcha 0.3.0 → 0.4.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/package.json +1 -1
- package/template/orcha/AGENTS.md +63 -56
- package/template/orcha/index.ts +0 -3
- package/template/orcha/orderSupport/actions/lookupOrder/index.json +0 -1
- package/template/orcha/approvalAgent/evaluations/index.js +0 -5
- package/template/orcha/approvalAgent/skills/index.js +0 -5
- package/template/orcha/approvalAgent/tests/index.js +0 -5
- package/template/orcha/assistant/evaluations/index.js +0 -5
- package/template/orcha/assistant/skills/index.js +0 -5
- package/template/orcha/assistant/tests/index.js +0 -5
- package/template/orcha/orderSupport/evaluations/index.js +0 -5
- package/template/orcha/orderSupport/skills/index.js +0 -5
- package/template/orcha/orderSupport/tests/index.js +0 -5
package/package.json
CHANGED
package/template/orcha/AGENTS.md
CHANGED
|
@@ -7,14 +7,14 @@ files under `.orcha/`.
|
|
|
7
7
|
## Commands
|
|
8
8
|
|
|
9
9
|
- `orcha init` creates the initial Orcha files without overwriting files.
|
|
10
|
-
- `orcha dev` validates
|
|
10
|
+
- `orcha dev` validates agent source and watches `orcha/**` for changes.
|
|
11
11
|
- `orcha playground` opens the built-in local UI and hot reloads agent source.
|
|
12
12
|
- `orcha run <agent> --input "…"` executes one registered agent.
|
|
13
13
|
- `orcha run <agent> --input-file request.json` accepts structured input.
|
|
14
14
|
- `orcha run <agent> --session <id> --input "…"` continues a session.
|
|
15
15
|
- `orcha run <agent> --session <id> --tool-results results.json` submits
|
|
16
16
|
pending client-action results.
|
|
17
|
-
- `orcha test` runs every
|
|
17
|
+
- `orcha test` runs every enabled discovered agent test.
|
|
18
18
|
- `orcha test <agent>` or `orcha test <agent>/<case>` narrows the run.
|
|
19
19
|
- `orcha build` creates the production Orcha bundle without calling models.
|
|
20
20
|
- Add `--json` to `run` and `test` for machine-readable output.
|
|
@@ -35,17 +35,16 @@ orcha.init({
|
|
|
35
35
|
anthropic: process.env.ANTHROPIC_API_KEY ?? "",
|
|
36
36
|
openai: process.env.OPENAI_API_KEY ?? "",
|
|
37
37
|
},
|
|
38
|
-
actions: { runtime: "native" },
|
|
39
38
|
agents: {
|
|
40
39
|
supportBot: "./supportBot",
|
|
41
40
|
},
|
|
42
41
|
});
|
|
43
42
|
```
|
|
44
43
|
|
|
45
|
-
Only registered
|
|
46
|
-
such as `orcha.supportBot`.
|
|
47
|
-
|
|
48
|
-
|
|
44
|
+
Only registered agents are compiled. Agent keys become runtime properties
|
|
45
|
+
such as `orcha.supportBot`. Local actions use the native Node.js runtime by
|
|
46
|
+
default. Set `actions.runtime: "sandbox"` only when isolated execution is
|
|
47
|
+
required.
|
|
49
48
|
|
|
50
49
|
`orcha.init()` fields:
|
|
51
50
|
|
|
@@ -53,8 +52,8 @@ allows action modules to execute in its Node.js process.
|
|
|
53
52
|
name.
|
|
54
53
|
- `agents` (required): runtime property names mapped to folders relative to
|
|
55
54
|
`orcha/`. At least one agent is required.
|
|
56
|
-
- `actions` (
|
|
57
|
-
|
|
55
|
+
- `actions` (optional): overrides the native local-action runtime or configures
|
|
56
|
+
its environment and sandbox settings.
|
|
58
57
|
- `storage.strategy` (optional): currently only `"node-jsonl"`.
|
|
59
58
|
- `storage.directory` (optional): session directory relative to project
|
|
60
59
|
root; defaults to `.orcha/sessions`.
|
|
@@ -363,7 +362,6 @@ actions/
|
|
|
363
362
|
{
|
|
364
363
|
"name": "lookup_account",
|
|
365
364
|
"description": "Look up one account.",
|
|
366
|
-
"execution": "local",
|
|
367
365
|
"parameters": {
|
|
368
366
|
"type": "object",
|
|
369
367
|
"properties": {
|
|
@@ -402,8 +400,10 @@ Action `index.json` fields:
|
|
|
402
400
|
- `name` (required): model-facing tool name, 1–64 letters, numbers,
|
|
403
401
|
underscores, or hyphens. `load_skill` is reserved.
|
|
404
402
|
- `description` (required): tells the model when and why to call the action.
|
|
405
|
-
- `
|
|
406
|
-
|
|
403
|
+
- `enabled` (optional): defaults to `true`. Set to `false` to keep a WIP
|
|
404
|
+
action inactive without deleting its folder.
|
|
405
|
+
- `execution` (optional): defaults to `"local"`, which executes `index.js`.
|
|
406
|
+
Set to `"client"` to pause and delegate execution to the application.
|
|
407
407
|
- `parameters` (required): JSON Schema for model-generated arguments.
|
|
408
408
|
- `outputSchema` (optional): JSON Schema validated against local or submitted
|
|
409
409
|
client output before the model receives it.
|
|
@@ -432,7 +432,7 @@ Global local-action configuration:
|
|
|
432
432
|
|
|
433
433
|
```ts
|
|
434
434
|
actions: {
|
|
435
|
-
runtime: "sandbox", //
|
|
435
|
+
runtime: "sandbox", // optional; native is the default
|
|
436
436
|
env: {
|
|
437
437
|
BILLING_API_TOKEN: process.env.BILLING_API_TOKEN,
|
|
438
438
|
},
|
|
@@ -450,18 +450,9 @@ allowlisted `env`, guarded `fetch`, and prefixed `log`.
|
|
|
450
450
|
|
|
451
451
|
## Skills
|
|
452
452
|
|
|
453
|
-
Skills are lazy-loaded procedural instructions.
|
|
454
|
-
|
|
455
|
-
|
|
456
|
-
// skills/index.js
|
|
457
|
-
import { defineSkills } from "orchajs/skills";
|
|
458
|
-
|
|
459
|
-
export default defineSkills({
|
|
460
|
-
incidentTriage: "./incidentTriage",
|
|
461
|
-
});
|
|
462
|
-
```
|
|
463
|
-
|
|
464
|
-
Each skill folder contains `index.json` metadata and `instructions.md`.
|
|
453
|
+
Skills are lazy-loaded procedural instructions. Direct child folders under
|
|
454
|
+
`skills/` are discovered automatically. Each folder contains `index.json`
|
|
455
|
+
metadata and `instructions.md`.
|
|
465
456
|
The model receives a compact catalog and can call the internal `load_skill`
|
|
466
457
|
tool. Loaded instructions remain active for the durable session. Lifecycle
|
|
467
458
|
events are `skill.requested`, `skill.loaded`, and `skill.failed`.
|
|
@@ -471,28 +462,23 @@ Skill `index.json` fields:
|
|
|
471
462
|
- `name` (required): model-facing name, 1–64 letters, numbers, underscores,
|
|
472
463
|
or hyphens; unique within the agent.
|
|
473
464
|
- `description` (required): compact catalog description shown before loading.
|
|
465
|
+
- `enabled` (optional): defaults to `true`. Set to `false` to keep a WIP
|
|
466
|
+
skill inactive.
|
|
474
467
|
- `triggers` (optional): non-empty array of non-empty situations describing
|
|
475
468
|
when the model should load the skill.
|
|
476
469
|
|
|
477
|
-
`instructions.md` is required and cannot be empty
|
|
478
|
-
`
|
|
479
|
-
name used by the model and durable events. Unregistered folders are ignored.
|
|
470
|
+
`instructions.md` is required and cannot be empty for enabled skills.
|
|
471
|
+
`index.json.name` is the name used by the model and durable events.
|
|
480
472
|
|
|
481
473
|
## Tests
|
|
482
474
|
|
|
483
|
-
|
|
484
|
-
|
|
485
|
-
|
|
486
|
-
|
|
487
|
-
|
|
488
|
-
|
|
489
|
-
|
|
490
|
-
});
|
|
491
|
-
```
|
|
492
|
-
|
|
493
|
-
Each case's `index.json` defines `input`, mocked responses for every action,
|
|
494
|
-
and `expect`. Tests run the real compiled agent and provider but never execute
|
|
495
|
-
real actions. The mocked action set must exactly match the compiled action set.
|
|
475
|
+
Direct child folders under `tests/` are discovered automatically. Each
|
|
476
|
+
case's `index.json` defines `input`, optional mocked action responses, and
|
|
477
|
+
`expect`. Tests run the real compiled agent and provider. Actions listed
|
|
478
|
+
under `actions` are mocked; unlisted local actions execute live. Orcha reports
|
|
479
|
+
live action use in the terminal, test report, and session trace.
|
|
480
|
+
Client actions must always be mocked because no application client is attached
|
|
481
|
+
to the test runner.
|
|
496
482
|
|
|
497
483
|
```json
|
|
498
484
|
{
|
|
@@ -523,12 +509,21 @@ through action-argument assertions.
|
|
|
523
509
|
|
|
524
510
|
Test `index.json` fields:
|
|
525
511
|
|
|
512
|
+
- `name` (optional): friendly display name. The folder name remains the CLI
|
|
513
|
+
selector.
|
|
526
514
|
- `description` (optional): human-readable purpose.
|
|
527
|
-
- `
|
|
515
|
+
- `enabled` (optional): defaults to `true`. Set to `false` to keep a WIP test
|
|
516
|
+
inactive.
|
|
517
|
+
- `input` (required): an inline agent input object or a path to a JSON file
|
|
518
|
+
containing that object. Relative paths resolve from the test case directory.
|
|
519
|
+
- `input.content` (required for inline input): string or multimodal content
|
|
520
|
+
array.
|
|
528
521
|
- `input.variables` (optional): string map available to the session.
|
|
529
522
|
- `input.metadata` (optional): string, number, boolean, or null values.
|
|
530
|
-
- `actions` (
|
|
531
|
-
local actions
|
|
523
|
+
- `actions` (optional): mocked actions keyed by compiled action name. Unlisted
|
|
524
|
+
local actions execute live and may cause side effects. Unknown action names
|
|
525
|
+
are rejected, and unlisted client actions are rejected. Every `responses`
|
|
526
|
+
array is consumed in call order.
|
|
532
527
|
- `responses[].output` (required): mocked action result.
|
|
533
528
|
- `responses[].isError` (optional): marks the mocked result as an error.
|
|
534
529
|
- `expect.status` (optional): `"completed"` or `"failed"`; defaults to
|
|
@@ -540,15 +535,13 @@ Test `index.json` fields:
|
|
|
540
535
|
- `expect.actions` (optional): ordered expected calls. Each may assert
|
|
541
536
|
`arguments.equals` or `arguments.partial`.
|
|
542
537
|
|
|
543
|
-
The
|
|
544
|
-
`orcha test agent/testName`; its value resolves to the case folder.
|
|
538
|
+
The test folder name is the selector used by `orcha test agent/testName`.
|
|
545
539
|
|
|
546
540
|
## Evaluations
|
|
547
541
|
|
|
548
|
-
Evaluations are asynchronous LLM judges
|
|
549
|
-
`evaluations
|
|
550
|
-
|
|
551
|
-
model, metrics, and thresholds:
|
|
542
|
+
Evaluations are asynchronous LLM judges discovered from direct child folders
|
|
543
|
+
under `evaluations/`. Each folder's `index.json` defines its provider, model,
|
|
544
|
+
metrics, and thresholds:
|
|
552
545
|
|
|
553
546
|
```json
|
|
554
547
|
{
|
|
@@ -581,8 +574,8 @@ Evaluation `index.json` fields:
|
|
|
581
574
|
- `description` (optional): short human-facing summary of the evaluator.
|
|
582
575
|
- `instructions` (optional): detailed prompt supplied to the judge. When
|
|
583
576
|
omitted, `description` is used for backward compatibility.
|
|
584
|
-
- `enabled` (optional): defaults to `true`. Disabled evaluations are
|
|
585
|
-
|
|
577
|
+
- `enabled` (optional): defaults to `true`. Disabled evaluations are ignored
|
|
578
|
+
by compilation and runtime.
|
|
586
579
|
- `provider` and `model` (required): independently select the judge. The
|
|
587
580
|
provider must also exist in `orcha.init().providers`.
|
|
588
581
|
- `maxTokens` (optional): positive integer; defaults to 2000 for judges.
|
|
@@ -917,8 +910,19 @@ type TestCompleted = SessionEvent<{
|
|
|
917
910
|
durationMs: number;
|
|
918
911
|
error?: { message: string };
|
|
919
912
|
}>;
|
|
913
|
+
warnings?: Array<{
|
|
914
|
+
code: "live_actions";
|
|
915
|
+
message: string;
|
|
916
|
+
actions: string[];
|
|
917
|
+
}>;
|
|
920
918
|
error?: ErrorData;
|
|
921
919
|
}>; // type "test.completed", no run
|
|
920
|
+
|
|
921
|
+
type TestWarning = SessionEvent<{
|
|
922
|
+
code: "live_actions";
|
|
923
|
+
message: string;
|
|
924
|
+
actions: string[];
|
|
925
|
+
}>; // type "test.warning", no run
|
|
922
926
|
```
|
|
923
927
|
|
|
924
928
|
Typical event order:
|
|
@@ -955,8 +959,9 @@ number. A conversational `resume(...content)` starts a new run number.
|
|
|
955
959
|
1. `orcha/index.ts` calls `orcha.init()` with explicit agent paths.
|
|
956
960
|
2. The compiler reads each registered agent's `index.json` and
|
|
957
961
|
`instructions.md`.
|
|
958
|
-
3.
|
|
959
|
-
evaluations are
|
|
962
|
+
3. Direct child folders under `actions/`, `skills/`, `tests/`, and
|
|
963
|
+
`evaluations/` are discovered automatically. Items with `"enabled": false`
|
|
964
|
+
remain inactive.
|
|
960
965
|
4. Local action source is bundled and SHA-256 hashed. Production bundles keep
|
|
961
966
|
only provider adapters required by agents and enabled evaluations.
|
|
962
967
|
5. Invalid paths, duplicate model-facing names, missing files, unsupported
|
|
@@ -1037,9 +1042,11 @@ needed for faithful continuation but are omitted from evaluation transcripts.
|
|
|
1037
1042
|
|
|
1038
1043
|
## Change rules
|
|
1039
1044
|
|
|
1040
|
-
- Register every new agent
|
|
1045
|
+
- Register every new agent and subagent explicitly. Agent-owned actions, skills,
|
|
1046
|
+
tests, and evaluations are auto-discovered; use `"enabled": false` for WIP.
|
|
1041
1047
|
- Keep runtime behavior provider-neutral.
|
|
1042
|
-
-
|
|
1048
|
+
- Mock actions when tests must avoid side effects; document intentional live
|
|
1049
|
+
action coverage.
|
|
1043
1050
|
- Do not commit `.orcha/`; it contains generated output and session data.
|
|
1044
1051
|
- Run `orcha dev` after filesystem changes and `orcha test` when behavior
|
|
1045
1052
|
changes.
|
package/template/orcha/index.ts
CHANGED