create-orcha 0.3.0 → 0.3.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/package.json +1 -1
- package/template/orcha/AGENTS.md +27 -7
package/package.json
CHANGED
package/template/orcha/AGENTS.md
CHANGED
|
@@ -490,9 +490,12 @@ export default defineTests({
|
|
|
490
490
|
});
|
|
491
491
|
```
|
|
492
492
|
|
|
493
|
-
Each case's `index.json` defines `input`, mocked responses
|
|
494
|
-
and `expect`. Tests run the real compiled agent and provider
|
|
495
|
-
|
|
493
|
+
Each case's `index.json` defines `input`, optional mocked action responses,
|
|
494
|
+
and `expect`. Tests run the real compiled agent and provider. Actions listed
|
|
495
|
+
under `actions` are mocked; unlisted local actions execute live. Orcha reports
|
|
496
|
+
live action use in the terminal, test report, and session trace.
|
|
497
|
+
Client actions must always be mocked because no application client is attached
|
|
498
|
+
to the test runner.
|
|
496
499
|
|
|
497
500
|
```json
|
|
498
501
|
{
|
|
@@ -524,11 +527,16 @@ through action-argument assertions.
|
|
|
524
527
|
Test `index.json` fields:
|
|
525
528
|
|
|
526
529
|
- `description` (optional): human-readable purpose.
|
|
527
|
-
- `input
|
|
530
|
+
- `input` (required): an inline agent input object or a path to a JSON file
|
|
531
|
+
containing that object. Relative paths resolve from the test case directory.
|
|
532
|
+
- `input.content` (required for inline input): string or multimodal content
|
|
533
|
+
array.
|
|
528
534
|
- `input.variables` (optional): string map available to the session.
|
|
529
535
|
- `input.metadata` (optional): string, number, boolean, or null values.
|
|
530
|
-
- `actions` (
|
|
531
|
-
local actions
|
|
536
|
+
- `actions` (optional): mocked actions keyed by compiled action name. Unlisted
|
|
537
|
+
local actions execute live and may cause side effects. Unknown action names
|
|
538
|
+
are rejected, and unlisted client actions are rejected. Every `responses`
|
|
539
|
+
array is consumed in call order.
|
|
532
540
|
- `responses[].output` (required): mocked action result.
|
|
533
541
|
- `responses[].isError` (optional): marks the mocked result as an error.
|
|
534
542
|
- `expect.status` (optional): `"completed"` or `"failed"`; defaults to
|
|
@@ -917,8 +925,19 @@ type TestCompleted = SessionEvent<{
|
|
|
917
925
|
durationMs: number;
|
|
918
926
|
error?: { message: string };
|
|
919
927
|
}>;
|
|
928
|
+
warnings?: Array<{
|
|
929
|
+
code: "live_actions";
|
|
930
|
+
message: string;
|
|
931
|
+
actions: string[];
|
|
932
|
+
}>;
|
|
920
933
|
error?: ErrorData;
|
|
921
934
|
}>; // type "test.completed", no run
|
|
935
|
+
|
|
936
|
+
type TestWarning = SessionEvent<{
|
|
937
|
+
code: "live_actions";
|
|
938
|
+
message: string;
|
|
939
|
+
actions: string[];
|
|
940
|
+
}>; // type "test.warning", no run
|
|
922
941
|
```
|
|
923
942
|
|
|
924
943
|
Typical event order:
|
|
@@ -1039,7 +1058,8 @@ needed for faithful continuation but are omitted from evaluation transcripts.
|
|
|
1039
1058
|
|
|
1040
1059
|
- Register every new agent, skill, test, and evaluation explicitly.
|
|
1041
1060
|
- Keep runtime behavior provider-neutral.
|
|
1042
|
-
-
|
|
1061
|
+
- Mock actions when tests must avoid side effects; document intentional live
|
|
1062
|
+
action coverage.
|
|
1043
1063
|
- Do not commit `.orcha/`; it contains generated output and session data.
|
|
1044
1064
|
- Run `orcha dev` after filesystem changes and `orcha test` when behavior
|
|
1045
1065
|
changes.
|