@ls-stack/agent-eval 0.24.0 → 0.26.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (18) hide show
  1. package/dist/{app-DYRmucgj.mjs → app-BWjrZUSt.mjs} +6 -3
  2. package/dist/apps/web/dist/assets/index-3bLi3uQJ.js +118 -0
  3. package/dist/apps/web/dist/assets/index-DNsZjOms.css +1 -0
  4. package/dist/apps/web/dist/index.html +2 -2
  5. package/dist/bin.mjs +1 -1
  6. package/dist/{cli-Be0x8CS3.mjs → cli-CmNNm6d0.mjs} +155 -45
  7. package/dist/index.d.mts +61 -6
  8. package/dist/index.mjs +4 -4
  9. package/dist/runChild.mjs +28 -18
  10. package/dist/{runOrchestration-D697g6Qe.mjs → runOrchestration-B31SV_Bq.mjs} +222 -87
  11. package/dist/{runner-jSujaSKt.mjs → runner-NPxmmHwa.mjs} +1 -1
  12. package/dist/{runner-B4SosWgD.mjs → runner-dnkl4pWd.mjs} +2 -2
  13. package/dist/src-DIFRuMuH.mjs +3 -0
  14. package/package.json +3 -3
  15. package/skills/agent-eval/SKILL.md +6 -1
  16. package/dist/apps/web/dist/assets/index-KbbX3NYr.js +0 -118
  17. package/dist/apps/web/dist/assets/index-r0dVFK0B.css +0 -1
  18. package/dist/src-D6cettg0.mjs +0 -3
package/skills/agent-eval/SKILL.md CHANGED
@@ -374,6 +374,10 @@ Mental model:
374
374
  across operations/evals, but the source-file fingerprint still participates
375
375
  in the final key. Shared namespaces are reusable across evals in the same
376
376
  file; evals in different files miss even with the same namespace and key.
377
+ - Authored eval ids are unique within one eval file. The exact eval identity is
378
+ the workspace-relative file path plus eval id, so the same id can be reused in
379
+ different files. Case ids must be unique within one eval; duplicate case ids
380
+ are reported as run errors.
377
381
  - Cache keys should be deterministic primitives, arrays, and plain objects.
378
382
  `Buffer`, `ArrayBuffer`, and typed arrays hash by bytes. Native `Blob`/`File`
379
383
  keys use stable metadata by default (`type`, `size`, plus
@@ -462,7 +466,8 @@ When adding or changing evals:
462
466
  `evalTracer.cache(...)`; never cache operations whose external side effects
463
467
  you depend on.
464
468
  7. Sanity-check after changes: `agent-evals list`, then
465
- `agent-evals run --eval <id>`.
469
+ `agent-evals run --eval <id>`; use `--file <path|glob>` to target one file
470
+ when multiple files use the same eval id.
466
471
  8. Locate saved artifacts with `agent-evals show-runs latest --json`, then read
467
472
  the relevant `summary.json`, `cases.jsonl`, `case-details/<case-id>.json`,
468
473
  or `traces/<case-id>.json` file directly.