@axlsdk/eval 0.20.0 → 0.20.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (2) hide show
  1. package/README.md +5 -0
  2. package/package.json +2 -2
package/README.md CHANGED
@@ -482,6 +482,11 @@ for (const item of results.items) {
482
482
 
483
483
  `captureTraces` strips verbose-mode `agent_call_start.data.messages` request snapshots and high-volume events (`token`, `partial_object`) from the captured array to keep memory bounded. The structural events you'd want for debugging — `agent_call_start`/`agent_call_end`, `tool_call_*`, gate events, `pipeline`, `verify`, `handoff_*` — are all retained.
484
484
 
485
+ For tool-using evals, these events let your own analysis distinguish a tool that
486
+ failed after starting from one that was rejected, denied, or cancelled. Axl Evals does
487
+ not assign scores to those states automatically; they are evidence for your scorers
488
+ and debugging.
489
+
485
490
  #### Per-item live observation with `ctx.events`
486
491
 
487
492
  `captureTraces` collects events for post-hoc inspection on `EvalItem.traces`. For **live** per-item observation — e.g., printing each item's `partial_object` snapshots as they stream during eval execution — use `ctx.events` inside your top-level `executeWorkflow` export. The two are complementary: `captureTraces` records the structural timeline; `ctx.events` is the live firehose.
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@axlsdk/eval",
3
- "version": "0.20.0",
3
+ "version": "0.20.1",
4
4
  "description": "Evaluation framework for Axl agentic workflows",
5
5
  "type": "module",
6
6
  "main": "./dist/index.cjs",
@@ -25,7 +25,7 @@
25
25
  "dist"
26
26
  ],
27
27
  "dependencies": {
28
- "@axlsdk/axl": "0.20.0"
28
+ "@axlsdk/axl": "0.20.1"
29
29
  },
30
30
  "peerDependencies": {
31
31
  "zod": "^4.0.0",