@tangle-network/agent-eval 0.145.3 → 0.145.4
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +13 -0
- package/dist/analyst/index.d.ts +1 -1
- package/dist/analyst/index.js +1 -1
- package/dist/{benchmark-command-Iaqlp3DU.js → benchmark-command-BANo4dTm.js} +2 -2
- package/dist/{benchmark-command-Iaqlp3DU.js.map → benchmark-command-BANo4dTm.js.map} +1 -1
- package/dist/benchmarks/index.d.ts +2 -2
- package/dist/benchmarks/index.js +2 -2
- package/dist/campaign/index.d.ts +4 -4
- package/dist/campaign/index.js +4 -4
- package/dist/{campaign-C5kujyqv.js → campaign-FY23WYzZ.js} +503 -24
- package/dist/campaign-FY23WYzZ.js.map +1 -0
- package/dist/cli.js +1 -1
- package/dist/contract/index.d.ts +3 -3
- package/dist/contract/index.js +3 -3
- package/dist/{define-agent-eval-sH24zBfM.d.ts → define-agent-eval-DcAcC1yQ.d.ts} +2 -2
- package/dist/{define-agent-eval-sH24zBfM.d.ts.map → define-agent-eval-DcAcC1yQ.d.ts.map} +1 -1
- package/dist/{define-agent-eval-CAL5HOz9.js → define-agent-eval-xaceIpPS.js} +2 -2
- package/dist/{define-agent-eval-CAL5HOz9.js.map → define-agent-eval-xaceIpPS.js.map} +1 -1
- package/dist/{index-CUczrQ7a.d.ts → index-Dytg99g1.d.ts} +101 -4
- package/dist/index-Dytg99g1.d.ts.map +1 -0
- package/dist/index.d.ts +2 -2
- package/dist/index.js +3 -3
- package/dist/{llm-judge-C2lQBv7h.js → llm-judge-BPKBdQTp.js} +7 -4
- package/dist/llm-judge-BPKBdQTp.js.map +1 -0
- package/dist/openapi.json +1 -1
- package/dist/{produced-state-Bgph_Qj-.js → produced-state-B_-TW9o0.js} +2 -2
- package/dist/{produced-state-Bgph_Qj-.js.map → produced-state-B_-TW9o0.js.map} +1 -1
- package/dist/{provenance-CCdxgLDT.d.ts → provenance-C_ln89b0.d.ts} +15 -1
- package/dist/provenance-C_ln89b0.d.ts.map +1 -0
- package/dist/{skillopt-optimization-method-C5cotF4E.d.ts → skillopt-optimization-method-CkySQ0ca.d.ts} +2 -2
- package/dist/{skillopt-optimization-method-C5cotF4E.d.ts.map → skillopt-optimization-method-CkySQ0ca.d.ts.map} +1 -1
- package/dist/{skillopt-optimization-method-B__Jnto4.js → skillopt-optimization-method-CxrOjp1S.js} +2 -2
- package/dist/{skillopt-optimization-method-B__Jnto4.js.map → skillopt-optimization-method-CxrOjp1S.js.map} +1 -1
- package/docs/eval-surface-map.md +11 -0
- package/package.json +1 -1
- package/dist/campaign-C5kujyqv.js.map +0 -1
- package/dist/index-CUczrQ7a.d.ts.map +0 -1
- package/dist/llm-judge-C2lQBv7h.js.map +0 -1
- package/dist/provenance-CCdxgLDT.d.ts.map +0 -1
package/docs/eval-surface-map.md
CHANGED
|
@@ -22,6 +22,17 @@ Set `maxConcurrency` for cases within one profile and `maxProfileConcurrency` fo
|
|
|
22
22
|
Every paid-call receipt must match the profile, and a moving alias must resolve to one snapshot across the entire profile column.
|
|
23
23
|
The caller commit and profile config are always part of cache identity; set `dispatchRef` when execution behavior can change without a new commit.
|
|
24
24
|
A failed profile cancels active sibling columns before the matrix rejects.
|
|
25
|
+
|
|
26
|
+
When one external grant cannot run the complete profile matrix, use
|
|
27
|
+
`createProfileMatrixPlan`, `runProfileMatrixSegment`, and
|
|
28
|
+
`finalizeProfileMatrix` from the same campaign surface.
|
|
29
|
+
The plan hashes the complete profiles × scenarios × reps design and assigns
|
|
30
|
+
one stable row identity to every cell.
|
|
31
|
+
Each segment claims explicit, disjoint rows and can reuse its segment identity
|
|
32
|
+
to retry failed cells from Eval's durable campaign cache.
|
|
33
|
+
Finalization refuses missing, overlapping, stale, corrupt, or duplicate rows,
|
|
34
|
+
then returns the ordinary `runProfileMatrix` result and its distributions.
|
|
35
|
+
Coverage reports missing, failed, and zero-score rows separately.
|
|
25
36
|
| `runAgentMatrix` | The bare N-axis cartesian scheduler with concurrency control. The layer beneath the eval surface: reach for it only when you need raw scheduling, not eval semantics. | cell results |
|
|
26
37
|
|
|
27
38
|
Mental model: **measure** (`runCampaign`/`runEval`) → **factor** (`runProfileMatrix`) →
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@tangle-network/agent-eval",
|
|
3
|
-
"version": "0.145.
|
|
3
|
+
"version": "0.145.4",
|
|
4
4
|
"description": "Evaluate and improve AI agents from runs, traces, judges, and feedback. Compare candidates, cluster failures, measure lift, and gate releases.",
|
|
5
5
|
"homepage": "https://github.com/tangle-network/agent-eval#readme",
|
|
6
6
|
"repository": {
|