@tangle-network/agent-eval 0.187.1 → 0.187.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +8 -0
- package/dist/analyst/index.d.ts +1 -1
- package/dist/analyst/index.js +1 -1
- package/dist/{benchmark-command-CERRH0E6.js → benchmark-command-DGokT8zd.js} +2 -2
- package/dist/{benchmark-command-CERRH0E6.js.map → benchmark-command-DGokT8zd.js.map} +1 -1
- package/dist/cli.js +1 -1
- package/dist/experiment/index.d.ts +55 -1
- package/dist/experiment/index.d.ts.map +1 -1
- package/dist/experiment/index.js +119 -1
- package/dist/experiment/index.js.map +1 -1
- package/dist/openapi.json +1 -1
- package/docs/experiment.md +8 -0
- package/package.json +1 -1
package/docs/experiment.md
CHANGED
|
@@ -115,6 +115,14 @@ if (decision.refusal !== null) throw new Error(decision.refusal)
|
|
|
115
115
|
```
|
|
116
116
|
|
|
117
117
|
`cohortReceipt` binds the post-cutoff complete roster and the fixed observation snapshot to the prospective protocol.
|
|
118
|
+
`joinProspectiveCanaryReceipts` joins separate assignment, checker, execution, and billing rows before this decision.
|
|
119
|
+
The join is disabled unless the host explicitly enables it.
|
|
120
|
+
It rejects reused source units, assignments outside the witnessed window, orphan receipts, duplicate source IDs, and provider-call mismatches.
|
|
121
|
+
An unsettled customer cost remains `null`, including a failed provider dispatch with unknown actual cost.
|
|
122
|
+
The host must independently attest the historical source roster, pre-traffic witness, assignment completeness, randomization, and receipt authority.
|
|
123
|
+
The append-only assignment ledger covered by `assignmentLedgerTipDigest` must retain each source unit and assignment time for that attestation.
|
|
124
|
+
The join does not route traffic or turn a synthetic fixture into a live cohort.
|
|
125
|
+
`src/experiment/prospective-canary-join.test.ts` exercises a 40-assignment synthetic fixture through `decideRandomizedCanary`.
|
|
118
126
|
The deterministic calibration in `src/experiment/randomized-canary.test.ts` used 500 trials per law, 60 customers, and four sessions per customer.
|
|
119
127
|
Each customer had a shared pass propensity of 0.25 or 0.65 with equal probability.
|
|
120
128
|
Two sessions entered each arm in random order; the positive law added 0.20 to candidate pass probability.
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@tangle-network/agent-eval",
|
|
3
|
-
"version": "0.187.
|
|
3
|
+
"version": "0.187.2",
|
|
4
4
|
"description": "Evaluate and improve AI agents from runs, traces, judges, and feedback. Compare candidates, cluster failures, measure lift, and gate releases.",
|
|
5
5
|
"homepage": "https://github.com/tangle-network/agent-eval#readme",
|
|
6
6
|
"repository": {
|