@tangle-network/agent-eval 0.202.2 → 0.203.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +10 -0
- package/dist/adapters/http.d.ts +10 -4
- package/dist/adapters/http.d.ts.map +1 -1
- package/dist/adapters/http.js +43 -38
- package/dist/adapters/http.js.map +1 -1
- package/dist/{agent-profile-DDfIjdyS.d.ts → agent-profile-EpO0HURy.d.ts} +4 -4
- package/dist/{agent-profile-DDfIjdyS.d.ts.map → agent-profile-EpO0HURy.d.ts.map} +1 -1
- package/dist/analyst/index.d.ts +12 -12
- package/dist/analyst/index.d.ts.map +1 -1
- package/dist/analyst/index.js +6 -6
- package/dist/{analyze-runs-CPiKLJFs.d.ts → analyze-runs-Q74o4Hfj.d.ts} +2 -2
- package/dist/{analyze-runs-CPiKLJFs.d.ts.map → analyze-runs-Q74o4Hfj.d.ts.map} +1 -1
- package/dist/{benchmark-command-DFs_i62N.js → benchmark-command-6OBcB0sG.js} +7 -6
- package/dist/benchmark-command-6OBcB0sG.js.map +1 -0
- package/dist/benchmarks/index.d.ts +4 -4
- package/dist/benchmarks/index.js +3 -3
- package/dist/campaign/index.d.ts +6 -6
- package/dist/campaign/index.js +4 -4
- package/dist/{campaign-Btk1yFfw.js → campaign-BbObY2c6.js} +6 -6
- package/dist/{campaign-Btk1yFfw.js.map → campaign-BbObY2c6.js.map} +1 -1
- package/dist/{chat-client-DUhWNQQS.js → chat-client-5LLl4Ctl.js} +2 -2
- package/dist/{chat-client-DUhWNQQS.js.map → chat-client-5LLl4Ctl.js.map} +1 -1
- package/dist/{chat-json-call-Dx_sHzqm.js → chat-json-call-Digk-pfF.js} +2 -2
- package/dist/{chat-json-call-Dx_sHzqm.js.map → chat-json-call-Digk-pfF.js.map} +1 -1
- package/dist/cli.js +5 -5
- package/dist/contract/index.d.ts +10 -10
- package/dist/contract/index.js +7 -7
- package/dist/{default-registry-DesnDBvl.d.ts → default-registry-CxEBVA6W.d.ts} +6 -6
- package/dist/{default-registry-DesnDBvl.d.ts.map → default-registry-CxEBVA6W.d.ts.map} +1 -1
- package/dist/{default-registry-6QOiTzqi.js → default-registry-wInsQWlQ.js} +3 -3
- package/dist/{default-registry-6QOiTzqi.js.map → default-registry-wInsQWlQ.js.map} +1 -1
- package/dist/{define-agent-eval-BDPV3ieA.d.ts → define-agent-eval-BbEXdM3H.d.ts} +7 -7
- package/dist/{define-agent-eval-BDPV3ieA.d.ts.map → define-agent-eval-BbEXdM3H.d.ts.map} +1 -1
- package/dist/{define-agent-eval-CJN9dTQw.js → define-agent-eval-Da8mHyc7.js} +3 -3
- package/dist/{define-agent-eval-CJN9dTQw.js.map → define-agent-eval-Da8mHyc7.js.map} +1 -1
- package/dist/diagnosis/index.d.ts +2 -2
- package/dist/{dspy-rlm-engine-US8YsYGq.js → dspy-rlm-engine-D0h-P2Mz.js} +2 -2
- package/dist/{dspy-rlm-engine-US8YsYGq.js.map → dspy-rlm-engine-D0h-P2Mz.js.map} +1 -1
- package/dist/{engine-neXmlmTu.d.ts → engine-CaSqEVlL.d.ts} +4 -4
- package/dist/{engine-neXmlmTu.d.ts.map → engine-CaSqEVlL.d.ts.map} +1 -1
- package/dist/evaluation.d.ts +2 -2
- package/dist/{exact-types-KGxBWH4b.d.ts → exact-types-BBMMYk3g.d.ts} +2 -2
- package/dist/{exact-types-KGxBWH4b.d.ts.map → exact-types-BBMMYk3g.d.ts.map} +1 -1
- package/dist/experiment/index.d.ts +1 -1
- package/dist/{exporters-DjCu6-lX.d.ts → exporters-ChTnR90l.d.ts} +2 -2
- package/dist/{exporters-DjCu6-lX.d.ts.map → exporters-ChTnR90l.d.ts.map} +1 -1
- package/dist/{exporters-ISzpUY3e.js → exporters-X4xwla8A.js} +2 -2
- package/dist/{exporters-ISzpUY3e.js.map → exporters-X4xwla8A.js.map} +1 -1
- package/dist/{external-optimizer-process-KLcw0Vnc.js → external-optimizer-process-CIV4EqiT.js} +2 -2
- package/dist/{external-optimizer-process-KLcw0Vnc.js.map → external-optimizer-process-CIV4EqiT.js.map} +1 -1
- package/dist/{external-optimizer-subprocess-CkyUUV7h.js → external-optimizer-subprocess-BmKtH3T6.js} +2 -2
- package/dist/{external-optimizer-subprocess-CkyUUV7h.js.map → external-optimizer-subprocess-BmKtH3T6.js.map} +1 -1
- package/dist/{extract-usage-DFATEuWW.js → extract-usage-BrnzRJ6H.js} +4 -3
- package/dist/extract-usage-BrnzRJ6H.js.map +1 -0
- package/dist/{feedback-trajectory-BXCGTRkz.d.ts → feedback-trajectory-C6NJxXEa.d.ts} +3 -3
- package/dist/{feedback-trajectory-BXCGTRkz.d.ts.map → feedback-trajectory-C6NJxXEa.d.ts.map} +1 -1
- package/dist/{hf-dataset-DdT7nNkf.js → hf-dataset-BR2yhuNn.js} +3 -3
- package/dist/{hf-dataset-DdT7nNkf.js.map → hf-dataset-BR2yhuNn.js.map} +1 -1
- package/dist/{index-BCBpEreJ.d.ts → index-BoVEyQwH.d.ts} +4 -4
- package/dist/{index-BCBpEreJ.d.ts.map → index-BoVEyQwH.d.ts.map} +1 -1
- package/dist/{index-CnKgaHKA.d.ts → index-CGAwP9l3.d.ts} +8 -8
- package/dist/{index-CnKgaHKA.d.ts.map → index-CGAwP9l3.d.ts.map} +1 -1
- package/dist/{index-BtcxmabC.d.ts → index-CIkxRsLS.d.ts} +8 -8
- package/dist/{index-BtcxmabC.d.ts.map → index-CIkxRsLS.d.ts.map} +1 -1
- package/dist/{index-BPCdNSMW.d.ts → index-Ce2HACcP.d.ts} +7 -7
- package/dist/{index-BPCdNSMW.d.ts.map → index-Ce2HACcP.d.ts.map} +1 -1
- package/dist/index.d.ts +22 -22
- package/dist/index.js +13 -13
- package/dist/{integrity-atFlABhR.js → integrity-BV1ZDf42.js} +38 -6
- package/dist/{integrity-atFlABhR.js.map → integrity-BV1ZDf42.js.map} +1 -1
- package/dist/{integrity-CaeNeusS.d.ts → integrity-BWO6RPxv.d.ts} +2 -2
- package/dist/{integrity-CaeNeusS.d.ts.map → integrity-BWO6RPxv.d.ts.map} +1 -1
- package/dist/{jev-review-DX6MI_jq.d.ts → jev-review-Cw06AKuF.d.ts} +2 -2
- package/dist/{jev-review-DX6MI_jq.d.ts.map → jev-review-Cw06AKuF.d.ts.map} +1 -1
- package/dist/jev.d.ts +3 -3
- package/dist/{llm-client-Bhqoglar.js → llm-client-DmCNzrc9.js} +77 -60
- package/dist/llm-client-DmCNzrc9.js.map +1 -0
- package/dist/{llm-judge-C-0P1WiT.js → llm-judge-EUMHOZOQ.js} +4 -4
- package/dist/{llm-judge-C-0P1WiT.js.map → llm-judge-EUMHOZOQ.js.map} +1 -1
- package/dist/{matrix-DRXCV70M.d.ts → matrix-BqHiMKIj.d.ts} +2 -2
- package/dist/{matrix-DRXCV70M.d.ts.map → matrix-BqHiMKIj.d.ts.map} +1 -1
- package/dist/meta-eval/index.d.ts +1 -1
- package/dist/{mint-BFTrlq4t.d.ts → mint-C1W8w5wo.d.ts} +2 -2
- package/dist/{mint-BFTrlq4t.d.ts.map → mint-C1W8w5wo.d.ts.map} +1 -1
- package/dist/{mint-DE6nQrX_.js → mint-CWG-s2mI.js} +2 -2
- package/dist/{mint-DE6nQrX_.js.map → mint-CWG-s2mI.js.map} +1 -1
- package/dist/multishot/golden/index.d.ts +1 -1
- package/dist/multishot/index.d.ts +2 -2
- package/dist/openapi.json +1 -1
- package/dist/{prime-protocol-T0i6xcAs.d.ts → prime-protocol-IoN4Qb6c.d.ts} +2 -2
- package/dist/{prime-protocol-T0i6xcAs.d.ts.map → prime-protocol-IoN4Qb6c.d.ts.map} +1 -1
- package/dist/{produced-state-CR2tAPLf.js → produced-state-g9BPgu4D.js} +3 -3
- package/dist/{produced-state-CR2tAPLf.js.map → produced-state-g9BPgu4D.js.map} +1 -1
- package/dist/{abort-signal-CtzAM_sJ.js → proposal-findings-CXblsskH.js} +2 -11
- package/dist/proposal-findings-CXblsskH.js.map +1 -0
- package/dist/{registry-Bu3lNole.d.ts → registry-CT-hMSkR.d.ts} +4 -4
- package/dist/{registry-Bu3lNole.d.ts.map → registry-CT-hMSkR.d.ts.map} +1 -1
- package/dist/{report-command-DzPpFgjB.js → report-command-D0XL2Cvi.js} +128 -15
- package/dist/report-command-D0XL2Cvi.js.map +1 -0
- package/dist/{researcher-DojEMb7E.d.ts → researcher-zTpxNJBy.d.ts} +3 -3
- package/dist/{researcher-DojEMb7E.d.ts.map → researcher-zTpxNJBy.d.ts.map} +1 -1
- package/dist/rl.d.ts +7 -7
- package/dist/rl.js +3 -3
- package/dist/rollout/index.d.ts +5 -5
- package/dist/rollout/index.js +5 -5
- package/dist/{rollout-DTunVdJE.js → rollout-BdCoWV9n.js} +5 -5
- package/dist/{rollout-DTunVdJE.js.map → rollout-BdCoWV9n.js.map} +1 -1
- package/dist/{schema-D08IIzui.js → schema-BuiujZ-g.js} +15 -5
- package/dist/schema-BuiujZ-g.js.map +1 -0
- package/dist/{schema-Bp8S20Ii.d.ts → schema-C-ziyJD7.d.ts} +17 -2
- package/dist/schema-C-ziyJD7.d.ts.map +1 -0
- package/dist/search/index.d.ts +2 -2
- package/dist/{search-kernel-zI-rsHoG.d.ts → search-kernel-C5jua_UB.d.ts} +2 -2
- package/dist/{search-kernel-zI-rsHoG.d.ts.map → search-kernel-C5jua_UB.d.ts.map} +1 -1
- package/dist/{semantic-concept-judge-BcjYg37z.js → semantic-concept-judge-BgWxa4L_.js} +4 -4
- package/dist/{semantic-concept-judge-BcjYg37z.js.map → semantic-concept-judge-BgWxa4L_.js.map} +1 -1
- package/dist/{series-convergence-DY7ha0kx.d.ts → series-convergence-DLy0bQMV.d.ts} +2 -2
- package/dist/{series-convergence-DY7ha0kx.d.ts.map → series-convergence-DLy0bQMV.d.ts.map} +1 -1
- package/dist/{server-B4jMhqQn.js → server-CoBZO1fj.js} +2 -2
- package/dist/{server-B4jMhqQn.js.map → server-CoBZO1fj.js.map} +1 -1
- package/dist/{skillopt-optimization-method-Bls9t-G_.js → skillopt-optimization-method-DZrdQGZl.js} +5 -5
- package/dist/{skillopt-optimization-method-Bls9t-G_.js.map → skillopt-optimization-method-DZrdQGZl.js.map} +1 -1
- package/dist/{statistical-heldout-BAbd_oXC.d.ts → statistical-heldout-CJYAO-4a.d.ts} +2 -2
- package/dist/{statistical-heldout-BAbd_oXC.d.ts.map → statistical-heldout-CJYAO-4a.d.ts.map} +1 -1
- package/dist/{store-tool-spans-Biio4_Nw.js → store-tool-spans-D5kOU0mr.js} +2 -2
- package/dist/{store-tool-spans-Biio4_Nw.js.map → store-tool-spans-D5kOU0mr.js.map} +1 -1
- package/dist/{store-tool-spans-CKRw7JHa.d.ts → store-tool-spans-fIE31bU8.d.ts} +5 -5
- package/dist/{store-tool-spans-CKRw7JHa.d.ts.map → store-tool-spans-fIE31bU8.d.ts.map} +1 -1
- package/dist/supervisor-run/index.d.ts +2 -2
- package/dist/supervisor-run/index.d.ts.map +1 -1
- package/dist/supervisor-run/index.js +2 -2
- package/dist/terminal-record-BHocwFw1.js.map +1 -1
- package/dist/{tool-groups-V23aGP_g.d.ts → tool-groups-DP8b5kjF.d.ts} +3 -3
- package/dist/tool-groups-DP8b5kjF.d.ts.map +1 -0
- package/dist/{trace-questions-B-Wf0W3Q.d.ts → trace-questions-Cd0Fyhbv.d.ts} +3 -3
- package/dist/{trace-questions-B-Wf0W3Q.d.ts.map → trace-questions-Cd0Fyhbv.d.ts.map} +1 -1
- package/dist/trace-repair/index.d.ts +1 -1
- package/dist/traces.d.ts +5 -5
- package/dist/traces.js +2 -2
- package/dist/{types-CeDfCRXp.d.ts → types-6VEehmEg.d.ts} +2 -2
- package/dist/{types-CeDfCRXp.d.ts.map → types-6VEehmEg.d.ts.map} +1 -1
- package/dist/{types-C0bWoSKC.d.ts → types-DLxe96rp.d.ts} +2 -2
- package/dist/{types-C0bWoSKC.d.ts.map → types-DLxe96rp.d.ts.map} +1 -1
- package/dist/{types-BR1LLIxm.d.ts → types-Duc6xOoW.d.ts} +4 -4
- package/dist/{types-BR1LLIxm.d.ts.map → types-Duc6xOoW.d.ts.map} +1 -1
- package/dist/{types-BbiT5NPH.d.ts → types-hH9APahd.d.ts} +2 -2
- package/dist/{types-BbiT5NPH.d.ts.map → types-hH9APahd.d.ts.map} +1 -1
- package/dist/{types-DUlbOCwL.d.ts → types-v6nRcIqg.d.ts} +11 -3
- package/dist/types-v6nRcIqg.d.ts.map +1 -0
- package/dist/wire/index.d.ts +2 -2
- package/dist/wire/index.js +1 -1
- package/docs/changes/llm-physical-attempts.md +30 -0
- package/docs/changes/shared-model-request-lifetime.md +30 -0
- package/docs/concepts.md +8 -0
- package/docs/http-dispatch.md +33 -0
- package/docs/proofs/router-cache-usage-20261001/after.json +2058 -0
- package/docs/proofs/router-cache-usage-20261001/component-projections.json +584 -0
- package/docs/proofs/router-cache-usage-20261001/component-projections.nested-only.json +306 -0
- package/docs/proofs/router-cache-usage-20261001/manifest.json +32 -0
- package/docs/proofs/router-cache-usage-20261001/summary.json +49 -0
- package/docs/proofs/router-cache-usage-20261001/summary.nested-only.json +29 -0
- package/docs/rollout.md +32 -0
- package/package.json +1 -1
- package/dist/abort-signal-CtzAM_sJ.js.map +0 -1
- package/dist/benchmark-command-DFs_i62N.js.map +0 -1
- package/dist/extract-usage-DFATEuWW.js.map +0 -1
- package/dist/llm-client-Bhqoglar.js.map +0 -1
- package/dist/report-command-DzPpFgjB.js.map +0 -1
- package/dist/schema-Bp8S20Ii.d.ts.map +0 -1
- package/dist/schema-D08IIzui.js.map +0 -1
- package/dist/tool-groups-V23aGP_g.d.ts.map +0 -1
- package/dist/types-DUlbOCwL.d.ts.map +0 -1
|
@@ -1,11 +1,11 @@
|
|
|
1
1
|
import { S as PaidCallResult, T as RunPaidCallInput, a as CostChannel, c as CostLedgerHandle, f as CostLedgerSummary, p as CostProvenance } from "./cost-ledger-DbQdN3nO.js";
|
|
2
2
|
import { d as RunTokenUsage, s as RunSearchCoordinates } from "./run-record-DF-fuu31.js";
|
|
3
|
-
import { _ as ProposalFinding } from "./types-
|
|
4
|
-
import { M as LlmCallMetadata } from "./types-
|
|
3
|
+
import { _ as ProposalFinding } from "./types-6VEehmEg.js";
|
|
4
|
+
import { M as LlmCallMetadata } from "./types-DLxe96rp.js";
|
|
5
5
|
import { t as SeriesDistribution } from "./descriptive-B2iPaT9J.js";
|
|
6
6
|
import { A as SearchLedgerHash, G as SearchSplit, I as SearchNodeStatus, L as SearchOpenedEvent, N as SearchNodeDecision, O as SearchLedgerEntry, P as SearchNodeRef, Q as SearchTaskOutcome, R as SearchOperationKind, U as SearchReservation, Z as SearchTask, b as SearchClosedEvent, d as SearchCellCancelledEvent, i as SearchArtifactRef, l as SearchCandidateSurface, o as SearchAudit, p as SearchCellStage, t as NodeEstimate, w as SearchEdgeRecordedEvent, z as SearchOperationRecordedEvent } from "./search-ledger-types-p2xKfiDJ.js";
|
|
7
7
|
import { p as LedgerProjector } from "./index-Bj96W0jZ.js";
|
|
8
|
-
import { i as RolloutSearchLineage } from "./mint-
|
|
8
|
+
import { i as RolloutSearchLineage } from "./mint-C1W8w5wo.js";
|
|
9
9
|
//#region src/campaign/search-state.d.ts
|
|
10
10
|
/** Deterministic node id: identical content in one search is one node. */
|
|
11
11
|
declare function searchNodeId(searchId: string, artifactDigest: string): string;
|
|
@@ -1009,4 +1009,4 @@ interface CampaignResult<TArtifact = unknown, TScenario extends Scenario = Scena
|
|
|
1009
1009
|
}
|
|
1010
1010
|
//#endregion
|
|
1011
1011
|
export { SearchNode as $, LabeledScenarioWrite as A, ScoredSurfaceOutcome as B, JudgeDimension as C, LabeledScenarioSampleArgs as D, LabeledScenarioRecord as E, ProposeContext as F, labelTrustRank as G, SurfaceProposer as H, ProposedCandidate as I, renderSearchSummary as J, SearchProposerView as K, RedactionStatus as L, OptimizerConfig as M, ParetoParent as N, LabeledScenarioSource as O, ProposalTrackContext as P, SearchDecisionRecord as Q, Scenario as R, JudgeConfig as S, LabelTrust as T, TraceSpan as U, SessionScript as V, isProposedCandidate as W, SearchCell as X, searchProposerView as Y, SearchCompletion as Z, GateDecision as _, CampaignResult as a, SearchUnitScore as at, GenerationRecord as b, CampaignTraceWriter as c, searchTaskSetDigest as ct, DispatchContext as d, SearchOperation as et, DispatchFn as f, GateContribution as g, GateContext as h, CampaignCostMeter as i, SearchStateView as it, MutableSurface as j, LabeledScenarioStore as k, CodeSurface as l, searchUnitScores as lt, GateCheckStatus as m, CampaignArtifactWriter as n, SearchSpend as nt, CampaignScenarioIdentity as o, searchCellId as ot, Gate as p, SearchSummaryOptions as q, CampaignCellResult as r, SearchState as rt, CampaignTokenUsage as s, searchNodeId as st, CampaignAggregates as t, SearchScoredCell as tt, ComponentSurface as u, GateResult as v, JudgeScore as w, JudgeAggregate as x, GenerationCandidate as y, ScenarioAggregate as z };
|
|
1012
|
-
//# sourceMappingURL=types-
|
|
1012
|
+
//# sourceMappingURL=types-Duc6xOoW.d.ts.map
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"types-
|
|
1
|
+
{"version":3,"file":"types-Duc6xOoW.d.ts","names":[],"sources":["../src/campaign/search-state.ts","../src/campaign/search-summary.ts","../src/campaign/types.ts"],"mappings":";;;;;;;;;;iBAwDgB,aAAa,kBAAkB;;iBAK/B,aACd,kBACA,gBACA,gBACA,OAAO,aACP;;iBAMc,oBAAoB,gBAAgB,eAAe;UAkBlD;EACf;EACA,UAAU;EACV,OAAO;EACP;EACA;;UAGe;EACf;EACA;EACA;EACA;;;;UAKe;EACf;EACA;EACA;EACA;;;UAIe;EACf;EACA;;EAEA;EACA;;;;;;;;iBASc,iBAAiB,gBAAgB,qBAAqB;UAoBrD;EACf;;EAEA;EACA,gBAAgB;EAChB,UAAU;EACV,mBAAmB;;;EAGnB;;EAEA;;;;EAIA,kBAAkB;EAClB;EACA;EACA,QAAQ;EACR;;EAEA;EACA,oBAAoB;EACpB;EACA,OAAO;EACP;EACA;;UAGe;EACf;EACA;EACA;EACA;EACA,OAAO;EACP;EACA,OAAO;EACP;EACA,aAAa;EACb;EACA;EACA,SAAS;;EAET;;EAEA;EACA,WAAW;;EAEX;;EAEA;EACA;EACA;EACA;;UAGe;EACf;EACA,eAAe;EACf,aAAa;EACb;;EAEA,SAAS;;;EAGT,WAAW;;;EAGX,oBAAoB;;EAEpB;;EAEA;;UAGe;EACf;EACA;;;;;;;cA6BW,uBAAuB,gBAAgB,mBAAmB;WAC5D;UACD;UACA;UACA;UACA;UACA;UACA;mBACS;mBACA;mBACA;mBACA;mBACA;mBACA;mBACA;mBACA;UACT;UACA;;mBAES;;UAET;mBACS;EAEjB,YAAY;EAgCZ,MAAM,OAAO,mBAAmB;EA6BhC,YAAY;;EAKZ,KAAK;UAIG;UAgCA;UA6CA;UAsBA;UA8BA;UA8CA;UAsHA;UAmEA;UA4EA;UAoBA;UAmDA;UAaA;;;UA2EA;;EASR;;;UAmBQ;;;UAkBA;;;;;;;;;;;UAoBA;;;EAsBR;IAAgB;IAA4B;;;EAa5C,cAAc;;EAKd,cAAc;;EAKd;IAAc;IAAkB,WAAW;;;EAK3C;;EAKA,aAAa;;EAcb,SAAS,iBAAiB;;EAM1B;;EAKA,YAAY;;EAKZ,oBAAoB;;EAKpB,SAAS,iBAAiB;;EAK1B;;EAKA,SAAS,iBAAiB;;EAM1B,YAAY;;EAMZ,cAAc,sBAAsB;;EAMpC,gBAAgB,gBAAgB,OAAO,cAAc;;;;;;;cAiB1C;WACF;WACA,QAAQ;WACR,QAAQ;WACR;IAAQ;IAAkB,WAAW;;;WAErC;WACA,OAAO;WACP,YAAY;;;;;;;WAOZ;IAAU;IAA4B;;mBAC9B;mBACA;EAEjB,YAAY,OAAO,aAAa;;MAgB5B;EAIJ,KAAK,iBAAiB;EAItB,QAAQ;;EAKR;;EAKA,SAAS;EAKT,gBAAgB;EAIhB,KAAK,iBAAiB;;EAKtB,SAAS;EAKT,KAAK,iBAAiB;;EAKtB,MAAM;IAAU;MAAyB;EAKzC,UAAU,sBAAsB;;;EAMhC,YAAY,gBAAgB,OAAO,cAAc;;EAKjD,WAAW,gBAAgB,OAAO,cAAc;;EAKhD,QAAQ,aAAa,uBAAuB;UAmCpC;;;;;;UCluCO;WACN;WACA;;;EAGT,YAAY,0BAA0B;;EAEtC,WAAW,0BAA0B;;;iBAIvB,mBAAmB,OAAO,kBAAkB;UAW3C;;;;;;;;EAQf,OAAO;;EAEP;;;;;;;;;;;iBAYc,oBAAoB,OAAO,iBAAiB,SAAS;;;;;UCtCpD;EACf;EACA;EACA;;;;;EAKA;;;UAIe,iCAAiC,KAAK;EACrD;;;;;UAMe;EACf;;EAEA;EACA;EACA;EACA;EACA,QAAQ;EACR,OAAO;EACP,WAAW;EACX,MAAM;;EAEN;;EAEA;;;;;;;;EAQA;;;;KAKU,WAAW,kBAAkB,UAAU,cACjD,UAAU,WACV,KAAK,oBACF,QAAQ;;;;UAOI,cAAc,WAAW;EACxC;EACA;EACA;;EAEA;;;EAGA,sBAAsB,UAAU,WAAW,sBAAsB,UAAU,cAAc;;UAK1E;;EAEf;;EAEA;;;;;;;;;UAUe,YAAY,WAAW,kBAAkB,WAAW;EACnE;EACA,YAAY;;;;EAIZ;;;EAGA,MAAM;IACJ,UAAU;IACV,UAAU;IACV,QAAQ;;IAER,aAAa;IACb;IACA,WAAW;MACT,aAAa,QAAQ;EACzB,aAAa,UAAU;;;;;;;;;;;UAYR;EACf,YAAY;EACZ;EACA;;EAEA,UAAU;;;;;;;EAOV;;;;EAIA,eAAe,eAAe;IAAQ;IAAe;;;;;EAIrD;;;EAGA;;EAEA;;EAEA,WAAW,eAAe;;;;;;;UAUX;WACN;;;WAGA;;WAEA;;WAEA;;WAEA;;WAEA;;WAEA;;;WAGA;aACE;aACA;aACA;;;WAGF;;;UAIM;WACN;WACA,YAAY,SAAS;;;;;;;;;;;KAYpB,0BAA0B,mBAAmB;;;;;;;UAQxC;EACf,SAAS;;EAET;;;;EAIA;;;;;;EAMA,cAAc,SAAS;;;;iBAKT,oBACd,OAAO,iBAAiB,oBACvB,SAAS;;;;;;;;;UAkBK;EACf,SAAS;EACT;;;EAGA,YAAY;;;EAGZ;;EAEA;EACA;EACA;;;UAIe;EACf;EACA;EACA;EACA;EACA;;;;;UAMe;;;EAGf;;EAEA;EACA;EACA;EACA,YAAY;;;;;EAKZ,WAAW;IAAQ;IAAoB;IAAmB;IAAgB;;EAC1E;IACE;IACA;;;;;UAMa,eAAe,YAAY;;;;WAIjC,gBAAgB;;;;;WAKhB;;;;WAIA,UAAU;IAAgB;IAAgB,UAAU;;;;WAGpD,QAAQ;;;;WAIR;;;WAGA,SAAS,cAAc;WACvB,UAAU,cAAc;;WAExB;WACA;WACA,QAAQ;;WAER,QAAQ;;;WAGR,kBAAkB;;;WAGlB,mBAAmB;;;;WAInB,gBAAgB;;;;WAIhB;;;;;;;WAOA,gBAAgB,cAAc;;WAE9B,aAAa;WACb;;;;;;;;;;;;;;UAeM,gBAAgB,YAAY;EAC3C;;;;;EAKA,QAAQ,KAAK,eAAe,aAAa,QAAQ,MAAM,iBAAiB;;;EAGxE,QAAQ;IAAQ,SAAS,cAAc;;IAAwB;IAAe;;;UAG/D;EACf;EACA;EACA,mBAAmB,qBAAqB;;UAGzB,wBAAwB;EACvC,UAAU;;;KAMA;;KAGA;UAEK;EACf;EACA,QAAQ;EACR;;UAGe,YAAY,WAAW,kBAAkB;EACxD,oBAAoB,YAAY;EAChC,oBAAoB,YAAY;;EAEhC,aAAa,YAAY,eAAe;;;;;EAKxC,sBAAsB,YAAY,eAAe;;;;;;;EAOjD,yBAAyB,YAAY,eAAe;;;EAGpD,uBAAuB,YAAY;EACnC,WAAW;EACX;IAAQ;IAAmB;;;EAE3B,aAAa;EACb;EACA,QAAQ;;UAGO;EACf,UAAU;EACV;EACA,mBAAmB;EACnB;;;UAIe,KAAK,qBAAqB,kBAAkB,WAAW;EACtE;EACA,OAAO,KAAK,YAAY,WAAW,aAAa,QAAQ;;;;UAOzC;EACf,KAAK,cAAc,aAAa,0BAA0B;EAC1D,SAAS;;UAGM;EACf,IAAI,aAAa;EACjB,aAAa,aAAa;;;;UAKX;EACf,MAAM,cAAc,kBAAkB,aAAa;EACnD,UAAU,cAAc,iBAAiB;;;;;;KAO/B,qBAAqB;;;;;UAMhB;;EAEf,YAAY,GACV,OAAO,KAAK,iBAAiB;IAC3B,UAAU;MAEX,QAAQ,eAAe;;;;;KAQhB;KAOA;;;;;;;;;;;;;;;KAgBA;;iBASI,eAAe,OAAO;;;;UAOrB,qBAAqB,kBAAkB,WAAW,UAAU;EAC3E,UAAU;EACV,UAAU;EACV,aAAa,eAAe;EAC5B,QAAQ;EACR;EACA;EACA,iBAAiB;;;;;EAKjB,aAAa;;EAEb;;UAGe,sBAAsB,kBAAkB,WAAW,UAAU,6BACpE,qBAAqB,WAAW;;EAExC;;;EAGA;;UAGe;EACf;;EAEA;;;;EAIA;EACA;IACE;IACA,SAAS,wBAAwB;IACjC;IACA;;;;;IAKA,WAAW;;;UAIE;EACf,QAAQ,OAAO,uBAAuB;EACtC,OAAO,MAAM,4BAA4B,QAAQ;EACjD,QAAQ;IACN;IACA;IACA,UAAU;;;IAGV,SAAS,OAAO;;;UAMH,mBAAmB;;;EAGlC;EACA;EACA;EACA;EACA;EACA,UAAU;EACV,aAAa,eAAe;;;EAG5B;;EAEA,gBAAgB;;EAEhB;;;EAGA,YAAY;;;EAGZ;;;EAGA;EACA;EACA;EACA;;;;EAIA;;EAEA;;EAEA;EACA;;UAGe;EACf;EACA;EACA;EACA;;;;;;EAMA,cAAc;;UAGC;EACf;EACA;EACA;;;EAGA,cAAc;;UAGC;EACf;EACA,YAAY;EACZ;;;;;;UAOe;EACf;;;;EAIA;;;;EAIA;;;;EAIA;IACE;IACA;IACA,iBAAiB;MAAQ;MAAgB;;;;;EAI3C,YAAY;;;;;;;;;;EAUZ,WAAW;IAAQ;IAAoB;IAAmB;IAAgB;;;;EAG1E;;;;EAIA;;;;EAIA,cAAc,SAAS;;UAGR;EACf,SAAS,eAAe;EACxB,YAAY,eAAe;;EAE3B,MAAM;;EAEN;EACA;EACA;;EAEA;;EAEA;;EAEA;;EAEA;;UAGe,eAAe,qBAAqB,kBAAkB,WAAW;;EAEhF;;EAEA;EACA;;EAEA;EACA;EACA;EACA;EACA,OAAO,MAAM,mBAAmB;EAChC,YAAY;EACZ;IACE,aAAa;IACb;;EAEF,OAAO;EACP;EACA;EACA,iBAAiB;;;EAGjB,WAAW,MAAM,2BAA2B,KAAK"}
|
|
@@ -1,5 +1,5 @@
|
|
|
1
1
|
import { c as RunSplitTag } from "./run-record-DF-fuu31.js";
|
|
2
|
-
import { R as Scenario, d as DispatchContext } from "./types-
|
|
2
|
+
import { R as Scenario, d as DispatchContext } from "./types-Duc6xOoW.js";
|
|
3
3
|
//#region src/benchmarks/types.d.ts
|
|
4
4
|
type BenchmarkTaskKind = 'retrieval' | 'rag-answer' | 'hallucination' | 'kb-improvement' | 'routing' | 'custom';
|
|
5
5
|
type BenchmarkFamily = 'beir' | 'mteb-retrieval' | 'msmarco' | 'trec-dl' | 'miracl' | 'lotte' | 'bright' | 'crag' | 'hotpotqa' | 'kilt' | 'ragtruth' | 'faithbench' | 'first-party' | 'custom';
|
|
@@ -99,4 +99,4 @@ declare const BENCHMARK_SPLIT_SEED = "agent-eval-v1";
|
|
|
99
99
|
declare function deterministicSplit(itemId: string, seed?: string): RunSplitTag;
|
|
100
100
|
//#endregion
|
|
101
101
|
export { BenchmarkFamily as a, BenchmarkSource as c, BenchmarkEvaluation as i, BenchmarkTaskKind as l, BenchmarkAdapter as n, BenchmarkResponder as o, BenchmarkDatasetItem as r, BenchmarkScenario as s, BENCHMARK_SPLIT_SEED as t, deterministicSplit as u };
|
|
102
|
-
//# sourceMappingURL=types-
|
|
102
|
+
//# sourceMappingURL=types-hH9APahd.d.ts.map
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"types-
|
|
1
|
+
{"version":3,"file":"types-hH9APahd.d.ts","names":[],"sources":["../src/benchmarks/types.ts"],"mappings":";;;KASY;KAQA;UAgBK,qBAAqB;;;EAGpC;;EAEA,SAAS;;EAET,QAAQ;;EAER,SAAS;;EAET,WAAW;;EAEX;;EAEA,SAAS;EACT,WAAW;;UAGI;;;;EAIf;;EAEA;;EAEA,aAAa;;;EAGb,KAAK;EACL;;UAGe;EACf;EACA;EACA;;EAEA;EACA;EACA;;EAEA;;;;;EAKA;;;UAQe,iBAAiB,kBAAkB,oBAAoB;;EAEtE;EACA,SAAS;EACT,WAAW;EACX;EACA,SAAS;EACT;;;;;EAKA,YAAY,OAAO,cAAc,QAAQ,qBAAqB;;EAE9D,SAAS,MAAM,qBAAqB,WAAW,UAAU,YAAY,QAAQ;;;;EAI7E,YAAY,iBAAiB;;UAGd,kBAAkB,4BAA4B;EAC7D;EACA;EACA,QAAQ;EACR,UAAU;EACV,UAAU;EACV,MAAM,qBAAqB;;KAGjB,mBAAmB,oBAAoB,uBAAuB;EACxE,UAAU,kBAAkB;EAC5B,MAAM,qBAAqB;EAC3B,SAAS;MACL,QAAQ,aAAa;;;cAoBd;;;;;;;;;iBAUG,mBACd,gBACA,gBACC"}
|
|
@@ -1,5 +1,5 @@
|
|
|
1
1
|
import { t as SeriesDistribution } from "./descriptive-B2iPaT9J.js";
|
|
2
|
-
import { h as RolloutLine } from "./schema-
|
|
2
|
+
import { C as RolloutWorkspaceCapture, h as RolloutLine } from "./schema-C-ziyJD7.js";
|
|
3
3
|
//#region src/supervisor-run/types.d.ts
|
|
4
4
|
/** A metric that could not be computed, with the reason its artifact was missing. */
|
|
5
5
|
interface Unavailable {
|
|
@@ -58,6 +58,8 @@ interface WorkerLogSource {
|
|
|
58
58
|
* store kept no receipt for this worker.
|
|
59
59
|
*/
|
|
60
60
|
readonly nativeSession?: WorkerNativeSession | null;
|
|
61
|
+
/** Capture receipts in output order. Null means none were retained; absent means this reader has no channel. */
|
|
62
|
+
readonly workspaceCaptures?: readonly RolloutWorkspaceCapture[] | null;
|
|
61
63
|
/** Where this worker's delivered patch lives. Null = the store keeps no patch per worker. */
|
|
62
64
|
readonly patchPath?: string | null;
|
|
63
65
|
/** This worker's own inference tokens, when the store records them per worker. */
|
|
@@ -176,6 +178,8 @@ interface SupervisorRunSources {
|
|
|
176
178
|
* retain one must set this to `null`, or omit the path entirely.
|
|
177
179
|
*/
|
|
178
180
|
readonly rootTranscriptRef?: string | null;
|
|
181
|
+
/** Root capture receipts in output order; null means no retained receipt. */
|
|
182
|
+
readonly rootWorkspaceCaptures?: readonly RolloutWorkspaceCapture[] | null;
|
|
179
183
|
/**
|
|
180
184
|
* The `traces` CLI command that covers this run's harness-session layer.
|
|
181
185
|
* Null falls back to the analyzer's default (an opencode worker fleet).
|
|
@@ -303,6 +307,8 @@ interface PerWorkerRow {
|
|
|
303
307
|
readonly passed: boolean | null;
|
|
304
308
|
/** Numeric verdict score exactly as recorded; null means no score was recorded. */
|
|
305
309
|
readonly score: number | null;
|
|
310
|
+
/** Exact Runtime capture receipts in journal order; null means capture evidence is unknown. */
|
|
311
|
+
readonly workspaceCaptures: readonly RolloutWorkspaceCapture[] | null;
|
|
306
312
|
}
|
|
307
313
|
/**
|
|
308
314
|
* `SeriesDistribution` (from `../statistics`) over per-worker wall
|
|
@@ -471,6 +477,8 @@ interface SupervisorRunReport {
|
|
|
471
477
|
readonly decision: DecisionMetrics;
|
|
472
478
|
readonly economics: EconomicsMetrics;
|
|
473
479
|
readonly outcome: OutcomeMetrics;
|
|
480
|
+
/** Exact root capture receipts; null means capture evidence is unknown. */
|
|
481
|
+
readonly rootWorkspaceCaptures: readonly RolloutWorkspaceCapture[] | null;
|
|
474
482
|
/** Artifacts that were missing, in read order — the provenance of every `unavailable`. */
|
|
475
483
|
readonly gaps: readonly string[];
|
|
476
484
|
/** The `traces` CLI command that covers the harness-session layer for this run. */
|
|
@@ -537,7 +545,7 @@ interface SupervisorRunTree {
|
|
|
537
545
|
readonly gaps: readonly SupervisorRunTreeGap[];
|
|
538
546
|
}
|
|
539
547
|
/** Stable machine-readable reasons emitted while minting a supervisor tree. */
|
|
540
|
-
type SupervisorRunTreeGapCode = 'journal-unavailable' | 'source-row-malformed' | 'root-spawn-unavailable' | 'root-reward-unavailable' | 'child-reward-unavailable' | 'node-role-unavailable' | 'node-schema-invalid';
|
|
548
|
+
type SupervisorRunTreeGapCode = 'journal-unavailable' | 'source-row-malformed' | 'root-spawn-unavailable' | 'root-reward-unavailable' | 'child-reward-unavailable' | 'node-role-unavailable' | 'workspace-capture-unavailable' | 'workspace-capture-incomplete' | 'node-schema-invalid';
|
|
541
549
|
interface SupervisorRunTreeGap {
|
|
542
550
|
readonly code: SupervisorRunTreeGapCode;
|
|
543
551
|
readonly message: string;
|
|
@@ -546,4 +554,4 @@ interface SupervisorRunTreeGap {
|
|
|
546
554
|
}
|
|
547
555
|
//#endregion
|
|
548
556
|
export { WallDistribution as A, SupervisorRunSources as C, SupervisorStatusSource as D, SupervisorRunTreeGapCode as E, showMeasured as F, unavailable as I, WorkerNativeSession as M, WorkerTurnCoverage as N, TerminalFailure as O, isUnavailable as P, SupervisorRunRollup as S, SupervisorRunTreeGap as T, SpendMeasurements as _, NamedResourceReceipt as a, SupervisorRunReader as b, PatchStats as c, RoleSpend as d, RollupCellRow as f, SpendMeasurement as g, SourceLimits as h, NO_SOURCE_LIMITS as i, WorkerLogSource as j, Unavailable as k, PerWorkerRow as l, SUPERVISOR_RUN_SCHEMA as m, EconomicsMetrics as n, OrchestrationMetrics as o, SUPERVISOR_RUN_ROLLUP_SCHEMA as p, Measured as r, OutcomeMetrics as s, DecisionMetrics as t, ResourceSpendRecord as u, SteerBreakdown as v, SupervisorRunTree as w, SupervisorRunReport as x, SupervisorRunNodeRole as y };
|
|
549
|
-
//# sourceMappingURL=types-
|
|
557
|
+
//# sourceMappingURL=types-v6nRcIqg.d.ts.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"types-v6nRcIqg.d.ts","names":[],"sources":["../src/supervisor-run/types.ts"],"mappings":";;;;UAkCiB;WACN;;;KAIC,SAAS,KAAK,IAAI;iBAEd,YAAY,iBAAiB;iBAI7B,cAAc,aAAa,KAAK;;iBAKhC,aAAa,GAAG;;KAWpB;;UAGK;WACN;WACA;;;KAIC;WACG;WAA8B;;WAC9B;WAAgC;;;;;;UAM9B;;;;;WAKN;;WAEA;;WAEA;;WAEA;;WAEA;;WAEA;;;;;;WAMA,QAAQ;;;;;;WAMR,gBAAgB;;WAEhB,6BAA6B;;WAE7B;;WAEA;WACA;WACA;WACA;;;;;;;;;;;;;UAcM;;WAEN;;WAEA;;WAEA;;WAEA;;WAEA;;;cAIE,kBAAkB;;;;;;;;;;UAiBd;;WAEN;WACA;;WAEA;;WAEA;;;;;;WAMA;;;;;;WAMA;;WAEA;;WAEA;;WAEA;;WAEA;;WAEA,kBAAkB;;WAElB;;;;WAIA;;;;;;WAMA;;;;;;;WAOA;;WAEA;;WAEA;;WAEA;;;;;WAKA;IACP;IACA;IACA;IACA;;IAEA;IACA;;WAEO;;WAEA,QAAQ;;;;;WAKR;;WAEA,iCAAiC;;;;;WAKjC;;;;;;UAOM;;WAEN;EACT,QAAQ,QAAQ;;cAOL;cACA;UAEI;;WAEN;WACA;;WAEA;;WAEA;;UAGM;WACN,gBAAgB;WAChB,gBAAgB;WAChB,kBAAkB;;WAElB,QAAQ;WACR,iBAAiB;WACjB,gBAAgB,kBAAkB;;WAElC,kBAAkB;;;;;;WAMlB,OAAO;WACP,WAAW;WACX,gBAAgB;;WAEhB,UAAU;;WAEV,gBAAgB;;WAEhB,iBAAiB;WACjB,oBAAoB;WACpB,kBAAkB;;;;;;;;;WASlB,sBAAsB;;WAEtB,QAAQ;WACR,SAAS;;WAET,mBAAmB;;UAGb;WACN,iBAAiB,SAAS;WAC1B,iBAAiB,SAAS;;;;;;;WAO1B,UAAU;;WAEV,UAAU;;WAEV,WAAW;;WAEX,oBAAoB;;WAEpB,wBAAwB;;WAExB,eAAe;WACf,qBAAqB;;UAGf;WACN,UAAU;WACV,WAAW;;;;;;WAMX,WAAW;WACX,YAAY;WACZ,KAAK;WACL;;UAGM;;WAEN;WACA;;WAEA,MAAM;;WAEN;;WAEA;;WAEA;;WAEA;;WAEA;WACA;;WAEA;WACA;WACA;WACA;WACA;;WAEA;;WAEA,4BAA4B;;;;;;;KAQ3B,mBAAmB;;UAGd;WACN,KAAK;;WAEL;;;;;;;WAOA;;WAEA;;WAEA;;;;;;;;;;;;;UAcM;WACN,gBAAgB;WAChB,aAAa;;;UAIP;WACN,MAAM;WACN,MAAM;WACN,QAAQ;WACR,OAAO;;;UAID;WACN;WACA;WACA;WACA,WAAW,kBAAkB;;UAGvB;;WAEN,kBAAkB,kBAAkB;;WAEpC,OAAO;;;;;;;;WAQP,kBAAkB;;WAElB,SAAS;;WAET,OAAO;;;;;;;;WAQP,UAAU;;;;;;WAMV;WACA,yBAAyB;WACzB,0BAA0B,SAAS;WACnC,WAAW,kBAAkB;;UAGvB;WACN;WACA;WACA;WACA;;;KAIC;;;;;;UAOK;;;;;WAKN;;WAEA;;WAEA;;WAEA;;;;;;;;WAQA;;UAGM;;;;;;WAMN,WAAW;WACX,iBAAiB,SAAS;;;;;;WAM1B,WAAW;;;;;;;WAOX,SAAS,SAAS;WAClB,YAAY;WACZ,WAAW;WACX,eAAe;WACf,YAAY;WACZ,aAAa;WACb,YAAY;WACZ,YAAY;WACZ,UAAU;WACV,OAAO,SAAS;;WAEhB;;UAGM;WACN,eAAe;;WAEf;WACA;WACA;WACA,cAAc;WACd,yBAAyB;WACzB;WACA,eAAe;WACf,UAAU;WACV,WAAW;WACX,SAAS;;WAET,gCAAgC;;WAEhC;;WAEA;;UAGM;;WAEN,kBAAkB,kBAAkB;WACpC;WACA;WACA,QAAQ;WACR,OAAO;WACP,aAAa;WACb,SAAS;WACT,UAAU;WACV,KAAK;;UAGC;WACN,eAAe;WACf;WACA,aAAa;WACb,iBAAiB;WACjB;WACA,WAAW;WACX,mBAAmB;WACnB,iBAAiB;WACjB,aAAa;WACb,qBAAqB;WACrB,eAAe;;;;;WAKf,UAAU;;;;;;;WAOV;aACE;eAA2B,OAAO;eAA2B;;aAC7D;eAAwB,OAAO;eAA2B;;;WAE5D,eAAe;WACf,kBAAkB;;;;;;;;UASZ;WACN;WACA,gBAAgB;;WAEhB,eAAe;;;KAId;UAWK;WACN,MAAM;WACN;WACA;WACA"}
|
package/dist/wire/index.d.ts
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
import { b as CustomTokenPricing, c as CostLedgerHandle } from "../cost-ledger-DbQdN3nO.js";
|
|
2
|
-
import { x as ChatClient } from "../types-
|
|
2
|
+
import { x as ChatClient } from "../types-DLxe96rp.js";
|
|
3
3
|
import { s as TraceStore } from "../store-C4avFjIi.js";
|
|
4
|
-
import { m as FeedbackTrajectoryStore } from "../feedback-trajectory-
|
|
4
|
+
import { m as FeedbackTrajectoryStore } from "../feedback-trajectory-C6NJxXEa.js";
|
|
5
5
|
import { z } from "zod";
|
|
6
6
|
import { ServerType } from "@hono/node-server";
|
|
7
7
|
import { Hono } from "hono";
|
package/dist/wire/index.js
CHANGED
|
@@ -1,2 +1,2 @@
|
|
|
1
|
-
import { A as WIRE_VERSION, C as RUBRIC_VERSION_SCHEME, D as TracesIngestRequestSchema, E as TraceEventSchema, O as TracesIngestResponseSchema, S as ListRubricsResponseSchema, T as RubricSchema, _ as FeedbackIngestResponseSchema, a as runRpcBatch, b as JudgeRequestSchema, c as WireError, d as handleListRubrics, f as handleTracesIngest, g as ErrorResponseSchema, h as listBuiltinRubrics, i as dispatchRpc, j as hashRubric, k as VersionResponseSchema, l as handleFeedbackIngest, m as getBuiltinRubric, n as startServer, o as runRpcOnce, p as handleVersion, r as startServerAsync, s as buildOpenApi, t as createApp, u as handleJudge, v as FeedbackTrajectorySchema, w as RubricDimensionSchema, x as JudgeResultSchema, y as HealthResponseSchema } from "../server-
|
|
1
|
+
import { A as WIRE_VERSION, C as RUBRIC_VERSION_SCHEME, D as TracesIngestRequestSchema, E as TraceEventSchema, O as TracesIngestResponseSchema, S as ListRubricsResponseSchema, T as RubricSchema, _ as FeedbackIngestResponseSchema, a as runRpcBatch, b as JudgeRequestSchema, c as WireError, d as handleListRubrics, f as handleTracesIngest, g as ErrorResponseSchema, h as listBuiltinRubrics, i as dispatchRpc, j as hashRubric, k as VersionResponseSchema, l as handleFeedbackIngest, m as getBuiltinRubric, n as startServer, o as runRpcOnce, p as handleVersion, r as startServerAsync, s as buildOpenApi, t as createApp, u as handleJudge, v as FeedbackTrajectorySchema, w as RubricDimensionSchema, x as JudgeResultSchema, y as HealthResponseSchema } from "../server-CoBZO1fj.js";
|
|
2
2
|
export { ErrorResponseSchema, FeedbackIngestResponseSchema, FeedbackTrajectorySchema, HealthResponseSchema, JudgeRequestSchema, JudgeResultSchema, ListRubricsResponseSchema, RUBRIC_VERSION_SCHEME, RubricDimensionSchema, RubricSchema, TraceEventSchema, TracesIngestRequestSchema, TracesIngestResponseSchema, VersionResponseSchema, WIRE_VERSION, WireError, buildOpenApi, createApp, dispatchRpc, getBuiltinRubric, handleFeedbackIngest, handleJudge, handleListRubrics, handleTracesIngest, handleVersion, hashRubric, listBuiltinRubrics, runRpcBatch, runRpcOnce, startServer, startServerAsync };
|
|
@@ -0,0 +1,30 @@
|
|
|
1
|
+
# Unreleased: one physical-attempt allowance for structured calls
|
|
2
|
+
|
|
3
|
+
`createChatClient` and structured JSON calls now count schema negotiation,
|
|
4
|
+
temperature negotiation and transient retries against one `maximumAttempts`
|
|
5
|
+
allowance. Schema fallback no longer starts a second retry loop, resets the
|
|
6
|
+
cross-attempt deadline, or restarts raw-provider attempt indices.
|
|
7
|
+
|
|
8
|
+
`maximumAttempts: 1` makes at most one HTTP request. A provider rejecting the
|
|
9
|
+
schema is returned to the caller rather than followed by an unadmitted call.
|
|
10
|
+
Callers that allow negotiation can supply enough attempts, or select
|
|
11
|
+
`jsonSchemaTransport: 'json-object'` when that protocol is already known.
|
|
12
|
+
Low-level `callLlm` retains its existing no-schema-negotiation behavior.
|
|
13
|
+
|
|
14
|
+
`maximumChargeForLlmRequest` uses the same physical-attempt allowance rather
|
|
15
|
+
than reserving two full batches for a schema request. Its input bound still
|
|
16
|
+
includes the native schema, and output limits remain enforced per request.
|
|
17
|
+
Failed-attempt raw observations keep their order and the operation's original
|
|
18
|
+
identity. This does not claim provider-side deduplication or final-response
|
|
19
|
+
aggregation of every paid failed attempt; retain the execution evidence and
|
|
20
|
+
account at the existing caller-owned boundary.
|
|
21
|
+
|
|
22
|
+
There is one retry owner and no new public framework, dependency, credential
|
|
23
|
+
source, scheduler or product-specific transport. Product fetch guards protecting
|
|
24
|
+
older installed versions must remain until a release containing this change is
|
|
25
|
+
published and the compatible consumer set is installed and verified.
|
|
26
|
+
|
|
27
|
+
Proof: `tests/llm-physical-attempts.test.ts` runs the public client against a real
|
|
28
|
+
loopback HTTP server. Existing client, transient-status and raw-capture checks
|
|
29
|
+
remain in place. These are transport and allowance checks, not live-provider
|
|
30
|
+
cost or outcome-quality benchmarks.
|
|
@@ -0,0 +1,30 @@
|
|
|
1
|
+
# Model request lifetime and cost interpretation
|
|
2
|
+
|
|
3
|
+
The existing physical-attempt loop now keeps `timeoutMs` active until the full
|
|
4
|
+
response body has been read, for both successful and failed HTTP responses.
|
|
5
|
+
`deadlineMs` also bounds each remaining request and retry wait; an already-expired
|
|
6
|
+
operation does not dispatch. Cancelling during `Retry-After` or exponential backoff
|
|
7
|
+
stops that wait immediately. A late transport response after the deadline fails
|
|
8
|
+
with the cancellation error rather than becoming a result or a new retry.
|
|
9
|
+
|
|
10
|
+
The client reuses `combineAbortSignals` instead of maintaining separate signal
|
|
11
|
+
composition. Raw captures retain the existing attempt indices, operation identity
|
|
12
|
+
and redaction policy. User-defined capture callbacks must still complete or honor
|
|
13
|
+
their own cancellation; this change does not promise to terminate arbitrary code.
|
|
14
|
+
|
|
15
|
+
Results and receipts use the same reported-cost parser. Invalid negative or
|
|
16
|
+
non-finite provider prices are not exposed as observed cost; an explicitly reported
|
|
17
|
+
zero remains zero. Token-price estimates, when available, remain estimates. This
|
|
18
|
+
is provider-level parsing, not evidence of a Router customer charge: product
|
|
19
|
+
consumers must keep their customer receipt and payer attribution.
|
|
20
|
+
|
|
21
|
+
The full operation continues to use the caller's physical request allowance,
|
|
22
|
+
including schema and temperature negotiation. No hidden extra request batch,
|
|
23
|
+
dependency, new budget owner, default provider, or automatic result replay is added.
|
|
24
|
+
|
|
25
|
+
Run `pnpm exec vitest run src/llm-client.test.ts tests/llm-physical-attempts.test.ts`
|
|
26
|
+
for the maintained client and real-loopback HTTP contracts. These include a stalled
|
|
27
|
+
body after headers, cancellation during server-directed backoff, exhausted deadlines,
|
|
28
|
+
request-count conservation, raw-capture ordering and malformed reported prices.
|
|
29
|
+
Regenerate only the live analyst implementation pin with
|
|
30
|
+
`pnpm analyst:pin`; historical evidence pins remain fixed.
|
package/docs/concepts.md
CHANGED
|
@@ -154,6 +154,14 @@ that can seed memory, replay scenarios, and optimization.
|
|
|
154
154
|
| **Provenance** | Where a number came from: the package version, the source revision, the run identity, the exact attempt. |
|
|
155
155
|
| **`RunRecord`** | The analysis-time projection of one run: who ran, on what, with which seed, at what cost, and what it scored. |
|
|
156
156
|
|
|
157
|
+
### Read provider usage
|
|
158
|
+
|
|
159
|
+
Use the public `extractUsage` reader for retained provider response bodies.
|
|
160
|
+
It preserves reported cache reads and writes, including Router prompt details and `prompt_cache` fields.
|
|
161
|
+
An explicit cache zero remains zero; an absent cache component remains omitted.
|
|
162
|
+
The reader does not establish complete physical-call coverage or authoritative billing.
|
|
163
|
+
Keep raw usage and its source identity beside the projection.
|
|
164
|
+
|
|
157
165
|
### Improving a surface
|
|
158
166
|
|
|
159
167
|
| Term | Plain English |
|
|
@@ -0,0 +1,33 @@
|
|
|
1
|
+
# Distributed dispatch: execution, retries and receipts
|
|
2
|
+
|
|
3
|
+
`httpDispatch()` and `runDispatchServer()` are the existing `/adapters/http` transport. They do not own a second scheduler, an execution database, or an exactly-once protocol. Keep durable execution ownership in the worker's existing Run/retained-dispatch mechanism.
|
|
4
|
+
|
|
5
|
+
## Migration
|
|
6
|
+
|
|
7
|
+
`httpDispatch()` now makes **one attempt by default** (`retries: 0`, previously 2). A disconnected response or HTTP 500 does not establish that a worker did nothing. Applications that explicitly enable retries must bind request identity and external effects to a durable worker-side deduplication contract first. Existing explicit `retries` configuration remains supported.
|
|
8
|
+
|
|
9
|
+
Caller cancellation, terminal HTTP refusals, malformed success bodies, missing artifacts and receipt-import failures do not redispatch work. An error response carrying paid receipts is also terminal: import the already-settled spend, then report the failure. Request bytes and run/cell identities stay unchanged across explicitly enabled transport attempts. Backoff is cancellable.
|
|
10
|
+
|
|
11
|
+
The worker's optional `contextFactory` must supply a real `CampaignCostMeter` for paid work. A missing meter refuses `runPaidCall` before its execute callback. Deterministic work remains usable without a cost factory. Work performed outside the meter is not thereby free; the host must require and reconcile the appropriate evidence.
|
|
12
|
+
|
|
13
|
+
## Accounting boundary
|
|
14
|
+
|
|
15
|
+
Workers authorize and cap spend **before execution**. The client imports receipts after the operation; that cannot retroactively enforce a coordinator ceiling. Successful responses, failures and deliverable cancellation responses carry the same available settled receipts. Unknown cost stays unknown.
|
|
16
|
+
|
|
17
|
+
If the client disconnects before receiving a result, reconcile the worker's authoritative ledger. Do not reset an allowance or repeat the operation to reconstruct a lost receipt. This adapter does not implement that reconciliation or authorize an outer workflow to retry.
|
|
18
|
+
|
|
19
|
+
## Verification
|
|
20
|
+
|
|
21
|
+
The maintained `src/adapters/http.test.ts` uses real loopback Node HTTP, the actual client/server, real filesystem effects, and the native cost ledger. Five regressions distinguish the old behavior from the corrected boundary: terminal authorization, failure after a file effect, charged failure, charged cancellation and failed coordinator receipt import. The two existing connection/cancellation cases remain.
|
|
22
|
+
|
|
23
|
+
```sh
|
|
24
|
+
pnpm install --frozen-lockfile
|
|
25
|
+
pnpm exec vitest run src/adapters/http.test.ts
|
|
26
|
+
pnpm typecheck
|
|
27
|
+
pnpm build
|
|
28
|
+
pnpm verify:package
|
|
29
|
+
```
|
|
30
|
+
|
|
31
|
+
Ledger amounts in the transport cases are synthetic; no provider call is needed to prove this boundary. Passing them is not a model-quality, task-success, live deployment, isolation, or customer-savings claim.
|
|
32
|
+
|
|
33
|
+
For a real application comparison, use the existing campaign/experiment APIs over complete application jobs. Preserve assigned failures, immutable checker and policy identities, full cost evidence, fresh final tasks and existing controlled publication. A transport check and a customer-benefit experiment answer different questions.
|