@tangle-network/agent-eval 0.203.0 → 0.203.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +6 -0
- package/dist/adapters/http.d.ts +10 -4
- package/dist/adapters/http.d.ts.map +1 -1
- package/dist/adapters/http.js +43 -38
- package/dist/adapters/http.js.map +1 -1
- package/dist/{agent-profile-DDfIjdyS.d.ts → agent-profile-EpO0HURy.d.ts} +4 -4
- package/dist/{agent-profile-DDfIjdyS.d.ts.map → agent-profile-EpO0HURy.d.ts.map} +1 -1
- package/dist/analyst/index.d.ts +12 -12
- package/dist/analyst/index.d.ts.map +1 -1
- package/dist/analyst/index.js +6 -6
- package/dist/{analyze-runs-CPiKLJFs.d.ts → analyze-runs-Q74o4Hfj.d.ts} +2 -2
- package/dist/{analyze-runs-CPiKLJFs.d.ts.map → analyze-runs-Q74o4Hfj.d.ts.map} +1 -1
- package/dist/{benchmark-command-DFs_i62N.js → benchmark-command-6OBcB0sG.js} +7 -6
- package/dist/benchmark-command-6OBcB0sG.js.map +1 -0
- package/dist/benchmarks/index.d.ts +4 -4
- package/dist/benchmarks/index.js +3 -3
- package/dist/campaign/index.d.ts +6 -6
- package/dist/campaign/index.js +4 -4
- package/dist/{campaign-Btk1yFfw.js → campaign-BbObY2c6.js} +6 -6
- package/dist/{campaign-Btk1yFfw.js.map → campaign-BbObY2c6.js.map} +1 -1
- package/dist/{chat-client-DUhWNQQS.js → chat-client-5LLl4Ctl.js} +2 -2
- package/dist/{chat-client-DUhWNQQS.js.map → chat-client-5LLl4Ctl.js.map} +1 -1
- package/dist/{chat-json-call-Dx_sHzqm.js → chat-json-call-Digk-pfF.js} +2 -2
- package/dist/{chat-json-call-Dx_sHzqm.js.map → chat-json-call-Digk-pfF.js.map} +1 -1
- package/dist/cli.js +3 -3
- package/dist/contract/index.d.ts +10 -10
- package/dist/contract/index.js +7 -7
- package/dist/{default-registry-B-utjsmX.d.ts → default-registry-CxEBVA6W.d.ts} +6 -6
- package/dist/{default-registry-B-utjsmX.d.ts.map → default-registry-CxEBVA6W.d.ts.map} +1 -1
- package/dist/{default-registry-CpswV_S2.js → default-registry-wInsQWlQ.js} +2 -2
- package/dist/{default-registry-CpswV_S2.js.map → default-registry-wInsQWlQ.js.map} +1 -1
- package/dist/{define-agent-eval-MAc6BQrq.d.ts → define-agent-eval-BbEXdM3H.d.ts} +7 -7
- package/dist/{define-agent-eval-MAc6BQrq.d.ts.map → define-agent-eval-BbEXdM3H.d.ts.map} +1 -1
- package/dist/{define-agent-eval-CJN9dTQw.js → define-agent-eval-Da8mHyc7.js} +3 -3
- package/dist/{define-agent-eval-CJN9dTQw.js.map → define-agent-eval-Da8mHyc7.js.map} +1 -1
- package/dist/diagnosis/index.d.ts +2 -2
- package/dist/{dspy-rlm-engine-US8YsYGq.js → dspy-rlm-engine-D0h-P2Mz.js} +2 -2
- package/dist/{dspy-rlm-engine-US8YsYGq.js.map → dspy-rlm-engine-D0h-P2Mz.js.map} +1 -1
- package/dist/{engine-4_Q4oujc.d.ts → engine-CaSqEVlL.d.ts} +4 -4
- package/dist/{engine-4_Q4oujc.d.ts.map → engine-CaSqEVlL.d.ts.map} +1 -1
- package/dist/evaluation.d.ts +2 -2
- package/dist/{exact-types-KGxBWH4b.d.ts → exact-types-BBMMYk3g.d.ts} +2 -2
- package/dist/{exact-types-KGxBWH4b.d.ts.map → exact-types-BBMMYk3g.d.ts.map} +1 -1
- package/dist/experiment/index.d.ts +1 -1
- package/dist/{external-optimizer-process-KLcw0Vnc.js → external-optimizer-process-CIV4EqiT.js} +2 -2
- package/dist/{external-optimizer-process-KLcw0Vnc.js.map → external-optimizer-process-CIV4EqiT.js.map} +1 -1
- package/dist/{external-optimizer-subprocess-CkyUUV7h.js → external-optimizer-subprocess-BmKtH3T6.js} +2 -2
- package/dist/{external-optimizer-subprocess-CkyUUV7h.js.map → external-optimizer-subprocess-BmKtH3T6.js.map} +1 -1
- package/dist/{extract-usage-DFATEuWW.js → extract-usage-BrnzRJ6H.js} +4 -3
- package/dist/extract-usage-BrnzRJ6H.js.map +1 -0
- package/dist/{feedback-trajectory-BXCGTRkz.d.ts → feedback-trajectory-C6NJxXEa.d.ts} +3 -3
- package/dist/{feedback-trajectory-BXCGTRkz.d.ts.map → feedback-trajectory-C6NJxXEa.d.ts.map} +1 -1
- package/dist/{index-Cuhxk05B.d.ts → index-CGAwP9l3.d.ts} +8 -8
- package/dist/{index-Cuhxk05B.d.ts.map → index-CGAwP9l3.d.ts.map} +1 -1
- package/dist/{index-DBk_4UPs.d.ts → index-CIkxRsLS.d.ts} +7 -7
- package/dist/{index-DBk_4UPs.d.ts.map → index-CIkxRsLS.d.ts.map} +1 -1
- package/dist/{index-CAtNOBQY.d.ts → index-Ce2HACcP.d.ts} +7 -7
- package/dist/{index-CAtNOBQY.d.ts.map → index-Ce2HACcP.d.ts.map} +1 -1
- package/dist/index.d.ts +19 -19
- package/dist/index.js +11 -11
- package/dist/{integrity-CaeNeusS.d.ts → integrity-BWO6RPxv.d.ts} +2 -2
- package/dist/{integrity-CaeNeusS.d.ts.map → integrity-BWO6RPxv.d.ts.map} +1 -1
- package/dist/{jev-review-DX6MI_jq.d.ts → jev-review-Cw06AKuF.d.ts} +2 -2
- package/dist/{jev-review-DX6MI_jq.d.ts.map → jev-review-Cw06AKuF.d.ts.map} +1 -1
- package/dist/jev.d.ts +3 -3
- package/dist/{llm-client-Bhqoglar.js → llm-client-DmCNzrc9.js} +77 -60
- package/dist/llm-client-DmCNzrc9.js.map +1 -0
- package/dist/{llm-judge-C-0P1WiT.js → llm-judge-EUMHOZOQ.js} +4 -4
- package/dist/{llm-judge-C-0P1WiT.js.map → llm-judge-EUMHOZOQ.js.map} +1 -1
- package/dist/{matrix-BucOjLF1.d.ts → matrix-BqHiMKIj.d.ts} +2 -2
- package/dist/{matrix-BucOjLF1.d.ts.map → matrix-BqHiMKIj.d.ts.map} +1 -1
- package/dist/meta-eval/index.d.ts +1 -1
- package/dist/multishot/golden/index.d.ts +1 -1
- package/dist/multishot/index.d.ts +2 -2
- package/dist/openapi.json +1 -1
- package/dist/{prime-protocol-T0i6xcAs.d.ts → prime-protocol-IoN4Qb6c.d.ts} +2 -2
- package/dist/{prime-protocol-T0i6xcAs.d.ts.map → prime-protocol-IoN4Qb6c.d.ts.map} +1 -1
- package/dist/{produced-state-CR2tAPLf.js → produced-state-g9BPgu4D.js} +3 -3
- package/dist/{produced-state-CR2tAPLf.js.map → produced-state-g9BPgu4D.js.map} +1 -1
- package/dist/{abort-signal-CtzAM_sJ.js → proposal-findings-CXblsskH.js} +2 -11
- package/dist/proposal-findings-CXblsskH.js.map +1 -0
- package/dist/{registry-Bu3lNole.d.ts → registry-CT-hMSkR.d.ts} +4 -4
- package/dist/{registry-Bu3lNole.d.ts.map → registry-CT-hMSkR.d.ts.map} +1 -1
- package/dist/{researcher-DojEMb7E.d.ts → researcher-zTpxNJBy.d.ts} +3 -3
- package/dist/{researcher-DojEMb7E.d.ts.map → researcher-zTpxNJBy.d.ts.map} +1 -1
- package/dist/rl.d.ts +4 -4
- package/dist/search/index.d.ts +2 -2
- package/dist/{search-kernel-B8RC6BlK.d.ts → search-kernel-C5jua_UB.d.ts} +2 -2
- package/dist/{search-kernel-B8RC6BlK.d.ts.map → search-kernel-C5jua_UB.d.ts.map} +1 -1
- package/dist/{semantic-concept-judge-BcjYg37z.js → semantic-concept-judge-BgWxa4L_.js} +4 -4
- package/dist/{semantic-concept-judge-BcjYg37z.js.map → semantic-concept-judge-BgWxa4L_.js.map} +1 -1
- package/dist/{series-convergence-DY7ha0kx.d.ts → series-convergence-DLy0bQMV.d.ts} +2 -2
- package/dist/{series-convergence-DY7ha0kx.d.ts.map → series-convergence-DLy0bQMV.d.ts.map} +1 -1
- package/dist/{server-B4jMhqQn.js → server-CoBZO1fj.js} +2 -2
- package/dist/{server-B4jMhqQn.js.map → server-CoBZO1fj.js.map} +1 -1
- package/dist/{skillopt-optimization-method-Bls9t-G_.js → skillopt-optimization-method-DZrdQGZl.js} +5 -5
- package/dist/{skillopt-optimization-method-Bls9t-G_.js.map → skillopt-optimization-method-DZrdQGZl.js.map} +1 -1
- package/dist/{statistical-heldout-DJtQAUZB.d.ts → statistical-heldout-CJYAO-4a.d.ts} +2 -2
- package/dist/{statistical-heldout-DJtQAUZB.d.ts.map → statistical-heldout-CJYAO-4a.d.ts.map} +1 -1
- package/dist/{store-tool-spans-Biio4_Nw.js → store-tool-spans-D5kOU0mr.js} +2 -2
- package/dist/{store-tool-spans-Biio4_Nw.js.map → store-tool-spans-D5kOU0mr.js.map} +1 -1
- package/dist/{store-tool-spans-fkNF7Q9Y.d.ts → store-tool-spans-fIE31bU8.d.ts} +5 -5
- package/dist/{store-tool-spans-fkNF7Q9Y.d.ts.map → store-tool-spans-fIE31bU8.d.ts.map} +1 -1
- package/dist/{tool-groups-Zm84Pqpk.d.ts → tool-groups-DP8b5kjF.d.ts} +3 -3
- package/dist/tool-groups-DP8b5kjF.d.ts.map +1 -0
- package/dist/{trace-questions--lnJTaou.d.ts → trace-questions-Cd0Fyhbv.d.ts} +3 -3
- package/dist/{trace-questions--lnJTaou.d.ts.map → trace-questions-Cd0Fyhbv.d.ts.map} +1 -1
- package/dist/trace-repair/index.d.ts +1 -1
- package/dist/traces.d.ts +5 -5
- package/dist/traces.js +2 -2
- package/dist/{types-CeDfCRXp.d.ts → types-6VEehmEg.d.ts} +2 -2
- package/dist/{types-CeDfCRXp.d.ts.map → types-6VEehmEg.d.ts.map} +1 -1
- package/dist/{types-C0bWoSKC.d.ts → types-DLxe96rp.d.ts} +2 -2
- package/dist/{types-C0bWoSKC.d.ts.map → types-DLxe96rp.d.ts.map} +1 -1
- package/dist/{types-khIUQ0O5.d.ts → types-Duc6xOoW.d.ts} +3 -3
- package/dist/{types-khIUQ0O5.d.ts.map → types-Duc6xOoW.d.ts.map} +1 -1
- package/dist/{types-B6JadxWL.d.ts → types-hH9APahd.d.ts} +2 -2
- package/dist/{types-B6JadxWL.d.ts.map → types-hH9APahd.d.ts.map} +1 -1
- package/dist/wire/index.d.ts +2 -2
- package/dist/wire/index.js +1 -1
- package/docs/changes/llm-physical-attempts.md +30 -0
- package/docs/changes/shared-model-request-lifetime.md +30 -0
- package/docs/concepts.md +8 -0
- package/docs/http-dispatch.md +33 -0
- package/docs/proofs/router-cache-usage-20261001/after.json +2058 -0
- package/docs/proofs/router-cache-usage-20261001/component-projections.json +584 -0
- package/docs/proofs/router-cache-usage-20261001/component-projections.nested-only.json +306 -0
- package/docs/proofs/router-cache-usage-20261001/manifest.json +32 -0
- package/docs/proofs/router-cache-usage-20261001/summary.json +49 -0
- package/docs/proofs/router-cache-usage-20261001/summary.nested-only.json +29 -0
- package/package.json +1 -1
- package/dist/abort-signal-CtzAM_sJ.js.map +0 -1
- package/dist/benchmark-command-DFs_i62N.js.map +0 -1
- package/dist/extract-usage-DFATEuWW.js.map +0 -1
- package/dist/llm-client-Bhqoglar.js.map +0 -1
- package/dist/tool-groups-Zm84Pqpk.d.ts.map +0 -1
|
@@ -365,7 +365,7 @@ interface LlmCallRequest {
|
|
|
365
365
|
* capped CostLedger to reject the call before execution. Pass
|
|
366
366
|
* `customTokenPricing` when package pricing does not cover the model or endpoint. */
|
|
367
367
|
interface LlmChargeBounds {
|
|
368
|
-
/** Total
|
|
368
|
+
/** Total physical requests, including retries and schema/temperature fallback. Default 3. */
|
|
369
369
|
maximumAttempts?: number;
|
|
370
370
|
/** The transport sends JSON mode instead of a response schema. */
|
|
371
371
|
jsonSchemaTransport?: 'native' | 'json-object';
|
|
@@ -862,4 +862,4 @@ interface CheckResult {
|
|
|
862
862
|
}
|
|
863
863
|
//#endregion
|
|
864
864
|
export { NoopRawProviderSink as $, createChatClient as A, LlmToolCall as B, ChatResponse as C, judgeFamily as Ct, MockTransportOpts as D, CustomTransportOpts as E, LlmChargeBounds as F, costReceiptFromLlmError as G, LlmToolDefinition as H, LlmMessage as I, stripFencedJson as J, isTransientLlmError as K, LlmResponseError as L, LlmCallMetadata as M, LlmCallRequest as N, OpenAiCompatibleTransportOpts as O, LlmCallResult as P, InMemoryRawProviderSinkOptions as Q, LlmThinkingMode as R, ChatRequest as S, assertCrossFamily as St, CreateChatClientOpts as T, LlmUsage as U, LlmToolChoice as V, costReceiptFromLlm as W, FileSystemRawProviderSinkOptions as X, FileSystemRawProviderSink as Y, InMemoryRawProviderSink as Z, ScenarioResult as _, checkServedModel as _t, CompletionCriterion as a, defaultProviderRedactor as at, ChatCallOpts as b, CrossFamilyError as bt, JudgeInput as c, AssertServedModelOptions as ct, PersonaConfig as d, ServedModelCheck as dt, ProviderRedactor as et, ProductClientConfig as f, ServedModelPolicy as ft, ScenarioFile as g, assertServedModels as gt, Scenario as h, assertServedModel as ht, CollectedArtifacts as i, RawProviderSinkFilter as it, LlmCallError as j, SandboxSdkTransportOpts as k, JudgeRubric as l, ModelSubstitutionError as lt, RubricDimension as m, assertCrossFamilyServed as mt, ArtifactResult as n, RawProviderEvent as nt, DriverState as o, providerFromBaseUrl as ot, RouteMap as p, ServedModelVerdict as pt, maximumChargeForLlmRequest as q, CheckResult as r, RawProviderSink as rt, JudgeFn as s, AssertCrossFamilyServedOptions as st, ArtifactCheck as t, RawProviderDirection as tt, JudgeScore as u, ServedCrossFamilyError as ut, Turn as v, servedModelAcceptable as vt, ChatTransport as w, ChatClient as x, JudgeFamily as xt, TurnResult as y, AssertCrossFamilyOptions as yt, LlmTokenLogprob as z };
|
|
865
|
-
//# sourceMappingURL=types-
|
|
865
|
+
//# sourceMappingURL=types-DLxe96rp.d.ts.map
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"types-
|
|
1
|
+
{"version":3,"file":"types-DLxe96rp.d.ts","names":[],"sources":["../src/judge-families.ts","../src/integrity/served-model.ts","../src/trace/raw-provider-sink.ts","../src/llm-client.ts","../src/analyst/chat-client.ts","../src/types.ts"],"mappings":";;;;;;;;;;;;;KAWY;;;;;;;iBAsEI,YAAY,kBAAkB;UAc7B;;EAEf;;;;EAIA;;cAGW,yBAAyB;WAGlB,UAAU;WACV;EAHlB,YACE,iBACgB,UAAU,eACV;;;;;;;;;;;;;;;iBAoBJ,kBACd,kBACA,OAAM,2BACL;;;;KCvFS;;;;;;;;;;;UAYK;;EAEf;;EAEA;EACA,iBAAiB;;EAEjB,cAAc;EACd,SAAS;;EAET;;;;;;;;;;;;iBA4Cc,iBACd,mBACA,oCACC;cA4CU,+BAA+B;WAGxB,QAAQ,cAAc;EAFxC,YACE,iBACgB,QAAQ,cAAc;;;;;;;;;KAc9B;UAWK;;;;;;EAMf;;;;;EAKA;;EAEA;;;;;;;iBAQc,sBACd,OAAO,kBACP,OAAM;;;;;;iBA6BQ,kBACd,mBACA,mCACA,OAAM,2BACL;;;;;iBAea,mBACd,OAAO;EAAgB;EAAmB;IAC1C,OAAM,2BACL;UAec,uCAAuC;;EAEtD;;EAEA;;cAGW,+BAA+B;WAGxB,UAAU;WACV,QAAQ,cAAc;EAHxC,YACE,iBACgB,UAAU,eACV,QAAQ,cAAc;;;;;;;;;;;iBAgB1B,wBACd,OAAO;EAAgB;EAAmB;IAC1C,OAAM,iCACL;;;;;;;;;;;;;;;;;;;;;;;;;;;;;KCjRS;UAEK;;EAEf;;EAEA;EACA;;;;;;EAMA;EACA;;EAEA;;EAEA;;EAEA;EACA,WAAW;;EAEX;;EAEA;EACA;EACA,iBAAiB;EACjB;EACA,kBAAkB;EAClB;;EAEA;;EAEA;;UAGe;EACf;EACA;EACA,YAAY;EACZ;;UAGe;EACf,OAAO,OAAO,mBAAmB;;EAEjC,MAAM,SAAS,wBAAwB,QAAQ;;EAE/C,UAAU;;KAGA,oBAAoB,OAAO,qBAAqB;;;;;iBAM5C,wBAAwB,OAAO,mBAAmB;UAqBjD;EACf,WAAW;;cAGA,mCAAmC;UACtC;UACA;EAER,YAAY,OAAM;EAIZ,OAAO,OAAO,mBAAmB;EAIjC,KAAK,SAAQ,wBAA6B,QAAQ;EAUxD;;cAKW,+BAA+B;EACpC,UAAU;;;;;;;EASV,QAAQ,QAAQ;;UAOP;;EAEf;;EAEA;;EAEA;EACA,WAAW;;cAGA,qCAAqC;UACxC;UACA;UACA;UACA;UACA;UACA;UACA;EAER,YAAY,MAAM;UAOJ;UAON;EAKF,OAAO,OAAO,mBAAmB;EAYjC,KAAK,SAAQ,wBAA6B,QAAQ;;;;;;iBAkC1C,oBAAoB;;;UCnLnB;EACf;;;;;EAKA,kBAEI;IACM;IAAc;;IACd;IAAmB;MAAa;MAAa;;;;EAGvD,YAAY;;EAEZ;;KAGU;;UAGK;EACf;EACA;IACE;IACA;IACA,YAAY;;;;KAKJ;EAIN;EAAkB;IAAY;;;;UAGnB;EACf;EACA;;EAEA;;UAGe;EACf;EACA,UAAU;;EAEV;;EAEA;IAAe;IAAc,QAAQ;;;EAErC,QAAQ;;EAER,aAAa;EACb;EACA;;;;;EAKA;IAAa;;;EAEb,WAAW;;EAEX;;;;;;UAOe;;EAEf;;EAEA;;EAEA,WAAW;;EAEX,qBAAqB;;iBAGP,2BACd,SAAS,KACP,0GAGF,UAAS,kBACR;UA+Bc;EACf;EACA;EACA;;EAEA;;EAEA;;EAEA;;;;UAKe;EACf;EACA;EACA,KAAK;IAAgB;IAAe;;;UAGrB;;EAEf;;EAEA,YAAY;EACZ,OAAO;;;;;EAKP;;;;;;;;;EASA;;;;;;;;;;;;EAYA;;EAEA;;;;;;;;;EASA;;;;;;;EAOA;;;;;;;;EAQA,WAAW,cAAc;;EAEzB,KAAK;;KAGK,kBAAkB,KAAK;;;;;;iBAOnB,mBACd,QAAQ,eACR,qBAAqB,qBACpB;;iBA6Ba,wBACd,OAAO,OACP,qBAAqB,qBACpB;cAMU,qBAAqB;WAGd;WACA;WACA;EAJlB,YACE,iBACgB,gBACA,cACA;;;;;cASP,yBAAyB;WAGlB,QAAQ;EAF1B,YACE,iBACgB,QAAQ,eACxB;IAAY;;;UAMC,yBAAyB;;EAExC;;EAEA;EACA;;EAEA;IAAe;IAAc;;;EAE7B;;EAEA;;;;;;;EAOA,SAAS;;;;;;;;EAQT;;;;;;EAMA;;EAEA,eAAe;;;;;;;;EAQf,UAAU;;;;;EAKV;;EAEA;IAAiB;IAAgB;;;EAEjC,WAAW;;;;;;;;;;EAUX,8BAA8B;;;;;;;;;;;;;iBAwEhB,oBAAoB;;;;;;iBAsPpB,gBAAgB;;;;;;UCpqBf;;WAEN,WAAW;;WAEX;;WAEA;;EAGT,KAAK,KAAK,aAAa,OAAO,eAAe,QAAQ;;KAG3C;UAMK,oBAAoB,KAAK;;EAExC;;KAGU,eAAe;UAEV;;EAEf,SAAS;;EAET;;EAEA;;EAEA;;KAKU,uBACR,0BACA,sBACA,gCACA;UAEM;EACR;;EAEA;;;;;;UAOe,gCAAgC;EAC/C;EACA,OAAO,KAAK,aAAa,OAAO,iBAAiB,QAAQ;;;UAI1C,4BAA4B;EAC3C;EACA,OAAO,KAAK,aAAa,OAAO,iBAAiB,QAAQ;;;;;;;;;;;;;;;;;;;;UAqB1C,sCACP,mBACN,KACE;EAaJ;;EAEA;;EAEA;;EAEA;;EAEA;IAAe;IAAc;;;;;;;UAOd,0BAA0B;EACzC;EACA,UAAU,KAAK,aAAa,OAAO,iBAAiB,QAAQ;;;;;;iBAO9C,iBAAiB,MAAM,uBAAuB;;;UC/J7C;EACf;EACA;EACA;EACA;EACA;EACA,OAAO;EACP,gBAAgB;EAChB;;UAGe;EACf;EACA;EACA;EACA;;UAKe;EACf;EAQA;EACA;EACA;EACA;;UAKe;EACf;EACA;EACA,YAAY;;UAGG;EACf;EACA;EACA;EACA;EACA;;UAKe;EACf;EACA;EACA,OAAO;EACP,iBAAiB;EACjB,aAAa;EACb;EACA;EACA;EACA,WAAW;;EAEX,OAAO;;UAGQ;EACf;EACA;EACA;EACA;EACA;IAAmB;IAAc;;EACjC;EACA;;UAGe;EACf,OAAO;EACP;EACA;;UAGe;EACf;EACA;EACA;EACA;EACA;;UAGe;EACf;IAAc;IAAc;;EAC5B;IAAmB;IAAc,QAAQ;;EACzC;IAAc;IAAkB;;EAChC;;UAKe;EACf;EACA;EACA;EACA;EACA;EACA;EACA;EACA;EACA;EACA;GACC;;UAGc;EACf;EACA,QAAQ;;EAER;;UAKe;EACf;EACA;EACA;EACA;EACA;EACA;EACA;IACE;MACE;MACA;MACA;;;EAGJ,OAAO;EACP,gBAAgB;;UAKD;EACf;EACA,QAAQ,OAAO;EACf,YAAY,OAAO;;;;;;;;;;KAWT;UAEK;EACf;EACA;EACA;EACA,oBAAoB;EACpB;;EAEA,QAAQ;;;;;;;EAOR;;;;;;;EAOA;;;;;;EAMA;;UAGe;EACf;EACA;EACA;IAAa;IAAiB;IAAkB;;EAChD;EACA;EACA;;UAGe;EACf,UAAU;EACV,OAAO;EACP,WAAW;;EAEX,aAAa;EACb;EACA,WAAW;EACX,SAAS;;KAGC,WAAW,MAAM,YAAY,OAAO,eAAe,QAAQ;UAYtD;EACf;EACA;EACA;EACA"}
|
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
import { S as PaidCallResult, T as RunPaidCallInput, a as CostChannel, c as CostLedgerHandle, f as CostLedgerSummary, p as CostProvenance } from "./cost-ledger-DbQdN3nO.js";
|
|
2
2
|
import { d as RunTokenUsage, s as RunSearchCoordinates } from "./run-record-DF-fuu31.js";
|
|
3
|
-
import { _ as ProposalFinding } from "./types-
|
|
4
|
-
import { M as LlmCallMetadata } from "./types-
|
|
3
|
+
import { _ as ProposalFinding } from "./types-6VEehmEg.js";
|
|
4
|
+
import { M as LlmCallMetadata } from "./types-DLxe96rp.js";
|
|
5
5
|
import { t as SeriesDistribution } from "./descriptive-B2iPaT9J.js";
|
|
6
6
|
import { A as SearchLedgerHash, G as SearchSplit, I as SearchNodeStatus, L as SearchOpenedEvent, N as SearchNodeDecision, O as SearchLedgerEntry, P as SearchNodeRef, Q as SearchTaskOutcome, R as SearchOperationKind, U as SearchReservation, Z as SearchTask, b as SearchClosedEvent, d as SearchCellCancelledEvent, i as SearchArtifactRef, l as SearchCandidateSurface, o as SearchAudit, p as SearchCellStage, t as NodeEstimate, w as SearchEdgeRecordedEvent, z as SearchOperationRecordedEvent } from "./search-ledger-types-p2xKfiDJ.js";
|
|
7
7
|
import { p as LedgerProjector } from "./index-Bj96W0jZ.js";
|
|
@@ -1009,4 +1009,4 @@ interface CampaignResult<TArtifact = unknown, TScenario extends Scenario = Scena
|
|
|
1009
1009
|
}
|
|
1010
1010
|
//#endregion
|
|
1011
1011
|
export { SearchNode as $, LabeledScenarioWrite as A, ScoredSurfaceOutcome as B, JudgeDimension as C, LabeledScenarioSampleArgs as D, LabeledScenarioRecord as E, ProposeContext as F, labelTrustRank as G, SurfaceProposer as H, ProposedCandidate as I, renderSearchSummary as J, SearchProposerView as K, RedactionStatus as L, OptimizerConfig as M, ParetoParent as N, LabeledScenarioSource as O, ProposalTrackContext as P, SearchDecisionRecord as Q, Scenario as R, JudgeConfig as S, LabelTrust as T, TraceSpan as U, SessionScript as V, isProposedCandidate as W, SearchCell as X, searchProposerView as Y, SearchCompletion as Z, GateDecision as _, CampaignResult as a, SearchUnitScore as at, GenerationRecord as b, CampaignTraceWriter as c, searchTaskSetDigest as ct, DispatchContext as d, SearchOperation as et, DispatchFn as f, GateContribution as g, GateContext as h, CampaignCostMeter as i, SearchStateView as it, MutableSurface as j, LabeledScenarioStore as k, CodeSurface as l, searchUnitScores as lt, GateCheckStatus as m, CampaignArtifactWriter as n, SearchSpend as nt, CampaignScenarioIdentity as o, searchCellId as ot, Gate as p, SearchSummaryOptions as q, CampaignCellResult as r, SearchState as rt, CampaignTokenUsage as s, searchNodeId as st, CampaignAggregates as t, SearchScoredCell as tt, ComponentSurface as u, GateResult as v, JudgeScore as w, JudgeAggregate as x, GenerationCandidate as y, ScenarioAggregate as z };
|
|
1012
|
-
//# sourceMappingURL=types-
|
|
1012
|
+
//# sourceMappingURL=types-Duc6xOoW.d.ts.map
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"types-
|
|
1
|
+
{"version":3,"file":"types-Duc6xOoW.d.ts","names":[],"sources":["../src/campaign/search-state.ts","../src/campaign/search-summary.ts","../src/campaign/types.ts"],"mappings":";;;;;;;;;;iBAwDgB,aAAa,kBAAkB;;iBAK/B,aACd,kBACA,gBACA,gBACA,OAAO,aACP;;iBAMc,oBAAoB,gBAAgB,eAAe;UAkBlD;EACf;EACA,UAAU;EACV,OAAO;EACP;EACA;;UAGe;EACf;EACA;EACA;EACA;;;;UAKe;EACf;EACA;EACA;EACA;;;UAIe;EACf;EACA;;EAEA;EACA;;;;;;;;iBASc,iBAAiB,gBAAgB,qBAAqB;UAoBrD;EACf;;EAEA;EACA,gBAAgB;EAChB,UAAU;EACV,mBAAmB;;;EAGnB;;EAEA;;;;EAIA,kBAAkB;EAClB;EACA;EACA,QAAQ;EACR;;EAEA;EACA,oBAAoB;EACpB;EACA,OAAO;EACP;EACA;;UAGe;EACf;EACA;EACA;EACA;EACA,OAAO;EACP;EACA,OAAO;EACP;EACA,aAAa;EACb;EACA;EACA,SAAS;;EAET;;EAEA;EACA,WAAW;;EAEX;;EAEA;EACA;EACA;EACA;;UAGe;EACf;EACA,eAAe;EACf,aAAa;EACb;;EAEA,SAAS;;;EAGT,WAAW;;;EAGX,oBAAoB;;EAEpB;;EAEA;;UAGe;EACf;EACA;;;;;;;cA6BW,uBAAuB,gBAAgB,mBAAmB;WAC5D;UACD;UACA;UACA;UACA;UACA;UACA;mBACS;mBACA;mBACA;mBACA;mBACA;mBACA;mBACA;mBACA;UACT;UACA;;mBAES;;UAET;mBACS;EAEjB,YAAY;EAgCZ,MAAM,OAAO,mBAAmB;EA6BhC,YAAY;;EAKZ,KAAK;UAIG;UAgCA;UA6CA;UAsBA;UA8BA;UA8CA;UAsHA;UAmEA;UA4EA;UAoBA;UAmDA;UAaA;;;UA2EA;;EASR;;;UAmBQ;;;UAkBA;;;;;;;;;;;UAoBA;;;EAsBR;IAAgB;IAA4B;;;EAa5C,cAAc;;EAKd,cAAc;;EAKd;IAAc;IAAkB,WAAW;;;EAK3C;;EAKA,aAAa;;EAcb,SAAS,iBAAiB;;EAM1B;;EAKA,YAAY;;EAKZ,oBAAoB;;EAKpB,SAAS,iBAAiB;;EAK1B;;EAKA,SAAS,iBAAiB;;EAM1B,YAAY;;EAMZ,cAAc,sBAAsB;;EAMpC,gBAAgB,gBAAgB,OAAO,cAAc;;;;;;;cAiB1C;WACF;WACA,QAAQ;WACR,QAAQ;WACR;IAAQ;IAAkB,WAAW;;;WAErC;WACA,OAAO;WACP,YAAY;;;;;;;WAOZ;IAAU;IAA4B;;mBAC9B;mBACA;EAEjB,YAAY,OAAO,aAAa;;MAgB5B;EAIJ,KAAK,iBAAiB;EAItB,QAAQ;;EAKR;;EAKA,SAAS;EAKT,gBAAgB;EAIhB,KAAK,iBAAiB;;EAKtB,SAAS;EAKT,KAAK,iBAAiB;;EAKtB,MAAM;IAAU;MAAyB;EAKzC,UAAU,sBAAsB;;;EAMhC,YAAY,gBAAgB,OAAO,cAAc;;EAKjD,WAAW,gBAAgB,OAAO,cAAc;;EAKhD,QAAQ,aAAa,uBAAuB;UAmCpC;;;;;;UCluCO;WACN;WACA;;;EAGT,YAAY,0BAA0B;;EAEtC,WAAW,0BAA0B;;;iBAIvB,mBAAmB,OAAO,kBAAkB;UAW3C;;;;;;;;EAQf,OAAO;;EAEP;;;;;;;;;;;iBAYc,oBAAoB,OAAO,iBAAiB,SAAS;;;;;UCtCpD;EACf;EACA;EACA;;;;;EAKA;;;UAIe,iCAAiC,KAAK;EACrD;;;;;UAMe;EACf;;EAEA;EACA;EACA;EACA;EACA,QAAQ;EACR,OAAO;EACP,WAAW;EACX,MAAM;;EAEN;;EAEA;;;;;;;;EAQA;;;;KAKU,WAAW,kBAAkB,UAAU,cACjD,UAAU,WACV,KAAK,oBACF,QAAQ;;;;UAOI,cAAc,WAAW;EACxC;EACA;EACA;;EAEA;;;EAGA,sBAAsB,UAAU,WAAW,sBAAsB,UAAU,cAAc;;UAK1E;;EAEf;;EAEA;;;;;;;;;UAUe,YAAY,WAAW,kBAAkB,WAAW;EACnE;EACA,YAAY;;;;EAIZ;;;EAGA,MAAM;IACJ,UAAU;IACV,UAAU;IACV,QAAQ;;IAER,aAAa;IACb;IACA,WAAW;MACT,aAAa,QAAQ;EACzB,aAAa,UAAU;;;;;;;;;;;UAYR;EACf,YAAY;EACZ;EACA;;EAEA,UAAU;;;;;;;EAOV;;;;EAIA,eAAe,eAAe;IAAQ;IAAe;;;;;EAIrD;;;EAGA;;EAEA;;EAEA,WAAW,eAAe;;;;;;;UAUX;WACN;;;WAGA;;WAEA;;WAEA;;WAEA;;WAEA;;WAEA;;;WAGA;aACE;aACA;aACA;;;WAGF;;;UAIM;WACN;WACA,YAAY,SAAS;;;;;;;;;;;KAYpB,0BAA0B,mBAAmB;;;;;;;UAQxC;EACf,SAAS;;EAET;;;;EAIA;;;;;;EAMA,cAAc,SAAS;;;;iBAKT,oBACd,OAAO,iBAAiB,oBACvB,SAAS;;;;;;;;;UAkBK;EACf,SAAS;EACT;;;EAGA,YAAY;;;EAGZ;;EAEA;EACA;EACA;;;UAIe;EACf;EACA;EACA;EACA;EACA;;;;;UAMe;;;EAGf;;EAEA;EACA;EACA;EACA,YAAY;;;;;EAKZ,WAAW;IAAQ;IAAoB;IAAmB;IAAgB;;EAC1E;IACE;IACA;;;;;UAMa,eAAe,YAAY;;;;WAIjC,gBAAgB;;;;;WAKhB;;;;WAIA,UAAU;IAAgB;IAAgB,UAAU;;;;WAGpD,QAAQ;;;;WAIR;;;WAGA,SAAS,cAAc;WACvB,UAAU,cAAc;;WAExB;WACA;WACA,QAAQ;;WAER,QAAQ;;;WAGR,kBAAkB;;;WAGlB,mBAAmB;;;;WAInB,gBAAgB;;;;WAIhB;;;;;;;WAOA,gBAAgB,cAAc;;WAE9B,aAAa;WACb;;;;;;;;;;;;;;UAeM,gBAAgB,YAAY;EAC3C;;;;;EAKA,QAAQ,KAAK,eAAe,aAAa,QAAQ,MAAM,iBAAiB;;;EAGxE,QAAQ;IAAQ,SAAS,cAAc;;IAAwB;IAAe;;;UAG/D;EACf;EACA;EACA,mBAAmB,qBAAqB;;UAGzB,wBAAwB;EACvC,UAAU;;;KAMA;;KAGA;UAEK;EACf;EACA,QAAQ;EACR;;UAGe,YAAY,WAAW,kBAAkB;EACxD,oBAAoB,YAAY;EAChC,oBAAoB,YAAY;;EAEhC,aAAa,YAAY,eAAe;;;;;EAKxC,sBAAsB,YAAY,eAAe;;;;;;;EAOjD,yBAAyB,YAAY,eAAe;;;EAGpD,uBAAuB,YAAY;EACnC,WAAW;EACX;IAAQ;IAAmB;;;EAE3B,aAAa;EACb;EACA,QAAQ;;UAGO;EACf,UAAU;EACV;EACA,mBAAmB;EACnB;;;UAIe,KAAK,qBAAqB,kBAAkB,WAAW;EACtE;EACA,OAAO,KAAK,YAAY,WAAW,aAAa,QAAQ;;;;UAOzC;EACf,KAAK,cAAc,aAAa,0BAA0B;EAC1D,SAAS;;UAGM;EACf,IAAI,aAAa;EACjB,aAAa,aAAa;;;;UAKX;EACf,MAAM,cAAc,kBAAkB,aAAa;EACnD,UAAU,cAAc,iBAAiB;;;;;;KAO/B,qBAAqB;;;;;UAMhB;;EAEf,YAAY,GACV,OAAO,KAAK,iBAAiB;IAC3B,UAAU;MAEX,QAAQ,eAAe;;;;;KAQhB;KAOA;;;;;;;;;;;;;;;KAgBA;;iBASI,eAAe,OAAO;;;;UAOrB,qBAAqB,kBAAkB,WAAW,UAAU;EAC3E,UAAU;EACV,UAAU;EACV,aAAa,eAAe;EAC5B,QAAQ;EACR;EACA;EACA,iBAAiB;;;;;EAKjB,aAAa;;EAEb;;UAGe,sBAAsB,kBAAkB,WAAW,UAAU,6BACpE,qBAAqB,WAAW;;EAExC;;;EAGA;;UAGe;EACf;;EAEA;;;;EAIA;EACA;IACE;IACA,SAAS,wBAAwB;IACjC;IACA;;;;;IAKA,WAAW;;;UAIE;EACf,QAAQ,OAAO,uBAAuB;EACtC,OAAO,MAAM,4BAA4B,QAAQ;EACjD,QAAQ;IACN;IACA;IACA,UAAU;;;IAGV,SAAS,OAAO;;;UAMH,mBAAmB;;;EAGlC;EACA;EACA;EACA;EACA;EACA,UAAU;EACV,aAAa,eAAe;;;EAG5B;;EAEA,gBAAgB;;EAEhB;;;EAGA,YAAY;;;EAGZ;;;EAGA;EACA;EACA;EACA;;;;EAIA;;EAEA;;EAEA;EACA;;UAGe;EACf;EACA;EACA;EACA;;;;;;EAMA,cAAc;;UAGC;EACf;EACA;EACA;;;EAGA,cAAc;;UAGC;EACf;EACA,YAAY;EACZ;;;;;;UAOe;EACf;;;;EAIA;;;;EAIA;;;;EAIA;IACE;IACA;IACA,iBAAiB;MAAQ;MAAgB;;;;;EAI3C,YAAY;;;;;;;;;;EAUZ,WAAW;IAAQ;IAAoB;IAAmB;IAAgB;;;;EAG1E;;;;EAIA;;;;EAIA,cAAc,SAAS;;UAGR;EACf,SAAS,eAAe;EACxB,YAAY,eAAe;;EAE3B,MAAM;;EAEN;EACA;EACA;;EAEA;;EAEA;;EAEA;;EAEA;;UAGe,eAAe,qBAAqB,kBAAkB,WAAW;;EAEhF;;EAEA;EACA;;EAEA;EACA;EACA;EACA;EACA,OAAO,MAAM,mBAAmB;EAChC,YAAY;EACZ;IACE,aAAa;IACb;;EAEF,OAAO;EACP;EACA;EACA,iBAAiB;;;EAGjB,WAAW,MAAM,2BAA2B,KAAK"}
|
|
@@ -1,5 +1,5 @@
|
|
|
1
1
|
import { c as RunSplitTag } from "./run-record-DF-fuu31.js";
|
|
2
|
-
import { R as Scenario, d as DispatchContext } from "./types-
|
|
2
|
+
import { R as Scenario, d as DispatchContext } from "./types-Duc6xOoW.js";
|
|
3
3
|
//#region src/benchmarks/types.d.ts
|
|
4
4
|
type BenchmarkTaskKind = 'retrieval' | 'rag-answer' | 'hallucination' | 'kb-improvement' | 'routing' | 'custom';
|
|
5
5
|
type BenchmarkFamily = 'beir' | 'mteb-retrieval' | 'msmarco' | 'trec-dl' | 'miracl' | 'lotte' | 'bright' | 'crag' | 'hotpotqa' | 'kilt' | 'ragtruth' | 'faithbench' | 'first-party' | 'custom';
|
|
@@ -99,4 +99,4 @@ declare const BENCHMARK_SPLIT_SEED = "agent-eval-v1";
|
|
|
99
99
|
declare function deterministicSplit(itemId: string, seed?: string): RunSplitTag;
|
|
100
100
|
//#endregion
|
|
101
101
|
export { BenchmarkFamily as a, BenchmarkSource as c, BenchmarkEvaluation as i, BenchmarkTaskKind as l, BenchmarkAdapter as n, BenchmarkResponder as o, BenchmarkDatasetItem as r, BenchmarkScenario as s, BENCHMARK_SPLIT_SEED as t, deterministicSplit as u };
|
|
102
|
-
//# sourceMappingURL=types-
|
|
102
|
+
//# sourceMappingURL=types-hH9APahd.d.ts.map
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"types-
|
|
1
|
+
{"version":3,"file":"types-hH9APahd.d.ts","names":[],"sources":["../src/benchmarks/types.ts"],"mappings":";;;KASY;KAQA;UAgBK,qBAAqB;;;EAGpC;;EAEA,SAAS;;EAET,QAAQ;;EAER,SAAS;;EAET,WAAW;;EAEX;;EAEA,SAAS;EACT,WAAW;;UAGI;;;;EAIf;;EAEA;;EAEA,aAAa;;;EAGb,KAAK;EACL;;UAGe;EACf;EACA;EACA;;EAEA;EACA;EACA;;EAEA;;;;;EAKA;;;UAQe,iBAAiB,kBAAkB,oBAAoB;;EAEtE;EACA,SAAS;EACT,WAAW;EACX;EACA,SAAS;EACT;;;;;EAKA,YAAY,OAAO,cAAc,QAAQ,qBAAqB;;EAE9D,SAAS,MAAM,qBAAqB,WAAW,UAAU,YAAY,QAAQ;;;;EAI7E,YAAY,iBAAiB;;UAGd,kBAAkB,4BAA4B;EAC7D;EACA;EACA,QAAQ;EACR,UAAU;EACV,UAAU;EACV,MAAM,qBAAqB;;KAGjB,mBAAmB,oBAAoB,uBAAuB;EACxE,UAAU,kBAAkB;EAC5B,MAAM,qBAAqB;EAC3B,SAAS;MACL,QAAQ,aAAa;;;cAoBd;;;;;;;;;iBAUG,mBACd,gBACA,gBACC"}
|
package/dist/wire/index.d.ts
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
import { b as CustomTokenPricing, c as CostLedgerHandle } from "../cost-ledger-DbQdN3nO.js";
|
|
2
|
-
import { x as ChatClient } from "../types-
|
|
2
|
+
import { x as ChatClient } from "../types-DLxe96rp.js";
|
|
3
3
|
import { s as TraceStore } from "../store-C4avFjIi.js";
|
|
4
|
-
import { m as FeedbackTrajectoryStore } from "../feedback-trajectory-
|
|
4
|
+
import { m as FeedbackTrajectoryStore } from "../feedback-trajectory-C6NJxXEa.js";
|
|
5
5
|
import { z } from "zod";
|
|
6
6
|
import { ServerType } from "@hono/node-server";
|
|
7
7
|
import { Hono } from "hono";
|
package/dist/wire/index.js
CHANGED
|
@@ -1,2 +1,2 @@
|
|
|
1
|
-
import { A as WIRE_VERSION, C as RUBRIC_VERSION_SCHEME, D as TracesIngestRequestSchema, E as TraceEventSchema, O as TracesIngestResponseSchema, S as ListRubricsResponseSchema, T as RubricSchema, _ as FeedbackIngestResponseSchema, a as runRpcBatch, b as JudgeRequestSchema, c as WireError, d as handleListRubrics, f as handleTracesIngest, g as ErrorResponseSchema, h as listBuiltinRubrics, i as dispatchRpc, j as hashRubric, k as VersionResponseSchema, l as handleFeedbackIngest, m as getBuiltinRubric, n as startServer, o as runRpcOnce, p as handleVersion, r as startServerAsync, s as buildOpenApi, t as createApp, u as handleJudge, v as FeedbackTrajectorySchema, w as RubricDimensionSchema, x as JudgeResultSchema, y as HealthResponseSchema } from "../server-
|
|
1
|
+
import { A as WIRE_VERSION, C as RUBRIC_VERSION_SCHEME, D as TracesIngestRequestSchema, E as TraceEventSchema, O as TracesIngestResponseSchema, S as ListRubricsResponseSchema, T as RubricSchema, _ as FeedbackIngestResponseSchema, a as runRpcBatch, b as JudgeRequestSchema, c as WireError, d as handleListRubrics, f as handleTracesIngest, g as ErrorResponseSchema, h as listBuiltinRubrics, i as dispatchRpc, j as hashRubric, k as VersionResponseSchema, l as handleFeedbackIngest, m as getBuiltinRubric, n as startServer, o as runRpcOnce, p as handleVersion, r as startServerAsync, s as buildOpenApi, t as createApp, u as handleJudge, v as FeedbackTrajectorySchema, w as RubricDimensionSchema, x as JudgeResultSchema, y as HealthResponseSchema } from "../server-CoBZO1fj.js";
|
|
2
2
|
export { ErrorResponseSchema, FeedbackIngestResponseSchema, FeedbackTrajectorySchema, HealthResponseSchema, JudgeRequestSchema, JudgeResultSchema, ListRubricsResponseSchema, RUBRIC_VERSION_SCHEME, RubricDimensionSchema, RubricSchema, TraceEventSchema, TracesIngestRequestSchema, TracesIngestResponseSchema, VersionResponseSchema, WIRE_VERSION, WireError, buildOpenApi, createApp, dispatchRpc, getBuiltinRubric, handleFeedbackIngest, handleJudge, handleListRubrics, handleTracesIngest, handleVersion, hashRubric, listBuiltinRubrics, runRpcBatch, runRpcOnce, startServer, startServerAsync };
|
|
@@ -0,0 +1,30 @@
|
|
|
1
|
+
# Unreleased: one physical-attempt allowance for structured calls
|
|
2
|
+
|
|
3
|
+
`createChatClient` and structured JSON calls now count schema negotiation,
|
|
4
|
+
temperature negotiation and transient retries against one `maximumAttempts`
|
|
5
|
+
allowance. Schema fallback no longer starts a second retry loop, resets the
|
|
6
|
+
cross-attempt deadline, or restarts raw-provider attempt indices.
|
|
7
|
+
|
|
8
|
+
`maximumAttempts: 1` makes at most one HTTP request. A provider rejecting the
|
|
9
|
+
schema is returned to the caller rather than followed by an unadmitted call.
|
|
10
|
+
Callers that allow negotiation can supply enough attempts, or select
|
|
11
|
+
`jsonSchemaTransport: 'json-object'` when that protocol is already known.
|
|
12
|
+
Low-level `callLlm` retains its existing no-schema-negotiation behavior.
|
|
13
|
+
|
|
14
|
+
`maximumChargeForLlmRequest` uses the same physical-attempt allowance rather
|
|
15
|
+
than reserving two full batches for a schema request. Its input bound still
|
|
16
|
+
includes the native schema, and output limits remain enforced per request.
|
|
17
|
+
Failed-attempt raw observations keep their order and the operation's original
|
|
18
|
+
identity. This does not claim provider-side deduplication or final-response
|
|
19
|
+
aggregation of every paid failed attempt; retain the execution evidence and
|
|
20
|
+
account at the existing caller-owned boundary.
|
|
21
|
+
|
|
22
|
+
There is one retry owner and no new public framework, dependency, credential
|
|
23
|
+
source, scheduler or product-specific transport. Product fetch guards protecting
|
|
24
|
+
older installed versions must remain until a release containing this change is
|
|
25
|
+
published and the compatible consumer set is installed and verified.
|
|
26
|
+
|
|
27
|
+
Proof: `tests/llm-physical-attempts.test.ts` runs the public client against a real
|
|
28
|
+
loopback HTTP server. Existing client, transient-status and raw-capture checks
|
|
29
|
+
remain in place. These are transport and allowance checks, not live-provider
|
|
30
|
+
cost or outcome-quality benchmarks.
|
|
@@ -0,0 +1,30 @@
|
|
|
1
|
+
# Model request lifetime and cost interpretation
|
|
2
|
+
|
|
3
|
+
The existing physical-attempt loop now keeps `timeoutMs` active until the full
|
|
4
|
+
response body has been read, for both successful and failed HTTP responses.
|
|
5
|
+
`deadlineMs` also bounds each remaining request and retry wait; an already-expired
|
|
6
|
+
operation does not dispatch. Cancelling during `Retry-After` or exponential backoff
|
|
7
|
+
stops that wait immediately. A late transport response after the deadline fails
|
|
8
|
+
with the cancellation error rather than becoming a result or a new retry.
|
|
9
|
+
|
|
10
|
+
The client reuses `combineAbortSignals` instead of maintaining separate signal
|
|
11
|
+
composition. Raw captures retain the existing attempt indices, operation identity
|
|
12
|
+
and redaction policy. User-defined capture callbacks must still complete or honor
|
|
13
|
+
their own cancellation; this change does not promise to terminate arbitrary code.
|
|
14
|
+
|
|
15
|
+
Results and receipts use the same reported-cost parser. Invalid negative or
|
|
16
|
+
non-finite provider prices are not exposed as observed cost; an explicitly reported
|
|
17
|
+
zero remains zero. Token-price estimates, when available, remain estimates. This
|
|
18
|
+
is provider-level parsing, not evidence of a Router customer charge: product
|
|
19
|
+
consumers must keep their customer receipt and payer attribution.
|
|
20
|
+
|
|
21
|
+
The full operation continues to use the caller's physical request allowance,
|
|
22
|
+
including schema and temperature negotiation. No hidden extra request batch,
|
|
23
|
+
dependency, new budget owner, default provider, or automatic result replay is added.
|
|
24
|
+
|
|
25
|
+
Run `pnpm exec vitest run src/llm-client.test.ts tests/llm-physical-attempts.test.ts`
|
|
26
|
+
for the maintained client and real-loopback HTTP contracts. These include a stalled
|
|
27
|
+
body after headers, cancellation during server-directed backoff, exhausted deadlines,
|
|
28
|
+
request-count conservation, raw-capture ordering and malformed reported prices.
|
|
29
|
+
Regenerate only the live analyst implementation pin with
|
|
30
|
+
`pnpm analyst:pin`; historical evidence pins remain fixed.
|
package/docs/concepts.md
CHANGED
|
@@ -154,6 +154,14 @@ that can seed memory, replay scenarios, and optimization.
|
|
|
154
154
|
| **Provenance** | Where a number came from: the package version, the source revision, the run identity, the exact attempt. |
|
|
155
155
|
| **`RunRecord`** | The analysis-time projection of one run: who ran, on what, with which seed, at what cost, and what it scored. |
|
|
156
156
|
|
|
157
|
+
### Read provider usage
|
|
158
|
+
|
|
159
|
+
Use the public `extractUsage` reader for retained provider response bodies.
|
|
160
|
+
It preserves reported cache reads and writes, including Router prompt details and `prompt_cache` fields.
|
|
161
|
+
An explicit cache zero remains zero; an absent cache component remains omitted.
|
|
162
|
+
The reader does not establish complete physical-call coverage or authoritative billing.
|
|
163
|
+
Keep raw usage and its source identity beside the projection.
|
|
164
|
+
|
|
157
165
|
### Improving a surface
|
|
158
166
|
|
|
159
167
|
| Term | Plain English |
|
|
@@ -0,0 +1,33 @@
|
|
|
1
|
+
# Distributed dispatch: execution, retries and receipts
|
|
2
|
+
|
|
3
|
+
`httpDispatch()` and `runDispatchServer()` are the existing `/adapters/http` transport. They do not own a second scheduler, an execution database, or an exactly-once protocol. Keep durable execution ownership in the worker's existing Run/retained-dispatch mechanism.
|
|
4
|
+
|
|
5
|
+
## Migration
|
|
6
|
+
|
|
7
|
+
`httpDispatch()` now makes **one attempt by default** (`retries: 0`, previously 2). A disconnected response or HTTP 500 does not establish that a worker did nothing. Applications that explicitly enable retries must bind request identity and external effects to a durable worker-side deduplication contract first. Existing explicit `retries` configuration remains supported.
|
|
8
|
+
|
|
9
|
+
Caller cancellation, terminal HTTP refusals, malformed success bodies, missing artifacts and receipt-import failures do not redispatch work. An error response carrying paid receipts is also terminal: import the already-settled spend, then report the failure. Request bytes and run/cell identities stay unchanged across explicitly enabled transport attempts. Backoff is cancellable.
|
|
10
|
+
|
|
11
|
+
The worker's optional `contextFactory` must supply a real `CampaignCostMeter` for paid work. A missing meter refuses `runPaidCall` before its execute callback. Deterministic work remains usable without a cost factory. Work performed outside the meter is not thereby free; the host must require and reconcile the appropriate evidence.
|
|
12
|
+
|
|
13
|
+
## Accounting boundary
|
|
14
|
+
|
|
15
|
+
Workers authorize and cap spend **before execution**. The client imports receipts after the operation; that cannot retroactively enforce a coordinator ceiling. Successful responses, failures and deliverable cancellation responses carry the same available settled receipts. Unknown cost stays unknown.
|
|
16
|
+
|
|
17
|
+
If the client disconnects before receiving a result, reconcile the worker's authoritative ledger. Do not reset an allowance or repeat the operation to reconstruct a lost receipt. This adapter does not implement that reconciliation or authorize an outer workflow to retry.
|
|
18
|
+
|
|
19
|
+
## Verification
|
|
20
|
+
|
|
21
|
+
The maintained `src/adapters/http.test.ts` uses real loopback Node HTTP, the actual client/server, real filesystem effects, and the native cost ledger. Five regressions distinguish the old behavior from the corrected boundary: terminal authorization, failure after a file effect, charged failure, charged cancellation and failed coordinator receipt import. The two existing connection/cancellation cases remain.
|
|
22
|
+
|
|
23
|
+
```sh
|
|
24
|
+
pnpm install --frozen-lockfile
|
|
25
|
+
pnpm exec vitest run src/adapters/http.test.ts
|
|
26
|
+
pnpm typecheck
|
|
27
|
+
pnpm build
|
|
28
|
+
pnpm verify:package
|
|
29
|
+
```
|
|
30
|
+
|
|
31
|
+
Ledger amounts in the transport cases are synthetic; no provider call is needed to prove this boundary. Passing them is not a model-quality, task-success, live deployment, isolation, or customer-savings claim.
|
|
32
|
+
|
|
33
|
+
For a real application comparison, use the existing campaign/experiment APIs over complete application jobs. Preserve assigned failures, immutable checker and policy identities, full cost evidence, fresh final tasks and existing controlled publication. A transport check and a customer-benefit experiment answer different questions.
|