@tangle-network/agent-eval 0.217.1 → 0.219.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +57 -1
- package/dist/adapters/http.d.ts +3 -3
- package/dist/adapters/http.d.ts.map +1 -1
- package/dist/adapters/http.js +1 -0
- package/dist/adapters/http.js.map +1 -1
- package/dist/{agent-profile-DTt8uGP9.d.ts → agent-profile-D01Io2Xw.d.ts} +4 -4
- package/dist/{agent-profile-DTt8uGP9.d.ts.map → agent-profile-D01Io2Xw.d.ts.map} +1 -1
- package/dist/analyst/index.d.ts +13 -13
- package/dist/analyst/index.d.ts.map +1 -1
- package/dist/analyst/index.js +10 -10
- package/dist/analyst/index.js.map +1 -1
- package/dist/{analyze-runs-DyeNcenW.d.ts → analyze-runs-BWAHBDAw.d.ts} +4 -4
- package/dist/{analyze-runs-DyeNcenW.d.ts.map → analyze-runs-BWAHBDAw.d.ts.map} +1 -1
- package/dist/{benchmark-command-xQK8nOI6.js → benchmark-command-BzmMTnvC.js} +664 -565
- package/dist/benchmark-command-BzmMTnvC.js.map +1 -0
- package/dist/benchmarks/index.d.ts +5 -5
- package/dist/benchmarks/index.js +3 -3
- package/dist/benchmarks/index.js.map +1 -1
- package/dist/campaign/index.d.ts +8 -8
- package/dist/campaign/index.js +11 -11
- package/dist/{campaign-CJGkxAQ6.js → campaign-BVPC6ODQ.js} +452 -282
- package/dist/campaign-BVPC6ODQ.js.map +1 -0
- package/dist/{campaign-evidence-CQYw8_0H.js → campaign-evidence-DunzcJGv.js} +2 -2
- package/dist/campaign-evidence-DunzcJGv.js.map +1 -0
- package/dist/{chat-client-ldqxYYTN.js → chat-client-CMQ8_h4J.js} +2 -2
- package/dist/{chat-client-ldqxYYTN.js.map → chat-client-CMQ8_h4J.js.map} +1 -1
- package/dist/{chat-json-call-DavzexSp.js → chat-json-call-BrZKn0L3.js} +2 -2
- package/dist/{chat-json-call-DavzexSp.js.map → chat-json-call-BrZKn0L3.js.map} +1 -1
- package/dist/cli.js +10 -10
- package/dist/{completion-verifier-B_jw6C9p.js → completion-verifier-Cigap8CG.js} +5 -5
- package/dist/{completion-verifier-B_jw6C9p.js.map → completion-verifier-Cigap8CG.js.map} +1 -1
- package/dist/contract/index.d.ts +11 -11
- package/dist/contract/index.js +91 -11
- package/dist/contract/index.js.map +1 -1
- package/dist/{cost-ledger-D0352Z4d.js → cost-ledger-BGZB6yY2.js} +75 -2
- package/dist/cost-ledger-BGZB6yY2.js.map +1 -0
- package/dist/{cost-ledger-Jk62OBfF.d.ts → cost-ledger-C1W3ICmP.d.ts} +26 -2
- package/dist/cost-ledger-C1W3ICmP.d.ts.map +1 -0
- package/dist/{default-registry-ClYjq3QE.js → default-registry-BrKmT01n.js} +310 -278
- package/dist/{default-registry-ClYjq3QE.js.map → default-registry-BrKmT01n.js.map} +1 -1
- package/dist/{default-registry-DXgLmoDI.d.ts → default-registry-Cwg_QN3d.d.ts} +6 -6
- package/dist/{default-registry-DXgLmoDI.d.ts.map → default-registry-Cwg_QN3d.d.ts.map} +1 -1
- package/dist/{define-agent-eval-BIkH_gAR.d.ts → define-agent-eval-C2nFhHPu.d.ts} +8 -8
- package/dist/{define-agent-eval-BIkH_gAR.d.ts.map → define-agent-eval-C2nFhHPu.d.ts.map} +1 -1
- package/dist/{define-agent-eval-BDxNlP3D.js → define-agent-eval-UQY3VbJ2.js} +165 -105
- package/dist/define-agent-eval-UQY3VbJ2.js.map +1 -0
- package/dist/diagnosis/index.d.ts +2 -2
- package/dist/{dspy-rlm-engine-DhdCr1Dc.js → dspy-rlm-engine-Bn_Mt_LW.js} +3 -3
- package/dist/{dspy-rlm-engine-DhdCr1Dc.js.map → dspy-rlm-engine-Bn_Mt_LW.js.map} +1 -1
- package/dist/{engine-BvZ0R-36.d.ts → engine-BpGGw6cd.d.ts} +3 -3
- package/dist/{engine-BvZ0R-36.d.ts.map → engine-BpGGw6cd.d.ts.map} +1 -1
- package/dist/eval-campaign-BUMIxKTE.js +393 -0
- package/dist/eval-campaign-BUMIxKTE.js.map +1 -0
- package/dist/{evaluation-jTjC_QQD.d.ts → evaluation-CBdQt_kr.d.ts} +4 -4
- package/dist/{evaluation-jTjC_QQD.d.ts.map → evaluation-CBdQt_kr.d.ts.map} +1 -1
- package/dist/evaluation.d.ts +1 -1
- package/dist/evaluation.js +1 -1
- package/dist/{exact-types-CflbJ9f3.d.ts → exact-types-BYFmYq44.d.ts} +2 -2
- package/dist/{exact-types-CflbJ9f3.d.ts.map → exact-types-BYFmYq44.d.ts.map} +1 -1
- package/dist/experiment/index.d.ts +24 -1
- package/dist/experiment/index.d.ts.map +1 -1
- package/dist/experiment/index.js +110 -71
- package/dist/experiment/index.js.map +1 -1
- package/dist/{exporters-Cl6IeNAu.js → exporters-DWvM16LE.js} +2 -2
- package/dist/{exporters-Cl6IeNAu.js.map → exporters-DWvM16LE.js.map} +1 -1
- package/dist/{external-optimizer-process-KVFGvYX1.js → external-optimizer-process-wOLfzsCC.js} +2 -2
- package/dist/{external-optimizer-process-KVFGvYX1.js.map → external-optimizer-process-wOLfzsCC.js.map} +1 -1
- package/dist/{external-optimizer-subprocess-XIHBOSKt.js → external-optimizer-subprocess-D670Oxf2.js} +4 -4
- package/dist/{external-optimizer-subprocess-XIHBOSKt.js.map → external-optimizer-subprocess-D670Oxf2.js.map} +1 -1
- package/dist/{feedback-trajectory-BWmhRBGu.d.ts → feedback-trajectory-DhB9WzYg.d.ts} +3 -3
- package/dist/{feedback-trajectory-BWmhRBGu.d.ts.map → feedback-trajectory-DhB9WzYg.d.ts.map} +1 -1
- package/dist/{findings-store-BMVQtEsm.js → findings-store-CvNo712L.js} +6 -6
- package/dist/{findings-store-BMVQtEsm.js.map → findings-store-CvNo712L.js.map} +1 -1
- package/dist/fuzz.d.ts +1 -1
- package/dist/fuzz.js +1 -1
- package/dist/{hf-dataset-D9f7-oX7.js → hf-dataset-CKxIJQba.js} +3 -3
- package/dist/{hf-dataset-D9f7-oX7.js.map → hf-dataset-CKxIJQba.js.map} +1 -1
- package/dist/hosted/index.js +1 -1
- package/dist/{index-CX-GAUpw.d.ts → index-CHxwfW5g.d.ts} +2 -2
- package/dist/{index-CX-GAUpw.d.ts.map → index-CHxwfW5g.d.ts.map} +1 -1
- package/dist/{index-BLGqZ81N.d.ts → index-CVFStrQD.d.ts} +8 -8
- package/dist/{index-BLGqZ81N.d.ts.map → index-CVFStrQD.d.ts.map} +1 -1
- package/dist/{index-9jJ2fKSX.d.ts → index-CalAsXMR.d.ts} +7 -5
- package/dist/index-CalAsXMR.d.ts.map +1 -0
- package/dist/{index-CiRwLv2R.d.ts → index-D7fmC-1K.d.ts} +3 -4
- package/dist/index-D7fmC-1K.d.ts.map +1 -0
- package/dist/{index-BiT6RtNH.d.ts → index-DAXfpVf7.d.ts} +10 -10
- package/dist/{index-BiT6RtNH.d.ts.map → index-DAXfpVf7.d.ts.map} +1 -1
- package/dist/{index-ZuWx52WG.d.ts → index-Q6DvdcwI.d.ts} +67 -24
- package/dist/index-Q6DvdcwI.d.ts.map +1 -0
- package/dist/index.d.ts +26 -26
- package/dist/index.d.ts.map +1 -1
- package/dist/index.js +615 -692
- package/dist/index.js.map +1 -1
- package/dist/{integrity-DOd9Q86O.js → integrity-C8oUeB0g.js} +322 -197
- package/dist/integrity-C8oUeB0g.js.map +1 -0
- package/dist/{integrity-_qgZLA5p.d.ts → integrity-CCwvezWS.d.ts} +2 -2
- package/dist/{integrity-_qgZLA5p.d.ts.map → integrity-CCwvezWS.d.ts.map} +1 -1
- package/dist/{journal-CR4QikYp.js → journal-C7om8Gum.js} +48 -38
- package/dist/journal-C7om8Gum.js.map +1 -0
- package/dist/judge-gate-Co5tuh57.js.map +1 -1
- package/dist/{judge-gate-DsL0yFcb.d.ts → judge-gate-Dc_mXQsL.d.ts} +2 -2
- package/dist/judge-gate-Dc_mXQsL.d.ts.map +1 -0
- package/dist/{kind-factory-Gy4fXixd.js → kind-factory-BhxbtUsQ.js} +2 -2
- package/dist/{kind-factory-Gy4fXixd.js.map → kind-factory-BhxbtUsQ.js.map} +1 -1
- package/dist/ledger-core/index.d.ts +1 -1
- package/dist/ledger-core/index.js +2 -2
- package/dist/ledger-core/lock-holder.d.ts +2 -0
- package/dist/ledger-core/lock-holder.js +181 -0
- package/dist/ledger-core/lock-holder.js.map +1 -0
- package/dist/{ledger-core-DrJhbms9.js → ledger-core-CAxUdDk0.js} +2 -2
- package/dist/{ledger-core-DrJhbms9.js.map → ledger-core-CAxUdDk0.js.map} +1 -1
- package/dist/{llm-client-BshCREn6.js → llm-client-DXuOTseU.js} +239 -249
- package/dist/llm-client-DXuOTseU.js.map +1 -0
- package/dist/{llm-judge-C8jMTNNG.js → llm-judge-Cc8vDJh_.js} +648 -459
- package/dist/llm-judge-Cc8vDJh_.js.map +1 -0
- package/dist/lock-holder-DaJbfpci.d.ts +86 -0
- package/dist/lock-holder-DaJbfpci.d.ts.map +1 -0
- package/dist/matrix/index.d.ts +2 -2
- package/dist/{matrix-DHe-dwVJ.d.ts → matrix-Cf-AGxGh.d.ts} +4 -4
- package/dist/{matrix-DHe-dwVJ.d.ts.map → matrix-Cf-AGxGh.d.ts.map} +1 -1
- package/dist/meta-eval/index.d.ts +2 -2
- package/dist/{mint-zszyYlpA.js → mint-CbdwjTPW.js} +2 -2
- package/dist/{mint-zszyYlpA.js.map → mint-CbdwjTPW.js.map} +1 -1
- package/dist/{mint-BRZXuwRi.d.ts → mint-CkzpVPdz.d.ts} +3 -3
- package/dist/{mint-BRZXuwRi.d.ts.map → mint-CkzpVPdz.d.ts.map} +1 -1
- package/dist/multishot/golden/index.d.ts +1 -1
- package/dist/multishot/index.d.ts +2 -2
- package/dist/multishot/index.js.map +1 -1
- package/dist/openapi.json +1 -1
- package/dist/{pareto-BzsxidFD.js → pareto-BiBnQXAE.js} +306 -212
- package/dist/pareto-BiBnQXAE.js.map +1 -0
- package/dist/pipelines/index.d.ts.map +1 -1
- package/dist/pipelines/index.js.map +1 -1
- package/dist/{prime-protocol-BWG2DQIk.d.ts → prime-protocol-CckHBxGu.d.ts} +3 -3
- package/dist/{prime-protocol-BWG2DQIk.d.ts.map → prime-protocol-CckHBxGu.d.ts.map} +1 -1
- package/dist/{promotion-gate-DaY95yU6.d.ts → promotion-gate-CkVZHagC.d.ts} +3 -3
- package/dist/{promotion-gate-DaY95yU6.d.ts.map → promotion-gate-CkVZHagC.d.ts.map} +1 -1
- package/dist/{proposal-findings-BbWidvUk.js → proposal-findings-E1aSC_cU.js} +2 -2
- package/dist/{proposal-findings-BbWidvUk.js.map → proposal-findings-E1aSC_cU.js.map} +1 -1
- package/dist/{registry-ssNoHzwD.d.ts → registry-B_yG-1jh.d.ts} +5 -5
- package/dist/{registry-ssNoHzwD.d.ts.map → registry-B_yG-1jh.d.ts.map} +1 -1
- package/dist/{render-C-w2EZ6p.js → render-Cs34y0iX.js} +404 -208
- package/dist/render-Cs34y0iX.js.map +1 -0
- package/dist/reporting.d.ts +3 -3
- package/dist/{researcher-0ABVIdGa.d.ts → researcher-H0p44OYC.d.ts} +5 -5
- package/dist/{researcher-0ABVIdGa.d.ts.map → researcher-H0p44OYC.d.ts.map} +1 -1
- package/dist/{reward-DrZ71vnf.d.ts → reward-BeUgVVC_.d.ts} +2 -2
- package/dist/{reward-DrZ71vnf.d.ts.map → reward-BeUgVVC_.d.ts.map} +1 -1
- package/dist/{reward-hacking-E52wq86n.d.ts → reward-hacking-WTxUbtbb.d.ts} +2 -2
- package/dist/{reward-hacking-E52wq86n.d.ts.map → reward-hacking-WTxUbtbb.d.ts.map} +1 -1
- package/dist/rl.d.ts +8 -8
- package/dist/rl.js +4 -4
- package/dist/rollout/index.d.ts +3 -3
- package/dist/rollout/index.js +5 -5
- package/dist/{rollout-D6C50mfA.js → rollout-Dd1ucT_M.js} +5 -5
- package/dist/{rollout-D6C50mfA.js.map → rollout-Dd1ucT_M.js.map} +1 -1
- package/dist/{rubric-predictive-validity-DiwT8PbL.d.ts → rubric-predictive-validity-noY7uE03.d.ts} +2 -2
- package/dist/{rubric-predictive-validity-DiwT8PbL.d.ts.map → rubric-predictive-validity-noY7uE03.d.ts.map} +1 -1
- package/dist/{run-record-CaxbLq6Y.d.ts → run-record-C3awiXPf.d.ts} +2 -2
- package/dist/{run-record-CaxbLq6Y.d.ts.map → run-record-C3awiXPf.d.ts.map} +1 -1
- package/dist/schema-C-ziyJD7.d.ts.map +1 -1
- package/dist/{schema-DxcKTzbi.js → schema-CvhfLjtZ.js} +77 -47
- package/dist/schema-CvhfLjtZ.js.map +1 -0
- package/dist/search/index.d.ts +2 -2
- package/dist/search/index.d.ts.map +1 -1
- package/dist/search/index.js +2 -2
- package/dist/{search-kernel-DxqjBxA-.d.ts → search-kernel-Dloxd6gs.d.ts} +5 -5
- package/dist/{search-kernel-DxqjBxA-.d.ts.map → search-kernel-Dloxd6gs.d.ts.map} +1 -1
- package/dist/{search-ledger-TUvB5gWH.js → search-ledger-CZRqtFbW.js} +252 -245
- package/dist/search-ledger-CZRqtFbW.js.map +1 -0
- package/dist/{search-ledger-file-UKCO2r8a.js → search-ledger-file-Bd5Tioq6.js} +3 -3
- package/dist/{search-ledger-file-UKCO2r8a.js.map → search-ledger-file-Bd5Tioq6.js.map} +1 -1
- package/dist/{search-shipper-Cw2z3-td.js → search-shipper-Dn3NLsuH.js} +3 -3
- package/dist/{search-shipper-Cw2z3-td.js.map → search-shipper-Dn3NLsuH.js.map} +1 -1
- package/dist/{search-summary-DQIvJsSB.js → search-summary-D4bwYnAE.js} +3 -3
- package/dist/{search-summary-DQIvJsSB.js.map → search-summary-D4bwYnAE.js.map} +1 -1
- package/dist/{server-DoVbqqtJ.js → server-8TS8XCvg.js} +3 -3
- package/dist/{server-DoVbqqtJ.js.map → server-8TS8XCvg.js.map} +1 -1
- package/dist/{single-run-lock-U58yf8N6.js → single-run-lock-Dekd8_Wv.js} +7 -7
- package/dist/single-run-lock-Dekd8_Wv.js.map +1 -0
- package/dist/{skillopt-optimization-method-kyL94-ri.js → skillopt-optimization-method-DLKqdhcR.js} +1384 -1059
- package/dist/skillopt-optimization-method-DLKqdhcR.js.map +1 -0
- package/dist/{statistical-heldout-C9AbgREN.d.ts → statistical-heldout-3z3I-kJN.d.ts} +3 -3
- package/dist/{statistical-heldout-C9AbgREN.d.ts.map → statistical-heldout-3z3I-kJN.d.ts.map} +1 -1
- package/dist/{store-otlp-BteDAbHK.js → store-otlp-dyWE3wAG.js} +2 -2
- package/dist/{store-otlp-BteDAbHK.js.map → store-otlp-dyWE3wAG.js.map} +1 -1
- package/dist/{store-tool-spans-D5l_OfTJ.js → store-tool-spans-DIUgZbzu.js} +3 -3
- package/dist/{store-tool-spans-D5l_OfTJ.js.map → store-tool-spans-DIUgZbzu.js.map} +1 -1
- package/dist/{store-tool-spans-DZS0HWVf.d.ts → store-tool-spans-DPwXHQkH.d.ts} +7 -7
- package/dist/{store-tool-spans-DZS0HWVf.d.ts.map → store-tool-spans-DPwXHQkH.d.ts.map} +1 -1
- package/dist/{summary-report-DM7tlml1.d.ts → summary-report-C8V49Jt3.d.ts} +18 -3
- package/dist/summary-report-C8V49Jt3.d.ts.map +1 -0
- package/dist/supervisor-run/index.d.ts.map +1 -1
- package/dist/supervisor-run/index.js +3 -3
- package/dist/{systemone-Bp_Ac0p1.d.ts → systemone-DVA7t76V.d.ts} +6 -6
- package/dist/{systemone-Bp_Ac0p1.d.ts.map → systemone-DVA7t76V.d.ts.map} +1 -1
- package/dist/{systemone-review-CwE6JGIp.d.ts → systemone-review-CUBMuTLi.d.ts} +2 -2
- package/dist/{systemone-review-CwE6JGIp.d.ts.map → systemone-review-CUBMuTLi.d.ts.map} +1 -1
- package/dist/systemone.d.ts +3 -3
- package/dist/{terminal-record-78g6E4vh.js → terminal-record-D4Rv-j1W.js} +241 -193
- package/dist/{terminal-record-78g6E4vh.js.map → terminal-record-D4Rv-j1W.js.map} +1 -1
- package/dist/{tool-groups-DyNz2JZ_.d.ts → tool-groups-LISbBWCg.d.ts} +3 -3
- package/dist/tool-groups-LISbBWCg.d.ts.map +1 -0
- package/dist/{trace-questions-xxwgsuMi.d.ts → trace-questions-C42fYEE4.d.ts} +4 -4
- package/dist/{trace-questions-xxwgsuMi.d.ts.map → trace-questions-C42fYEE4.d.ts.map} +1 -1
- package/dist/trace-repair/index.d.ts +3 -3
- package/dist/traces.d.ts +6 -6
- package/dist/traces.js +3 -3
- package/dist/trajectory-replay/index.d.ts +4 -1
- package/dist/trajectory-replay/index.d.ts.map +1 -1
- package/dist/trajectory-replay/index.js +854 -823
- package/dist/trajectory-replay/index.js.map +1 -1
- package/dist/{tree-BGRglkKd.js → tree-DSqwJpL9.js} +122 -14
- package/dist/tree-DSqwJpL9.js.map +1 -0
- package/dist/{types-CW8ewATZ.d.ts → types-8yQzNm8u.d.ts} +4 -4
- package/dist/{types-CW8ewATZ.d.ts.map → types-8yQzNm8u.d.ts.map} +1 -1
- package/dist/{types-D8GhJoHk.d.ts → types-CHfn1zg5.d.ts} +70 -72
- package/dist/types-CHfn1zg5.d.ts.map +1 -0
- package/dist/{types-DtMbFw6U.d.ts → types-D8GXK9Z5.d.ts} +3 -3
- package/dist/{types-DtMbFw6U.d.ts.map → types-D8GXK9Z5.d.ts.map} +1 -1
- package/dist/{types-D-mgRzlh.d.ts → types-eMhdacVp.d.ts} +2 -2
- package/dist/{types-D-mgRzlh.d.ts.map → types-eMhdacVp.d.ts.map} +1 -1
- package/dist/wire/index.d.ts +3 -3
- package/dist/wire/index.js +1 -1
- package/docs/campaign-proposers.md +16 -0
- package/docs/concepts.md +31 -0
- package/docs/design/statistics-decisions.md +1 -1
- package/docs/eval-surface-map.md +1 -0
- package/docs/profile-json-identity.md +1 -2
- package/docs/public-api.md +100 -88
- package/package.json +8 -6
- package/dist/benchmark-command-xQK8nOI6.js.map +0 -1
- package/dist/campaign-CJGkxAQ6.js.map +0 -1
- package/dist/campaign-evidence-CQYw8_0H.js.map +0 -1
- package/dist/cost-ledger-D0352Z4d.js.map +0 -1
- package/dist/cost-ledger-Jk62OBfF.d.ts.map +0 -1
- package/dist/define-agent-eval-BDxNlP3D.js.map +0 -1
- package/dist/eval-campaign-BLEfBGbA.js +0 -343
- package/dist/eval-campaign-BLEfBGbA.js.map +0 -1
- package/dist/index-9jJ2fKSX.d.ts.map +0 -1
- package/dist/index-CiRwLv2R.d.ts.map +0 -1
- package/dist/index-ZuWx52WG.d.ts.map +0 -1
- package/dist/integrity-DOd9Q86O.js.map +0 -1
- package/dist/journal-CR4QikYp.js.map +0 -1
- package/dist/judge-gate-DsL0yFcb.d.ts.map +0 -1
- package/dist/llm-client-BshCREn6.js.map +0 -1
- package/dist/llm-judge-C8jMTNNG.js.map +0 -1
- package/dist/pareto-BzsxidFD.js.map +0 -1
- package/dist/render-C-w2EZ6p.js.map +0 -1
- package/dist/schema-DxcKTzbi.js.map +0 -1
- package/dist/search-ledger-TUvB5gWH.js.map +0 -1
- package/dist/single-run-lock-U58yf8N6.js.map +0 -1
- package/dist/skillopt-optimization-method-kyL94-ri.js.map +0 -1
- package/dist/summary-report-DM7tlml1.d.ts.map +0 -1
- package/dist/tool-groups-DyNz2JZ_.d.ts.map +0 -1
- package/dist/tree-BGRglkKd.js.map +0 -1
- package/dist/types-D8GhJoHk.d.ts.map +0 -1
|
@@ -1,6 +1,6 @@
|
|
|
1
|
-
import {
|
|
2
|
-
import { a as RunRecord, d as RunTokenUsage } from "./run-record-
|
|
3
|
-
import { l as JudgeInput, x as ChatClient } from "./types-
|
|
1
|
+
import { o as CostLedgerHandle, u as CostProvenance } from "./cost-ledger-C1W3ICmP.js";
|
|
2
|
+
import { a as RunRecord, d as RunTokenUsage } from "./run-record-C3awiXPf.js";
|
|
3
|
+
import { l as JudgeInput, x as ChatClient } from "./types-eMhdacVp.js";
|
|
4
4
|
import { z } from "zod";
|
|
5
5
|
import { SpanKind } from "@tangle-network/agent-trace-contract";
|
|
6
6
|
import { RE2JS } from "re2js";
|
|
@@ -862,4 +862,4 @@ type AnalystRunEvent = {
|
|
|
862
862
|
};
|
|
863
863
|
//#endregion
|
|
864
864
|
export { TraceAnalystByteBudgets as $, FindingReaderLabel as A, ReadSpanSourceInput as B, FINDING_LAYERS as C, FindingMeasure as D, FindingLayer as E, lineageRoots as F, TraceAnalysisStoreContext as G, SpanSourceReader as H, parseAnalystFinding as I, ErrorCluster as J, DEFAULT_TRACE_ANALYST_BUDGETS as K, readFindingScope as L, FindingScopeReading as M, ProposalFindingSchema as N, FindingMethod as O, independentUnitCount as P, SpanMatchRecord as Q, createBoundedTraceAnalysisStore as R, makeProposalFinding as S, FindingEvidenceUnit as T, TRACE_ANALYSIS_LIMITS as U, ReadSpanSourceResult as V, TraceAnalysisStore as W, SearchSpanResult as X, QueryTracesPage as Y, SearchTraceResult as Z, ExecutionProbeRequest as _, AnalystFindingSchemaVersion as a, ViewSpansResult as at, computeFindingId as b, AnalystRunEvent as c, AnalystRunSummary as d, TraceAnalystFilters as et, AnalystSeverity as f, ExecutionProbeOutcome as g, ExecutionProbe as h, AnalystFinding as i, TraceAnalystTraceSummary as it, FindingScope as j, FindingParseOutcome as k, AnalystRunInputs as l, EvidenceRef as m, AnalystContext as n, TraceAnalystSpanKind as nt, AnalystInputKind as o, ViewTraceOversized as ot, AnalystUsageReceipt as p, DatasetOverview as q, AnalystCost as r, TraceAnalystSpanStatus as rt, AnalystRequirements as s, ViewTraceResult as st, Analyst as t, TraceAnalystSpan as tt, AnalystRunResult as u, ProposalFinding as v, FindingCalibration as w, makeFinding as x, ProposalFindingOrigin as y, BoundedTraceAnalysisStoreOptions as z };
|
|
865
|
-
//# sourceMappingURL=types-
|
|
865
|
+
//# sourceMappingURL=types-8yQzNm8u.d.ts.map
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"types-
|
|
1
|
+
{"version":3,"file":"types-8yQzNm8u.d.ts","names":[],"sources":["../src/trace-analyst/types.ts","../src/trace-analyst/store-contract.ts","../src/trace-analyst/store-boundary.ts","../src/analyst/finding-scope.ts","../src/analyst/types.ts"],"mappings":";;;;;;;;;;;;KAsBY,uBAAuB;KAEvB;;;;UAKK;EACf;EACA;EACA;EACA;EACA,MAAM;EACN;EACA;EACA;EACA,QAAQ;EACR;EACA;EACA;EACA;EACA;;;EAGA,YAAY;;UAGG;EACf;EACA;EACA;EACA;EACA;EACA;EACA;EACA;EACA;EACA;EACA;;UAGe;;EAEf;;EAEA;;EAEA;;EAEA;;EAEA;;EAEA;;EAEA;;;EAGA;;;;;;;UAQe;;EAEf;;EAEA;;EAEA;;EAEA;EACA;EACA;;EAEA;;EAEA;;EAEA;;UAGe;EACf;EACA;EACA;EACA;EACA;EACA;;EAEA;EACA;IAAU;IAAqB;;;;;;EAK/B,gBAAgB;EAChB;IAAc;IAAkB;;;UAGjB;EACf,QAAQ;EACR;EACA;;;;;UAMe;EACf;EACA,QAAQ;EACR,YAAY;;UAGG;EACf;;EAEA,gBAAgB;;EAEhB;EACA;;UAGe;EACf;EACA,OAAO;;EAEP;;EAEA;;EAEA;;EAEA;;UAGe;EACf;EACA;EACA;EACA,WAAW;;;EAGX;EACA;EACA;EACA;EACA;;UAGe;EACf;EACA,MAAM;EACN;;UAGe;EACf;EACA;EACA,MAAM;EACN;;;UAIe;;;EAGf;;;EAGA;;;EAGA;;;EAGA;;cAGW,+BAA+B;;;cClM/B;WACX;WACA;WACA;WACA;WACA;WACA;WACA;WACA;;UAGe;EACf,SAAS;;;UAIM;EACf;EACA;EACA;EACA;EACA;EACA;;KAGU;EAEN;EACA;EACA;EACA;EACA;EACA;;EAGA;EACA;EACA;EACA;EACA;EACA;EACA;EACA;EACA;EACA;IACE;IACA;IACA;IACA;IACA;;;;KAKI,oBACV,OAAO,qBACP,UAAU,8BACP,QAAQ;;;;;;;;UASI;;EAEf,iBAAiB;EAEjB,SAAS,kBAAkB,UAAU,4BAA4B;EAEjE,SACE;IAAS;IAAkB;KAC3B,UAAU,4BACT;EAEH,YACE,UAAU,qBACV,UAAU,4BACT,QAAQ;EAEX,YACE;IAAS,UAAU;IAAqB;IAAe;KACvD,UAAU,4BACT,QAAQ;EAEX,YAAY,UAAU,qBAAqB,UAAU,4BAA4B;EAEjF,UACE;IACE;IACA;KAEF,UAAU,4BACT,QAAQ;EAEX,UACE;IACE;IACA;IACA;KAEF,UAAU,4BACT,QAAQ;EAEX,YACE;IACE;IACA;IACA;KAEF,UAAU,4BACT,QAAQ;EAEX,WACE;IACE;IACA;IACA;IACA;KAEF,UAAU,4BACT,QAAQ;;UAGI;EACf,UAAU,QAAQ;;;;;iBC1GJ,gCACd,QAAQ,oBACR,UAAS,mCACR;;;;;;;;cCCU;KAaD,uBAAuB;UAOlB;;;;;;;;EAQf;;EAEA;;EAEA;IAAU;IAAc;;;EAExB;;UAGe;;EAEf;;EAEA;;EAEA;;UAGe;EACf;EACA;EACA;EACA,cAAc;;UAGC;EACf;EACA;EACA;EACA;;EAEA;;;;;;;;UASe;EACf;EACA,OAAO;EACP;;;cAuKW,uBAAqB,EAAA;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;GAGN,EAAA,KAAA;KAQhB;EACN;EAAiB,OAAO;;EACxB;EAAkB;IAAS;IAAc;;;;iBAG/B,oBAAoB,iBAAiB;;;;;UAiBpC;EACf,OAAO;EACP;;EAEA;EACA;EACA,QAAQ;;;;;;;;KASE;iBAEI,iBAAiB,SAAS,iBAAiB;;;;;;iBAkC3C,aAAa,OAAO,cAAc,uBAAuB;;iBA0BzD,qBAAqB,OAAO,cAAc;;;;;;;;;;;;;UCvVzC;EACf,gBAAgB;;;;;;;EAOhB;EACA;EACA;EACA,UAAU;;;;;;;EAOV;EACA;EACA;EACA,eAAe;EACf;EACA;;EAEA;;;;;;EAMA;;;;EAIA;;EAEA,WAAW;;EAEX,QAAQ;;EAER,QAAQ;;EAER,SAAS;;EAET,UAAU;;EAEV;;EAEA;;;;;EAKA;;;;;EAKA,iBAAiB;;;cAIb;KAEM,sCAAsC;;;cAIrC;KAED,0BAA0B;;KAG1B;;KAGA,kBAAkB;WACnB,iBAAiB;;UAGX;;;;;;;EAOf;EACA;EACA;;;;;;;;KAWU;UAOK;;EAEf;;EAEA;;EAEA;;EAEA;;UAGe;;EAEf;;EAEA;;;;;;UAOe;EACf,aAAa;EACb;EACA,YAAY;EACZ,aAAa;;EAEb,SAAS;;UAGM;EACf;;EAEA;;EAEA;;EAEA;;EAEA,aAAa;;EAEb;;;;;;;EAOA,OAAO;;;;;;;;;;EAUP,gBAAgB,cAAc;;;;;;;EAO9B,mBAAmB,cAAc;;;;;EAKjC,eAAe,SAAS;;EAExB,OAAO;;EAEP,OAAO,aAAa,SAAS;;EAE7B,SAAS;;;;;;;;EAQT,QAAQ;;;UAMO;EACf;;EAEA;;EAEA;;EAEA;EACA,SAAS;;;;;;;KAQC;EAEN;EACA;EACA;EACA;EACA;;EAEA;;EAEA;EAAkB;IAAS;IAAe;;;;;;;;UAO/B;;WAEN;EACT,QAAQ,SAAS,wBAAwB,QAAQ;;;;;;;;UASlC,QAAQ;;WAEd;;WAEA;WACA,WAAW;WACX,MAAM;WACN,WAAW;;WAEX;EACT,QAAQ,OAAO,QAAQ,KAAK,iBAAiB,QAAQ;;;UAItC;;EAEf;;EAEA,QAAQ;;EAER,MAAM;;EAEN;;;;;;;;EAQA;IAAkB;IAAsB;;;;;;;;EAOxC;;;;;;;;;;iBAac,iBAAiB;EAC/B;EACA;EACA;EACA;;EAEA;;;;;;iBAqCc,YACd,MAAM,KAAK;EACT;EACA;IAED;;iBAmBa,oBACd,MAAM,KAAK;EACT;EACA;IAED;UAOc;EACf;EACA;;EAEA;EACA;EACA;;EAEA,OAAO;;EAEP;IAAU;IAAe;;;UAGV;EACf;EACA;EACA;EACA;EACA,UAAU;EACV,aAAa;;EAEb;;;;;EAKA,wBAAwB;;;;;;;;;;;;;;;KAkBd;EAEN;EACA;EACA;EACA;;EAEA,aAAa;;EAGb;EACA,SAAS;;EAGT;EACA;EACA;;EAGA;;EAEA,SAAS;EACT,UAAU,cAAc;;EAGxB;EACA,QAAQ"}
|
|
@@ -1,11 +1,11 @@
|
|
|
1
|
-
import {
|
|
2
|
-
import { d as RunTokenUsage, s as RunSearchCoordinates } from "./run-record-
|
|
3
|
-
import { v as ProposalFinding } from "./types-
|
|
4
|
-
import { j as LlmCallMetadata } from "./types-
|
|
1
|
+
import { g as PaidCallResult, i as CostChannel, l as CostLedgerSummary, o as CostLedgerHandle, r as CostBreakdown, u as CostProvenance, y as RunPaidCallInput } from "./cost-ledger-C1W3ICmP.js";
|
|
2
|
+
import { d as RunTokenUsage, s as RunSearchCoordinates } from "./run-record-C3awiXPf.js";
|
|
3
|
+
import { v as ProposalFinding } from "./types-8yQzNm8u.js";
|
|
4
|
+
import { j as LlmCallMetadata } from "./types-eMhdacVp.js";
|
|
5
5
|
import { t as SeriesDistribution } from "./descriptive-B2iPaT9J.js";
|
|
6
|
-
import { A as SearchLedgerHash, G as SearchSplit, I as SearchNodeStatus, L as SearchOpenedEvent, N as SearchNodeDecision, O as SearchLedgerEntry, P as SearchNodeRef, Q as SearchTaskOutcome, R as SearchOperationKind, U as SearchReservation, Z as SearchTask, b as SearchClosedEvent, d as SearchCellCancelledEvent, i as SearchArtifactRef, l as SearchCandidateSurface, o as SearchAudit, p as SearchCellStage, t as NodeEstimate, w as SearchEdgeRecordedEvent, z as SearchOperationRecordedEvent } from "./search-ledger-types-p2xKfiDJ.js";
|
|
7
|
-
import { y as LedgerProjector } from "./index-
|
|
8
|
-
import { i as RolloutSearchLineage } from "./mint-
|
|
6
|
+
import { A as SearchLedgerHash, F as SearchNodeRegisteredEvent, G as SearchSplit, I as SearchNodeStatus, L as SearchOpenedEvent, N as SearchNodeDecision, O as SearchLedgerEntry, P as SearchNodeRef, Q as SearchTaskOutcome, R as SearchOperationKind, U as SearchReservation, Z as SearchTask, b as SearchClosedEvent, d as SearchCellCancelledEvent, i as SearchArtifactRef, l as SearchCandidateSurface, o as SearchAudit, p as SearchCellStage, t as NodeEstimate, w as SearchEdgeRecordedEvent, z as SearchOperationRecordedEvent } from "./search-ledger-types-p2xKfiDJ.js";
|
|
7
|
+
import { y as LedgerProjector } from "./index-CalAsXMR.js";
|
|
8
|
+
import { i as RolloutSearchLineage } from "./mint-CkzpVPdz.js";
|
|
9
9
|
//#region src/campaign/search-state-identity.d.ts
|
|
10
10
|
/** Deterministic node id: identical content in one search is one node. */
|
|
11
11
|
declare function searchNodeId(searchId: string, artifactDigest: string): string;
|
|
@@ -120,6 +120,48 @@ interface SearchCompletion {
|
|
|
120
120
|
complete: boolean;
|
|
121
121
|
reasons: string[];
|
|
122
122
|
}
|
|
123
|
+
interface NodeRecord {
|
|
124
|
+
event: SearchNodeRegisteredEvent;
|
|
125
|
+
ordinal: number;
|
|
126
|
+
registeredSequence: number;
|
|
127
|
+
updatedSequence: number;
|
|
128
|
+
edgeIds: string[];
|
|
129
|
+
parents: SearchNodeRef[];
|
|
130
|
+
parentKeys: Set<string>;
|
|
131
|
+
primaryParentId: string | null;
|
|
132
|
+
depth: number | null;
|
|
133
|
+
children: string[];
|
|
134
|
+
decisions: SearchDecisionRecord[];
|
|
135
|
+
finalist: boolean;
|
|
136
|
+
cellIds: string[];
|
|
137
|
+
scoredCells: number;
|
|
138
|
+
spend: SearchSpend;
|
|
139
|
+
}
|
|
140
|
+
//#endregion
|
|
141
|
+
//#region src/campaign/search-state-indexes.d.ts
|
|
142
|
+
declare class SearchLedgerIndexes {
|
|
143
|
+
readonly tasks: Map<string, {
|
|
144
|
+
split: SearchSplit;
|
|
145
|
+
unitId: string;
|
|
146
|
+
}>;
|
|
147
|
+
readonly nodes: Map<string, NodeRecord>;
|
|
148
|
+
readonly nodeOrder: string[];
|
|
149
|
+
readonly nodeByDigest: Map<string, string>;
|
|
150
|
+
readonly edges: Map<string, SearchEdgeRecordedEvent>;
|
|
151
|
+
readonly edgeOrder: string[];
|
|
152
|
+
readonly cells: Map<string, SearchCell>;
|
|
153
|
+
readonly operations: Map<string, SearchOperation>;
|
|
154
|
+
node(nodeId: string): SearchNode | undefined;
|
|
155
|
+
nodeIds(): readonly string[];
|
|
156
|
+
hasNode(nodeId: string): boolean;
|
|
157
|
+
nodeIdForDigest(artifactDigest: string): string | undefined;
|
|
158
|
+
edge(edgeId: string): SearchEdgeRecordedEvent | undefined;
|
|
159
|
+
edgeIds(): readonly string[];
|
|
160
|
+
cell(cellId: string): SearchCell | undefined;
|
|
161
|
+
cellIds(nodeId?: string): readonly string[];
|
|
162
|
+
operation(operationId: string): SearchOperation | undefined;
|
|
163
|
+
scoredCells(nodeId: string, split: SearchSplit): SearchScoredCell[];
|
|
164
|
+
}
|
|
123
165
|
//#endregion
|
|
124
166
|
//#region src/campaign/search-state-view.d.ts
|
|
125
167
|
/**
|
|
@@ -201,21 +243,15 @@ declare class SearchState implements LedgerProjector<SearchLedgerEntry, SearchSt
|
|
|
201
243
|
private head;
|
|
202
244
|
private lastOccurredAt;
|
|
203
245
|
private lastOccurredAtText;
|
|
204
|
-
private readonly tasks;
|
|
205
|
-
private readonly nodes;
|
|
206
|
-
private readonly nodeOrder;
|
|
207
|
-
private readonly nodeByDigest;
|
|
208
|
-
private readonly edges;
|
|
209
|
-
private readonly edgeOrder;
|
|
210
|
-
private readonly cells;
|
|
211
|
-
private readonly operations;
|
|
212
246
|
private selectedNodeId;
|
|
213
|
-
private claimUsedUsd;
|
|
214
247
|
/** Nodes ever decided `finalist`. */
|
|
215
248
|
private readonly finalists;
|
|
216
249
|
/** Claim cells allocated. Once one exists, the finalists are fixed. */
|
|
217
250
|
private claimCells;
|
|
218
251
|
private readonly audit;
|
|
252
|
+
private readonly spend;
|
|
253
|
+
/** @internal The live indexes a `SearchStateView` reads. */
|
|
254
|
+
readonly index: SearchLedgerIndexes;
|
|
219
255
|
constructor(searchId: string);
|
|
220
256
|
apply(entry: SearchLedgerEntry, index: number): void;
|
|
221
257
|
snapshot(): SearchStateView;
|
|
@@ -238,62 +274,22 @@ declare class SearchState implements LedgerProjector<SearchLedgerEntry, SearchSt
|
|
|
238
274
|
private assertSearching;
|
|
239
275
|
/** @internal */
|
|
240
276
|
incompleteReasons(): string[];
|
|
241
|
-
/**
|
|
242
|
-
*
|
|
243
|
-
|
|
244
|
-
|
|
245
|
-
|
|
246
|
-
|
|
247
|
-
|
|
248
|
-
|
|
249
|
-
|
|
250
|
-
|
|
251
|
-
|
|
252
|
-
|
|
253
|
-
|
|
254
|
-
|
|
255
|
-
|
|
256
|
-
* back; refusing them would leave a search that can never close. */
|
|
257
|
-
private admit;
|
|
258
|
-
/** @internal The largest hold a new non-claim cell or operation may take,
|
|
259
|
-
* and the claim reserve still unspent. */
|
|
260
|
-
readBudget(): {
|
|
261
|
-
headroomUsd: number | null;
|
|
262
|
-
claimReserveUsd: number;
|
|
277
|
+
/** @internal What a `SearchStateView` copies at construction: the header,
|
|
278
|
+
* close, head and last timestamp, a snapshot of the audit, and the budget room. */
|
|
279
|
+
readPosition(): {
|
|
280
|
+
header: SearchOpenedEvent | null;
|
|
281
|
+
closed: SearchClosedEvent | null;
|
|
282
|
+
head: {
|
|
283
|
+
sequence: number;
|
|
284
|
+
entryHash: SearchLedgerHash;
|
|
285
|
+
} | null;
|
|
286
|
+
lastOccurredAt: string | null;
|
|
287
|
+
audit: SearchAudit;
|
|
288
|
+
budget: {
|
|
289
|
+
headroomUsd: number | null;
|
|
290
|
+
claimReserveUsd: number;
|
|
291
|
+
};
|
|
263
292
|
};
|
|
264
|
-
/** @internal */
|
|
265
|
-
readHeader(): SearchOpenedEvent | null;
|
|
266
|
-
/** @internal */
|
|
267
|
-
readClosed(): SearchClosedEvent | null;
|
|
268
|
-
/** @internal */
|
|
269
|
-
readHead(): {
|
|
270
|
-
sequence: number;
|
|
271
|
-
entryHash: SearchLedgerHash;
|
|
272
|
-
} | null;
|
|
273
|
-
/** @internal */
|
|
274
|
-
readLastOccurredAt(): string | null;
|
|
275
|
-
/** @internal */
|
|
276
|
-
readAudit(): SearchAudit;
|
|
277
|
-
/** @internal */
|
|
278
|
-
readNode(nodeId: string): SearchNode | undefined;
|
|
279
|
-
/** @internal */
|
|
280
|
-
readNodeIds(): readonly string[];
|
|
281
|
-
/** @internal */
|
|
282
|
-
readHasNode(nodeId: string): boolean;
|
|
283
|
-
/** @internal */
|
|
284
|
-
readNodeIdForDigest(artifactDigest: string): string | undefined;
|
|
285
|
-
/** @internal */
|
|
286
|
-
readEdge(edgeId: string): SearchEdgeRecordedEvent | undefined;
|
|
287
|
-
/** @internal */
|
|
288
|
-
readEdgeIds(): readonly string[];
|
|
289
|
-
/** @internal */
|
|
290
|
-
readCell(cellId: string): SearchCell | undefined;
|
|
291
|
-
/** @internal */
|
|
292
|
-
readCellIds(nodeId?: string): readonly string[];
|
|
293
|
-
/** @internal */
|
|
294
|
-
readOperation(operationId: string): SearchOperation | undefined;
|
|
295
|
-
/** @internal */
|
|
296
|
-
readScoredCells(nodeId: string, split: SearchSplit): SearchScoredCell[];
|
|
297
293
|
}
|
|
298
294
|
//#endregion
|
|
299
295
|
//#region src/campaign/search-summary.d.ts
|
|
@@ -851,6 +847,8 @@ interface LabeledScenarioStore {
|
|
|
851
847
|
}>;
|
|
852
848
|
}
|
|
853
849
|
interface CampaignCellResult<TArtifact> {
|
|
850
|
+
/** Router USD, sandbox estimate and standard seat credits remain separate units. */
|
|
851
|
+
costBreakdown?: CostBreakdown;
|
|
854
852
|
/** Manifest that produced this cell. Resumability refuses to reuse a cell
|
|
855
853
|
* whose manifest differs from the current run. */
|
|
856
854
|
manifestHash?: string;
|
|
@@ -1019,4 +1017,4 @@ interface CampaignResult<TArtifact = unknown, TScenario extends Scenario = Scena
|
|
|
1019
1017
|
}
|
|
1020
1018
|
//#endregion
|
|
1021
1019
|
export { SearchCompletion as $, LabeledScenarioWrite as A, ScoredSurfaceOutcome as B, JudgeDimension as C, LabeledScenarioSampleArgs as D, LabeledScenarioRecord as E, ProposeContext as F, SearchProposerView as G, SurfaceProposer as H, ProposedCandidate as I, searchProposerView as J, SearchSummaryOptions as K, RedactionStatus as L, OptimizerConfig as M, ParetoParent as N, LabeledScenarioSource as O, ProposalTrackContext as P, SearchCell as Q, Scenario as R, JudgeConfig as S, LabelTrust as T, TraceSpan as U, SessionScript as V, isProposedCandidate as W, SearchStateView as X, SearchState as Y, searchUnitScores as Z, GateDecision as _, CampaignResult as a, SearchUnitScore as at, GenerationRecord as b, CampaignTraceWriter as c, searchTaskSetDigest as ct, DispatchContext as d, SearchDecisionRecord as et, DispatchFn as f, GateContribution as g, GateContext as h, CampaignCostMeter as i, SearchSpend as it, MutableSurface as j, LabeledScenarioStore as k, CodeSurface as l, GateCheckStatus as m, CampaignArtifactWriter as n, SearchOperation as nt, CampaignScenarioIdentity as o, searchCellId as ot, Gate as p, renderSearchSummary as q, CampaignCellResult as r, SearchScoredCell as rt, CampaignTokenUsage as s, searchNodeId as st, CampaignAggregates as t, SearchNode as tt, ComponentSurface as u, GateResult as v, JudgeScore as w, JudgeAggregate as x, GenerationCandidate as y, ScenarioAggregate as z };
|
|
1022
|
-
//# sourceMappingURL=types-
|
|
1020
|
+
//# sourceMappingURL=types-CHfn1zg5.d.ts.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"types-CHfn1zg5.d.ts","names":[],"sources":["../src/campaign/search-state-identity.ts","../src/campaign/search-state-types.ts","../src/campaign/search-state-indexes.ts","../src/campaign/search-state-view.ts","../src/campaign/search-state-projection.ts","../src/campaign/search-summary.ts","../src/campaign/types.ts"],"mappings":";;;;;;;;;;iBAOgB,aAAa,kBAAkB;;iBAK/B,aACd,kBACA,gBACA,gBACA,OAAO,aACP;;iBAMc,oBAAoB,gBAAgB,eAAe;;;UCHlD;EACf;EACA,UAAU;EACV,OAAO;EACP;EACA;;UAGe;EACf;EACA;EACA;EACA;;;;UAKe;EACf;EACA;EACA;EACA;;;UAIe;EACf;EACA;;EAEA;EACA;;UAGe;EACf;;EAEA;EACA,gBAAgB;EAChB,UAAU;EACV,mBAAmB;;;EAGnB;;EAEA;;;;EAIA,kBAAkB;EAClB;EACA;EACA,QAAQ;EACR;;EAEA;EACA,oBAAoB;EACpB;EACA,OAAO;EACP;EACA;;UAGe;EACf;EACA;EACA;EACA;EACA,OAAO;EACP;EACA,OAAO;EACP;EACA,aAAa;EACb;EACA;EACA,SAAS;;EAET;;EAEA;EACA,WAAW;;EAEX;;EAEA;EACA;EACA;EACA;;UAGe;EACf;EACA,eAAe;EACf,aAAa;EACb;;EAEA,SAAS;;;EAGT,WAAW;;;EAGX,oBAAoB;;EAEpB;;EAEA;;UAGe;EACf;EACA;;UAGe;EACf,OAAO;EACP;EACA;EACA;EACA;EACA,SAAS;EACT,YAAY;EACZ;EACA;EACA;EACA,WAAW;EACX;EACA;EACA;EACA,OAAO;;;;cCvII;WACF,OAAK;IAA4B,OAAA;IAAqB;;WACtD,OAAK,YAAA;WACL;WACA,cAAY;WACZ,OAAK,YAAA;WACL;WACA,OAAK,YAAA;WACL,YAAU,YAAA;EAEnB,KAAK,iBAAiB;EAKtB;EAIA,QAAQ;EAIR,gBAAgB;EAIhB,KAAK,iBAAiB;EAItB;EAIA,KAAK,iBAAiB;EAKtB,QAAQ;EAKR,UAAU,sBAAsB;EAKhC,YAAY,gBAAgB,OAAO,cAAc;;;;;;;;;;iBChCnC,iBAAiB,gBAAgB,qBAAqB;;;;;;cAyBzD;WACF;WACA,QAAQ;WACR,QAAQ;WACR;IAAQ;IAAkB,WAAW;;;WAErC;WACA,OAAO;WACP,YAAY;;;;;;;WAOZ;IAAU;IAA4B;;mBAC9B;mBACA;EAEjB,YAAY,OAAO,aAAa;;MAiB5B;EAIJ,KAAK,iBAAiB;EAItB,QAAQ;;EAKR;;EAKA,SAAS;EAKT,gBAAgB;EAIhB,KAAK,iBAAiB;;EAKtB,SAAS;EAKT,KAAK,iBAAiB;;EAKtB,MAAM;IAAU;MAAyB;EAKzC,UAAU,sBAAsB;;;EAMhC,YAAY,gBAAgB,OAAO,cAAc;;EAKjD,WAAW,gBAAgB,OAAO,cAAc;;EAKhD,QAAQ,aAAa,uBAAuB;UAmCpC;;;;;;;;;cCnJG,uBAAuB,gBAAgB,mBAAmB;WAC5D;UACD;UACA;UACA;UACA;UACA;UACA;UACA;;mBAES;;UAET;mBACS;mBACA;;WAER,OAAK;EAEd,YAAY;EAiCZ,MAAM,OAAO,mBAAmB;EA6BhC,YAAY;;EAKZ,KAAK;UAIG;UAgCA;UA6CA;UA2BA;UA8BA;UA8CA;UAsHA;UAwEA;UAyEA;UAiBA;UAmDA;UAaA;;;UA2EA;;EASR;;;EAmBA;IACE,QAAQ;IACR,QAAQ;IACR;MAAQ;MAAkB,WAAW;;IACrC;IACA,OAAO;IACP;MAAU;MAA4B;;;;;;;;UCjuBzB;WACN;WACA;;;EAGT,YAAY,0BAA0B;;EAEtC,WAAW,0BAA0B;;;iBAIvB,mBAAmB,OAAO,kBAAkB;UAW3C;;;;;;;;EAQf,OAAO;;EAEP;;;;;;;;;;;iBAYc,oBAAoB,OAAO,iBAAiB,SAAS;;;;;UCrCpD;EACf;EACA;EACA;;;;;EAKA;;;UAIe,iCAAiC,KAAK;EACrD;;;;;UAMe;EACf;;EAEA;EACA;EACA;EACA;EACA,QAAQ;EACR,OAAO;EACP,WAAW;EACX,MAAM;;EAEN;;EAEA;;;;;;;;EAQA;;;;KAKU,WAAW,kBAAkB,UAAU,cACjD,UAAU,WACV,KAAK,oBACF,QAAQ;;;;UAOI,cAAc,WAAW;EACxC;EACA;EACA;;EAEA;;;EAGA,sBAAsB,UAAU,WAAW,sBAAsB,UAAU,cAAc;;UAK1E;;EAEf;;EAEA;;;;;;;;;UAUe,YAAY,WAAW,kBAAkB,WAAW;EACnE;EACA,YAAY;;;;EAIZ;;;EAGA,MAAM;IACJ,UAAU;IACV,UAAU;IACV,QAAQ;;IAER,aAAa;IACb;IACA,WAAW;MACT,aAAa,QAAQ;EACzB,aAAa,UAAU;;;;;;;;;;;UAYR;EACf,YAAY;EACZ;EACA;;EAEA,UAAU;;;;;;;EAOV;;;;EAIA,eAAe,eAAe;IAAQ;IAAe;;;;;EAIrD;;;EAGA;;EAEA;;EAEA,WAAW,eAAe;;;;;;;UAUX;WACN;;;WAGA;;WAEA;;WAEA;;WAEA;;WAEA;;WAEA;;;WAGA;aACE;aACA;aACA;;;WAGF;;;UAIM;WACN;WACA,YAAY,SAAS;;;;;;;;;;;KAYpB,0BAA0B,mBAAmB;;;;;;;UAQxC;EACf,SAAS;;EAET;;;;EAIA;;;;;;EAMA,cAAc,SAAS;;;;iBAKT,oBACd,OAAO,iBAAiB,oBACvB,SAAS;;;;;;;;;UAkBK;EACf,SAAS;EACT;;;EAGA,YAAY;;;EAGZ;;EAEA;EACA;EACA;;;UAIe;EACf;EACA;EACA;EACA;EACA;;;;;UAMe;;;EAGf;;EAEA;EACA;EACA;EACA,YAAY;;;;;EAKZ,WAAW;IAAQ;IAAoB;IAAmB;IAAgB;;EAC1E;IACE;IACA;;;;;UAMa,eAAe,YAAY;;;;WAIjC,gBAAgB;;;;;WAKhB;;;;WAIA,UAAU;IAAgB;IAAgB,UAAU;;;;WAGpD,QAAQ;;;;WAIR;;;WAGA,SAAS,cAAc;WACvB,UAAU,cAAc;;WAExB;WACA;WACA,QAAQ;;WAER,QAAQ;;;WAGR,kBAAkB;;;WAGlB,mBAAmB;;;;WAInB,gBAAgB;;;;WAIhB;;;;;;;WAOA,gBAAgB,cAAc;;WAE9B,aAAa;WACb;;;;;;;;;;;;;;UAeM,gBAAgB,YAAY;EAC3C;;;;;EAKA,QAAQ,KAAK,eAAe,aAAa,QAAQ,MAAM,iBAAiB;;;EAGxE,QAAQ;IAAQ,SAAS,cAAc;;IAAwB;IAAe;;;UAGtE;EACR;EACA;EACA,mBAAmB,qBAAqB;;UAGzB,wBAAwB;EACvC,UAAU;;;KAMA;;KAGA;UAEK;EACf;EACA,QAAQ;EACR;;UAGe,YAAY,WAAW,kBAAkB;EACxD,oBAAoB,YAAY;EAChC,oBAAoB,YAAY;;EAEhC,aAAa,YAAY,eAAe;;;;;EAKxC,sBAAsB,YAAY,eAAe;;;;;;;EAOjD,yBAAyB,YAAY,eAAe;;;EAGpD,uBAAuB,YAAY;EACnC,WAAW;EACX;IAAQ;IAAmB;;;EAE3B,aAAa;EACb;EACA,QAAQ;;UAGO;EACf,UAAU;EACV;EACA,mBAAmB;EACnB;;;UAIe,KAAK,qBAAqB,kBAAkB,WAAW;EACtE;EACA,OAAO,KAAK,YAAY,WAAW,aAAa,QAAQ;;;;UAOzC;EACf,KAAK,cAAc,aAAa,0BAA0B;EAC1D,SAAS;;UAGM;EACf,IAAI,aAAa;EACjB,aAAa,aAAa;;;;;UAMX;EACf,MAAM,cAAc,kBAAkB,aAAa;EACnD,UAAU,cAAc,iBAAiB;;;;;;KAO/B,qBAAqB;;;;;UAMhB;;EAEf,YAAY,GACV,OAAO,KAAK,iBAAiB;IAC3B,UAAU;MAEX,QAAQ,eAAe;;;;;KAQhB;KAOA;;;;;;;;;;;;;;;KAgBA;;;;UAgBK,qBAAqB,kBAAkB,WAAW,UAAU;EAC3E,UAAU;EACV,UAAU;EACV,aAAa,eAAe;EAC5B,QAAQ;EACR;EACA;EACA,iBAAiB;;;;;EAKjB,aAAa;;EAEb;;UAGe,sBAAsB,kBAAkB,WAAW,UAAU,6BACpE,qBAAqB,WAAW;;EAExC;;;EAGA;;UAGe;EACf;;EAEA;;;;EAIA;EACA;IACE;IACA,SAAS,wBAAwB;IACjC;IACA;;;;;IAKA,WAAW;;;UAIE;EACf,QAAQ,OAAO,uBAAuB;EACtC,OAAO,MAAM,4BAA4B,QAAQ;EACjD,QAAQ;IACN;IACA;IACA,UAAU;;;IAGV,SAAS,OAAO;;;UAMH,mBAAmB;;EAElC,gBAAgB;;;EAGhB;EACA;;EAEA;IAAY;IAAsB;;EAClC;EACA;EACA;EACA,UAAU;EACV,aAAa,eAAe;;;EAG5B;;EAEA,gBAAgB;;EAEhB;;;EAGA,YAAY;;;EAGZ;;;EAGA;EACA;EACA;EACA;;;;EAIA;;EAEA;;EAEA;EACA;;UAGe;EACf;EACA;EACA;EACA;;;;;;EAMA,cAAc;;UAGC;EACf;EACA;EACA;;;EAGA,cAAc;;UAGC;EACf;EACA,YAAY;EACZ;;;;;;UAOe;EACf;;;;EAIA;;;;EAIA;;;;EAIA;IACE;IACA;IACA,iBAAiB;MAAQ;MAAgB;;;;;EAI3C,YAAY;;;;;;;;;;EAUZ,WAAW;IAAQ;IAAoB;IAAmB;IAAgB;;;;EAG1E;;;;EAIA;;;;EAIA,cAAc,SAAS;;UAGR;EACf,SAAS,eAAe;EACxB,YAAY,eAAe;;EAE3B,MAAM;;EAEN;EACA;EACA;;EAEA;;EAEA;;EAEA;;EAEA;;UAGe,eAAe,qBAAqB,kBAAkB,WAAW;;EAEhF;;EAEA;EACA;;EAEA;EACA;EACA;EACA;EACA,OAAO,MAAM,mBAAmB;EAChC,YAAY;EACZ;IACE,aAAa;IACb;;EAEF,OAAO;EACP;EACA;EACA,iBAAiB;;;EAGjB,WAAW,MAAM,2BAA2B,KAAK"}
|
|
@@ -1,5 +1,5 @@
|
|
|
1
|
-
import { c as RunSplitTag } from "./run-record-
|
|
2
|
-
import { R as Scenario, d as DispatchContext } from "./types-
|
|
1
|
+
import { c as RunSplitTag } from "./run-record-C3awiXPf.js";
|
|
2
|
+
import { R as Scenario, d as DispatchContext } from "./types-CHfn1zg5.js";
|
|
3
3
|
//#region src/benchmarks/types.d.ts
|
|
4
4
|
type BenchmarkTaskKind = 'retrieval' | 'rag-answer' | 'hallucination' | 'kb-improvement' | 'routing' | 'custom';
|
|
5
5
|
type BenchmarkFamily = 'beir' | 'mteb-retrieval' | 'msmarco' | 'trec-dl' | 'miracl' | 'lotte' | 'bright' | 'crag' | 'hotpotqa' | 'kilt' | 'ragtruth' | 'faithbench' | 'first-party' | 'custom';
|
|
@@ -99,4 +99,4 @@ declare const BENCHMARK_SPLIT_SEED = "agent-eval-v1";
|
|
|
99
99
|
declare function deterministicSplit(itemId: string, seed?: string): RunSplitTag;
|
|
100
100
|
//#endregion
|
|
101
101
|
export { BenchmarkFamily as a, BenchmarkSource as c, BenchmarkEvaluation as i, BenchmarkTaskKind as l, BenchmarkAdapter as n, BenchmarkResponder as o, BenchmarkDatasetItem as r, BenchmarkScenario as s, BENCHMARK_SPLIT_SEED as t, deterministicSplit as u };
|
|
102
|
-
//# sourceMappingURL=types-
|
|
102
|
+
//# sourceMappingURL=types-D8GXK9Z5.d.ts.map
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"types-
|
|
1
|
+
{"version":3,"file":"types-D8GXK9Z5.d.ts","names":[],"sources":["../src/benchmarks/types.ts"],"mappings":";;;KASY;KAQA;UAgBK,qBAAqB;;;EAGpC;;EAEA,SAAS;;EAET,QAAQ;;EAER,SAAS;;EAET,WAAW;;EAEX;;EAEA,SAAS;EACT,WAAW;;UAGI;;;;EAIf;;EAEA;;EAEA,aAAa;;;EAGb,KAAK;EACL;;UAGe;EACf;EACA;EACA;;EAEA;EACA;EACA;;EAEA;;;;;EAKA;;;UAQe,iBAAiB,kBAAkB,oBAAoB;;EAEtE;EACA,SAAS;EACT,WAAW;EACX;EACA,SAAS;EACT;;;;;EAKA,YAAY,OAAO,cAAc,QAAQ,qBAAqB;;EAE9D,SAAS,MAAM,qBAAqB,WAAW,UAAU,YAAY,QAAQ;;;;EAI7E,YAAY,iBAAiB;;UAGd,kBAAkB,4BAA4B;EAC7D;EACA;EACA,QAAQ;EACR,UAAU;EACV,UAAU;EACV,MAAM,qBAAqB;;KAGjB,mBAAmB,oBAAoB,uBAAuB;EACxE,UAAU,kBAAkB;EAC5B,MAAM,qBAAqB;EAC3B,SAAS;MACL,QAAQ,aAAa;;;cAoBd;;;;;;;;;iBAUG,mBACd,gBACA,gBACC"}
|
|
@@ -1,4 +1,4 @@
|
|
|
1
|
-
import {
|
|
1
|
+
import { l as CostLedgerSummary, m as CustomTokenPricing, o as CostLedgerHandle } from "./cost-ledger-C1W3ICmP.js";
|
|
2
2
|
import { t as AgentEvalError } from "./errors-DEE6u6ot.js";
|
|
3
3
|
//#region src/judge-families.d.ts
|
|
4
4
|
/**
|
|
@@ -806,4 +806,4 @@ interface CheckResult {
|
|
|
806
806
|
}
|
|
807
807
|
//#endregion
|
|
808
808
|
export { AssertServedModelOptions as $, createChatClient as A, LlmToolDefinition as B, ChatResponse as C, MockTransportOpts as D, CustomTransportOpts as E, LlmMessage as F, InMemoryRawProviderSinkOptions as G, FileSystemRawProviderSink as H, LlmThinkingMode as I, RawProviderDirection as J, NoopRawProviderSink as K, LlmTokenLogprob as L, LlmCallRequest as M, LlmCallResult as N, OpenAiCompatibleTransportOpts as O, LlmChargeBounds as P, AssertCrossFamilyServedOptions as Q, LlmToolCall as R, ChatRequest as S, CreateChatClientOpts as T, FileSystemRawProviderSinkOptions as U, LlmUsage as V, InMemoryRawProviderSink as W, RawProviderSink as X, RawProviderEvent as Y, RawProviderSinkFilter as Z, ScenarioResult as _, CompletionCriterion as a, assertServedModel as at, ChatCallOpts as b, JudgeFn as c, AssertCrossFamilyOptions as ct, PersonaConfig as d, assertCrossFamily as dt, ModelSubstitutionError as et, ProductClientConfig as f, judgeFamily as ft, ScenarioFile as g, RubricDimension as h, CollectedArtifacts as i, assertCrossFamilyServed as it, LlmCallMetadata as j, SandboxSdkTransportOpts as k, JudgeInput as l, CrossFamilyError as lt, RouteMap as m, ArtifactResult as n, ServedModelPolicy as nt, DimensionJudgeScore as o, assertServedModels as ot, ProductScenario as p, ProviderRedactor as q, CheckResult as r, ServedModelVerdict as rt, DriverState as s, checkServedModel as st, ArtifactCheck as t, ServedModelCheck as tt, JudgeRubric as u, JudgeFamily as ut, Turn as v, ChatTransport as w, ChatClient as x, TurnResult as y, LlmToolChoice as z };
|
|
809
|
-
//# sourceMappingURL=types-
|
|
809
|
+
//# sourceMappingURL=types-eMhdacVp.d.ts.map
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"types-
|
|
1
|
+
{"version":3,"file":"types-eMhdacVp.d.ts","names":[],"sources":["../src/judge-families.ts","../src/integrity/served-model.ts","../src/trace/raw-provider-sink.ts","../src/llm-client/contract.ts","../src/analyst/chat-client.ts","../src/types.ts"],"mappings":";;;;;;;;;;;;;KAWY;;;;;;;iBAsEI,YAAY,kBAAkB;UAc7B;;EAEf;;;;EAIA;;cAGW,yBAAyB;WAGlB,UAAU;WACV;EAHlB,YACE,iBACgB,UAAU,eACV;;;;;;;;;;;;;;;iBAoBJ,kBACd,kBACA,OAAM,2BACL;;;;KCvFS;;;;;;;;;;;UAYK;;EAEf;;EAEA;EACA,iBAAiB;;EAEjB,cAAc;EACd,SAAS;;EAET;;;;;;;;;;;;iBA4Cc,iBACd,mBACA,oCACC;cA4CU,+BAA+B;WAGxB,QAAQ,cAAc;EAFxC,YACE,iBACgB,QAAQ,cAAc;;;;;;;;;KAc9B;UAWK;;;;;;EAMf;;;;;EAKA;;EAEA;;;;;;;iBAuCc,kBACd,mBACA,mCACA,OAAM,2BACL;;;;;iBAea,mBACd,OAAO;EAAgB;EAAmB;IAC1C,OAAM,2BACL;UAec,uCAAuC;;EAEtD;;EAEA;;;;;;;;;;;iBAuBc,wBACd,OAAO;EAAgB;EAAmB;IAC1C,OAAM,iCACL;;;;;;;;;;;;;;;;;;;;;;;;;;;;;KCjRS;UAEK;;EAEf;;EAEA;EACA;;;;;;EAMA;EACA;;EAEA;;EAEA;;EAEA;EACA,WAAW;;EAEX;;EAEA;EACA;EACA,iBAAiB;EACjB;EACA,kBAAkB;EAClB;;EAEA;;EAEA;;UAGe;EACf;EACA;EACA,YAAY;EACZ;;UAGe;EACf,OAAO,OAAO,mBAAmB;;EAEjC,MAAM,SAAS,wBAAwB,QAAQ;;EAE/C,UAAU;;KAGA,oBAAoB,OAAO,qBAAqB;UA2B3C;EACf,WAAW;;cAGA,mCAAmC;UACtC;UACA;EAER,YAAY,OAAM;EAIZ,OAAO,OAAO,mBAAmB;EAIjC,KAAK,SAAQ,wBAA6B,QAAQ;EAUxD;;cAKW,+BAA+B;EACpC,UAAU;;;;;;;EASV,QAAQ,QAAQ;;UAOP;;EAEf;;EAEA;;EAEA;EACA,WAAW;;cAGA,qCAAqC;UACxC;UACA;UACA;UACA;UACA;UACA;UACA;EAER,YAAY,MAAM;UAOJ;UAON;EAKF,OAAO,OAAO,mBAAmB;EAYjC,KAAK,SAAQ,wBAA6B,QAAQ;;;;UC1MzC;EACf;;;;;EAKA,kBAEI;IACM;IAAc;;IACd;IAAmB;MAAa;MAAa;;;;EAGvD,YAAY;;EAEZ;;KAGU;;UAGK;EACf;EACA;IACE;IACA;IACA,YAAY;;;;KAKJ;EAIN;EAAkB;IAAY;;;;UAGnB;EACf;EACA;;EAEA;;UAGe;EACf;EACA,UAAU;;EAEV;;EAEA;IAAe;IAAc,QAAQ;;;EAErC,QAAQ;;EAER,aAAa;EACb;EACA;;;;;EAKA;IAAa;;;EAEb,WAAW;;EAEX;;;;;;UAOe;;EAEf;;EAEA;;EAEA,WAAW;;EAEX,qBAAqB;;UAGN;EACf;EACA;EACA;;EAEA;;EAEA;;EAEA;;;;UAKe;EACf;EACA;EACA,KAAK;IAAgB;IAAe;;;UAGrB;;EAEf;;EAEA,YAAY;EACZ,OAAO;;;;;EAKP;;;;;;;;;EASA;;;;;;;;;;;;EAYA;;EAEA;;;;;;;;;EASA;;;;;;;EAOA;;;;;;;;EAQA,WAAW,cAAc;;EAEzB,KAAK;;KAGK,kBAAkB,KAAK;UAElB,yBAAyB;;EAExC;;EAEA;EACA;;EAEA;IAAe;IAAc;;;EAE7B;;EAEA;;;;;;;EAOA;;;;;;;EAOA,SAAS;;;;;;;;EAQT;;;;;;EAMA;;EAEA,eAAe;;;;;;;;EAQf,UAAU;;;;;EAKV;;EAEA;IAAiB;IAAgB;;;EAEjC,WAAW;;;;;;;;;;EAUX,8BAA8B;;;;;;;UCzMf;;WAEN,WAAW;;WAEX;;WAEA;;EAGT,KAAK,KAAK,aAAa,OAAO,eAAe,QAAQ;;KAG3C;UAMK,oBAAoB,KAAK;;EAExC;;KAGU,eAAe;UAEV;;EAEf,SAAS;;EAET;;EAEA;;EAEA;;KAKU,uBACR,0BACA,sBACA,gCACA;UAEM;EACR;;EAEA;;;;;;UAOe,gCAAgC;EAC/C;EACA,OAAO,KAAK,aAAa,OAAO,iBAAiB,QAAQ;;;UAI1C,4BAA4B;EAC3C;EACA,OAAO,KAAK,aAAa,OAAO,iBAAiB,QAAQ;;;;;;;;;;;;;;;;;;;;UAqB1C,sCACP,mBACN,KACE;EAcJ;;EAEA;;EAEA;;EAEA;;EAEA;IAAe;IAAc;;;;;;;UAOd,0BAA0B;EACzC;EACA,UAAU,KAAK,aAAa,OAAO,iBAAiB,QAAQ;;;;;;iBAO9C,iBAAiB,MAAM,uBAAuB;;;UChK7C;EACf;EACA;EACA;EACA;EACA;EACA,OAAO;EACP,gBAAgB;EAChB;;UAGe;EACf;EACA;EACA;EACA;;UAKe;EACf;EAQA;EACA;EACA;EACA;;UAKe;EACf;EACA;EACA,YAAY;;UAGG;EACf;EACA;EACA;EACA;EACA;;UAKe;EACf;EACA;EACA,OAAO;EACP,iBAAiB;EACjB,aAAa;EACb;EACA;EACA;EACA,WAAW;;EAEX,OAAO;;UAGQ;EACf;EACA;EACA;EACA;EACA;IAAmB;IAAc;;EACjC;EACA;;UAGe;EACf,OAAO;EACP;EACA;;UAGe;EACf;EACA;EACA;EACA;EACA;;UAGe;EACf;IAAc;IAAc;;EAC5B;IAAmB;IAAc,QAAQ;;EACzC;IAAc;IAAkB;;EAChC;;UAKe;EACf;EACA;EACA;EACA;EACA;EACA;EACA;EACA;EACA;EACA;GACC;;UAGc;EACf;EACA,QAAQ;;EAER;;UAKe;EACf;EACA;EACA;EACA;EACA;EACA;EACA;IACE;MACE;MACA;MACA;;;EAGJ,OAAO;EACP,gBAAgB;;UAKD;EACf;EACA,QAAQ,OAAO;EACf,YAAY,OAAO;;;;;;;;;;KAWhB;UAEY;EACf;EACA;EACA;EACA,oBAAoB;EACpB;;EAEA,QAAQ;;;;;;;EAOR;;;;;;;EAOA;;;;;;EAMA;;UAGe;EACf;EACA;EACA;IAAa;IAAiB;IAAkB;;EAChD;EACA;EACA;;UAGe;EACf,UAAU;EACV,OAAO;EACP,WAAW;;EAEX,aAAa;EACb;EACA,WAAW;EACX,SAAS;;KAGC,WAAW,MAAM,YAAY,OAAO,eAAe,QAAQ;UAMtD;EACf;EACA;EACA;EACA"}
|
package/dist/wire/index.d.ts
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
|
-
import {
|
|
2
|
-
import { x as ChatClient } from "../types-
|
|
1
|
+
import { m as CustomTokenPricing, o as CostLedgerHandle } from "../cost-ledger-C1W3ICmP.js";
|
|
2
|
+
import { x as ChatClient } from "../types-eMhdacVp.js";
|
|
3
3
|
import { s as TraceStore } from "../store-B3nWRy3B.js";
|
|
4
|
-
import { m as FeedbackTrajectoryStore } from "../feedback-trajectory-
|
|
4
|
+
import { m as FeedbackTrajectoryStore } from "../feedback-trajectory-DhB9WzYg.js";
|
|
5
5
|
import { z } from "zod";
|
|
6
6
|
import { ServerType } from "@hono/node-server";
|
|
7
7
|
import { Hono } from "hono";
|
package/dist/wire/index.js
CHANGED
|
@@ -1,2 +1,2 @@
|
|
|
1
|
-
import { a as dispatchRpc, c as WireError, d as FeedbackTrajectorySchema, f as TraceEventSchema, n as startServer, p as WIRE_VERSION, r as startServerAsync, t as createApp, u as ErrorResponseSchema } from "../server-
|
|
1
|
+
import { a as dispatchRpc, c as WireError, d as FeedbackTrajectorySchema, f as TraceEventSchema, n as startServer, p as WIRE_VERSION, r as startServerAsync, t as createApp, u as ErrorResponseSchema } from "../server-8TS8XCvg.js";
|
|
2
2
|
export { ErrorResponseSchema, FeedbackTrajectorySchema, TraceEventSchema, WIRE_VERSION, WireError, createApp, dispatchRpc, startServer, startServerAsync };
|
|
@@ -642,3 +642,19 @@ Running it again on the same run directory continues an interrupted search and r
|
|
|
642
642
|
- Bridge children inherit a small environment allowlist; pass unproxied provider credentials only through `runner.env`.
|
|
643
643
|
- The metered proxy path replaces provider credentials with a loopback URL and an ephemeral key.
|
|
644
644
|
- Resumed state must match every input that can change the result.
|
|
645
|
+
|
|
646
|
+
### Creative artifacts in GEPA reflection
|
|
647
|
+
|
|
648
|
+
`gepaOptimizationMethod` forwards each successful campaign dimension to `info.scores`
|
|
649
|
+
(higher is better), retaining `info.dimensions` for existing consumers. Failed cells
|
|
650
|
+
carry a failed status and no objective scores; their protocol penalty is not evidence.
|
|
651
|
+
Normalize lower-is-better measures such as element count in the owning judge.
|
|
652
|
+
Use the upstream hybrid frontier configuration to retain objective specialists.
|
|
653
|
+
|
|
654
|
+
`describeArtifact` may include an inline PNG at any depth:
|
|
655
|
+
`{ type: 'image', mimeType: 'image/png', data: pngBytes.toString('base64') }`.
|
|
656
|
+
The Python bridge converts exactly this shape to `gepa.Image` so the upstream
|
|
657
|
+
reflection prompt contains an image part. It never reads a caller path or fetches
|
|
658
|
+
an image URL. Include only train/selection artifacts, keep them in the run record,
|
|
659
|
+
and size `maxEvidenceChars` and callback limits for their base64 payloads.
|
|
660
|
+
The reflection executor must itself support image parts; delivery is not a taste score.
|
package/docs/concepts.md
CHANGED
|
@@ -450,3 +450,34 @@ The root also exports contingency-table inference, normal and Student-t tails, a
|
|
|
450
450
|
`twoByTwoPValue` chooses Fisher exact inference for sparse expected counts and the Pearson chi-squared tail otherwise.
|
|
451
451
|
A regression with insufficient residual degrees of freedom or constant x retains a null slope p-value.
|
|
452
452
|
These primitives do not choose product acceptance thresholds or authorize release.
|
|
453
|
+
|
|
454
|
+
### Matrix cost units
|
|
455
|
+
|
|
456
|
+
`measureCampaignCost` from `/campaign` reads Sandbox's live pricing endpoint and
|
|
457
|
+
Z.AI's published coding-plan credit calculation at execution time. Pass observed
|
|
458
|
+
Router `routerUsd` (including a measured zero), the actual sandbox resource size
|
|
459
|
+
and elapsed seconds, and seat model plus known raw usage. `inputTokens` includes
|
|
460
|
+
`cachedTokens`; cached usage must be explicit, even when zero. Attach its result
|
|
461
|
+
as `costBreakdown` on the existing `ctx.cost.runPaidCall` receipt. Keep
|
|
462
|
+
`actualCostUsd` equal to the observed Router charge, or `costUnknown: true` when
|
|
463
|
+
that charge is unavailable. The receipt validates that these observations agree.
|
|
464
|
+
|
|
465
|
+
The ledger summary and campaign cell retain Router USD, additional sandbox USD
|
|
466
|
+
and standard model credits separately, with live source URLs, fetch times,
|
|
467
|
+
hashes and the rates actually used. Existing `costUsd` stays the provider USD
|
|
468
|
+
subtotal and its ceiling retains its existing meaning; it is not an all-in total.
|
|
469
|
+
Sandbox seconds are supplied by the execution owner, not assumed from judge time.
|
|
470
|
+
Seat credits are estimates before off-peak discounts and MCP charges, not billed
|
|
471
|
+
credits or amortized seat USD. A changed/unreadable live rate source yields an
|
|
472
|
+
uncaptured component. Unknown cached usage cannot produce a credit estimate.
|
|
473
|
+
Mixed legacy receipts and pending calls keep the split unknown; legacy results
|
|
474
|
+
without a split remain unchanged. Do not add component values across units.
|
|
475
|
+
|
|
476
|
+
The retained-artifact integration command is
|
|
477
|
+
`node --import tsx scripts/creative-matrix-proof.ts manifest.json output-dir python`.
|
|
478
|
+
Its manifest names three or more real PNG paths, their SHA-256 hashes, source
|
|
479
|
+
measurements and higher-is-better objective maps. An optional `receipt` with
|
|
480
|
+
`receiptSource` replays already-settled raw seat usage into a persisted campaign
|
|
481
|
+
cell. No model or image requests are made. The command runs the official GEPA
|
|
482
|
+
baseline frontier and builds its actual multimodal reflection prompt; it checks
|
|
483
|
+
that every objective and image survives. It proves transport, not creative gain.
|
|
@@ -54,7 +54,7 @@ All are repaired, each with a regression test that fails on the pristine blob an
|
|
|
54
54
|
| `studentTCdf` / `incompleteBeta` at `df ≤ 100` | defective, **fixed** | keep-and-fix | The continued fraction omits the mandatory symmetry branch (`x < (a+1)/(a+b+2)`, else `1 − I(1−x, b, a)`), so it is evaluated outside its convergence domain for small `|t|` and collapses as `x → 1`. `studentTCdf(0.005, 100)` returns `0.89152130` against a true `0.50198972`, an error of `0.3895`. Through `pairedTTest` at `df = 7`: `t = 0.001` reports `p = 0.15130173` against a true `0.99923002`, and `t = 1e-6` reports `p = 0.00015251`. There is a discontinuity at zero — `t = 1e-8` returns exactly `p = 1.0` because `x ≥ 1` short-circuits. | A perfectly null paired result reports `p < 0.05`. The materially wrong band is `|t| ≲ 0.02` at `df = 2–10`, widening to `|t| ≲ 0.058` near `df = 100`; outside it `pairedTTest` matched `scipy.stats.ttest_rel` to `≤ 1e-6` relative across 166 cases. |
|
|
55
55
|
| `interRaterReliability` | defective, **fixed** | keep-and-fix | The grouping loop iterates judge-major and opens a new item bucket whenever the last holds `judgeScores.length` entries, so each bucket collects consecutive scores from the *same* judge. It measures within-judge spread, not between-judge agreement, and is anti-correlated with the truth. Two identical judges scoring `[0,100]` return **`−0.500000`** where the true α is `+1.0`; two maximally disagreeing judges (`[0,0]` versus `[100,100]`) return **`+1.000000`** where the true α is `−0.5`. Perfect agreement on three items returns `−0.250000`, on four items `+0.650000`. | The result is also unstable in the item count, because bucket boundaries depend on how the item count divides by the judge count. Zero test coverage repo-wide; consumed by `src/pipelines/judge-agreement.ts:73`. |
|
|
56
56
|
| `mannWhitneyU` (NaN input) | defective, **fixed** | keep-and-fix | The tie-grouping loop advances via `while (j < len && combined[j].v === combined[i].v) j++` then `i = j`. `NaN === NaN` is false, so `j` never leaves `i` and the process spins forever. `mannWhitneyU([1,2,3,4,5,6],[NaN,2,3,4,5,6])` did not return within 8 s and blocked the event loop hard enough that a 4 s `setTimeout` never fired. | A single NaN score hangs a campaign rather than failing it. `ranks` is safe only because it uses `i = j + 1`. |
|
|
57
|
-
| `wilcoxonSignedRank` (NaN input) | defective, **fixed** | keep-and-fix | The same construct at the absolute-rank grouping loop, same hang. | Reachable from `held-out-gate.ts
|
|
57
|
+
| `wilcoxonSignedRank` (NaN input) | defective, **fixed** | keep-and-fix | The same construct at the absolute-rank grouping loop, same hang. | Reachable from `HeldOutGate.decidePairedDelta` in `held-out-gate.ts`, so a NaN in a holdout set hangs the gate. |
|
|
58
58
|
| `mannWhitneyU` (tie and continuity correction) | defective, **fixed** | keep-and-fix | The variance uses the no-ties formula `n₁n₂(n₁+n₂+1)/12` and there is no continuity correction. `[1,2,3]` versus `[4,5,6]` (zero ties) and `[0,0,0]` versus `[1,1,1]` (maximal ties) both return the byte-identical `p = 0.04953448`. scipy's tie-and-continuity-corrected asymptotic separates them at `0.080856` and `0.046854`. | Both omissions push p downward, so they compound the CDF defect rather than offsetting it. `u` is returned as `min(u1,u2)`, which discards the direction of the effect. |
|
|
59
59
|
| `wilcoxonSignedRank` (tie and continuity correction) | defective, **fixed** | keep-and-fix | The variance uses `n(n+1)(2n+1)/24` with no tie term `−(1/48)Σ(t³−t)` and no continuity correction, despite the function computing average ranks for ties. | Same compounding direction. The statistic convention also differs from scipy without being documented: this returns `W⁺`, scipy returns `min(W⁺, W⁻)`. |
|
|
60
60
|
| `wilcoxonSignedRank` (`n < 6` branch) | defective, **fixed** | **branch deleted** | Line 219 hard-returns `{w: 0, p: 1}` below six non-zero differences, with no exception, no flag, and nothing in the result to distinguish it from a measured null. A clean `+0.5` on all five of five pairs returns `p = 1` where the exact answer is `0.0625`. Because zero deltas are dropped first, ties silently push `n` under the threshold: ten pairs with five zero deltas also returned `p = 1`. | This is the single most consequential open defect at 3–10 reps, and it violates the package's own *no fallbacks, fail loud* rule. It is a false-negative generator precisely in the target regime. |
|
package/docs/eval-surface-map.md
CHANGED
|
@@ -85,6 +85,7 @@ Use the decision diagnostics and intervals to distinguish insufficient evidence
|
|
|
85
85
|
| `/rl` | Build reward, preference, and supervised datasets from eligible evidence. |
|
|
86
86
|
| `/profile-cell` | Create and validate portable agent-profile identities. |
|
|
87
87
|
| `/authenticity`, `/ledger-core` | Check evidence authenticity and maintain canonical hash-chained journals. |
|
|
88
|
+
| `/lock-holder` | Decide whether a lock's recorded holder is `dead`, `alive` or `unknown` with `lockHolderVerdict`; record a holder with `readLockHolderIdentity`. It imports only Node built-ins, so any lock owner can use it without loading the rest of the package. |
|
|
88
89
|
| [`/rollout`](./rollout.md), `/storyboard` | Serialize training rows and render recorded work. |
|
|
89
90
|
| `/hosted`, `/wire`, `/adapters/http` | Connect hosted storage or expose evaluation through HTTP and RPC. |
|
|
90
91
|
|
|
@@ -26,8 +26,7 @@ contributes to the cell ID, so a source identity join should compare both kind
|
|
|
26
26
|
and hash. A loose product-specific fingerprint must not claim to be a validated
|
|
27
27
|
canonical AgentProfile merely because it is JSON-serializable.
|
|
28
28
|
|
|
29
|
-
Regression coverage: `tests/profile-
|
|
30
|
-
existing `tests/agent-profile-cell.test.ts`. Run the repository typecheck, full
|
|
29
|
+
Regression coverage: `tests/agent-profile-cell.test.ts`. Run the repository typecheck, full
|
|
31
30
|
suite, build and package checks before release. This document does not allocate
|
|
32
31
|
or announce a package version; release metadata and downstream peer ranges must
|
|
33
32
|
be updated together by the normal release process.
|