@tangle-network/agent-eval 0.118.3 → 0.119.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +27 -0
- package/README.md +26 -0
- package/dist/analyst/index.d.ts +172 -59
- package/dist/analyst/index.js +50 -12
- package/dist/analyst/index.js.map +1 -1
- package/dist/benchmarks/index.d.ts +50 -40
- package/dist/benchmarks/index.js +8 -8
- package/dist/campaign/index.d.ts +258 -184
- package/dist/campaign/index.js +7 -7
- package/dist/{chunk-JTMFT5QZ.js → chunk-4I2E3LLO.js} +2 -2
- package/dist/{chunk-KK2O4VWA.js → chunk-D7AEXSM5.js} +2 -2
- package/dist/{chunk-MLHRFWQK.js → chunk-JHOJHHU7.js} +85 -16
- package/dist/chunk-JHOJHHU7.js.map +1 -0
- package/dist/{chunk-F5A4GDQW.js → chunk-JM2SKQMS.js} +3 -15
- package/dist/chunk-JM2SKQMS.js.map +1 -0
- package/dist/{chunk-EGNRE7VA.js → chunk-PAHNGS65.js} +2 -2
- package/dist/{chunk-QWRLW4CT.js → chunk-PSXJ32SR.js} +160 -6
- package/dist/chunk-PSXJ32SR.js.map +1 -0
- package/dist/{chunk-6QIM2EAP.js → chunk-PXD6ZFNY.js} +99 -1
- package/dist/chunk-PXD6ZFNY.js.map +1 -0
- package/dist/{chunk-267UCQWI.js → chunk-QWMPPZ3X.js} +3 -3
- package/dist/{chunk-267UCQWI.js.map → chunk-QWMPPZ3X.js.map} +1 -1
- package/dist/{chunk-VNLWDTDA.js → chunk-S5TT5R3L.js} +1010 -285
- package/dist/chunk-S5TT5R3L.js.map +1 -0
- package/dist/{chunk-ZYHJNKI3.js → chunk-ULOKLHIQ.js} +42 -5
- package/dist/chunk-ULOKLHIQ.js.map +1 -0
- package/dist/{chunk-FIUFRXP3.js → chunk-WW2A73HW.js} +49 -91
- package/dist/chunk-WW2A73HW.js.map +1 -0
- package/dist/{chunk-7V3QDWNL.js → chunk-XJ7JVCHB.js} +2 -2
- package/dist/{chunk-EKHHBKS6.js → chunk-XMBOU5W7.js} +48 -190
- package/dist/chunk-XMBOU5W7.js.map +1 -0
- package/dist/cli.js +2 -2
- package/dist/contract/index.d.ts +281 -175
- package/dist/contract/index.js +284 -8
- package/dist/contract/index.js.map +1 -1
- package/dist/fuzz.d.ts +19 -1
- package/dist/fuzz.js +1 -1
- package/dist/index.d.ts +223 -114
- package/dist/index.js +38 -37
- package/dist/index.js.map +1 -1
- package/dist/openapi.json +1 -1
- package/dist/primeintellect/index.d.ts +311 -0
- package/dist/primeintellect/index.js +348 -0
- package/dist/primeintellect/index.js.map +1 -0
- package/dist/rl.d.ts +4 -0
- package/dist/{run-campaign-DWC67KJP.js → run-campaign-OBXSN5WK.js} +3 -3
- package/dist/traces.d.ts +39 -25
- package/dist/traces.js +3 -5
- package/dist/wire/index.d.ts +19 -1
- package/dist/wire/index.js +2 -2
- package/docs/trace-analysis.md +6 -2
- package/package.json +7 -2
- package/dist/chunk-6QIM2EAP.js.map +0 -1
- package/dist/chunk-EKHHBKS6.js.map +0 -1
- package/dist/chunk-F5A4GDQW.js.map +0 -1
- package/dist/chunk-FIUFRXP3.js.map +0 -1
- package/dist/chunk-MLHRFWQK.js.map +0 -1
- package/dist/chunk-QWRLW4CT.js.map +0 -1
- package/dist/chunk-VNLWDTDA.js.map +0 -1
- package/dist/chunk-ZYHJNKI3.js.map +0 -1
- /package/dist/{chunk-JTMFT5QZ.js.map → chunk-4I2E3LLO.js.map} +0 -0
- /package/dist/{chunk-KK2O4VWA.js.map → chunk-D7AEXSM5.js.map} +0 -0
- /package/dist/{chunk-EGNRE7VA.js.map → chunk-PAHNGS65.js.map} +0 -0
- /package/dist/{chunk-7V3QDWNL.js.map → chunk-XJ7JVCHB.js.map} +0 -0
- /package/dist/{run-campaign-DWC67KJP.js.map → run-campaign-OBXSN5WK.js.map} +0 -0
package/CHANGELOG.md
CHANGED
|
@@ -4,6 +4,33 @@ All notable changes to `@tangle-network/agent-eval` and its sibling `agent-eval-
|
|
|
4
4
|
|
|
5
5
|
---
|
|
6
6
|
|
|
7
|
+
## [0.119.1] — 2026-07-14 — portable improvement evidence
|
|
8
|
+
|
|
9
|
+
### Added
|
|
10
|
+
|
|
11
|
+
- `measuredComparisonFromSelfImproveResult()` converts paired held-out quality, cost, latency, uncertainty, power, decision, and provenance into the shared `AgentImprovementMeasuredComparison` contract.
|
|
12
|
+
|
|
13
|
+
## [0.119.0] — 2026-07-14 — chained, metered trace analysis
|
|
14
|
+
|
|
15
|
+
### Added
|
|
16
|
+
|
|
17
|
+
- Trace analysts can consume findings produced earlier in the same ordered run and emit multiple evidence citations without changing the original singular-citation callback API.
|
|
18
|
+
- Analyst summaries report provider calls, input, output, reasoning, cache-read, cache-write, dollar provenance, and known partial spend independently of finding count.
|
|
19
|
+
|
|
20
|
+
### Fixed
|
|
21
|
+
|
|
22
|
+
- All Ax analyst calls reserve spend before dispatch, disable hidden provider retries, honor cancellation, wait a bounded time for late receipts, and preserve known charges when token usage is unavailable.
|
|
23
|
+
- Trace-analysis proposers record each model call directly in the campaign cost ledger instead of replacing them with one estimated wrapper receipt.
|
|
24
|
+
- Direct Gemini 3 analysis keeps its output limit without sending Ax's incompatible thinking-level option.
|
|
25
|
+
- Recovery findings pass through the same subject, evidence, and post-processing rules as primary findings, and malformed recovery calls fail visibly.
|
|
26
|
+
- Budget allocation rejects invalid values and cannot exceed the remaining run budget or regain spend through malformed finding metadata.
|
|
27
|
+
|
|
28
|
+
### Breaking
|
|
29
|
+
|
|
30
|
+
- `SemanticConceptJudgeAdapterOpts.options` no longer accepts `costLedger` or `signal`; remove those fields because `AnalystRegistry` now supplies the run budget and cancellation signal and records the resulting usage.
|
|
31
|
+
- `createTraceAnalystKind()` now requires `model` when passed an externally constructed Ax service; supply the service's model explicitly or construct it with `createAnalystAi()` so the model can be recovered safely.
|
|
32
|
+
- `TraceAnalystGolden.expected` now uses `CanonicalRawAnalystFinding` with an `evidence` array; migrate singular `evidence_uri` and `evidence_excerpt` fields into the first array entry.
|
|
33
|
+
|
|
7
34
|
## [0.118.2] — 2026-07-13 — interoperable contracts and trace accounting
|
|
8
35
|
|
|
9
36
|
### Fixed
|
package/README.md
CHANGED
|
@@ -95,6 +95,31 @@ const traceRuns = fromOtelSpans({ spans: yourOtelSpans })
|
|
|
95
95
|
await analyzeRuns({ runs: [...runs, ...traceRuns], raterScores })
|
|
96
96
|
```
|
|
97
97
|
|
|
98
|
+
### 4. Export to PrimeIntellect
|
|
99
|
+
|
|
100
|
+
Use this when you want PrimeIntellect Verifiers to run or train against tasks produced by Tangle evals, runtime runs, or knowledge runs.
|
|
101
|
+
|
|
102
|
+
```ts
|
|
103
|
+
import {
|
|
104
|
+
primeIntellectRowsFromRunRecords,
|
|
105
|
+
writePrimeIntellectEnvironmentPackage,
|
|
106
|
+
} from '@tangle-network/agent-eval/primeintellect'
|
|
107
|
+
|
|
108
|
+
const rows = primeIntellectRowsFromRunRecords({
|
|
109
|
+
records, // RunRecord[] with scenarioId
|
|
110
|
+
scenarios, // prompts plus answer or requiredSubstrings
|
|
111
|
+
})
|
|
112
|
+
|
|
113
|
+
await writePrimeIntellectEnvironmentPackage('prime-envs/refund-policy', {
|
|
114
|
+
name: 'refund-policy',
|
|
115
|
+
rows,
|
|
116
|
+
runRecords: records,
|
|
117
|
+
})
|
|
118
|
+
```
|
|
119
|
+
|
|
120
|
+
The generated package contains `load_environment()`, `data/dataset.jsonl`, `data/run_records.jsonl`, and a manifest.
|
|
121
|
+
Run it with `uv run vf-eval refund-policy` or upload it with `prime env push`.
|
|
122
|
+
|
|
98
123
|
---
|
|
99
124
|
|
|
100
125
|
## Core concepts
|
|
@@ -124,6 +149,7 @@ Each example: `README.md` + a single `index.ts` runnable via `pnpm tsx`. Prints
|
|
|
124
149
|
|---|---|
|
|
125
150
|
| `…/contract` | **The headline, frozen surface — new code starts here.** `defineAgentEval`, `selfImprove`, `analyzeRuns`, `runEval`, `runCampaign`, `runImprovementLoop`, `diffRuns`; intake adapters (`fromFeedbackTable`, `fromOtelSpans`); proposers (`gepaProposer`, `evolutionaryProposer`); gates (`defaultProductionGate`, `heldOutGate`, `paretoSignificanceGate`, `neutralizationGate`, `composeGate`); the deployment-outcome store; storage; and the five core types `Scenario` / `Dispatch` / `JudgeConfig` / `SurfaceProposer` / `Gate`. |
|
|
126
151
|
| `…/hosted` | `createHostedClient` / `hostedClientFromEnv` + the wire types to ship eval-run events + trace spans to a hosted orchestrator (ours or your own implementation of the spec) |
|
|
152
|
+
| `…/primeintellect` | Export validated `RunRecord`s plus scenario prompts into a PrimeIntellect Verifiers package: Python `load_environment()`, `data/dataset.jsonl`, `data/run_records.jsonl`, and export manifest |
|
|
127
153
|
| `…/adapters/otel` | `createOtelBridge` — forwards OpenTelemetry-shape spans into the hosted-tier ingest, no `@opentelemetry/*` dependency |
|
|
128
154
|
| `…/adapters/langchain` | Wrap any LangChain `Runnable` as a `Dispatch` (or `JudgeConfig`), no `@langchain/core` peer dep |
|
|
129
155
|
| `…/adapters/http` | `httpDispatch` + `runDispatchServer` — run a campaign's worker on another machine (multi-region, remote worker execution) |
|
package/dist/analyst/index.d.ts
CHANGED
|
@@ -1,5 +1,5 @@
|
|
|
1
1
|
import { TCloud } from '@tangle-network/tcloud';
|
|
2
|
-
import { AxAIService, AxFunction } from '@ax-llm/ax';
|
|
2
|
+
import { AxAIArgs, AxAIService, AxFunction } from '@ax-llm/ax';
|
|
3
3
|
import { z } from 'zod';
|
|
4
4
|
|
|
5
5
|
/**
|
|
@@ -800,8 +800,14 @@ declare class RunCritic {
|
|
|
800
800
|
type CostChannel = 'agent' | 'judge' | 'verifier' | 'analyst' | 'driver' | (string & {});
|
|
801
801
|
interface CostUsage {
|
|
802
802
|
inputTokens: number;
|
|
803
|
+
/** Includes reasoning tokens when the provider bills them as output. */
|
|
803
804
|
outputTokens: number;
|
|
805
|
+
/** Reasoning-token subset of outputTokens, when reported. */
|
|
806
|
+
reasoningTokens?: number;
|
|
807
|
+
/** Prompt tokens served from a provider cache. */
|
|
804
808
|
cachedTokens?: number;
|
|
809
|
+
/** Prompt tokens written into a provider cache. */
|
|
810
|
+
cacheWriteTokens?: number;
|
|
805
811
|
}
|
|
806
812
|
interface CostCallBase {
|
|
807
813
|
callId: string;
|
|
@@ -868,7 +874,9 @@ interface ChannelRollup {
|
|
|
868
874
|
calls: number;
|
|
869
875
|
inputTokens: number;
|
|
870
876
|
outputTokens: number;
|
|
877
|
+
reasoningTokens?: number;
|
|
871
878
|
cachedTokens: number;
|
|
879
|
+
cacheWriteTokens?: number;
|
|
872
880
|
costUsd: number;
|
|
873
881
|
unpricedCalls: number;
|
|
874
882
|
unknownUsageCalls: number;
|
|
@@ -880,7 +888,9 @@ interface CostLedgerSummary {
|
|
|
880
888
|
reservedCostUsd: number;
|
|
881
889
|
inputTokens: number;
|
|
882
890
|
outputTokens: number;
|
|
891
|
+
reasoningTokens?: number;
|
|
883
892
|
cachedTokens: number;
|
|
893
|
+
cacheWriteTokens?: number;
|
|
884
894
|
totalCostUsd: number;
|
|
885
895
|
byChannel: ChannelRollup[];
|
|
886
896
|
unpricedModels: string[];
|
|
@@ -894,6 +904,10 @@ interface CostLedgerFilter {
|
|
|
894
904
|
phase?: string;
|
|
895
905
|
tags?: Record<string, string>;
|
|
896
906
|
}
|
|
907
|
+
interface CostLedgerWaitOptions {
|
|
908
|
+
/** Maximum time to wait for active provider calls. Default 5 seconds. */
|
|
909
|
+
timeoutMs?: number;
|
|
910
|
+
}
|
|
897
911
|
/** Append-only storage. `append` must atomically reject stale revisions. */
|
|
898
912
|
interface CostLedgerPersistence {
|
|
899
913
|
read(): {
|
|
@@ -913,6 +927,7 @@ declare class CostLedger {
|
|
|
913
927
|
private readonly records;
|
|
914
928
|
private readonly activeCallIds;
|
|
915
929
|
private readonly lateCallIds;
|
|
930
|
+
private readonly idleWaiters;
|
|
916
931
|
private completedTasks;
|
|
917
932
|
private revision;
|
|
918
933
|
private costLimitPersisted;
|
|
@@ -920,6 +935,8 @@ declare class CostLedger {
|
|
|
920
935
|
private readonly persistence?;
|
|
921
936
|
constructor(input?: number | CostLedgerOptions);
|
|
922
937
|
runPaidCall<T>(input: RunPaidCallInput<T>): Promise<PaidCallResult<T>>;
|
|
938
|
+
/** Wait until every call started by this ledger has produced a durable outcome. */
|
|
939
|
+
waitForIdle(options?: CostLedgerWaitOptions): Promise<boolean>;
|
|
923
940
|
/** Settle a call left pending by a crashed process after reconciling with the provider. */
|
|
924
941
|
reconcile(callId: string, observed: CostReceiptInput, options?: {
|
|
925
942
|
error?: string;
|
|
@@ -930,6 +947,7 @@ declare class CostLedger {
|
|
|
930
947
|
costPerCompletedTask(): number | null;
|
|
931
948
|
private execute;
|
|
932
949
|
private captureLateOutcome;
|
|
950
|
+
private releaseActiveCall;
|
|
933
951
|
private commitOutcome;
|
|
934
952
|
private captureFailure;
|
|
935
953
|
private commitReceipt;
|
|
@@ -945,7 +963,7 @@ declare class CostLedger {
|
|
|
945
963
|
* Keeping callback contracts structural lets those subpaths compose while the
|
|
946
964
|
* concrete {@link CostLedger} retains its private durable state.
|
|
947
965
|
*/
|
|
948
|
-
type CostLedgerHandle = Pick<CostLedger, keyof CostLedger
|
|
966
|
+
type CostLedgerHandle = Pick<CostLedger, Exclude<keyof CostLedger, 'waitForIdle'>> & Partial<Pick<CostLedger, 'waitForIdle'>>;
|
|
949
967
|
|
|
950
968
|
/**
|
|
951
969
|
* LLM client with graceful degrade.
|
|
@@ -1688,6 +1706,8 @@ interface AnalystCost {
|
|
|
1688
1706
|
est_usd_per_run?: number;
|
|
1689
1707
|
/** Models the analyst expects to use (informational). */
|
|
1690
1708
|
models?: string[];
|
|
1709
|
+
/** Maximum post-cancellation wait for provider usage. Model analysts default to 5 seconds. */
|
|
1710
|
+
settlement_timeout_ms?: number;
|
|
1691
1711
|
}
|
|
1692
1712
|
interface AnalystRequirements {
|
|
1693
1713
|
/** Min number of shots / samples the analyst needs to produce signal. */
|
|
@@ -1711,10 +1731,14 @@ interface AnalystContext {
|
|
|
1711
1731
|
runId: string;
|
|
1712
1732
|
/** Stable correlation id so logs from a single registry.run() share a tag. */
|
|
1713
1733
|
correlationId: string;
|
|
1714
|
-
/**
|
|
1734
|
+
/** Enforced wall-clock deadline (epoch ms). */
|
|
1715
1735
|
deadlineMs?: number;
|
|
1716
1736
|
/** Per-analyst USD budget. Analysts MAY check before issuing LLM calls. */
|
|
1717
1737
|
budgetUsd?: number;
|
|
1738
|
+
/** Shared paid-call account when the analyst runs inside a larger campaign. */
|
|
1739
|
+
costLedger?: CostLedgerHandle;
|
|
1740
|
+
/** Attribution phase used when writing to the shared paid-call account. */
|
|
1741
|
+
costPhase?: string;
|
|
1718
1742
|
/**
|
|
1719
1743
|
* Shared chat client. Analysts that call an LLM go through this so
|
|
1720
1744
|
* the operator picks transport (sandbox-sdk | router | cli-bridge |
|
|
@@ -1732,6 +1756,18 @@ interface AnalystContext {
|
|
|
1732
1756
|
* filter. Empty / absent means no cross-run context.
|
|
1733
1757
|
*/
|
|
1734
1758
|
priorFindings?: ReadonlyArray<AnalystFinding>;
|
|
1759
|
+
/**
|
|
1760
|
+
* Findings emitted by analysts that completed earlier in this registry run.
|
|
1761
|
+
* This is separate from `priorFindings`: upstream findings are dependency
|
|
1762
|
+
* context for the current pass, while prior findings are cross-run memory.
|
|
1763
|
+
* The registry populates this only when `RegistryRunOpts.chainFindings` is on.
|
|
1764
|
+
*/
|
|
1765
|
+
upstreamFindings?: ReadonlyArray<AnalystFinding>;
|
|
1766
|
+
/**
|
|
1767
|
+
* Report metered work independently of findings. This keeps an empty finding
|
|
1768
|
+
* set from erasing token/cost telemetry. Multiple receipts are accumulated.
|
|
1769
|
+
*/
|
|
1770
|
+
recordUsage?: (receipt: AnalystUsageReceipt) => void;
|
|
1735
1771
|
/** Free-form runtime tags (env, host, op). Findings can echo these into metadata. */
|
|
1736
1772
|
tags?: Record<string, string>;
|
|
1737
1773
|
/** Logger callback — analysts SHOULD prefer this over console.* for testability. */
|
|
@@ -1757,6 +1793,17 @@ interface Analyst<TInput = unknown> {
|
|
|
1757
1793
|
readonly version: string;
|
|
1758
1794
|
analyze(input: TInput, ctx: AnalystContext): Promise<AnalystFinding[]>;
|
|
1759
1795
|
}
|
|
1796
|
+
/** Metered work performed by one analyst call. */
|
|
1797
|
+
interface AnalystUsageReceipt {
|
|
1798
|
+
/** Number of model-usage records observed at the provider boundary. */
|
|
1799
|
+
calls: number | null;
|
|
1800
|
+
/** Null when the provider did not return token accounting. */
|
|
1801
|
+
tokens: RunTokenUsage | null;
|
|
1802
|
+
/** Observed, estimated, or explicitly uncaptured dollar cost. */
|
|
1803
|
+
cost: RunCostProvenance;
|
|
1804
|
+
/** Known lower bound when one or more calls have uncaptured cost. */
|
|
1805
|
+
knownCostUsd?: number;
|
|
1806
|
+
}
|
|
1760
1807
|
/**
|
|
1761
1808
|
* Compute the stable finding_id from the identity-defining fields.
|
|
1762
1809
|
* Default implementation hashes {analyst_id, area, subject, normalized claim}.
|
|
@@ -1789,6 +1836,12 @@ interface AnalystRunSummary {
|
|
|
1789
1836
|
findings_count: number;
|
|
1790
1837
|
latency_ms: number;
|
|
1791
1838
|
cost_usd: number;
|
|
1839
|
+
/**
|
|
1840
|
+
* Additive receipt for model usage. Registry-produced summaries populate it
|
|
1841
|
+
* even when the analyst emits no findings. `cost_usd` remains the legacy
|
|
1842
|
+
* numeric field; inspect `usage.cost` before treating zero as observed.
|
|
1843
|
+
*/
|
|
1844
|
+
usage?: AnalystUsageReceipt;
|
|
1792
1845
|
/** When `status='failed'`: the error class + message, never the full stack. */
|
|
1793
1846
|
error?: {
|
|
1794
1847
|
class: string;
|
|
@@ -1804,6 +1857,11 @@ interface AnalystRunResult {
|
|
|
1804
1857
|
per_analyst: AnalystRunSummary[];
|
|
1805
1858
|
/** Total LLM cost in USD across all analysts in this registry.run(). */
|
|
1806
1859
|
total_cost_usd: number;
|
|
1860
|
+
/**
|
|
1861
|
+
* Provenance for `total_cost_usd`. When uncaptured, the numeric field is only
|
|
1862
|
+
* the known subtotal and must not be treated as the run's total spend.
|
|
1863
|
+
*/
|
|
1864
|
+
total_cost_provenance?: RunCostProvenance;
|
|
1807
1865
|
}
|
|
1808
1866
|
/**
|
|
1809
1867
|
* Events emitted by `AnalystRegistry.runStream(...)` in real time as
|
|
@@ -1896,7 +1954,10 @@ declare function createJudgeAdapter(opts: JudgeAdapterOpts): Analyst<JudgeInput>
|
|
|
1896
1954
|
interface SemanticConceptJudgeAdapterOpts {
|
|
1897
1955
|
id?: string;
|
|
1898
1956
|
area?: string;
|
|
1899
|
-
|
|
1957
|
+
/** Registry context owns cancellation and the per-analyst cost ledger. */
|
|
1958
|
+
options?: Omit<SemanticConceptJudgeOptions, 'costLedger' | 'signal'>;
|
|
1959
|
+
/** Maximum post-cancellation wait for a provider receipt. Default 5 seconds. */
|
|
1960
|
+
settlementTimeoutMs?: number;
|
|
1900
1961
|
}
|
|
1901
1962
|
declare function createSemanticConceptJudgeAdapter(opts?: SemanticConceptJudgeAdapterOpts): Analyst<SemanticConceptJudgeInput>;
|
|
1902
1963
|
|
|
@@ -1906,11 +1967,13 @@ interface CreateAnalystAiConfig {
|
|
|
1906
1967
|
apiKey: string;
|
|
1907
1968
|
/** OpenAI-compatible base URL — e.g. `https://router.tangle.tools/v1` or a
|
|
1908
1969
|
* cli-bridge loopback. */
|
|
1909
|
-
baseUrl
|
|
1910
|
-
/**
|
|
1970
|
+
baseUrl?: string;
|
|
1971
|
+
/** Additional headers required by the gateway, such as tenant or execution policy. */
|
|
1972
|
+
headers?: Record<string, string>;
|
|
1973
|
+
/** Model id forwarded to analyst calls. */
|
|
1911
1974
|
model: string;
|
|
1912
1975
|
/** Ax provider name. Defaults to the OpenAI-compatible client. */
|
|
1913
|
-
provider?: '
|
|
1976
|
+
provider?: AxAIArgs<unknown>['name'];
|
|
1914
1977
|
}
|
|
1915
1978
|
/**
|
|
1916
1979
|
* Construct the `AxAIService` an analyst kind calls through
|
|
@@ -2016,7 +2079,15 @@ declare function behavioralAnalyst(): Analyst<TraceAnalysisStore>;
|
|
|
2016
2079
|
*/
|
|
2017
2080
|
|
|
2018
2081
|
declare const ANALYST_SEVERITIES: readonly ["critical", "high", "medium", "low", "info"];
|
|
2082
|
+
declare const RawAnalystEvidenceSchema: z.ZodObject<{
|
|
2083
|
+
uri: z.ZodString;
|
|
2084
|
+
excerpt: z.ZodOptional<z.ZodString>;
|
|
2085
|
+
}, z.core.$strict>;
|
|
2086
|
+
type RawAnalystEvidence = z.infer<typeof RawAnalystEvidenceSchema>;
|
|
2087
|
+
/** Original public schema retained for stored rows and callback contracts. */
|
|
2019
2088
|
declare const RawAnalystFindingSchema: z.ZodObject<{
|
|
2089
|
+
evidence_uri: z.ZodString;
|
|
2090
|
+
evidence_excerpt: z.ZodOptional<z.ZodString>;
|
|
2020
2091
|
severity: z.ZodEnum<{
|
|
2021
2092
|
critical: "critical";
|
|
2022
2093
|
info: "info";
|
|
@@ -2026,35 +2097,63 @@ declare const RawAnalystFindingSchema: z.ZodObject<{
|
|
|
2026
2097
|
}>;
|
|
2027
2098
|
claim: z.ZodString;
|
|
2028
2099
|
subject: z.ZodOptional<z.ZodString>;
|
|
2029
|
-
evidence_uri: z.ZodString;
|
|
2030
|
-
evidence_excerpt: z.ZodOptional<z.ZodString>;
|
|
2031
2100
|
confidence: z.ZodNumber;
|
|
2032
2101
|
rationale: z.ZodOptional<z.ZodString>;
|
|
2033
2102
|
recommended_action: z.ZodOptional<z.ZodString>;
|
|
2034
2103
|
}, z.core.$strict>;
|
|
2035
2104
|
type RawAnalystFinding = z.infer<typeof RawAnalystFindingSchema>;
|
|
2105
|
+
/**
|
|
2106
|
+
* Canonical plural-evidence contract. The preprocessor accepts the original
|
|
2107
|
+
* `evidence_uri` / `evidence_excerpt` pair and normalizes it into one evidence
|
|
2108
|
+
* item so persisted rows and older model fixtures remain readable. New output
|
|
2109
|
+
* always receives the plural shape.
|
|
2110
|
+
*/
|
|
2111
|
+
declare const CanonicalRawAnalystFindingSchema: z.ZodPipe<z.ZodTransform<unknown, unknown>, z.ZodObject<{
|
|
2112
|
+
evidence: z.ZodArray<z.ZodObject<{
|
|
2113
|
+
uri: z.ZodString;
|
|
2114
|
+
excerpt: z.ZodOptional<z.ZodString>;
|
|
2115
|
+
}, z.core.$strict>>;
|
|
2116
|
+
severity: z.ZodEnum<{
|
|
2117
|
+
critical: "critical";
|
|
2118
|
+
info: "info";
|
|
2119
|
+
low: "low";
|
|
2120
|
+
high: "high";
|
|
2121
|
+
medium: "medium";
|
|
2122
|
+
}>;
|
|
2123
|
+
claim: z.ZodString;
|
|
2124
|
+
subject: z.ZodOptional<z.ZodString>;
|
|
2125
|
+
confidence: z.ZodNumber;
|
|
2126
|
+
rationale: z.ZodOptional<z.ZodString>;
|
|
2127
|
+
recommended_action: z.ZodOptional<z.ZodString>;
|
|
2128
|
+
}, z.core.$strict>>;
|
|
2129
|
+
type CanonicalRawAnalystFinding = z.infer<typeof CanonicalRawAnalystFindingSchema>;
|
|
2036
2130
|
/**
|
|
2037
2131
|
* Description embedded into the actor prompt so the LLM knows what
|
|
2038
2132
|
* shape to emit. Kept here so kinds share one source of truth rather
|
|
2039
2133
|
* than restating the schema in every prompt.
|
|
2040
2134
|
*/
|
|
2041
|
-
declare const RAW_FINDING_SCHEMA_PROMPT = "Each finding MUST be a JSON object with
|
|
2135
|
+
declare const RAW_FINDING_SCHEMA_PROMPT = "Each finding MUST be a strict JSON object with:\n - severity: \"critical\" | \"high\" | \"medium\" | \"low\" | \"info\"\n - claim: one-sentence statement (max 2000 chars)\n - subject?: one exact subject form listed by this kind; omit rather than guess\n - evidence: REQUIRED non-empty array of {\"uri\": string, \"excerpt\"?: string}. Use real identifiers with span://, event://, artifact://, metric://, or finding://. Include a short exact quote in excerpt when available. If nothing is citable, do not emit the finding.\n - confidence: number 0..1 (0.9+ exact evidence; 0.6-0.8 inferred pattern; <0.5 speculative)\n - rationale?: one or two reasoning sentences\n - recommended_action?: concrete imperative change; omit for descriptive findings\n\nUnknown fields are rejected. Do not emit area; the factory assigns it. Emit [] when there are no findings. Never fabricate evidence.";
|
|
2136
|
+
/** Convert canonical raw citations into the public finding evidence envelope. */
|
|
2137
|
+
declare function evidenceRefsFromRawFinding(finding: CanonicalRawAnalystFinding): EvidenceRef[];
|
|
2042
2138
|
/**
|
|
2043
|
-
* Validate
|
|
2044
|
-
*
|
|
2045
|
-
*
|
|
2139
|
+
* Validate the original singular-evidence shape. This public parser retains
|
|
2140
|
+
* its pre-canonicalization result type so existing callback code and stored
|
|
2141
|
+
* rows continue to receive exactly the object accepted by
|
|
2142
|
+
* {@link RawAnalystFindingSchema}.
|
|
2046
2143
|
*/
|
|
2047
2144
|
declare function parseRawFinding(row: unknown, log?: (msg: string, fields?: Record<string, unknown>) => void): RawAnalystFinding | null;
|
|
2145
|
+
/** Validate model output and normalize original singular citations. */
|
|
2146
|
+
declare function parseCanonicalRawFinding(row: unknown, log?: (msg: string, fields?: Record<string, unknown>) => void): CanonicalRawAnalystFinding | null;
|
|
2048
2147
|
|
|
2049
2148
|
/**
|
|
2050
2149
|
* Analyst-kind factory — the typed way to define trace analysts.
|
|
2051
2150
|
*
|
|
2052
2151
|
* A "kind" is a specialized analyst whose actor prompt, tool subset,
|
|
2053
|
-
* and Ax
|
|
2054
|
-
* classification, knowledge gap discovery, knowledge poisoning,
|
|
2055
|
-
* self-improvement, ...). Kinds emit findings in the typed
|
|
2056
|
-
* shape via a JSON-array Ax output; the factory
|
|
2057
|
-
* Zod and lifts it into `AnalystFinding[]
|
|
2152
|
+
* and bounded Ax subqueries target one failure-mode lens (failure-mode
|
|
2153
|
+
* classification, knowledge gap discovery, knowledge poisoning,
|
|
2154
|
+
* self-improvement, ...). Kinds emit findings in the typed
|
|
2155
|
+
* `CanonicalRawAnalystFinding` shape via a JSON-array Ax output; the factory
|
|
2156
|
+
* validates each row with Zod and lifts it into `AnalystFinding[]`.
|
|
2058
2157
|
*
|
|
2059
2158
|
* Composition rules:
|
|
2060
2159
|
* - Each kind owns its actor description. No generic "answer this
|
|
@@ -2062,12 +2161,11 @@ declare function parseRawFinding(row: unknown, log?: (msg: string, fields?: Reco
|
|
|
2062
2161
|
* - Each kind picks a narrow tool subset from `ANALYST_TOOL_GROUPS`.
|
|
2063
2162
|
* A kind that never needs full-trace dumps can drop `viewTrace` /
|
|
2064
2163
|
* `viewSpans` and stay cheap.
|
|
2065
|
-
* - Each kind declares its
|
|
2066
|
-
*
|
|
2067
|
-
* (poisoning) usually stay at 0 since they have a tighter brief.
|
|
2164
|
+
* - Each kind declares its subquery + parallelism budget. Discovery-heavy
|
|
2165
|
+
* kinds can fan out more bounded semantic questions than narrow lenses.
|
|
2068
2166
|
*
|
|
2069
2167
|
* Optimizer hook: kinds may declare `goldens` — labeled examples used
|
|
2070
|
-
* by `
|
|
2168
|
+
* by `AxBootstrapFewShot` / `AxGEPA` to fit the actor
|
|
2071
2169
|
* description programmatically. Stored on the kind, not the registry,
|
|
2072
2170
|
* because the right metric is kind-specific.
|
|
2073
2171
|
*/
|
|
@@ -2087,23 +2185,25 @@ interface TraceAnalystKindSpec {
|
|
|
2087
2185
|
version: string;
|
|
2088
2186
|
/** Actor system prompt. Must instruct the LLM to emit `findings` per the schema. */
|
|
2089
2187
|
actorDescription: string;
|
|
2090
|
-
/** Responder system prompt; falls back to a minimal "format the findings" instruction. */
|
|
2091
|
-
responderDescription?: string;
|
|
2092
2188
|
/** Tool functions the actor may call. Pick narrow subsets via `ANALYST_TOOL_GROUPS`. */
|
|
2093
2189
|
buildTools: (store: TraceAnalysisStore) => AxFunction[];
|
|
2094
|
-
/**
|
|
2095
|
-
|
|
2096
|
-
|
|
2097
|
-
|
|
2190
|
+
/** Bounded semantic subqueries. `maxCalls: 0` disables model fan-out. */
|
|
2191
|
+
subqueries?: {
|
|
2192
|
+
maxCalls: number;
|
|
2193
|
+
maxParallel?: number;
|
|
2098
2194
|
};
|
|
2099
2195
|
/** Actor turn cap. Default 12. */
|
|
2100
2196
|
maxTurns?: number;
|
|
2101
2197
|
/** Runtime char cap. Default 6000. */
|
|
2102
2198
|
maxRuntimeChars?: number;
|
|
2199
|
+
/** Maximum output tokens for every actor and subquery model call. Default 4096. */
|
|
2200
|
+
maxOutputTokens?: number;
|
|
2103
2201
|
/** Cost classification surfaced in `registry.list()` and budget enforcement. */
|
|
2104
2202
|
cost: AnalystCost;
|
|
2105
2203
|
/** Per-finding-row hook — kinds may reject / rewrite before lifting. */
|
|
2106
2204
|
postProcess?: (row: RawAnalystFinding, ctx: AnalystContext) => RawAnalystFinding | null;
|
|
2205
|
+
/** Minimum citations per finding. Default 1; rows below it are rejected. */
|
|
2206
|
+
minimumEvidenceCitations?: number;
|
|
2107
2207
|
/** Optional optimizer hook — populated when a kind wants to fit its prompt against labeled examples. */
|
|
2108
2208
|
goldens?: TraceAnalystGolden[];
|
|
2109
2209
|
}
|
|
@@ -2115,12 +2215,12 @@ interface TraceAnalystKindSpec {
|
|
|
2115
2215
|
*/
|
|
2116
2216
|
interface TraceAnalystGolden {
|
|
2117
2217
|
question: string;
|
|
2118
|
-
expected: ReadonlyArray<Omit<
|
|
2218
|
+
expected: ReadonlyArray<Omit<CanonicalRawAnalystFinding, 'confidence'>>;
|
|
2119
2219
|
}
|
|
2120
2220
|
interface CreateTraceAnalystKindOpts {
|
|
2121
2221
|
/** AxAIService bound at registration time. */
|
|
2122
2222
|
ai: AxAIService;
|
|
2123
|
-
/**
|
|
2223
|
+
/** Required unless `ai` was created by {@link createAnalystAi}. */
|
|
2124
2224
|
model?: string;
|
|
2125
2225
|
/** Override the spec's `version` (e.g. when an optimizer has fitted a new prompt). */
|
|
2126
2226
|
versionSuffix?: string;
|
|
@@ -2137,6 +2237,8 @@ interface CreateTraceAnalystKindOpts {
|
|
|
2137
2237
|
model?: string;
|
|
2138
2238
|
fetchImpl?: typeof fetch;
|
|
2139
2239
|
};
|
|
2240
|
+
/** Maximum post-cancellation wait for a provider receipt. Default 5 seconds. */
|
|
2241
|
+
settlementTimeoutMs?: number;
|
|
2140
2242
|
}
|
|
2141
2243
|
/**
|
|
2142
2244
|
* Build an `Analyst<TraceAnalysisStore>` from a kind spec.
|
|
@@ -2162,6 +2264,8 @@ declare function createTraceAnalystKind(spec: TraceAnalystKindSpec, opts: Create
|
|
|
2162
2264
|
* prompts (e.g. specialized analysts living outside the default kinds).
|
|
2163
2265
|
*/
|
|
2164
2266
|
declare function renderPriorFindings(prior: AnalystContext['priorFindings']): string;
|
|
2267
|
+
/** Render findings produced earlier in this same registry run. */
|
|
2268
|
+
declare function renderUpstreamFindings(upstream: AnalystContext['upstreamFindings']): string;
|
|
2165
2269
|
|
|
2166
2270
|
/**
|
|
2167
2271
|
* AnalystRegistry — orchestrate N analysts against one run.
|
|
@@ -2218,7 +2322,8 @@ interface BudgetPolicy {
|
|
|
2218
2322
|
/**
|
|
2219
2323
|
* Custom allocator — receives the analyst, remaining/total budget, and
|
|
2220
2324
|
* the count of analysts that will run. Returns the per-analyst budget
|
|
2221
|
-
* (or undefined
|
|
2325
|
+
* (or undefined only when the run has no overall cap). Overrides weights
|
|
2326
|
+
* when set.
|
|
2222
2327
|
*/
|
|
2223
2328
|
allocate?: (args: {
|
|
2224
2329
|
analyst: Analyst;
|
|
@@ -2244,10 +2349,14 @@ interface RegistryRunOpts {
|
|
|
2244
2349
|
skip?: string[];
|
|
2245
2350
|
/** Budget policy — totalUsd + optional weights/allocator. Falls back to options.defaultBudget. */
|
|
2246
2351
|
budget?: BudgetPolicy;
|
|
2247
|
-
/**
|
|
2352
|
+
/** Active-work cap for the complete registry run. Model receipt settlement may follow. */
|
|
2248
2353
|
timeoutMs?: number;
|
|
2249
2354
|
/** Abort signal — forwarded into every analyst's context. */
|
|
2250
2355
|
signal?: AbortSignal;
|
|
2356
|
+
/** Shared paid-call account forwarded to every analyst. */
|
|
2357
|
+
costLedger?: CostLedgerHandle;
|
|
2358
|
+
/** Attribution phase for calls written to `costLedger`. */
|
|
2359
|
+
costPhase?: string;
|
|
2251
2360
|
/** Tags echoed into AnalystContext.tags — useful for tracking environment/version in findings. */
|
|
2252
2361
|
tags?: Record<string, string>;
|
|
2253
2362
|
/**
|
|
@@ -2255,10 +2364,16 @@ interface RegistryRunOpts {
|
|
|
2255
2364
|
* analyst via `ctx.priorFindings`. The registry forwards the slice
|
|
2256
2365
|
* whose `analyst_id` matches each registered analyst so a kind sees
|
|
2257
2366
|
* only its own history. Pass `{ '*': findings }` to broadcast to
|
|
2258
|
-
* every analyst (useful
|
|
2259
|
-
*
|
|
2367
|
+
* every analyst (useful when several kinds share the same historical
|
|
2368
|
+
* context). For findings from this run, use `chainFindings` instead.
|
|
2260
2369
|
*/
|
|
2261
2370
|
priorFindings?: ReadonlyArray<AnalystFinding> | Record<string, ReadonlyArray<AnalystFinding>>;
|
|
2371
|
+
/**
|
|
2372
|
+
* Pass findings produced earlier in this registry run to each later analyst
|
|
2373
|
+
* via `ctx.upstreamFindings`. Registration order is dependency order.
|
|
2374
|
+
* Disabled by default because independent analyst suites must opt in.
|
|
2375
|
+
*/
|
|
2376
|
+
chainFindings?: boolean;
|
|
2262
2377
|
}
|
|
2263
2378
|
declare class AnalystRegistry {
|
|
2264
2379
|
private readonly analysts;
|
|
@@ -2302,7 +2417,7 @@ declare class AnalystRegistry {
|
|
|
2302
2417
|
interface DefaultAnalystRegistryOptions {
|
|
2303
2418
|
/** Ax service for the agentic RLM kinds. Omit → only the deterministic analyst. */
|
|
2304
2419
|
ai?: AxAIService;
|
|
2305
|
-
/**
|
|
2420
|
+
/** Required unless `ai` was created by `createAnalystAi`. */
|
|
2306
2421
|
model?: string;
|
|
2307
2422
|
/** Which agentic kinds to register when `ai` is present. Default = the shipped suite. */
|
|
2308
2423
|
kinds?: readonly TraceAnalystKindSpec[];
|
|
@@ -2570,19 +2685,14 @@ declare function diffFindings(previous: PersistedFinding[], current: PersistedFi
|
|
|
2570
2685
|
* findings. The actor's job is *taxonomy + evidence*, not fix-design —
|
|
2571
2686
|
* that's the improvement-analyst's job.
|
|
2572
2687
|
*
|
|
2573
|
-
*
|
|
2574
|
-
*
|
|
2575
|
-
* into candidate clusters, each cluster spawns a focused investigator
|
|
2576
|
-
* that drills into representative traces, and a deeply-recursed
|
|
2577
|
-
* investigator may itself split a confounded mode into two sub-modes.
|
|
2578
|
-
* Each level fans out 4-way, so the analyst can investigate up to
|
|
2579
|
-
* ~16 leaf clusters before hitting the depth ceiling.
|
|
2688
|
+
* Eight bounded model subqueries let the actor compare candidate
|
|
2689
|
+
* clusters in parallel after it has loaded representative evidence.
|
|
2580
2690
|
*/
|
|
2581
2691
|
|
|
2582
2692
|
declare const FAILURE_MODE_KIND_SPEC: TraceAnalystKindSpec;
|
|
2583
2693
|
|
|
2584
2694
|
/**
|
|
2585
|
-
* Improvement analyst — actionable
|
|
2695
|
+
* Improvement analyst — actionable self-improvement findings.
|
|
2586
2696
|
*
|
|
2587
2697
|
* Brief: read findings from upstream analysts (failure-mode,
|
|
2588
2698
|
* knowledge-gap, knowledge-poisoning) AND the trace dataset itself,
|
|
@@ -2592,14 +2702,11 @@ declare const FAILURE_MODE_KIND_SPEC: TraceAnalystKindSpec;
|
|
|
2592
2702
|
* finding is one proposed edit with the locus, the diff, and the
|
|
2593
2703
|
* expected effect.
|
|
2594
2704
|
*
|
|
2595
|
-
* This is the
|
|
2705
|
+
* This is the self-improvement loop's last mile: the prior
|
|
2596
2706
|
* kinds describe *what's wrong*; this kind describes *what to change*.
|
|
2597
2707
|
*
|
|
2598
|
-
*
|
|
2599
|
-
*
|
|
2600
|
-
* fix directions (tighten prompt vs add tool vs adjust scaffolding),
|
|
2601
|
-
* and the actor should explore each with a focused subagent before
|
|
2602
|
-
* picking the highest-leverage one to recommend.
|
|
2708
|
+
* Eight bounded model subqueries let the actor compare competing fix
|
|
2709
|
+
* directions over the same cited evidence before recommending one.
|
|
2603
2710
|
*/
|
|
2604
2711
|
|
|
2605
2712
|
declare const IMPROVEMENT_KIND_SPEC: TraceAnalystKindSpec;
|
|
@@ -2626,8 +2733,8 @@ declare const IMPROVEMENT_KIND_SPEC: TraceAnalystKindSpec;
|
|
|
2626
2733
|
* knowledge-gap names the *information* whose absence (or staleness)
|
|
2627
2734
|
* caused the break. One failure-mode often maps to several gaps.
|
|
2628
2735
|
*
|
|
2629
|
-
*
|
|
2630
|
-
*
|
|
2736
|
+
* Five bounded model subqueries let the actor compare candidate gaps
|
|
2737
|
+
* across source layers after it has loaded the relevant excerpts.
|
|
2631
2738
|
*/
|
|
2632
2739
|
|
|
2633
2740
|
declare const KNOWLEDGE_GAP_KIND_SPEC: TraceAnalystKindSpec;
|
|
@@ -2646,10 +2753,8 @@ declare const KNOWLEDGE_GAP_KIND_SPEC: TraceAnalystKindSpec;
|
|
|
2646
2753
|
* surface as questions / self-correction; poisonings surface as
|
|
2647
2754
|
* confident-but-wrong actions that downstream evidence contradicts.
|
|
2648
2755
|
*
|
|
2649
|
-
*
|
|
2650
|
-
*
|
|
2651
|
-
* the agent acted on the false belief, one to confirm the belief
|
|
2652
|
-
* itself is actually false in ground truth.
|
|
2756
|
+
* Eight bounded model subqueries let the actor independently assess
|
|
2757
|
+
* the action and contradiction excerpts for candidate poisonings.
|
|
2653
2758
|
*/
|
|
2654
2759
|
|
|
2655
2760
|
declare const KNOWLEDGE_POISONING_KIND_SPEC: TraceAnalystKindSpec;
|
|
@@ -2661,8 +2766,8 @@ declare const KNOWLEDGE_POISONING_KIND_SPEC: TraceAnalystKindSpec;
|
|
|
2661
2766
|
* The four kinds chain: failure-mode classifies; knowledge-gap and
|
|
2662
2767
|
* knowledge-poisoning explain *why* in two orthogonal ways; improvement
|
|
2663
2768
|
* proposes concrete edits. Register all four against the same trace
|
|
2664
|
-
* store
|
|
2665
|
-
*
|
|
2769
|
+
* store in this order and run the registry with `chainFindings: true`
|
|
2770
|
+
* to pass each completed kind's findings to the kinds that follow it.
|
|
2666
2771
|
*/
|
|
2667
2772
|
|
|
2668
2773
|
/**
|
|
@@ -2778,7 +2883,7 @@ declare function coerceJson(text: string): unknown;
|
|
|
2778
2883
|
* Coerce arbitrary actor/structurer output into an array of candidate finding
|
|
2779
2884
|
* rows: a JSON string → parse; a single object → 1-element array; an array →
|
|
2780
2885
|
* as-is; anything else → []. Callers still run each row through Zod
|
|
2781
|
-
* (`
|
|
2886
|
+
* (`parseCanonicalRawFinding`) — this only fixes the SHAPE, never invents fields.
|
|
2782
2887
|
*/
|
|
2783
2888
|
declare function coerceToFindingRows(raw: unknown): unknown[];
|
|
2784
2889
|
|
|
@@ -2950,9 +3055,17 @@ interface StructureFindingsOptions {
|
|
|
2950
3055
|
/** Optional ledger for direct use. */
|
|
2951
3056
|
costLedger?: CostLedgerHandle;
|
|
2952
3057
|
costPhase?: string;
|
|
3058
|
+
costTags?: Record<string, string>;
|
|
2953
3059
|
maxTokens?: number;
|
|
3060
|
+
signal?: AbortSignal;
|
|
2954
3061
|
/** Max reask attempts after a zero/invalid extraction. Default 1. */
|
|
2955
3062
|
maxReasks?: number;
|
|
3063
|
+
/** Apply the caller's normal finding rules before a recovered row is lifted. */
|
|
3064
|
+
processRow?: (row: RawAnalystFinding) => RawAnalystFinding | null;
|
|
3065
|
+
/** Apply canonical multi-citation rules after any original callback. */
|
|
3066
|
+
processCanonicalRow?: (row: CanonicalRawAnalystFinding) => CanonicalRawAnalystFinding | null;
|
|
3067
|
+
/** Provenance copied onto every recovered finding. */
|
|
3068
|
+
findingMetadata?: Record<string, unknown>;
|
|
2956
3069
|
/** Test seam: inject a fetch (no network in unit tests). */
|
|
2957
3070
|
fetchImpl?: LlmClientOptions['fetch'];
|
|
2958
3071
|
}
|
|
@@ -2995,4 +3108,4 @@ type TraceToolGroupName =
|
|
|
2995
3108
|
*/
|
|
2996
3109
|
declare function buildTraceToolsForGroup(group: TraceToolGroupName, store: TraceAnalysisStore): AxFunction[];
|
|
2997
3110
|
|
|
2998
|
-
export { ANALYST_SEVERITIES, type Analyst, type AnalystContext, type AnalystCost, type AnalystFinding, type AnalystHooks, type AnalystInputKind, AnalystRegistry, type AnalystRegistryOptions, type AnalystRequirements, type AnalystRunEvent, type AnalystRunInputs, type AnalystRunResult, type AnalystRunSummary, type AnalystSeverity, type BudgetPolicy, type ChatCallOpts, type ChatClient, type ChatRequest, type ChatResponse, type ChatTransport, type CliBridgeTransportOpts, type CreateAnalystAiConfig, type CreateChatClientOpts, type CreateTraceAnalystKindOpts, DEFAULT_TRACE_ANALYST_KINDS, type DefaultAnalystRegistryOptions, type DiffPolicy, type DirectProviderTransportOpts, type EvidenceRef, FAILURE_MODE_KIND_SPEC, FINDING_SUBJECT_GRAMMAR_PROMPT, FINDING_SUBJECT_KINDS, FINDING_SUBJECT_SYNTAX, type FindingSubject, type FindingSubjectKind, FindingSubjectStringSchema, type FindingToPolicyEditOptions, type FindingsDiff, FindingsStore, IMPROVEMENT_KIND_SPEC, type JudgeAdapterOpts, KIND_EXPECTED_SUBJECTS, KNOWLEDGE_GAP_KIND_SPEC, KNOWLEDGE_POISONING_KIND_SPEC, type MockTransportOpts, POLICY_EDIT_AXES, POLICY_EDIT_CANDIDATE_RECORD_SCHEMA, POLICY_EDIT_TARGET_SURFACES, type PersistedFinding, type PolicyEdit, type PolicyEditAdmission, type PolicyEditAdmissionOptions, type PolicyEditAxis, type PolicyEditCandidateRecord, type PolicyEditChange, type PolicyEditExpectedGain, type PolicyEditGainDirection, type PolicyEditGainUnit, type PolicyEditInit, type PolicyEditRisk, type PolicyEditSchemaVersion, type PolicyEditSource, type PolicyEditTarget, type PolicyEditTargetSurface, PolicyEditValidationError, RAW_FINDING_SCHEMA_PROMPT, type RawAnalystFinding, RawAnalystFindingSchema, type RegistryRunOpts, type RouterTransportOpts, type RunCriticAdapterOpts, SKILL_USAGE_ANALYST, type SandboxSdkTransportOpts, type SemanticConceptJudgeAdapterOpts, SkillUsageAnalyst, type SkillUsageRecord, type SkillUsageReport, type SkillUsageScanConfig, type StructureFindingsOptions, type StructureFindingsResult, type TraceAnalystGolden, type TraceAnalystKindSpec, type TraceToolGroupName, type VerifierAdapterOpts, admitPolicyEdit, applyPolicyEditToSurface, assertNoJudgeVerdict, behavioralAnalyst, buildDefaultAnalystRegistry, buildSkillUsageReport, buildTraceToolsForGroup, coerceJson, coerceToFindingRows, computeFindingId, computePolicyEditId, createAnalystAi, createChatClient, createJudgeAdapter, createRunCriticAdapter, createSemanticConceptJudgeAdapter, createTraceAnalystKind, createVerifierAdapter, defaultIsMaterial, deriveEfficiencyFindings, diffFindings, emitSkillUsageFindings, findingSubjectGrammarPromptFor, isJudgeVerdict, isPolicyEdit, isTraceObservable, liftSeverity, makeFinding, makePolicyEdit, makePolicyEditCandidateRecord, parseFindingSubject, parseRawFinding, policyEditFromFinding, policyEditsFromFindings, renderFindingSubject, renderPriorFindings, scorePolicyEditReadiness, stripCodeFences, structureFindings, validatePolicyEdit, validatePolicyEditCandidateRecord };
|
|
3111
|
+
export { ANALYST_SEVERITIES, type Analyst, type AnalystContext, type AnalystCost, type AnalystFinding, type AnalystHooks, type AnalystInputKind, AnalystRegistry, type AnalystRegistryOptions, type AnalystRequirements, type AnalystRunEvent, type AnalystRunInputs, type AnalystRunResult, type AnalystRunSummary, type AnalystSeverity, type AnalystUsageReceipt, type BudgetPolicy, type CanonicalRawAnalystFinding, CanonicalRawAnalystFindingSchema, type ChatCallOpts, type ChatClient, type ChatRequest, type ChatResponse, type ChatTransport, type CliBridgeTransportOpts, type CreateAnalystAiConfig, type CreateChatClientOpts, type CreateTraceAnalystKindOpts, DEFAULT_TRACE_ANALYST_KINDS, type DefaultAnalystRegistryOptions, type DiffPolicy, type DirectProviderTransportOpts, type EvidenceRef, FAILURE_MODE_KIND_SPEC, FINDING_SUBJECT_GRAMMAR_PROMPT, FINDING_SUBJECT_KINDS, FINDING_SUBJECT_SYNTAX, type FindingSubject, type FindingSubjectKind, FindingSubjectStringSchema, type FindingToPolicyEditOptions, type FindingsDiff, FindingsStore, IMPROVEMENT_KIND_SPEC, type JudgeAdapterOpts, KIND_EXPECTED_SUBJECTS, KNOWLEDGE_GAP_KIND_SPEC, KNOWLEDGE_POISONING_KIND_SPEC, type MockTransportOpts, POLICY_EDIT_AXES, POLICY_EDIT_CANDIDATE_RECORD_SCHEMA, POLICY_EDIT_TARGET_SURFACES, type PersistedFinding, type PolicyEdit, type PolicyEditAdmission, type PolicyEditAdmissionOptions, type PolicyEditAxis, type PolicyEditCandidateRecord, type PolicyEditChange, type PolicyEditExpectedGain, type PolicyEditGainDirection, type PolicyEditGainUnit, type PolicyEditInit, type PolicyEditRisk, type PolicyEditSchemaVersion, type PolicyEditSource, type PolicyEditTarget, type PolicyEditTargetSurface, PolicyEditValidationError, RAW_FINDING_SCHEMA_PROMPT, type RawAnalystEvidence, RawAnalystEvidenceSchema, type RawAnalystFinding, RawAnalystFindingSchema, type RegistryRunOpts, type RouterTransportOpts, type RunCriticAdapterOpts, SKILL_USAGE_ANALYST, type SandboxSdkTransportOpts, type SemanticConceptJudgeAdapterOpts, SkillUsageAnalyst, type SkillUsageRecord, type SkillUsageReport, type SkillUsageScanConfig, type StructureFindingsOptions, type StructureFindingsResult, type TraceAnalystGolden, type TraceAnalystKindSpec, type TraceToolGroupName, type VerifierAdapterOpts, admitPolicyEdit, applyPolicyEditToSurface, assertNoJudgeVerdict, behavioralAnalyst, buildDefaultAnalystRegistry, buildSkillUsageReport, buildTraceToolsForGroup, coerceJson, coerceToFindingRows, computeFindingId, computePolicyEditId, createAnalystAi, createChatClient, createJudgeAdapter, createRunCriticAdapter, createSemanticConceptJudgeAdapter, createTraceAnalystKind, createVerifierAdapter, defaultIsMaterial, deriveEfficiencyFindings, diffFindings, emitSkillUsageFindings, evidenceRefsFromRawFinding, findingSubjectGrammarPromptFor, isJudgeVerdict, isPolicyEdit, isTraceObservable, liftSeverity, makeFinding, makePolicyEdit, makePolicyEditCandidateRecord, parseCanonicalRawFinding, parseFindingSubject, parseRawFinding, policyEditFromFinding, policyEditsFromFindings, renderFindingSubject, renderPriorFindings, renderUpstreamFindings, scorePolicyEditReadiness, stripCodeFences, structureFindings, validatePolicyEdit, validatePolicyEditCandidateRecord };
|