@tangle-network/agent-eval 0.20.2 → 0.20.4

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -1,343 +1,162 @@
1
1
  # @tangle-network/agent-eval
2
2
 
3
- **A library for deciding whether an LLM-driven generator did its job.**
4
-
5
- You hand it the thing the generator produced — a code scaffold, a patch, a tweet, a JSON config — and you get back a structured verdict: pass/fail, dimension scores, plain-English rationale. Built to catch the LLM failure modes that LLM-as-judge alone misses.
6
-
7
- ```ts
8
- import { BuilderSession, SubprocessSandboxDriver, InMemoryTraceStore } from '@tangle-network/agent-eval'
9
-
10
- const session = new BuilderSession(new InMemoryTraceStore(), { projectId: 'my-app' }, new SubprocessSandboxDriver())
11
- await session.startChat()
12
- const ship = await session.ship({
13
- harness: { setupCommand: 'pnpm install', testCommand: 'pnpm exec tsc --noEmit', cwd: scaffoldDir, timeoutMs: 180_000 },
14
- })
15
- console.log(ship.result.passed, ship.result.score)
3
+ Trace-first evaluation infrastructure for agent systems.
4
+
5
+ `agent-eval` provides the contracts and runtime primitives for measuring agent
6
+ behavior: traces, harnesses, verifier pipelines, judges, datasets, holdout
7
+ gates, failure classification, optimization loops, and release reports.
8
+
9
+ It does not own your product state, credentials, UI, or model routing. Product
10
+ teams keep those boundaries; this package standardizes how runs are recorded,
11
+ checked, compared, and promoted.
12
+
13
+ ## Contents
14
+
15
+ - [When To Use It](#when-to-use-it)
16
+ - [Architecture](#architecture)
17
+ - [Install](#install)
18
+ - [Core Primitives](#core-primitives)
19
+ - [Examples](#examples)
20
+ - [Documentation](#documentation)
21
+ - [Development](#development)
22
+ - [Related Packages](#related-packages)
23
+
24
+ ## When To Use It
25
+
26
+ Use `agent-eval` when you need one or more of these:
27
+
28
+ - A reproducible eval harness for coding agents, builder agents, or multi-tool
29
+ workflows.
30
+ - Structured traces for agent runs: spans, artifacts, events, budgets, tool
31
+ calls, retrieval, judge output, and sandbox execution.
32
+ - Deterministic gates around build/test/deploy checks.
33
+ - LLM-as-judge or deterministic judge fleets with calibration and canaries.
34
+ - Dataset splits, holdouts, paired statistics, and release confidence gates.
35
+ - Failure taxonomy that distinguishes prompt, tool, sandbox, retrieval,
36
+ evaluator, and knowledge-readiness failures.
37
+ - Optimization loops over prompts, steering, code mutations, or full multi-shot
38
+ trajectories.
39
+ - Report data for internal launch reviews, CI gates, and research analysis.
40
+
41
+ ## Architecture
42
+
43
+ ```txt
44
+ agent/product run
45
+ -> TraceEmitter / TraceStore
46
+ -> SandboxHarness / MultiLayerVerifier / JudgeRunner
47
+ -> failure taxonomy + metrics
48
+ -> paired stats + held-out gates
49
+ -> optimization + release confidence + reports
16
50
  ```
17
51
 
18
- ## Who this is for
19
-
20
- - You ship a code generator (scaffolder, patcher, refactor agent) and need to gate on whether its output actually works.
21
- - You ship a content generator and need quality signal beyond "the LLM said it's good".
22
- - You want a release gate that fails on regressions you can name, not vibes.
52
+ Package responsibilities:
23
53
 
24
- If that's you, start with [`docs/concepts.md`](./docs/concepts.md) — 5-minute mental model then use [`docs/feature-guide.md`](./docs/feature-guide.md) to choose the right primitive.
54
+ - `agent-eval`: run evidence, eval contracts, verification, statistics,
55
+ optimization, reporting.
56
+ - Product app: domain state, tools, credentials, UI, storage, deployment, model
57
+ gateway.
58
+ - `agent-runtime`: production agent-loop/session runtime.
59
+ - `agent-knowledge`: evidence stores, claim/page synthesis, retrieval, knowledge
60
+ readiness implementation.
25
61
 
26
- ## Quickstart
62
+ ## Install
27
63
 
28
- ### From any language: HTTP or RPC
64
+ ```sh
65
+ pnpm add @tangle-network/agent-eval
66
+ ```
29
67
 
30
- The fastest path. agent-eval ships a CLI that runs as either an HTTP server or a stdio RPC binary. Drive it from Python, Rust, Go, anything.
68
+ Wire protocol / CLI:
31
69
 
32
70
  ```sh
33
71
  npm i -g @tangle-network/agent-eval
34
-
35
- # HTTP — long-running
36
72
  agent-eval serve --port 5005
37
-
38
- # stdio RPC — one-shot, batch
39
- echo '{"rubricName":"anti-slop","content":"…"}' | agent-eval rpc judge
40
73
  ```
41
74
 
42
- Python:
75
+ Python client:
76
+
43
77
  ```sh
44
78
  pip install tangle-agent-eval
45
79
  ```
46
- ```python
47
- from tangle_agent_eval import Client
48
- c = Client()
49
- r = c.judge(content="our scaffold ships zero-copy IO", rubric_name="anti-slop")
50
- print(r.composite, r.failure_modes)
51
- ```
52
-
53
- See [`docs/wire-protocol.md`](./docs/wire-protocol.md) for the full surface.
54
80
 
55
- ### From TypeScript: import directly
56
-
57
- In-process; no wire round-trip. Use this when your eval lives in the same Node process as your generator.
81
+ ## Core Primitives
82
+
83
+ | Primitive | Purpose |
84
+ |---|---|
85
+ | `TraceEmitter`, `TraceStore` | Append-only run/span/event/artifact/budget records. |
86
+ | `SandboxHarness` | Build/test/runtime checks with captured stdout, stderr, exit codes, wall time, and parsed test counts. |
87
+ | `MultiLayerVerifier` | Ordered verification stages with dependencies, skip-on-fail, findings, scores, and time caps. |
88
+ | `JudgeRunner` | Parallel deterministic or LLM-backed judges over the same artifact/run. |
89
+ | `runAgentControlLoop` | Observe/validate/decide/act loop with budgets, stop policies, and structured eval results. |
90
+ | `Dataset`, `RunRecord`, `HeldOutGate` | Versioned corpora, reproducible run metadata, and held-out promotion decisions. |
91
+ | `pairedBootstrap`, `pairedWilcoxon`, `bhAdjust` | Paired experiment statistics and multiple-comparison correction. |
92
+ | `classifyFailure` | Rule-based failure classification for agent, tool, sandbox, retrieval, and knowledge failures. |
93
+ | `runMultiShotOptimization` | Optimization over full agent trajectories with actionable side information. |
94
+ | `runPromptEvolution` | Prompt/steering/code evolution over scenario scores. |
95
+ | `evaluateReleaseConfidence` | Release scorecard across evidence volume, pass rate, score, overfit, cost, latency, and gates. |
96
+ | `summaryTable`, `paretoChart`, `gainHistogram` | Report-ready structured outputs. |
97
+ | `KnowledgeRequirement`, `KnowledgeBundle` | Shared contracts for knowledge readiness. |
98
+
99
+ ## Examples
100
+
101
+ Runnable examples live in [`examples/`](./examples):
102
+
103
+ - [`examples/same-sandbox-harness`](./examples/same-sandbox-harness) - run
104
+ multiple eval passes against the same workspace.
105
+ - [`examples/multi-shot-optimization`](./examples/multi-shot-optimization) -
106
+ optimize full agent trajectories with held-out promotion.
107
+ - [`examples/benchmarks`](./examples/benchmarks) - benchmark adapter shape and
108
+ reference benchmark wrappers.
109
+
110
+ The examples are intentionally kept outside the README so they can be expanded,
111
+ tested, and copied without turning this page into a tutorial.
112
+
113
+ ## Documentation
114
+
115
+ - [Concepts](./docs/concepts.md)
116
+ - [Feature Guide](./docs/feature-guide.md)
117
+ - [Control Runtime](./docs/control-runtime.md)
118
+ - [Knowledge Readiness](./docs/knowledge-readiness.md)
119
+ - [Multi-Shot Optimization](./docs/multi-shot-optimization.md)
120
+ - [Feedback Trajectories](./docs/feedback-trajectories.md)
121
+ - [Wire Protocol](./docs/wire-protocol.md)
122
+
123
+ ## Development
58
124
 
59
125
  ```sh
60
- pnpm add @tangle-network/agent-eval
61
- ```
62
-
63
- The recipe for a code-generator eval is in [`SKILL.md` §Minimal working path](./.claude/skills/agent-eval/SKILL.md#minimal-working-path-builder-of-builders).
64
-
65
- ## Two ways to read this repo
66
-
67
- - **You're a human onboarding** — read [`docs/concepts.md`](./docs/concepts.md) for the mental model, then [`docs/wire-protocol.md`](./docs/wire-protocol.md) if you'll call from another language, or `SKILL.md` if you'll embed in TS.
68
- - **You're deciding what to integrate** — read [`docs/feature-guide.md`](./docs/feature-guide.md) for the layman explanation, use cases, feature map, and guardrails.
69
- - **You're an LLM agent writing integration code** — read `SKILL.md`. Every directive there encodes a shipped bug; skipping one reintroduces the bug class.
70
-
71
- ## What's in the box
72
-
73
- | Module | What it does | Doc |
74
- |---|---|---|
75
- | `BuilderSession` | Three-layer eval orchestrator (builder → app-build → app-runtime) for code generators. | concepts.md §three-layer eval |
76
- | `MultiLayerVerifier` | Pipeline of layers (install → typecheck → build → semantic). Skip-on-fail, weighted aggregate. | concepts.md §verifiers |
77
- | `judges`, `createCustomJudge`, `createAntiSlopJudge` | LLM and deterministic judges. | SKILL.md |
78
- | Wire protocol (`agent-eval serve` / `rpc`) | HTTP and stdio RPC interface for cross-language clients. | wire-protocol.md |
79
- | `clients/python/` | First-party Python client (`tangle-agent-eval` on PyPI). Version-locked to npm. | clients/python/README.md |
80
- | `BenchmarkRunner`, `executeScenario`, `ConvergenceTracker` | Multi-turn scenario execution + cross-run tracking. | SKILL.md |
81
- | `runAgentControlLoop` | Policy-based runtime for agentic tasks: observe typed state, validate, decide, act, repeat with budgets, tracing, and stuck-loop guards. | [control-runtime.md](./docs/control-runtime.md) |
82
- | `FeedbackTrajectory`, `InMemoryFeedbackTrajectoryStore`, `FileSystemFeedbackTrajectoryStore` | Human/environment feedback loops: capture approvals, rejections, choices, revisions, metrics, and policy blocks as train/dev/test/holdout examples. | [feedback-trajectories.md](./docs/feedback-trajectories.md) |
83
- | `evaluateActionPolicy` | Generic action preflight for approval, budget, expected-outcome, and kill-criteria checks. | [feature-guide.md](./docs/feature-guide.md) |
84
- | `ExperimentTracker`, steering optimizers, `bisector` | A/B prompts, optimize steering, bisect regressions. | SKILL.md |
85
- | `runMultiShotOptimization`, `trialTraceFromMultiShotTrial` | GEPA-style optimization for variable-length agent trajectories with ASI, paired seeds, and optional held-out promotion gating. | [multi-shot-optimization.md](./docs/multi-shot-optimization.md) |
86
- | `evaluateReleaseConfidence`, `assertReleaseConfidence` | Release scorecard that composes corpus coverage, search/holdout run evidence, ASI diagnostics, overfit checks, and cost/latency budgets. | §Release confidence |
87
- | `runPromptEvolution`, `createCompositeMutator`, `createSandboxPool`, `createSandboxCodeMutator`, `MutationTelemetry`, `LineageRecorder`, `CostLedger`, `JsonlTrialCache` | Prompt + code evolution loops with bounded sandbox pools, durable JSONL telemetry, plateau-detecting composite mutators, crash-resumable trial cache. | §Evolution loop |
88
- | `reflective-mutation` (`buildReflectionPrompt`, `parseReflectionResponse`, `DEFAULT_MUTATION_PRIMITIVES`) | Trace-conditioned LLM mutator that reasons over top/bottom trials instead of blind rewrites. | inline JSDoc |
89
- | `correlationStudy`, `OutcomeStore`, `ProductRegistry` | Meta-eval: do our scores predict deployment outcomes (revenue, retention)? | inline JSDoc |
90
- | Telemetry (`telemetry/`, `telemetry/file`) | OTLP export, trace replay, file sinks. | inline JSDoc |
91
-
92
- ## Release confidence
93
-
94
- Use `evaluateReleaseConfidence` at the release boundary for every consuming
95
- agent surface. It fails closed unless the release has a versioned corpus,
96
- search and holdout run evidence, score/pass-rate evidence, ASI for failures,
97
- and budget/overfit checks. Single-shot and multi-shot apps use the same path:
98
- single-shot traces are just trace evidence with `turnCount: 1`.
99
-
100
- ```ts
101
- import {
102
- evaluateReleaseConfidence,
103
- releaseTraceEvidenceFromMultiShotTrials,
104
- } from '@tangle-network/agent-eval'
105
-
106
- const scorecard = evaluateReleaseConfidence({
107
- target: 'blueprint-agent/autoresearch',
108
- candidateId: 'candidate-v3',
109
- baselineId: 'baseline',
110
- dataset: await dataset.manifest(),
111
- runs: [...candidateRuns, ...baselineRuns],
112
- traces: releaseTraceEvidenceFromMultiShotTrials(result.evolution.generations.flatMap((g) => g.trials)),
113
- gateDecision: result.gate?.decision,
114
- thresholds: {
115
- minScenarioCount: 50,
116
- minSearchRuns: 50,
117
- minHoldoutRuns: 20,
118
- minPassRate: 0.9,
119
- minMeanScore: 0.8,
120
- maxOverfitGap: 0.1,
121
- maxMeanCostUsd: 0.05,
122
- maxP95WallMs: 120_000,
123
- },
124
- })
125
-
126
- if (!scorecard.promote) throw new Error(scorecard.summary)
126
+ pnpm install
127
+ pnpm typecheck
128
+ pnpm test
129
+ pnpm build
130
+ pnpm openapi
127
131
  ```
128
132
 
129
- ## Evolution loop
130
-
131
- For agent tasks that run across many chat turns or tool calls, start with
132
- [`runMultiShotOptimization`](./docs/multi-shot-optimization.md). It runs the
133
- same prompt-evolution core over full trajectories, carries actionable side
134
- information into reflection, and separates the search winner from the variant
135
- that actually passes held-out promotion.
136
-
137
- Closing the loop on a prompt or codebase is **two adapters + a config**. Compose `runPromptEvolution` with `createCompositeMutator` (plateau policy) and you get prompt-only optimization until improvement stalls, then automatic switch to code-channel mutations from a coding agent inside a `SandboxPool`.
138
-
139
- ```ts
140
- import {
141
- createSandboxPool,
142
- createSandboxCodeMutator,
143
- createCompositeMutator,
144
- buildReflectionPrompt,
145
- parseReflectionResponse,
146
- runPromptEvolution,
147
- MutationTelemetry,
148
- LineageRecorder,
149
- CostLedger,
150
- JsonlTrialCache,
151
- } from '@tangle-network/agent-eval'
152
-
153
- // 1. Prompt mutator — reflective-mutation reasons over top/bottom trials
154
- const promptMutator = {
155
- async mutate({ parent, topTrials, bottomTrials, childCount }) {
156
- const ctx = { target: 'forge-prompt', parentPayload: parent.payload, topTrials, bottomTrials, childCount }
157
- const reflection = buildReflectionPrompt(ctx)
158
- const raw = await yourLlm(reflection)
159
- return parseReflectionResponse(raw, childCount).map((p, i) => ({
160
- id: `${parent.id}.g${parent.generation + 1}.prompt.${i}`,
161
- payload: p.payload,
162
- generation: parent.generation + 1,
163
- parentId: parent.id,
164
- label: p.label,
165
- rationale: p.rationale,
166
- }))
167
- },
168
- }
169
-
170
- // 2. Code mutator — runs a coding agent in a sandbox slot, captures the diff
171
- const pool = createSandboxPool({
172
- size: 4,
173
- factory: {
174
- async create(id) { return await yourSandboxClient.create({ name: id }) },
175
- async reset(slot) { await slot.resource.exec('git reset --hard origin/main && git clean -fd') },
176
- async destroy(slot) { await slot.resource.delete() },
177
- },
178
- })
179
- const codeMutator = createSandboxCodeMutator({
180
- pool,
181
- runner: async ({ slot, parent, topTrials, bottomTrials }) => {
182
- const result = await slot.resource.task(`Improve the prompt at /repo/forge-prompt.ts...`)
183
- return [{ ok: true, latencyMs: result.durationMs, costUsd: result.costUsd, artifact: { diff: result.diff } }]
184
- },
185
- toVariantPayload: (outcome, parent) => ({ ...parent.payload, codeMutation: outcome.artifact }),
186
- })
187
-
188
- // 3. Compose — plateau policy auto-switches when prompt evolution stalls
189
- const composite = createCompositeMutator({
190
- primary: promptMutator,
191
- secondary: codeMutator,
192
- policy: 'plateau',
193
- plateauThreshold: 0.02,
194
- plateauPatience: 2,
195
- })
196
-
197
- // 4. Run — durable telemetry to disk, crash-resumable
198
- const result = await runPromptEvolution({
199
- runId: `forge_${Date.now()}`,
200
- target: 'forge-prompt',
201
- seedVariants: [{ id: 'v0', payload: { text: currentPrompt }, generation: 0, label: 'baseline' }],
202
- scenarioIds: referenceCorpus.map(s => s.id),
203
- reps: 3,
204
- generations: 5,
205
- populationSize: 4,
206
- scoreAdapter: { /* runs your eval against (variant, scenario, rep) */ },
207
- mutateAdapter: composite,
208
- cache: new JsonlTrialCache('.evolve/cache.jsonl'),
209
- objectives: [
210
- { name: 'score', direction: 'maximize', value: a => a.meanScore },
211
- { name: 'cost', direction: 'minimize', value: a => a.meanCost },
212
- ],
213
- })
214
- ```
133
+ Run the local server:
215
134
 
216
- The `MutationTelemetry`, `LineageRecorder`, and `CostLedger` pass into the `code-mutator` (and any consumer that wants them) — they emit append-only JSONL of every attempt (success + failure with reason) and a snapshot lineage tree, so a finished run leaves a forensically complete trail under one directory.
217
-
218
- For the full primitive surface and rationale, read each module's JSDoc — `prompt-evolution.ts`, `composite-mutator.ts`, `sandbox-pool.ts`, `code-mutator.ts`, `reflective-mutation.ts`, `evolution-telemetry.ts`.
219
-
220
- ## Feedback trajectory loop
221
-
222
- When normal agent usage should generate training/eval signal, use feedback
223
- trajectories. They turn approvals, rejections, option choices, edits, metrics,
224
- and policy blocks into reusable examples.
225
-
226
- ```ts
227
- import {
228
- createFeedbackTrajectory,
229
- summarizePreferenceMemory,
230
- feedbackTrajectoriesToDatasetScenarios,
231
- feedbackTrajectoriesToOptimizerRows,
232
- } from '@tangle-network/agent-eval'
233
-
234
- const trajectory = createFeedbackTrajectory({
235
- projectId: 'research-agent',
236
- scenarioId: 'brief-review',
237
- task: { intent: 'Revise a research brief until it is specific and sourced.' },
238
- attempts: [{
239
- id: 'draft-1',
240
- stepIndex: 0,
241
- artifactType: 'research',
242
- artifact: { summary: 'Initial brief with weak sourcing.' },
243
- createdAt: new Date().toISOString(),
244
- }],
245
- labels: [{
246
- source: 'user',
247
- kind: 'revision_request',
248
- value: 'needs stronger evidence',
249
- reason: 'add primary sources and remove unsupported claims',
250
- severity: 'error',
251
- createdAt: new Date().toISOString(),
252
- }],
253
- })
254
-
255
- const memory = summarizePreferenceMemory([trajectory])
256
- const scenarios = feedbackTrajectoriesToDatasetScenarios([trajectory])
257
- const optimizerRows = feedbackTrajectoriesToOptimizerRows([trajectory])
135
+ ```sh
136
+ pnpm build
137
+ node dist/cli.js serve --port 5005
258
138
  ```
259
139
 
260
- This is the bridge between feedback and optimization: review signals become
261
- immediate memory, replayable eval scenarios, and prompt/signature/code optimizer
262
- input. See [`docs/feedback-trajectories.md`](./docs/feedback-trajectories.md).
263
-
264
- ## v0.16 highlights — production-rigor primitives
265
-
266
- These are the primitives any team running prompt-optimization in production needs, regardless of whether they're writing a paper. v0.15 shipped them under "paper-grade" naming; v0.16 corrects that — they're production-first, paper-grade as a side effect.
267
-
268
- - `HeldOutGate` — held-out paired-delta gate with `few_runs` /
269
- `negative_delta` / `overfit_gap` rejection codes and a full evidence
270
- block on every decision. Sits alongside the existing bootstrap-CI
271
- `promotion-gate.ts`: that one asks "is this real or noise?", this one
272
- asks "is this a real win on held-out and not overfit?". Use both.
273
- - `RunRecord` — typed run schema with mandatory snapshot-pinned `model`,
274
- `promptHash`, `configHash`, `commitSha`, `costUsd`, `splitTag`.
275
- Runtime validator throws on missing fields. Reproducibility falls
276
- out for free.
277
- - `pairedBootstrap`, `pairedWilcoxon`, `bhAdjust` — statistical
278
- primitives every rigorous A/B test needs. Already-existing primitives
279
- are re-exported for paper-style aliases.
280
- - `runCanaries` — silent judge-fallback, calibration drift (KS test),
281
- distribution shift (chi-square). Catches the failure mode where your
282
- judge silently degrades to a constant-0.30 confidence and you ship
283
- configs graded by a stub.
284
- - `summaryTable`, `paretoChart`, `gainHistogram` — A/B reporting
285
- helpers. `summaryTable` emits markdown with means + 95% bootstrap
286
- CIs + paired Wilcoxon p (BH-adjusted) + Cohen's d. Useful for both
287
- internal status reports and paper Table 1s.
288
- - `Researcher` — stable interface for an external agent that drives the
289
- meta-loop (`inspectFailures` → `proposeChange` → `applyChange` →
290
- `evaluateChange`). Ship a `NoopResearcher` as a placeholder; real
291
- implementations live downstream.
292
- - `benchmarks/routing` — synthetic 16-task router benchmark we own.
293
- Ships in the package. Reference wrappers for GSM8K and SWE-Bench
294
- Lite live under `examples/benchmarks/` — read, copy, adapt. All
295
- three implement one `BenchmarkAdapter` shape with deterministic
296
- splits and fail-loud env-var configuration.
297
-
298
- ### v0.16 changes from v0.15
299
-
300
- - Renamed `paperTable` → `summaryTable`, `paretoFigure` → `paretoChart`,
301
- `gainDistributionFigure` → `gainHistogram`. Underlying semantics
302
- unchanged. Type names follow (`SummaryTable`, `SummaryTableOptions`,
303
- `SummaryTableRow`).
304
- - File: `src/paper-report.ts` → `src/summary-report.ts`.
305
- - Drop the "paper-grade" framing — the primitives are production-first.
306
-
307
- See `CHANGELOG.md` for the full list. `.claude/skills/agent-eval/SKILL.md`
308
- covers usage directives and pitfalls.
309
-
310
- ## Tech stack
311
-
312
- - TypeScript strict, no semicolons, single quotes, 2-space indent
313
- - `tsup` for bundling, `vitest` for tests
314
- - `@tangle-network/tcloud` for LLM calls (judges, driver)
315
- - `hono` + `@asteasolutions/zod-to-openapi` for the wire protocol
316
-
317
- ## Develop
140
+ Python client tests:
318
141
 
319
142
  ```sh
320
- pnpm install
321
- pnpm typecheck
322
- pnpm test
323
143
  pnpm build
324
- pnpm openapi # write dist/openapi.json from the wire schemas
325
-
326
- # Run the server locally
327
- node dist/cli.js serve --port 5005
328
-
329
- # Python client tests (require pnpm build first)
330
- cd clients/python && pip install -e ".[dev]" && pytest
144
+ cd clients/python
145
+ pip install -e ".[dev]"
146
+ pytest
331
147
  ```
332
148
 
333
149
  ## Release
334
150
 
335
- `@tangle-network/agent-eval` (npm) and `tangle-agent-eval` (PyPI) ship from the same git tag in the same CI workflow. If either fails to publish, neither does. Versions are locked.
151
+ `@tangle-network/agent-eval` publishes to npm. The Python client lives under
152
+ `clients/python` and is versioned from this repository.
336
153
 
337
- ## Related
154
+ ## Related Packages
338
155
 
156
+ - [`@tangle-network/agent-runtime`](https://github.com/tangle-network/agent-runtime)
157
+ - [`@tangle-network/agent-knowledge`](https://github.com/tangle-network/agent-knowledge)
158
+ - [`@tangle-network/agent-integrations`](https://github.com/tangle-network/agent-integrations)
339
159
  - [`@tangle-network/agent-gateway`](https://github.com/tangle-network/agent-gateway)
340
- - [`@tangle-network/agent-client`](https://github.com/tangle-network/agent-client)
341
160
  - [`@tangle-network/tcloud`](https://github.com/tangle-network/tcloud)
342
161
 
343
162
  ## License
@@ -1,6 +1,6 @@
1
1
  import {
2
2
  callLlmJson
3
- } from "./chunk-ITN4YOZY.js";
3
+ } from "./chunk-JAOLXRIA.js";
4
4
 
5
5
  // src/wire/schemas.ts
6
6
  import { extendZodWithOpenApi } from "@asteasolutions/zod-to-openapi";
@@ -591,4 +591,4 @@ export {
591
591
  runRpcOnce,
592
592
  runRpcBatch
593
593
  };
594
- //# sourceMappingURL=chunk-OZPRSK4A.js.map
594
+ //# sourceMappingURL=chunk-CJJSB6ZQ.js.map
@@ -76,6 +76,56 @@ function stripFencedJson(raw) {
76
76
  const m = trimmed.match(/^```(?:json)?\s*\n?([\s\S]*?)\n?```\s*$/);
77
77
  return m ? m[1].trim() : trimmed;
78
78
  }
79
+ function extractJsonPayload(raw) {
80
+ const stripped = stripFencedJson(raw);
81
+ try {
82
+ JSON.parse(stripped);
83
+ return stripped;
84
+ } catch {
85
+ }
86
+ const starts = [...stripped.matchAll(/[\[{]/g)].map((match) => match.index).filter((index) => index != null);
87
+ for (const start of starts) {
88
+ const candidate = extractBalancedJson(stripped, start);
89
+ if (!candidate) continue;
90
+ try {
91
+ JSON.parse(candidate);
92
+ return candidate;
93
+ } catch {
94
+ }
95
+ }
96
+ return stripped;
97
+ }
98
+ function extractBalancedJson(input, start) {
99
+ const opener = input[start];
100
+ const closer = opener === "{" ? "}" : opener === "[" ? "]" : null;
101
+ if (!closer) return null;
102
+ const stack = [closer];
103
+ let isInString = false;
104
+ let isEscaped = false;
105
+ for (let i = start + 1; i < input.length; i++) {
106
+ const char = input[i];
107
+ if (isEscaped) {
108
+ isEscaped = false;
109
+ continue;
110
+ }
111
+ if (char === "\\") {
112
+ isEscaped = isInString;
113
+ continue;
114
+ }
115
+ if (char === '"') {
116
+ isInString = !isInString;
117
+ continue;
118
+ }
119
+ if (isInString) continue;
120
+ if (char === "{") stack.push("}");
121
+ else if (char === "[") stack.push("]");
122
+ else if (char === stack[stack.length - 1]) {
123
+ stack.pop();
124
+ if (stack.length === 0) return input.slice(start, i + 1);
125
+ }
126
+ }
127
+ return null;
128
+ }
79
129
  async function callLlm(req, opts = {}) {
80
130
  const baseUrl = (opts.baseUrl ?? DEFAULT_BASE_URL).replace(/\/+$/, "");
81
131
  const url = `${baseUrl}/chat/completions`;
@@ -159,7 +209,7 @@ async function callLlmJson(req, opts = {}) {
159
209
  }
160
210
  }
161
211
  function parseJsonSafely(content, model) {
162
- const stripped = stripFencedJson(content);
212
+ const stripped = extractJsonPayload(content);
163
213
  try {
164
214
  return JSON.parse(stripped);
165
215
  } catch (err) {
@@ -212,4 +262,4 @@ export {
212
262
  probeLlm,
213
263
  LlmClient
214
264
  };
215
- //# sourceMappingURL=chunk-ITN4YOZY.js.map
265
+ //# sourceMappingURL=chunk-JAOLXRIA.js.map
@@ -0,0 +1 @@
1
+ {"version":3,"sources":["../src/llm-client.ts"],"sourcesContent":["/**\n * LLM client with graceful degrade.\n *\n * OpenAI-compatible `/v1/chat/completions` client with:\n * - Exponential-backoff retry on 429 + 5xx gateway errors (502/503/504).\n * - Retry on transient network errors (fetch failed, AbortError, ECONNRESET).\n * - Graceful json_schema → json_object degrade on 400 with schema-reject body.\n * - Fenced-JSON stripping (```json ... ```) for models that wrap structured output.\n * - Configurable base URL + api key / bearer, works with LiteLLM proxies, OpenAI\n * directly, cli-bridge subscriptions, and any router that speaks the spec.\n *\n * Usage:\n * const { value, result } = await callLlmJson<MyType>(\n * { model: 'gpt-4o', messages: [...], jsonSchema: { name: 'x', schema: {...} } },\n * { baseUrl: 'https://router.tangle.tools/v1', apiKey: process.env.KEY },\n * )\n *\n * This is THE llm-calling seam for agent-eval primitives that need structured\n * output (semantic concept judge, reviewer directives, critic scores). Primitives\n * that need free-form text use `callLlm` and parse output themselves.\n */\n\n// ─── Types ──────────────────────────────────────────────────────────────\n\nexport interface LlmMessage {\n role: 'system' | 'user' | 'assistant'\n /**\n * Either a plain text content string OR a multimodal content array\n * (text + image_url parts) for vision-capable models.\n */\n content:\n | string\n | Array<\n | { type: 'text'; text: string }\n | { type: 'image_url'; image_url: { url: string; detail?: 'auto' | 'low' | 'high' } }\n >\n}\n\nexport interface LlmCallRequest {\n model: string\n messages: LlmMessage[]\n /** Optional JSON-mode response format (response_format: json_object). */\n jsonMode?: boolean\n /** Optional structured output via JSON Schema. Falls back to json_object on 400. */\n jsonSchema?: { name: string; schema: Record<string, unknown> }\n temperature?: number\n maxTokens?: number\n /** Per-call timeout, default 60s. */\n timeoutMs?: number\n}\n\nexport interface LlmUsage {\n promptTokens: number\n completionTokens: number\n totalTokens: number\n /** Proxies populate this when prompt caching is on. */\n cachedPromptTokens?: number\n}\n\nexport interface LlmCallResult {\n /** The text content of the first choice. Empty string if none. */\n content: string\n usage: LlmUsage\n /**\n * Cost in USD. Pulled from proxy's `_response_cost` field when present;\n * `null` when neither the proxy nor the caller can derive it.\n */\n costUsd: number | null\n /** Model name actually used (echoed from response). */\n model: string\n /** Wall-clock duration of the HTTP call (last attempt, if retried). */\n durationMs: number\n /** Raw response body. */\n raw: Record<string, unknown>\n}\n\nexport class LlmCallError extends Error {\n constructor(\n message: string,\n public readonly status: number,\n public readonly body: string,\n public readonly model: string,\n ) {\n super(message)\n this.name = 'LlmCallError'\n }\n}\n\nexport interface LlmClientOptions {\n /** Base URL (without trailing slash). Must end at the `/v1` prefix. */\n baseUrl?: string\n /** Bearer token — either `apiKey` or `bearer` populates `Authorization: Bearer ...`. */\n apiKey?: string\n bearer?: string\n /** Override for the `Authorization` header (e.g. `X-Auth: ...`). Takes precedence over apiKey/bearer. */\n authHeader?: { name: string; value: string }\n /** Default timeout in ms. Per-call can override. */\n defaultTimeoutMs?: number\n /** Max retry attempts on retriable errors. Default 3 (1 initial + 2 retries). */\n maxRetries?: number\n /** Fetch implementation — defaults to global `fetch`. Override for custom transport (e.g. tests). */\n fetch?: typeof fetch\n}\n\n// ─── Internals ──────────────────────────────────────────────────────────\n\nconst DEFAULT_BASE_URL = 'https://router.tangle.tools/v1'\nconst DEFAULT_TIMEOUT_MS = 60_000\nconst DEFAULT_MAX_RETRIES = 3\n\nconst RETRYABLE_STATUS = new Set([429, 502, 503, 504])\n\nfunction isRetryableError(err: unknown): boolean {\n if (err instanceof LlmCallError) return RETRYABLE_STATUS.has(err.status)\n if (err instanceof Error) {\n return (\n err.name === 'AbortError' ||\n err.name === 'TimeoutError' ||\n /fetch failed|ECONNRESET|ETIMEDOUT|EAI_AGAIN/i.test(err.message)\n )\n }\n return false\n}\n\nfunction parseRetryAfter(headers: Headers): number | null {\n const h = headers.get('retry-after')\n if (!h) return null\n const asNumber = Number(h)\n if (Number.isFinite(asNumber) && asNumber > 0) return asNumber * 1000\n const asDate = Date.parse(h)\n if (Number.isFinite(asDate)) return Math.max(0, asDate - Date.now())\n return null\n}\n\nfunction backoffMs(attempt: number): number {\n // 500ms, 1s, 2s, 4s, ...\n return Math.min(500 * Math.pow(2, attempt), 16_000)\n}\n\nfunction buildHeaders(opts: LlmClientOptions): Record<string, string> {\n const headers: Record<string, string> = {\n 'Content-Type': 'application/json',\n Accept: 'application/json',\n }\n if (opts.authHeader) {\n headers[opts.authHeader.name] = opts.authHeader.value\n } else if (opts.bearer || opts.apiKey) {\n headers.Authorization = `Bearer ${opts.bearer ?? opts.apiKey}`\n }\n return headers\n}\n\nfunction isSchemaRejection(status: number, body: string): boolean {\n if (status !== 400) return false\n const lower = body.toLowerCase()\n return (\n lower.includes('response_format') ||\n lower.includes('json_schema') ||\n lower.includes('is unavailable') ||\n lower.includes('not supported')\n )\n}\n\nfunction buildBody(req: LlmCallRequest, forceJsonObject: boolean): Record<string, unknown> {\n const body: Record<string, unknown> = {\n model: req.model,\n messages: req.messages,\n temperature: req.temperature ?? 0,\n }\n if (req.maxTokens != null) body.max_tokens = req.maxTokens\n\n if (req.jsonSchema && !forceJsonObject) {\n body.response_format = {\n type: 'json_schema',\n json_schema: { name: req.jsonSchema.name, schema: req.jsonSchema.schema, strict: true },\n }\n } else if (req.jsonMode || req.jsonSchema) {\n body.response_format = { type: 'json_object' }\n }\n\n return body\n}\n\nasync function sleep(ms: number): Promise<void> {\n return new Promise((resolve) => setTimeout(resolve, ms))\n}\n\n// ─── Public API ─────────────────────────────────────────────────────────\n\n/**\n * Strip a ```json / ``` code fence if the model emitted one.\n * Idempotent for naked JSON. Some models (claude-code via router, certain\n * deepseek models) wrap output even under json_object.\n */\nexport function stripFencedJson(raw: string): string {\n const trimmed = raw.trim()\n const m = trimmed.match(/^```(?:json)?\\s*\\n?([\\s\\S]*?)\\n?```\\s*$/)\n return m ? m[1]!.trim() : trimmed\n}\n\nexport function extractJsonPayload(raw: string): string {\n const stripped = stripFencedJson(raw)\n try {\n JSON.parse(stripped)\n return stripped\n } catch {\n // Continue with balanced extraction below.\n }\n\n const starts = [...stripped.matchAll(/[\\[{]/g)].map((match) => match.index).filter((index) => index != null)\n for (const start of starts) {\n const candidate = extractBalancedJson(stripped, start)\n if (!candidate) continue\n try {\n JSON.parse(candidate)\n return candidate\n } catch {\n // Keep scanning; earlier braces may belong to prose.\n }\n }\n\n return stripped\n}\n\nfunction extractBalancedJson(input: string, start: number): string | null {\n const opener = input[start]\n const closer = opener === '{' ? '}' : opener === '[' ? ']' : null\n if (!closer) return null\n\n const stack: string[] = [closer]\n let isInString = false\n let isEscaped = false\n\n for (let i = start + 1; i < input.length; i++) {\n const char = input[i]!\n if (isEscaped) {\n isEscaped = false\n continue\n }\n if (char === '\\\\') {\n isEscaped = isInString\n continue\n }\n if (char === '\"') {\n isInString = !isInString\n continue\n }\n if (isInString) continue\n\n if (char === '{') stack.push('}')\n else if (char === '[') stack.push(']')\n else if (char === stack[stack.length - 1]) {\n stack.pop()\n if (stack.length === 0) return input.slice(start, i + 1)\n }\n }\n\n return null\n}\n\n/**\n * Low-level call. Returns raw content + usage + cost. Retries on transient\n * failures; does NOT degrade schema here — callers that want graceful\n * degrade use `callLlmJson`.\n */\nexport async function callLlm(\n req: LlmCallRequest,\n opts: LlmClientOptions = {},\n): Promise<LlmCallResult> {\n const baseUrl = (opts.baseUrl ?? DEFAULT_BASE_URL).replace(/\\/+$/, '')\n const url = `${baseUrl}/chat/completions`\n const timeoutMs = req.timeoutMs ?? opts.defaultTimeoutMs ?? DEFAULT_TIMEOUT_MS\n const maxRetries = opts.maxRetries ?? DEFAULT_MAX_RETRIES\n const fetchFn = opts.fetch ?? globalThis.fetch\n const headers = buildHeaders(opts)\n\n let lastErr: unknown\n for (let attempt = 0; attempt < maxRetries; attempt++) {\n const controller = new AbortController()\n const timeoutHandle = setTimeout(() => controller.abort(), timeoutMs)\n const started = Date.now()\n\n try {\n const res = await fetchFn(url, {\n method: 'POST',\n headers,\n body: JSON.stringify(buildBody(req, false)),\n signal: controller.signal,\n })\n clearTimeout(timeoutHandle)\n\n if (!res.ok) {\n const body = await res.text()\n const err = new LlmCallError(\n `LLM call ${res.status}: ${body.slice(0, 300)}`,\n res.status,\n body,\n req.model,\n )\n if (RETRYABLE_STATUS.has(res.status) && attempt < maxRetries - 1) {\n lastErr = err\n const retryAfter = parseRetryAfter(res.headers)\n await sleep(retryAfter ?? backoffMs(attempt))\n continue\n }\n throw err\n }\n\n const json = (await res.json()) as Record<string, unknown>\n const choice = (json.choices as Array<{ message?: { content?: string } }> | undefined)?.[0]\n const usageRaw = (json.usage as Record<string, unknown> | undefined) ?? {}\n const costFromProxy = (json._response_cost ?? json.cost_usd) as number | undefined\n\n return {\n content: choice?.message?.content ?? '',\n usage: {\n promptTokens: Number(usageRaw.prompt_tokens ?? 0),\n completionTokens: Number(usageRaw.completion_tokens ?? 0),\n totalTokens: Number(usageRaw.total_tokens ?? 0),\n cachedPromptTokens:\n usageRaw.prompt_tokens_details &&\n typeof usageRaw.prompt_tokens_details === 'object'\n ? Number(\n (usageRaw.prompt_tokens_details as Record<string, unknown>).cached_tokens ?? 0,\n )\n : undefined,\n },\n costUsd: typeof costFromProxy === 'number' ? costFromProxy : null,\n model: (json.model as string) ?? req.model,\n durationMs: Date.now() - started,\n raw: json,\n }\n } catch (err) {\n clearTimeout(timeoutHandle)\n lastErr = err\n if (attempt < maxRetries - 1 && isRetryableError(err)) {\n await sleep(backoffMs(attempt))\n continue\n }\n throw err\n }\n }\n throw lastErr instanceof Error ? lastErr : new Error(String(lastErr))\n}\n\n/**\n * Structured-output call. Returns parsed JSON plus the raw result envelope.\n * Degrades `jsonSchema` → `jsonMode` on a 400 that names the schema param —\n * critical for deepseek-v3/v4, kimi-k2.6, and other models that don't accept\n * the `response_format.json_schema` shape but DO accept `json_object`.\n */\nexport async function callLlmJson<T = unknown>(\n req: LlmCallRequest,\n opts: LlmClientOptions = {},\n): Promise<{ value: T; result: LlmCallResult }> {\n try {\n const result = await callLlm({ ...req, jsonMode: req.jsonMode ?? !req.jsonSchema }, opts)\n const value = parseJsonSafely<T>(result.content, result.model)\n return { value, result }\n } catch (err) {\n if (err instanceof LlmCallError && isSchemaRejection(err.status, err.body) && req.jsonSchema) {\n // Degrade to json_object + retry.\n const degradedReq: LlmCallRequest = { ...req, jsonMode: true, jsonSchema: undefined }\n const result = await callLlm(degradedReq, opts)\n const value = parseJsonSafely<T>(result.content, result.model)\n return { value, result }\n }\n throw err\n }\n}\n\nfunction parseJsonSafely<T>(content: string, model: string): T {\n const stripped = extractJsonPayload(content)\n try {\n return JSON.parse(stripped) as T\n } catch (err) {\n throw new Error(\n `LLM returned non-JSON content (model=${model}): ${\n err instanceof Error ? err.message : String(err)\n }\\n--- raw content ---\\n${content.slice(0, 800)}`,\n )\n }\n}\n\n/**\n * Probe whether a model is reachable. Returns latency + null error on\n * success; `ok=false` + error message on any failure (HTTP, timeout,\n * network, parse). Designed for sweep preflights — fail loud at the\n * boundary before burning a 30-leaf run on a misconfigured router.\n *\n * Sends a tiny `ping` message with `maxTokens=64`. Reasoning models\n * (glm-5.1, deepseek-v4) can burn the entire budget on internal reasoning\n * for short prompts, so don't tighten this further. We don't validate\n * content; HTTP 200 means reachable.\n */\nexport async function probeLlm(\n model: string,\n opts: LlmClientOptions & { timeoutMs?: number } = {},\n): Promise<{ ok: boolean; latencyMs: number; error: string | null }> {\n const start = Date.now()\n try {\n await callLlm(\n {\n model,\n messages: [{ role: 'user', content: 'ping' }],\n maxTokens: 64,\n timeoutMs: opts.timeoutMs ?? 30_000,\n },\n opts,\n )\n return { ok: true, latencyMs: Date.now() - start, error: null }\n } catch (err) {\n return {\n ok: false,\n latencyMs: Date.now() - start,\n error: err instanceof Error ? err.message : String(err),\n }\n }\n}\n\n/**\n * Stateful client — construct once with defaults, call many times.\n * Thin wrapper around the free functions; exists for callers that want\n * to inject a single configured instance into multiple primitives.\n */\nexport class LlmClient {\n constructor(private readonly opts: LlmClientOptions = {}) {}\n\n call(req: LlmCallRequest, per?: LlmClientOptions): Promise<LlmCallResult> {\n return callLlm(req, { ...this.opts, ...per })\n }\n\n callJson<T = unknown>(\n req: LlmCallRequest,\n per?: LlmClientOptions,\n ): Promise<{ value: T; result: LlmCallResult }> {\n return callLlmJson<T>(req, { ...this.opts, ...per })\n }\n}\n"],"mappings":";AA4EO,IAAM,eAAN,cAA2B,MAAM;AAAA,EACtC,YACE,SACgB,QACA,MACA,OAChB;AACA,UAAM,OAAO;AAJG;AACA;AACA;AAGhB,SAAK,OAAO;AAAA,EACd;AAAA,EANkB;AAAA,EACA;AAAA,EACA;AAKpB;AAoBA,IAAM,mBAAmB;AACzB,IAAM,qBAAqB;AAC3B,IAAM,sBAAsB;AAE5B,IAAM,mBAAmB,oBAAI,IAAI,CAAC,KAAK,KAAK,KAAK,GAAG,CAAC;AAErD,SAAS,iBAAiB,KAAuB;AAC/C,MAAI,eAAe,aAAc,QAAO,iBAAiB,IAAI,IAAI,MAAM;AACvE,MAAI,eAAe,OAAO;AACxB,WACE,IAAI,SAAS,gBACb,IAAI,SAAS,kBACb,+CAA+C,KAAK,IAAI,OAAO;AAAA,EAEnE;AACA,SAAO;AACT;AAEA,SAAS,gBAAgB,SAAiC;AACxD,QAAM,IAAI,QAAQ,IAAI,aAAa;AACnC,MAAI,CAAC,EAAG,QAAO;AACf,QAAM,WAAW,OAAO,CAAC;AACzB,MAAI,OAAO,SAAS,QAAQ,KAAK,WAAW,EAAG,QAAO,WAAW;AACjE,QAAM,SAAS,KAAK,MAAM,CAAC;AAC3B,MAAI,OAAO,SAAS,MAAM,EAAG,QAAO,KAAK,IAAI,GAAG,SAAS,KAAK,IAAI,CAAC;AACnE,SAAO;AACT;AAEA,SAAS,UAAU,SAAyB;AAE1C,SAAO,KAAK,IAAI,MAAM,KAAK,IAAI,GAAG,OAAO,GAAG,IAAM;AACpD;AAEA,SAAS,aAAa,MAAgD;AACpE,QAAM,UAAkC;AAAA,IACtC,gBAAgB;AAAA,IAChB,QAAQ;AAAA,EACV;AACA,MAAI,KAAK,YAAY;AACnB,YAAQ,KAAK,WAAW,IAAI,IAAI,KAAK,WAAW;AAAA,EAClD,WAAW,KAAK,UAAU,KAAK,QAAQ;AACrC,YAAQ,gBAAgB,UAAU,KAAK,UAAU,KAAK,MAAM;AAAA,EAC9D;AACA,SAAO;AACT;AAEA,SAAS,kBAAkB,QAAgB,MAAuB;AAChE,MAAI,WAAW,IAAK,QAAO;AAC3B,QAAM,QAAQ,KAAK,YAAY;AAC/B,SACE,MAAM,SAAS,iBAAiB,KAChC,MAAM,SAAS,aAAa,KAC5B,MAAM,SAAS,gBAAgB,KAC/B,MAAM,SAAS,eAAe;AAElC;AAEA,SAAS,UAAU,KAAqB,iBAAmD;AACzF,QAAM,OAAgC;AAAA,IACpC,OAAO,IAAI;AAAA,IACX,UAAU,IAAI;AAAA,IACd,aAAa,IAAI,eAAe;AAAA,EAClC;AACA,MAAI,IAAI,aAAa,KAAM,MAAK,aAAa,IAAI;AAEjD,MAAI,IAAI,cAAc,CAAC,iBAAiB;AACtC,SAAK,kBAAkB;AAAA,MACrB,MAAM;AAAA,MACN,aAAa,EAAE,MAAM,IAAI,WAAW,MAAM,QAAQ,IAAI,WAAW,QAAQ,QAAQ,KAAK;AAAA,IACxF;AAAA,EACF,WAAW,IAAI,YAAY,IAAI,YAAY;AACzC,SAAK,kBAAkB,EAAE,MAAM,cAAc;AAAA,EAC/C;AAEA,SAAO;AACT;AAEA,eAAe,MAAM,IAA2B;AAC9C,SAAO,IAAI,QAAQ,CAAC,YAAY,WAAW,SAAS,EAAE,CAAC;AACzD;AASO,SAAS,gBAAgB,KAAqB;AACnD,QAAM,UAAU,IAAI,KAAK;AACzB,QAAM,IAAI,QAAQ,MAAM,yCAAyC;AACjE,SAAO,IAAI,EAAE,CAAC,EAAG,KAAK,IAAI;AAC5B;AAEO,SAAS,mBAAmB,KAAqB;AACtD,QAAM,WAAW,gBAAgB,GAAG;AACpC,MAAI;AACF,SAAK,MAAM,QAAQ;AACnB,WAAO;AAAA,EACT,QAAQ;AAAA,EAER;AAEA,QAAM,SAAS,CAAC,GAAG,SAAS,SAAS,QAAQ,CAAC,EAAE,IAAI,CAAC,UAAU,MAAM,KAAK,EAAE,OAAO,CAAC,UAAU,SAAS,IAAI;AAC3G,aAAW,SAAS,QAAQ;AAC1B,UAAM,YAAY,oBAAoB,UAAU,KAAK;AACrD,QAAI,CAAC,UAAW;AAChB,QAAI;AACF,WAAK,MAAM,SAAS;AACpB,aAAO;AAAA,IACT,QAAQ;AAAA,IAER;AAAA,EACF;AAEA,SAAO;AACT;AAEA,SAAS,oBAAoB,OAAe,OAA8B;AACxE,QAAM,SAAS,MAAM,KAAK;AAC1B,QAAM,SAAS,WAAW,MAAM,MAAM,WAAW,MAAM,MAAM;AAC7D,MAAI,CAAC,OAAQ,QAAO;AAEpB,QAAM,QAAkB,CAAC,MAAM;AAC/B,MAAI,aAAa;AACjB,MAAI,YAAY;AAEhB,WAAS,IAAI,QAAQ,GAAG,IAAI,MAAM,QAAQ,KAAK;AAC7C,UAAM,OAAO,MAAM,CAAC;AACpB,QAAI,WAAW;AACb,kBAAY;AACZ;AAAA,IACF;AACA,QAAI,SAAS,MAAM;AACjB,kBAAY;AACZ;AAAA,IACF;AACA,QAAI,SAAS,KAAK;AAChB,mBAAa,CAAC;AACd;AAAA,IACF;AACA,QAAI,WAAY;AAEhB,QAAI,SAAS,IAAK,OAAM,KAAK,GAAG;AAAA,aACvB,SAAS,IAAK,OAAM,KAAK,GAAG;AAAA,aAC5B,SAAS,MAAM,MAAM,SAAS,CAAC,GAAG;AACzC,YAAM,IAAI;AACV,UAAI,MAAM,WAAW,EAAG,QAAO,MAAM,MAAM,OAAO,IAAI,CAAC;AAAA,IACzD;AAAA,EACF;AAEA,SAAO;AACT;AAOA,eAAsB,QACpB,KACA,OAAyB,CAAC,GACF;AACxB,QAAM,WAAW,KAAK,WAAW,kBAAkB,QAAQ,QAAQ,EAAE;AACrE,QAAM,MAAM,GAAG,OAAO;AACtB,QAAM,YAAY,IAAI,aAAa,KAAK,oBAAoB;AAC5D,QAAM,aAAa,KAAK,cAAc;AACtC,QAAM,UAAU,KAAK,SAAS,WAAW;AACzC,QAAM,UAAU,aAAa,IAAI;AAEjC,MAAI;AACJ,WAAS,UAAU,GAAG,UAAU,YAAY,WAAW;AACrD,UAAM,aAAa,IAAI,gBAAgB;AACvC,UAAM,gBAAgB,WAAW,MAAM,WAAW,MAAM,GAAG,SAAS;AACpE,UAAM,UAAU,KAAK,IAAI;AAEzB,QAAI;AACF,YAAM,MAAM,MAAM,QAAQ,KAAK;AAAA,QAC7B,QAAQ;AAAA,QACR;AAAA,QACA,MAAM,KAAK,UAAU,UAAU,KAAK,KAAK,CAAC;AAAA,QAC1C,QAAQ,WAAW;AAAA,MACrB,CAAC;AACD,mBAAa,aAAa;AAE1B,UAAI,CAAC,IAAI,IAAI;AACX,cAAM,OAAO,MAAM,IAAI,KAAK;AAC5B,cAAM,MAAM,IAAI;AAAA,UACd,YAAY,IAAI,MAAM,KAAK,KAAK,MAAM,GAAG,GAAG,CAAC;AAAA,UAC7C,IAAI;AAAA,UACJ;AAAA,UACA,IAAI;AAAA,QACN;AACA,YAAI,iBAAiB,IAAI,IAAI,MAAM,KAAK,UAAU,aAAa,GAAG;AAChE,oBAAU;AACV,gBAAM,aAAa,gBAAgB,IAAI,OAAO;AAC9C,gBAAM,MAAM,cAAc,UAAU,OAAO,CAAC;AAC5C;AAAA,QACF;AACA,cAAM;AAAA,MACR;AAEA,YAAM,OAAQ,MAAM,IAAI,KAAK;AAC7B,YAAM,SAAU,KAAK,UAAoE,CAAC;AAC1F,YAAM,WAAY,KAAK,SAAiD,CAAC;AACzE,YAAM,gBAAiB,KAAK,kBAAkB,KAAK;AAEnD,aAAO;AAAA,QACL,SAAS,QAAQ,SAAS,WAAW;AAAA,QACrC,OAAO;AAAA,UACL,cAAc,OAAO,SAAS,iBAAiB,CAAC;AAAA,UAChD,kBAAkB,OAAO,SAAS,qBAAqB,CAAC;AAAA,UACxD,aAAa,OAAO,SAAS,gBAAgB,CAAC;AAAA,UAC9C,oBACE,SAAS,yBACT,OAAO,SAAS,0BAA0B,WACtC;AAAA,YACG,SAAS,sBAAkD,iBAAiB;AAAA,UAC/E,IACA;AAAA,QACR;AAAA,QACA,SAAS,OAAO,kBAAkB,WAAW,gBAAgB;AAAA,QAC7D,OAAQ,KAAK,SAAoB,IAAI;AAAA,QACrC,YAAY,KAAK,IAAI,IAAI;AAAA,QACzB,KAAK;AAAA,MACP;AAAA,IACF,SAAS,KAAK;AACZ,mBAAa,aAAa;AAC1B,gBAAU;AACV,UAAI,UAAU,aAAa,KAAK,iBAAiB,GAAG,GAAG;AACrD,cAAM,MAAM,UAAU,OAAO,CAAC;AAC9B;AAAA,MACF;AACA,YAAM;AAAA,IACR;AAAA,EACF;AACA,QAAM,mBAAmB,QAAQ,UAAU,IAAI,MAAM,OAAO,OAAO,CAAC;AACtE;AAQA,eAAsB,YACpB,KACA,OAAyB,CAAC,GACoB;AAC9C,MAAI;AACF,UAAM,SAAS,MAAM,QAAQ,EAAE,GAAG,KAAK,UAAU,IAAI,YAAY,CAAC,IAAI,WAAW,GAAG,IAAI;AACxF,UAAM,QAAQ,gBAAmB,OAAO,SAAS,OAAO,KAAK;AAC7D,WAAO,EAAE,OAAO,OAAO;AAAA,EACzB,SAAS,KAAK;AACZ,QAAI,eAAe,gBAAgB,kBAAkB,IAAI,QAAQ,IAAI,IAAI,KAAK,IAAI,YAAY;AAE5F,YAAM,cAA8B,EAAE,GAAG,KAAK,UAAU,MAAM,YAAY,OAAU;AACpF,YAAM,SAAS,MAAM,QAAQ,aAAa,IAAI;AAC9C,YAAM,QAAQ,gBAAmB,OAAO,SAAS,OAAO,KAAK;AAC7D,aAAO,EAAE,OAAO,OAAO;AAAA,IACzB;AACA,UAAM;AAAA,EACR;AACF;AAEA,SAAS,gBAAmB,SAAiB,OAAkB;AAC7D,QAAM,WAAW,mBAAmB,OAAO;AAC3C,MAAI;AACF,WAAO,KAAK,MAAM,QAAQ;AAAA,EAC5B,SAAS,KAAK;AACZ,UAAM,IAAI;AAAA,MACR,wCAAwC,KAAK,MAC3C,eAAe,QAAQ,IAAI,UAAU,OAAO,GAAG,CACjD;AAAA;AAAA,EAA0B,QAAQ,MAAM,GAAG,GAAG,CAAC;AAAA,IACjD;AAAA,EACF;AACF;AAaA,eAAsB,SACpB,OACA,OAAkD,CAAC,GACgB;AACnE,QAAM,QAAQ,KAAK,IAAI;AACvB,MAAI;AACF,UAAM;AAAA,MACJ;AAAA,QACE;AAAA,QACA,UAAU,CAAC,EAAE,MAAM,QAAQ,SAAS,OAAO,CAAC;AAAA,QAC5C,WAAW;AAAA,QACX,WAAW,KAAK,aAAa;AAAA,MAC/B;AAAA,MACA;AAAA,IACF;AACA,WAAO,EAAE,IAAI,MAAM,WAAW,KAAK,IAAI,IAAI,OAAO,OAAO,KAAK;AAAA,EAChE,SAAS,KAAK;AACZ,WAAO;AAAA,MACL,IAAI;AAAA,MACJ,WAAW,KAAK,IAAI,IAAI;AAAA,MACxB,OAAO,eAAe,QAAQ,IAAI,UAAU,OAAO,GAAG;AAAA,IACxD;AAAA,EACF;AACF;AAOO,IAAM,YAAN,MAAgB;AAAA,EACrB,YAA6B,OAAyB,CAAC,GAAG;AAA7B;AAAA,EAA8B;AAAA,EAA9B;AAAA,EAE7B,KAAK,KAAqB,KAAgD;AACxE,WAAO,QAAQ,KAAK,EAAE,GAAG,KAAK,MAAM,GAAG,IAAI,CAAC;AAAA,EAC9C;AAAA,EAEA,SACE,KACA,KAC8C;AAC9C,WAAO,YAAe,KAAK,EAAE,GAAG,KAAK,MAAM,GAAG,IAAI,CAAC;AAAA,EACrD;AACF;","names":[]}
package/dist/cli.js CHANGED
@@ -5,8 +5,8 @@ import {
5
5
  runRpcBatch,
6
6
  runRpcOnce,
7
7
  startServer
8
- } from "./chunk-OZPRSK4A.js";
9
- import "./chunk-ITN4YOZY.js";
8
+ } from "./chunk-CJJSB6ZQ.js";
9
+ import "./chunk-JAOLXRIA.js";
10
10
  import "./chunk-PZ5AY32C.js";
11
11
 
12
12
  // src/cli.ts