@tangle-network/agent-runtime 0.95.0 → 0.97.0

package/skills/build-with-agent-runtime/SKILL.md

@@ -1,217 +1,126 @@
 ---
 name: build-with-agent-runtime
-description: Use before hand-rolling a tool loop, driver, corpus, or optimizer wrapper. Create an agent genome, run it on a benchmark, optimize+gate it, observe/ship it with @tangle-network/agent-runtime. The genome→run→optimize→observe spine.
+description: Use before adding an agent loop, benchmark, optimizer, trace wrapper, or candidate activation path. Find and compose the existing Runtime, Eval, Knowledge, and Interface primitives.
 ---
 
-# build-with-agent-runtime
-
-The one create→run→optimize→observe seam for `@tangle-network/agent-runtime`. A
-**genome** (an `AgentProfile`/`AgentSurfaces` — systemPrompt + skills + tools +
-mcp + knowledge + memory + rag as ONE combined surface) runs as a
-**driver⟷worker** shape over a **benchmark**, gets **optimized by a gated loop**
-that evolves the genome and certifies wins on a **frozen holdout**, and is
-**observed** through the one lifecycle stream. The selector is never the judge;
-observation attaches to the loop, never to the portable genome.
-
-If you are about to write a `runConversation`, a "skill optimizer", a
-"profile-seam", a depth-vs-breadth A/B harness, a bootstrap loop, or a
-`new Sandbox(...)` + stream + read dance — **stop.** It exists, and a parallel
-silently breaks a load-bearing invariant (equal-k, selector≠judge,
-capture-integrity, or eval/prod parity).
-
-## Load order — point at source, never freeze snippets
-
-This skill carries **no API snippets**. The barrel MOVES (`./loops` is the
-runtime barrel), the agent-eval pin drifts, and signatures get
-corrected in place. Freezing a snippet here guarantees rot. Instead, read, in
-order, and re-verify against source:
-
-1. **`docs/canonical-api.md`** — the source of truth: the §2 decision table
-   ("I want to X → use Y → NOT Z"), §3 per-subsystem signatures (each cited
-   `file:line`), §4 the end-to-end recipe, §5 the recursive atom, §6 the
-   two-substrate map. Every signature there was read from source.
-2. **`grep` the export barrel** — `grep -nE 'export (function|const|type)' src/runtime/index.ts`
-   (and `src/agent/index.ts`, `src/improvement/index.ts`, `src/mcp/index.ts`,
-   `src/intelligence/index.ts`) for the live names + subpaths. `./loops` is the
-   runtime barrel (`package.json` maps it to `src/runtime/index.ts`).
-3. **`bench/HARNESS.md`** — the experiment-harness map: commands, the
-   `rollout → corpus → selector → CI → gate` flow, and the `ADAPTERS` registry
-   (a harness-local export, `bench/src/adapters.ts`, not a package export).
-
-**Code wins.** If a name, subpath, or signature here or in `docs/canonical-api.md`
-disagrees with source, the **source is right** — fix the map in the *same turn*
-(the anti-rediscovery law). Verify with Read/Edit, don't re-read to confirm.
-
-## Decision table — by altitude (each row → ONE source, not a snippet)
-
-Read the cited `docs/canonical-api.md` row before writing; it carries the live
-signature + the exact "do NOT build".
-
-**The §1.5 law, inline:** an agent IS its full authored `AgentProfile`
-(prompt+skills+tools+mcp+subagents+hooks); you change behavior by AUTHORING the
-profile and letting the substrate materialize it into harness shapes —
-self-verification, iteration, and audit are profile levers (hooks/skills/
-subagents), never glue code.
-
-`AgentProfile` is owned by `@tangle-network/agent-interface` (the `/loops`
-barrel re-exports the sandbox alias as a one-stop import), which also owns
-`HarnessType` + `ReasoningEffort` and a capability layer
-(`harnessSupportsModel` / `reasoningEffortsFor`) — so harness/model/reasoning
-compatibility is a queryable contract, not an assumption.
-
-Harness is a RUN-layer coordinate, not part of the portable genome: it rides on
-agent-runtime's `AgentSpec { profile, harness }`. To sweep it as an eval axis,
-don't hand-declare a harness list — expand one base profile across
-`CODING_HARNESSES` with `expandProfileAxes` (agent-eval), run with
-`runProfileMatrix`, and pivot results by the stamped `AgentProfileCell`
-(`groupRunsByAgentProfileCell`); `harnessSupportsModel` filters per harness,
-and a vendor-locked harness that supports none of the requested models SNAPS
-to its native default (`HARNESS_NATIVE_MODEL`) — never silently dropped.
-
-| Altitude — I want to… | Use | Source |
-|---|---|---|
-| **Define a genome** (who the agent is + what it can do, ONE surface) | `AgentProfile` (runnable) / `AgentSurfaces` (the editable-coordinate map) — `/loops`, `/agent` | canonical-api §3.2 |
-| **Define the personified-run record** (model+prompt+tools+role+seams) | `definePersona(input)` — `/loops` | canonical-api §3.1 |
-| **Run a genome driver⟷worker, end-to-end** | `runPersonified({ persona, shape, task, budget })` — `/loops` | canonical-api §3.1 |
-| **Loop a worker over one evolving artifact, K rounds, stop-when-good** | `loopUntil(seed, spec)` as the `shape` — `/loops` | canonical-api §3.1 |
-| **Best-of-N / parallel-research at equal compute** | `fanout(items, opts)` — `/loops` | canonical-api §3.1 |
-| **Produce-then-gate / multi-judge quorum / fixed chain** | `verify` / `panel` / `pipeline` — `/loops` | canonical-api §3.1 |
-| **Run depth-vs-breadth (or a custom strategy) over a stateful tool domain** | `runAgentic({ surface, task, mode\|strategy, budget })` — `/loops` | canonical-api §3.3 |
-| **Author a new topology/strategy compactly** | `defineStrategy(name, body)` w/ `ctx.shot()`+`ctx.critique()` — `/loops` | canonical-api §3.3 |
-| **Add a stateful tool-using domain** | implement `AgenticSurface` (5 hooks) — `/loops` | canonical-api §3.3 |
-| **Drive a team of agents over a graded `AgenticSurface` task** (workers settle on its check, driver self-improves from the failing tests) | `superviseSurface(profile, task, { surface, worker })` — `/loops` | canonical-api §2 |
-| **Benchmark: compare strategies + significance + Pareto on a domain** | `runBenchmark({ environment, tasks, worker, strategies })` — `/loops` | canonical-api §3.3 |
-| **Author a PRODUCT eval leaderboard** (cases + prompt + grader → ranked board, standard flags, fresh run-dir, export, `toBenchmarkAdapter()`) | `defineLeaderboard({ name, cases, prompt, score, axis?, backends?, flags?, setup?/teardown?, onCellEvents?, resolveModel?, export?, dispatch?, judges?, matrix? })` — `/loops` — NOT a hand-assembled flag-parsing + `runProfileMatrix` frame; `runProfileMatrix` is the escape floor, the level-2 `dispatch` override is how in-process products plug in | `src/runtime/define-leaderboard.ts` (verify vs source) |
-| **Resolve a harness-in-box backend** (box / local cli-bridge / router leaf, one `SandboxClient` shape) | `resolveSandboxClient({ backend: 'sandbox' \| 'bridge' \| 'router' })` — `/loops` — NOT a per-product backend factory or a hand-faked box (`inlineSandboxClient` / the bridge executor already exist) | `src/runtime/resolve-sandbox-client.ts` |
-| **Resolve an in-process chat backend** (the one `--backend` branch for `runChatThroughRuntime` / `runAgentTaskStream`) | `resolveAgentBackend({ kind: 'router' \| 'tcloud' \| 'cli-bridge' \| 'sandbox' })` — root `.` | `src/resolve-agent-backend.ts` |
-| **Run ONE agent turn as one normalized event stream** (box, executor, or chat backend; guaranteed terminal result+usage) | `streamAgentTurn(backend, prompt, { signal, timeoutMs })` + `collectAgentTurn(stream)` — `/loops` | canonical-api §2 |
-| **Benchmark report: multi-profile × multi-axis leaderboard** (ranked board + score matrix + SVG/HTML charts, any `RunRecord[]`) | `leaderboard(records)` + `renderLeaderboardMarkdown` / `renderLeaderboardSvg` / `renderLeaderboardHtml` — `/loops` | canonical-api §2 |
-| **Meter one `openSandboxRun` cell's token/cost usage** | `sumSandboxUsage(events)` — `/loops` | canonical-api §2 |
-| **Sweep harness × model as an eval axis** (turn one base profile into the full harness × model set) | `expandProfileAxes({ base, harnesses, models })` over `CODING_HARNESSES` → `runProfileMatrix(...)`, pivot with `groupRunsByAgentProfileCell` — `agent-eval` root — NOT a hand-declared `HARNESSES` list | agent-eval root (verify vs source) |
-| **Benchmark: add/run an external benchmark from the harness** | `ADAPTERS`/`resolveAdapter(key)` + a bench gate (`*-gate.mts`) over `openSandboxRun` + `sandboxAgentRun` (`bench/src/sandbox-run.ts`) | canonical-api §3.3 |
-| **Spawn N coding agents on isolated git worktrees, keep the one whose patch passes checks** | `worktreeFanout` + `createWorktreeCliExecutor` + `gateOnDeliverable(DeliverableSpec)` over a raw `WorktreePatchArtifact`, winner via `selectValidWinner` — `/loops` — NOT a hand-rolled spawn-loop / "coder" role | canonical-api §3.1 / §5 |
-| **Sandbox coding rollout** (fresh box/round, or persistent+resume) | `runLoop(options)` / `openSandboxRun(client, opts, deliverable)` — `/loops` | canonical-api §3.1 |
-| **Optimize a CODE surface** in a gated loop | `improvementDriver({ worktree, generator })` — root `.` | canonical-api §3.4 |
-| **Freeze a measured profile/diff + code surface for execution** | `buildAgentCandidateBundle(...)` then `verifyAgentCandidateBundle(...)` — `/candidate-execution` | canonical-api §2 |
-| **Optimize any agent/code surface** (one call) — START HERE | `improve(profile, findings, { surface, gate })` — root `.`; prompt, skill-document, and curated-memory surfaces have built-in proposers, config/whole-profile surfaces accept a proposer, and code gets isolated incumbent/candidate worktrees. Workflow and rollout-policy files use code plus agent-eval's `parameterSweepProposer` for JSON sweeps; drop to `selfImprove({ agent, scenarios, judge, baselineSurface })` only for lower-level control | canonical-api §3.4 |
-| **Gate: ship/hold a candidate** (campaign ctx) | `defaultProductionGate` / `heldOutGate` / `composeGate` — `agent-eval/contract`; `neutralizationGate` (footprint-matched PLACEBO gate — proves a held-out lift is CONTENT, not added prompt/mount footprint) — `agent-eval/campaign` | canonical-api §3.4 |
-| **Gate: ship/hold from a `BenchmarkReport`** (per-task cells) | `promotionGate({ report, incumbent, candidate })` — `/loops` | canonical-api §3.4 |
-| **Run the full multi-generation flywheel + certify** | `runStrategyEvolution(config)` — `/loops` | canonical-api §3.4 |
-| **Observe a run** (cost/time waterfall, OTLP) | `createWaterfallCollector()` — `/loops`; `createOtelExporter` attached via `composeRuntimeHooks(...)` — root `.` | canonical-api §2 |
-| **State any A/B claim** | `pairedLift` (bench) over `pairedBootstrap`/`heldoutSignificance` (substrate) | canonical-api §3.5 |
-| **Observe + deliver on a live agent with billing-boundary** | `withIntelligence(agent, { project, target, effort })` — `/intelligence` | canonical-api §2 |
-| **Pull the certified profile from the Intelligence plane** (pull-by-default delivery: fold the gate-certified prompt onto the base surface) | `pullCertified` / `withIntelligence` / `composeCertifiedPrompt` — `/intelligence` | `src/intelligence/delivery.ts` |
-
-## Do-NOT-reinvent — the traps this skill exists to stop
-
-Each of these gets hand-rolled every session; the canonical primitive already
-holds the load-bearing invariant the parallel breaks:
-
-- `runConversation` / persona-runner / `while(!done)` steering loop **≈**
-  `loopUntil` + `runPersonified` (threads executor seams; equal-k; selector≠judge
-  firewall; journal/replay — a parallel runner silently fails to wire the seams).
-- "skill optimizer" / "topology mutator" that opens branches + applies patches
-  **≈** `improvementDriver` (code surface) or `selfImprove`/`gepaProposer` (prompt
-  surface) — both gated on a frozen holdout.
-- "profile-seam" / agent-config wrapper carrying model+prompt+tools+role **≈**
-  `AgentProfile` (it IS that bundle) + `definePersona` (the run record);
-  `sandboxAgentRun({ profile })` is the box seam — never pass a router key into
-  the box.
-- "drive N coding agents in worktrees + pick the passing patch" / a "coder" role
-  **≈** `worktreeFanout` + `gateOnDeliverable` (each on its own worktree-CLI leaf,
-  settled ⟺ delivered, winner via `selectValidWinner` — a hand-rolled spawn-loop
-  skips the deliverable gate and the valid-only selection).
-- `new Sandbox()` + acquire + stream + `box.fs.read` + delete **≈**
-  `openSandboxRun` (persistent + resume) or `runLoop` (fresh box/round).
-- `Promise.all` over N calls + manual argmax/merge **≈** `fanout` (bypassing the
-  budget pool breaks equal-compute claims).
-- a per-step cost/token tally over events **≈** `createWaterfallCollector` (the
-  sum of spans IS the billed run cost; a parallel tally drifts).
-- your own bootstrap loop / PRNG per gate **≈** `pairedLift` / `promotionGate`
-  (seeded, identical run-to-run; never report a point lift without `low/high/pairs`).
-- a per-product `HARNESSES` / `HarnessBackend` list + a metadata-harness reader
-  **≈** `CODING_HARNESSES` + `expandProfileAxes` (the one canonical harness list;
-  vendor-locked harnesses SNAP to their native model via `HARNESS_NATIVE_MODEL`,
-  never dropped) and the
-  `AgentProfileCell` stamped by `runProfileMatrix`, pivoted via
-  `groupRunsByAgentProfileCell` — never bake the harness into the model id so the
-  same model can run under multiple harnesses.
-- a per-product leaderboard CLI (flag parsing + run-dir management + axis
-  expansion + a `runProfileMatrix` call + export/markdown) **≈**
-  `defineLeaderboard` (0.84+; it owns that whole frame — FRESH default run-dir,
-  standard `--backend`/`--harnesses`/`--models`/`--cases`/`--shots`/`--reps`
-  flags, `toBenchmarkAdapter()`; the product writes ~150-250 domain lines).
-- a backend factory / `if (backend === 'router') ... else ...` branch or a
-  hand-faked box around a non-box executor **≈** `resolveSandboxClient`
-  (harness-in-box: `'sandbox' | 'bridge' | 'router'`) or `resolveAgentBackend`
-  (in-process: `'router' | 'tcloud' | 'cli-bridge' | 'sandbox'`) — grep the
-  substrate first; `inlineSandboxClient` and the bridge executor exist.
-- a per-provider stream→event mapper for a single agent turn **≈**
-  `streamAgentTurn` + `collectAgentTurn` (0.85+; one `RuntimeStreamEvent`
-  contract over box / executor / chat, guaranteed terminal result+usage).
-
-## End-to-end recipe
-
-`docs/canonical-api.md` §4 is the real composition — copy it from there, don't
-re-derive: **define a genome → run driver⟷worker via the reactive substrate over
-a multi-turn `AgenticSurface` → measure with `runBenchmark` → optimize a prompt
-surface with `selfImprove` → certify on a frozen holdout with the gate.** For the
-multi-generation flywheel, replace the measure/certify steps with one
-`runStrategyEvolution(...)` and read `report.verdict` (NOT `report.trajectory`)
-as the evidence. For a sandbox coding rollout judged by an external deterministic
-checker, use the bench-gate path: `resolveAdapter(...)` to pick the benchmark,
-then `openSandboxRun(client, { agentRun: sandboxAgentRun({ profile }), ... },
-deliverable)` per task, A/B-ing a blind arm against an `llmAnalyst`-steered arm
-at equal compute (both helpers live in `bench/src/sandbox-run.ts`; the blind arm
-is the mandatory equal-compute control). See `bench/src/commit0-gate.mts` /
-`gate.ts` for the live shape.
-
-## Two substrates — pick one, don't invent a third
-
-Both implement the same recursive-decision atom over the one `Executor` port and
-share `defaultSelectWinner`. **Reactive** (`Supervisor`/`Scope` + personify
-combinators: `runPersonified`/`runAgentic`/`runBenchmark`) — prefer for NEW
-recursive work; equal-k by construction. **Round-synchronous** (`runLoop` driven
-by a caller-supplied `Driver`, plus the bench gates over `openSandboxRun`) —
-sandbox coding rollouts against external benchmarks. The full when-which map is
-`docs/canonical-api.md` §6.
-
-## Observe / ship with the Intelligence SDK
-
-One hook wraps any agent with send + receive + billing boundary:
-`withIntelligence(agent, { project, target, effort })`, `effort ∈
-off|eco|standard|thorough|max` (`'off'` is the provable passthrough floor —
-intelligence spend clamped to 0). It SENDs a typed `RunRecord` and RECEIVEs the
-certified profile, building on `createOtelExporter` + `buildLoopOtelSpans` —
-don't hand-roll a trace-wrapper, a second receive path, or effort/tier config.
-Verify the live subpath against `src/intelligence/with-intelligence.ts`.
-
-Two operational facts every consumer must know:
-
-- **Send is a silent no-op without a tenant apiKey.** The export leg only ships
-  when an `apiKey` (or `TANGLE_API_KEY`) is present — the ingest requires the
-  tenant Bearer. One `baseUrl` (`TANGLE_INTELLIGENCE_URL`, default
-  `https://intelligence.tangle.tools`) drives both the OTLP send (`/v1/otlp`)
-  and the receive pull. The client's `doctor().exportConfigured` is the check
-  that a send will actually land.
-- **Delivery pulls the certified profile from the plane.** `pullCertified` /
-  `withIntelligence` hit
-  `GET {baseUrl}/v1/profiles/:target/composed` with `Bearer TANGLE_API_KEY`;
-  the pull deserializes the typed `agentProfileDiffs` (surfaced as PROPOSALS,
-  never auto-applied) alongside the certified prompt. `withIntelligence` folds
-  the certified prompt onto the base surface, refreshes at most every 5 minutes,
-  and is fail-closed — a failed pull runs the agent on its base surface.
-
-## Final check
-
-- Picked a primitive from the decision table, not a hand-rolled parallel?
-- Genome is ONE `AgentProfile`/`AgentSurfaces` surface, not split skill/tool/prompt knobs?
-- Equal compute preserved (budget pool, or `arms[0]` control) — no `Promise.all` zoo?
-- Selector ≠ judge: no judge score feeding a driver/another judge; holdout score write-only?
-- Any win certified on a FROZEN holdout via a gate, never on the training composite?
-- Map fixed in the same turn if source disagreed with `docs/canonical-api.md`?
-
-See `_common.md` for shared conventions (frontmatter, fail-loud, no AI attribution).
-
-Next: build the genome/loop/optimizer against `docs/canonical-api.md` §3–§4; if a
-strategy beats incumbent on the holdout gate, `/ship` it.
+# Build with agent-runtime
+
+Use this skill before writing product-local agent infrastructure.
+The goal is one portable agent definition, one execution path, one measurement system, and one reviewed activation path.
+
+## Read first
+
+1. Read `docs/canonical-api.md` for the current decision table.
+2. Check exports in `src/index.ts`, `src/runtime/index.ts`, `src/improvement/index.ts`, `src/intelligence/index.ts`, and `src/knowledge/index.ts`.
+3. Read the nearest runnable example.
+4. Treat source as authoritative when docs disagree, then correct the stale doc in the same change.
+
+## Ownership
+
+| Concern | Owner |
+|---|---|
+| Portable prompt, skills, tools, MCP, hooks, subagents, model hints | `AgentProfile` from `@tangle-network/agent-interface` |
+| Agent execution, supervision, budgets, streaming, candidate execution | `@tangle-network/agent-runtime` |
+| Tasks, graders, search, paired statistics, cost and latency comparison | `@tangle-network/agent-eval` |
+| Sources, retrieval, citations, freshness, memory adapters, knowledge promotion | `@tangle-network/agent-knowledge` |
+| Product records, permissions, funding, UI, and atomic storage writes | The consuming product |
+
+Do not move shared measurement into Runtime or product code.
+Do not move product storage transactions into a provider-neutral package.
+
+## Choose the entry point
+
+| Need | Use |
+|---|---|
+| One product chat turn | `handleChatTurn(...)` |
+| One normalized streamed agent turn | `streamAgentTurn(...)` and `collectAgentTurn(...)` |
+| One task or multi-turn loop | `runAgentTask(...)`, `runAgentTaskStream(...)`, or `runLoop(...)` |
+| Supervisor and workers | `supervise(...)` or `superviseSurface(...)` |
+| Parallel work with a shared budget | `fanout(...)` |
+| Fixed composition | `pipeline(...)`, `panel(...)`, or `verify(...)` |
+| Product benchmark | `defineLeaderboard(...)` |
+| Profile matrix | `expandProfileAxes(...)` and `runProfileMatrix(...)` from agent-eval |
+| Search one agent surface | `improve(...)` |
+| Analyze traces through a measured proposal | `proposeAgentImprovement(...)` |
+| Review and authorize an exact proposal | `reviewAgentImprovementProposal(...)` and `createAgentImprovementActivation(...)` |
+| Apply or restore an approved candidate | `executeAgentImprovementActivation(...)` with a product transaction |
+| Build a knowledge candidate | `runKnowledgeImprovementJob(...)` |
+| Apply a knowledge candidate | `createKnowledgeImprovementActivationExecutor(...)` through the same activation path |
+| Observe and pull approved changes on a live agent | `withIntelligence(...)` |
+
+## Improvement flow
+
+`improve(profile, findings, options)` searches one surface and returns a detached winner.
+It never changes a profile, document, repository, memory store, or knowledge base.
+
+Supported profile surfaces are prompt, one named inline skill, tools, MCP, hooks, subagents, whole profile, and curated memory in `profile.resources.instructions`.
+Code uses isolated worktrees and returns a sealed patch candidate.
+Knowledge uses `runKnowledgeImprovementJob(...)` and returns paired snapshots.
+
+Use `proposeAgentImprovement(...)` for a production proposal.
+It performs these steps in order:
+
+1. Analyze completed traces.
+2. Search for a candidate on development tasks.
+3. Freeze the baseline and winner into one exact experiment.
+4. Reject the experiment if its candidate differs from the search winner.
+5. Run baseline and candidate on the same held-back tasks.
+6. Produce findings, confidence intervals, quality, cost, latency, and a decision.
+
+After a person or tenant policy approves the proposal, call `createAgentImprovementActivation(...)` with target identities, funding owner, authority, intent, and expiry.
+Runtime derives the expected current digests from the measured experiment.
+Call `executeAgentImprovementActivation(...)` with one product-owned transaction that compares current state, writes every target atomically, and stores the result under the activation digest.
+Pass a read-only reconciliation function so retries can distinguish committed, uncommitted, and uncertain outcomes.
+
+Never apply a change from analyst confidence alone.
+Never measure one candidate and apply another.
+Never let search code write live state.
+Never treat a lost response as a failed write without reconciling it.
+
+## Surface rules
+
+- Prompt changes `profile.prompt` only.
+- Skill optimization selects one inline skill by `skills.resourceName`; profile resources must fail closed.
+- Curated memory changes `profile.resources.instructions`; retrieval stores and memory databases belong in the knowledge flow.
+- Tools, MCP, hooks, subagents, and whole-profile changes require an explicit proposer because Runtime cannot invent domain capabilities safely.
+- Code candidates must come from the Runtime worktree path so patch identity and cleanup stay intact.
+- Workflow and policy files are code surfaces; parameter sweeps come from agent-eval.
+- Knowledge candidates remain detached until the shared activation path applies or restores their frozen snapshots.
+
+## Product integration
+
+The product supplies only the pieces that vary by deployment:
+
+- How traces and current profiles are loaded.
+- How exact candidate execution is placed on compute.
+- How proposal, review, activation, and result records are persisted.
+- How a target is changed atomically.
+- Who may approve, reject, request changes, fund, apply, or restore.
+- How those records and actions appear in the UI or API.
+
+The product must not recreate candidate hashing, paired comparison, confidence intervals, review binding, expiry, retry identity, or result validation.
+
+## Do not duplicate
+
+- Do not write a provider-specific profile wrapper; extend `AgentProfile` and its materializer.
+- Do not write a second optimizer loop; compose Eval proposers through `improve(...)`.
+- Do not write a second candidate catalog; persist the immutable proposal records.
+- Do not let an analyst or adapter commit, push, open a pull request, or edit a live store.
+- Do not hand-roll SSE parsing, usage totals, profile matrices, bootstrap statistics, sandbox acquisition, or worktree cleanup.
+- Do not add a product-local approval format for knowledge, code, or profile changes.
+
+## Finish
+
+- The same agent definition runs in product and measurement paths.
+- The held-back tasks were not visible during search.
+- Candidate identity is checked before execution and again before activation.
+- Quality, cost, latency, sample count, and uncertainty are retained.
+- Rejection and request-changes are first-class outcomes.
+- Activation is authorized, expiring, idempotent, and reconcilable.
+- No customer write, message, trigger, or billing occurs in read-only proof mode.
+- Public examples, package exports, generated API docs, type checks, tests, build, and package verification pass.
+
+## Then consider
+
+- Use `critical-audit` when the change introduces or alters a public contract.
+- Use `verify` before publishing or adopting the package in a product.

package/dist/chunk-6O73TRHW.js

@@ -1,142 +0,0 @@
-// src/improvement/build-prompts.ts
-function findingLines(findings) {
-  return findings.map((f) => {
-    const where = f.subject ? ` [${f.subject}]` : "";
-    const action = f.recommended_action ? ` \u2192 ${f.recommended_action}` : "";
-    return `- (${f.severity})${where} ${f.claim}${action}`;
-  });
-}
-function toolBuildPrompt(args) {
-  return [
-    "You are building a new TOOL for this codebase to address the gaps below.",
-    "Write the tool as a small, self-contained module PLUS tests that exercise it.",
-    "The tool must compile and its tests must pass \u2014 they will be run automatically;",
-    "if verification fails you will get the error and another attempt. Do not commit;",
-    "leave the changes in the working tree.",
-    "",
-    "Gaps the tool should close:",
-    ...findingLines(args.findings)
-  ].join("\n");
-}
-function mcpBuildPrompt(args) {
-  return [
-    "You are building a new MCP SERVER (Model Context Protocol) that exposes",
-    "tool(s) addressing the gaps below, so any harness can mount it.",
-    "Requirements that WILL be checked by booting the server:",
-    "- it starts over stdio and answers the MCP `initialize` handshake,",
-    "- `tools/list` returns at least one tool with a valid input schema.",
-    "Newline-delimited JSON-RPC 2.0, protocol version 2024-11-05. Include a start",
-    "command (e.g. a package.json `start` script or a clear entrypoint). If the",
-    "boot-and-probe fails you will get the error and another attempt. Do not",
-    "commit; leave the changes in the working tree.",
-    "",
-    "Capabilities the server should provide:",
-    ...findingLines(args.findings)
-  ].join("\n");
-}
-
-// src/improvement/mcp-serve-verifier.ts
-import { spawn } from "child_process";
-import { createInterface } from "readline";
-var PROTOCOL_VERSION = "2024-11-05";
-function mcpServeVerifier(spec) {
-  const timeoutMs = spec.timeoutMs ?? 3e4;
-  const minTools = spec.minTools ?? 1;
-  return (worktreePath) => new Promise((resolve, reject) => {
-    const child = spawn(spec.command, spec.args ?? [], {
-      cwd: worktreePath,
-      stdio: ["pipe", "pipe", "pipe"],
-      env: { ...process.env, ...spec.env }
-    });
-    const stderr = [];
-    let settled = false;
-    let nextId = 1;
-    const initId = nextId++;
-    let listId = -1;
-    const settle = (fn) => {
-      if (settled) return;
-      settled = true;
-      clearTimeout(timer);
-      rl.close();
-      child.kill("SIGKILL");
-      fn();
-    };
-    const withStderr = (msg) => stderr.length > 0 ? `${msg}
-stderr:
-${stderr.join("").slice(-2e3)}` : msg;
-    const pass = () => settle(() => resolve({ ok: true }));
-    const failCandidate = (msg) => settle(() => resolve({ ok: false, feedback: withStderr(msg) }));
-    const setupFault = (err) => settle(() => reject(err));
-    const failStdin = (err) => failCandidate(`writing to MCP server stdin failed: ${err.message}`);
-    const send = (msg) => {
-      try {
-        child.stdin.write(`${JSON.stringify(msg)}
-`);
-        return true;
-      } catch (err) {
-        failStdin(err);
-        return false;
-      }
-    };
-    child.on("error", (err) => {
-      const code = err.code;
-      setupFault(
-        code === "ENOENT" ? new Error(
-          `mcpServeVerifier: '${spec.command}' not found in PATH (setup bug, not a failed candidate)`
-        ) : new Error(`mcpServeVerifier: '${spec.command}' failed to spawn: ${err.message}`)
-      );
-    });
-    child.on("exit", (code, signal) => {
-      failCandidate(`MCP server exited (code ${code}, signal ${signal}) before serving`);
-    });
-    child.stdin.on("error", failStdin);
-    child.stderr.on("data", (d) => stderr.push(String(d)));
-    const rl = createInterface({ input: child.stdout });
-    rl.on("line", (line) => {
-      let msg;
-      try {
-        msg = JSON.parse(line);
-      } catch {
-        return;
-      }
-      if (!msg || typeof msg !== "object") return;
-      if (msg.id === initId) {
-        if (msg.error) return failCandidate(`initialize errored: ${JSON.stringify(msg.error)}`);
-        if (!send({ jsonrpc: "2.0", method: "notifications/initialized" })) return;
-        listId = nextId++;
-        send({ jsonrpc: "2.0", id: listId, method: "tools/list" });
-        return;
-      }
-      if (msg.id === listId) {
-        if (msg.error) return failCandidate(`tools/list errored: ${JSON.stringify(msg.error)}`);
-        const tools = msg.result?.tools;
-        if (!Array.isArray(tools)) return failCandidate("tools/list result has no tools array");
-        if (tools.length < minTools) {
-          return failCandidate(`tools/list returned ${tools.length} tool(s), need >= ${minTools}`);
-        }
-        return pass();
-      }
-    });
-    const timer = setTimeout(
-      () => failCandidate(`MCP server did not complete the handshake within ${timeoutMs}ms`),
-      timeoutMs
-    );
-    send({
-      jsonrpc: "2.0",
-      id: initId,
-      method: "initialize",
-      params: {
-        protocolVersion: PROTOCOL_VERSION,
-        capabilities: {},
-        clientInfo: { name: "agent-runtime-mcp-verify", version: "0" }
-      }
-    });
-  });
-}
-
-export {
-  toolBuildPrompt,
-  mcpBuildPrompt,
-  mcpServeVerifier
-};
-//# sourceMappingURL=chunk-6O73TRHW.js.map

package/dist/chunk-6O73TRHW.js.map

@@ -1 +0,0 @@
-{"version":3,"sources":["../src/improvement/build-prompts.ts","../src/improvement/mcp-serve-verifier.ts"],"sourcesContent":["/**\n * Build-prompt starting points for the two buildable artifact types. There is\n * NO `toolGenerator`/`mcpGenerator` wrapper — the factory is `agenticGenerator`\n * + a verifier (docs/artifact-lifecycle-frontier.md), so a tool or an MCP\n * server is built by composing the pieces directly:\n *\n *   // a tool:\n *   agenticGenerator({ buildPrompt: toolBuildPrompt, verify: commandVerifier('pnpm', ['test']) })\n *   // an MCP server:\n *   agenticGenerator({ buildPrompt: mcpBuildPrompt, verify: mcpServeVerifier({ command: 'node', args: ['server.mjs'] }) })\n *\n * These are the only type-specific bit (the phrasing that points the agent at a\n * tool vs. an MCP); the worktree, resume-on-failure loop, and improvement-loop\n * wrapper are shared. MCP is the load-bearing target — it is how a harness\n * acquires tools; raw tools matter where we control the loader.\n */\n\nimport type { AnalystFinding } from '@tangle-network/agent-eval'\n\ntype FindingsArg = { report: unknown; findings: AnalystFinding[] }\n\nfunction findingLines(findings: AnalystFinding[]): string[] {\n  return findings.map((f) => {\n    const where = f.subject ? ` [${f.subject}]` : ''\n    const action = f.recommended_action ? ` → ${f.recommended_action}` : ''\n    return `- (${f.severity})${where} ${f.claim}${action}`\n  })\n}\n\n/** Build the starting instruction for a coder agent tasked with implementing a new tool. */\nexport function toolBuildPrompt(args: FindingsArg): string {\n  return [\n    'You are building a new TOOL for this codebase to address the gaps below.',\n    'Write the tool as a small, self-contained module PLUS tests that exercise it.',\n    'The tool must compile and its tests must pass — they will be run automatically;',\n    'if verification fails you will get the error and another attempt. Do not commit;',\n    'leave the changes in the working tree.',\n    '',\n    'Gaps the tool should close:',\n    ...findingLines(args.findings),\n  ].join('\\n')\n}\n\n/** Build the starting instruction for a coder agent tasked with implementing a new MCP server. */\nexport function mcpBuildPrompt(args: FindingsArg): string {\n  return [\n    'You are building a new MCP SERVER (Model Context Protocol) that exposes',\n    'tool(s) addressing the gaps below, so any harness can mount it.',\n    'Requirements that WILL be checked by booting the server:',\n    '- it starts over stdio and answers the MCP `initialize` handshake,',\n    '- `tools/list` returns at least one tool with a valid input schema.',\n    'Newline-delimited JSON-RPC 2.0, protocol version 2024-11-05. Include a start',\n    'command (e.g. a package.json `start` script or a clear entrypoint). If the',\n    'boot-and-probe fails you will get the error and another attempt. Do not',\n    'commit; leave the changes in the working tree.',\n    '',\n    'Capabilities the server should provide:',\n    ...findingLines(args.findings),\n  ].join('\\n')\n}\n","/**\n * `mcpServeVerifier` — the intrinsic verifier for a built MCP server: the\n * boot-and-probe checker named in docs/artifact-lifecycle-frontier.md. A\n * generated MCP server is only a candidate if it actually *serves* — so this\n * boots it over stdio (the default local MCP transport) and runs the real\n * handshake: `initialize` → `notifications/initialized` → `tools/list`, and\n * asserts the server answers with at least `minTools` tools.\n *\n * Outcomes follow the `Verifier` contract: a server that fails to start, exits\n * early, errors the handshake, times out, or exposes no tools is a FAILED\n * candidate (`{ok:false}`, fed back into the next generation shot); a missing\n * start binary or spawn fault THROWS (a setup bug, never a silent fallback).\n *\n * Protocol matches the runtime's own stdio MCP server (src/mcp/server.ts):\n * newline-delimited JSON-RPC 2.0, protocol version 2024-11-05.\n */\n\nimport { spawn } from 'node:child_process'\nimport { createInterface } from 'node:readline'\nimport type { Verifier, VerifyResult } from './agentic-generator'\n\nconst PROTOCOL_VERSION = '2024-11-05'\n\nexport interface McpServeSpec {\n  /** Command that starts the built MCP server in the worktree (stdio transport). */\n  command: string\n  args?: string[]\n  /** Extra env for the server process (merged over `process.env`). */\n  env?: Record<string, string>\n  /** Handshake timeout (ms). Default 30s. */\n  timeoutMs?: number\n  /** Minimum tools the server must expose to pass. Default 1. */\n  minTools?: number\n}\n\ninterface JsonRpcResponse {\n  jsonrpc?: string\n  id?: number | string | null\n  result?: unknown\n  error?: { code: number; message: string }\n}\n\n/** Build a `Verifier` that boots a generated MCP server over stdio and checks it exposes tools. */\nexport function mcpServeVerifier(spec: McpServeSpec): Verifier {\n  const timeoutMs = spec.timeoutMs ?? 30_000\n  const minTools = spec.minTools ?? 1\n\n  return (worktreePath: string): Promise<VerifyResult> =>\n    new Promise<VerifyResult>((resolve, reject) => {\n      const child = spawn(spec.command, spec.args ?? [], {\n        cwd: worktreePath,\n        stdio: ['pipe', 'pipe', 'pipe'],\n        env: { ...process.env, ...spec.env },\n      })\n\n      const stderr: string[] = []\n      let settled = false\n      let nextId = 1\n      const initId = nextId++\n      let listId = -1\n\n      const settle = (fn: () => void) => {\n        if (settled) return\n        settled = true\n        clearTimeout(timer)\n        rl.close()\n        child.kill('SIGKILL')\n        fn()\n      }\n      const withStderr = (msg: string) =>\n        stderr.length > 0 ? `${msg}\\nstderr:\\n${stderr.join('').slice(-2000)}` : msg\n      const pass = () => settle(() => resolve({ ok: true }))\n      const failCandidate = (msg: string) =>\n        settle(() => resolve({ ok: false, feedback: withStderr(msg) }))\n      const setupFault = (err: Error) => settle(() => reject(err))\n      const failStdin = (err: Error) =>\n        failCandidate(`writing to MCP server stdin failed: ${err.message}`)\n\n      const send = (msg: Record<string, unknown>): boolean => {\n        try {\n          child.stdin.write(`${JSON.stringify(msg)}\\n`)\n          return true\n        } catch (err) {\n          // EPIPE: the server died mid-handshake — a failed candidate, not a fault.\n          failStdin(err as Error)\n          return false\n        }\n      }\n\n      child.on('error', (err) => {\n        const code = (err as NodeJS.ErrnoException).code\n        setupFault(\n          code === 'ENOENT'\n            ? new Error(\n                `mcpServeVerifier: '${spec.command}' not found in PATH (setup bug, not a failed candidate)`,\n              )\n            : new Error(`mcpServeVerifier: '${spec.command}' failed to spawn: ${err.message}`),\n        )\n      })\n      child.on('exit', (code, signal) => {\n        // An exit before the handshake completes is a failed candidate (the\n        // server crashed on boot); after we settle, our own SIGKILL fires here.\n        failCandidate(`MCP server exited (code ${code}, signal ${signal}) before serving`)\n      })\n      // Pipe failures normally arrive asynchronously, outside send()'s try/catch.\n      child.stdin.on('error', failStdin)\n      child.stderr.on('data', (d) => stderr.push(String(d)))\n\n      const rl = createInterface({ input: child.stdout })\n      rl.on('line', (line) => {\n        let msg: JsonRpcResponse | undefined\n        try {\n          msg = JSON.parse(line) as JsonRpcResponse\n        } catch {\n          return // servers log to stdout too; skip non-JSON lines\n        }\n        if (!msg || typeof msg !== 'object') return\n\n        if (msg.id === initId) {\n          if (msg.error) return failCandidate(`initialize errored: ${JSON.stringify(msg.error)}`)\n          if (!send({ jsonrpc: '2.0', method: 'notifications/initialized' })) return\n          listId = nextId++\n          send({ jsonrpc: '2.0', id: listId, method: 'tools/list' })\n          return\n        }\n        if (msg.id === listId) {\n          if (msg.error) return failCandidate(`tools/list errored: ${JSON.stringify(msg.error)}`)\n          const tools = (msg.result as { tools?: unknown[] } | undefined)?.tools\n          if (!Array.isArray(tools)) return failCandidate('tools/list result has no tools array')\n          if (tools.length < minTools) {\n            return failCandidate(`tools/list returned ${tools.length} tool(s), need >= ${minTools}`)\n          }\n          return pass()\n        }\n      })\n\n      const timer = setTimeout(\n        () => failCandidate(`MCP server did not complete the handshake within ${timeoutMs}ms`),\n        timeoutMs,\n      )\n\n      send({\n        jsonrpc: '2.0',\n        id: initId,\n        method: 'initialize',\n        params: {\n          protocolVersion: PROTOCOL_VERSION,\n          capabilities: {},\n          clientInfo: { name: 'agent-runtime-mcp-verify', version: '0' },\n        },\n      })\n    })\n}\n"],"mappings":";AAqBA,SAAS,aAAa,UAAsC;AAC1D,SAAO,SAAS,IAAI,CAAC,MAAM;AACzB,UAAM,QAAQ,EAAE,UAAU,KAAK,EAAE,OAAO,MAAM;AAC9C,UAAM,SAAS,EAAE,qBAAqB,WAAM,EAAE,kBAAkB,KAAK;AACrE,WAAO,MAAM,EAAE,QAAQ,IAAI,KAAK,IAAI,EAAE,KAAK,GAAG,MAAM;AAAA,EACtD,CAAC;AACH;AAGO,SAAS,gBAAgB,MAA2B;AACzD,SAAO;AAAA,IACL;AAAA,IACA;AAAA,IACA;AAAA,IACA;AAAA,IACA;AAAA,IACA;AAAA,IACA;AAAA,IACA,GAAG,aAAa,KAAK,QAAQ;AAAA,EAC/B,EAAE,KAAK,IAAI;AACb;AAGO,SAAS,eAAe,MAA2B;AACxD,SAAO;AAAA,IACL;AAAA,IACA;AAAA,IACA;AAAA,IACA;AAAA,IACA;AAAA,IACA;AAAA,IACA;AAAA,IACA;AAAA,IACA;AAAA,IACA;AAAA,IACA;AAAA,IACA,GAAG,aAAa,KAAK,QAAQ;AAAA,EAC/B,EAAE,KAAK,IAAI;AACb;;;AC1CA,SAAS,aAAa;AACtB,SAAS,uBAAuB;AAGhC,IAAM,mBAAmB;AAsBlB,SAAS,iBAAiB,MAA8B;AAC7D,QAAM,YAAY,KAAK,aAAa;AACpC,QAAM,WAAW,KAAK,YAAY;AAElC,SAAO,CAAC,iBACN,IAAI,QAAsB,CAAC,SAAS,WAAW;AAC7C,UAAM,QAAQ,MAAM,KAAK,SAAS,KAAK,QAAQ,CAAC,GAAG;AAAA,MACjD,KAAK;AAAA,MACL,OAAO,CAAC,QAAQ,QAAQ,MAAM;AAAA,MAC9B,KAAK,EAAE,GAAG,QAAQ,KAAK,GAAG,KAAK,IAAI;AAAA,IACrC,CAAC;AAED,UAAM,SAAmB,CAAC;AAC1B,QAAI,UAAU;AACd,QAAI,SAAS;AACb,UAAM,SAAS;AACf,QAAI,SAAS;AAEb,UAAM,SAAS,CAAC,OAAmB;AACjC,UAAI,QAAS;AACb,gBAAU;AACV,mBAAa,KAAK;AAClB,SAAG,MAAM;AACT,YAAM,KAAK,SAAS;AACpB,SAAG;AAAA,IACL;AACA,UAAM,aAAa,CAAC,QAClB,OAAO,SAAS,IAAI,GAAG,GAAG;AAAA;AAAA,EAAc,OAAO,KAAK,EAAE,EAAE,MAAM,IAAK,CAAC,KAAK;AAC3E,UAAM,OAAO,MAAM,OAAO,MAAM,QAAQ,EAAE,IAAI,KAAK,CAAC,CAAC;AACrD,UAAM,gBAAgB,CAAC,QACrB,OAAO,MAAM,QAAQ,EAAE,IAAI,OAAO,UAAU,WAAW,GAAG,EAAE,CAAC,CAAC;AAChE,UAAM,aAAa,CAAC,QAAe,OAAO,MAAM,OAAO,GAAG,CAAC;AAC3D,UAAM,YAAY,CAAC,QACjB,cAAc,uCAAuC,IAAI,OAAO,EAAE;AAEpE,UAAM,OAAO,CAAC,QAA0C;AACtD,UAAI;AACF,cAAM,MAAM,MAAM,GAAG,KAAK,UAAU,GAAG,CAAC;AAAA,CAAI;AAC5C,eAAO;AAAA,MACT,SAAS,KAAK;AAEZ,kBAAU,GAAY;AACtB,eAAO;AAAA,MACT;AAAA,IACF;AAEA,UAAM,GAAG,SAAS,CAAC,QAAQ;AACzB,YAAM,OAAQ,IAA8B;AAC5C;AAAA,QACE,SAAS,WACL,IAAI;AAAA,UACF,sBAAsB,KAAK,OAAO;AAAA,QACpC,IACA,IAAI,MAAM,sBAAsB,KAAK,OAAO,sBAAsB,IAAI,OAAO,EAAE;AAAA,MACrF;AAAA,IACF,CAAC;AACD,UAAM,GAAG,QAAQ,CAAC,MAAM,WAAW;AAGjC,oBAAc,2BAA2B,IAAI,YAAY,MAAM,kBAAkB;AAAA,IACnF,CAAC;AAED,UAAM,MAAM,GAAG,SAAS,SAAS;AACjC,UAAM,OAAO,GAAG,QAAQ,CAAC,MAAM,OAAO,KAAK,OAAO,CAAC,CAAC,CAAC;AAErD,UAAM,KAAK,gBAAgB,EAAE,OAAO,MAAM,OAAO,CAAC;AAClD,OAAG,GAAG,QAAQ,CAAC,SAAS;AACtB,UAAI;AACJ,UAAI;AACF,cAAM,KAAK,MAAM,IAAI;AAAA,MACvB,QAAQ;AACN;AAAA,MACF;AACA,UAAI,CAAC,OAAO,OAAO,QAAQ,SAAU;AAErC,UAAI,IAAI,OAAO,QAAQ;AACrB,YAAI,IAAI,MAAO,QAAO,cAAc,uBAAuB,KAAK,UAAU,IAAI,KAAK,CAAC,EAAE;AACtF,YAAI,CAAC,KAAK,EAAE,SAAS,OAAO,QAAQ,4BAA4B,CAAC,EAAG;AACpE,iBAAS;AACT,aAAK,EAAE,SAAS,OAAO,IAAI,QAAQ,QAAQ,aAAa,CAAC;AACzD;AAAA,MACF;AACA,UAAI,IAAI,OAAO,QAAQ;AACrB,YAAI,IAAI,MAAO,QAAO,cAAc,uBAAuB,KAAK,UAAU,IAAI,KAAK,CAAC,EAAE;AACtF,cAAM,QAAS,IAAI,QAA8C;AACjE,YAAI,CAAC,MAAM,QAAQ,KAAK,EAAG,QAAO,cAAc,sCAAsC;AACtF,YAAI,MAAM,SAAS,UAAU;AAC3B,iBAAO,cAAc,uBAAuB,MAAM,MAAM,qBAAqB,QAAQ,EAAE;AAAA,QACzF;AACA,eAAO,KAAK;AAAA,MACd;AAAA,IACF,CAAC;AAED,UAAM,QAAQ;AAAA,MACZ,MAAM,cAAc,oDAAoD,SAAS,IAAI;AAAA,MACrF;AAAA,IACF;AAEA,SAAK;AAAA,MACH,SAAS;AAAA,MACT,IAAI;AAAA,MACJ,QAAQ;AAAA,MACR,QAAQ;AAAA,QACN,iBAAiB;AAAA,QACjB,cAAc,CAAC;AAAA,QACf,YAAY,EAAE,MAAM,4BAA4B,SAAS,IAAI;AAAA,MAC/D;AAAA,IACF,CAAC;AAAA,EACH,CAAC;AACL;","names":[]}