@tangle-network/agent-runtime 0.194.0 → 0.195.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/dist/{activation-BQFIiyUG.js → activation-DtIbcFOX.js} +3 -3
- package/dist/{activation-BQFIiyUG.js.map → activation-DtIbcFOX.js.map} +1 -1
- package/dist/agent.d.ts +1 -1
- package/dist/agent.js +1 -1
- package/dist/candidate-execution/index.js +4 -4
- package/dist/{candidate-execution-B6CDW-fp.js → candidate-execution-nvqVIMyS.js} +5 -4
- package/dist/{candidate-execution-B6CDW-fp.js.map → candidate-execution-nvqVIMyS.js.map} +1 -1
- package/dist/durable.d.ts +1 -1
- package/dist/graph.d.ts +1 -1
- package/dist/{improve-COGzLCiY.d.ts → improve-7VTwse1c.d.ts} +4 -4
- package/dist/{improvement-cycle-DeS1ZC5B.js → improvement-cycle-bTBySunH.js} +19 -5
- package/dist/{improvement-cycle-DeS1ZC5B.js.map → improvement-cycle-bTBySunH.js.map} +1 -1
- package/dist/{index-CrBgLCIf.d.ts → index-CVD30soi.d.ts} +2 -2
- package/dist/{index-BEPjOPwH.d.ts → index-DNx0nWZH.d.ts} +9 -2
- package/dist/index.d.ts +6 -5
- package/dist/index.js +71 -25
- package/dist/index.js.map +1 -1
- package/dist/intelligence.d.ts +1 -1
- package/dist/intelligence.js +5 -4
- package/dist/intelligence.js.map +1 -1
- package/dist/kernel.d.ts +1 -1
- package/dist/kernel.js +2 -2
- package/dist/{knowledge-B5okOzBQ.js → knowledge-U8WJ5xCI.js} +6 -16
- package/dist/knowledge-U8WJ5xCI.js.map +1 -0
- package/dist/knowledge.d.ts +1 -1
- package/dist/knowledge.js +1 -1
- package/dist/{loop-runner-bin-U1J6x_UF.js → loop-runner-bin-BU2NACsF.js} +2 -2
- package/dist/{loop-runner-bin-U1J6x_UF.js.map → loop-runner-bin-BU2NACsF.js.map} +1 -1
- package/dist/{loop-runner-bin-Clc5IAwu.d.ts → loop-runner-bin-b4585v9Z.d.ts} +2 -2
- package/dist/loop-runner-bin.d.ts +1 -1
- package/dist/loop-runner-bin.js +1 -1
- package/dist/mcp/index.d.ts +1 -1
- package/dist/mcp/index.js +1 -1
- package/dist/{prepare-C29kNAon.js → prepare-DDGp0-rW.js} +4 -54
- package/dist/prepare-DDGp0-rW.js.map +1 -0
- package/dist/profiles.js +454 -1
- package/dist/profiles.js.map +1 -1
- package/dist/{protected-model-port-CKYND416.js → protected-model-port-DxFN8DLS.js} +3 -2
- package/dist/{protected-model-port-CKYND416.js.map → protected-model-port-DxFN8DLS.js.map} +1 -1
- package/dist/{protected-redaction--F3v1oo8.js → protected-redaction-wGo44k2K.js} +54 -3
- package/dist/protected-redaction-wGo44k2K.js.map +1 -0
- package/dist/{runtime-BVrqSu0-.js → runtime-CuNJ0opC.js} +133 -58
- package/dist/runtime-CuNJ0opC.js.map +1 -0
- package/dist/{structural-rollout-BcPXFzVb.js → structural-rollout-DxEQ77H7.js} +22 -9
- package/dist/structural-rollout-DxEQ77H7.js.map +1 -0
- package/dist/testing.d.ts +1 -1
- package/dist/testing.js +11 -10
- package/dist/testing.js.map +1 -1
- package/dist/tui/index.d.ts +1 -1
- package/dist/{workspace-archive-BCezkeIk.js → workspace-archive-Ybomp7AN.js} +3 -2
- package/dist/{workspace-archive-BCezkeIk.js.map → workspace-archive-Ybomp7AN.js.map} +1 -1
- package/package.json +4 -4
- package/skills/agent-graphs/SKILL.md +28 -123
- package/skills/agent-graphs/references/authoring.md +34 -0
- package/skills/agent-graphs/references/measurement.md +18 -0
- package/skills/build-with-agent-runtime/SKILL.md +31 -117
- package/skills/build-with-agent-runtime/references/improvement.md +32 -0
- package/skills/codemode/SKILL.md +22 -34
- package/skills/codemode/references/runtime-execution.md +14 -0
- package/skills/generate-eval/SKILL.md +2 -1
- package/skills/loop-writer/SKILL.md +30 -84
- package/skills/supervise/SKILL.md +43 -95
- package/skills/supervise/references/profile-authoring.md +19 -0
- package/dist/knowledge-B5okOzBQ.js.map +0 -1
- package/dist/prepare-C29kNAon.js.map +0 -1
- package/dist/protected-redaction--F3v1oo8.js.map +0 -1
- package/dist/researcher-Cp4JRbFp.js +0 -457
- package/dist/researcher-Cp4JRbFp.js.map +0 -1
- package/dist/runtime-BVrqSu0-.js.map +0 -1
- package/dist/structural-rollout-BcPXFzVb.js.map +0 -1
- package/skills/agent-graphs/IMPROVE.md +0 -58
|
@@ -1,136 +1,50 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: build-with-agent-runtime
|
|
3
|
-
description: Choose and compose
|
|
3
|
+
description: Choose maintained runtime APIs and compose execution, evaluation, and controlled improvement.
|
|
4
4
|
---
|
|
5
5
|
|
|
6
|
-
# Build with
|
|
6
|
+
# Build with Agent Runtime
|
|
7
7
|
|
|
8
|
-
|
|
9
|
-
|
|
8
|
+
Build on the maintained execution path while keeping product policy and storage in the consumer.
|
|
9
|
+
Read the current [API decision table](https://github.com/tangle-network/agent-runtime/blob/main/docs/canonical-api.md) and [package exports](https://github.com/tangle-network/agent-runtime/blob/main/package.json).
|
|
10
|
+
Follow the chosen entrypoint to its implementation and nearest runnable example.
|
|
11
|
+
For an existing consumer, confirm the actual installed package supports the chosen contract.
|
|
10
12
|
|
|
11
|
-
##
|
|
13
|
+
## Choose by the required outcome
|
|
12
14
|
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
3. Read the nearest runnable example.
|
|
16
|
-
4. Treat source as authoritative when docs disagree, then correct the stale doc in the same change.
|
|
17
|
-
|
|
18
|
-
## Ownership
|
|
15
|
+
Use the existing entrypoint for one turn, a bounded task, fixed composition, dynamic supervision, or a measured improvement.
|
|
16
|
+
Avoid copying the API catalog into product code or creating a wrapper that only renames it.
|
|
19
17
|
|
|
20
18
|
| Concern | Owner |
|
|
21
19
|
|---|---|
|
|
22
|
-
| Portable prompt, skills, tools, MCP, hooks,
|
|
23
|
-
|
|
|
24
|
-
|
|
|
25
|
-
|
|
|
26
|
-
|
|
|
27
|
-
|
|
28
|
-
Do not move shared measurement into Runtime or product code.
|
|
29
|
-
Do not move product storage transactions into a provider-neutral package.
|
|
30
|
-
|
|
31
|
-
## Choose the entry point
|
|
32
|
-
|
|
33
|
-
| Need | Use |
|
|
34
|
-
|---|---|
|
|
35
|
-
| One product chat turn | `handleChatTurn(...)` |
|
|
36
|
-
| One normalized streamed agent turn | `streamAgentTurn(...)` and `collectAgentTurn(...)` |
|
|
37
|
-
| One task or multi-turn loop | `runAgentTask(...)`, `runAgentTaskStream(...)`, or `runAgentRounds(...)` |
|
|
38
|
-
| Supervisor and workers | `supervise(...)` or `superviseSurface(...)` |
|
|
39
|
-
| Static roles with versioned delegation and analysis directives | `runGraph(...)` |
|
|
40
|
-
| Parallel work with a shared budget | `fanout(...)` |
|
|
41
|
-
| Fixed composition | `pipeline(...)`, `panel(...)`, or `verify(...)` |
|
|
42
|
-
| Product benchmark | `defineLeaderboard(...)` |
|
|
43
|
-
| Profile matrix | `expandProfileAxes(...)` and `runProfileMatrix(...)` from agent-eval |
|
|
44
|
-
| Search one agent surface | `improve(...)` |
|
|
45
|
-
| Analyze traces through a measured proposal | `proposeAgentImprovement(...)` |
|
|
46
|
-
| Review and authorize an exact proposal | `reviewAgentImprovementProposal(...)` and `createAgentImprovementActivation(...)` |
|
|
47
|
-
| Apply or restore an approved candidate | `executeAgentImprovementActivation(...)` with a product transaction |
|
|
48
|
-
| Build a knowledge candidate | `runKnowledgeImprovementJob(...)` |
|
|
49
|
-
| Apply a knowledge candidate | `createKnowledgeImprovementActivationExecutor(...)` through the same activation path |
|
|
50
|
-
| Observe and pull approved changes on a live agent | `withIntelligence(...)` |
|
|
51
|
-
|
|
52
|
-
## Improvement flow
|
|
53
|
-
|
|
54
|
-
`improve(profile, options)` searches one surface and returns a detached winner.
|
|
55
|
-
It never changes a profile, document, repository, memory store, or knowledge base.
|
|
56
|
-
|
|
57
|
-
For a profile field, pass one complete agent-eval `OptimizationMethod`, explicit train, selection, and final-test partitions, judges, and the candidate execution function.
|
|
58
|
-
Use `officialGepa(...)` with an explicit recipe when upstream GEPA should own search.
|
|
59
|
-
Use `officialSkillOpt(...)` when Microsoft's SkillOpt should own search.
|
|
60
|
-
Both require `evaluationId`; change it whenever dispatch, judges, models, or scoring behavior changes.
|
|
61
|
-
Resumable runs accept `never`, `if-compatible`, or `required` and reuse state only when agent-eval derives the same run identity.
|
|
62
|
-
Runtime has no local prompt, skill, memory, or profile optimizer fallback.
|
|
63
|
-
Code uses Runtime's isolated worktrees and returns a sealed patch candidate.
|
|
64
|
-
Knowledge uses `runKnowledgeImprovementJob(...)` and returns paired snapshots.
|
|
65
|
-
|
|
66
|
-
Use `proposeAgentImprovement(...)` for a production proposal.
|
|
67
|
-
It performs these steps in order:
|
|
68
|
-
|
|
69
|
-
1. Analyze completed traces.
|
|
70
|
-
2. Search for a candidate on development tasks.
|
|
71
|
-
3. Build the frozen baseline, candidate, and held-back work.
|
|
72
|
-
4. Return only baseline, candidate, held-back tasks, and policy; Runtime adds the optimizer ancestry and seals the final experiment.
|
|
73
|
-
5. Reject the experiment if its candidate differs from the search winner.
|
|
74
|
-
6. Run baseline and candidate on the same held-back tasks.
|
|
75
|
-
7. Produce findings, confidence intervals, quality, cost, latency, and a decision.
|
|
76
|
-
|
|
77
|
-
After a person or tenant policy approves the proposal, call `createAgentImprovementActivation(...)` with target identities, funding owner, authority, intent, and expiry.
|
|
78
|
-
Runtime derives the expected current digests from the measured experiment.
|
|
79
|
-
Call `executeAgentImprovementActivation(...)` with one product-owned transaction that compares current state, writes every target atomically, and stores the result under the activation digest.
|
|
80
|
-
Pass a read-only reconciliation function so retries can distinguish committed, uncommitted, and uncertain outcomes.
|
|
81
|
-
|
|
82
|
-
Never apply a change from analyst confidence alone.
|
|
83
|
-
Never measure one candidate and apply another.
|
|
84
|
-
Never let search code write live state.
|
|
85
|
-
Never treat a lost response as a failed write without reconciling it.
|
|
86
|
-
|
|
87
|
-
## Surface rules
|
|
88
|
-
|
|
89
|
-
- Prompt changes `profile.prompt` only and requires a complete method.
|
|
90
|
-
- Skill optimization selects one inline skill by `skills.resourceName`, requires a complete method, and requires profile resources to fail closed.
|
|
91
|
-
- Curated memory changes `profile.resources.instructions`; retrieval stores and memory databases belong in the knowledge flow.
|
|
92
|
-
- Tools, MCP, hooks, subagents, curated memory, rollout policy, and whole-profile changes require a complete method.
|
|
93
|
-
- Code candidates must come from the Runtime worktree path so patch identity and cleanup stay intact.
|
|
94
|
-
- Workflow files are code surfaces. Parameter sweeps belong in a complete agent-eval method.
|
|
95
|
-
- Knowledge candidates remain detached until the shared activation path applies or restores their frozen snapshots.
|
|
96
|
-
|
|
97
|
-
## Product integration
|
|
98
|
-
|
|
99
|
-
The product supplies only the pieces that vary by deployment:
|
|
20
|
+
| Portable prompt, skills, tools, MCP, hooks, and model hints | AgentProfile from agent-interface |
|
|
21
|
+
| Execution, supervision, budgets, streaming, and candidate execution | Agent-runtime |
|
|
22
|
+
| Cases, grading, search, statistics, and comparison | Agent-eval |
|
|
23
|
+
| Retrieval, citations, freshness, memory stores, and knowledge promotion | Agent-knowledge |
|
|
24
|
+
| Users, permissions, funding, UI, persistence, and atomic writes | The product |
|
|
100
25
|
|
|
101
|
-
|
|
102
|
-
|
|
103
|
-
- How proposal, review, activation, and result records are persisted.
|
|
104
|
-
- How a target is changed atomically.
|
|
105
|
-
- Who may approve, reject, request changes, fund, apply, or restore.
|
|
106
|
-
- How those records and actions appear in the UI or API.
|
|
26
|
+
Keep measurement in Eval and product storage transactions in the consumer.
|
|
27
|
+
Use the same agent definition and execution path in the product and its evaluation.
|
|
107
28
|
|
|
108
|
-
|
|
29
|
+
## Integrate the selected capability
|
|
109
30
|
|
|
110
|
-
|
|
31
|
+
Search for the existing product adapter and current package usage before adding infrastructure.
|
|
32
|
+
Supply only the policy, storage, credential, and execution-placement boundaries the consumer needs.
|
|
33
|
+
Preserve explicit failures, cost and usage capture, cancellation, and recovery behavior.
|
|
111
34
|
|
|
112
|
-
|
|
113
|
-
|
|
114
|
-
- Do not use Runtime's code generator to approximate GEPA, SkillOpt, or another upstream profile optimizer.
|
|
115
|
-
- Do not write a second candidate catalog; persist the immutable proposal records.
|
|
116
|
-
- Do not let an analyst or adapter commit, push, open a pull request, or edit a live store.
|
|
117
|
-
- Do not hand-roll SSE parsing, usage totals, profile matrices, bootstrap statistics, sandbox acquisition, or worktree cleanup.
|
|
118
|
-
- Do not attach completed Runtime totals to an Eval campaign. Use `loopDispatch` or `loopCampaignDispatch` so admission and receipt capture surround the paid work.
|
|
119
|
-
- Do not add a product-local approval format for knowledge, code, or profile changes.
|
|
35
|
+
When changing prompts, skills, code, or knowledge through measured search, read [improvement and activation](references/improvement.md) before implementing that path.
|
|
36
|
+
Ordinary execution work does not need an optimizer or activation workflow.
|
|
120
37
|
|
|
121
|
-
##
|
|
38
|
+
## Prove the integration
|
|
122
39
|
|
|
123
|
-
|
|
124
|
-
|
|
125
|
-
-
|
|
126
|
-
|
|
127
|
-
|
|
128
|
-
- Activation is authorized, expiring, idempotent, and reconcilable.
|
|
129
|
-
- No customer write, message, trigger, or billing occurs in read-only proof mode.
|
|
130
|
-
- Public examples, package exports, generated API docs, type checks, tests, build, and package verification pass.
|
|
40
|
+
Run a real task through the selected backend and inspect its result and execution evidence.
|
|
41
|
+
Test the changed contract's denial, failure, cancellation, or recovery cases as applicable.
|
|
42
|
+
An in-process test proves only its own path; it does not prove a deployed sandbox path.
|
|
43
|
+
Run the repository's required checks, public-import checks when exports change, and the consumer's affected flow.
|
|
44
|
+
Report retained product adapters, adopted exports, observable results, and unchecked boundaries.
|
|
131
45
|
|
|
132
46
|
## Then consider
|
|
133
47
|
|
|
134
|
-
-
|
|
135
|
-
-
|
|
136
|
-
-
|
|
48
|
+
- `build-with-agent-knowledge` when the remaining work concerns retrieval or memory integration.
|
|
49
|
+
- `critical-audit` when a changed public contract needs independent review.
|
|
50
|
+
- `verify` when implementation is complete and release checks remain.
|
|
@@ -0,0 +1,32 @@
|
|
|
1
|
+
# Measured improvement and activation
|
|
2
|
+
|
|
3
|
+
Read the current [improvement exports](https://github.com/tangle-network/agent-runtime/blob/main/src/improvement/index.ts), [intelligence exports](https://github.com/tangle-network/agent-runtime/blob/main/src/intelligence/index.ts), and relevant sections of the [API decision table](https://github.com/tangle-network/agent-runtime/blob/main/docs/canonical-api.md).
|
|
4
|
+
For knowledge changes, also read the [knowledge exports](https://github.com/tangle-network/agent-runtime/blob/main/src/knowledge/index.ts).
|
|
5
|
+
Use the selected function's current types and maintained example rather than copying a method signature from this guide.
|
|
6
|
+
|
|
7
|
+
## Search without changing the live system
|
|
8
|
+
|
|
9
|
+
Use the existing improvement API and a complete Eval optimization method for the chosen surface.
|
|
10
|
+
Keep development, selection, and final decision cases separate.
|
|
11
|
+
Record the delivered profile resources and execution identity so resumed work cannot silently reuse incompatible measurements.
|
|
12
|
+
Search returns a detached candidate; it cannot edit the live product, knowledge store, or repository.
|
|
13
|
+
|
|
14
|
+
Prompt, tool, resource, and profile changes remain portable profile data.
|
|
15
|
+
Code candidates use Runtime's isolated worktree and patch identity path.
|
|
16
|
+
Knowledge candidates use the existing snapshot and promotion contract.
|
|
17
|
+
Do not rebuild candidate hashing, statistics, or search history in the consumer.
|
|
18
|
+
|
|
19
|
+
## Apply only the measured candidate
|
|
20
|
+
|
|
21
|
+
Use the maintained proposal, review, and activation path.
|
|
22
|
+
The proposal must compare the unchanged baseline and exact candidate on tasks hidden during search.
|
|
23
|
+
Keep candidate identity checked before execution and before activation.
|
|
24
|
+
Retain quality, cost, latency, sample count, uncertainty, and rejected outcomes.
|
|
25
|
+
|
|
26
|
+
The product supplies authority, funding, target identities, persistence, and an atomic transaction.
|
|
27
|
+
That transaction compares expected current state, writes the authorized targets, and records the activation outcome under its retry identity.
|
|
28
|
+
Use read-only reconciliation to distinguish committed, uncommitted, and uncertain outcomes after a lost response.
|
|
29
|
+
Preserve expiry and authority checks; review evidence does not itself grant write authority.
|
|
30
|
+
|
|
31
|
+
Prove rejection, successful activation, expired or mismatched activation, and retry after an uncertain write.
|
|
32
|
+
A read-only experiment must not send customer messages, mutate customer data, or incur product billing side effects.
|
package/skills/codemode/SKILL.md
CHANGED
|
@@ -1,49 +1,37 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: codemode
|
|
3
|
-
description: Batch mechanical tool work
|
|
3
|
+
description: Batch mechanical tool work in code while preserving judgment, authorization, and accounting.
|
|
4
4
|
---
|
|
5
5
|
|
|
6
6
|
# Codemode
|
|
7
7
|
|
|
8
|
-
|
|
9
|
-
|
|
10
|
-
Write one program instead: the loop, the branch, and the intermediates stay in the program, and only the decision-relevant summary returns.
|
|
8
|
+
Batch stretches of mechanical tool work whose intermediate results require no judgment.
|
|
9
|
+
Keep decisions that could change the plan in the agent's turn.
|
|
11
10
|
|
|
12
|
-
|
|
13
|
-
In a coding harness you already have the whole capability: a shell, a filesystem, and the tools this profile grants.
|
|
11
|
+
## Batch the work
|
|
14
12
|
|
|
15
|
-
|
|
13
|
+
Identify independent calls and the points where an observed result must change the next action.
|
|
14
|
+
Use the session's permitted execution tool to hold intermediate data in variables or workspace files.
|
|
15
|
+
Return the decision-relevant values, failures, and artifact locations.
|
|
16
|
+
Inspect every result; a successful batch must not hide a failed item.
|
|
17
|
+
Retain results needed later instead of rerunning expensive work to recover them.
|
|
16
18
|
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
5. Prefer one script that fans out over N items to N separate tool calls with identical shape.
|
|
22
|
-
6. Stop batching the moment a result changes what you would do next; read it, decide, then batch again.
|
|
19
|
+
Respect each tool's concurrency, cancellation, and authorization contract.
|
|
20
|
+
A batch does not expand the permission of its individual operations.
|
|
21
|
+
Meter paid operations on the owning execution path and preserve their usage records.
|
|
22
|
+
Keep dependent actions sequential unless their contract supports safe composition.
|
|
23
23
|
|
|
24
|
-
##
|
|
24
|
+
## Runtime-supervised code
|
|
25
25
|
|
|
26
|
-
|
|
27
|
-
|
|
28
|
-
|
|
29
|
-
The lint on authored code refuses imports, `process`, and network access; it is a lint, not a sandbox, so treat generated code you did not review as untrusted.
|
|
26
|
+
When configuring code execution for a Runtime supervisor, read [the execution boundary](references/runtime-execution.md).
|
|
27
|
+
That branch supplies a generated API over Runtime's existing coordination tools and requires an explicit runner.
|
|
28
|
+
For ordinary shell or session-tool batching, no additional runtime is needed.
|
|
30
29
|
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
A raw chat model has no shell, so give it the runtime's code mode: pass `codeModeSupervisorTools()` as `resolveSupervisorTools` and the supervisor's tool surface becomes `search` and `execute`.
|
|
34
|
-
`search` answers a TypeScript API generated from the live coordination grant; `execute` runs the model's program through a caller-supplied runner, and every `api.spawn_worker` call crosses the kernel's pool, authorization, and journal.
|
|
35
|
-
Supply a jailed runner for an untrusted model: the in-process runner is not an isolation boundary.
|
|
36
|
-
The lifecycle verbs (`submit_result`, `stop`, `ask_parent`) stay model tools: the program does the mechanics, the model keeps the judgment.
|
|
37
|
-
|
|
38
|
-
## Common Mistakes
|
|
39
|
-
|
|
40
|
-
- Batching a step whose output should have changed your plan, then discovering it three steps later.
|
|
41
|
-
- Printing a whole dataset into the reply instead of writing it to a file and printing the summary.
|
|
42
|
-
- Re-running an expensive script to re-read a value the first run already produced; write results to files.
|
|
43
|
-
- Moving supervision into a script because the coordination verbs are reachable over local HTTP.
|
|
30
|
+
Complete the requested work and inspect the resulting artifact.
|
|
31
|
+
Code reduces mechanical round trips; it does not replace judgment or prove the quality of the result.
|
|
44
32
|
|
|
45
33
|
## Then consider
|
|
46
34
|
|
|
47
|
-
- `supervise` when the
|
|
48
|
-
- `agent-graphs` when
|
|
49
|
-
- `loop-writer` when no
|
|
35
|
+
- `supervise` when the work needs workers with their own judgment.
|
|
36
|
+
- `agent-graphs` when fixed roles require explicit Runtime relationships and shared accounting.
|
|
37
|
+
- `loop-writer` when no maintained composition expresses the required control policy.
|
|
@@ -0,0 +1,14 @@
|
|
|
1
|
+
# Runtime code execution boundary
|
|
2
|
+
|
|
3
|
+
Read [code-mode implementation and types](https://github.com/tangle-network/agent-runtime/blob/main/src/runtime/supervise/code-mode.ts) and its [contract tests](https://github.com/tangle-network/agent-runtime/blob/main/tests/kernel/code-mode.test.ts).
|
|
4
|
+
Use `codeModeSupervisorTools(runner)` with an explicit `CodeModeRunner`; there is no default runner.
|
|
5
|
+
For untrusted model output, supply a real isolated execution environment.
|
|
6
|
+
The in-process runner and source lint are not security boundaries.
|
|
7
|
+
|
|
8
|
+
The generated API follows the live coordination grant.
|
|
9
|
+
Code can spawn or steer through Runtime-provided bindings such as `api.spawn_worker`; these retain authorization, shared budgets, and journal records.
|
|
10
|
+
Direct coordination requests over HTTP or a second scheduler bypass that contract.
|
|
11
|
+
|
|
12
|
+
Lifecycle decisions remain model tools: `submit_result`, `stop`, and `ask_parent` are outside the generated code API.
|
|
13
|
+
Keep judgment in the model and mechanics in the program.
|
|
14
|
+
Before broad use, exercise one allowed call, one denied call, a failed operation, and cancellation through the selected runner.
|
|
@@ -13,8 +13,9 @@ Do not use it for general coding quality or subjective output.
|
|
|
13
13
|
- `TARGET`: a pinned package version, repository commit, or release.
|
|
14
14
|
- `OUT`: the path for one candidate JSON object.
|
|
15
15
|
|
|
16
|
-
Read
|
|
16
|
+
Read the current [candidate schema](https://github.com/tangle-network/agent-runtime/blob/main/bench/src/generate-eval/schema.ts) and [execution checks](https://github.com/tangle-network/agent-runtime/blob/main/bench/src/generate-eval/certify.ts) before authoring the candidate.
|
|
17
17
|
Those files define the current format and checks.
|
|
18
|
+
Use a maintained target for new cases, then freeze its exact identity so later runs compare the same behavior.
|
|
18
19
|
|
|
19
20
|
## Build One Case
|
|
20
21
|
|
|
@@ -1,103 +1,49 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: loop-writer
|
|
3
|
-
description:
|
|
3
|
+
description: Build a custom execution policy only when maintained Runtime composition cannot express it.
|
|
4
4
|
---
|
|
5
5
|
|
|
6
6
|
# Loop Writer
|
|
7
7
|
|
|
8
|
-
Use this
|
|
9
|
-
Read
|
|
10
|
-
|
|
8
|
+
Use this for a required execution policy that maintained Runtime APIs cannot express.
|
|
9
|
+
Read the [current decision table](https://github.com/tangle-network/agent-runtime/blob/main/docs/canonical-api.md), [exports](https://github.com/tangle-network/agent-runtime/blob/main/package.json), selected implementation, and nearest test.
|
|
10
|
+
If an existing entrypoint fits, use it and stop.
|
|
11
|
+
A wrapper that only renames inputs or outputs does not justify a custom loop.
|
|
11
12
|
|
|
12
|
-
##
|
|
13
|
+
## Define the missing behavior
|
|
13
14
|
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
| Two or more actors taking turns | `defineConversation(...)` and `runConversation(...)` |
|
|
19
|
-
| A driver coordinating workers | `supervise(...)` or `superviseSurface(...)` |
|
|
20
|
-
| Parallel or fixed composition | `fanout(...)`, `pipeline(...)`, `panel(...)`, `verify(...)`, or `loopUntil(...)` |
|
|
21
|
-
| Parallel repository workers with isolated branches | `worktreeFanout(...)` |
|
|
22
|
-
| Repeated work in a graded tool environment | `runAgentic(...)` |
|
|
23
|
-
| Equal-budget comparison over that environment | `runBenchmark(...)` |
|
|
24
|
-
| Low-level round policy with custom planning and stopping | `runAgentRounds(...)` |
|
|
15
|
+
Name the consumer, required decision, and why existing composition cannot express it.
|
|
16
|
+
Reuse Runtime's execution, accounting, cancellation, questions, and recovery contracts.
|
|
17
|
+
The custom policy chooses work and continuation from the task and checked prior outcomes.
|
|
18
|
+
It must not introduce another scheduler or measurement system.
|
|
25
19
|
|
|
26
|
-
|
|
27
|
-
|
|
20
|
+
Keep planning separate from consequential writes.
|
|
21
|
+
The caller owns credentials, persistence, and authority; executors return observed artifacts; independent checks decide whether those artifacts satisfy the task.
|
|
22
|
+
A model score cannot override failed objective checks, denied actions, or service failures.
|
|
28
23
|
|
|
29
|
-
##
|
|
24
|
+
## Preserve observable state
|
|
30
25
|
|
|
31
|
-
|
|
26
|
+
Represent successful delivery, exhausted resources, cancellation, unresolved questions, execution failure, and interrupted recovery where they apply.
|
|
27
|
+
Keep measurement and service errors distinct from agent failure.
|
|
28
|
+
Retain task and attempt identities, artifacts, spend, and the reason for continuing or stopping.
|
|
29
|
+
Resume from durable facts and reconcile uncertain writes before retrying.
|
|
32
30
|
|
|
33
|
-
|
|
34
|
-
|
|
35
|
-
|
|
31
|
+
Steering and questions use the existing typed events and delivery records.
|
|
32
|
+
Keep unanswered required questions visible.
|
|
33
|
+
Parallel workers receive isolated state or an explicit shared-mutation contract.
|
|
34
|
+
Children remain within parent authority and the same accounting and cancellation rules.
|
|
36
35
|
|
|
37
|
-
|
|
36
|
+
## Prove the policy
|
|
38
37
|
|
|
39
|
-
|
|
40
|
-
|
|
41
|
-
|
|
42
|
-
- Trace emission records plans, attempts, tool effects, checks, spend, and decisions.
|
|
43
|
-
- The caller owns policy, budgets, credentials, persistence, and side-effect authority.
|
|
38
|
+
Test the custom decision on a representative real task.
|
|
39
|
+
Exercise its relevant failure and recovery paths, including rejected output followed by correction, resource exhaustion, cancellation, and duplicate-side-effect prevention.
|
|
40
|
+
Use existing contract tests for unchanged Runtime behavior; add checks that distinguish the new policy.
|
|
44
41
|
|
|
45
|
-
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
## Required States
|
|
49
|
-
|
|
50
|
-
Model every terminal and resumable state explicitly:
|
|
51
|
-
|
|
52
|
-
- succeeded with the accepted artifact;
|
|
53
|
-
- exhausted by rounds, tokens, money, time, or concurrency;
|
|
54
|
-
- cancelled by the caller;
|
|
55
|
-
- blocked on a question or permission;
|
|
56
|
-
- failed before an attempt;
|
|
57
|
-
- failed during execution or checking;
|
|
58
|
-
- interrupted with enough durable state to resume safely.
|
|
59
|
-
|
|
60
|
-
Use stable run, attempt, task, and parent IDs.
|
|
61
|
-
Persist the accepted artifact, every attempted artifact identity, spend, and final reason.
|
|
62
|
-
Resume from durable facts, not an in-memory counter or summary alone.
|
|
63
|
-
|
|
64
|
-
## Steering And Questions
|
|
65
|
-
|
|
66
|
-
Steering is a typed input to a running or replacement attempt.
|
|
67
|
-
Record who sent it, why, which evidence motivated it, whether delivery succeeded, and which attempt consumed it.
|
|
68
|
-
|
|
69
|
-
Questions are explicit events.
|
|
70
|
-
Route them to the responsible parent or user, preserve unanswered blockers, and fail closed when a required answer is unavailable.
|
|
71
|
-
Do not bury a permission request in free-form worker output.
|
|
72
|
-
|
|
73
|
-
## Parallel And Recursive Work
|
|
74
|
-
|
|
75
|
-
Give parallel workers isolated state unless shared mutation is the point of the task.
|
|
76
|
-
For repository changes, use one worktree per worker and explicit merge outcomes.
|
|
77
|
-
For external writes, use idempotency keys and product-owned transactions.
|
|
78
|
-
|
|
79
|
-
Recursive supervisors use the same budget, cancellation, trace, question, and completion contracts at every depth.
|
|
80
|
-
Do not grant a child more authority than its parent.
|
|
81
|
-
|
|
82
|
-
## Tests
|
|
83
|
-
|
|
84
|
-
Cover:
|
|
85
|
-
|
|
86
|
-
- success on the first and later rounds;
|
|
87
|
-
- invalid output followed by a corrected attempt;
|
|
88
|
-
- every budget limit;
|
|
89
|
-
- abort propagation to in-flight work;
|
|
90
|
-
- service and check failures remaining distinct from agent failure;
|
|
91
|
-
- unresolved blocking questions;
|
|
92
|
-
- interrupted run recovery without duplicate side effects;
|
|
93
|
-
- deterministic replay of decisions from saved outcomes where supported.
|
|
94
|
-
|
|
95
|
-
## Completion
|
|
96
|
-
|
|
97
|
-
The change is complete when the public entrypoint is smaller than the policy it replaces, all states above are observable, a real task exercises the custom behavior, and package typecheck, tests, build, docs, and package verification pass.
|
|
42
|
+
Complete when the required behavior works through its public entrypoint, final and resumable states remain observable, and the relevant package and consumer checks pass.
|
|
43
|
+
Report the exact missing capability supplied, retained Runtime contracts, real result, and limits.
|
|
98
44
|
|
|
99
45
|
## Then consider
|
|
100
46
|
|
|
101
47
|
- `critical-audit` when the loop changes a public contract or authority boundary.
|
|
102
|
-
- `eval-engineering` when the
|
|
103
|
-
- `verify`
|
|
48
|
+
- `eval-engineering` when the stopping condition lacks an adequate evaluation case.
|
|
49
|
+
- `verify` when implementation is complete and release checks remain.
|
|
@@ -1,110 +1,58 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: supervise
|
|
3
|
-
description: Author and drive recursive AgentProfiles with
|
|
3
|
+
description: Author and drive recursive AgentProfiles with explicit capabilities, evidence, and recovery.
|
|
4
4
|
---
|
|
5
5
|
|
|
6
6
|
# Supervise
|
|
7
7
|
|
|
8
8
|
Use this when Runtime coordination tools are attached.
|
|
9
|
-
Author agents that can perform the work, then
|
|
10
|
-
|
|
11
|
-
##
|
|
12
|
-
|
|
13
|
-
Put research choices, methods, revision rules, and
|
|
14
|
-
Runtime owns
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
|
|
34
|
-
|
|
35
|
-
|
|
36
|
-
|
|
37
|
-
|
|
38
|
-
|
|
39
|
-
|
|
40
|
-
|
|
41
|
-
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
|
|
45
|
-
"kind": "inline",
|
|
46
|
-
"name": "profile-authoring/SKILL.md",
|
|
47
|
-
"content": "<the complete profile-authoring skill text>"
|
|
48
|
-
}
|
|
49
|
-
]
|
|
50
|
-
}
|
|
51
|
-
}
|
|
52
|
-
```
|
|
53
|
-
|
|
54
|
-
The example shows placement, not required values.
|
|
55
|
-
Every agent is the same `AgentProfile` shape.
|
|
56
|
-
An agent becomes a recursive lead only by declaring `agent_runtime_coordination_spawn_worker: true`.
|
|
57
|
-
Declare each other Runtime verb it will use, such as `await_event`, `steer_agent`, or `read_journal`.
|
|
58
|
-
Runtime mounts only the declared bare verbs through its coordination surface; the provider receives a profile projection without Runtime-owned declarations.
|
|
59
|
-
Metadata can describe the work, but it never grants execution authority.
|
|
60
|
-
Every profile that can spawn workers carries the complete profile-authoring skill in `resources.skills`.
|
|
61
|
-
Make that resource an immutable inline snapshot or a pinned reference, and set `resources.failOnError: true`.
|
|
62
|
-
This is taught through the profile, not injected or enforced by Runtime: the authored profile remains the complete record of why it can delegate.
|
|
63
|
-
Omit Runtime coordination tools for a leaf.
|
|
64
|
-
|
|
65
|
-
The task argument names the concrete artifact and a check that can fail.
|
|
66
|
-
The profile names how the agent works and which capabilities it receives.
|
|
67
|
-
Together they must leave no acceptance criterion for the worker to invent.
|
|
68
|
-
|
|
69
|
-
## Drive the tree
|
|
70
|
-
|
|
71
|
-
1. Split the objective into independent checked artifacts.
|
|
72
|
-
2. Start each new assignment with a stable semantic `key` and a deliberate per-spawn `budget`.
|
|
73
|
-
3. Fill available parallel capacity while `freeSlots > 0` and distinct useful assignments remain.
|
|
74
|
-
4. Pull settlements and findings with `await_event`.
|
|
75
|
-
A bounded wait returns control; it does not prove failure.
|
|
76
|
-
5. Inspect a quiet or `stalled` worker with `observe_agent`.
|
|
77
|
-
Steer only when its recorded work shows a wrong path, missing requirement, or useful new evidence.
|
|
78
|
-
6. Check every settled artifact before using it.
|
|
79
|
-
Feed accepted results, contradictions, and negative results into the next profile or task revision.
|
|
80
|
-
7. After a refusal or failed check, preserve the attempt and author a materially changed profile or assignment.
|
|
81
|
-
8. Use `continuity: 'resume'` only to continue the most recent settled worker with the same profile name.
|
|
82
|
-
A resume is a new execution and cannot carry a run-once `key`.
|
|
83
|
-
9. After recovery, call `read_journal` and reconcile Runtime's restored roster, settlements, questions, findings, and spend before spawning.
|
|
84
|
-
Completed keys resolve to their committed results; do not replace them.
|
|
85
|
-
|
|
86
|
-
Do not add a deadline because a worker is quiet.
|
|
87
|
-
Stop research only at checked success, a declared resource limit, cancellation, or a demonstrated dead end.
|
|
9
|
+
Author agents that can perform the required work, then direct them from checked evidence.
|
|
10
|
+
|
|
11
|
+
## Author complete agents
|
|
12
|
+
|
|
13
|
+
Put research choices, methods, revision rules, and stopping criteria in each profile's prompt.
|
|
14
|
+
Runtime owns shared budgets, recursion limits, concurrency, cancellation, journals, and recovery.
|
|
15
|
+
The caller configures root execution through [SuperviseOptions](https://github.com/tangle-network/agent-runtime/blob/main/src/runtime/supervise/supervise.ts).
|
|
16
|
+
Per-assignment budgets, continuity, and keys belong to the [coordination tools](https://github.com/tangle-network/agent-runtime/blob/main/src/mcp/tools/coordination.ts), not invented profile fields.
|
|
17
|
+
|
|
18
|
+
An agent receives recursive authority by declaring `agent_runtime_coordination_spawn_worker: true` in its tools.
|
|
19
|
+
Declare each other Runtime tool it needs explicitly.
|
|
20
|
+
Metadata describes work; it does not grant authority.
|
|
21
|
+
Every profile that can spawn workers carries the complete `profile-authoring/SKILL.md` resource, using an immutable snapshot and `resources.failOnError: true`.
|
|
22
|
+
The authored profile must explain its own ability to delegate; Runtime does not invent that policy.
|
|
23
|
+
|
|
24
|
+
When creating or changing a descendant profile, read [profile authoring](references/profile-authoring.md) for the exact contract and resource placement.
|
|
25
|
+
The task names the concrete artifact and completion check; the profile names the method and granted capabilities.
|
|
26
|
+
|
|
27
|
+
## Direct the work
|
|
28
|
+
|
|
29
|
+
1. Split the objective into independent artifacts with checkable outcomes.
|
|
30
|
+
2. Start each assignment with a stable semantic key and a deliberate budget.
|
|
31
|
+
Fill available capacity while distinct useful assignments remain.
|
|
32
|
+
3. Pull settlements, findings, and questions with `await_event`.
|
|
33
|
+
A bounded wait returning does not establish failure.
|
|
34
|
+
4. Inspect quiet or stalled workers with `observe_agent` before steering them.
|
|
35
|
+
Correct an observed wrong path, missing requirement, or new evidence.
|
|
36
|
+
5. Check each returned artifact before using it.
|
|
37
|
+
Preserve failed attempts and change the profile or assignment when evidence supports another attempt.
|
|
38
|
+
6. After recovery, reconcile the journal, roster, settlements, questions, findings, and spend before creating replacements.
|
|
39
|
+
|
|
40
|
+
Use `continuity: 'resume'` to continue the most recent settled worker with that profile name.
|
|
41
|
+
A resumed assignment is a new execution and cannot carry a run-once key.
|
|
42
|
+
Completed keys resolve to committed results; an uncertain prior dispatch must be reconciled before replacement.
|
|
43
|
+
Stop at checked success, an authorized resource limit, cancellation, or a demonstrated dead end.
|
|
44
|
+
A quiet worker alone is not a reason to invent a deadline.
|
|
88
45
|
|
|
89
46
|
## Accept delivery
|
|
90
47
|
|
|
91
48
|
Inspect the artifact and its independent completion result.
|
|
92
|
-
Preserve
|
|
93
|
-
Worker prose cannot
|
|
94
|
-
|
|
95
|
-
|
|
96
|
-
Calling `stop` ends coordination; it does not turn missing evidence into success.
|
|
97
|
-
|
|
98
|
-
## Exact contracts
|
|
99
|
-
|
|
100
|
-
When a field is unclear, read the owner instead of inventing it:
|
|
101
|
-
|
|
102
|
-
- [`AgentProfile`](https://github.com/tangle-network/agent-sdk/blob/main/packages/agent-interface/src/agent-profile.ts) and its [exact schema](https://github.com/tangle-network/agent-sdk/blob/main/packages/agent-interface/src/profile-schema.ts) own authored fields.
|
|
103
|
-
- [`spawn_worker` tool schema](https://github.com/tangle-network/agent-runtime/blob/main/src/mcp/tools/coordination.ts) owns per-assignment arguments.
|
|
104
|
-
- [`SuperviseOptions`](https://github.com/tangle-network/agent-runtime/blob/main/src/runtime/supervise/supervise.ts) owns root execution policy.
|
|
49
|
+
Preserve profile identities, assignment keys, parent-child links, continuations, costs, failures, and missing accounting.
|
|
50
|
+
Worker prose cannot approve its own result.
|
|
51
|
+
Use `submit_result` only when the attached check can validate this agent's artifact.
|
|
52
|
+
Calling `stop` ends coordination; it does not establish completion.
|
|
105
53
|
|
|
106
54
|
## Then consider
|
|
107
55
|
|
|
108
|
-
- `build-with-agent-runtime` when
|
|
56
|
+
- `build-with-agent-runtime` when the product must change Runtime limits or adapters.
|
|
109
57
|
- `eval-engineering` when no existing check can separate success from failure.
|
|
110
|
-
- `verify`
|
|
58
|
+
- `verify` when required artifacts pass and delivery checks remain.
|