melchizedek-agents 1.5.0 → 1.7.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (44) hide show
  1. package/CHANGELOG.md +115 -0
  2. package/DOCUMENTATION.md +28 -2
  3. package/README.md +1 -1
  4. package/config/agents/syndicate.schema.json +63 -0
  5. package/config/agents/syndicateSchema.yaml +50 -0
  6. package/db/hardening.sql +5 -2
  7. package/db/migrations/0015_model_pricing.sql +107 -0
  8. package/db/telemetry.sql +79 -1
  9. package/dist/lib/a2a/executor.js +21 -0
  10. package/dist/lib/compile.d.ts +4 -3
  11. package/dist/lib/compile.js +8 -3
  12. package/dist/lib/index.d.ts +9 -1
  13. package/dist/lib/index.js +5 -0
  14. package/dist/lib/loadSyndicate.d.ts +13 -0
  15. package/dist/lib/loadSyndicate.js +11 -1
  16. package/dist/lib/observability/supabaseSpanExporter.js +2 -1
  17. package/dist/lib/observability/tracer.d.ts +11 -2
  18. package/dist/lib/observability/tracer.js +17 -4
  19. package/dist/lib/runtime/native/agentLoop.d.ts +6 -0
  20. package/dist/lib/runtime/native/checkpoint.d.ts +2 -1
  21. package/dist/lib/runtime/native/checkpoint.js +6 -2
  22. package/dist/lib/runtime/native/delegate.js +30 -1
  23. package/dist/lib/runtime/native/interrupts.js +9 -2
  24. package/dist/lib/runtime/nativeTurn.d.ts +2 -1
  25. package/dist/lib/runtime/nativeTurn.js +5 -1
  26. package/dist/lib/runtime/syndicateTurn.d.ts +25 -2
  27. package/dist/lib/runtime/syndicateTurn.js +387 -5
  28. package/dist/lib/runtime/turnControl.d.ts +27 -0
  29. package/dist/lib/sandbox/sandbox.d.ts +121 -0
  30. package/dist/lib/sandbox/sandbox.js +276 -0
  31. package/dist/lib/sandbox/worker.d.ts +29 -0
  32. package/dist/lib/sandbox/worker.js +280 -0
  33. package/dist/lib/script/config.d.ts +35 -0
  34. package/dist/lib/script/config.js +17 -0
  35. package/dist/lib/script/journal.d.ts +137 -0
  36. package/dist/lib/script/journal.js +193 -0
  37. package/dist/lib/script/run.d.ts +138 -0
  38. package/dist/lib/script/run.js +364 -0
  39. package/dist/lib/syndicateSchema.d.ts +20 -0
  40. package/dist/lib/syndicateSchema.js +98 -6
  41. package/dist/lib/toolRegistry.js +4 -0
  42. package/dist/lib/tools/runCodeTool.d.ts +50 -0
  43. package/dist/lib/tools/runCodeTool.js +59 -0
  44. package/package.json +5 -1
package/CHANGELOG.md CHANGED
@@ -6,6 +6,121 @@ the starter pack and the templates), not the repo's full history.
6
6
 
7
7
  ## Unreleased
8
8
 
9
+ ## 1.7.0 — 2026-10-17
10
+
11
+ A minor release that lets a syndicate hold code (ADR 0137). A `script:`
12
+ block is the fourth orchestration method: one JavaScript module, run in
13
+ a QuickJS sandbox on a worker thread, calls the syndicate's agents, its
14
+ allowed tools and the person as functions, so routing, per-call models
15
+ and human-in-the-loop change with a YAML deploy instead of an engine
16
+ release. Agents on any provider get `run_code`, the same sandbox as a
17
+ registry tool. Both are off until the operator sets
18
+ `MELCHIZEDEK_CODE_SANDBOX=on`; nothing changes for a deployment that
19
+ does not.
20
+
21
+ ### Added
22
+
23
+ - **`script:` orchestration (ADR 0137 Part B).** `script: { source,
24
+ models?, tools? }` at the root; cannot be combined with `dispatch` or
25
+ `workflow`. The default export is called with `{ message, transcript,
26
+ variables, agent, tool, askUser, parallel, now, random, log }` and its
27
+ return is the answer; guards run on it once. `agent(name, input, {
28
+ model?, reasoning?, temperature?, conversation? })` runs a declared
29
+ agent (parsed JSON when it has an output schema, else text; a failed run
30
+ throws into the script); `conversation: true` answers in the
31
+ conversation as a dispatch route does. `askUser(id, question, {
32
+ payload?, schema? })` pauses the turn `input-required` with the same
33
+ `result.input` an `ask_user` node writes; the next message resumes the
34
+ run by replaying its journal (one content-less event per call, stored in
35
+ the conversation only when the run pauses) on the source the pause pinned, so a redeploy never breaks
36
+ a waiting conversation. A resume whose calls no longer match fails
37
+ `SCRIPT_DIVERGED`; a run past 1,000 calls fails `SCRIPT_CALL_LIMIT`.
38
+ The schema refuses `require_approval`, the `ask_user` tool, local skill
39
+ scripts and authorization_code grants on a script's agents (a pause
40
+ inside a called agent cannot be replayed yet), `context:` and root
41
+ `max_concurrency`; `mode: task` is allowed (a pause reached inside a
42
+ called agent anyway fails `PAUSE_INSIDE_AGENT`). `{{token}}`
43
+ interpolation skips `script.source`; the script reads the bound
44
+ `variables`. `script.require_approval` gates `tool()` calls with the
45
+ standard approval request (`result.approval`). Every `script.models`
46
+ pairing is checked before the run (`SCRIPT_MODEL_INVALID`). Under
47
+ `runDurableTurn` each call's journal event is a checkpoint boundary, so
48
+ a stopped worker resumes mid-script. `agent()` calls report through
49
+ `events.onAgentEvent`; spans `script.invoke` and `script.call`
50
+ (exported to `adk_telemetry`, no arguments or results) trace a run, and
51
+ each agent call's turn span carries `script.run` and `script.call.seq`.
52
+ A `yaml_reference` to a script syndicate runs its whole script as a
53
+ subagent, route or node, its pauses reaching the turn; `agent()` may
54
+ call nested workflow, dispatch and script syndicates.
55
+ `startEngineSpan` takes an optional parent context.
56
+ `TurnStage` gains `'script'`; `SyndicateTurnResult.failedStage` may be
57
+ `'script'`.
58
+ - **`run_code` (ADR 0137 Part A).** A registry tool: `{ code, input? }`,
59
+ an ES module whose default export is called with `input`, returning
60
+ `{ result, logs, error }`, each bounded at 20,000 characters. No
61
+ network, files, host, clock or randomness. An agent's
62
+ `run_code: { attach: true }` puts each run on the reply:
63
+ `SyndicateTurnResult.codeRuns` and, on the A2A server, a `code_runs`
64
+ artifact of `code_run` data parts. `DrainedRun` gains `codeRuns`.
65
+ - **The sandbox.** QuickJS (`quickjs-emscripten-core` and
66
+ `@jitl/quickjs-wasmfile-release-sync`, both pinned at 0.32.0) on a
67
+ worker thread per run: 64 MB memory, 1 MB stack, 5 s of compute (waits
68
+ on the host not counted), a watchdog and the turn's signal terminate
69
+ it; `MELCHIZEDEK_CODE_SANDBOX_WORKERS` (default 16) caps concurrent
70
+ runs. A syndicate with a script or a `run_code` agent is refused with
71
+ `CodeSandboxOffError` when a turn starts while the switch is off.
72
+ - **Package surface.** New subpaths `melchizedek-agents/script`
73
+ (`runScript`, `ScriptFatal`, `SCRIPT_CALL_LIMIT`, and the journal reader
74
+ `openRun` / `scriptPauseIn` with the `SCRIPT_*` marker keys and the
75
+ `JournalEntry`, `OpenRun`, `ScriptCallRecord`, `ScriptPauseRecord`,
76
+ `ScriptPinRecord` types) and
77
+ `melchizedek-agents/sandbox` (`runInSandbox`, `codeSandboxEnabled`,
78
+ `refuseWithoutSandbox`, `CodeSandboxOffError`, `SandboxHalt`,
79
+ `DEFAULT_SANDBOX_LIMITS`). The barrel adds those plus
80
+ `isScriptSyndicate`, `openRun`, `runCodeTool`, `RUN_CODE` and the types
81
+ `ScriptConfig`, `ScriptHost`, `ScriptCallSite`, `JournalEntry`, `OpenRun`,
82
+ `ScriptPauseRecord`, `ScriptPinRecord`, `ScriptRunInput`, `ScriptRunOutcome`,
83
+ `SandboxLimits`, `SandboxResult`, `SandboxRunOptions`, `CodeRun`,
84
+ `RunCodeOutput`.
85
+
86
+ ### For melch-app
87
+
88
+ - An Electron build must list `quickjs-emscripten-core`,
89
+ `@jitl/quickjs-wasmfile-release-sync` and `melchizedek-agents/dist/lib/sandbox`
90
+ in `asarUnpack`: the wasm file and the worker script load from disk.
91
+
92
+ ## 1.6.0 — 2026-10-15
93
+
94
+ A minor release for applications that show a turn as it runs and account
95
+ for what it costs. Delegated subagents report live through
96
+ `events.onAgentEvent`, with their path and the call that started them
97
+ (ADR 0135), and the ledger records dollars: `model_pricing` and `cost_usd`
98
+ on every turn and model-call row, computed at write time (ADR 0136).
99
+ Nothing changes for a consumer that passes no new callback; a ledger
100
+ deployment re-runs `npx melchizedek-db apply --telemetry` to get the
101
+ column.
102
+
103
+ ### Added
104
+
105
+ - **Live events from delegated subagents (ADR 0135).** `TurnEvents` gains
106
+ `onAgentEvent(event, branch)`: every event a delegated run yields, as it
107
+ yields it (thinking and text partials, tool calls, results, the answer),
108
+ at any depth, with `branch.agent`, `branch.path` (the delegation path from
109
+ the caller at the top) and `branch.callId` (the caller's call that started
110
+ it). New type export `AgentBranch` from the barrel and
111
+ `melchizedek-agents/runtime`; `TurnControl` gains the optional
112
+ `agentEvents` it is carried on. Passing the callback makes delegated runs
113
+ stream; the stored sessions, `onEvent` and `onTextDelta` are unchanged.
114
+
115
+ - **Cost in the ledger (ADR 0136).** Migration `0015_model_pricing` adds
116
+ `model_pricing` (USD per million tokens, `verified_at` per row, seeded
117
+ from melch-obs's `pricing.toml`, `ollama/*` at zero) and the functions
118
+ `melchizedek_cost_usd` and `melchizedek_usage_cost`. `db/telemetry.sql`
119
+ adds `cost_usd` to `adk_turns` and `adk_telemetry`, filled at write time by
120
+ a trigger (NULL for a model with no row), and `cost_usd` to
121
+ `adk_kpi_daily`. The root span gains `syndicate.usage` (each model's
122
+ calls and tokens). Re-run `npx melchizedek-db apply --telemetry`.
123
+
9
124
  ## 1.5.0 — 2026-10-15
10
125
 
11
126
  A minor release that makes the engine embeddable in a single-process
package/DOCUMENTATION.md CHANGED
@@ -35,6 +35,8 @@ lib/tools/mcpToolFactory.ts MCP client: remote tools → the engine's own Tools
35
35
  scripts/demo_mcp_server.ts demo MCP server (library catalog, SSE)
36
36
  lib/runtime/native/… the native loop: steps, delegation, compaction, interrupts
37
37
  lib/workflow/… the workflow scheduler for `workflow:` syndicates
38
+ lib/script/… the runner for `script:` syndicates: journal, replay, pause (ADR 0137)
39
+ lib/sandbox/… the QuickJS code sandbox on a worker thread (run_code, script:)
38
40
  lib/session/… Supabase-backed session service
39
41
  lib/memory/… pgvector long-term memory service
40
42
  lib/observability/… OpenTelemetry run tracing
@@ -63,7 +65,7 @@ agent. The native runtime is the only runtime
63
65
  ([ADR 0107](./wiki/decisions/0107-release-1-0-0-removes-adk.md)): it owns the
64
66
  loop, the model contract and its adapters, the tools, the sessions and
65
67
  the workflow scheduler, and runs a single-agent, delegating,
66
- plan-dispatch or `workflow:` syndicate, `context:` compaction
68
+ plan-dispatch, `workflow:` or `script:` syndicate, `context:` compaction
67
69
  ([ADR 0078](./wiki/decisions/0078-native-compaction-ports-adk-compactor.md)),
68
70
  `mode: task`, and pause and resume included
69
71
  ([ADR 0095](./wiki/decisions/0095-native-workflow-turn-drains-through-the-adk-reader.md)).
@@ -130,11 +132,13 @@ Field reference:
130
132
  | `memory_retention_days` | root | Days a fact in this syndicate's namespace is kept; the server deletes older facts when it loads the syndicate and daily after. Requires `memory_namespace` (never applied to the shared default namespace). |
131
133
  | `dispatch` | root | Switches the syndicate from DELEGATE to PLAN-DISPATCH routing (§6). `default_route` (required) names the fail-static subagent; `route_key` / `reason_key` name the router's JSON properties (defaults `route` / `reason`). Honoured by every surface (the CLI, the server, the worker, evals). |
132
134
  | `workflow` | root | Switches the syndicate to a WORKFLOW (§6): a graph whose nodes are its agents plus declared `join`, `map`, `tool` and `ask_user` nodes, and whose `edges` say what runs after what and on which route. The orchestrator is a node like any other; nothing delegates. Cannot be combined with `dispatch`. Contract: `lib/workflow.ts`; worked example: `examples/pipeline.yaml`. |
135
+ | `script` | root | Switches the syndicate to a SCRIPT ([ADR 0137](./wiki/decisions/0137-script-orchestration-and-sandboxed-code.md)): `source` is one JavaScript module, run in the engine's QuickJS sandbox, whose default export gets `{ message, transcript, variables, agent, tool, askUser, parallel, now, random, log }` and returns the answer. `models` lists the models `agent(name, input, { model })` may move a call to; `tools` the registry tools `tool(name, args)` may run. `askUser` pauses the turn; the next message replays the run's journal on the source it pinned. Needs `MELCHIZEDEK_CODE_SANDBOX=on`; cannot be combined with `dispatch` or `workflow`; the agents may not pause inside themselves (no `require_approval`, `ask_user` tool, local skill scripts or authorization_code grants). `{{token}}` does not touch `source`. |
133
136
  | `retries` | root | Self-correction, on by default ([ADR 0034](./wiki/decisions/0034-self-correction.md)): `model_errors` (default 2) retries a model reply marked malformed (`MALFORMED_FUNCTION_CALL`) instead of failing the turn; `tool_errors` (default 3) answers a tool that threw with structured reflection guidance and caps its retries. `0` turns either off. Every retry is a model call under `max_steps`. |
134
137
  | `guards` | root | Optional list of post-answer guard NAMES (`lib/guards/index.ts`). Each runs after the answering turn and before the reply publishes, receiving the final text plus every tool-result text of that turn, and rewrites in place rather than re-asking the model; its notes land in the `[STATUS]` stream. Guards declared by a syndicate reached through `yaml_reference:` count too — the server resolves the union via `collectGuards()`. Resolved by name, never by module path, so adding one is a deliberate act in code; **the published registry ships one, `science`** (citation checks for `research.yaml`); `registerGuard()` adds your own from code, and an unregistered name is warned about and skipped. |
135
138
  | `name` / `model` / `instruction` | agent | The agent triple. Any Gemini id, `claude-*`, or `ollama/*` for open-weight local models (see §5). |
136
139
  | `description` | subagent | **The delegation API.** The orchestrator reads this when deciding to hand off — write it like a function signature ("Use this subagent to…, pass it…"). |
137
140
  | `tools` | agent | Names resolved by the tool registry (§3). Long-term memory agents add `preload_memory` / `load_memory`. |
141
+ | `run_code` | agent | With `run_code` in `tools`, the agent runs JavaScript in the engine's sandbox on any provider (`{ code, input? }` → `{ result, logs, error }`; no network, host, clock or randomness; needs `MELCHIZEDEK_CODE_SANDBOX=on`, [ADR 0137](./wiki/decisions/0137-script-orchestration-and-sandboxed-code.md)). `run_code: { attach: true }` puts each run's code and result on the reply as a `code_run` data part (`result.codeRuns`). |
138
142
  | `reasoning` | agent | How hard the agent reasons, on any provider: `none`, `low`, `medium`, `high`, `xhigh`, `max`, or `{ budget_tokens: <int> }`. The compiler sends each provider the field it reads: a thinking level on Gemini 3, a thinking budget on Claude 4.6 and earlier (2,048 / 8,192 / 16,384 tokens for low / medium / high), adaptive thinking with that effort on later Claude models ([ADR 0049](./wiki/decisions/0049-claude-requests-by-model-generation.md)), an effort word on GPT, Grok, Kimi, Ollama and the gateway ([ADR 0047](./wiki/decisions/0047-provider-neutral-reasoning-key.md)). `xhigh` and `max` reach the models that take them (both on Claude from Opus 4.7 and Sonnet 5, `max` on Kimi K3, `xhigh` on GPT-5.2 and later and grok-4.7); elsewhere the model's highest setting is sent and the span carries `llm.reasoning.weakened` (`npm run doctor -- --matrix` lists each path, [ADR 0117](./wiki/decisions/0117-tool-choice-and-effort-above-high-in-v2.md)). Unset, each adapter keeps its own default. |
139
143
  | `sampling` | agent | `{ temperature, top_p, max_output_tokens, stop }`, the fields the engine sends to every provider; each adapter sends the ones its API takes ([ADR 0115](./wiki/decisions/0115-yaml-schema-v2-provider-neutral-keys.md)). |
140
144
  | `output` | agent | `{ schema, mime }`. `mime` is `application/json` (JSON mode, with or without a schema) or `text/plain` (the default). `schema` is the structured-JSON contract: the agent ends its turn on one JSON object matching it. An orchestrator may hold one beside its subagents: it delegates first, then answers in the schema itself. The schema travels in the provider's own structured-output field beside the tools on Claude from Opus 4.8, Sonnet 5 and Haiku 5.5, on OpenAI, and on Gemini 2+ through Vertex AI, and as a `set_model_response` tool elsewhere (the capability matrix's `structured_output_with_tools`, [ADR 0109](./wiki/decisions/0109-structured-output-beside-tools.md)). Worked example: `examples/structured_critic.yaml`. A leaf that holds the schema (`critic.yaml`) and plan-dispatch remain the choices when the answer should be a specialist's. |
@@ -634,7 +638,17 @@ a clean call's row comes from `model.call` and holds the request and the
634
638
  adapter's response in the contract's shapes. Rows written before 1.0.0
635
639
  under ADK's `invoke_agent`, `call_llm` and `execute_tool` spans hold ADK's
636
640
  request, and the ledger still reads them (ADR 0076). The view `adk_turns_production` excludes
637
- eval and classifier turns. Operate it with `npm run telemetry:stats`,
641
+ eval and classifier turns. `adk_turns.cost_usd` and `adk_telemetry.cost_usd`
642
+ are filled at write time by a trigger from `model_pricing` (migration 0015,
643
+ USD per million tokens, seeded from melch-obs's `pricing.toml` with a
644
+ `verified_at` per row; NULL means unchecked): a model call from its model and
645
+ tokens, a turn from its per-model usage (`syndicate.usage` on the root span,
646
+ input summed per call), thinking at the output rate unless the row sets
647
+ `thinking_per_mtok`. A model with no row leaves the call and its turn NULL,
648
+ never a partial sum; `ollama/*` costs zero. Add or change a rate with an
649
+ `INSERT … ON CONFLICT (model) DO UPDATE` on `model_pricing`; rows already
650
+ written keep the rate in force when they were written (ADR 0136).
651
+ Operate it with `npm run telemetry:stats`,
638
652
  `telemetry:prune` and `telemetry:replay` (the exporter spools failed
639
653
  batches to `outputs/telemetry-deadletter.ndjson`).
640
654
 
@@ -847,6 +861,18 @@ in flight, and the task ends `canceled`. Final states: `completed`,
847
861
  capacity). Message parts may be `text` or `data` (sent to the model as
848
862
  JSON); `file` parts are refused.
849
863
 
864
+ In code, `runSyndicateTurn`'s `events` follow a turn as it runs:
865
+ `onProgress` (the `[STATUS]` lines), `onEvent` (every event of the
866
+ answering agent), `onTextDelta`/`onTextReset` (its reply as it is written,
867
+ with `streaming: true`), and `onAgentEvent(event, branch)` for delegated
868
+ runs (ADR 0135). The last receives every event a delegated subagent yields
869
+ while it runs: its thinking and text as partial events, then its tool
870
+ calls, their results and its answer, at any depth. `branch.agent` names the
871
+ subagent, `branch.path` lists the agents from the caller at the top down to
872
+ it, and `branch.callId` is the id of the caller's call that started it, so
873
+ calls running at once can be told apart. Passing it makes delegated runs
874
+ stream; the sessions store the same events either way.
875
+
850
876
  #### Approval gates (`require_approval`)
851
877
 
852
878
  A skill script run (`skills.scripts: local`) pauses the same way, without being listed: every `run_skill_script` call waits for the person's answer.
package/README.md CHANGED
@@ -104,7 +104,7 @@ const result = await runSyndicateTurn({
104
104
  console.log(result.status, result.text);
105
105
  ```
106
106
 
107
- `runSyndicateTurn` is the same runtime the server, CLI, and eval harness use. It runs every syndicate on the engine's own agent loop and, for a workflow, its own scheduler (the `native` runtime, the only one). `MELCHIZEDEK_RUNTIME=adk` is an error that names 1.0.0; `npx melchizedek-doctor` prints the runtime in use (ADR 0107). To read syndicates from somewhere else, pass `loadSyndicate(file, { agentsDir })` or set `MELCHIZEDEK_AGENTS_DIR`. To serve inside your own Express app, mount `(await createA2AApp({ defaultSyndicate: 'mine.yaml', serverSecret })).app`.
107
+ `runSyndicateTurn` is the same runtime the server, CLI, and eval harness use. It runs every syndicate on the engine's own agent loop and, for a workflow, its own scheduler (the `native` runtime, the only one). `MELCHIZEDEK_RUNTIME=adk` is an error that names 1.0.0; `npx melchizedek-doctor` prints the runtime in use (ADR 0107). To read syndicates from somewhere else, pass `loadSyndicate(file, { agentsDir })` or set `MELCHIZEDEK_AGENTS_DIR`. To serve inside your own Express app, mount `(await createA2AApp({ defaultSyndicate: 'mine.yaml', serverSecret })).app`. `events.onEvent` and `events.onTextDelta` follow the answering agent; `events.onAgentEvent(event, branch)` follows every delegated subagent live, its thinking, text and tool calls, with `branch.path` and `branch.callId` saying where it runs (ADR 0135).
108
108
 
109
109
  The example syndicates ship inside the package at `node_modules/melchizedek-agents/config/agents/examples/`; copy one out as a starting point.
110
110
 
@@ -943,6 +943,17 @@
943
943
  "type": "string",
944
944
  "const": "gemini"
945
945
  },
946
+ "run_code": {
947
+ "description": "How this agent's run_code tool behaves (ADR 0137). Needs run_code in `tools`.",
948
+ "type": "object",
949
+ "properties": {
950
+ "attach": {
951
+ "description": "Put each run's code and result on the reply as a code_run data part. Default false.",
952
+ "type": "boolean"
953
+ }
954
+ },
955
+ "additionalProperties": false
956
+ },
946
957
  "context": {
947
958
  "description": "Summarize a long conversation instead of overflowing the window. The orchestrator of a delegate syndicate only.",
948
959
  "type": "object",
@@ -1967,6 +1978,17 @@
1967
1978
  "type": "string",
1968
1979
  "const": "gemini"
1969
1980
  },
1981
+ "run_code": {
1982
+ "description": "How this agent's run_code tool behaves (ADR 0137). Needs run_code in `tools`.",
1983
+ "type": "object",
1984
+ "properties": {
1985
+ "attach": {
1986
+ "description": "Put each run's code and result on the reply as a code_run data part. Default false.",
1987
+ "type": "boolean"
1988
+ }
1989
+ },
1990
+ "additionalProperties": false
1991
+ },
1970
1992
  "context": {
1971
1993
  "description": "Summarize a long conversation instead of overflowing the window. The orchestrator of a delegate syndicate only.",
1972
1994
  "type": "object",
@@ -2302,6 +2324,47 @@
2302
2324
  "additionalProperties": false,
2303
2325
  "description": "Opts into a WORKFLOW: the syndicate as a graph (lib/workflow.ts). Cannot be combined with dispatch."
2304
2326
  },
2327
+ "script": {
2328
+ "type": "object",
2329
+ "properties": {
2330
+ "source": {
2331
+ "type": "string",
2332
+ "minLength": 1,
2333
+ "maxLength": 65536,
2334
+ "description": "One JavaScript module whose default export is an async function of { message, variables, agent, tool, askUser, parallel, now, random, log }: the orchestration. Its return value is the answer. {{token}} interpolation does not touch it; it reads `variables`."
2335
+ },
2336
+ "models": {
2337
+ "description": "The models agent(name, input, { model }) may move a call to; each agent is compiled and checked on every one at load.",
2338
+ "maxItems": 16,
2339
+ "type": "array",
2340
+ "items": {
2341
+ "type": "string",
2342
+ "minLength": 1
2343
+ }
2344
+ },
2345
+ "tools": {
2346
+ "description": "The registry tools tool(name, args) may run; the only path from the script to the network or the host.",
2347
+ "type": "array",
2348
+ "items": {
2349
+ "type": "string",
2350
+ "minLength": 1
2351
+ }
2352
+ },
2353
+ "require_approval": {
2354
+ "description": "Tools from script.tools whose every call waits for a person to approve the exact call (ADR 0028): the turn ends input-required; approved, the pinned arguments run, refused, the script gets { error }.",
2355
+ "type": "array",
2356
+ "items": {
2357
+ "type": "string",
2358
+ "minLength": 1
2359
+ }
2360
+ }
2361
+ },
2362
+ "required": [
2363
+ "source"
2364
+ ],
2365
+ "additionalProperties": false,
2366
+ "description": "Opts into SCRIPT orchestration (lib/script/): the syndicate's logic as code run in the engine's sandbox. Needs MELCHIZEDEK_CODE_SANDBOX=on. Cannot be combined with dispatch or workflow."
2367
+ },
2305
2368
  "retries": {
2306
2369
  "description": "Self-correction on model and tool errors: the engine's reflect-and-retry. On by default.",
2307
2370
  "type": "object",
@@ -190,6 +190,48 @@ dispatch:
190
190
  # Confirm: { ask_user: "Publish? Reply yes, or say what to change." }
191
191
  # max_concurrency: 4
192
192
 
193
+ # [optional] Switches this syndicate to a SCRIPT (ADR 0137): one
194
+ # JavaScript module, run in the engine's QuickJS sandbox, is the
195
+ # orchestration. It calls the agents below, the tools it lists and the
196
+ # person as functions, so routing, per-call models and pauses live here
197
+ # and change with a deploy. Needs MELCHIZEDEK_CODE_SANDBOX=on. Cannot be
198
+ # combined with `dispatch` or `workflow`. Contract: lib/script/.
199
+ #
200
+ # - The default export is called with { message, transcript, variables,
201
+ # agent, tool, askUser, parallel, now, random, log }; its return (text,
202
+ # or JSON) is the answer, and `guards` run on it once.
203
+ # - agent(name, input, { model?, reasoning?, temperature?,
204
+ # conversation? }) runs a declared agent seeing only `input` (null:
205
+ # the message); it returns parsed JSON when the agent has an output
206
+ # schema, else its text, and throws when the run fails. With
207
+ # `conversation: true` it answers in the conversation, as a dispatch
208
+ # route does. `model` must be in script.models.
209
+ # - tool(name, args) runs a registry tool listed in script.tools; one
210
+ # also in script.require_approval waits for a person to approve the
211
+ # exact call (refused, the script gets { error }).
212
+ # - askUser(id, question, { payload?, schema? }) pauses the turn
213
+ # input-required; the next message resumes it, returning
214
+ # { reply, input }. The run replays its journal from the top, so a
215
+ # script must make the same calls on every run: use now() and
216
+ # random(), never Date or Math.random (they throw), and never
217
+ # Promise.race.
218
+ # - require_approval, the ask_user tool, skills.scripts: local and
219
+ # authorization_code grants are refused on a script's agents in this
220
+ # version (a pause inside a called agent cannot be replayed).
221
+ # - {{token}} interpolation does not touch `source`; read `variables`.
222
+ # - A yaml_reference to a script syndicate runs its whole script as a
223
+ # subagent, a route or a node, its pauses reaching the turn.
224
+ # script:
225
+ # models: ["gemini-3.8-flash", "claude-opus-5-5"]
226
+ # tools: ["web_extract", "publish"]
227
+ # require_approval: ["publish"]
228
+ # source: |
229
+ # export default async ({ message, agent, askUser, log }) => {
230
+ # const triage = await agent("OrchestratorName", message);
231
+ # log(`Routed to ${triage.route}`);
232
+ # return agent(triage.route, null, { conversation: true });
233
+ # };
234
+
193
235
  # ── Orchestrator ────────────────────────────────────────────
194
236
  # The turn's root agent: the engine's agent loop runs it. Its keys are
195
237
  # spelled after ADK's LlmAgentConfig.
@@ -293,6 +335,14 @@ orchestrator:
293
335
  # Google's server-side sandbox; nothing runs on this host. Gemini only.
294
336
  # code_execution: "gemini"
295
337
 
338
+ # [optional] With `run_code` in tools, the agent runs JavaScript in the
339
+ # engine's QuickJS sandbox on any provider and rests its answer on the
340
+ # result (no network, no host, no clock; needs MELCHIZEDEK_CODE_SANDBOX=on,
341
+ # ADR 0137). `attach: true` puts each run's code and result on the reply
342
+ # as a code_run data part. Default off.
343
+ # run_code:
344
+ # attach: true
345
+
296
346
  # [optional] Summarize a long conversation instead of overflowing the
297
347
  # window: past `compact_after_tokens` (the last request's prompt), earlier
298
348
  # turns become one summary and the last `keep_recent_events` stay
package/db/hardening.sql CHANGED
@@ -75,7 +75,8 @@ DO $$
75
75
  DECLARE t text;
76
76
  BEGIN
77
77
  FOREACH t IN ARRAY ARRAY['adk_telemetry', 'adk_turns', 'adk_payloads', 'adk_verdicts', 'adk_labels',
78
- 'adk_session_events', 'adk_a2a_tasks', 'melchizedek_usage', 'melchizedek_audit'] LOOP
78
+ 'adk_session_events', 'adk_a2a_tasks', 'melchizedek_usage', 'melchizedek_audit',
79
+ 'model_pricing'] LOOP
79
80
  IF to_regclass('public.' || t) IS NOT NULL THEN
80
81
  EXECUTE format('ALTER TABLE %I ENABLE ROW LEVEL SECURITY', t);
81
82
  PERFORM pg_temp.melchizedek_revoke_api(format('%I', t));
@@ -163,7 +164,9 @@ BEGIN
163
164
  'melchizedek_registry_versions_immutable', 'melchizedek_registry_publish',
164
165
  'melchizedek_registry_activate', 'melchizedek_memory_commit',
165
166
  'melchizedek_memory_dimensions', 'melchizedek_prune_memory_ingest',
166
- 'melchizedek_prune_memory', 'melchizedek_prune_audit')
167
+ 'melchizedek_prune_memory', 'melchizedek_prune_audit',
168
+ 'melchizedek_cost_usd', 'melchizedek_usage_cost', 'melchizedek_usage_text_cost',
169
+ 'melchizedek_turns_cost', 'melchizedek_telemetry_cost')
167
170
  LOOP
168
171
  EXECUTE format('REVOKE ALL ON FUNCTION %s FROM PUBLIC', fn);
169
172
  PERFORM pg_temp.melchizedek_revoke_api(format('FUNCTION %s', fn));
@@ -0,0 +1,107 @@
1
+ -- ============================================================================
2
+ -- 0015_model_pricing — what a model call costs, so the ledger records dollars
3
+ -- (ADR 0136)
4
+ -- ============================================================================
5
+ -- model_pricing: USD per million tokens, one row per model id as the YAML
6
+ -- declares it (`claude-opus-5`, `gemini-3.8-flash`, `grok-4.7`). A row whose
7
+ -- id ends in `/*` prices every id under that prefix (`ollama/*`: local
8
+ -- models cost nothing). `thinking_per_mtok` NULL means thinking bills at the
9
+ -- output rate, as every provider here bills it. `verified_at` NULL means
10
+ -- nobody has checked the row against the vendor's price list: spend built on
11
+ -- it is a steering estimate (the convention ymir_model_pricing set).
12
+ --
13
+ -- Seeded from melch-obs's pricing.toml (2026-10). The seed never overwrites a
14
+ -- row: an operator's own rates survive a re-run. A rate change ships as a
15
+ -- later migration that UPDATEs the row and its verified_at.
16
+ --
17
+ -- The cost functions below are what db/telemetry.sql's ledger trigger calls
18
+ -- to fill adk_turns.cost_usd and adk_telemetry.cost_usd at write time:
19
+ -- melchizedek_cost_usd(model, input, output, thinking) one call's cost,
20
+ -- NULL for a model with no row
21
+ -- melchizedek_usage_cost(usage) a turn's cost from its per-model usage
22
+ -- ({"<model>": {"input": n, "output": n, "thinking": n}}), NULL when
23
+ -- any model in it has no row: a partial sum would read as the total.
24
+ -- Ledger output_tokens exclude thinking (genaiMapping's usage meaning), so
25
+ -- thinking is priced on top of output.
26
+ --
27
+ -- Nothing secret, nothing personal; hardening.sql closes it to the API roles
28
+ -- like every other table. Idempotent; safe to re-run.
29
+ -- ============================================================================
30
+
31
+ CREATE TABLE IF NOT EXISTS model_pricing (
32
+ model TEXT PRIMARY KEY,
33
+ input_per_mtok NUMERIC NOT NULL CHECK (input_per_mtok >= 0),
34
+ output_per_mtok NUMERIC NOT NULL CHECK (output_per_mtok >= 0),
35
+ thinking_per_mtok NUMERIC CHECK (thinking_per_mtok >= 0),
36
+ verified_at TIMESTAMPTZ,
37
+ source TEXT,
38
+ note TEXT,
39
+ updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
40
+ );
41
+
42
+ INSERT INTO model_pricing (model, input_per_mtok, output_per_mtok, verified_at, source, note) VALUES
43
+ -- Anthropic first-party rates; pricing.toml dates the check to the month (2026-06).
44
+ ('claude-fable-5-1', 10.00, 50.00, '2026-06-01', 'melch-obs/pricing.toml', 'Anthropic first-party, checked 2026-06'),
45
+ ('claude-fable-5', 10.00, 50.00, '2026-06-01', 'melch-obs/pricing.toml', 'Anthropic first-party, checked 2026-06'),
46
+ ('claude-opus-5', 5.00, 25.00, '2026-06-01', 'melch-obs/pricing.toml', 'Anthropic first-party, checked 2026-06'),
47
+ ('claude-opus-4-8', 5.00, 25.00, '2026-06-01', 'melch-obs/pricing.toml', 'Anthropic first-party, checked 2026-06'),
48
+ ('claude-opus-4-7', 5.00, 25.00, '2026-06-01', 'melch-obs/pricing.toml', 'Anthropic first-party, checked 2026-06'),
49
+ ('claude-opus-4-6', 5.00, 25.00, '2026-06-01', 'melch-obs/pricing.toml', 'Anthropic first-party, checked 2026-06'),
50
+ ('claude-sonnet-5', 2.00, 10.00, '2026-06-01', 'melch-obs/pricing.toml', 'Anthropic first-party, checked 2026-06'),
51
+ ('claude-sonnet-4-6', 3.00, 15.00, '2026-06-01', 'melch-obs/pricing.toml', 'Anthropic first-party, checked 2026-06'),
52
+ ('claude-haiku-4-5', 1.00, 5.00, '2026-06-01', 'melch-obs/pricing.toml', 'Anthropic first-party, checked 2026-06'),
53
+ ('gemini-3.5-flash-lite', 0.30, 2.50, '2026-10-01', 'melch-obs/pricing.toml', 'Gemini API standard tier, checked 2026-10-01'),
54
+ -- Carried over from ymir_model_pricing, never checked against a vendor list.
55
+ ('gemini-3.1-flash-lite', 0.10, 0.40, NULL, 'melch-obs/pricing.toml', 'unverified estimate'),
56
+ ('gemini-3.7-flash', 0.30, 2.50, NULL, 'melch-obs/pricing.toml', 'unverified estimate'),
57
+ ('gemini-3.8-flash', 0.30, 2.50, NULL, 'melch-obs/pricing.toml', 'placeholder at 3.7''s rate, unverified'),
58
+ ('grok-4.5', 3.00, 15.00, NULL, 'melch-obs/pricing.toml', 'unverified estimate'),
59
+ ('grok-4.7', 3.00, 15.00, NULL, 'melch-obs/pricing.toml', 'unverified estimate'),
60
+ -- A local model bills nothing; that needs no vendor list.
61
+ ('ollama/*', 0, 0, '2026-10-15', 'engine', 'local models: no provider charge')
62
+ ON CONFLICT (model) DO NOTHING;
63
+
64
+ -- The row that prices `model`: its own, else its prefix's `<prefix>/*`.
65
+ CREATE OR REPLACE FUNCTION melchizedek_cost_usd(model TEXT, input BIGINT, output BIGINT, thinking BIGINT)
66
+ RETURNS NUMERIC
67
+ LANGUAGE sql
68
+ STABLE
69
+ SET search_path = public
70
+ AS $$
71
+ SELECT (coalesce(input, 0) * p.input_per_mtok
72
+ + coalesce(output, 0) * p.output_per_mtok
73
+ + coalesce(thinking, 0) * coalesce(p.thinking_per_mtok, p.output_per_mtok)) / 1000000
74
+ FROM model_pricing p
75
+ WHERE p.model = melchizedek_cost_usd.model
76
+ OR (position('/' IN melchizedek_cost_usd.model) > 0
77
+ AND p.model = split_part(melchizedek_cost_usd.model, '/', 1) || '/*')
78
+ ORDER BY (p.model = melchizedek_cost_usd.model) DESC
79
+ LIMIT 1
80
+ $$;
81
+
82
+ CREATE OR REPLACE FUNCTION melchizedek_usage_cost(usage JSONB)
83
+ RETURNS NUMERIC
84
+ LANGUAGE sql
85
+ STABLE
86
+ SET search_path = public
87
+ AS $$
88
+ SELECT CASE
89
+ WHEN usage IS NULL OR jsonb_typeof(usage) <> 'object' THEN NULL
90
+ -- bool_and over no rows is NULL: a turn that made no model call costs nothing.
91
+ WHEN coalesce(bool_and(c.cost IS NOT NULL), true) THEN coalesce(sum(c.cost), 0)
92
+ ELSE NULL
93
+ END
94
+ FROM (
95
+ SELECT melchizedek_cost_usd(
96
+ e.key,
97
+ CASE WHEN jsonb_typeof(e.value->'input') = 'number' THEN (e.value->>'input')::numeric::bigint END,
98
+ CASE WHEN jsonb_typeof(e.value->'output') = 'number' THEN (e.value->>'output')::numeric::bigint END,
99
+ CASE WHEN jsonb_typeof(e.value->'thinking') = 'number' THEN (e.value->>'thinking')::numeric::bigint END
100
+ ) AS cost
101
+ FROM jsonb_each(CASE WHEN jsonb_typeof(usage) = 'object' THEN usage ELSE '{}'::jsonb END) e
102
+ ) c
103
+ $$;
104
+
105
+ INSERT INTO melchizedek_schema_version (version, name)
106
+ VALUES (15, '0015_model_pricing')
107
+ ON CONFLICT (version) DO NOTHING;
package/db/telemetry.sql CHANGED
@@ -163,6 +163,83 @@ CREATE INDEX IF NOT EXISTS idx_adk_turns_surface_user
163
163
  -- at CREATE time. It does not gain these columns until it is re-created, so
164
164
  -- run the whole file, in order.
165
165
 
166
+ -- ── Cost: dollars per turn and per model call (ADR 0136) ─────────────────
167
+ -- cost_usd is filled at write time by the trigger below from model_pricing
168
+ -- (db/migrations/0015_model_pricing.sql), so both sinks and a replayed
169
+ -- dead-letter spool get it, and the price table is the one source:
170
+ -- adk_telemetry an llm.request row: its model and tokens. A root row: the
171
+ -- turn's per-model usage on the span (syndicate.usage).
172
+ -- adk_turns the turn's per-model usage (attributes->'syndicate.usage',
173
+ -- the tracer sums each model's calls), so a turn that used
174
+ -- two models is priced per model, input summed per call.
175
+ -- NULL when a model has no price row (a partial sum would read as the
176
+ -- total), or when the row carries no usage (a row written before this
177
+ -- column). A value the writer set itself is kept. The rate in force at
178
+ -- write time is the rate recorded: a later price change does not rewrite
179
+ -- history. Re-price on purpose with an UPDATE ... SET cost_usd = ... .
180
+ ALTER TABLE adk_turns ADD COLUMN IF NOT EXISTS cost_usd NUMERIC;
181
+ ALTER TABLE adk_telemetry ADD COLUMN IF NOT EXISTS cost_usd NUMERIC;
182
+
183
+ -- SECURITY DEFINER: the inserting role (service_role, or an application's
184
+ -- own role) need not be able to execute the pricing functions or read
185
+ -- model_pricing. Each reads only NEW and the price table. Unreadable usage
186
+ -- leaves the cost NULL; it never fails the insert.
187
+ CREATE OR REPLACE FUNCTION melchizedek_usage_text_cost(usage_text TEXT)
188
+ RETURNS NUMERIC
189
+ LANGUAGE plpgsql
190
+ STABLE
191
+ SET search_path = public
192
+ AS $$
193
+ BEGIN
194
+ IF usage_text IS NULL OR to_regclass('model_pricing') IS NULL THEN
195
+ RETURN NULL;
196
+ END IF;
197
+ RETURN melchizedek_usage_cost(usage_text::jsonb);
198
+ EXCEPTION WHEN others THEN
199
+ RETURN NULL;
200
+ END;
201
+ $$;
202
+
203
+ CREATE OR REPLACE FUNCTION melchizedek_turns_cost()
204
+ RETURNS trigger
205
+ LANGUAGE plpgsql
206
+ SECURITY DEFINER
207
+ SET search_path = public
208
+ AS $$
209
+ BEGIN
210
+ IF NEW.cost_usd IS NULL THEN
211
+ NEW.cost_usd := melchizedek_usage_text_cost(NEW.attributes->>'syndicate.usage');
212
+ END IF;
213
+ RETURN NEW;
214
+ END;
215
+ $$;
216
+
217
+ CREATE OR REPLACE FUNCTION melchizedek_telemetry_cost()
218
+ RETURNS trigger
219
+ LANGUAGE plpgsql
220
+ SECURITY DEFINER
221
+ SET search_path = public
222
+ AS $$
223
+ BEGIN
224
+ IF NEW.cost_usd IS NOT NULL OR to_regclass('model_pricing') IS NULL THEN
225
+ RETURN NEW;
226
+ END IF;
227
+ IF NEW.model IS NOT NULL THEN
228
+ NEW.cost_usd := melchizedek_cost_usd(NEW.model, NEW.input_tokens, NEW.output_tokens, NEW.thinking_tokens);
229
+ ELSE
230
+ NEW.cost_usd := melchizedek_usage_text_cost(NEW.span->'attributes'->>'syndicate.usage');
231
+ END IF;
232
+ RETURN NEW;
233
+ END;
234
+ $$;
235
+
236
+ DROP TRIGGER IF EXISTS melchizedek_turns_cost ON adk_turns;
237
+ CREATE TRIGGER melchizedek_turns_cost BEFORE INSERT ON adk_turns
238
+ FOR EACH ROW EXECUTE FUNCTION melchizedek_turns_cost();
239
+ DROP TRIGGER IF EXISTS melchizedek_telemetry_cost ON adk_telemetry;
240
+ CREATE TRIGGER melchizedek_telemetry_cost BEFORE INSERT ON adk_telemetry
241
+ FOR EACH ROW EXECUTE FUNCTION melchizedek_telemetry_cost();
242
+
166
243
  -- ── adk_payloads: full prompt/response per model call, by policy, expiring ─
167
244
  CREATE TABLE IF NOT EXISTS adk_payloads (
168
245
  id BIGINT GENERATED ALWAYS AS IDENTITY PRIMARY KEY,
@@ -360,7 +437,8 @@ CREATE OR REPLACE VIEW adk_kpi_daily AS
360
437
  sum(output_tokens) AS output_tokens,
361
438
  sum(thinking_tokens) AS thinking_tokens,
362
439
  avg(llm_calls) AS avg_llm_calls,
363
- avg(tool_calls) AS avg_tool_calls
440
+ avg(tool_calls) AS avg_tool_calls,
441
+ sum(cost_usd) AS cost_usd
364
442
  FROM adk_turns_production
365
443
  GROUP BY 1, 2;
366
444
 
@@ -375,6 +375,24 @@ export function approvalAnswer(rawParts, pending) {
375
375
  }
376
376
  return undefined;
377
377
  }
378
+ /** The `code_runs` artifact: one `code_run` data part per run an attaching agent made (ADR 0137 item 3). */
379
+ function publishCodeRuns(eventBus, taskId, contextId, runs) {
380
+ eventBus.publish(AgentEvent.artifactUpdate({
381
+ taskId,
382
+ contextId,
383
+ artifact: {
384
+ artifactId: 'code_runs',
385
+ name: 'code_runs',
386
+ description: 'The code the answer rests on, and what it returned',
387
+ parts: runs.map((run) => ({ content: { $case: 'data', value: run }, metadata: undefined, filename: '', mediaType: 'application/json' })),
388
+ metadata: undefined,
389
+ extensions: [],
390
+ },
391
+ append: false,
392
+ lastChunk: true,
393
+ metadata: undefined,
394
+ }));
395
+ }
378
396
  function publishFinal(eventBus, taskId, contextId, state, text) {
379
397
  eventBus.publish(AgentEvent.statusUpdate({
380
398
  taskId,
@@ -687,6 +705,9 @@ export class SyndicateExecutor {
687
705
  return;
688
706
  }
689
707
  log(`✓ Task ${short} complete — ${result.text.length.toLocaleString()} chars, ${spent}`);
708
+ // run_code's attach (ADR 0137): the code an answer rests on, and its result, as data parts beside it.
709
+ if (result.codeRuns?.length)
710
+ publishCodeRuns(eventBus, taskId, contextId, result.codeRuns);
690
711
  publishFinal(eventBus, taskId, contextId, 'completed', result.text || undefined);
691
712
  await report(ctx, 'completed', undefined, u);
692
713
  // Long-term memory: there is no "session end" on a server, so ingest
@@ -115,9 +115,10 @@ export type SpecTool = {
115
115
  dispatch: DispatchSpec;
116
116
  };
117
117
  /**
118
- * A nested dispatch (plan-dispatch) syndicate (ADR 0120): it runs as it
119
- * runs at the top, as a turn of its own on its own conversation (its
120
- * classifier picks a route, the route answers), so what it compiles to is
118
+ * A nested dispatch (plan-dispatch) syndicate (ADR 0120), or a nested
119
+ * script syndicate (ADR 0137 item 15): it runs as it runs at the top, as a
120
+ * turn of its own on its own conversation (its classifier picks a route and
121
+ * the route answers, or its script runs), so what it compiles to is
121
122
  * the file and the options its turn loads its routes with. The turn runner
122
123
  * runs it (lib/runtime/syndicateTurn.ts, TurnControl.nestedDispatch), as a
123
124
  * delegated call, a dispatch route or a workflow node.