melchizedek-agents 1.0.3 → 1.2.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (172) hide show
  1. package/.env.example +29 -2
  2. package/AGENT_SETUP.md +3 -1
  3. package/CHANGELOG.md +524 -0
  4. package/DOCUMENTATION.md +178 -43
  5. package/ONBOARDING.md +339 -0
  6. package/QUICKSTART.md +15 -2
  7. package/README.md +6 -2
  8. package/config/agents/README.md +2 -2
  9. package/config/agents/examples/ares.yaml +6 -10
  10. package/config/agents/examples/assistant.yaml +6 -6
  11. package/config/agents/examples/augustin.yaml +9 -15
  12. package/config/agents/examples/cartographers.yaml +6 -6
  13. package/config/agents/examples/council.yaml +7 -7
  14. package/config/agents/examples/critic.yaml +24 -24
  15. package/config/agents/examples/harness.yaml +4 -4
  16. package/config/agents/examples/image_production.yaml +37 -35
  17. package/config/agents/examples/librarian.yaml +4 -4
  18. package/config/agents/examples/patient_advocate.yaml +6 -10
  19. package/config/agents/examples/pipeline.yaml +20 -19
  20. package/config/agents/examples/research.yaml +36 -43
  21. package/config/agents/examples/scribe.yaml +31 -32
  22. package/config/agents/examples/scriptorium.yaml +6 -6
  23. package/config/agents/examples/structured_critic.yaml +82 -0
  24. package/config/agents/examples/style_council.yaml +9 -11
  25. package/config/agents/examples/syndicate.yaml +3 -5
  26. package/config/agents/examples/tutor.yaml +2 -2
  27. package/config/agents/examples/weather.yaml +2 -2
  28. package/config/agents/syndicate.schema.json +859 -13
  29. package/config/agents/syndicateSchema.yaml +122 -73
  30. package/config/agents/templates/README.md +1 -1
  31. package/config/agents/templates/account_memory.yaml +4 -4
  32. package/config/agents/templates/case_desk.yaml +6 -6
  33. package/config/agents/templates/conversational.yaml +2 -2
  34. package/config/agents/templates/draft_review.yaml +6 -6
  35. package/config/agents/templates/intake_extractor.yaml +64 -62
  36. package/config/agents/templates/research_brief.yaml +6 -6
  37. package/config/agents/templates/research_desk.yaml +6 -6
  38. package/config/agents/templates/review_panel.yaml +8 -8
  39. package/config/agents/templates/support_triage.yaml +30 -29
  40. package/config/agents/templates/systems_operator.yaml +24 -5
  41. package/db/migrations/0014_durable_runs.sql +31 -0
  42. package/dist/lib/a2a/app.d.ts +27 -0
  43. package/dist/lib/a2a/app.js +64 -5
  44. package/dist/lib/a2a/executor.d.ts +7 -1
  45. package/dist/lib/a2a/executor.js +43 -5
  46. package/dist/lib/a2a/oauthSetup.d.ts +106 -0
  47. package/dist/lib/a2a/oauthSetup.js +259 -0
  48. package/dist/lib/agentDialect.d.ts +76 -0
  49. package/dist/lib/agentDialect.js +151 -0
  50. package/dist/lib/compile.d.ts +42 -6
  51. package/dist/lib/compile.js +97 -45
  52. package/dist/lib/compileNative.d.ts +11 -3
  53. package/dist/lib/compileNative.js +20 -1
  54. package/dist/lib/dispatch.d.ts +10 -5
  55. package/dist/lib/dispatch.js +10 -5
  56. package/dist/lib/doctor.d.ts +110 -1
  57. package/dist/lib/doctor.js +205 -9
  58. package/dist/lib/loadSyndicate.d.ts +54 -2
  59. package/dist/lib/loadSyndicate.js +22 -2
  60. package/dist/lib/memory/namespace.d.ts +6 -5
  61. package/dist/lib/memory/namespace.js +6 -5
  62. package/dist/lib/memory/providers.js +3 -4
  63. package/dist/lib/models/capabilities.d.ts +25 -1
  64. package/dist/lib/models/capabilities.js +79 -1
  65. package/dist/lib/models/chatCompletionsAdapter.d.ts +11 -3
  66. package/dist/lib/models/chatCompletionsAdapter.js +19 -5
  67. package/dist/lib/models/claudeAdapter.js +4 -0
  68. package/dist/lib/models/contract.d.ts +5 -2
  69. package/dist/lib/models/gateway.d.ts +2 -1
  70. package/dist/lib/models/gateway.js +4 -2
  71. package/dist/lib/models/gatewayAdapter.d.ts +7 -1
  72. package/dist/lib/models/gatewayAdapter.js +8 -0
  73. package/dist/lib/models/geminiAdapter.js +8 -2
  74. package/dist/lib/models/genaiMapping.d.ts +8 -4
  75. package/dist/lib/models/genaiMapping.js +23 -9
  76. package/dist/lib/models/gptAdapter.d.ts +4 -2
  77. package/dist/lib/models/gptAdapter.js +11 -4
  78. package/dist/lib/models/grokAdapter.d.ts +3 -1
  79. package/dist/lib/models/grokAdapter.js +11 -3
  80. package/dist/lib/models/kimiAdapter.d.ts +2 -1
  81. package/dist/lib/models/kimiAdapter.js +3 -3
  82. package/dist/lib/models/reasoning.d.ts +30 -2
  83. package/dist/lib/models/reasoning.js +56 -5
  84. package/dist/lib/observability/tracer.d.ts +8 -1
  85. package/dist/lib/observability/tracer.js +33 -5
  86. package/dist/lib/onboarding.d.ts +134 -0
  87. package/dist/lib/onboarding.js +539 -0
  88. package/dist/lib/runtime/approvals.d.ts +8 -0
  89. package/dist/lib/runtime/approvals.js +4 -3
  90. package/dist/lib/runtime/credentials.d.ts +6 -0
  91. package/dist/lib/runtime/native/agentLoop.d.ts +23 -3
  92. package/dist/lib/runtime/native/agentLoop.js +107 -11
  93. package/dist/lib/runtime/native/checkpoint.d.ts +171 -0
  94. package/dist/lib/runtime/native/checkpoint.js +359 -0
  95. package/dist/lib/runtime/native/delegate.d.ts +222 -18
  96. package/dist/lib/runtime/native/delegate.js +390 -39
  97. package/dist/lib/runtime/native/interrupts.d.ts +184 -2
  98. package/dist/lib/runtime/native/interrupts.js +391 -1
  99. package/dist/lib/runtime/native/request.d.ts +13 -3
  100. package/dist/lib/runtime/native/request.js +28 -4
  101. package/dist/lib/runtime/native/telemetry.js +10 -15
  102. package/dist/lib/runtime/questions.d.ts +7 -4
  103. package/dist/lib/runtime/sessions.d.ts +1 -1
  104. package/dist/lib/runtime/syndicateTurn.d.ts +11 -3
  105. package/dist/lib/runtime/syndicateTurn.js +353 -68
  106. package/dist/lib/runtime/turnControl.d.ts +56 -0
  107. package/dist/lib/runtime/turnControl.js +10 -0
  108. package/dist/lib/session/transcript.d.ts +9 -0
  109. package/dist/lib/session/transcript.js +20 -1
  110. package/dist/lib/skills.d.ts +18 -0
  111. package/dist/lib/skills.js +73 -1
  112. package/dist/lib/storage/postgres/index.d.ts +12 -2
  113. package/dist/lib/storage/postgres/index.js +8 -3
  114. package/dist/lib/storage/postgres/taskQueue.d.ts +20 -4
  115. package/dist/lib/storage/postgres/taskQueue.js +29 -5
  116. package/dist/lib/storage/postgres/taskStore.d.ts +26 -2
  117. package/dist/lib/storage/postgres/taskStore.js +74 -8
  118. package/dist/lib/syndicateSchema.d.ts +316 -1
  119. package/dist/lib/syndicateSchema.js +203 -29
  120. package/dist/lib/tools/auth.d.ts +20 -6
  121. package/dist/lib/tools/auth.js +21 -5
  122. package/dist/lib/tools/credentialEnv.d.ts +28 -0
  123. package/dist/lib/tools/credentialEnv.js +52 -0
  124. package/dist/lib/tools/credentialHosts.d.ts +88 -0
  125. package/dist/lib/tools/credentialHosts.js +165 -0
  126. package/dist/lib/tools/credentialStore.js +9 -2
  127. package/dist/lib/tools/credentialUses.d.ts +92 -0
  128. package/dist/lib/tools/credentialUses.js +112 -0
  129. package/dist/lib/tools/mcpToolFactory.d.ts +25 -2
  130. package/dist/lib/tools/mcpToolFactory.js +179 -21
  131. package/dist/lib/tools/oauthConsent.js +8 -1
  132. package/dist/lib/tools/oauthHosts.d.ts +73 -0
  133. package/dist/lib/tools/oauthHosts.js +154 -0
  134. package/dist/lib/tools/oauthTools.d.ts +242 -0
  135. package/dist/lib/tools/oauthTools.js +447 -0
  136. package/dist/lib/tools/openapiTools.d.ts +11 -6
  137. package/dist/lib/tools/openapiTools.js +88 -38
  138. package/dist/lib/tools/taskTools.d.ts +61 -6
  139. package/dist/lib/tools/taskTools.js +145 -21
  140. package/dist/lib/workflow/turn.d.ts +22 -3
  141. package/dist/lib/workflow/turn.js +151 -31
  142. package/dist/lib/workflow.d.ts +14 -7
  143. package/dist/lib/workflow.js +14 -7
  144. package/dist/lib/workflowConfig.d.ts +6 -0
  145. package/dist/scripts/a2a_server.d.ts +36 -0
  146. package/dist/scripts/a2a_server.js +131 -1
  147. package/dist/scripts/assistant_worker.d.ts +12 -0
  148. package/dist/scripts/assistant_worker.js +81 -22
  149. package/dist/scripts/doctor.d.ts +7 -2
  150. package/dist/scripts/doctor.js +9 -4
  151. package/dist/scripts/setup.d.ts +23 -0
  152. package/dist/scripts/setup.js +154 -0
  153. package/dist/scripts/skills_install.d.ts +3 -1
  154. package/dist/scripts/skills_install.js +17 -2
  155. package/dist/scripts/yaml_codemod.d.ts +62 -0
  156. package/dist/scripts/yaml_codemod.js +652 -0
  157. package/package.json +8 -2
  158. package/skills/README.md +11 -2
  159. package/skills/briefs/_shared.md +1 -1
  160. package/skills/briefs/melchizedek-author.md +7 -5
  161. package/skills/briefs/melchizedek-models.md +7 -4
  162. package/skills/briefs/melchizedek.md +1 -0
  163. package/skills/briefs/skills-readme.md +3 -1
  164. package/skills/melchizedek/SKILL.md +1 -0
  165. package/skills/melchizedek-author/SKILL.md +7 -4
  166. package/skills/melchizedek-author/assets/minimal.yaml +13 -13
  167. package/skills/melchizedek-models/SKILL.md +11 -4
  168. package/skills/melchizedek-onboard/SKILL.md +49 -0
  169. package/skills/melchizedek-onboard-cloud/SKILL.md +38 -0
  170. package/skills/melchizedek-onboard-keys/SKILL.md +35 -0
  171. package/skills/melchizedek-onboard-local/SKILL.md +41 -0
  172. package/skills/melchizedek-onboard-serve/SKILL.md +43 -0
@@ -10,7 +10,9 @@
10
10
  * a list of records with a status — so both live in one store:
11
11
  *
12
12
  * kind: todo the user's own tasks. open → done | cancelled.
13
- * kind: background a job for the worker. queued → running → done | failed.
13
+ * kind: background a job for the worker. queued → running → done | failed,
14
+ * and cancelled from queued or running (a running job's
15
+ * worker sees it at its next renewal and stops).
14
16
  *
15
17
  * The tools only WRITE the queue. They never run a job: a tool that
16
18
  * orchestrates agents is exactly what the tool-contract doctrine refuses
@@ -28,11 +30,18 @@
28
30
  * gitignored). Every operation re-reads the file, so the chat process
29
31
  * and the worker see each other's writes. It is SINGLE-USER: every
30
32
  * caller shares one list. Run ONE worker: the claim is a
31
- * read-modify-write, not a lock.
33
+ * read-modify-write, not a lock. A running job's step checkpoints
34
+ * (ADR 0113) sit in a sidecar, <store>.checkpoints.json, keyed by id;
35
+ * every write of the store drops those of jobs no longer running or
36
+ * queued. A checkpoint above the cap (TaskBackendOptions.
37
+ * checkpointMaxBytes, default 5 MiB of JSON) is not saved, on either
38
+ * store: the job keeps its previous one and one line names the job id
39
+ * and the sizes.
32
40
  * - With Postgres (postgresStorage().taskQueue, migration 0009) each
33
41
  * caller has its own list, scoped by the caller the tool call carries
34
42
  * (the A2A server's scope key), and any number of workers claim jobs
35
- * with FOR UPDATE SKIP LOCKED under a renewed lease (ADR 0021).
43
+ * with FOR UPDATE SKIP LOCKED under a renewed lease (ADR 0021). The
44
+ * checkpoint is a column beside the record (migration 0014).
36
45
  *
37
46
  * SECURITY:
38
47
  * On the file store every caller of a shared endpoint shares one list: do
@@ -95,8 +104,13 @@ export interface TaskBackend {
95
104
  mutate<T>(owner: string, change: (store: TaskStore) => T): Promise<T>;
96
105
  /** The oldest queued background job of any owner, now running and leased. */
97
106
  claimNext(worker: WorkerLease): Promise<OwnedTask | null>;
98
- /** Keeps a claimed job leased while it runs. */
99
- renew(worker: WorkerLease, job: OwnedTask): Promise<void>;
107
+ /**
108
+ * Keeps a claimed job leased while it runs. Resolves false when the claim
109
+ * is gone (the job left `running`, for instance cancelled with
110
+ * task_update, or another worker holds it): the worker must stop the run.
111
+ * A backend that resolves void makes no such claim.
112
+ */
113
+ renew(worker: WorkerLease, job: OwnedTask): Promise<void | boolean>;
100
114
  /** Records a job's outcome; a record no longer running is left alone. */
101
115
  finish(job: OwnedTask, outcome: JobOutcome): Promise<void>;
102
116
  /** Jobs whose worker died: queued again, or failed after MAX_ATTEMPTS. */
@@ -104,7 +118,45 @@ export interface TaskBackend {
104
118
  requeued: string[];
105
119
  failed: string[];
106
120
  }>;
121
+ /**
122
+ * Durable runs (ADR 0113): stores the run's latest step checkpoint beside
123
+ * the job. Resolves false when the job is no longer this worker's running
124
+ * job (cancelled, finished, or re-leased): the run must stop.
125
+ *
126
+ * A checkpoint lives only while its job is `running` or `queued` (a job
127
+ * re-queued after an interruption keeps it, so the next claim resumes);
128
+ * any other status drops it, and so does a pruned or deleted record. It is
129
+ * kept beside the record, never in it, so task_get never shows it.
130
+ */
131
+ saveCheckpoint?(worker: WorkerLease, job: OwnedTask, checkpoint: object): Promise<boolean>;
132
+ /** The checkpoint a re-queued job left, or null. */
133
+ loadCheckpoint?(job: OwnedTask): Promise<object | null>;
107
134
  }
135
+ /**
136
+ * The largest checkpoint a backend stores by default: 5 MiB of serialized
137
+ * JSON (ADR 0113). A save above the cap is skipped and the job's previous
138
+ * checkpoint kept, so a huge run resumes from an older step boundary instead
139
+ * of failing or bloating the table.
140
+ */
141
+ export declare const DEFAULT_CHECKPOINT_MAX_BYTES: number;
142
+ /** The worker's variable that moves the cap, in bytes (scripts/assistant_worker.ts). */
143
+ export declare const CHECKPOINT_MAX_BYTES_ENV = "MELCHIZEDEK_CHECKPOINT_MAX_BYTES";
144
+ /** How a backend stores checkpoints (fileTaskBackendWith, postgresTaskBackend). */
145
+ export interface TaskBackendOptions {
146
+ /** The largest checkpoint stored, in bytes of serialized JSON. Default DEFAULT_CHECKPOINT_MAX_BYTES; a value that is not a positive integer is the default. */
147
+ checkpointMaxBytes?: number;
148
+ /** Where a skipped save is reported: one line naming the job id and the sizes, never the checkpoint. Default console.warn. */
149
+ log?: (line: string) => void;
150
+ }
151
+ /** The cap a variable's value sets: a positive whole number of bytes, else the default. */
152
+ export declare function checkpointMaxBytesSetting(raw: string | undefined): number;
153
+ /**
154
+ * The checkpoint serialized once, as the backend stores it, or null when it
155
+ * is above the cap: then one line is logged with the job id and the sizes
156
+ * only (the checkpoint carries the user's text and tool results), and the
157
+ * caller keeps the previous checkpoint.
158
+ */
159
+ export declare function checkpointJson(job: Pick<OwnedTask, 'id'>, checkpoint: object, options?: TaskBackendOptions): string | null;
108
160
  /** Records kept at most; finished ones are pruned oldest-first to make room. */
109
161
  export declare const MAX_TASKS = 500;
110
162
  /** A worker result is stored up to this length, then cut with a marker. */
@@ -112,6 +164,7 @@ export declare const MAX_RESULT_CHARS = 20000;
112
164
  /** A job interrupted this many times is marked failed instead of re-queued. */
113
165
  export declare const MAX_ATTEMPTS = 2;
114
166
  export declare function taskStorePath(): string;
167
+ export declare function taskCheckpointPath(storePath?: string): string;
115
168
  /**
116
169
  * Called once at worker start: a record left `running` means a worker died
117
170
  * mid-job. It goes back to the queue, or fails once it has been tried
@@ -129,7 +182,9 @@ export declare function finishJob(id: string, outcome: JobOutcome): void;
129
182
  export declare function applyFinish(store: TaskStore, id: string, outcome: JobOutcome): void;
130
183
  /** The interrupted-job rule for one record: back to the queue, or failed. */
131
184
  export declare function applyInterrupted(t: TaskRecord): 'requeued' | 'failed';
132
- /** The single-user JSON file (the default backend). */
185
+ /** The single-user JSON file, its checkpoints capped by `options` (the default backend is fileTaskBackend). */
186
+ export declare function fileTaskBackendWith(options?: TaskBackendOptions): TaskBackend;
187
+ /** The single-user JSON file (the default backend), checkpoints capped at DEFAULT_CHECKPOINT_MAX_BYTES. */
133
188
  export declare const fileTaskBackend: TaskBackend;
134
189
  /** Plugs in where tasks live (postgresStorage().taskQueue); default the JSON file. */
135
190
  export declare function setTaskBackend(backend: TaskBackend): void;
@@ -10,7 +10,9 @@
10
10
  * a list of records with a status — so both live in one store:
11
11
  *
12
12
  * kind: todo the user's own tasks. open → done | cancelled.
13
- * kind: background a job for the worker. queued → running → done | failed.
13
+ * kind: background a job for the worker. queued → running → done | failed,
14
+ * and cancelled from queued or running (a running job's
15
+ * worker sees it at its next renewal and stops).
14
16
  *
15
17
  * The tools only WRITE the queue. They never run a job: a tool that
16
18
  * orchestrates agents is exactly what the tool-contract doctrine refuses
@@ -28,11 +30,18 @@
28
30
  * gitignored). Every operation re-reads the file, so the chat process
29
31
  * and the worker see each other's writes. It is SINGLE-USER: every
30
32
  * caller shares one list. Run ONE worker: the claim is a
31
- * read-modify-write, not a lock.
33
+ * read-modify-write, not a lock. A running job's step checkpoints
34
+ * (ADR 0113) sit in a sidecar, <store>.checkpoints.json, keyed by id;
35
+ * every write of the store drops those of jobs no longer running or
36
+ * queued. A checkpoint above the cap (TaskBackendOptions.
37
+ * checkpointMaxBytes, default 5 MiB of JSON) is not saved, on either
38
+ * store: the job keeps its previous one and one line names the job id
39
+ * and the sizes.
32
40
  * - With Postgres (postgresStorage().taskQueue, migration 0009) each
33
41
  * caller has its own list, scoped by the caller the tool call carries
34
42
  * (the A2A server's scope key), and any number of workers claim jobs
35
- * with FOR UPDATE SKIP LOCKED under a renewed lease (ADR 0021).
43
+ * with FOR UPDATE SKIP LOCKED under a renewed lease (ADR 0021). The
44
+ * checkpoint is a column beside the record (migration 0014).
36
45
  *
37
46
  * SECURITY:
38
47
  * On the file store every caller of a shared endpoint shares one list: do
@@ -48,6 +57,43 @@ import { existsSync, mkdirSync, readFileSync, renameSync, writeFileSync } from '
48
57
  import { dirname, join, resolve } from 'node:path';
49
58
  import { z } from 'zod';
50
59
  import { defineTool } from './toolContract.js';
60
+ /**
61
+ * The largest checkpoint a backend stores by default: 5 MiB of serialized
62
+ * JSON (ADR 0113). A save above the cap is skipped and the job's previous
63
+ * checkpoint kept, so a huge run resumes from an older step boundary instead
64
+ * of failing or bloating the table.
65
+ */
66
+ export const DEFAULT_CHECKPOINT_MAX_BYTES = 5 * 1024 * 1024;
67
+ /** The worker's variable that moves the cap, in bytes (scripts/assistant_worker.ts). */
68
+ export const CHECKPOINT_MAX_BYTES_ENV = 'MELCHIZEDEK_CHECKPOINT_MAX_BYTES';
69
+ const validCap = (n) => typeof n === 'number' && Number.isSafeInteger(n) && n > 0;
70
+ /** The cap a variable's value sets: a positive whole number of bytes, else the default. */
71
+ export function checkpointMaxBytesSetting(raw) {
72
+ const value = raw?.trim() ?? '';
73
+ if (!/^\d+$/.test(value))
74
+ return DEFAULT_CHECKPOINT_MAX_BYTES;
75
+ const n = Number(value);
76
+ return validCap(n) ? n : DEFAULT_CHECKPOINT_MAX_BYTES;
77
+ }
78
+ /**
79
+ * The checkpoint serialized once, as the backend stores it, or null when it
80
+ * is above the cap: then one line is logged with the job id and the sizes
81
+ * only (the checkpoint carries the user's text and tool results), and the
82
+ * caller keeps the previous checkpoint.
83
+ */
84
+ export function checkpointJson(job, checkpoint, options = {}) {
85
+ const cap = validCap(options.checkpointMaxBytes) ? options.checkpointMaxBytes : DEFAULT_CHECKPOINT_MAX_BYTES;
86
+ const json = JSON.stringify(checkpoint);
87
+ // One UTF-16 unit is at most three UTF-8 bytes: a short string is under the cap without counting.
88
+ if (json.length * 3 <= cap)
89
+ return json;
90
+ const bytes = Buffer.byteLength(json, 'utf8');
91
+ if (bytes <= cap)
92
+ return json;
93
+ const log = options.log ?? ((line) => console.warn(`[tasks] ${line}`));
94
+ log(`checkpoint for job ${job.id} not saved: ${bytes} bytes is over the ${cap}-byte cap; the previous checkpoint is kept`);
95
+ return null;
96
+ }
51
97
  /** Records kept at most; finished ones are pruned oldest-first to make room. */
52
98
  export const MAX_TASKS = 500;
53
99
  /** A worker result is stored up to this length, then cut with a marker. */
@@ -73,18 +119,62 @@ function readStore(path = taskStorePath()) {
73
119
  }
74
120
  return { version: 1, next_id: parsed.next_id, tasks: parsed.tasks };
75
121
  }
76
- function writeStore(store, path = taskStorePath()) {
122
+ /** Atomic write: temp file + rename, so a reader never sees half a file. */
123
+ function writeText(text, path) {
77
124
  mkdirSync(dirname(path), { recursive: true });
78
125
  const tmp = `${path}.${process.pid}.tmp`;
79
- writeFileSync(tmp, `${JSON.stringify(store, null, 2)}\n`);
126
+ writeFileSync(tmp, text);
80
127
  renameSync(tmp, path);
81
128
  }
129
+ /** Atomic JSON write. */
130
+ function writeJson(value, path) {
131
+ writeText(`${JSON.stringify(value, null, 2)}\n`, path);
132
+ }
133
+ function writeStore(store, path = taskStorePath()) {
134
+ writeJson(store, path);
135
+ }
136
+ /** Statuses a job's checkpoint survives in. */
137
+ const CHECKPOINTED = ['running', 'queued'];
138
+ export function taskCheckpointPath(storePath = taskStorePath()) {
139
+ return `${storePath}.checkpoints.json`;
140
+ }
141
+ /** The sidecar, or empty when missing or unreadable (a checkpoint is an optimisation, never the record). */
142
+ function readCheckpoints(storePath) {
143
+ const path = taskCheckpointPath(storePath);
144
+ if (!existsSync(path))
145
+ return {};
146
+ try {
147
+ const parsed = JSON.parse(readFileSync(path, 'utf8'));
148
+ return parsed && typeof parsed === 'object' && !Array.isArray(parsed) ? parsed : {};
149
+ }
150
+ catch {
151
+ return {};
152
+ }
153
+ }
154
+ const keepsCheckpoint = (t) => !!t && t.kind === 'background' && CHECKPOINTED.includes(t.status);
155
+ /** Drops sidecar entries whose job is gone or no longer running/queued. */
156
+ function pruneCheckpoints(store, storePath) {
157
+ if (!existsSync(taskCheckpointPath(storePath)))
158
+ return;
159
+ const entries = readCheckpoints(storePath);
160
+ const byId = new Map(store.tasks.map((t) => [t.id, t]));
161
+ let changed = false;
162
+ for (const id of Object.keys(entries)) {
163
+ if (!keepsCheckpoint(byId.get(id))) {
164
+ delete entries[id];
165
+ changed = true;
166
+ }
167
+ }
168
+ if (changed)
169
+ writeJson(entries, taskCheckpointPath(storePath));
170
+ }
82
171
  /** Read, change, write — one synchronous step, so no await splits it. */
83
172
  function mutate(change) {
84
173
  const path = taskStorePath();
85
174
  const store = readStore(path);
86
175
  const out = change(store);
87
176
  writeStore(store, path);
177
+ pruneCheckpoints(store, path);
88
178
  return out;
89
179
  }
90
180
  const now = () => new Date().toISOString();
@@ -204,18 +294,51 @@ export function applyInterrupted(t) {
204
294
  t.status = 'queued';
205
295
  return 'requeued';
206
296
  }
207
- /** The single-user JSON file (the default backend). */
208
- export const fileTaskBackend = {
209
- read: async () => readStore(),
210
- mutate: async (_owner, change) => mutate(change),
211
- claimNext: async () => {
212
- const job = claimNextJob();
213
- return job ? { ...job, owner: '' } : null;
214
- },
215
- renew: async () => { },
216
- finish: async (job, outcome) => finishJob(job.id, outcome),
217
- recover: async () => recoverInterruptedJobs(),
218
- };
297
+ /**
298
+ * The sidecar with one job's checkpoint set from its serialized JSON, which
299
+ * is spliced in as it is rather than parsed and serialized again. The other
300
+ * entries are written compact; both shapes read back the same.
301
+ */
302
+ function writeCheckpointEntry(storePath, id, json) {
303
+ const { [id]: _replaced, ...rest } = readCheckpoints(storePath);
304
+ const others = JSON.stringify(rest);
305
+ const entry = `${JSON.stringify(id)}:{"checkpoint":${json},"at":${JSON.stringify(now())}}`;
306
+ writeText(`${others === '{}' ? `{${entry}}` : `${others.slice(0, -1)},${entry}}`}\n`, taskCheckpointPath(storePath));
307
+ }
308
+ /** The single-user JSON file, its checkpoints capped by `options` (the default backend is fileTaskBackend). */
309
+ export function fileTaskBackendWith(options = {}) {
310
+ return {
311
+ read: async () => readStore(),
312
+ mutate: async (_owner, change) => mutate(change),
313
+ claimNext: async () => {
314
+ const job = claimNextJob();
315
+ return job ? { ...job, owner: '' } : null;
316
+ },
317
+ // No leases on the single-user file: the claim holds while the record is running.
318
+ renew: async (_worker, job) => readStore().tasks.find((t) => t.id === job.id)?.status === 'running',
319
+ finish: async (job, outcome) => finishJob(job.id, outcome),
320
+ recover: async () => recoverInterruptedJobs(),
321
+ saveCheckpoint: async (_worker, job, checkpoint) => {
322
+ const path = taskStorePath();
323
+ const record = readStore(path).tasks.find((t) => t.id === job.id);
324
+ if (!record || record.kind !== 'background' || record.status !== 'running')
325
+ return false;
326
+ // Over the cap: nothing written, the previous checkpoint kept, the claim still held.
327
+ const json = checkpointJson(job, checkpoint, options);
328
+ if (json !== null)
329
+ writeCheckpointEntry(path, job.id, json);
330
+ return true;
331
+ },
332
+ loadCheckpoint: async (job) => {
333
+ const path = taskStorePath();
334
+ if (!keepsCheckpoint(readStore(path).tasks.find((t) => t.id === job.id)))
335
+ return null;
336
+ return readCheckpoints(path)[job.id]?.checkpoint ?? null;
337
+ },
338
+ };
339
+ }
340
+ /** The single-user JSON file (the default backend), checkpoints capped at DEFAULT_CHECKPOINT_MAX_BYTES. */
341
+ export const fileTaskBackend = fileTaskBackendWith();
219
342
  let activeBackend = fileTaskBackend;
220
343
  /** Plugs in where tasks live (postgresStorage().taskQueue); default the JSON file. */
221
344
  export function setTaskBackend(backend) {
@@ -318,7 +441,8 @@ export const taskGetContract = defineTool({
318
441
  export const taskUpdateContract = defineTool({
319
442
  name: 'task_update',
320
443
  description: 'Change a task: mark it done, cancel it, reopen it, or edit its title, notes, or due date. ' +
321
- 'For a background job, "cancelled" withdraws it before it runs and "queued" retries a failed or cancelled job. ' +
444
+ 'For a background job, "cancelled" withdraws it before it runs, or stops it if it is running now ' +
445
+ '(the worker stops at its next step and keeps no result); "queued" retries a failed or cancelled job. ' +
322
446
  'Call task_list first if you do not have the id.',
323
447
  schema: z.object({
324
448
  id: taskId,
@@ -340,9 +464,9 @@ export const taskUpdateContract = defineTool({
340
464
  return `Error: a ${task.kind} task cannot be set to "${status}". Allowed: ${allowed[task.kind].join(', ')}.`;
341
465
  }
342
466
  if (task.kind === 'background') {
343
- if (task.status === 'running') {
344
- return `Error: ${id} is running now. Wait for it to finish, then read it with task_get.`;
345
- }
467
+ // Cancelling a running job is allowed: its worker's next lease
468
+ // renewal or checkpoint sees the record left `running` and stops
469
+ // the run, and the worker's outcome is then left alone (applyFinish).
346
470
  if (status === 'queued' && !['failed', 'cancelled'].includes(task.status)) {
347
471
  return `Error: only a failed or cancelled job can be queued again; ${id} is ${task.status}.`;
348
472
  }
@@ -63,6 +63,7 @@
63
63
  */
64
64
  import type { ModelAdapter } from '../models/contract.ts';
65
65
  import type { TurnEvent } from '../runtime/events.ts';
66
+ import type { WorkflowSubagentEnd } from '../runtime/native/delegate.ts';
66
67
  import type { MemoryService } from '../runtime/memoryService.ts';
67
68
  import type { NativeAgent } from '../runtime/native/request.ts';
68
69
  import type { SelfCorrection } from '../runtime/native/selfCorrection.ts';
@@ -93,15 +94,33 @@ export interface NativeWorkflowParams {
93
94
  selfCorrection?: SelfCorrection;
94
95
  /** The run's tool credentials, pinned to its app (ADR 0072). */
95
96
  credentials?: Pick<CredentialStore, 'get'>;
96
- /** The agent nodes that are a nested workflow syndicate, by YAML name (ADR 0106). */
97
- workflows?: ReadonlyMap<string, NestedWorkflow>;
97
+ /** The agent nodes that are a nested workflow syndicate (ADR 0106) or a nested dispatch syndicate (ADR 0120), by YAML name. */
98
+ workflows?: ReadonlyMap<string, NestedWorkflow | NestedRun>;
98
99
  }
99
100
  /** A workflow syndicate run as a node of another (ADR 0106): its graph, its agents, its tool nodes' lookup, and its own nested nodes. */
100
101
  export interface NestedWorkflow {
101
102
  graph: WorkflowGraph;
102
103
  agents: ReadonlyMap<string, NativeAgent>;
103
104
  resolveTool: (name: string) => unknown;
104
- workflows?: ReadonlyMap<string, NestedWorkflow>;
105
+ workflows?: ReadonlyMap<string, NestedWorkflow | NestedRun>;
106
+ }
107
+ /** Where a node's own run goes: its child session, the message, the turn's signal. */
108
+ export interface NestedRunParams {
109
+ sessions: SessionService;
110
+ appName: string;
111
+ userId: string;
112
+ sessionId: string;
113
+ userParts: unknown[];
114
+ signal?: AbortSignal;
115
+ }
116
+ /**
117
+ * A node that runs as something other than a graph (ADR 0120): a nested
118
+ * dispatch syndicate, its own turn (lib/runtime/native/delegate.ts
119
+ * nestedDispatchWalk). It runs where a nested workflow's walk runs, on the
120
+ * same child session, and ends as a walk ends.
121
+ */
122
+ export interface NestedRun {
123
+ walk(run: NestedRunParams): AsyncGenerator<TurnEvent, WorkflowSubagentEnd | undefined>;
105
124
  }
106
125
  /** How the walk ended. */
107
126
  export interface NativeWorkflowEnd {
@@ -62,7 +62,10 @@
62
62
  * carries, never instructions this module acts on.
63
63
  */
64
64
  import { randomUUID } from 'node:crypto';
65
- import { createTurnEvent } from '../runtime/events.js';
65
+ import { APPROVAL_REQUEST } from '../runtime/approvals.js';
66
+ import { createTurnEvent, getFunctionCalls } from '../runtime/events.js';
67
+ import { entrySession } from '../runtime/native/delegate.js';
68
+ import { INPUT_REQUEST } from '../workflowConfig.js';
66
69
  import { traceNodeExecution, traceToolNodeCall, traceWorkflowInvocation } from '../runtime/native/telemetry.js';
67
70
  import { TEMP_STATE_PREFIX } from '../runtime/sessions.js';
68
71
  import { NodeStoppedError, agentNodeRuntime, nodeInputContent } from './agentNode.js';
@@ -78,41 +81,150 @@ function lastText(event) {
78
81
  .filter((t) => t)
79
82
  .join('\n');
80
83
  }
84
+ /** The requests a walk's interrupts are, as a node raises them: an approval request or an input request. */
85
+ const RAISED = new Set([APPROVAL_REQUEST, INPUT_REQUEST]);
86
+ /** The latest request call, by id, among `ids` that `events` hold (an agent's or a node's, never the user's). */
87
+ function requestCalls(events, ids, author) {
88
+ const wanted = new Set(ids);
89
+ const calls = new Map();
90
+ for (let i = events.length - 1; i >= 0 && calls.size < wanted.size; i--) {
91
+ const event = events[i];
92
+ if (!event.author || event.author === 'user' || (author !== undefined && event.author !== author))
93
+ continue;
94
+ for (const call of getFunctionCalls(event)) {
95
+ if (call.id && call.name && RAISED.has(call.name) && wanted.has(call.id) && !calls.has(call.id))
96
+ calls.set(call.id, call);
97
+ }
98
+ }
99
+ return calls;
100
+ }
101
+ /**
102
+ * Whether a resumed walk's answer to `call` answers it: a decision for an
103
+ * approval request (a ToolConfirmation, or ADK's `{ response: <json> }`
104
+ * form, as lib/workflow/agentNode.ts reads one), any answer for an input
105
+ * request.
106
+ */
107
+ function answers(call, value) {
108
+ if (value === undefined || !call)
109
+ return false;
110
+ if (call.name !== APPROVAL_REQUEST)
111
+ return true;
112
+ if (typeof value !== 'object' || value === null || Array.isArray(value))
113
+ return false;
114
+ const answer = value;
115
+ return typeof answer.confirmed === 'boolean' || typeof answer.response === 'string';
116
+ }
117
+ /** The part that carries `value` down to the nested walk, as the answer to `call`: its explicit reply, under the request's name. */
118
+ function answerPart(call, value) {
119
+ const response = call.name === APPROVAL_REQUEST ? value : { result: value };
120
+ return { functionResponse: { id: call.id, name: call.name, response } };
121
+ }
122
+ /**
123
+ * A node whose answered interrupts an earlier resume already carried down
124
+ * (another node's answer was the one since): the output it stored after
125
+ * the last event that raised them, else undefined.
126
+ */
127
+ function outputAfterRaise(events, path, ids) {
128
+ const open = new Set(ids);
129
+ let start = -1;
130
+ events.forEach((event, i) => {
131
+ if (event.nodeInfo?.path === path && (event.longRunningToolIds ?? []).some((id) => open.has(id)))
132
+ start = i;
133
+ });
134
+ if (start === -1)
135
+ return undefined;
136
+ for (let i = events.length - 1; i > start; i--) {
137
+ const event = events[i];
138
+ if (event.nodeInfo?.path === path && event.output !== undefined && !event.partial)
139
+ return event.output;
140
+ }
141
+ return undefined;
142
+ }
143
+ /**
144
+ * The node's pause on its caller's walk (ADR 0119): every request its own
145
+ * walk waits on, raised again as one event of the node's (a copy of each
146
+ * call, stamped with the node's path, the ids in `longRunningToolIds`, the
147
+ * node's input recorded for the resume), so the caller's walk pauses on
148
+ * the same ids, its resume reruns the node with their answers, and the
149
+ * turn finds the request (lib/runtime/native/interrupts.ts follows it down
150
+ * by id to the node that asked). Carries the walk's state writes so far.
151
+ */
152
+ function raiseAgain(name, run, invocationId, calls, stateDelta) {
153
+ const event = createTurnEvent({
154
+ author: name,
155
+ invocationId,
156
+ content: { role: 'model', parts: calls.map((call) => ({ functionCall: structuredClone(call) })) },
157
+ ...(Object.keys(stateDelta).length ? { actions: { stateDelta } } : {}),
158
+ });
159
+ event.longRunningToolIds = calls.map((call) => call.id);
160
+ enrichNodeEvent(event, run, { invocationId });
161
+ event.actions = { ...event.actions, agentState: { ...(event.actions?.agentState ?? {}), input: run.input } };
162
+ return event;
163
+ }
81
164
  /**
82
165
  * Runs `workflow` as the node `run` names (ADR 0106), as a delegated nested
83
166
  * workflow runs (ADR 0098): the whole graph walked on the child session
84
- * under the node's name (created from the caller's state the first time,
85
- * `temp:` keys dropped, and kept), the node's input as its message, its last
86
- * yielded event's text the node's output. The caller stores one event for
87
- * the node, carrying that output and the walk's state writes, so a resumed
88
- * walk completes the node from it. A walk that ends paused is refused by
89
- * name: a pause cannot reach the caller's walk yet (WS6-2).
167
+ * filed under the walk's app name and the node's behind its kind
168
+ * (`<walk's app>/node:<node>`, entryAppName, ADR 0119, ADR 0120; the one
169
+ * under an older key is continued when it ran there before), created from the caller's state the first time, `temp:` keys
170
+ * dropped, and kept; the node's input as its message, its last yielded
171
+ * event's text the node's output. The caller stores one event for the
172
+ * node, carrying that output and the walk's state writes, so a resumed
173
+ * walk completes the node from it.
174
+ *
175
+ * A walk that ends paused (an ask_user node, a gated agent node, a node of
176
+ * its own that is a nested workflow) pauses the node on the same ids
177
+ * (raiseAgain), and the caller's walk with it. Rerun on the resume, the
178
+ * node walks its graph again on the answers among its ids, each as its
179
+ * explicit reply (an approval decision as it came, an input request's
180
+ * answer as `{ result }`), which the nested walk's resume reads as a
181
+ * top-level one's; with none of them answered it waits again without
182
+ * walking; once an earlier resume finished it, its stored output is the
183
+ * node's.
90
184
  */
91
185
  async function runWorkflowNode(name, workflow, run, params, parent) {
92
- const { sessions, userId, sessionId } = params;
93
- const key = { appName: name, userId, sessionId };
186
+ const { sessions } = params;
187
+ const resumed = run.resumedInterruptIds ?? [];
188
+ let userParts;
189
+ if (resumed.length > 0) {
190
+ const raised = requestCalls(parent.session.events, resumed, name);
191
+ const answered = resumed.filter((id) => answers(raised.get(id), run.resumeInputs?.[id]));
192
+ if (answered.length === 0) {
193
+ parent.store(raiseAgain(name, run, parent.invocationId, resumed.map((id) => raised.get(id)).filter((c) => !!c), {}));
194
+ return { interruptIds: [...resumed] };
195
+ }
196
+ const finished = answered.length === resumed.length ? outputAfterRaise(parent.session.events, run.path, resumed) : undefined;
197
+ if (finished !== undefined)
198
+ return { output: finished };
199
+ userParts = answered.map((id) => answerPart(raised.get(id), run.resumeInputs[id]));
200
+ }
201
+ else {
202
+ const input = run.input === undefined || run.input === null ? parent.userContent : nodeInputContent(run.input);
203
+ userParts = input.parts ?? [];
204
+ }
94
205
  const state = Object.fromEntries(Object.entries(parent.session.state ?? {}).filter(([k]) => !k.startsWith(TEMP_STATE_PREFIX)));
95
- if (!(await sessions.get(key)))
96
- await sessions.create({ ...key, state });
97
- const input = run.input === undefined || run.input === null ? parent.userContent : nodeInputContent(run.input);
98
- const walk = runNativeWorkflow({
99
- graph: workflow.graph,
100
- agents: workflow.agents,
101
- resolveTool: workflow.resolveTool,
102
- ...(workflow.workflows ? { workflows: workflow.workflows } : {}),
103
- sessions,
104
- appName: name,
105
- userId,
106
- sessionId,
107
- userParts: input.parts ?? [],
108
- adapterFor: params.adapterFor,
109
- signal: run.signal,
110
- stream: false,
111
- ...(params.memory ? { memory: params.memory } : {}),
112
- ...(params.log ? { log: params.log } : {}),
113
- ...(params.selfCorrection ? { selfCorrection: params.selfCorrection } : {}),
114
- ...(params.credentials ? { credentials: params.credentials } : {}),
115
- });
206
+ const child = await entrySession(sessions, { appName: parent.session.appName, userId: parent.session.userId, sessionId: parent.session.id, events: parent.session.events }, name, 'node', state, resumed.length > 0);
207
+ // A nested dispatch syndicate runs its own turn there (ADR 0120); a nested workflow walks its graph.
208
+ const walk = 'walk' in workflow
209
+ ? workflow.walk({ sessions, appName: child.appName, userId: child.userId, sessionId: child.id, userParts, ...(run.signal ? { signal: run.signal } : {}) })
210
+ : runNativeWorkflow({
211
+ graph: workflow.graph,
212
+ agents: workflow.agents,
213
+ resolveTool: workflow.resolveTool,
214
+ ...(workflow.workflows ? { workflows: workflow.workflows } : {}),
215
+ sessions,
216
+ appName: child.appName,
217
+ userId: child.userId,
218
+ sessionId: child.id,
219
+ userParts,
220
+ adapterFor: params.adapterFor,
221
+ signal: run.signal,
222
+ stream: false,
223
+ ...(params.memory ? { memory: params.memory } : {}),
224
+ ...(params.log ? { log: params.log } : {}),
225
+ ...(params.selfCorrection ? { selfCorrection: params.selfCorrection } : {}),
226
+ ...(params.credentials ? { credentials: params.credentials } : {}),
227
+ });
116
228
  let last;
117
229
  const stateDelta = {};
118
230
  let end;
@@ -133,7 +245,15 @@ async function runWorkflowNode(name, workflow, run, params, parent) {
133
245
  throw new NodeStoppedError(name, undefined);
134
246
  const paused = end.run?.interruptIds ?? [];
135
247
  if (paused.length > 0) {
136
- throw new Error(`Node '${name}': the workflow it runs paused on ${paused.join(', ')}; a pause inside a workflow run as a node cannot reach the walk yet.`);
248
+ const walked = (await sessions.get({ appName: child.appName, userId: child.userId, sessionId: child.id }))?.events ?? [];
249
+ // A nested dispatch syndicate names the calls it waits on: they may sit in a route's delegation below its conversation (ADR 0120).
250
+ const named = 'requests' in end && end.requests ? new Map(end.requests.map((call) => [call.id, call])) : undefined;
251
+ const raised = named ?? requestCalls(walked, paused);
252
+ const missing = paused.filter((id) => !raised.has(id));
253
+ if (missing.length > 0)
254
+ throw new Error(`Node '${name}': the ${'walk' in workflow ? 'syndicate' : 'workflow'} it runs paused on ${missing.join(', ')}, which is not an approval request or a question.`);
255
+ parent.store(raiseAgain(name, run, parent.invocationId, paused.map((id) => raised.get(id)), stateDelta));
256
+ return { interruptIds: [...paused] };
137
257
  }
138
258
  const output = lastText(last);
139
259
  const event = createTurnEvent({
@@ -41,16 +41,23 @@
41
41
  * A delegated `yaml_reference` to a workflow syndicate is the whole graph,
42
42
  * walked under the subagent entry's name and description on a child
43
43
  * session (lib/compileNative.ts workflowSubagentOf); its answer is the
44
- * graph's last event's text. A nested workflow with an
45
- * `ask_user` node is refused by name (lib/compile.ts compileWorkflowSpec): a
46
- * pause inside a tool call cannot reach the caller (ADR 0028).
44
+ * graph's last event's text. A walk that pauses (an `ask_user` node, a
45
+ * gated agent node) leaves the call open, and the turn ends input-required
46
+ * with the agent path down to the node; the answer walks the graph again
47
+ * (lib/runtime/native/delegate.ts resumeWorkflowSubagent, ADR 0111).
47
48
  *
48
49
  * ── As a dispatch route or a workflow node (ADR 0106) ────────────────────
49
50
  * A `yaml_reference` to a workflow syndicate runs its whole graph there too,
50
- * on the child session filed under the entry's name, as a subagent does. A
51
- * route's answer is what the graph would answer as its own syndicate
52
- * (lib/runtime/syndicateTurn.ts); a node's is the nested walk's
53
- * (lib/workflow/turn.ts). A map over one is refused (lib/compile.ts).
51
+ * on the child session filed under the agent path and its kind
52
+ * (`<app>/route:<entry>`, `<walk's app>/node:<entry>`, ADR 0119, ADR 0120),
53
+ * as a subagent does. A route's answer is what the graph would
54
+ * answer as its own syndicate (lib/runtime/syndicateTurn.ts); a node's is
55
+ * the nested walk's (lib/workflow/turn.ts). A map over one is refused
56
+ * (lib/compile.ts). Its pauses (an `ask_user` node, a gate) reach the turn
57
+ * with the path from the entry down to the node that asked: a route's
58
+ * through the route's pause record in the conversation, a node's through
59
+ * its own pause on the caller's walk; the answer walks the nested graph
60
+ * again (ADR 0119).
54
61
  *
55
62
  * ── Approval gates (ADR 0098, ADR 0106) ──────────────────────────────────
56
63
  * A tool in a node agent's `require_approval`, and a skill script