akm-cli 0.9.10 → 0.9.12
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +226 -0
- package/STABILITY.md +28 -15
- package/dist/assets/hints/cli-hints-full.md +1 -1
- package/dist/assets/improve-strategies/consolidate.json +1 -1
- package/dist/assets/improve-strategies/default.json +1 -1
- package/dist/assets/improve-strategies/thorough.json +1 -2
- package/dist/cli/shared.js +16 -4
- package/dist/cli.js +15 -13
- package/dist/commands/agent/agent-dispatch.js +8 -0
- package/dist/commands/command/execution-source-loader.js +25 -22
- package/dist/commands/command/portable-template.js +4 -26
- package/dist/commands/config-cli.js +10 -4
- package/dist/commands/env/env-binding.js +10 -3
- package/dist/commands/env/env-cli.js +7 -0
- package/dist/commands/env/secret-cli.js +15 -4
- package/dist/commands/health/checks.js +209 -78
- package/dist/commands/health/improve-metrics.js +12 -0
- package/dist/commands/health.js +16 -4
- package/dist/commands/improve/distill/quality-gate.js +15 -7
- package/dist/commands/improve/distill.js +28 -12
- package/dist/commands/improve/eval-cases.js +9 -2
- package/dist/commands/improve/execution.js +1 -2
- package/dist/commands/improve/extract.js +82 -56
- package/dist/commands/improve/improve-strategies.js +26 -8
- package/dist/commands/improve/improve.js +32 -4
- package/dist/commands/improve/loop-stages.js +13 -3
- package/dist/commands/improve/preparation.js +9 -6
- package/dist/commands/improve/reflect.js +61 -77
- package/dist/commands/lint/index.js +3 -1
- package/dist/commands/migrate-cli.js +6 -4
- package/dist/commands/proposal/drain-policies.js +22 -2
- package/dist/commands/proposal/repository.js +4 -4
- package/dist/commands/proposal/validators/proposal-quality-validators.js +23 -2
- package/dist/commands/proposal/validators/proposals.js +10 -19
- package/dist/commands/read/show.js +42 -31
- package/dist/commands/registry-cli.js +4 -2
- package/dist/commands/sources/init.js +4 -8
- package/dist/commands/sources/self-update.js +2 -2
- package/dist/commands/sources/source-clone.js +5 -7
- package/dist/commands/sources/sources-cli.js +3 -5
- package/dist/commands/tasks/tasks-cli.js +36 -12
- package/dist/commands/tasks/tasks.js +38 -35
- package/dist/commands/tasks/validate.js +186 -0
- package/dist/commands/url-checker.js +75 -16
- package/dist/commands/workflow-cli.js +17 -15
- package/dist/core/activation-policy.js +31 -3
- package/dist/core/adapter/execution-source.js +39 -11
- package/dist/core/asset/stash-meta.js +7 -41
- package/dist/core/bundle-id.js +7 -1
- package/dist/core/common.js +8 -17
- package/dist/core/config/config-schema.js +3 -23
- package/dist/core/config/config-walker.js +50 -4
- package/dist/core/config/config.js +21 -5
- package/dist/core/config/legacy-source-shape-shim.js +79 -0
- package/dist/core/config/schema/embedding.js +2 -2
- package/dist/core/config/schema/engines.js +19 -2
- package/dist/core/config/schema/index-config.js +19 -21
- package/dist/core/config/schema/primitives.js +21 -10
- package/dist/core/config/schema/sources-bundles.js +1 -6
- package/dist/core/errors.js +2 -3
- package/dist/core/improve-result.js +8 -0
- package/dist/core/improve-types.js +17 -0
- package/dist/core/json-schema.js +1 -11
- package/dist/core/maintenance-barrier.js +17 -2
- package/dist/core/paths.js +124 -15
- package/dist/core/state/migrations.js +28 -0
- package/dist/core/state-db.js +28 -1
- package/dist/core/write-source.js +6 -6
- package/dist/indexer/bundle-identity-guard.js +3 -0
- package/dist/indexer/ensure-index.js +5 -0
- package/dist/indexer/indexer.js +11 -3
- package/dist/indexer/lookup/adapter-concept-owner.js +14 -3
- package/dist/indexer/passes/metadata.js +16 -5
- package/dist/indexer/search/search-fields.js +1 -30
- package/dist/indexer/search/search-source.js +3 -2
- package/dist/integrations/agent/engine-resolution.js +107 -4
- package/dist/integrations/agent/execution-lowering.js +15 -2
- package/dist/integrations/agent/model-map.js +16 -10
- package/dist/integrations/agent/prompts.js +13 -6
- package/dist/integrations/agent/runner-dispatch.js +16 -3
- package/dist/integrations/agent/runner.js +2 -0
- package/dist/integrations/lockfile.js +22 -7
- package/dist/llm/client.js +16 -0
- package/dist/llm/index-passes.js +3 -2
- package/dist/output/shapes/passthrough.js +10 -3
- package/dist/output/shapes.js +5 -3
- package/dist/output/text/workflow-format.js +8 -1
- package/dist/scripts/akm-migrate-node.js +2028 -1613
- package/dist/scripts/akm-migrate.js +2027 -1612
- package/dist/setup/setup.js +14 -21
- package/dist/sources/include.js +150 -20
- package/dist/sources/providers/git-install.js +14 -12
- package/dist/sources/providers/git-provider.js +3 -3
- package/dist/sources/snapshot-fetchers/website-ingest.js +54 -16
- package/dist/sources/website-url.js +12 -4
- package/dist/storage/engines/sqlite-migrations.js +40 -10
- package/dist/storage/like-pattern.js +7 -0
- package/dist/storage/repositories/extract-sessions-repository.js +23 -0
- package/dist/storage/repositories/index-connection.js +27 -10
- package/dist/storage/repositories/index-entry-schema.js +19 -2
- package/dist/storage/repositories/index-schema.js +30 -9
- package/dist/storage/repositories/proposals-repository.js +2 -1
- package/dist/storage/repositories/task-history-repository.js +14 -7
- package/dist/storage/repositories/workflow-runs-repository.js +15 -1
- package/dist/storage/sqlite-read-snapshot.js +11 -9
- package/dist/tasks/backends/cron.js +34 -5
- package/dist/tasks/backends/launchd.js +23 -26
- package/dist/tasks/backends/schtasks.js +50 -3
- package/dist/tasks/frozen-script.js +2 -0
- package/dist/tasks/prepare/prepare.js +2 -7
- package/dist/tasks/prepare/script-capture.js +38 -6
- package/dist/tasks/schedule.js +154 -13
- package/dist/tasks/scheduler-sync.js +51 -25
- package/dist/tasks/source/task-source-v3-frozen.js +0 -1
- package/dist/tasks/source/task-source-v4.js +0 -1
- package/dist/workflows/exec/child-workflow.js +2 -3
- package/dist/workflows/exec/dispatch-redaction.js +21 -7
- package/dist/workflows/exec/exec-unit.js +3 -4
- package/dist/workflows/exec/run-workflow.js +19 -10
- package/dist/workflows/exec/step-work.js +35 -56
- package/dist/workflows/freeze/resolve-steps.js +19 -11
- package/dist/workflows/freeze/source-freeze.js +7 -0
- package/dist/workflows/freeze/targets/child-workflow.js +12 -18
- package/dist/workflows/freeze/targets/command.js +14 -2
- package/dist/workflows/ir/environment-v4.js +4 -2
- package/dist/workflows/ir/freeze-v4.js +2 -5
- package/dist/workflows/ir/plan-hash.js +0 -3
- package/dist/workflows/ir/schema-v4.js +14 -9
- package/dist/workflows/ir/schema.js +1 -3
- package/dist/workflows/resource-limits.js +35 -48
- package/dist/workflows/runtime/plan-classifier.js +89 -41
- package/dist/workflows/runtime/run-outputs.js +1 -21
- package/dist/workflows/runtime/runs.js +76 -150
- package/dist/workflows/source-files.js +28 -54
- package/dist/workflows/source-ir/program.js +2 -2
- package/dist/workflows/source-ir/semantics.js +5 -23
- package/docs/integration/bundling-akm.md +1 -1
- package/docs/migration/v0.8-to-v0.9.md +32 -0
- package/docs/migration/v0.9.1-to-v0.9.2.md +20 -0
- package/docs/reference/cli.md +105 -22
- package/docs/reference/configuration.md +12 -2
- package/docs/reference/data-and-telemetry.md +1 -1
- package/docs/reference/tasks.md +8 -0
- package/package.json +1 -1
- package/schemas/akm-config.json +11 -8
|
@@ -23,7 +23,7 @@ import { withWorkflowRunsRepo, } from "../../storage/repositories/workflow-runs-
|
|
|
23
23
|
import { canonicalJson } from "../ir/plan-hash.js";
|
|
24
24
|
import { parseReference, resolveReferenceString, } from "../program/expressions.js";
|
|
25
25
|
import { clip, WORKFLOW_UNIT_DIAGNOSTIC_CLIP } from "../resource-limits.js";
|
|
26
|
-
import { completeWorkflowStep
|
|
26
|
+
import { completeWorkflowStep } from "../runtime/runs.js";
|
|
27
27
|
import { GATE_EVALUATION_PHASE } from "../runtime/unit-phases.js";
|
|
28
28
|
import { parseJudgeVerdict } from "../validate-summary.js";
|
|
29
29
|
import { gateNodeId } from "./frozen-judge.js";
|
|
@@ -65,30 +65,38 @@ function validateFanOutItems(stepId, items) {
|
|
|
65
65
|
return (`Step "${stepId}" fan-out list contains a null item (index ${nullIndex}). ` +
|
|
66
66
|
`Every item must be a concrete value — fix the producing step's output.`);
|
|
67
67
|
}
|
|
68
|
-
const firstIndexByCanonical = new Map();
|
|
69
|
-
for (let i = 0; i < items.length; i++) {
|
|
70
|
-
const canonical = canonicalJson(items[i]) ?? "null";
|
|
71
|
-
const firstIndex = firstIndexByCanonical.get(canonical);
|
|
72
|
-
if (firstIndex !== undefined) {
|
|
73
|
-
return (`Step "${stepId}" fan-out list contains duplicate items (indices ${firstIndex} and ${i}: ` +
|
|
74
|
-
`${clip(canonical, 200)}). Content-derived unit identity requires distinct items — ` +
|
|
75
|
-
`deduplicate the list this workflow fans out over.`);
|
|
76
|
-
}
|
|
77
|
-
firstIndexByCanonical.set(canonical, i);
|
|
78
|
-
}
|
|
79
68
|
return undefined;
|
|
80
69
|
}
|
|
81
70
|
/**
|
|
82
71
|
* Resolve one whole-value reference, refusing a value a persisted TRUNCATION
|
|
83
72
|
* ENVELOPE stands in for (`clipStepEvidenceForPersistence`, runtime/runs.ts).
|
|
84
73
|
*
|
|
85
|
-
* The
|
|
86
|
-
*
|
|
87
|
-
*
|
|
88
|
-
*
|
|
89
|
-
*
|
|
90
|
-
*
|
|
91
|
-
*
|
|
74
|
+
* The first occurrence of a given canonical value keeps the byte-identical id
|
|
75
|
+
* {@link unitIdFor} always produced for it — so a plan with no duplicates (the
|
|
76
|
+
* overwhelming common case) is completely unaffected, and no prior journal
|
|
77
|
+
* entry is ever invalidated by this change. Only the SECOND and later
|
|
78
|
+
* occurrences gain a `#<n>` suffix (`#2`, `#3`, …), computed purely from each
|
|
79
|
+
* item's position in `items` — deterministic across a fresh run and a
|
|
80
|
+
* resumed one, since both call this from the same place in
|
|
81
|
+
* {@link computeStepWorkList} over the same resolved list.
|
|
82
|
+
*/
|
|
83
|
+
function occurrenceSuffixedUnitIds(nodeId, items) {
|
|
84
|
+
const occurrenceByCanonical = new Map();
|
|
85
|
+
return items.map((item) => {
|
|
86
|
+
const base = unitIdFor(nodeId, item, true, true);
|
|
87
|
+
const canonical = canonicalJson(item) ?? "null";
|
|
88
|
+
const occurrence = (occurrenceByCanonical.get(canonical) ?? 0) + 1;
|
|
89
|
+
occurrenceByCanonical.set(canonical, occurrence);
|
|
90
|
+
return occurrence === 1 ? base : `${base}#${occurrence}`;
|
|
91
|
+
});
|
|
92
|
+
}
|
|
93
|
+
/**
|
|
94
|
+
* Resolve one whole-value reference (`inputs[]`, `map.over`, `route.input`).
|
|
95
|
+
*
|
|
96
|
+
* Every step artifact is now persisted whole (issue C), so this is a thin
|
|
97
|
+
* wrapper: source adapters may retain GitHub's whole-value `${{ ... }}`
|
|
98
|
+
* spelling, and this work-list seam unwraps only an exact whole-value
|
|
99
|
+
* wrapper — it never interpolates prose.
|
|
92
100
|
*/
|
|
93
101
|
function resolveStepReference(reference, scope) {
|
|
94
102
|
// Source adapters may retain GitHub's whole-value `${{ ... }}` spelling.
|
|
@@ -96,40 +104,7 @@ function resolveStepReference(reference, scope) {
|
|
|
96
104
|
// unwraps only an exact whole-value wrapper and never interpolates prose.
|
|
97
105
|
const exactWrapper = /^\$\{\{\s*([^{}]+?)\s*\}\}$/.exec(reference);
|
|
98
106
|
const canonicalReference = exactWrapper?.[1] ?? reference;
|
|
99
|
-
|
|
100
|
-
const truncated = truncatedReferenceTarget(canonicalReference, scope, resolved);
|
|
101
|
-
if (!truncated)
|
|
102
|
-
return resolved;
|
|
103
|
-
return {
|
|
104
|
-
ok: false,
|
|
105
|
-
error: {
|
|
106
|
-
reference,
|
|
107
|
-
message: `${reference} reads a step artifact that was NOT persisted (${truncated.originalBytes} bytes exceeded the ` +
|
|
108
|
-
`${truncated.limitBytes}-byte evidence_json cap, so the row stores a truncation marker). This run was ` +
|
|
109
|
-
`resumed from rows that no longer hold the value — it cannot be recovered. Start a new run, or have the ` +
|
|
110
|
-
`producing step emit a reference (path, id) instead of inline bulk data.`,
|
|
111
|
-
},
|
|
112
|
-
};
|
|
113
|
-
}
|
|
114
|
-
/** The envelope a reference lands on or walks through, if any. */
|
|
115
|
-
function truncatedReferenceTarget(reference, scope, resolved) {
|
|
116
|
-
if (resolved.ok)
|
|
117
|
-
return isTruncatedEvidence(resolved.value) ? resolved.value : undefined;
|
|
118
|
-
// A FAILED resolution is re-walked: the envelope is an object with none of
|
|
119
|
-
// the original's keys, so the raw failure is whatever property went missing
|
|
120
|
-
// along the way, several segments past the truncation.
|
|
121
|
-
const parsed = parseReference(reference);
|
|
122
|
-
if (!parsed.ok || parsed.expr.kind !== "stepOutput")
|
|
123
|
-
return undefined;
|
|
124
|
-
let current = scope.stepOutputs[parsed.expr.stepId];
|
|
125
|
-
for (const segment of parsed.expr.path) {
|
|
126
|
-
if (isTruncatedEvidence(current))
|
|
127
|
-
return current;
|
|
128
|
-
if (typeof current !== "object" || current === null)
|
|
129
|
-
return undefined;
|
|
130
|
-
current = current[segment];
|
|
131
|
-
}
|
|
132
|
-
return isTruncatedEvidence(current) ? current : undefined;
|
|
107
|
+
return resolveReferenceString(canonicalReference, scope);
|
|
133
108
|
}
|
|
134
109
|
/**
|
|
135
110
|
* Pre-attempt resolution of a task-composing step's frozen `inputBindings`
|
|
@@ -248,9 +223,13 @@ export function computeStepWorkList(plan, input) {
|
|
|
248
223
|
const fanOutProblem = isFanOut ? validateFanOutItems(plan.stepId, items) : undefined;
|
|
249
224
|
if (fanOutProblem)
|
|
250
225
|
return { ok: false, error: fanOutProblem };
|
|
251
|
-
// Content-derived unit identity: compute every id up front
|
|
252
|
-
//
|
|
253
|
-
|
|
226
|
+
// Content-derived unit identity: compute every id up front. A fan-out's
|
|
227
|
+
// canonical duplicates are disambiguated by occurrence ordinal rather than
|
|
228
|
+
// rejected (issue 6); a solo (non-fan-out) step has exactly one item, so
|
|
229
|
+
// there is nothing to disambiguate.
|
|
230
|
+
const unitIds = isFanOut
|
|
231
|
+
? occurrenceSuffixedUnitIds(template.id, items)
|
|
232
|
+
: [unitIdFor(template.id, undefined, false, true)];
|
|
254
233
|
const gateLoop = input.gateLoop ?? 1;
|
|
255
234
|
const target = template.frozenTarget;
|
|
256
235
|
const frozenExec = target.kind === "shell" || target.kind === "script" ? target.exec : undefined;
|
|
@@ -62,17 +62,25 @@ function rejectNonTaskBindingWith(source, ref, kind) {
|
|
|
62
62
|
const family = kind === "command" ? "commands" : "scripts";
|
|
63
63
|
throw new UsageError(`Workflow step ${source.id} cannot pass with: to ${family} target ${ref}; a ${kind} ref is not a binding surface.`, "COMPOSITION_INVALID");
|
|
64
64
|
}
|
|
65
|
+
const NO_ENGINE_AVAILABLE_MESSAGE = "the fixed opencode-sdk fallback is unavailable";
|
|
66
|
+
function isNoEngineAvailable(err) {
|
|
67
|
+
return err instanceof ConfigError && err.message.includes(NO_ENGINE_AVAILABLE_MESSAGE);
|
|
68
|
+
}
|
|
65
69
|
export function resolveJudge(source, context) {
|
|
66
|
-
const
|
|
67
|
-
if (!engine) {
|
|
68
|
-
throw new ConfigError("This workflow declares completion criteria but no verification engine is configured. Set workflow.judgeEngine to a named LLM or agent engine.", "INVALID_CONFIG_FILE");
|
|
69
|
-
}
|
|
70
|
+
const configuredEngine = context.config.workflow?.judgeEngine;
|
|
70
71
|
const content = source.gate?.rubric?.trim() ?? "Judge workflow completion.";
|
|
71
|
-
|
|
72
|
-
|
|
73
|
-
|
|
74
|
-
|
|
75
|
-
|
|
76
|
-
|
|
77
|
-
|
|
72
|
+
try {
|
|
73
|
+
const prepared = prepareInlineExecution({
|
|
74
|
+
content,
|
|
75
|
+
config: context.config,
|
|
76
|
+
invocationKind: "workflow",
|
|
77
|
+
...(configuredEngine ? { current: { engine: configuredEngine } } : {}),
|
|
78
|
+
});
|
|
79
|
+
return commandResult(source, { onError: "fail", source: sourceStepRef(source) }, prepared, context);
|
|
80
|
+
}
|
|
81
|
+
catch (err) {
|
|
82
|
+
if (configuredEngine || !isNoEngineAvailable(err))
|
|
83
|
+
throw err;
|
|
84
|
+
return undefined;
|
|
85
|
+
}
|
|
78
86
|
}
|
|
@@ -2,6 +2,7 @@
|
|
|
2
2
|
// License, v. 2.0. If a copy of the MPL was not distributed with this
|
|
3
3
|
// file, You can obtain one at https://mozilla.org/MPL/2.0/.
|
|
4
4
|
import { COMPOSITION_INVALID_MULTI_JOB_HINT, UsageError } from "../../core/errors.js";
|
|
5
|
+
import { warn } from "../../core/warn.js";
|
|
5
6
|
import { compileWorkflowSource } from "../source-ir/compile.js";
|
|
6
7
|
import { resolveJudge, resolveStep } from "./resolve-steps.js";
|
|
7
8
|
/**
|
|
@@ -42,6 +43,12 @@ export async function resolveWorkflowSourceV4(asset, workflowSource, config, col
|
|
|
42
43
|
}
|
|
43
44
|
if (sourceStep.gate?.rubric?.trim()) {
|
|
44
45
|
const judge = resolveJudge(sourceStep, context);
|
|
46
|
+
if (judge === undefined) {
|
|
47
|
+
warn(`Workflow step "${sourceStep.id}" has completion criteria but no verification engine is available ` +
|
|
48
|
+
"(set workflow.judgeEngine, defaults.engine, or install an opencode-sdk binary). The step will block " +
|
|
49
|
+
"for `akm workflow resume` once it is reached.");
|
|
50
|
+
continue;
|
|
51
|
+
}
|
|
45
52
|
if (judge.target.kind !== "command")
|
|
46
53
|
throw new Error(`workflow judge ${sourceStep.id} did not resolve to a command target`);
|
|
47
54
|
judges.set(sourceStep.id, judge.target);
|
|
@@ -28,10 +28,11 @@
|
|
|
28
28
|
import { createHash } from "node:crypto";
|
|
29
29
|
import { parseBundleRef } from "../../../core/asset/asset-ref.js";
|
|
30
30
|
import { UsageError } from "../../../core/errors.js";
|
|
31
|
+
import { warn } from "../../../core/warn.js";
|
|
31
32
|
import { GuardedExecutionSourceCollector } from "../../../execution/guarded-source.js";
|
|
32
33
|
import { workflowParamContract } from "../../ir/params.js";
|
|
33
34
|
import { canonicalJson, canonicalPlanJson } from "../../ir/plan-hash.js";
|
|
34
|
-
import { utf8Bytes
|
|
35
|
+
import { utf8Bytes } from "../../resource-limits.js";
|
|
35
36
|
import { loadWorkflowAsset } from "../../runtime/workflow-asset-loader.js";
|
|
36
37
|
import { resolveOwnedAsset } from "../environment.js";
|
|
37
38
|
import { declaredParamNames, earlierStepIds } from "../step-values.js";
|
|
@@ -62,15 +63,14 @@ function childWorkflowContentHash(fields) {
|
|
|
62
63
|
* `freezeEnvironment`): literal `env:` values and `unit: {env: [...]}` refs.
|
|
63
64
|
* An absent/empty `env:` is not authored and stays valid.
|
|
64
65
|
*/
|
|
65
|
-
function
|
|
66
|
+
function warnIfStepEnvironment(stepId, childRef, source) {
|
|
66
67
|
const hasLiteralEnv = Object.keys(source.env ?? {}).length > 0;
|
|
67
68
|
const hasEnvRefs = (source.unit?.env ?? []).length > 0;
|
|
68
69
|
if (!hasLiteralEnv && !hasEnvRefs)
|
|
69
70
|
return;
|
|
70
|
-
|
|
71
|
-
`environment inside its own plan, so
|
|
72
|
-
`
|
|
73
|
-
"own source — a composing step's environment is never delivered into a child run.");
|
|
71
|
+
warn(`Workflow step ${stepId} declares env: while composing ${childRef}: a child run carries its own frozen ` +
|
|
72
|
+
`environment inside its own plan, so this composing step's env: (or unit: env:) is not delivered into it ` +
|
|
73
|
+
`and has no effect. Set those variables inside ${childRef}'s own source instead.`);
|
|
74
74
|
}
|
|
75
75
|
/** A workflow entry of a composition `refPath` — the only entries a cycle can close through (§4.5: a task target can never itself be a task, so no task->task chain exists to close one). */
|
|
76
76
|
function isWorkflowRef(ref) {
|
|
@@ -98,14 +98,8 @@ function assertNoCompositionCycle(stepId, childRef, refPath) {
|
|
|
98
98
|
* total exactly as it was, matching every other freeze-time failure's
|
|
99
99
|
* no-partial-effect shape.
|
|
100
100
|
*/
|
|
101
|
-
function chargeEmbeddedBudget(
|
|
102
|
-
|
|
103
|
-
if (projected > WORKFLOW_MAX_EMBEDDED_CHILD_PLAN_BYTES) {
|
|
104
|
-
throw new UsageError(`Workflow step ${stepId} cannot compose ${childRef}: the embedded child plans would total ${projected} ` +
|
|
105
|
-
`bytes, over the ${WORKFLOW_MAX_EMBEDDED_CHILD_PLAN_BYTES}-byte limit for one workflow run.`, "COMPOSITION_INVALID", "Reduce the number or size of workflows composed into this run — split the work across separate " +
|
|
106
|
-
"top-level runs, or trim the composed children's own plans.");
|
|
107
|
-
}
|
|
108
|
-
budget.embeddedBytes = projected;
|
|
101
|
+
function chargeEmbeddedBudget(_stepId, _childRef, childPlanBytes, budget) {
|
|
102
|
+
budget.embeddedBytes += childPlanBytes;
|
|
109
103
|
}
|
|
110
104
|
export async function childWorkflowDispatch(input) {
|
|
111
105
|
const { source, baseUnit, childRefInput, context, via, taskRef, authoredInputs } = input;
|
|
@@ -115,10 +109,10 @@ export async function childWorkflowDispatch(input) {
|
|
|
115
109
|
const owned = await resolveOwnedAsset(childRefInput, "workflow", context);
|
|
116
110
|
const childAsset = await loadWorkflowAsset(owned.ref);
|
|
117
111
|
const childRef = childAsset.ref;
|
|
118
|
-
//
|
|
119
|
-
//
|
|
120
|
-
//
|
|
121
|
-
|
|
112
|
+
// Issue 10: an authored env: on the composing step has no path to reach
|
|
113
|
+
// the child run — warn rather than vanish silently (see
|
|
114
|
+
// warnIfStepEnvironment's doc comment).
|
|
115
|
+
warnIfStepEnvironment(source.id, childRef, source);
|
|
122
116
|
// §4.2 step 2: the composition cycle check, before any child compilation.
|
|
123
117
|
assertNoCompositionCycle(source.id, childRef, context.composition.refPath);
|
|
124
118
|
const childDepth = context.composition.depth + 1;
|
|
@@ -2,7 +2,9 @@
|
|
|
2
2
|
// License, v. 2.0. If a copy of the MPL was not distributed with this
|
|
3
3
|
// file, You can obtain one at https://mozilla.org/MPL/2.0/.
|
|
4
4
|
import { createHash } from "node:crypto";
|
|
5
|
+
import { parseBuiltinCommandAction } from "../../../commands/command/builtin-action.js";
|
|
5
6
|
import { prepareCommandInvocation } from "../../../commands/command/command-execution.js";
|
|
7
|
+
import { PORTABLE_ARGUMENTS_PLACEHOLDER } from "../../../commands/command/portable-template.js";
|
|
6
8
|
import { captureFrozenDirectoryIdentity } from "../../../execution/directory-identity.js";
|
|
7
9
|
import { freezeExecutableIdentity } from "../../../execution/executable-identity.js";
|
|
8
10
|
import { canonicalResolvedExecutionRequest, } from "../../../execution/resolved-request.js";
|
|
@@ -13,15 +15,25 @@ import { prepareInlineExecution } from "../../../integrations/agent/inline-execu
|
|
|
13
15
|
import { freezeEnvironment, guardedExecutionSource } from "../environment.js";
|
|
14
16
|
import { gitIdentity } from "../identity.js";
|
|
15
17
|
import { durableRequest, executionUnitValues, executionValues, targetConcurrency, } from "../step-values.js";
|
|
18
|
+
function inlineWorkflowCommandAction(action, commandMode) {
|
|
19
|
+
if (commandMode !== "portable-template")
|
|
20
|
+
return action;
|
|
21
|
+
const parsed = parseBuiltinCommandAction(action);
|
|
22
|
+
if (parsed.kind !== "inline")
|
|
23
|
+
return action;
|
|
24
|
+
return { content: parsed.content.split(PORTABLE_ARGUMENTS_PLACEHOLDER).join(parsed.arguments ?? "") };
|
|
25
|
+
}
|
|
16
26
|
export async function commandDispatch(source, baseUnit, action, context) {
|
|
17
27
|
const prepared = await prepareCommandInvocation({
|
|
18
|
-
action,
|
|
28
|
+
action: inlineWorkflowCommandAction(action, source.commandMode),
|
|
19
29
|
config: context.config,
|
|
20
30
|
invocationKind: "workflow",
|
|
21
31
|
...(context.sourceIr.defaults
|
|
22
32
|
? { invocationDefaults: executionUnitValues(context.sourceIr.defaults, context.asset.sourcePath) }
|
|
23
33
|
: {}),
|
|
24
|
-
...(source.commandMode === "literal"
|
|
34
|
+
...(source.commandMode === "literal" || source.commandMode === "portable-template"
|
|
35
|
+
? { inlineContentMode: "literal" }
|
|
36
|
+
: {}),
|
|
25
37
|
current: executionValues(source, context.asset.sourcePath),
|
|
26
38
|
sourceLoader: (ref, kind) => guardedExecutionSource(ref, kind, context),
|
|
27
39
|
});
|
|
@@ -16,6 +16,7 @@ import dotenv from "dotenv";
|
|
|
16
16
|
import { assetPathForName } from "../../core/asset/asset-placement.js";
|
|
17
17
|
import { isWithin } from "../../core/common.js";
|
|
18
18
|
import { NotFoundError, UsageError } from "../../core/errors.js";
|
|
19
|
+
import { warn } from "../../core/warn.js";
|
|
19
20
|
import { captureGuardedExecutionSource, GuardedExecutionSourceCollector } from "../../execution/guarded-source.js";
|
|
20
21
|
const SECRET_TOKEN_RE = /\$\{secret:([A-Za-z0-9_./-]+)\}/g;
|
|
21
22
|
const ENVIRONMENT_NAME_RE = /^[A-Za-z_][A-Za-z0-9_]*$/;
|
|
@@ -33,7 +34,7 @@ export function freezeWorkflowEnvironment(refs, options) {
|
|
|
33
34
|
throw new UsageError(`Workflow env ref ${JSON.stringify(inputRef)} did not resolve to a canonical fully-qualified owner.`, "WORKFLOW_SOURCE_INVALID");
|
|
34
35
|
}
|
|
35
36
|
if (logical.has(resolved.ref)) {
|
|
36
|
-
|
|
37
|
+
continue;
|
|
37
38
|
}
|
|
38
39
|
logical.add(resolved.ref);
|
|
39
40
|
trackParentDirectories(collector, resolved.root, resolved.path);
|
|
@@ -48,7 +49,8 @@ export function freezeWorkflowEnvironment(refs, options) {
|
|
|
48
49
|
const physicalKey = `${captured.containmentPhysicalIdentity}\0${captured.physicalIdentity}`;
|
|
49
50
|
const alias = physical.get(physicalKey);
|
|
50
51
|
if (alias !== undefined && alias !== resolved.ref) {
|
|
51
|
-
|
|
52
|
+
warn(`Workflow env ref ${resolved.ref} aliases the same physical environment source as ${alias} under a different logical owner. Using ${alias}.`);
|
|
53
|
+
continue;
|
|
52
54
|
}
|
|
53
55
|
physical.set(physicalKey, resolved.ref);
|
|
54
56
|
const parsed = dotenv.parse(Buffer.from(captured.bytesBase64, "base64"));
|
|
@@ -51,11 +51,8 @@ export async function compileResolveFreezeWorkflowV4(asset, config, options = {}
|
|
|
51
51
|
throw new UsageError(compiled.errors.map((error) => `${asset.path}:${error.line}: ${error.message}`).join("\n"), code, isMultiJob ? COMPOSITION_INVALID_MULTI_JOB_HINT : undefined);
|
|
52
52
|
}
|
|
53
53
|
const steps = compiled.plan.steps.map((step) => {
|
|
54
|
-
const frozenJudge = step.gate.criteria.length === 0 ? null : resolved.judges.get(step.stepId);
|
|
55
|
-
|
|
56
|
-
throw new Error(`resolved workflow judge missing for step ${step.stepId}`);
|
|
57
|
-
}
|
|
58
|
-
const gate = Object.freeze({ ...step.gate, maxLoops: step.gate.maxLoops ?? 1, frozenJudge: frozenJudge ?? null });
|
|
54
|
+
const frozenJudge = step.gate.criteria.length === 0 ? null : (resolved.judges.get(step.stepId) ?? null);
|
|
55
|
+
const gate = Object.freeze({ ...step.gate, maxLoops: step.gate.maxLoops ?? 1, frozenJudge });
|
|
59
56
|
if (!step.root) {
|
|
60
57
|
const { root: _root, ...withoutRoot } = step;
|
|
61
58
|
return Object.freeze({ ...withoutRoot, gate });
|
|
@@ -13,7 +13,6 @@
|
|
|
13
13
|
* Pure module: no IO beyond node:crypto, no engine imports.
|
|
14
14
|
*/
|
|
15
15
|
import { createHash } from "node:crypto";
|
|
16
|
-
import { utf8Bytes, WORKFLOW_MAX_PLAN_BYTES } from "../resource-limits.js";
|
|
17
16
|
import { decodeWorkflowPlanV4, WORKFLOW_IR_V5_VERSION } from "./schema-v4.js";
|
|
18
17
|
/** sha256 hex of the canonical (recursively sorted-keys) JSON of the plan. */
|
|
19
18
|
export function computePlanHash(plan) {
|
|
@@ -29,8 +28,6 @@ export function canonicalJson(value) {
|
|
|
29
28
|
}
|
|
30
29
|
/** Decode, require stored canonical bytes, then verify the stored SHA-256. */
|
|
31
30
|
export function decodeCanonicalPlan(runId, planJson, planHash, expectedVersion) {
|
|
32
|
-
if (utf8Bytes(planJson) > WORKFLOW_MAX_PLAN_BYTES)
|
|
33
|
-
throw new Error(`Workflow run ${runId} frozen plan exceeds the 2 MiB resource limit.`);
|
|
34
31
|
let parsed;
|
|
35
32
|
try {
|
|
36
33
|
parsed = JSON.parse(planJson);
|
|
@@ -19,7 +19,7 @@ import { decodeExecutionSourceIdentity } from "../../execution/source.js";
|
|
|
19
19
|
import { decodeFrozenRunnerSpec } from "../../integrations/agent/execution-lowering.js";
|
|
20
20
|
import { parseReference } from "../program/expressions.js";
|
|
21
21
|
import { PROGRAM_PARAM_NAME_PATTERN } from "../program/schema.js";
|
|
22
|
-
import { utf8Bytes
|
|
22
|
+
import { utf8Bytes } from "../resource-limits.js";
|
|
23
23
|
import { decodeWorkflowExecSpec, validateWorkflowPlanStructure, } from "./schema.js";
|
|
24
24
|
export const WORKFLOW_IR_V5_VERSION = 5;
|
|
25
25
|
/**
|
|
@@ -152,8 +152,18 @@ function decodeGateV4(value, stepId, requiredSources) {
|
|
|
152
152
|
frozenJudge: null,
|
|
153
153
|
});
|
|
154
154
|
}
|
|
155
|
-
if (!Object.hasOwn(gate, "frozenJudge")
|
|
156
|
-
fail(`gate ${stepId} with criteria
|
|
155
|
+
if (!Object.hasOwn(gate, "frozenJudge")) {
|
|
156
|
+
fail(`gate ${stepId} with criteria is missing its frozenJudge field`);
|
|
157
|
+
}
|
|
158
|
+
if (gate.frozenJudge === null) {
|
|
159
|
+
return Object.freeze({
|
|
160
|
+
kind: "gate",
|
|
161
|
+
id: gate.id,
|
|
162
|
+
stepId,
|
|
163
|
+
criteria,
|
|
164
|
+
maxLoops: gate.maxLoops,
|
|
165
|
+
frozenJudge: null,
|
|
166
|
+
});
|
|
157
167
|
}
|
|
158
168
|
const identity = {
|
|
159
169
|
kind: "unit",
|
|
@@ -239,12 +249,7 @@ function decodeChildWorkflowTarget(target, unit, depth, budget) {
|
|
|
239
249
|
if (actualPlanHash !== planHash) {
|
|
240
250
|
fail(`unit ${unit.id} child workflow embedded plan does not match its frozen planHash`);
|
|
241
251
|
}
|
|
242
|
-
|
|
243
|
-
if (projectedBytes > WORKFLOW_MAX_EMBEDDED_CHILD_PLAN_BYTES) {
|
|
244
|
-
fail(`unit ${unit.id} child workflow ${target.ref} embedded plans total ${projectedBytes} bytes, over the ` +
|
|
245
|
-
`${WORKFLOW_MAX_EMBEDDED_CHILD_PLAN_BYTES}-byte limit`);
|
|
246
|
-
}
|
|
247
|
-
budget.embeddedBytes = projectedBytes;
|
|
252
|
+
budget.embeddedBytes += utf8Bytes(embeddedPlanJson);
|
|
248
253
|
return Object.freeze({
|
|
249
254
|
kind: "child-workflow",
|
|
250
255
|
ref: target.ref,
|
|
@@ -5,7 +5,7 @@ import { isContainedRelativePath, isRecord } from "../../core/common.js";
|
|
|
5
5
|
import { UsageError } from "../../core/errors.js";
|
|
6
6
|
import { parseReference } from "../program/expressions.js";
|
|
7
7
|
import { PROGRAM_PARAM_NAME_PATTERN, PROGRAM_RETRY_REASONS, PROGRAM_STEP_ID_PATTERN } from "../program/schema.js";
|
|
8
|
-
import { jsonBytes, utf8Bytes, WORKFLOW_ENV_VAR_NAME_PATTERN, WORKFLOW_MAX_CONCURRENCY, WORKFLOW_MAX_INSTRUCTION_BYTES,
|
|
8
|
+
import { jsonBytes, utf8Bytes, WORKFLOW_ENV_VAR_NAME_PATTERN, WORKFLOW_MAX_CONCURRENCY, WORKFLOW_MAX_INSTRUCTION_BYTES, WORKFLOW_MAX_SCHEMA_BYTES, WORKFLOW_MAX_TIMEOUT_MS, } from "../resource-limits.js";
|
|
9
9
|
// Shared dispatch-significant bounds now live in `../resource-limits` so the
|
|
10
10
|
// parser, the published JSON Schema, and this decoder enforce identical
|
|
11
11
|
// values. Re-exported here for existing importers (e.g. `commands/workflow-cli.ts`).
|
|
@@ -18,8 +18,6 @@ export function validateWorkflowPlanStructure(input, options) {
|
|
|
18
18
|
fail(`irVersion must be ${options.expectedVersion}`);
|
|
19
19
|
}
|
|
20
20
|
assertJson(input);
|
|
21
|
-
if (jsonBytes(input) > WORKFLOW_MAX_PLAN_BYTES)
|
|
22
|
-
fail("plan exceeds the 2 MiB resource limit");
|
|
23
21
|
const plan = input;
|
|
24
22
|
assertKeys(input, ["irVersion", "title", "params", "paramSchemas", "budget", "execution", "steps", ...(options.planExtraKeys ?? [])], "plan");
|
|
25
23
|
assertString(plan.title, "title");
|
|
@@ -2,7 +2,6 @@
|
|
|
2
2
|
// License, v. 2.0. If a copy of the MPL was not distributed with this
|
|
3
3
|
// file, You can obtain one at https://mozilla.org/MPL/2.0/.
|
|
4
4
|
import { EXECUTION_MAX_TIMEOUT_MS } from "../execution/limits.js";
|
|
5
|
-
export const WORKFLOW_MAX_PLAN_BYTES = 2 * 1024 * 1024;
|
|
6
5
|
export const WORKFLOW_MAX_SOURCE_BYTES = 1024 * 1024;
|
|
7
6
|
export const WORKFLOW_MAX_INSTRUCTION_BYTES = 256 * 1024;
|
|
8
7
|
export const WORKFLOW_MAX_SCHEMA_BYTES = 256 * 1024;
|
|
@@ -73,24 +72,22 @@ export const DEFAULT_EXEC_TIMEOUT_MS = 600_000;
|
|
|
73
72
|
* corrupt every downstream reference. That is the residual failure the cap
|
|
74
73
|
* genuinely justifies.
|
|
75
74
|
*
|
|
76
|
-
* 8 MiB is deliberately generous
|
|
77
|
-
*
|
|
78
|
-
*
|
|
79
|
-
* is nowhere near it.
|
|
75
|
+
* 8 MiB is deliberately generous, so any output a step is likely to produce
|
|
76
|
+
* fits many times over and an ordinary full test/build log is nowhere near
|
|
77
|
+
* it.
|
|
80
78
|
*/
|
|
81
79
|
export const WORKFLOW_MAX_EXEC_OUTPUT_BYTES = 8 * 1024 * 1024;
|
|
82
80
|
/**
|
|
83
81
|
* Marker stamped on an exec artifact that was RETAINED ONLY IN PART because the
|
|
84
82
|
* command wrote past {@link WORKFLOW_MAX_EXEC_OUTPUT_BYTES}.
|
|
85
83
|
*
|
|
86
|
-
* Deliberately ugly and unique
|
|
87
|
-
*
|
|
88
|
-
*
|
|
89
|
-
*
|
|
90
|
-
*
|
|
91
|
-
*
|
|
92
|
-
*
|
|
93
|
-
* the head of a real log), and the block says exactly how much is missing.
|
|
84
|
+
* Deliberately ugly and unique: a truncated value must NEVER be mistakable
|
|
85
|
+
* for a complete one by a downstream `steps.<id>.output` reference, by a gate
|
|
86
|
+
* judge, by `akm workflow status`, or by a human reading the row. The
|
|
87
|
+
* artifact is TEXT here rather than a JSON value, so the marker is appended
|
|
88
|
+
* as a trailing block instead of replacing the value with an envelope: the
|
|
89
|
+
* retained prefix is still genuinely useful (it is the head of a real log),
|
|
90
|
+
* and the block says exactly how much is missing.
|
|
94
91
|
*/
|
|
95
92
|
export const WORKFLOW_EXEC_OUTPUT_TRUNCATED_MARKER = "__akm_exec_output_truncated__";
|
|
96
93
|
// Per-var: Win32 `SetEnvironmentVariable` caps one variable at 32 767 UTF-16
|
|
@@ -143,27 +140,17 @@ export function clip(text, max) {
|
|
|
143
140
|
return text.length > max ? `${text.slice(0, max)}…` : text;
|
|
144
141
|
}
|
|
145
142
|
// ── Persistence bounds ───────────────────────────────────────────────────────
|
|
146
|
-
|
|
147
|
-
|
|
148
|
-
|
|
149
|
-
|
|
150
|
-
|
|
151
|
-
|
|
152
|
-
|
|
153
|
-
|
|
154
|
-
|
|
155
|
-
|
|
156
|
-
|
|
157
|
-
* silently shortening them.
|
|
158
|
-
*
|
|
159
|
-
* 1 MiB is deliberately generous: it is 4× the per-instruction cap and half the
|
|
160
|
-
* whole-plan cap, so no realistic authored workflow reaches it, while a runaway
|
|
161
|
-
* fan-out is still bounded to something SQLite and `akm workflow status` can
|
|
162
|
-
* handle.
|
|
163
|
-
*/
|
|
164
|
-
export const WORKFLOW_MAX_EVIDENCE_JSON_BYTES = 1024 * 1024;
|
|
165
|
-
/** Chars of the original value retained (as a marked preview) in a truncation envelope. */
|
|
166
|
-
export const WORKFLOW_EVIDENCE_TRUNCATION_PREVIEW_CHARS = 1000;
|
|
143
|
+
//
|
|
144
|
+
// `workflow_run_steps.evidence_json` is persisted WHOLE and unclipped. It used
|
|
145
|
+
// to be capped at 1 MiB, past which the row was replaced by a marked
|
|
146
|
+
// truncation envelope — but the run still looked successful, and the NEXT
|
|
147
|
+
// invocation (a resume, or any later step referencing the artifact) failed
|
|
148
|
+
// permanently: the value was gone from the only place a resumed run can read
|
|
149
|
+
// it from, so every prior paid step had to be re-run from scratch. SQLite has
|
|
150
|
+
// no practical row-size problem here (its own ceiling is ~1 GB), so there is
|
|
151
|
+
// nothing this cap protected that a full write does not already handle
|
|
152
|
+
// correctly. `JSON.stringify` throwing on genuinely unserializable evidence
|
|
153
|
+
// is unchanged.
|
|
167
154
|
export function utf8Bytes(value) {
|
|
168
155
|
return Buffer.byteLength(value, "utf8");
|
|
169
156
|
}
|
|
@@ -173,21 +160,21 @@ export function jsonBytes(value) {
|
|
|
173
160
|
// ── Recursive child-workflow composition bounds (spec docs/plans/specs/
|
|
174
161
|
// ── p3a-plan-v5-child-freeze.md §4.5, A-N6) ──────────────────────────────────
|
|
175
162
|
//
|
|
176
|
-
// Enforced ONCE, at freeze, before publication, in
|
|
177
|
-
// `src/workflows/freeze/targets/child-workflow.ts` — the ONE resolver both the
|
|
178
|
-
// direct `uses: workflows/<ref>` form and the task-wrapped form route through
|
|
179
|
-
// — and re-enforced as a corruption gate whenever a parent plan is DECODED
|
|
180
|
-
// (`src/workflows/ir/schema-v4.ts`'s recursive `decodeChildWorkflowTarget`).
|
|
181
|
-
// Full design history, including the rejected alternative for the byte cap:
|
|
182
|
-
// docs/architecture/decisions/0007-workflow-composition-bounds.md.
|
|
183
|
-
//
|
|
184
163
|
// Composition DEPTH is unbounded — `assertNoCompositionCycle` (freeze/targets/
|
|
185
164
|
// child-workflow.ts) already makes infinite composition mathematically
|
|
186
165
|
// impossible, so a depth ceiling on top of it only bounded how many
|
|
187
166
|
// legitimately distinct workflows an author could nest.
|
|
188
|
-
|
|
189
|
-
|
|
190
|
-
|
|
191
|
-
|
|
192
|
-
|
|
193
|
-
|
|
167
|
+
//
|
|
168
|
+
// This section used to also cap the AGGREGATE canonical-JSON bytes of every
|
|
169
|
+
// embedded child plan in one root freeze (`WORKFLOW_MAX_EMBEDDED_CHILD_PLAN_BYTES`,
|
|
170
|
+
// enforced in `freeze/targets/child-workflow.ts` and re-checked on every
|
|
171
|
+
// decode in `ir/schema-v4.ts`'s `decodeChildWorkflowTarget`), alongside a
|
|
172
|
+
// standalone cap on any one frozen plan's own bytes
|
|
173
|
+
// (`WORKFLOW_MAX_PLAN_BYTES`, enforced in `ir/plan-hash.ts` and
|
|
174
|
+
// `ir/schema.ts`). A large plan is not a wrong plan: composing several
|
|
175
|
+
// substantial workflows together is a legitimate, deliberate authoring
|
|
176
|
+
// choice, and SQLite has no practical row-size problem here (its own
|
|
177
|
+
// ceiling is ~1 GB) — there was nothing left for either cap to protect that
|
|
178
|
+
// `planHash`/`contentHash` verification does not already cover. Both caps
|
|
179
|
+
// are removed; the hash/canonical-JSON integrity checks they sat next to
|
|
180
|
+
// are unchanged.
|