@multileaf/ai-workflow 0.7.0 → 0.9.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -4,6 +4,21 @@ All notable changes to this project are documented in this file. The format foll
4
4
 
5
5
  ## [Unreleased]
6
6
 
7
+ ## [0.9.0] - 2026-10-21
8
+
9
+ ## [0.8.0] - 2026-10-20
10
+
11
+ ### Added
12
+
13
+ - Standing-context benchmark: `aiw context-bench` measures the persistent context AI Workflow renders into every agent session (root context file, neutral resources under `.aiw/resources/`, `.aiw/context/` layers) per category, offline and deterministically. Byte and file counts are measured; tokens are estimated (characters/4) and labeled as such; a content hash pins the fixture and the report contains no timestamps. `--save` records a checkpoint under `.aiw/checkpoints/` and `--baseline=<path>` exits nonzero on any token regression, giving context changes a cost gate.
14
+ - Task parity suite: `aiw parity [--tasks=directory]` runs YAML parity tasks (`.aiw/parity/` by default) that pin quality constraints on context assembly. Each task compares the full composed context against the just-in-time task-retrieval assembler and fails, naming the exact fact, when a critical fact or completion marker does not survive retrieval. Exit nonzero on any failure makes it a CI gate for context changes; estimated token cost is reported but never gates here. Inter-stage tasks load real artifacts via `sources` (project-relative paths into any layer) and enforce `required_patterns` — every extracted identifier (e.g. `REQ-\d+` from the approved specification) must survive assembly, guarding the identifiers traceability links depend on.
15
+ - Usage sidecar: `aiw usage-sidecar record/report` maintains a versioned ledger at `.aiw/checkpoints/usage-sidecar.yml` with per-run tokens split into billing tiers (cache-read, fresh-input, cache-write, output), a `measured`/`estimated` label that propagates pessimistically to aggregates, and attribution by model and harness (with version). The report exposes `cache_hit_rate` (cache-read over billable input); dollar conversion is intentionally left out. `aiw scan` auto-records interpreter and embedding usage into the ledger with their real measurement label.
16
+ - Cost attribution: `aiw cost-report` joins the usage sidecar with the traceability graph (`aiw trace`) to report tokens per verified requirement — the outcome-denominated cost metric — with attributed and unattributed tokens shown side by side. Sidecar entries accept `--requirement=REQ-###`, and `parseTraceabilityGraph` round-trips the serialized graph.
17
+ - Model routing table: `aiw routing init|show|resolve --category=name` maintains a static, evidence-linked table at `.aiw/routing.yml`. Rules map task categories to models deterministically; each rule names its evidence, and the resolver reports its advisory level (`evidenced`, `unevidenced-rule`, `default-fallback`, `default-unset`). Routing is advisory only and never invokes a model.
18
+ - Task-lifecycle token usage: `aiw task-usage [--workflow=id-or-ticket]` aggregates sidecar entries correlated by `--workflow` into one task's totals — input tokens (all billing tiers), output tokens, cache hit rate, per-stage breakdown with models, requirements touched, and harness attribution. Tokens are the model-invariant metric, so lifecycle totals stay comparable across model routing changes; `usage-sidecar record` accepts `--workflow` for correlation.
19
+ - Harness log import: `aiw usage-sidecar import --from=claude-code|codex|gemini|pi` turns local harness session logs into measured ledger entries with all billing tiers, model, and harness attribution. Claude Code imports deduplicate stream chunks by message id; Codex sessions are scoped to the project via `session_meta.cwd` with cached input separated from fresh; Gemini planner responses are imported without project scoping (transcripts record no working directory); Pi imports per-message usage with provider attribution and dedicated usage entries (for example cache warming), scoped by project directory. Import is idempotent via stable per-event identities, so reruns never double-count. Sidecar entries and reports now attribute tokens by provider alongside model and harness, and the docs ship a harness support matrix declaring per-harness capabilities (with Pi recommended as the fullest experience).
20
+ - Session heartbeat: `aiw heartbeat` records beats from harness hooks (`aiw hooks install --target=claude-code` wires Stop and UserPromptSubmit) and decides by idle gap — inside the warn window (default 4 min) it saves a compact checkpoint while the provider cache is still warm; past the TTL it saves one so the next gap resumes cheap. No timers or keep-alive pings: the gap is measured on the next beat, guaranteeing a session never pays the full-price prefix rebuild twice in a row. Thresholds are configurable, state lives in `.aiw/heartbeats.yml`, and hook installation is project-scoped, merge-safe, and idempotent.
21
+
7
22
  ## [0.7.0] - 2026-10-13
8
23
 
9
24
  ### Added
@@ -0,0 +1,52 @@
1
+ import type { FileSystem } from "./types.js";
2
+ /**
3
+ * Standing-context benchmark (deterministic, offline, zero API cost).
4
+ *
5
+ * Measures the persistent context AI Workflow renders into every agent
6
+ * session: the root context file for the active target, neutral resources
7
+ * under `.aiw/resources/`, and composed context layers under `.aiw/context/`.
8
+ *
9
+ * Honest-measurement rules (see docs/token-efficiency-study-notes.md):
10
+ * - byte and file counts are `measured`; token counts are `estimated`
11
+ * (characters/4) and always labeled as such;
12
+ * - the bundle hash pins the exact fixture content behind every number;
13
+ * - output contains no timestamps or randomness, so reruns are
14
+ * byte-identical for unchanged projects.
15
+ */
16
+ export declare const CONTEXT_BENCH_SCHEMA = 1;
17
+ export declare const ROOT_CONTEXT_FILES: readonly ["AGENTS.md", "CLAUDE.md", "GEMINI.md"];
18
+ export declare const RESOURCE_CATEGORIES: readonly ["skills", "rules", "agents", "hooks", "templates"];
19
+ export type ResourceCategory = (typeof RESOURCE_CATEGORIES)[number];
20
+ export type ContextBenchCategory = ResourceCategory | "root-context" | "context-layers" | "other-resources";
21
+ export declare const CONTEXT_BENCH_CATEGORIES: readonly ContextBenchCategory[];
22
+ export type ContextBenchCategoryMetrics = {
23
+ name: ContextBenchCategory;
24
+ files: number;
25
+ characters: number;
26
+ estimatedTokens: number;
27
+ paths: string[];
28
+ };
29
+ export type ContextBenchReport = {
30
+ schema: number;
31
+ bundleHash: string;
32
+ totalCharacters: number;
33
+ totalEstimatedTokens: number;
34
+ categories: ContextBenchCategoryMetrics[];
35
+ };
36
+ /** Estimate tokens from characters. Always labeled `estimated`, never `measured`. */
37
+ export declare function estimateTokens(text: string): number;
38
+ /** Collect the standing-context report for a project. Deterministic: same files, same bytes, same report. */
39
+ export declare function collectContextBench(root: string, fs: FileSystem): ContextBenchReport;
40
+ export declare function serializeContextBench(report: ContextBenchReport): string;
41
+ /** Parse a previously saved report for baseline comparison. */
42
+ export declare function parseContextBench(content: string): ContextBenchReport;
43
+ export type ContextBenchComparison = {
44
+ bundleChanged: boolean;
45
+ totalTokenDelta: number;
46
+ regressions: string[];
47
+ };
48
+ /**
49
+ * Compare a current report against a saved baseline.
50
+ * Follows the parity-harness rule: any token increase is a regression.
51
+ */
52
+ export declare function compareContextBench(baseline: ContextBenchReport, current: ContextBenchReport): ContextBenchComparison;
@@ -0,0 +1,170 @@
1
+ import { createHash } from "node:crypto";
2
+ import { join } from "node:path";
3
+ /**
4
+ * Standing-context benchmark (deterministic, offline, zero API cost).
5
+ *
6
+ * Measures the persistent context AI Workflow renders into every agent
7
+ * session: the root context file for the active target, neutral resources
8
+ * under `.aiw/resources/`, and composed context layers under `.aiw/context/`.
9
+ *
10
+ * Honest-measurement rules (see docs/token-efficiency-study-notes.md):
11
+ * - byte and file counts are `measured`; token counts are `estimated`
12
+ * (characters/4) and always labeled as such;
13
+ * - the bundle hash pins the exact fixture content behind every number;
14
+ * - output contains no timestamps or randomness, so reruns are
15
+ * byte-identical for unchanged projects.
16
+ */
17
+ export const CONTEXT_BENCH_SCHEMA = 1;
18
+ export const ROOT_CONTEXT_FILES = ["AGENTS.md", "CLAUDE.md", "GEMINI.md"];
19
+ export const RESOURCE_CATEGORIES = ["skills", "rules", "agents", "hooks", "templates"];
20
+ export const CONTEXT_BENCH_CATEGORIES = [
21
+ "root-context",
22
+ ...RESOURCE_CATEGORIES,
23
+ "context-layers",
24
+ "other-resources",
25
+ ];
26
+ /** Estimate tokens from characters. Always labeled `estimated`, never `measured`. */
27
+ export function estimateTokens(text) {
28
+ return Math.ceil(text.length / 4);
29
+ }
30
+ function hashBundle(entries) {
31
+ const hash = createHash("sha256");
32
+ for (const entry of entries.sort((a, b) => (a.path < b.path ? -1 : a.path > b.path ? 1 : 0)))
33
+ hash.update(`${entry.path}\0${Buffer.byteLength(entry.content, "utf8")}\0`);
34
+ return hash.digest("hex").slice(0, 16);
35
+ }
36
+ function emptyCategory(name) {
37
+ return { name, files: 0, characters: 0, estimatedTokens: 0, paths: [] };
38
+ }
39
+ function addFile(categories, name, path, content) {
40
+ const category = categories.get(name) ?? emptyCategory(name);
41
+ category.files += 1;
42
+ category.characters += content.length;
43
+ category.estimatedTokens += estimateTokens(content);
44
+ category.paths.push(path);
45
+ categories.set(name, category);
46
+ }
47
+ /** Collect the standing-context report for a project. Deterministic: same files, same bytes, same report. */
48
+ export function collectContextBench(root, fs) {
49
+ const categories = new Map();
50
+ const bundle = [];
51
+ const record = (name, path, content) => {
52
+ addFile(categories, name, path, content);
53
+ bundle.push({ path, content });
54
+ };
55
+ for (const file of ROOT_CONTEXT_FILES) {
56
+ const path = join(root, file);
57
+ if (fs.pathType?.(path) === "file")
58
+ record("root-context", file, fs.read(path));
59
+ }
60
+ const resourceRoot = join(root, ".aiw/resources");
61
+ if (fs.pathType?.(resourceRoot) === "directory" && fs.listFiles) {
62
+ for (const relative of fs.listFiles(resourceRoot)) {
63
+ const absolute = join(resourceRoot, relative);
64
+ if (fs.pathType?.(absolute) !== "file")
65
+ continue;
66
+ const top = relative.split(/[\\/]/)[0];
67
+ const name = RESOURCE_CATEGORIES.includes(top)
68
+ ? top
69
+ : "other-resources";
70
+ record(name, `.aiw/resources/${relative.split("\\").join("/")}`, fs.read(absolute));
71
+ }
72
+ }
73
+ const contextRoot = join(root, ".aiw/context");
74
+ if (fs.pathType?.(contextRoot) === "directory") {
75
+ for (const file of (fs.list?.(contextRoot) ?? [])
76
+ .filter((name) => name.endsWith(".md"))
77
+ .sort()) {
78
+ const absolute = join(contextRoot, file);
79
+ if (fs.pathType?.(absolute) !== "file")
80
+ continue;
81
+ record("context-layers", `.aiw/context/${file}`, fs.read(absolute));
82
+ }
83
+ }
84
+ const ordered = CONTEXT_BENCH_CATEGORIES.map((name) => categories.get(name) ?? emptyCategory(name));
85
+ return {
86
+ schema: CONTEXT_BENCH_SCHEMA,
87
+ bundleHash: hashBundle(bundle),
88
+ totalCharacters: ordered.reduce((total, category) => total + category.characters, 0),
89
+ totalEstimatedTokens: ordered.reduce((total, category) => total + category.estimatedTokens, 0),
90
+ categories: ordered,
91
+ };
92
+ }
93
+ export function serializeContextBench(report) {
94
+ const lines = [
95
+ `schema: ${report.schema}`,
96
+ `bundle_hash: ${report.bundleHash}`,
97
+ `total_characters: ${report.totalCharacters}`,
98
+ `total_estimated_tokens: ${report.totalEstimatedTokens}`,
99
+ "token_estimate: characters/4 (estimated; byte counts are measured)",
100
+ "categories:",
101
+ ];
102
+ for (const category of report.categories) {
103
+ lines.push(` - name: ${category.name}`, ` files: ${category.files}`, ` characters: ${category.characters}`, ` estimated_tokens: ${category.estimatedTokens}`);
104
+ if (category.paths.length) {
105
+ lines.push(" paths:");
106
+ lines.push(...category.paths.map((path) => ` - ${path}`));
107
+ }
108
+ }
109
+ return lines.join("\n") + "\n";
110
+ }
111
+ /** Parse a previously saved report for baseline comparison. */
112
+ export function parseContextBench(content) {
113
+ const lines = content.split("\n");
114
+ if (lines[0]?.trim() !== `schema: ${CONTEXT_BENCH_SCHEMA}`)
115
+ throw new Error("Unrecognized context benchmark report schema.");
116
+ const report = {
117
+ schema: CONTEXT_BENCH_SCHEMA,
118
+ bundleHash: requiredValue(lines, "bundle_hash"),
119
+ totalCharacters: Number(requiredValue(lines, "total_characters")),
120
+ totalEstimatedTokens: Number(requiredValue(lines, "total_estimated_tokens")),
121
+ categories: [],
122
+ };
123
+ let current;
124
+ for (const line of lines.slice(1)) {
125
+ const category = line.match(/^ {2}- name: (.+)$/);
126
+ if (category) {
127
+ current = emptyCategory(category[1]);
128
+ report.categories.push(current);
129
+ continue;
130
+ }
131
+ const path = line.match(/^ {6}- (.+)$/);
132
+ if (path && current)
133
+ current.paths.push(path[1]);
134
+ const match = line.match(/^ {4}(files|characters|estimated_tokens): (\d+)$/);
135
+ if (match && current) {
136
+ const key = match[1] === "estimated_tokens" ? "estimatedTokens" : match[1];
137
+ current[key] = Number(match[2]);
138
+ }
139
+ }
140
+ if (!report.categories.length)
141
+ throw new Error("Context benchmark report has no categories.");
142
+ return report;
143
+ }
144
+ function requiredValue(lines, key) {
145
+ const value = lines.find((line) => line.startsWith(`${key}: `))?.slice(key.length + 2);
146
+ if (!value)
147
+ throw new Error(`Context benchmark report is missing ${key}.`);
148
+ return value;
149
+ }
150
+ /**
151
+ * Compare a current report against a saved baseline.
152
+ * Follows the parity-harness rule: any token increase is a regression.
153
+ */
154
+ export function compareContextBench(baseline, current) {
155
+ const baselineByCategory = new Map(baseline.categories.map((category) => [category.name, category]));
156
+ const regressions = [];
157
+ for (const category of current.categories) {
158
+ const previous = baselineByCategory.get(category.name);
159
+ if (previous && category.estimatedTokens > previous.estimatedTokens)
160
+ regressions.push(`${category.name}: ${previous.estimatedTokens} -> ${category.estimatedTokens} estimated tokens`);
161
+ }
162
+ const totalTokenDelta = current.totalEstimatedTokens - baseline.totalEstimatedTokens;
163
+ if (totalTokenDelta > 0)
164
+ regressions.push(`total: ${baseline.totalEstimatedTokens} -> ${current.totalEstimatedTokens} estimated tokens`);
165
+ return {
166
+ bundleChanged: baseline.bundleHash !== current.bundleHash,
167
+ totalTokenDelta,
168
+ regressions,
169
+ };
170
+ }
@@ -0,0 +1,65 @@
1
+ import { type UsageLabel, type UsageSidecarEntry } from "./usage-sidecar.js";
2
+ import type { TraceabilityGraph } from "./traceability.js";
3
+ /**
4
+ * Cost attribution per verified requirement — the outcome-denominated cost
5
+ * metric (Uber's "cost per merged PR", adapted to AI Workflow's unit of
6
+ * value: a requirement with verification evidence).
7
+ *
8
+ * Honesty rules:
9
+ * - tokens attributed to requirements and unattributed tokens are reported
10
+ * side by side, so the ratio cannot be gamed by leaving expensive work
11
+ * unattributed;
12
+ * - the denominator counts only requirements the traceability graph marks
13
+ * as verified (linked evidence present);
14
+ * - labels propagate pessimistically (any estimated entry poisons the total).
15
+ */
16
+ export type RequirementCost = {
17
+ requirement: string;
18
+ tokens: number;
19
+ label: UsageLabel;
20
+ verified: boolean;
21
+ };
22
+ export type CostAttribution = {
23
+ attributed: RequirementCost[];
24
+ unattributedTokens: number;
25
+ unattributedLabel: UsageLabel;
26
+ verifiedRequirements: number;
27
+ unverifiedRequirements: number;
28
+ totalTokens: number;
29
+ totalLabel: UsageLabel;
30
+ /** Outcome-denominated metric: attributed tokens per verified requirement. */
31
+ tokensPerVerifiedRequirement?: number;
32
+ };
33
+ export declare function computeCostAttribution(entries: UsageSidecarEntry[], graph: TraceabilityGraph): CostAttribution;
34
+ export declare function serializeCostAttribution(attribution: CostAttribution): string;
35
+ /**
36
+ * Task-lifecycle usage: total tokens from task start to merged PR.
37
+ *
38
+ * Tokens — not dollars — are the model-invariant metric: switching between a
39
+ * frontier and an entry-level model changes the price per token but not the
40
+ * input/output consumption, so lifecycle token totals stay comparable across
41
+ * model routing changes and harness migrations. All input tiers count as
42
+ * input tokens processed (cached reads still consume tokens; they just bill
43
+ * less), and the tier split stays visible for cache-behavior analysis.
44
+ */
45
+ export type TaskUsageStage = {
46
+ stage: string;
47
+ tokens: number;
48
+ label: UsageLabel;
49
+ models: string[];
50
+ };
51
+ export type TaskUsageSummary = {
52
+ workflow: string;
53
+ entries: number;
54
+ inputTokens: number;
55
+ outputTokens: number;
56
+ totalTokens: number;
57
+ label: UsageLabel;
58
+ cacheHitRate?: number;
59
+ stages: TaskUsageStage[];
60
+ requirements: string[];
61
+ models: string[];
62
+ harnesses: string[];
63
+ };
64
+ export declare function summarizeTaskUsage(entries: UsageSidecarEntry[], workflow: string): TaskUsageSummary;
65
+ export declare function serializeTaskUsage(summary: TaskUsageSummary): string;
@@ -0,0 +1,119 @@
1
+ import { USAGE_TIERS } from "./usage-sidecar.js";
2
+ const totalTokens = (entry) => USAGE_TIERS.reduce((total, tier) => total + entry.tiers[tier], 0);
3
+ export function computeCostAttribution(entries, graph) {
4
+ const verifiedByRequirement = new Map(graph.links.map((link) => [link.requirement, link.evidence.length > 0]));
5
+ const byRequirement = new Map();
6
+ const unattributed = [];
7
+ for (const entry of entries) {
8
+ if (entry.requirement) {
9
+ const bucket = byRequirement.get(entry.requirement) ?? [];
10
+ bucket.push(entry);
11
+ byRequirement.set(entry.requirement, bucket);
12
+ }
13
+ else
14
+ unattributed.push(entry);
15
+ }
16
+ const labelOf = (group) => group.every((entry) => entry.label === "measured") ? "measured" : "estimated";
17
+ const attributed = [...byRequirement]
18
+ .map(([requirement, group]) => ({
19
+ requirement,
20
+ tokens: group.reduce((total, entry) => total + totalTokens(entry), 0),
21
+ label: labelOf(group),
22
+ verified: verifiedByRequirement.get(requirement) ?? false,
23
+ }))
24
+ .sort((a, b) => b.tokens - a.tokens || a.requirement.localeCompare(b.requirement));
25
+ const graphRequirements = new Set(graph.links.map((link) => link.requirement));
26
+ const verifiedRequirements = attributed.filter(({ verified }) => verified).length;
27
+ const unverifiedRequirements = attributed.filter(({ requirement }) => graphRequirements.has(requirement)).length -
28
+ verifiedRequirements;
29
+ const attributedToVerified = attributed
30
+ .filter(({ verified }) => verified)
31
+ .reduce((total, entry) => total + entry.tokens, 0);
32
+ return {
33
+ attributed,
34
+ unattributedTokens: unattributed.reduce((total, entry) => total + totalTokens(entry), 0),
35
+ unattributedLabel: labelOf(unattributed),
36
+ verifiedRequirements,
37
+ unverifiedRequirements,
38
+ totalTokens: entries.reduce((total, entry) => total + totalTokens(entry), 0),
39
+ totalLabel: labelOf(entries),
40
+ ...(verifiedRequirements > 0
41
+ ? { tokensPerVerifiedRequirement: Math.ceil(attributedToVerified / verifiedRequirements) }
42
+ : {}),
43
+ };
44
+ }
45
+ export function serializeCostAttribution(attribution) {
46
+ if (!attribution.attributed.length && !attribution.unattributedTokens)
47
+ return "No usage recorded against requirements yet. Record with `aiw usage-sidecar record --requirement=REQ-###`.";
48
+ const lines = [
49
+ `total_tokens: ${attribution.totalTokens.toLocaleString("en-US")} (${attribution.totalLabel})`,
50
+ `unattributed_tokens: ${attribution.unattributedTokens.toLocaleString("en-US")} (${attribution.unattributedLabel})`,
51
+ `verified_requirements: ${attribution.verifiedRequirements}`,
52
+ `unverified_requirements: ${attribution.unverifiedRequirements}`,
53
+ ...(attribution.tokensPerVerifiedRequirement === undefined
54
+ ? ["tokens_per_verified_requirement: n/a (no verified requirements yet)"]
55
+ : [
56
+ `tokens_per_verified_requirement: ${attribution.tokensPerVerifiedRequirement.toLocaleString("en-US")} (estimated; outcome-denominated)`,
57
+ ]),
58
+ "by requirement:",
59
+ ...attribution.attributed.map(({ requirement, tokens, label, verified }) => ` - ${requirement}: ${tokens.toLocaleString("en-US")} tokens (${label}, ${verified ? "verified" : "unverified"})`),
60
+ ];
61
+ return lines.join("\n");
62
+ }
63
+ export function summarizeTaskUsage(entries, workflow) {
64
+ const scoped = entries.filter((entry) => entry.workflow === workflow);
65
+ const labelOf = (group) => group.every((entry) => entry.label === "measured") ? "measured" : "estimated";
66
+ const inputTokens = scoped.reduce((total, entry) => total + entry.tiers["cache-read"] + entry.tiers["fresh-input"] + entry.tiers["cache-write"], 0);
67
+ const billableInput = scoped.reduce((total, entry) => total + entry.tiers["cache-read"] + entry.tiers["fresh-input"], 0);
68
+ const byCommand = new Map();
69
+ for (const entry of scoped) {
70
+ const bucket = byCommand.get(entry.command) ?? [];
71
+ bucket.push(entry);
72
+ byCommand.set(entry.command, bucket);
73
+ }
74
+ const stages = [...byCommand]
75
+ .map(([stage, group]) => ({
76
+ stage,
77
+ tokens: group.reduce((total, entry) => total + totalTokens(entry), 0),
78
+ label: labelOf(group),
79
+ models: [...new Set(group.map((entry) => entry.model ?? "unattributed"))].sort(),
80
+ }))
81
+ .sort((a, b) => b.tokens - a.tokens || a.stage.localeCompare(b.stage));
82
+ const uniqueSorted = (values) => [...new Set(values.filter((value) => value !== undefined))].sort();
83
+ return {
84
+ workflow,
85
+ entries: scoped.length,
86
+ inputTokens,
87
+ outputTokens: scoped.reduce((total, entry) => total + entry.tiers.output, 0),
88
+ totalTokens: scoped.reduce((total, entry) => total + totalTokens(entry), 0),
89
+ label: labelOf(scoped),
90
+ ...(billableInput > 0
91
+ ? {
92
+ cacheHitRate: scoped.reduce((total, entry) => total + entry.tiers["cache-read"], 0) / billableInput,
93
+ }
94
+ : {}),
95
+ stages,
96
+ requirements: uniqueSorted(scoped.map((entry) => entry.requirement)),
97
+ models: uniqueSorted(scoped.map((entry) => entry.model)),
98
+ harnesses: uniqueSorted(scoped.map((entry) => entry.harness)),
99
+ };
100
+ }
101
+ export function serializeTaskUsage(summary) {
102
+ if (!summary.entries)
103
+ return `No usage recorded for workflow ${summary.workflow}. Correlate runs with \`aiw usage-sidecar record --workflow=${summary.workflow}\`.`;
104
+ return [
105
+ `workflow: ${summary.workflow} (${summary.entries} entries, ${summary.label})`,
106
+ `input_tokens: ${summary.inputTokens.toLocaleString("en-US")} (cache-read + fresh-input + cache-write)`,
107
+ `output_tokens: ${summary.outputTokens.toLocaleString("en-US")}`,
108
+ `total_tokens: ${summary.totalTokens.toLocaleString("en-US")}`,
109
+ ...(summary.cacheHitRate === undefined
110
+ ? []
111
+ : [`cache_hit_rate: ${Math.round(summary.cacheHitRate * 100)}%`]),
112
+ ...(summary.models.length ? [`models: ${summary.models.join(", ")}`] : []),
113
+ ...(summary.harnesses.length ? [`harnesses: ${summary.harnesses.join(", ")}`] : []),
114
+ ...(summary.requirements.length ? [`requirements: ${summary.requirements.join(", ")}`] : []),
115
+ "by stage:",
116
+ ...summary.stages.map(({ stage, tokens, label, models }) => ` - ${stage}: ${tokens.toLocaleString("en-US")} tokens (${label}${models.length ? `; ${models.join(", ")}` : ""})`),
117
+ "metric: tokens are model-invariant — comparable across model routing changes and harness migrations.",
118
+ ].join("\n");
119
+ }
@@ -0,0 +1,61 @@
1
+ /**
2
+ * Session heartbeat: event-driven cache-TTL management.
3
+ *
4
+ * Harness hooks (Claude Code Stop/UserPromptSubmit, Gemini
5
+ * PostInvocation/Stop, Codex turn notifications) call `aiw heartbeat` on
6
+ * every event. Each beat measures the gap since the previous one:
7
+ *
8
+ * - gap < warn (default 4 min): active session, nothing to do;
9
+ * - warn <= gap < ttl (default 5 min): the provider cache is still warm —
10
+ * this request just read it at cache price — so a compact checkpoint is
11
+ * written now: if the session goes idle past the TTL, the next resume
12
+ * loads a small prefix instead of rebuilding the full one;
13
+ * - gap >= ttl: the cache already expired and this request paid the full
14
+ * rebuild — a compact checkpoint is written so the *next* gap hurts an
15
+ * order of magnitude less.
16
+ *
17
+ * The guarantee: a session never pays the full-price prefix rebuild twice
18
+ * in a row on the same context. No timers, no keep-alive pings: the gap is
19
+ * always measured on the next beat, which costs nothing locally.
20
+ *
21
+ * Timestamps are legitimate here: this is operational state (like workflow
22
+ * checkpoints), not a deterministic report.
23
+ */
24
+ export declare const HEARTBEAT_SCHEMA = 1;
25
+ export declare const HEARTBEAT_STATE_PATH = "heartbeats.yml";
26
+ export declare const HEARTBEAT_WARN_SECONDS = 240;
27
+ export declare const HEARTBEAT_TTL_SECONDS = 300;
28
+ export type HeartbeatDecision = "session-start" | "active" | "compact-before-expiry" | "cache-expired";
29
+ export type HeartbeatSession = {
30
+ id: string;
31
+ lastBeat: string;
32
+ beats: number;
33
+ status: "warm" | "cold";
34
+ actions: number;
35
+ };
36
+ export type HeartbeatState = {
37
+ schema: number;
38
+ sessions: HeartbeatSession[];
39
+ };
40
+ export type HeartbeatOutcome = {
41
+ decision: HeartbeatDecision;
42
+ gapSeconds: number | undefined;
43
+ session: HeartbeatSession;
44
+ /** Set when a compact checkpoint should be written (decision tier warn/ttl). */
45
+ checkpoint: {
46
+ session: string;
47
+ index: number;
48
+ decision: HeartbeatDecision;
49
+ } | undefined;
50
+ };
51
+ export declare function validateHeartbeatThresholds(warnSeconds: number, ttlSeconds: number): void;
52
+ export declare function emptyHeartbeatState(): HeartbeatState;
53
+ export declare function parseHeartbeatState(content: string): HeartbeatState;
54
+ export declare function serializeHeartbeatState(state: HeartbeatState): string;
55
+ /** Record a beat and decide. `now` is injectable for deterministic tests. */
56
+ export declare function recordBeat(state: HeartbeatState, sessionId: string, now: Date, warnSeconds?: number, ttlSeconds?: number): {
57
+ state: HeartbeatState;
58
+ outcome: HeartbeatOutcome;
59
+ };
60
+ export declare function serializeHeartbeatCheckpoint(checkpoint: NonNullable<HeartbeatOutcome["checkpoint"]>, gapSeconds: number, ttlSeconds: number): string;
61
+ export declare function describeHeartbeatOutcome(outcome: HeartbeatOutcome): string;
@@ -0,0 +1,158 @@
1
+ import { parseDocument } from "yaml";
2
+ /**
3
+ * Session heartbeat: event-driven cache-TTL management.
4
+ *
5
+ * Harness hooks (Claude Code Stop/UserPromptSubmit, Gemini
6
+ * PostInvocation/Stop, Codex turn notifications) call `aiw heartbeat` on
7
+ * every event. Each beat measures the gap since the previous one:
8
+ *
9
+ * - gap < warn (default 4 min): active session, nothing to do;
10
+ * - warn <= gap < ttl (default 5 min): the provider cache is still warm —
11
+ * this request just read it at cache price — so a compact checkpoint is
12
+ * written now: if the session goes idle past the TTL, the next resume
13
+ * loads a small prefix instead of rebuilding the full one;
14
+ * - gap >= ttl: the cache already expired and this request paid the full
15
+ * rebuild — a compact checkpoint is written so the *next* gap hurts an
16
+ * order of magnitude less.
17
+ *
18
+ * The guarantee: a session never pays the full-price prefix rebuild twice
19
+ * in a row on the same context. No timers, no keep-alive pings: the gap is
20
+ * always measured on the next beat, which costs nothing locally.
21
+ *
22
+ * Timestamps are legitimate here: this is operational state (like workflow
23
+ * checkpoints), not a deterministic report.
24
+ */
25
+ export const HEARTBEAT_SCHEMA = 1;
26
+ export const HEARTBEAT_STATE_PATH = "heartbeats.yml";
27
+ export const HEARTBEAT_WARN_SECONDS = 240;
28
+ export const HEARTBEAT_TTL_SECONDS = 300;
29
+ export function validateHeartbeatThresholds(warnSeconds, ttlSeconds) {
30
+ for (const [name, value] of [
31
+ ["warn", warnSeconds],
32
+ ["ttl", ttlSeconds],
33
+ ])
34
+ if (!Number.isInteger(value) || value <= 0)
35
+ throw new Error(`Heartbeat ${name} threshold must be a positive integer (seconds).`);
36
+ if (warnSeconds >= ttlSeconds)
37
+ throw new Error("Heartbeat warn threshold must be smaller than the TTL threshold.");
38
+ }
39
+ export function emptyHeartbeatState() {
40
+ return { schema: HEARTBEAT_SCHEMA, sessions: [] };
41
+ }
42
+ export function parseHeartbeatState(content) {
43
+ if (!content.trim())
44
+ return emptyHeartbeatState();
45
+ const document = parseDocument(content);
46
+ if (document.errors.length)
47
+ throw new Error("Heartbeat state is not valid YAML.");
48
+ const schema = document.get("schema");
49
+ if (schema !== HEARTBEAT_SCHEMA)
50
+ throw new Error(`Heartbeat state schema must be version ${HEARTBEAT_SCHEMA}.`);
51
+ const sessionsNode = document.get("sessions");
52
+ const sessions = [];
53
+ if (sessionsNode !== null && sessionsNode !== undefined) {
54
+ const items = Array.isArray(sessionsNode)
55
+ ? sessionsNode
56
+ : (sessionsNode.items ?? []);
57
+ for (const raw of items) {
58
+ const read = (key) => typeof raw.get === "function" ? raw.get(key) : raw[key];
59
+ const id = String(read("id"));
60
+ if (!id.trim())
61
+ throw new Error("Heartbeat session id must not be empty.");
62
+ const lastBeat = String(read("last_beat"));
63
+ if (Number.isNaN(Date.parse(lastBeat)))
64
+ throw new Error(`Heartbeat session ${id} has an invalid last_beat timestamp.`);
65
+ sessions.push({
66
+ id,
67
+ lastBeat,
68
+ beats: Number(read("beats")),
69
+ status: read("status") === "cold" ? "cold" : "warm",
70
+ actions: Number(read("actions") ?? 0),
71
+ });
72
+ }
73
+ }
74
+ return { schema: HEARTBEAT_SCHEMA, sessions };
75
+ }
76
+ export function serializeHeartbeatState(state) {
77
+ const lines = [`schema: ${state.schema}`, "sessions:"];
78
+ if (!state.sessions.length)
79
+ lines.push(" []");
80
+ for (const session of state.sessions)
81
+ lines.push(` - id: ${session.id}`, ` last_beat: ${session.lastBeat}`, ` beats: ${session.beats}`, ` status: ${session.status}`, ` actions: ${session.actions}`);
82
+ return lines.join("\n") + "\n";
83
+ }
84
+ /** Record a beat and decide. `now` is injectable for deterministic tests. */
85
+ export function recordBeat(state, sessionId, now, warnSeconds = HEARTBEAT_WARN_SECONDS, ttlSeconds = HEARTBEAT_TTL_SECONDS) {
86
+ if (!sessionId.trim())
87
+ throw new Error("Heartbeat session id must not be empty.");
88
+ validateHeartbeatThresholds(warnSeconds, ttlSeconds);
89
+ const iso = now.toISOString();
90
+ const previous = state.sessions.find((session) => session.id === sessionId);
91
+ let decision;
92
+ let gapSeconds;
93
+ if (!previous) {
94
+ decision = "session-start";
95
+ }
96
+ else {
97
+ gapSeconds = Math.max(0, Math.round((now.getTime() - Date.parse(previous.lastBeat)) / 1000));
98
+ decision =
99
+ gapSeconds < warnSeconds
100
+ ? "active"
101
+ : gapSeconds < ttlSeconds
102
+ ? "compact-before-expiry"
103
+ : "cache-expired";
104
+ }
105
+ const updated = {
106
+ id: sessionId,
107
+ lastBeat: iso,
108
+ beats: (previous?.beats ?? 0) + 1,
109
+ status: decision === "cache-expired" ? "cold" : "warm",
110
+ actions: previous?.actions ?? 0,
111
+ };
112
+ const shouldCheckpoint = decision === "compact-before-expiry" || decision === "cache-expired";
113
+ if (shouldCheckpoint)
114
+ updated.actions += 1;
115
+ const sessions = state.sessions.filter((session) => session.id !== sessionId).concat(updated);
116
+ sessions.sort((a, b) => (a.id < b.id ? -1 : a.id > b.id ? 1 : 0));
117
+ return {
118
+ state: { schema: state.schema, sessions },
119
+ outcome: {
120
+ decision,
121
+ gapSeconds,
122
+ session: updated,
123
+ checkpoint: shouldCheckpoint
124
+ ? { session: sessionId, index: updated.actions, decision }
125
+ : undefined,
126
+ },
127
+ };
128
+ }
129
+ export function serializeHeartbeatCheckpoint(checkpoint, gapSeconds, ttlSeconds) {
130
+ return ([
131
+ "schema: 1",
132
+ `session: ${checkpoint.session}`,
133
+ `decision: ${checkpoint.decision}`,
134
+ `gap_seconds: ${gapSeconds}`,
135
+ `ttl_seconds: ${ttlSeconds}`,
136
+ "action: resume from this checkpoint; prefer harness compaction (/compact) before continuing the session",
137
+ ].join("\n") + "\n");
138
+ }
139
+ export function describeHeartbeatOutcome(outcome) {
140
+ const gap = outcome.gapSeconds === undefined ? "" : ` (gap ${formatGap(outcome.gapSeconds)})`;
141
+ switch (outcome.decision) {
142
+ case "session-start":
143
+ return `heartbeat: session ${outcome.session.id} started`;
144
+ case "active":
145
+ return `heartbeat: session ${outcome.session.id} active${gap}`;
146
+ case "compact-before-expiry":
147
+ return `heartbeat: session ${outcome.session.id} idle${gap} — cache still warm; compact checkpoint #${outcome.session.actions} saved for a cheap resume`;
148
+ case "cache-expired":
149
+ return `heartbeat: session ${outcome.session.id} idle${gap} — cache expired and this request paid the rebuild; compact checkpoint #${outcome.session.actions} saved so the next gap is cheap`;
150
+ }
151
+ }
152
+ function formatGap(seconds) {
153
+ if (seconds < 60)
154
+ return `${seconds}s`;
155
+ const minutes = Math.floor(seconds / 60);
156
+ const rest = seconds % 60;
157
+ return rest ? `${minutes}m${rest}s` : `${minutes}m`;
158
+ }