baldart 3.29.0 → 3.30.0

package/CHANGELOG.md

@@ -5,6 +5,31 @@ All notable changes to BALDART will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [3.30.0] - 2026-05-29
+
+Hardens the **overlay** system — the layer that turns generic framework skills into project-specific knowledge. An audit surfaced a structural asymmetry: agent/command overlays are *compile-time merged* (the overlay is materially fused into the generated file → guaranteed delivery), but skill overlays use *runtime concatenation* (a line of prose tells the agent to read `.baldart/overlays/<skill>.md` → best-effort, nothing verifies it loaded). The most important overlays for knowledge transfer had the weakest guarantee. This release ships the high-certainty authoring/DX fixes now and adds **observation-only telemetry** to gather real data on whether skill overlays actually load — before deciding whether a stronger enforcement mechanism is worth building. **No new `baldart.config.yml` keys** — the telemetry hook is always-on (like `framework-edit-gate` / `agent-discovery-gate`), so the schema-propagation rule does not apply.
+
+### Added — overlay-load telemetry (observation only)
+
+- **[framework/.claude/hooks/overlay-telemetry.sh](framework/.claude/hooks/overlay-telemetry.sh)**: a fail-safe `PostToolUse` hook (matcher `Read`) that appends one JSON line to `.baldart/telemetry/overlay-loads.jsonl` every time a `.baldart/overlays/*.md` file is read. Objective, not self-reported — the failure mode being measured is "the agent forgot the overlay", which a self-report would corrupt in exactly that case. Written in **bash, not node**, on purpose: it fires on every `Read` (the hottest tool) in the critical path, so a per-read node cold-start (~50ms) would tax every file read — a bash fork (~5ms) keeps the hot path cheap. **Numerator-only by design**: skill invocation is prompt-expansion, not a tool call, so the denominator (skill runs that *should* have loaded an overlay) is not hook-observable (claude-code [#43630](https://github.com/anthropics/claude-code/issues/43630), [#22655](https://github.com/anthropics/claude-code/issues/22655)). Registered in `HOOK_REGISTRY` (`baldart-overlay-telemetry`) → auto-registered by `add`, backfilled by `update`, reported by `doctor` — no call-site changes (registry-driven).
+- **[framework/scripts/analyze-overlay-loads.js](framework/scripts/analyze-overlay-loads.js)**: zero-dependency aggregator that cross-references the read log against the overlays that exist on disk. Headline signal: a **skill** overlay that exists but is read ~never (`NEVER-LOADED`). Agent/command overlays (compile-time merged) are deliberately not flagged on zero reads.
+- **[framework/docs/OVERLAY-TELEMETRY.md](framework/docs/OVERLAY-TELEMETRY.md)**: full reading guide, with an explicit "conservative proxy — UNDER-counts" section (in-context reads and prompt-expansion are invisible to the hook; a low count is a reason to investigate, not proof of a miss).
+
+### Changed — overlay authoring/DX parity for skills
+
+- **[src/commands/overlay.js](src/commands/overlay.js)** `validate` (skill path): now renders a **merge dry-run** for skill overlays — section counts, concatenated size, and a per-marker check flagging `[OVERRIDE]/[APPEND]/[PREPEND]` headings that don't anchor to any base H2. Previously it printed only "valid, concat model", so a skill-overlay author never saw what the merge would look like (agent/command overlays already got this). Powered by a new pure, unit-tested `analyzeSkillConcat()` helper.
+- **[src/commands/overlay.js](src/commands/overlay.js)** `drift` (skill path): when the base file changed, now lists *which* overlay markers were orphaned by the change (heading renamed/removed upstream) — the same actionable feedback agent/command overlays get from the merger, instead of a bare sha-mismatch warning.
+- **[src/commands/overlay.js](src/commands/overlay.js)** `scaffoldBody` (skill path): the `overlay scaffold skill/<x>` skeleton is now a guided template — explains the runtime-concat model honestly, enumerates the typical project-fact sections (canonical paths, domain vocabulary, BLOCKING rules, mandated stack), and shows a ready-to-edit `[OVERRIDE]` block. Improves authoring for all skills, not just the two that ship an example overlay.
+- **[framework/agents/project-context.md](framework/agents/project-context.md)** §5 precedence rules: corrected the overclaim that a skill overlay's `## [OVERRIDE] <topic>` "replaces" the base section. For skills nothing parses the markers — they are textual conventions the executing agent honours (best-effort), unlike agent/command overlays which are materially fused (§5.bis). The doc now matches `scaffoldBody`'s honest framing instead of the middle.
+
+### Changed — drift-check counter
+
+- **[framework/.claude/skills/baldart-update/SKILL.md](framework/.claude/skills/baldart-update/SKILL.md)**: `hook_registry_entries` 3 → 4 (the new telemetry hook). The `add`/`update`/`doctor` integration is registry-driven, so no narration changes were needed.
+
+### Added — tests
+
+- **[src/utils/__tests__/analyze-skill-concat.test.js](src/utils/__tests__/analyze-skill-concat.test.js)**: 5 cases covering marker matching, orphan detection, plain (new) sections, line counts, and the empty-overlay edge case.
+
 ## [3.29.0] - 2026-05-29
 
 Closes a two-part process gap in `npx baldart update` reported from a non-TTY agent context on a shared consumer repo: (1) plumbing/collaboration merge commits were mislabeled `custom-other`, downgrading an otherwise overlay-able divergence set into `mixed`; (2) an automated agent had **no** non-destructive path to complete an update when the consumer carried local `.framework/` commits — `--yes` refused, the interactive prompt needed a TTY, and `--reset` is gated on a clean working tree. **No new `baldart.config.yml` keys** — `--on-divergence` is a CLI flag, not a config flag, so the schema-propagation rule does not apply.

package/VERSION

@@ -1 +1 @@
-3.29.0
+3.30.0

package/framework/.claude/hooks/overlay-telemetry.sh

@@ -0,0 +1,88 @@
+#!/usr/bin/env bash
+#
+# overlay-telemetry.sh — BALDART PostToolUse hook (matcher: Read).
+#
+# OBSERVATION ONLY. Appends one JSON line to
+# `.baldart/telemetry/overlay-loads.jsonl` every time a `.baldart/overlays/*.md`
+# file is read. This is the *numerator* of the overlay-load question: when the
+# agent runs a skill that HAS an overlay, does it actually load it? A `Read` of
+# the overlay file is the one objective signal — skill self-report would be
+# corrupted in exactly the case we measure ("the agent forgot the overlay").
+#
+# Bash, NOT node, on purpose: this fires on EVERY `Read` (the hottest tool) in
+# the consumer's critical path. A per-read node cold-start (~50ms) would tax
+# every file read; a bash fork (~5ms) keeps the hot path cheap. The work is just
+# a path match + a conditional append, so bash is sufficient. Same precedent as
+# agent-discovery-info.sh.
+#
+# Numerator-only by design — skill invocation is prompt-expansion, not a tool
+# call, so the denominator is not hook-observable (claude-code #43630, #22655).
+# See framework/docs/OVERLAY-TELEMETRY.md for the under-count caveats.
+#
+# Fail-safe: always exit 0, never disrupt the session.
+
+set -u
+exec 2>/dev/null  # never surface stderr to the user
+
+INPUT=""
+if [ ! -t 0 ]; then
+  INPUT="$(cat)"
+fi
+[ -z "$INPUT" ] && exit 0
+
+get_field() {
+  # $1 = JSON key (top-level or nested — sed fallback matches anywhere)
+  if command -v jq >/dev/null 2>&1; then
+    printf '%s' "$INPUT" | jq -r "$2 // empty" 2>/dev/null
+  else
+    printf '%s' "$INPUT" | sed -n "s/.*\"$1\"[[:space:]]*:[[:space:]]*\"\\([^\"]*\\)\".*/\\1/p" | head -1
+  fi
+}
+
+TOOL_NAME="$(get_field tool_name '.tool_name')"
+[ "$TOOL_NAME" = "Read" ] || exit 0
+
+FILE_PATH="$(get_field file_path '.tool_input.file_path')"
+[ -n "$FILE_PATH" ] || exit 0
+
+CWD="$(get_field cwd '.cwd')"
+[ -n "$CWD" ] || CWD="$PWD"
+SESSION_ID="$(get_field session_id '.session_id')"
+
+# Resolve relative paths against cwd.
+case "$FILE_PATH" in
+  /*) ABS="$FILE_PATH" ;;
+  *)  ABS="$CWD/$FILE_PATH" ;;
+esac
+
+# Must live under .baldart/overlays/.
+case "$ABS" in
+  */.baldart/overlays/*) ;;
+  *) exit 0 ;;
+esac
+REL="${ABS##*/.baldart/overlays/}"   # everything after the last marker
+
+# Classify — mirror src/utils/overlay-merger / overlay.js path logic.
+# Order matters: deeper/excluded patterns before the catch-alls.
+KIND=""
+NAME=""
+case "$REL" in
+  *.example.md) exit 0 ;;
+  README.md) exit 0 ;;
+  */*/*) exit 0 ;;                                                       # 3+ levels — ignore
+  agents/*.md)   KIND="agent";   NAME="${REL#agents/}";   NAME="${NAME%.md}" ;;
+  commands/*.md) KIND="command"; NAME="${REL#commands/}"; NAME="${NAME%.md}" ;;
+  */*) exit 0 ;;                                                         # 2 levels, not agents/commands — ignore
+  *.md) KIND="skill"; NAME="${REL%.md}" ;;
+  *) exit 0 ;;
+esac
+[ -n "$KIND" ] && [ -n "$NAME" ] || exit 0
+
+TS="$(date -u +%Y-%m-%dT%H:%M:%SZ)"
+if [ -n "$SESSION_ID" ]; then SID="\"$SESSION_ID\""; else SID="null"; fi
+
+mkdir -p "$CWD/.baldart/telemetry" || exit 0
+printf '{"ts":"%s","event":"overlay-read","kind":"%s","name":"%s","session_id":%s}\n' \
+  "$TS" "$KIND" "$NAME" "$SID" >> "$CWD/.baldart/telemetry/overlay-loads.jsonl"
+
+exit 0

package/framework/.claude/skills/baldart-update/SKILL.md

@@ -13,7 +13,7 @@ sources, a warning is surfaced in the workflow run. The decision whether
 this skill needs an update is left to a human reviewer.
 
 update_js_prompts: 7
-hook_registry_entries: 3
+hook_registry_entries: 4
 config_template_top_level_keys: features,git,identity,lsp,paths,stack,tools,version
 last_verified: v3.26.0
 -->

package/framework/agents/project-context.md

@@ -86,7 +86,7 @@ mode: extend                # extend | override
 ### Precedence rules
 
 1. **Default**: overlays *extend* base skills. If overlay and base both apply, both run.
-2. **Explicit override**: inside an overlay (regardless of `mode`), a section marked `## [OVERRIDE] <topic>` replaces the same `<topic>` from the base skill.
+2. **Explicit override**: inside a skill overlay, a section marked `## [OVERRIDE] <topic>` signals the agent to prefer the overlay's version of `<topic>` over the base skill's. **This is a textual convention, not a parsed merge.** Skill overlays use runtime concatenation (§5.bis): nothing in the CLI parses `[OVERRIDE]`/`[APPEND]`/`[PREPEND]` for skills — the merger (`overlay-merger.js`) parses those markers **only for agent/command overlays**, which are compiled at `add`/`update` time. For skills, the marker is an instruction the executing agent honours when it reads the overlay; honouring it is best-effort, not structurally enforced. (Agent/command overrides, by contrast, are materially fused into the generated file — see §5.bis.)
 3. **Stack/identity conflicts**: when an overlay's stack/identity rule contradicts `baldart.config.yml`, the overlay wins (the user wrote it deliberately) — but the skill MUST log a one-line "overlay overrides config" notice in its output.
 
 ### Version drift handling

package/framework/docs/OVERLAY-TELEMETRY.md

@@ -0,0 +1,77 @@
+# Overlay-Load Telemetry
+
+This doc describes the telemetry layer that records, **objectively**, whether skill overlays are actually loaded at runtime. The layer is **observation only** — it changes no behaviour. It exists so we can decide, with real data, whether skill overlays need a stronger delivery guarantee than the current runtime-concat model.
+
+## Why this exists
+
+Skill overlays and agent/command overlays have **opposite delivery guarantees**:
+
+- **Agent / command overlays** are *compile-time merged* — the overlay is materially fused into the generated `.claude/agents/<x>.md` at `add`/`update` (`overlay-merger.js`). When Claude loads the subagent, the overlay is already there. The guarantee is structural.
+- **Skill overlays** use *runtime concatenation* — `SKILL.md` stays a symlink, the overlay lives separately in `.baldart/overlays/<skill>.md`, and a line of prose tells the agent "load `.baldart/overlays/<skill>.md` if present". Nothing verifies the agent did. The guarantee is best-effort.
+
+So the *most* important overlays for knowledge transfer (skills) have the *weakest* guarantee. The architectural fixes for that (SessionStart awareness injection, PreToolUse-on-Skill injection, or compile-time merge of `SKILL.md`) are all non-trivial — and we have **no evidence** overlays actually fail to load in practice. Before building any of them, we measure. This layer is that measurement.
+
+## What it captures (and what it can't)
+
+A single PostToolUse hook (`framework/.claude/hooks/overlay-telemetry.sh`, matcher `Read`) appends one JSON line to `.baldart/telemetry/overlay-loads.jsonl` every time a `.baldart/overlays/*.md` file is read:
+
+```json
+{"ts":"2026-05-29T08:00:00Z","event":"overlay-read","kind":"skill","name":"bug","session_id":"..."}
+```
+
+### Why a hook, not agent self-report
+
+The failure mode we measure is *"the agent forgot to load the overlay."* A self-reported `loaded=yes/no` would be corrupted in exactly that case — an agent that forgot to load would also forget to log, or log `yes` without having done it. Only an objective `Read` observation is trustworthy here.
+
+### Numerator only — the known limitation
+
+We capture the **numerator** (overlay reads), not the **denominator** (skill runs that *should* have loaded an overlay). The denominator is not hook-observable: skills are *prompt-expansion* in Claude Code, not tool calls, so there is no `PreToolUse` on skill invocation and the tool-call payload does not carry the invoking skill's name (anthropics/claude-code [#43630](https://github.com/anthropics/claude-code/issues/43630), [#22655](https://github.com/anthropics/claude-code/issues/22655)).
+
+This makes the metric a **conservative proxy that UNDER-counts**:
+
+- Overlay content already in the agent's context (no explicit `Read`) is invisible.
+- A `Read` is not proof the content influenced the output — only that it was loaded.
+- Reads performed in ways the main-loop hook doesn't observe are missed.
+
+Every bias points the same direction: the observed load-rate looks **worse** than reality. That matters because under-counting biases toward "build the enforcement" — the very thing this telemetry exists to gate. **Treat a low count as a reason to investigate, not as proof of a miss.** (Same honesty convention as `agent-discovery-info.sh`'s #10373 caveat and the "conservative proxy" language in `FIX-APPLICATION-TELEMETRY.md`.)
+
+## How to read the data
+
+```bash
+node framework/scripts/analyze-overlay-loads.js
+# or, non-default locations:
+node framework/scripts/analyze-overlay-loads.js --log <path.jsonl> --overlays <dir>
+```
+
+The aggregator cross-references the read log against the overlays that **exist on disk** and prints per-overlay read counts. The headline signal:
+
+| Column / footer | What it means |
+|---|---|
+| `Flag = NEVER-LOADED` | A **skill** overlay exists but was never read. The alarm signal — authored knowledge the agent isn't picking up. (Subject to the under-count caveat above.) |
+| `Reads` low but non-zero | Overlay loads sometimes. Compare against how often the skill ran (you'll have to estimate the denominator from your own usage). |
+| Agent/command rows, zero reads | **Expected, not a finding.** These overlays are compile-time merged into the generated file — they're not meant to be read at runtime. The table does not flag them. |
+| `Reads logged for overlays no longer on disk` | Stale/renamed overlays. Cosmetic. |
+
+## When to act on the data
+
+After a few weeks of real usage with at least one authored skill overlay:
+
+1. **`NEVER-LOADED` skill overlays.** If a skill overlay you actively rely on shows zero reads across many sessions where that skill clearly ran, that's the strongest available evidence that runtime concat is dropping it — and the case for an enforcement mechanism (SessionStart awareness, PreToolUse injection, or `SKILL.md` compile-time merge) becomes data-backed rather than speculative.
+2. **Healthy load pattern.** If your authored skill overlays show steady reads, the runtime-concat model is working in practice and enforcement is not worth its cost. Ship the DX improvements and leave the architecture alone.
+
+## Cost
+
+Be honest about where the cost lands: the hook fires on **every `Read`** — the most frequent tool — not just on overlay reads. So the per-read cost is what matters, and it is paid hundreds of times per session in the PostToolUse critical path (the model waits for the hook before seeing the tool result).
+
+- **Every read** (overwhelmingly non-overlay): one bash fork + a path-prefix match, then exit. This is why the hook is bash, not node — a node cold-start (~50ms) per read would be a real tax; a bash fork (~5ms) keeps the hot path cheap. There is no cheaper event: observing "an overlay was read" *requires* hooking `Read`, so the only lever is making the hook itself cheap.
+- **Only on an actual overlay read** (rare): one `mkdir -p` + one ~120-byte append.
+
+The hook is fail-safe (any error → exit 0, never blocks). The aggregator has zero dependencies and runs in well under a second. The log is append-only and rotates only by hand (delete the file to reset the window).
+
+## Pointers
+
+- Hook: `framework/.claude/hooks/overlay-telemetry.sh`
+- Registration: `src/utils/hooks.js` → `HOOK_REGISTRY` entry `baldart-overlay-telemetry` (auto-registered by `add`, backfilled by `update`, reported by `doctor`)
+- Aggregator: `framework/scripts/analyze-overlay-loads.js`
+- Overlay model: `framework/agents/project-context.md` §5 (skills, runtime concat) and §5.bis (agent/command, compile-time merge)
+- Sibling telemetry layer: `framework/docs/FIX-APPLICATION-TELEMETRY.md`

package/framework/scripts/analyze-overlay-loads.js

@@ -0,0 +1,130 @@
+#!/usr/bin/env node
+// Aggregate overlay-load telemetry. Cross-references the objective read log
+// (.baldart/telemetry/overlay-loads.jsonl, written by the overlay-telemetry
+// PostToolUse hook) against the overlays that actually EXIST on disk
+// (.baldart/overlays/), and reports per-overlay load counts.
+//
+// The headline signal is the LAST column: a SKILL overlay that exists but is
+// read ~never is the alarm — runtime concatenation is not structurally
+// enforced, so a never-read skill overlay is knowledge the project authored but
+// the agent isn't picking up. (Agent/command overlays are compile-time merged
+// into the generated file, so they're NOT expected to be read at runtime — a
+// zero count there is normal, not a problem. The table marks them accordingly.)
+//
+// This is a CONSERVATIVE proxy. The hook can only see explicit `Read` calls, so
+// it UNDER-counts: overlay content already in context (no Read) and reads done
+// in ways the main-loop hook doesn't observe are invisible. A low count is a
+// reason to investigate, not proof of a miss. See framework/docs/OVERLAY-TELEMETRY.md.
+//
+// Usage:
+//   node framework/scripts/analyze-overlay-loads.js
+//   node framework/scripts/analyze-overlay-loads.js --log <path.jsonl> --overlays <dir>
+
+'use strict';
+const fs = require('fs');
+const path = require('path');
+
+function argVal(flag, fallback) {
+  const i = process.argv.indexOf(flag);
+  return i !== -1 && process.argv[i + 1] ? process.argv[i + 1] : fallback;
+}
+
+const LOG_PATH = argVal('--log', path.join('.baldart', 'telemetry', 'overlay-loads.jsonl'));
+const OVERLAYS_DIR = argVal('--overlays', path.join('.baldart', 'overlays'));
+
+// --- read the objective read log -------------------------------------------
+
+const reads = new Map(); // "kind/name" → { count, sessions:Set, first, last }
+let logLines = 0;
+let logBad = 0;
+
+if (fs.existsSync(LOG_PATH)) {
+  const raw = fs.readFileSync(LOG_PATH, 'utf8');
+  for (const line of raw.split(/\r?\n/)) {
+    if (!line.trim()) continue;
+    logLines += 1;
+    let row;
+    try { row = JSON.parse(line); } catch (_) { logBad += 1; continue; }
+    if (!row || row.event !== 'overlay-read' || !row.kind || !row.name) { logBad += 1; continue; }
+    const key = `${row.kind}/${row.name}`;
+    const e = reads.get(key) || { count: 0, sessions: new Set(), first: null, last: null };
+    e.count += 1;
+    if (row.session_id) e.sessions.add(row.session_id);
+    if (row.ts) {
+      if (!e.first || row.ts < e.first) e.first = row.ts;
+      if (!e.last || row.ts > e.last) e.last = row.ts;
+    }
+    reads.set(key, e);
+  }
+}
+
+// --- enumerate overlays that exist on disk ---------------------------------
+
+function listOverlays(dir) {
+  const out = []; // { kind, name, key }
+  if (!fs.existsSync(dir)) return out;
+  const add = (sub, kind) => {
+    const d = sub ? path.join(dir, sub) : dir;
+    if (!fs.existsSync(d)) return;
+    for (const f of fs.readdirSync(d)) {
+      if (!f.endsWith('.md') || f.endsWith('.example.md') || f === 'README.md') continue;
+      const name = f.replace(/\.md$/, '');
+      out.push({ kind, name, key: `${kind}/${name}` });
+    }
+  };
+  add('', 'skill');
+  add('agents', 'agent');
+  add('commands', 'command');
+  return out;
+}
+
+const overlays = listOverlays(OVERLAYS_DIR);
+const overlayKeys = new Set(overlays.map((o) => o.key));
+
+// --- build rows -------------------------------------------------------------
+
+const rows = overlays.map((o) => {
+  const r = reads.get(o.key);
+  return {
+    kind: o.kind,
+    name: o.name,
+    reads: r ? r.count : 0,
+    sessions: r ? r.sessions.size : 0,
+    last: r && r.last ? r.last.slice(0, 10) : '-',
+    // Only skills are expected to be read at runtime; agent/command overlays are
+    // fused at build time, so a zero there is expected, not a finding.
+    flag: (o.kind === 'skill' && (!r || r.count === 0)) ? 'NEVER-LOADED' : '',
+  };
+});
+rows.sort((a, b) => a.reads - b.reads || a.kind.localeCompare(b.kind) || a.name.localeCompare(b.name));
+
+// Reads of overlays that no longer exist on disk (stale/renamed) — surfaced separately.
+const orphanReads = [...reads.keys()].filter((k) => !overlayKeys.has(k));
+
+// --- render -----------------------------------------------------------------
+
+const headers = ['Kind', 'Name', 'Reads', 'Sessions', 'Last-read', 'Flag'];
+const cols = ['kind', 'name', 'reads', 'sessions', 'last', 'flag'];
+const widths = headers.map((h, i) => Math.max(h.length, ...rows.map((r) => String(r[cols[i]]).length), 1));
+const fmt = (cells) => cells.map((c, i) => String(c).padEnd(widths[i])).join('  ');
+
+console.log(fmt(headers));
+console.log(widths.map((w) => '-'.repeat(w)).join('  '));
+for (const r of rows) console.log(fmt(cols.map((c) => r[c])));
+
+console.log('');
+console.log(`Overlays on disk: ${overlays.length}  |  Log rows: ${logLines}${logBad ? ` (${logBad} skipped)` : ''}  |  Log: ${fs.existsSync(LOG_PATH) ? LOG_PATH : 'NOT FOUND (no telemetry yet)'}`);
+
+const neverLoaded = rows.filter((r) => r.flag === 'NEVER-LOADED');
+if (neverLoaded.length) {
+  console.log('');
+  console.log('⚠  Skill overlays that EXIST but were never read (runtime concat may not be loading them):');
+  for (const r of neverLoaded) console.log(`   ${r.kind}/${r.name}`);
+  console.log('   Conservative proxy — see framework/docs/OVERLAY-TELEMETRY.md before acting (in-context reads are invisible to the hook).');
+}
+
+if (orphanReads.length) {
+  console.log('');
+  console.log('ℹ  Reads logged for overlays no longer on disk (renamed/removed):');
+  for (const k of orphanReads) console.log(`   ${k}`);
+}

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "baldart",
-  "version": "3.29.0",
+  "version": "3.30.0",
   "description": "Claude Agent Framework - Reusable framework for coordinating AI agents and humans in software projects",
   "bin": {
     "baldart": "./bin/baldart.js"

package/src/commands/overlay.js

@@ -2,7 +2,7 @@ const fs = require('fs');
 const path = require('path');
 const yaml = require('js-yaml');
 const UI = require('../utils/ui');
-const { mergeOverlay, computeBaseFileSha } = require('../utils/overlay-merger');
+const { mergeOverlay, computeBaseFileSha, parseMarkdown } = require('../utils/overlay-merger');
 
 const FRAMEWORK_DIR = '.framework';
 const FRAMEWORK_PAYLOAD = path.join(FRAMEWORK_DIR, 'framework');
@@ -72,18 +72,68 @@ function parseFrontmatter(content) {
   }
 }
 
+// Skill overlays use the runtime-concat model — the executing agent reads the
+// base SKILL.md and the overlay and applies the overlay's guidance itself.
+// Unlike agent/command overlays, NOTHING parses the [OVERRIDE]/[APPEND]/[PREPEND]
+// markers here (see framework/agents/project-context.md § 5). This helper gives
+// the skill-overlay author the same "what does the merge look like?" feedback the
+// marker model already gives for agents/commands: it lists the overlay's sections,
+// flags markers whose target heading no longer exists in the base (the agent has
+// nothing to anchor the override to), and reports the concatenated size. Pure — no IO.
+function analyzeSkillConcat(baseContent, overlayContent) {
+  const base = parseMarkdown(baseContent);
+  const overlay = parseMarkdown(overlayContent);
+  const baseHeadings = new Set(base.sections.map((s) => s.heading).filter(Boolean));
+
+  const markerSections = [];
+  let plainSections = 0;
+  for (const s of overlay.sections) {
+    if (s.marker) {
+      markerSections.push({ marker: s.marker, heading: s.heading, orphan: !baseHeadings.has(s.heading) });
+    } else if (s.heading) {
+      plainSections += 1;
+    }
+  }
+  return {
+    baseLines: baseContent.split('\n').length,
+    overlayLines: overlayContent.split('\n').length,
+    baseSectionCount: base.sections.length,
+    markerSections,
+    plainSections,
+    orphans: markerSections.filter((m) => m.orphan),
+  };
+}
+
 function scaffoldBody(kind, name) {
   if (kind === 'skill') {
     return [
       '',
-      `> Skill overlays are concatenated to the base SKILL.md at runtime by the Claude`,
-      `> agent. The merger does NOT parse [OVERRIDE]/[APPEND]/[PREPEND] markers here —`,
-      `> they are textual conventions for the agent. Use \`mode: override\` in`,
-      `> frontmatter to replace the base skill entirely.`,
+      `> HOW SKILL OVERLAYS WORK`,
+      `> When the "${name}" skill runs, the agent reads this file and applies it on top`,
+      `> of the base SKILL.md (runtime concatenation). NOTHING in the CLI parses the`,
+      `> [OVERRIDE]/[APPEND]/[PREPEND] markers below — for skills they are textual`,
+      `> conventions the agent honours (best-effort), not a compile-time merge (that`,
+      `> model is agent/command-only). Keep this file the single home for everything`,
+      `> project-specific the base skill deliberately leaves generic.`,
+      `>`,
+      `> Frontmatter above: bump base_skill_version when you re-review against a newer`,
+      `> framework; \`mode: override\` tells the agent to prefer this file wholesale.`,
+      `> Fill in the sections that apply and delete the rest, then run`,
+      `> \`npx baldart overlay validate .baldart/overlays/${name}.md\` to preview the merge.`,
       '',
       `## [APPEND] Project-specific guidance`,
       '',
-      `<!-- Add the rules / vocabulary / opinions that apply to THIS project. -->`,
+      `<!-- The default home for project facts the base "${name}" skill cannot know: -->`,
+      `<!--   - canonical paths / module entry points (cite REAL files in this repo) -->`,
+      `<!--   - domain vocabulary, naming conventions, terminology -->`,
+      `<!--   - project-specific BLOCKING rules ("never X", "always read Y first") -->`,
+      `<!--   - mandated stack / forbidden libraries beyond baldart.config.yml stack.* -->`,
+      '',
+      `## [OVERRIDE] <exact base H2 to replace>`,
+      '',
+      `<!-- OPTIONAL. Copy an H2 heading VERBATIM from the base SKILL.md to tell the -->`,
+      `<!-- agent to prefer this version over the base. \`overlay validate\` warns if -->`,
+      `<!-- the heading matches no base section. Delete this block if you only append. -->`,
       '',
     ].join('\n');
   }
@@ -261,10 +311,33 @@ async function validate(overlayPath) {
   }
 
   if (t.kind === 'skill') {
-    UI.success(`Overlay valid (skill, concat model).`);
+    const baseContent = fs.readFileSync(baseAbs, 'utf8');
+    const a = analyzeSkillConcat(baseContent, overlayContent);
+    UI.success(`Overlay valid (skill, runtime-concat model).`);
     UI.info(`Targets v${fm.base_skill_version || 'unknown'}, mode=${fm.mode || 'extend'}.`);
+    UI.newline();
+    // Dry-run preview. For agent/command we render the actual merged output;
+    // skills can't be merged at the CLI (the executing agent reads base + overlay
+    // and applies it), so we show what the agent will see instead of a fake merge.
+    UI.info('Merge preview (runtime concat: base SKILL.md + overlay):');
+    UI.list([
+      `base sections: ${a.baseSectionCount} · overlay sections: ${a.markerSections.length + a.plainSections} (${a.markerSections.length} marked, ${a.plainSections} new)`,
+      `concatenated size: ~${a.baseLines + a.overlayLines} lines (${a.baseLines} base + ${a.overlayLines} overlay)`,
+    ]);
+    if (a.markerSections.length) {
+      UI.newline();
+      UI.info('Section markers (textual conventions — honoured by the agent at runtime, NOT parsed by the CLI):');
+      a.markerSections.forEach((m) => {
+        if (m.orphan) {
+          UI.warning(`  [${m.marker}] ${m.heading} — no matching H2 in base SKILL.md; the agent has no base section to anchor this to. Fix the heading or drop the marker.`);
+        } else {
+          UI.info(`  [${m.marker}] ${m.heading} — matches a base section`);
+        }
+      });
+    }
+    UI.newline();
     if (fm.base_file_sha) {
-      const currentSha = computeBaseFileSha(fs.readFileSync(baseAbs, 'utf8'));
+      const currentSha = computeBaseFileSha(baseContent);
       if (currentSha !== fm.base_file_sha) {
         UI.warning(`Base file changed since overlay was authored (sha ${fm.base_file_sha} → ${currentSha}). Review for drift.`);
       } else {
@@ -384,6 +457,15 @@ async function drift(target, options = {}) {
 
     if (shaDrift === true) {
       UI.warning(`${t.rel}: base file changed since overlay was authored (sha ${fm.base_file_sha} → ${baseSha}).`);
+      // For skills, surface WHICH overlay markers the base change orphaned — the
+      // same actionable feedback agent/command overlays get from the merger. A
+      // marker whose heading vanished upstream needs re-anchoring by hand.
+      if (t.kind === 'skill') {
+        const a = analyzeSkillConcat(fs.readFileSync(baseAbs, 'utf8'), content);
+        a.orphans.forEach((m) =>
+          UI.warning(`  └─ overlay marker [${m.marker}] "${m.heading}" no longer matches any base H2 — likely renamed/removed upstream. Re-anchor it.`)
+        );
+      }
       drifted++;
     } else if (shaDrift === false) {
       UI.success(`${t.rel}: clean (sha matches base v${fwVersion}).`);
@@ -404,4 +486,4 @@ async function drift(target, options = {}) {
   UI.info(`Summary: ${clean} clean, ${drifted} drifted, ${unknown} unknown — of ${targets.length} overlay(s).`);
 }
 
-module.exports = { scaffold, scaffoldFile, validate, drift, parseTarget };
+module.exports = { scaffold, scaffoldFile, validate, drift, parseTarget, analyzeSkillConcat };

package/src/utils/__tests__/analyze-skill-concat.test.js

@@ -0,0 +1,89 @@
+'use strict';
+const { test } = require('node:test');
+const assert = require('node:assert');
+const { analyzeSkillConcat } = require('../../commands/overlay');
+
+// analyzeSkillConcat powers the skill-overlay dry-run in `overlay validate` and
+// the orphaned-marker warning in `overlay drift`. Skills use runtime concat, so
+// the markers are NOT parsed — this helper only DESCRIBES what the agent will
+// see and flags markers whose heading no longer anchors to a base section.
+
+const BASE = `---
+name: bug
+---
+
+## Phase 1
+reproduce
+
+## Phase 2
+fix
+`;
+
+test('marker matching a base heading is not an orphan', () => {
+  const overlay = `---
+base_skill: bug
+---
+
+## [APPEND] Phase 1
+project entry points
+`;
+  const a = analyzeSkillConcat(BASE, overlay);
+  assert.strictEqual(a.markerSections.length, 1);
+  assert.strictEqual(a.markerSections[0].orphan, false);
+  assert.strictEqual(a.orphans.length, 0);
+});
+
+test('marker with no matching base heading is flagged orphan', () => {
+  const overlay = `---
+base_skill: bug
+---
+
+## [OVERRIDE] Phase 9
+heading absent from base
+`;
+  const a = analyzeSkillConcat(BASE, overlay);
+  assert.strictEqual(a.orphans.length, 1);
+  assert.strictEqual(a.orphans[0].heading, 'Phase 9');
+  assert.strictEqual(a.orphans[0].marker, 'OVERRIDE');
+});
+
+test('unmarked overlay sections count as new (plain), not markers', () => {
+  const overlay = `---
+base_skill: bug
+---
+
+## Brand vocabulary
+project terms
+
+## [APPEND] Phase 2
+extra
+`;
+  const a = analyzeSkillConcat(BASE, overlay);
+  assert.strictEqual(a.plainSections, 1);
+  assert.strictEqual(a.markerSections.length, 1);
+  assert.strictEqual(a.baseSectionCount, 2);
+});
+
+test('reports line counts for both base and overlay', () => {
+  const overlay = `---
+base_skill: bug
+---
+
+## [APPEND] Phase 1
+x
+`;
+  const a = analyzeSkillConcat(BASE, overlay);
+  assert.ok(a.baseLines > 0);
+  assert.ok(a.overlayLines > 0);
+});
+
+test('empty overlay body yields no sections and no orphans', () => {
+  const overlay = `---
+base_skill: bug
+---
+`;
+  const a = analyzeSkillConcat(BASE, overlay);
+  assert.strictEqual(a.markerSections.length, 0);
+  assert.strictEqual(a.plainSections, 0);
+  assert.strictEqual(a.orphans.length, 0);
+});

package/src/utils/hooks.js

@@ -87,6 +87,23 @@ const HOOK_REGISTRY = [
     command: 'bash .framework/framework/.claude/hooks/agent-discovery-info.sh',
     order: 100,
   },
+  {
+    // Observation-only telemetry: records every Read of a
+    // `.baldart/overlays/*.md` file to `.baldart/telemetry/overlay-loads.jsonl`.
+    // Measures, objectively, whether skill overlays are actually loaded at
+    // runtime (skill overlays use runtime-concat, which is NOT structurally
+    // enforced — unlike agent/command overlays, which are compile-time merged).
+    // Numerator-only by design: skill invocation is prompt-expansion, not a
+    // tool call, so the denominator is not hook-observable (claude-code #43630,
+    // #22655). bash, not node — it fires on EVERY Read (hottest tool) in the
+    // critical path, so a per-read node cold-start would tax every file read.
+    // Fail-safe — never blocks. See framework/docs/OVERLAY-TELEMETRY.md.
+    id: 'baldart-overlay-telemetry',
+    event: 'PostToolUse',
+    matcher: 'Read',
+    command: 'bash .framework/framework/.claude/hooks/overlay-telemetry.sh',
+    order: 100,
+  },
   // Future hooks (T1.3, T1.4, T2.3) will be appended here, e.g.:
   // {
   //   id: 'baldart-capture-detect',