baldart 3.28.3 → 3.30.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +52 -0
- package/VERSION +1 -1
- package/bin/baldart.js +1 -0
- package/framework/.claude/hooks/overlay-telemetry.sh +88 -0
- package/framework/.claude/skills/baldart-update/SKILL.md +17 -6
- package/framework/agents/project-context.md +1 -1
- package/framework/docs/OVERLAY-TELEMETRY.md +77 -0
- package/framework/scripts/analyze-overlay-loads.js +130 -0
- package/package.json +1 -1
- package/src/commands/overlay.js +148 -40
- package/src/commands/update.js +160 -28
- package/src/utils/__tests__/analyze-skill-concat.test.js +89 -0
- package/src/utils/__tests__/classify-divergence.test.js +71 -0
- package/src/utils/git.js +32 -11
- package/src/utils/hooks.js +17 -0
package/CHANGELOG.md
CHANGED
|
@@ -5,6 +5,58 @@ All notable changes to BALDART will be documented in this file.
|
|
|
5
5
|
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
|
|
6
6
|
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
|
7
7
|
|
|
8
|
+
## [3.30.0] - 2026-05-29
|
|
9
|
+
|
|
10
|
+
Hardens the **overlay** system — the layer that turns generic framework skills into project-specific knowledge. An audit surfaced a structural asymmetry: agent/command overlays are *compile-time merged* (the overlay is materially fused into the generated file → guaranteed delivery), but skill overlays use *runtime concatenation* (a line of prose tells the agent to read `.baldart/overlays/<skill>.md` → best-effort, nothing verifies it loaded). The most important overlays for knowledge transfer had the weakest guarantee. This release ships the high-certainty authoring/DX fixes now and adds **observation-only telemetry** to gather real data on whether skill overlays actually load — before deciding whether a stronger enforcement mechanism is worth building. **No new `baldart.config.yml` keys** — the telemetry hook is always-on (like `framework-edit-gate` / `agent-discovery-gate`), so the schema-propagation rule does not apply.
|
|
11
|
+
|
|
12
|
+
### Added — overlay-load telemetry (observation only)
|
|
13
|
+
|
|
14
|
+
- **[framework/.claude/hooks/overlay-telemetry.sh](framework/.claude/hooks/overlay-telemetry.sh)**: a fail-safe `PostToolUse` hook (matcher `Read`) that appends one JSON line to `.baldart/telemetry/overlay-loads.jsonl` every time a `.baldart/overlays/*.md` file is read. Objective, not self-reported — the failure mode being measured is "the agent forgot the overlay", which a self-report would corrupt in exactly that case. Written in **bash, not node**, on purpose: it fires on every `Read` (the hottest tool) in the critical path, so a per-read node cold-start (~50ms) would tax every file read — a bash fork (~5ms) keeps the hot path cheap. **Numerator-only by design**: skill invocation is prompt-expansion, not a tool call, so the denominator (skill runs that *should* have loaded an overlay) is not hook-observable (claude-code [#43630](https://github.com/anthropics/claude-code/issues/43630), [#22655](https://github.com/anthropics/claude-code/issues/22655)). Registered in `HOOK_REGISTRY` (`baldart-overlay-telemetry`) → auto-registered by `add`, backfilled by `update`, reported by `doctor` — no call-site changes (registry-driven).
|
|
15
|
+
- **[framework/scripts/analyze-overlay-loads.js](framework/scripts/analyze-overlay-loads.js)**: zero-dependency aggregator that cross-references the read log against the overlays that exist on disk. Headline signal: a **skill** overlay that exists but is read ~never (`NEVER-LOADED`). Agent/command overlays (compile-time merged) are deliberately not flagged on zero reads.
|
|
16
|
+
- **[framework/docs/OVERLAY-TELEMETRY.md](framework/docs/OVERLAY-TELEMETRY.md)**: full reading guide, with an explicit "conservative proxy — UNDER-counts" section (in-context reads and prompt-expansion are invisible to the hook; a low count is a reason to investigate, not proof of a miss).
|
|
17
|
+
|
|
18
|
+
### Changed — overlay authoring/DX parity for skills
|
|
19
|
+
|
|
20
|
+
- **[src/commands/overlay.js](src/commands/overlay.js)** `validate` (skill path): now renders a **merge dry-run** for skill overlays — section counts, concatenated size, and a per-marker check flagging `[OVERRIDE]/[APPEND]/[PREPEND]` headings that don't anchor to any base H2. Previously it printed only "valid, concat model", so a skill-overlay author never saw what the merge would look like (agent/command overlays already got this). Powered by a new pure, unit-tested `analyzeSkillConcat()` helper.
|
|
21
|
+
- **[src/commands/overlay.js](src/commands/overlay.js)** `drift` (skill path): when the base file changed, now lists *which* overlay markers were orphaned by the change (heading renamed/removed upstream) — the same actionable feedback agent/command overlays get from the merger, instead of a bare sha-mismatch warning.
|
|
22
|
+
- **[src/commands/overlay.js](src/commands/overlay.js)** `scaffoldBody` (skill path): the `overlay scaffold skill/<x>` skeleton is now a guided template — explains the runtime-concat model honestly, enumerates the typical project-fact sections (canonical paths, domain vocabulary, BLOCKING rules, mandated stack), and shows a ready-to-edit `[OVERRIDE]` block. Improves authoring for all skills, not just the two that ship an example overlay.
|
|
23
|
+
- **[framework/agents/project-context.md](framework/agents/project-context.md)** §5 precedence rules: corrected the overclaim that a skill overlay's `## [OVERRIDE] <topic>` "replaces" the base section. For skills nothing parses the markers — they are textual conventions the executing agent honours (best-effort), unlike agent/command overlays which are materially fused (§5.bis). The doc now matches `scaffoldBody`'s honest framing instead of the middle.
|
|
24
|
+
|
|
25
|
+
### Changed — drift-check counter
|
|
26
|
+
|
|
27
|
+
- **[framework/.claude/skills/baldart-update/SKILL.md](framework/.claude/skills/baldart-update/SKILL.md)**: `hook_registry_entries` 3 → 4 (the new telemetry hook). The `add`/`update`/`doctor` integration is registry-driven, so no narration changes were needed.
|
|
28
|
+
|
|
29
|
+
### Added — tests
|
|
30
|
+
|
|
31
|
+
- **[src/utils/__tests__/analyze-skill-concat.test.js](src/utils/__tests__/analyze-skill-concat.test.js)**: 5 cases covering marker matching, orphan detection, plain (new) sections, line counts, and the empty-overlay edge case.
|
|
32
|
+
|
|
33
|
+
## [3.29.0] - 2026-05-29
|
|
34
|
+
|
|
35
|
+
Closes a two-part process gap in `npx baldart update` reported from a non-TTY agent context on a shared consumer repo: (1) plumbing/collaboration merge commits were mislabeled `custom-other`, downgrading an otherwise overlay-able divergence set into `mixed`; (2) an automated agent had **no** non-destructive path to complete an update when the consumer carried local `.framework/` commits — `--yes` refused, the interactive prompt needed a TTY, and `--reset` is gated on a clean working tree. **No new `baldart.config.yml` keys** — `--on-divergence` is a CLI flag, not a config flag, so the schema-propagation rule does not apply.
|
|
36
|
+
|
|
37
|
+
### Fixed — divergence classifier no longer poisoned by merge commits
|
|
38
|
+
|
|
39
|
+
- **[src/utils/git.js](src/utils/git.js)** `classifyDivergence()`: merge commits are now detected **structurally by parent count** (`%P` in the `git log` format → `parents.length >= 2`) instead of relying solely on the brittle subject regex `^Merge … into <branch>$`. That regex missed real `git subtree pull --squash` merges that present as a bare `Merge commit '<sha>'` (git omits the `into <branch>` suffix when the merge lands on `master`) and shared-repo collaboration merges shaped like `Merge branch 'x' of <url> into <branch>` / `Merge remote-tracking branch '…'`. Such merges fell through to the path classifier, which returns `custom-other` for the empty diff-tree of a merge — collapsing an `overlay-able` set into `mixed` and hiding the overlay-migration option. Merges are now labeled `subtree-merge` (noise) and excluded before the aggregate decision.
|
|
40
|
+
- **[src/utils/git.js](src/utils/git.js)**: aggregation extracted to a pure, unit-testable static `GitUtils.aggregateDivergenceClass(commits)`. Each divergence commit record now also carries `parents` and `touched` (the latter reused by the new overlay-scaffold flow below).
|
|
41
|
+
- **[src/utils/__tests__/classify-divergence.test.js](src/utils/__tests__/classify-divergence.test.js)**: +9 fixtures — bare-merge / `of <url>` / remote-tracking subjects (documenting the regex blind spot now covered by parent count) and an `aggregateDivergenceClass` suite asserting `overlay-able + plumbing-merge → overlay-able` (not `mixed`).
|
|
42
|
+
|
|
43
|
+
### Added — `--on-divergence` non-interactive escape hatch for agents / CI
|
|
44
|
+
|
|
45
|
+
- **[src/commands/update.js](src/commands/update.js)** + **[bin/baldart.js](bin/baldart.js)**: `npx baldart update --on-divergence <strategy>` resolves a divergent update without a TTY:
|
|
46
|
+
- `scaffold-overlays` → auto-creates overlay skeleton(s) for the overlay-able framework files touched by the divergent commits (reusing `overlay.scaffoldFile`), then stops so the edits can be filled in and the update re-run. For `mixed`, scaffolds the overlay-able subset and refuses to pull while `custom-other` commits remain (would be destructive).
|
|
47
|
+
- `pull` → keeps the commits and lets the subtree pull create a merge (non-destructive; conflicts are surfaced, never auto-resolved). Works for `overlay-able`, `mixed`, and `real-custom`.
|
|
48
|
+
- `abort` → explicit no-op.
|
|
49
|
+
- Bare `--yes` on a divergent class still refuses (nothing destructive by accident) but now prints the exact flag to re-run with instead of dead-ending.
|
|
50
|
+
|
|
51
|
+
### Changed — interactive `overlay` choice now actually scaffolds
|
|
52
|
+
|
|
53
|
+
- **[src/commands/update.js](src/commands/update.js)**: the interactive "migrate to overlay" option previously printed *"run `/overlay` then re-run"* and exited (a no-op). It now scaffolds the overlay skeleton(s) in-place via `overlay.scaffoldFile` and points the user to `/overlay` for guided filling.
|
|
54
|
+
- **[src/commands/overlay.js](src/commands/overlay.js)**: extracted a non-exiting `scaffoldFile(cwd, target, options)` (returns a structured result) from the `process.exit`-driven `scaffold()` CLI wrapper, so it can be called in a loop from `update` without a single missing/existing overlay terminating the run. Exported alongside `parseTarget`.
|
|
55
|
+
|
|
56
|
+
### Changed — `/baldart-update` skill narration synced
|
|
57
|
+
|
|
58
|
+
- **[framework/.claude/skills/baldart-update/SKILL.md](framework/.claude/skills/baldart-update/SKILL.md)**: documents the parent-count merge detection and the new `--on-divergence` strategies; corrects the stale "real user-custom commits ABORT in --yes mode" claim.
|
|
59
|
+
|
|
8
60
|
## [3.28.3] - 2026-05-28
|
|
9
61
|
|
|
10
62
|
Release B of the 3-release plan on `/new` fix-application authority. Closes two inline-apply violation patterns observed in real `/new` runs (doc fixes applied by the orchestrator instead of delegated to coder; Codex HIGH security findings written inline) and shuts the structural loophole in `SKILL.md`'s sub-agent failure protocol that was being used to bypass delegation even when no agent had crashed. Telemetry from v3.28.2 will measure the effectiveness of these changes; Release C (conditional threshold rule) waits for that data.
|
package/VERSION
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
3.
|
|
1
|
+
3.30.0
|
package/bin/baldart.js
CHANGED
|
@@ -72,6 +72,7 @@ program
|
|
|
72
72
|
.option('--auto-stash', 'When the working tree is dirty, stash automatically instead of prompting.')
|
|
73
73
|
.option('--reset', 'Nuclear option: rm -rf .framework/ + fresh install, preserving baldart.config.yml + .baldart/ + custom .claude/{agents,skills,commands}/. Requires clean working tree.')
|
|
74
74
|
.option('--i-know', 'Acknowledges --reset will wipe untracked/ignored files inside .framework/ (required to use --reset --yes).')
|
|
75
|
+
.option('--on-divergence <strategy>', 'Non-interactive resolution when the consumer has local commits on .framework/ (for agents / CI): "scaffold-overlays" (auto-create overlay skeletons, then stop), "pull" (keep commits + merge — non-destructive), or "abort".')
|
|
75
76
|
.action(async (options) => {
|
|
76
77
|
const updateCommand = require('../src/commands/update');
|
|
77
78
|
await updateCommand(options);
|
|
@@ -0,0 +1,88 @@
|
|
|
1
|
+
#!/usr/bin/env bash
|
|
2
|
+
#
|
|
3
|
+
# overlay-telemetry.sh — BALDART PostToolUse hook (matcher: Read).
|
|
4
|
+
#
|
|
5
|
+
# OBSERVATION ONLY. Appends one JSON line to
|
|
6
|
+
# `.baldart/telemetry/overlay-loads.jsonl` every time a `.baldart/overlays/*.md`
|
|
7
|
+
# file is read. This is the *numerator* of the overlay-load question: when the
|
|
8
|
+
# agent runs a skill that HAS an overlay, does it actually load it? A `Read` of
|
|
9
|
+
# the overlay file is the one objective signal — skill self-report would be
|
|
10
|
+
# corrupted in exactly the case we measure ("the agent forgot the overlay").
|
|
11
|
+
#
|
|
12
|
+
# Bash, NOT node, on purpose: this fires on EVERY `Read` (the hottest tool) in
|
|
13
|
+
# the consumer's critical path. A per-read node cold-start (~50ms) would tax
|
|
14
|
+
# every file read; a bash fork (~5ms) keeps the hot path cheap. The work is just
|
|
15
|
+
# a path match + a conditional append, so bash is sufficient. Same precedent as
|
|
16
|
+
# agent-discovery-info.sh.
|
|
17
|
+
#
|
|
18
|
+
# Numerator-only by design — skill invocation is prompt-expansion, not a tool
|
|
19
|
+
# call, so the denominator is not hook-observable (claude-code #43630, #22655).
|
|
20
|
+
# See framework/docs/OVERLAY-TELEMETRY.md for the under-count caveats.
|
|
21
|
+
#
|
|
22
|
+
# Fail-safe: always exit 0, never disrupt the session.
|
|
23
|
+
|
|
24
|
+
set -u
|
|
25
|
+
exec 2>/dev/null # never surface stderr to the user
|
|
26
|
+
|
|
27
|
+
INPUT=""
|
|
28
|
+
if [ ! -t 0 ]; then
|
|
29
|
+
INPUT="$(cat)"
|
|
30
|
+
fi
|
|
31
|
+
[ -z "$INPUT" ] && exit 0
|
|
32
|
+
|
|
33
|
+
get_field() {
|
|
34
|
+
# $1 = JSON key (top-level or nested — sed fallback matches anywhere)
|
|
35
|
+
if command -v jq >/dev/null 2>&1; then
|
|
36
|
+
printf '%s' "$INPUT" | jq -r "$2 // empty" 2>/dev/null
|
|
37
|
+
else
|
|
38
|
+
printf '%s' "$INPUT" | sed -n "s/.*\"$1\"[[:space:]]*:[[:space:]]*\"\\([^\"]*\\)\".*/\\1/p" | head -1
|
|
39
|
+
fi
|
|
40
|
+
}
|
|
41
|
+
|
|
42
|
+
TOOL_NAME="$(get_field tool_name '.tool_name')"
|
|
43
|
+
[ "$TOOL_NAME" = "Read" ] || exit 0
|
|
44
|
+
|
|
45
|
+
FILE_PATH="$(get_field file_path '.tool_input.file_path')"
|
|
46
|
+
[ -n "$FILE_PATH" ] || exit 0
|
|
47
|
+
|
|
48
|
+
CWD="$(get_field cwd '.cwd')"
|
|
49
|
+
[ -n "$CWD" ] || CWD="$PWD"
|
|
50
|
+
SESSION_ID="$(get_field session_id '.session_id')"
|
|
51
|
+
|
|
52
|
+
# Resolve relative paths against cwd.
|
|
53
|
+
case "$FILE_PATH" in
|
|
54
|
+
/*) ABS="$FILE_PATH" ;;
|
|
55
|
+
*) ABS="$CWD/$FILE_PATH" ;;
|
|
56
|
+
esac
|
|
57
|
+
|
|
58
|
+
# Must live under .baldart/overlays/.
|
|
59
|
+
case "$ABS" in
|
|
60
|
+
*/.baldart/overlays/*) ;;
|
|
61
|
+
*) exit 0 ;;
|
|
62
|
+
esac
|
|
63
|
+
REL="${ABS##*/.baldart/overlays/}" # everything after the last marker
|
|
64
|
+
|
|
65
|
+
# Classify — mirror src/utils/overlay-merger / overlay.js path logic.
|
|
66
|
+
# Order matters: deeper/excluded patterns before the catch-alls.
|
|
67
|
+
KIND=""
|
|
68
|
+
NAME=""
|
|
69
|
+
case "$REL" in
|
|
70
|
+
*.example.md) exit 0 ;;
|
|
71
|
+
README.md) exit 0 ;;
|
|
72
|
+
*/*/*) exit 0 ;; # 3+ levels — ignore
|
|
73
|
+
agents/*.md) KIND="agent"; NAME="${REL#agents/}"; NAME="${NAME%.md}" ;;
|
|
74
|
+
commands/*.md) KIND="command"; NAME="${REL#commands/}"; NAME="${NAME%.md}" ;;
|
|
75
|
+
*/*) exit 0 ;; # 2 levels, not agents/commands — ignore
|
|
76
|
+
*.md) KIND="skill"; NAME="${REL%.md}" ;;
|
|
77
|
+
*) exit 0 ;;
|
|
78
|
+
esac
|
|
79
|
+
[ -n "$KIND" ] && [ -n "$NAME" ] || exit 0
|
|
80
|
+
|
|
81
|
+
TS="$(date -u +%Y-%m-%dT%H:%M:%SZ)"
|
|
82
|
+
if [ -n "$SESSION_ID" ]; then SID="\"$SESSION_ID\""; else SID="null"; fi
|
|
83
|
+
|
|
84
|
+
mkdir -p "$CWD/.baldart/telemetry" || exit 0
|
|
85
|
+
printf '{"ts":"%s","event":"overlay-read","kind":"%s","name":"%s","session_id":%s}\n' \
|
|
86
|
+
"$TS" "$KIND" "$NAME" "$SID" >> "$CWD/.baldart/telemetry/overlay-loads.jsonl"
|
|
87
|
+
|
|
88
|
+
exit 0
|
|
@@ -13,7 +13,7 @@ sources, a warning is surfaced in the workflow run. The decision whether
|
|
|
13
13
|
this skill needs an update is left to a human reviewer.
|
|
14
14
|
|
|
15
15
|
update_js_prompts: 7
|
|
16
|
-
hook_registry_entries:
|
|
16
|
+
hook_registry_entries: 4
|
|
17
17
|
config_template_top_level_keys: features,git,identity,lsp,paths,stack,tools,version
|
|
18
18
|
last_verified: v3.26.0
|
|
19
19
|
-->
|
|
@@ -139,7 +139,7 @@ post-flight assert:
|
|
|
139
139
|
- **Hooks drift**: missing hooks auto-registered silently; only command-level drift prompts (and only if interactive).
|
|
140
140
|
- **Schema config drift**: surfaces as a warning post-update; never blocks.
|
|
141
141
|
- **BALDART-managed auto-commit**: `--yes` commits silently with `chore(baldart):` subject.
|
|
142
|
-
- **Subtree-merge / chore divergence commits**: classified by the CLI; auto-resolved silently when `all-noise`, prompts
|
|
142
|
+
- **Subtree-merge / chore divergence commits**: classified by the CLI (merges detected by parent count, so plumbing/collaboration merges never poison the class); auto-resolved silently when `all-noise`, prompts for `overlay-able` / `real-custom` / `mixed` — or resolve non-interactively with `--on-divergence scaffold-overlays|pull|abort`.
|
|
143
143
|
- **CLI self-upgrade**: transparent relaunch via `npx baldart@latest` when global CLI is stale.
|
|
144
144
|
|
|
145
145
|
> **IMPORTANT — do NOT use `git -C .framework fetch`.** `.framework/` is a git
|
|
@@ -272,10 +272,21 @@ Do NOT pass `--auto-stash` (redundant with `--yes`).
|
|
|
272
272
|
files it touched during the reconcile with `chore(baldart): post-update
|
|
273
273
|
reconcile to vX.Y.Z`. User-owned files are never staged.
|
|
274
274
|
- **Subtree-merge / chore divergence commits**: when the consumer has
|
|
275
|
-
local commits that are just `git subtree pull` plumbing
|
|
276
|
-
|
|
277
|
-
|
|
278
|
-
|
|
275
|
+
local commits that are just `git subtree pull` plumbing, ordinary
|
|
276
|
+
collaboration merges (detected by parent count, not just subject), or
|
|
277
|
+
CLI-generated `[CHORE]` commits, the CLI auto-resolves silently. Real
|
|
278
|
+
user-custom commits do NOT auto-resolve under bare `--yes` — but an agent
|
|
279
|
+
is no longer dead-ended: pass `--on-divergence` to resolve non-interactively
|
|
280
|
+
(since v3.29.0):
|
|
281
|
+
- `--on-divergence scaffold-overlays` → auto-creates overlay skeleton(s)
|
|
282
|
+
for the overlay-able commits, then stops so the edits can be filled in
|
|
283
|
+
(then re-run update). Use this when the divergence is `overlay-able`.
|
|
284
|
+
- `--on-divergence pull` → keeps the commits and lets the subtree pull
|
|
285
|
+
create a merge (non-destructive; resolve any conflicts that surface).
|
|
286
|
+
Works for `overlay-able`, `mixed`, and `real-custom`.
|
|
287
|
+
- `--on-divergence abort` → explicit no-op.
|
|
288
|
+
Bare `--yes` without a strategy still refuses (and now prints exactly which
|
|
289
|
+
flag to re-run with), so nothing destructive happens by accident.
|
|
279
290
|
- **CLI self-upgrade**: if the global CLI is older than npm latest, the
|
|
280
291
|
update transparently relaunches itself via `npx baldart@latest update
|
|
281
292
|
--yes`. Loop-guarded by `BALDART_RELAUNCHED=1`.
|
|
@@ -86,7 +86,7 @@ mode: extend # extend | override
|
|
|
86
86
|
### Precedence rules
|
|
87
87
|
|
|
88
88
|
1. **Default**: overlays *extend* base skills. If overlay and base both apply, both run.
|
|
89
|
-
2. **Explicit override**: inside
|
|
89
|
+
2. **Explicit override**: inside a skill overlay, a section marked `## [OVERRIDE] <topic>` signals the agent to prefer the overlay's version of `<topic>` over the base skill's. **This is a textual convention, not a parsed merge.** Skill overlays use runtime concatenation (§5.bis): nothing in the CLI parses `[OVERRIDE]`/`[APPEND]`/`[PREPEND]` for skills — the merger (`overlay-merger.js`) parses those markers **only for agent/command overlays**, which are compiled at `add`/`update` time. For skills, the marker is an instruction the executing agent honours when it reads the overlay; honouring it is best-effort, not structurally enforced. (Agent/command overrides, by contrast, are materially fused into the generated file — see §5.bis.)
|
|
90
90
|
3. **Stack/identity conflicts**: when an overlay's stack/identity rule contradicts `baldart.config.yml`, the overlay wins (the user wrote it deliberately) — but the skill MUST log a one-line "overlay overrides config" notice in its output.
|
|
91
91
|
|
|
92
92
|
### Version drift handling
|
|
@@ -0,0 +1,77 @@
|
|
|
1
|
+
# Overlay-Load Telemetry
|
|
2
|
+
|
|
3
|
+
This doc describes the telemetry layer that records, **objectively**, whether skill overlays are actually loaded at runtime. The layer is **observation only** — it changes no behaviour. It exists so we can decide, with real data, whether skill overlays need a stronger delivery guarantee than the current runtime-concat model.
|
|
4
|
+
|
|
5
|
+
## Why this exists
|
|
6
|
+
|
|
7
|
+
Skill overlays and agent/command overlays have **opposite delivery guarantees**:
|
|
8
|
+
|
|
9
|
+
- **Agent / command overlays** are *compile-time merged* — the overlay is materially fused into the generated `.claude/agents/<x>.md` at `add`/`update` (`overlay-merger.js`). When Claude loads the subagent, the overlay is already there. The guarantee is structural.
|
|
10
|
+
- **Skill overlays** use *runtime concatenation* — `SKILL.md` stays a symlink, the overlay lives separately in `.baldart/overlays/<skill>.md`, and a line of prose tells the agent "load `.baldart/overlays/<skill>.md` if present". Nothing verifies the agent did. The guarantee is best-effort.
|
|
11
|
+
|
|
12
|
+
So the *most* important overlays for knowledge transfer (skills) have the *weakest* guarantee. The architectural fixes for that (SessionStart awareness injection, PreToolUse-on-Skill injection, or compile-time merge of `SKILL.md`) are all non-trivial — and we have **no evidence** overlays actually fail to load in practice. Before building any of them, we measure. This layer is that measurement.
|
|
13
|
+
|
|
14
|
+
## What it captures (and what it can't)
|
|
15
|
+
|
|
16
|
+
A single PostToolUse hook (`framework/.claude/hooks/overlay-telemetry.sh`, matcher `Read`) appends one JSON line to `.baldart/telemetry/overlay-loads.jsonl` every time a `.baldart/overlays/*.md` file is read:
|
|
17
|
+
|
|
18
|
+
```json
|
|
19
|
+
{"ts":"2026-05-29T08:00:00Z","event":"overlay-read","kind":"skill","name":"bug","session_id":"..."}
|
|
20
|
+
```
|
|
21
|
+
|
|
22
|
+
### Why a hook, not agent self-report
|
|
23
|
+
|
|
24
|
+
The failure mode we measure is *"the agent forgot to load the overlay."* A self-reported `loaded=yes/no` would be corrupted in exactly that case — an agent that forgot to load would also forget to log, or log `yes` without having done it. Only an objective `Read` observation is trustworthy here.
|
|
25
|
+
|
|
26
|
+
### Numerator only — the known limitation
|
|
27
|
+
|
|
28
|
+
We capture the **numerator** (overlay reads), not the **denominator** (skill runs that *should* have loaded an overlay). The denominator is not hook-observable: skills are *prompt-expansion* in Claude Code, not tool calls, so there is no `PreToolUse` on skill invocation and the tool-call payload does not carry the invoking skill's name (anthropics/claude-code [#43630](https://github.com/anthropics/claude-code/issues/43630), [#22655](https://github.com/anthropics/claude-code/issues/22655)).
|
|
29
|
+
|
|
30
|
+
This makes the metric a **conservative proxy that UNDER-counts**:
|
|
31
|
+
|
|
32
|
+
- Overlay content already in the agent's context (no explicit `Read`) is invisible.
|
|
33
|
+
- A `Read` is not proof the content influenced the output — only that it was loaded.
|
|
34
|
+
- Reads performed in ways the main-loop hook doesn't observe are missed.
|
|
35
|
+
|
|
36
|
+
Every bias points the same direction: the observed load-rate looks **worse** than reality. That matters because under-counting biases toward "build the enforcement" — the very thing this telemetry exists to gate. **Treat a low count as a reason to investigate, not as proof of a miss.** (Same honesty convention as `agent-discovery-info.sh`'s #10373 caveat and the "conservative proxy" language in `FIX-APPLICATION-TELEMETRY.md`.)
|
|
37
|
+
|
|
38
|
+
## How to read the data
|
|
39
|
+
|
|
40
|
+
```bash
|
|
41
|
+
node framework/scripts/analyze-overlay-loads.js
|
|
42
|
+
# or, non-default locations:
|
|
43
|
+
node framework/scripts/analyze-overlay-loads.js --log <path.jsonl> --overlays <dir>
|
|
44
|
+
```
|
|
45
|
+
|
|
46
|
+
The aggregator cross-references the read log against the overlays that **exist on disk** and prints per-overlay read counts. The headline signal:
|
|
47
|
+
|
|
48
|
+
| Column / footer | What it means |
|
|
49
|
+
|---|---|
|
|
50
|
+
| `Flag = NEVER-LOADED` | A **skill** overlay exists but was never read. The alarm signal — authored knowledge the agent isn't picking up. (Subject to the under-count caveat above.) |
|
|
51
|
+
| `Reads` low but non-zero | Overlay loads sometimes. Compare against how often the skill ran (you'll have to estimate the denominator from your own usage). |
|
|
52
|
+
| Agent/command rows, zero reads | **Expected, not a finding.** These overlays are compile-time merged into the generated file — they're not meant to be read at runtime. The table does not flag them. |
|
|
53
|
+
| `Reads logged for overlays no longer on disk` | Stale/renamed overlays. Cosmetic. |
|
|
54
|
+
|
|
55
|
+
## When to act on the data
|
|
56
|
+
|
|
57
|
+
After a few weeks of real usage with at least one authored skill overlay:
|
|
58
|
+
|
|
59
|
+
1. **`NEVER-LOADED` skill overlays.** If a skill overlay you actively rely on shows zero reads across many sessions where that skill clearly ran, that's the strongest available evidence that runtime concat is dropping it — and the case for an enforcement mechanism (SessionStart awareness, PreToolUse injection, or `SKILL.md` compile-time merge) becomes data-backed rather than speculative.
|
|
60
|
+
2. **Healthy load pattern.** If your authored skill overlays show steady reads, the runtime-concat model is working in practice and enforcement is not worth its cost. Ship the DX improvements and leave the architecture alone.
|
|
61
|
+
|
|
62
|
+
## Cost
|
|
63
|
+
|
|
64
|
+
Be honest about where the cost lands: the hook fires on **every `Read`** — the most frequent tool — not just on overlay reads. So the per-read cost is what matters, and it is paid hundreds of times per session in the PostToolUse critical path (the model waits for the hook before seeing the tool result).
|
|
65
|
+
|
|
66
|
+
- **Every read** (overwhelmingly non-overlay): one bash fork + a path-prefix match, then exit. This is why the hook is bash, not node — a node cold-start (~50ms) per read would be a real tax; a bash fork (~5ms) keeps the hot path cheap. There is no cheaper event: observing "an overlay was read" *requires* hooking `Read`, so the only lever is making the hook itself cheap.
|
|
67
|
+
- **Only on an actual overlay read** (rare): one `mkdir -p` + one ~120-byte append.
|
|
68
|
+
|
|
69
|
+
The hook is fail-safe (any error → exit 0, never blocks). The aggregator has zero dependencies and runs in well under a second. The log is append-only and rotates only by hand (delete the file to reset the window).
|
|
70
|
+
|
|
71
|
+
## Pointers
|
|
72
|
+
|
|
73
|
+
- Hook: `framework/.claude/hooks/overlay-telemetry.sh`
|
|
74
|
+
- Registration: `src/utils/hooks.js` → `HOOK_REGISTRY` entry `baldart-overlay-telemetry` (auto-registered by `add`, backfilled by `update`, reported by `doctor`)
|
|
75
|
+
- Aggregator: `framework/scripts/analyze-overlay-loads.js`
|
|
76
|
+
- Overlay model: `framework/agents/project-context.md` §5 (skills, runtime concat) and §5.bis (agent/command, compile-time merge)
|
|
77
|
+
- Sibling telemetry layer: `framework/docs/FIX-APPLICATION-TELEMETRY.md`
|
|
@@ -0,0 +1,130 @@
|
|
|
1
|
+
#!/usr/bin/env node
|
|
2
|
+
// Aggregate overlay-load telemetry. Cross-references the objective read log
|
|
3
|
+
// (.baldart/telemetry/overlay-loads.jsonl, written by the overlay-telemetry
|
|
4
|
+
// PostToolUse hook) against the overlays that actually EXIST on disk
|
|
5
|
+
// (.baldart/overlays/), and reports per-overlay load counts.
|
|
6
|
+
//
|
|
7
|
+
// The headline signal is the LAST column: a SKILL overlay that exists but is
|
|
8
|
+
// read ~never is the alarm — runtime concatenation is not structurally
|
|
9
|
+
// enforced, so a never-read skill overlay is knowledge the project authored but
|
|
10
|
+
// the agent isn't picking up. (Agent/command overlays are compile-time merged
|
|
11
|
+
// into the generated file, so they're NOT expected to be read at runtime — a
|
|
12
|
+
// zero count there is normal, not a problem. The table marks them accordingly.)
|
|
13
|
+
//
|
|
14
|
+
// This is a CONSERVATIVE proxy. The hook can only see explicit `Read` calls, so
|
|
15
|
+
// it UNDER-counts: overlay content already in context (no Read) and reads done
|
|
16
|
+
// in ways the main-loop hook doesn't observe are invisible. A low count is a
|
|
17
|
+
// reason to investigate, not proof of a miss. See framework/docs/OVERLAY-TELEMETRY.md.
|
|
18
|
+
//
|
|
19
|
+
// Usage:
|
|
20
|
+
// node framework/scripts/analyze-overlay-loads.js
|
|
21
|
+
// node framework/scripts/analyze-overlay-loads.js --log <path.jsonl> --overlays <dir>
|
|
22
|
+
|
|
23
|
+
'use strict';
|
|
24
|
+
const fs = require('fs');
|
|
25
|
+
const path = require('path');
|
|
26
|
+
|
|
27
|
+
function argVal(flag, fallback) {
|
|
28
|
+
const i = process.argv.indexOf(flag);
|
|
29
|
+
return i !== -1 && process.argv[i + 1] ? process.argv[i + 1] : fallback;
|
|
30
|
+
}
|
|
31
|
+
|
|
32
|
+
const LOG_PATH = argVal('--log', path.join('.baldart', 'telemetry', 'overlay-loads.jsonl'));
|
|
33
|
+
const OVERLAYS_DIR = argVal('--overlays', path.join('.baldart', 'overlays'));
|
|
34
|
+
|
|
35
|
+
// --- read the objective read log -------------------------------------------
|
|
36
|
+
|
|
37
|
+
const reads = new Map(); // "kind/name" → { count, sessions:Set, first, last }
|
|
38
|
+
let logLines = 0;
|
|
39
|
+
let logBad = 0;
|
|
40
|
+
|
|
41
|
+
if (fs.existsSync(LOG_PATH)) {
|
|
42
|
+
const raw = fs.readFileSync(LOG_PATH, 'utf8');
|
|
43
|
+
for (const line of raw.split(/\r?\n/)) {
|
|
44
|
+
if (!line.trim()) continue;
|
|
45
|
+
logLines += 1;
|
|
46
|
+
let row;
|
|
47
|
+
try { row = JSON.parse(line); } catch (_) { logBad += 1; continue; }
|
|
48
|
+
if (!row || row.event !== 'overlay-read' || !row.kind || !row.name) { logBad += 1; continue; }
|
|
49
|
+
const key = `${row.kind}/${row.name}`;
|
|
50
|
+
const e = reads.get(key) || { count: 0, sessions: new Set(), first: null, last: null };
|
|
51
|
+
e.count += 1;
|
|
52
|
+
if (row.session_id) e.sessions.add(row.session_id);
|
|
53
|
+
if (row.ts) {
|
|
54
|
+
if (!e.first || row.ts < e.first) e.first = row.ts;
|
|
55
|
+
if (!e.last || row.ts > e.last) e.last = row.ts;
|
|
56
|
+
}
|
|
57
|
+
reads.set(key, e);
|
|
58
|
+
}
|
|
59
|
+
}
|
|
60
|
+
|
|
61
|
+
// --- enumerate overlays that exist on disk ---------------------------------
|
|
62
|
+
|
|
63
|
+
function listOverlays(dir) {
|
|
64
|
+
const out = []; // { kind, name, key }
|
|
65
|
+
if (!fs.existsSync(dir)) return out;
|
|
66
|
+
const add = (sub, kind) => {
|
|
67
|
+
const d = sub ? path.join(dir, sub) : dir;
|
|
68
|
+
if (!fs.existsSync(d)) return;
|
|
69
|
+
for (const f of fs.readdirSync(d)) {
|
|
70
|
+
if (!f.endsWith('.md') || f.endsWith('.example.md') || f === 'README.md') continue;
|
|
71
|
+
const name = f.replace(/\.md$/, '');
|
|
72
|
+
out.push({ kind, name, key: `${kind}/${name}` });
|
|
73
|
+
}
|
|
74
|
+
};
|
|
75
|
+
add('', 'skill');
|
|
76
|
+
add('agents', 'agent');
|
|
77
|
+
add('commands', 'command');
|
|
78
|
+
return out;
|
|
79
|
+
}
|
|
80
|
+
|
|
81
|
+
const overlays = listOverlays(OVERLAYS_DIR);
|
|
82
|
+
const overlayKeys = new Set(overlays.map((o) => o.key));
|
|
83
|
+
|
|
84
|
+
// --- build rows -------------------------------------------------------------
|
|
85
|
+
|
|
86
|
+
const rows = overlays.map((o) => {
|
|
87
|
+
const r = reads.get(o.key);
|
|
88
|
+
return {
|
|
89
|
+
kind: o.kind,
|
|
90
|
+
name: o.name,
|
|
91
|
+
reads: r ? r.count : 0,
|
|
92
|
+
sessions: r ? r.sessions.size : 0,
|
|
93
|
+
last: r && r.last ? r.last.slice(0, 10) : '-',
|
|
94
|
+
// Only skills are expected to be read at runtime; agent/command overlays are
|
|
95
|
+
// fused at build time, so a zero there is expected, not a finding.
|
|
96
|
+
flag: (o.kind === 'skill' && (!r || r.count === 0)) ? 'NEVER-LOADED' : '',
|
|
97
|
+
};
|
|
98
|
+
});
|
|
99
|
+
rows.sort((a, b) => a.reads - b.reads || a.kind.localeCompare(b.kind) || a.name.localeCompare(b.name));
|
|
100
|
+
|
|
101
|
+
// Reads of overlays that no longer exist on disk (stale/renamed) — surfaced separately.
|
|
102
|
+
const orphanReads = [...reads.keys()].filter((k) => !overlayKeys.has(k));
|
|
103
|
+
|
|
104
|
+
// --- render -----------------------------------------------------------------
|
|
105
|
+
|
|
106
|
+
const headers = ['Kind', 'Name', 'Reads', 'Sessions', 'Last-read', 'Flag'];
|
|
107
|
+
const cols = ['kind', 'name', 'reads', 'sessions', 'last', 'flag'];
|
|
108
|
+
const widths = headers.map((h, i) => Math.max(h.length, ...rows.map((r) => String(r[cols[i]]).length), 1));
|
|
109
|
+
const fmt = (cells) => cells.map((c, i) => String(c).padEnd(widths[i])).join(' ');
|
|
110
|
+
|
|
111
|
+
console.log(fmt(headers));
|
|
112
|
+
console.log(widths.map((w) => '-'.repeat(w)).join(' '));
|
|
113
|
+
for (const r of rows) console.log(fmt(cols.map((c) => r[c])));
|
|
114
|
+
|
|
115
|
+
console.log('');
|
|
116
|
+
console.log(`Overlays on disk: ${overlays.length} | Log rows: ${logLines}${logBad ? ` (${logBad} skipped)` : ''} | Log: ${fs.existsSync(LOG_PATH) ? LOG_PATH : 'NOT FOUND (no telemetry yet)'}`);
|
|
117
|
+
|
|
118
|
+
const neverLoaded = rows.filter((r) => r.flag === 'NEVER-LOADED');
|
|
119
|
+
if (neverLoaded.length) {
|
|
120
|
+
console.log('');
|
|
121
|
+
console.log('⚠ Skill overlays that EXIST but were never read (runtime concat may not be loading them):');
|
|
122
|
+
for (const r of neverLoaded) console.log(` ${r.kind}/${r.name}`);
|
|
123
|
+
console.log(' Conservative proxy — see framework/docs/OVERLAY-TELEMETRY.md before acting (in-context reads are invisible to the hook).');
|
|
124
|
+
}
|
|
125
|
+
|
|
126
|
+
if (orphanReads.length) {
|
|
127
|
+
console.log('');
|
|
128
|
+
console.log('ℹ Reads logged for overlays no longer on disk (renamed/removed):');
|
|
129
|
+
for (const k of orphanReads) console.log(` ${k}`);
|
|
130
|
+
}
|
package/package.json
CHANGED
package/src/commands/overlay.js
CHANGED
|
@@ -2,7 +2,7 @@ const fs = require('fs');
|
|
|
2
2
|
const path = require('path');
|
|
3
3
|
const yaml = require('js-yaml');
|
|
4
4
|
const UI = require('../utils/ui');
|
|
5
|
-
const { mergeOverlay, computeBaseFileSha } = require('../utils/overlay-merger');
|
|
5
|
+
const { mergeOverlay, computeBaseFileSha, parseMarkdown } = require('../utils/overlay-merger');
|
|
6
6
|
|
|
7
7
|
const FRAMEWORK_DIR = '.framework';
|
|
8
8
|
const FRAMEWORK_PAYLOAD = path.join(FRAMEWORK_DIR, 'framework');
|
|
@@ -72,18 +72,68 @@ function parseFrontmatter(content) {
|
|
|
72
72
|
}
|
|
73
73
|
}
|
|
74
74
|
|
|
75
|
+
// Skill overlays use the runtime-concat model — the executing agent reads the
|
|
76
|
+
// base SKILL.md and the overlay and applies the overlay's guidance itself.
|
|
77
|
+
// Unlike agent/command overlays, NOTHING parses the [OVERRIDE]/[APPEND]/[PREPEND]
|
|
78
|
+
// markers here (see framework/agents/project-context.md § 5). This helper gives
|
|
79
|
+
// the skill-overlay author the same "what does the merge look like?" feedback the
|
|
80
|
+
// marker model already gives for agents/commands: it lists the overlay's sections,
|
|
81
|
+
// flags markers whose target heading no longer exists in the base (the agent has
|
|
82
|
+
// nothing to anchor the override to), and reports the concatenated size. Pure — no IO.
|
|
83
|
+
function analyzeSkillConcat(baseContent, overlayContent) {
|
|
84
|
+
const base = parseMarkdown(baseContent);
|
|
85
|
+
const overlay = parseMarkdown(overlayContent);
|
|
86
|
+
const baseHeadings = new Set(base.sections.map((s) => s.heading).filter(Boolean));
|
|
87
|
+
|
|
88
|
+
const markerSections = [];
|
|
89
|
+
let plainSections = 0;
|
|
90
|
+
for (const s of overlay.sections) {
|
|
91
|
+
if (s.marker) {
|
|
92
|
+
markerSections.push({ marker: s.marker, heading: s.heading, orphan: !baseHeadings.has(s.heading) });
|
|
93
|
+
} else if (s.heading) {
|
|
94
|
+
plainSections += 1;
|
|
95
|
+
}
|
|
96
|
+
}
|
|
97
|
+
return {
|
|
98
|
+
baseLines: baseContent.split('\n').length,
|
|
99
|
+
overlayLines: overlayContent.split('\n').length,
|
|
100
|
+
baseSectionCount: base.sections.length,
|
|
101
|
+
markerSections,
|
|
102
|
+
plainSections,
|
|
103
|
+
orphans: markerSections.filter((m) => m.orphan),
|
|
104
|
+
};
|
|
105
|
+
}
|
|
106
|
+
|
|
75
107
|
function scaffoldBody(kind, name) {
|
|
76
108
|
if (kind === 'skill') {
|
|
77
109
|
return [
|
|
78
110
|
'',
|
|
79
|
-
`>
|
|
80
|
-
`> agent
|
|
81
|
-
`>
|
|
82
|
-
`>
|
|
111
|
+
`> HOW SKILL OVERLAYS WORK`,
|
|
112
|
+
`> When the "${name}" skill runs, the agent reads this file and applies it on top`,
|
|
113
|
+
`> of the base SKILL.md (runtime concatenation). NOTHING in the CLI parses the`,
|
|
114
|
+
`> [OVERRIDE]/[APPEND]/[PREPEND] markers below — for skills they are textual`,
|
|
115
|
+
`> conventions the agent honours (best-effort), not a compile-time merge (that`,
|
|
116
|
+
`> model is agent/command-only). Keep this file the single home for everything`,
|
|
117
|
+
`> project-specific the base skill deliberately leaves generic.`,
|
|
118
|
+
`>`,
|
|
119
|
+
`> Frontmatter above: bump base_skill_version when you re-review against a newer`,
|
|
120
|
+
`> framework; \`mode: override\` tells the agent to prefer this file wholesale.`,
|
|
121
|
+
`> Fill in the sections that apply and delete the rest, then run`,
|
|
122
|
+
`> \`npx baldart overlay validate .baldart/overlays/${name}.md\` to preview the merge.`,
|
|
83
123
|
'',
|
|
84
124
|
`## [APPEND] Project-specific guidance`,
|
|
85
125
|
'',
|
|
86
|
-
`<!--
|
|
126
|
+
`<!-- The default home for project facts the base "${name}" skill cannot know: -->`,
|
|
127
|
+
`<!-- - canonical paths / module entry points (cite REAL files in this repo) -->`,
|
|
128
|
+
`<!-- - domain vocabulary, naming conventions, terminology -->`,
|
|
129
|
+
`<!-- - project-specific BLOCKING rules ("never X", "always read Y first") -->`,
|
|
130
|
+
`<!-- - mandated stack / forbidden libraries beyond baldart.config.yml stack.* -->`,
|
|
131
|
+
'',
|
|
132
|
+
`## [OVERRIDE] <exact base H2 to replace>`,
|
|
133
|
+
'',
|
|
134
|
+
`<!-- OPTIONAL. Copy an H2 heading VERBATIM from the base SKILL.md to tell the -->`,
|
|
135
|
+
`<!-- agent to prefer this version over the base. \`overlay validate\` warns if -->`,
|
|
136
|
+
`<!-- the heading matches no base section. Delete this block if you only append. -->`,
|
|
87
137
|
'',
|
|
88
138
|
].join('\n');
|
|
89
139
|
}
|
|
@@ -116,60 +166,86 @@ function buildFrontmatter({ kind, name, frameworkVersion, baseSha, mode }) {
|
|
|
116
166
|
|
|
117
167
|
// ─── scaffold ──────────────────────────────────────────────────────────────
|
|
118
168
|
|
|
119
|
-
|
|
120
|
-
|
|
121
|
-
|
|
122
|
-
|
|
169
|
+
// Pure scaffolder — performs the filesystem work and returns a structured
|
|
170
|
+
// result WITHOUT printing or calling process.exit. The interactive `scaffold`
|
|
171
|
+
// wrapper renders the result; `update --on-divergence scaffold-overlays` calls
|
|
172
|
+
// this in a loop (so one already-existing or missing overlay can't terminate
|
|
173
|
+
// the whole update). Keeping the path/frontmatter logic here honours the
|
|
174
|
+
// "CLI is the source of truth, never re-implement" rule.
|
|
175
|
+
function scaffoldFile(cwd, target, options = {}) {
|
|
123
176
|
const parsed = parseTarget(target);
|
|
124
|
-
if (!parsed) {
|
|
125
|
-
UI.error('Invalid target. Use: skill/<name>, agents/<name>, or commands/<name>.');
|
|
126
|
-
UI.info('Example: npx baldart overlay scaffold agents/coder');
|
|
127
|
-
process.exit(1);
|
|
128
|
-
}
|
|
177
|
+
if (!parsed) return { ok: false, reason: 'invalid-target', target };
|
|
129
178
|
const { kind, name } = parsed;
|
|
130
179
|
|
|
131
180
|
const baseAbs = baseFilePathFor(cwd, kind, name);
|
|
132
181
|
if (!fs.existsSync(baseAbs)) {
|
|
133
|
-
|
|
134
|
-
UI.info(`Run \`ls ${path.relative(cwd, path.dirname(baseAbs))}\` to list available ${kind}s.`);
|
|
135
|
-
process.exit(1);
|
|
182
|
+
return { ok: false, reason: 'base-missing', kind, name, baseRel: path.relative(cwd, baseAbs) };
|
|
136
183
|
}
|
|
137
184
|
|
|
138
185
|
const overlayAbs = overlayPathFor(cwd, kind, name);
|
|
139
186
|
const overlayRel = path.relative(cwd, overlayAbs);
|
|
140
|
-
|
|
141
187
|
if (fs.existsSync(overlayAbs)) {
|
|
142
|
-
|
|
143
|
-
UI.info('Run `npx baldart overlay drift ' + overlayRel + '` to check it, or edit it directly to extend.');
|
|
144
|
-
process.exit(0);
|
|
188
|
+
return { ok: false, reason: 'exists', kind, name, overlayRel };
|
|
145
189
|
}
|
|
146
190
|
|
|
147
|
-
fs.mkdirSync(path.dirname(overlayAbs), { recursive: true });
|
|
148
|
-
|
|
149
|
-
const baseContent = fs.readFileSync(baseAbs, 'utf8');
|
|
150
|
-
const baseSha = computeBaseFileSha(baseContent);
|
|
151
|
-
const fwVersion = readFrameworkVersion(cwd);
|
|
152
191
|
const mode = (options.mode || 'extend').trim();
|
|
153
192
|
if (!['extend', 'override'].includes(mode)) {
|
|
154
|
-
|
|
155
|
-
process.exit(1);
|
|
193
|
+
return { ok: false, reason: 'bad-mode', mode };
|
|
156
194
|
}
|
|
157
195
|
|
|
196
|
+
fs.mkdirSync(path.dirname(overlayAbs), { recursive: true });
|
|
197
|
+
const baseContent = fs.readFileSync(baseAbs, 'utf8');
|
|
198
|
+
const baseSha = computeBaseFileSha(baseContent);
|
|
199
|
+
const fwVersion = readFrameworkVersion(cwd);
|
|
158
200
|
const content =
|
|
159
201
|
buildFrontmatter({ kind, name, frameworkVersion: fwVersion, baseSha, mode }) +
|
|
160
202
|
scaffoldBody(kind, name);
|
|
161
|
-
|
|
162
203
|
fs.writeFileSync(overlayAbs, content);
|
|
163
|
-
|
|
164
|
-
|
|
165
|
-
|
|
204
|
+
|
|
205
|
+
return {
|
|
206
|
+
ok: true, reason: 'created', kind, name, overlayRel,
|
|
207
|
+
baseRel: path.relative(cwd, baseAbs), fwVersion, baseSha,
|
|
208
|
+
};
|
|
209
|
+
}
|
|
210
|
+
|
|
211
|
+
async function scaffold(target, options = {}) {
|
|
212
|
+
const cwd = process.cwd();
|
|
213
|
+
ensureConsumer(cwd);
|
|
214
|
+
|
|
215
|
+
const res = scaffoldFile(cwd, target, options);
|
|
216
|
+
if (!res.ok) {
|
|
217
|
+
if (res.reason === 'invalid-target') {
|
|
218
|
+
UI.error('Invalid target. Use: skill/<name>, agents/<name>, or commands/<name>.');
|
|
219
|
+
UI.info('Example: npx baldart overlay scaffold agents/coder');
|
|
220
|
+
process.exit(1);
|
|
221
|
+
}
|
|
222
|
+
if (res.reason === 'base-missing') {
|
|
223
|
+
UI.error(`Base ${res.kind} "${res.name}" not found at ${res.baseRel}.`);
|
|
224
|
+
UI.info(`Run \`ls ${path.dirname(res.baseRel)}\` to list available ${res.kind}s.`);
|
|
225
|
+
process.exit(1);
|
|
226
|
+
}
|
|
227
|
+
if (res.reason === 'exists') {
|
|
228
|
+
UI.warning(`Overlay already exists: ${res.overlayRel}`);
|
|
229
|
+
UI.info('Run `npx baldart overlay drift ' + res.overlayRel + '` to check it, or edit it directly to extend.');
|
|
230
|
+
process.exit(0);
|
|
231
|
+
}
|
|
232
|
+
if (res.reason === 'bad-mode') {
|
|
233
|
+
UI.error(`Invalid --mode "${res.mode}". Use "extend" or "override".`);
|
|
234
|
+
process.exit(1);
|
|
235
|
+
}
|
|
236
|
+
process.exit(1);
|
|
237
|
+
}
|
|
238
|
+
|
|
239
|
+
UI.success(`Created ${res.overlayRel}`);
|
|
240
|
+
UI.info(`Base file: ${res.baseRel}`);
|
|
241
|
+
UI.info(`Targets framework v${res.fwVersion || 'unknown'} (base_file_sha=${res.baseSha}).`);
|
|
166
242
|
UI.newline();
|
|
167
243
|
UI.list([
|
|
168
|
-
`Edit ${overlayRel} to add your customisations.`,
|
|
169
|
-
`Validate the merge with: npx baldart overlay validate ${overlayRel}`,
|
|
170
|
-
kind === 'skill'
|
|
244
|
+
`Edit ${res.overlayRel} to add your customisations.`,
|
|
245
|
+
`Validate the merge with: npx baldart overlay validate ${res.overlayRel}`,
|
|
246
|
+
res.kind === 'skill'
|
|
171
247
|
? `Run any /-command that uses this skill to see the overlay applied at runtime.`
|
|
172
|
-
: `Apply with: npx baldart update (regenerates .claude/${kind}s/${name}.md from base + overlay).`,
|
|
248
|
+
: `Apply with: npx baldart update (regenerates .claude/${res.kind}s/${res.name}.md from base + overlay).`,
|
|
173
249
|
]);
|
|
174
250
|
}
|
|
175
251
|
|
|
@@ -235,10 +311,33 @@ async function validate(overlayPath) {
|
|
|
235
311
|
}
|
|
236
312
|
|
|
237
313
|
if (t.kind === 'skill') {
|
|
238
|
-
|
|
314
|
+
const baseContent = fs.readFileSync(baseAbs, 'utf8');
|
|
315
|
+
const a = analyzeSkillConcat(baseContent, overlayContent);
|
|
316
|
+
UI.success(`Overlay valid (skill, runtime-concat model).`);
|
|
239
317
|
UI.info(`Targets v${fm.base_skill_version || 'unknown'}, mode=${fm.mode || 'extend'}.`);
|
|
318
|
+
UI.newline();
|
|
319
|
+
// Dry-run preview. For agent/command we render the actual merged output;
|
|
320
|
+
// skills can't be merged at the CLI (the executing agent reads base + overlay
|
|
321
|
+
// and applies it), so we show what the agent will see instead of a fake merge.
|
|
322
|
+
UI.info('Merge preview (runtime concat: base SKILL.md + overlay):');
|
|
323
|
+
UI.list([
|
|
324
|
+
`base sections: ${a.baseSectionCount} · overlay sections: ${a.markerSections.length + a.plainSections} (${a.markerSections.length} marked, ${a.plainSections} new)`,
|
|
325
|
+
`concatenated size: ~${a.baseLines + a.overlayLines} lines (${a.baseLines} base + ${a.overlayLines} overlay)`,
|
|
326
|
+
]);
|
|
327
|
+
if (a.markerSections.length) {
|
|
328
|
+
UI.newline();
|
|
329
|
+
UI.info('Section markers (textual conventions — honoured by the agent at runtime, NOT parsed by the CLI):');
|
|
330
|
+
a.markerSections.forEach((m) => {
|
|
331
|
+
if (m.orphan) {
|
|
332
|
+
UI.warning(` [${m.marker}] ${m.heading} — no matching H2 in base SKILL.md; the agent has no base section to anchor this to. Fix the heading or drop the marker.`);
|
|
333
|
+
} else {
|
|
334
|
+
UI.info(` [${m.marker}] ${m.heading} — matches a base section`);
|
|
335
|
+
}
|
|
336
|
+
});
|
|
337
|
+
}
|
|
338
|
+
UI.newline();
|
|
240
339
|
if (fm.base_file_sha) {
|
|
241
|
-
const currentSha = computeBaseFileSha(
|
|
340
|
+
const currentSha = computeBaseFileSha(baseContent);
|
|
242
341
|
if (currentSha !== fm.base_file_sha) {
|
|
243
342
|
UI.warning(`Base file changed since overlay was authored (sha ${fm.base_file_sha} → ${currentSha}). Review for drift.`);
|
|
244
343
|
} else {
|
|
@@ -358,6 +457,15 @@ async function drift(target, options = {}) {
|
|
|
358
457
|
|
|
359
458
|
if (shaDrift === true) {
|
|
360
459
|
UI.warning(`${t.rel}: base file changed since overlay was authored (sha ${fm.base_file_sha} → ${baseSha}).`);
|
|
460
|
+
// For skills, surface WHICH overlay markers the base change orphaned — the
|
|
461
|
+
// same actionable feedback agent/command overlays get from the merger. A
|
|
462
|
+
// marker whose heading vanished upstream needs re-anchoring by hand.
|
|
463
|
+
if (t.kind === 'skill') {
|
|
464
|
+
const a = analyzeSkillConcat(fs.readFileSync(baseAbs, 'utf8'), content);
|
|
465
|
+
a.orphans.forEach((m) =>
|
|
466
|
+
UI.warning(` └─ overlay marker [${m.marker}] "${m.heading}" no longer matches any base H2 — likely renamed/removed upstream. Re-anchor it.`)
|
|
467
|
+
);
|
|
468
|
+
}
|
|
361
469
|
drifted++;
|
|
362
470
|
} else if (shaDrift === false) {
|
|
363
471
|
UI.success(`${t.rel}: clean (sha matches base v${fwVersion}).`);
|
|
@@ -378,4 +486,4 @@ async function drift(target, options = {}) {
|
|
|
378
486
|
UI.info(`Summary: ${clean} clean, ${drifted} drifted, ${unknown} unknown — of ${targets.length} overlay(s).`);
|
|
379
487
|
}
|
|
380
488
|
|
|
381
|
-
module.exports = { scaffold, validate, drift };
|
|
489
|
+
module.exports = { scaffold, scaffoldFile, validate, drift, parseTarget, analyzeSkillConcat };
|
package/src/commands/update.js
CHANGED
|
@@ -20,6 +20,68 @@ function readEnabledTools(cwd = process.cwd()) {
|
|
|
20
20
|
return ['claude'];
|
|
21
21
|
}
|
|
22
22
|
|
|
23
|
+
// Allowed values for `--on-divergence` — the non-interactive escape hatch that
|
|
24
|
+
// lets an agent / CI complete (or deliberately stop) an update when the
|
|
25
|
+
// consumer has local commits on `.framework/`. Without it a non-TTY caller has
|
|
26
|
+
// no non-destructive path: `--yes` refuses, the interactive prompt needs a TTY,
|
|
27
|
+
// and `--reset` is gated on a clean working tree (see Finding 2 in the
|
|
28
|
+
// agent-overlay feedback that prompted this).
|
|
29
|
+
const DIVERGENCE_STRATEGIES = ['abort', 'pull', 'scaffold-overlays'];
|
|
30
|
+
|
|
31
|
+
// Map a touched framework path to an `overlay scaffold` target string.
|
|
32
|
+
// .framework/framework/.claude/agents/coder.md → 'agents/coder'
|
|
33
|
+
// .framework/framework/.claude/commands/check.md → 'commands/check'
|
|
34
|
+
// .framework/framework/.claude/skills/bug/SKILL.md → 'skill/bug'
|
|
35
|
+
// Returns null for anything that isn't an overlay-able framework primitive.
|
|
36
|
+
function frameworkPathToOverlayTarget(p) {
|
|
37
|
+
const m = /(?:^|\/)\.framework\/framework\/\.claude\/(agents|commands|skills)\/(.+)$/.exec(p);
|
|
38
|
+
if (!m) return null;
|
|
39
|
+
const [, dir, rest] = m;
|
|
40
|
+
if (dir === 'skills') {
|
|
41
|
+
const name = rest.split('/')[0];
|
|
42
|
+
return name ? `skill/${name}` : null;
|
|
43
|
+
}
|
|
44
|
+
const name = rest.replace(/\.md$/, '');
|
|
45
|
+
if (!name || name.includes('/')) return null;
|
|
46
|
+
return `${dir}/${name}`;
|
|
47
|
+
}
|
|
48
|
+
|
|
49
|
+
// Scaffold overlay skeletons for every overlay-able framework file touched by
|
|
50
|
+
// the given commits. Reuses overlay.scaffoldFile (the CLI is the source of
|
|
51
|
+
// truth — we never re-implement frontmatter/path logic here). Idempotent:
|
|
52
|
+
// existing overlays are left untouched.
|
|
53
|
+
function scaffoldOverlaysForCommits(commits) {
|
|
54
|
+
const overlay = require('./overlay');
|
|
55
|
+
const cwd = process.cwd();
|
|
56
|
+
const targets = new Set();
|
|
57
|
+
for (const c of commits) {
|
|
58
|
+
for (const p of (c.touched || [])) {
|
|
59
|
+
const t = frameworkPathToOverlayTarget(p);
|
|
60
|
+
if (t) targets.add(t);
|
|
61
|
+
}
|
|
62
|
+
}
|
|
63
|
+
const created = [];
|
|
64
|
+
const skipped = [];
|
|
65
|
+
if (targets.size === 0) {
|
|
66
|
+
UI.warning('Could not derive any overlay target from the divergent commits.');
|
|
67
|
+
UI.info('They may touch files outside `.claude/{agents,skills,commands}/`. Use `/baldart-push` or `--on-divergence pull`.');
|
|
68
|
+
return { created, skipped };
|
|
69
|
+
}
|
|
70
|
+
for (const t of [...targets].sort()) {
|
|
71
|
+
const r = overlay.scaffoldFile(cwd, t);
|
|
72
|
+
if (r.ok) {
|
|
73
|
+
created.push(r.overlayRel);
|
|
74
|
+
UI.success(`Scaffolded overlay: ${r.overlayRel} (base: ${r.baseRel})`);
|
|
75
|
+
} else if (r.reason === 'exists') {
|
|
76
|
+
skipped.push(r.overlayRel);
|
|
77
|
+
UI.info(`Overlay already exists, left as-is: ${r.overlayRel}`);
|
|
78
|
+
} else {
|
|
79
|
+
UI.warning(`Could not scaffold ${t}: ${r.reason}${r.baseRel ? ` (${r.baseRel})` : ''}`);
|
|
80
|
+
}
|
|
81
|
+
}
|
|
82
|
+
return { created, skipped };
|
|
83
|
+
}
|
|
84
|
+
|
|
23
85
|
// Path prefixes BALDART itself writes during install/update. Anything outside
|
|
24
86
|
// these patterns is treated as user-owned and never auto-staged.
|
|
25
87
|
const BALDART_MANAGED_PATTERNS = [
|
|
@@ -359,6 +421,14 @@ async function update(options = {}) {
|
|
|
359
421
|
const autoStash = autoYes || options.autoStash === true;
|
|
360
422
|
const confirm = async (msg, def = true) => (autoYes ? def : UI.confirm(msg, def));
|
|
361
423
|
|
|
424
|
+
// Non-interactive divergence strategy (since v3.29.0). Validated up-front so
|
|
425
|
+
// a typo fails fast instead of mid-flow after the network fetch.
|
|
426
|
+
const divStrategy = options.onDivergence;
|
|
427
|
+
if (divStrategy !== undefined && !DIVERGENCE_STRATEGIES.includes(divStrategy)) {
|
|
428
|
+
UI.error(`Invalid --on-divergence "${divStrategy}". Use one of: ${DIVERGENCE_STRATEGIES.join(', ')}.`);
|
|
429
|
+
process.exit(1);
|
|
430
|
+
}
|
|
431
|
+
|
|
362
432
|
try {
|
|
363
433
|
// Step 1: Verify installation
|
|
364
434
|
UI.header('STEP 1/5: Verify Installation');
|
|
@@ -470,45 +540,107 @@ async function update(options = {}) {
|
|
|
470
540
|
} else if (status.divergenceClass === 'overlay-able') {
|
|
471
541
|
UI.warning('Detected custom edits to framework files inside `.framework/.claude/{agents,skills,commands}/`.');
|
|
472
542
|
UI.info('These are better expressed as overlays under `.baldart/overlays/` — they survive updates without divergence.');
|
|
473
|
-
|
|
543
|
+
const overlayCommits = status.divergenceCommits.filter((c) => c.category === 'custom-overlay-able');
|
|
544
|
+
for (const c of overlayCommits) {
|
|
474
545
|
UI.info(` • ${c.sha.slice(0, 7)} — ${c.subject}`);
|
|
475
546
|
}
|
|
476
|
-
|
|
477
|
-
|
|
478
|
-
|
|
479
|
-
|
|
480
|
-
|
|
481
|
-
|
|
482
|
-
|
|
483
|
-
|
|
484
|
-
|
|
485
|
-
|
|
486
|
-
|
|
487
|
-
|
|
488
|
-
UI.info('Run `/overlay` from Claude Code to migrate the custom edits, then re-run `npx baldart update`.');
|
|
547
|
+
|
|
548
|
+
// Non-interactive resolution (agents / CI) via --on-divergence.
|
|
549
|
+
if (divStrategy === 'scaffold-overlays') {
|
|
550
|
+
UI.newline();
|
|
551
|
+
scaffoldOverlaysForCommits(overlayCommits);
|
|
552
|
+
UI.newline();
|
|
553
|
+
UI.info('Next steps:');
|
|
554
|
+
UI.list([
|
|
555
|
+
'Fill in the overlay skeleton(s) above to capture your customisations (the base file shows what you changed).',
|
|
556
|
+
'Then complete the update with `npx baldart update --reset --yes --i-know` — this discards the now-redundant `.framework/` edits and reinstalls clean; your `.baldart/overlays/` are preserved and re-applied.',
|
|
557
|
+
'Alternatively, push the edits upstream with `/baldart-push` if they are generic improvements.',
|
|
558
|
+
], 'cyan');
|
|
489
559
|
process.exit(0);
|
|
490
560
|
}
|
|
491
|
-
|
|
561
|
+
if (divStrategy === 'abort') { UI.info('Update cancelled (--on-divergence abort).'); process.exit(0); }
|
|
562
|
+
if (divStrategy === 'pull') {
|
|
563
|
+
UI.info('Keeping the custom commits (--on-divergence pull). The subtree pull will attempt a merge.');
|
|
564
|
+
// fall through
|
|
565
|
+
} else if (autoYes) {
|
|
566
|
+
// --yes without an explicit strategy: never silently overwrite. Tell
|
|
567
|
+
// the (likely non-TTY) caller exactly which flag completes the update.
|
|
568
|
+
UI.error('Custom edits present — refusing to resolve them unattended under --yes.');
|
|
569
|
+
UI.info('Re-run with one of:');
|
|
570
|
+
UI.list([
|
|
571
|
+
'--on-divergence scaffold-overlays → auto-create overlay skeleton(s), then stop so you can fill them',
|
|
572
|
+
'--on-divergence pull → keep the commits and merge the update (non-destructive)',
|
|
573
|
+
'--reset --yes --i-know → discard .framework/ edits and reinstall clean (destructive)',
|
|
574
|
+
], 'cyan');
|
|
575
|
+
process.exit(1);
|
|
576
|
+
} else {
|
|
577
|
+
const choice = await UI.select('How would you like to proceed?', [
|
|
578
|
+
{ name: 'Scaffold overlay skeleton(s) now (recommended — then fill them in & re-run)', value: 'overlay' },
|
|
579
|
+
{ name: 'Keep the custom commits (subtree pull may conflict)', value: 'keep' },
|
|
580
|
+
{ name: 'Abort', value: 'abort' },
|
|
581
|
+
]);
|
|
582
|
+
if (choice === 'abort') { UI.info('Update cancelled.'); process.exit(0); }
|
|
583
|
+
if (choice === 'overlay') {
|
|
584
|
+
UI.newline();
|
|
585
|
+
scaffoldOverlaysForCommits(overlayCommits);
|
|
586
|
+
UI.newline();
|
|
587
|
+
UI.info('Next steps:');
|
|
588
|
+
UI.list([
|
|
589
|
+
'Edit the overlay skeleton(s) above to capture your customisations (`/overlay` from Claude Code gives a guided flow).',
|
|
590
|
+
'Then complete the update with `npx baldart update --reset` — discards the redundant `.framework/` edits, reinstalls clean, re-applies your overlays.',
|
|
591
|
+
], 'cyan');
|
|
592
|
+
process.exit(0);
|
|
593
|
+
}
|
|
594
|
+
// 'keep' → fall through, subtree pull will attempt merge
|
|
595
|
+
}
|
|
492
596
|
} else if (status.divergenceClass === 'real-custom' || status.divergenceClass === 'mixed') {
|
|
493
597
|
UI.warning(`Detected ${status.divergenceCommits.filter((c) => c.category === 'custom-other').length} custom commit(s) on .framework/ that are NOT auto-classifiable as overlays.`);
|
|
494
598
|
for (const c of status.divergenceCommits.filter((c) => !['subtree-merge', 'subtree-squash', 'chore-wrapper'].includes(c.category))) {
|
|
495
599
|
UI.info(` • ${c.sha.slice(0, 7)} [${c.category}] — ${c.subject}`);
|
|
496
600
|
}
|
|
497
|
-
|
|
498
|
-
|
|
499
|
-
|
|
500
|
-
|
|
501
|
-
|
|
502
|
-
|
|
503
|
-
|
|
504
|
-
|
|
505
|
-
|
|
506
|
-
|
|
507
|
-
|
|
508
|
-
UI.
|
|
601
|
+
const overlayCommits = status.divergenceCommits.filter((c) => c.category === 'custom-overlay-able');
|
|
602
|
+
|
|
603
|
+
// Non-interactive resolution (agents / CI) via --on-divergence.
|
|
604
|
+
if (divStrategy === 'scaffold-overlays') {
|
|
605
|
+
if (overlayCommits.length === 0) {
|
|
606
|
+
UI.error('No overlay-able commits to scaffold — every custom commit touches non-overlayable paths (src/, CHANGELOG, …).');
|
|
607
|
+
UI.info('Use `/baldart-push` to contribute them upstream, or `--on-divergence pull` to merge.');
|
|
608
|
+
process.exit(1);
|
|
609
|
+
}
|
|
610
|
+
UI.newline();
|
|
611
|
+
scaffoldOverlaysForCommits(overlayCommits);
|
|
612
|
+
UI.newline();
|
|
613
|
+
UI.warning('Scaffolded overlays for the overlay-able commits, but custom-other commit(s) remain — NOT pulling (would be destructive to those edits).');
|
|
614
|
+
UI.info('Resolve the remaining commits (push upstream with `/baldart-push`, or revert), then re-run `npx baldart update`.');
|
|
509
615
|
process.exit(0);
|
|
510
616
|
}
|
|
511
|
-
|
|
617
|
+
if (divStrategy === 'abort') { UI.info('Update cancelled (--on-divergence abort).'); process.exit(0); }
|
|
618
|
+
if (divStrategy === 'pull') {
|
|
619
|
+
UI.info('Pulling over the custom commits (--on-divergence pull). A merge commit will be created — resolve any conflicts that surface.');
|
|
620
|
+
// fall through
|
|
621
|
+
} else if (autoYes) {
|
|
622
|
+
UI.error('Custom commits present — refusing to pull over them unattended under --yes.');
|
|
623
|
+
UI.info('Re-run with one of:');
|
|
624
|
+
UI.list([
|
|
625
|
+
'--on-divergence pull → create a merge commit over your edits (non-destructive)',
|
|
626
|
+
'--on-divergence scaffold-overlays → scaffold overlays for the overlay-able subset (custom-other still needs handling)',
|
|
627
|
+
'--on-divergence abort → do nothing',
|
|
628
|
+
], 'cyan');
|
|
629
|
+
UI.info('Best practice: contribute generic improvements upstream first with `/baldart-push`.');
|
|
630
|
+
process.exit(1);
|
|
631
|
+
} else {
|
|
632
|
+
const choice = await UI.select('How would you like to proceed?', [
|
|
633
|
+
{ name: 'Push these commits upstream first (recommended — `/baldart-push`)', value: 'push' },
|
|
634
|
+
{ name: 'Pull anyway (will create a merge commit — possible conflicts)', value: 'pull' },
|
|
635
|
+
{ name: 'Abort', value: 'abort' },
|
|
636
|
+
]);
|
|
637
|
+
if (choice === 'abort') { UI.info('Update cancelled.'); process.exit(0); }
|
|
638
|
+
if (choice === 'push') {
|
|
639
|
+
UI.info('Run `/baldart-push` from Claude Code (or `npx baldart push`) to contribute upstream, then re-run update.');
|
|
640
|
+
process.exit(0);
|
|
641
|
+
}
|
|
642
|
+
// 'pull' → fall through
|
|
643
|
+
}
|
|
512
644
|
}
|
|
513
645
|
// 'unknown' → conservative: continue silently, the subtree pull will
|
|
514
646
|
// surface any real conflict.
|
|
@@ -0,0 +1,89 @@
|
|
|
1
|
+
'use strict';
|
|
2
|
+
const { test } = require('node:test');
|
|
3
|
+
const assert = require('node:assert');
|
|
4
|
+
const { analyzeSkillConcat } = require('../../commands/overlay');
|
|
5
|
+
|
|
6
|
+
// analyzeSkillConcat powers the skill-overlay dry-run in `overlay validate` and
|
|
7
|
+
// the orphaned-marker warning in `overlay drift`. Skills use runtime concat, so
|
|
8
|
+
// the markers are NOT parsed — this helper only DESCRIBES what the agent will
|
|
9
|
+
// see and flags markers whose heading no longer anchors to a base section.
|
|
10
|
+
|
|
11
|
+
const BASE = `---
|
|
12
|
+
name: bug
|
|
13
|
+
---
|
|
14
|
+
|
|
15
|
+
## Phase 1
|
|
16
|
+
reproduce
|
|
17
|
+
|
|
18
|
+
## Phase 2
|
|
19
|
+
fix
|
|
20
|
+
`;
|
|
21
|
+
|
|
22
|
+
test('marker matching a base heading is not an orphan', () => {
|
|
23
|
+
const overlay = `---
|
|
24
|
+
base_skill: bug
|
|
25
|
+
---
|
|
26
|
+
|
|
27
|
+
## [APPEND] Phase 1
|
|
28
|
+
project entry points
|
|
29
|
+
`;
|
|
30
|
+
const a = analyzeSkillConcat(BASE, overlay);
|
|
31
|
+
assert.strictEqual(a.markerSections.length, 1);
|
|
32
|
+
assert.strictEqual(a.markerSections[0].orphan, false);
|
|
33
|
+
assert.strictEqual(a.orphans.length, 0);
|
|
34
|
+
});
|
|
35
|
+
|
|
36
|
+
test('marker with no matching base heading is flagged orphan', () => {
|
|
37
|
+
const overlay = `---
|
|
38
|
+
base_skill: bug
|
|
39
|
+
---
|
|
40
|
+
|
|
41
|
+
## [OVERRIDE] Phase 9
|
|
42
|
+
heading absent from base
|
|
43
|
+
`;
|
|
44
|
+
const a = analyzeSkillConcat(BASE, overlay);
|
|
45
|
+
assert.strictEqual(a.orphans.length, 1);
|
|
46
|
+
assert.strictEqual(a.orphans[0].heading, 'Phase 9');
|
|
47
|
+
assert.strictEqual(a.orphans[0].marker, 'OVERRIDE');
|
|
48
|
+
});
|
|
49
|
+
|
|
50
|
+
test('unmarked overlay sections count as new (plain), not markers', () => {
|
|
51
|
+
const overlay = `---
|
|
52
|
+
base_skill: bug
|
|
53
|
+
---
|
|
54
|
+
|
|
55
|
+
## Brand vocabulary
|
|
56
|
+
project terms
|
|
57
|
+
|
|
58
|
+
## [APPEND] Phase 2
|
|
59
|
+
extra
|
|
60
|
+
`;
|
|
61
|
+
const a = analyzeSkillConcat(BASE, overlay);
|
|
62
|
+
assert.strictEqual(a.plainSections, 1);
|
|
63
|
+
assert.strictEqual(a.markerSections.length, 1);
|
|
64
|
+
assert.strictEqual(a.baseSectionCount, 2);
|
|
65
|
+
});
|
|
66
|
+
|
|
67
|
+
test('reports line counts for both base and overlay', () => {
|
|
68
|
+
const overlay = `---
|
|
69
|
+
base_skill: bug
|
|
70
|
+
---
|
|
71
|
+
|
|
72
|
+
## [APPEND] Phase 1
|
|
73
|
+
x
|
|
74
|
+
`;
|
|
75
|
+
const a = analyzeSkillConcat(BASE, overlay);
|
|
76
|
+
assert.ok(a.baseLines > 0);
|
|
77
|
+
assert.ok(a.overlayLines > 0);
|
|
78
|
+
});
|
|
79
|
+
|
|
80
|
+
test('empty overlay body yields no sections and no orphans', () => {
|
|
81
|
+
const overlay = `---
|
|
82
|
+
base_skill: bug
|
|
83
|
+
---
|
|
84
|
+
`;
|
|
85
|
+
const a = analyzeSkillConcat(BASE, overlay);
|
|
86
|
+
assert.strictEqual(a.markerSections.length, 0);
|
|
87
|
+
assert.strictEqual(a.plainSections, 0);
|
|
88
|
+
assert.strictEqual(a.orphans.length, 0);
|
|
89
|
+
});
|
|
@@ -41,6 +41,65 @@ const SUBJECT_FIXTURES = [
|
|
|
41
41
|
{ subject: "Squashed 'docs/' content from abc..def", expected: null },
|
|
42
42
|
// Chore without 'baldart' keyword — NOT chore-wrapper
|
|
43
43
|
{ subject: 'chore: update dependencies', expected: null },
|
|
44
|
+
|
|
45
|
+
// ─── subject-regex BLIND SPOTS (now covered by parent-count detection) ──
|
|
46
|
+
// git omits the "into <branch>" suffix when the merge lands on `master`,
|
|
47
|
+
// so a real subtree-pull merge can present as a bare "Merge commit 'x'".
|
|
48
|
+
// The subject classifier returns null here; classifyDivergence() then
|
|
49
|
+
// relabels it `subtree-merge` because the commit has 2+ parents. We assert
|
|
50
|
+
// the subject classifier's null so the gap (and why parents matter) is
|
|
51
|
+
// documented and regression-locked.
|
|
52
|
+
{ subject: "Merge commit 'abc123'", expected: null },
|
|
53
|
+
// Shared-repo collaboration merge — the " of <url>" segment breaks the
|
|
54
|
+
// narrow `Merge branch 'x' into <branch>$` shape. Also null → parent count.
|
|
55
|
+
{ subject: "Merge branch 'develop' of github.com:org/repo into develop", expected: null },
|
|
56
|
+
{ subject: "Merge remote-tracking branch 'origin/main'", expected: null },
|
|
57
|
+
];
|
|
58
|
+
|
|
59
|
+
// aggregateDivergenceClass(commits) — pure reduction of per-commit categories
|
|
60
|
+
// to the divergence class. The headline Finding-1 case: a single overlay-able
|
|
61
|
+
// commit alongside a plumbing merge must stay `overlay-able`, NOT collapse to
|
|
62
|
+
// `mixed`, because the merge is excluded as noise before the decision.
|
|
63
|
+
const AGGREGATE_FIXTURES = [
|
|
64
|
+
{
|
|
65
|
+
name: 'overlay-able commit + plumbing merge → overlay-able (not mixed)',
|
|
66
|
+
commits: [
|
|
67
|
+
{ category: 'custom-overlay-able' },
|
|
68
|
+
{ category: 'subtree-merge' },
|
|
69
|
+
],
|
|
70
|
+
expected: 'overlay-able',
|
|
71
|
+
},
|
|
72
|
+
{
|
|
73
|
+
name: 'overlay-able + subtree-squash + chore-wrapper → overlay-able',
|
|
74
|
+
commits: [
|
|
75
|
+
{ category: 'custom-overlay-able' },
|
|
76
|
+
{ category: 'subtree-squash' },
|
|
77
|
+
{ category: 'chore-wrapper' },
|
|
78
|
+
],
|
|
79
|
+
expected: 'overlay-able',
|
|
80
|
+
},
|
|
81
|
+
{
|
|
82
|
+
name: 'overlay-able + custom-other → mixed (genuine mix preserved)',
|
|
83
|
+
commits: [
|
|
84
|
+
{ category: 'custom-overlay-able' },
|
|
85
|
+
{ category: 'custom-other' },
|
|
86
|
+
],
|
|
87
|
+
expected: 'mixed',
|
|
88
|
+
},
|
|
89
|
+
{
|
|
90
|
+
name: 'only custom-other → real-custom',
|
|
91
|
+
commits: [{ category: 'custom-other' }, { category: 'subtree-merge' }],
|
|
92
|
+
expected: 'real-custom',
|
|
93
|
+
},
|
|
94
|
+
{
|
|
95
|
+
name: 'only noise → all-noise',
|
|
96
|
+
commits: [
|
|
97
|
+
{ category: 'subtree-merge' },
|
|
98
|
+
{ category: 'subtree-squash' },
|
|
99
|
+
{ category: 'chore-wrapper' },
|
|
100
|
+
],
|
|
101
|
+
expected: 'all-noise',
|
|
102
|
+
},
|
|
44
103
|
];
|
|
45
104
|
|
|
46
105
|
const PATH_FIXTURES = [
|
|
@@ -116,5 +175,17 @@ for (const f of PATH_FIXTURES) {
|
|
|
116
175
|
}
|
|
117
176
|
}
|
|
118
177
|
|
|
178
|
+
console.log('\n─── Aggregate class ───');
|
|
179
|
+
for (const f of AGGREGATE_FIXTURES) {
|
|
180
|
+
const actual = GitUtils.aggregateDivergenceClass(f.commits);
|
|
181
|
+
if (actual === f.expected) {
|
|
182
|
+
pass++;
|
|
183
|
+
console.log(` ✓ ${f.name} → ${actual}`);
|
|
184
|
+
} else {
|
|
185
|
+
fail++;
|
|
186
|
+
console.log(` ✗ ${f.name} → ${actual} (expected ${f.expected})`);
|
|
187
|
+
}
|
|
188
|
+
}
|
|
189
|
+
|
|
119
190
|
console.log(`\nResult: ${pass} pass, ${fail} fail`);
|
|
120
191
|
process.exit(fail > 0 ? 1 : 0);
|
package/src/utils/git.js
CHANGED
|
@@ -263,43 +263,64 @@ class GitUtils {
|
|
|
263
263
|
const raw = await this.git.raw([
|
|
264
264
|
'log',
|
|
265
265
|
'FETCH_HEAD..HEAD',
|
|
266
|
-
'--pretty=format:%H%x00%s',
|
|
266
|
+
'--pretty=format:%H%x00%P%x00%s',
|
|
267
267
|
'--',
|
|
268
268
|
FRAMEWORK_DIR
|
|
269
269
|
]);
|
|
270
270
|
for (const line of raw.split('\n')) {
|
|
271
271
|
if (!line.trim()) continue;
|
|
272
|
-
const [sha, ...rest] = line.split('');
|
|
272
|
+
const [sha, parentStr, ...rest] = line.split('');
|
|
273
273
|
const subject = rest.join('');
|
|
274
|
+
const parents = (parentStr || '').trim().split(/\s+/).filter(Boolean);
|
|
275
|
+
const isMerge = parents.length >= 2;
|
|
274
276
|
let category = this.classifyCommitSubject(subject);
|
|
277
|
+
let touched;
|
|
278
|
+
if (!category && isMerge) {
|
|
279
|
+
// A merge commit is git plumbing, never a user customization. The
|
|
280
|
+
// real content (if any) lives in the merged commits, which appear
|
|
281
|
+
// in this range on their own when they aren't already upstream.
|
|
282
|
+
// `git subtree pull --squash` merges vary in subject across git
|
|
283
|
+
// versions and default-branch name ("Merge commit 'x'" with no
|
|
284
|
+
// "into <branch>" suffix when landing on `master`; "Merge branch
|
|
285
|
+
// 'x' of <url> into <branch>" on shared repos), so the subject
|
|
286
|
+
// regex alone is brittle. Detect merges structurally by parent
|
|
287
|
+
// count and treat them as noise — otherwise a single plumbing
|
|
288
|
+
// merge poisons an otherwise overlay-able set into `mixed`.
|
|
289
|
+
category = 'subtree-merge';
|
|
290
|
+
}
|
|
275
291
|
if (!category) {
|
|
276
292
|
// Need file list to disambiguate custom-overlay-able vs custom-other.
|
|
277
|
-
|
|
293
|
+
touched = [];
|
|
278
294
|
try {
|
|
279
295
|
const filesRaw = await this.git.raw(['diff-tree', '--no-commit-id', '--name-only', '-r', sha]);
|
|
280
296
|
touched = filesRaw.split('\n').filter((l) => l.trim());
|
|
281
297
|
} catch (_) { /* leave empty → custom-other */ }
|
|
282
298
|
category = this.classifyCommitByPaths(touched);
|
|
283
299
|
}
|
|
284
|
-
commits.push({ sha, subject, category });
|
|
300
|
+
commits.push({ sha, subject, category, parents, touched: touched || [] });
|
|
285
301
|
}
|
|
286
302
|
} catch (_) {
|
|
287
303
|
return { class: 'unknown', commits: [] };
|
|
288
304
|
}
|
|
289
305
|
|
|
290
306
|
if (commits.length === 0) return { class: 'all-noise', commits: [] };
|
|
307
|
+
return { class: GitUtils.aggregateDivergenceClass(commits), commits };
|
|
308
|
+
}
|
|
291
309
|
|
|
310
|
+
// Pure aggregation of per-commit categories into the divergence class.
|
|
311
|
+
// Extracted as a static method so it can be unit-tested without a live repo
|
|
312
|
+
// (see src/utils/__tests__/classify-divergence.test.js). Noise categories
|
|
313
|
+
// (subtree plumbing + CLI chore wrappers + merge commits) are excluded
|
|
314
|
+
// BEFORE deciding overlay-able vs mixed, so plumbing never downgrades the UX.
|
|
315
|
+
static aggregateDivergenceClass(commits) {
|
|
292
316
|
const noiseCategories = new Set(['subtree-merge', 'subtree-squash', 'chore-wrapper']);
|
|
293
317
|
const nonNoise = commits.filter((c) => !noiseCategories.has(c.category));
|
|
294
|
-
if (nonNoise.length === 0) return
|
|
295
|
-
|
|
318
|
+
if (nonNoise.length === 0) return 'all-noise';
|
|
296
319
|
const hasOverlayable = nonNoise.some((c) => c.category === 'custom-overlay-able');
|
|
297
320
|
const hasOther = nonNoise.some((c) => c.category === 'custom-other');
|
|
298
|
-
|
|
299
|
-
if (hasOverlayable &&
|
|
300
|
-
|
|
301
|
-
|
|
302
|
-
return { class: aggregate, commits };
|
|
321
|
+
if (hasOverlayable && !hasOther) return 'overlay-able';
|
|
322
|
+
if (hasOverlayable && hasOther) return 'mixed';
|
|
323
|
+
return 'real-custom';
|
|
303
324
|
}
|
|
304
325
|
|
|
305
326
|
async getUpdateStatus(repo, branch = 'main') {
|
package/src/utils/hooks.js
CHANGED
|
@@ -87,6 +87,23 @@ const HOOK_REGISTRY = [
|
|
|
87
87
|
command: 'bash .framework/framework/.claude/hooks/agent-discovery-info.sh',
|
|
88
88
|
order: 100,
|
|
89
89
|
},
|
|
90
|
+
{
|
|
91
|
+
// Observation-only telemetry: records every Read of a
|
|
92
|
+
// `.baldart/overlays/*.md` file to `.baldart/telemetry/overlay-loads.jsonl`.
|
|
93
|
+
// Measures, objectively, whether skill overlays are actually loaded at
|
|
94
|
+
// runtime (skill overlays use runtime-concat, which is NOT structurally
|
|
95
|
+
// enforced — unlike agent/command overlays, which are compile-time merged).
|
|
96
|
+
// Numerator-only by design: skill invocation is prompt-expansion, not a
|
|
97
|
+
// tool call, so the denominator is not hook-observable (claude-code #43630,
|
|
98
|
+
// #22655). bash, not node — it fires on EVERY Read (hottest tool) in the
|
|
99
|
+
// critical path, so a per-read node cold-start would tax every file read.
|
|
100
|
+
// Fail-safe — never blocks. See framework/docs/OVERLAY-TELEMETRY.md.
|
|
101
|
+
id: 'baldart-overlay-telemetry',
|
|
102
|
+
event: 'PostToolUse',
|
|
103
|
+
matcher: 'Read',
|
|
104
|
+
command: 'bash .framework/framework/.claude/hooks/overlay-telemetry.sh',
|
|
105
|
+
order: 100,
|
|
106
|
+
},
|
|
90
107
|
// Future hooks (T1.3, T1.4, T2.3) will be appended here, e.g.:
|
|
91
108
|
// {
|
|
92
109
|
// id: 'baldart-capture-detect',
|