baldart 3.29.0 → 3.31.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +44 -0
- package/VERSION +1 -1
- package/framework/.claude/hooks/overlay-telemetry.sh +88 -0
- package/framework/.claude/skills/baldart-update/SKILL.md +115 -12
- package/framework/agents/project-context.md +1 -1
- package/framework/docs/OVERLAY-TELEMETRY.md +77 -0
- package/framework/scripts/analyze-overlay-loads.js +130 -0
- package/package.json +1 -1
- package/src/commands/overlay.js +91 -9
- package/src/utils/__tests__/analyze-skill-concat.test.js +89 -0
- package/src/utils/hooks.js +17 -0
- package/src/utils/routine-adapters/claude-code-cloud.js +7 -0
- package/src/utils/routine-adapters/cron.js +7 -0
- package/src/utils/routine-adapters/github-actions.js +7 -0
package/CHANGELOG.md
CHANGED
|
@@ -5,6 +5,50 @@ All notable changes to BALDART will be documented in this file.
|
|
|
5
5
|
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
|
|
6
6
|
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
|
7
7
|
|
|
8
|
+
## [3.31.0] - 2026-05-30
|
|
9
|
+
|
|
10
|
+
Gives `/baldart-update` an **autonomous mode** so agents and scheduled routines can bring the framework up-to-date with **zero human interaction** — closing the gap that the skill, as shipped, was *purely* human-in-the-loop (it replicated every CLI decision point as a chat confirmation, which an unattended agent cannot answer). The new mode is **auto-detected** from a positive automation signal and keeps every one of the CLI's safety gates: it drives `npx baldart update --yes` and **stops hard** (never forces a merge or reset) the moment the CLI refuses on custom divergence or a conflict — exactly the calls a human should own. **No new `baldart.config.yml` keys** — `BALDART_AUTONOMOUS` is a runtime env var, not a config flag, so the schema-propagation rule does not apply.
|
|
11
|
+
|
|
12
|
+
### Added — autonomous (unattended) mode for `/baldart-update`
|
|
13
|
+
|
|
14
|
+
- **[framework/.claude/skills/baldart-update/SKILL.md](framework/.claude/skills/baldart-update/SKILL.md)**: new **Mode detection** first step (`printenv | grep -qE '^(BALDART_AUTONOMOUS|CI|GITHUB_ACTIONS)='`) branches the skill into INTERACTIVE (a human is present → existing chat-confirmation flow, unchanged) or AUTONOMOUS (no human → new flow). Detection is **positive-signal-only**: absence of every signal defaults to INTERACTIVE so a human never silently loses control. Documents *why* the TTY test (`[ -t 0 ]`) cannot be used — Claude's Bash tool never has a TTY on stdin, even with a human present, so it can't tell the two situations apart.
|
|
15
|
+
- **`## Autonomous mode`** section: pre-flight (shared Step 0 + CLI version drift gate) → `npx baldart update --yes` (bare, `timeout: 600000`) → exit-code interpretation → shared post-flight assert → structured log-friendly report. **Hard guardrail**: on a non-zero exit (custom-overlay-able / real-custom divergence, subtree-pull conflict, stale-CLI halt) the skill STOPS and reports the exact flag a human would re-run with — it **never** retries with `--on-divergence pull|scaffold-overlays` or `--reset` to force past a refusal. A refusal is a *correct* outcome that needs a human.
|
|
16
|
+
|
|
17
|
+
### Added — automation marker injected by the routine-adapters
|
|
18
|
+
|
|
19
|
+
- **[src/utils/routine-adapters/github-actions.js](src/utils/routine-adapters/github-actions.js)**: the generated workflow's run step now sets `BALDART_AUTONOMOUS: '1'` in its `env:` block (alongside `ANTHROPIC_API_KEY`), inherited by the `claude` process and every child Bash it spawns.
|
|
20
|
+
- **[src/utils/routine-adapters/cron.js](src/utils/routine-adapters/cron.js)**: the generated shell wrapper `export`s `BALDART_AUTONOMOUS=1` before invoking `claude`.
|
|
21
|
+
- **[src/utils/routine-adapters/claude-code-cloud.js](src/utils/routine-adapters/claude-code-cloud.js)**: the generated routine config now carries `env: { BALDART_AUTONOMOUS: '1' }` (best-effort — relies on the RemoteTrigger surfacing `env` into the run environment; github-actions / cron inject it directly).
|
|
22
|
+
|
|
23
|
+
### Changed — drift-check counter + honest framing
|
|
24
|
+
|
|
25
|
+
- **[framework/.claude/skills/baldart-update/SKILL.md](framework/.claude/skills/baldart-update/SKILL.md)**: `last_verified` v3.26.0 → v3.31.0; added an **AUTONOMY NOTE** recording that `BALDART_AUTONOMOUS` is a runtime env var (not a config key) so the schema-propagation rule does not apply. Hard rules gain a rule #0 (run mode detection first; default-on-absence is INTERACTIVE); rule #2 now qualifies the chat-replication mandate as INTERACTIVE-only. The closing "both keep the user in the loop" claim is corrected to reflect that autonomous mode drops confirmations while keeping the safety gates.
|
|
26
|
+
|
|
27
|
+
## [3.30.0] - 2026-05-29
|
|
28
|
+
|
|
29
|
+
Hardens the **overlay** system — the layer that turns generic framework skills into project-specific knowledge. An audit surfaced a structural asymmetry: agent/command overlays are *compile-time merged* (the overlay is materially fused into the generated file → guaranteed delivery), but skill overlays use *runtime concatenation* (a line of prose tells the agent to read `.baldart/overlays/<skill>.md` → best-effort, nothing verifies it loaded). The most important overlays for knowledge transfer had the weakest guarantee. This release ships the high-certainty authoring/DX fixes now and adds **observation-only telemetry** to gather real data on whether skill overlays actually load — before deciding whether a stronger enforcement mechanism is worth building. **No new `baldart.config.yml` keys** — the telemetry hook is always-on (like `framework-edit-gate` / `agent-discovery-gate`), so the schema-propagation rule does not apply.
|
|
30
|
+
|
|
31
|
+
### Added — overlay-load telemetry (observation only)
|
|
32
|
+
|
|
33
|
+
- **[framework/.claude/hooks/overlay-telemetry.sh](framework/.claude/hooks/overlay-telemetry.sh)**: a fail-safe `PostToolUse` hook (matcher `Read`) that appends one JSON line to `.baldart/telemetry/overlay-loads.jsonl` every time a `.baldart/overlays/*.md` file is read. Objective, not self-reported — the failure mode being measured is "the agent forgot the overlay", which a self-report would corrupt in exactly that case. Written in **bash, not node**, on purpose: it fires on every `Read` (the hottest tool) in the critical path, so a per-read node cold-start (~50ms) would tax every file read — a bash fork (~5ms) keeps the hot path cheap. **Numerator-only by design**: skill invocation is prompt-expansion, not a tool call, so the denominator (skill runs that *should* have loaded an overlay) is not hook-observable (claude-code [#43630](https://github.com/anthropics/claude-code/issues/43630), [#22655](https://github.com/anthropics/claude-code/issues/22655)). Registered in `HOOK_REGISTRY` (`baldart-overlay-telemetry`) → auto-registered by `add`, backfilled by `update`, reported by `doctor` — no call-site changes (registry-driven).
|
|
34
|
+
- **[framework/scripts/analyze-overlay-loads.js](framework/scripts/analyze-overlay-loads.js)**: zero-dependency aggregator that cross-references the read log against the overlays that exist on disk. Headline signal: a **skill** overlay that exists but is read ~never (`NEVER-LOADED`). Agent/command overlays (compile-time merged) are deliberately not flagged on zero reads.
|
|
35
|
+
- **[framework/docs/OVERLAY-TELEMETRY.md](framework/docs/OVERLAY-TELEMETRY.md)**: full reading guide, with an explicit "conservative proxy — UNDER-counts" section (in-context reads and prompt-expansion are invisible to the hook; a low count is a reason to investigate, not proof of a miss).
|
|
36
|
+
|
|
37
|
+
### Changed — overlay authoring/DX parity for skills
|
|
38
|
+
|
|
39
|
+
- **[src/commands/overlay.js](src/commands/overlay.js)** `validate` (skill path): now renders a **merge dry-run** for skill overlays — section counts, concatenated size, and a per-marker check flagging `[OVERRIDE]/[APPEND]/[PREPEND]` headings that don't anchor to any base H2. Previously it printed only "valid, concat model", so a skill-overlay author never saw what the merge would look like (agent/command overlays already got this). Powered by a new pure, unit-tested `analyzeSkillConcat()` helper.
|
|
40
|
+
- **[src/commands/overlay.js](src/commands/overlay.js)** `drift` (skill path): when the base file changed, now lists *which* overlay markers were orphaned by the change (heading renamed/removed upstream) — the same actionable feedback agent/command overlays get from the merger, instead of a bare sha-mismatch warning.
|
|
41
|
+
- **[src/commands/overlay.js](src/commands/overlay.js)** `scaffoldBody` (skill path): the `overlay scaffold skill/<x>` skeleton is now a guided template — explains the runtime-concat model honestly, enumerates the typical project-fact sections (canonical paths, domain vocabulary, BLOCKING rules, mandated stack), and shows a ready-to-edit `[OVERRIDE]` block. Improves authoring for all skills, not just the two that ship an example overlay.
|
|
42
|
+
- **[framework/agents/project-context.md](framework/agents/project-context.md)** §5 precedence rules: corrected the overclaim that a skill overlay's `## [OVERRIDE] <topic>` "replaces" the base section. For skills nothing parses the markers — they are textual conventions the executing agent honours (best-effort), unlike agent/command overlays which are materially fused (§5.bis). The doc now matches `scaffoldBody`'s honest framing instead of the middle.
|
|
43
|
+
|
|
44
|
+
### Changed — drift-check counter
|
|
45
|
+
|
|
46
|
+
- **[framework/.claude/skills/baldart-update/SKILL.md](framework/.claude/skills/baldart-update/SKILL.md)**: `hook_registry_entries` 3 → 4 (the new telemetry hook). The `add`/`update`/`doctor` integration is registry-driven, so no narration changes were needed.
|
|
47
|
+
|
|
48
|
+
### Added — tests
|
|
49
|
+
|
|
50
|
+
- **[src/utils/__tests__/analyze-skill-concat.test.js](src/utils/__tests__/analyze-skill-concat.test.js)**: 5 cases covering marker matching, orphan detection, plain (new) sections, line counts, and the empty-overlay edge case.
|
|
51
|
+
|
|
8
52
|
## [3.29.0] - 2026-05-29
|
|
9
53
|
|
|
10
54
|
Closes a two-part process gap in `npx baldart update` reported from a non-TTY agent context on a shared consumer repo: (1) plumbing/collaboration merge commits were mislabeled `custom-other`, downgrading an otherwise overlay-able divergence set into `mixed`; (2) an automated agent had **no** non-destructive path to complete an update when the consumer carried local `.framework/` commits — `--yes` refused, the interactive prompt needed a TTY, and `--reset` is gated on a clean working tree. **No new `baldart.config.yml` keys** — `--on-divergence` is a CLI flag, not a config flag, so the schema-propagation rule does not apply.
|
package/VERSION
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
3.
|
|
1
|
+
3.31.0
|
|
@@ -0,0 +1,88 @@
|
|
|
1
|
+
#!/usr/bin/env bash
|
|
2
|
+
#
|
|
3
|
+
# overlay-telemetry.sh — BALDART PostToolUse hook (matcher: Read).
|
|
4
|
+
#
|
|
5
|
+
# OBSERVATION ONLY. Appends one JSON line to
|
|
6
|
+
# `.baldart/telemetry/overlay-loads.jsonl` every time a `.baldart/overlays/*.md`
|
|
7
|
+
# file is read. This is the *numerator* of the overlay-load question: when the
|
|
8
|
+
# agent runs a skill that HAS an overlay, does it actually load it? A `Read` of
|
|
9
|
+
# the overlay file is the one objective signal — skill self-report would be
|
|
10
|
+
# corrupted in exactly the case we measure ("the agent forgot the overlay").
|
|
11
|
+
#
|
|
12
|
+
# Bash, NOT node, on purpose: this fires on EVERY `Read` (the hottest tool) in
|
|
13
|
+
# the consumer's critical path. A per-read node cold-start (~50ms) would tax
|
|
14
|
+
# every file read; a bash fork (~5ms) keeps the hot path cheap. The work is just
|
|
15
|
+
# a path match + a conditional append, so bash is sufficient. Same precedent as
|
|
16
|
+
# agent-discovery-info.sh.
|
|
17
|
+
#
|
|
18
|
+
# Numerator-only by design — skill invocation is prompt-expansion, not a tool
|
|
19
|
+
# call, so the denominator is not hook-observable (claude-code #43630, #22655).
|
|
20
|
+
# See framework/docs/OVERLAY-TELEMETRY.md for the under-count caveats.
|
|
21
|
+
#
|
|
22
|
+
# Fail-safe: always exit 0, never disrupt the session.
|
|
23
|
+
|
|
24
|
+
set -u
|
|
25
|
+
exec 2>/dev/null # never surface stderr to the user
|
|
26
|
+
|
|
27
|
+
INPUT=""
|
|
28
|
+
if [ ! -t 0 ]; then
|
|
29
|
+
INPUT="$(cat)"
|
|
30
|
+
fi
|
|
31
|
+
[ -z "$INPUT" ] && exit 0
|
|
32
|
+
|
|
33
|
+
get_field() {
|
|
34
|
+
# $1 = JSON key (top-level or nested — sed fallback matches anywhere)
|
|
35
|
+
if command -v jq >/dev/null 2>&1; then
|
|
36
|
+
printf '%s' "$INPUT" | jq -r "$2 // empty" 2>/dev/null
|
|
37
|
+
else
|
|
38
|
+
printf '%s' "$INPUT" | sed -n "s/.*\"$1\"[[:space:]]*:[[:space:]]*\"\\([^\"]*\\)\".*/\\1/p" | head -1
|
|
39
|
+
fi
|
|
40
|
+
}
|
|
41
|
+
|
|
42
|
+
TOOL_NAME="$(get_field tool_name '.tool_name')"
|
|
43
|
+
[ "$TOOL_NAME" = "Read" ] || exit 0
|
|
44
|
+
|
|
45
|
+
FILE_PATH="$(get_field file_path '.tool_input.file_path')"
|
|
46
|
+
[ -n "$FILE_PATH" ] || exit 0
|
|
47
|
+
|
|
48
|
+
CWD="$(get_field cwd '.cwd')"
|
|
49
|
+
[ -n "$CWD" ] || CWD="$PWD"
|
|
50
|
+
SESSION_ID="$(get_field session_id '.session_id')"
|
|
51
|
+
|
|
52
|
+
# Resolve relative paths against cwd.
|
|
53
|
+
case "$FILE_PATH" in
|
|
54
|
+
/*) ABS="$FILE_PATH" ;;
|
|
55
|
+
*) ABS="$CWD/$FILE_PATH" ;;
|
|
56
|
+
esac
|
|
57
|
+
|
|
58
|
+
# Must live under .baldart/overlays/.
|
|
59
|
+
case "$ABS" in
|
|
60
|
+
*/.baldart/overlays/*) ;;
|
|
61
|
+
*) exit 0 ;;
|
|
62
|
+
esac
|
|
63
|
+
REL="${ABS##*/.baldart/overlays/}" # everything after the last marker
|
|
64
|
+
|
|
65
|
+
# Classify — mirror src/utils/overlay-merger / overlay.js path logic.
|
|
66
|
+
# Order matters: deeper/excluded patterns before the catch-alls.
|
|
67
|
+
KIND=""
|
|
68
|
+
NAME=""
|
|
69
|
+
case "$REL" in
|
|
70
|
+
*.example.md) exit 0 ;;
|
|
71
|
+
README.md) exit 0 ;;
|
|
72
|
+
*/*/*) exit 0 ;; # 3+ levels — ignore
|
|
73
|
+
agents/*.md) KIND="agent"; NAME="${REL#agents/}"; NAME="${NAME%.md}" ;;
|
|
74
|
+
commands/*.md) KIND="command"; NAME="${REL#commands/}"; NAME="${NAME%.md}" ;;
|
|
75
|
+
*/*) exit 0 ;; # 2 levels, not agents/commands — ignore
|
|
76
|
+
*.md) KIND="skill"; NAME="${REL%.md}" ;;
|
|
77
|
+
*) exit 0 ;;
|
|
78
|
+
esac
|
|
79
|
+
[ -n "$KIND" ] && [ -n "$NAME" ] || exit 0
|
|
80
|
+
|
|
81
|
+
TS="$(date -u +%Y-%m-%dT%H:%M:%SZ)"
|
|
82
|
+
if [ -n "$SESSION_ID" ]; then SID="\"$SESSION_ID\""; else SID="null"; fi
|
|
83
|
+
|
|
84
|
+
mkdir -p "$CWD/.baldart/telemetry" || exit 0
|
|
85
|
+
printf '{"ts":"%s","event":"overlay-read","kind":"%s","name":"%s","session_id":%s}\n' \
|
|
86
|
+
"$TS" "$KIND" "$NAME" "$SID" >> "$CWD/.baldart/telemetry/overlay-loads.jsonl"
|
|
87
|
+
|
|
88
|
+
exit 0
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: baldart-update
|
|
3
|
-
description: "Guidance for updating BALDART framework to the latest version in a consumer repo. Use when the user says /baldart-update, 'aggiorna baldart', 'aggiorna framework', 'update del framework', 'pull baldart latest', or wants to bring the framework up-to-date.
|
|
3
|
+
description: "Guidance for updating BALDART framework to the latest version in a consumer repo. Use when the user says /baldart-update, 'aggiorna baldart', 'aggiorna framework', 'update del framework', 'pull baldart latest', or wants to bring the framework up-to-date. Two modes, auto-detected: INTERACTIVE (a human is present) replicates the native CLI's user-decision points (preview diff, working-tree stash) as chat-side confirmations; AUTONOMOUS (unattended — CI / scheduled routine, signalled by BALDART_AUTONOMOUS / CI / GITHUB_ACTIONS env) runs `npx baldart update --yes` with zero prompts and hard-STOP guardrails (never forces a merge or reset on custom divergence / conflicts). Both let the CLI auto-resolve mechanical noise (hook backfill, subtree-merge commits, schema migration) and assert post-flight that .framework/VERSION actually changed. For non-update drift, defers to `npx baldart` (smart doctor)."
|
|
4
4
|
contamination_scan: skip
|
|
5
5
|
---
|
|
6
6
|
|
|
@@ -12,10 +12,20 @@ pull_request). When any of the numbers below changes against the actual
|
|
|
12
12
|
sources, a warning is surfaced in the workflow run. The decision whether
|
|
13
13
|
this skill needs an update is left to a human reviewer.
|
|
14
14
|
|
|
15
|
-
update_js_prompts:
|
|
16
|
-
hook_registry_entries:
|
|
15
|
+
update_js_prompts: 6
|
|
16
|
+
hook_registry_entries: 4
|
|
17
17
|
config_template_top_level_keys: features,git,identity,lsp,paths,stack,tools,version
|
|
18
|
-
last_verified: v3.
|
|
18
|
+
last_verified: v3.31.0
|
|
19
|
+
-->
|
|
20
|
+
|
|
21
|
+
<!--
|
|
22
|
+
AUTONOMY NOTE (v3.31.0)
|
|
23
|
+
`BALDART_AUTONOMOUS` is a RUNTIME ENV VAR, not a baldart.config.yml key — the
|
|
24
|
+
schema-change propagation rule does NOT apply (no template/configure/update
|
|
25
|
+
detector change). It is injected by the routine-adapters (github-actions: env
|
|
26
|
+
block; cron: `export`; claude-code-cloud: best-effort `env` field in the run
|
|
27
|
+
config) so scheduled routines run unattended. Any orchestrator that wants a
|
|
28
|
+
child agent to update autonomously can set it the same way.
|
|
19
29
|
-->
|
|
20
30
|
|
|
21
31
|
|
|
@@ -42,7 +52,31 @@ CLI would normally ask in the terminal — so the CLI can be driven with
|
|
|
42
52
|
╚══════════════════════════════════════════════════════════════════════╝
|
|
43
53
|
```
|
|
44
54
|
|
|
45
|
-
##
|
|
55
|
+
## Mode detection — FIRST STEP, BLOCKING (v3.31.0+)
|
|
56
|
+
|
|
57
|
+
Before anything else, decide which of two modes you are in. This gates the
|
|
58
|
+
entire rest of the skill.
|
|
59
|
+
|
|
60
|
+
```bash
|
|
61
|
+
# A single positive signal of automation means "no human is in the loop".
|
|
62
|
+
# Absence of ALL signals means "human present" → stay interactive.
|
|
63
|
+
printenv | grep -qE '^(BALDART_AUTONOMOUS|CI|GITHUB_ACTIONS)=' && echo AUTONOMOUS || echo INTERACTIVE
|
|
64
|
+
```
|
|
65
|
+
|
|
66
|
+
- **`AUTONOMOUS`** → no human can answer chat prompts (scheduled routine, CI,
|
|
67
|
+
or an orchestrator that exported the marker). Follow **`## Autonomous mode`**
|
|
68
|
+
below. Do NOT ask any chat questions — there is no one to read them.
|
|
69
|
+
- **`INTERACTIVE`** → a human is present. Follow the interactive flow
|
|
70
|
+
(Execution model + Workflow steps), which replicates every CLI decision
|
|
71
|
+
point as a chat confirmation.
|
|
72
|
+
|
|
73
|
+
> **Why not `[ -t 0 ]`?** When Claude runs a command via the Bash tool, stdin
|
|
74
|
+
> is never a TTY — not even with a human present. The TTY test cannot tell the
|
|
75
|
+
> two situations apart. Only a *positive* automation signal (env var) is
|
|
76
|
+
> reliable, and we default to INTERACTIVE on its absence so a human never
|
|
77
|
+
> silently loses control.
|
|
78
|
+
|
|
79
|
+
## Execution model (INTERACTIVE mode) — IMPORTANT
|
|
46
80
|
|
|
47
81
|
This is an **agent-driven skill with full chat-side replication** of every
|
|
48
82
|
decision point the CLI would pose interactively. The flow is:
|
|
@@ -64,6 +98,60 @@ pure-guidance (the user runs the CLI themselves). Here the user has asked
|
|
|
64
98
|
for an agent that drives the update, so the skill drives it — but it
|
|
65
99
|
replicates every confirmation so nothing is silently auto-accepted.
|
|
66
100
|
|
|
101
|
+
## Autonomous mode (AUTONOMOUS) — unattended, zero prompts (v3.31.0+)
|
|
102
|
+
|
|
103
|
+
When mode detection returned `AUTONOMOUS`, there is no human to confirm
|
|
104
|
+
anything. The skill drives the update end-to-end with NO chat questions, and
|
|
105
|
+
leans entirely on the CLI's own safety gates — which already **refuse** to do
|
|
106
|
+
anything destructive under bare `--yes`.
|
|
107
|
+
|
|
108
|
+
**The contract: autonomy over mechanical work, hard STOP on anything that
|
|
109
|
+
touches the user's custom content.**
|
|
110
|
+
|
|
111
|
+
### Autonomous workflow
|
|
112
|
+
|
|
113
|
+
1. **Pre-flight (read-only)** — same as interactive Step 0: run
|
|
114
|
+
`npx baldart version`, parse the `Installed:` and `Remote:` lines with the
|
|
115
|
+
tolerant matcher. Apply the same **CLI version drift gate** (a too-old
|
|
116
|
+
global CLI crossing the v3.27.0 payload boundary → STOP and report; do not
|
|
117
|
+
launch).
|
|
118
|
+
- Aligned / already up-to-date → report "already current", exit success.
|
|
119
|
+
2. **Launch directly — no preview/stash confirmation:**
|
|
120
|
+
```bash
|
|
121
|
+
npx baldart update --yes
|
|
122
|
+
```
|
|
123
|
+
Use the Bash tool with `timeout: 600000`. `--yes` implies `--auto-stash`,
|
|
124
|
+
so a dirty tree is handled (stashed + re-applied) without asking.
|
|
125
|
+
**Do NOT add `--on-divergence pull` or `--reset`.** Bare `--yes` is the
|
|
126
|
+
safe ceiling: the CLI auto-resolves only mechanical noise and *refuses*
|
|
127
|
+
(exits non-zero) on real custom divergence.
|
|
128
|
+
3. **Interpret the exit code (the whole point of staying safe):**
|
|
129
|
+
- **Exit 0** → update applied (or already aligned). Go to post-flight.
|
|
130
|
+
- **Exit non-zero** → the CLI hit a wall it will not cross unattended:
|
|
131
|
+
custom-overlay-able / real-custom divergence, a subtree-pull conflict, or
|
|
132
|
+
a stale-CLI halt. **STOP. Do NOT retry with `--on-divergence` or
|
|
133
|
+
`--reset`.** Report loudly (see below) so the human who reads the routine
|
|
134
|
+
log knows exactly what needs a hands-on decision. Forcing a merge or a
|
|
135
|
+
reset unattended is precisely the silent destruction this mode forbids.
|
|
136
|
+
4. **Post-flight assert (MANDATORY)** — identical to interactive Step 4:
|
|
137
|
+
`cat .framework/VERSION`, compare to the pre-update version. If the CLI
|
|
138
|
+
exited 0 but VERSION did not change (and stdout is not "Already up to
|
|
139
|
+
date"), **FAIL LOUD** — do not report success.
|
|
140
|
+
5. **Report** — emit a structured, log-friendly summary (no questions):
|
|
141
|
+
version delta, backup tag + rollback command, any agent-layout migration
|
|
142
|
+
(v3.27.0+), and — on a non-zero exit — the verbatim CLI stderr plus the
|
|
143
|
+
exact flag a human would re-run with (`--on-divergence pull` /
|
|
144
|
+
`scaffold-overlays` / `--reset --yes --i-know`). The skill names the
|
|
145
|
+
options; it never picks one autonomously.
|
|
146
|
+
|
|
147
|
+
### What autonomous mode must NEVER do
|
|
148
|
+
|
|
149
|
+
- Ask a chat question (no one is there; a blocked prompt hangs the routine).
|
|
150
|
+
- Pass `--on-divergence pull|scaffold-overlays` or `--reset` to "get past" a
|
|
151
|
+
refusal. A refusal is a *correct* outcome that needs a human.
|
|
152
|
+
- Report success when the post-flight assert fails.
|
|
153
|
+
- Auto-resolve a stash-pop or merge conflict with `--hard` / `--no-verify`.
|
|
154
|
+
|
|
67
155
|
## When to use
|
|
68
156
|
|
|
69
157
|
- The user invokes `/baldart-update`.
|
|
@@ -100,13 +188,20 @@ for the protocol.
|
|
|
100
188
|
|
|
101
189
|
## Hard rules
|
|
102
190
|
|
|
191
|
+
0. **MUST run mode detection FIRST** (the `printenv | grep` probe above) and
|
|
192
|
+
branch accordingly. Defaulting to AUTONOMOUS when no signal is present —
|
|
193
|
+
or to INTERACTIVE when a signal IS present — are both wrong. The default on
|
|
194
|
+
*absence of signal* is INTERACTIVE (human keeps control).
|
|
103
195
|
1. **MUST NOT re-implement the logic of `update.js`.** Same rule as
|
|
104
196
|
`/baldart-push`: the CLI is the source of truth. The skill only
|
|
105
197
|
replicates prompts and parses output.
|
|
106
|
-
2. **MUST replicate in chat
|
|
107
|
-
preview
|
|
108
|
-
|
|
109
|
-
|
|
198
|
+
2. **(INTERACTIVE mode) MUST replicate in chat the 2 remaining decision
|
|
199
|
+
points** before passing `--yes`: preview and working-tree stash (the other
|
|
200
|
+
three — hooks drift, schema config drift, BALDART-managed auto-commit — are
|
|
201
|
+
auto-resolved silently by the CLI since v3.26.0). Silencing a real prompt
|
|
202
|
+
is dishonest — the user thinks they confirmed everything, but they didn't
|
|
203
|
+
see that prompt. **(AUTONOMOUS mode) MUST NOT ask any chat question** — see
|
|
204
|
+
`## Autonomous mode`; a blocked prompt hangs the routine.
|
|
110
205
|
3. **MUST pass `timeout: 600000` (10 minutes)** to the `Bash` call running
|
|
111
206
|
`npx baldart update --yes`. The default 120s is not enough for subtree
|
|
112
207
|
pull + symlink reconcile + post-update wizard + hooks register +
|
|
@@ -150,7 +245,11 @@ post-flight assert:
|
|
|
150
245
|
> preview. The repo URL is exposed by `npx baldart version` in the
|
|
151
246
|
> `Repository:` line.
|
|
152
247
|
|
|
153
|
-
## Workflow
|
|
248
|
+
## Workflow (INTERACTIVE mode)
|
|
249
|
+
|
|
250
|
+
> In AUTONOMOUS mode follow `## Autonomous mode` instead. Step 0 (pre-flight
|
|
251
|
+
> probe + CLI version drift gate) and Step 4 (post-flight assert) are shared
|
|
252
|
+
> by both modes; Steps 1–3 (chat confirmations) are interactive-only.
|
|
154
253
|
|
|
155
254
|
### Step 0 — Pre-flight (read-only)
|
|
156
255
|
|
|
@@ -391,5 +490,9 @@ spine of BALDART maintenance:
|
|
|
391
490
|
- `/baldart-update` — pull from upstream.
|
|
392
491
|
- `/baldart-push` — push to upstream.
|
|
393
492
|
|
|
394
|
-
Both wrap the CLI without re-implementing it;
|
|
395
|
-
|
|
493
|
+
Both wrap the CLI without re-implementing it; the CLI remains the source of
|
|
494
|
+
truth. `/baldart-push` always keeps the user in the loop. `/baldart-update`
|
|
495
|
+
does too **when a human is present** (interactive mode); when run unattended
|
|
496
|
+
(autonomous mode — CI / scheduled routine) it drops the confirmations but
|
|
497
|
+
keeps the CLI's safety gates, stopping hard rather than forcing any
|
|
498
|
+
destructive resolution a human would otherwise own.
|
|
@@ -86,7 +86,7 @@ mode: extend # extend | override
|
|
|
86
86
|
### Precedence rules
|
|
87
87
|
|
|
88
88
|
1. **Default**: overlays *extend* base skills. If overlay and base both apply, both run.
|
|
89
|
-
2. **Explicit override**: inside
|
|
89
|
+
2. **Explicit override**: inside a skill overlay, a section marked `## [OVERRIDE] <topic>` signals the agent to prefer the overlay's version of `<topic>` over the base skill's. **This is a textual convention, not a parsed merge.** Skill overlays use runtime concatenation (§5.bis): nothing in the CLI parses `[OVERRIDE]`/`[APPEND]`/`[PREPEND]` for skills — the merger (`overlay-merger.js`) parses those markers **only for agent/command overlays**, which are compiled at `add`/`update` time. For skills, the marker is an instruction the executing agent honours when it reads the overlay; honouring it is best-effort, not structurally enforced. (Agent/command overrides, by contrast, are materially fused into the generated file — see §5.bis.)
|
|
90
90
|
3. **Stack/identity conflicts**: when an overlay's stack/identity rule contradicts `baldart.config.yml`, the overlay wins (the user wrote it deliberately) — but the skill MUST log a one-line "overlay overrides config" notice in its output.
|
|
91
91
|
|
|
92
92
|
### Version drift handling
|
|
@@ -0,0 +1,77 @@
|
|
|
1
|
+
# Overlay-Load Telemetry
|
|
2
|
+
|
|
3
|
+
This doc describes the telemetry layer that records, **objectively**, whether skill overlays are actually loaded at runtime. The layer is **observation only** — it changes no behaviour. It exists so we can decide, with real data, whether skill overlays need a stronger delivery guarantee than the current runtime-concat model.
|
|
4
|
+
|
|
5
|
+
## Why this exists
|
|
6
|
+
|
|
7
|
+
Skill overlays and agent/command overlays have **opposite delivery guarantees**:
|
|
8
|
+
|
|
9
|
+
- **Agent / command overlays** are *compile-time merged* — the overlay is materially fused into the generated `.claude/agents/<x>.md` at `add`/`update` (`overlay-merger.js`). When Claude loads the subagent, the overlay is already there. The guarantee is structural.
|
|
10
|
+
- **Skill overlays** use *runtime concatenation* — `SKILL.md` stays a symlink, the overlay lives separately in `.baldart/overlays/<skill>.md`, and a line of prose tells the agent "load `.baldart/overlays/<skill>.md` if present". Nothing verifies the agent did. The guarantee is best-effort.
|
|
11
|
+
|
|
12
|
+
So the *most* important overlays for knowledge transfer (skills) have the *weakest* guarantee. The architectural fixes for that (SessionStart awareness injection, PreToolUse-on-Skill injection, or compile-time merge of `SKILL.md`) are all non-trivial — and we have **no evidence** overlays actually fail to load in practice. Before building any of them, we measure. This layer is that measurement.
|
|
13
|
+
|
|
14
|
+
## What it captures (and what it can't)
|
|
15
|
+
|
|
16
|
+
A single PostToolUse hook (`framework/.claude/hooks/overlay-telemetry.sh`, matcher `Read`) appends one JSON line to `.baldart/telemetry/overlay-loads.jsonl` every time a `.baldart/overlays/*.md` file is read:
|
|
17
|
+
|
|
18
|
+
```json
|
|
19
|
+
{"ts":"2026-05-29T08:00:00Z","event":"overlay-read","kind":"skill","name":"bug","session_id":"..."}
|
|
20
|
+
```
|
|
21
|
+
|
|
22
|
+
### Why a hook, not agent self-report
|
|
23
|
+
|
|
24
|
+
The failure mode we measure is *"the agent forgot to load the overlay."* A self-reported `loaded=yes/no` would be corrupted in exactly that case — an agent that forgot to load would also forget to log, or log `yes` without having done it. Only an objective `Read` observation is trustworthy here.
|
|
25
|
+
|
|
26
|
+
### Numerator only — the known limitation
|
|
27
|
+
|
|
28
|
+
We capture the **numerator** (overlay reads), not the **denominator** (skill runs that *should* have loaded an overlay). The denominator is not hook-observable: skills are *prompt-expansion* in Claude Code, not tool calls, so there is no `PreToolUse` on skill invocation and the tool-call payload does not carry the invoking skill's name (anthropics/claude-code [#43630](https://github.com/anthropics/claude-code/issues/43630), [#22655](https://github.com/anthropics/claude-code/issues/22655)).
|
|
29
|
+
|
|
30
|
+
This makes the metric a **conservative proxy that UNDER-counts**:
|
|
31
|
+
|
|
32
|
+
- Overlay content already in the agent's context (no explicit `Read`) is invisible.
|
|
33
|
+
- A `Read` is not proof the content influenced the output — only that it was loaded.
|
|
34
|
+
- Reads performed in ways the main-loop hook doesn't observe are missed.
|
|
35
|
+
|
|
36
|
+
Every bias points the same direction: the observed load-rate looks **worse** than reality. That matters because under-counting biases toward "build the enforcement" — the very thing this telemetry exists to gate. **Treat a low count as a reason to investigate, not as proof of a miss.** (Same honesty convention as `agent-discovery-info.sh`'s #10373 caveat and the "conservative proxy" language in `FIX-APPLICATION-TELEMETRY.md`.)
|
|
37
|
+
|
|
38
|
+
## How to read the data
|
|
39
|
+
|
|
40
|
+
```bash
|
|
41
|
+
node framework/scripts/analyze-overlay-loads.js
|
|
42
|
+
# or, non-default locations:
|
|
43
|
+
node framework/scripts/analyze-overlay-loads.js --log <path.jsonl> --overlays <dir>
|
|
44
|
+
```
|
|
45
|
+
|
|
46
|
+
The aggregator cross-references the read log against the overlays that **exist on disk** and prints per-overlay read counts. The headline signal:
|
|
47
|
+
|
|
48
|
+
| Column / footer | What it means |
|
|
49
|
+
|---|---|
|
|
50
|
+
| `Flag = NEVER-LOADED` | A **skill** overlay exists but was never read. The alarm signal — authored knowledge the agent isn't picking up. (Subject to the under-count caveat above.) |
|
|
51
|
+
| `Reads` low but non-zero | Overlay loads sometimes. Compare against how often the skill ran (you'll have to estimate the denominator from your own usage). |
|
|
52
|
+
| Agent/command rows, zero reads | **Expected, not a finding.** These overlays are compile-time merged into the generated file — they're not meant to be read at runtime. The table does not flag them. |
|
|
53
|
+
| `Reads logged for overlays no longer on disk` | Stale/renamed overlays. Cosmetic. |
|
|
54
|
+
|
|
55
|
+
## When to act on the data
|
|
56
|
+
|
|
57
|
+
After a few weeks of real usage with at least one authored skill overlay:
|
|
58
|
+
|
|
59
|
+
1. **`NEVER-LOADED` skill overlays.** If a skill overlay you actively rely on shows zero reads across many sessions where that skill clearly ran, that's the strongest available evidence that runtime concat is dropping it — and the case for an enforcement mechanism (SessionStart awareness, PreToolUse injection, or `SKILL.md` compile-time merge) becomes data-backed rather than speculative.
|
|
60
|
+
2. **Healthy load pattern.** If your authored skill overlays show steady reads, the runtime-concat model is working in practice and enforcement is not worth its cost. Ship the DX improvements and leave the architecture alone.
|
|
61
|
+
|
|
62
|
+
## Cost
|
|
63
|
+
|
|
64
|
+
Be honest about where the cost lands: the hook fires on **every `Read`** — the most frequent tool — not just on overlay reads. So the per-read cost is what matters, and it is paid hundreds of times per session in the PostToolUse critical path (the model waits for the hook before seeing the tool result).
|
|
65
|
+
|
|
66
|
+
- **Every read** (overwhelmingly non-overlay): one bash fork + a path-prefix match, then exit. This is why the hook is bash, not node — a node cold-start (~50ms) per read would be a real tax; a bash fork (~5ms) keeps the hot path cheap. There is no cheaper event: observing "an overlay was read" *requires* hooking `Read`, so the only lever is making the hook itself cheap.
|
|
67
|
+
- **Only on an actual overlay read** (rare): one `mkdir -p` + one ~120-byte append.
|
|
68
|
+
|
|
69
|
+
The hook is fail-safe (any error → exit 0, never blocks). The aggregator has zero dependencies and runs in well under a second. The log is append-only and rotates only by hand (delete the file to reset the window).
|
|
70
|
+
|
|
71
|
+
## Pointers
|
|
72
|
+
|
|
73
|
+
- Hook: `framework/.claude/hooks/overlay-telemetry.sh`
|
|
74
|
+
- Registration: `src/utils/hooks.js` → `HOOK_REGISTRY` entry `baldart-overlay-telemetry` (auto-registered by `add`, backfilled by `update`, reported by `doctor`)
|
|
75
|
+
- Aggregator: `framework/scripts/analyze-overlay-loads.js`
|
|
76
|
+
- Overlay model: `framework/agents/project-context.md` §5 (skills, runtime concat) and §5.bis (agent/command, compile-time merge)
|
|
77
|
+
- Sibling telemetry layer: `framework/docs/FIX-APPLICATION-TELEMETRY.md`
|
|
@@ -0,0 +1,130 @@
|
|
|
1
|
+
#!/usr/bin/env node
|
|
2
|
+
// Aggregate overlay-load telemetry. Cross-references the objective read log
|
|
3
|
+
// (.baldart/telemetry/overlay-loads.jsonl, written by the overlay-telemetry
|
|
4
|
+
// PostToolUse hook) against the overlays that actually EXIST on disk
|
|
5
|
+
// (.baldart/overlays/), and reports per-overlay load counts.
|
|
6
|
+
//
|
|
7
|
+
// The headline signal is the LAST column: a SKILL overlay that exists but is
|
|
8
|
+
// read ~never is the alarm — runtime concatenation is not structurally
|
|
9
|
+
// enforced, so a never-read skill overlay is knowledge the project authored but
|
|
10
|
+
// the agent isn't picking up. (Agent/command overlays are compile-time merged
|
|
11
|
+
// into the generated file, so they're NOT expected to be read at runtime — a
|
|
12
|
+
// zero count there is normal, not a problem. The table marks them accordingly.)
|
|
13
|
+
//
|
|
14
|
+
// This is a CONSERVATIVE proxy. The hook can only see explicit `Read` calls, so
|
|
15
|
+
// it UNDER-counts: overlay content already in context (no Read) and reads done
|
|
16
|
+
// in ways the main-loop hook doesn't observe are invisible. A low count is a
|
|
17
|
+
// reason to investigate, not proof of a miss. See framework/docs/OVERLAY-TELEMETRY.md.
|
|
18
|
+
//
|
|
19
|
+
// Usage:
|
|
20
|
+
// node framework/scripts/analyze-overlay-loads.js
|
|
21
|
+
// node framework/scripts/analyze-overlay-loads.js --log <path.jsonl> --overlays <dir>
|
|
22
|
+
|
|
23
|
+
'use strict';
|
|
24
|
+
const fs = require('fs');
|
|
25
|
+
const path = require('path');
|
|
26
|
+
|
|
27
|
+
function argVal(flag, fallback) {
|
|
28
|
+
const i = process.argv.indexOf(flag);
|
|
29
|
+
return i !== -1 && process.argv[i + 1] ? process.argv[i + 1] : fallback;
|
|
30
|
+
}
|
|
31
|
+
|
|
32
|
+
const LOG_PATH = argVal('--log', path.join('.baldart', 'telemetry', 'overlay-loads.jsonl'));
|
|
33
|
+
const OVERLAYS_DIR = argVal('--overlays', path.join('.baldart', 'overlays'));
|
|
34
|
+
|
|
35
|
+
// --- read the objective read log -------------------------------------------
|
|
36
|
+
|
|
37
|
+
const reads = new Map(); // "kind/name" → { count, sessions:Set, first, last }
|
|
38
|
+
let logLines = 0;
|
|
39
|
+
let logBad = 0;
|
|
40
|
+
|
|
41
|
+
if (fs.existsSync(LOG_PATH)) {
|
|
42
|
+
const raw = fs.readFileSync(LOG_PATH, 'utf8');
|
|
43
|
+
for (const line of raw.split(/\r?\n/)) {
|
|
44
|
+
if (!line.trim()) continue;
|
|
45
|
+
logLines += 1;
|
|
46
|
+
let row;
|
|
47
|
+
try { row = JSON.parse(line); } catch (_) { logBad += 1; continue; }
|
|
48
|
+
if (!row || row.event !== 'overlay-read' || !row.kind || !row.name) { logBad += 1; continue; }
|
|
49
|
+
const key = `${row.kind}/${row.name}`;
|
|
50
|
+
const e = reads.get(key) || { count: 0, sessions: new Set(), first: null, last: null };
|
|
51
|
+
e.count += 1;
|
|
52
|
+
if (row.session_id) e.sessions.add(row.session_id);
|
|
53
|
+
if (row.ts) {
|
|
54
|
+
if (!e.first || row.ts < e.first) e.first = row.ts;
|
|
55
|
+
if (!e.last || row.ts > e.last) e.last = row.ts;
|
|
56
|
+
}
|
|
57
|
+
reads.set(key, e);
|
|
58
|
+
}
|
|
59
|
+
}
|
|
60
|
+
|
|
61
|
+
// --- enumerate overlays that exist on disk ---------------------------------
|
|
62
|
+
|
|
63
|
+
function listOverlays(dir) {
|
|
64
|
+
const out = []; // { kind, name, key }
|
|
65
|
+
if (!fs.existsSync(dir)) return out;
|
|
66
|
+
const add = (sub, kind) => {
|
|
67
|
+
const d = sub ? path.join(dir, sub) : dir;
|
|
68
|
+
if (!fs.existsSync(d)) return;
|
|
69
|
+
for (const f of fs.readdirSync(d)) {
|
|
70
|
+
if (!f.endsWith('.md') || f.endsWith('.example.md') || f === 'README.md') continue;
|
|
71
|
+
const name = f.replace(/\.md$/, '');
|
|
72
|
+
out.push({ kind, name, key: `${kind}/${name}` });
|
|
73
|
+
}
|
|
74
|
+
};
|
|
75
|
+
add('', 'skill');
|
|
76
|
+
add('agents', 'agent');
|
|
77
|
+
add('commands', 'command');
|
|
78
|
+
return out;
|
|
79
|
+
}
|
|
80
|
+
|
|
81
|
+
const overlays = listOverlays(OVERLAYS_DIR);
|
|
82
|
+
const overlayKeys = new Set(overlays.map((o) => o.key));
|
|
83
|
+
|
|
84
|
+
// --- build rows -------------------------------------------------------------
|
|
85
|
+
|
|
86
|
+
const rows = overlays.map((o) => {
|
|
87
|
+
const r = reads.get(o.key);
|
|
88
|
+
return {
|
|
89
|
+
kind: o.kind,
|
|
90
|
+
name: o.name,
|
|
91
|
+
reads: r ? r.count : 0,
|
|
92
|
+
sessions: r ? r.sessions.size : 0,
|
|
93
|
+
last: r && r.last ? r.last.slice(0, 10) : '-',
|
|
94
|
+
// Only skills are expected to be read at runtime; agent/command overlays are
|
|
95
|
+
// fused at build time, so a zero there is expected, not a finding.
|
|
96
|
+
flag: (o.kind === 'skill' && (!r || r.count === 0)) ? 'NEVER-LOADED' : '',
|
|
97
|
+
};
|
|
98
|
+
});
|
|
99
|
+
rows.sort((a, b) => a.reads - b.reads || a.kind.localeCompare(b.kind) || a.name.localeCompare(b.name));
|
|
100
|
+
|
|
101
|
+
// Reads of overlays that no longer exist on disk (stale/renamed) — surfaced separately.
|
|
102
|
+
const orphanReads = [...reads.keys()].filter((k) => !overlayKeys.has(k));
|
|
103
|
+
|
|
104
|
+
// --- render -----------------------------------------------------------------
|
|
105
|
+
|
|
106
|
+
const headers = ['Kind', 'Name', 'Reads', 'Sessions', 'Last-read', 'Flag'];
|
|
107
|
+
const cols = ['kind', 'name', 'reads', 'sessions', 'last', 'flag'];
|
|
108
|
+
const widths = headers.map((h, i) => Math.max(h.length, ...rows.map((r) => String(r[cols[i]]).length), 1));
|
|
109
|
+
const fmt = (cells) => cells.map((c, i) => String(c).padEnd(widths[i])).join(' ');
|
|
110
|
+
|
|
111
|
+
console.log(fmt(headers));
|
|
112
|
+
console.log(widths.map((w) => '-'.repeat(w)).join(' '));
|
|
113
|
+
for (const r of rows) console.log(fmt(cols.map((c) => r[c])));
|
|
114
|
+
|
|
115
|
+
console.log('');
|
|
116
|
+
console.log(`Overlays on disk: ${overlays.length} | Log rows: ${logLines}${logBad ? ` (${logBad} skipped)` : ''} | Log: ${fs.existsSync(LOG_PATH) ? LOG_PATH : 'NOT FOUND (no telemetry yet)'}`);
|
|
117
|
+
|
|
118
|
+
const neverLoaded = rows.filter((r) => r.flag === 'NEVER-LOADED');
|
|
119
|
+
if (neverLoaded.length) {
|
|
120
|
+
console.log('');
|
|
121
|
+
console.log('⚠ Skill overlays that EXIST but were never read (runtime concat may not be loading them):');
|
|
122
|
+
for (const r of neverLoaded) console.log(` ${r.kind}/${r.name}`);
|
|
123
|
+
console.log(' Conservative proxy — see framework/docs/OVERLAY-TELEMETRY.md before acting (in-context reads are invisible to the hook).');
|
|
124
|
+
}
|
|
125
|
+
|
|
126
|
+
if (orphanReads.length) {
|
|
127
|
+
console.log('');
|
|
128
|
+
console.log('ℹ Reads logged for overlays no longer on disk (renamed/removed):');
|
|
129
|
+
for (const k of orphanReads) console.log(` ${k}`);
|
|
130
|
+
}
|
package/package.json
CHANGED
package/src/commands/overlay.js
CHANGED
|
@@ -2,7 +2,7 @@ const fs = require('fs');
|
|
|
2
2
|
const path = require('path');
|
|
3
3
|
const yaml = require('js-yaml');
|
|
4
4
|
const UI = require('../utils/ui');
|
|
5
|
-
const { mergeOverlay, computeBaseFileSha } = require('../utils/overlay-merger');
|
|
5
|
+
const { mergeOverlay, computeBaseFileSha, parseMarkdown } = require('../utils/overlay-merger');
|
|
6
6
|
|
|
7
7
|
const FRAMEWORK_DIR = '.framework';
|
|
8
8
|
const FRAMEWORK_PAYLOAD = path.join(FRAMEWORK_DIR, 'framework');
|
|
@@ -72,18 +72,68 @@ function parseFrontmatter(content) {
|
|
|
72
72
|
}
|
|
73
73
|
}
|
|
74
74
|
|
|
75
|
+
// Skill overlays use the runtime-concat model — the executing agent reads the
|
|
76
|
+
// base SKILL.md and the overlay and applies the overlay's guidance itself.
|
|
77
|
+
// Unlike agent/command overlays, NOTHING parses the [OVERRIDE]/[APPEND]/[PREPEND]
|
|
78
|
+
// markers here (see framework/agents/project-context.md § 5). This helper gives
|
|
79
|
+
// the skill-overlay author the same "what does the merge look like?" feedback the
|
|
80
|
+
// marker model already gives for agents/commands: it lists the overlay's sections,
|
|
81
|
+
// flags markers whose target heading no longer exists in the base (the agent has
|
|
82
|
+
// nothing to anchor the override to), and reports the concatenated size. Pure — no IO.
|
|
83
|
+
function analyzeSkillConcat(baseContent, overlayContent) {
|
|
84
|
+
const base = parseMarkdown(baseContent);
|
|
85
|
+
const overlay = parseMarkdown(overlayContent);
|
|
86
|
+
const baseHeadings = new Set(base.sections.map((s) => s.heading).filter(Boolean));
|
|
87
|
+
|
|
88
|
+
const markerSections = [];
|
|
89
|
+
let plainSections = 0;
|
|
90
|
+
for (const s of overlay.sections) {
|
|
91
|
+
if (s.marker) {
|
|
92
|
+
markerSections.push({ marker: s.marker, heading: s.heading, orphan: !baseHeadings.has(s.heading) });
|
|
93
|
+
} else if (s.heading) {
|
|
94
|
+
plainSections += 1;
|
|
95
|
+
}
|
|
96
|
+
}
|
|
97
|
+
return {
|
|
98
|
+
baseLines: baseContent.split('\n').length,
|
|
99
|
+
overlayLines: overlayContent.split('\n').length,
|
|
100
|
+
baseSectionCount: base.sections.length,
|
|
101
|
+
markerSections,
|
|
102
|
+
plainSections,
|
|
103
|
+
orphans: markerSections.filter((m) => m.orphan),
|
|
104
|
+
};
|
|
105
|
+
}
|
|
106
|
+
|
|
75
107
|
function scaffoldBody(kind, name) {
|
|
76
108
|
if (kind === 'skill') {
|
|
77
109
|
return [
|
|
78
110
|
'',
|
|
79
|
-
`>
|
|
80
|
-
`> agent
|
|
81
|
-
`>
|
|
82
|
-
`>
|
|
111
|
+
`> HOW SKILL OVERLAYS WORK`,
|
|
112
|
+
`> When the "${name}" skill runs, the agent reads this file and applies it on top`,
|
|
113
|
+
`> of the base SKILL.md (runtime concatenation). NOTHING in the CLI parses the`,
|
|
114
|
+
`> [OVERRIDE]/[APPEND]/[PREPEND] markers below — for skills they are textual`,
|
|
115
|
+
`> conventions the agent honours (best-effort), not a compile-time merge (that`,
|
|
116
|
+
`> model is agent/command-only). Keep this file the single home for everything`,
|
|
117
|
+
`> project-specific the base skill deliberately leaves generic.`,
|
|
118
|
+
`>`,
|
|
119
|
+
`> Frontmatter above: bump base_skill_version when you re-review against a newer`,
|
|
120
|
+
`> framework; \`mode: override\` tells the agent to prefer this file wholesale.`,
|
|
121
|
+
`> Fill in the sections that apply and delete the rest, then run`,
|
|
122
|
+
`> \`npx baldart overlay validate .baldart/overlays/${name}.md\` to preview the merge.`,
|
|
83
123
|
'',
|
|
84
124
|
`## [APPEND] Project-specific guidance`,
|
|
85
125
|
'',
|
|
86
|
-
`<!--
|
|
126
|
+
`<!-- The default home for project facts the base "${name}" skill cannot know: -->`,
|
|
127
|
+
`<!-- - canonical paths / module entry points (cite REAL files in this repo) -->`,
|
|
128
|
+
`<!-- - domain vocabulary, naming conventions, terminology -->`,
|
|
129
|
+
`<!-- - project-specific BLOCKING rules ("never X", "always read Y first") -->`,
|
|
130
|
+
`<!-- - mandated stack / forbidden libraries beyond baldart.config.yml stack.* -->`,
|
|
131
|
+
'',
|
|
132
|
+
`## [OVERRIDE] <exact base H2 to replace>`,
|
|
133
|
+
'',
|
|
134
|
+
`<!-- OPTIONAL. Copy an H2 heading VERBATIM from the base SKILL.md to tell the -->`,
|
|
135
|
+
`<!-- agent to prefer this version over the base. \`overlay validate\` warns if -->`,
|
|
136
|
+
`<!-- the heading matches no base section. Delete this block if you only append. -->`,
|
|
87
137
|
'',
|
|
88
138
|
].join('\n');
|
|
89
139
|
}
|
|
@@ -261,10 +311,33 @@ async function validate(overlayPath) {
|
|
|
261
311
|
}
|
|
262
312
|
|
|
263
313
|
if (t.kind === 'skill') {
|
|
264
|
-
|
|
314
|
+
const baseContent = fs.readFileSync(baseAbs, 'utf8');
|
|
315
|
+
const a = analyzeSkillConcat(baseContent, overlayContent);
|
|
316
|
+
UI.success(`Overlay valid (skill, runtime-concat model).`);
|
|
265
317
|
UI.info(`Targets v${fm.base_skill_version || 'unknown'}, mode=${fm.mode || 'extend'}.`);
|
|
318
|
+
UI.newline();
|
|
319
|
+
// Dry-run preview. For agent/command we render the actual merged output;
|
|
320
|
+
// skills can't be merged at the CLI (the executing agent reads base + overlay
|
|
321
|
+
// and applies it), so we show what the agent will see instead of a fake merge.
|
|
322
|
+
UI.info('Merge preview (runtime concat: base SKILL.md + overlay):');
|
|
323
|
+
UI.list([
|
|
324
|
+
`base sections: ${a.baseSectionCount} · overlay sections: ${a.markerSections.length + a.plainSections} (${a.markerSections.length} marked, ${a.plainSections} new)`,
|
|
325
|
+
`concatenated size: ~${a.baseLines + a.overlayLines} lines (${a.baseLines} base + ${a.overlayLines} overlay)`,
|
|
326
|
+
]);
|
|
327
|
+
if (a.markerSections.length) {
|
|
328
|
+
UI.newline();
|
|
329
|
+
UI.info('Section markers (textual conventions — honoured by the agent at runtime, NOT parsed by the CLI):');
|
|
330
|
+
a.markerSections.forEach((m) => {
|
|
331
|
+
if (m.orphan) {
|
|
332
|
+
UI.warning(` [${m.marker}] ${m.heading} — no matching H2 in base SKILL.md; the agent has no base section to anchor this to. Fix the heading or drop the marker.`);
|
|
333
|
+
} else {
|
|
334
|
+
UI.info(` [${m.marker}] ${m.heading} — matches a base section`);
|
|
335
|
+
}
|
|
336
|
+
});
|
|
337
|
+
}
|
|
338
|
+
UI.newline();
|
|
266
339
|
if (fm.base_file_sha) {
|
|
267
|
-
const currentSha = computeBaseFileSha(
|
|
340
|
+
const currentSha = computeBaseFileSha(baseContent);
|
|
268
341
|
if (currentSha !== fm.base_file_sha) {
|
|
269
342
|
UI.warning(`Base file changed since overlay was authored (sha ${fm.base_file_sha} → ${currentSha}). Review for drift.`);
|
|
270
343
|
} else {
|
|
@@ -384,6 +457,15 @@ async function drift(target, options = {}) {
|
|
|
384
457
|
|
|
385
458
|
if (shaDrift === true) {
|
|
386
459
|
UI.warning(`${t.rel}: base file changed since overlay was authored (sha ${fm.base_file_sha} → ${baseSha}).`);
|
|
460
|
+
// For skills, surface WHICH overlay markers the base change orphaned — the
|
|
461
|
+
// same actionable feedback agent/command overlays get from the merger. A
|
|
462
|
+
// marker whose heading vanished upstream needs re-anchoring by hand.
|
|
463
|
+
if (t.kind === 'skill') {
|
|
464
|
+
const a = analyzeSkillConcat(fs.readFileSync(baseAbs, 'utf8'), content);
|
|
465
|
+
a.orphans.forEach((m) =>
|
|
466
|
+
UI.warning(` └─ overlay marker [${m.marker}] "${m.heading}" no longer matches any base H2 — likely renamed/removed upstream. Re-anchor it.`)
|
|
467
|
+
);
|
|
468
|
+
}
|
|
387
469
|
drifted++;
|
|
388
470
|
} else if (shaDrift === false) {
|
|
389
471
|
UI.success(`${t.rel}: clean (sha matches base v${fwVersion}).`);
|
|
@@ -404,4 +486,4 @@ async function drift(target, options = {}) {
|
|
|
404
486
|
UI.info(`Summary: ${clean} clean, ${drifted} drifted, ${unknown} unknown — of ${targets.length} overlay(s).`);
|
|
405
487
|
}
|
|
406
488
|
|
|
407
|
-
module.exports = { scaffold, scaffoldFile, validate, drift, parseTarget };
|
|
489
|
+
module.exports = { scaffold, scaffoldFile, validate, drift, parseTarget, analyzeSkillConcat };
|
|
@@ -0,0 +1,89 @@
|
|
|
1
|
+
'use strict';
|
|
2
|
+
const { test } = require('node:test');
|
|
3
|
+
const assert = require('node:assert');
|
|
4
|
+
const { analyzeSkillConcat } = require('../../commands/overlay');
|
|
5
|
+
|
|
6
|
+
// analyzeSkillConcat powers the skill-overlay dry-run in `overlay validate` and
|
|
7
|
+
// the orphaned-marker warning in `overlay drift`. Skills use runtime concat, so
|
|
8
|
+
// the markers are NOT parsed — this helper only DESCRIBES what the agent will
|
|
9
|
+
// see and flags markers whose heading no longer anchors to a base section.
|
|
10
|
+
|
|
11
|
+
const BASE = `---
|
|
12
|
+
name: bug
|
|
13
|
+
---
|
|
14
|
+
|
|
15
|
+
## Phase 1
|
|
16
|
+
reproduce
|
|
17
|
+
|
|
18
|
+
## Phase 2
|
|
19
|
+
fix
|
|
20
|
+
`;
|
|
21
|
+
|
|
22
|
+
test('marker matching a base heading is not an orphan', () => {
|
|
23
|
+
const overlay = `---
|
|
24
|
+
base_skill: bug
|
|
25
|
+
---
|
|
26
|
+
|
|
27
|
+
## [APPEND] Phase 1
|
|
28
|
+
project entry points
|
|
29
|
+
`;
|
|
30
|
+
const a = analyzeSkillConcat(BASE, overlay);
|
|
31
|
+
assert.strictEqual(a.markerSections.length, 1);
|
|
32
|
+
assert.strictEqual(a.markerSections[0].orphan, false);
|
|
33
|
+
assert.strictEqual(a.orphans.length, 0);
|
|
34
|
+
});
|
|
35
|
+
|
|
36
|
+
test('marker with no matching base heading is flagged orphan', () => {
|
|
37
|
+
const overlay = `---
|
|
38
|
+
base_skill: bug
|
|
39
|
+
---
|
|
40
|
+
|
|
41
|
+
## [OVERRIDE] Phase 9
|
|
42
|
+
heading absent from base
|
|
43
|
+
`;
|
|
44
|
+
const a = analyzeSkillConcat(BASE, overlay);
|
|
45
|
+
assert.strictEqual(a.orphans.length, 1);
|
|
46
|
+
assert.strictEqual(a.orphans[0].heading, 'Phase 9');
|
|
47
|
+
assert.strictEqual(a.orphans[0].marker, 'OVERRIDE');
|
|
48
|
+
});
|
|
49
|
+
|
|
50
|
+
test('unmarked overlay sections count as new (plain), not markers', () => {
|
|
51
|
+
const overlay = `---
|
|
52
|
+
base_skill: bug
|
|
53
|
+
---
|
|
54
|
+
|
|
55
|
+
## Brand vocabulary
|
|
56
|
+
project terms
|
|
57
|
+
|
|
58
|
+
## [APPEND] Phase 2
|
|
59
|
+
extra
|
|
60
|
+
`;
|
|
61
|
+
const a = analyzeSkillConcat(BASE, overlay);
|
|
62
|
+
assert.strictEqual(a.plainSections, 1);
|
|
63
|
+
assert.strictEqual(a.markerSections.length, 1);
|
|
64
|
+
assert.strictEqual(a.baseSectionCount, 2);
|
|
65
|
+
});
|
|
66
|
+
|
|
67
|
+
test('reports line counts for both base and overlay', () => {
|
|
68
|
+
const overlay = `---
|
|
69
|
+
base_skill: bug
|
|
70
|
+
---
|
|
71
|
+
|
|
72
|
+
## [APPEND] Phase 1
|
|
73
|
+
x
|
|
74
|
+
`;
|
|
75
|
+
const a = analyzeSkillConcat(BASE, overlay);
|
|
76
|
+
assert.ok(a.baseLines > 0);
|
|
77
|
+
assert.ok(a.overlayLines > 0);
|
|
78
|
+
});
|
|
79
|
+
|
|
80
|
+
test('empty overlay body yields no sections and no orphans', () => {
|
|
81
|
+
const overlay = `---
|
|
82
|
+
base_skill: bug
|
|
83
|
+
---
|
|
84
|
+
`;
|
|
85
|
+
const a = analyzeSkillConcat(BASE, overlay);
|
|
86
|
+
assert.strictEqual(a.markerSections.length, 0);
|
|
87
|
+
assert.strictEqual(a.plainSections, 0);
|
|
88
|
+
assert.strictEqual(a.orphans.length, 0);
|
|
89
|
+
});
|
package/src/utils/hooks.js
CHANGED
|
@@ -87,6 +87,23 @@ const HOOK_REGISTRY = [
|
|
|
87
87
|
command: 'bash .framework/framework/.claude/hooks/agent-discovery-info.sh',
|
|
88
88
|
order: 100,
|
|
89
89
|
},
|
|
90
|
+
{
|
|
91
|
+
// Observation-only telemetry: records every Read of a
|
|
92
|
+
// `.baldart/overlays/*.md` file to `.baldart/telemetry/overlay-loads.jsonl`.
|
|
93
|
+
// Measures, objectively, whether skill overlays are actually loaded at
|
|
94
|
+
// runtime (skill overlays use runtime-concat, which is NOT structurally
|
|
95
|
+
// enforced — unlike agent/command overlays, which are compile-time merged).
|
|
96
|
+
// Numerator-only by design: skill invocation is prompt-expansion, not a
|
|
97
|
+
// tool call, so the denominator is not hook-observable (claude-code #43630,
|
|
98
|
+
// #22655). bash, not node — it fires on EVERY Read (hottest tool) in the
|
|
99
|
+
// critical path, so a per-read node cold-start would tax every file read.
|
|
100
|
+
// Fail-safe — never blocks. See framework/docs/OVERLAY-TELEMETRY.md.
|
|
101
|
+
id: 'baldart-overlay-telemetry',
|
|
102
|
+
event: 'PostToolUse',
|
|
103
|
+
matcher: 'Read',
|
|
104
|
+
command: 'bash .framework/framework/.claude/hooks/overlay-telemetry.sh',
|
|
105
|
+
order: 100,
|
|
106
|
+
},
|
|
90
107
|
// Future hooks (T1.3, T1.4, T2.3) will be appended here, e.g.:
|
|
91
108
|
// {
|
|
92
109
|
// id: 'baldart-capture-detect',
|
|
@@ -43,6 +43,13 @@ class ClaudeCodeCloudAdapter {
|
|
|
43
43
|
prompt: spec.prompt,
|
|
44
44
|
output: spec.output,
|
|
45
45
|
backend: this.name,
|
|
46
|
+
// Marks this as an unattended (no-human) run. Autonomy-aware skills
|
|
47
|
+
// (e.g. /baldart-update) auto-detect this and drop their chat
|
|
48
|
+
// confirmations, with safe guardrails (STOP on custom divergence /
|
|
49
|
+
// conflicts, never forces a destructive merge). Best-effort for this
|
|
50
|
+
// backend: RemoteTrigger must surface `env` into the run environment for
|
|
51
|
+
// the runtime probe to see it; github-actions / cron inject it directly.
|
|
52
|
+
env: { BALDART_AUTONOMOUS: '1' },
|
|
46
53
|
installed_at: new Date().toISOString()
|
|
47
54
|
};
|
|
48
55
|
fs.writeFileSync(this.configPath(spec.name), JSON.stringify(config, null, 2) + '\n');
|
|
@@ -101,6 +101,13 @@ if ! command -v claude >/dev/null 2>&1; then
|
|
|
101
101
|
fi
|
|
102
102
|
: "\${ANTHROPIC_API_KEY:?ANTHROPIC_API_KEY must be set (export it from the cron env or your shell profile)}"
|
|
103
103
|
|
|
104
|
+
# Marks this as an unattended (no-human) run. The /baldart-update skill (and
|
|
105
|
+
# any other autonomy-aware skill) auto-detects this and switches to
|
|
106
|
+
# non-interactive mode — no chat confirmations, safe guardrails (STOP on
|
|
107
|
+
# custom divergence / conflicts, never forces a destructive merge). Exported
|
|
108
|
+
# so the \`claude\` process and every child Bash it spawns inherit it.
|
|
109
|
+
export BALDART_AUTONOMOUS=1
|
|
110
|
+
|
|
104
111
|
DATE=$(date -u +%Y%m%d)
|
|
105
112
|
OUT_PATH="${outputPath}"
|
|
106
113
|
OUT_PATH="\${OUT_PATH//\\{\\{YYYYMMDD\\}\\}/$DATE}"
|
|
@@ -105,6 +105,13 @@ jobs:
|
|
|
105
105
|
- name: Run ${spec.name}
|
|
106
106
|
env:
|
|
107
107
|
ANTHROPIC_API_KEY: \${{ secrets.ANTHROPIC_API_KEY }}
|
|
108
|
+
# Marks this as an unattended (no-human) run. The /baldart-update
|
|
109
|
+
# skill (and any other autonomy-aware skill) auto-detects this and
|
|
110
|
+
# switches to non-interactive mode — no chat confirmations, safe
|
|
111
|
+
# guardrails (STOP on custom divergence / conflicts, never forces a
|
|
112
|
+
# destructive merge). Inherited by the \`claude\` process and every
|
|
113
|
+
# child Bash it spawns.
|
|
114
|
+
BALDART_AUTONOMOUS: '1'
|
|
108
115
|
run: |
|
|
109
116
|
DATE=$(date -u +%Y%m%d)
|
|
110
117
|
OUT_PATH="${outputPath}"
|