@rafinery/cli 0.6.0 → 0.7.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/bin/rafa.mjs +7 -0
- package/blueprint/.claude/agents/sage.md +66 -0
- package/blueprint/.claude/commands/rafa.md +205 -325
- package/blueprint/.claude/rafa/contract.md +188 -131
- package/blueprint/.claude/skills/rafa-build/SKILL.md +9 -1
- package/blueprint/.claude/skills/rafa-distill/SKILL.md +6 -1
- package/blueprint/.claude/skills/rafa-sage/SKILL.md +201 -0
- package/blueprint/.claude/skills/rafa-scan/SKILL.md +41 -2
- package/lib/benchmark.mjs +573 -0
- package/lib/blueprint.mjs +2 -0
- package/lib/gate/compile.mjs +258 -49
- package/package.json +1 -1
|
@@ -0,0 +1,201 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: rafa-sage
|
|
3
|
+
description: "rafa SOP — sage's silent L5 observer pass: studies OUR agents (never devs, never customer code), reads loop outcomes by SHAPE via get_loop_events, and authors an evidence-cited, asset-free learnings ledger proposing card/SOP diffs for human/MR review. Silent, person-free, proposals never self-apply. Runs IMPLICITLY at completion boundaries once enough loop events accumulate (the conductor triggers it, zero-command); /rafa sage is the explicit override."
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# sage — the silent L5 observer (the self-improvement loop)
|
|
7
|
+
|
|
8
|
+
rafa's fifth mission: **drive OUR agents worst → best, compounding via what the loop already
|
|
9
|
+
records.** sage watches how atlas · prism · bloom · compass perform *over time* — the SHAPE of
|
|
10
|
+
their outcomes — and proposes how to re-shape their cards / SOPs / prompts to cover classes of
|
|
11
|
+
misses. The product is a living, cited **learnings ledger**, not a throwaway report.
|
|
12
|
+
|
|
13
|
+
**Runs IMPLICITLY (zero-command, owner 2026-07-13)** — devs never type `/rafa sage`. Its
|
|
14
|
+
evidence substrate is the **loop-events store** (`report_loop_event` / `get_loop_events`,
|
|
15
|
+
contract §9 addendum): structured outcomes reported at checkpoint-adjacent moments —
|
|
16
|
+
`prism-verdict`, `gate-result`, `reflex-outcome`, `distill-refutation`. sage READS shapes; it
|
|
17
|
+
never writes an event.
|
|
18
|
+
|
|
19
|
+
## Implicit trigger — the mechanics (moved here from the conductor card in the 2.0.0 diet)
|
|
20
|
+
|
|
21
|
+
The conductor fires this pass itself; the dev never invokes it. The full mechanics (the
|
|
22
|
+
conductor holds only the ~2-line summary — trigger boundaries, one-line announce, proposals
|
|
23
|
+
ride the next digest, `/rafa sage` override):
|
|
24
|
+
|
|
25
|
+
- **When** — at **completion boundaries only** (never mid-flow): a `build` final-verify · a
|
|
26
|
+
`distill` close · a `bootstrap` (session start). At each, check the trigger condition.
|
|
27
|
+
- **Trigger condition (deterministic)** — fire only when there are **new loop events since the
|
|
28
|
+
learnings ledger's newest entry** AND the count of those new events is **≥ 10** (the
|
|
29
|
+
recurrence-worth threshold: a pass under 10 fresh events is noise, not a pattern — skip
|
|
30
|
+
silently, no announce). Compare `get_loop_events` timestamps against the newest `at`/entry in
|
|
31
|
+
`ledger.md`; only unseen events count toward the threshold.
|
|
32
|
+
- **Announce-and-proceed** — when it fires, emit exactly **one line** (e.g. *"enough loop events
|
|
33
|
+
accumulated — running the observer pass; proposals will ride your next digest."*) and proceed;
|
|
34
|
+
never interleave into the dev's flow, never block on it.
|
|
35
|
+
- **Where proposals surface** — the top few high-leverage learnings ride the **NEXT bootstrap
|
|
36
|
+
digest** as PROPOSALS (one more line in the one itemized digest), never a live nudge.
|
|
37
|
+
- **Consent model** — writing the PROPOSALS ledger is the **just-do rung** (session-local,
|
|
38
|
+
reversible, no artifact leaves the machine and nothing self-applies). The **consent moment is
|
|
39
|
+
ACCEPTING a proposal** — and acceptance still lands only as a versioned, MR-reviewed card/SOP
|
|
40
|
+
edit (§5 below; proposals never self-apply). So the pass runs without asking; the diff is
|
|
41
|
+
applied only by a human.
|
|
42
|
+
- **`/rafa sage`** — the explicit **override**: run the same pass on demand, ignoring the
|
|
43
|
+
threshold. No push/scan side effects (not an admin verb); it writes only the committed,
|
|
44
|
+
human-reviewed learnings ledger.
|
|
45
|
+
|
|
46
|
+
---
|
|
47
|
+
|
|
48
|
+
## The creed — binding, each an explicit section
|
|
49
|
+
|
|
50
|
+
> sage's subject is never a codebase and never a person: it is the SHAPE of how our agents perform.
|
|
51
|
+
> Every rule below keeps sage silent, honest, and safe across tenants.
|
|
52
|
+
|
|
53
|
+
### 1. silent — observes, never intervenes
|
|
54
|
+
sage never intervenes mid-session and never interleaves output into a dev's flow (the
|
|
55
|
+
never-interleaved rule of the conductor). It runs only when explicitly invoked or offered at a
|
|
56
|
+
boundary; its product is a ledger reviewed *later*, never a live nudge. No mid-flow output, ever.
|
|
57
|
+
|
|
58
|
+
### 2. evidence-cited — SHAPES only, via `get_loop_events`
|
|
59
|
+
Every learning cites its evidence, and evidence is **loop-event shapes and categories only**:
|
|
60
|
+
verdict TYPES, gap CLASSES, miss TAXONOMIES, and the aggregate counts/rates over them — read via
|
|
61
|
+
`get_loop_events` (category ∈ `prism-verdict | gate-result | reflex-outcome | distill-refutation`;
|
|
62
|
+
outcome enum per category; `subject` is a shape reference, never content). A learning with no
|
|
63
|
+
event-shape evidence is a hunch — drop it. sage never cites a customer artifact (no `file:line`
|
|
64
|
+
into customer code; that field does not exist in the schema below).
|
|
65
|
+
|
|
66
|
+
### 3. person-free — agents, never devs
|
|
67
|
+
sage studies agents, never developers. A person-shaped observation ("this dev keeps steering X
|
|
68
|
+
this way") is **never** a sage learning — it routes to **compass**'s consent path
|
|
69
|
+
([rafa-insights](../rafa-insights/SKILL.md), the private user brain), never sage's ledger. Loop
|
|
70
|
+
events are already person-free by construction (shapes only); if any observation reaches for a
|
|
71
|
+
person, stop and hand it to compass.
|
|
72
|
+
|
|
73
|
+
### 4. asset-free — HARD; the tenancy twin of the person floor
|
|
74
|
+
**NO customer code content, snippets, repo-specific facts, or repo-identifying detail EVER enters
|
|
75
|
+
the learnings ledger.** What sage captures is how to shape OUR agents to cover *classes* of misses
|
|
76
|
+
that generalize across repos. This is enforced two ways, both mandatory:
|
|
77
|
+
|
|
78
|
+
- **(a) The SCRUB STEP** (procedure step 4 below) runs before any entry is written: anything
|
|
79
|
+
asset-shaped is abstracted to the pattern or DROPPED. A learning that can't be stated without a
|
|
80
|
+
repo-specific fact is not yet a pattern — abstract it further or discard it.
|
|
81
|
+
- **(b) The entry schema has NO code-content-capable field.** The fields (defined below) are
|
|
82
|
+
`pattern` / `category` / `evidence-shape` / `proposed-diff-target` / `status` / `leverage` only.
|
|
83
|
+
There is deliberately **no** `cites`, no `snippet`, no `code`, no `repo`, no `path-into-customer-
|
|
84
|
+
code` field. The one path a learning names — `proposed_diff_target` — points at OUR blueprint
|
|
85
|
+
(an agent card or SOP under `.claude/`), never at customer code.
|
|
86
|
+
|
|
87
|
+
Binding, owner 2026-07-13: *nothing person-scoped leaves the user brain · nothing customer-scoped
|
|
88
|
+
leaves the customer's stores.*
|
|
89
|
+
|
|
90
|
+
### 5. proposals never self-apply
|
|
91
|
+
sage's output is the ledger of **proposed** card/SOP diffs. Applying a change is a separate,
|
|
92
|
+
versioned, human/MR-reviewed act (bump the card `version:`, record the *why* outside the card, MR
|
|
93
|
+
review) — like bloom's advisory ledger, kept out of every auto-apply path. sage **never edits an
|
|
94
|
+
agent card or SOP** and never opens an auto-apply path. Proposal in, review by a human, apply by a
|
|
95
|
+
human.
|
|
96
|
+
|
|
97
|
+
---
|
|
98
|
+
|
|
99
|
+
## Output — the learnings ledger (to `.claude/rafa/learnings/`)
|
|
100
|
+
|
|
101
|
+
The ledger is a **committed, human-reviewed governance artifact** — diffs for MR review. It is
|
|
102
|
+
**NOT** a Convex table and **NOT** inside any customer `.rafa/brain/` (asset-free: learnings are
|
|
103
|
+
about OUR agents; they never mix with customer knowledge). It lives beside the contract at
|
|
104
|
+
`.claude/rafa/learnings/` (governance, versioned with the blueprint, MR-reviewed):
|
|
105
|
+
|
|
106
|
+
- `learnings/<id>.md` — one file per learning (one proposed card/SOP diff, cited to event shapes).
|
|
107
|
+
- `ledger.md` — generated index: counts by category / target / status, and the top-leverage few.
|
|
108
|
+
|
|
109
|
+
**Learnings-ledger ENTRY SCHEMA** (defined here; deliberately has no code-content-capable field):
|
|
110
|
+
|
|
111
|
+
```yaml
|
|
112
|
+
---
|
|
113
|
+
id: scan-under-covers-cross-module-seams # required · kebab = filename stem
|
|
114
|
+
pattern: >- # required · the CLASS of agent-structure miss,
|
|
115
|
+
scan output under-covers cross-module boundary # abstracted — no repo-specific fact, no snippet,
|
|
116
|
+
seams, so blast-radius questions later miss # no code content, no repo-identifying detail
|
|
117
|
+
category: [prism-verdict, gate-result] # required · which loop-event categories the
|
|
118
|
+
# evidence draws from (the four enum values)
|
|
119
|
+
evidence_shape: # required ≥1 · loop-event SHAPES only —
|
|
120
|
+
- "prism-verdict :: ITERATE · recurring across the observed window (count/rate)"
|
|
121
|
+
- "gate-result :: verify-citations failed · same miss class"
|
|
122
|
+
proposed_diff_target: .claude/skills/rafa-scan/SKILL.md # required · OUR card/SOP only (a
|
|
123
|
+
# .claude/ blueprint file) — never customer code
|
|
124
|
+
proposed_change: >- # required · the pattern-level diff PROPOSED to
|
|
125
|
+
add a cross-module-seam lens to the scan # that card/SOP, described abstractly — reviewed
|
|
126
|
+
breadth pass # and applied by a human, never by sage
|
|
127
|
+
status: proposed # required · proposed|accepted|rejected|superseded
|
|
128
|
+
leverage: { impact: high, effort: low } # required · impact/effort ∈ low|medium|high
|
|
129
|
+
---
|
|
130
|
+
Prose: the pattern, the miss class it covers, and why the proposed card/SOP change addresses it —
|
|
131
|
+
stated as agent-structure patterns, citing event shapes, with NO customer code content, snippet,
|
|
132
|
+
repo fact, or repo-identifying detail. (No `cites`, `snippet`, `code`, `repo`, or customer-path
|
|
133
|
+
field exists — by design.)
|
|
134
|
+
```
|
|
135
|
+
|
|
136
|
+
The ledger is **not** compile-gated (only the sage card and this SOP are — they live under
|
|
137
|
+
`.claude/agents/` and `.claude/skills/`). The ledger's guarantee is the scrub step + the schema,
|
|
138
|
+
enforced by sage and by MR review.
|
|
139
|
+
|
|
140
|
+
---
|
|
141
|
+
|
|
142
|
+
## Procedure (`/rafa sage`)
|
|
143
|
+
|
|
144
|
+
1. **Read the loop-events store — it is sage's index.** `get_loop_events` (optionally per
|
|
145
|
+
`category`, with `limit`) → the shapes: `category`, `outcome`, `subject` (shape ref), `at`. No
|
|
146
|
+
bodies, no customer artifacts — the store is shapes-only by construction. Also read the agent
|
|
147
|
+
cards (`.claude/agents/*.md`) and their SOPs (`.claude/skills/rafa-*`) as the surface you may
|
|
148
|
+
propose to change, and the existing `ledger.md` to reconcile against.
|
|
149
|
+
2. **Aggregate into miss CLASSES.** Group outcomes by category and by the miss taxonomy — e.g.
|
|
150
|
+
recurring `prism-verdict :: ITERATE` on a Done-check class, `gate-result :: failed` on a check
|
|
151
|
+
class, `reflex-outcome :: session-only` where knowledge should have been durable,
|
|
152
|
+
`distill-refutation :: refuted` on a note class. A PATTERN is a recurring shape, not a single
|
|
153
|
+
event. One-offs are noise; recurrence is signal.
|
|
154
|
+
3. **Attribute to agent structure.** For each pattern, ask: which agent's card/SOP could be
|
|
155
|
+
re-shaped to cover this class of miss? (scan under-covers a seam class → rafa-scan; a Done-check
|
|
156
|
+
is chronically ambiguous → rafa-plan / rafa-build; a gate keeps catching the same class →
|
|
157
|
+
rafa-validate.) The proposal targets OUR blueprint, never customer code.
|
|
158
|
+
4. **SCRUB STEP — run before writing any entry (asset-free gate (a)).** For each candidate
|
|
159
|
+
learning, scrub every field: is anything **asset-shaped** — code content, a snippet, a
|
|
160
|
+
repo-specific fact, a repo name/path, or any repo-identifying detail? If yes → **abstract it to
|
|
161
|
+
the pattern** (state the class, not the instance) or, if it can't be abstracted without the
|
|
162
|
+
asset, **DROP the learning**. Evidence must reference event shapes/categories only. A learning
|
|
163
|
+
that survives the scrub is a portable pattern; one that doesn't is not yet a learning.
|
|
164
|
+
5. **Route person-shaped observations OUT.** Anything about a *dev* (how they ask, steer, prefer)
|
|
165
|
+
is not a sage learning — hand it to compass's consent path (rafa-insights), never the ledger.
|
|
166
|
+
6. **Write learnings.** One file per learning per the entry schema (gate (b): no code-content
|
|
167
|
+
field). Cite event shapes. **Reconcile** against the existing ledger: dedup; mark superseded
|
|
168
|
+
patterns `status: superseded`; preserve prior human triage (`accepted`/`rejected` stay).
|
|
169
|
+
7. **Ledger + leverage.** Regenerate `ledger.md`: counts by category / target / status, and the
|
|
170
|
+
**top-leverage few** (impact × ease) — the single card/SOP change that would cover the widest
|
|
171
|
+
class of misses, first.
|
|
172
|
+
8. **Present, don't nag, don't apply.** Surface the top few high-leverage learnings succinctly as
|
|
173
|
+
PROPOSALS. sage never applies a diff; a human reviews the ledger, and any accepted change lands
|
|
174
|
+
as a versioned, MR-reviewed card/SOP edit (bump `version:`, record the why). Silent otherwise.
|
|
175
|
+
|
|
176
|
+
---
|
|
177
|
+
|
|
178
|
+
## Leverage
|
|
179
|
+
|
|
180
|
+
Rank learnings by **impact × ease** — the widest miss-class covered by the smallest card/SOP
|
|
181
|
+
change, first. A learning is worth proposing only if a plausible card/SOP edit would measurably
|
|
182
|
+
shrink a recurring miss class. Leverage is sage's *proposal*; the human reviewer sets the call.
|
|
183
|
+
|
|
184
|
+
---
|
|
185
|
+
|
|
186
|
+
## Anti-patterns (each is a way sage becomes unsafe or unwelcome)
|
|
187
|
+
- **Any customer asset in the ledger** — a snippet, a repo fact, a customer file path. The
|
|
188
|
+
cardinal sin: it breaks the tenancy floor. Abstract to the pattern or drop it.
|
|
189
|
+
- **A person-shaped "learning"** — that's compass's plane, under consent, in the user brain.
|
|
190
|
+
- **Self-applying a diff** — sage proposes; humans apply. No auto-apply path, ever.
|
|
191
|
+
- **Mid-session output** — sage is silent; a live nudge is a bug.
|
|
192
|
+
- **A learning from a single event** — a one-off is noise; only recurring shapes are patterns.
|
|
193
|
+
- **Uncited learnings** — no event-shape evidence = a hunch; drop it.
|
|
194
|
+
|
|
195
|
+
---
|
|
196
|
+
|
|
197
|
+
## Future hardening (not v1)
|
|
198
|
+
- `prism`-style validation of the learnings ledger (are the patterns real, the proposals sound).
|
|
199
|
+
- Recurrence thresholds calibrated from real observed windows.
|
|
200
|
+
- Wider loop-event categories as the loop grows (new checkpoint beats → new evidence shapes),
|
|
201
|
+
absorbed as new `category` values — never a new code-content-capable field.
|
|
@@ -221,8 +221,47 @@ retrieval index. Bodies read like a senior engineer explaining that one concept
|
|
|
221
221
|
Producing the brain + a **green checker (gate 1)** completes *this capability's* job. The
|
|
222
222
|
independent QA (prism) and the fix→re-check→re-validate loop are owned by the **conductor**
|
|
223
223
|
— the `/rafa` command, running in the main session — because subagents can't spawn
|
|
224
|
-
subagents, so the loop must live there. **scan.md never spawns prism.**
|
|
225
|
-
|
|
224
|
+
subagents, so the loop must live there. **scan.md never spawns prism.** The full loop is
|
|
225
|
+
the conductor orchestration below (moved here from the conductor card in the 2.0.0 diet, so
|
|
226
|
+
this SOP stays self-contained).
|
|
227
|
+
|
|
228
|
+
## Conductor orchestration — the `scan` / `init` pipeline (the conductor runs this)
|
|
229
|
+
|
|
230
|
+
The conductor (not atlas) drives this loop from the main session; atlas is the spawned
|
|
231
|
+
subagent it calls in step 1/5. **`init`** = ensure structure idempotently (`.rafa/active.md`
|
|
232
|
+
= `# No active plan`), then run the full scan below. **`scan`** default runs the whole
|
|
233
|
+
pipeline; **`--brain-only`** stops after the brain is validated (step 5 PASS) — skips improve
|
|
234
|
+
+ push, a cheap knowledge refresh.
|
|
235
|
+
|
|
236
|
+
1. **Scan — spawn `atlas`** context-isolated: *"Run the scan per this SOP: comprehensive,
|
|
237
|
+
breadth-before-depth, cited notes → `.rafa/brain/{rules,playbooks}/` + `coverage.md`. Run
|
|
238
|
+
`npx @rafinery/cli verify-citations` until it **exits 0**. Return a coverage summary only —
|
|
239
|
+
not the raw reads."*
|
|
240
|
+
2. **Gate 1 — checker (trust-but-verify):** re-run `npx @rafinery/cli verify-citations`
|
|
241
|
+
yourself. Must **exit 0** (else re-spawn atlas to fix). It writes `citation-check.md`.
|
|
242
|
+
3. **Gate 2 — prism:** spawn `prism` **context-isolated**, passing ONLY: *"Validate the scan
|
|
243
|
+
in `.rafa/brain/` against the repo per your SOP; write `.rafa/brain/checklist.md`."* Never
|
|
244
|
+
pass atlas's reasoning — prism judges blind.
|
|
245
|
+
4. **Read** `.rafa/brain/checklist.md`. Append this round to `.rafa/brain/log.md`.
|
|
246
|
+
5. **`verdict: PASS`** → continue to Improve. **`ITERATE`** → **spawn `atlas`**: *"Fix every
|
|
247
|
+
blocker + major per `checklist.md`, re-run the checker to exit 0, return what changed."*
|
|
248
|
+
Then back to step 2. **Max 3 rounds**; if still not PASS, **STOP** — surface the findings,
|
|
249
|
+
do **not** improve or push (never improve/push an unvalidated brain).
|
|
250
|
+
6. **Improve** *(skip if `--brain-only`)* — run the improve pass ([rafa-improve](../rafa-improve/SKILL.md)):
|
|
251
|
+
spawn `bloom` → `.rafa/improve/`. It reads the *validated* brain as its index, so it only
|
|
252
|
+
runs after PASS.
|
|
253
|
+
7. **Push** — present the full summary (brain verdict/score + top improvements). **On the
|
|
254
|
+
dev's explicit approval**, `npx @rafinery/cli push` — commits `.rafa/` and pushes to the
|
|
255
|
+
brain remote using the dev's own git auth, stamped `brain-for: <code sha>`. Never without approval.
|
|
256
|
+
8. **The coach offer (founding scan only)** — if this was the repo's FIRST scan and the dev's
|
|
257
|
+
user brain is empty (`list_dev_insights` → none), offer ONCE: *"the code side is mapped —
|
|
258
|
+
want me to bootstrap YOUR insights from your usage report?"* Accepted = run `## insights`
|
|
259
|
+
(compass; every candidate offered, banked only on yes). The offer rung of the consent
|
|
260
|
+
ladder — never auto-run, never re-asked after a no.
|
|
261
|
+
|
|
262
|
+
atlas scans + fixes; prism judges; bloom improves; the conductor orchestrates + pushes on
|
|
263
|
+
approval. The conductor owns `log.md`; it never edits `checklist.md` or the brain itself —
|
|
264
|
+
it spawns atlas for that.
|
|
226
265
|
|
|
227
266
|
## Acceptance criteria (strict)
|
|
228
267
|
|