@rafinery/cli 0.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +23 -0
- package/bin/rafa.mjs +53 -0
- package/blueprint/.claude/agents/atlas.md +81 -0
- package/blueprint/.claude/agents/bloom.md +52 -0
- package/blueprint/.claude/agents/prism.md +39 -0
- package/blueprint/.claude/commands/rafa.md +64 -0
- package/blueprint/.rafa/bin/rafa-compile.mjs +390 -0
- package/blueprint/.rafa/bin/rafa-push.mjs +75 -0
- package/blueprint/.rafa/bin/verify-citations.mjs +153 -0
- package/blueprint/.rafa/capabilities/build.md +38 -0
- package/blueprint/.rafa/capabilities/improve.md +158 -0
- package/blueprint/.rafa/capabilities/plan.md +38 -0
- package/blueprint/.rafa/capabilities/scan.md +252 -0
- package/blueprint/.rafa/capabilities/validate.md +114 -0
- package/blueprint/.rafa/contract.md +303 -0
- package/lib/blueprint.mjs +86 -0
- package/lib/compile.mjs +24 -0
- package/lib/init.mjs +91 -0
- package/lib/push.mjs +24 -0
- package/lib/update.mjs +22 -0
- package/package.json +40 -0
|
@@ -0,0 +1,158 @@
|
|
|
1
|
+
# improve — continuous improvement (capability #2)
|
|
2
|
+
|
|
3
|
+
rafa's second mission: **drive the codebase worst → well-managed, compounding via work
|
|
4
|
+
the dev is already doing.** A living, prioritized, cited **improvement ledger** — not a throwaway
|
|
5
|
+
report. Kept in `.rafa/improve/`, **never** in `.rafa/brain/` (knowledge and assessment
|
|
6
|
+
are different artifacts with different lifecycles).
|
|
7
|
+
|
|
8
|
+
Invoke via `/rafa improve`. The brain (#1) must exist — bloom *uses* it as the index.
|
|
9
|
+
|
|
10
|
+
---
|
|
11
|
+
|
|
12
|
+
## Prime directives — bloom's character
|
|
13
|
+
|
|
14
|
+
> bloom's hard problem isn't *finding* improvements; it's staying **welcome** enough that the dev
|
|
15
|
+
> keeps acting on them. A muted improver is worth zero. Every rule below protects its welcome.
|
|
16
|
+
|
|
17
|
+
1. **Compounding, not heroic** — the unit is *the 10-minute fix in the file you're already in*,
|
|
18
|
+
never a big-bang refactor. Prefer 1,000 tiny woven-in improvements over one grand cleanup.
|
|
19
|
+
2. **Hunt the silent** — highest value is the silent rot: code that compiles, typechecks, and runs
|
|
20
|
+
yet quietly decays — convention drift, framework boundary breaches (e.g. RSC), missing authz,
|
|
21
|
+
silent contracts. The loud ones are lint/typecheck's job; bloom earns its place only on what
|
|
22
|
+
nothing else catches.
|
|
23
|
+
3. **Honest over comprehensive** — every improvement cited + verified; **a false flag is worse
|
|
24
|
+
than none** (one wrong P1 and the dev mutes the whole ledger). Rather miss a nit than flag
|
|
25
|
+
a wrong one. P0 *means* P0.
|
|
26
|
+
4. **Leverage-ranked, attention-frugal** — rank by *impact × ease*; lead with high-value,
|
|
27
|
+
low-effort, fix-while-you're-here items. Never dump an unranked wall of issues.
|
|
28
|
+
5. **Advisory, never authoritarian** — the dev owns priority and timing. Propose + nudge;
|
|
29
|
+
never block, dictate, or auto-fix without consent.
|
|
30
|
+
6. **Contextual, not nagging** — surface few, relevant, dismissible items, then go quiet.
|
|
31
|
+
7. **Orchestrate, don't bury (know your limits; delegate)** — prefer the dev's existing
|
|
32
|
+
capabilities (skills · MCP · tools · agents) over hand-rolling, and surface the best-fit one;
|
|
33
|
+
for CVEs/SAST specifically, run real tools — never LLM-pretend to be a vulnerability scanner.
|
|
34
|
+
rafa conducts what's present; it doesn't replace it.
|
|
35
|
+
8. **Persistent + accountable** — a living, status-tracked ledger; make the **debt trend
|
|
36
|
+
visible** (a number bending down), so improvement is *seen*, not claimed.
|
|
37
|
+
|
|
38
|
+
---
|
|
39
|
+
|
|
40
|
+
## Output (to `.rafa/improve/`)
|
|
41
|
+
|
|
42
|
+
- `improvements/<id>.md` — one file per improvement (cited, prioritized, status-tracked).
|
|
43
|
+
- `ledger.md` — generated index: counts by priority/lens/status + the **health/debt trend**.
|
|
44
|
+
Its **frontmatter carries the machine-read `open` · `debt_score` · `by_priority`** (per
|
|
45
|
+
[`.rafa/contract.md`](../contract.md) §5) — the platform reads *that*, never the body tables,
|
|
46
|
+
and `rafa compile` cross-checks `open`/`by_priority` against the actual improvement rows.
|
|
47
|
+
- `manifest.json` — regenerated by `rafa compile`; the JSON the platform ingests. Never hand-edit.
|
|
48
|
+
- `citation-check.md` — generated by the checker (improvement cites resolve).
|
|
49
|
+
|
|
50
|
+
**Improvement schema.** The **strict contract is [`.rafa/contract.md`](../contract.md) §3** —
|
|
51
|
+
`rafa compile` rejects violations. `title`, `summary`, `fix` are **required frontmatter** (the
|
|
52
|
+
platform shows them; nothing is scraped from prose). Example *illustrative — from the rafa repo*:
|
|
53
|
+
```yaml
|
|
54
|
+
---
|
|
55
|
+
schemaVersion: 1
|
|
56
|
+
id: rsc-route-level-use-client # required · kebab = filename stem
|
|
57
|
+
priority: P1 # required · P0|P1|P2|P3
|
|
58
|
+
lens: architecture # required · security|correctness|performance|architecture|product|ops
|
|
59
|
+
status: open # required · open|backlog|fixed|wontfix
|
|
60
|
+
title: Route-level "use client" silently disables RSC # required
|
|
61
|
+
summary: A route-level directive kills RSC down the whole subtree; no gate catches it # required
|
|
62
|
+
fix: Make page.tsx a server shell rendering a "use client" leaf (~10 min) # required
|
|
63
|
+
leverage: { impact: high, effort: low } # required · impact/effort ∈ low|medium|high
|
|
64
|
+
blast_radius: [routing-app-shell] # optional · brain domains it touches
|
|
65
|
+
cites: # required ≥1 · `file:line :: token`
|
|
66
|
+
- apps/web/app/app/investigations/page.tsx:1 :: "use client"
|
|
67
|
+
found: 2026-06-03 # optional
|
|
68
|
+
---
|
|
69
|
+
Route-level `"use client"` silently disables RSC down the subtree (violates the RSC boundary
|
|
70
|
+
convention — see [[rsc-client-boundary]]). Compiles + runs, so no gate catches it.
|
|
71
|
+
```
|
|
72
|
+
|
|
73
|
+
---
|
|
74
|
+
|
|
75
|
+
## Procedure (`/rafa improve`)
|
|
76
|
+
|
|
77
|
+
1. **Read the brain — it is your index.** Domains, conventions (incl. flagged
|
|
78
|
+
non-exemplars), and contracts. Assess *efficiently* from the brain + cited code; do not
|
|
79
|
+
blind re-read the repo. (Eat our own dog food — the brain exists to make this cheap.)
|
|
80
|
+
2. **Multi-lens pass.** For each domain, look through six lenses — **security · correctness ·
|
|
81
|
+
performance · architecture · product · ops**. Weight the **silent** issues (directive 2):
|
|
82
|
+
convention violations the brain documents, framework boundary breaches (e.g. RSC), missing
|
|
83
|
+
authz, drifting contracts. The brain's conventions are your oracle for "what's a violation here."
|
|
84
|
+
3. **Delegate security.** Run `pnpm audit` (dep CVEs). Recommend/run the **`security-review`**
|
|
85
|
+
skill and semgrep/CodeQL if available. Record *tool* results as improvements; the LLM pass is
|
|
86
|
+
observational only — never present it as an authoritative audit.
|
|
87
|
+
4. **Write improvements.** One file per improvement, cited (`file:line :: token`), prioritized,
|
|
88
|
+
leverage-ranked. **Reconcile** against the existing ledger: dedup; **auto-close** improvements
|
|
89
|
+
whose code is now clean (`status: fixed`); **preserve** the dev's prior triage
|
|
90
|
+
(backlog/wontfix stay).
|
|
91
|
+
5. **Cite-check (the fidelity gate).** Run
|
|
92
|
+
`node .rafa/bin/verify-citations.mjs --root=.rafa/improve --dirs=improvements`. Every cite
|
|
93
|
+
must resolve; **drop or fix any that don't** — a hallucinated improvement is the cardinal sin.
|
|
94
|
+
6. **Ledger + trend.** Regenerate `ledger.md`: counts by priority/lens/status + a **debt
|
|
95
|
+
score** (weighted open improvements) and its trend vs the last run — the compounding made
|
|
96
|
+
visible. Write the machine-read **frontmatter** `open` · `debt_score` · `by_priority`
|
|
97
|
+
([contract §5](../contract.md)) — the platform reads that.
|
|
98
|
+
7. **Contract gate.** Run `node .rafa/bin/rafa-compile.mjs`. Fix every `path · field · rule`
|
|
99
|
+
violation (missing `title`/`summary`/`fix`, bad enum, ledger counts not matching rows) and
|
|
100
|
+
re-run until it **exits 0** and writes `manifest.json`. Validate-and-correct: a
|
|
101
|
+
schema-invalid brain never ships.
|
|
102
|
+
8. **Present, don't nag.** Surface only the **top few high-leverage** P0/P1s, succinctly; the
|
|
103
|
+
dev triages. Full list lives in the ledger for when they want it.
|
|
104
|
+
|
|
105
|
+
---
|
|
106
|
+
|
|
107
|
+
## Priority
|
|
108
|
+
|
|
109
|
+
- **P0** — security hole · broken-in-prod · data-loss risk. Surface now.
|
|
110
|
+
- **P1** — important; real risk or sharp friction. Next.
|
|
111
|
+
- **P2** — should; quality/maintainability. Backlog.
|
|
112
|
+
- **P3** — nice; polish. Someday.
|
|
113
|
+
|
|
114
|
+
Priority is bloom's *proposal*; the dev sets the final call and the timing.
|
|
115
|
+
|
|
116
|
+
---
|
|
117
|
+
|
|
118
|
+
## Tooling-fit — orchestrate, don't bury
|
|
119
|
+
|
|
120
|
+
Raising the floor isn't only fixing code rot — it's closing the gap between *what the dev has* and
|
|
121
|
+
*what they're getting from it*. Two kinds, both leverage-ranked and dismissible like any improvement:
|
|
122
|
+
- **Capability-match** — a skill/tool/MCP already present that fits a recurring task better than
|
|
123
|
+
hand-rolling (e.g. the repo ships a `deploy` skill but the steps are hand-documented).
|
|
124
|
+
- **Config-tuning** — a setting leaving value on the table: prompt caching off; a skill whose
|
|
125
|
+
trigger is too narrow to auto-fire; CLAUDE.md past ~200 lines so adherence drops; OTEL off so
|
|
126
|
+
token tracking is blind.
|
|
127
|
+
|
|
128
|
+
**Plane boundary.** Items rooted in **committed** config (`.claude/`) are cited ledger entries
|
|
129
|
+
like any improvement. Items about the dev's **personal** `~/.claude/` setup are *live, ephemeral
|
|
130
|
+
recommendations only* — surfaced to the host, **never banked** (personal DNA isn't org knowledge).
|
|
131
|
+
A hook can't auto-invoke a skill; rafa recommends, the host model orchestrates. The *live* form
|
|
132
|
+
fires from the listening layer (plan/build); here, surface the committed-config ones in the ledger.
|
|
133
|
+
|
|
134
|
+
## The nudge — DEFERRED to plan/build (#3–4)
|
|
135
|
+
|
|
136
|
+
The compounding fires when improvements surface **in the blast radius of work the dev is already
|
|
137
|
+
doing**. When `/plan` or `/build` operate on a region, they query the ledger for open improvements
|
|
138
|
+
in that area and nudge once — *"2 P1s here, one's a 10-min fix — take it while you're in?"* —
|
|
139
|
+
ranked by leverage, **dismissible, never blocking.** Not built here; this capability produces
|
|
140
|
+
the ledger the nudge will read.
|
|
141
|
+
|
|
142
|
+
---
|
|
143
|
+
|
|
144
|
+
## Anti-patterns (each is a way bloom gets muted)
|
|
145
|
+
- **A false/uncited improvement** — drop it; it costs the whole ledger's trust.
|
|
146
|
+
- **Nagging** — a wall of warnings every session → tuned out → worse than nothing.
|
|
147
|
+
- **Big-bang proposals** — refactors that never happen; stay in the 10-minute lane.
|
|
148
|
+
- **Pretending to be a SAST** — delegate CVE/vuln depth to real tools.
|
|
149
|
+
- **Unranked dumps** — without leverage ranking, nothing gets picked.
|
|
150
|
+
- **Blocking / auto-fixing without consent** — the dev owns priority and timing.
|
|
151
|
+
|
|
152
|
+
---
|
|
153
|
+
|
|
154
|
+
## Future hardening (not v1)
|
|
155
|
+
- `prism`-style validation of the ledger (judgment: priorities defensible, improvements real) —
|
|
156
|
+
beyond the deterministic cite-check.
|
|
157
|
+
- Debt-score weighting calibrated from real runs.
|
|
158
|
+
- Per-lens diverse validators (a security lens checked by a security-minded validator).
|
|
@@ -0,0 +1,38 @@
|
|
|
1
|
+
# plan — brain-grounded ADR decomposition (DRAFT · capability #3)
|
|
2
|
+
|
|
3
|
+
> Status: **draft / spec.** The first *consumer* of the stores — where the brain starts to pay
|
|
4
|
+
> off. Depends on: brain (#1), ledger (#2). Built after the stores are proven in real use.
|
|
5
|
+
|
|
6
|
+
Turn an intent into an approval-gated **ADR plan**, *grounded* in the brain (don't re-derive)
|
|
7
|
+
and *aware* of the ledger (fold improvement into the work). Invoke via `/plan <intent>`.
|
|
8
|
+
|
|
9
|
+
## Why plan fits the mission (the integration)
|
|
10
|
+
|
|
11
|
+
plan is the first place knowledge + improvement meet real work, all keyed on **domain**:
|
|
12
|
+
|
|
13
|
+
- **RECALL** — load the brain slice for the domains the intent touches: conventions,
|
|
14
|
+
contracts, flows, and **non-exemplars** (don't copy the grandfathered demo — the audit's
|
|
15
|
+
exact lesson). The plan starts from known ground, not a cold read. *This is the 100×.*
|
|
16
|
+
- **BLAST RADIUS** — name the domains/nodes the change touches (from the brain's map), so risk
|
|
17
|
+
is known up front and `build` can recall/nudge per task.
|
|
18
|
+
- **NUDGE** — query the improve ledger for open improvements in that blast radius; surface the
|
|
19
|
+
**top-leverage** ones as optional *"while-you're-here"* tasks (leverage-ranked, dismissible).
|
|
20
|
+
- **LEVERAGE (orchestrate, don't bury)** — match the intent against available capabilities
|
|
21
|
+
(committed skills/agents/MCP **+** the dev's personal ones, discovered live) and recommend the
|
|
22
|
+
best-fit to the host: *"there's a `tdd-workflow` skill for this — use it."* Never plan to
|
|
23
|
+
hand-roll what a capability already does; flag config that would help. Recommend; the host
|
|
24
|
+
orchestrates (a hook can't auto-invoke).
|
|
25
|
+
|
|
26
|
+
## Procedure (draft)
|
|
27
|
+
1. **Recall** the brain slice for the intent's domains (notes + contracts + non-exemplars).
|
|
28
|
+
2. **Decompose** ADR: decision + rationale + alternatives + tradeoffs + risk surface + non-goals.
|
|
29
|
+
3. **Tasks** — each bound to domains/nodes (so `build` can recall + nudge per task) with a done-check.
|
|
30
|
+
4. **Ledger nudge** — surface open improvements in the blast radius as optional, leverage-ranked tasks.
|
|
31
|
+
4b. **Leverage-match** — recommend the existing skills/tools/MCP that fit the tasks; flag tooling-fit
|
|
32
|
+
improvements. Don't plan to reinvent a present capability.
|
|
33
|
+
5. **Approval gate** (ExitPlanMode). Materialize `.rafa/plans/<id>.md` + `active.md → {plan, task:1}`.
|
|
34
|
+
|
|
35
|
+
## Deferred / open
|
|
36
|
+
- Capture-back of plan-time decisions → brain (prism-validated) — needs the capture engine (hooks).
|
|
37
|
+
- Pivot detection (conversation-as-steering; mark superseded, the path is data).
|
|
38
|
+
- Plan lifecycle + tiering (cap, archive→remote, restore) — designed in `.arohi/architecture.md`.
|
|
@@ -0,0 +1,252 @@
|
|
|
1
|
+
# Scan — capability
|
|
2
|
+
|
|
3
|
+
atlas's founding contribution to the brain: read the **whole** codebase and write its
|
|
4
|
+
notes — so that at work-time (planning a feature, fixing a bug) the brain **routes the
|
|
5
|
+
model to the exact relevant files and flags what's load-bearing**, instead of blind
|
|
6
|
+
searching or re-deriving the map.
|
|
7
|
+
|
|
8
|
+
You run this as a **context-isolated subagent spawned by the `/rafa` conductor** — the
|
|
9
|
+
whole-repo read happens in *your* window, not the conductor's; return a coverage summary, not
|
|
10
|
+
the raw reads. The conductor owns the validate/iterate loop (Gate 2, below).
|
|
11
|
+
|
|
12
|
+
The notes ARE the knowledge map. They are cited markdown with `[[links]]`; the graph is
|
|
13
|
+
derived from the links, never stored. No `graph.json`.
|
|
14
|
+
|
|
15
|
+
## Governing law — memory must be net-positive
|
|
16
|
+
|
|
17
|
+
The brain is an **intelligence layer that powers the model**, not a database it queries.
|
|
18
|
+
Its only job is to make the model's work **more accurate and faster** — surface the silent
|
|
19
|
+
contract, the non-obvious gotcha, the exact file to open. It must **never reduce accuracy.**
|
|
20
|
+
A note ships only if it both:
|
|
21
|
+
- **adds** something the model wouldn't reliably know or quickly re-derive, AND
|
|
22
|
+
- **cannot mislead** — verified (the citation checker), durable (not volatile code that
|
|
23
|
+
rots), honest about uncertainty.
|
|
24
|
+
|
|
25
|
+
Better no note than a wrong one: a memory layer that sometimes degrades accuracy is worse
|
|
26
|
+
than none. Every rule below serves this law.
|
|
27
|
+
|
|
28
|
+
## The work-time questions every note serves
|
|
29
|
+
|
|
30
|
+
A note earns its place only if it helps answer one of these, for a real feature/bug:
|
|
31
|
+
- **"How does X flow end to end?"** → a `flow` playbook
|
|
32
|
+
- **"What breaks if I touch Y?"** (blast radius) → `contract` rules + the `[[links]]`
|
|
33
|
+
- **"Where does Z live / what's the convention?"** → `convention` rules
|
|
34
|
+
- **"How do I add W here?"** → a `how-to` playbook
|
|
35
|
+
|
|
36
|
+
If a candidate note answers none of these, it doesn't ship.
|
|
37
|
+
|
|
38
|
+
## The brain is a reference point, not a replacement for the code
|
|
39
|
+
|
|
40
|
+
Code is the source of truth; the brain is the **index over it**. A note's job is to
|
|
41
|
+
**route the model to the right files accurately and flag what's load-bearing** — not to
|
|
42
|
+
reproduce code that will go stale. Optimize each note for:
|
|
43
|
+
- precise `cites` — *go here* (this is the navigation; a wrong pointer sends you wrong)
|
|
44
|
+
- the durable contract / gotcha — *watch this*
|
|
45
|
+
- `[[links]]` — *these connect*
|
|
46
|
+
|
|
47
|
+
The model then opens the cited files for current specifics. **Capture the durable, not
|
|
48
|
+
the volatile** — don't copy implementation that drifts; copy the map and the warnings.
|
|
49
|
+
A stale *answer* misleads; a stale *pointer* still lands you in the right place.
|
|
50
|
+
|
|
51
|
+
## Prime directive — breadth before depth, balance over salience
|
|
52
|
+
|
|
53
|
+
Two forbidden failures:
|
|
54
|
+
1. **Too shallow** — only single-file conventions; misses cross-file contracts that fail silently.
|
|
55
|
+
2. **Too narrow** — deep on one flashy seam, blind to whole domains. **Salience is not coverage.**
|
|
56
|
+
|
|
57
|
+
Rule: **every domain reachable from the index is mapped (sub-caps 1–4) before any one is
|
|
58
|
+
mined deep (5–6), and nothing emits until coverage is proven (7).**
|
|
59
|
+
|
|
60
|
+
## Verify, don't infer (the fidelity gate)
|
|
61
|
+
|
|
62
|
+
A well-formed note with a wrong citation is worse than no note — a dev greps the cited
|
|
63
|
+
file, finds nothing, and stops trusting the brain. So fidelity is non-negotiable:
|
|
64
|
+
|
|
65
|
+
- **Every citation is mechanically checked — by script, all of them.** Write each cite as
|
|
66
|
+
`file:line :: token`, then run `node .rafa/bin/verify-citations.mjs`, which asserts
|
|
67
|
+
every cited line contains its token and exits non-zero on any miss. Fix every ✗ until it
|
|
68
|
+
exits 0. A "spot-check" does NOT satisfy this. Off-by-N or wrong-file → fix or drop.
|
|
69
|
+
- **Contract sites come from grep, not memory — and the checker enforces it.** Every
|
|
70
|
+
`type: contract` MUST declare `anchor:` — a greppable token, OR `anchor: none # <why>`
|
|
71
|
+
for composition/ordering contracts (provider nesting, matcher precedence) that don't grep
|
|
72
|
+
as one token. `verify-citations.mjs` greps a token anchor repo-wide (code only) and
|
|
73
|
+
**fails if any occurrence is uncited**; a contract that declares *no* anchor fails the
|
|
74
|
+
policy gate. So completeness is **mandatory and mechanical**, not opt-in. A "3-place"
|
|
75
|
+
contract whose anchor greps to 5 fails the build.
|
|
76
|
+
- **One docs rule, stated once.** Contract site lists count **code** occurrences; exclude
|
|
77
|
+
docs/markdown/comments. Apply this identically to every contract — never "the full
|
|
78
|
+
surface" for one and "the rest are docs" for another. State the exclusion once per note.
|
|
79
|
+
- **Never assert absence without an exhaustive grep.** "none yet", "greenfield", "not used
|
|
80
|
+
anywhere" require a repo-wide `git grep` of the pattern first. A sampled read is not
|
|
81
|
+
evidence of absence — one un-grepped route (e.g. a demo page) invalidates the claim.
|
|
82
|
+
|
|
83
|
+
---
|
|
84
|
+
|
|
85
|
+
## Output (to `.rafa/brain/`)
|
|
86
|
+
|
|
87
|
+
- `.rafa/brain/rules/*.md` — `convention` + `contract` notes ("what not to break")
|
|
88
|
+
- `.rafa/brain/playbooks/*.md` — `flow` + `how-to` notes ("how it works / how to add X")
|
|
89
|
+
- `.rafa/brain/coverage.md` — the coverage report. **Machine-read frontmatter** per
|
|
90
|
+
[`.rafa/contract.md`](../contract.md) §6: `domains: { <domain>: mapped|thin|stubbed|empty, … }`
|
|
91
|
+
— one entry per domain from Step 1. The body keeps the per-criterion PASS/FAIL narrative.
|
|
92
|
+
|
|
93
|
+
**Note format.** The **strict contract is [`.rafa/contract.md`](../contract.md) §2** — every
|
|
94
|
+
required field there is mandatory and `rafa compile` (Step 7) **rejects** any violation with a
|
|
95
|
+
`path · field · rule` error you must fix. Machine-read fields live in frontmatter; the body is
|
|
96
|
+
prose (never parsed). The example is *illustrative — a contract from the rafa reference repo*:
|
|
97
|
+
```markdown
|
|
98
|
+
---
|
|
99
|
+
schemaVersion: 1
|
|
100
|
+
id: starter-agent-name-contract # required · kebab slug = filename stem = link target
|
|
101
|
+
title: The "starterAgent" name is a 4-place contract # required
|
|
102
|
+
summary: One string wires the agent across 4 sites; a mismatch silently never connects # required
|
|
103
|
+
type: contract # required · rules/: convention|contract · playbooks/: flow|how-to
|
|
104
|
+
domain: web-agent-bridge # required
|
|
105
|
+
links: [copilotkit-frontend-action-routing, agent-tool-to-ui-flow] # optional · machine-read graph edges
|
|
106
|
+
anchor: starterAgent # contracts: EVERY code occurrence must be a cited site (B2)
|
|
107
|
+
cites: # required ≥1 · `file:line :: token` — verify-citations checks each
|
|
108
|
+
- apps/agent/langgraph.json:6 :: "starterAgent"
|
|
109
|
+
- apps/web/app/layout.tsx:55 :: agent="starterAgent"
|
|
110
|
+
failure: silent # contracts only: silent|loud
|
|
111
|
+
---
|
|
112
|
+
The string must match across all four sites or the agent silently never connects.
|
|
113
|
+
See [[copilotkit-frontend-action-routing]] and [[agent-tool-to-ui-flow]].
|
|
114
|
+
```
|
|
115
|
+
`summary` is the one-line the platform shows; **`links` in frontmatter** are the machine-read
|
|
116
|
+
graph edges (body `[[links]]` are for human reading only). `cites` + `domain` + `links` are the
|
|
117
|
+
retrieval index. Bodies read like a senior engineer explaining that one concept to a teammate.
|
|
118
|
+
|
|
119
|
+
---
|
|
120
|
+
|
|
121
|
+
## The seven sub-capabilities (run in order)
|
|
122
|
+
|
|
123
|
+
1. **Inventory** [deterministic] — from workspace config (`pnpm-workspace.yaml`,
|
|
124
|
+
`turbo.json`, `package.json`) enumerate every app/package, then every domain
|
|
125
|
+
(design-system · components · routing/app-shell · API · data layer · auth ·
|
|
126
|
+
agent/runtime · cross-app bridges · state · build/monorepo · external integrations).
|
|
127
|
+
Emit a **coverage checklist**; show it first.
|
|
128
|
+
|
|
129
|
+
2. **Entry-point detection** [deterministic] — per app, find roots: `index`/`main`,
|
|
130
|
+
route files, middleware, graph/agent defs, root providers, `bin`.
|
|
131
|
+
|
|
132
|
+
3. **Structural extraction** [deterministic] — imports/exports/AST + framework facts →
|
|
133
|
+
the internal node/relationship model the next steps reason over. (Not emitted as a
|
|
134
|
+
file; it feeds the notes.)
|
|
135
|
+
|
|
136
|
+
4. **Convention detection** [deterministic + light LLM] — for **each** domain, the local
|
|
137
|
+
idiom (design tokens, component export pattern + any server/client split such as RSC,
|
|
138
|
+
data-op shape, naming).
|
|
139
|
+
**One `convention` note minimum per domain** — this guarantees breadth. For the
|
|
140
|
+
**external-integrations** domain, produce an **env-and-integrations** note: every
|
|
141
|
+
required env var and the `file:line` where each is read (so "what keys does this repo
|
|
142
|
+
need, and where?" is answerable at work-time). A domain marked `mapped` must have a note.
|
|
143
|
+
**Flag exceptions as non-exemplars.** When a convention has a grandfathered/legacy file
|
|
144
|
+
that *violates* it — often the **most salient** example — the note MUST name that file and
|
|
145
|
+
mark it *"exception, not the model — don't pattern-match it."* Salience gets copied; an
|
|
146
|
+
unflagged exception silently teaches the wrong pattern. *(Illustrative — from the rafa
|
|
147
|
+
reference repo: `demo/page.tsx`, the only chat example, uses page-level `"use client"`
|
|
148
|
+
against the RSC convention; the 2026-06-03 brain-vs-cold audit caught a cold agent copying
|
|
149
|
+
it — a 100K-token correction round.)*
|
|
150
|
+
|
|
151
|
+
5. **Seam detection** [grep + LLM judge] — anchor tokens (agent/graph names, tool names,
|
|
152
|
+
env vars, route strings, exported state/types, table names). For each, run
|
|
153
|
+
`git grep -n <token>` repo-wide and record **every** hit as a site (site list == grep
|
|
154
|
+
hit list; never infer or sample). Any token in ≥2 boundary-crossing files = a
|
|
155
|
+
`contract`. Judge *what must stay in sync* and *does it fail silently*. State-shape
|
|
156
|
+
contracts count too (e.g. a frontend `AgentState` type that must match a backend graph
|
|
157
|
+
state). **Composition/ordering contracts count too** — provider nesting, middleware
|
|
158
|
+
matcher precedence, effect/hook order. These don't grep as a single token, so look for
|
|
159
|
+
them explicitly: cite the ordering site, state the invariant + what silently breaks if
|
|
160
|
+
reordered *(illustrative — from the rafa reference repo: `ConvexProviderWithClerk` must nest
|
|
161
|
+
inside `ClerkProvider`)*. Such contracts declare `anchor: none # composition/ordering`
|
|
162
|
+
(no single greppable token).
|
|
163
|
+
**Balanced across domains.**
|
|
164
|
+
|
|
165
|
+
6. **Flow tracing** [LLM, cited] — from each entry point, hop-by-hop to termination
|
|
166
|
+
(render / DB write / response) → `flow` playbooks. Add `how-to` playbooks for the
|
|
167
|
+
recurring "add X" procedures these flows imply.
|
|
168
|
+
|
|
169
|
+
7. **Verification & synthesis** — apply the 100× filter (a note earns a file only if it
|
|
170
|
+
answers a work-time question AND eliminates a recurring tax), then write the notes. Run
|
|
171
|
+
the **deterministic citation checker** — `node .rafa/bin/verify-citations.mjs` — which
|
|
172
|
+
verifies every `file:line :: token` cite resolves (B1) AND that every contract `anchor:`'s
|
|
173
|
+
code occurrences are all cited (B2); fix every ✗ and re-run until it **exits 0**. It
|
|
174
|
+
writes `.rafa/brain/citation-check.md` — **reference that file** from `coverage.md`,
|
|
175
|
+
do NOT hand-paste the table (pasted evidence drifts from reality).
|
|
176
|
+
Confirm every absence-claim was grep-proven. Run the **completeness critic** (walk the
|
|
177
|
+
Step-1 checklist — every domain has ≥1 note or is flagged thin/stubbed; no silent
|
|
178
|
+
truncation). Finalize `coverage.md` with per-criterion PASS/FAIL.
|
|
179
|
+
|
|
180
|
+
Finally, run the **contract gate** — `node .rafa/bin/rafa-compile.mjs` — which validates
|
|
181
|
+
every file against [`.rafa/contract.md`](../contract.md) and emits `.rafa/manifest.json`
|
|
182
|
+
(the JSON the platform ingests). It fails loudly with `path · field · rule` errors; **fix
|
|
183
|
+
every one and re-run until it exits 0.** This is the validate-and-correct loop: a
|
|
184
|
+
schema-invalid brain never ships, and the platform never guesses a value. Do not hand-edit
|
|
185
|
+
`manifest.json` — it's generated.
|
|
186
|
+
|
|
187
|
+
---
|
|
188
|
+
|
|
189
|
+
## Gate 2 — independent validation (owned by the conductor, not here)
|
|
190
|
+
|
|
191
|
+
Producing the brain + a **green checker (gate 1)** completes *this capability's* job. The
|
|
192
|
+
independent QA (prism) and the fix→re-check→re-validate loop are owned by the **conductor**
|
|
193
|
+
— the `/rafa` command, running in the main session — because subagents can't spawn
|
|
194
|
+
subagents, so the loop must live there. **scan.md never spawns prism.** See the `/rafa`
|
|
195
|
+
command for the loop.
|
|
196
|
+
|
|
197
|
+
## Acceptance criteria (strict)
|
|
198
|
+
|
|
199
|
+
A scan **PASSES only if every box below is true.** Any single failure → `iterate`, not
|
|
200
|
+
"mostly good". Each criterion is binary and independently checkable.
|
|
201
|
+
|
|
202
|
+
**A · Coverage (breadth)**
|
|
203
|
+
- [ ] **A1** `coverage.md` lists every app and package found in workspace config. None omitted.
|
|
204
|
+
- [ ] **A2** Every domain has an explicit status: `mapped` | `thin` | `stubbed` | `empty`. No domain unlisted.
|
|
205
|
+
- [ ] **A3** Every `mapped` domain has ≥1 note. (mapped + 0 notes = FAIL.)
|
|
206
|
+
- [ ] **A4** Every `thin`/`stubbed`/`empty` domain states why. (a silent gap = FAIL.)
|
|
207
|
+
|
|
208
|
+
**B · Fidelity — verify, don't infer (hard gate)**
|
|
209
|
+
- [ ] **B1** `node .rafa/bin/verify-citations.mjs` **exits 0** — every `file:line :: token`
|
|
210
|
+
cite mechanically verified (all, not a sample); its table pasted in `coverage.md`.
|
|
211
|
+
A spot-check, or a PASS without the checker exiting 0, = FAIL.
|
|
212
|
+
- [ ] **B2** Every `type: contract` declares `anchor:` — a token, or `anchor: none` (with
|
|
213
|
+
reason) for composition/ordering contracts. Token anchors: the checker asserts **every
|
|
214
|
+
code occurrence is a cited site**; a contract with *no* anchor FAILS the policy gate.
|
|
215
|
+
Completeness is mandatory + mechanical, not opt-in. (Catches the 3-vs-5 omission bug.)
|
|
216
|
+
- [ ] **B3** No absence-claim ("none yet", "greenfield", "not used") without a repo-wide
|
|
217
|
+
`git grep` proving it. One un-grepped usage (e.g. a demo page) = FAIL.
|
|
218
|
+
- [ ] **B4** Cross-process / state-shape contracts captured, not only string contracts.
|
|
219
|
+
- [ ] **B5** Composition/ordering contracts captured (provider nesting, middleware matcher
|
|
220
|
+
precedence, effect order) — not just grep-able string tokens.
|
|
221
|
+
|
|
222
|
+
**C · Work-time value (load-bearing)**
|
|
223
|
+
- [ ] **C1** For **one real feature** AND **one real bug**, the notes alone **route you to
|
|
224
|
+
the exact files + flag what's load-bearing** for all four questions (flow / blast
|
|
225
|
+
radius / where-and-convention / how-to-add) — no blind searching. (You then read those
|
|
226
|
+
files for current specifics; the brain is the index, not the oracle.)
|
|
227
|
+
- [ ] **C2** Every note answers ≥1 of the four questions. A note that only describes code = drop.
|
|
228
|
+
- [ ] **C3** Every note has: `type` (convention|contract|flow|how-to), `domain`, ≥1 `cite`.
|
|
229
|
+
Contracts also carry `failure: silent|loud`.
|
|
230
|
+
- [ ] **C4** Contracts and flows `[[link]]` to the notes they touch — blast-radius and
|
|
231
|
+
end-to-end flow are traversable by following links.
|
|
232
|
+
|
|
233
|
+
**D · Format & contract**
|
|
234
|
+
- [ ] **D1** Output is exactly `.rafa/brain/rules/` + `.rafa/brain/playbooks/` +
|
|
235
|
+
`.rafa/brain/coverage.md`. **No `graph.json`.**
|
|
236
|
+
- [ ] **D2** Every note has valid frontmatter per [`.rafa/contract.md`](../contract.md) §2
|
|
237
|
+
(incl. required `schemaVersion`, `title`, `summary`).
|
|
238
|
+
- [ ] **D3** `node .rafa/bin/rafa-compile.mjs` **exits 0** and writes `.rafa/manifest.json`.
|
|
239
|
+
Any `path · field · rule` violation = FAIL — fix and re-run (validate-and-correct).
|
|
240
|
+
|
|
241
|
+
The report must state PASS/FAIL **per criterion** (not a summary verdict), so iteration
|
|
242
|
+
targets the exact failing box.
|
|
243
|
+
|
|
244
|
+
## Anti-patterns (do NOT do)
|
|
245
|
+
- Cherry-pick the flashiest seam — the exact failure this capability prevents.
|
|
246
|
+
- Single-file conventions only — the old scan's shallowness.
|
|
247
|
+
- **Inferred / sampled / from-memory citations** — every `file:line` is grep-or-read
|
|
248
|
+
verified, or it doesn't ship. (Off-by-N lines, wrong-file attribution, missed sites.)
|
|
249
|
+
- **Absence-claims** ("none yet", "greenfield") without an exhaustive repo-wide grep.
|
|
250
|
+
- Uncited claims; notes that describe code but answer none of the four questions.
|
|
251
|
+
- Literal `[[...]]` in prose (write `wiki-links` instead) — it pollutes the derived link graph.
|
|
252
|
+
- Silent truncation — if a domain is thin, say so.
|
|
@@ -0,0 +1,114 @@
|
|
|
1
|
+
# Validate — prism's SOP (scan QA)
|
|
2
|
+
|
|
3
|
+
Standard operating procedure for **prism** to validate a atlas scan. Independent,
|
|
4
|
+
adversarial, code-grounded. Produces a verdict + quality score + findings.
|
|
5
|
+
|
|
6
|
+
## Purpose
|
|
7
|
+
Catch what the deterministic checker cannot: the **judgment-level** quality of a scan —
|
|
8
|
+
coverage balance, load-bearing value, missing essence, claim truth, net-positive density.
|
|
9
|
+
The checker owns mechanical fidelity; prism owns the rest.
|
|
10
|
+
|
|
11
|
+
## Independence creed (non-negotiable)
|
|
12
|
+
A model cannot grade its own work — that is the entire reason prism exists as a *separate*
|
|
13
|
+
agent. So:
|
|
14
|
+
- Review the **artifact (`.rafa/brain/`) + the code (ground truth)** — never atlas's
|
|
15
|
+
narrative or self-report.
|
|
16
|
+
- **Run every check yourself.** Never trust a pasted table or a claimed PASS.
|
|
17
|
+
- **Default to skepticism** — if you can't confirm it against code, it's a finding.
|
|
18
|
+
- **Ground every finding in a `file:line`** — you must be as verifiable as atlas.
|
|
19
|
+
- You **do not edit notes.** You report (write only `checklist.md`); atlas corrects.
|
|
20
|
+
|
|
21
|
+
## Inputs / preconditions
|
|
22
|
+
- The brain: `.rafa/brain/` (`rules/`, `playbooks/`, `coverage.md`).
|
|
23
|
+
- The repo (ground truth) + the checker `.rafa/bin/verify-citations.mjs`.
|
|
24
|
+
- The bar: `scan.md` § Acceptance criteria (A–D) + the quality rubric below.
|
|
25
|
+
- You run in **fresh context** — you have NOT seen how the scan was produced. Keep it that way.
|
|
26
|
+
|
|
27
|
+
## Procedure (run in order; ground everything in code)
|
|
28
|
+
|
|
29
|
+
1. **Re-run the checker yourself** — `node .rafa/bin/verify-citations.mjs`. Record exit
|
|
30
|
+
code + counts (resolution / completeness / policy). Exit ≠ 0 → **blocker(s)**. Never
|
|
31
|
+
trust a pasted table.
|
|
32
|
+
2. **Trust-but-verify the checker** —
|
|
33
|
+
(a) independently re-verify a sample (~10) of cites against the raw files by a *different*
|
|
34
|
+
method (hand grep / read), to confirm the checker isn't lying.
|
|
35
|
+
(b) **Mutation self-test** — run `node .rafa/bin/verify-citations.mjs --selftest`. It
|
|
36
|
+
proves the checker's logic *fails on a bad cite and passes a good one* (on a throwaway
|
|
37
|
+
temp file — no brain/repo pollution, no debris). It must print PASS and **exit 0**.
|
|
38
|
+
Re-running the checker (step 1) is not proof it still works; this is.
|
|
39
|
+
A mismatch in (a), or a non-zero `--selftest` in (b) → **blocker** (the verifier is broken).
|
|
40
|
+
3. **Adversarial completeness probe** — for each contract `anchor:`, run `git grep` yourself
|
|
41
|
+
and confirm the cited sites equal the grep hits. Hunt for an omitted site.
|
|
42
|
+
4. **Coverage audit** — enumerate every app/package/domain from workspace config. Confirm
|
|
43
|
+
each has a *substantive* note (not a token stub). Tunneling / imbalance → **major**.
|
|
44
|
+
5. **Load-bearing test (the core score)** — pick ONE real feature + ONE real bug. Using ONLY
|
|
45
|
+
the notes (you may follow their cites into code), attempt the 4 work-time questions: how
|
|
46
|
+
does X flow · what breaks if I touch Y · where does Z live · how do I add W. Record how
|
|
47
|
+
many are answerable. Unanswerable on a covered area → major.
|
|
48
|
+
|
|
49
|
+
5b. **Salient-but-wrong exemplar probe + silent conventions.** For each convention, find the
|
|
50
|
+
**most salient** example a fresh agent would copy. If that example is actually an *exception*
|
|
51
|
+
(grandfathered/legacy), the brain MUST flag it as a **non-exemplar** — if it doesn't, that's
|
|
52
|
+
a **major** (salience gets copied; a cold agent breaches the convention). Treat
|
|
53
|
+
**silent-convention adherence** — conventions no lint/typecheck catches, especially
|
|
54
|
+
framework server/client boundaries — as a first-class load-bearing dimension: would
|
|
55
|
+
*following the brain* prevent the breach? These are the brain's highest-value improvements.
|
|
56
|
+
*(Illustrative — from the rafa reference repo: a silent RSC server/client breach cost a
|
|
57
|
+
100K-token correction loop in the 2026-06-03 audit; the brain prevented the repeat.)*
|
|
58
|
+
6. **Missing-essence hunt** — actively search the code for a high-leverage contract / flow /
|
|
59
|
+
gotcha that is NOT captured. Each genuine miss → major (the cherry-pick guard).
|
|
60
|
+
7. **Claim-truth check** — for a sample of notes, verify the note's *claim* matches the code
|
|
61
|
+
(the checker only proved the cite *resolves*). A false/misleading claim → **blocker**
|
|
62
|
+
(it violates the net-positive law: a wrong note is worse than none).
|
|
63
|
+
8. **Connectivity** — orphan notes, dangling `[[links]]`, literal `[[...]]` in prose → minor.
|
|
64
|
+
|
|
65
|
+
## Severity + decision
|
|
66
|
+
- **Blocker** — hard-gate failure, broken checker, or a wrong/misleading note. Must be **0**.
|
|
67
|
+
- **Major** — a real gap: missing essence, under-covered domain, unanswerable question.
|
|
68
|
+
- **Minor** — nits (orphan, cosmetic). Logged, not looped on.
|
|
69
|
+
- **PASS** = hard gates pass · quality score ≥ 85 · 0 blockers · majors ≤ 2. Else **ITERATE**.
|
|
70
|
+
|
|
71
|
+
## Quality score (0–100) — only meaningful once hard gates (fidelity + coverage) pass
|
|
72
|
+
| Weight | Dimension | Measure |
|
|
73
|
+
|---|---|---|
|
|
74
|
+
| 35 | Load-bearing | questions answerable / questions asked (step 5) |
|
|
75
|
+
| 25 | Coverage balance | notes spread across domains; tunneling penalty (step 4) |
|
|
76
|
+
| 25 | Net-positive density | notes adding non-obvious intelligence / total |
|
|
77
|
+
| 15 | Connectivity | 1 − (orphan + dangling) rate (step 8) |
|
|
78
|
+
|
|
79
|
+
**Non-obvious** (for net-positive density) = a competent dev *couldn't derive it from a
|
|
80
|
+
single file* — it's a cross-file contract, a flow, a scar, or a *why*. A note that restates
|
|
81
|
+
one file's content is obvious and doesn't count. Apply this test literally so two runs on
|
|
82
|
+
the same brain score it the same.
|
|
83
|
+
|
|
84
|
+
## Output → `.rafa/brain/checklist.md` (the report card)
|
|
85
|
+
|
|
86
|
+
A frontmatter **header** the conductor branches on, then a **checklist** of every criterion
|
|
87
|
+
with two columns — atlas's self-check (read from `coverage.md`) and your independent
|
|
88
|
+
verdict (a `✓/✗` split on a row is itself a finding) — then the scorecard, then findings.
|
|
89
|
+
|
|
90
|
+
```yaml
|
|
91
|
+
---
|
|
92
|
+
schemaVersion: 1 # required · contract §4 — the platform ingests this header
|
|
93
|
+
verdict: ITERATE # PASS | ITERATE ← the conductor reads this
|
|
94
|
+
round: 1
|
|
95
|
+
score: 78 # 0–100 (required int; a hard-gate fail is score 0 + verdict ITERATE)
|
|
96
|
+
gates: { fidelity: pass, coverage: pass } # each pass | fail
|
|
97
|
+
counts: { blockers: 1, majors: 2, minors: 3 } # non-negative ints
|
|
98
|
+
---
|
|
99
|
+
```
|
|
100
|
+
`checklist.md`'s frontmatter is machine-read by the platform (contract §4) and validated by
|
|
101
|
+
`rafa compile` — `schemaVersion`, `verdict`, `score`, `gates`, `counts` are all required. On a
|
|
102
|
+
hard-gate fail write `score: 0` (not `null` — the contract requires an int).
|
|
103
|
+
- **Checklist** — one row per criterion (A1, A3, B1, B2, policy, C1, C2, D1…): `checked-by`
|
|
104
|
+
(checker | atlas | prism) · atlas (self) · prism (verify) · evidence (`file:line` / artifact).
|
|
105
|
+
- **Scorecard** — the 4 graded dimensions + composite.
|
|
106
|
+
- **Findings ledger** — blocker / major / minor, each cited (`file:line`) + a suggested fix.
|
|
107
|
+
|
|
108
|
+
You write **`checklist.md` only**. The **conductor** owns `log.md` (the round-by-round trail).
|
|
109
|
+
PASS = hard gates pass · score ≥ 85 · 0 blockers · majors ≤ 2.
|
|
110
|
+
|
|
111
|
+
## The ratchet
|
|
112
|
+
Any judgment finding you raise repeatedly that *could* be mechanized → recommend it become a
|
|
113
|
+
deterministic check in `verify-citations.mjs` (as B2 did). Push checks down into the machine;
|
|
114
|
+
keep prism for what only judgment can catch.
|