@balpal4495/quorum 0.4.0 → 1.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CLAUDE.md +102 -42
- package/README.md +226 -176
- package/SETUP.md +24 -10
- package/bin/commands/advisor.js +301 -0
- package/bin/commands/commit.js +42 -52
- package/bin/commands/evolve.js +285 -0
- package/bin/commands/growth.js +139 -0
- package/bin/commands/init.js +8 -0
- package/bin/commands/sentinel.js +1 -1
- package/bin/init.js +9 -1
- package/bin/quorum.js +28 -0
- package/bin/shared/llm.js +228 -0
- package/modules/AGENTS.md +8 -0
- package/modules/CLAUDE.md +8 -2
- package/modules/README.md +72 -13
- package/modules/advisor/ask.ts +87 -0
- package/modules/advisor/index.ts +2 -0
- package/modules/advisor/prompt.ts +50 -0
- package/modules/advisor/types.ts +26 -0
- package/modules/council/deliberate.ts +15 -5
- package/modules/council/types.ts +1 -1
- package/modules/setup.ts +15 -0
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -1,158 +1,241 @@
|
|
|
1
1
|
# Quorum
|
|
2
2
|
|
|
3
|
-
**Quorum gives your AI coding assistant memory and judgment.**
|
|
3
|
+
**Quorum gives your AI coding assistant persistent memory and judgment — and keeps it getting smarter over time.**
|
|
4
4
|
|
|
5
|
-
When Claude Code, Copilot, or
|
|
5
|
+
When Claude Code, Copilot, Cursor, or Codex works in your codebase, it forgets everything between sessions. It retries approaches that already failed. It contradicts decisions made last week. It has no idea what the team has already learned.
|
|
6
6
|
|
|
7
|
-
Quorum fixes this. It installs a persistent knowledge store into your project
|
|
7
|
+
Quorum fixes this. It installs a persistent knowledge store (Chronicle) into your project, gives your AI a structured workflow for querying it before proposing solutions, validates designs before acting, and writes new knowledge back — with you approving every write.
|
|
8
8
|
|
|
9
9
|
---
|
|
10
10
|
|
|
11
11
|
## Get started in one command
|
|
12
12
|
|
|
13
|
-
Run this from your project root:
|
|
14
|
-
|
|
15
13
|
```bash
|
|
16
14
|
npx @balpal4495/quorum@latest init
|
|
17
15
|
```
|
|
18
16
|
|
|
19
|
-
Then
|
|
17
|
+
Then `npm install`. That's it.
|
|
18
|
+
|
|
19
|
+
Quorum copies its modules into `quorum/`, merges instruction files for your AI (`CLAUDE.md`, `AGENTS.md`, `.github/copilot-instructions.md`), and creates the Chronicle knowledge store at `.chronicle/`.
|
|
20
|
+
|
|
21
|
+
---
|
|
22
|
+
|
|
23
|
+
## How Quorum learns over time
|
|
24
|
+
|
|
25
|
+
This is the core loop. Every session makes the project smarter.
|
|
26
|
+
|
|
27
|
+
```
|
|
28
|
+
session start
|
|
29
|
+
└─ AI reads Chronicle (quorum advisor brief + query)
|
|
30
|
+
└─ work happens informed by accumulated knowledge
|
|
31
|
+
└─ decisions and learnings staged as proposals (oracle.propose)
|
|
32
|
+
└─ you approve from terminal (quorum commit)
|
|
33
|
+
└─ Chronicle grows
|
|
34
|
+
└─ PR merged → growth comment posted automatically
|
|
35
|
+
└─ periodic: quorum evolve consolidates + improves entries
|
|
36
|
+
```
|
|
37
|
+
|
|
38
|
+
**Session start** — the AI runs `quorum advisor brief` to see what Chronicle knows, then `quorum advisor query "topic"` to get relevant entries before touching any code.
|
|
39
|
+
|
|
40
|
+
**During work** — Oracle is queried before every significant decision. Refuted entries are treated as hard stops. Validated entries inform the approach.
|
|
41
|
+
|
|
42
|
+
**Session end** — the AI stages Chronicle proposals for every meaningful decision made. You review and commit them with `quorum commit`.
|
|
43
|
+
|
|
44
|
+
**On every PR merge** — a growth comment is posted automatically showing exactly what Chronicle learned from that PR.
|
|
45
|
+
|
|
46
|
+
**Periodically** — `quorum evolve` reviews all entries and proposes consolidations, resolves contradictions, and promotes confirmed knowledge.
|
|
20
47
|
|
|
21
|
-
|
|
48
|
+
**Visibility at any time** — `quorum growth` shows whether learning is actually happening, how fast, and what was learned recently.
|
|
22
49
|
|
|
23
50
|
---
|
|
24
51
|
|
|
25
52
|
## CLI commands
|
|
26
53
|
|
|
27
|
-
|
|
54
|
+
```bash
|
|
55
|
+
npm install -g @balpal4495/quorum
|
|
56
|
+
# or: npx @balpal4495/quorum <command>
|
|
57
|
+
```
|
|
28
58
|
|
|
29
|
-
| Command | What it does | LLM
|
|
59
|
+
| Command | What it does | LLM |
|
|
30
60
|
|---|---|---|
|
|
31
|
-
| `quorum
|
|
32
|
-
| `quorum
|
|
61
|
+
| `quorum advisor "question"` | Ask a plain-language question — answer synthesised from Chronicle evidence | Auto¹ |
|
|
62
|
+
| `quorum advisor query "topic"` | Search Chronicle entries by keyword | No |
|
|
63
|
+
| `quorum advisor brief` | High-level Chronicle summary | No |
|
|
64
|
+
| `quorum growth` | Chronicle health — growth rate, recent learnings, weekly sparkline | No |
|
|
65
|
+
| `quorum evolve` | Consolidate and improve Chronicle entries | Auto¹ |
|
|
66
|
+
| `quorum status` | Chronicle health — pending proposals, committed entries | No |
|
|
33
67
|
| `quorum check --outcome X --design Y` | Deterministic preflight + risk classifier | No |
|
|
34
68
|
| `quorum commit <id>` | Approve and index a pending proposal | No |
|
|
35
69
|
| `quorum sentinel [coverage]` | Chronicle coverage of your source files | No |
|
|
70
|
+
| `quorum init` | Scaffold Quorum into a project | No |
|
|
71
|
+
|
|
72
|
+
¹ **Auto-detect** — Quorum finds whichever LLM is available: `ANTHROPIC_API_KEY`, `OPENAI_API_KEY`, `GEMINI_API_KEY`, `OPENAI_BASE_URL`, Ollama at localhost:11434, or an authenticated `gemini` CLI. When running inside Claude Code, Copilot, Codex, or any other AI agent without a separate key, these commands output Chronicle evidence and a synthesis request directly — the agent answers inline. No key required.
|
|
73
|
+
|
|
74
|
+
---
|
|
36
75
|
|
|
37
|
-
|
|
76
|
+
## `quorum advisor` — ask Chronicle a question
|
|
38
77
|
|
|
39
78
|
```bash
|
|
40
|
-
quorum
|
|
41
|
-
|
|
42
|
-
|
|
79
|
+
quorum advisor "what did we decide about authentication?"
|
|
80
|
+
quorum advisor query "session handling" # keyword search, no LLM
|
|
81
|
+
quorum advisor brief # full Chronicle summary, no LLM
|
|
43
82
|
```
|
|
44
83
|
|
|
45
84
|
```
|
|
46
|
-
|
|
47
|
-
⚠ Sensitive areas: auth
|
|
48
|
-
✗ No rollback strategy mentioned
|
|
49
|
-
✗ No test strategy mentioned
|
|
85
|
+
Question: what did we decide about authentication?
|
|
50
86
|
|
|
51
|
-
|
|
52
|
-
|
|
53
|
-
|
|
54
|
-
|
|
55
|
-
· authentication or authorisation logic
|
|
87
|
+
What we know
|
|
88
|
+
The team settled on RS256 JWT after rejecting HS256 — key rotation without
|
|
89
|
+
invalidating active sessions was the blocker. Tokens are 15-min expiry with
|
|
90
|
+
refresh rotation in httpOnly cookies.
|
|
56
91
|
|
|
57
|
-
|
|
92
|
+
Recommendation
|
|
93
|
+
Follow the RS256 pattern. Entry [abc-123] is validated at 0.91 confidence.
|
|
94
|
+
|
|
95
|
+
Risks
|
|
96
|
+
· OAuth migration is planned for Q3 — coordinate before adding new auth surfaces
|
|
97
|
+
|
|
98
|
+
Next step
|
|
99
|
+
quorum advisor query "oauth migration" to check current status
|
|
58
100
|
```
|
|
59
101
|
|
|
60
|
-
|
|
61
|
-
Also accepts JSON on stdin: `echo '{"outcome":"…","design":"…"}' | quorum check --json`
|
|
102
|
+
Advisor validates its own answer internally — if confidence is below 0.7 or blockers exist, it retries up to 2 times with the previous answer as context before returning.
|
|
62
103
|
|
|
63
|
-
|
|
104
|
+
---
|
|
105
|
+
|
|
106
|
+
## `quorum growth` — is Chronicle actually learning?
|
|
64
107
|
|
|
65
108
|
```bash
|
|
66
|
-
quorum
|
|
109
|
+
quorum growth
|
|
110
|
+
quorum growth --json # machine-readable, for CI
|
|
67
111
|
```
|
|
68
112
|
|
|
69
113
|
```
|
|
70
|
-
Chronicle
|
|
114
|
+
Chronicle growth
|
|
115
|
+
|
|
116
|
+
Status THRIVING
|
|
117
|
+
Total entries 17
|
|
118
|
+
Last 7 days 6 commits
|
|
119
|
+
Last 30 days 17 commits
|
|
120
|
+
Last commit 0 days ago 2026-05-16
|
|
121
|
+
Pending 2 proposals awaiting quorum commit
|
|
122
|
+
|
|
123
|
+
Weekly commits
|
|
124
|
+
w/c 2026-05-11 ▪▪▪▪▪▪ 6
|
|
125
|
+
w/c 2026-05-04 ▪▪▪▪▪▪▪▪▪▪▪ 11
|
|
126
|
+
|
|
127
|
+
Recent learnings
|
|
128
|
+
bf448871 Low-risk designs skip Council entirely — Jury alone is sufficient… 2026-05-16
|
|
129
|
+
3efb1789 Advisor validates answers before returning — retries up to 2 times… 2026-05-16
|
|
130
|
+
090c7dc6 Advisor is a read-only path — never calls oracle.propose()… 2026-05-16
|
|
131
|
+
e57c30d5 Releases trigger from PR labels, not manual tag pushes… 2026-05-16
|
|
132
|
+
```
|
|
71
133
|
|
|
72
|
-
|
|
73
|
-
2 pending proposals
|
|
134
|
+
Status levels: `EMPTY` → `STALLED` (14 days with no commits) → `SLOW` (7 days) → `HEALTHY` → `THRIVING` (3+ commits this week). When stalled, it tells you exactly what to do.
|
|
74
135
|
|
|
75
|
-
|
|
76
|
-
a1b2c3d4 JWT key rotation approach needs RS256 not HS256
|
|
77
|
-
oracle/propose.ts, modules/auth/
|
|
136
|
+
---
|
|
78
137
|
|
|
79
|
-
|
|
80
|
-
|
|
138
|
+
## `quorum evolve` — Chronicle self-improvement
|
|
139
|
+
|
|
140
|
+
```bash
|
|
141
|
+
quorum evolve # analyse and stage improvement proposals
|
|
142
|
+
quorum evolve --dry-run # preview without writing
|
|
81
143
|
```
|
|
82
144
|
|
|
83
|
-
|
|
145
|
+
```
|
|
146
|
+
Quorum evolve 17 entries · via Anthropic
|
|
84
147
|
|
|
85
|
-
|
|
86
|
-
|
|
87
|
-
|
|
88
|
-
|
|
148
|
+
✓ Analysis complete
|
|
149
|
+
|
|
150
|
+
2 improvements found
|
|
151
|
+
|
|
152
|
+
✓ consolidate 10b848a2 + d93b6f40
|
|
153
|
+
Both entries describe Mermaid rendering failures — distinct symptoms, same root cause
|
|
154
|
+
→ Mermaid diagrams have three known failure modes in GitHub PR descriptions…
|
|
155
|
+
|
|
156
|
+
✓ promote 55278b3d → validated (0.88)
|
|
157
|
+
Confirmed by three subsequent entries referencing SUMMARY.md temporal context
|
|
158
|
+
|
|
159
|
+
2 proposals staged — run quorum commit --list to review
|
|
89
160
|
```
|
|
90
161
|
|
|
91
|
-
|
|
162
|
+
Three improvement types:
|
|
163
|
+
|
|
164
|
+
- **consolidate** — two entries covering the same ground → one sharper entry with `supersedes`
|
|
165
|
+
- **resolve** — a validated entry contradicted by a newer one → mark it `refuted`
|
|
166
|
+
- **promote** — an `open` entry confirmed by later entries → elevate to `validated`
|
|
167
|
+
|
|
168
|
+
Every proposed improvement goes through the human gate (`quorum commit`). Nothing is auto-committed.
|
|
92
169
|
|
|
93
|
-
|
|
170
|
+
---
|
|
171
|
+
|
|
172
|
+
## `quorum check` — instant risk triage
|
|
94
173
|
|
|
95
174
|
```bash
|
|
96
|
-
quorum
|
|
175
|
+
quorum check \
|
|
176
|
+
--outcome "migrate auth from sessions to JWT" \
|
|
177
|
+
--design "replace session middleware with HS256 tokens on all routes"
|
|
97
178
|
```
|
|
98
179
|
|
|
99
180
|
```
|
|
100
|
-
|
|
101
|
-
|
|
102
|
-
|
|
181
|
+
Preflight
|
|
182
|
+
⚠ Sensitive areas: auth
|
|
183
|
+
✗ No rollback strategy mentioned
|
|
184
|
+
✗ No test strategy mentioned
|
|
103
185
|
|
|
104
|
-
|
|
105
|
-
|
|
106
|
-
|
|
186
|
+
Risk
|
|
187
|
+
Level: CRITICAL
|
|
188
|
+
Council mode: full
|
|
189
|
+
Reasons:
|
|
190
|
+
· authentication or authorisation logic
|
|
107
191
|
|
|
108
|
-
|
|
109
|
-
✗ council/chairman.ts
|
|
110
|
-
✗ jury/evaluate.ts
|
|
111
|
-
…
|
|
192
|
+
⚠ Critical risk — human architecture review required before proceeding.
|
|
112
193
|
```
|
|
113
194
|
|
|
114
|
-
|
|
115
|
-
|
|
116
|
-
## Then just talk to your AI
|
|
195
|
+
Exit codes: `0` = low/medium, `1` = high, `2` = critical — pipe directly into CI scripts.
|
|
196
|
+
Also accepts JSON on stdin: `echo '{"outcome":"…","design":"…"}' | quorum check --json`
|
|
117
197
|
|
|
118
|
-
|
|
198
|
+
---
|
|
119
199
|
|
|
120
|
-
|
|
200
|
+
## `quorum commit` — the human gate
|
|
121
201
|
|
|
122
|
-
|
|
202
|
+
```bash
|
|
203
|
+
quorum commit --list # see all pending proposals
|
|
204
|
+
quorum commit a1b2c3d4 # approve and index (partial ID prefix works)
|
|
205
|
+
quorum commit a1b2c3d4 --dry-run # preview without writing
|
|
206
|
+
```
|
|
123
207
|
|
|
124
|
-
|
|
125
|
-
- Claude Code (`claude` CLI or VS Code extension)
|
|
126
|
-
- GitHub Copilot (agent mode)
|
|
127
|
-
- Cursor
|
|
128
|
-
- Any other AI that can read files and run terminal commands
|
|
208
|
+
Writes to `.chronicle/committed/`, updates `SUMMARY.md`, removes the proposal. Always works — no extra dependencies required. Install `@xenova/transformers` and `vectordb` to also embed and index in the vector store for semantic search.
|
|
129
209
|
|
|
130
210
|
---
|
|
131
211
|
|
|
132
212
|
## What changes after setup
|
|
133
213
|
|
|
134
|
-
### Your AI
|
|
214
|
+
### Your AI starts every session with full project context
|
|
135
215
|
|
|
136
|
-
Before
|
|
216
|
+
Before touching any code, your AI reads Chronicle:
|
|
217
|
+
|
|
218
|
+
```bash
|
|
219
|
+
quorum advisor brief # what has the project learned?
|
|
220
|
+
quorum advisor query "topic of the work" # what's relevant to today's task?
|
|
221
|
+
```
|
|
137
222
|
|
|
138
223
|
> *"I queried Chronicle before proposing the Redis session approach. Entry `[abc-123]` shows we rejected this in March — key rotation wasn't viable. I'm proposing JWT with RS256 instead."*
|
|
139
224
|
|
|
140
|
-
###
|
|
225
|
+
### Designs are validated before they reach you
|
|
141
226
|
|
|
142
|
-
Every proposal goes through Jury (confidence scoring against evidence) and Council (adversarial panel
|
|
227
|
+
Every proposal goes through Jury (confidence scoring against evidence) and Council (adversarial panel) before it surfaces. Low-confidence or contested ideas get challenged internally first.
|
|
143
228
|
|
|
144
|
-
> *"Jury scored this 0.41 — gaps in lock strategy and rollback plan. Council flagged the same
|
|
229
|
+
> *"Jury scored this 0.41 — gaps in lock strategy and rollback plan. Council flagged the same. I've revised the migration to use a shadow column approach before bringing it to you."*
|
|
145
230
|
|
|
146
231
|
### You approve what gets remembered
|
|
147
232
|
|
|
148
|
-
When a decision is made, your AI stages a Chronicle entry using `oracle.propose()`. You approve it from the terminal:
|
|
149
|
-
|
|
150
233
|
```bash
|
|
151
234
|
quorum commit --list # see what's pending
|
|
152
235
|
quorum commit <id> # approve and index
|
|
153
236
|
```
|
|
154
237
|
|
|
155
|
-
Nothing is indexed without your
|
|
238
|
+
Nothing is indexed without your sign-off.
|
|
156
239
|
|
|
157
240
|
```
|
|
158
241
|
.chronicle/
|
|
@@ -161,13 +244,31 @@ Nothing is indexed without your explicit sign-off.
|
|
|
161
244
|
SUMMARY.md ← auto-generated weekly context for your AI to read
|
|
162
245
|
```
|
|
163
246
|
|
|
164
|
-
Commit `.chronicle/committed/` to git.
|
|
247
|
+
Commit `.chronicle/committed/` to git. Every future session — yours and your teammates' — starts with that context.
|
|
248
|
+
|
|
249
|
+
### Every merged PR shows what Chronicle learned
|
|
250
|
+
|
|
251
|
+
Quorum ships two GitHub Actions workflows. Enable them by copying `.github/workflows/` from the [Quorum repo](https://github.com/balpal4495/Quorum):
|
|
252
|
+
|
|
253
|
+
**`chronicle-on-merge.yml`** — fires on every PR merge. Creates a Chronicle proposal from the PR metadata and posts a growth comment:
|
|
254
|
+
|
|
255
|
+
```
|
|
256
|
+
## Quorum Chronicle — what this PR taught
|
|
257
|
+
|
|
258
|
+
Chronicle grew from 14 → 17 entries
|
|
165
259
|
|
|
166
|
-
|
|
260
|
+
Committed this PR:
|
|
261
|
+
✅ [bf448871] Low-risk designs skip Council entirely — jury-only, 0 LLM calls
|
|
262
|
+
✅ [3efb1789] Advisor validates answers before returning — retries up to 2 times
|
|
263
|
+
✅ [090c7dc6] Advisor is a read-only path — never calls oracle.propose()
|
|
167
264
|
|
|
168
|
-
|
|
265
|
+
2 proposals pending — run quorum commit --list to review.
|
|
169
266
|
|
|
170
|
-
|
|
267
|
+
---
|
|
268
|
+
Run quorum growth for full Chronicle health · quorum evolve to consolidate entries
|
|
269
|
+
```
|
|
270
|
+
|
|
271
|
+
**`sentinel-pr.yml`** — posts a coverage table and Mermaid heatmap on every PR showing which files Chronicle knows about and which are blind spots.
|
|
171
272
|
|
|
172
273
|
---
|
|
173
274
|
|
|
@@ -180,7 +281,7 @@ Your AI is about to propose symmetric JWT signing. Oracle returns:
|
|
|
180
281
|
```
|
|
181
282
|
[abc-123] Tried HS256 JWT in March. Rejected — no way to rotate keys without
|
|
182
283
|
invalidating all active sessions. Decision: RS256 with short-lived tokens.
|
|
183
|
-
status:
|
|
284
|
+
status: validated · confidence: 0.91
|
|
184
285
|
```
|
|
185
286
|
|
|
186
287
|
Jury flags it as a direct conflict. The agent revises before Council even sees it.
|
|
@@ -192,16 +293,16 @@ Jury flags it as a direct conflict. The agent revises before Council even sees i
|
|
|
192
293
|
Day one of a new Claude Code session. Before touching anything:
|
|
193
294
|
|
|
194
295
|
```
|
|
195
|
-
|
|
296
|
+
quorum advisor query "authentication, session handling, token strategy"
|
|
196
297
|
|
|
197
298
|
6 entries found:
|
|
198
|
-
|
|
199
|
-
|
|
200
|
-
|
|
201
|
-
|
|
299
|
+
· HS256 rejected (key rotation problem) → use RS256
|
|
300
|
+
· Redis sessions tried and removed (memory overhead at scale)
|
|
301
|
+
· Current: RS256 JWT, 15-min expiry, refresh rotation in httpOnly cookies
|
|
302
|
+
· Upcoming: OAuth migration planned for Q3
|
|
202
303
|
```
|
|
203
304
|
|
|
204
|
-
|
|
305
|
+
Full project context from the first message — no archaeology through git history.
|
|
205
306
|
|
|
206
307
|
---
|
|
207
308
|
|
|
@@ -215,24 +316,7 @@ gaps: ["no lock strategy documented", "no rollback plan"]
|
|
|
215
316
|
council_brief: challenge
|
|
216
317
|
```
|
|
217
318
|
|
|
218
|
-
Council
|
|
219
|
-
|
|
220
|
-
```json
|
|
221
|
-
{
|
|
222
|
-
"satisfied": false,
|
|
223
|
-
"blockers": [
|
|
224
|
-
{
|
|
225
|
-
"issue": "Naive ALTER TABLE takes an exclusive lock for minutes on a 50M-row table",
|
|
226
|
-
"evidence": ["db-017"],
|
|
227
|
-
"required_fix": "Use shadow column pattern or pg_repack. Add rollback path."
|
|
228
|
-
}
|
|
229
|
-
],
|
|
230
|
-
"warnings": [],
|
|
231
|
-
"advisor_split": { "proceed": 0, "redesign": 4, "investigate-more": 1 }
|
|
232
|
-
}
|
|
233
|
-
```
|
|
234
|
-
|
|
235
|
-
The agent revises the plan. You approve the Chronicle entry once it's solid. The reasoning — including alternatives considered and why they were rejected — is on record for the next time someone touches that table:
|
|
319
|
+
Council gives a structured verdict with blockers that must be resolved before proceeding. The agent revises. You approve the Chronicle entry once it's solid — including alternatives considered and why they were rejected — so the next person touching that table has the full reasoning:
|
|
236
320
|
|
|
237
321
|
```json
|
|
238
322
|
{
|
|
@@ -240,8 +324,7 @@ The agent revises the plan. You approve the Chronicle entry once it's solid. The
|
|
|
240
324
|
"alternatives_considered": ["naive ALTER TABLE", "pg_repack"],
|
|
241
325
|
"rejected_reason": ["ALTER TABLE takes exclusive lock for minutes on 50M rows"],
|
|
242
326
|
"scope": ["database", "migrations"],
|
|
243
|
-
"
|
|
244
|
-
"validation_plan": ["Confirm 100% backfill before applying NOT NULL constraint", "Test rollback path on staging"],
|
|
327
|
+
"validation_plan": ["Confirm 100% backfill before applying NOT NULL constraint"],
|
|
245
328
|
"review_after": "2026-08-01"
|
|
246
329
|
}
|
|
247
330
|
```
|
|
@@ -250,14 +333,15 @@ The agent revises the plan. You approve the Chronicle entry once it's solid. The
|
|
|
250
333
|
|
|
251
334
|
## What's inside
|
|
252
335
|
|
|
253
|
-
|
|
336
|
+
Five portable TypeScript modules installed into `quorum/modules/`:
|
|
254
337
|
|
|
255
|
-
| Module | What it does |
|
|
256
|
-
|
|
257
|
-
| **
|
|
258
|
-
| **
|
|
259
|
-
| **
|
|
260
|
-
| **
|
|
338
|
+
| Module | What it does | LLM |
|
|
339
|
+
|---|---|---|
|
|
340
|
+
| **Advisor** | Plain-language interface to Chronicle. Ask a question, get a concise answer synthesised from evidence, validated with an internal retry loop. | Yes |
|
|
341
|
+
| **Oracle** | Query and write interface to Chronicle. Two-pass retrieval (vector + BM25). | No |
|
|
342
|
+
| **Jury** | Evaluates a design against Chronicle evidence. Four-dimension confidence score, deterministic preflight, hard-blocker gaps. | Yes |
|
|
343
|
+
| **Council** | Adversarial panel — advisors challenge independently, reviewers critique anonymously, Chairman gives a structured verdict. Risk-scaled fan-out. | Yes |
|
|
344
|
+
| **Sentinel** | Coverage reporting (which files Chronicle knows about), drift detection (are entries still accurate), PR coverage maps. | Optional |
|
|
261
345
|
|
|
262
346
|
The modules live in your repo — readable by any AI working in the codebase. Nothing is hidden in `node_modules`.
|
|
263
347
|
|
|
@@ -265,9 +349,9 @@ The modules live in your repo — readable by any AI working in the codebase. No
|
|
|
265
349
|
|
|
266
350
|
## How Jury works
|
|
267
351
|
|
|
268
|
-
Before calling the LLM, Jury runs a **deterministic preflight**
|
|
352
|
+
Before calling the LLM, Jury runs a **deterministic preflight** that checks whether the design touches sensitive areas (auth, database migrations, crypto, payments, PII, secrets), mentions a rollback strategy, and whether any refuted Chronicle entries conflict with the design. These facts are injected into the prompt as hard ground truth.
|
|
269
353
|
|
|
270
|
-
The LLM
|
|
354
|
+
The LLM scores the design across four dimensions:
|
|
271
355
|
|
|
272
356
|
| Dimension | What it measures |
|
|
273
357
|
|---|---|
|
|
@@ -276,68 +360,42 @@ The LLM then scores the design across four dimensions:
|
|
|
276
360
|
| Risk | How well does the design address known failure modes? |
|
|
277
361
|
| Completeness | Does the design cover the full outcome? |
|
|
278
362
|
|
|
279
|
-
Confidence is recomputed as the exact average
|
|
363
|
+
Confidence is recomputed as the exact average — the LLM's stated value is discarded. Jury separates `blocking_gaps` (must resolve before proceeding) from `gaps` (useful but not critical).
|
|
280
364
|
|
|
281
365
|
---
|
|
282
366
|
|
|
283
367
|
## How Council works
|
|
284
368
|
|
|
285
|
-
|
|
369
|
+
A **risk classifier** runs before the panel and scales fan-out accordingly:
|
|
286
370
|
|
|
287
|
-
| Risk | Council mode | LLM calls |
|
|
288
|
-
|
|
289
|
-
| Low |
|
|
290
|
-
| Medium | 1 advisor + 2 reviewers | 5 |
|
|
291
|
-
| High | 5 advisors + 5 reviewers | 12 |
|
|
292
|
-
| Critical |
|
|
371
|
+
| Risk | Triggers | Council mode | LLM calls |
|
|
372
|
+
|---|---|---|---|
|
|
373
|
+
| Low | Nothing sensitive | jury-only — Council skipped entirely | 0 |
|
|
374
|
+
| Medium | Cache, queues, deployments | lite — 1 advisor + 2 reviewers | 5 |
|
|
375
|
+
| High | DB migrations, PII, permissions | full — 5 advisors + 5 reviewers | 12 |
|
|
376
|
+
| Critical | Auth, payments, crypto, data deletion | full + human flag | 12 |
|
|
293
377
|
|
|
294
|
-
|
|
378
|
+
Refuted entries in the evidence pack always elevate risk by at least one level.
|
|
295
379
|
|
|
296
|
-
The Chairman's verdict is
|
|
297
|
-
|
|
298
|
-
```json
|
|
299
|
-
{
|
|
300
|
-
"blockers": [
|
|
301
|
-
{
|
|
302
|
-
"issue": "No rollback plan for destructive migration",
|
|
303
|
-
"evidence": ["db-017"],
|
|
304
|
-
"required_fix": "Add shadow-column migration and rollback path before execution"
|
|
305
|
-
}
|
|
306
|
-
],
|
|
307
|
-
"warnings": [
|
|
308
|
-
{
|
|
309
|
-
"issue": "No integration test for token expiry edge case",
|
|
310
|
-
"suggested_fix": "Add test covering token rotation during concurrent requests"
|
|
311
|
-
}
|
|
312
|
-
],
|
|
313
|
-
"advisor_split": { "proceed": 2, "redesign": 2, "investigate-more": 1 }
|
|
314
|
-
}
|
|
315
|
-
```
|
|
316
|
-
|
|
317
|
-
Blockers must be resolved before the human gate. Warnings can be ticketed. High `advisor_split` disagreement is surfaced explicitly — it means genuine uncertainty, not a safe proceed.
|
|
318
|
-
|
|
319
|
-
Every Oracle ID cited in the verdict is also validated against the evidence pack that was actually sent. Hallucinated citations are flagged in `citation_validation.hallucinated_ids` and stripped from the Chronicle proposal.
|
|
380
|
+
The Chairman's verdict is structured with `blockers` (must resolve), `warnings` (should address), `advisor_split` (shows disagreement), and `citation_validation` (hallucinated Oracle IDs are stripped before the Chronicle proposal is written).
|
|
320
381
|
|
|
321
382
|
---
|
|
322
383
|
|
|
323
384
|
## Eval suite
|
|
324
385
|
|
|
325
|
-
`evals/` contains canonical test cases — known-bad proposals that
|
|
386
|
+
`evals/` contains canonical test cases — known-bad proposals that should block, known-good ones that should pass:
|
|
326
387
|
|
|
327
|
-
| Case | Expected
|
|
388
|
+
| Case | Expected |
|
|
328
389
|
|---|---|
|
|
329
390
|
| Naive NOT NULL migration on large table | Block — no lock strategy |
|
|
330
|
-
| HS256 JWT when RS256 was already chosen | Block — cites refuted entry
|
|
391
|
+
| HS256 JWT when RS256 was already chosen | Block — cites refuted entry |
|
|
331
392
|
| PII fields logged to stdout | Block — GDPR violation in evidence |
|
|
332
393
|
| Payment charge without idempotency key | Block — duplicate charge risk |
|
|
333
|
-
| Redis sessions (previously removed) | Block — memory overhead already documented |
|
|
334
|
-
| Cache without stampede protection | Block — prior incident in Chronicle |
|
|
335
394
|
| Safe internal rename | Proceed — low risk, no conflicts |
|
|
336
|
-
| RS256 JWT (approved pattern) | Proceed — matches validated
|
|
395
|
+
| RS256 JWT (approved pattern) | Proceed — matches validated entry |
|
|
337
396
|
| Migration with rollback + shadow column | Proceed — addresses documented failure mode |
|
|
338
|
-
| Novel WebSocket design, no evidence | Investigate-more — no Chronicle evidence either way |
|
|
339
397
|
|
|
340
|
-
Deterministic assertions
|
|
398
|
+
Deterministic assertions run on every CI pass. LLM assertions activate with `EVAL_LLM=1`.
|
|
341
399
|
|
|
342
400
|
```bash
|
|
343
401
|
npx vitest run evals/
|
|
@@ -347,49 +405,41 @@ npx vitest run evals/
|
|
|
347
405
|
|
|
348
406
|
## Sentinel — coverage and drift
|
|
349
407
|
|
|
350
|
-
Sentinel surfaces two things Chronicle can't tell you about itself.
|
|
351
|
-
|
|
352
408
|
**Coverage** — which parts of your codebase has the AI never documented?
|
|
353
409
|
|
|
354
410
|
```bash
|
|
355
|
-
quorum sentinel coverage --path src
|
|
356
|
-
quorum sentinel coverage --json
|
|
411
|
+
quorum sentinel coverage --path src
|
|
412
|
+
quorum sentinel coverage --json
|
|
357
413
|
```
|
|
358
414
|
|
|
359
|
-
**Drift** —
|
|
360
|
-
|
|
361
|
-
Add `sentinel-pr.yml` (included in `quorum/`) to your GitHub Actions and every PR gets a comment showing a full-project coverage table and a colour-coded heatmap. Changed modules are highlighted. Reviewers see exactly where knowledge is solid and where it goes dark.
|
|
415
|
+
**Drift** — are existing Chronicle entries still accurate? Requires an LLM; use `sentinelAssertions({ llm })` in your test suite.
|
|
362
416
|
|
|
363
417
|
---
|
|
364
418
|
|
|
365
419
|
## For custom agent pipelines
|
|
366
420
|
|
|
367
|
-
|
|
421
|
+
Wire the modules directly into any TypeScript agent:
|
|
368
422
|
|
|
369
423
|
```typescript
|
|
370
424
|
import { setup } from "./quorum/modules/setup"
|
|
371
425
|
|
|
372
|
-
const { oracle, evaluate, deliberate } = await setup({ llm: myLLMProvider })
|
|
426
|
+
const { oracle, evaluate, deliberate, ask } = await setup({ llm: myLLMProvider })
|
|
427
|
+
|
|
428
|
+
// Ask a plain-language question
|
|
429
|
+
const answer = await ask("what did the team decide about authentication?")
|
|
373
430
|
|
|
431
|
+
// Full evaluation pipeline
|
|
374
432
|
const evidence = await oracle.query("authentication patterns")
|
|
375
433
|
const jury = await evaluate({ outcome, design, evidence })
|
|
376
434
|
const verdict = await deliberate({ outcome, design, evidence, jury_output: jury })
|
|
377
435
|
```
|
|
378
436
|
|
|
379
|
-
The `LLMProvider` type is a simple function — wire OpenAI, Anthropic, or anything else:
|
|
380
|
-
|
|
381
437
|
```typescript
|
|
382
|
-
//
|
|
383
|
-
const llm = async (messages, model = "claude-3-5-sonnet-20241022") => {
|
|
438
|
+
// Wire any LLM provider
|
|
439
|
+
const llm: LLMProvider = async (messages, model = "claude-3-5-sonnet-20241022") => {
|
|
384
440
|
const res = await anthropic.messages.create({ model, messages, max_tokens: 2048 })
|
|
385
441
|
return res.content[0].type === "text" ? res.content[0].text : ""
|
|
386
442
|
}
|
|
387
|
-
|
|
388
|
-
// OpenAI
|
|
389
|
-
const llm = async (messages, model = "gpt-4o") => {
|
|
390
|
-
const res = await openai.chat.completions.create({ model, messages })
|
|
391
|
-
return res.choices[0].message.content ?? ""
|
|
392
|
-
}
|
|
393
443
|
```
|
|
394
444
|
|
|
395
445
|
Full API reference: [modules/README.md](modules/README.md)
|
|
@@ -398,13 +448,13 @@ Full API reference: [modules/README.md](modules/README.md)
|
|
|
398
448
|
|
|
399
449
|
## Releases
|
|
400
450
|
|
|
401
|
-
|
|
451
|
+
Published as `@balpal4495/quorum`. Releases trigger automatically on PR merge via label (`release:patch`, `release:minor`, `release:major`) — GitHub Actions bumps the version, tags, and publishes via OIDC Trusted Publishing.
|
|
402
452
|
|
|
403
453
|
---
|
|
404
454
|
|
|
405
455
|
## Docs
|
|
406
456
|
|
|
407
|
-
- [SETUP.md](SETUP.md) — full bootstrap sequence (
|
|
457
|
+
- [SETUP.md](SETUP.md) — full bootstrap sequence (point your AI at this)
|
|
408
458
|
- [modules/README.md](modules/README.md) — TypeScript API reference
|
|
409
459
|
- [modules/AGENTS.md](modules/AGENTS.md) — file ownership map
|
|
410
460
|
- [modules/CLAUDE.md](modules/CLAUDE.md) — design decisions and invariants
|