the-high-council 0.8.1 → 0.8.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CONTRIBUTING.md +32 -5
- package/README.md +191 -91
- package/chains/advise-premium.json +2 -3
- package/chains/advise-single-astra.json +2 -3
- package/chains/advise-single-deepseek.json +4 -5
- package/chains/advise-single-gemini.json +2 -3
- package/chains/advise-single-glm-flash.json +40 -0
- package/chains/advise-single-opus.json +1 -2
- package/chains/advise-single-qwen.json +38 -0
- package/chains/advise-single.json +2 -3
- package/chains/advise-standard.json +2 -3
- package/chains/cheap-7-v2.json +5 -0
- package/chains/mock-advise-deep.json +1 -2
- package/chains/mock-advise-quick.json +1 -2
- package/chains/mock-advise-single.json +1 -2
- package/chains/mock-advise-standard.json +1 -2
- package/chains/plan-daily-7.json +175 -36
- package/chains/plan-debate-roles-c1.json +4 -0
- package/chains/plan-highest-7.json +163 -18
- package/chains/plan-lanes-4.json +95 -0
- package/chains/plan-open-7.json +5 -0
- package/chains/plan-premium-7.json +5 -0
- package/chains/verify.json +1 -1
- package/package.json +1 -1
- package/schemas/contract-v1.json +71 -0
- package/schemas/gate-ledger-v1.json +16 -1
- package/schemas/report-v1.json +95 -7
- package/schemas/spend-reservations-v1.json +23 -0
- package/src/advice-guards.js +12 -6
- package/src/advice-retention.json +38 -0
- package/src/answer-back.js +90 -0
- package/src/api.js +1 -1
- package/src/canary.js +12 -2
- package/src/chain-lint.js +75 -7
- package/src/chain-name.js +19 -1
- package/src/chain.js +367 -126
- package/src/claims.js +2 -1
- package/src/cli.js +272 -72
- package/src/context-files.js +6 -0
- package/src/contract-cli.js +183 -0
- package/src/contract-draft.js +50 -0
- package/src/contract-lint.js +101 -0
- package/src/contract-record.js +509 -0
- package/src/cost.js +40 -12
- package/src/council-replay.js +2 -1
- package/src/criteria-kinds.js +11 -1
- package/src/criteria-ledger.js +64 -3
- package/src/debate-order.js +48 -0
- package/src/draft-disputes.js +16 -4
- package/src/dry-run.js +23 -4
- package/src/errors.js +7 -1
- package/src/gate-cli.js +59 -4
- package/src/gate-ledger.js +8 -5
- package/src/gate.js +44 -4
- package/src/handoff-contract-text.js +42 -0
- package/src/handoff-from-run.js +30 -4
- package/src/handoff-prompt.js +14 -0
- package/src/held-roles.js +20 -0
- package/src/lanes.js +21 -0
- package/src/mcp/advice.js +11 -8
- package/src/mcp/contract.js +72 -0
- package/src/mcp/server.js +117 -50
- package/src/mcp/untrusted.js +7 -2
- package/src/metrics.js +3 -1
- package/src/milestones.js +127 -0
- package/src/objection-ids.js +32 -0
- package/src/policy.js +14 -5
- package/src/post-quotes.js +56 -0
- package/src/pricing.json +5 -0
- package/src/providers.js +110 -9
- package/src/reasoning-table.json +26 -24
- package/src/reasoning.js +11 -0
- package/src/report-shape.js +37 -3
- package/src/resume-brief.js +14 -4
- package/src/roles.js +273 -4
- package/src/run-budget.js +4 -1
- package/src/run-files.js +110 -0
- package/src/run-status.js +6 -1
- package/src/seeded-shuffle.js +26 -0
- package/src/send-path-refusals.js +16 -5
- package/src/send-path.js +19 -10
- package/src/send-profiles.js +5 -3
- package/src/spend-reservations.js +79 -0
- package/src/spend.js +7 -0
- package/src/stage-contract.js +23 -4
- package/src/stage-submission.js +8 -4
- package/src/text-labels.js +81 -0
- package/src/thin-contract-check.js +88 -0
- package/src/thin-contract.js +60 -0
- package/src/tools.js +15 -1
- package/src/unpriced.js +18 -0
- package/chains/cheap-7.json +0 -148
- package/chains/cheap.json +0 -53
- package/chains/gp-judge-v1.json +0 -109
- package/chains/idea-open-c2.json +0 -100
- package/chains/plan-auto-mistral.json +0 -131
- package/chains/plan-auto.json +0 -136
- package/chains/plan-cheap.json +0 -85
- package/chains/plan-debate-c2.json +0 -91
- package/chains/plan-debate-open-c2.json +0 -92
- package/chains/plan-debate.json +0 -121
- package/chains/plan-fast.json +0 -66
- package/chains/plan-proposals-best-of-5.json +0 -101
- package/chains/plan-proposals.json +0 -99
- package/chains/plan-relay.json +0 -86
- package/chains/plan-thorough.json +0 -85
- package/chains/plan-two-strong.json +0 -110
- package/chains/plan-unanimous.json +0 -85
- package/chains/seven-cheap.json +0 -88
- package/chains/seven.json +0 -81
- package/chains/smo-v2-cheap.json +0 -85
- package/chains/smo.json +0 -72
package/CONTRIBUTING.md
CHANGED
|
@@ -1,11 +1,11 @@
|
|
|
1
1
|
# Contributing
|
|
2
2
|
|
|
3
3
|
This project has one maintainer. The bottleneck is reviewing pull requests, not writing code -
|
|
4
|
-
an unscoped PR costs more of that time than it saves. The
|
|
4
|
+
an unscoped PR costs more of that time than it saves. The four shapes below are pre-scoped and
|
|
5
5
|
easy to land; a PR outside them is welcome but will take longer to review, since it needs a scope
|
|
6
6
|
discussion first.
|
|
7
7
|
|
|
8
|
-
## The
|
|
8
|
+
## The four smallest landable contributions
|
|
9
9
|
|
|
10
10
|
1. **A new provider adapter** (`src/providers.js`). Every provider except `mock`/`external` speaks
|
|
11
11
|
either the Anthropic Messages API or the OpenAI chat-completions shape via `OPENAI_COMPAT`.
|
|
@@ -16,7 +16,7 @@ discussion first.
|
|
|
16
16
|
2. **A new chain config** (`chains/*.json`). A chain is data, not code - it names which seats run
|
|
17
17
|
which stages. `council doctor --chain chains/<yours>.json` lints it (the same rules every run
|
|
18
18
|
applies, including the denied-model and `key-host` rules), and `council doctor` prices it
|
|
19
|
-
worst-case, both with zero network calls; run both against your new file before opening a PR. Look at `chains/
|
|
19
|
+
worst-case, both with zero network calls; run both against your new file before opening a PR. Look at `chains/cheap-7-v2.json` or `chains/verify.json` as a starting
|
|
20
20
|
shape depending on whether you want a debate-and-panel chain or a single-critic chain.
|
|
21
21
|
|
|
22
22
|
3. **A new mock fixture** (`src/providers.js`'s `callMock`, exercised from `test/*.test.js`). If
|
|
@@ -26,6 +26,30 @@ discussion first.
|
|
|
26
26
|
`mock-provider-error` - for the shape: a model name that triggers one deterministic scripted
|
|
27
27
|
reply, documented with which real-world failure it stands in for.
|
|
28
28
|
|
|
29
|
+
4. **A change to a skill** (`skills/<name>/SKILL.md`). The bar is in [`skills/README.md`](skills/README.md):
|
|
30
|
+
a good proposal names the failure it prevents, where that failure was seen, and what the rule
|
|
31
|
+
should say. Skills are generic and public, so a change also passes the same leftover-content
|
|
32
|
+
check every skill shipped through: no project names, internal paths, private model or session
|
|
33
|
+
names, or third-party text copied in.
|
|
34
|
+
|
|
35
|
+
## Before you open a PR
|
|
36
|
+
|
|
37
|
+
These checks run in `npm test`, so they will fail a PR rather than surprise it in review:
|
|
38
|
+
|
|
39
|
+
- **No xAI/Grok seat in any chain.** Chain lint's `denied-model` rule refuses it, with no override.
|
|
40
|
+
- **A shipped chain's seats reason at high.** Chain lint's `reasoning-not-high` rule refuses a
|
|
41
|
+
setting that lowers a seat's reasoning effort.
|
|
42
|
+
- **Prompt wording lives in `src/roles.js` only.** Changing it fails `test/prompt-pin.test.js` on
|
|
43
|
+
purpose; say in the PR what the prompt now tells the model and why, and the maintainer re-records
|
|
44
|
+
the pin.
|
|
45
|
+
- **Documents that list what the repo holds are checked against it.** `skills/README.md`,
|
|
46
|
+
`docs/skills.html` and the landing page (`docs/index.html`) are compared with the repo by tests;
|
|
47
|
+
a failing one names the document to update.
|
|
48
|
+
- **Run output is never committed.** `runs/` is gitignored; keep it that way.
|
|
49
|
+
|
|
50
|
+
Two stances, where a PR is closed rather than scoped: bring-your-own-key only (no default key,
|
|
51
|
+
proxy or hosted path), and no claim about output quality until something has been measured.
|
|
52
|
+
|
|
29
53
|
## Everything else
|
|
30
54
|
|
|
31
55
|
Bug fixes, docs fixes, and typo fixes are always welcome with no scoping step. Anything larger -
|
|
@@ -35,7 +59,10 @@ from a diff.
|
|
|
35
59
|
|
|
36
60
|
## Running the tests
|
|
37
61
|
|
|
38
|
-
npm test
|
|
62
|
+
npm test # the whole suite
|
|
63
|
+
node --test test/chain.test.js # one file
|
|
64
|
+
node --test --test-name-pattern 'parseJson: raw newline' # one test
|
|
39
65
|
|
|
40
|
-
Offline, no API key, no network call, no cost.
|
|
66
|
+
Offline, no API key, no network call, no cost. The whole suite is over 2,000 tests and takes
|
|
67
|
+
several minutes; run the files you touched first. `node --test` matches the project's own test
|
|
41
68
|
runner - no additional test framework to install.
|
package/README.md
CHANGED
|
@@ -1,16 +1,51 @@
|
|
|
1
|
+
<div align="center">
|
|
2
|
+
|
|
3
|
+
<img src="https://raw.githubusercontent.com/muad-yasin/the-high-council-mcp/master/docs/logo-diamond.png" alt="Sower Industries diamond logo" width="120">
|
|
4
|
+
|
|
1
5
|
# The High Council
|
|
2
6
|
|
|
3
|
-
|
|
4
|
-
|
|
5
|
-
|
|
7
|
+
Several AI models from different labs argue about your idea, then debate on Software Engineering choices.<br>
|
|
8
|
+
The goal is to get everyone to work together, in harmony and with manners. Effectiveness is unproven, but it is a fun way to spend your tokens!
|
|
9
|
+
|
|
10
|
+
[](https://www.npmjs.com/package/the-high-council)
|
|
11
|
+
[](https://registry.modelcontextprotocol.io/v0/servers?search=io.github.muad-yasin/the-high-council)
|
|
12
|
+
[](LICENSE)
|
|
13
|
+
|
|
14
|
+
[Website](https://sower-industries.de/en/THC/) · [Releases](https://github.com/muad-yasin/the-high-council-mcp/releases) · [Changelog](CHANGELOG.md) · [Known limits](#known-limits-stated-plainly)
|
|
15
|
+
|
|
16
|
+
<img src="https://raw.githubusercontent.com/muad-yasin/the-high-council-mcp/master/media/council-water-collector.gif" width="480" alt="The council chamber from the project website: the debate graph of a real recorded run (a water-collector design), with the Claude seat in the middle.">
|
|
17
|
+
|
|
18
|
+
</div>
|
|
19
|
+
|
|
20
|
+
## Try it in 30 seconds
|
|
21
|
+
|
|
22
|
+
npx the-high-council council demo
|
|
23
|
+
|
|
24
|
+
No keys, no network, $0. Add it to Claude Code as an MCP server:
|
|
25
|
+
|
|
26
|
+
claude mcp add council -- npx -y the-high-council council --mcp
|
|
27
|
+
|
|
28
|
+
For Claude Desktop, download the `.mcpb` bundle (`the-high-council-<version>.mcpb`) from [Releases](https://github.com/muad-yasin/the-high-council-mcp/releases/latest).
|
|
29
|
+
|
|
30
|
+
## Contents
|
|
31
|
+
|
|
32
|
+
- [Your first ten minutes](#your-first-ten-minutes) · [Words used here](#words-used-here)
|
|
33
|
+
- [How it works](#how-it-works) · [Reading a run with a program](#reading-a-run-with-a-program)
|
|
34
|
+
- [Requirements](#requirements) · [Setup](#setup) · [Quick start (CLI)](#quick-start-cli) · [Quick start (MCP)](#quick-start-mcp)
|
|
35
|
+
- [Available tools](#available-tools) · [The spend cap](#the-spend-cap) · [Chains](#chains)
|
|
36
|
+
- [How this was built](#how-this-was-built) · [Skills](#skills)
|
|
37
|
+
- [Known limits](#known-limits-stated-plainly) · [Privacy](#privacy) · [Accessibility](#accessibility) · [Reporting a bug](#reporting-a-bug) · [License](#license)
|
|
38
|
+
|
|
6
39
|
You choose the number of debate rounds, the seats and labs (any lab except xAI), and the token
|
|
7
40
|
limit for each model's replies.
|
|
8
41
|
|
|
9
|
-
We recommend our own chains: `
|
|
42
|
+
We recommend our own chains: `plan-daily-7` (`plan-highest-7`'s stages at a lower price, for planning
|
|
43
|
+
one feature; it needs an OpenRouter key, a Claude subscription for its Claude Opus 5.5 seats and a Claude Code session to write the plan; this roster has not run yet), `cheap-7-v2`, `plan-premium-7` and `plan-highest-7` (the top
|
|
10
44
|
models, GPT-6 Astra, Claude Opus 5.5, DeepSeek V4 Pro
|
|
11
45
|
and GLM-5.3, write the alternative
|
|
12
|
-
architectures
|
|
13
|
-
your own machine with `local-ollama`.
|
|
46
|
+
architectures; cheaper seats write the proposals and the debate; in `plan-highest-7` and `plan-daily-7` every seat in the chain also votes, blind, the plan's writer too). You can also edit a chain to suit yourself, or run every seat on
|
|
47
|
+
your own machine with `local-ollama`. For a quick second opinion on one decision, the advisor add-on asks a single seat:
|
|
48
|
+
GPT-6.1 Sol by default (`council_advise` with `mode: "single"`, see [Available tools](#available-tools)).
|
|
14
49
|
|
|
15
50
|
See it work right now - no keys, no setup, no cost:
|
|
16
51
|
|
|
@@ -23,10 +58,11 @@ model reply in it is written in advance; what is real is the code that runs the
|
|
|
23
58
|
them.
|
|
24
59
|
|
|
25
60
|
**What it costs.** You pay the model providers directly, with your own API keys. The demo,
|
|
26
|
-
`doctor`, `init` and the `mock` chains cost $0 and make no network call. `--dry-run` prints
|
|
27
|
-
|
|
28
|
-
|
|
29
|
-
|
|
61
|
+
`doctor`, `init` and the `mock` chains cost $0 and make no network call. `--dry-run` prints two
|
|
62
|
+
figures before anything is called: what the chain's own size assumptions expect a run to cost, and the most
|
|
63
|
+
the planned calls could cost if every one used its seat's whole output allowance (re-asks and the cut-off retry of a seat that is not Anthropic are not priced; the spend cap is the only bound). Every run stops before any stage that could
|
|
64
|
+
take it past its spend cap ($7 unless you set `--max-usd`). Most runs cost less than either figure: a panel
|
|
65
|
+
that signs off early skips the remaining rounds.
|
|
30
66
|
|
|
31
67
|
Three commands from nothing to a priced run of your own idea (no key needed until you drop
|
|
32
68
|
`--dry-run`; [Your first ten minutes](#your-first-ten-minutes) walks through them):
|
|
@@ -35,19 +71,22 @@ Three commands from nothing to a priced run of your own idea (no key needed unti
|
|
|
35
71
|
npx the-high-council init # writes tasks/my-first-task.md for your idea, plus a $0 mock run
|
|
36
72
|
npx the-high-council --task tasks/my-first-task.md --chain cheap-7-v2 --dry-run
|
|
37
73
|
|
|
38
|
-
**Which chain.**
|
|
74
|
+
**Which chain.** Prices as `council doctor` prints them (expected / at most), from the shipped price table
|
|
39
75
|
(prices as of 2026-09-06) and the default task-size estimate; a long task costs more.
|
|
40
76
|
|
|
41
|
-
| Chain | Who does what | Needs |
|
|
77
|
+
| Chain | Who does what | Needs | Expected / at most |
|
|
42
78
|
|---|---|---|---|
|
|
43
|
-
| `
|
|
44
|
-
| `
|
|
45
|
-
| `plan-
|
|
46
|
-
| `
|
|
47
|
-
|
|
48
|
-
|
|
49
|
-
|
|
50
|
-
|
|
79
|
+
| `plan-daily-7` | `plan-highest-7`'s stages at a lower price, for planning one feature; **every seat, the plan's writer included, also votes, blind**; Claude Opus 5.5 on your Claude subscription takes the criteria, one architecture and one vote, and a Claude Code session writes the skeleton, the plan and the handoff (details under "Chains" below). This roster has not run. | OpenRouter key + a Claude subscription + a Claude Code session | $12.59 / $33.25 |
|
|
80
|
+
| `cheap-7-v2` | Claude Sonnet 5 writes the plan; low-cost models from seven other labs review it, and all seven must sign off. Up to 7 rounds. | one OpenRouter key | $11.22 / $22.36 |
|
|
81
|
+
| `plan-premium-7` | A Claude Code session writes the plan; a seven-lab panel of larger models (GPT-6 Astra, Claude Fable 5.1, Gemini 3.8 Flash, Muse Spark 1.2, Qwen 3.8 Max, DeepSeek V4 Pro, GLM-5.3) proposes, debates and reviews. | OpenRouter key + a Claude Code session | $36.75 / $78.59 |
|
|
82
|
+
| `plan-highest-7` | Seven low-cost models propose and debate; four top models write whole architectures; one deep-dive seat checks the first draft against the task; **every seat, the plan's writer included, also votes, blind** (the owner's deliberate choice, 7 Oct 2026). **Untested with real models.** | OpenRouter key + Claude Code sessions | $18.49 / $50.43 |
|
|
83
|
+
| `local-ollama` | Every seat on your own machine, through Ollama. | Ollama and the models pulled | $0 / $0 |
|
|
84
|
+
|
|
85
|
+
The first figure is what the chain's own assumptions expect (the typical output of a review, every round up to the cap); the second is the most the planned calls could cost
|
|
86
|
+
if every one wrote its seat's whole output allowance (an Anthropic seat's one retry included; re-asks and the cut-off retry of other seats are not priced, so the spend cap is the only bound). The second figure is the larger one by a wide margin because the reasoning models can spend that
|
|
87
|
+
allowance thinking (since 0.8.1 every shipped seat reasons at high). `cheap-7-v2`, `plan-premium-7`, `plan-highest-7` and `plan-daily-7` expect to cost more than the default cap ($7), so give them a `--max-usd`
|
|
88
|
+
of your own: at or above the second figure to rule out a stop at the cap, or lower if you accept that a run that uses much of its allowance stops at the cap with no plan yet (it keeps what it paid for and
|
|
89
|
+
resumes with a higher `--max-usd`). A run that signs off early costs far less than either figure. The writer seats of `plan-daily-7`, `plan-premium-7`, `plan-highest-7` and `plan-open-7`, and the Claude Opus 5.5 subscription seats of `plan-daily-7` and `plan-highest-7`, are [external](#quick-start-mcp): the run pauses
|
|
51
90
|
and a Claude Code session (or you) writes that stage, at no API cost. Nothing about any chain's
|
|
52
91
|
output quality has been measured.
|
|
53
92
|
|
|
@@ -87,16 +126,17 @@ before anything is spent.
|
|
|
87
126
|
(one account at [openrouter.ai](https://openrouter.ai) that reaches many labs' models) is
|
|
88
127
|
enough for a seven-lab panel. Put the key in a file called `.env` in the folder you run from,
|
|
89
128
|
as `OPENROUTER_API_KEY=...`, and add `.env` to your `.gitignore` so it is never committed.
|
|
90
|
-
4. **Price it, then run it.** Add `--dry-run` to see the
|
|
91
|
-
it without:
|
|
129
|
+
4. **Price it, then run it.** Add `--dry-run` to see the expected and the maximum price for your task,
|
|
130
|
+
then run it without:
|
|
92
131
|
|
|
93
132
|
npx the-high-council --task tasks/my-first-task.md --chain cheap-7-v2 --dry-run
|
|
94
|
-
npx the-high-council --task tasks/my-first-task.md --chain cheap-7-v2 --max-usd
|
|
133
|
+
npx the-high-council --task tasks/my-first-task.md --chain cheap-7-v2 --max-usd 22.37
|
|
95
134
|
|
|
96
|
-
`cheap-7-v2`
|
|
97
|
-
cap, set just above
|
|
98
|
-
you set `--max-usd`). A run stopped by the cap keeps what it paid for but has no plan yet
|
|
99
|
-
|
|
135
|
+
`cheap-7-v2` expects $11.22 and could cost up to $22.36, both above the default cap, so the run command
|
|
136
|
+
carries its own cap, set just above the maximum (re-asks and retries are not in it, so only the cap is a bound). Every run stops before it would pass its spend cap
|
|
137
|
+
($7 unless you set `--max-usd`). A run stopped by the cap keeps what it paid for but has no plan yet; a cap
|
|
138
|
+
below the maximum can stop a run that uses a lot of its allowance (resume it with a higher `--max-usd`).
|
|
139
|
+
A panel that signs off early costs less than either figure.
|
|
100
140
|
|
|
101
141
|
The plan lands in `runs/<time>/deliverable.md`; the argument behind it is in `BOARD.md`.
|
|
102
142
|
|
|
@@ -154,7 +194,21 @@ A run moves through fixed stages. Which stages fire depends on the chain you pic
|
|
|
154
194
|
harness writes, not a model: the acceptance criteria the run settled, how each checkable one is
|
|
155
195
|
checked, and a fingerprint for each of the two. `council check-lock HANDOFF.md` ($0) says whether a
|
|
156
196
|
copy still carries them; it catches an edit or
|
|
157
|
-
a slip, and is not a signature.
|
|
197
|
+
a slip, and is not a signature. `report.json` also carries a small `thin_contract` (0.8.2): full-width
|
|
198
|
+
hashes of the task, the criteria, the evidence the run recorded as shown to the seats (the `--context` documents and the tool
|
|
199
|
+
results; never a repository, seats do not see one, and not material handed in with `--draft` or `--from-run`) and the
|
|
200
|
+
text the panel signed, plus one hash over the record. `council contract check runs/<id>` ($0) recomputes what the run folder still holds and says
|
|
201
|
+
`sealed`, `DRIFT` (what changed) or `cannot check` (never a pass for what it could not read). Like the
|
|
202
|
+
lock, it is tamper-evident, not tamper-proof: whoever can edit `report.json` can edit the record too.
|
|
203
|
+
`HANDOFF.md` also carries two short sections the harness writes (no model does): "Before you build" (read the
|
|
204
|
+
plan and write down blockers before coding; the criteria are not to be edited; the record's hash in plain
|
|
205
|
+
text, which `council contract check runs/<id> --handoff HANDOFF.md` compares with the run) and "Replan
|
|
206
|
+
triggers" (stop and send the plan back when the contract check or check-lock reports a difference, when a
|
|
207
|
+
named check fails twice in a row, and so on). The "twice" is `handoff_contract.check_failures` in a chain,
|
|
208
|
+
default 2. A chain can also set `handoff_contract.milestones` (off by default; on in the seven shipped planning chains, where the handoff seat is asked for the milestone shape):
|
|
209
|
+
the harness then lints the milestones the handoff seat wrote ($0: every criterion discharged by a milestone
|
|
210
|
+
that has a named check and an expected result) and records what it found in `report.json`
|
|
211
|
+
(`handoff_milestones`), `WARNINGS.md` and the "Before you build" section; a finding never changes `passed`.
|
|
158
212
|
|
|
159
213
|
Some chains add optional stages to this: whole alternative architectures written blind before
|
|
160
214
|
the skeleton, a deep-dive check of the first draft, a dispute stage that records unresolved
|
|
@@ -392,22 +446,28 @@ echo "Plan the data model for a bookmarking app. Single user, offline-first." >
|
|
|
392
446
|
|
|
393
447
|
# see what's available and what it would cost
|
|
394
448
|
council --help
|
|
395
|
-
council --task tasks/your-idea.md --chain
|
|
449
|
+
council --task tasks/your-idea.md --chain cheap-7-v2 --dry-run
|
|
396
450
|
|
|
397
451
|
# a real run
|
|
398
|
-
council --task tasks/your-idea.md --chain
|
|
452
|
+
council --task tasks/your-idea.md --chain cheap-7-v2
|
|
399
453
|
```
|
|
400
454
|
|
|
401
455
|
Your requests and everything the council writes about them stay on your machine, in `tasks/` and
|
|
402
456
|
`runs/` under the directory you ran from. In a clone of this repo both are gitignored, so a fork
|
|
403
457
|
will not accidentally publish them.
|
|
404
458
|
|
|
405
|
-
|
|
406
|
-
needs
|
|
407
|
-
with `cheap-7-v2` instead; `council doctor` lists every chain that runs on the keys you have.
|
|
459
|
+
There is no default chain: since 0.8.2 `council --task x.md` without `--chain` stops and names the recommended ones.
|
|
460
|
+
`cheap-7-v2` needs one OpenRouter key; `council doctor` lists every chain that runs on the keys you have.
|
|
408
461
|
`--chain` takes a chain's name (`cheap-7-v2`), never a file path: a `chains/<name>.json` of your
|
|
409
462
|
own in the working directory is found by its name.
|
|
410
463
|
|
|
464
|
+
**If your task already contains its own acceptance list.** A task written like a specification often ends with what the answer must contain and when it fails. The criteria seat can mirror that
|
|
465
|
+
list: criteria about the format of the answer, not about what makes it right. The criteria stage has two checks that catch some of these (criteria about the criteria list or its JSON shape, or criteria that demand several documents: it asks once more, and stops the run before any paid review
|
|
466
|
+
round if the second try does the same), after one or two paid criteria calls; a mirrored list that avoids those patterns passes them silently. Write the criteria yourself and hand them in instead: `council --task tasks/x.md --chain cheap-7-v2 --criteria criteria.md`. The file holds one
|
|
467
|
+
criterion per line (a leading `- `, `* ` or `1. ` is dropped, lines starting with `#` are skipped), or JSON (an array, or `{ "criteria": [...] }`). A hand-written list skips the criteria stage and
|
|
468
|
+
gets the same two checks before any paid review: criteria about the criteria list itself, or ones that demand documents one build stage cannot produce, stop the run. `council lint-criteria --criteria criteria.md`
|
|
469
|
+
runs the $0 word-level lints on the file first.
|
|
470
|
+
|
|
411
471
|
**If your task names files.** A task that names a file (`src/app.js`) without including its
|
|
412
472
|
content is stopped before any call (exit 9, `BLOCKED-ARTIFACTS.md`), because the seats would
|
|
413
473
|
otherwise plan against code they never saw. Put the real content in with
|
|
@@ -474,8 +534,8 @@ stage - a stage that could take the run past the ceiling is never called, so the
|
|
|
474
534
|
than reporting the overspend afterwards.
|
|
475
535
|
|
|
476
536
|
```bash
|
|
477
|
-
council --task tasks/x.md --max-usd 2 # this run stops at $2
|
|
478
|
-
council --task tasks/x.md --max-usd none # no ceiling (0 is refused: it used to mean this)
|
|
537
|
+
council --task tasks/x.md --chain cheap-7-v2 --max-usd 2 # this run stops at $2
|
|
538
|
+
council --task tasks/x.md --chain cheap-7-v2 --max-usd none # no ceiling (0 is refused: it used to mean this)
|
|
479
539
|
export MAX_USD_PER_RUN=20 # change the default
|
|
480
540
|
```
|
|
481
541
|
|
|
@@ -609,7 +669,7 @@ Then drive it with these tools:
|
|
|
609
669
|
|
|
610
670
|
| Tool | What it does |
|
|
611
671
|
|---|---|
|
|
612
|
-
| `list_chains` | Available chains with descriptions
|
|
672
|
+
| `list_chains` | Available chains with descriptions (price one with `dry_run`) |
|
|
613
673
|
| `write_task` | Write the request the harness plans against |
|
|
614
674
|
| `dry_run` | Price a run before spending anything |
|
|
615
675
|
| `start_run` | Start a run in the background (`max_usd` sets its ceiling). `chain` is a chain name; `task`, `draft`, `context` and `from_run` must be inside the working directory |
|
|
@@ -626,6 +686,8 @@ Then drive it with these tools:
|
|
|
626
686
|
| `plan_outline` | Section tree of a run's deliverable (or a `.md` file in the working directory) with word counts |
|
|
627
687
|
| `council_quote` | Experimental. Preview and price a request for advice from other labs: who would receive the text, its exact size and `sha256` after masking, the worst-case and expected price, a `quote_id`. Sends and spends nothing. See [Asking other labs for advice](#asking-other-labs-for-advice-council_quote-council_advise-experimental) |
|
|
628
688
|
| `council_advise` | Experimental. Spends. Asks other labs for advice on one decision, once you approve it: a leaning and every dissent in the models' own words. `start_run` and `resume_run` refuse advice chains, so this is the only way in over MCP |
|
|
689
|
+
| `contract_read` | Read the locked contract of a run: its version, the obligations a person approved and the amendment requests filed against it. Checked first ($0): if the record, its approval or the ledger disagree, nothing is returned. The text comes back marked as model-written |
|
|
690
|
+
| `contract_amend` | Ask for an obligation of a locked contract to be changed. Free; it only files a request, and a person decides it at a terminal with `council contract amend --decide`. See [The contract record](#the-contract-record-council-contract) |
|
|
629
691
|
|
|
630
692
|
**Text from a run comes back marked as model-written.** A deliverable, a board, a log line or an
|
|
631
693
|
advisor's answer was written by AI models, and a model can be fooled into writing instructions aimed
|
|
@@ -668,8 +730,10 @@ rating: a typo in a hand edit of that file makes every later advice call refuse)
|
|
|
668
730
|
own terminal (`council --chain advise-single --task brief.md`) is your own call and has no gate; its
|
|
669
731
|
record says so (`gate: null` in `advise-log.json`). The terminal check is friction, not proof
|
|
670
732
|
that a person answered: a program can open a pseudo-terminal. Advice calls therefore need a person
|
|
671
|
-
at the keyboard; scripted or unattended use is not supported. `mode: "single"` asks one seat
|
|
672
|
-
`advisor` argument
|
|
733
|
+
at the keyboard; scripted or unattended use is not supported. `mode: "single"` asks one seat: GPT-6.1 Sol through OpenRouter by default, or with the
|
|
734
|
+
`advisor` argument GPT-6 Astra (`astra`), GLM-5.3 Flash (`glm-flash`, new in 0.8.2) or Qwen3.8 Max (`qwen`, new in 0.8.2; it has no zero-retention endpoint,
|
|
735
|
+
so a quote for it is refused unless the operator set the sensitivity floor to `public`). Three more seats are reachable by name only: `gemini`, `deepseek` and `opus` (Claude Opus 5.5, the experimental control);
|
|
736
|
+
`mode: "council"` asks three labs (OpenAI GPT-6.1 Sol, Z.ai GLM-5.3, Google Gemini 3.8 Flash)
|
|
673
737
|
blind, lets them read each other once, and a seat changes its answer only by quoting an argument.
|
|
674
738
|
The call holds for up to `wait_seconds` (default 25, at most 30, the same for every client), your approval included;
|
|
675
739
|
a run that outlives the hold returns its id, and `run_status(run, wait_seconds: 25, until: "settled")` (also at most
|
|
@@ -694,18 +758,22 @@ rating: a typo in a hand edit of that file makes every later advice call refuse)
|
|
|
694
758
|
folder you name in `COUNCIL_ADVICE_CHAINS_DIR` (an absolute path outside the project; anything else stops
|
|
695
759
|
the server), and never from the project's `chains/`: a repository's chain file would otherwise choose who
|
|
696
760
|
receives the text. Your chain must pass the same checks: priced seats, not denied, zero-retention routing
|
|
697
|
-
where the model has it
|
|
761
|
+
where the model has it. `COUNCIL_ADVISE_MOCK=1` runs the mock chains, so the whole exchange
|
|
698
762
|
can be tried with no key and no money.
|
|
699
763
|
- **When an advisor asks for more.** An answer can list what was missing from the brief. That list is model
|
|
700
764
|
text, shown to the agent as untrusted, and the next call does not go out on the agent's say-so: its
|
|
701
765
|
approval text starts with "This call follows an advisor's request for more material", so you see that
|
|
702
766
|
the new text answers a model's request, and decide whether that material should leave your machine.
|
|
703
|
-
- **Stopping.** A client that cancels the call or goes away, `council stop runs/<id>` at your
|
|
704
|
-
terminal,
|
|
705
|
-
|
|
706
|
-
|
|
707
|
-
|
|
708
|
-
|
|
767
|
+
- **Stopping.** A client that cancels the call or goes away, or `council stop runs/<id>` at your
|
|
768
|
+
terminal, stops the run before its next paid step. **A stop does not cut a call that is already running:**
|
|
769
|
+
with no clock, it waits for that call, which for a slow reasoning model can take up to two hours (the request
|
|
770
|
+
deadline), and the call finishes and is billed. The run then exits 18, and the answers that were paid for are kept in `report-partial.json`, which
|
|
771
|
+
`run_status` hands back. A stopped call is not resumed; ask again. **No shipped advice chain has a time
|
|
772
|
+
limit** (owner, 6 Oct 2026: if a call takes time it takes time, and if it times out you ask again): a slow
|
|
773
|
+
reasoning model took 6 to 13 minutes for one answer in the run that measured it, and `council_advise` hands
|
|
774
|
+
back a run id within 30 seconds either way. A chain of your own can still set `advise.max_wall_ms` to end
|
|
775
|
+
the run at a deadline (then a call still running is cut off, which may still be billed); every request has
|
|
776
|
+
a 2-hour deadline regardless.
|
|
709
777
|
|
|
710
778
|
What this does **not** do, stated plainly: "ZDR" in the preview is OpenRouter's routing tag for an
|
|
711
779
|
endpoint, not a guarantee about what that host does with your text, and the retention table
|
|
@@ -728,34 +796,53 @@ same debate on external, based on one measured comparison, not a guarantee. Not
|
|
|
728
796
|
anywhere in this repo's config, README ordering, or example chains - external stays recommended
|
|
729
797
|
for anyone with a subscription, since the project's cost story depends on it.
|
|
730
798
|
|
|
799
|
+
### The contract record (`council contract`)
|
|
800
|
+
|
|
801
|
+
After a run has ended, a person can lock a **contract** from it: a short list of obligations (an id and a sentence or two each) that a build session works against. A model drafts the list. Nothing
|
|
802
|
+
becomes the contract until a person has read the exact text and approved it at a terminal, through the same approval step the advice add-on uses.
|
|
803
|
+
|
|
804
|
+
```
|
|
805
|
+
council contract lock runs/<id> # shows the exact text, asks, then writes runs/<id>/contract/v1.json and CONTRACT.md
|
|
806
|
+
council contract show runs/<id> [--md] # the current version, each obligation, the requests filed against it
|
|
807
|
+
council contract check runs/<id> # $0: do the record, its approval, the ledger and CONTRACT.md still agree?
|
|
808
|
+
council contract amend --decide runs/<id> [request] # a person's decision on an amendment request
|
|
809
|
+
```
|
|
810
|
+
|
|
811
|
+
A version is never edited. A build session that finds an obligation wrong or impossible does not change it: it calls `contract_amend`, which is free, files a request and decides nothing. A person
|
|
812
|
+
decides the request at a terminal, with the exact current and the exact proposed words in front of them. An approval writes the next version, which names the one before it and the request;
|
|
813
|
+
a refusal is recorded. `contract_read` hands the build session the current version, after the same check `council contract check` makes; the obligation text comes back marked as model-written.
|
|
814
|
+
|
|
815
|
+
The record is **tamper-evident, not tamper-proof**. It lives in the run folder together with its approval and the ledger; whoever can write that folder can rewrite all three together, and nothing
|
|
816
|
+
here detects it. `council contract check` catches an edit of one of them without the others. A 0.8.1 `council gate verify` reads a ledger that holds contract events as broken; verify it with 0.8.2.
|
|
817
|
+
|
|
731
818
|
## Chains
|
|
732
819
|
|
|
733
|
-
|
|
820
|
+
46 chain configs live in `chains/`. Each one is plain JSON - the seat roster, which models fill
|
|
734
821
|
which seat, the round cap, and whether proposals/debate/handoff stages run. They are meant to be
|
|
735
822
|
copied and edited.
|
|
736
823
|
|
|
737
|
-
The
|
|
738
|
-
in the table at the top. A few more worth knowing:
|
|
739
|
-
|
|
740
|
-
- **`verify`** - two labs, two rounds.
|
|
741
|
-
|
|
824
|
+
The five recommended ones (`cheap-7-v2`, `plan-daily-7`, `plan-premium-7`, `plan-highest-7`, `local-ollama`) are
|
|
825
|
+
in the table at the top. The chains the 0.8.2 cleanup moved out of `chains/` are in `archive/chains/` in the repository (not in the npm package); copy one into your project's own `chains/` to run it by name. A run recorded on one of them needs its file there too before it can be resumed, rematched or replayed (the error says so). A few more worth knowing:
|
|
826
|
+
|
|
827
|
+
- **`verify`** - two labs, two rounds; needs Anthropic, OpenAI and Google keys. It was the default when you named no chain until 0.8.1.
|
|
828
|
+
- **`plan-lanes-4`** (new in 0.8.2, optional and experimental) - one writer (Claude Sonnet 5.5, no panel seat) and
|
|
829
|
+
three reviewers (GPT-5.6 Luna, Gemini 3.8 Flash, GLM 5.3 Flash), each with an assigned lane: correctness and
|
|
830
|
+
interfaces, security and failure behaviour, implementation and verification. No blind proposals and no debate;
|
|
831
|
+
unanimous sign-off, up to 7 rounds, one OpenRouter key. A lane is where a reviewer looks first, never a smaller
|
|
832
|
+
set of criteria: every reviewer still checks every criterion. Dry run: $1.62 expected at the chain's own task-size estimate, $6.34 at most (every call at its full output allowance, GLM at 360,000 tokens for its panel
|
|
833
|
+
review, over seven rounds) (under the default $7 at the estimate sizes, by a margin that shrinks as the task grows), and about $8.9 if every GLM review were cut off
|
|
834
|
+
and retried at its larger cap, in which case the cap stops the run before it is exceeded. The words that tell a
|
|
835
|
+
reviewer its lane are recorded in the judge prompt (`LANE_TEXT` in `src/roles.js`), so each of its three judges is told where its attention goes first, and every judge still writes a row for every criterion.
|
|
836
|
+
Nothing here claims it plans better than another chain: that cannot be measured without ground truth.
|
|
742
837
|
- **`plan-open-7`** - the `cheap-7-v2` panel, with a Claude Code session writing the plan instead
|
|
743
838
|
of a paid seat.
|
|
744
|
-
- **`plan-daily-7`** - `plan-highest-7`'s stages at a lower price, for planning one feature
|
|
745
|
-
|
|
746
|
-
|
|
747
|
-
|
|
748
|
-
(including `-retry` and `-reask1`/`-reask2` forms), Sonnet 5 the writer stages
|
|
749
|
-
(0.8.
|
|
750
|
-
|
|
751
|
-
**Run once with real models (2026-09-28); nothing about it has been measured.**
|
|
752
|
-
- **`cheap`** - small models throughout. For testing the harness itself, not for real work.
|
|
753
|
-
- **`plan-debate`** - five labs propose blind, debate each other's proposals anonymised, then a
|
|
754
|
-
blind panel grades the integrated draft.
|
|
755
|
-
- **`plan-auto`** - the full open-scope chain with every seat on a real API, so no stage waits for
|
|
756
|
-
an external session. It still pauses once for your answers to its questions.
|
|
757
|
-
- **`plan-unanimous`** - every critic must independently sign off on the *same* draft. Three rounds
|
|
758
|
-
here means up to fifteen critic calls, not five.
|
|
839
|
+
- **`plan-daily-7`** - `plan-highest-7`'s stages at a lower price, for planning one feature.
|
|
840
|
+
**Every seat votes, blind, the plan's writer too** (the owner's deliberate choice of 7 Oct 2026; it reverses the September rule that no seat both argues and votes, for this chain and `plan-highest-7` only):
|
|
841
|
+
the criteria seat, one of the architects and one of the judges are Claude Opus 5.5 on your Claude subscription (external seats); GPT-6.1 Sol and GLM-5.3 Flash write the other architectures; six low-cost labs
|
|
842
|
+
propose and debate; thirteen judges (those seats, DeepSeek V4 Pro, GPT-6 Luna, MiniMax M3, the deep-dive seat's lab and the writer) sign off unanimously. Two Claude Code sessions answer its
|
|
843
|
+
external seats: Opus 5.5 every stage whose label contains `opus5.5-sub` (including `-retry` and `-reask1`/`-reask2` forms) and the criteria stage (`criteria`, `criteria-retry`, `criteria-feasibility-retry`: their labels carry no lab name), Sonnet 5 the writer stages and the writer's own panel review.
|
|
844
|
+
$12.59 expected, $33.25 at most (the 0.8.2 dry run of 7 Oct 2026, on the shipped price table, as of 2026-09-06; pass a `--max-usd` above the $7 default).
|
|
845
|
+
**The roster before 7 Oct ran once with real models (2026-09-28); this one has not run, and nothing about it has been measured.**
|
|
759
846
|
|
|
760
847
|
Read a chain's `description` field before running it; they say what they cost you in calls.
|
|
761
848
|
|
|
@@ -785,8 +872,8 @@ for any model the table has a field for.
|
|
|
785
872
|
- **Models the table has no field for** (for example Cohere, Groq, Mistral Large, the models on Together
|
|
786
873
|
other than Qwen3.5-9B and DeepSeek V4 Flash, local Ollama models) are sent nothing. DeepSeek V4 Flash on
|
|
787
874
|
Together is sent the chat-template switch vLLM documents for it; Together's own page for it was not found.
|
|
788
|
-
- **`gp-judge-v1`**
|
|
789
|
-
from
|
|
875
|
+
- **`gp-judge-v1`** (since 0.8.2 in `archive/chains/`) ran its Sonnet and Qwen seats with reasoning on (it ran with it off before). Scores
|
|
876
|
+
from that version on come from the high-effort judge, so earlier and later GP scores are not
|
|
790
877
|
directly comparable.
|
|
791
878
|
|
|
792
879
|
### Optional chain-config fields
|
|
@@ -812,11 +899,11 @@ for any model the table has a field for.
|
|
|
812
899
|
a round short of it is recorded as `notQuorate`, the run does not pass, and the loop stops.
|
|
813
900
|
- **Tiered councils (experimental, used by `plan-highest-7`, `plan-daily-7` and `mock-tiered`).**
|
|
814
901
|
`seats.alternatives` names who writes the whole alternative architectures, separately from the
|
|
815
|
-
proposers. `deep_dive` + `seats.deep_dive` add one
|
|
902
|
+
proposers. `deep_dive` + `seats.deep_dive` add one seat with its own dollar cap
|
|
816
903
|
(`usd`, required) inside the run's cap. `majority_guard.enabled` shows each author every
|
|
817
904
|
argument once, with no lab name or count, and keeps a proposal whose withdrawal does not quote
|
|
818
905
|
the argument it gives in to. chain-lint refuses a tiered chain where a lab or model sits
|
|
819
|
-
both among the proposers and among the reviewers or architecture authors, or where the deep-dive seat shares a lab or model with a reviewer. See
|
|
906
|
+
both among the proposers and among the reviewers or architecture authors, or where the deep-dive seat shares a lab or model with a reviewer, unless the chain says `"judging": "all-roles"` at the top level (the owner's deliberate override of 7 Oct 2026, set in `plan-daily-7` and `plan-highest-7` only: there every seat with another role also judges and votes, blind; `"selfReview": "allowed"` seats the plan's writer on the panel too). See
|
|
820
907
|
CHANGELOG.md's 0.7.8 entry.
|
|
821
908
|
|
|
822
909
|
## How this was built
|
|
@@ -876,6 +963,10 @@ what each rule guards against and why. Copy the folders you want into your proje
|
|
|
876
963
|
|
|
877
964
|
## Known limits, stated plainly
|
|
878
965
|
|
|
966
|
+
- **The panel signs one draft; later stages can change what you receive, and nobody re-reads it.** The dispute pass, the post-sign-off challenge and the final edit are model edits made after
|
|
967
|
+
the last review, and the dissent and cold-read blocks are written by the harness. The run labels this and does not repair it: `report.json` has `signed_text` and `delivered_text`
|
|
968
|
+
(hashes, `same_as_signed`, `changed_by`), `BOARD.md` says so when it applies, and `HANDOFF.md` opens with a banner when a model edit changed the text after the panel signed it (a run the panel did not sign keeps its own "not a signed-off plan" banner). A cold-read or dissent block the harness adds is named in `delivered_text.harness_notes` and gets no banner. The panel is not run
|
|
969
|
+
again on the changed text (that would be a further paid pass per run), and `run_status` says so after "every lab signed off" when the delivered text differs (`delivered_text.same_as_signed` in `report.json`).
|
|
879
970
|
- The spend cap is enforced against a **worst case**, not a prediction: the whole prompt billed as
|
|
880
971
|
input plus the seat's entire `maxTokens` budget billed as output (for Anthropic seats, plus the
|
|
881
972
|
bigger cap of the one retry they may make). Real stages almost never cost that much, so a run can stop with headroom
|
|
@@ -883,11 +974,11 @@ what each rule guards against and why. Copy the folders you want into your proje
|
|
|
883
974
|
- The dry run (`--dry-run`, MCP `dry_run`) prices the task alone and does not read `--context`
|
|
884
975
|
documents, so for a run with context its estimate is too low. The spend cap is not affected: it
|
|
885
976
|
checks every real prompt, context included, before each call.
|
|
886
|
-
- A seat whose model has no entry in `src/pricing.json`
|
|
887
|
-
|
|
888
|
-
|
|
889
|
-
|
|
890
|
-
|
|
977
|
+
- A seat whose model has no entry in `src/pricing.json` cannot be held by a spend cap, because the cap would count every call to it as $0. Since 0.8.2 a run **refuses** such a
|
|
978
|
+
seat before it sends anything (exit 5; the message names the seat and the two ways out: add a pricing row, or run with `--max-usd none`, which sets no spend ceiling at all), and
|
|
979
|
+
`--dry-run` shows the same refusal at $0. `mock`, `external` and `ollama` seats are exempt: `ollama` prices at an explicit $0 for any model name, because it is genuinely free to run.
|
|
980
|
+
What remains: the cap is only as good as the price table (it is dated, and the dry run says when it is old), and a model that answers in place of a priced one (an OpenRouter fallback in
|
|
981
|
+
`extra.models`) is charged at the priced seat's price, which can differ from its own.
|
|
891
982
|
- A call that fails **after** it reached the provider - the connection drops while a 200 reply is
|
|
892
983
|
being read, a 200 carries a body that is not JSON, or the request times out waiting for headers -
|
|
893
984
|
is not retried, because the generation was probably already billed. Its real usage cannot be
|
|
@@ -921,14 +1012,18 @@ what each rule guards against and why. Copy the folders you want into your proje
|
|
|
921
1012
|
through 2026-11-21; Gemini 3.8 Flash, which doubles on 2027-01-01): past it, `council doctor` and
|
|
922
1013
|
`--dry-run` warn, but the projection and the cap keep the old price, so they can project low
|
|
923
1014
|
until the table is updated.
|
|
924
|
-
- `plan-highest-7` has not been run with real models, and `plan-daily-7`
|
|
925
|
-
(2026-09-28). None of the tiered councils' mechanisms (tiers, the deep-dive seat, the majority
|
|
1015
|
+
- `plan-highest-7` has not been run with real models, and `plan-daily-7` ran once (2026-09-28) with the roster it had before the 7 Oct 2026 redesign. None of the tiered councils' mechanisms (tiers, the deep-dive seat, the majority
|
|
926
1016
|
guard) has been measured. Their dry-run prices are projections.
|
|
1017
|
+
- **`run_tests` runs a repository's own test files as trusted code, with no sandbox.** It belongs to the opt-in seat tools (`verify.tools` naming `run_tests`, as `coder-gate-v1` does; a seat
|
|
1018
|
+
may request it mid-stage only when the chain also sets `tools.seat_requests`). Since 0.8.2 the test process gets an allowlisted environment: `PATH`, `HOME` (`USERPROFILE`), the temporary-folder
|
|
1019
|
+
variables, `LANG`, `LANGUAGE`, `LC_*`, `TERM`, `CI` and the Windows system variables (`SystemRoot`, `ComSpec`, `PATHEXT`, `WINDIR`). No provider key is in its environment, but a test is code you trust: it can still read the parent process's environment and files on disk (for example `/proc/<pid>/environ`), so run it only on a repository you trust. A project's tests that need another
|
|
1020
|
+
variable (a database URL, say) will not see it. It can still read every file you can read, write files and use the network, so use it only on a repository you trust.
|
|
1021
|
+
- **The symbolic-link containment of the MCP tools is untested on Windows.** Since 0.8.2 the tools refuse a run folder that holds a symbolic link and open run-folder files with `O_NOFOLLOW`. That flag does not exist on Windows, so there the
|
|
1022
|
+
protection rests on the `lstat` checks alone, and it has been tested on Linux only: not on a real Windows machine, and the packaged `.exe`'s smoke test under Wine does not exercise it.
|
|
927
1023
|
- Seats marked `external` (the writer seats in `plan-premium-7`, `plan-highest-7`,
|
|
928
|
-
`plan-daily-7` and `plan-open-7`, and the Opus 5.5
|
|
1024
|
+
`plan-daily-7` and `plan-open-7`, and the Opus 5.5 subscription seats in `plan-daily-7` and `plan-highest-7`) pause the run until someone answers them, and are $0 only because a Claude Code
|
|
929
1025
|
session on a subscription writes them. Nothing checks who wrote the answer.
|
|
930
|
-
- Chains with many labs and high round caps get expensive quickly
|
|
931
|
-
is fifteen critic calls; `plan-auto` runs several frontier models over multiple rounds. Price
|
|
1026
|
+
- Chains with many labs and high round caps get expensive quickly: every round asks every critic seat. Price
|
|
932
1027
|
before you run.
|
|
933
1028
|
- A panel that signs off is not a guarantee the output is correct. It means every critic seat
|
|
934
1029
|
checked it against the stated acceptance criteria and found nothing. Criteria that are vague
|
|
@@ -963,6 +1058,9 @@ what each rule guards against and why. Copy the folders you want into your proje
|
|
|
963
1058
|
- U5: what each provider bills for an aborted request, and a real client's cancel (the tests send `notifications/cancelled` by hand).
|
|
964
1059
|
- U6: Sol's real output length and latency, and GLM-5.3's latency, as advisors.
|
|
965
1060
|
- U7: cancellation semantics under the protocol revision the SDK negotiates (generic cancellation, not the newer stdio requirement).
|
|
1061
|
+
- **The contract record (`council contract`, `contract_read`, `contract_amend`) was checked only against mock providers and a pretend terminal.** Its drafting call was never made with a real model
|
|
1062
|
+
(its wording, recorded in 0.8.2, has only been read by a mock seat). The person's answer is friction, not enforcement: a shell-capable agent can open a pseudo-terminal, as
|
|
1063
|
+
with `council gate answer`. A contract is an agreement between a person and a build session's instructions; it does not make a build session follow it, and nothing here measures whether it helps.
|
|
966
1064
|
- **The approval gate is friction, not enforcement.** Every advice call needs a person to answer a gate through a terminal or an elicitation, so scripted or
|
|
967
1065
|
unattended use is not supported: a client that cannot ask a person (headless Claude Code, a script) gets `awaiting_approval`, and nothing is sent until someone answers at a terminal. The terminal answer can be typed by anything that runs as you. The gate's ledger is
|
|
968
1066
|
tamper-evident, not tamper-proof: an edit before its last line is detected; the last line, the tail and a rewrite of both the ledger and its view are not.
|
|
@@ -974,16 +1072,16 @@ what each rule guards against and why. Copy the folders you want into your proje
|
|
|
974
1072
|
- **The advice brief is written into the run folder** (`runs/<id>/advice-brief.md`), so a commit of the run folder publishes it.
|
|
975
1073
|
- **`advise-single`'s ceiling is $1.32 (the "at most" its tool description shows).** It covers the first call and one retry of a cut-off reply; the typical
|
|
976
1074
|
cost of one call is far lower. A person's own terminal run of an advice chain is not gated (`gate: null`), and the add-on's money limits and its chain choice can
|
|
977
|
-
also be set from a project `.env` (as can the audit-log key `AUDIT_HMAC_KEY` and the quote lifetime `COUNCIL_ADVISE_QUOTE_TTL_MS`): whether those should be refused is open. The
|
|
978
|
-
(Astra, Gemini) keep fixed output caps.
|
|
1075
|
+
also be set from a project `.env` (as can the audit-log key `AUDIT_HMAC_KEY` and the quote lifetime `COUNCIL_ADVISE_QUOTE_TTL_MS`): whether those should be refused is open. The single-seat swaps
|
|
1076
|
+
(Astra, Gemini) keep fixed output caps. The three added or raised in 0.8.2 were priced by dry run only, and no live call was made: Qwen3.8 Max has a 36,000-token cap (whether that is enough when the model reasons at its own default, above high, is not verified); GLM-5.3 Flash and DeepSeek V4.1 Flash are set to 360,000 tokens, which only some of their zero-data-retention endpoints accept (8 of 17 and 3 of 5, list read 2026-10-06), and whether OpenRouter routes around an endpoint that does not or refuses the call is not verified. No advice chain has a time limit of its own, so a stop (`council stop`, a client cancel) waits for calls already in flight, up to the 2-hour request deadline.
|
|
979
1077
|
- **A review cut-off in the security-review seat:** `src/security-review.js` cuts reviewer fields at 2,000 characters with a visible marker and no record.
|
|
980
|
-
- **Untested on case-insensitive file systems (macOS, Windows).** The fix for `handoff.md` and `HANDOFF.md` being one file (FX-9: `handoff --from-run` now also writes with an exclusive create, so it cannot replace an existing file whatever the disk's case rules) and the working-folder path checks were tested on Linux only. No pass on a Mac or a Windows machine was run for 0.8.1.
|
|
1078
|
+
- **Untested on case-insensitive file systems (macOS, Windows).** The fix for `handoff.md` and `HANDOFF.md` being one file (FX-9: `handoff --from-run` now also writes with an exclusive create, so it cannot replace an existing file whatever the disk's case rules) and the working-folder path checks were tested on Linux only. No pass on a Mac or a Windows machine was run for 0.8.1 or 0.8.2.
|
|
981
1079
|
- **The dry run does not price every cap.** A planning chain's dry run prices a critic at the chain's own estimate, not at its output cap, so the larger caps of
|
|
982
1080
|
0.8.1 (360,000 tokens for DeepSeek V4.1 Flash and the panel review of GLM-5.3 Flash) appear only in the spend cap's per-stage projection, which can stop a run before
|
|
983
|
-
the dry-run total. A retry of 720,000 tokens (a 360,000-token seat at the model's maximum)
|
|
1081
|
+
the dry-run total. A retry of 720,000 tokens (a 360,000-token seat at the model's maximum) would take about 90 minutes at the slowest speed measured (about 134 tokens/s), close to the 2-hour request deadline, and a call that deadline cuts is a paid timeout.
|
|
984
1082
|
- **A failed start after approval is counted at the full ceiling.** If an advisor run cannot start after a person approved it (for example an unreadable audit-key file named in a project `.env`), the approval is used up, nothing is sent, and the call is still counted in the limits at its full ceiling (`advise-single`: $1.32). Two of these and one real call pass the $3 session limit, which then refuses advice for six hours with nothing spent.
|
|
985
1083
|
- **One approval can start two runs if the approved run folder is copied:** the approval is bound to the text, not to the run folder.
|
|
986
|
-
- **
|
|
1084
|
+
- **Two shipped chains seat Gemini 2.5** (`us-only` and `verify`, through the direct Google provider; the archived `plan-two-strong`, `seven`, `cheap` and `smo` did too). Google has closed Gemini 2.5 to new projects, so a run on those chains by a new user will probably fail at that seat; use a chain with another seat, or change the seat's model.
|
|
987
1085
|
- **The retry of a direct Anthropic seat whose thinking used the whole cap may never fire.** It keys on a thinking-token count in the reply's `usage`
|
|
988
1086
|
(`output_tokens_details.thinking_tokens`) that Anthropic's public Messages API reference does not list (thinking counts inside `output_tokens`); OpenRouter reports
|
|
989
1087
|
`reasoning_tokens`, so it fires for an Anthropic seat sent through a single-vendor route (`"transport": "openrouter"`). Where the count is absent, a reply cut off at its cap is retried by the panel's ordinary cut-off rule (one bigger call),
|
|
@@ -992,13 +1090,15 @@ what each rule guards against and why. Copy the folders you want into your proje
|
|
|
992
1090
|
external seat's claim as ordinary fields; text from a cloned repository's run folder or chain file could carry an instruction there. Only the model-written fields named in
|
|
993
1091
|
`untrusted_fields` are marked. A stage bundle for a subagent now says its referenced files are data; the field marking is planned for 0.8.2.
|
|
994
1092
|
- **The terminal command for approving a call was not run under the `.mcpb` route.** It names the server's own Node binary and `cli.js` with absolute paths (or the standalone binary alone). Under Claude Desktop's bundled Node, whether that binary can be run from a terminal was not checked; `council gate answer` from an installed `council` always works.
|
|
995
|
-
- **A process killed in the middle of a call
|
|
996
|
-
|
|
997
|
-
|
|
998
|
-
|
|
1093
|
+
- **A process killed in the middle of a paid call leaves that call charged at its worst case, not at what it cost.** Since 0.8.2 a call's worst case is written to
|
|
1094
|
+
`runs/<id>/spend-reservations.jsonl` before the request goes out, and a settle line follows when the call returns. If the process dies in between (there are no signal handlers), the provider may bill
|
|
1095
|
+
the call and no usage file exists, so the unsettled reservation stays counted at its worst case in a resume's cap, in `council --spend` and in the spend report. No command clears one: the figure errs high,
|
|
1096
|
+
and goes when the run folder goes. Whoever can write the run folder can edit the file. A brief of non-ASCII text (CJK, say) is projected low in a planning chain (four characters a token); the add-on prices such a brief from its bytes.
|
|
1097
|
+
- **A run paused under 0.8.1 on a chain or a prompt that 0.8.2 changed is not replayed when resumed:** the config fingerprint or the prompt hash differs (0.8.2 changed seven planning chains and the criteria and judge prompts of every chain), so its API stages are
|
|
1098
|
+
paid for again and the answers already given to its external stages are set aside and asked again (the same holds for a run paused under 0.8.0 on a chain 0.8.1 changed). Finish such a run on the version that paused it, or start it again.
|
|
999
1099
|
- **`npm test` turns red on dates, by design:** 2026-10-31 (the retention table, 30 days old), 2026-11-21 (the `gpt-5.6-sol` promotional price row, and the OpenRouter
|
|
1000
|
-
price rows of `src/pricing.json`, 45 days after `scripts/refresh-openrouter-prices.mjs` last compared them with OpenRouter's list) and 2026-12-
|
|
1001
|
-
(`src/reasoning-table.json`, 60 days old). Re-read the pages, re-date the file, then the suite is green again. The price rows of the direct providers
|
|
1100
|
+
price rows of `src/pricing.json`, 45 days after `scripts/refresh-openrouter-prices.mjs` last compared them with OpenRouter's list) and 2026-12-06
|
|
1101
|
+
(`src/reasoning-table.json`, 60 days old; regenerated 2026-10-07). Re-read the pages, re-date the file, then the suite is green again. The price rows of the direct providers
|
|
1002
1102
|
(Anthropic, OpenAI, Google, ...) carry their own dates in `_source` and are not covered by that test.
|
|
1003
1103
|
- **DeepSeek V4 Flash on Together** is sent `chat_template_kwargs {thinking: true, reasoning_effort: "high"}` from vLLM's serving recipe; Together's own page for it was
|
|
1004
1104
|
not found, so the on-value is unconfirmed on Together.
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "advise-premium",
|
|
3
|
-
"description": "Council advisor, the premium option (experimental; decided rule 4: a premium-7-style setup). Selected only by the operator through COUNCIL_ADVISE_CHAIN_COUNCIL, never by a tool argument. The critics of plan-premium-7 minus its Anthropic seat (the caller is Claude), with Muse Spark 1.3 and DeepSeek V4.1 Flash for the ids the retention table classifies: GPT-6 Astra, Gemini 3.8 Flash, Muse Spark 1.3, Qwen3.8 Max, DeepSeek V4.1 Flash and GLM-5.3, each at 36,000 output tokens. They answer the brief blind, read each other once, and the harness rolls the answers up; no synthesis seat. Several of these labs have no endpoint OpenRouter tags zero-data-retention, so this chain takes public material only. Its worst case is far above the add-on's default money limits (per call COUNCIL_ADVISE_MAX_USD_PER_CALL $1.50, per session COUNCIL_ADVISE_MAX_USD_PER_SESSION $3, per day COUNCIL_ADVISE_MAX_USD_PER_DAY $10): an operator raises all three to use it. advise.usd is this call's own ceiling
|
|
3
|
+
"description": "Council advisor, the premium option (experimental; decided rule 4: a premium-7-style setup). Selected only by the operator through COUNCIL_ADVISE_CHAIN_COUNCIL, never by a tool argument. The critics of plan-premium-7 minus its Anthropic seat (the caller is Claude), with Muse Spark 1.3 and DeepSeek V4.1 Flash for the ids the retention table classifies: GPT-6 Astra, Gemini 3.8 Flash, Muse Spark 1.3, Qwen3.8 Max, DeepSeek V4.1 Flash and GLM-5.3, each at 36,000 output tokens. They answer the brief blind, read each other once, and the harness rolls the answers up; no synthesis seat. Several of these labs have no endpoint OpenRouter tags zero-data-retention, so this chain takes public material only. Its worst case is far above the add-on's default money limits (per call COUNCIL_ADVISE_MAX_USD_PER_CALL $1.50, per session COUNCIL_ADVISE_MAX_USD_PER_SESSION $3, per day COUNCIL_ADVISE_MAX_USD_PER_DAY $10): an operator raises all three to use it. advise.usd is this call's own ceiling. maxRounds is unused by advise chains.",
|
|
4
4
|
"resumeAfterStop": false,
|
|
5
5
|
"maxRounds": 1,
|
|
6
6
|
"estimate": {
|
|
@@ -13,8 +13,7 @@
|
|
|
13
13
|
"usd": 6.5,
|
|
14
14
|
"rounds": 1,
|
|
15
15
|
"skipDebateWhenUnanimous": true,
|
|
16
|
-
"synthesis": "none"
|
|
17
|
-
"max_wall_ms": 1500000
|
|
16
|
+
"synthesis": "none"
|
|
18
17
|
},
|
|
19
18
|
"seats": {
|
|
20
19
|
"critics": [
|