@expert-council/pi-package 0.4.0 → 0.5.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +117 -7
- package/package.json +3 -3
- package/skills/expert-council/SKILL.md +3 -3
package/README.md
CHANGED
|
@@ -4,8 +4,6 @@
|
|
|
4
4
|
|
|
5
5
|
Expert Council is a local, multi-model, cost-aware expert orchestration system for Pi and MCP hosts such as Codex. It discovers the models connected through Pi's registered LLM APIs and coding plans, combines runtime metadata with user-defined billing policy, capability profiles, and local reliability data, dynamically assembles a small semantic expert team, executes bounded tasks through Pi, and returns compact structured results to the Main Agent.
|
|
6
6
|
|
|
7
|
-
> ⚠️ **Codex plugin not yet released**: the Codex plugin has not completed functional testing. Its installation and usage documentation is temporarily withheld from this version and is expected to ship with the next official release. For now, use Expert Council through the native Pi Package, the CLI, or the generic MCP Server.
|
|
8
|
-
|
|
9
7
|
Key advantages:
|
|
10
8
|
|
|
11
9
|
| Advantage | Description |
|
|
@@ -19,7 +17,7 @@ In practice, the theoretically strongest model is not automatically the best exe
|
|
|
19
17
|
|
|
20
18
|
## Current status
|
|
21
19
|
|
|
22
|
-
The current release (0.
|
|
20
|
+
The current release (0.5.1) includes:
|
|
23
21
|
|
|
24
22
|
- A host-independent Core: configuration validation, model normalization, billing policy, profile layering, role scoring, task classification, dynamic team sizing, retry/escalation, and telemetry aggregation.
|
|
25
23
|
- An execution runtime built on Pi's current `ModelRuntime` and `createAgentSession` APIs.
|
|
@@ -31,7 +29,7 @@ The current release (0.4.0) includes:
|
|
|
31
29
|
- Runtime availability markers: when a call fails with dead-model evidence, the model is recorded into the persisted assessment and hard-rejected by later council building, delegation, and escalation; markers expire and are retried automatically after 24 hours.
|
|
32
30
|
- Provider session error surfacing: upstream denials such as `403 AccessDenied` are no longer swallowed; they return to the Main Agent with the real diagnostic and the correct failure class.
|
|
33
31
|
- Cross-process shared model assessment: with multiple instances running in parallel, availability markers become visible to each other without a restart.
|
|
34
|
-
- A Codex plugin with the shared Skill and a bundled stdio MCP Server
|
|
32
|
+
- A Codex plugin with the shared Skill and a bundled stdio MCP Server, using Codex's host-owned `codex/sandbox-state-meta` capability for workspace discovery (no hooks).
|
|
35
33
|
- Deterministic automated tests that never consume model quota.
|
|
36
34
|
|
|
37
35
|
## Architecture
|
|
@@ -81,16 +79,27 @@ Requirements:
|
|
|
81
79
|
- Pi installed and configured with at least one usable model.
|
|
82
80
|
- When writable experts need worktree isolation, the Git repository must have at least one commit.
|
|
83
81
|
|
|
84
|
-
### Install the Pi Package
|
|
82
|
+
### Install the Pi Package
|
|
85
83
|
|
|
86
|
-
|
|
84
|
+
Install from npm (recommended):
|
|
87
85
|
|
|
88
86
|
```bash
|
|
89
87
|
pi install npm:@expert-council/pi-package
|
|
88
|
+
pi list
|
|
89
|
+
pi --verbose
|
|
90
|
+
```
|
|
91
|
+
|
|
92
|
+
`pi list` should show `npm:@expert-council/pi-package` and its resolved directory; a newly started verbose Pi session should load `dist/extension.js`, the `expert-council` Skill, and the eight semantic tools. To upgrade later:
|
|
93
|
+
|
|
94
|
+
```bash
|
|
90
95
|
pi update npm:@expert-council/pi-package
|
|
91
96
|
```
|
|
92
97
|
|
|
93
|
-
|
|
98
|
+
### Install the Codex plugin (optional)
|
|
99
|
+
|
|
100
|
+
To run Codex as the Main Agent, build the repository and install the bundled plugin through a Codex plugin marketplace — see [Codex plugin](#codex-plugin) for the full walkthrough.
|
|
101
|
+
|
|
102
|
+
### Build from source (development)
|
|
94
103
|
|
|
95
104
|
```bash
|
|
96
105
|
npm install
|
|
@@ -414,6 +423,8 @@ Environment overrides and CLI path flags are trusted operator inputs. In particu
|
|
|
414
423
|
|
|
415
424
|
## Native Pi Package
|
|
416
425
|
|
|
426
|
+
For daily use, install from npm (see [Quick start](#install-the-pi-package)); this section covers source development and local candidate validation.
|
|
427
|
+
|
|
417
428
|
Build and install the local candidate from the repository root. Even on Windows, use forward slashes whenever a command may pass through Pi's Bash-compatible shell; an unquoted `.\packages\pi-package` loses its backslashes before reaching Pi.
|
|
418
429
|
|
|
419
430
|
```bash
|
|
@@ -454,6 +465,105 @@ Pi loads `dist/extension.js` and the synchronized `expert-council` Skill through
|
|
|
454
465
|
|
|
455
466
|
Pi delegation is non-blocking; a single call can start up to eight independent background assignments before returning. When an expert finishes, the extension sends compact JSON containing the completed `executionId` and, only when supplied at dispatch, the `taskDescription`; it never carries feedback directly. Notifications use `steer` while the Main Agent is working and a `triggerTurn` `followUp` when it is idle. The Main Agent then calls `expert_result` for the structured feedback. Dispatch the entire ready batch before ending the turn, and avoid polling or silently waiting afterwards.
|
|
456
467
|
|
|
468
|
+
## Codex plugin
|
|
469
|
+
|
|
470
|
+
The Codex plugin turns Codex into the Main Agent: it bundles the shared `expert-council` Skill and a stdio MCP Server, while experts execute through Pi. It ships no hooks — the server uses MCP client roots when available, otherwise derives the task workspace from Codex's host-owned `codex/sandbox-state-meta` capability, then falls back to the `EXPERT_COUNCIL_WORKSPACE` override, and refuses to use the plugin installation directory as a workspace.
|
|
471
|
+
|
|
472
|
+
The built plugin root is:
|
|
473
|
+
|
|
474
|
+
```text
|
|
475
|
+
packages/codex-integration/plugin/expert-council/
|
|
476
|
+
.codex-plugin/plugin.json
|
|
477
|
+
.mcp.json
|
|
478
|
+
skills/expert-council/SKILL.md
|
|
479
|
+
dist/server.mjs
|
|
480
|
+
dist/roles/*.md
|
|
481
|
+
```
|
|
482
|
+
|
|
483
|
+
### Installation
|
|
484
|
+
|
|
485
|
+
Release `v0.5.1` includes the prebuilt MCP server, so Codex can install the plugin directly from the repository as a pinned Git marketplace. Node.js 22.19 or newer and a working Pi installation are required at runtime; cloning and building this repository is not required.
|
|
486
|
+
|
|
487
|
+
```bash
|
|
488
|
+
codex plugin marketplace add Labiey/expert-council-router --ref v0.5.1 --json
|
|
489
|
+
codex plugin marketplace list --json
|
|
490
|
+
codex plugin list --marketplace expert-council-router --available --json
|
|
491
|
+
codex plugin add expert-council@expert-council-router --json
|
|
492
|
+
codex plugin list --json
|
|
493
|
+
```
|
|
494
|
+
|
|
495
|
+
`marketplace add` is needed only once for this release. If the marketplace name is already registered from an older or local source, remove that source first or follow the upgrade procedure below. `plugin list --json` should show `expert-council` as installed from `expert-council-router`.
|
|
496
|
+
|
|
497
|
+
Codex Desktop on Windows includes the CLI, but it may not be on `PATH`. In PowerShell, resolve the running Desktop binary and use it for the same remote installation:
|
|
498
|
+
|
|
499
|
+
```powershell
|
|
500
|
+
$ecCodex = (Get-Command codex.exe -ErrorAction SilentlyContinue).Source
|
|
501
|
+
if (-not $ecCodex) {
|
|
502
|
+
$ecCodex = Get-Process codex -ErrorAction SilentlyContinue |
|
|
503
|
+
Where-Object Path |
|
|
504
|
+
Select-Object -First 1 -ExpandProperty Path
|
|
505
|
+
}
|
|
506
|
+
if (-not $ecCodex) {
|
|
507
|
+
$ecCodex = Get-ChildItem (Join-Path $env:LOCALAPPDATA "OpenAI/Codex/bin") `
|
|
508
|
+
-Filter codex.exe -File -Recurse -ErrorAction SilentlyContinue |
|
|
509
|
+
Sort-Object LastWriteTime -Descending |
|
|
510
|
+
Select-Object -First 1 -ExpandProperty FullName
|
|
511
|
+
}
|
|
512
|
+
if (-not $ecCodex) { throw "Codex Desktop CLI was not found." }
|
|
513
|
+
|
|
514
|
+
& $ecCodex plugin marketplace add Labiey/expert-council-router --ref v0.5.1 --json
|
|
515
|
+
& $ecCodex plugin marketplace list --json
|
|
516
|
+
& $ecCodex plugin list --marketplace expert-council-router --available --json
|
|
517
|
+
& $ecCodex plugin add "expert-council@expert-council-router" --json
|
|
518
|
+
& $ecCodex plugin list --json
|
|
519
|
+
```
|
|
520
|
+
|
|
521
|
+
Fully quit Codex Desktop, wait for its backend process to exit, reopen it, and start a new task. Merely opening another task is not a reliable MCP reload boundary in every Desktop build.
|
|
522
|
+
|
|
523
|
+
For local plugin development, clone the repository, run `npm ci && npm run build`, and pass its absolute root to `codex plugin marketplace add` instead of the GitHub repository name. The pinned remote release is recommended for normal use.
|
|
524
|
+
|
|
525
|
+
A correct load exposes the `expert-council` Skill and all nine `expert_*` MCP tools. If the Skill is present but the tools are absent, treat the installation as failed: restart or reinstall the plugin instead of launching `dist/server.mjs` manually or sending hand-written JSON-RPC.
|
|
526
|
+
|
|
527
|
+
To verify the installed workflow, use a new Codex task and ask:
|
|
528
|
+
|
|
529
|
+
```text
|
|
530
|
+
Use Expert Council to inspect the currently available Pi models, providers, billing classification, and model-assessment status. Return only a compact summary; do not build a council or delegate experts.
|
|
531
|
+
```
|
|
532
|
+
|
|
533
|
+
The task should invoke `expert_inspect`. It should not ask for hook trust, manually launch the MCP server, or treat the plugin cache as the project workspace.
|
|
534
|
+
|
|
535
|
+
### Upgrade
|
|
536
|
+
|
|
537
|
+
A marketplace pinned with `--ref` intentionally stays on that release. To upgrade, remove the installed plugin and old marketplace registration, then add the new tag and reinstall:
|
|
538
|
+
|
|
539
|
+
```bash
|
|
540
|
+
codex plugin remove expert-council@expert-council-router --json
|
|
541
|
+
codex plugin marketplace remove expert-council-router --json
|
|
542
|
+
codex plugin marketplace add Labiey/expert-council-router --ref vX.Y.Z --json
|
|
543
|
+
codex plugin add expert-council@expert-council-router --json
|
|
544
|
+
```
|
|
545
|
+
|
|
546
|
+
Replace `vX.Y.Z` with the intended release. Users who deliberately track the default branch can omit `--ref` and later run `codex plugin marketplace upgrade expert-council-router --json`, but pinned tags are safer for normal use. After reinstalling, fully restart Codex Desktop and test in a new task. Avoid installing multiple copies that all declare the `expert_council` MCP server.
|
|
547
|
+
|
|
548
|
+
### Behavior highlights
|
|
549
|
+
|
|
550
|
+
- The bundled `.mcp.json` raises the host tool-call ceiling to 3660 seconds so a single bounded `expert_wait` can block until completion; the Skill still requires explicit per-operation deadlines rather than treating that ceiling as a default budget.
|
|
551
|
+
- On the first council of a conversation the Main Agent establishes exactly one cost policy with you (economy, balanced, or speed); until then `expert_build` and `expert_delegate` responses carry reminders to ask.
|
|
552
|
+
- Writable experts mutate inside a detached Git worktree under the trusted workspace; changes come back for Main Agent review and are never auto-merged.
|
|
553
|
+
|
|
554
|
+
### Removal
|
|
555
|
+
|
|
556
|
+
Remove the plugin and its marketplace registration:
|
|
557
|
+
|
|
558
|
+
```bash
|
|
559
|
+
codex plugin remove expert-council@expert-council-router --json
|
|
560
|
+
codex plugin marketplace remove expert-council-router --json
|
|
561
|
+
codex plugin list --json
|
|
562
|
+
codex plugin marketplace list --json
|
|
563
|
+
```
|
|
564
|
+
|
|
565
|
+
On Windows, replace `codex` with `& $ecCodex` when using the PowerShell variable above. Fully quit Codex Desktop before starting new tasks.
|
|
566
|
+
|
|
457
567
|
## Testing
|
|
458
568
|
|
|
459
569
|
```bash
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@expert-council/pi-package",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.5.1",
|
|
4
4
|
"description": "Native Pi package exposing Expert Council tools and host guidance",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"main": "./dist/extension.js",
|
|
@@ -17,8 +17,8 @@
|
|
|
17
17
|
"skills": ["./skills"]
|
|
18
18
|
},
|
|
19
19
|
"dependencies": {
|
|
20
|
-
"@expert-council/core": "0.
|
|
21
|
-
"@expert-council/pi-runtime": "0.
|
|
20
|
+
"@expert-council/core": "0.5.1",
|
|
21
|
+
"@expert-council/pi-runtime": "0.5.1"
|
|
22
22
|
},
|
|
23
23
|
"peerDependencies": {
|
|
24
24
|
"@earendil-works/pi-coding-agent": ">=0.84.0 <1",
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: expert-council
|
|
3
|
-
description: Use
|
|
3
|
+
description: Use the Pi expert council by default for substantial work — code review, architecture analysis, bug investigation, feature implementation, refactoring, repository exploration, test writing, or verification — whenever delegation saves Main Agent context, execution effort, or scarce-model quota. Engage it in every conversation turn where such work appears, including later turns; skip it only for trivial single-file lookups or one-line edits.
|
|
4
4
|
---
|
|
5
5
|
|
|
6
6
|
# Expert Council
|
|
@@ -11,8 +11,8 @@ Use Expert Council only when delegation is likely to save significant context, e
|
|
|
11
11
|
2. Before building the first council in a new conversation, establish exactly one assembly preference. If the user has not already stated one, ask whether to optimize for `economy` (lowest effective marginal cost), `balanced` (cost, completion time, and success probability), or `speed` (fastest completion). Stop and wait for the answer. Reuse that choice for later councils in the same conversation; ask again only if the user changes it. The native Pi tool enforces this fallback and stores the choice in session state.
|
|
12
12
|
3. The Main Agent, not the user, assesses which models fit each role. If `modelAssessment.status` is `required`, do not call `expert_build` or `expert_delegate` yet. Use an already available network research tool to cross-check current evidence only for keys in `researchModels`. Preserve saved scores for the remaining `requiredModels` entries. If no network research tool is available, stop and tell the user that the mandatory audit cannot be completed; never silently fall back to unaudited defaults. Never install a browser, Skill, plugin, or executable package automatically.
|
|
13
13
|
4. For the mandatory audit, consider only models returned by `expert_inspect`. Prefer current primary or first-party evidence: Artificial Analysis for coding/agentic scores, price, throughput, and latency; LiveBench for coding and agentic coding; Arena for human preference; provider documentation for model/version/runtime and access-method facts. Treat OpenRouter usage rankings only as adoption evidence, not proof of quality. Triangulate into 1–12 consolidated source URLs, read the actual host clock for `asOf` (never synthesize a future timestamp), normalize only supported capability dimensions to 0–10, merge researched entries with saved entries to cover every `requiredModels` key, and include verified provider billing/access classifications when known; leave them `unknown` rather than guessing. Do not infer personal billing from published per-token prices. Pass the complete result as `modelAssessment` to `expert_build`. If the gate reports `future-dated`, retain the same evidence and scores, correct only `asOf`, and do not browse again. When `modelAssessment.status` is `current`, omit `modelAssessment` from `expert_build` so the saved snapshot is reused; do not send an empty or reconstructed table and do not browse again unless the user requests a re-audit.
|
|
14
|
-
5. Call `expert_build` only after the assessment gate is current, with the task and established cost policy. If the tool still returns `model-assessment-required`, follow its instructions and retry once after completing the audit; do not repeatedly call it unchanged. Let deterministic routing combine the Main Agent assessment, objective Pi metadata, billing policy when known, and locally observed reliability. Do not make the user rank individual models manually and do not assume a newer or more popular model is automatically best for every role.
|
|
15
|
-
6. Delegate bounded semantic assignments by role with `expert_delegate`. It starts background work and immediately returns execution IDs. Set an explicit finite `timeoutMs` for every assignment according to task difficulty instead of relying on the runtime fallback: normally 2–5 minutes for focused read-only work, 10–20 minutes for ordinary implementation/debugging, and at most 30–60 minutes for justified long-running work. When two or more independent assignments are ready, send them together in `assignments` as a real JSON array, never a quoted/stringified JSON value, so the entire batch is dispatched before the host turn ends. Add a short `taskDescription` when recognizing a task later would help.
|
|
14
|
+
5. Call `expert_build` only after the assessment gate is current, with the task and the established cost policy passed as `constraints.costPolicy`. If the tool still returns `model-assessment-required`, follow its instructions and retry once after completing the audit; do not repeatedly call it unchanged. When the response warns that no costPolicy was supplied, stop and ask the user once for economy, balanced, or speed before the next build. Let deterministic routing combine the Main Agent assessment, objective Pi metadata, billing policy when known, and locally observed reliability. Do not make the user rank individual models manually and do not assume a newer or more popular model is automatically best for every role.
|
|
15
|
+
6. Delegate bounded semantic assignments by role with `expert_delegate`. It starts background work and immediately returns execution IDs. If no cost policy has been established in this conversation, ask the user once (economy, balanced, or speed) before the first dispatch; an `expert_delegate` or `expert_build` response carrying `reminders` means the policy is still missing. Set an explicit finite `timeoutMs` for every assignment according to task difficulty instead of relying on the runtime fallback: normally 2–5 minutes for focused read-only work, 10–20 minutes for ordinary implementation/debugging, and at most 30–60 minutes for justified long-running work. When two or more independent assignments are ready, send them together in `assignments` as a real JSON array, never a quoted/stringified JSON value, so the entire batch is dispatched before the host turn ends. Add a short `taskDescription` when recognizing a task later would help. If a writable assignment is rejected because the runtime provides no mutation, retry once with `workspace` set to the target project's Git repository root inside the trusted boundary — the conversation's startup folder itself may not be a repository.
|
|
16
16
|
7. After dispatching the complete batch, continue any independent Main Agent work. Follow the host-specific completion workflow appended by the distribution; do not poll repeatedly or spend model tokens on an idle loop.
|
|
17
17
|
8. When the host reports a completed `executionId`, call `expert_result` with that exact ID before using the feedback. Prefer read-only investigation before mutation, retain architecture and final acceptance in the Main Agent, and never create a redundant lead expert when the host Main Agent already owns that role.
|
|
18
18
|
9. Verify important results, then call `expert_feedback` so local routing can learn. Inspect isolated worktree changes before integration and call `expert_cleanup` after integrating or rejecting them. Do not delegate trivial work or allow recursive expert delegation.
|