@sleepy_panda_srl/portulan 0.2.0 → 0.3.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +2 -2
- package/cli/README.md +6 -3
- package/cli/compile.mjs +318 -13
- package/cli/doctor.mjs +46 -7
- package/cli/drills.mjs +26 -4
- package/cli/ledger.mjs +110 -6
- package/cli/portulan.mjs +1 -1
- package/cli/review-meter.mjs +14 -9
- package/cli/telemetry.mjs +57 -15
- package/core/operating/context.md +79 -4
- package/core/personas/README.md +33 -0
- package/core/skills/clarify/SKILL.md +1 -1
- package/core/skills/consolidate/SKILL.md +1 -1
- package/evals/README.md +23 -7
- package/evals/releases/0.3.0.json +150 -0
- package/evals/releases/0.3.0.md +68 -0
- package/package.json +2 -1
- package/packs/.claude-plugin/plugin.json +2 -1
- package/packs/README.md +7 -4
- package/packs/tools/README.md +8 -8
- package/packs/tools/local-llm/README.md +136 -0
- package/packs/tools/local-llm/pack.json +32 -0
- package/packs/tools/local-llm/personas/local-worker.md +44 -0
- package/packs/tools/local-llm/runner/local-llm.mjs +1665 -0
- package/packs/tools/local-llm/skills/delegate-local/SKILL.md +64 -0
- package/packs/tools/local-llm/verify/README.md +18 -0
- package/spec/README.md +1 -0
- package/spec/slots.md +10 -6
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: consolidate
|
|
3
|
-
description: Bring a memory store back under its budget by merging, compressing, splitting, and retiring records
|
|
3
|
+
description: Bring a memory store back under its budget by merging, compressing, splitting, and retiring records. Use when the index verify recipe reports a breach, or on the librarian's scheduled pass. The output is drafts through the human gate; raising the budget is not one of the moves.
|
|
4
4
|
---
|
|
5
5
|
|
|
6
6
|
# Skill — Consolidate
|
package/evals/README.md
CHANGED
|
@@ -122,6 +122,9 @@ node cli/review-meter.mjs --fetch --repo <owner/name> --out <file> the on
|
|
|
122
122
|
with `--register evals/review-loop/register.md --write` regenerates it, and the recipe byte-compares it.
|
|
123
123
|
The criterion's *"rounds per pull request"* is **submissions** per pull request — its unit before *round*
|
|
124
124
|
was redefined — and the tool prints that name, never the bare word.
|
|
125
|
+
**The register is historical.** The loop it measures is Copilot's, and Copilot review left this
|
|
126
|
+
repository's review path on 2026-10-30, by the maintainer's decision, so the capture is the record of that
|
|
127
|
+
loop rather than a reading of a live one.
|
|
125
128
|
What each figure is, at length, is in the header at `git show a31da46:cli/review-meter.mjs`.
|
|
126
129
|
|
|
127
130
|
**What it does NOT establish:**
|
|
@@ -133,9 +136,9 @@ What each figure is, at length, is in the header at `git show a31da46:cli/review
|
|
|
133
136
|
workspace-layer matcher the engine tool refused to copy, since two spellings of one rule on opposite
|
|
134
137
|
sides of that boundary cannot see each other drift; the matcher left with its workflow on 2026-09-23
|
|
135
138
|
([#355](https://github.com/sleepy-panda-srl/portulan/issues/355)).
|
|
136
|
-
- **A bound.** It is a meter
|
|
137
|
-
[`a-review-loop-needs-a-bound.md`](
|
|
138
|
-
|
|
139
|
+
- **A bound.** It is a meter, and nothing here stops anything. The bound was rule 4 of
|
|
140
|
+
[`a-review-loop-needs-a-bound.md`](https://github.com/sleepy-panda-srl/portulan/blob/7d72356/.portulan/memory/a-review-loop-needs-a-bound.md),
|
|
141
|
+
retired on 2026-10-30.
|
|
139
142
|
- **Current data.** The recipe compares the register to the snapshot, never the snapshot to the world, so a
|
|
140
143
|
stale capture and a current one are the same green. Refreshing is `--fetch`, run by a person
|
|
141
144
|
([#356](https://github.com/sleepy-panda-srl/portulan/issues/356)).
|
|
@@ -232,12 +235,14 @@ is a new consent question rather than a covered one. It neither queues nor retri
|
|
|
232
235
|
path. The send is proven only in the suite, against an injected transport, and no yielded recipe may
|
|
233
236
|
ever exercise `--export`.
|
|
234
237
|
- **A network mode reached indirectly.** The suite derives the set of network-capable modules from the
|
|
235
|
-
tree, and the offline audit matches a module as a path suffix and a flag as a token
|
|
236
|
-
|
|
238
|
+
tree, and the offline audit matches a module as a path suffix and a flag as a token, or, for the
|
|
239
|
+
local-llm runner, any invocation but `--verify-offline` alone or `eval --verify` alone on its command; a mode reached through
|
|
240
|
+
another script, or through a flag built at runtime, is invisible to it.
|
|
237
241
|
- **Scheduled emission.** Nothing fires when nobody runs it —
|
|
238
242
|
[#344](https://github.com/sleepy-panda-srl/portulan/issues/344)'s silence again.
|
|
239
|
-
- **Enforcement of the review-loop bound.** An emitter with no backend reading it
|
|
240
|
-
[`a-review-loop-needs-a-bound.md`](
|
|
243
|
+
- **Enforcement of the review-loop bound.** An emitter with no backend reading it was never the checker
|
|
244
|
+
[`a-review-loop-needs-a-bound.md`](https://github.com/sleepy-panda-srl/portulan/blob/7d72356/.portulan/memory/a-review-loop-needs-a-bound.md) lacked, and that record
|
|
245
|
+
retired on 2026-10-30.
|
|
241
246
|
- **Traces.** Metrics only.
|
|
242
247
|
|
|
243
248
|
## The A/B baseline
|
|
@@ -262,6 +267,17 @@ capture. The recipes run no agent; the modes that do — `ab --stop-probe`, `ab-
|
|
|
262
267
|
and the register cites it rather than restating it. **The baseline is scoped to the vendored-and-compiled
|
|
263
268
|
tier** [`ab/arm.md`](ab/arm.md) specifies, and closes row 8 for no other configuration of *Portulan on*.
|
|
264
269
|
|
|
270
|
+
## The local-worker routing capture — milestone 14
|
|
271
|
+
|
|
272
|
+
[`ab/local-worker.json`](ab/local-worker.json) is the immutable acquired local/host capture;
|
|
273
|
+
[`ab/local-worker.md`](ab/local-worker.md) is its generated register. The
|
|
274
|
+
[acquisition note](ab/local-worker-acquisition.md) records conditions, recovery and limitations.
|
|
275
|
+
`node packs/tools/local-llm/runner/local-llm.mjs eval --verify` regrades and byte-compares without a
|
|
276
|
+
model. `node cli/local-routing.mjs` additionally compares the shipped role flags, model IDs and
|
|
277
|
+
capture digest with an independent derivation. Both run beside the original baseline in `ab-run`.
|
|
278
|
+
The acquired five-turn cells support code, while the shared prose role is disabled. These rates
|
|
279
|
+
support no general quality claim, and CI cannot replay the events that produced them.
|
|
280
|
+
|
|
265
281
|
## A release carries an eval result
|
|
266
282
|
|
|
267
283
|
```
|
|
@@ -0,0 +1,150 @@
|
|
|
1
|
+
{
|
|
2
|
+
"portulan": {
|
|
3
|
+
"releaseEval": "1"
|
|
4
|
+
},
|
|
5
|
+
"version": "0.3.0",
|
|
6
|
+
"captured": "2026-11-03",
|
|
7
|
+
"source": {
|
|
8
|
+
"commit": "c08515fee0693cb0fc07fc6077fb152aa46bc28b",
|
|
9
|
+
"clean": false
|
|
10
|
+
},
|
|
11
|
+
"host": {
|
|
12
|
+
"node": "v26.12.1",
|
|
13
|
+
"platform": "darwin"
|
|
14
|
+
},
|
|
15
|
+
"recipes": [
|
|
16
|
+
{
|
|
17
|
+
"id": "docs",
|
|
18
|
+
"exit": 0
|
|
19
|
+
},
|
|
20
|
+
{
|
|
21
|
+
"id": "json",
|
|
22
|
+
"exit": 0
|
|
23
|
+
},
|
|
24
|
+
{
|
|
25
|
+
"id": "doctor",
|
|
26
|
+
"exit": 0
|
|
27
|
+
},
|
|
28
|
+
{
|
|
29
|
+
"id": "tests",
|
|
30
|
+
"exit": 0
|
|
31
|
+
},
|
|
32
|
+
{
|
|
33
|
+
"id": "plugin",
|
|
34
|
+
"exit": 0
|
|
35
|
+
},
|
|
36
|
+
{
|
|
37
|
+
"id": "compile",
|
|
38
|
+
"exit": 0
|
|
39
|
+
},
|
|
40
|
+
{
|
|
41
|
+
"id": "workflow-filters",
|
|
42
|
+
"exit": 0
|
|
43
|
+
},
|
|
44
|
+
{
|
|
45
|
+
"id": "index",
|
|
46
|
+
"exit": 0
|
|
47
|
+
},
|
|
48
|
+
{
|
|
49
|
+
"id": "context",
|
|
50
|
+
"exit": 0
|
|
51
|
+
},
|
|
52
|
+
{
|
|
53
|
+
"id": "ledger",
|
|
54
|
+
"exit": 0
|
|
55
|
+
},
|
|
56
|
+
{
|
|
57
|
+
"id": "control-chars",
|
|
58
|
+
"exit": 0
|
|
59
|
+
},
|
|
60
|
+
{
|
|
61
|
+
"id": "comments",
|
|
62
|
+
"exit": 0
|
|
63
|
+
},
|
|
64
|
+
{
|
|
65
|
+
"id": "rule-carriers",
|
|
66
|
+
"exit": 0
|
|
67
|
+
},
|
|
68
|
+
{
|
|
69
|
+
"id": "pack-version",
|
|
70
|
+
"exit": 0
|
|
71
|
+
},
|
|
72
|
+
{
|
|
73
|
+
"id": "pack-identity",
|
|
74
|
+
"exit": 0
|
|
75
|
+
},
|
|
76
|
+
{
|
|
77
|
+
"id": "payload",
|
|
78
|
+
"exit": 0
|
|
79
|
+
},
|
|
80
|
+
{
|
|
81
|
+
"id": "eval-bundle",
|
|
82
|
+
"exit": 0
|
|
83
|
+
},
|
|
84
|
+
{
|
|
85
|
+
"id": "goldens",
|
|
86
|
+
"exit": 0
|
|
87
|
+
},
|
|
88
|
+
{
|
|
89
|
+
"id": "mutants",
|
|
90
|
+
"exit": 0
|
|
91
|
+
},
|
|
92
|
+
{
|
|
93
|
+
"id": "fuzz-shell",
|
|
94
|
+
"exit": 0
|
|
95
|
+
},
|
|
96
|
+
{
|
|
97
|
+
"id": "version-carriers",
|
|
98
|
+
"exit": 0
|
|
99
|
+
},
|
|
100
|
+
{
|
|
101
|
+
"id": "skill-goldens",
|
|
102
|
+
"exit": 0
|
|
103
|
+
},
|
|
104
|
+
{
|
|
105
|
+
"id": "review-loop",
|
|
106
|
+
"exit": 0
|
|
107
|
+
},
|
|
108
|
+
{
|
|
109
|
+
"id": "telemetry",
|
|
110
|
+
"exit": 0
|
|
111
|
+
},
|
|
112
|
+
{
|
|
113
|
+
"id": "drills",
|
|
114
|
+
"exit": 0
|
|
115
|
+
},
|
|
116
|
+
{
|
|
117
|
+
"id": "ab",
|
|
118
|
+
"exit": 0
|
|
119
|
+
},
|
|
120
|
+
{
|
|
121
|
+
"id": "ab-grade",
|
|
122
|
+
"exit": 0
|
|
123
|
+
},
|
|
124
|
+
{
|
|
125
|
+
"id": "ab-run",
|
|
126
|
+
"exit": 0
|
|
127
|
+
},
|
|
128
|
+
{
|
|
129
|
+
"id": "tools/github:actions-pinned",
|
|
130
|
+
"exit": 0
|
|
131
|
+
},
|
|
132
|
+
{
|
|
133
|
+
"id": "tools/local-llm:runner-rails",
|
|
134
|
+
"exit": 0
|
|
135
|
+
}
|
|
136
|
+
],
|
|
137
|
+
"excluded": [
|
|
138
|
+
{
|
|
139
|
+
"id": "release-eval",
|
|
140
|
+
"why": "a capture cannot be accurate about the record it is inside — captured before the record is written it is red for the record's absence, and captured after it is stale the moment the register is rendered. Its verdict for this release is the rail's own run on the pull request carrying this file."
|
|
141
|
+
}
|
|
142
|
+
],
|
|
143
|
+
"abBaseline": {
|
|
144
|
+
"snapshot": "evals/ab/baseline.json",
|
|
145
|
+
"register": "evals/ab/baseline.md",
|
|
146
|
+
"captured": "2026-08-31T09:05:32.699Z",
|
|
147
|
+
"commit": "a642d551eec6c146c67ef0acfe3f8ba535c5ce6e",
|
|
148
|
+
"clean": false
|
|
149
|
+
}
|
|
150
|
+
}
|
|
@@ -0,0 +1,68 @@
|
|
|
1
|
+
# Eval result — Portulan 0.3.0
|
|
2
|
+
|
|
3
|
+
> **Generated. Do not edit.** Rendered from [`evals/releases/0.3.0.json`](0.3.0.json) by [`cli/release-eval.mjs`](../../cli/release-eval.mjs) and byte-compared by the `release-eval` verify recipe. This is the eval result milestone 8 requires a release to carry — `docs/plan.md`, Protocol → Versioning.
|
|
4
|
+
|
|
5
|
+
## What this release was measured at
|
|
6
|
+
|
|
7
|
+
| | |
|
|
8
|
+
|---|---|
|
|
9
|
+
| Version | `0.3.0` |
|
|
10
|
+
| Captured | 2026-11-03 |
|
|
11
|
+
| Commit | `c08515fee0693cb0fc07fc6077fb152aa46bc28b` |
|
|
12
|
+
| Tree at capture | **not clean** |
|
|
13
|
+
| Node | `v26.12.1` |
|
|
14
|
+
| Platform | `darwin` |
|
|
15
|
+
|
|
16
|
+
## The rails, and what each returned
|
|
17
|
+
|
|
18
|
+
30 of the 31 recipes this workspace yielded at capture. The set is derived from [`cli/recipe-set.mjs`](../../cli/recipe-set.mjs) and never listed by hand.
|
|
19
|
+
|
|
20
|
+
| Recipe | Verdict |
|
|
21
|
+
|---|---|
|
|
22
|
+
| `docs` | green |
|
|
23
|
+
| `json` | green |
|
|
24
|
+
| `doctor` | green |
|
|
25
|
+
| `tests` | green |
|
|
26
|
+
| `plugin` | green |
|
|
27
|
+
| `compile` | green |
|
|
28
|
+
| `workflow-filters` | green |
|
|
29
|
+
| `index` | green |
|
|
30
|
+
| `context` | green |
|
|
31
|
+
| `ledger` | green |
|
|
32
|
+
| `control-chars` | green |
|
|
33
|
+
| `comments` | green |
|
|
34
|
+
| `rule-carriers` | green |
|
|
35
|
+
| `pack-version` | green |
|
|
36
|
+
| `pack-identity` | green |
|
|
37
|
+
| `payload` | green |
|
|
38
|
+
| `eval-bundle` | green |
|
|
39
|
+
| `goldens` | green |
|
|
40
|
+
| `mutants` | green |
|
|
41
|
+
| `fuzz-shell` | green |
|
|
42
|
+
| `version-carriers` | green |
|
|
43
|
+
| `skill-goldens` | green |
|
|
44
|
+
| `review-loop` | green |
|
|
45
|
+
| `telemetry` | green |
|
|
46
|
+
| `drills` | green |
|
|
47
|
+
| `ab` | green |
|
|
48
|
+
| `ab-grade` | green |
|
|
49
|
+
| `ab-run` | green |
|
|
50
|
+
| `tools/github:actions-pinned` | green |
|
|
51
|
+
| `tools/local-llm:runner-rails` | green |
|
|
52
|
+
|
|
53
|
+
**`release-eval` is excluded**, and the reason is printed rather than the row silently dropped: a capture cannot be accurate about the record it is inside — captured before the record is written it is red for the record's absence, and captured after it is stale the moment the register is rendered. Its verdict for this release is the rail's own run on the pull request carrying this file.
|
|
54
|
+
|
|
55
|
+
## The A/B baseline this release ships against
|
|
56
|
+
|
|
57
|
+
[`evals/ab/baseline.md`](https://github.com/sleepy-panda-srl/portulan/blob/v0.3.0/evals/ab/baseline.md), rendered from `evals/ab/baseline.json` — captured 2026-08-31T09:05:32.699Z at `a642d551eec6c146c67ef0acfe3f8ba535c5ce6e`, over a tree that was **not clean**.
|
|
58
|
+
|
|
59
|
+
**Its figures are not repeated here.** They have one carrier and this is not it.
|
|
60
|
+
|
|
61
|
+
## What this record does not establish
|
|
62
|
+
|
|
63
|
+
- **A green says this record agrees with its own capture — never that the release is good.** Every line below is what the recipes returned at one commit, and a recipe's own green establishes only what that recipe's documentation says it establishes.
|
|
64
|
+
- **The recipes were not run at the tag.** They ran at `c08515fee0693cb0fc07fc6077fb152aa46bc28b`, and this record is committed *in* the cut change — so the tagged tree is this commit plus whatever that change still adds after the capture, the record itself at minimum. A record cannot be captured at a commit that does not exist yet, and printing the one it was captured at is the only honest form.
|
|
65
|
+
- **`release-eval` is excluded from the rows above**, because a capture cannot be accurate about the record it is inside. Its verdict for this release is the rail's own run on the pull request that carries this file.
|
|
66
|
+
- **This record measures the Portulan repository's own build**, at the commit named above — never the workspace, project or package this release is installed into. Nothing here has looked at your tree.
|
|
67
|
+
- **The tree was NOT clean at capture** — the diff of the cut change is the only record of what was uncommitted, which is weaker than a sha and is printed rather than hidden.
|
|
68
|
+
- **The A/B baseline's figures are NOT restated here.** This record names which baseline the release ships against; the figures and everything that may not be concluded from them live in that register alone.
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@sleepy_panda_srl/portulan",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.3.0",
|
|
4
4
|
"description": "Portulan — an agentic-engineering framework whose product is its files. The CLI wraps the file-based mechanics for hosts beyond Claude Code.",
|
|
5
5
|
"license": "Apache-2.0",
|
|
6
6
|
"type": "module",
|
|
@@ -26,6 +26,7 @@
|
|
|
26
26
|
"!cli/ab.mjs",
|
|
27
27
|
"!cli/ab-run.mjs",
|
|
28
28
|
"!cli/ab-grade.mjs",
|
|
29
|
+
"!cli/local-routing.mjs",
|
|
29
30
|
"!cli/warm.mjs",
|
|
30
31
|
"!cli/payload.mjs",
|
|
31
32
|
"!cli/roster.mjs"
|
package/packs/README.md
CHANGED
|
@@ -31,7 +31,9 @@ registers nothing, silently, while a validator walking deeper counts three;
|
|
|
31
31
|
manifests therefore sit in this tree and they answer different questions: the repository's own at
|
|
32
32
|
[`../.claude-plugin/plugin.json`](../.claude-plugin/plugin.json), which declares
|
|
33
33
|
`./packs/rituals/checkpoints/skills/` because *its* root is the repository; and this one, whose root is
|
|
34
|
-
`packs/`. Neither is redundant, and the paths differ by exactly the prefix their roots differ by.
|
|
34
|
+
`packs/`. Neither is redundant, and the paths differ by exactly the prefix their roots differ by. Both
|
|
35
|
+
also declare `tools/local-llm`'s `skills/` root, the same way, since this repository's workspace composes
|
|
36
|
+
that pack too; the plugin's name and version here are the checkpoints pack's.
|
|
35
37
|
|
|
36
38
|
**The version here mirrors the pack's own** — `portulan.version` in
|
|
37
39
|
[`rituals/checkpoints/pack.json`](rituals/checkpoints/pack.json), which the Pack Definition names as how
|
|
@@ -45,7 +47,8 @@ observed through a *local* marketplace over a copy of this directory. The quoted
|
|
|
45
47
|
exists and the feed's pin moves to it. Until then the green is demonstrated on the payload and inferred
|
|
46
48
|
for the feed — a distinction this repository would rather write down than round off.
|
|
47
49
|
|
|
48
|
-
> **
|
|
49
|
-
> ritual;
|
|
50
|
-
>
|
|
50
|
+
> **Three packs exist.** [`rituals/checkpoints`](rituals/checkpoints/README.md), the supervised-build
|
|
51
|
+
> ritual; [`tools/github`](tools/github/README.md), one verify recipe; and
|
|
52
|
+
> [`tools/local-llm`](tools/local-llm/README.md), hybrid local-model execution under a zero-floor input
|
|
53
|
+
> budget and a verify-gated rollback loop. `stacks/` is still empty and says why.
|
|
51
54
|
|
package/packs/tools/README.md
CHANGED
|
@@ -3,12 +3,12 @@
|
|
|
3
3
|
Tool packs — integrations for the tools and MCP servers an agent drives (issue trackers, cloud
|
|
4
4
|
CLIs, design tools, databases), with the how-to and the gate classification for each.
|
|
5
5
|
|
|
6
|
-
> **
|
|
7
|
-
> host-neutral vocabulary, dispatched on by
|
|
8
|
-
> cascade's missing middle (`core < pack < workspace`)
|
|
9
|
-
> fragments are resolved and merged into the policy,
|
|
6
|
+
> **Two tool packs exist** — [`github/`](github/README.md) and [`local-llm/`](local-llm/README.md). The
|
|
7
|
+
> machinery that reads one is a gate policy in a host-neutral vocabulary, dispatched on by
|
|
8
|
+
> [`../../cli/compile.mjs`](../../cli/compile.mjs) — and the cascade's missing middle (`core < pack < workspace`)
|
|
9
|
+
> is bridged: a pack's `contributes.gates` fragments are resolved and merged into the policy,
|
|
10
|
+
> **tighten-only**, by the same compiler.
|
|
10
11
|
>
|
|
11
|
-
>
|
|
12
|
-
>
|
|
13
|
-
>
|
|
14
|
-
> a workspace here drives a tool that needs one.
|
|
12
|
+
> [`local-llm/`](local-llm/README.md) is the first tool pack to ship a gate classification
|
|
13
|
+
> (`local-llm-off-loopback` at `propose`) alongside a persona and an offline verify recipe,
|
|
14
|
+
> fulfilling the promise this file has carried since it was written.
|
|
@@ -0,0 +1,136 @@
|
|
|
1
|
+
# `tools/local-llm` — Hybrid local-model execution under Portulan's rails
|
|
2
|
+
|
|
3
|
+
A tool pack for **Hybrid AI Orchestration**, argued in Proposal [`0039`](../../../.portulan/proposals/0039-a-local-model-runs-without-the-host-floor-and-under-a-thinking-cap.md). It lets a frontier orchestrator session offload bounded code transformations, verify-gated red→green retries and a screen of a curated slot for duplicate files to a local OpenAI-compatible runtime (such as **LM Studio** on `http://127.0.0.1:1234/v1` or **Ollama** on `http://127.0.0.1:11434/v1`) at **zero API token cost** and with a **zero-floor prompt prefix**.
|
|
4
|
+
|
|
5
|
+
Composed through a workspace's `packs` array as `tools/local-llm`.
|
|
6
|
+
|
|
7
|
+
| Contributes | What |
|
|
8
|
+
|---|---|
|
|
9
|
+
| Skill | [`delegate-local`](skills/delegate-local/SKILL.md) — the pass a frontier session follows to hand a bounded edit or digest to the runner; its description names the precondition, a workspace composing this pack and a server answering on loopback |
|
|
10
|
+
| Persona | [`local-worker`](personas/local-worker.md) — a context-firewall worker operating with an omitted host floor (`omitClaudeMd`) and a hard input-byte ceiling |
|
|
11
|
+
| Verify | [`runner-rails`](verify/README.md) — offline verification of the runner's loopback-only guard, input-budget rail, rollback contract, and `dedup`'s refusals (zero network calls) |
|
|
12
|
+
| Gate | `local-llm-off-loopback` (`propose`) — marks leaving loopback (`127.0.0.1` / `localhost` / `[::1]`) as a decision requiring explicit approval |
|
|
13
|
+
|
|
14
|
+
The skill registers where a plugin manifest declares `skills/`, and this repository's manifests may
|
|
15
|
+
declare only the skills of packs its own workspace composes, as its `.portulan/workspace.json` composes
|
|
16
|
+
this one.
|
|
17
|
+
|
|
18
|
+
## Why this pack exists
|
|
19
|
+
|
|
20
|
+
Proposal [`0036`](../../../.portulan/proposals/0036-what-a-host-loads-into-every-context-is-budgeted.md) budgets what a host loads into every fresh context, and Proposal [`0038`](../../../.portulan/proposals/0038-what-a-change-spends-is-measured.md) prices what a change spends across turns (`m_r × (F·T + g·T²/2)`). Together they expose two structural facts about hybrid local+cloud development:
|
|
21
|
+
|
|
22
|
+
1. **Parent-context turns are quadratic; surgical local calls are flat and free.** Every tool call in a frontier session re-reads the entire session history. Offloading a 10-file comment cleanup, a test-fixture draft, or a file summary to a local 27B–31B model (`qwen/qwen3.8-27b`, `google/gemma-4-31b`) keeps intermediate turns and raw file reads out of the frontier context window completely.
|
|
23
|
+
2. **Local models choke on the host floor (`F`), not on the task.** A cloud subagent inherits a 44k–85k token prefix (`F`) subsidized by provider prompt caching. On local hardware, pre-filling 85k tokens into a 30B model's KV cache takes tens of seconds before the first token appears, whereas a **surgical prompt of a few hundred tokens** (built using [`cli/symbols.mjs`](../../../cli/symbols.mjs) spans) pre-filled in 0.85–1.9 seconds in proposal 0039's measurement.
|
|
24
|
+
|
|
25
|
+
## What the runner enforces in code
|
|
26
|
+
|
|
27
|
+
[`runner/local-llm.mjs`](runner/local-llm.mjs) has **zero npm dependencies** (Node.js `>=20` built-in `fetch` and `node:child_process`) and enforces five rails:
|
|
28
|
+
|
|
29
|
+
1. **Loopback-only by default.** Only `127.0.0.1`, `localhost`, and `[::1]` are permitted unless `--allow-remote` is explicitly passed, and a redirect is refused rather than followed off the machine. A proxy listening on loopback that forwards elsewhere is not visible to it, which is what the `local-llm-off-loopback` gate fragment records.
|
|
30
|
+
2. **Zero-floor input budget (`--max-input-bytes`, default `32768`).** Refuses oversized prompts, and for `dedup` any file over the ceiling, before hitting the local server, directing the caller to scope with [`cli/symbols.mjs`](../../../cli/symbols.mjs) and `--lines` first. A value that is not a whole number is refused rather than read as no limit.
|
|
31
|
+
3. **Explicit thinking cap (`--effort low`).** Passes `reasoning_effort: "low"` and `enable_thinking: false` by default so models whose chat templates default to `'xhigh'` thinking do not exhaust `max_tokens` inside `<think>` blocks.
|
|
32
|
+
4. **Verify-gated transforms (`transform --file <path> --verify <cmd>`, `--verify` required).** Applies the local model's edit, runs the verify command, retries with the end of its output on exit `1`, and **restores the original file byte-for-byte** unless the command reaches exit `0`. Any exit other than `0` or `1` — `2`, a missing command, a signal — is no verdict: the file is restored at once and the runner exits `2`. SIGINT, SIGTERM and SIGHUP, whether they arrive during the model call or while the verify command runs, stop the verify, restore the file and exit `2`. **The limits:** this is a rollback, not an atomic write — any other signal that ends the runner (SIGKILL and SIGQUIT among them) or a power loss between the edit and its verdict leaves the edit in place, and since the verify runs in a process group of its own, such a signal leaves the verify running too, even one sent to the runner's whole group. Signal handling is POSIX's: on Windows no SIGTERM arrives, and stopping the verify stops only its shell.
|
|
33
|
+
5. **Role screening (`eval`).** Grades an installed local model against Portulan's own [`cli/comments.mjs`](../../../cli/comments.mjs), [`cli/doctor.mjs`](../../../cli/doctor.mjs), and [`cli/skill-goldens.mjs`](../../../cli/skill-goldens.mjs) predicates across `code`, `codify`, and `handoff` tasks. It takes **one sample per task and has no baseline arm**, so it is a screen, not the A/B run `0038` requires before a role is routed: a model that fails it is not routed, and passing it earns nothing. It needs `cli/` beside `packs/`, as the repository and the npm package carry it, exiting `2` where only `packs/` was installed. The A/B run is [*The A/B record*](#the-ab-record) below.
|
|
34
|
+
|
|
35
|
+
A receipt prints a figure the server did not report as `?`, never as `0`, and `time=` is the model's request time, summed over a transform's attempts like `in=` and `out=`: the runner does not stream, so it cannot see a first token.
|
|
36
|
+
|
|
37
|
+
## Screening a slot for duplicates
|
|
38
|
+
|
|
39
|
+
`dedup --dir <slot-dir> [--threshold 0.80] [--model <id>] [--max-input-bytes <per file>]` embeds every file directly in the directory whose name ends in `.md`, in any letter case, sorted by name, in one `POST /v1/embeddings` request to the loopback endpoint, with `text-embedding-nomic-embed-text-v1.5` (`PORTULAN_LOCAL_EMBED_MODEL`) unless `--model` names another; `--role` picks a chat model and is refused here. It prints each pair whose cosine similarity is at or above the threshold, highest first and ties in file-name order, one line each, `0.829 0036-….md 0038-….md`, and two lines on stderr: `dedup files=8 pairs=1 threshold=0.8`, followed by the shared receipt `OK mode=dedup model=… attempts=1 in=…t out=? time=…ms rate=? spend=$0.00`. The ledger reads the receipt as embedding work at $0.00; output tokens and rate are unreported, never zero.
|
|
40
|
+
|
|
41
|
+
- **A slot's `README.md` is left out**, in any letter case: it describes the slot rather than being one of its entries, and would rank against every entry it summarizes. Subdirectories, symbolic links and other files are left out too.
|
|
42
|
+
- **The ceiling is per file**, since the model embeds each input on its own, and it measures the file's text: its size on disk before it is read, and its UTF-8 bytes after. A file over it refuses the run with exit `1`, naming every such file with its size, and nothing is truncated. The request as a whole is not capped: a slot of `n` files sends at most `n` times the ceiling of file text, which JSON escaping can enlarge in the request body, to about six times for control characters. Raise `--max-input-bytes` only as far as the embedding model's context holds.
|
|
43
|
+
- **The threshold is a plain decimal from `-1` to `1`**, the range of a cosine; `-1` lists every pair. Anything else is exit `2`, as every malformed argument to the runner is, and never read as the default.
|
|
44
|
+
- **Exit `0` is a run, with pairs or with none**, since a candidate pair is for a human to judge, not a red. Exit `1` is a refusal before any request: an endpoint off loopback, or a file over the ceiling. Exit `2` is no server, a redirect, an HTTP error, a response that is not one finite vector of one length per file, or a directory with fewer than two entries to compare. A zero vector scores `0` against everything rather than `NaN`.
|
|
45
|
+
|
|
46
|
+
A score is a candidate, not a verdict: the runner neither merges nor edits a file.
|
|
47
|
+
|
|
48
|
+
## Delegating to it
|
|
49
|
+
|
|
50
|
+
The pass is the [`delegate-local`](skills/delegate-local/SKILL.md) skill: check the server with
|
|
51
|
+
`status`, scope the input with [`cli/symbols.mjs`](../../../cli/symbols.mjs) and `--lines`, choose the
|
|
52
|
+
supported role by task shape, gate every edit with `--verify`, and read only the receipt back.
|
|
53
|
+
The acquired [record](../../../evals/ab/local-worker.md) supports `code` at low effort; shared `prose`
|
|
54
|
+
is disabled because codify failed all five local turns. `summarize` defaults to that disabled role and
|
|
55
|
+
refuses. An explicit `--model` without `--role` is an operator-selected candidate, not automatic routing.
|
|
56
|
+
|
|
57
|
+
## The A/B record
|
|
58
|
+
|
|
59
|
+
The acquired [capture](../../../evals/ab/local-worker.json) and its generated
|
|
60
|
+
[register](../../../evals/ab/local-worker.md) record five turns per task and arm on 2026-11-03:
|
|
61
|
+
code L5/H5, codify L0/H3, handoff L5/H5. The [acquisition note](../../../evals/ab/local-worker-acquisition.md)
|
|
62
|
+
keeps the failed authentication acquisition and the authorized host-only replacement auditable.
|
|
63
|
+
|
|
64
|
+
The rule fixed before outcomes requires all five local turns to pass and all five host turns to
|
|
65
|
+
complete, with local passes at least host passes. A positive 5/5 tie qualifies; a 0/5 tie does not.
|
|
66
|
+
Both tasks must qualify for the shared prose role. Thus **code is enabled and prose is disabled**.
|
|
67
|
+
Operational defaults and explicit `--role` calls require the measured model at low effort; a model
|
|
68
|
+
override or environment variable cannot silently substitute another. Explicit `--model` without
|
|
69
|
+
`--role`, `eval` and `eval --ab` remain candidate paths and never enable routing by themselves.
|
|
70
|
+
|
|
71
|
+
`EARNED_ROUTING` embeds the capture's SHA-256, measured model IDs and support bits, so packs-only
|
|
72
|
+
installs need no root `evals/` directory at runtime. The repository's
|
|
73
|
+
[`cli/local-routing.mjs`](../../../cli/local-routing.mjs) regrades the record, reproduces its register,
|
|
74
|
+
and independently derives those bits and models before comparing them with the shipped table.
|
|
75
|
+
It is repository workshop tooling, excluded from npm like the earlier A/B instruments.
|
|
76
|
+
|
|
77
|
+
- **`eval --ab --k 5 --seed <s> --host-cmd "<command>" --host-model <id> --write` captures it.** Arm L
|
|
78
|
+
is each task's model as routed (`code` to `--role code`; `codify` and `handoff` to `--role prose`)
|
|
79
|
+
at `--effort low`, on loopback, with a sampling seed per turn derived from `--seed`. Arm H is the host
|
|
80
|
+
agent, given the same prompt as one text, as the last argument of the command `--host-cmd` names, in
|
|
81
|
+
an empty directory it removes afterwards. Both arms are graded by `eval`'s predicates, `k = 5` per
|
|
82
|
+
(task, arm) cell, the k ruled for `evals/ab/baseline.md`. The record names both models, the host's
|
|
83
|
+
version, the commit, the machine and the seed, and the sha256 fingerprints of the loaded runner source, the predicates and the
|
|
84
|
+
prompts it was graded under, as the capture recorded them, since nothing can later verify the first; the register carries `baseline.md`'s limit: `k = 5` per cell supports a
|
|
85
|
+
recorded rate and nothing else. The machine is the CPU model, the memory, the platform and
|
|
86
|
+
architecture, and the Node version, and never the hostname.
|
|
87
|
+
- **Arm H consumes host-account usage and its prompts leave the machine**, fifteen host calls, so it crosses the loopback
|
|
88
|
+
rail the rest of the runner holds and that `local-llm-off-loopback` records. Usage may be charged
|
|
89
|
+
through an API account or covered by a subscription; this acquisition used Claude Max sign-in with
|
|
90
|
+
Sonnet 5.5 and a configured total ceiling of $2.25, not an independently measured invoice amount. It runs only where
|
|
91
|
+
`--host-cmd` names the host command; without it `--ab` refuses with exit `1`, the answer the runner
|
|
92
|
+
gives any refused endpoint, before anything is contacted. **It runs under the operator's full
|
|
93
|
+
environment and credentials**, which is how it reaches the host's account and also how anything the
|
|
94
|
+
operator's configuration loads reaches arm H. A command carrying `--dangerously-skip-permissions` or
|
|
95
|
+
`--permission-mode bypassPermissions` is refused with exit `2`, by the check `cli/ab-run.mjs` uses.
|
|
96
|
+
Arm L runs first, at `$0.00`, so a local failure stops the capture before any host call.
|
|
97
|
+
`--host-model` is recorded as declared: the runner does not observe the host's model.
|
|
98
|
+
- **The command is split on whitespace, with no shell**, so no argument of it can hold a space, and a
|
|
99
|
+
quoted one arrives in pieces. Its version is read by running the whole command with `--version`
|
|
100
|
+
appended, so a wrapper such as `env` reports the host it runs rather than itself; a host that read
|
|
101
|
+
`--version` as prompt text would answer it as a turn, and the record would show that answer as its
|
|
102
|
+
version.
|
|
103
|
+
- **Every turn is journalled before the next starts**, under the repository's git directory
|
|
104
|
+
(`git rev-parse --git-path portulan/local-worker-journal`), where no commit and no `git status` sees
|
|
105
|
+
it. SIGINT, SIGTERM or SIGHUP stops the running host turn, removes its directory, keeps the journal
|
|
106
|
+
and exits `2`; the same command run again resumes, re-running only turns the journal lacks, and
|
|
107
|
+
refuses with exit `2` a journal taken under other conditions (commit, tree, machine, seed, models,
|
|
108
|
+
endpoint, host command, version and model, or any fingerprint), since its arm-H turns were paid
|
|
109
|
+
for and removing it is the operator's act. A host that writes more than 16 MiB stops the capture with
|
|
110
|
+
exit `2` and the turn is re-run on resume. **The limit:** SIGKILL, or a power loss, leaves the running
|
|
111
|
+
host turn's directory behind and, since the host runs in a process group of its own, the host still
|
|
112
|
+
running and billing a turn the journal will not hold; the journal still resumes, and that turn runs again. The journal is removed once the
|
|
113
|
+
record is written, and a capture is refused where `evals/ab/local-worker.json` exists: a record is
|
|
114
|
+
captured once, and removing it is the operator's deliberate act. The final record is created
|
|
115
|
+
exclusively too: if another record appears during capture, it and its register are preserved,
|
|
116
|
+
exit `2` leaves the current journal for recovery. This prevents overwrites; it is not a lock on
|
|
117
|
+
concurrent journal writers or a crash-atomic publication. Different loaded runner bytes refuse
|
|
118
|
+
journal resume even at the same commit and with both trees dirty.
|
|
119
|
+
- **`eval --verify` checks it without running a model**: the record's shape, its matrix total over `k`,
|
|
120
|
+
every arm-L seed against `--seed`, every turn's model against the routing, **every completed turn
|
|
121
|
+
regraded** by the predicates as they ship, the prompts' fingerprint, the cells re-folded from the
|
|
122
|
+
turns, and the register byte-compared with a fresh render. A verdict that regrades differently is
|
|
123
|
+
exit `1`, and the finding says whether the predicates moved since capture or the record no longer
|
|
124
|
+
matches its own output: either way the figures would not hold under the screen that ships. A
|
|
125
|
+
predicate file that changed while every verdict regrades the same is no finding. A missing record is
|
|
126
|
+
exit `2`, and so is a payload with no `cli/` beside `packs/`, since nothing there can regrade. A
|
|
127
|
+
record carrying the synthetic fixture's `fixture` field is exit `1` from `--verify` and `--write`
|
|
128
|
+
alike: a synthetic fixture is not a capture.
|
|
129
|
+
`eval --write` re-renders the register from the record, refusing one `--verify` would red.
|
|
130
|
+
|
|
131
|
+
The `ab-run` recipe runs the earlier baseline verifier, exact no-model `eval --verify`, and the
|
|
132
|
+
shipped-routing verifier on every pull request. A red or could-not-run result survives later greens;
|
|
133
|
+
exit `2` takes precedence over exit `1`. No model is rerun in CI. `k = 5` supports these recorded rates,
|
|
134
|
+
not general reliability, arbitrary task quality or independently observed host model identity.
|
|
135
|
+
|
|
136
|
+
Not for multi-file architectural planning, ambiguous requirements (`clarify`), or pre-merge verdicts (`reviewer`).
|
|
@@ -0,0 +1,32 @@
|
|
|
1
|
+
{
|
|
2
|
+
"portulan": {
|
|
3
|
+
"pack": "1.0",
|
|
4
|
+
"version": "0.2.0"
|
|
5
|
+
},
|
|
6
|
+
"name": "local-llm",
|
|
7
|
+
"category": "tools",
|
|
8
|
+
"summary": "Zero-dependency local LLM runner (LM Studio / Ollama) with a verify-gated red-to-green file transform loop, a zero-floor input budget rail, role evals, an embedding screen for duplicate files in a curated slot, and loopback-only enforcement.",
|
|
9
|
+
"doc": "README.md",
|
|
10
|
+
"contributes": {
|
|
11
|
+
"skills": ["skills/"],
|
|
12
|
+
"personas": ["personas/local-worker.md"],
|
|
13
|
+
"verify": [
|
|
14
|
+
{
|
|
15
|
+
"id": "runner-rails",
|
|
16
|
+
"run": "node ${PACK_ROOT}/runner/local-llm.mjs --verify-offline",
|
|
17
|
+
"requires": ["node"],
|
|
18
|
+
"doc": "verify/README.md"
|
|
19
|
+
}
|
|
20
|
+
],
|
|
21
|
+
"gates": [
|
|
22
|
+
{
|
|
23
|
+
"id": "local-llm-off-loopback",
|
|
24
|
+
"tier": "propose",
|
|
25
|
+
"action": {
|
|
26
|
+
"none": "No honest shell matcher. Whether a local-model invocation stays on the machine depends on the resolved endpoint (`--base-url`, `PORTULAN_LOCAL_LLM_URL`, or a proxy config), not on the binary name. `runner/local-llm.mjs` refuses non-loopback hosts in code unless `--allow-remote` is passed; this fragment records the policy boundary that a host permission rule cannot see."
|
|
27
|
+
},
|
|
28
|
+
"reason": "Routing a task to a local model is a spend and confidentiality boundary only while the endpoint resolves to loopback (`127.0.0.1`, `localhost`, `[::1]`). Pointing a local worker at a remote endpoint sends workspace context off-machine under a local tool's name; propose the remote endpoint explicitly before running."
|
|
29
|
+
}
|
|
30
|
+
]
|
|
31
|
+
}
|
|
32
|
+
}
|
|
@@ -0,0 +1,44 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: local-worker
|
|
3
|
+
description: Executes bounded file transformations, test-fixture drafts, handoff compressions, and research digests on a local OpenAI-compatible model (LM Studio / Ollama) with an omitted host floor and a verify-gated rollback loop. Delegate surgical, single-file or read-parallel summarization work to it; never ask it to own multi-file architecture or merge decisions.
|
|
4
|
+
tools: [read, search, edit-code, run-verify]
|
|
5
|
+
effort: low
|
|
6
|
+
omitClaudeMd: true
|
|
7
|
+
---
|
|
8
|
+
|
|
9
|
+
# Persona — Local Worker
|
|
10
|
+
|
|
11
|
+
> A context-firewall worker whose whole value is **what it does not load and what it does not spend**.
|
|
12
|
+
> A frontier session re-reads its entire prefix (`F + g·T`) on every tool call; a cloud subagent inherits
|
|
13
|
+
> the host floor (`F ≈ 44k–85k` tokens). The local worker runs against a loopback model (`127.0.0.1`) with
|
|
14
|
+
> the host instruction floor omitted (`omitClaudeMd: true`), a hard input-byte ceiling, and a verify
|
|
15
|
+
> recipe that gates its output before anything returns to the parent loop.
|
|
16
|
+
|
|
17
|
+
## Charter
|
|
18
|
+
|
|
19
|
+
- **For:** executing one bounded, verify-gated code transformation (`transform --file <path> --verify <cmd>`),
|
|
20
|
+
drafting unit tests or fixtures against an explicit signature, compressing a session log into a
|
|
21
|
+
bounded handoff, or distilling a file span into a compact digest for the parent session.
|
|
22
|
+
- **Not for:** cross-cutting architectural design, multi-branch coordination, or final pre-merge verdicts
|
|
23
|
+
(`reviewer` / `supervisor`). It does not accept rules (`librarian`) and it does not take outward actions.
|
|
24
|
+
|
|
25
|
+
## Autonomy reach
|
|
26
|
+
|
|
27
|
+
Acts in **Auto** to read scoped inputs, run local loopback inference, apply edits in place, kept only on a
|
|
28
|
+
green verify and rolled back otherwise, and run the workspace's verify recipe. Its output into any curated
|
|
29
|
+
slot (memory, proposals, handoffs) is a draft into a **Propose** or **Gated** decision owned by the
|
|
30
|
+
parent session or human. **Prohibited is not a reach** and does not appear here as a permission: it is
|
|
31
|
+
the tier no role may act in.
|
|
32
|
+
|
|
33
|
+
## Read / write posture
|
|
34
|
+
|
|
35
|
+
Reads only the scoped files or symbol spans passed to it — never the full repository tree or the
|
|
36
|
+
always-loaded host floor. Writes are isolated to a single target file under a rollback guard: unless the
|
|
37
|
+
verify command exits `0` within the retry cap, the original bytes are restored before the worker exits,
|
|
38
|
+
short of the signals and power loss the pack's README names.
|
|
39
|
+
|
|
40
|
+
## Memory scope
|
|
41
|
+
|
|
42
|
+
Its own `local-worker` scope in the adopting workspace's memory layer: which task shapes, prompt budgets,
|
|
43
|
+
and local model identifiers pass the workspace's verify recipes on the first try versus which require
|
|
44
|
+
frontier escalation. Kept separate from `implementer` and `reviewer` memory.
|