@sleepy_panda_srl/portulan 0.2.0 → 0.4.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +15 -4
- package/cli/README.md +61 -6
- package/cli/advisory.mjs +167 -53
- package/cli/cache-risk.mjs +69 -0
- package/cli/comments.mjs +8 -5
- package/cli/compile.mjs +781 -130
- package/cli/context.mjs +204 -43
- package/cli/doctor.mjs +248 -31
- package/cli/drills.mjs +56 -9
- package/cli/finish.md +34 -0
- package/cli/finish.mjs +8 -67
- package/cli/form.mjs +34 -18
- package/cli/frontmatter.mjs +39 -0
- package/cli/gate.mjs +14 -12
- package/cli/guidance-files.mjs +335 -0
- package/cli/handoff-policy.mjs +76 -0
- package/cli/html-entities.mjs +2130 -0
- package/cli/import-files.mjs +15 -0
- package/cli/index.mjs +77 -24
- package/cli/init-preview.mjs +553 -0
- package/cli/init.mjs +261 -46
- package/cli/instructions-journal.mjs +1185 -0
- package/cli/instructions.mjs +332 -57
- package/cli/ledger.mjs +230 -40
- package/cli/librarian.mjs +103 -19
- package/cli/persona-routing.mjs +209 -0
- package/cli/plugin-lint.mjs +77 -56
- package/cli/portulan.mjs +1 -1
- package/cli/recipe-output.mjs +91 -0
- package/cli/recipe-runner.md +68 -0
- package/cli/recipe-runner.mjs +131 -0
- package/cli/recipe-set.mjs +12 -3
- package/cli/review-meter.mjs +14 -9
- package/cli/ritual-contexts.mjs +103 -0
- package/cli/sessions.mjs +2 -1
- package/cli/spend-analysis.mjs +111 -0
- package/cli/spend-offer.md +53 -0
- package/cli/spend-offer.mjs +93 -0
- package/cli/stop-diagnostics.mjs +114 -0
- package/cli/stop-gate.mjs +133 -90
- package/cli/stop-policy.mjs +14 -0
- package/cli/symbols.mjs +55 -8
- package/cli/telemetry.mjs +66 -17
- package/cli/upgrade.mjs +62 -44
- package/cli/vendor.mjs +207 -125
- package/core/operating/context.md +401 -18
- package/core/operating/evolution.md +3 -2
- package/core/operating/loop.md +9 -4
- package/core/operating/sessions.md +6 -15
- package/core/personas/README.md +75 -0
- package/core/personas/implementer.md +3 -0
- package/core/personas/librarian.md +3 -0
- package/core/personas/reviewer.md +3 -0
- package/core/skills/clarify/SKILL.md +1 -1
- package/core/skills/consolidate/SKILL.md +1 -1
- package/core/templates/handoff.md +6 -2
- package/evals/README.md +23 -7
- package/evals/ab/personas/README.md +111 -0
- package/evals/ab/personas/acquisition.json +330 -0
- package/evals/ab/personas/bindings/implementer.txt +43 -0
- package/evals/ab/personas/bindings/librarian.txt +40 -0
- package/evals/ab/personas/bindings/reviewer.txt +41 -0
- package/evals/ab/personas/corpus.json +69 -0
- package/evals/ab/personas/grader.mjs +171 -0
- package/evals/ab/personas/implementer.json +774 -0
- package/evals/ab/personas/implementer.md +22 -0
- package/evals/ab/personas/librarian.json +758 -0
- package/evals/ab/personas/librarian.md +22 -0
- package/evals/ab/personas/protocol/continuation.mjs.txt +284 -0
- package/evals/ab/personas/protocol/first-outcome.mjs.txt +227 -0
- package/evals/ab/personas/protocol/first-probe.mjs.txt +172 -0
- package/evals/ab/personas/protocol/publisher.mjs.txt +126 -0
- package/evals/ab/personas/protocol/second-probe.mjs.txt +193 -0
- package/evals/ab/personas/repairs/first-control-answer.txt +23 -0
- package/evals/ab/personas/repairs/grader-before-decode-repair.mjs.txt +143 -0
- package/evals/ab/personas/reviewer.json +758 -0
- package/evals/ab/personas/reviewer.md +22 -0
- package/evals/releases/0.3.0.json +150 -0
- package/evals/releases/0.3.0.md +68 -0
- package/evals/releases/0.4.0.json +154 -0
- package/evals/releases/0.4.0.md +69 -0
- package/package.json +7 -2
- package/packs/.claude-plugin/plugin.json +2 -1
- package/packs/README.md +7 -4
- package/packs/rituals/checkpoints/README.md +35 -0
- package/packs/rituals/checkpoints/pack.json +3 -2
- package/packs/tools/README.md +8 -8
- package/packs/tools/local-llm/README.md +176 -0
- package/packs/tools/local-llm/pack.json +32 -0
- package/packs/tools/local-llm/personas/local-worker.md +44 -0
- package/packs/tools/local-llm/runner/local-llm.mjs +1779 -0
- package/packs/tools/local-llm/skills/delegate-local/SKILL.md +88 -0
- package/packs/tools/local-llm/verify/README.md +18 -0
- package/spec/README.md +37 -3
- package/spec/migrations/0009-instruction-sections-on-read.mjs +60 -11
- package/spec/migrations/0011-persona-bindings-compiled.mjs +23 -0
- package/spec/migrations/README.md +6 -0
- package/spec/pack.schema.json +12 -3
- package/spec/slots.md +272 -123
- package/spec/workspace.schema.json +67 -11
|
@@ -0,0 +1,22 @@
|
|
|
1
|
+
# reviewer — acquired routing comparison
|
|
2
|
+
|
|
3
|
+
Host: 2.1.321 (Claude Code). Concrete model: claude-sonnet-5-5.
|
|
4
|
+
|
|
5
|
+
A uses the shipped inherited model and undeclared child effort (observed high). B requests medium through the native child tool. Each case has one attempt per arm.
|
|
6
|
+
|
|
7
|
+
| Arm | Semantic passes | Observed cost index | Cold-normalized cost index |
|
|
8
|
+
|---|---:|---:|---:|
|
|
9
|
+
| A | 3/3 | 150520.60 | 426477.60 |
|
|
10
|
+
| B | 3/3 | 161948.05 | 426186.55 |
|
|
11
|
+
|
|
12
|
+
Medium wins this observed corpus: **yes**. The shipped policy remains inherited model, undeclared effort, instruction floor retained and five-minute cache lifetime.
|
|
13
|
+
|
|
14
|
+
These are three observations, not a reliability estimate. A result on this concrete model does not establish medium effort for other inherited models. Cost indexes use the frozen declared vector, include parent and child requests, and are not subscription bills. First-request input is normalized separately in each fresh parent and child context.
|
|
15
|
+
|
|
16
|
+
| Case | A | B |
|
|
17
|
+
|---|---|---|
|
|
18
|
+
| reviewer-verdict | PASS | PASS |
|
|
19
|
+
| reviewer-clean | PASS | PASS |
|
|
20
|
+
| reviewer-boundaries | PASS | PASS |
|
|
21
|
+
|
|
22
|
+
The common acquisition record accounts for both mechanism probes and discloses the one retained-answer decoder repair after the first outcome. The original failed decode score and the repaired score are both retained; that child was never rerun.
|
|
@@ -0,0 +1,150 @@
|
|
|
1
|
+
{
|
|
2
|
+
"portulan": {
|
|
3
|
+
"releaseEval": "1"
|
|
4
|
+
},
|
|
5
|
+
"version": "0.3.0",
|
|
6
|
+
"captured": "2026-11-03",
|
|
7
|
+
"source": {
|
|
8
|
+
"commit": "c08515fee0693cb0fc07fc6077fb152aa46bc28b",
|
|
9
|
+
"clean": false
|
|
10
|
+
},
|
|
11
|
+
"host": {
|
|
12
|
+
"node": "v26.12.1",
|
|
13
|
+
"platform": "darwin"
|
|
14
|
+
},
|
|
15
|
+
"recipes": [
|
|
16
|
+
{
|
|
17
|
+
"id": "docs",
|
|
18
|
+
"exit": 0
|
|
19
|
+
},
|
|
20
|
+
{
|
|
21
|
+
"id": "json",
|
|
22
|
+
"exit": 0
|
|
23
|
+
},
|
|
24
|
+
{
|
|
25
|
+
"id": "doctor",
|
|
26
|
+
"exit": 0
|
|
27
|
+
},
|
|
28
|
+
{
|
|
29
|
+
"id": "tests",
|
|
30
|
+
"exit": 0
|
|
31
|
+
},
|
|
32
|
+
{
|
|
33
|
+
"id": "plugin",
|
|
34
|
+
"exit": 0
|
|
35
|
+
},
|
|
36
|
+
{
|
|
37
|
+
"id": "compile",
|
|
38
|
+
"exit": 0
|
|
39
|
+
},
|
|
40
|
+
{
|
|
41
|
+
"id": "workflow-filters",
|
|
42
|
+
"exit": 0
|
|
43
|
+
},
|
|
44
|
+
{
|
|
45
|
+
"id": "index",
|
|
46
|
+
"exit": 0
|
|
47
|
+
},
|
|
48
|
+
{
|
|
49
|
+
"id": "context",
|
|
50
|
+
"exit": 0
|
|
51
|
+
},
|
|
52
|
+
{
|
|
53
|
+
"id": "ledger",
|
|
54
|
+
"exit": 0
|
|
55
|
+
},
|
|
56
|
+
{
|
|
57
|
+
"id": "control-chars",
|
|
58
|
+
"exit": 0
|
|
59
|
+
},
|
|
60
|
+
{
|
|
61
|
+
"id": "comments",
|
|
62
|
+
"exit": 0
|
|
63
|
+
},
|
|
64
|
+
{
|
|
65
|
+
"id": "rule-carriers",
|
|
66
|
+
"exit": 0
|
|
67
|
+
},
|
|
68
|
+
{
|
|
69
|
+
"id": "pack-version",
|
|
70
|
+
"exit": 0
|
|
71
|
+
},
|
|
72
|
+
{
|
|
73
|
+
"id": "pack-identity",
|
|
74
|
+
"exit": 0
|
|
75
|
+
},
|
|
76
|
+
{
|
|
77
|
+
"id": "payload",
|
|
78
|
+
"exit": 0
|
|
79
|
+
},
|
|
80
|
+
{
|
|
81
|
+
"id": "eval-bundle",
|
|
82
|
+
"exit": 0
|
|
83
|
+
},
|
|
84
|
+
{
|
|
85
|
+
"id": "goldens",
|
|
86
|
+
"exit": 0
|
|
87
|
+
},
|
|
88
|
+
{
|
|
89
|
+
"id": "mutants",
|
|
90
|
+
"exit": 0
|
|
91
|
+
},
|
|
92
|
+
{
|
|
93
|
+
"id": "fuzz-shell",
|
|
94
|
+
"exit": 0
|
|
95
|
+
},
|
|
96
|
+
{
|
|
97
|
+
"id": "version-carriers",
|
|
98
|
+
"exit": 0
|
|
99
|
+
},
|
|
100
|
+
{
|
|
101
|
+
"id": "skill-goldens",
|
|
102
|
+
"exit": 0
|
|
103
|
+
},
|
|
104
|
+
{
|
|
105
|
+
"id": "review-loop",
|
|
106
|
+
"exit": 0
|
|
107
|
+
},
|
|
108
|
+
{
|
|
109
|
+
"id": "telemetry",
|
|
110
|
+
"exit": 0
|
|
111
|
+
},
|
|
112
|
+
{
|
|
113
|
+
"id": "drills",
|
|
114
|
+
"exit": 0
|
|
115
|
+
},
|
|
116
|
+
{
|
|
117
|
+
"id": "ab",
|
|
118
|
+
"exit": 0
|
|
119
|
+
},
|
|
120
|
+
{
|
|
121
|
+
"id": "ab-grade",
|
|
122
|
+
"exit": 0
|
|
123
|
+
},
|
|
124
|
+
{
|
|
125
|
+
"id": "ab-run",
|
|
126
|
+
"exit": 0
|
|
127
|
+
},
|
|
128
|
+
{
|
|
129
|
+
"id": "tools/github:actions-pinned",
|
|
130
|
+
"exit": 0
|
|
131
|
+
},
|
|
132
|
+
{
|
|
133
|
+
"id": "tools/local-llm:runner-rails",
|
|
134
|
+
"exit": 0
|
|
135
|
+
}
|
|
136
|
+
],
|
|
137
|
+
"excluded": [
|
|
138
|
+
{
|
|
139
|
+
"id": "release-eval",
|
|
140
|
+
"why": "a capture cannot be accurate about the record it is inside — captured before the record is written it is red for the record's absence, and captured after it is stale the moment the register is rendered. Its verdict for this release is the rail's own run on the pull request carrying this file."
|
|
141
|
+
}
|
|
142
|
+
],
|
|
143
|
+
"abBaseline": {
|
|
144
|
+
"snapshot": "evals/ab/baseline.json",
|
|
145
|
+
"register": "evals/ab/baseline.md",
|
|
146
|
+
"captured": "2026-08-31T09:05:32.699Z",
|
|
147
|
+
"commit": "a642d551eec6c146c67ef0acfe3f8ba535c5ce6e",
|
|
148
|
+
"clean": false
|
|
149
|
+
}
|
|
150
|
+
}
|
|
@@ -0,0 +1,68 @@
|
|
|
1
|
+
# Eval result — Portulan 0.3.0
|
|
2
|
+
|
|
3
|
+
> **Generated. Do not edit.** Rendered from [`evals/releases/0.3.0.json`](0.3.0.json) by [`cli/release-eval.mjs`](../../cli/release-eval.mjs) and byte-compared by the `release-eval` verify recipe. This is the eval result milestone 8 requires a release to carry — `docs/plan.md`, Protocol → Versioning.
|
|
4
|
+
|
|
5
|
+
## What this release was measured at
|
|
6
|
+
|
|
7
|
+
| | |
|
|
8
|
+
|---|---|
|
|
9
|
+
| Version | `0.3.0` |
|
|
10
|
+
| Captured | 2026-11-03 |
|
|
11
|
+
| Commit | `c08515fee0693cb0fc07fc6077fb152aa46bc28b` |
|
|
12
|
+
| Tree at capture | **not clean** |
|
|
13
|
+
| Node | `v26.12.1` |
|
|
14
|
+
| Platform | `darwin` |
|
|
15
|
+
|
|
16
|
+
## The rails, and what each returned
|
|
17
|
+
|
|
18
|
+
30 of the 31 recipes this workspace yielded at capture. The set is derived from [`cli/recipe-set.mjs`](../../cli/recipe-set.mjs) and never listed by hand.
|
|
19
|
+
|
|
20
|
+
| Recipe | Verdict |
|
|
21
|
+
|---|---|
|
|
22
|
+
| `docs` | green |
|
|
23
|
+
| `json` | green |
|
|
24
|
+
| `doctor` | green |
|
|
25
|
+
| `tests` | green |
|
|
26
|
+
| `plugin` | green |
|
|
27
|
+
| `compile` | green |
|
|
28
|
+
| `workflow-filters` | green |
|
|
29
|
+
| `index` | green |
|
|
30
|
+
| `context` | green |
|
|
31
|
+
| `ledger` | green |
|
|
32
|
+
| `control-chars` | green |
|
|
33
|
+
| `comments` | green |
|
|
34
|
+
| `rule-carriers` | green |
|
|
35
|
+
| `pack-version` | green |
|
|
36
|
+
| `pack-identity` | green |
|
|
37
|
+
| `payload` | green |
|
|
38
|
+
| `eval-bundle` | green |
|
|
39
|
+
| `goldens` | green |
|
|
40
|
+
| `mutants` | green |
|
|
41
|
+
| `fuzz-shell` | green |
|
|
42
|
+
| `version-carriers` | green |
|
|
43
|
+
| `skill-goldens` | green |
|
|
44
|
+
| `review-loop` | green |
|
|
45
|
+
| `telemetry` | green |
|
|
46
|
+
| `drills` | green |
|
|
47
|
+
| `ab` | green |
|
|
48
|
+
| `ab-grade` | green |
|
|
49
|
+
| `ab-run` | green |
|
|
50
|
+
| `tools/github:actions-pinned` | green |
|
|
51
|
+
| `tools/local-llm:runner-rails` | green |
|
|
52
|
+
|
|
53
|
+
**`release-eval` is excluded**, and the reason is printed rather than the row silently dropped: a capture cannot be accurate about the record it is inside — captured before the record is written it is red for the record's absence, and captured after it is stale the moment the register is rendered. Its verdict for this release is the rail's own run on the pull request carrying this file.
|
|
54
|
+
|
|
55
|
+
## The A/B baseline this release ships against
|
|
56
|
+
|
|
57
|
+
[`evals/ab/baseline.md`](https://github.com/sleepy-panda-srl/portulan/blob/v0.3.0/evals/ab/baseline.md), rendered from `evals/ab/baseline.json` — captured 2026-08-31T09:05:32.699Z at `a642d551eec6c146c67ef0acfe3f8ba535c5ce6e`, over a tree that was **not clean**.
|
|
58
|
+
|
|
59
|
+
**Its figures are not repeated here.** They have one carrier and this is not it.
|
|
60
|
+
|
|
61
|
+
## What this record does not establish
|
|
62
|
+
|
|
63
|
+
- **A green says this record agrees with its own capture — never that the release is good.** Every line below is what the recipes returned at one commit, and a recipe's own green establishes only what that recipe's documentation says it establishes.
|
|
64
|
+
- **The recipes were not run at the tag.** They ran at `c08515fee0693cb0fc07fc6077fb152aa46bc28b`, and this record is committed *in* the cut change — so the tagged tree is this commit plus whatever that change still adds after the capture, the record itself at minimum. A record cannot be captured at a commit that does not exist yet, and printing the one it was captured at is the only honest form.
|
|
65
|
+
- **`release-eval` is excluded from the rows above**, because a capture cannot be accurate about the record it is inside. Its verdict for this release is the rail's own run on the pull request that carries this file.
|
|
66
|
+
- **This record measures the Portulan repository's own build**, at the commit named above — never the workspace, project or package this release is installed into. Nothing here has looked at your tree.
|
|
67
|
+
- **The tree was NOT clean at capture** — the diff of the cut change is the only record of what was uncommitted, which is weaker than a sha and is printed rather than hidden.
|
|
68
|
+
- **The A/B baseline's figures are NOT restated here.** This record names which baseline the release ships against; the figures and everything that may not be concluded from them live in that register alone.
|
|
@@ -0,0 +1,154 @@
|
|
|
1
|
+
{
|
|
2
|
+
"portulan": {
|
|
3
|
+
"releaseEval": "1"
|
|
4
|
+
},
|
|
5
|
+
"version": "0.4.0",
|
|
6
|
+
"captured": "2026-11-11",
|
|
7
|
+
"source": {
|
|
8
|
+
"commit": "660cbbfd7c9a4cf234e5a85d82d2ae21d248d3b1",
|
|
9
|
+
"clean": true
|
|
10
|
+
},
|
|
11
|
+
"host": {
|
|
12
|
+
"node": "v26.13.0",
|
|
13
|
+
"platform": "darwin"
|
|
14
|
+
},
|
|
15
|
+
"recipes": [
|
|
16
|
+
{
|
|
17
|
+
"id": "cache-stability",
|
|
18
|
+
"exit": 0
|
|
19
|
+
},
|
|
20
|
+
{
|
|
21
|
+
"id": "docs",
|
|
22
|
+
"exit": 0
|
|
23
|
+
},
|
|
24
|
+
{
|
|
25
|
+
"id": "json",
|
|
26
|
+
"exit": 0
|
|
27
|
+
},
|
|
28
|
+
{
|
|
29
|
+
"id": "doctor",
|
|
30
|
+
"exit": 0
|
|
31
|
+
},
|
|
32
|
+
{
|
|
33
|
+
"id": "tests",
|
|
34
|
+
"exit": 0
|
|
35
|
+
},
|
|
36
|
+
{
|
|
37
|
+
"id": "plugin",
|
|
38
|
+
"exit": 0
|
|
39
|
+
},
|
|
40
|
+
{
|
|
41
|
+
"id": "compile",
|
|
42
|
+
"exit": 0
|
|
43
|
+
},
|
|
44
|
+
{
|
|
45
|
+
"id": "workflow-filters",
|
|
46
|
+
"exit": 0
|
|
47
|
+
},
|
|
48
|
+
{
|
|
49
|
+
"id": "index",
|
|
50
|
+
"exit": 0
|
|
51
|
+
},
|
|
52
|
+
{
|
|
53
|
+
"id": "context",
|
|
54
|
+
"exit": 0
|
|
55
|
+
},
|
|
56
|
+
{
|
|
57
|
+
"id": "ledger",
|
|
58
|
+
"exit": 0
|
|
59
|
+
},
|
|
60
|
+
{
|
|
61
|
+
"id": "control-chars",
|
|
62
|
+
"exit": 0
|
|
63
|
+
},
|
|
64
|
+
{
|
|
65
|
+
"id": "comments",
|
|
66
|
+
"exit": 0
|
|
67
|
+
},
|
|
68
|
+
{
|
|
69
|
+
"id": "rule-carriers",
|
|
70
|
+
"exit": 0
|
|
71
|
+
},
|
|
72
|
+
{
|
|
73
|
+
"id": "pack-version",
|
|
74
|
+
"exit": 0
|
|
75
|
+
},
|
|
76
|
+
{
|
|
77
|
+
"id": "pack-identity",
|
|
78
|
+
"exit": 0
|
|
79
|
+
},
|
|
80
|
+
{
|
|
81
|
+
"id": "payload",
|
|
82
|
+
"exit": 0
|
|
83
|
+
},
|
|
84
|
+
{
|
|
85
|
+
"id": "eval-bundle",
|
|
86
|
+
"exit": 0
|
|
87
|
+
},
|
|
88
|
+
{
|
|
89
|
+
"id": "goldens",
|
|
90
|
+
"exit": 0
|
|
91
|
+
},
|
|
92
|
+
{
|
|
93
|
+
"id": "mutants",
|
|
94
|
+
"exit": 0
|
|
95
|
+
},
|
|
96
|
+
{
|
|
97
|
+
"id": "fuzz-shell",
|
|
98
|
+
"exit": 0
|
|
99
|
+
},
|
|
100
|
+
{
|
|
101
|
+
"id": "version-carriers",
|
|
102
|
+
"exit": 0
|
|
103
|
+
},
|
|
104
|
+
{
|
|
105
|
+
"id": "skill-goldens",
|
|
106
|
+
"exit": 0
|
|
107
|
+
},
|
|
108
|
+
{
|
|
109
|
+
"id": "review-loop",
|
|
110
|
+
"exit": 0
|
|
111
|
+
},
|
|
112
|
+
{
|
|
113
|
+
"id": "telemetry",
|
|
114
|
+
"exit": 0
|
|
115
|
+
},
|
|
116
|
+
{
|
|
117
|
+
"id": "drills",
|
|
118
|
+
"exit": 0
|
|
119
|
+
},
|
|
120
|
+
{
|
|
121
|
+
"id": "ab",
|
|
122
|
+
"exit": 0
|
|
123
|
+
},
|
|
124
|
+
{
|
|
125
|
+
"id": "ab-grade",
|
|
126
|
+
"exit": 0
|
|
127
|
+
},
|
|
128
|
+
{
|
|
129
|
+
"id": "ab-run",
|
|
130
|
+
"exit": 0
|
|
131
|
+
},
|
|
132
|
+
{
|
|
133
|
+
"id": "tools/github:actions-pinned",
|
|
134
|
+
"exit": 0
|
|
135
|
+
},
|
|
136
|
+
{
|
|
137
|
+
"id": "tools/local-llm:runner-rails",
|
|
138
|
+
"exit": 0
|
|
139
|
+
}
|
|
140
|
+
],
|
|
141
|
+
"excluded": [
|
|
142
|
+
{
|
|
143
|
+
"id": "release-eval",
|
|
144
|
+
"why": "a capture cannot be accurate about the record it is inside — captured before the record is written it is red for the record's absence, and captured after it is stale the moment the register is rendered. Its verdict for this release is the rail's own run on the pull request carrying this file."
|
|
145
|
+
}
|
|
146
|
+
],
|
|
147
|
+
"abBaseline": {
|
|
148
|
+
"snapshot": "evals/ab/baseline.json",
|
|
149
|
+
"register": "evals/ab/baseline.md",
|
|
150
|
+
"captured": "2026-08-31T09:05:32.699Z",
|
|
151
|
+
"commit": "a642d551eec6c146c67ef0acfe3f8ba535c5ce6e",
|
|
152
|
+
"clean": false
|
|
153
|
+
}
|
|
154
|
+
}
|
|
@@ -0,0 +1,69 @@
|
|
|
1
|
+
# Eval result — Portulan 0.4.0
|
|
2
|
+
|
|
3
|
+
> **Generated. Do not edit.** Rendered from [`evals/releases/0.4.0.json`](0.4.0.json) by [`cli/release-eval.mjs`](../../cli/release-eval.mjs) and byte-compared by the `release-eval` verify recipe. This is the eval result milestone 8 requires a release to carry — `docs/plan.md`, Protocol → Versioning.
|
|
4
|
+
|
|
5
|
+
## What this release was measured at
|
|
6
|
+
|
|
7
|
+
| | |
|
|
8
|
+
|---|---|
|
|
9
|
+
| Version | `0.4.0` |
|
|
10
|
+
| Captured | 2026-11-11 |
|
|
11
|
+
| Commit | `660cbbfd7c9a4cf234e5a85d82d2ae21d248d3b1` |
|
|
12
|
+
| Tree at capture | clean |
|
|
13
|
+
| Node | `v26.13.0` |
|
|
14
|
+
| Platform | `darwin` |
|
|
15
|
+
|
|
16
|
+
## The rails, and what each returned
|
|
17
|
+
|
|
18
|
+
31 of the 32 recipes this workspace yielded at capture. The set is derived from [`cli/recipe-set.mjs`](../../cli/recipe-set.mjs) and never listed by hand.
|
|
19
|
+
|
|
20
|
+
| Recipe | Verdict |
|
|
21
|
+
|---|---|
|
|
22
|
+
| `cache-stability` | green |
|
|
23
|
+
| `docs` | green |
|
|
24
|
+
| `json` | green |
|
|
25
|
+
| `doctor` | green |
|
|
26
|
+
| `tests` | green |
|
|
27
|
+
| `plugin` | green |
|
|
28
|
+
| `compile` | green |
|
|
29
|
+
| `workflow-filters` | green |
|
|
30
|
+
| `index` | green |
|
|
31
|
+
| `context` | green |
|
|
32
|
+
| `ledger` | green |
|
|
33
|
+
| `control-chars` | green |
|
|
34
|
+
| `comments` | green |
|
|
35
|
+
| `rule-carriers` | green |
|
|
36
|
+
| `pack-version` | green |
|
|
37
|
+
| `pack-identity` | green |
|
|
38
|
+
| `payload` | green |
|
|
39
|
+
| `eval-bundle` | green |
|
|
40
|
+
| `goldens` | green |
|
|
41
|
+
| `mutants` | green |
|
|
42
|
+
| `fuzz-shell` | green |
|
|
43
|
+
| `version-carriers` | green |
|
|
44
|
+
| `skill-goldens` | green |
|
|
45
|
+
| `review-loop` | green |
|
|
46
|
+
| `telemetry` | green |
|
|
47
|
+
| `drills` | green |
|
|
48
|
+
| `ab` | green |
|
|
49
|
+
| `ab-grade` | green |
|
|
50
|
+
| `ab-run` | green |
|
|
51
|
+
| `tools/github:actions-pinned` | green |
|
|
52
|
+
| `tools/local-llm:runner-rails` | green |
|
|
53
|
+
|
|
54
|
+
**`release-eval` is excluded**, and the reason is printed rather than the row silently dropped: a capture cannot be accurate about the record it is inside — captured before the record is written it is red for the record's absence, and captured after it is stale the moment the register is rendered. Its verdict for this release is the rail's own run on the pull request carrying this file.
|
|
55
|
+
|
|
56
|
+
## The A/B baseline this release ships against
|
|
57
|
+
|
|
58
|
+
[`evals/ab/baseline.md`](https://github.com/sleepy-panda-srl/portulan/blob/v0.4.0/evals/ab/baseline.md), rendered from `evals/ab/baseline.json` — captured 2026-08-31T09:05:32.699Z at `a642d551eec6c146c67ef0acfe3f8ba535c5ce6e`, over a tree that was **not clean**.
|
|
59
|
+
|
|
60
|
+
**Its figures are not repeated here.** They have one carrier and this is not it.
|
|
61
|
+
|
|
62
|
+
## What this record does not establish
|
|
63
|
+
|
|
64
|
+
- **A green says this record agrees with its own capture — never that the release is good.** Every line below is what the recipes returned at one commit, and a recipe's own green establishes only what that recipe's documentation says it establishes.
|
|
65
|
+
- **The recipes were not run at the tag.** They ran at `660cbbfd7c9a4cf234e5a85d82d2ae21d248d3b1`, and this record is committed *in* the cut change — so the tagged tree is this commit plus whatever that change still adds after the capture, the record itself at minimum. A record cannot be captured at a commit that does not exist yet, and printing the one it was captured at is the only honest form.
|
|
66
|
+
- **`release-eval` is excluded from the rows above**, because a capture cannot be accurate about the record it is inside. Its verdict for this release is the rail's own run on the pull request that carries this file.
|
|
67
|
+
- **This record measures the Portulan repository's own build**, at the commit named above — never the workspace, project or package this release is installed into. Nothing here has looked at your tree.
|
|
68
|
+
- **The tree was clean at capture**, so the commit named above is what was measured.
|
|
69
|
+
- **The A/B baseline's figures are NOT restated here.** This record names which baseline the release ships against; the figures and everything that may not be concluded from them live in that register alone.
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@sleepy_panda_srl/portulan",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.4.0",
|
|
4
4
|
"description": "Portulan — an agentic-engineering framework whose product is its files. The CLI wraps the file-based mechanics for hosts beyond Claude Code.",
|
|
5
5
|
"license": "Apache-2.0",
|
|
6
6
|
"type": "module",
|
|
@@ -16,6 +16,7 @@
|
|
|
16
16
|
"spec/",
|
|
17
17
|
"packs/",
|
|
18
18
|
"evals/releases/",
|
|
19
|
+
"evals/ab/personas/",
|
|
19
20
|
"README.md",
|
|
20
21
|
"LICENSE",
|
|
21
22
|
"NOTICE",
|
|
@@ -26,9 +27,13 @@
|
|
|
26
27
|
"!cli/ab.mjs",
|
|
27
28
|
"!cli/ab-run.mjs",
|
|
28
29
|
"!cli/ab-grade.mjs",
|
|
30
|
+
"!cli/local-routing.mjs",
|
|
31
|
+
"!cli/acquire-personas.mjs",
|
|
32
|
+
"!cli/normalize-personas.mjs",
|
|
29
33
|
"!cli/warm.mjs",
|
|
30
34
|
"!cli/payload.mjs",
|
|
31
|
-
"!cli/roster.mjs"
|
|
35
|
+
"!cli/roster.mjs",
|
|
36
|
+
"!cli/cache-stability.mjs"
|
|
32
37
|
],
|
|
33
38
|
"repository": {
|
|
34
39
|
"type": "git",
|
package/packs/README.md
CHANGED
|
@@ -31,7 +31,9 @@ registers nothing, silently, while a validator walking deeper counts three;
|
|
|
31
31
|
manifests therefore sit in this tree and they answer different questions: the repository's own at
|
|
32
32
|
[`../.claude-plugin/plugin.json`](../.claude-plugin/plugin.json), which declares
|
|
33
33
|
`./packs/rituals/checkpoints/skills/` because *its* root is the repository; and this one, whose root is
|
|
34
|
-
`packs/`. Neither is redundant, and the paths differ by exactly the prefix their roots differ by.
|
|
34
|
+
`packs/`. Neither is redundant, and the paths differ by exactly the prefix their roots differ by. Both
|
|
35
|
+
also declare `tools/local-llm`'s `skills/` root, the same way, since this repository's workspace composes
|
|
36
|
+
that pack too; the plugin's name and version here are the checkpoints pack's.
|
|
35
37
|
|
|
36
38
|
**The version here mirrors the pack's own** — `portulan.version` in
|
|
37
39
|
[`rituals/checkpoints/pack.json`](rituals/checkpoints/pack.json), which the Pack Definition names as how
|
|
@@ -45,7 +47,8 @@ observed through a *local* marketplace over a copy of this directory. The quoted
|
|
|
45
47
|
exists and the feed's pin moves to it. Until then the green is demonstrated on the payload and inferred
|
|
46
48
|
for the feed — a distinction this repository would rather write down than round off.
|
|
47
49
|
|
|
48
|
-
> **
|
|
49
|
-
> ritual;
|
|
50
|
-
>
|
|
50
|
+
> **Three packs exist.** [`rituals/checkpoints`](rituals/checkpoints/README.md), the supervised-build
|
|
51
|
+
> ritual; [`tools/github`](tools/github/README.md), one verify recipe; and
|
|
52
|
+
> [`tools/local-llm`](tools/local-llm/README.md), hybrid local-model execution under a zero-floor input
|
|
53
|
+
> budget and a verify-gated rollback loop. `stacks/` is still empty and says why.
|
|
51
54
|
|
|
@@ -18,8 +18,43 @@ sentence reads as a citation now and did not before.
|
|
|
18
18
|
|---|---|
|
|
19
19
|
| Skills | [`session-open`](skills/session-open/SKILL.md) · [`pre-commit`](skills/pre-commit/SKILL.md) · [`milestone-close`](skills/milestone-close/SKILL.md) |
|
|
20
20
|
| Persona | [`supervisor`](personas/supervisor.md) — grades, never implements or merges |
|
|
21
|
+
| Context admission | One ordinary fresh native Agent at a time per Claude session; see the scope below. |
|
|
21
22
|
| Gate fragments | `commit-without-the-hooks` (gated) · `self-certify-a-checkpoint` (prohibited) |
|
|
22
23
|
|
|
24
|
+
## Context admission
|
|
25
|
+
|
|
26
|
+
Pack Definition 1.1 declares `contributes.contexts.max_concurrent_subagents: 1`.
|
|
27
|
+
One supervisor at a time fits this sequential ritual; the three checkpoints can
|
|
28
|
+
still open three or more contexts over the work's lifetime. The ledger reports
|
|
29
|
+
actual contexts separately. Composed ritual counts tighten by minimum, independent
|
|
30
|
+
of pack order. An unresolved pack beside a known cap refuses compilation because
|
|
31
|
+
its contribution is unknown. No workspace `rituals` slot is introduced.
|
|
32
|
+
|
|
33
|
+
Claude compilation writes project `env.CLAUDE_CODE_MAX_CONCURRENT_SUBAGENTS`, even
|
|
34
|
+
without a gate policy. It applies across rituals within one session and excludes
|
|
35
|
+
the parent. It governs **ordinary fresh native Agent admission**, not all child
|
|
36
|
+
activity: completed-agent resumes and `/subtask` forks can exceed it; workflow
|
|
37
|
+
agents and team teammates have independent limits. Higher-precedence settings,
|
|
38
|
+
experimental flags and ultracode paths can bypass the guard. It bounds neither
|
|
39
|
+
lifetime children nor requests, time or tokens. Other adapters report the count
|
|
40
|
+
as not compiled; that is a limit of these adapters, not a claim about their hosts.
|
|
41
|
+
|
|
42
|
+
The [native trace](../../../cli/fixtures/ritual-cap/README.md) tests Claude Code
|
|
43
|
+
2.1.321 with synthetic loopback responses. The [current host documentation](https://code.claude.com/docs/en/sub-agents#concurrent-subagent-limit)
|
|
44
|
+
was checked on 2026-11-11. Neither establishes a universal hard ceiling. The
|
|
45
|
+
compiler composes caps with default restart advice in authored governing workspaces,
|
|
46
|
+
independently of a gate policy. It owns this dedicated native settings artifact
|
|
47
|
+
only while its complete payload, internal marker and digest remain intact; this
|
|
48
|
+
is accidental-edit detection, not authentication. Foreign content, additions,
|
|
49
|
+
links and old policy settings refuse before any output write. Formatting and key
|
|
50
|
+
order may normalize. Exact legacy cap-only output can migrate after validating
|
|
51
|
+
its shape and source/count consistency; it has no historical digest to authenticate
|
|
52
|
+
plausible edits. Removing a cap leaves advice settings in place. Missing manifests
|
|
53
|
+
and pointers do not adopt or remove those settings. The legacy internal cap-only
|
|
54
|
+
path removes its own file when its cap disappears. Normal policy compilation
|
|
55
|
+
retains its existing ownership. Removing both policy and cap while normal settings
|
|
56
|
+
still carry a cap refuses in every mode and preserves the file for explicit resolution.
|
|
57
|
+
|
|
23
58
|
## The three checkpoints
|
|
24
59
|
|
|
25
60
|
| Checkpoint | When | What it grades |
|
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
{
|
|
2
2
|
"portulan": {
|
|
3
|
-
"pack": "1.
|
|
4
|
-
"version": "0.3.
|
|
3
|
+
"pack": "1.1",
|
|
4
|
+
"version": "0.3.1"
|
|
5
5
|
},
|
|
6
6
|
|
|
7
7
|
"name": "checkpoints",
|
|
@@ -10,6 +10,7 @@
|
|
|
10
10
|
"doc": "README.md",
|
|
11
11
|
|
|
12
12
|
"contributes": {
|
|
13
|
+
"contexts": { "max_concurrent_subagents": 1 },
|
|
13
14
|
"skills": ["skills/"],
|
|
14
15
|
|
|
15
16
|
"personas": ["personas/supervisor.md"],
|
package/packs/tools/README.md
CHANGED
|
@@ -3,12 +3,12 @@
|
|
|
3
3
|
Tool packs — integrations for the tools and MCP servers an agent drives (issue trackers, cloud
|
|
4
4
|
CLIs, design tools, databases), with the how-to and the gate classification for each.
|
|
5
5
|
|
|
6
|
-
> **
|
|
7
|
-
> host-neutral vocabulary, dispatched on by
|
|
8
|
-
> cascade's missing middle (`core < pack < workspace`)
|
|
9
|
-
> fragments are resolved and merged into the policy,
|
|
6
|
+
> **Two tool packs exist** — [`github/`](github/README.md) and [`local-llm/`](local-llm/README.md). The
|
|
7
|
+
> machinery that reads one is a gate policy in a host-neutral vocabulary, dispatched on by
|
|
8
|
+
> [`../../cli/compile.mjs`](../../cli/compile.mjs) — and the cascade's missing middle (`core < pack < workspace`)
|
|
9
|
+
> is bridged: a pack's `contributes.gates` fragments are resolved and merged into the policy,
|
|
10
|
+
> **tighten-only**, by the same compiler.
|
|
10
11
|
>
|
|
11
|
-
>
|
|
12
|
-
>
|
|
13
|
-
>
|
|
14
|
-
> a workspace here drives a tool that needs one.
|
|
12
|
+
> [`local-llm/`](local-llm/README.md) is the first tool pack to ship a gate classification
|
|
13
|
+
> (`local-llm-off-loopback` at `propose`) alongside a persona and an offline verify recipe,
|
|
14
|
+
> fulfilling the promise this file has carried since it was written.
|