pi-braid 0.43.17 → 0.43.19
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +32 -0
- package/README.md +4 -4
- package/build-info.json +4 -4
- package/dist/extensions/core/skills-tool/auto-inject.d.ts +16 -9
- package/dist/extensions/core/skills-tool/auto-inject.js +16 -9
- package/dist/extensions/core/skills-tool/reach.d.ts +3 -5
- package/dist/extensions/core/skills-tool/reach.js +3 -5
- package/dist/extensions/core/skills-tool/skill-triggers.d.ts +1 -1
- package/dist/extensions/core/skills-tool/skill-triggers.js +1 -1
- package/dist/extensions/scheduling/timer/wake-retry.js +9 -3
- package/docs/LOCAL_DEV.md +1 -1
- package/docs/ROADMAP.md +10 -6
- package/docs/SETUP.md +1 -1
- package/package.json +6 -6
- package/skills/meta-improvement/SKILL.md +5 -74
- package/skills/skill-authoring/SKILL.md +135 -0
- package/src/extensions/core/skills-tool/auto-inject.ts +16 -9
- package/src/extensions/core/skills-tool/reach.ts +3 -5
- package/src/extensions/core/skills-tool/skill-triggers.ts +1 -1
- package/src/extensions/scheduling/timer/wake-retry.ts +9 -3
- package/src/extensions/self/health-maintenance/index.ts +11 -1
package/CHANGELOG.md
CHANGED
|
@@ -53,6 +53,38 @@ record; the tag scheme above applies unchanged.
|
|
|
53
53
|
|
|
54
54
|
---
|
|
55
55
|
|
|
56
|
+
## Core — 0.43.19 (2026-11-03: the pack learns how to write a skill)
|
|
57
|
+
|
|
58
|
+
- **A skill for writing skills, taken from the two best public teachers and finished with this repository's
|
|
59
|
+
own contract.** The doctrine (a failing baseline before any writing, the form matched to the failure type,
|
|
60
|
+
a description that states conditions rather than the workflow) comes from `obra/superpowers`;
|
|
61
|
+
the should-fire and should-not-fire evaluation set comes from `anthropics/skills`. Neither could supply the
|
|
62
|
+
format half, because both are written for a runtime that has no phrase matcher, no phrase caps and no
|
|
63
|
+
contract script. The landed file passes the shared contract against a planted control that fails by name,
|
|
64
|
+
declares eleven phrases of which none is shared, and carries the six gates in the order a change meets
|
|
65
|
+
them.
|
|
66
|
+
- **`meta-improvement` keeps the half it owns.** Its authoring Part 1 moves out with the phrases that
|
|
67
|
+
described it, so one phrase still names one skill; what remains is the experiment loop, with a pointer to
|
|
68
|
+
the new file for everything about the file itself.
|
|
69
|
+
- **One clause was reworded to keep a route.** The 23rd skill pushed the word `edit` to the corpus-common
|
|
70
|
+
cut at five of twenty-three documents, and the verify-worker prompt stopped reaching
|
|
71
|
+
`subagent-verification` because of it. The description now says "an ordinary text change", the fixture was
|
|
72
|
+
left alone rather than re-pinned to the new behaviour, and the open item is that the cut was derived at
|
|
73
|
+
twenty-two documents and has not been re-measured at twenty-three.
|
|
74
|
+
|
|
75
|
+
## Core — 0.43.18 (2026-11-03: braid follows the 1.5.0 harness, and its home link leaves the repository)
|
|
76
|
+
|
|
77
|
+
- **The harness pins move to 1.5.0 and the three surfaces it changed are handled where they live.** 1.5.0
|
|
78
|
+
validates tool schemas through `pi-schema`, and a `StringEnum` now loses its static type at its one read,
|
|
79
|
+
so the action union is named in the module that reads it; `pi-ai` no longer re-exports typebox's `Type`, so
|
|
80
|
+
the providers builder imports it from the package that owns it and declares the peer floor; and the
|
|
81
|
+
release's fatal-error vocabulary gains one token, mirrored in the two lists that already hold that class of
|
|
82
|
+
literal. Two fixtures pin the behaviour rather than the version: a JSON-Schema coercion case and a cost
|
|
83
|
+
reader that keeps `total` and `totalFromProvider` apart.
|
|
84
|
+
- **The package's home link points at the deck, not the repository.** npm metadata is read per version, so
|
|
85
|
+
this release carries the move: `homepage` is `https://kifirkin.github.io` while `repository` stays the
|
|
86
|
+
monorepo, which is why the package page showed the same destination twice.
|
|
87
|
+
|
|
56
88
|
## Core — 0.43.17 (2026-11-02: a worker session stops being able to message the owner)
|
|
57
89
|
|
|
58
90
|
- **The telegram extension is a factory-level gate now, and the launch that sets it exists.** The worker
|
package/README.md
CHANGED
|
@@ -3,7 +3,7 @@
|
|
|
3
3
|
# pi-braid
|
|
4
4
|
|
|
5
5
|
[](https://opensource.org/licenses/MIT)
|
|
6
|
-
[](https://github.com/earendil-works/pi)
|
|
7
7
|
|
|
8
8
|
<!-- braid:intro:start -->
|
|
9
9
|
|
|
@@ -23,7 +23,7 @@ pi install npm:pi-braid
|
|
|
23
23
|
|
|
24
24
|
`pi install` registers the package with pi, writes the declaration to
|
|
25
25
|
`~/.pi/agent/settings.json`, and unpacks it under pi's npm directory. The next `pi` start loads the
|
|
26
|
-
suite and the
|
|
26
|
+
suite and the 23 skills that ship with it.
|
|
27
27
|
|
|
28
28
|
The package also installs one command, `pi-braid-fleet`: a supervisor process that drives local node
|
|
29
29
|
sessions against a single SQLite journal. [docs/FLEET.md](docs/FLEET.md) is its guide.
|
|
@@ -68,8 +68,8 @@ reference: every key, its default, and worked examples. Module toggling has its
|
|
|
68
68
|
|
|
69
69
|
## Skills
|
|
70
70
|
|
|
71
|
-
|
|
72
|
-
separate `pi-braid-skills` package carries
|
|
71
|
+
23 skills ship in this package under `skills/`, declared by `pi.skills` in `package.json`. The
|
|
72
|
+
separate `pi-braid-skills` package carries nine more for project-scoped loading. Skills are read on
|
|
73
73
|
demand, not pasted into the prompt: `list_skills` ranks the catalog against the current task and
|
|
74
74
|
prices each read, `skill_stats` counts what was suggested and what was actually opened.
|
|
75
75
|
|
package/build-info.json
CHANGED
|
@@ -1,9 +1,9 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "pi-braid",
|
|
3
|
-
"version": "0.43.
|
|
4
|
-
"sha": "
|
|
5
|
-
"shaShort": "
|
|
3
|
+
"version": "0.43.19",
|
|
4
|
+
"sha": "50acf1bbbf5a1eb4faee48aeba2f766190c25f9c",
|
|
5
|
+
"shaShort": "50acf1bb",
|
|
6
6
|
"branch": "detached",
|
|
7
7
|
"dirty": false,
|
|
8
|
-
"packedAt": "2026-11-
|
|
8
|
+
"packedAt": "2026-11-03T05:41:48.545Z"
|
|
9
9
|
}
|
|
@@ -151,8 +151,8 @@ export declare const MAX_EXCERPT_CHARS = 300;
|
|
|
151
151
|
export declare const MIN_NON_NAME_SCORE = 2;
|
|
152
152
|
/**
|
|
153
153
|
* A keyword present in this many skill documents or more is corpus-common: it
|
|
154
|
-
* appears in a large share of the catalog
|
|
155
|
-
*
|
|
154
|
+
* appears in a large share of the catalog, so it cannot distinguish one skill
|
|
155
|
+
* from another and contributes NOTHING to the weak legs
|
|
156
156
|
* (description, catalog path) — or to the name leg's token-prefix rule, the one
|
|
157
157
|
* name hit that can rest on a shared word.
|
|
158
158
|
*
|
|
@@ -167,13 +167,20 @@ export declare const MIN_NON_NAME_SCORE = 2;
|
|
|
167
167
|
* line); below it the words still describe work types (review, check, test,
|
|
168
168
|
* edit, documentation).
|
|
169
169
|
*
|
|
170
|
-
* Labelled set. 26 recorded prompts, recall@2 by threshold
|
|
171
|
-
*
|
|
172
|
-
*
|
|
173
|
-
*
|
|
174
|
-
* only raise a top-2 recall figure.
|
|
175
|
-
*
|
|
176
|
-
*
|
|
170
|
+
* Labelled set. 26 recorded prompts, recall@2 by threshold, re-measured at the
|
|
171
|
+
* 23-skill catalog over 24 labelled pairs: df ≥ 4 → 11/24, df ≥ 5 → 13/24,
|
|
172
|
+
* df ≥ 6 → 13/24, df ≥ 7 → 14/24, no gate at all → 14/24. Recall alone cannot
|
|
173
|
+
* choose the number: every step that weakens the gate admits more skills, and
|
|
174
|
+
* admitting more can only raise a top-2 recall figure. The rows above 5 are a
|
|
175
|
+
* plateau rather than a preference: every cut from 7 to 23 returns 14/24, and
|
|
176
|
+
* no gate returns it too, so that number reads "at least 7" and prices
|
|
177
|
+
* nothing about the precision a weaker gate spends. 5 is where the gate
|
|
178
|
+
* removes a quarter of the corpus-common mass without also deleting the
|
|
179
|
+
* work-type words that are sometimes a skill's only reachable evidence.
|
|
180
|
+
*
|
|
181
|
+
* The 400-prompt mass above was measured against the 22-skill catalog and has
|
|
182
|
+
* not been re-run since the catalog reached 23, so its shares are a record of
|
|
183
|
+
* that revision rather than a current reading.
|
|
177
184
|
*/
|
|
178
185
|
export declare const COMMON_KEYWORD_DF = 5;
|
|
179
186
|
export interface InjectedSkill {
|
|
@@ -166,8 +166,8 @@ export const MAX_EXCERPT_CHARS = 300;
|
|
|
166
166
|
export const MIN_NON_NAME_SCORE = 2;
|
|
167
167
|
/**
|
|
168
168
|
* A keyword present in this many skill documents or more is corpus-common: it
|
|
169
|
-
* appears in a large share of the catalog
|
|
170
|
-
*
|
|
169
|
+
* appears in a large share of the catalog, so it cannot distinguish one skill
|
|
170
|
+
* from another and contributes NOTHING to the weak legs
|
|
171
171
|
* (description, catalog path) — or to the name leg's token-prefix rule, the one
|
|
172
172
|
* name hit that can rest on a shared word.
|
|
173
173
|
*
|
|
@@ -182,13 +182,20 @@ export const MIN_NON_NAME_SCORE = 2;
|
|
|
182
182
|
* line); below it the words still describe work types (review, check, test,
|
|
183
183
|
* edit, documentation).
|
|
184
184
|
*
|
|
185
|
-
* Labelled set. 26 recorded prompts, recall@2 by threshold
|
|
186
|
-
*
|
|
187
|
-
*
|
|
188
|
-
*
|
|
189
|
-
* only raise a top-2 recall figure.
|
|
190
|
-
*
|
|
191
|
-
*
|
|
185
|
+
* Labelled set. 26 recorded prompts, recall@2 by threshold, re-measured at the
|
|
186
|
+
* 23-skill catalog over 24 labelled pairs: df ≥ 4 → 11/24, df ≥ 5 → 13/24,
|
|
187
|
+
* df ≥ 6 → 13/24, df ≥ 7 → 14/24, no gate at all → 14/24. Recall alone cannot
|
|
188
|
+
* choose the number: every step that weakens the gate admits more skills, and
|
|
189
|
+
* admitting more can only raise a top-2 recall figure. The rows above 5 are a
|
|
190
|
+
* plateau rather than a preference: every cut from 7 to 23 returns 14/24, and
|
|
191
|
+
* no gate returns it too, so that number reads "at least 7" and prices
|
|
192
|
+
* nothing about the precision a weaker gate spends. 5 is where the gate
|
|
193
|
+
* removes a quarter of the corpus-common mass without also deleting the
|
|
194
|
+
* work-type words that are sometimes a skill's only reachable evidence.
|
|
195
|
+
*
|
|
196
|
+
* The 400-prompt mass above was measured against the 22-skill catalog and has
|
|
197
|
+
* not been re-run since the catalog reached 23, so its shares are a record of
|
|
198
|
+
* that revision rather than a current reading.
|
|
192
199
|
*/
|
|
193
200
|
export const COMMON_KEYWORD_DF = 5;
|
|
194
201
|
/**
|
|
@@ -25,11 +25,9 @@
|
|
|
25
25
|
*
|
|
26
26
|
* Headings are parsed fence-aware. Skill files carry long fenced blocks of
|
|
27
27
|
* shell, TypeScript and markdown samples whose comment lines start with `#`.
|
|
28
|
-
*
|
|
29
|
-
*
|
|
30
|
-
*
|
|
31
|
-
* commentary, and a fence-blind parse advertises them as document structure,
|
|
32
|
-
* sending a reader to a line that holds no section.
|
|
28
|
+
* Those lines are commentary, not structure: a reader sent to one lands on no
|
|
29
|
+
* section. A fence-blind parse cannot tell the two apart, so this one skips
|
|
30
|
+
* fenced blocks.
|
|
33
31
|
*
|
|
34
32
|
* Two fence edges are known limits, not open bugs. A fence opener indented
|
|
35
33
|
* four spaces is not a fence - the fence and ATX rules both stop at three
|
|
@@ -25,11 +25,9 @@
|
|
|
25
25
|
*
|
|
26
26
|
* Headings are parsed fence-aware. Skill files carry long fenced blocks of
|
|
27
27
|
* shell, TypeScript and markdown samples whose comment lines start with `#`.
|
|
28
|
-
*
|
|
29
|
-
*
|
|
30
|
-
*
|
|
31
|
-
* commentary, and a fence-blind parse advertises them as document structure,
|
|
32
|
-
* sending a reader to a line that holds no section.
|
|
28
|
+
* Those lines are commentary, not structure: a reader sent to one lands on no
|
|
29
|
+
* section. A fence-blind parse cannot tell the two apart, so this one skips
|
|
30
|
+
* fenced blocks.
|
|
33
31
|
*
|
|
34
32
|
* Two fence edges are known limits, not open bugs. A fence opener indented
|
|
35
33
|
* four spaces is not a fence - the fence and ATX rules both stop at three
|
|
@@ -9,7 +9,7 @@
|
|
|
9
9
|
* Authors wrote the phrases in three places, and all three are read:
|
|
10
10
|
* 1. frontmatter, under `trigger:` (a comma-separated string) or
|
|
11
11
|
* `triggers:` (a YAML list) — the canonical field;
|
|
12
|
-
* 2. a `## trigger:` body heading, quoted or bare (9 of core's
|
|
12
|
+
* 2. a `## trigger:` body heading, quoted or bare (9 of core's 23 skills
|
|
13
13
|
* carry the phrases here and nowhere else);
|
|
14
14
|
* 3. the description's trailing "Triggers: …" segment, the pack's older
|
|
15
15
|
* convention.
|
|
@@ -9,7 +9,7 @@
|
|
|
9
9
|
* Authors wrote the phrases in three places, and all three are read:
|
|
10
10
|
* 1. frontmatter, under `trigger:` (a comma-separated string) or
|
|
11
11
|
* `triggers:` (a YAML list) — the canonical field;
|
|
12
|
-
* 2. a `## trigger:` body heading, quoted or bare (9 of core's
|
|
12
|
+
* 2. a `## trigger:` body heading, quoted or bare (9 of core's 23 skills
|
|
13
13
|
* carry the phrases here and nowhere else);
|
|
14
14
|
* 3. the description's trailing "Triggers: …" segment, the pack's older
|
|
15
15
|
* convention.
|
|
@@ -154,9 +154,11 @@ export const WAKE_RETRY_MAX_ATTEMPTS = MAX_WAKE_ATTEMPTS_DEFAULT;
|
|
|
154
154
|
*
|
|
155
155
|
* Pinned by test/runtime/lib/test-wake-retry-nonretryable-mirror.mjs, which
|
|
156
156
|
* reads the token array out of the installed pi-ai module and requires each
|
|
157
|
-
*
|
|
158
|
-
*
|
|
159
|
-
*
|
|
157
|
+
* entry to be vetoed here - including the one pi-ai declares as a pattern
|
|
158
|
+
* rather than a literal (Zhipu/Z.ai's plan-limit code, matched as a JSON body
|
|
159
|
+
* so the bare number does not veto the turn). The retryable side below is NOT
|
|
160
|
+
* pinned: it is a hand-written subset of pi-ai's retryable classes, so a
|
|
161
|
+
* failure pi-ai retries but this mirror does not match settles with no hold. */
|
|
160
162
|
const WAKE_NON_RETRYABLE_ERROR_PATTERN = new RegExp([
|
|
161
163
|
"GoUsageLimitError",
|
|
162
164
|
"FreeUsageLimitError",
|
|
@@ -167,6 +169,10 @@ const WAKE_NON_RETRYABLE_ERROR_PATTERN = new RegExp([
|
|
|
167
169
|
"quota exceeded",
|
|
168
170
|
"billing",
|
|
169
171
|
"subscription_sharing_usage_limit_exceeded",
|
|
172
|
+
"subscription_sharing_workplace_individual_usage_limit_exceeded",
|
|
173
|
+
// Zhipu/Z.ai plan-limit code, matched as the JSON body pi-ai matches; the
|
|
174
|
+
// digit guard keeps a longer number from matching a 1308 prefix.
|
|
175
|
+
'"code"\\s*:\\s*"?1308(?![0-9])',
|
|
170
176
|
].join("|"), "i");
|
|
171
177
|
/** The retryable classes braid mirrors from pi-core's
|
|
172
178
|
* RETRYABLE_PROVIDER_ERROR_PATTERN (pi-ai/dist/utils/retry.js): transport
|
package/docs/LOCAL_DEV.md
CHANGED
|
@@ -138,7 +138,7 @@ before and after a full run. A non-zero delta is the leak this rule prevents.
|
|
|
138
138
|
### New fixtures: use `node:test`
|
|
139
139
|
|
|
140
140
|
A new fixture is written on `node:test`. The runner does not care which style a
|
|
141
|
-
file uses -
|
|
141
|
+
file uses - 103 of the core suite's 338 fixtures already use `node:test` - so the
|
|
142
142
|
reason is the failure the other style produces. A hand-rolled helper that calls
|
|
143
143
|
`fn()` without awaiting it discards the case's promise, so an `async` case runs
|
|
144
144
|
detached and the file can print its green tally before that case settles, with
|
package/docs/ROADMAP.md
CHANGED
|
@@ -58,12 +58,16 @@ provenance; they are no longer work.
|
|
|
58
58
|
(`~/.pi/agent/…`, `~/pi-braid/…`) and never a private home layout;
|
|
59
59
|
`check-doc-links` fails on the latter, and every in-repo reference must
|
|
60
60
|
resolve.
|
|
61
|
-
5. **Skills pack** - finish validating the
|
|
62
|
-
`verdict: UNVALIDATED` (`
|
|
63
|
-
`placement-discipline`, `probe-evaluation`, `
|
|
64
|
-
catalog fresh as decision rows land. The
|
|
65
|
-
|
|
66
|
-
|
|
61
|
+
5. **Skills pack** - finish validating the six skills still marked
|
|
62
|
+
`verdict: UNVALIDATED` (`cv-improvement`, `misdirection-audit`,
|
|
63
|
+
`placement-discipline`, `probe-evaluation`, `skill-authoring`,
|
|
64
|
+
`voice-replies`); keep the pack catalog fresh as decision rows land. The
|
|
65
|
+
set is whatever
|
|
66
|
+
`grep -l '^verdict: UNVALIDATED' packages/core/skills/*/SKILL.md packages/skills/skills/*/SKILL.md`
|
|
67
|
+
prints, so the next pass re-derives it rather than trusting this line. The
|
|
68
|
+
machine-local `tunnel-control` skill is `KEEP` with its v2RayTun arm
|
|
69
|
+
validated; its Happ and Incy arms stay unvalidated until a host installs
|
|
70
|
+
those clients.
|
|
67
71
|
6. **Extension surface** - ongoing de-bloat of tool/extension counts;
|
|
68
72
|
subagent-RPC hardening. Worker-stats reporting shipped (`list-workers`
|
|
69
73
|
`includeStats`); the orchestrator's metrics-emission half (writing
|
package/docs/SETUP.md
CHANGED
|
@@ -238,7 +238,7 @@ braid's skills come in two tiers.
|
|
|
238
238
|
`pi.skills` in the package manifest). Registering `packages/core` loads it; it
|
|
239
239
|
needs no settings entry of its own.
|
|
240
240
|
- **On-demand set** - the separate `pi-braid-skills` pack
|
|
241
|
-
(`packages/skills/skills/`:
|
|
241
|
+
(`packages/skills/skills/`: nine `SKILL.md` plus shared content under
|
|
242
242
|
`skills/_shared/`), registered in the settings file of the project that wants
|
|
243
243
|
it (`pi install --local npm:pi-braid-skills`, or a `file:` spec from a
|
|
244
244
|
checkout).
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "pi-braid",
|
|
3
|
-
"version": "0.43.
|
|
3
|
+
"version": "0.43.19",
|
|
4
4
|
"description": "🥨 Braid - Metacognitive extension suite for pi coding agent: Adds memory, plan tracking, structured cognitive signals and completion checks on top of pi core loop, so long sessions keep their context and stop claiming unfinished work.",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"main": "./dist/index.js",
|
|
@@ -35,7 +35,7 @@
|
|
|
35
35
|
"bugs": {
|
|
36
36
|
"url": "https://github.com/kifirkin/pi-braid/issues"
|
|
37
37
|
},
|
|
38
|
-
"homepage": "https://github.
|
|
38
|
+
"homepage": "https://kifirkin.github.io",
|
|
39
39
|
"pi": {
|
|
40
40
|
"extensions": [
|
|
41
41
|
"./src/index.ts"
|
|
@@ -81,10 +81,10 @@
|
|
|
81
81
|
"typescript-eslint": "8.59.1"
|
|
82
82
|
},
|
|
83
83
|
"peerDependencies": {
|
|
84
|
-
"@earendil-works/pi-agent-core": "^1.
|
|
85
|
-
"@earendil-works/pi-ai": "^1.
|
|
86
|
-
"@earendil-works/pi-coding-agent": "^1.
|
|
87
|
-
"@earendil-works/pi-tui": "^1.
|
|
84
|
+
"@earendil-works/pi-agent-core": "^1.5.0",
|
|
85
|
+
"@earendil-works/pi-ai": "^1.5.0",
|
|
86
|
+
"@earendil-works/pi-coding-agent": "^1.5.0",
|
|
87
|
+
"@earendil-works/pi-tui": "^1.5.0",
|
|
88
88
|
"typebox": "^1.1.38"
|
|
89
89
|
},
|
|
90
90
|
"keywords": [
|
|
@@ -1,88 +1,19 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: meta-improvement
|
|
3
|
-
description: "
|
|
3
|
+
description: "Run self-improving experiments and measure skill performance. Covers: archive-based evolution with staged evaluation, and testing improvements against baselines. Not for authoring a skill; `skill-authoring` owns the file, its frontmatter and its trigger phrases."
|
|
4
4
|
validated: 2026-08-16
|
|
5
5
|
verdict: KEEP
|
|
6
6
|
---
|
|
7
7
|
|
|
8
|
-
## trigger: "
|
|
8
|
+
## trigger: "self-improvement experiments, skill performance measurement, archive-based evolution, staged evaluation, testing improvements against baselines"
|
|
9
9
|
|
|
10
10
|
# Meta-Improvement
|
|
11
11
|
|
|
12
|
-
|
|
12
|
+
Run self-improving experiments against an archive that keeps every variant, not just the best.
|
|
13
13
|
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
## Part 1: Skill Creation
|
|
17
|
-
|
|
18
|
-
### Core Loop
|
|
19
|
-
|
|
20
|
-
1. **Understand** - What should this skill enable?
|
|
21
|
-
2. **Draft** - Write the SKILL.md
|
|
22
|
-
3. **Test** - Run test cases
|
|
23
|
-
4. **Evaluate** - Does it trigger correctly? Is guidance clear?
|
|
24
|
-
5. **Improve** - Refine based on results
|
|
25
|
-
6. **Repeat** - Until satisfied
|
|
26
|
-
|
|
27
|
-
### Anatomy of a Skill
|
|
28
|
-
|
|
29
|
-
```
|
|
30
|
-
skill-name/
|
|
31
|
-
├── SKILL.md (required)
|
|
32
|
-
│ ├── YAML frontmatter (name, description)
|
|
33
|
-
│ └── Markdown instructions
|
|
34
|
-
└── Bundled Resources (optional)
|
|
35
|
-
├── scripts/
|
|
36
|
-
├── references/
|
|
37
|
-
└── assets/
|
|
38
|
-
```
|
|
39
|
-
|
|
40
|
-
### Frontmatter
|
|
41
|
-
|
|
42
|
-
| Field | Required | Description |
|
|
43
|
-
| -------------------------- | -------- | ---------------------------------------------- |
|
|
44
|
-
| `name` | Yes | Lowercase, hyphens, max 64 chars |
|
|
45
|
-
| `description` | Yes | When to trigger + what it does, max 1024 chars |
|
|
46
|
-
| `disable-model-invocation` | No | Skip loading to agent prompt |
|
|
47
|
-
|
|
48
|
-
### Writing the Description
|
|
49
|
-
|
|
50
|
-
Be specific about triggers - include contexts and user phrases:
|
|
51
|
-
|
|
52
|
-
```
|
|
53
|
-
# WEAK
|
|
54
|
-
"Build dashboards."
|
|
55
|
-
|
|
56
|
-
# SPECIFIC
|
|
57
|
-
"Build dashboards to display metrics.
|
|
58
|
-
Use when: dashboards, data visualization, metrics,
|
|
59
|
-
internal data, charts, graphs, or displaying company data."
|
|
60
|
-
```
|
|
61
|
-
|
|
62
|
-
### Progressive Disclosure
|
|
63
|
-
|
|
64
|
-
Skills load in layers:
|
|
65
|
-
|
|
66
|
-
1. **Metadata** - name + description (~100 words)
|
|
67
|
-
2. **SKILL.md body** - in context when triggered (<500 lines)
|
|
68
|
-
3. **Bundled resources** - loaded as needed
|
|
69
|
-
|
|
70
|
-
### Testing
|
|
71
|
-
|
|
72
|
-
Create test cases, run the skill on realistic prompts, compare output to expected.
|
|
73
|
-
|
|
74
|
-
### Common Issues
|
|
75
|
-
|
|
76
|
-
| Issue | Fix |
|
|
77
|
-
| ------------------ | ------------------------------------- |
|
|
78
|
-
| Under-triggering | More specific contexts in description |
|
|
79
|
-
| Over-triggering | Narrow description, add exclusions |
|
|
80
|
-
| Unclear guidance | Add examples, explain the why |
|
|
81
|
-
| Missing edge cases | Add sections for edge cases |
|
|
82
|
-
|
|
83
|
-
---
|
|
14
|
+
Authoring a skill is a different job: `skill-authoring` covers the file, its frontmatter, its trigger phrases, and the run that proves both. This file owns the loop that compares versions of one.
|
|
84
15
|
|
|
85
|
-
##
|
|
16
|
+
## Self-Improving Experiments
|
|
86
17
|
|
|
87
18
|
Based on: [Hyperagents (arXiv:2603.19461)](https://arxiv.org/abs/2603.19461)
|
|
88
19
|
|
|
@@ -0,0 +1,135 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: skill-authoring
|
|
3
|
+
description: "Use when a skill has to be written, repaired or retired in this repository: a SKILL.md whose frontmatter a validator refuses, a skill that is never suggested or fires on every prompt, a body nobody follows, or a shipped skill with no read evidence. Produces the file, its trigger phrases and the run that proves both. Not for an ordinary text change or a single tool call fix."
|
|
4
|
+
validated: 2026-11-02
|
|
5
|
+
verdict: UNVALIDATED
|
|
6
|
+
triggers:
|
|
7
|
+
- "create a skill"
|
|
8
|
+
- "new skill"
|
|
9
|
+
- "author a skill"
|
|
10
|
+
- "write a skill.md"
|
|
11
|
+
- "write the skill.md"
|
|
12
|
+
- "skill frontmatter"
|
|
13
|
+
- "trigger phrases"
|
|
14
|
+
- "skill never fires"
|
|
15
|
+
- "skill does not fire"
|
|
16
|
+
- "skill fires on everything"
|
|
17
|
+
- "retire a skill"
|
|
18
|
+
---
|
|
19
|
+
|
|
20
|
+
# Skill Authoring
|
|
21
|
+
|
|
22
|
+
A skill is two artifacts in one file: a declaration the matcher reads, and a procedure an agent follows. Both halves are machine-checked in this repository, and a file can satisfy one while failing the other. Author against the check, not against the last skill you happened to open.
|
|
23
|
+
|
|
24
|
+
## Is a skill the right carrier?
|
|
25
|
+
|
|
26
|
+
Two questions, in order.
|
|
27
|
+
|
|
28
|
+
1. **Does the situation recur?** A skill is a prediction that this situation will come up again, in another session, with no memory of this one. One-off advice belongs in the session.
|
|
29
|
+
2. **Is the knowledge a judgment rule?** If a test, a lint or a type can enforce it, add the check where the invariant lives and say nothing here. A skill that restates an enforceable rule trains agents to treat prose as the requirement; the check is what actually fails.
|
|
30
|
+
|
|
31
|
+
A skill that answers yes to both is worth its file. A skill that answers no to either is a comment with a frontmatter block.
|
|
32
|
+
|
|
33
|
+
## Step 1: capture the failing baseline
|
|
34
|
+
|
|
35
|
+
Run one realistic task without the skill, in a fresh context, and keep what goes wrong word for word. If the run does the right thing, stop: there is nothing to teach, and the file you were about to write would be decoration. The failures are the body's outline and the phrases' vocabulary. One run per claim; a failure nobody observed is a failure you invented.
|
|
36
|
+
|
|
37
|
+
This is the cheapest gate in the process and the one most often skipped. Its cost is one run. Its value is that every later sentence can point at something an agent actually did.
|
|
38
|
+
|
|
39
|
+
## Step 2: frontmatter is the interface
|
|
40
|
+
|
|
41
|
+
Five fields carry weight, and each is read by a different piece of code:
|
|
42
|
+
|
|
43
|
+
| Field | Read by | Rule |
|
|
44
|
+
| ------------- | ------------------------------------------- | ---------------------------------------------------------------------------------------------------------- |
|
|
45
|
+
| `name` | the catalog, the matcher, the validator | equals the directory name; `a-z`, `0-9`, hyphen |
|
|
46
|
+
| `description` | the matcher's keyword leg, the catalog line | one line, at most 1024 chars, and it must name a situation (`Use when ...`); a topic list never fires |
|
|
47
|
+
| `triggers` | the trigger reader, then the scorer | at least 2, at most 16 phrases, each at most 66 chars; one phrase names one skill across the installed set |
|
|
48
|
+
| `validated` | readers deciding whether to trust it | an ISO date, the last time evidence supported the file |
|
|
49
|
+
| `verdict` | readers deciding what to do next | one of `KEEP`, `IMPROVE`, `MOVE`, `DEPRECATE`, `UNVALIDATED` |
|
|
50
|
+
|
|
51
|
+
Four traps follow.
|
|
52
|
+
|
|
53
|
+
- **The description must not summarize the workflow.** An agent that reads a workflow summary follows the summary and skips the body: a description saying "code review between tasks" produced one review where the body specified two. Conditions only, in the description; the procedure belongs in the body.
|
|
54
|
+
- **Keep the description on one line.** The validator reads the first `description:` line and strips quotes, so a `>` folded block is read as the single character `>`, and a description that starts on the next line reads as empty.
|
|
55
|
+
- **Declare phrases under `triggers:`, nowhere else.** The reader also accepts a `## trigger:` body heading and a trailing `Triggers:` sentence in the description, but the validator refuses both shapes and counts those files as declaring zero phrases. One format exists; use it.
|
|
56
|
+
- **`governs:` is optional and mostly absent.** It names the tool families a body directs, and the admitted families are the pack's own (`browser`, `document`). Advice about prose, a checklist or a file format directs none of them: leave the key out rather than inventing a family.
|
|
57
|
+
|
|
58
|
+
## Step 3: phrases a user actually types
|
|
59
|
+
|
|
60
|
+
A phrase is what the user writes when the situation occurs, not a word from the skill's subject. "Skill frontmatter" is a phrase; "skills" is a subject word that appears in every prompt and would fire the skill on unrelated work.
|
|
61
|
+
|
|
62
|
+
Matching is a case-folded, word-bounded literal test: the phrase occurs where neither neighbour is a letter or digit, so `pane` does not match `paneled`. Four properties follow.
|
|
63
|
+
|
|
64
|
+
- **A phrase is one wording, not a family of phrasings.** "create a skill" does not occur in "create a new skill", and "write a skill.md" does not occur in "write the SKILL.md frontmatter". Declare the wordings the failing baseline actually used; the articles and the verbs are part of the phrase, and a phrase list that has grown past a dozen entries is usually a list of subject words wearing situation clothes.
|
|
65
|
+
- **A phrase hit alone qualifies the skill.** The scorer awards it the weight of a name match and exempts the skill from the qualification bar, so a phrase is the strongest evidence a file can declare and the easiest thing to over-declare.
|
|
66
|
+
- **A shared phrase makes the choice a coin flip.** Two skills declaring one phrase are ranked against each other on every matching prompt, and the loser is never suggested. The validator refuses the duplicate by name, which is the rule working as intended.
|
|
67
|
+
- **A phrase over 66 characters is silently dropped** by the reader, and the validator refuses the file so the drop never ships. Count characters, do not eyeball them.
|
|
68
|
+
|
|
69
|
+
## Step 4: the body
|
|
70
|
+
|
|
71
|
+
- **Mechanism before instruction.** One sentence on why the step exists, then the step. A rule whose reason is on the page survives contact with a case the author did not anticipate.
|
|
72
|
+
- **Name the artifact, not the intention.** "Write the row into the ledger" is actionable; "record the decision" is not.
|
|
73
|
+
- **Name no tool the reader cannot call.** A skill that names an unreachable tool teaches the agent to invent the missing call, which is worse than teaching nothing. The pack refuses such files by name at validation time; check before you ship.
|
|
74
|
+
- **The first 300 characters decide whether the body is read.** The auto-injector carries at most 300 characters of the body in its suggestion and points at the file for the rest. Put the situation and the mechanism at the top; put provenance, history and worked examples below it.
|
|
75
|
+
- **Ship a script only when the skill owns the instrument.** Core skills may carry `scripts/`; the standalone pack may not carry code at all. Logic that belongs to an extension lives in the extension, and the skill names the command.
|
|
76
|
+
|
|
77
|
+
## Step 5: run the gates, in this order
|
|
78
|
+
|
|
79
|
+
From the repository root:
|
|
80
|
+
|
|
81
|
+
1. `node packages/skills/test/skills-contract.mjs`: frontmatter, trigger count and length, phrase uniqueness, description situation, and the tool/project allowlist. It is the only gate that reads the trigger declarations the matcher will use.
|
|
82
|
+
2. `node packages/skills/test/governs-declaration.mjs`: the `governs:` key, admitted and refused. Run it whenever the file declares one.
|
|
83
|
+
3. The reader's own count, not a character count by eye:
|
|
84
|
+
|
|
85
|
+
```bash
|
|
86
|
+
node -e 'import("./packages/core/src/extensions/core/skills-tool/skill-triggers.ts").then(t=>{const p=process.argv[1];const e=t.readSkillTriggerEntries(p);console.log(e.length, Math.max(...e.map(x=>x.phrase.length)), e)})' packages/core/skills/<name>/SKILL.md
|
|
87
|
+
```
|
|
88
|
+
|
|
89
|
+
4. **Tamper test.** Break one field on purpose, run the gate, read the failure by name, restore the file. A green run you have never watched fail is a claim about the gate, not about the file.
|
|
90
|
+
|
|
91
|
+
The contract is shared code, so its frontmatter half applies to any skill tree, including the core set. Its allowlist half is scoped to the pack that owns the table; a core skill legitimately names tools the pack cannot reach.
|
|
92
|
+
|
|
93
|
+
## Step 6: prove it fires, and prove it does not over-fire
|
|
94
|
+
|
|
95
|
+
Pick at least three prompts the skill must serve and at least two near-misses that share its vocabulary and must not fire; a should-not-fire prompt that is obviously unrelated tests nothing. Write the prompts down before you tune the phrases: a probe set assembled after you know what the phrases match measures nothing. Run them through the shipped matcher rather than reasoning about them:
|
|
96
|
+
|
|
97
|
+
```bash
|
|
98
|
+
node -e 'import("./packages/core/src/extensions/core/skills-tool/applicability.ts").then(a=>console.log(a.skillApplicability({name:"<name>",triggers:["<phrase>"]}, process.argv[1])))' "<prompt text>"
|
|
99
|
+
```
|
|
100
|
+
|
|
101
|
+
Then measure it in production, where the interesting number is not the suggestion count but the conversion. The adoption counters give three buckets: a skill the whisper suggests and nobody opens is `needs-refresh` and the fix is usually the description, not the body; a skill with no suggestion and no load in the window is a `prune` candidate; a skill with loads is `active`. A shipped skill with no read evidence is `UNVALIDATED`, never `KEEP`, and one session with zero loads is not a trend.
|
|
102
|
+
|
|
103
|
+
## Retire or merge
|
|
104
|
+
|
|
105
|
+
Retiring a skill takes three things, not one: `status: deprecated`, a `successor:` naming what to do instead in terms a user can act on, and a description that opens with "Deprecated:" so the file cannot be read as a live instruction. A deprecated file without a successor is a tombstone standing where the instruction used to be. The deprecation convention the instrument reads (date and reason) is meta-improvement's; the three fields above are the contract's, and the contract is what fails a release.
|
|
106
|
+
|
|
107
|
+
Merging is usually the better move. The core set's code-search is the shipped fold: `status: merged`, `successor: code-refactoring`, and a banner at the top of the body. The contract defines only the deprecated branch, so a merged file keeps its own catalog row and its own read cost; if it declares no phrase it never fires, which is the right end state for a pointer that exists so a search for the old name lands somewhere.
|
|
108
|
+
|
|
109
|
+
## Match the form to the failure
|
|
110
|
+
|
|
111
|
+
The form of the guidance is a decision, not a style preference. The same wording that fixes one failure type measurably worsens another.
|
|
112
|
+
|
|
113
|
+
| Baseline failure | Right form | Wrong form |
|
|
114
|
+
| ----------------------------------------------------------- | ------------------------------------------------------- | ------------------------------------- |
|
|
115
|
+
| Skips a known rule under pressure | prohibition, rationalization table, red-flag list | soft guidance ("prefer", "consider") |
|
|
116
|
+
| Complies but the output has the wrong shape | a recipe stating what the output is, its parts in order | a prohibition list |
|
|
117
|
+
| Omits a required element from something it already produces | a required field in the template it fills | a reminder in prose near the template |
|
|
118
|
+
| Behavior should depend on a condition | a conditional on an observable predicate | an unconditional rule plus exemptions |
|
|
119
|
+
|
|
120
|
+
Two corollaries hold across all four: a nuance clause ("don't X unless it matters") reopens the negotiation the rule was meant to close, and an exemption clause does not scope: "this limit does not apply to code blocks" still suppresses code blocks.
|
|
121
|
+
|
|
122
|
+
## Symptom to fix
|
|
123
|
+
|
|
124
|
+
| Symptom | Mechanism, and the fix |
|
|
125
|
+
| ---------------------------------- | ------------------------------------------------------------------------------------------------------- |
|
|
126
|
+
| Never suggested | fewer than two phrases, or phrases nobody types; add phrases from the failing baseline's vocabulary |
|
|
127
|
+
| Fires on unrelated prompts | a phrase names the subject rather than the situation; subject words are in every prompt |
|
|
128
|
+
| Read, then ignored | the description summarized the workflow, so the agent followed the summary; strip it to conditions |
|
|
129
|
+
| Body advice ignored under pressure | a discipline failure answered with soft guidance; switch to prohibition plus rationalizations |
|
|
130
|
+
| Same helper rewritten every run | the skill is missing its instrument; move it into `scripts/` of the owning package and name the command |
|
|
131
|
+
| Validator refuses a field | the contract's reader is line-based and literal; match its shape rather than arguing with it |
|
|
132
|
+
|
|
133
|
+
## What this skill does not cover
|
|
134
|
+
|
|
135
|
+
It does not supply the domain knowledge the skill teaches, and it does not run the experiment loop that compares two versions of a body; that measurement belongs to meta-improvement, and the adoption counters belong to skill-usage. This file only makes the carrier correct.
|
|
@@ -198,8 +198,8 @@ export const MIN_NON_NAME_SCORE = 2;
|
|
|
198
198
|
|
|
199
199
|
/**
|
|
200
200
|
* A keyword present in this many skill documents or more is corpus-common: it
|
|
201
|
-
* appears in a large share of the catalog
|
|
202
|
-
*
|
|
201
|
+
* appears in a large share of the catalog, so it cannot distinguish one skill
|
|
202
|
+
* from another and contributes NOTHING to the weak legs
|
|
203
203
|
* (description, catalog path) — or to the name leg's token-prefix rule, the one
|
|
204
204
|
* name hit that can rest on a shared word.
|
|
205
205
|
*
|
|
@@ -214,13 +214,20 @@ export const MIN_NON_NAME_SCORE = 2;
|
|
|
214
214
|
* line); below it the words still describe work types (review, check, test,
|
|
215
215
|
* edit, documentation).
|
|
216
216
|
*
|
|
217
|
-
* Labelled set. 26 recorded prompts, recall@2 by threshold
|
|
218
|
-
*
|
|
219
|
-
*
|
|
220
|
-
*
|
|
221
|
-
* only raise a top-2 recall figure.
|
|
222
|
-
*
|
|
223
|
-
*
|
|
217
|
+
* Labelled set. 26 recorded prompts, recall@2 by threshold, re-measured at the
|
|
218
|
+
* 23-skill catalog over 24 labelled pairs: df ≥ 4 → 11/24, df ≥ 5 → 13/24,
|
|
219
|
+
* df ≥ 6 → 13/24, df ≥ 7 → 14/24, no gate at all → 14/24. Recall alone cannot
|
|
220
|
+
* choose the number: every step that weakens the gate admits more skills, and
|
|
221
|
+
* admitting more can only raise a top-2 recall figure. The rows above 5 are a
|
|
222
|
+
* plateau rather than a preference: every cut from 7 to 23 returns 14/24, and
|
|
223
|
+
* no gate returns it too, so that number reads "at least 7" and prices
|
|
224
|
+
* nothing about the precision a weaker gate spends. 5 is where the gate
|
|
225
|
+
* removes a quarter of the corpus-common mass without also deleting the
|
|
226
|
+
* work-type words that are sometimes a skill's only reachable evidence.
|
|
227
|
+
*
|
|
228
|
+
* The 400-prompt mass above was measured against the 22-skill catalog and has
|
|
229
|
+
* not been re-run since the catalog reached 23, so its shares are a record of
|
|
230
|
+
* that revision rather than a current reading.
|
|
224
231
|
*/
|
|
225
232
|
export const COMMON_KEYWORD_DF = 5;
|
|
226
233
|
|
|
@@ -25,11 +25,9 @@
|
|
|
25
25
|
*
|
|
26
26
|
* Headings are parsed fence-aware. Skill files carry long fenced blocks of
|
|
27
27
|
* shell, TypeScript and markdown samples whose comment lines start with `#`.
|
|
28
|
-
*
|
|
29
|
-
*
|
|
30
|
-
*
|
|
31
|
-
* commentary, and a fence-blind parse advertises them as document structure,
|
|
32
|
-
* sending a reader to a line that holds no section.
|
|
28
|
+
* Those lines are commentary, not structure: a reader sent to one lands on no
|
|
29
|
+
* section. A fence-blind parse cannot tell the two apart, so this one skips
|
|
30
|
+
* fenced blocks.
|
|
33
31
|
*
|
|
34
32
|
* Two fence edges are known limits, not open bugs. A fence opener indented
|
|
35
33
|
* four spaces is not a fence - the fence and ATX rules both stop at three
|
|
@@ -9,7 +9,7 @@
|
|
|
9
9
|
* Authors wrote the phrases in three places, and all three are read:
|
|
10
10
|
* 1. frontmatter, under `trigger:` (a comma-separated string) or
|
|
11
11
|
* `triggers:` (a YAML list) — the canonical field;
|
|
12
|
-
* 2. a `## trigger:` body heading, quoted or bare (9 of core's
|
|
12
|
+
* 2. a `## trigger:` body heading, quoted or bare (9 of core's 23 skills
|
|
13
13
|
* carry the phrases here and nowhere else);
|
|
14
14
|
* 3. the description's trailing "Triggers: …" segment, the pack's older
|
|
15
15
|
* convention.
|
|
@@ -225,9 +225,11 @@ export const WAKE_RETRY_MAX_ATTEMPTS = MAX_WAKE_ATTEMPTS_DEFAULT;
|
|
|
225
225
|
*
|
|
226
226
|
* Pinned by test/runtime/lib/test-wake-retry-nonretryable-mirror.mjs, which
|
|
227
227
|
* reads the token array out of the installed pi-ai module and requires each
|
|
228
|
-
*
|
|
229
|
-
*
|
|
230
|
-
*
|
|
228
|
+
* entry to be vetoed here - including the one pi-ai declares as a pattern
|
|
229
|
+
* rather than a literal (Zhipu/Z.ai's plan-limit code, matched as a JSON body
|
|
230
|
+
* so the bare number does not veto the turn). The retryable side below is NOT
|
|
231
|
+
* pinned: it is a hand-written subset of pi-ai's retryable classes, so a
|
|
232
|
+
* failure pi-ai retries but this mirror does not match settles with no hold. */
|
|
231
233
|
const WAKE_NON_RETRYABLE_ERROR_PATTERN = new RegExp(
|
|
232
234
|
[
|
|
233
235
|
"GoUsageLimitError",
|
|
@@ -239,6 +241,10 @@ const WAKE_NON_RETRYABLE_ERROR_PATTERN = new RegExp(
|
|
|
239
241
|
"quota exceeded",
|
|
240
242
|
"billing",
|
|
241
243
|
"subscription_sharing_usage_limit_exceeded",
|
|
244
|
+
"subscription_sharing_workplace_individual_usage_limit_exceeded",
|
|
245
|
+
// Zhipu/Z.ai plan-limit code, matched as the JSON body pi-ai matches; the
|
|
246
|
+
// digit guard keeps a longer number from matching a 1308 prefix.
|
|
247
|
+
'"code"\\s*:\\s*"?1308(?![0-9])',
|
|
242
248
|
].join("|"),
|
|
243
249
|
"i",
|
|
244
250
|
);
|
|
@@ -42,6 +42,16 @@ const EVOLVE_ACTIONS = [
|
|
|
42
42
|
"benchmark",
|
|
43
43
|
] as const;
|
|
44
44
|
|
|
45
|
+
/**
|
|
46
|
+
* The action union the schema declares. `StringEnum` returns a pi-schema
|
|
47
|
+
* `Schema<T>`, which TypeBox's `Static` cannot read, so the framework infers
|
|
48
|
+
* `params.action` as `unknown` while every hand-written field beside it keeps
|
|
49
|
+
* its type. The alias restores the declared union at the one read that needs a
|
|
50
|
+
* string; it coerces nothing, so a value the schema would not accept still
|
|
51
|
+
* cannot reach this file.
|
|
52
|
+
*/
|
|
53
|
+
type EvolveAction = (typeof EVOLVE_ACTIONS)[number];
|
|
54
|
+
|
|
45
55
|
const healthMaintenanceSchema = Type.Object({
|
|
46
56
|
action: StringEnum([...EVOLVE_ACTIONS], { description: "Action to perform" }),
|
|
47
57
|
checkpoint: Type.Optional(Type.String({ description: "Checkpoint hash for rollback" })),
|
|
@@ -209,7 +219,7 @@ export default function healthMaintenanceExtension(pi: ExtensionAPI) {
|
|
|
209
219
|
};
|
|
210
220
|
} catch (error) {
|
|
211
221
|
const errorResult: HealthMaintenanceResult = {
|
|
212
|
-
action: params.action,
|
|
222
|
+
action: params.action as EvolveAction,
|
|
213
223
|
success: false,
|
|
214
224
|
error: error instanceof Error ? error.message : String(error),
|
|
215
225
|
};
|