cyber-aced 0.4.1 → 0.4.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude-plugin/plugin.json +1 -1
- package/.codex-plugin/plugin.json +1 -1
- package/.plugin/plugin.json +1 -1
- package/agents/aced-impl-judge.md +1 -1
- package/agents/aced-scenario-writer.md +1 -1
- package/package.json +1 -1
- package/skills/aced-architect-impl/SKILL.md +1 -1
- package/skills/aced-builder-impl/SKILL.md +2 -2
- package/skills/aced-builder-spec/SKILL.md +7 -8
- package/skills/aced-fit/SKILL.md +4 -5
- package/skills/aced-impl-producer/SKILL.md +4 -4
- package/skills/check-freshness/SKILL.md +4 -5
- package/skills/define-agent/SKILL.md +2 -2
- package/skills/define-command/SKILL.md +3 -3
- package/skills/define-governance/SKILL.md +2 -2
- package/skills/define-skill/SKILL.md +5 -6
- package/skills/extract-situation/SKILL.md +1 -1
- package/skills/improve/SKILL.md +2 -2
- package/skills/improve-skill/SKILL.md +1 -1
- package/skills/improve-skill/references/check-definitions.md +1 -1
- package/skills/improve-skill/scripts/validate.mts +2 -2
- package/skills/init-aced/SKILL.md +2 -2
- package/skills/list-skills/SKILL.md +1 -1
- package/skills/manage-model-runners/SKILL.md +1 -1
- package/skills/manage-skill-dirs/SKILL.md +1 -1
- package/skills/repair-private-skills/SKILL.md +1 -1
- package/skills/report/SKILL.md +2 -2
- package/skills/skillify/SKILL.md +3 -3
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
{
|
|
2
2
|
"$schema": "https://json.schemastore.org/claude-code-plugin-manifest.json",
|
|
3
3
|
"name": "aced",
|
|
4
|
-
"version": "0.4.
|
|
4
|
+
"version": "0.4.2",
|
|
5
5
|
"description": "Agent Config Evaluation & Development. Spec, evaluate, and improve agent configurations: skills, AGENTS.md sections, subagent definitions, and commands.",
|
|
6
6
|
"author": {
|
|
7
7
|
"name": "unional"
|
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
{
|
|
2
2
|
"$schema": "https://raw.githubusercontent.com/cyberuni/marketplace/main/schema/claude/plugin.json",
|
|
3
3
|
"name": "aced",
|
|
4
|
-
"version": "0.4.
|
|
4
|
+
"version": "0.4.2",
|
|
5
5
|
"description": "Agent Config Evaluation & Development. Spec, evaluate, and improve agent configurations: skills, AGENTS.md sections, subagent definitions, and commands.",
|
|
6
6
|
"author": {
|
|
7
7
|
"name": "unional"
|
package/.plugin/plugin.json
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
{
|
|
2
2
|
"$schema": "https://raw.githubusercontent.com/cyberuni/marketplace/main/schema/claude/plugin.json",
|
|
3
3
|
"name": "aced",
|
|
4
|
-
"version": "0.4.
|
|
4
|
+
"version": "0.4.2",
|
|
5
5
|
"description": "Agent Config Evaluation & Development. Spec, evaluate, and improve agent configurations: skills, AGENTS.md sections, subagent definitions, and commands.",
|
|
6
6
|
"author": {
|
|
7
7
|
"name": "unional"
|
|
@@ -28,7 +28,7 @@ VERIFICATION_PATHS: the frozen .feature (the eval source) + eval.md (run poli
|
|
|
28
28
|
|
|
29
29
|
1. **Read the frozen `.feature` as the eval source.** Enumerate its scenarios in file order. Each scenario carries its own eval: a `@rubric` scenario's inline rubric docstring (named dimensions + `max` + `threshold`), a `@trigger` `Scenario Outline`'s `Examples` rows, or a deterministic boolean `Then`. You **run** it, you do not author it. A `@rubric` scenario whose inline rubric block is **absent** is a `BLOCKER` — you do not free-author the missing rubric.
|
|
30
30
|
|
|
31
|
-
2. **Set the run policy from `eval.md`.** Read the run policy under the `eval:` block: `eval.trigger.{runs, activation_threshold}` for `@trigger` scenarios; `eval.judge.model` and `eval.judge.default_threshold` for `@behavior`/`@quality` scenarios (an inline `threshold` in a `@rubric` docstring overrides the default). Defaults when `eval.md` omits them: judge model
|
|
31
|
+
2. **Set the run policy from `eval.md`.** Read the run policy under the `eval:` block: `eval.trigger.{runs, activation_threshold}` for `@trigger` scenarios; `eval.judge.model` and `eval.judge.default_threshold` for `@behavior`/`@quality` scenarios (an inline `threshold` in a `@rubric` docstring overrides the default). Defaults when `eval.md` omits them: judge model the session's current model, default_threshold 4, trigger activation_threshold 0.5, trigger runs 3.
|
|
32
32
|
|
|
33
33
|
3. **Run the evals.** For each scenario, invoke `aced-case-judge` with the SUBJECT, the **`.feature` path and the scenario name**, and its threshold, over its run count — for a `@trigger` `Scenario Outline`, **once per `Examples` row**, passing its zero-based `ROW` (one row is one case; the judge reports one invoke decision, and aggregating accuracy across rows is yours). **Never pass the scenario's steps, its `Then`, or its rubric** — the judge blinds its own simulating context (composing that context's brief with the `extract-situation` engine) and scores the returned transcript in a separate one; handing it the scenario body would put the answer key back in the context that has to reach the answer. One invocation covers both passes; never sequence them here. It returns a score per named dimension against that dimension's own `max`, plus a total. Aggregate the totals across runs. <!-- field-mandate-ignore: ROW is aced-case-judge's input, which this judge constructs -->
|
|
34
34
|
|
|
@@ -56,7 +56,7 @@ USER_ANSWERS: <answers to previously returned QUESTIONS — or null>
|
|
|
56
56
|
- **Select the form before you grade it — the substitutability test.** A `@rubric` is a **compensatory** model (strength on one dimension pays for weakness on another), so a criterion belongs in one **only if** it is **substitutable**: only if you accept that trade — *"great scope makes up for shipping an npx dependency"* is a trade nobody accepts, so `no_npx_dependency` is a boolean `Then`, not a dimension. A rule graded as a dimension becomes **tradeable**, which is the one thing a rule must never be, and no threshold repairs it. Split a **double-barreled** dimension (`harness_agnostic_and_mcp_free`) before selecting — its halves routinely land in different forms. **Write the trade down** for each dimension you author or revise, in the same record that carries the cut's reason, naming it and **what pays for it** — a trade nobody wrote is one nobody can disagree with. The duty is **yours alone**: no judge reports a missing record, so nothing catches you skipping it. **Keep a property a boolean scenario in the suite already decides out of the rubric** — author it as the boolean only, never both; scoring it as a dimension too smuggles an untradeable rule into the compensatory sum (the boolean-smuggling tell: if your rubrics begin restating booleans, the boundary was set too high). This is a **same-object** check, not same-criterion: two dimensions may share a criterion with no boolean twin deciding either without either being a smuggled boolean.
|
|
57
57
|
- **Ask the span question before the trade question — a whole-output criterion is untradeable by construction.** When a graded criterion covers **the whole of what the subject produces** (a persona's voice, the fitness of the artifact as a whole), there is no *rest of the output* left to pay for weakness in it, so the compensatory sum has nothing to trade against. Author it as **one boolean `@quality` scenario**, never as a dimension — and never go hunting for a boolean twin the suite does not have; the same-object check above requires one, this does not. `@quality` names the **result axis** while `@rubric` is the orthogonal, optional **assertion form**, so a boolean `@quality` scenario is ordinary boolean Gherkin, not a new form. The boolean is strictly **stronger**: the gate requires every scenario to pass, so the property cannot be zeroed and bought back on unrelated points the way a dimension can. Do not let *"but it is graded, so it must be a `@rubric` dimension"* route you past this — that inference is exactly what failed. **The test is monadic:** read the single criterion against the output and ask *can I name an attribute of the output this criterion does not touch?* If you can, it is per-attribute and stays a dimension. Sibling dimensions tell you how the **output decomposes**; they are never what you compare the criterion against. Because span is asked **first**, a whole-output criterion authored alongside per-attribute siblings does not make the case ambiguous — the spanning one becomes the boolean and the siblings stay dimensions. A whole-output-*sounding* name (`coherence`, `clarity`, `overall_quality`) is not the test: judge what the dimension **scores**, not how broadly it is worded.
|
|
58
58
|
- **Read your scenarios against each other before returning.** No two sharing a `When` may demand opposite verdicts on one constructible snapshot; narrow one `Given` to exclude the overlap. Overlap whose `Then`s agree, and scenarios whose `When`s name different operations, are not contradictions.
|
|
59
|
-
- **On a backfill, re-derive the scenario set from the CFG's edges** — one per `(path class, edge)` pair, every guard paired with a positive companion — rather than reading the standing `.feature` and patching the gaps a diff notices. Any pre-existing suite or retired golden set is **reference only**: a claim to verify against the current configuration, never the baseline to patch (`sdd:suite-format-governance
|
|
59
|
+
- **On a backfill, re-derive the scenario set from the CFG's edges** — one per `(path class, edge)` pair, every guard paired with a positive companion — rather than reading the standing `.feature` and patching the gaps a diff notices. Any pre-existing suite or retired golden set is **reference only**: a claim to verify against the current configuration, never the baseline to patch (`sdd:suite-format-governance`).
|
|
60
60
|
|
|
61
61
|
4. **Pre-flight before you freeze.** Before returning the `spec.md` + `.feature`, run the produce-time pre-flight — the cheap check that stops a wrong assumption from being frozen into an already-shipped suite:
|
|
62
62
|
- **Ground every asserted harness/tool behavior in the harness's own primary docs.** When a `Then` (or the `spec.md`) asserts *how a harness or tool behaves* — a flag's effect, what a field controls, when a mechanism fires — confirm it against the harness's own primary documentation before freezing. Never assert harness behavior from memory; a wrong assumption frozen into a scenario forces a re-open of a shipped suite.
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "cyber-aced",
|
|
3
|
-
"version": "0.4.
|
|
3
|
+
"version": "0.4.2",
|
|
4
4
|
"description": "Agent Config Evaluation & Development. Spec, evaluate, and improve agent configurations: skills, AGENTS.md sections, subagent definitions, and commands.",
|
|
5
5
|
"repository": {
|
|
6
6
|
"type": "git",
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: aced-architect-impl
|
|
3
|
-
description: "
|
|
3
|
+
description: "By name only"
|
|
4
4
|
user-invocable: false
|
|
5
5
|
metadata:
|
|
6
6
|
actor: architect
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: aced-builder-impl
|
|
3
|
-
description: "
|
|
3
|
+
description: "By name only"
|
|
4
4
|
user-invocable: false
|
|
5
5
|
metadata:
|
|
6
6
|
actor: builder
|
|
@@ -29,7 +29,7 @@ agent level.
|
|
|
29
29
|
- **Run policy is explicit — from `eval.md`.** Read the `eval:` block: `@trigger` scenarios run
|
|
30
30
|
`eval.trigger.runs` against `eval.trigger.activation_threshold` (accuracy); `@behavior`/`@quality`
|
|
31
31
|
scenarios run N times against the scenario's inline `threshold` (else `eval.judge.default_threshold`),
|
|
32
|
-
scored by `eval.judge.model`. Defaults when omitted: model
|
|
32
|
+
scored by `eval.judge.model`. Defaults when omitted: model the session's current model, default_threshold 4,
|
|
33
33
|
trigger activation_threshold 0.5, trigger runs 3.
|
|
34
34
|
- **Collapse to a boolean.** A scenario passes when its aggregate **total across the rubric's named
|
|
35
35
|
dimensions** ≥ `threshold` and no must-not-do was triggered (trigger scenarios: accuracy ≥
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: aced-builder-spec
|
|
3
|
-
description: "
|
|
3
|
+
description: "By name only"
|
|
4
4
|
user-invocable: false
|
|
5
5
|
metadata:
|
|
6
6
|
actor: builder
|
|
@@ -78,8 +78,8 @@ bar. Two criteria below are **conditional on tier**; a **missing** `**Fit:**` de
|
|
|
78
78
|
- **The test is monadic.** Read the **single** criterion against the output: *can you name an
|
|
79
79
|
attribute of the output this criterion does not touch?* If you can, it is per-attribute and stays
|
|
80
80
|
a dimension. Sibling dimensions are evidence of how the **output decomposes** — never the object
|
|
81
|
-
of the comparison. Comparing one candidate dimension against another is the twin-scan
|
|
82
|
-
|
|
81
|
+
of the comparison. Comparing one candidate dimension against another is the twin-scan the
|
|
82
|
+
suite format rejects, on this ground exactly as much as on Ground 2.
|
|
83
83
|
- **Span is checked before substitutability.** A whole-output criterion never reaches the trade
|
|
84
84
|
question. So when one is authored **alongside** per-attribute siblings — the shape the
|
|
85
85
|
persona-voice recurrence actually took — the span ruling governs the spanning criterion and the
|
|
@@ -99,9 +99,8 @@ bar. Two criteria below are **conditional on tier**; a **missing** `**Fit:**` de
|
|
|
99
99
|
`only` bounds Ground 2's own test, not selection as a whole, so it never licenses passing a
|
|
100
100
|
criterion that fails Ground 1. Two `@rubric` dimensions that merely
|
|
101
101
|
**share a criterion**, with no boolean twin deciding either, do **not** fail selection for sharing
|
|
102
|
-
it — comparing two dimensions to each other is the twin-scan
|
|
103
|
-
|
|
104
|
-
hole) and this Selection verdict (a smuggled boolean, out of the sum) are orthogonal reads of one
|
|
102
|
+
it — comparing two dimensions to each other is the twin-scan the suite format rejects; the
|
|
103
|
+
discrimination verdict (noise, not a hole) and this Selection verdict (a smuggled boolean, out of the sum) are orthogonal reads of one
|
|
105
104
|
dimension.
|
|
106
105
|
- **Discrimination** *(all tiers; every scenario and every `@rubric` dimension).* Each must be able
|
|
107
106
|
to **register a miss** — a plausible wrong config must fail it, or score below the dimension's
|
|
@@ -126,5 +125,5 @@ bar. Two criteria below are **conditional on tier**; a **missing** `**Fit:**` de
|
|
|
126
125
|
Validate-section coverage requires a scenario for.
|
|
127
126
|
- `sdd:suite-format-governance` — the miss test, the wrong-subject table, the three anti-patterns,
|
|
128
127
|
the substitutability test, and the pairwise-consistency rule this bar specializes.
|
|
129
|
-
- `design/
|
|
130
|
-
|
|
128
|
+
- `design/test-levels.md` — the boolean-smuggling tell and the same-object, not same-criterion,
|
|
129
|
+
reconciliation the Selection bullet encodes.
|
package/skills/aced-fit/SKILL.md
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: aced-fit
|
|
3
|
-
description: "
|
|
3
|
+
description: "By name only"
|
|
4
4
|
user-invocable: false
|
|
5
5
|
metadata:
|
|
6
6
|
type: governance
|
|
@@ -35,7 +35,7 @@ name-only description) makes no activation decision, so it is never `strong`.
|
|
|
35
35
|
## How each role uses it
|
|
36
36
|
|
|
37
37
|
- **`aced-scenario-writer` (producer, explore) — decides.** Classify fit **first**; declare it as a
|
|
38
|
-
`**Fit:** strong | partial` line in the subject's node spec (its `
|
|
38
|
+
`**Fit:** strong | partial` line in the subject's node spec (its `spec.md`) `## Use Cases`. Then author to the tier:
|
|
39
39
|
`strong` → author should-trigger + same-keyword near-miss; `partial` → author behavior/edge/rule,
|
|
40
40
|
**no fabricated near-miss**; `wrong-squad` → **recuse**, produce nothing, recommend the SDD default.
|
|
41
41
|
- **`aced-spec-validator` (judge, gate) — enforces.** Read the declared tier; apply trigger-context /
|
|
@@ -83,7 +83,7 @@ is **not** a `CONTENT_GAP` (unlike the tier's).
|
|
|
83
83
|
Assertions about **the subject being classified**, run by the producer before handoff and re-run by
|
|
84
84
|
the cold judge against the artifact — never against the producer's account of what it did:
|
|
85
85
|
|
|
86
|
-
1. The subject's node spec (`
|
|
86
|
+
1. The subject's node spec (`spec.md`) carries exactly one `**Fit:** strong` or `**Fit:** partial`
|
|
87
87
|
line; a subject recused as wrong-squad has neither a `**Fit:**` line nor an ACED
|
|
88
88
|
`eval.md` (its node, if any, was written by the SDD-default chain). *(mechanical)*
|
|
89
89
|
2. A subject declared `strong` does not have a name-only description (one beginning `By name only`
|
|
@@ -98,8 +98,7 @@ the cold judge against the artifact — never against the producer's account of
|
|
|
98
98
|
|
|
99
99
|
## References
|
|
100
100
|
|
|
101
|
-
- `design/fit.md` (`.agents/specs/aced/design/fit.md`) — the normative model
|
|
102
|
-
ADR 0003 (measured axis).
|
|
101
|
+
- `design/fit.md` (`.agents/specs/aced/design/fit.md`) — the normative model (tier and measured axis).
|
|
103
102
|
- `aced:aced-builder-spec` — the spec bar whose trigger-context / trigger-balance criteria this
|
|
104
103
|
governance makes conditional.
|
|
105
104
|
- `aced:aced-builder-impl` — the impl bar (which eval layers get evals follows the tier).
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: aced-impl-producer
|
|
3
|
-
description: "
|
|
3
|
+
description: "By name only"
|
|
4
4
|
user-invocable: false
|
|
5
5
|
metadata:
|
|
6
6
|
actor: producer
|
|
@@ -13,7 +13,7 @@ Diagnose failing eval cases and propose targeted edits to the target agent confi
|
|
|
13
13
|
|
|
14
14
|
## Role: the ACED impl-producer
|
|
15
15
|
|
|
16
|
-
When the conductor dispatches `define-agent`, `define-skill`, or `define-governance` as a generic builder (`produced-by sdd:automaton`) for the **impl-producer** role (implement mode, against a frozen `.feature`), it builds the **agent configuration** to pass the frozen suite. The **verification is the frozen `.feature` itself** — its inline `@rubric` scenarios and `@trigger` `Examples`, authored by `aced-scenario-writer` at explore and frozen at the spec gate — so the impl-producer does **not** author a separate eval suite. As impl-producer it self-aligns to `sdd:ownership-governance` plus the resolved **builder-impl + architect-impl** bars (the ACED builder-impl is `aced:aced-builder-impl`). When the **impl-judge** (`aced-impl-judge`) reports scenario failures, load this skill to run the diagnose-and-refine loop below. Independence holds because the evals are anchored to the frozen `.feature` and executed by a separate runner (`aced-case-judge`), which is not the producer.
|
|
16
|
+
When the conductor dispatches `define-agent`, `define-skill`, or `define-governance` as a generic builder (`produced-by sdd:sdd-automaton`) for the **impl-producer** role (implement mode, against a frozen `.feature`), it builds the **agent configuration** to pass the frozen suite. The **verification is the frozen `.feature` itself** — its inline `@rubric` scenarios and `@trigger` `Examples`, authored by `aced-scenario-writer` at explore and frozen at the spec gate — so the impl-producer does **not** author a separate eval suite. As impl-producer it self-aligns to `sdd:ownership-governance` plus the resolved **builder-impl + architect-impl** bars (the ACED builder-impl is `aced:aced-builder-impl`). When the **impl-judge** (`aced-impl-judge`) reports scenario failures, load this skill to run the diagnose-and-refine loop below. Independence holds because the evals are anchored to the frozen `.feature` and executed by a separate runner (`aced-case-judge`), which is not the producer.
|
|
17
17
|
|
|
18
18
|
## Load context
|
|
19
19
|
|
|
@@ -23,7 +23,7 @@ and its colocated `eval.md` (subject + run policy):
|
|
|
23
23
|
- Read `eval.md` for the `subject` agent configuration path and the `eval:` run policy
|
|
24
24
|
- Read the `subject` agent configuration in full
|
|
25
25
|
- Read the frozen `<node>.feature` (the eval source)
|
|
26
|
-
- Read the most recent result file from
|
|
26
|
+
- Read the most recent result file from `.agents/aced/results/<target-slug>/` (sort by filename descending, take first)
|
|
27
27
|
|
|
28
28
|
If no results exist, run `run` first.
|
|
29
29
|
|
|
@@ -54,7 +54,7 @@ For each pattern, propose a concrete change to the agent configuration. Show exa
|
|
|
54
54
|
Examples of edit types:
|
|
55
55
|
|
|
56
56
|
- **Trigger false-positive/negative** → rewrite the `description:` field; add explicit "when NOT to use" section
|
|
57
|
-
- **Missing step** →
|
|
57
|
+
- **Missing step** → state the step and why it matters; break it into sub-steps if it is compound
|
|
58
58
|
- **Ambiguous rule** → replace vague language ("prefer X") with decision rule ("use X when Y, use Z when W")
|
|
59
59
|
- **Conflicting instruction** → add explicit precedence rule or split into separate instructions
|
|
60
60
|
- **Description mismatch** → align `description:` to match what the body actually instructs
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: check-freshness
|
|
3
|
-
description: "
|
|
3
|
+
description: "By name only"
|
|
4
4
|
user-invocable: false
|
|
5
5
|
metadata:
|
|
6
6
|
internal: true
|
|
@@ -70,7 +70,7 @@ about what it measured.
|
|
|
70
70
|
|
|
71
71
|
**Trust boundary.** `evaluated` is what the run **reports** consuming, not a verified trace. An
|
|
72
72
|
under-reporting run records a shorter set whose entries all match, and this answers `current` with
|
|
73
|
-
full confidence — a silent failure pointing the unsafe way
|
|
73
|
+
full confidence — a silent failure pointing the unsafe way. What is checkable is
|
|
74
74
|
**coherence**: a set omitting the `.feature` whose scenarios the record scores, or the configuration
|
|
75
75
|
its own `target` names, contradicts the record it accompanies and reads `absent`. That catches only
|
|
76
76
|
an *inconsistent* under-reporter; a uniform one escapes.
|
|
@@ -81,8 +81,7 @@ it should have" is not, and this engine never claims the second.
|
|
|
81
81
|
## Boundaries
|
|
82
82
|
|
|
83
83
|
Scores nothing (`run`), compares no two versions (`compare`), rolls nothing up (`report`), and does
|
|
84
|
-
not judge whether a recorded *pass* was well-founded (a judge-protocol question
|
|
85
|
-
a verdict and reports it; **what a caller does with that verdict is the caller's
|
|
86
|
-
`improve` to consult it is `#476`.
|
|
84
|
+
not judge whether a recorded *pass* was well-founded (a judge-protocol question). It decides
|
|
85
|
+
a verdict and reports it; **what a caller does with that verdict is the caller's**.
|
|
87
86
|
|
|
88
87
|
Spec: `.agents/specs/aced/eval-run/check-freshness/`.
|
|
@@ -7,7 +7,7 @@ description: 'Use this skill when the user wants to create or improve an agent d
|
|
|
7
7
|
|
|
8
8
|
Create or improve an agent definition — a named role encoded in a single file.
|
|
9
9
|
|
|
10
|
-
When the conductor dispatches this skill as a generic builder (`produced-by sdd:automaton`) for the ACED **impl-producer** role (implement mode, against a frozen `.feature`), it builds the **agent definition** to pass the frozen suite. The **verification is the frozen `.feature` itself** — its inline `@rubric` scenarios and `@trigger` `Examples`, authored by `aced-scenario-writer` at explore — so no separate eval suite is authored here; `eval.md` carries only the `subject` binding and run policy. As impl-producer it self-aligns to `sdd:ownership-governance` plus the resolved **builder-impl + architect-impl** bars (the ACED builder-impl is `aced:aced-builder-impl`). (Invoked standalone — no frozen `.feature` — only the agent definition is produced.) If the impl-judge reports scenario failures, load `aced-impl-producer` to run the diagnose-and-refine loop rather than re-deriving it here.
|
|
10
|
+
When the conductor dispatches this skill as a generic builder (`produced-by sdd:sdd-automaton`) for the ACED **impl-producer** role (implement mode, against a frozen `.feature`), it builds the **agent definition** to pass the frozen suite. The **verification is the frozen `.feature` itself** — its inline `@rubric` scenarios and `@trigger` `Examples`, authored by `aced-scenario-writer` at explore — so no separate eval suite is authored here; `eval.md` carries only the `subject` binding and run policy. As impl-producer it self-aligns to `sdd:ownership-governance` plus the resolved **builder-impl + architect-impl** bars (the ACED builder-impl is `aced:aced-builder-impl`). (Invoked standalone — no frozen `.feature` — only the agent definition is produced.) If the impl-judge reports scenario failures, load `aced-impl-producer` to run the diagnose-and-refine loop rather than re-deriving it here.
|
|
11
11
|
|
|
12
12
|
## First: confirm an agent definition is the right artifact
|
|
13
13
|
|
|
@@ -103,7 +103,7 @@ The canonical file lives at the canonical path. All runtime locations are symlin
|
|
|
103
103
|
|
|
104
104
|
Ask the user:
|
|
105
105
|
|
|
106
|
-
1. **Name** — kebab-case slug (e.g. `conductor`, `code-reviewer`). If the agent's role is to **score or verify a specific gate or case**, name it by that gate/scope, not a bare action verb: `<domain>-<gate>-judge` for a gate scorer (e.g. `sdd-impl-judge`, `aced-impl-judge`), `<domain>-case-judge` for a case scorer (e.g. `
|
|
106
|
+
1. **Name** — kebab-case slug (e.g. `conductor`, `code-reviewer`). If the agent's role is to **score or verify a specific gate or case**, name it by that gate/scope, not a bare action verb: `<domain>-<gate>-judge` for a gate scorer (e.g. `sdd-impl-judge`, `aced-impl-judge`), `<domain>-case-judge` for a case scorer (e.g. `aced-case-judge`). Flag any gate/case scorer whose name is not in that form — bare verdict verbs (`implementer`, `judge`, `validator`, `reviewer`, `checker`) and non-verdict action names (`eval-runner`, `grader`) alike. A producer/worker agent (e.g. `scenario-writer`, `doc-writer`) keeps its action-oriented name — this convention only applies to gate/case scorers.
|
|
107
107
|
2. **Role** — one sentence: "You are a [seniority] [role] focused on [bounded concern]."
|
|
108
108
|
3. **Responsibilities** — what does this agent do? (3–6 bounded concerns)
|
|
109
109
|
4. **Output format** — what does it produce? (file, report, JSON, confirmation, etc.)
|
|
@@ -8,7 +8,7 @@ description: 'Use this skill when the user wants to create or improve a command
|
|
|
8
8
|
Create or improve a **command** — a workflow the user invokes explicitly via `/name`, never
|
|
9
9
|
triggered automatically by the model.
|
|
10
10
|
|
|
11
|
-
When the conductor dispatches this skill as a generic builder (`produced-by sdd:automaton`) for the
|
|
11
|
+
When the conductor dispatches this skill as a generic builder (`produced-by sdd:sdd-automaton`) for the
|
|
12
12
|
ACED **impl-producer** role (implement mode, against a frozen `.feature`), it builds the **command
|
|
13
13
|
file** to pass the frozen suite. The **verification is the frozen `.feature` itself** — its inline
|
|
14
14
|
`@rubric` scenarios and `@trigger` `Examples`, authored by `aced-scenario-writer` at explore — so no
|
|
@@ -51,7 +51,7 @@ harnesses to target if not stated):
|
|
|
51
51
|
| Mechanism | File | Portability | When to pick it |
|
|
52
52
|
|---|---|---|---|
|
|
53
53
|
| **`commands/` folder** (legacy) | A standalone markdown file, not a `SKILL.md` | Widest support — Claude Code, and GitHub Copilot CLI (reads `.claude/commands/`); the only option for cross-harness plugin commands targeting Cursor too | Default choice; use for anything shipped in a plugin or targeting more than Claude Code |
|
|
54
|
-
| **`disable-model-invocation: true`** (modern) | A `SKILL.md` with the frontmatter flag | Claude Code, Windsurf; **not** Copilot CLI, Codex CLI, or Gemini CLI;
|
|
54
|
+
| **`disable-model-invocation: true`** (modern) | A `SKILL.md` with the frontmatter flag | Claude Code, Windsurf; **not** Copilot CLI, Codex CLI, or Gemini CLI; may not surface plugin-delivered skills in Cursor's `/` menu — verify before relying on it | Only when targeting Claude Code / Windsurf exclusively and the command benefits from full `SKILL.md` progressive disclosure (a `references/` or `scripts/` dir) |
|
|
55
55
|
|
|
56
56
|
If improving an existing file, read it first and keep its current mechanism unless the user asks to
|
|
57
57
|
migrate.
|
|
@@ -135,7 +135,7 @@ For the `disable-model-invocation` mechanism, run the structural audit and fix a
|
|
|
135
135
|
finding before presenting the skill:
|
|
136
136
|
|
|
137
137
|
```bash
|
|
138
|
-
npx cyberplace
|
|
138
|
+
npx cyberplace@<version> audit validate --path <placement-dir>/<name> # resolve <version> via: npm view cyberplace version — never @latest
|
|
139
139
|
```
|
|
140
140
|
|
|
141
141
|
The `commands/` folder mechanism is not a `SKILL.md` — the audit tool does not apply; check by hand
|
|
@@ -59,7 +59,7 @@ Write the file at the canonical path using this structure:
|
|
|
59
59
|
```markdown
|
|
60
60
|
---
|
|
61
61
|
name: <name>
|
|
62
|
-
description: "
|
|
62
|
+
description: "By name only"
|
|
63
63
|
user-invocable: false
|
|
64
64
|
metadata:
|
|
65
65
|
type: governance
|
|
@@ -90,7 +90,7 @@ dimensions already are the verification.>
|
|
|
90
90
|
|
|
91
91
|
Rules for the body:
|
|
92
92
|
- Set `user-invocable: false` — the classifier for a partial skill (a governance is loaded by name, never user-triggered)
|
|
93
|
-
-
|
|
93
|
+
- Set `description` to exactly `"By name only"` — the harness still sees the description, so anything added is another handle for a spurious match; put identity and caller in the body
|
|
94
94
|
- Do not include `## Why`, `## Rationale`, or causal prose — put that in ADRs
|
|
95
95
|
- Encode decisions and criteria, not facts the model already knows
|
|
96
96
|
- For a **Constraint set**, **Checklist**, or **Decision table** governance, write a `## Validate`
|
|
@@ -8,7 +8,7 @@ description: 'Creates, scaffolds, or formalizes a workflow skill — a process,
|
|
|
8
8
|
Create or improve a **workflow skill** — a process, tool-based, or standard SKILL.md — then hand it
|
|
9
9
|
to the ACED eval loop to spec and score.
|
|
10
10
|
|
|
11
|
-
When the conductor dispatches this skill as a generic builder (`produced-by sdd:automaton`) for the
|
|
11
|
+
When the conductor dispatches this skill as a generic builder (`produced-by sdd:sdd-automaton`) for the
|
|
12
12
|
ACED **impl-producer** role (implement mode, against a frozen `.feature`), it builds the **SKILL.md**
|
|
13
13
|
to pass the frozen suite. The **verification is the frozen `.feature` itself** — its inline `@rubric`
|
|
14
14
|
scenarios and `@trigger` `Examples`, authored by `aced-scenario-writer` at explore — so no separate
|
|
@@ -133,10 +133,9 @@ description: <capability> + "Use when <trigger>" + an implicit-phrasing example
|
|
|
133
133
|
with the capability, then a "Use when…" trigger clause, then an implicit phrasing. Never open with a
|
|
134
134
|
fixed lead-in such as "Use this skill when" or "When to use". Keep it ≤1024 characters, the
|
|
135
135
|
capability and main triggers first. For a
|
|
136
|
-
**
|
|
137
|
-
`user-invocable: false` and
|
|
138
|
-
|
|
139
|
-
non-trigger-shaped so it does not self-activate.
|
|
136
|
+
**name-only skill** (a reusable part other skills call by name, never matched to a situation), set
|
|
137
|
+
`user-invocable: false` and the description to exactly `"By name only"` and nothing else; identity
|
|
138
|
+
and caller go in the body.
|
|
140
139
|
- **Body** — step-by-step, under 500 lines. Keep the *when* here; move any **deterministic,
|
|
141
140
|
fixed-output** step to a `scripts/` file or an existing CLI and have the body only say when to run
|
|
142
141
|
it.
|
|
@@ -164,7 +163,7 @@ is `improve`.
|
|
|
164
163
|
For a subagent this skill scaffolds (realized as a partial skill, e.g. loaded via the ACED
|
|
165
164
|
impl-producer/impl-judge pattern): if its role is to **score or verify a specific gate or case**,
|
|
166
165
|
name it by that gate/scope, not a bare action verb — `<domain>-<gate>-judge` for a gate scorer (e.g.
|
|
167
|
-
`aced-impl-judge`), `<domain>-case-judge` for a case scorer (e.g. `
|
|
166
|
+
`aced-impl-judge`), `<domain>-case-judge` for a case scorer (e.g. `aced-case-judge`). Reject
|
|
168
167
|
`implementer`, `judge`, `validator`, `reviewer`, `checker` alone. A producer subagent (e.g.
|
|
169
168
|
`scenario-writer`, `doc-writer`) keeps its action-oriented name — this check does not fire for it.
|
|
170
169
|
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: extract-situation
|
|
3
|
-
description: "
|
|
3
|
+
description: "By name only"
|
|
4
4
|
user-invocable: false
|
|
5
5
|
metadata:
|
|
6
6
|
internal: true
|
package/skills/improve/SKILL.md
CHANGED
|
@@ -31,7 +31,7 @@ Check for the target's node in the project spec — `.agents/specs/<project>/…
|
|
|
31
31
|
through the SDD spec tree) — carrying a colocated `eval.md` for this target.
|
|
32
32
|
|
|
33
33
|
- **ACED-tracked (eval suite exists):** ensure a recent result exists — run `run` first if the
|
|
34
|
-
latest
|
|
34
|
+
latest file in `.agents/aced/results/<target-slug>/` is stale or missing. Then load `aced-impl-producer` to identify failing
|
|
35
35
|
scenarios, classify them by pattern, and propose concrete before/after edits.
|
|
36
36
|
- **Not yet tracked (no eval suite):** there is nothing to diagnose failures against. Do a general
|
|
37
37
|
review instead:
|
|
@@ -94,7 +94,7 @@ any agent, SDD or not.
|
|
|
94
94
|
If failures are caused by inherent non-determinism (high score variance across similar cases),
|
|
95
95
|
recommend:
|
|
96
96
|
1. Adding more specific examples to the config
|
|
97
|
-
2. Lowering the threshold in `eval.md`
|
|
97
|
+
2. Lowering the bar — but a per-scenario `threshold` is inline in the frozen `.feature`, so lowering it is a narrowing edit that needs a re-open and Clearance at the spec gate, not a casual `eval.md` change (only `eval.judge.default_threshold`, the fallback, lives in `eval.md`)
|
|
98
98
|
3. Splitting the config into two narrower ones
|
|
99
99
|
|
|
100
100
|
Do not propose removing test cases to fix failing evals — that defeats the purpose.
|
|
@@ -90,7 +90,7 @@ If you need the exact criteria for any check, read `references/check-definitions
|
|
|
90
90
|
| S9 | Structure | Extra documentation files in `references/`, not skill root | LOW | |
|
|
91
91
|
| Q1 | Quality | Description contains triggering context (situations, user requests) | HIGH | |
|
|
92
92
|
| Q2 | Quality | Description is specific (not vague / matches-everything) | HIGH | |
|
|
93
|
-
| Q3 | Quality | Partial skill (`user-invocable: false`) has `Partial Skill:` prefix
|
|
93
|
+
| Q3 | Quality | Partial skill (`user-invocable: false`) has description `By name only` (or `Partial Skill:` prefix) | MEDIUM | |
|
|
94
94
|
| Q4 | Quality | Skill has actionable instruction body (not just description) | MEDIUM | |
|
|
95
95
|
| Q5 | Quality | `description` ≤1024 characters (spec hard limit) | HIGH | |
|
|
96
96
|
| Q6 | Quality | No baked-in stack assumptions | MEDIUM | |
|
|
@@ -59,7 +59,7 @@ Warn if the description:
|
|
|
59
59
|
- Would plausibly match any user request (too broad to discriminate)
|
|
60
60
|
|
|
61
61
|
**Q3 — Partial-skill prefix (MEDIUM)**
|
|
62
|
-
Warn if the skill is a partial skill (top-level `user-invocable: false` — a reusable part invoked by name by another skill) but its `description`
|
|
62
|
+
Warn if the skill is a partial skill (top-level `user-invocable: false` — a reusable part invoked by name by another skill) but its `description` is neither exactly `"By name only"` nor starts with `"Partial Skill:"`. The mechanical engine keys this on `user-invocable: false`; the recommended form is exactly `"By name only"`. A partial skill stays `disable-model-invocation: false` so its caller can invoke it, so a non-declarative or trigger-shaped description may activate unintentionally.
|
|
63
63
|
|
|
64
64
|
**Q4 — Instruction body (MEDIUM)**
|
|
65
65
|
Warn if the skill body contains only a description and no actionable steps, numbered instructions, or decision logic. A skill with no instructions gives the agent nothing to execute.
|
|
@@ -675,13 +675,13 @@ export function runChecks(filePath: string, scanRoots?: Set<string>): CheckResul
|
|
|
675
675
|
)
|
|
676
676
|
}
|
|
677
677
|
|
|
678
|
-
if (isPartialSkill && fmDesc && !/^Partial Skill
|
|
678
|
+
if (isPartialSkill && fmDesc && !/^(Partial Skill:|By name only$)/i.test(fmDesc.trim())) {
|
|
679
679
|
warn(
|
|
680
680
|
'MEDIUM',
|
|
681
681
|
'Q3',
|
|
682
682
|
'Partial skill description missing the "Partial Skill:" prefix',
|
|
683
683
|
`description: ${fmDesc}`,
|
|
684
|
-
'
|
|
684
|
+
'Set the description to exactly "By name only" (or lead with "Partial Skill:")',
|
|
685
685
|
)
|
|
686
686
|
}
|
|
687
687
|
|
|
@@ -50,14 +50,14 @@ Do not reorder or reformat other entries.
|
|
|
50
50
|
"builder-spec": "aced-builder-spec",
|
|
51
51
|
"builder-impl": "aced-builder-impl",
|
|
52
52
|
"architect-spec": null,
|
|
53
|
-
"architect-impl":
|
|
53
|
+
"architect-impl": "aced-architect-impl"
|
|
54
54
|
}
|
|
55
55
|
}
|
|
56
56
|
]
|
|
57
57
|
}
|
|
58
58
|
```
|
|
59
59
|
|
|
60
|
-
ACED serves its four agent-config artifact-types with one squad. `impl-producer: null` — writing the agent config is done by the `define-agent` / `define-command` / `improve` skills or the SDD-default impl-producer (the conductor running `impl-producer-governance` via a spawned builder), not a bound impl-producer agent. `solution-producer: null` uses the SDD default. ACED binds its own `builder-spec` (`aced-builder-spec`)
|
|
60
|
+
ACED serves its four agent-config artifact-types with one squad. `impl-producer: null` — writing the agent config is done by the `define-agent` / `define-command` / `improve` skills or the SDD-default impl-producer (the conductor running `impl-producer-governance` via a spawned builder), not a bound impl-producer agent; `aced-impl-producer` is the diagnose-and-refine loop those skills load, not a role binding. `solution-producer: null` uses the SDD default. ACED binds its own `builder-spec` (`aced-builder-spec`), `builder-impl` (`aced-builder-impl`), and `architect-impl` (`aced-architect-impl`) bars; each remaining `null` governance uses the SDD default actor-gate bar.
|
|
61
61
|
|
|
62
62
|
**Each squad's `governances` block is required.** Every squad must carry a `governances` map (each binding may be `null`, but the block itself must be present). Reject a payload with a squad missing its `governances` block — fail with an error and **do not write** the file.
|
|
63
63
|
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: list-skills
|
|
3
|
-
description: "
|
|
3
|
+
description: "By name only"
|
|
4
4
|
user-invocable: false
|
|
5
5
|
metadata:
|
|
6
6
|
internal: true
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: manage-model-runners
|
|
3
|
-
description: "
|
|
3
|
+
description: "By name only"
|
|
4
4
|
user-invocable: false
|
|
5
5
|
metadata:
|
|
6
6
|
internal: true
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: manage-skill-dirs
|
|
3
|
-
description: "
|
|
3
|
+
description: "By name only"
|
|
4
4
|
user-invocable: false
|
|
5
5
|
metadata:
|
|
6
6
|
internal: true
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: repair-private-skills
|
|
3
|
-
description: "
|
|
3
|
+
description: "By name only"
|
|
4
4
|
user-invocable: false
|
|
5
5
|
metadata:
|
|
6
6
|
internal: true
|
package/skills/report/SKILL.md
CHANGED
|
@@ -11,8 +11,8 @@ Generate a project-wide health dashboard across all eval suites.
|
|
|
11
11
|
|
|
12
12
|
Scan the project spec (`.agents/specs/`) for behavioral-leaf nodes carrying an `eval.md` (subject + run policy) alongside their `<node>.feature`. For each, read:
|
|
13
13
|
- `eval.md` for target path and threshold
|
|
14
|
-
- The most recent file in
|
|
15
|
-
- The second-most-recent file
|
|
14
|
+
- The most recent file in `.agents/aced/results/<target-slug>/` (sort by filename descending)
|
|
15
|
+
- The second-most-recent file there for trend (if it exists)
|
|
16
16
|
|
|
17
17
|
If no matching directories are found, report that no eval suites are initialized and suggest `sdd:start-mission` (the conductor resolves the ACED roles).
|
|
18
18
|
|
package/skills/skillify/SKILL.md
CHANGED
|
@@ -31,7 +31,7 @@ The tell is the *target artifact*, not the "reuse this" framing: an agent/person
|
|
|
31
31
|
Load the **skill-design** governance before drafting:
|
|
32
32
|
|
|
33
33
|
```bash
|
|
34
|
-
npx cyberplace
|
|
34
|
+
npx cyberplace@<version> governance show skill-design # resolve <version> via: npm view cyberplace version — never @latest
|
|
35
35
|
```
|
|
36
36
|
|
|
37
37
|
Read stdout as the authoritative rules for principles, progressive disclosure, and description structure.
|
|
@@ -98,9 +98,9 @@ description: Use this skill when <trigger>. <One-line summary.>
|
|
|
98
98
|
### 2. <Step title>
|
|
99
99
|
...
|
|
100
100
|
|
|
101
|
-
##
|
|
101
|
+
## Pitfalls
|
|
102
102
|
|
|
103
|
-
- <
|
|
103
|
+
- <Mistake — the failure it causes and what to do instead>
|
|
104
104
|
```
|
|
105
105
|
|
|
106
106
|
Rules:
|