cyber-aced 0.3.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (77) hide show
  1. package/.claude-plugin/plugin.json +18 -0
  2. package/.plugin/pins.json +1 -0
  3. package/.plugin/plugin.json +18 -0
  4. package/agents/aced-case-judge.md +165 -0
  5. package/agents/aced-impl-judge.md +55 -0
  6. package/agents/aced-scenario-writer.md +76 -0
  7. package/agents/aced-spec-validator.md +66 -0
  8. package/governances/README.md +52 -0
  9. package/governances/agent-tool-output.md +86 -0
  10. package/governances/cli-resolution.md +102 -0
  11. package/governances/skill-design.md +258 -0
  12. package/governances/skill-repo-structure.md +173 -0
  13. package/package.json +44 -0
  14. package/readme.md +124 -0
  15. package/skills/aced-architect-impl/README.md +9 -0
  16. package/skills/aced-architect-impl/SKILL.md +73 -0
  17. package/skills/aced-builder-impl/README.md +14 -0
  18. package/skills/aced-builder-impl/SKILL.md +46 -0
  19. package/skills/aced-builder-spec/README.md +15 -0
  20. package/skills/aced-builder-spec/SKILL.md +130 -0
  21. package/skills/aced-fit/README.md +21 -0
  22. package/skills/aced-fit/SKILL.md +105 -0
  23. package/skills/aced-impl-producer/SKILL.md +75 -0
  24. package/skills/add-scenario/README.md +25 -0
  25. package/skills/add-scenario/SKILL.md +75 -0
  26. package/skills/bench/README.md +64 -0
  27. package/skills/bench/SKILL.md +199 -0
  28. package/skills/bench/scripts/bench-claude-code.mts +187 -0
  29. package/skills/bench/scripts/bench-compare.mts +369 -0
  30. package/skills/bench/scripts/bench-record.mts +133 -0
  31. package/skills/bench/scripts/bench.mts +930 -0
  32. package/skills/check-freshness/README.md +57 -0
  33. package/skills/check-freshness/SKILL.md +76 -0
  34. package/skills/check-freshness/scripts/check-freshness.mts +399 -0
  35. package/skills/compare/README.md +25 -0
  36. package/skills/compare/SKILL.md +87 -0
  37. package/skills/contribute-skill/README.md +25 -0
  38. package/skills/contribute-skill/SKILL.md +241 -0
  39. package/skills/contribute-skill/scripts/source.mts +188 -0
  40. package/skills/define-agent/README.md +40 -0
  41. package/skills/define-agent/SKILL.md +279 -0
  42. package/skills/define-command/README.md +36 -0
  43. package/skills/define-command/SKILL.md +174 -0
  44. package/skills/define-governance/SKILL.md +151 -0
  45. package/skills/define-skill/README.md +37 -0
  46. package/skills/define-skill/SKILL.md +200 -0
  47. package/skills/extract-situation/README.md +32 -0
  48. package/skills/extract-situation/SKILL.md +54 -0
  49. package/skills/extract-situation/scripts/extract-situation.mts +287 -0
  50. package/skills/improve/README.md +25 -0
  51. package/skills/improve/SKILL.md +100 -0
  52. package/skills/improve-skill/README.md +38 -0
  53. package/skills/improve-skill/SKILL.md +169 -0
  54. package/skills/improve-skill/references/check-definitions.md +215 -0
  55. package/skills/improve-skill/scripts/validate.mts +1033 -0
  56. package/skills/init-aced/README.md +5 -0
  57. package/skills/init-aced/SKILL.md +76 -0
  58. package/skills/init-aced/scripts/ensure-results-ignored.mts +117 -0
  59. package/skills/list-skills/README.md +19 -0
  60. package/skills/list-skills/SKILL.md +55 -0
  61. package/skills/list-skills/scripts/list-skills.mts +210 -0
  62. package/skills/manage/README.md +35 -0
  63. package/skills/manage/SKILL.md +65 -0
  64. package/skills/manage-model-runners/README.md +25 -0
  65. package/skills/manage-model-runners/SKILL.md +113 -0
  66. package/skills/manage-skill-dirs/README.md +17 -0
  67. package/skills/manage-skill-dirs/SKILL.md +54 -0
  68. package/skills/manage-skill-dirs/scripts/manage-skill-dirs.mts +328 -0
  69. package/skills/repair-private-skills/README.md +17 -0
  70. package/skills/repair-private-skills/SKILL.md +59 -0
  71. package/skills/repair-private-skills/scripts/repair-private-skills.mts +279 -0
  72. package/skills/report/README.md +25 -0
  73. package/skills/report/SKILL.md +74 -0
  74. package/skills/run/README.md +25 -0
  75. package/skills/run/SKILL.md +220 -0
  76. package/skills/skillify/README.md +25 -0
  77. package/skills/skillify/SKILL.md +141 -0
@@ -0,0 +1,37 @@
1
+ # define-skill
2
+
3
+ Create or improve a workflow skill — a process, tool-based, or standard SKILL.md — then hand it to the ACED eval loop to spec and score.
4
+
5
+ ## When to use
6
+
7
+ Use this skill when you want to author a new workflow skill or formalize an existing ad-hoc workflow into one.
8
+
9
+ Good triggers include:
10
+
11
+ - "Create a skill that does X" / "I want a skill for X"
12
+ - Turning a documented multi-step process into a reusable skill
13
+ - Filling out an incomplete or weakly-triggering existing SKILL.md
14
+ - Scaffolding the SKILL.md + README + runtime symlinks in one pass
15
+
16
+ Defer to a sibling when the request is really about an agent or persona (`define-agent`), a reference-only rule set (`define-governance`), extracting the current session (`skillify`), or scoring / diagnosing an existing config (`run` / `add-scenario` / `improve`).
17
+
18
+ ## What it does
19
+
20
+ The skill walks you through the shape before writing anything:
21
+
22
+ 1. **Route** — confirm this is a workflow skill, not an agent, governance, or session extraction
23
+ 2. **Settle scope** — the five design questions (scope, trigger, output contract, quality bar, out-of-scope)
24
+ 3. **Pattern** — process, tool-based, or standard (personas go to `define-agent`)
25
+ 4. **Placement** — user-global, project-private, or project-public, plus which runtime symlinks to create
26
+
27
+ It then drafts the SKILL.md (kebab-case name, a 150–400 char trigger-bearing description, a step body under the size bar), adds a README for a public skill, runs the structural audit and fixes CRITICAL/HIGH findings, and points you at the ACED eval loop (`start-mission` / `add-scenario` / `run`) to spec and score it.
28
+
29
+ ## Enhanced from create-skill
30
+
31
+ `define-skill` is the ACED-native successor to the legacy `create-skill` skill: same scaffold → audit → link lifecycle, but the test step is the ACED eval loop (a frozen `.feature` with inline `@rubric` scenarios scored over N runs) instead of a one-shot trigger-query file, and fit is classified up front so authoring effort matches the subject.
32
+
33
+ ## Install
34
+
35
+ ```bash
36
+ npx skills add cyberuni/cyberplace --skill aced/define-skill
37
+ ```
@@ -0,0 +1,200 @@
1
+ ---
2
+ name: define-skill
3
+ description: 'Use this skill when the user wants to create, scaffold, or formalize a workflow skill — a process, tool-based, or standard SKILL.md — from scratch or from an existing ad-hoc workflow. Trigger on "create a skill", "write a skill for X", "I want a skill that…", "turn this workflow into a skill", or "help me build this skill", even if they do not say "skill" explicitly. Not for agent definitions or personas (define-agent), reference-only rule sets (define-governance), or extracting the current session (skillify).'
4
+ ---
5
+
6
+ # Define Skill
7
+
8
+ Create or improve a **workflow skill** — a process, tool-based, or standard SKILL.md — then hand it
9
+ to the ACED eval loop to spec and score.
10
+
11
+ When the conductor dispatches this skill as a generic builder (`produced-by sdd:automaton`) for the
12
+ ACED **impl-producer** role (implement mode, against a frozen `.feature`), it builds the **SKILL.md**
13
+ to pass the frozen suite. The **verification is the frozen `.feature` itself** — its inline `@rubric`
14
+ scenarios and `@trigger` `Examples`, authored by `aced-scenario-writer` at explore — so no separate
15
+ eval suite is authored here; `eval.md` carries only the `subject` binding and run policy. As
16
+ impl-producer it self-aligns to `sdd:ownership-governance` plus the resolved **builder-impl +
17
+ architect-impl** bars (the ACED builder-impl is `aced:aced-builder-impl`). If the impl-judge reports
18
+ scenario failures, load `aced-impl-producer` to run the diagnose-and-refine loop rather than
19
+ re-deriving it here.
20
+
21
+ There are two other entry points, both with **no frozen `.feature`** and **only the SKILL.md**
22
+ produced:
23
+
24
+ - **Standalone** — the user invokes this skill directly, outside any CR. Scaffold, then offer the
25
+ ACED eval loop (see "Report and hand off" below) rather than assuming it.
26
+ - **Escaped** — the gateway or `start-mission` invokes this skill directly after resolving the
27
+ request `non-durable` (the escape hatch, before any CR opens). There is no mission to hand off
28
+ within: scaffold, audit, report, and **stop** — do not mention the ACED eval loop.
29
+
30
+ ## Route the request first
31
+
32
+ `define-skill` owns **workflow skills**. Defer when the intent belongs to a sibling that carries the
33
+ same config vocabulary:
34
+
35
+ | The request is really about… | Defer to |
36
+ |---|---|
37
+ | a named reusable **agent** / delegated worker / an in-context **persona** | `define-agent` |
38
+ | a **reference-only rule set / governance** other skills load but never execute as steps | `define-governance` |
39
+ | generalizing **the current session's** work into a skill | `skillify` |
40
+ | **scoring** an existing config, or **adding** an eval case | `run` / `add-scenario` |
41
+ | **diagnosing** why an existing skill's evals fail | `improve` |
42
+
43
+ A persona request that surfaces mid-authoring is handed to `define-agent` — do not scaffold a persona
44
+ skill here.
45
+
46
+ ## Load the governance
47
+
48
+ Before writing content, load the **skill-design** governance and read stdout as the authoritative
49
+ rules for principles, progressive disclosure, description structure, and when to extract deterministic
50
+ logic to a script:
51
+
52
+ ```bash
53
+ npx cyberplace@0.2.4 governance show skill-design # resolve <version> via: npm view cyberplace version — never @latest
54
+ ```
55
+
56
+ If the skill will ship a `scripts/` directory or document CLI commands agents run, also load
57
+ **agent-tool-output** for stdout / JSON / non-interactive / stderr rules.
58
+
59
+ ## Settle the shape — five questions
60
+
61
+ Answer all five before writing a line of SKILL.md. If any cannot be inferred from what the user has
62
+ said, **ask the user to resolve it** — never invent it:
63
+
64
+ | Question | What to decide |
65
+ |---|---|
66
+ | Scope | What exactly does this skill do? One workflow only. |
67
+ | Trigger condition | When should it fire? List both explicit and implicit phrasings. |
68
+ | Output contract | What artifact does it produce — a file path, a report format, a decision? |
69
+ | Quality bar | How will you know a run succeeded? A concrete pass condition. |
70
+ | Out of scope | What must this skill explicitly NOT do? |
71
+
72
+ ## Choose the pattern
73
+
74
+ Capture the workflow shape (separate from placement). A persona is **not** a pattern here — route it
75
+ to `define-agent`.
76
+
77
+ | Pattern | When to use it | Body shape |
78
+ |---|---|---|
79
+ | Process | Ordered multi-step workflow with decision points | Numbered steps |
80
+ | Tool-based | Consistent use of tools, systems, or connectors | Tool usage + guardrails |
81
+ | Standard | Tone, format, structure, or quality enforcement | Rules + pass conditions |
82
+
83
+ ## Resolve placement and runtimes
84
+
85
+ Pick placement before scaffolding; if unclear from context, ask:
86
+
87
+ | Placement | Location | Notes |
88
+ |---|---|---|
89
+ | User-global | `~/.agents/skills/<name>/` | Personal, across all projects; no README |
90
+ | Project-private | `.agents/skills/<name>/` | Contributor tooling; add `metadata: internal: true` |
91
+ | Project-public | `skills/<name>/` (or `plugins/<plugin>/skills/<name>/`) | Shipped with a package; **requires a README** |
92
+
93
+ Check whether `npx skills` is available, then scaffold at the derived path:
94
+
95
+ ```bash
96
+ npx skills --version 2>/dev/null && npx skills init <name> --dir <placement-dir> # else: mkdir -p <placement-dir>/<name>
97
+ ```
98
+
99
+ Ask which runtimes to target; the SKILL.md is the canonical file, each runtime location is a symlink
100
+ to it. Create and **verify** one symlink per selected runtime:
101
+
102
+ | Runtime | Symlink location |
103
+ |---|---|
104
+ | Claude Code | `~/.claude/skills/<name>` (or `.claude/skills/<name>`) |
105
+ | Cursor | `~/.cursor/skills/<name>` |
106
+ | Codex | `~/.codex/skills/<name>` |
107
+
108
+ ```bash
109
+ ln -sf <relative-path-to-canonical> <runtime-location> && ls <runtime-location> # verify it resolves
110
+ ```
111
+
112
+ ## Draft the SKILL.md
113
+
114
+ Write the file at the canonical path:
115
+
116
+ ```markdown
117
+ ---
118
+ name: <name> # kebab-case, must match the directory name exactly
119
+ description: <capability> + "Use when <trigger>" + an implicit-phrasing example ("even if they say …")
120
+ ---
121
+
122
+ # <Title>
123
+
124
+ ## When to use
125
+ <the trigger condition, restated for the body>
126
+
127
+ ## Instructions
128
+ 1. <step>
129
+ 2. <step>
130
+ ```
131
+
132
+ - **Description** — the only field loaded at startup; it carries the whole triggering burden. Target
133
+ 150–400 characters (≤1024 hard limit): capability + "Use when…" + an implicit phrasing. For a
134
+ **partial skill** (a reusable part other skills call by name, not user-triggered), set
135
+ `user-invocable: false` and lead the description with the `"Partial Skill:"` prefix (recommended
136
+ form `"Partial Skill: invoke by name only — <identity>. <caller>."`), kept minimal and
137
+ non-trigger-shaped so it does not self-activate.
138
+ - **Body** — step-by-step, under 500 lines. Keep the *when* here; move any **deterministic,
139
+ fixed-output** step to a `scripts/` file or an existing CLI and have the body only say when to run
140
+ it.
141
+ - **Project-public** skills get a `README.md` beside the SKILL.md (title, when-to-use, what-it-does,
142
+ install line). User-global skills get none.
143
+ - **Binding rules get a `## Validate` section.** If the body states rules the agent must, must not,
144
+ always, or never do — most commonly the **Standard** pattern, but any pattern can carry one —
145
+ add a `## Validate` section: one assertion per rule, phrased as an observable, falsifiable claim
146
+ about the resulting artifact (a diff, a file's presence, a structural shape), not a restatement of
147
+ the rule or a claim about the agent's intent. Write it so it works run twice, from two vantage
148
+ points: as the agent's own pre-handoff self-check before declaring the work done, and as an
149
+ independent re-check by a cold reader later, against the same artifact rather than against
150
+ anything the first agent said about itself. Prefer a mechanical check (script, grep, diff) wherever
151
+ the rule permits one. A skill whose body is only ordered steps with no binding rules on the
152
+ *outcome* does not need one — this is for rules a result can violate, not for the steps themselves.
153
+
154
+ ### Improving an existing skill
155
+
156
+ If the target SKILL.md already exists, **read it first** and change **only** the gaps or issues found
157
+ — leave every sound part intact. This fills gaps in a skill's *definition*; diagnosing failing evals
158
+ is `improve`.
159
+
160
+ ## Name gate/case scorers by role
161
+
162
+ For a subagent this skill scaffolds (realized as a partial skill, e.g. loaded via the ACED
163
+ impl-producer/impl-judge pattern): if its role is to **score or verify a specific gate or case**,
164
+ name it by that gate/scope, not a bare action verb — `<domain>-<gate>-judge` for a gate scorer (e.g.
165
+ `aced-impl-judge`), `<domain>-case-judge` for a case scorer (e.g. `aces-case-judge`). Reject
166
+ `implementer`, `judge`, `validator`, `reviewer`, `checker` alone. A producer subagent (e.g.
167
+ `scenario-writer`, `doc-writer`) keeps its action-oriented name — this check does not fire for it.
168
+
169
+ Evaluate this **in-skill quality check** — separate from the mechanical `audit validate` below — before
170
+ handing the skill back: a drafted gate/case scorer named with a bare action verb is a **HIGH** severity
171
+ finding; fix it (rename to the gate-and-scope form) before presenting the skill.
172
+
173
+ ## Audit before handing back
174
+
175
+ Run the structural audit and fix any CRITICAL or HIGH finding **before** presenting the skill:
176
+
177
+ ```bash
178
+ npx cyberplace@0.2.4 audit validate --path <placement-dir>/<name>
179
+ ```
180
+
181
+ For a fuller pass, invoke the `improve-skill` skill. Do not present a skill with an
182
+ open CRITICAL finding.
183
+
184
+ ## Report and hand off to the ACED eval loop
185
+
186
+ Summarize:
187
+
188
+ - Canonical SKILL.md path
189
+ - README path (project-public only)
190
+ - Runtime symlinks created and verified
191
+ - Audit outcome
192
+
193
+ **Escaped** entry: stop here — the artifact resolved `non-durable`, so there is no mission to hand
194
+ off to.
195
+
196
+ **Standalone or impl-producer** entry: point the user at the **ACED eval loop** to spec and score
197
+ the skill — run `sdd:start-mission` (the conductor resolves the ACED roles for the `skill`
198
+ artifact-type) to author its frozen `.feature` (with inline `@rubric`), or `add-scenario` / `run` to
199
+ grow and score it. Do **not** embed a legacy trigger-query eval file as the test step — scoring is the
200
+ ACED loop's job.
@@ -0,0 +1,32 @@
1
+ # extract-situation
2
+
3
+ Internal, non-invokable ACED engine — loaded by `aced-case-judge`, not triggered directly. It reads
4
+ one scenario out of a frozen `.feature` and emits only the situation a simulating agent may see.
5
+
6
+ ## What it does
7
+
8
+ Parses the `.feature`, finds the named scenario, and emits its `Given` and `When` steps verbatim, in
9
+ file order. It withholds the scenario name, every `Then`, the inline `@rubric` docstring, the tags,
10
+ the `Feature:` description, and every sibling scenario.
11
+
12
+ An `And` inherits the keyword above it, so `Then A / And B` withholds `And B` while
13
+ `Given X / And Y` emits `And Y`. For a `Scenario Outline`, `<placeholder>` tokens are preserved and
14
+ an `Examples` column survives only when an emitted step references it.
15
+
16
+ Every failure exits non-zero and emits nothing: an unreadable or unparseable file, a scenario name
17
+ that is absent or matches twice, or a missing argument.
18
+
19
+ ## Why it exists
20
+
21
+ ACED's per-case scorer used to be handed a scenario's name, its `Then`, and its rubric — and then
22
+ asked to simulate an agent and score that simulation in one context. The simulator was told the
23
+ answer before it answered, so a passing score could not distinguish an agent that reasons from one
24
+ handed the key. This engine composes the simulator's brief mechanically, so the redaction is done by
25
+ tested code rather than by a judge deciding what to withhold from itself.
26
+
27
+ ## Usage
28
+
29
+ ```
30
+ node "<skill>/scripts/extract-situation.mts" --feature <path-to-.feature> --scenario "<exact name>"
31
+ node "<skill>/scripts/extract-situation.mts" --feature <path> --scenario "<name>" --format json
32
+ ```
@@ -0,0 +1,54 @@
1
+ ---
2
+ name: extract-situation
3
+ description: "Partial Skill: invoke by name only — the ACED blind-brief extractor — loaded by aced-case-judge to compose a simulating context's brief, not user-triggered."
4
+ user-invocable: false
5
+ metadata:
6
+ internal: true
7
+ ---
8
+
9
+ # extract-situation
10
+
11
+ The **blind-brief extractor** — the deterministic engine that reads one scenario out of a frozen
12
+ `.feature` and emits **only the situation**: its `Given` and `When` steps. The scenario **name**,
13
+ every `Then`, the inline `@rubric`, and the tags are **never** emitted. `aced-case-judge` composes a
14
+ simulating context's brief **only** through this engine; it never decides by its own judgment what to
15
+ withhold. It is **read-only** — it never writes a `.feature`, a brief, or any other file.
16
+
17
+ ## Invocation
18
+
19
+ ```
20
+ node "<skill>/scripts/extract-situation.mts" --feature <path-to-.feature> --scenario "<exact name>"
21
+ ```
22
+
23
+ `--format json` emits `{ steps, given, when, placeholders, examples }` instead of the markdown brief —
24
+ `steps` is the emitted brief in file order (`given`/`when` regroup the same steps by keyword), and
25
+ `examples` carries a `Scenario Outline`'s surviving columns.
26
+
27
+ ## What it emits, and what it must never emit
28
+
29
+ | Element | Emitted | Why |
30
+ |---|---|---|
31
+ | `Given` steps (and an `And`/`But` inheriting `Given`) | **yes** | the situation the agent is in |
32
+ | `When` steps (and an `And`/`But` inheriting `When`) | **yes** | the act under evaluation |
33
+ | Scenario **name** | never | names state the verdict (`red tests block the commit`) |
34
+ | `Then` steps, and any `And`/`But` inheriting `Then` | never | the expected outcome is the answer key |
35
+ | Inline `@rubric` docstring | never | the scoring ladders are worked answers |
36
+ | Tags, `Feature:` description, sibling scenarios | never | not the situation |
37
+
38
+ **Keyword inheritance is the load-bearing rule.** An `And` takes the keyword of the step above it:
39
+ `Given X / And Y` makes `And Y` a `Given` (emit); `Then A / And B` makes `And B` a `Then` (withhold).
40
+ Tracking the current keyword is what keeps the answer key out of the brief.
41
+
42
+ For a `Scenario Outline`, `<placeholder>` tokens stay intact and an `Examples` column is emitted
43
+ **only when an emitted step references its placeholder** — a column only a `Then` reads is withheld.
44
+
45
+ ## Fail closed
46
+
47
+ Every failure exits non-zero and emits **no brief** — an empty brief must never be mistakable for a
48
+ valid one. Unreadable or unparseable `.feature`; scenario name absent; the name matching **two**
49
+ scenarios; `--feature` or `--scenario` missing.
50
+
51
+ ## References
52
+
53
+ The contract this engine serves: `aced-case-judge`'s blind two-pass protocol, spec'd at
54
+ `.agents/specs/aced/sdd-roles/judge/`.
@@ -0,0 +1,287 @@
1
+ #!/usr/bin/env node
2
+ // extract-situation — mechanically redacts a named .feature scenario down to its SITUATION: the
3
+ // Given/When steps only. This is the fix for the "eval cannot fail" defect in
4
+ // plugins/aced/agents/aced-case-judge.md — that judge is handed the scenario's Then/And/But
5
+ // assertions AND its inline @rubric in the SAME context it simulates the subject in, so the
6
+ // simulator sees the answer key before it acts. A simulating context must see ONLY this engine's
7
+ // output: never the scenario name, never a Then/And/But, never a docstring, never a tag, never a
8
+ // sibling scenario.
9
+ //
10
+ // The keyword-inheritance rule is the load-bearing logic: `And`/`But` inherit the keyword of the
11
+ // concrete step above them (`Given X` / `And Y` → Y is a Given; `Then A` / `And B` → B is a
12
+ // Then). Each step's EFFECTIVE keyword — never its literal `And`/`But` text — decides inclusion.
13
+ //
14
+ // Pure functions are exported for node:test; running the file directly drives the CLI. No
15
+ // dependencies — plain node strips the types.
16
+
17
+ import { readFileSync, realpathSync } from 'node:fs'
18
+ import { pathToFileURL } from 'node:url'
19
+
20
+ // ─── types ────────────────────────────────────────────────────────────────────
21
+
22
+ export type ConcreteKeyword = 'Given' | 'When' | 'Then'
23
+
24
+ // An `And`/`But` with no concrete step above it has no keyword to inherit. It resolves to
25
+ // `unresolved` and is withheld: a step whose keyword cannot be established is never emitted.
26
+ export type EffectiveKeyword = ConcreteKeyword | 'unresolved'
27
+
28
+ export interface Step {
29
+ text: string
30
+ effectiveKeyword: EffectiveKeyword
31
+ // A docstring belongs to the step above it and inherits that step's fate. Under a `Given`/`When`
32
+ // it IS the situation (the prompt under test); under a `Then` it is the answer key (the rubric).
33
+ docstring: { fence: string; lines: string[] } | null
34
+ }
35
+
36
+ export interface ParsedScenario {
37
+ name: string
38
+ steps: Step[]
39
+ isOutline: boolean
40
+ examples: { header: string[]; rows: string[][] } | null
41
+ }
42
+
43
+ export interface Situation {
44
+ // Every emitted step, in the order the file lists them. `given`/`when` regroup the same steps
45
+ // by keyword; the brief is rendered from `steps`, so an interleaved scenario reads as written.
46
+ steps: string[]
47
+ given: string[]
48
+ when: string[]
49
+ placeholders: string[]
50
+ examples: { header: string[]; rows: string[][] } | null
51
+ }
52
+
53
+ // ─── parse ────────────────────────────────────────────────────────────────────
54
+
55
+ // Finds every scenario in the file (name, steps with resolved keywords, outline/Examples). Steps
56
+ // carry the ORIGINAL line text (minus indentation) so the brief is verbatim; only the keyword used
57
+ // for inclusion is resolved.
58
+ //
59
+ // Docstring lines are skipped wholesale rather than left to fall through the step regex. A rubric
60
+ // ladder routinely opens a line with a step keyword ("When the agent stages only related files,
61
+ // award 3"); read as a step, that line both leaks the answer key into the brief AND overwrites
62
+ // `lastKeyword`, so the collapsing `And ... at least the threshold` below it inherits `When` and
63
+ // leaks too.
64
+ export function parseScenarios(text: string): ParsedScenario[] {
65
+ const lines = text.split('\n')
66
+ const scenarios: ParsedScenario[] = []
67
+ let current: ParsedScenario | null = null
68
+ let lastKeyword: ConcreteKeyword | null = null
69
+ let docstringFence: string | null = null
70
+ let docstringOwner: Step | null = null
71
+
72
+ for (const raw of lines) {
73
+ const line = raw.trimStart()
74
+
75
+ // Docstring lines are captured onto their owning step, never re-read as steps. Letting them
76
+ // fall through the step regex is what leaks a rubric ladder that opens with a step keyword
77
+ // ("When the agent stages only related files, award 3") into the brief, and overwrites
78
+ // `lastKeyword` so the collapsing `And ... at least the threshold` below it leaks too.
79
+ const fence = /^("""|```)/.exec(line)?.[1]
80
+ if (docstringFence) {
81
+ if (fence === docstringFence) {
82
+ docstringFence = null
83
+ docstringOwner = null
84
+ } else if (docstringOwner?.docstring) {
85
+ docstringOwner.docstring.lines.push(raw)
86
+ }
87
+ continue
88
+ }
89
+ if (fence) {
90
+ docstringFence = fence
91
+ const owner = current?.steps.at(-1) ?? null
92
+ if (owner) {
93
+ owner.docstring = { fence, lines: [] }
94
+ docstringOwner = owner
95
+ }
96
+ continue
97
+ }
98
+
99
+ if (/^Scenario:/i.test(line) || /^Scenario Outline:/i.test(line)) {
100
+ if (current) scenarios.push(current)
101
+ const isOutline = /^Scenario Outline:/i.test(line)
102
+ const name = line.replace(/^Scenario(?: Outline)?:/i, '').trim()
103
+ current = { name, steps: [], isOutline, examples: null }
104
+ lastKeyword = null
105
+ continue
106
+ }
107
+
108
+ if (current && /^Examples:/i.test(line)) {
109
+ current.examples = { header: [], rows: [] }
110
+ continue
111
+ }
112
+
113
+ if (current?.examples && line.startsWith('|')) {
114
+ const cells = line
115
+ .split('|')
116
+ .slice(1, -1)
117
+ .map((c) => c.trim())
118
+ if (current.examples.header.length === 0) current.examples.header = cells
119
+ else current.examples.rows.push(cells)
120
+ continue
121
+ }
122
+
123
+ const stepMatch = /^(Given|When|Then|And|But)\b/i.exec(line)
124
+ if (current && stepMatch) {
125
+ const literal = stepMatch[1]
126
+ const resolved: EffectiveKeyword = /^(Given|When|Then)$/i.test(literal)
127
+ ? ((literal[0].toUpperCase() + literal.slice(1).toLowerCase()) as ConcreteKeyword)
128
+ : (lastKeyword ?? 'unresolved')
129
+ current.steps.push({ text: line, effectiveKeyword: resolved, docstring: null })
130
+ if (resolved !== 'unresolved') lastKeyword = resolved
131
+ }
132
+ }
133
+
134
+ if (current) scenarios.push(current)
135
+ return scenarios
136
+ }
137
+
138
+ // ─── extraction ───────────────────────────────────────────────────────────────
139
+
140
+ export class ScenarioNotFoundError extends Error {}
141
+ export class AmbiguousScenarioError extends Error {}
142
+ export class UnparseableFeatureError extends Error {}
143
+ // A scenario the file holds but which carries no Given/When has no situation to simulate from. It
144
+ // fails closed rather than emitting an empty brief: a simulator handed nothing simulates nothing,
145
+ // the case scores low, and the eval reads as a defect in the SUBJECT rather than in the extraction.
146
+ export class EmptySituationError extends Error {}
147
+ // A `Scenario Outline` row index that the Examples table does not hold.
148
+ export class RowOutOfRangeError extends Error {}
149
+
150
+ // Extracts the redacted situation for the named scenario: its Given/When steps only, by EFFECTIVE
151
+ // keyword (never literal text) — every Then/And/But-after-Then step, every tag, the scenario name
152
+ // itself, and every sibling scenario are excluded by construction (never read into the result).
153
+ // Fails closed: an unparseable file, a missing name, or a duplicate name throws rather than
154
+ // returning an empty-but-plausible brief a caller could mistake for a valid one.
155
+ export function extractSituation(text: string, scenarioName: string, fileLabel: string, row?: number): Situation {
156
+ if (!/^\s*Feature:/im.test(text)) {
157
+ throw new UnparseableFeatureError(`${fileLabel}: not a valid .feature file (no Feature: line)`)
158
+ }
159
+
160
+ const scenarios = parseScenarios(text)
161
+ const matches = scenarios.filter((s) => s.name === scenarioName)
162
+
163
+ if (matches.length === 0) {
164
+ throw new ScenarioNotFoundError(`${fileLabel}: scenario not found: "${scenarioName}"`)
165
+ }
166
+ if (matches.length > 1) {
167
+ throw new AmbiguousScenarioError(
168
+ `${fileLabel}: scenario name is ambiguous (${matches.length} matches): "${scenarioName}"`,
169
+ )
170
+ }
171
+
172
+ const scenario = matches[0]
173
+ const emitted = scenario.steps.filter((s) => s.effectiveKeyword === 'Given' || s.effectiveKeyword === 'When')
174
+ const steps = emitted.flatMap((s) =>
175
+ s.docstring ? [s.text, s.docstring.fence, ...s.docstring.lines, s.docstring.fence] : [s.text],
176
+ )
177
+ const withDoc = (s: Step) =>
178
+ s.docstring ? [s.text, s.docstring.fence, ...s.docstring.lines, s.docstring.fence] : [s.text]
179
+ const given = emitted.filter((s) => s.effectiveKeyword === 'Given').flatMap(withDoc)
180
+ const when = emitted.filter((s) => s.effectiveKeyword === 'When').flatMap(withDoc)
181
+
182
+ if (steps.length === 0) {
183
+ throw new EmptySituationError(`${fileLabel}: scenario has no Given or When steps: "${scenarioName}"`)
184
+ }
185
+
186
+ const placeholders = [...new Set(steps.flatMap((s) => [...s.matchAll(/<([^>]+)>/g)].map((m) => m[1])))]
187
+
188
+ let examples: Situation['examples'] = null
189
+ if (scenario.isOutline && scenario.examples && placeholders.length > 0) {
190
+ const keepIdx = scenario.examples.header
191
+ .map((col, i) => (placeholders.includes(col) ? i : -1))
192
+ .filter((i) => i !== -1)
193
+ if (keepIdx.length > 0) {
194
+ // One row is one case: judge scores a single invoke decision per invocation, so an outline
195
+ // is extracted a row at a time rather than handed over whole.
196
+ const allRows = scenario.examples.rows
197
+ if (row !== undefined && (row < 0 || row >= allRows.length)) {
198
+ throw new RowOutOfRangeError(
199
+ `${fileLabel}: row ${row} is outside the Examples table (${allRows.length} rows): "${scenarioName}"`,
200
+ )
201
+ }
202
+ const picked = row === undefined ? allRows : [allRows[row]]
203
+ examples = {
204
+ header: keepIdx.map((i) => scenario.examples!.header[i]),
205
+ rows: picked.map((r) => keepIdx.map((i) => r[i])),
206
+ }
207
+ }
208
+ }
209
+
210
+ return { steps, given, when, placeholders, examples }
211
+ }
212
+
213
+ // ─── format ───────────────────────────────────────────────────────────────────
214
+
215
+ function formatTable(table: { header: string[]; rows: string[][] }): string {
216
+ const line = (cells: string[]) => `| ${cells.join(' | ')} |`
217
+ return [line(table.header), ...table.rows.map(line)].join('\n')
218
+ }
219
+
220
+ export function formatMarkdown(situation: Situation): string {
221
+ const lines: string[] = ['## Situation', '']
222
+ for (const step of situation.steps) lines.push(step)
223
+ if (situation.examples) {
224
+ lines.push('', 'Examples:', formatTable(situation.examples))
225
+ }
226
+ return `${lines.join('\n')}\n`
227
+ }
228
+
229
+ export function formatJson(situation: Situation): string {
230
+ return `${JSON.stringify(situation, null, 2)}\n`
231
+ }
232
+
233
+ // ─── CLI ──────────────────────────────────────────────────────────────────────
234
+
235
+ function flag(argv: string[], name: string): string | undefined {
236
+ const i = argv.indexOf(name)
237
+ return i === -1 ? undefined : argv[i + 1]
238
+ }
239
+
240
+ export function main(argv: string[]): number {
241
+ const featurePath = flag(argv, '--feature')
242
+ const scenarioName = flag(argv, '--scenario')
243
+ const rowRaw = flag(argv, '--row')
244
+ const format = flag(argv, '--format') ?? (argv.includes('--json') ? 'json' : 'text')
245
+
246
+ if (!featurePath || !scenarioName) {
247
+ console.error('✗ usage: extract-situation.mts --feature <path> --scenario "<name>" [--row <n>] [--format json]')
248
+ return 1
249
+ }
250
+
251
+ let row: number | undefined
252
+ if (rowRaw !== undefined) {
253
+ row = Number(rowRaw)
254
+ if (!Number.isInteger(row) || row < 0) {
255
+ console.error(`✗ --row must be a non-negative integer, got "${rowRaw}"`)
256
+ return 1
257
+ }
258
+ }
259
+
260
+ let text: string
261
+ try {
262
+ text = readFileSync(featurePath, 'utf8')
263
+ } catch {
264
+ console.error(`✗ ${featurePath}: cannot read file`)
265
+ return 1
266
+ }
267
+
268
+ try {
269
+ const situation = extractSituation(text, scenarioName, featurePath, row)
270
+ process.stdout.write(format === 'json' ? formatJson(situation) : formatMarkdown(situation))
271
+ return 0
272
+ } catch (err) {
273
+ console.error(`✗ ${(err as Error).message}`)
274
+ return 1
275
+ }
276
+ }
277
+
278
+ // `import.meta.main` is Node >=24.2, but this repo's engines floor is >=22, where it is `undefined`
279
+ // — the CLI would never run, printing nothing and exiting 0. A caller keying its BLOCKER on a
280
+ // non-zero exit would read that empty brief as a success and simulate from nothing.
281
+ //
282
+ // `pathToFileURL`, not `file://${process.argv[1]}`: `import.meta.url` percent-encodes, so the naive
283
+ // concat mismatches on any path holding a space (or #, ?, %) and silently reproduces that same
284
+ // never-fires bug — the trigger merely moves from a Node version to an install path.
285
+ if (process.argv[1] && import.meta.url === pathToFileURL(realpathSync(process.argv[1])).href) {
286
+ process.exit(main(process.argv.slice(2)))
287
+ }
@@ -0,0 +1,25 @@
1
+ # improve
2
+
3
+ Improve an existing agent configuration of any kind — a skill, subagent, command, or AGENTS.md section — whether by diagnosing failing ACED evals or by general review against fit and quality bars.
4
+
5
+ ## When to use
6
+
7
+ Use this skill when a config isn't working as well as it should and you want concrete proposed edits, not just a diagnosis.
8
+
9
+ Good triggers include:
10
+
11
+ - "Improve this skill" / "Make this agent better"
12
+ - "Why does this config keep failing?"
13
+ - Reviewing a config that has no eval suite yet, before deciding whether it's worth building one
14
+
15
+ Defer to a sibling when the request is really about scaffolding something new (`define-skill` / `define-agent` / `define-governance`), just scoring (`run`), just adding a case (`add-scenario`), just diffing two versions (`compare`), or auditing a `SKILL.md`'s structure specifically (`improve-skill`).
16
+
17
+ ## What it does
18
+
19
+ If the target is ACED-tracked (has an `eval.md`), it loads the internal impl-producer loop to classify failing cases by pattern and propose before/after edits, then verifies with `compare`. If not yet tracked, it runs a general review — fit classification plus the relevant builder bar — and proposes edits for any gap found, offering to hand off to `sdd:start-mission` to formalize an eval suite.
20
+
21
+ ## Install
22
+
23
+ ```bash
24
+ npx skills add cyberuni/cyberplace --skill aced/improve
25
+ ```