cyber-aced 0.3.0 → 0.4.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -1,7 +1,7 @@
1
1
  {
2
2
  "$schema": "https://json.schemastore.org/claude-code-plugin-manifest.json",
3
3
  "name": "aced",
4
- "version": "0.3.0",
4
+ "version": "0.4.1",
5
5
  "description": "Agent Config Evaluation & Development. Spec, evaluate, and improve agent configurations: skills, AGENTS.md sections, subagent definitions, and commands.",
6
6
  "author": {
7
7
  "name": "unional"
@@ -0,0 +1,18 @@
1
+ {
2
+ "$schema": "https://raw.githubusercontent.com/cyberuni/marketplace/main/schema/claude/plugin.json",
3
+ "name": "aced",
4
+ "version": "0.4.1",
5
+ "description": "Agent Config Evaluation & Development. Spec, evaluate, and improve agent configurations: skills, AGENTS.md sections, subagent definitions, and commands.",
6
+ "author": {
7
+ "name": "unional"
8
+ },
9
+ "homepage": "https://github.com/cyberuni/cyber-sdd",
10
+ "repository": "https://github.com/cyberuni/cyber-sdd",
11
+ "skills": "./skills",
12
+ "agents": [
13
+ "./agents/aced-case-judge.md",
14
+ "./agents/aced-impl-judge.md",
15
+ "./agents/aced-scenario-writer.md",
16
+ "./agents/aced-spec-validator.md"
17
+ ]
18
+ }
@@ -1,7 +1,7 @@
1
1
  {
2
2
  "$schema": "https://raw.githubusercontent.com/cyberuni/marketplace/main/schema/claude/plugin.json",
3
3
  "name": "aced",
4
- "version": "0.3.0",
4
+ "version": "0.4.1",
5
5
  "description": "Agent Config Evaluation & Development. Spec, evaluate, and improve agent configurations: skills, AGENTS.md sections, subagent definitions, and commands.",
6
6
  "author": {
7
7
  "name": "unional"
@@ -19,7 +19,7 @@ The **impl-judge** for agent-configuration domain types — it grades the **buil
19
19
  ## Input
20
20
 
21
21
  ```
22
- DOMAIN, DOMAIN_PATH, SPEC_PATH, FEATURE_PATH
22
+ DOMAIN, DOMAIN_PATH, SPEC_PATH, FEATURE_PATH: the domain under work, its spec folder, its spec.md, and its .feature
23
23
  IMPLEMENTATION_PATHS: the agent configuration under evaluation (the SUBJECT)
24
24
  VERIFICATION_PATHS: the frozen .feature (the eval source) + eval.md (run policy only)
25
25
  ```
@@ -30,7 +30,7 @@ VERIFICATION_PATHS: the frozen .feature (the eval source) + eval.md (run poli
30
30
 
31
31
  2. **Set the run policy from `eval.md`.** Read the run policy under the `eval:` block: `eval.trigger.{runs, activation_threshold}` for `@trigger` scenarios; `eval.judge.model` and `eval.judge.default_threshold` for `@behavior`/`@quality` scenarios (an inline `threshold` in a `@rubric` docstring overrides the default). Defaults when `eval.md` omits them: judge model `claude-sonnet-4-6`, default_threshold 4, trigger activation_threshold 0.5, trigger runs 3.
32
32
 
33
- 3. **Run the evals.** For each scenario, invoke `aced-case-judge` with the SUBJECT, the **`.feature` path and the scenario name**, and its threshold, over its run count — for a `@trigger` `Scenario Outline`, **once per `Examples` row**, passing its zero-based `ROW` (one row is one case; the judge reports one invoke decision, and aggregating accuracy across rows is yours). **Never pass the scenario's steps, its `Then`, or its rubric** — the judge blinds its own simulating context (composing that context's brief with the `extract-situation` engine) and scores the returned transcript in a separate one; handing it the scenario body would put the answer key back in the context that has to reach the answer. One invocation covers both passes; never sequence them here. It returns a score per named dimension against that dimension's own `max`, plus a total. Aggregate the totals across runs.
33
+ 3. **Run the evals.** For each scenario, invoke `aced-case-judge` with the SUBJECT, the **`.feature` path and the scenario name**, and its threshold, over its run count — for a `@trigger` `Scenario Outline`, **once per `Examples` row**, passing its zero-based `ROW` (one row is one case; the judge reports one invoke decision, and aggregating accuracy across rows is yours). **Never pass the scenario's steps, its `Then`, or its rubric** — the judge blinds its own simulating context (composing that context's brief with the `extract-situation` engine) and scores the returned transcript in a separate one; handing it the scenario body would put the answer key back in the context that has to reach the answer. One invocation covers both passes; never sequence them here. It returns a score per named dimension against that dimension's own `max`, plus a total. Aggregate the totals across runs. <!-- field-mandate-ignore: ROW is aced-case-judge's input, which this judge constructs -->
34
34
 
35
35
  **Wait for each case judge, and bank only its last word.** Dispatching a case judge is asynchronous, and the case judge is itself waiting on its own blind simulator — so a judge can report before its measurement exists and correct itself afterwards. Two rules follow. **Only the final notification per agent id is trustworthy**: when an agent reports more than once, the later report supersedes the earlier one entirely — never bank the first and never merge them. And **a verdict is a result only when the agent has finished**: do not collapse to boolean or roll up `IMPLEMENTATION_PASS` while any dispatched case judge is still live. If a run ends with cases still outstanding, that is `STATUS: needs-input` with those scenarios named — not a pass and not a fail.
36
36
 
@@ -30,7 +30,7 @@ Before writing anything, classify the subject's fit tier (`aced:aced-fit`) and *
30
30
  ## Input
31
31
 
32
32
  ```
33
- DOMAIN, DOMAIN_PATH, SPEC_PATH
33
+ DOMAIN, DOMAIN_PATH, SPEC_PATH: the domain under work, its spec folder, and its spec.md
34
34
  SUBJECT: <full text of the agent configuration under spec, or null for a new one>
35
35
  COMMAND_SURFACE: <the configuration's trigger surface / interface — or null>
36
36
  DESIGN_DECISIONS: <known choices — or null>
@@ -24,7 +24,7 @@ The per-scenario **Checks** below are the `aced:aced-builder-spec` bar (its cano
24
24
  ## Input
25
25
 
26
26
  ```
27
- DOMAIN, DOMAIN_PATH, FEATURE_PATH, SPEC_PATH
27
+ DOMAIN, DOMAIN_PATH, FEATURE_PATH, SPEC_PATH: the domain under work, its spec folder, its .feature, and its spec.md
28
28
  SUBJECT: <full text of the agent configuration under spec, or null>
29
29
  ```
30
30