@claude-flow/cli 3.16.3 → 3.18.0

package/plugins/ruflo-metaharness/skills/harness-evolve/SKILL.md

@@ -1,7 +1,7 @@
 ---
 name: harness-evolve
 description: Run `@metaharness/darwin evolve <repo>` to mutate a harness's seven policy surfaces (planner/contextBuilder/reviewer/retryPolicy/toolPolicy/memoryPolicy/scorePolicy), sandbox-score each variant, and promote only measured wins. The model is frozen; the harness evolves. Closes the loop ADR-150 opens (score+genome describe; evolve changes). Degrades gracefully when @metaharness/darwin is absent (ADR-150 + ADR-153 architectural constraints).
-argument-hint: "--repo <path> [--generations 3] [--children 3] [--concurrency 2] [--sandbox real|mock|agent] [--selection pareto|quality-diversity|...] [--mutator deterministic|ruvllm] [--confirm]"
+argument-hint: "--repo <path> [--generations 3] [--children 3] [--concurrency 2] [--sandbox real|mock|agent] [--selection pareto|quality-diversity|...] [--mutator deterministic|ruvllm] [--diagnose] [--confirm]"
 allowed-tools: Bash
 ---
 
@@ -35,7 +35,7 @@ Implementation: [`scripts/evolve.mjs`](../../scripts/evolve.mjs).
    ≤ 20, `--concurrency` ≤ 8, sandbox/selection/mutator are known values).
 2. Without `--confirm`: print plan + exit 0 (mirrors `harness-mint` safety
    convention; defense in depth over the upstream `safety.ts` checks).
-3. With `--confirm`: shell to `npx -y @metaharness/darwin@~0.3.1 metaharness-darwin evolve <repo> ...`
+3. With `--confirm`: shell to `npx -y @metaharness/darwin@~0.8.0 metaharness-darwin evolve <repo> ...`
    via the shared `_darwin.mjs` async helper. Per-generation progress is
    forwarded to stderr; final champion JSON is captured from stdout.
 4. Compute timeout from `generations × children × per-variant` (per-variant
@@ -73,7 +73,46 @@ Reports land under `<repo>/.metaharness/`:
   reports/winner.json  # final champion + score delta vs parent
 ```
 
-Skill stdout = JSON `{success, data: {champion, plan, durationMs, improved}}`.
+Skill stdout = JSON `{success, data: {champion, plan, durationMs, improved}}`
+(plus `data.diagnosis` when `--diagnose` is passed — see below).
+
+## Failure diagnosis (`--diagnose`)
+
+GEPA's key trick is natural-language failure diagnosis from execution traces
+feeding the next mutation — not just scalar fitness. `--diagnose` adds a
+modest slice of that: after the evolution completes, the losing / failed
+variants' transcripts are run through darwin's GEPA library ops
+(`analyzeTranscript` + `classifyFailure`, via the shared `importGepa`
+resolver in `scripts/_darwin.mjs`) and a `diagnosis` section is appended to
+the emitted JSON:
+
+```json
+"diagnosis": {
+  "available": true,
+  "scope": "losing-variants",
+  "variants": [
+    { "id": "g1_v0", "transcripts": 2,
+      "failureClasses": { "exploration-loop": 1, "edit-mechanics": 1 },
+      "dominantClass": "exploration-loop" }
+  ],
+  "totals": { "exploration-loop": 1, "edit-mechanics": 1 }
+}
+```
+
+Upstream shape caveats (verified against `@metaharness/darwin@0.8.0`):
+
+- `metaharness-darwin evolve --json` prints a TEXT leaderboard — the stdout
+  carries no JSON and no transcripts. Per-variant run records live at
+  `<repo>/.metaharness/runs/<id>.json`.
+- Those run records hold sandbox exec traces (`{taskId, exitCode, stdout,
+  stderr}`), which are NOT GEPA `{actionRaw, obs}` transcripts. Diagnosis
+  therefore uses GEPA-shaped transcripts when a run record embeds them
+  (agent sandbox / future upstream), falls back to the champion's transcript,
+  and otherwise emits `diagnosis: {available: false, reason, traceSummary}`
+  where `traceSummary` is a mechanical per-variant tally (tasks / failed /
+  timedOut / blockedActions).
+- `--diagnose` NEVER fails the run — any internal error degrades to
+  `{available: false, reason: "diagnosis-failed: ..."}`.
 
 ## Exit codes
 

package/plugins/ruflo-metaharness/skills/harness-gepa/SKILL.md

@@ -0,0 +1,65 @@
+---
+name: harness-gepa
+description: Inspect and audit GEPA genomes via the `@metaharness/darwin/gepa` library entry (darwin 0.8.0) — load/validate a genome (default: the shipped cand-6 promotion), render the system prompt a genome compiles to, or classify failure modes in a run transcript. The `gepaOptimize` loop itself is library-only (bring your own evaluator) and not surfaced here — use `harness-evolve` for sandbox-scored evolution. Degrades gracefully when @metaharness/darwin is absent.
+argument-hint: "--op genome|validate|render|analyze [--path <genome.json>] [--transcript <t.json>] [--alert-on-invalid]"
+allowed-tools: Bash
+---
+
+Surfaces the GEPA (genetic-evolution prompt-adaptation) *library* exports
+from `@metaharness/darwin/gepa`. Unlike the other skills in this plugin
+there is no CLI binary behind this — the script dynamic-imports the library
+(local resolution first, versioned cache install as fallback) and calls the
+subprocess-safe subset.
+
+## When to use
+
+- **Adopting an evolved policy**: `--op render` shows the actual system
+  prompt a genome compiles to — read THAT, not the raw JSON, before
+  wiring a genome into a harness.
+- **Auditing a promotion**: `--op genome` loads + validates the shipped
+  cand-6 genome (first holdout-confirmed cheap-tier promotion; provenance
+  ships in the package) or any genome file you point at.
+- **CI gate on genome edits**: `--op validate --alert-on-invalid` exits 1
+  on structural errors.
+- **Debugging a bad run**: `--op analyze --transcript run.json` classifies
+  failure modes (GEPA's failure-class taxonomy) from a transcript array.
+
+## What is deliberately NOT here
+
+`gepaOptimize` — the optimization loop takes an in-process
+`evaluate(candidate)` callback ("bring your own evaluator") that cannot
+cross a subprocess boundary. Two supported paths instead:
+
+1. **Library consumers**: `import { gepaOptimize, loadCand6Genome } from '@metaharness/darwin/gepa'`
+2. **Sandbox-scored evolution**: `harness-evolve` (darwin CLI `evolve`),
+   which pairs GEPA with its own sandbox evaluators.
+
+## Algorithm
+
+Implementation: [`scripts/gepa.mjs`](../../scripts/gepa.mjs).
+
+1. `import('@metaharness/darwin/gepa')`; on MODULE_NOT_FOUND fall back to a
+   one-time `npm install --prefix ~/.ruflo/darwin-cache-0.8.0` and import
+   the cached `dist/gepa/index.js` (versioned dir → pin bumps invalidate).
+2. Dispatch `--op`:
+   - `genome`  → `loadGenome(fs, path)` or `loadCand6Genome()` + `validateGenome`
+   - `validate` → `validateGenome(rawJson)` (raw parse so broken files reach
+     the validator instead of throwing in the loader)
+   - `render`  → `buildSystemFromGenome(genome, ext?, glob?)`
+   - `analyze` → `analyzeTranscript(entries)`
+3. Emit one JSON object; exit 0 (or 1 under `--alert-on-invalid`, 2 on bad input).
+
+## Examples
+
+```bash
+node scripts/gepa.mjs --op genome                          # cand-6 + validation
+node scripts/gepa.mjs --op render | jq -r .system         # what does cand-6 SAY?
+node scripts/gepa.mjs --op validate --path my-genome.json --alert-on-invalid
+node scripts/gepa.mjs --op analyze --transcript run.json
+```
+
+## Exit codes
+
+- `0` — op completed (or degraded — darwin not installable)
+- `1` — `--alert-on-invalid` and validation found errors
+- `2` — config error (unknown op, missing/broken input file)

package/plugins/ruflo-metaharness/skills/harness-learn/SKILL.md

@@ -0,0 +1,65 @@
+---
+name: harness-learn
+description: Run a GEPA learning cycle via `metaharness learn` (upstream ADR-235, metaharness@0.3.0) — optimizes a harness genome against a SWE-bench-style slice manifest. $0 dry-run by default; `--run` is the explicit spend opt-in. Requires a metaharness repo checkout (`--repo` or $METAHARNESS_REPO) — without one it reports `checkout-required` with clone instructions. Degrades gracefully when metaharness is absent.
+argument-hint: "--host <h> --model <m> --slice <manifest> [--repo <checkout>] [--run] [--alert-on-fail]"
+allowed-tools: Bash
+---
+
+Surfaces `metaharness learn` — the upstream GEPA learning harness that
+evolves harness policy genomes against a scored task corpus instead of
+hand-editing prompts. Candidates are scored on held-out slices and only
+measured winners promote (the shipped cand-6 genome is the first such
+promotion: holdout gold 2/12 → 3/12, zero regressions).
+
+## When to use
+
+- A harness's policy prompt underperforms on a task family and you want a
+  measured improvement loop rather than manual prompt iteration.
+- Pricing a learning run before committing spend — the default dry-run
+  resolves the slice manifest and reports cost without any model calls.
+- After a learn run promotes a genome: pair with `harness-gepa --op render`
+  to inspect what the promoted policy actually says.
+
+## Preconditions (upstream design)
+
+The learning harness (GEPA + SWE-bench + Docker) is too heavy for the npm
+package, so `learn` needs a local clone:
+
+```bash
+git clone https://github.com/ruvnet/metaharness.git
+node scripts/learn.mjs --repo ./metaharness --host claude-code --model haiku --slice slices/lite.json
+```
+
+Without a checkout the script emits `{status: "checkout-required"}` and
+exits 0 — a precondition report, not an error (distinct from
+`degraded: true`, which means the npm package itself is absent). The
+managed-service path (gateway-side learn jobs, no checkout) is upstream's
+ADR-235 follow-up and not available yet.
+
+## Algorithm
+
+Implementation: [`scripts/learn.mjs`](../../scripts/learn.mjs).
+
+1. Validate `--repo` exists when given; export it as `$METAHARNESS_REPO`.
+2. Invoke the pinned `metaharness` binary (`metaharness@~0.3.0`, local install
+   or one-time versioned cache — never `@latest`): `metaharness learn --host <h>
+   --model <m> --slice <s> [--run]` via `_harness.mjs` (graceful degradation,
+   hard timeout).
+3. Default timeouts: 120s dry-run, 600s with `--run` — real runs on larger
+   slices need an explicit `--timeout-ms` matched to slice size × model cost.
+4. Detect the checkout-required message → structured payload, exit 0.
+5. Parse the trailing JSON report when upstream emits one; otherwise return
+   the raw report text under `rawReport`.
+
+## Cost note
+
+`--run` is the ONLY path that spends. Everything else — dry-run, checkout
+probe, degraded path — is $0. The MCP tool (`metaharness_learn`) has a 120s
+subprocess budget; run real learning cycles from a terminal via
+`ruflo metaharness learn ... --run --timeout-ms <big>`.
+
+## Exit codes
+
+- `0` — report produced (or dry-run, checkout-required, degraded)
+- `1` — `--alert-on-fail` and the learn run reported failure
+- `2` — config error (bad `--repo` path)

package/plugins/ruflo-metaharness/skills/harness-mcp-scan/SKILL.md

@@ -13,7 +13,9 @@ tool; pure static analysis.
 
 Implementation: [`scripts/mcp-scan.mjs`](../../scripts/mcp-scan.mjs).
 
-1. Shell `npx -p metaharness@latest harness mcp-scan <path> --json`.
+1. Invoke the pinned `harness` binary (`metaharness@~0.3.0`, resolved from a
+   local install or the one-time `~/.ruflo/metaharness-cache-<pin>` cache —
+   never `@latest`): `harness mcp-scan <path> --json`.
 2. Parse `findings[]` with `{ severity, id, server, tool, message }`.
 3. `--fail-on <severity>`: exit 1 when any finding is at or above that
    level. Default `high`.

package/plugins/ruflo-metaharness/skills/harness-score/SKILL.md

@@ -58,7 +58,7 @@ no network, registry unreachable), the script emits:
 {
   "degraded": true,
   "reason": "metaharness-not-available",
-  "hint": "Install metaharness manually with `npm i -D metaharness` or run `npx metaharness@latest --version` to verify network access."
+  "hint": "Install with `npm i -D metaharness@~0.3.0` (pinned range — this plugin never fetches @latest) or verify network access for the one-time cache install."
 }
 ```
 

package/plugins/ruflo-metaharness/skills/harness-security-bench/SKILL.md

@@ -29,7 +29,7 @@ ground-truth set. Running this nightly gives us:
 
 Implementation: [`scripts/security-bench.mjs`](../../scripts/security-bench.mjs).
 
-1. Shell to `npx -y @metaharness/darwin@~0.3.1 metaharness-darwin security bench --population N --cycles N [--seed S]`.
+1. Shell to `npx -y @metaharness/darwin@~0.8.0 metaharness-darwin security bench --population N --cycles N [--seed S]`.
 2. Default timeout = `3s × 19 evaluations × population × cycles + 30s overhead`.
    At default `--population 2 --cycles 1` ≈ 144s; at `--population 4 --cycles 3` ≈ 12 min.
 3. Parse the markdown report — overall PASS/FAIL plus per-gate

package/plugins/ruflo-metaharness/skills/harness-threat-model/SKILL.md

@@ -13,7 +13,9 @@ categorized report suitable for sharing with an InfoSec team.
 
 Implementation: [`scripts/threat-model.mjs`](../../scripts/threat-model.mjs).
 
-1. Shell `npx -p metaharness@latest harness threat-model <path> --json`.
+1. Invoke the pinned `harness` binary (`metaharness@~0.3.0`, resolved from a
+   local install or the one-time `~/.ruflo/metaharness-cache-<pin>` cache —
+   never `@latest`): `harness threat-model <path> --json`.
 2. Parse `{ worst, findings[] }`.
 3. `--fail-on <severity>`: exit 1 when `worst >= fail-on`. Default `high`.