@garygentry/feature-forge 0.2.13 → 0.3.0

package/adapters/pi/references/process-overview.md

@@ -0,0 +1,143 @@
+# Feature Forge Pipeline Overview
+
+This document describes the end-to-end feature development pipeline managed by the feature-forge plugin. All forge skills reference this document to understand the overall flow and their position within it.
+
+The pipeline compiles a fuzzy feature idea into a machine-executable `backlog.json`. Each stage narrows the idea down and adds structure, verification gates catch gaps before they reach later stages, and a swappable autonomous loop runner (rauf by default) implements the backlog in fresh per-item sessions.
+
+## Pipeline Stages
+
+```
+[forge-0-epic] → forge-1-prd → forge-2-tech → forge-3-specs → forge-verify → forge-4-backlog → forge-verify → forge-5-loop → forge-verify → forge-6-docs
+   (optional)
+```
+
+### Stage 0: Epic (`/skill:forge-0-epic <epic>`), optional
+**Input:** A change too large for one feature
+**Output:** `{specsDir}/{epic}/epic-manifest.json` + `EPIC.md` + one member-feature dir per feature
+**Method:** Decomposition interview splitting the change into member features with declared dependencies and `exposes`/`consumes` contracts. Purely additive: single-feature flows are unchanged. See [docs/architecture/epic-orchestration/README.md](../docs/architecture/epic-orchestration/README.md).
+
+### Stage 1: PRD (`/skill:forge-1-prd <feature>`)
+**Input:** User's feature idea and domain knowledge
+**Output:** `{specsDir}/{feature}/PRD.md`
+**Method:** Structured interview focused exclusively on requirements. No technology decisions.
+
+### Stage 2: Tech Spec (`/skill:forge-2-tech <feature>`)
+**Input:** PRD.md
+**Output:** `{specsDir}/{feature}/tech-spec.md`
+**Method:** Structured interview for technology decisions, grounded in PRD requirements.
+
+### Stage 3: Implementation Specs (`/skill:forge-3-specs <feature>`)
+**Input:** PRD.md + tech-spec.md
+**Output:** `{specsDir}/{feature}/##-<name>.md` (suite of numbered documents)
+**Method:** Generate detailed implementation documents from spec archetypes.
+
+### Verification Gate (`/skill:forge-verify <feature>`)
+**Input:** All artifacts from current and prior stages
+**Output:** `{specsDir}/{feature}/.verification/VERIFY-<stage>-<timestamp>.md` (includes both findings and a Fix Execution Plan)
+**Method:** Clean-context analysis producing actionable findings with an ordered fix plan.
+
+**Manual or automatic.** By default the navigator offers verification after each stage. Because
+it runs in a fresh, read-only subagent (clean-room by construction, never needs a `/clear`), it is
+safe to automate: set `autoVerify: true` (or per-stage via `autoVerifyStages`) in
+`forge.config.json` and the navigator runs `forge-verify` automatically once a stage completes,
+returning only a compact digest to the session. The freshness ledger (verify entries record the
+artifact `version` they ran against) means a stage is re-verified only when its artifact changes,
+and an explicitly `skipped` verify is respected. Fixing stays human-gated unless `autoFix: true` is
+also set, which chains `forge-fix` only when preconditions hold (zero unresolved decisions, clean
+tree, passing re-verify). See `references/shared-conventions.md` for the config keys.
+
+After verification, fixes can be applied via:
+- `/skill:forge-fix <feature>` — reads the Fix Execution Plan from the findings document and applies changes (works in any session)
+- Plan mode workflow — enter plan mode, run verify, review plan, exit and execute
+- Manual — read findings and apply fixes by hand
+
+### Stage 4: Backlog (`/skill:forge-4-backlog <feature>`)
+**Input:** Full spec suite
+**Output:** `{specsDir}/{feature}/backlog.json` (or `{backlogDir}/{feature}/backlog.json` if backlogDir is configured)
+**Method:** Generate structured backlog items with spec references, acceptance criteria, and dependencies. Backlog is collocated with feature specs by default.
+
+### Stage 5: Rauf Loop (`/skill:forge-5-loop <feature>`)
+**Input:** `backlog.json` from Stage 4
+**Output:** Implemented source code (committed per-item by rauf)
+**Method:** Execute the autonomous coding loop against the feature's backlog. Spawns a fresh session per backlog item with full spec context. rauf is the default runner but is swappable via the `loopRunner` block in `forge.config.json`; see [`references/ralph-loop-contract.md`](./ralph-loop-contract.md).
+
+### Stage 6: Documentation (`/skill:forge-6-docs <feature>`)
+**Input:** Specs + implementation
+**Output:** `{docsDir}/{feature}/` documentation suite
+**Method:** Generate developer-focused architecture documentation.
+
+## Pipeline State
+
+State is tracked in `{specsDir}/{feature}/.pipeline-state.json` and persists across session clears. The `/skill:forge <feature>` navigator reads this file to show current progress.
+
+## Git Discipline
+
+Every stage completion should be followed by a git commit:
+- Commit message format: `{commitPrefix}({feature}): <action>`
+- Examples: `forge(auth): complete PRD v1`, `forge(auth): apply spec verification fixes`
+- Before applying verification fixes, commit current state first (so pre-fix state is recoverable)
+- After applying fixes, commit again
+
+## Configuration
+
+Read `forge.config.json` from project root for path overrides. See `references/forge-config-schema.json` for the full schema. If no config file exists, use defaults:
+- specsDir: `./specs`
+- docsDir: `./docs/architecture`
+- backlogDir: (null — backlog defaults to {specsDir}/{feature}/backlog.json)
+- gitCommitAfterStage: `true`
+- commitPrefix: `forge`
+
+## Git Workflow
+
+Recommended: forge work lives on an isolated `forge/{feature}` (or `forge/{epic}`) branch so all commits land together and review as one branch. The entry stages (`forge-1-prd`, `forge-0-epic`) detect whether you're on the default branch (main/master) and **strongly recommend** creating that branch when you are — still letting you decline and stay. The chosen branch is recorded in `.pipeline-state.json`, and `forge-5-loop` re-checks it before the autonomous loop starts committing per item. After implementation, merge to your development branch.
+
+To customize: `branchPrefix` (default `forge/`) sets the branch name, and `branchPerFeature: false` disables the prompt entirely (forge then works on whatever branch is checked out). Branch Setup is gated only on the project using git — independent of `gitCommitAfterStage`.
+
+If you prefer manual commit control, set `gitCommitAfterStage: false` in `forge.config.json`.
+
+## Cross-Cutting Concerns
+
+### Feature Name is Required
+Every forge skill requires a feature name as the first argument. If not provided, STOP and ask. Never guess.
+
+### Reference Existing Code
+When creating specs, always examine the existing codebase for patterns, conventions, and integration points. Check:
+- Other feature specs in `{specsDir}/`
+- Existing documentation in `{docsDir}/`
+- Module/package structure and exports in the project
+- Shared types, utilities, and conventions
+
+### Stack Context
+The project's stack is detected during forge-2-tech and persisted in `forge.config.json` (the `stack`, `typeCheckCommand`, and `testCommand` fields). See `references/stack-resolution.md` for the full resolution protocol. The project may also have a `stack-decisions.md` (preferred at `.feature-forge/`; legacy `.claude/references/`) with established technology decisions — if present, it takes highest precedence.
+
+## Subagents
+
+The plugin includes three specialized subagents in `agents/` that enhance specific
+pipeline steps. They use **model aliases** (`opus`/`sonnet`) rather than pinned IDs, so
+they track the current model tier automatically.
+
+### forge-verifier
+- **Purpose:** Read-only verification of pipeline artifacts
+- **Used by:** `forge-verify` skill (delegation via Agent tool)
+- **Tools:** Read, Glob, Grep, Bash (read-only operations only)
+- **Model:** Opus (judgement-heavy gap/inconsistency analysis)
+- **Memory:** Project-scoped persistent memory — accumulates knowledge about recurring issues and project-specific patterns across sessions
+- **Parallel:** For large modes (specs/backlog/impl), `forge-verify` dispatches several instances in parallel, one per **dimension group** (e.g. types/contracts, traceability, testing), and the parent merges their findings. An opt-in adversarial "deep verify" pass re-checks high-severity findings with a skeptic instance to cut false positives. A purely mechanical dimension (e.g. traceability validation) could run on a cheaper tier (Haiku) if cost matters.
+- **Why a subagent:** Verification reads the entire spec suite, backlog, and potentially source code. Running this in a separate context window prevents context pressure on the main conversation. The read-only tool restriction also makes it impossible to accidentally modify specs during verification.
+
+### forge-researcher
+- **Purpose:** Codebase exploration and integration mapping
+- **Used by:** `forge-2-tech` skill (spawned before the tech-spec interview)
+- **Tools:** Read, Glob, Grep, Bash (read-only)
+- **Model:** Sonnet (cost-efficient for exploration tasks)
+- **Parallel:** For a large codebase or uncertain scope, `forge-2-tech` may dispatch several researchers in parallel, each with a disjoint focus (structure/conventions, per-subsystem integration surfaces), and merge the reports.
+- **Why a subagent:** Tech-spec planning requires reading many files across the project to understand integration points. Running this in a separate context returns a concise report without consuming the main session's context, keeping the interview focused.
+
+### forge-spec-writer
+- **Purpose:** Author exactly one numbered implementation spec document to the forge-3-specs quality bar
+- **Used by:** `forge-3-specs` skill (parallel fan-out after the shared foundation docs are written)
+- **Tools:** Read, Glob, Grep, Bash, **Write** (the only authoring agent — constrained to write its single assigned file)
+- **Model:** Opus (spec authoring is detail- and judgement-heavy)
+- **Why a subagent:** Authoring the whole suite in the main session serializes the work and pressures context. Writing the foundation (00/01) first, then fanning out one writer per remaining doc, parallelizes authoring while the parent keeps the cross-reference + traceability finish in view.
+
+All three subagents are optional. If the agents are not installed or the environment doesn't support subagents, the corresponding skills fall back to running inline (or, for spec authoring, batched in the main session).

package/adapters/pi/references/ralph-loop-contract.md

@@ -0,0 +1,221 @@
+# The Loop-Runner Contract (consumer side)
+
+feature-forge's pipeline ends by handing a `backlog.json` to an autonomous
+**loop runner** that implements each item. feature-forge does not embed any
+runner's internals — it talks to the runner through one indirection point: the
+**`loopRunner`** block in `forge.config.json`.
+
+## The seam
+
+Every command feature-forge runs against the runner is a template in
+`loopRunner`, with `{bin}`, `{backlogDir}`, `{specsDir}`, and `{iterations}`
+substituted at call time. `forge-5-loop` (execution), `forge-4-backlog`
+(validation), and `forge-verify` (backlog validation) all render their commands
+from this block — there are no hardcoded `rauf …` commands in the skills, and
+even the human log filename is tokenized via `{loopRunner.logFile}`.
+
+When `forge.config.json` has no `loopRunner` block, feature-forge uses the
+built-in defaults (see `references/forge-config-schema.json`) and announces
+"defaulting to the rauf loop runner."
+
+## The contract a runner MUST satisfy
+
+A conforming runner MUST implement the **backlog-tool / loop-runner contract**
+defined authoritatively in rauf's
+[`SPEC-BACKLOG-TOOL-CONTRACT.md`](https://github.com/garygentry/rauf/blob/main/docs/SPEC-BACKLOG-TOOL-CONTRACT.md)
+(Part A). In summary, it must provide:
+
+- **A backlog schema** with the published `$id` and an optional `schemaVersion`,
+  whose `type`/`status` vocabularies match the contract
+  (`type ∈ bug|bugfix|refactor|feature|chore|test`,
+  `status ∈ pending|in_progress|done|blocked`).
+- **A `validate` verb** with exit codes `0` (valid) / `1` (findings) / `2`
+  (usage/IO) that emits `{ valid, findings[] }` under `--json`. This is the
+  single check feature-forge trusts — it never re-implements validation.
+- **The signal protocol** (`RAUF_DONE` / `RAUF_BLOCKED` / `RAUF_NEEDS_HUMAN` /
+  `RAUF_REVIEW`). These are emitted by the *coding agent* into its stdout and
+  parsed by the runner — they are not runner-authored log lines, so consumers key
+  off the runner's parsed events (below), never the raw `RAUF_*` tokens (which can
+  also appear inside an agent's prose and produce false matches).
+- **A machine-readable event stream** for live supervision (`loopRunner.eventStreamCommand`,
+  rauf: `loop run … --ndjson`): one JSON event per line with a stable `type`
+  vocabulary — `item_completed` / `item_blocked` / `needs_human` / `signal_parsed`
+  / `loop_completed` / `loop_error` / `loop_cancelled` / `llm_stuck_warning` (a
+  circuit-breaker halt surfaces as `loop_error`) — plus a
+  derived-status JSON (`loopRunner.statusJsonCommand`, rauf: `status … --json`) and
+  per-iteration telemetry with a `stuckWarning` flag (`loopRunner.watchCommand`,
+  rauf: `status … --json` — the `loop watch` verb was removed in v0.5.0). `forge-5-loop` supervises the run through these,
+  **not** by parsing the human log. `followCommand` / `logCommand` are
+  human-formatted streams for a person watching in a terminal, not machine surfaces.
+- **The state-dir layout** (per-`--backlog` isolation under `loopRunner.stateDir`).
+- **The CLI verbs** mapped by `loopRunner`: run (+ event-stream) / validate /
+  status (+ `--json`) / list / follow / log / version.
+- **A `version` verb** (`{bin} version --json` → `{ version: <semver> }`) so
+  feature-forge can enforce `loopRunner.minRunnerVersion` before running.
+
+> **The runner does not pause for human input.** When the coding agent signals
+> `RAUF_NEEDS_HUMAN`/`RAUF_BLOCKED`/`RAUF_REVIEW`, a conforming runner sets that
+> item aside and **keeps working other runnable items to completion** (rauf:
+> `runner.ts` needs_human handler). So a supervising session can surface those
+> events live (visibility) and cancel early, but it cannot inject an answer and
+> resume the set-aside item mid-run — resolution is a follow-up retry pass. A
+> first-class pause/resume-with-answer capability is a desirable runner
+> enhancement (see `plans/rauf-enhancement-recommendations.md`).
+
+## rauf is the default and reference implementation
+
+rauf owns the contract spec and is the runner feature-forge defaults to. The
+authoring craft itself (how to decompose specs into well-scoped, verifiable
+items) lives in rauf's **`author-backlog`** skill, which `forge-4-backlog`
+delegates to — so the only thing binding feature-forge to a particular runner is
+the schema + `validate` verb that this contract formalizes. A future runner swap
+supplies its own `loopRunner` block (its own `bin`, schema, and `validate`
+command) without touching any pipeline skill.
+
+The coding-agent dimension this contract adds (below) is **additive and
+presence-gated**: it exists only when the `loopRunner` block advertises it via
+`agentArgument`. A runner that omits that field has no agent dimension at all —
+the seam degrades to exactly today's behavior with no error, so default-to-rauf
+and pluggability are unchanged. See `## Agent selection`.
+
+## Agent selection
+
+This section is **contract-level**: it states what a conforming runner exposes
+and what forge does with it, and defers every algorithm to the owning specs
+(`02`/`03`/`04`/`05`).
+
+### What a conforming runner exposes (the consumed surface)
+
+A runner that carries a coding-agent dimension exposes, in its `loopRunner`
+block:
+
+- an **`agentArgument`** template (rauf default `--agent {agent}`) — the
+  tokenized launch-time flag; its **presence** advertises the agent surface;
+- an **`agentsProbeCommand`** (rauf default `{bin} agents --json`) emitting
+  `{ agents: [{ id, displayName, available, detail? }] }` and **always exiting
+  0**;
+- an optional **`defaultAgent`** project-default id.
+
+These three fields are specified in full in `references/forge-config-schema.json`
+and are the schema half of this contract. forge consumes rauf's existing
+`--agent <id>` flag, `rauf agents` probe, `BacklogItem.provider`, and 5-layer
+precedence — it conforms to them, it does not redesign them.
+
+### Precedence and the run-layer mapping
+
+The agent-selection precedence is **`item > run > project > default`**,
+deliberately parallel to the model-selection precedence (`item.model >
+--model/options > project default > provider default`). It is realized as:
+
+- **item** — `BacklogItem.provider`, applied by **rauf** from the backlog. forge
+  **never reads, writes, or overrides** it (pass-through), so a deliberate
+  per-item agent always wins.
+- **run** — forge's per-run selector (`forge-5-loop` Step 2d).
+- **project** — forge's `loopRunner.defaultAgent`.
+- **default** — the runner's own default (`claude-cli` for rauf) when forge sends
+  nothing.
+
+forge owns **only** the run and project layers. It collapses run-over-project
+*inside itself* into the **single** `--agent {agent}` value it emits at the
+**run layer**, and lets rauf apply the item override *above* that. forge **never
+re-implements rauf's resolver**. The resolution algorithm itself lives in
+`03-selection-resolution-observability.md`.
+
+### Availability probe + unknown/unavailable disambiguation
+
+When the resolved agent is a **non-default** id, forge runs `agentsProbeCommand`
+**once (no retries)** before any loop side-effect, parses the advertised
+`agents[]`, and builds the advertised id set. Because the probe **always exits
+0**, an unknown id is distinguished from a known-but-unavailable one **only by
+set membership**, not by exit code:
+
+- **Unknown id** (not in the advertised set — a typo or unsupported agent):
+  **hard-reject before launch**, listing the valid ids. No proceed-anyway path.
+  No value is interpolated into `{agent}`.
+- **Known but unavailable** (`available: false`): **warn** (showing the probe's
+  `detail`) and let the user **proceed-anyway or choose another** — never
+  silently abort, never silently proceed.
+- **Available**: proceed.
+
+The advertised id set is also the **allow-list**: the only value ever
+interpolated into `{agent}` is a validated, advertised id. The **default /
+claude path never reaches the probe** — it incurs no extra cost. The
+`classify(...)` algorithm and the rejection-error text live in
+`04-availability-precheck.md`.
+
+### Capability gate + version floor
+
+Agent selection is **capability-gated** on the runner advertising
+`agentArgument`: a runner whose `loopRunner` omits (or empties) that field
+exposes no agent surface, so the per-run selector, the probe, and any `{agent}`
+substitution **vanish entirely** and no agent argument is sent — byte-identical
+to today. Degradation is **silent, not an error**, keeping alternate (non-rauf)
+runners first-class. The gate condition is owned by
+`02-config-schema-and-gating.md`.
+
+Independently, the **version gate** floors at the runner version that ships the
+agent surface. For rauf that is **0.6.0** (`loopRunner.minRunnerVersion`): the
+`--agent` flag, the `agents` probe, and the preset agent registry are present in
+rauf source at 0.6.0. A successful gate therefore guarantees those surfaces
+exist before any run. See `## Version gating` and
+`05-runner-discovery-version-gate.md`.
+
+**This document — the `## Agent selection` section, the `## Per-stage agent
+applicability` table, and the `## validate is agent-agnostic` note — together
+with the augmented `loopRunner` schema block in
+`references/forge-config-schema.json` constitute the `forge-loop-runner-contract`
+expose, consumed by the `packaging-docs-ci` capstone as documentation input.**
+
+## Per-stage agent applicability
+
+Every forge stage that invokes the loop runner is classified here. Only
+`forge-5-loop` (execution) carries the coding-agent dimension; the two
+validation-only stages are agent-agnostic.
+
+| Stage | Runner verbs | Agent dimension |
+|-------|-------------|-----------------|
+| `forge-5-loop` | run / eventStream / status / version | **Full** — selector, probe, `--agent` |
+| `forge-4-backlog` | `validate` | **None** — agent-agnostic |
+| `forge-verify` | `validate` | **None** — agent-agnostic |
+
+- **`forge-5-loop`** is the executor: it drives the run, so it renders the run /
+  event-stream / status / version verbs (and `list`) and carries the full agent
+  surface — the Step 2d selector, the availability probe, and the rendered
+  `agentArgument`.
+- **`forge-4-backlog`** authors and then *validates* the backlog; its only runner
+  call is `validateCommand`. It also reads `versionCommand` for a graceful-degrade
+  check, but **passes no agent** and never runs `loop run`.
+- **`forge-verify`** (backlog mode) re-runs the same `validateCommand` to surface
+  validation findings. It carries no agent dimension. This is **contract
+  coverage** of forge-verify (it is classified here), with an explicit agnostic
+  note (below) — **not** a new agent-driven run in forge-verify.
+
+### `validate` is agent-agnostic
+
+The `validate` verb (`loopRunner.validateCommand`) checks a `backlog.json`
+against the backlog schema and spec references. **It does not run a coding agent
+and has no agent dimension.** No agent argument — `--agent`, the `{agent}` token,
+or any agent id — may **ever** be passed to backlog validation, in **any** stage
+(`forge-4-backlog`, `forge-verify`, or any future caller). Backlog validation is
+a pure, deterministic check; the coding agent is irrelevant to it. A contributor
+who later adds agent selection to a *new* stage MUST confirm that stage runs the
+*execution* surface (like `forge-5-loop`), not `validate` — agent selection
+belongs to execution only. If you find yourself adding `--agent` near a
+`validateCommand` render, that is a bug.
+
+## Version gating
+
+feature-forge requires a runner exposing `backlog validate` + backlog
+`schemaVersion`, and the unified exit-code/status contract it reads. The floor is
+now the **agent-surface floor**: the runner version that ships the `--agent`
+flag, the `agents` probe, and the preset agent registry. For rauf that is
+**0.6.0** (`loopRunner.minRunnerVersion`).
+`forge-5-loop` runs `{bin} version --json`, semver-compares the reported version
+against `minRunnerVersion`, and on a missing-or-too-old runner stops with
+`loopRunner.installHint` (the CLI install/upgrade command) — **before** invoking
+the loop. See `COMPATIBILITY.md` for the version matrix.
+
+> **Two different "installs."** `installHint` obtains/upgrades the runner **CLI
+> binary** (e.g. rauf's `install-binary.sh`). `setupHint` installs the runner's
+> **per-project artifacts** (rauf: `rauf install .`). A failing version gate is
+> always the former, never the latter.