valent-pipeline 0.1.2 → 0.1.3

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "valent-pipeline",
-  "version": "0.1.2",
+  "version": "0.1.3",
   "description": "v3 multi-agent AI pipeline for software development lifecycle",
   "type": "module",
   "bin": {

package/skills/v3-configure/SKILL.md

@@ -0,0 +1,171 @@
+---
+name: v3-configure
+description: 'Configure or reconfigure the v3 pipeline. Use when the user says "configure pipeline", "setup v3", "v3 configure", or "change pipeline settings"'
+---
+
+# v3 Pipeline Configuration Wizard
+
+You are an interactive configuration wizard for the v3 story execution pipeline. Your job is to create or update `v3/pipeline-config.yaml`.
+
+## Determine Mode
+
+First, check whether `v3/pipeline-config.yaml` already exists.
+
+- **If it exists:** Read the current file, display a summary of current settings, and ask the user what they want to change. Do NOT force a full reconfiguration -- let them update individual sections.
+- **If it does not exist:** Run the full first-time setup flow below.
+
+---
+
+## First-Time Setup Flow
+
+Walk the user through each section interactively. Show smart defaults and skip irrelevant fields based on earlier answers.
+
+### Step 1: Project Type
+
+Ask the user to pick one:
+
+| Type | Description |
+|------|-------------|
+| `fullstack-web` | Backend API + frontend UI + E2E tests (all agents active) |
+| `backend-api` | API-only, no frontend (skips FEND, UXA, PMCP) |
+| `frontend-only` | UI consuming existing APIs (skips BEND) |
+| `data-pipeline` | ETL/ELT, transformations, data quality (skips FEND, UXA, PMCP) |
+| `mcp-server` | Model Context Protocol server (skips FEND, UXA, PMCP) |
+| `document-generation` | Templates, content pipelines, output validation |
+| `library` | Reusable package/module with public API (skips FEND, UXA, PMCP) |
+
+Default: `fullstack-web`
+
+### Step 2: Tech Stack
+
+Ask about each field. Use the project type to skip irrelevant questions:
+
+- **If project type has no frontend** (`backend-api`, `data-pipeline`, `mcp-server`, `library`): skip `frontend_framework`, `state_management`, and set them to `"none"`.
+- **If project type has no backend** (`frontend-only`): skip `database_orm` and set it to `"none"`.
+
+Fields to ask (in order):
+
+1. **Language** -- e.g., typescript, python, go, java, rust, csharp, ruby
+2. **Backend framework** -- e.g., nestjs, express, fastapi, django, gin, spring, hono, none
+3. **Frontend framework** -- e.g., react, vue, svelte, angular, solid, none
+4. **Database ORM** -- e.g., drizzle, prisma, typeorm, sqlalchemy, gorm, sequelize, none
+5. **Unit test framework** -- e.g., vitest, jest, pytest, go-test, junit, mocha
+6. **E2E test framework** -- e.g., playwright, agent-browser, cypress, none
+7. **Browser automation MCP** -- e.g., playwright, agent-browser, none
+8. **State management** -- e.g., tanstack-query, redux, zustand, pinia, jotai, none
+
+For each field, suggest a sensible default based on the language and project type. For example:
+- typescript + backend -> suggest vitest for unit tests
+- python -> suggest pytest for unit tests
+- No frontend -> skip browser automation MCP, default to none
+
+Present all tech stack fields together as a batch (not one-by-one) with proposed defaults, and let the user confirm or override.
+
+### Step 3: Git Configuration
+
+Ask for the target branch where story code should land.
+
+- Examples: `feature/sprint-12`, `develop`, `main`
+- Default: leave empty (pipeline will prompt at story start)
+
+### Step 4: Quality Thresholds
+
+Present defaults and let the user adjust:
+
+| Setting | Default | Description |
+|---------|---------|-------------|
+| `max_rejection_cycles` | `5` | Max times an agent can be rejected before escalating to user (range: 3-7) |
+| `retrospective_every_n_stories` | `5` | How often the Retrospective Agent runs (range: 3-10) |
+| `stall_threshold_minutes` | `15` | Minutes before a stalled agent gets a check-in (range: 5-30) |
+
+### Step 5: Knowledge Store Mode
+
+Ask the user to pick one:
+
+| Mode | Description |
+|------|-------------|
+| `sqlite` | Local SQLite + FTS5. Zero infrastructure, no Docker needed. **(recommended)** |
+| `none` | Curated files only. Zero dependencies. |
+| `local-docker` | Local ChromaDB via Docker (legacy). Pipeline provides a docker-compose file. |
+| `connect-to-existing` | Remote/hosted ChromaDB instance for shared team knowledge (legacy). |
+
+**If `sqlite`:** Set `sqlite_db_path` to `./v3/pipeline.db` (default). Inform the user to run `valent-pipeline db init` to create the database, or it will be created automatically on first story run.
+
+Also show the `knowledge_base_path` setting:
+- Default: `"./knowledge"`
+- Let the user change the path if desired.
+
+**If `local-docker`:** Inform the user that a `docker-compose.chromadb.yml` file will be created at `v3/docker-compose.chromadb.yml`. They can start it with:
+```
+docker compose -f v3/docker-compose.chromadb.yml up -d
+```
+
+**If `connect-to-existing`:** Ask for:
+- ChromaDB host URL (e.g., `http://chromadb.internal:8000`)
+- Collection prefix (e.g., `my-project-`) to namespace collections
+
+### Step 6: Model Assignments
+
+Show the default agent-to-model-tier assignments:
+
+| Tier | Agents | Rationale |
+|------|--------|-----------|
+| **opus** | bend, fend, critic | Complex code generation and deep review |
+| **sonnet** | reqs, uxa, qa_a, qa_b, judge_g1, judge_g2, pmcp, retrospective | Analysis, spec writing, coordination |
+| **haiku** | knowledge, embed, help | Mechanical retrieval and indexing |
+
+Ask the user if they want to adjust any assignments. Common adjustments:
+- Move `critic` to sonnet to save cost
+- Move `reqs` or `qa_a` to opus for higher quality specs/tests
+- Move all agents to sonnet for a budget-friendly setup
+
+Note: agents that were skipped due to project type (e.g., FEND for backend-api) should still appear in the model list -- they are inactive but remain configured.
+
+---
+
+## Writing the Config File
+
+After collecting all answers, generate the complete `v3/pipeline-config.yaml` file. Use the existing file at `v3/pipeline-config.yaml` as the canonical template -- preserve its structure, section headers, and comments exactly. Only change the values based on user input.
+
+When writing the knowledge section, handle fields based on the selected mode:
+- When mode is `sqlite`: set `sqlite_db_path` to `./v3/pipeline.db`. Leave ChromaDB fields commented out.
+- When mode is `none`: leave `sqlite_db_path`, `chromadb_host`, and `chromadb_collection_prefix` commented out.
+- When mode is `local-docker`: uncomment and set `chromadb_host` to `http://localhost:8000`. Leave `chromadb_collection_prefix` commented out (or set to default if user specifies).
+- When mode is `connect-to-existing`: uncomment and populate both `chromadb_host` and `chromadb_collection_prefix` with user-provided values.
+
+Read the existing template file before writing to ensure you preserve its exact format.
+
+**If `local-docker` was selected for knowledge mode**, also create `v3/docker-compose.chromadb.yml`:
+
+```yaml
+version: "3.8"
+services:
+  chromadb:
+    image: chromadb/chroma:latest
+    ports:
+      - "8000:8000"
+    volumes:
+      - ./knowledge/chromadb-data:/chroma/chroma
+    environment:
+      - ANONYMIZED_TELEMETRY=false
+      - PERSIST_DIRECTORY=/chroma/chroma
+```
+
+## After Writing
+
+1. Read back the generated file to verify it is valid YAML.
+2. Print a summary of the configuration to the user.
+3. If knowledge mode is `local-docker`, remind them to run the docker-compose command.
+4. If knowledge mode is `connect-to-existing`, remind them to verify ChromaDB connectivity.
+
+---
+
+## Reconfiguration Mode
+
+When the config file already exists and the user wants to change settings:
+
+1. Read the current `v3/pipeline-config.yaml`.
+2. Show a summary of all current settings grouped by section.
+3. Ask what the user wants to change. Accept natural language like "switch to python", "change the test framework to pytest", "move critic to sonnet".
+4. Apply only the requested changes, preserving everything else.
+5. Write the updated file and show a diff of what changed.

package/skills/v3-run-epic/SKILL.md

@@ -0,0 +1,171 @@
+---
+name: v3-run-epic
+description: 'Run all stories in an epic through the v3 pipeline sequentially. Use when the user says "run epic", "execute epic", or "v3 epic"'
+argument-hint: '<epic-id>'
+---
+
+# v3-run-epic
+
+Run all stories tagged with an epic sequentially through the v3 multiagent pipeline. The epic orchestrator is stateless between stories — all state lives on disk.
+
+## Arguments
+
+The user provides an epic ID (e.g., `KANBAN-MVP`). This is matched against the `epic` field on backlog items.
+
+If no argument is provided, ask the user for the epic ID.
+
+## Context Window Advisory
+
+Use the standard 200k context window. Per `pipeline-config.yaml` `orchestration.recommended_context_window`, 200k provides 93-98% MRCR v2 retrieval accuracy vs 76% at 1M. Auto-compression fires every 2-3 stories, keeping context clean. The PostCompact hook re-injects state from disk.
+
+## Execution Steps
+
+### Step 1: Load Pipeline Config
+
+Read and follow `v3/steps/orchestration/load-pipeline-config.md`.
+
+Set `{epic_id}` from the user's argument.
+
+### Step 2: Validate Epic
+
+Read `{backlog_path}`. Filter items where `epic` field matches `{epic_id}`.
+
+- If no items match, stop: "No backlog items found for epic `{epic_id}`. Add `epic: {epic_id}` to stories in `{backlog_path}`."
+- Count total, pending, shipped, blocked, and cancelled items for this epic.
+- Present summary to the user and confirm: "Epic `{epic_id}`: {total} stories ({pending} pending, {shipped} shipped, {blocked} blocked). Start from first pending story?"
+- If some stories are already shipped (resume scenario), note which are complete.
+
+### Step 3: Initialize or Resume Epic Progress
+
+Read `{epic_progress_path}`.
+
+**If the file exists and `epic_id` matches:** This is a **resume**. Display prior progress (stories completed, stories remaining). Confirm with the user: "Resuming epic `{epic_id}` — {completed} stories done, {remaining} remaining. Continue?"
+
+**If the file does not exist or `epic_id` differs:** Create initial `epic-progress.md`:
+
+```markdown
+---
+epic_id: {epic_id}
+stories_completed: []
+stories_remaining: [{pending story IDs from backlog, filtered by epic}]
+current_story: null
+stories_since_retro: 0
+total_completed: 0
+last_updated: {ISO-8601}
+---
+
+## Story Outcomes (compact — max 2 lines per story)
+```
+
+### Step 4: Story Loop
+
+Loop until all epic stories are shipped, blocked, or cancelled.
+
+#### 4a. Re-read State from Disk
+
+**ALWAYS** read `{epic_progress_path}` and `{backlog_path}` from disk at the top of each iteration. **NEVER** rely on in-context memory of prior stories. Context compression may have fired — disk is the source of truth.
+
+#### 4b. Resolve Next Story
+
+Read and follow `v3/steps/orchestration/resolve-next-work-item.md`.
+
+Set `{epic_filter}` = `{epic_id}` to scope backlog scanning to this epic.
+
+If no unblocked stories remain in this epic, exit the loop and go to Step 5.
+
+#### 4c. Resolve Story Path
+
+Read and follow `v3/steps/orchestration/resolve-story-path.md`.
+
+#### 4d. Validate Story Inputs
+
+Read and follow `v3/steps/orchestration/validate-story-inputs.md`.
+
+On validation failure: mark story `blocked-on-user` in backlog, update `epic-progress.md`, continue to next iteration (Step 4a).
+
+#### 4e. Load Agents Manifest
+
+Read and follow `v3/steps/orchestration/load-agents-manifest.md`.
+
+#### 4f. Adopt Lead and Run Story
+
+Read and follow `v3/steps/orchestration/adopt-lead-and-create-team.md`.
+
+Set `{is_epic_run}` = true. This enables Phase 3 Step 5b (epic progress checkpoint) in `lead.md`.
+
+Enter monitoring mode — follow `v3/prompts/lead.md` through story completion (Phase 2 monitor, Phase 3 ship/teardown).
+
+#### 4g. Update Epic Progress
+
+After the story ships (or is cancelled/blocked), update `{epic_progress_path}`:
+- Move the story from `stories_remaining` to `stories_completed`
+- Add a compact outcome line (~2 lines max): `{STORY-ID}: SHIPPED {date}. {one-line summary}. {test counts}. {bug count}. {clean|conditional}.`
+- Update `total_completed`, `stories_since_retro`, `last_updated`
+- **Keep the file under 50 lines total** regardless of story count
+
+#### 4h. Update Backlog
+
+Read and follow `v3/steps/orchestration/update-backlog-status.md`.
+
+#### 4i. Retrospective Check (BLOCKING)
+
+If `stories_since_retro >= {retrospective_every_n_stories}`:
+1. Spawn the Retrospective Agent with the last N story reports
+2. **Wait for the retrospective to complete** (blocking — do not start the next story)
+3. The retrospective updates correction directives, which feed into subsequent stories
+4. Reset `stories_since_retro` to 0
+5. Update `pipeline-state.json`
+
+#### 4j. Continue Loop
+
+Return to Step 4a. Context compression may fire here — that is expected. The PostCompact hook re-injects `epic-progress.md`, `pipeline-state.json`, and `pipeline-backlog.yaml`.
+
+### Step 5: Epic Complete
+
+When all stories in the epic are shipped (or remaining are all blocked/cancelled):
+
+#### 5a. Write Epic Report
+
+Write `epic-report.md` adjacent to `{epic_progress_path}`:
+- Epic ID, total stories, shipped count, blocked count, cancelled count
+- Aggregate metrics: total bugs filed, total rejection cycles, total escalations
+- Per-story one-line summaries (from `epic-progress.md`)
+- Cross-epic bugs: list any conditional bugs from shipped stories that are still pending
+
+#### 5b. Report to User
+
+Present the epic completion summary. If any stories are blocked or cancelled, list them with reasons.
+
+## Error Recovery
+
+### Mid-Story Failure
+If a story fails (circuit breaker exhausted, user cancels): mark it `cancelled` in the backlog, update `epic-progress.md`, continue to the next story. Do not abort the entire epic for a single story failure.
+
+### Context Compression
+Expected behavior during multi-story runs. The PostCompact hook re-injects state files. After compression, Step 4a re-reads ALL state from disk before continuing. No in-context memory is trusted.
+
+### Full Conversation Reset
+If the conversation dies entirely, the user re-runs `/v3-run-epic {epic_id}`. Step 3 detects the existing `epic-progress.md` and resumes from where it left off. Shipped stories are not re-run.
+
+## Notes
+
+- You are the Lead for each story. The lead prompt (`v3/prompts/lead.md`) is YOUR operating manual — read and follow it directly.
+- **Do NOT read agent output files.** Quality is JUDGE gates' and CRITIC's job.
+- **Always read from disk, never from memory** at the top of each loop iteration. This is the compression survival rule.
+- **Epic-progress.md is the crash recovery substrate.** Keep it under 50 lines. It must be readable standalone.
+- **Retrospectives run BLOCKING** during epic loops to ensure correction directives are applied to subsequent stories.
+- **Agent lifecycle — do NOT shut down agents early.** All per-story teammates stay alive until Phase 3 teardown. Only send `shutdown_request` during Phase 3, Step 4.
+- **Shutdown must be sent individually, not broadcast.** Send a separate `shutdown_request` to each teammate by name.
+- Correction directives may not exist yet for new pipelines — proceed without them gracefully.
+
+### Knowledge Agent Persistence
+
+The Knowledge Agent persists across stories during epic runs to avoid respawn overhead (~15-20k tokens per story).
+
+**Team lifecycle:**
+- Use team name `v3-{epic_id}` (e.g., `v3-kanban-mvp`) for the entire epic, NOT `v3-{story_id}` per story.
+- At story teardown (Phase 3, Step 4): send `shutdown_request` to all per-story teammates EXCEPT Knowledge. Do NOT call `TeamDelete` between stories.
+- At the start of each new story: Knowledge is already alive. Send `[STORY-RESET]` instead of spawning (see `v3/steps/orchestration/adopt-lead-and-create-team.md`). Wait for `[KNOWLEDGE-READY]` before spawning other agents.
+- At epic completion (Step 5): tear down Knowledge, then call `TeamDelete` to clean up the team.
+
+**First story in epic:** Knowledge is spawned normally (no existing teammate to reset). All subsequent stories use the reset protocol.

package/skills/v3-run-retrospective/SKILL.md

@@ -0,0 +1,82 @@
+---
+name: v3-run-retrospective
+description: 'Run a batch retrospective over recent stories. Use when the user says "run retrospective", "analyze recent stories", or "v3 retro"'
+argument-hint: '[count]'
+---
+
+# v3 Run Retrospective
+
+Manually trigger a retrospective analysis over the last N completed stories.
+
+## Steps
+
+### 1. Load Configuration
+
+Read `v3/pipeline-config.yaml` and extract:
+- `quality.retrospective_every_n_stories` (default count, typically 5)
+- `project.story_output_directory` (pattern: `./stories/{story-id}/output`)
+- `project.knowledge_base_path`
+- `knowledge.mode`
+- `knowledge.curated_files_path`
+- `knowledge.correction_directives_path`
+
+### 2. Determine Count
+
+If the user provided a count argument, use that. Otherwise default to the `retrospective_every_n_stories` value from config.
+
+### 3. Find Completed Story Output Directories
+
+Scan the story output directory pattern from config. Convert the `{story-id}` placeholder in the pattern to `*` for globbing (e.g., `./stories/{story-id}/output` becomes `./stories/*/output`). A story is "completed" if its output directory contains a `story-report.md` OR `judge-g2-decision.md` file (fallback for older stories that predate story-report.md generation).
+
+Sort by the `date` field in the `story-report.md` YAML frontmatter (most recent first). If story-report.md is absent, fall back to the `date` or `lastCheckpoint` in `judge-g2-decision.md` frontmatter, or the git commit date. Take the last N.
+
+### 4. Validate
+
+- If fewer than 2 completed stories exist, stop and inform the user: "Retrospective requires at least 2 completed stories. Found {count}."
+- List the stories that will be included and confirm with the user before proceeding.
+
+### 5. Determine Batch Number
+
+Check for existing `retrospective-batch-*.md` files in `{knowledge_base_path}` to determine the next batch number. If none exist, start at batch 1.
+
+### 6. Spawn Retrospective Agent
+
+Use the Agent tool to spawn an ephemeral Sonnet-tier subagent with the Retrospective prompt (`v3/prompts/retrospective.md`).
+
+Provide these context variables:
+- `batch_number` -- the next batch sequence number
+- `story_output_dirs` -- array of absolute paths to each story's output directory
+- `story_ids` -- array of story identifiers extracted from directory names
+- `correction_directives_path` -- resolved absolute path to `correction-directives.yaml`
+- `curated_files_path` -- resolved absolute path to curated knowledge directory
+- `knowledge_mode` -- value from config (`none`, `local-docker`, or `connect-to-existing`)
+
+The Retrospective Agent will:
+- Analyze all story outputs for patterns
+- Update correction directives
+- Write the retrospective report
+- Generate embed instructions if applicable
+
+**Note:** When the lead auto-triggers retrospectives during normal pipeline operation, it spawns the Retrospective Agent as a teammate with team context, so inbox messaging to the lead works. This skill spawns it as a standalone Agent subagent without team context, so inbox messaging is not available. The retrospective prompt handles this conditionally -- the agent skips inbox reporting when spawned standalone, and this skill handles reporting to the user in Step 8.
+
+### 7. Post-Retrospective: Embed Agent (Conditional)
+
+After the Retrospective Agent completes, check the `knowledge.mode` from config:
+- If `none` -- skip embedding, retrospective is complete.
+- If `local-docker` or `connect-to-existing` -- spawn the Embed Agent (`v3/prompts/embed.md`) with the `embed-instructions.md` file produced by the Retrospective Agent.
+
+### 8. Update Pipeline State
+
+Read `pipeline-state.json` and update:
+- Set `stories_completed_since_retro` to `0`
+- Set `last_retrospective_batch` to the batch number used in this retrospective
+
+This step is critical — without it, the lead's auto-trigger threshold (`retrospective_every_n_stories`) will never reset, causing the retro to appear perpetually overdue.
+
+### 9. Report Completion
+
+Inform the user that the retrospective is complete and summarize:
+- Which stories were analyzed
+- Path to the retrospective report
+- Whether correction directives were updated
+- Whether embed instructions were generated (and if Embed Agent was triggered)

package/skills/v3-run-story/SKILL.md

@@ -0,0 +1,59 @@
+---
+name: v3-run-story
+description: 'Run a story through the v3 pipeline. Use when the user says "run story", "execute story", or "v3 run"'
+argument-hint: '<story-id or path>'
+---
+
+# v3-run-story
+
+Run a single story through the v3 multiagent pipeline by validating inputs, resolving config, adopting the Lead persona, and orchestrating teammates directly.
+
+## Arguments
+
+The user provides a story ID or path as the argument:
+- Story ID: `STORY-042` (resolved to `{story_directory}/STORY-042`)
+- Relative path: `./stories/STORY-042` (used directly)
+
+If no argument is provided, resolve the next work item from the backlog (see Step 1b).
+
+## Execution Steps
+
+### Step 1: Load Pipeline Config
+
+Read and follow `v3/steps/orchestration/load-pipeline-config.md`.
+
+### Step 1b: Resolve Next Work Item (when no argument provided)
+
+Read and follow `v3/steps/orchestration/resolve-next-work-item.md`.
+
+Do NOT set `{epic_filter}` — standalone story runs use the full backlog.
+
+### Step 2: Resolve Story Path
+
+Read and follow `v3/steps/orchestration/resolve-story-path.md`.
+
+### Step 3: Validate Story Inputs
+
+Read and follow `v3/steps/orchestration/validate-story-inputs.md`.
+
+### Step 4: Load Agents Manifest
+
+Read and follow `v3/steps/orchestration/load-agents-manifest.md`.
+
+### Step 5: Adopt Lead Persona and Create Team
+
+Read and follow `v3/steps/orchestration/adopt-lead-and-create-team.md`.
+
+Set `{is_epic_run}` = false. This is a standalone story run.
+
+Enter monitoring mode — follow `v3/prompts/lead.md` for monitoring, exception handling, and ship/teardown.
+
+## Notes
+
+- You are the Lead. The lead prompt (`v3/prompts/lead.md`) is YOUR operating manual — read and follow it directly.
+- **Do NOT read agent output files.** Quality is JUDGE gates' and CRITIC's job. When you see a task complete or receive a `[STATUS]` CC, that's sufficient. Do NOT read the handoff document.
+- **Spawn Knowledge first, then all others in parallel.** Knowledge is spawned in Wave 1 so it's ready for queries. All remaining roster agents are spawned together in Wave 2. They self-trigger via their Trigger Protocol (inbox messages between peers).
+- Correction directives may not exist yet for new pipelines — that is expected. Pass the path regardless; proceed without directives gracefully.
+- **Agent lifecycle — do NOT shut down agents early.** All per-story teammates stay alive until Phase 3 teardown (after JUDGE-G2 approves). Agents communicate peer-to-peer via inbox: CRITIC sends rejections directly to BEND/FEND, QA-B routes bugs directly to devs, any agent can message any other living teammate. Premature shutdown breaks this communication. Only send `shutdown_request` during Phase 3, Step 4 ("Tear Down Teammates") when the story is complete or cancelled.
+- **Team cleanup at teardown is mandatory.** After all teammates have been shut down in Phase 3, call `TeamDelete` to remove the team. This prevents stale team errors on the next pipeline run. The Lead prompt's Phase 3 Step 4 ("Tear Down Teammates") must end with `TeamDelete`.
+- **Shutdown must be sent individually, not broadcast.** `SendMessage` with `to: "*"` does not support structured `shutdown_request` messages — only plain text. Send a separate `shutdown_request` to each teammate by name during Phase 3 teardown.