@davidorex/pi-workflows 0.1.0

package/CHANGELOG.md

@@ -0,0 +1,28 @@
+# Changelog
+
+All notable changes to this project will be documented in this file.
+
+## v0.1.0
+
+Initial release.
+
+### Added
+
+- Workflow orchestration via `.workflow.yaml` specs with DAG-based execution planning
+- Step types: agent, command, transform, gate, parallel, foreach, loop, pause
+- Expression evaluator (`${{ }}`) with filters: length, keys, filter, json, upper, lower, trim, default, first, last, join, split, replace, includes, map, sum, min, max, sort, unique, flatten, zip, group_by, count_by, chunk, pick, omit, entries, from_entries, merge, values, not, and, or
+- Agent dispatch via subprocess (`pi --mode json`) with thinking inheritance
+- Nunjucks template compilation for agent prompts with template inheritance
+- Agent spec loader (`.agent.yaml` format with model, thinking, tools, output schema)
+- DAG planner inferring parallelism from `${{ steps.X }}` references
+- Atomic state persistence after each step (tmp + rename, failure is fatal)
+- Checkpoint/resume: incomplete runs resume from last completed step
+- `completion` field for post-workflow messages to main LLM
+- Workflow SDK: `stepTypes()`, `filterNames()`, `expressionRoots()`, vocabulary discovery
+- Workflow discovery: `availableAgents(cwd)`, `availableWorkflows(cwd)`, `availableTemplates(cwd)` with three-tier search (project > user > package builtin)
+- Spec introspection: `extractExpressions`, `declaredSteps`, `declaredAgentRefs`, `declaredSchemaRefs`
+- `/workflow` command with `run`, `list`, `resume` subcommands
+- TUI progress widget for workflow execution
+- Bundled agents: investigator, decomposer, executor, verifier, refresher
+- Bundled workflows: do-gap, gap-to-phase, create-phase, refresh-blocks
+- Bundled output schemas and Nunjucks templates

package/README.md

@@ -0,0 +1,169 @@
+# pi-workflows
+
+Workflow orchestration extension for [Pi](https://github.com/badlogic/pi-mono). Define multi-step workflows in YAML, execute them as DAGs with typed data flow between agents, and resume from checkpoints on failure.
+
+## Install
+
+```bash
+pi install pi-workflows
+```
+
+Requires `pi-project` as a peer dependency (installed separately).
+
+## What It Does
+
+pi-workflows replaces ad-hoc agent chaining with composable, typed workflow orchestration. Workflows are YAML specs. Steps run as subprocesses (`pi --mode json`) with their own context windows. The main conversation is the control plane; workflows are subordinate.
+
+**Tool registered:**
+- `workflow` — run a named workflow with typed input
+
+**Commands registered:**
+- `/workflow list` — discover and select a workflow to run
+- `/workflow run <name> [--input '<json>']` — execute a workflow
+- `/workflow resume <name>` — resume an incomplete run from checkpoint
+
+**Keybindings:**
+- `Ctrl+H` — pause running workflow after current step
+- `Ctrl+J` — resume a paused/incomplete workflow
+
+## Workflow Spec Format
+
+Workflows are `.workflow.yaml` files discovered from `.pi/workflows/` (project), `~/.pi/agent/workflows/` (user), or the package's `workflows/` directory (builtin).
+
+```yaml
+name: my-workflow
+description: What this workflow does
+input:
+  type: object
+  properties:
+    target: { type: string, description: "What to analyze" }
+  required: [target]
+
+steps:
+  investigate:
+    agent: investigator
+    input: |
+      Investigate: ${{ input.target }}
+    output:
+      schema: investigation-findings
+
+  synthesize:
+    agent: synthesizer
+    input: |
+      Findings: ${{ steps.investigate.output | json }}
+
+completion:
+  message: |
+    Analysis complete. Key findings: ${{ steps.synthesize.output.summary }}
+```
+
+### Step Types
+
+| Type | Purpose |
+|------|---------|
+| `agent` | Dispatch to a Pi agent subprocess |
+| `command` | Run a shell command |
+| `transform` | Map/reshape data between steps |
+| `gate` | Conditional branching (`check` expression) |
+| `parallel` | Run nested steps concurrently |
+| `foreach` | Iterate over an array |
+| `loop` | Repeat steps with a condition |
+| `pause` | Halt execution for human review |
+
+### Expressions
+
+`${{ }}` expressions access step outputs, inputs, and apply filters:
+
+```yaml
+${{ input.target }}                    # workflow input
+${{ steps.investigate.output }}        # step output
+${{ steps.investigate.output | json }} # pipe through filter
+${{ steps.gather.output | length }}    # array length
+```
+
+Available filters: `length`, `keys`, `filter`, `json`, `upper`, `lower`, `trim`, `default`, `first`, `last`, `join`, `split`, `replace`, `includes`, `map`, `sum`, `min`, `max`, `sort`, `unique`, `flatten`, `zip`, `group_by`, `count_by`, `chunk`, `pick`, `omit`, `entries`, `from_entries`, `merge`, `values`, `not`, `and`, `or`.
+
+## Source Files
+
+| File | Purpose |
+|------|---------|
+| `src/index.ts` | Extension entry point — tool, command, keybinding registration |
+| `src/workflow-executor.ts` | Main orchestration loop |
+| `src/workflow-spec.ts` | YAML parsing, `STEP_TYPES` registry |
+| `src/workflow-sdk.ts` | SDK: vocabulary, discovery, introspection |
+| `src/workflow-discovery.ts` | Three-tier workflow discovery (project > user > builtin) |
+| `src/expression.ts` | `${{ }}` evaluator, `FILTER_NAMES` registry |
+| `src/template.ts` | Nunjucks template environment |
+| `src/dispatch.ts` | Agent subprocess spawn (`pi --mode json`) |
+| `src/dag.ts` | Dependency graph, execution plan from `${{ steps.X }}` refs |
+| `src/agent-spec.ts` | `.agent.yaml` parser |
+| `src/state.ts` | Atomic run state persistence |
+| `src/checkpoint.ts` | Checkpoint detection, resume validation |
+| `src/output.ts` | Step output persistence |
+| `src/completion.ts` | Post-workflow message resolution |
+| `src/tui.ts` | Terminal progress widget |
+| `src/types.ts` | Shared type definitions |
+| `src/step-*.ts` | Step type executors (one per type) |
+
+## Bundled Resources
+
+| Directory | Contents |
+|-----------|----------|
+| `agents/` | 13 agent specs (`.agent.yaml`): investigator, decomposer, verifier, synthesizer, etc. |
+| `schemas/` | 10 output schemas (`.schema.json`): investigation-findings, execution-results, etc. |
+| `workflows/` | 10 workflow specs (`.workflow.yaml`): do-gap, create-phase, parallel-analysis, etc. |
+| `templates/` | Nunjucks prompt templates organized by agent role |
+
+## SDK (`src/workflow-sdk.ts`)
+
+Single queryable surface for the extension's capabilities:
+
+```typescript
+// Vocabulary (derived from code registries)
+stepTypes(): StepTypeDescriptor[]
+filterNames(): string[]
+expressionRoots(): readonly string[]
+
+// Discovery (derived from filesystem, three-tier search)
+availableAgents(cwd): AgentSpec[]
+availableWorkflows(cwd): WorkflowSpec[]
+availableTemplates(cwd): string[]
+availableSchemas(cwd): string[]
+
+// Introspection (derived from parsed spec)
+extractExpressions(spec): ExpressionRef[]
+declaredSteps(spec): string[]
+declaredAgentRefs(spec): string[]
+declaredSchemaRefs(spec): string[]
+```
+
+## Architecture
+
+- Each workflow step runs as a subprocess (`pi --mode json`) with its own context window
+- DAG planner infers parallelism from `${{ steps.X }}` references — no manual dependency declaration
+- Agent specs are `.agent.yaml` only. Compiled to prompts via Nunjucks at dispatch time.
+- State persisted atomically after each step (tmp + rename). State write failure is fatal.
+- Three-tier resource search: project `.pi/` > user `~/.pi/agent/` > package builtin
+- `completion` field controls what message is sent back to the main conversation after a workflow finishes
+
+## For LLMs
+
+When working with this extension:
+
+- **Read `src/workflow-sdk.ts`** for the full vocabulary, discovery, and introspection API
+- **Read `src/workflow-spec.ts`** for the `STEP_TYPES` registry and YAML parsing rules
+- **Read `src/expression.ts`** for `FILTER_NAMES` and expression evaluation rules
+- **Read `src/types.ts`** for `WorkflowSpec`, `StepSpec`, `AgentSpec`, and `ExecutionState` type definitions
+- **Read `src/dag.ts`** to understand how execution order is inferred from expressions
+- **Read agent `.agent.yaml` files** in `agents/` to understand available agent capabilities
+- **Read workflow `.workflow.yaml` files** in `workflows/` for examples of workflow structure
+- Use the `workflow` tool to execute workflows — it handles discovery, input validation, checkpoint detection, and result formatting
+- The `/workflow list` command provides an interactive picker; `/workflow run <name>` runs directly
+
+## Tests
+
+```bash
+npm test
+```
+
+Runs `node --experimental-strip-types run-tests.ts` which discovers and executes all `src/*.test.ts` files. 500+ tests covering step types, expressions, DAG planning, state persistence, checkpoint/resume, and template compilation.

package/agents/audit-fixer.agent.yaml

@@ -0,0 +1,18 @@
+name: audit-fixer
+role: action
+description: Fixes a cluster of related audit findings in source code
+
+tools:
+  - read
+  - write
+  - edit
+  - bash
+  - grep
+  - find
+
+output:
+  format: json
+  schema: schemas/execution-results.schema.json
+
+prompt:
+  task: audit-fixer/task.md

package/agents/code-explorer.agent.yaml

@@ -0,0 +1,11 @@
+name: code-explorer
+role: sensor
+description: Explore a codebase path and produce a structural overview
+tools: [read, bash, ls]
+
+output:
+  format: json
+
+prompt:
+  system: explorer/system.md
+  task: explorer/task.md

package/agents/decomposer.agent.yaml

@@ -0,0 +1,51 @@
+name: decomposer
+description: Decomposes investigation findings into implementation specs with parallelization decisions
+role: decomposer
+prompt:
+  system: |
+    <objective>
+    You decompose a gap into implementation specs. Each spec is a unit of work for a single implementing agent — one context window, one concern. You receive investigation findings and optional research findings.
+    </objective>
+
+    <workflow>
+    1. Review affected files, constraints, and risks from investigation
+    2. Identify the minimal set of changes needed
+    3. Group changes into specs — each spec modifies a coherent set of files
+    4. Set dependencies: if spec B requires interfaces from spec A, B depends_on A
+    5. Set parallel_safe based on file overlap and semantic dependency
+    6. Write acceptance criteria that are verifiable (grep patterns, test commands, not "it works")
+    7. Estimate complexity per spec
+    </workflow>
+
+    <constraints>
+    - Output MUST be valid JSON conforming exactly to the output schema
+    - Every field marked required in the schema MUST be present
+    - Specs modifying the same file MUST have depends_on relationships (NOT parallel_safe)
+    - Specs where one introduces an interface the other consumes MUST be sequential
+    - Specs modifying different files with no semantic dependency SHOULD be parallel_safe: true
+    - Each spec must fit in a single agent context window — split if too large
+    - estimated_complexity MUST be one of: low, medium, high
+    - depends_on is an empty array if no dependencies; contains spec names otherwise
+    - Acceptance criteria must be machine-verifiable where possible (grep, test commands, file existence)
+    </constraints>
+
+    <anti_patterns>
+    - One giant spec that does everything — split into focused units
+    - Specs with vague acceptance criteria ("code is clean", "tests pass")
+    - Marking specs as parallel_safe when they modify the same file
+    - Creating specs for work not required by the gap
+    - Acceptance criteria that require human judgment
+    </anti_patterns>
+
+    <success_criteria>
+    - Output validates against the schema with zero errors
+    - Every spec has at least one file target
+    - Every spec has at least one verifiable acceptance criterion
+    - Dependency graph has no cycles
+    - parallel_safe flags are consistent with file targets (no shared-file parallelism)
+    - ordering_rationale explains the structure in one paragraph
+    </success_criteria>
+  task: decomposer/task.md
+output:
+  format: json
+  schema: schemas/decomposition-specs.schema.json

package/agents/investigator.agent.yaml

@@ -0,0 +1,49 @@
+name: investigator
+description: Investigates a gap by reading the codebase — produces structured findings
+role: investigator
+tools: [read, bash, grep, find]
+prompt:
+  system: |
+    <objective>
+    You investigate a code gap and produce structured JSON findings. You read code to understand what exists, what's affected, and what constraints apply. You do NOT plan or implement — you produce understanding.
+    </objective>
+
+    <workflow>
+    1. Read the gap description and identify which files are likely affected
+    2. Read those files (minimum reads — use grep/find to locate, read only what's needed)
+    3. For each affected file: document its path, role, and current behavior
+    4. List constraints (tests, interfaces, conventions that must not break)
+    5. List risks only if non-obvious
+    6. Set complexity: low (single file, mechanical), medium (multiple files, design choices), high (architectural)
+    7. Set needs_research to true ONLY if the solution requires knowledge not in the codebase
+    8. Write a one-paragraph summary
+    </workflow>
+
+    <constraints>
+    - Output MUST be valid JSON conforming exactly to the output schema
+    - Every field marked required in the schema MUST be present
+    - Do NOT write to any files — read only
+    - Do NOT produce implementation plans or code changes
+    - Match investigation depth to gap priority: low-priority cleanup gaps need minimal investigation
+    - If needs_research is false, research_questions MUST be an empty array
+    </constraints>
+
+    <anti_patterns>
+    - Reading every file in the project — use grep to find relevant code
+    - Setting needs_research: true for gaps solvable from codebase context alone
+    - Producing a plan instead of findings
+    - Omitting required fields from the output
+    - Writing or modifying any project files
+    </anti_patterns>
+
+    <success_criteria>
+    - Output validates against the schema with zero errors
+    - Every affected_files entry has path, role, and current_behavior
+    - constraints lists real constraints from the codebase (not generic advice)
+    - complexity rating matches actual scope
+    - Summary is one paragraph, not a plan
+    </success_criteria>
+  task: investigator/task.md
+output:
+  format: json
+  schema: schemas/investigation-findings.schema.json

package/agents/pattern-analyzer.agent.yaml

@@ -0,0 +1,21 @@
+name: pattern-analyzer
+role: sensor
+description: Analyze design patterns and return typed findings
+tools: [read, bash, ls]
+
+input:
+  type: object
+  required: [exploration, path]
+  properties:
+    exploration:
+      description: Prior exploration output (typed JSON)
+    path:
+      type: string
+
+output:
+  format: json
+  schema: schemas/pattern-analysis.schema.json
+
+prompt:
+  system: analyzers/patterns.md
+  task: analyzers/patterns-task.md

package/agents/phase-author.agent.yaml

@@ -0,0 +1,16 @@
+name: phase-author
+role: reasoning
+description: Converts unstructured intent from conversation into a structured phase spec with numbered specs, acceptance criteria, and success criteria
+
+tools:
+  - read
+  - bash
+  - grep
+  - find
+
+output:
+  format: json
+  schema: ../../.workflow/schemas/phase.schema.json
+
+prompt:
+  task: templates/phase-author/task.md

package/agents/plan-decomposer.agent.yaml

@@ -0,0 +1,12 @@
+name: plan-decomposer
+role: reasoning
+description: Decomposes a phase spec into focused implementation plans with context decisions
+
+tools: [read, bash, grep, find]
+
+output:
+  format: json
+  schema: schemas/plan-breakdown.schema.json
+
+prompt:
+  task: plan-decomposer/task.md

package/agents/quality-analyzer.agent.yaml

@@ -0,0 +1,21 @@
+name: quality-analyzer
+role: sensor
+description: Analyze code quality and return typed findings
+tools: [read, bash, ls]
+
+input:
+  type: object
+  required: [exploration, path]
+  properties:
+    exploration:
+      description: Prior exploration output (typed JSON)
+    path:
+      type: string
+
+output:
+  format: json
+  schema: schemas/quality-analysis.schema.json
+
+prompt:
+  system: analyzers/quality.md
+  task: analyzers/quality-task.md

package/agents/researcher.agent.yaml

@@ -0,0 +1,38 @@
+name: researcher
+description: Researches known solutions, patterns, and heuristics relevant to a gap
+role: researcher
+prompt:
+  system: |
+    <objective>
+    You answer specific technical research questions about a software engineering problem. You draw on knowledge of patterns, best practices, prior art, and established solutions.
+    </objective>
+
+    <workflow>
+    1. Read each research question
+    2. Answer with specific, actionable information — not general advice
+    3. Rate confidence: high (well-known pattern), medium (established but context-dependent), low (best guess)
+    4. Cite sources: documentation URLs, project names, pattern names, book references
+    5. Identify applicable patterns with concrete applicability statements
+    6. Produce recommendations that are specific enough to implement
+    </workflow>
+
+    <constraints>
+    - Output MUST be valid JSON conforming exactly to the output schema
+    - Every field marked required in the schema MUST be present
+    - Every question entry MUST have: question, answer, sources (array), confidence
+    - Every pattern entry MUST have: name, description, applicability
+    - confidence MUST be one of: high, medium, low
+    - Do NOT fabricate sources — if you don't know a source, use an empty array
+    - Recommendations must be actionable, not generic ("use caching" is too vague; "add LRU cache with 100-entry limit on the resolve() call" is specific)
+    </constraints>
+
+    <success_criteria>
+    - Output validates against the schema with zero errors
+    - Every question receives a direct answer (not "it depends" without resolution)
+    - Confidence ratings are honest — low when uncertain
+    - Recommendations are specific enough that an implementer can act on them without further research
+    </success_criteria>
+  task: researcher/task.md
+output:
+  format: json
+  schema: schemas/research-findings.schema.json

package/agents/spec-implementer.agent.yaml

@@ -0,0 +1,12 @@
+name: spec-implementer
+role: action
+description: Implements a single plan from a phase decomposition
+
+tools: [read, write, edit, bash, grep, find]
+
+output:
+  format: json
+  schema: schemas/execution-results.schema.json
+
+prompt:
+  task: spec-implementer/task.md

package/agents/structure-analyzer.agent.yaml

@@ -0,0 +1,21 @@
+name: structure-analyzer
+role: sensor
+description: Analyze code structure and return typed findings
+tools: [read, bash, ls]
+
+input:
+  type: object
+  required: [exploration, path]
+  properties:
+    exploration:
+      description: Prior exploration output (typed JSON)
+    path:
+      type: string
+
+output:
+  format: json
+  schema: schemas/structure-analysis.schema.json
+
+prompt:
+  system: analyzers/structure.md
+  task: analyzers/structure-task.md

package/agents/synthesizer.agent.yaml

@@ -0,0 +1,23 @@
+name: synthesizer
+role: reasoning
+description: Synthesize typed analysis findings into a unified report
+tools: [read]
+
+input:
+  type: object
+  required: [structure, quality, patterns]
+  properties:
+    structure:
+      description: Structure analysis (typed JSON)
+    quality:
+      description: Quality analysis (typed JSON)
+    patterns:
+      description: Pattern analysis (typed JSON)
+
+output:
+  format: json
+  schema: schemas/synthesis.schema.json
+
+prompt:
+  system: synthesizer/system.md
+  task: synthesizer/task.md

package/agents/verifier.agent.yaml

@@ -0,0 +1,12 @@
+name: verifier
+role: quality
+description: Verifies step outputs against declared intent and success criteria
+
+tools: [read, bash, grep, find]
+
+output:
+  format: json
+  schema: schemas/verifier-output.schema.json
+
+prompt:
+  task: verifier/task.md

package/package.json

@@ -0,0 +1,40 @@
+{
+  "name": "@davidorex/pi-workflows",
+  "version": "0.1.0",
+  "description": "Workflow orchestration extension for Pi",
+  "license": "MIT",
+  "author": "David Ryan",
+  "type": "module",
+  "keywords": ["pi-package"],
+  "repository": {
+    "type": "git",
+    "url": "git+https://github.com/davidorex/pi-project-workflows.git",
+    "directory": "packages/pi-workflows"
+  },
+  "homepage": "https://github.com/davidorex/pi-project-workflows/tree/main/packages/pi-workflows",
+  "pi": {
+    "extensions": ["./src/index.ts"]
+  },
+  "files": ["src/", "agents/", "schemas/", "workflows/", "templates/", "*.md"],
+  "scripts": {
+    "test": "node --experimental-strip-types run-tests.ts",
+    "release": "npx changelogen --bump --release",
+    "release:patch": "npx changelogen --bump --release --patch",
+    "release:minor": "npx changelogen --bump --release --minor",
+    "release:major": "npx changelogen --bump --release --major",
+    "release:push": "git push --follow-tags"
+  },
+  "dependencies": {
+    "nunjucks": "^3.2.4",
+    "yaml": "^2.7.1"
+  },
+  "peerDependencies": {
+    "@mariozechner/pi-coding-agent": "*",
+    "@mariozechner/pi-tui": "*",
+    "@sinclair/typebox": "*",
+    "@davidorex/pi-project": "*"
+  },
+  "devDependencies": {
+    "@types/nunjucks": "^3.2.6"
+  }
+}

package/schemas/decomposition-specs.schema.json

@@ -0,0 +1,50 @@
+{
+  "type": "object",
+  "required": ["specs", "ordering_rationale"],
+  "properties": {
+    "specs": {
+      "type": "array",
+      "items": {
+        "type": "object",
+        "required": ["name", "description", "files", "acceptance_criteria", "parallel_safe", "estimated_complexity"],
+        "properties": {
+          "name": {
+            "type": "string",
+            "description": "Kebab-case identifier for this spec"
+          },
+          "description": {
+            "type": "string",
+            "description": "What this spec implements — the unit of work"
+          },
+          "files": {
+            "type": "array",
+            "items": { "type": "string" },
+            "description": "File paths this spec will modify"
+          },
+          "depends_on": {
+            "type": "array",
+            "items": { "type": "string" },
+            "description": "Names of other specs that must complete before this one"
+          },
+          "acceptance_criteria": {
+            "type": "array",
+            "items": { "type": "string" },
+            "description": "How to verify this spec is correctly implemented"
+          },
+          "parallel_safe": {
+            "type": "boolean",
+            "description": "Whether this spec can run in parallel with other parallel_safe specs that have no file overlap"
+          },
+          "estimated_complexity": {
+            "type": "string",
+            "enum": ["low", "medium", "high"]
+          }
+        }
+      }
+    },
+    "ordering_rationale": {
+      "type": "string",
+      "description": "Why this ordering and parallelization structure was chosen"
+    }
+  }
+}

package/schemas/execution-results.schema.json

@@ -0,0 +1,50 @@
+{
+  "type": "object",
+  "required": ["status", "tasks"],
+  "properties": {
+    "status": {
+      "type": "string",
+      "enum": ["complete", "partial", "failed"]
+    },
+    "tasks": {
+      "type": "array",
+      "items": {
+        "type": "object",
+        "required": ["name", "status"],
+        "properties": {
+          "name": { "type": "string" },
+          "status": { "type": "string", "enum": ["done", "partial", "skipped", "failed"] },
+          "files_modified": { "type": "array", "items": { "type": "string" } },
+          "commit_hash": { "type": "string" },
+          "notes": { "type": "string" }
+        }
+      }
+    },
+    "decisions": {
+      "type": "array",
+      "items": {
+        "type": "object",
+        "required": ["id", "decision", "rationale", "status"],
+        "properties": {
+          "id": { "type": "string" },
+          "decision": { "type": "string" },
+          "rationale": { "type": "string" },
+          "status": { "type": "string", "enum": ["decided", "deferred", "reversed"] }
+        }
+      }
+    },
+    "issues": {
+      "type": "array",
+      "items": {
+        "type": "object",
+        "required": ["severity", "description"],
+        "properties": {
+          "severity": { "type": "string", "enum": ["info", "warning", "error", "critical"] },
+          "description": { "type": "string" }
+        }
+      }
+    },
+    "test_count": { "type": "integer" },
+    "commit_hash": { "type": "string" }
+  }
+}