@exaudeus/workrail 3.42.0 → 3.44.0

package/docs/design/adaptive-coordinator-context-candidates.md

@@ -0,0 +1,265 @@
+# Inter-Phase Context Passing: Design Candidates
+
+*Working analysis document -- raw investigative material, not a final decision.*
+*Produced during wr.discovery workflow for adaptive-coordinator-context.md.*
+
+---
+
+## Problem Understanding
+
+### Core tensions
+
+1. **Concern separation vs. convenience:** ContextAssembler is the only session injection mechanism but was designed for startup context (git diff, prior session notes), not inter-phase structured contracts. Using it for inter-phase handoff is convenient but muddles its purpose and change rate.
+
+2. **Coupling vs. simplicity:** File-based handoff (pitch.md) is simple and already works for Shaping->Coding. Extending the pattern to all transitions requires each workflow's final step to write to a fixed `.workrail/` path -- coupling pipeline topology to workflow authoring.
+
+3. **Uniform bridge vs. typed contracts:** Coordinator-injected freetext (assembledContextSummary) is uniform across all transitions but imprecise -- no schema enforcement. Typed artifacts per transition are precise and Zod-validated but add per-transition schema overhead.
+
+4. **Coordinator responsibility vs. session responsibility:** Coordinator can bridge at spawn time (reads lastStepNotes, builds context, injects at spawn). Sessions can self-discover (Phase 0.5 already does this for pitch.md). Self-discovery is more robust but requires each session to know the file conventions.
+
+### Likely seam
+
+The problem does NOT live in ContextAssembler. The coordinator already passes arbitrary context at spawn time via the 4th parameter to `spawnSession`. The real seam is: **what to put in that context object and how to extract it from the previous phase's output.**
+
+The problem is narrower than it appears:
+- Shaping->Coding: already solved (Phase 0.5 + pitch.md)
+- Review->Fix: already solved (wr.review_verdict + pr-review.ts)
+- Remaining gap: Discovery->Shaping only
+
+### What makes this hard
+
+- Coordinator and session are in different systems; workflow prompt changes can silently break coordinator parsing
+- The only injection point (`assembledContextSummary`) is a freetext markdown string, not a typed contract
+- Two transitions need structured coordinator routing (D->S needs direction; Review->Fix already solved). Three transitions are pass-through.
+- Silent failure is the dominant risk
+
+---
+
+## Philosophy Constraints
+
+**Principles that constrain this design:**
+- Immutability by default: all new types use `readonly`
+- Make illegal states unrepresentable: discriminated unions for AssemblyTask; typed artifacts for coordinator contracts
+- Errors are data: new ContextBundle sources should use `Result<T, string>`
+- Validate at boundaries: Zod for external inputs (typed artifacts, notes parsing)
+- Compose with small pure functions: handoff builder/parser functions must be pure
+- Dependency injection: file reads go through CoordinatorDeps
+- YAGNI: only add typed artifacts for transitions where coordinator branches on structured values
+
+**Active conflict:** YAGNI vs. "make illegal states unrepresentable" -- typed artifacts everywhere would enforce all contracts but add schema overhead for 3 transitions that don't need it.
+
+---
+
+## Impact Surface
+
+Must stay consistent with:
+- `src/context-assembly/types.ts` (AssemblyTask union)
+- `src/context-assembly/index.ts` (assemble() switch on task.kind)
+- `src/coordinators/pr-review.ts` (CoordinatorDeps interface)
+- `src/v2/durable-core/schemas/artifacts/` (typed artifact schemas)
+- `workflows/wr.discovery.json` (Phase 7 -- if emitting artifact)
+- `workflows/wr.shaping.json` (Step 1 -- if adding file search)
+- `workflows/coding-task-workflow-agentic.json` (Phase 0.5 -- no changes expected)
+
+---
+
+## Candidates
+
+### Candidate A: Goal string composition + notes injection
+
+**Summary:** Coordinator composes a rich goal string from the previous phase's `lastStepNotes` and passes notes as `assembledContextSummary` at spawn time -- no new mechanisms required.
+
+**Mechanism:**
+When spawning wr.shaping, coordinator constructs:
+```
+goal = "Shape the following problem discovered in our discovery session:\n\n[lastStepNotes summary]"
+context = { assembledContextSummary: lastStepNotes }
+```
+wr.shaping Step 1 reads from "goal text, discovery notes, tickets, user stories" -- both injection points land in session context.
+
+**Tensions resolved:** Workflow independence, coordinator simplicity, coherence
+
+**Tensions accepted:** No mechanism clarity, no schema validation, silent failure when notes format changes
+
+**Boundary:** Coordinator spawn site only
+
+**Failure mode:** wr.discovery notes format changes silently. Shaping session starts from degraded context. No error surfaced.
+
+**Repo pattern:** Follows existing assembledContextSummary pattern. No departure.
+
+**Gains:** Zero new code outside coordinator. Zero workflow changes. Works today.
+**Losses:** No machine-parseable contract. Coordinator cannot extract structured fields programmatically.
+
+**Scope judgment:** Best-fit for pass-through transitions. Too narrow for transitions where coordinator branches on structured values.
+
+**Philosophy:** Honors YAGNI, DI. Conflicts with "make illegal states unrepresentable", "validate at boundaries".
+
+---
+
+### Candidate B: File convention extension (adapt pitch.md)
+
+**Summary:** wr.discovery Phase 7 writes `.workrail/current-discovery.md`; wr.shaping Step 1 searches for it -- the same file convention that already works for Shaping->Coding.
+
+**Mechanism:**
+- wr.discovery Phase 7 updated to write `.workrail/current-discovery.md`
+- wr.shaping Step 1 updated to search for `.workrail/current-discovery.md` when no goal text provides discovery context
+- Coordinator passes correct workspacePath -- that's the only coordinator involvement
+
+**Tensions resolved:** Discoverable handoff (session self-discovers), coordinator logic (no bridging for D->S)
+
+**Tensions accepted:** 2 workflow changes, stale file risk (failed discovery leaves wrong file)
+
+**Boundary:** Filesystem at `.workrail/`
+
+**Failure mode:** Discovery fails before Phase 7. File is stale from prior session. Shaping reads wrong discovery output. No timestamp check.
+
+**Repo pattern:** Directly adapts pitch.md convention from wr.shaping Step 9.
+
+**Gains:** Sessions self-discover. Coordinator unchanged for D->S. Clean separation.
+**Losses:** Two workflow changes. Stale file risk. Conventions must be maintained.
+
+**Scope judgment:** Best-fit for D->S specifically. Over-engineered for other transitions.
+
+**Philosophy:** Honors YAGNI, "make illegal states unrepresentable" (file presence checkable). Conflicts with "validate at boundaries" (stale file not caught at coordinator boundary).
+
+---
+
+### Candidate C: Coordinator-injected structured context (pure builder functions)
+
+**Summary:** Add a pure `buildDiscoveryHandoffContext(notes: string): PhaseHandoffContext` function in the coordinator that extracts structured fields from discovery notes and passes them as spawn context.
+
+**Type shape:**
+```typescript
+interface PhaseHandoffContext {
+  readonly phaseFrom: string;
+  readonly phaseTo: string;
+  readonly keyFindings: string;
+  readonly selectedDirection?: string;
+  readonly designDocPath?: string;
+}
+```
+
+**Tensions resolved:** Coordinator simplicity (named pure function per transition), no workflow changes, coordinator owns contracts
+
+**Tensions accepted:** Notes format still unconstrained. Builder functions must be maintained when format changes.
+
+**Boundary:** Coordinator spawn site + named pure function (slightly better than A)
+
+**Failure mode:** Notes format changes silently break builder. Same failure as A but slightly more visible (named function can be tested).
+
+**Repo pattern:** Adapts parseFindingsFromNotes() pattern (pure function + coordinator uses result). Direct precedent in pr-review.ts.
+
+**Gains:** Named, testable, pure functions. Coordinator owns contracts. No workflow changes.
+**Losses:** Notes format still unconstrained. Per-transition builder functions as pipeline grows.
+
+**Scope judgment:** Best-fit for coordinator-centric design with intermediate rigor.
+
+**Philosophy:** Honors "compose with small pure functions", DI. Conflicts with "validate at boundaries", "make illegal states unrepresentable".
+
+---
+
+### Candidate D: Typed handoff artifact for decision-critical transitions only (wr.discovery_handoff)
+
+**Summary:** Add a `wr.discovery_handoff` typed artifact emitted in wr.discovery Phase 7, Zod-validated by the coordinator -- applied only to D->S; all other transitions use goal string composition.
+
+**Schema:**
+```typescript
+interface DiscoveryHandoffArtifactV1 {
+  readonly kind: 'wr.discovery_handoff';
+  readonly version: 1;
+  readonly selectedDirection: string;
+  readonly designDocPath: string;
+  readonly confidenceBand: 'high' | 'medium' | 'low';
+  readonly keyInvariants: readonly string[];
+}
+```
+
+**Coordinator parsing:**
+```typescript
+const handoffArtifact = readDiscoveryHandoffArtifact(lastStepArtifacts, sessionHandle);
+const spawnContext = handoffArtifact
+  ? { selectedDirection: handoffArtifact.selectedDirection, designDocPath: handoffArtifact.designDocPath, assembledContextSummary: render(handoffArtifact) }
+  : { assembledContextSummary: lastStepNotes };
+await deps.spawnSession('wr.shaping', buildShapingGoal(handoffArtifact ?? notes), workspace, spawnContext);
+```
+
+**Transition coverage:**
+- D->S: wr.discovery_handoff (NEW)
+- Review->Fix: wr.review_verdict (already exists)
+- S->C: goal string + pitch.md self-discovery via Phase 0.5 (no typed artifact)
+- C->PR: PR number in goal string (no typed artifact)
+- PR->Review: PR number (no typed artifact)
+
+**Tensions resolved:** Mechanism clarity (schema IS the contract), validates at boundaries (Zod), makes illegal states unrepresentable for D->S
+
+**Tensions accepted:** 1 workflow change (wr.discovery Phase 7 must emit artifact). Two-tier fallback complexity. Schema maintenance.
+
+**Boundary:** wr.discovery Phase 7 (emitter) + coordinator lastStepArtifacts handling (consumer)
+
+**Failure mode:** Agent doesn't emit artifact. Coordinator falls back to notes parsing. Two-tier failure, same as wr.review_verdict today.
+
+**Repo pattern:** Directly follows wr.review_verdict pattern in every structural detail.
+
+**Gains:** Machine-parseable D->S contract. Coordinator can route on confidenceBand. Type-safe. Explicit failure.
+**Losses:** 1 workflow change. Schema maintenance. Two-tier parsing. Agent must reliably emit artifact.
+
+**Scope judgment:** Best-fit for D->S. Combined with A for other transitions = minimum typed surface.
+
+**Philosophy:** Fully honors "make illegal states unrepresentable", "validate at boundaries", "type safety as first line of defense", YAGNI (applied only to 2 decision-critical transitions).
+
+---
+
+## Comparison and Recommendation
+
+### Comparison matrix
+
+| Criterion | A | B | C | D |
+|-----------|---|---|---|---|
+| Mechanism clarity | Low | Medium | Medium | High |
+| Coordinator logic minimality | High | High | Medium | Medium |
+| Workflow independence | High | Low | High | Medium |
+| Coherence with existing patterns | Medium | High | Medium | High |
+| Discoverable handoff | Low | High | Low | Low |
+| Philosophy fit | Medium | Medium | Medium | High |
+| Failure mode explicitness | Low | Medium | Medium | High |
+
+### Recommendation: Hybrid D + A
+
+**D for Discovery->Shaping:**
+Coordinator branches on `selectedDirection` -- this is a routing decision, not just context enrichment. Mirrors the wr.review_verdict precedent. Schema overhead proportionate to decision weight.
+
+**A for all other pass-through transitions:**
+- Shaping->Coding: Phase 0.5 finds pitch.md automatically (confirmed in workflow prompt)
+- Coding->PR: PR number in goal string
+- PR->Review: pr-review.ts already handles this
+
+**No ContextAssembler change needed:**
+Coordinator passes `{ selectedDirection, designDocPath, assembledContextSummary }` as spawn context keys. The existing `context: Record<string, unknown>` parameter on `spawnSession` is sufficient.
+
+---
+
+## Self-Critique
+
+**Strongest counter-argument against D:**
+The workflow change to wr.discovery Phase 7 is a point of failure. If the agent emits a malformed artifact or omits it, coordinator falls back to notes parsing -- same fragility as Candidate A. The typed contract only helps when the artifact IS correctly emitted.
+
+**Why A is a legitimate simpler answer:**
+wr.shaping Step 1 explicitly reads "goal text, discovery notes, tickets, user stories." If coordinator goal string includes discovery summary, this may be sufficient. No structured routing on selectedDirection may be needed. Validate this when the first adaptive coordinator is written.
+
+**Pivot condition to A:**
+When the first adaptive coordinator is written, if wr.shaping Step 1 produces sound results without structured `selectedDirection` access, Candidate A is correct and D's overhead is unjustified.
+
+**Broader option:**
+D for all transitions (add wr.shaping_handoff, wr.coding_handoff). Justified only if coordinator needs to branch on pitch content or implementation scope. Not justified now.
+
+---
+
+## Open Questions for the Main Agent
+
+1. Does the coordinator need to branch on `selectedDirection` from discovery, or is passing the full notes blob sufficient for wr.shaping Step 1? This determines whether D is needed at all.
+
+2. Is there a planned use case where the adaptive coordinator skips Shaping based on discovery confidence band? If yes, D is clearly justified. If not, A may be sufficient.
+
+3. What does the routing agent (adaptive-coordinator-routing.md) expect from context passing? Its design may have specific requirements for structured fields at routing decision time.
+
+4. Is `.workrail/current-pitch.md` reliably found by Phase 0.5 in practice? If Phase 0.5 sometimes misses it, the coordinator may need to inject `pitchPath` explicitly.

package/docs/design/adaptive-coordinator-context-review.md

@@ -0,0 +1,101 @@
+# Design Review Findings: Adaptive Coordinator Context Passing
+
+*Review of selected direction: Hybrid D + A (wr.discovery_handoff artifact for D->S + goal string/notes for other transitions)*
+
+---
+
+## Tradeoff Review
+
+**Tradeoff 1: 1 workflow change to wr.discovery Phase 7**
+- Acceptable. Phase 7 already has all required context variables (`selectedDirection`, `designDocPath`, `recommendationConfidenceBand`) set by prior steps. Agent can construct the artifact reliably.
+- Safe with versioned workflows: fallback path (notes parsing) exists, so old versions degrade gracefully.
+
+**Tradeoff 2: Two-tier fallback complexity for D->S**
+- Acceptable. Exact same pattern as wr.review_verdict + parseFindingsFromNotes. Code structure is proven. No novel complexity.
+
+**Tradeoff 3: Agent must emit artifact or fallback activates**
+- Acceptable with monitoring. wr.review_verdict evidence shows agents reliably emit typed artifacts when explicitly instructed. Fallback logs a `[WARN]`, not silent.
+
+---
+
+## Failure Mode Review
+
+**FM1: Agent doesn't emit artifact**
+- Design handles it (fallback to notes). **Missing mitigation:** coordinator should check `lastStepNotes.trim().length > 50` before using notes as fallback. Without this check, an empty notes string causes shaping to start from scratch with no information.
+- Severity: YELLOW
+
+**FM2: Artifact emitted with wrong schema**
+- Handled (Zod rejects, WARN logged, fallback activates). Same as wr.review_verdict. No action needed.
+- Severity: GREEN
+
+**FM3: pitch.md not found by Phase 0.5 (Shaping->Coding)**
+- Phase 0.5 does active search but success is probabilistic. **Missing mitigation:** coordinator should pass `pitchPath: '.workrail/current-pitch.md'` explicitly in the Shaping->Coding spawn context. Cost: zero (trigger context injection already works). Without this, a missed Phase 0.5 search results in unnecessary design ideation.
+- Severity: ORANGE
+
+**FM4: Discovery session fails before Phase 7**
+- Not a design gap. Coordinator checks session outcome (`_tag: 'success'`) before reading artifacts. Standard pattern.
+- Severity: GREEN
+
+**FM5: Stale context**
+- Not a risk. Coordinator awaits discovery before reading results.
+- Severity: GREEN
+
+---
+
+## Runner-Up / Simpler Alternative Review
+
+**Runner-up (Candidate A only) strengths incorporated:**
+- Coordinator always composes a rich goal string using `selectedDirection` from the artifact (if present) or from notes parsing (fallback). Goal string is the first orientation signal; it must be rich regardless of other context injection.
+
+**Simpler variant (plain context key instead of Zod artifact) analysis:**
+- Collapsed back to Candidate A with a named key. Same silent failure mode (notes format changes break the value without error). D is justified only when coordinator needs a validated `selectedDirection` string. Simpler variant is NOT simpler for decision-critical routing.
+
+**Hybrid opportunity (implementation sequencing):**
+- Define schema + coordinator parsing FIRST; update wr.discovery Phase 7 SECOND (after confirming structured routing is needed in practice). This prevents a premature workflow change.
+
+---
+
+## Philosophy Alignment
+
+**Satisfied clearly:** Immutability, errors-as-data, dependency injection, compose-small-pure-functions, YAGNI
+
+**Under acceptable tension:**
+- "Make illegal states unrepresentable": fully satisfied for D-path (typed artifact); notes-based A-path is unconstrained. Acceptable because A-path transitions don't require coordinator routing decisions.
+- "Validate at boundaries": D-path uses Zod. A-path notes fallback is a weak boundary. Acceptable: notes are informational only.
+
+**Inherent limitation (not a design flaw):**
+- Type safety at LLM inference boundary: trigger context keys are read by LLM inference, not TypeScript. No compile-time guarantee on how the session interprets `selectedDirection`. Monitor via prompt engineering quality.
+
+---
+
+## Findings
+
+### RED (must fix before finalizing)
+- None
+
+### ORANGE (should fix -- meaningful risk if missed)
+- **O1: Missing pitchPath injection for S->C transition.** Coordinator should pass `pitchPath: '.workrail/current-pitch.md'` in the Shaping->Coding spawn context to guarantee Phase 0.5 finds the pitch even if file search misses it. Zero additional code complexity; just an extra key in the spawn context.
+
+### YELLOW (low risk, but worth noting)
+- **Y1: Missing notes length check in fallback.** Coordinator should check `lastStepNotes.trim().length > 50` before using notes as assembledContextSummary fallback. Prevents shaping from starting completely blind if discovery notes are empty.
+- **Y2: Implementation ordering.** Define schema and coordinator parsing code first, before updating the wr.discovery Phase 7 workflow. This avoids a premature workflow change and provides a validation checkpoint.
+- **Y3: Trigger context injection undocumented.** The fact that ALL trigger.context keys are injected as JSON in the initial prompt (`Trigger context: { ... }`) is an important coordinator-authoring invariant that must be documented in the design doc. It is currently only visible by reading workflow-runner.ts lines 3191-3193.
+
+---
+
+## Recommended Revisions
+
+1. **Add pitchPath injection** to the Shaping->Coding spawn context specification in the design doc
+2. **Add notes length check** to the coordinator implementation requirements
+3. **Document trigger context injection mechanism** explicitly in the design doc (it is how structured fields reach the downstream session)
+4. **State implementation ordering** explicitly in the design doc: schema + parsing code first, workflow change second
+
+---
+
+## Residual Concerns
+
+1. **LLM inference reliability** at the trigger context boundary: the downstream session must correctly interpret `selectedDirection` from the `Trigger context: { ... }` JSON block. Prompt engineering quality of the wr.shaping Step 1 prompt determines this. Not solvable at the coordinator design level; worth monitoring in early pipeline runs.
+
+2. **Two-tier fallback debt** for D->S: the fallback (notes parsing for Discovery->Shaping) is a weak boundary that may accumulate technical debt as wr.discovery evolves. Consider adding an integration test that checks the coordinator falls back gracefully when wr.discovery Phase 7 does not emit the artifact.
+
+3. **No contract for Coding->PR transition:** the design explicitly leaves this as "goal string with PR number." If the adaptive coordinator needs to know the PR number programmatically (to verify it was created correctly, to pass it to the review coordinator), this should be documented as a known gap rather than left implicit.