valent-pipeline 0.5.3 → 0.5.4

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "valent-pipeline",
-  "version": "0.5.3",
+  "version": "0.5.4",
   "description": "v3 multi-agent AI pipeline for software development lifecycle",
   "type": "module",
   "bin": {

package/pipeline/prompts/qa-a.md

@@ -73,6 +73,7 @@ Before finalizing, verify:
 - No behavior is spec'd at multiple tiers
 - Every error test case has a specific error code and message pattern
 - Every P0 test case has DB state verification
+- Every P0/P1 AC on an interactive surface (`api`, `ui`) has a human-readable acceptance scenario realized as a Live Smoke Test (api) or UI Integration Smoke Test (ui) row for QA-B to execute against live infrastructure — never left to unit tests alone
 - Seed data uses factory patterns, not raw SQL
 - NFR requirements are tagged ([NFR-PERF], [NFR-SEC], [NFR-REL])
 - Visual validation checkpoints cover all 5 page states for each page (if UI story)

package/pipeline/scripts/query-kb.ts

@@ -78,10 +78,16 @@ switch (mode) {
 
   case 'directives': {
     const agent = flags['agent'];
+    // ALL-DEV is a cross-cutting directive that applies to every developer agent. Include it when
+    // the queried agent is one of them, so a per-agent query still sees shared dev directives.
+    const DEV_AGENTS = ['BEND', 'FEND', 'DATA', 'MCP-DEV', 'LIBDEV', 'DOCGEN', 'IAC', 'MOBILE'];
     let rows;
     if (agent) {
+      const includeAllDev = DEV_AGENTS.includes(agent.toUpperCase());
       rows = db.prepare(
-        "SELECT id, directive, reason FROM correction_directives WHERE target_agent = ? AND status = 'active'"
+        includeAllDev
+          ? "SELECT id, target_agent, directive, reason FROM correction_directives WHERE (target_agent = ? OR target_agent = 'ALL-DEV') AND status = 'active'"
+          : "SELECT id, directive, reason FROM correction_directives WHERE target_agent = ? AND status = 'active'"
       ).all(agent) as { id: string; directive: string; reason: string }[];
     } else {
       rows = db.prepare(

package/pipeline/steps/common/agent-protocol.md

@@ -42,7 +42,7 @@ When `signal_delivery` is `thread`: Lead steers you with the Design Council ques
 When you need information about project conventions, architectural patterns, existing code structure, or known pitfalls: self-serve from the knowledge base before exploring the codebase directly. Curated knowledge and correction directives answer in seconds what codebase exploration takes minutes to discover.
 
 **How to self-serve:**
-1. Read correction directives from `{correction_directives}` — filter for `status: active` entries targeting your agent role.
+1. Read correction directives from `{correction_directives}` — filter for `status: active` entries whose `target_agent` is your agent role (or `ALL-DEV` if you are a developer agent).
 2. Read curated knowledge files in `{curated_files_path}` — scan file names and section headers for entries relevant to your current task.
 3. If `{knowledge_mode}` is `sqlite`: query the database via CLI (see your read-inputs step for commands).
 4. If `{story_output_dir}/knowledge-context.md` exists: read it directly — this is a pre-compiled reference containing all relevant knowledge for the current story.
@@ -51,7 +51,7 @@ Reserve direct codebase exploration for when curated knowledge does not have the
 
 ## Correction Directives
 
-Read active correction directives from `{correction_directives}`. If the file does not exist or is empty, proceed without directives -- this is expected for new pipelines. Apply ALL directives targeting your agent role. If a directive conflicts with these instructions, the directive takes precedence. Log each applied directive in your YAML frontmatter under `correctionsApplied`.
+Read active correction directives from `{correction_directives}`. If the file does not exist or is empty, proceed without directives -- this is expected for new pipelines. Apply ALL directives whose `target_agent` matches your agent role **OR** is `ALL-DEV` when you are a developer agent (BEND, FEND, DATA, MCP-DEV, LIBDEV, DOCGEN, IAC, MOBILE) — `ALL-DEV` is a cross-cutting directive that applies to every developer agent, not one surface. If a directive conflicts with these instructions, the directive takes precedence. Log each applied directive in your YAML frontmatter under `correctionsApplied`.
 
 ## YAML Frontmatter
 

package/pipeline/steps/common/quality-standards.md

@@ -10,8 +10,22 @@ These are non-negotiable. CRITIC and QA-B enforce them. Every developer agent (B
 - **<1.5 minutes per test** -- any test exceeding this is a design problem, not a timeout problem.
 - **Self-cleaning via fixture auto-teardown** -- tests must not leave state behind. Use framework teardown hooks, not manual cleanup.
 - **Explicit assertions in test bodies** -- never hide assertions in helpers. Every test body must contain at least one visible `expect`/`assert`.
+- **Falsifiable assertions** -- every assertion must be able to FAIL when the behavior is wrong. No unfalsifiable disjunctive gates like `expect(a === 0 || b === 0 || fallback).toBe(true)` — a disjunction with a catch-all that is almost always true asserts nothing. Assert the real invariant directly (e.g. "exactly one container exists AND `SELECT 1` succeeds AND no corruption log"), not a condition that passes regardless of outcome.
+- **Bind to the real artifact under test** -- resolve the actual artifact dynamically (built image id, running service/container name, the file the build emits) and assert against THAT. Never assert against a hardcoded tag/name or an artifact that merely happens to exist on your machine. The test: would it still pass on a clean/CI checkout with no leftover state? If it passes only because of an ambient/side-channel artifact, it is broken. (Prove the artifact exists — `expect(imageId).toBeTruthy()` — before inspecting it.)
+- **Statistical assertions match the spec** -- when the spec states a percentile or sample count (e.g. NFR-PERF "P95 < 200ms"), assert exactly that over real samples with warm-up where appropriate. A single-shot `elapsed < 200` is not a P95 and is flaky — it does not satisfy the spec.
 - **Parallel-safe** -- no shared mutable state between tests. Must run cleanly with `--workers=4`.
 
+## Pre-Handoff Self-Check — MANDATORY before declaring done
+
+CRITIC enforces every item below and will reject on any High finding, costing a full rework cycle. Run this checklist against your own tests BEFORE writing your handoff; fix anything that fails first. (Each recurred across real stories — this is the gap that buys a guaranteed rework cycle if skipped.)
+
+- [ ] **Every P0 qa-test-spec case has a named, implemented test.** Count the spec's P0 cases; count your `it(...)`/`test(...)` blocks; a P0 case with no implementation is a High finding. Do not treat a spec note (e.g. "idempotency") as covered unless there is a named test for it.
+- [ ] **No `if`/`switch`/ternary in any test body** (same path every run).
+- [ ] **No unfalsifiable assertions** — no disjunctive/catch-all gates; each assert can fail.
+- [ ] **Tests bind to the real artifact** — no hardcoded tag/name; would pass on a clean/CI machine with no leftover state.
+- [ ] **Statistical/perf assertions match the spec's wording** (percentile + samples, not single-shot).
+- [ ] **No assertion weakening, no infra mocking on happy paths, no hard waits, assertions visible in test bodies.**
+
 ## Live Infrastructure Standards
 
 - **Live tests against running infrastructure** -- tests hit real systems. No mocking databases, APIs, pipelines, servers, or external services for happy-path verification.

package/pipeline/steps/critic/test-review.md

@@ -21,6 +21,9 @@
 - **No test deletion** -- all qa-test-spec test cases must have corresponding tests. A missing test is a High finding.
 - **No hard waits** -- `sleep()`, `setTimeout()`, `page.waitForTimeout()` in tests is a High finding.
 - **No conditionals** -- `if` statements in test bodies is a High finding.
+- **Falsifiable assertions** -- a disjunctive/catch-all gate that is almost always true (e.g. `expect(a === 0 || b === 0 || transient).toBe(true)`) asserts nothing. A test that cannot fail when the behavior is wrong is a High finding. Require the real invariant.
+- **Artifact binding** -- a test that asserts against a hardcoded tag/name, or only passes because of an ambient/leftover artifact on the dev's machine (would fail on a clean/CI checkout), is a High finding. The artifact under test must be resolved dynamically and proven to exist before inspection.
+- **Statistical assertions match spec** -- when the spec states a percentile/sample count (NFR-PERF "P95 < Nms"), a single-shot threshold instead is a Med finding (the assertion does not match the spec).
 - **Explicit assertions** -- assertions hidden inside helper functions is a Med finding. Every test body must contain visible `expect`/`assert`.
 
 ## Output

package/pipeline/steps/qa-a/write-spec.md

@@ -31,6 +31,17 @@ Structure per AC:
 - **Edge cases** (P0: required, P1: key boundaries, P2: optional, P3: omit)
 - **Concurrency** (P0: required, P1: if applicable, P2-P3: omit)
 
+### The Given-When-Then cases are human-readable acceptance scenarios — QA-B executes them
+
+Your Given-When-Then cases are not just developer test stubs; they are the **human-readable acceptance contract** for the story, written in plain user/consumer language (what a person does and observes), independent of implementation. They describe behavior the way a real developer or QA would verify it by hand.
+
+For **interactive surfaces this is mandatory and first-class** — it is how the pipeline tests like a real-world human developer, not just by trusting unit tests:
+
+- **`api` profile →** realize each acceptance scenario as a row in the **Live Smoke Tests** table (see `api.md`): real HTTP method/URL/body → expected status + response, plus a verification request for every mutation. QA-B starts the real server and drives these with real requests.
+- **`ui` profile →** realize each user-facing scenario as a row in the **UI Integration Smoke Tests** table (see `ui.md`): user action → real API call → expected UI state → DB verification, backed by a real-browser E2E test. QA-B runs these against the live UI + API + DB (and PMCP validates the visual checkpoints).
+
+These acceptance scenarios are **owned and executed by QA-B against live infrastructure** — they are the real-world, black-box evidence JUDGE relies on, distinct from (and never replaced by) the developer's own unit tests. Every P0/P1 AC on an interactive surface MUST have one. The Step 9b quality bar below governs the *automated test code*; it complements these acceptance scenarios — it does not replace them.
+
 ## Step 4: Database State Verification
 
 Per-risk DB verification:
@@ -72,6 +83,15 @@ For each NFR-sensitive path: `[NFR-PERF]` response time + load patterns; `[NFR-S
 - If `ui` in `{testing_profiles}` → **MANDATORY.** Read `uxa-spec.md`. If `uxa-spec.md` is missing, send `[BLOCKER]` to Lead — do NOT proceed without it. For each page state define: Checkpoint ID (VV-{NNN}), Page/Route, State (Default/Loading/Empty/Error/Success or custom), AC Reference, Area labels in scope, Screenshot filename (`{story_id}_VV-{NNN}_{page}_{state}.png`), Expected visual elements, Setup instructions, Pass criteria. Write to `{story_output_dir}/visual-validation-checklist.md`.
 - If `ui` NOT in `{testing_profiles}` → skip, note "N/A — no UI profile."
 
+## Step 9b: Test Quality Bar — make every case implementable AND falsifiable
+
+The dev agents implement exactly what you specify, and CRITIC rejects tests that are weak, unfalsifiable, or bound to the wrong artifact. Spec the bar IN so it is built right the first time (each rule below traces to a real rework cycle):
+
+- **Every P0 case is a named, must-implement test** — never leave a P0 (or an idempotency/concurrency requirement for stateful resources) as a prose note. If it matters, give it its own case ID and assertion target so the dev agent implements it directly. For stateful resources (volumes, DBs), enumerate an explicit idempotency case (apply twice → assert no-recreate / no re-init / data intact).
+- **Falsifiable assertion target** — state the real invariant and the exact expected value (status code, row/column value, count, error code + message regex). Forbid disjunctive/catch-all expectations: never spec "passes if A or B or fallback." The assertion must be able to fail when the behavior is wrong.
+- **Bind to the real artifact** — when a case inspects a built artifact (image, container, file), specify that the test resolves it dynamically (the compose-built image id, the actual service/container name) and proves it exists before inspecting it — never a hardcoded tag. Note it must pass on a clean/CI machine with no leftover state.
+- **NFR-PERF wording is statistical** — specify the percentile AND sample count + warm-up (e.g. "P95 < 200ms over 20 sequential samples after one warm-up request"), not a single-shot threshold.
+
 ## Step 10: Write Final Outputs
 
 - Write to `{story_output_dir}/qa-test-spec.md` and `{story_output_dir}/visual-validation-checklist.md` (if applicable)

package/pipeline/steps/retrospective/directives.md

@@ -12,13 +12,18 @@ If pattern touches an invariant, do NOT write a CD. Instead:
 
 For patterns NOT conflicting with invariants:
 
+**Scope the directive to the failure mode, not the agent that tripped it.** Before setting `target_agent`, ask: *could another agent hit this same pattern?* Many failure modes are cross-cutting — test quality (missing P0 cases, conditionals/unfalsifiable assertions in tests, artifact-binding, hard waits), handoff-format violations, infra-mocking — and apply to **every** developer agent, not just the one that happened to surface it this sprint. For those:
+- Set `target_agent: ALL-DEV` (BEND, FEND, DATA, MCP-DEV, LIBDEV, DOCGEN, IAC, MOBILE) so the lesson generalizes; do NOT pin a cross-cutting test-quality directive to the single agent that tripped it.
+- Prefer reinforcing the shared contract: if the pattern is a standing rule, the durable fix is `.valent-pipeline/steps/common/quality-standards.md` (the dev self-check) or the QA-A spec / CRITIC checklist — note that in the directive's `reason`. A per-agent directive that restates a shared standard is noise.
+- Only use a single `target_agent` when the pattern is genuinely specific to that agent's surface (e.g. an IAC-only compose idiom, a FEND-only component pattern).
+
 **ADD** new directives:
 ```yaml
 - id: CD-{batch_number}-{seq}
   status: active
-  target_agent: {agent}
+  target_agent: {agent | ALL-DEV}
   directive: "{what to do differently}"
-  reason: "{evidence from batch}"
+  reason: "{evidence from batch; if it restates a shared standard, name the file it belongs in}"
   impact_level: low | medium | high
   created_batch: {batch_number}
   last_reinforced_batch: {batch_number}