@ivorycanvas/qamap 0.4.1 → 0.4.3

package/CHANGELOG.md

@@ -2,6 +2,42 @@
 
 ## Unreleased
 
+## 0.4.3 - 2026-07-14
+
+### Added
+
+- Added evidence-ranked QA scenario routing. Each intent-backed scenario is now classified as `required`, `recommended`, or `review-only` from its priority, confidence, review requirement, and exact diff sources instead of treating every inferred path as equally actionable.
+- Added scenario-level automation receipts to human, JSON, and agent output. QAMap reports whether each selected scenario was `compiled`, `partial`, `not-compiled`, or `review-only`, including mapped step and assertion counts plus a concrete blocker when automation is incomplete.
+- Added benchmark contracts for selected-to-compiled coverage so public fixtures fail CI when required QA is proposed without a traceable automation handoff.
+
+### Changed
+
+- E2E readiness now includes required scenario compilation gaps. A syntactically valid draft cannot be promoted as runnable evidence when a required QA scenario was omitted or only partially mapped.
+- Playwright failure-path generation now requires an unchanged mockable endpoint boundary, an action selector related to the changed flow, and a repository-observed failure outcome before it emits executable setup, action, and assertion steps.
+- Contributor guidance now requires domain-neutral product rules, positive evidence combinations, negative controls, and unrelated fixture coverage before a specialized inference can enter the shared engine.
+
+### Fixed
+
+- Unrelated controls can no longer be reused merely because they are the first stable selector near a changed endpoint.
+- Maestro automation receipts now reflect commands and assertions that were actually generated instead of assuming coverage from the selected runner alone.
+
+## 0.4.2 - 2026-07-13
+
+### Added
+
+- Added benchmark contracts for draft readiness, runnable candidates, self-check passes, review-only files, TODO markers, and execution blockers. Public fixtures can now fail CI when QAMap finds the right flow but produces an unusable automation draft.
+- Added deterministic Playwright failure-path compilation when repository evidence exposes a mockable endpoint, a stable action control, and an observable failure message. Generated drafts replace the success route with a 4xx/5xx response, repeat the user action, and assert the failure UI instead of falling back to a body-visible smoke check.
+
+### Changed
+
+- Draft execution readiness now measures whether a generated file can be tried locally from runner, entrypoint, selector, fixture, and self-check evidence. Missing pre-existing validation coverage remains required PR guidance, but no longer falsely claims that the new file cannot execute.
+- A partial fixture match can qualify as a runnable candidate when QAMap generated executable mock scaffolding and every other execution check passes. It still remains a review item before the draft becomes trusted regression evidence.
+- Playwright self-checks now report body-only smoke assertions as domain-assertion warnings. Such files remain tryable, but cannot be called runnable candidates until they assert an observable product outcome.
+
+### Fixed
+
+- Draft reports no longer tell users to replace TODO locators when the generated files contain no TODO markers.
+
 ## 0.4.1 - 2026-07-13
 
 ### Added

package/README.md

@@ -4,6 +4,8 @@
 [![npm version](https://img.shields.io/npm/v/@ivorycanvas/qamap.svg)](https://www.npmjs.com/package/@ivorycanvas/qamap)
 [![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](LICENSE)
 
+![QAMap: local PR QA design from commits and diffs](docs/assets/qamap-cover.png)
+
 **A local, zero-LLM PR QA designer. QAMap turns commit intent and code diffs into evidence-backed behavior lifecycles, missing QA, and deterministic automation drafts. No cloud. No source upload. No token.**
 
 QAMap reads commit subjects and bodies, git changes, project structure, selectors, existing tests, and optional repo QA memory, then answers the question every reviewer asks: *"What behavior did this PR intend to change, how does that behavior move through the product, and what should we verify before merge?"* Playwright, Maestro, and manual checklists are output adapters after that judgment, not the product recommendation itself.
@@ -26,7 +28,7 @@ Run one read-only command on a branch. A manifest and test runner are not requir
 pnpm dlx @ivorycanvas/qamap qa . --base origin/main --head HEAD
 ```
 
-The report starts with the inferred behavior and keeps each proposed scenario connected to the commit or exact base/head diff line that caused it. Removed guards and validation remain visible as base-side evidence; each source is labeled `direct`, `supporting`, or `contextual` (trimmed from the committed lifecycle benchmark):
+The report starts with the inferred behavior and keeps each proposed scenario connected to the commit or exact base/head diff line that caused it. Removed guards and validation remain visible as base-side evidence; each source is labeled `direct`, `supporting`, or `contextual`. QAMap then shows two separate receipts: why the scenario was routed as `required`, `recommended`, or `review-only`, and whether the E2E adapter compiled it fully, partially, or not at all (trimmed from the committed lifecycle benchmark):
 
 ```txt
 Change intent: Submit notification preferences and show the saved state [high]
@@ -34,15 +36,21 @@ Behavior lifecycle: trigger -> state-change -> side-effect -> observable-outcome
 
 QA scenarios:
 - [critical] changed preference lifecycle (confidence: high)
+  - Routing: required — 3 supporting diff hunks
+  - E2E mapping: partial — steps 0/3, assertions 1/1
   - Source: src/pages/preferences.tsx:17, symbol onClick [supporting, head]
   - Source: src/pages/preferences.tsx:8, symbol setSaved [supporting, head]
   - Assert: the saved state becomes observable
 - [recommended] failure, timeout, and retry handling (confidence: medium; review required)
+  - Routing: recommended — 1 supporting diff hunk
+  - E2E mapping: not compiled — no deterministic failure compiler matched all required evidence
   - Source: src/pages/preferences.tsx:7, symbol fetch [supporting, head]
   - Assert: retries do not duplicate requests or side effects
 
 Optional automation:
 - Adapter candidate: Playwright
+- Scenario routing: 2 required, 2 recommended, 0 review-only
+- E2E mapping: 0 compiled, 1 partial, 3 not compiled
 - qamap e2e draft . --base origin/main --head HEAD
 ```
 
@@ -76,6 +84,7 @@ One minified JSON object (`schema: qamap.qa`) with change intents, lifecycle sta
 - **Judgment first, generation second.** Deciding *what deserves testing* for a given change is the missing layer. QAMap makes that judgment statically, deterministically, and locally for free.
 - **The repo remembers.** Team QA knowledge lives in `.qamap/manifest.yaml`, reviewed once and reused on every PR — instead of re-prompting an agent each session.
 - **Honest output.** Drafts state what blocks them from being trusted; changed endpoints are observed, never mocked away; generated specs never assert what cannot pass. Configuration, docs, generated artifacts, and changed tests stay in verification mode instead of fabricating a product-journey E2E.
+- **Selected is not compiled.** Every evidence-routed scenario keeps a static automation receipt. Required behavior that remains partial or not compiled lowers readiness instead of being hidden behind a smoke test. `compiled` still does not claim the application was executed.
 
 Positioning against recorders, LLM test generation, and impact-analysis tools: [where QAMap fits](docs/adoption.md#where-qamap-fits).
 

package/dist/e2e.d.ts

@@ -8,6 +8,7 @@ import type { LocalHistoryReference } from "./history.js";
 import type { CoverageEvidence, TestSuiteSummary } from "./test-evidence.js";
 import type { BehaviorGraph } from "./behavior.js";
 import type { BehaviorLifecycleStage, ChangeIntentAnalysis, ChangeIntentConfidence, ChangeIntentEvidence, IntentQaScenario } from "./change-intent.js";
+import type { QaScenarioDecision } from "./scenario-routing.js";
 export type E2eProjectType = "expo-react-native" | "react-native" | "web" | "api-service" | "design-tokens" | "data-catalog" | "cli" | "unknown";
 export type E2eRunnerName = "maestro" | "playwright" | "manual";
 export type E2eFlowKind = "ui" | "api" | "state" | "content" | "config" | "test-evidence" | "documentation" | "generated-artifact" | "artifact" | "catalog" | "command" | "domain" | "changed-file";
@@ -247,6 +248,23 @@ export interface E2eDraftFile {
     intentConfidence?: ChangeIntentConfidence;
     lifecycle?: BehaviorLifecycleStage[];
     qaScenarios?: IntentQaScenario[];
+    scenarioAutomation?: E2eScenarioAutomationReceipt[];
+}
+export type E2eScenarioAutomationStatus = "compiled" | "partial" | "not-compiled" | "review-only";
+export interface E2eScenarioAutomationReceipt {
+    scenarioId: string;
+    title: string;
+    kind: IntentQaScenario["kind"];
+    priority: IntentQaScenario["priority"];
+    decision: QaScenarioDecision;
+    status: E2eScenarioAutomationStatus;
+    requiredSourceCount: number;
+    referenceSourceCount: number;
+    mappedSteps: number;
+    totalSteps: number;
+    mappedAssertions: number;
+    totalAssertions: number;
+    blockers: string[];
 }
 export type E2eDraftActionKind = "assertion" | "fixture" | "manifest" | "runner" | "selector" | "setup" | "validation";
 export type E2eDraftActionPriority = "required" | "recommended";
@@ -299,6 +317,13 @@ export interface E2eDraftReadinessSummary {
     totalTodos: number;
     filesWithExecutionBlockers: number;
     totalExecutionBlockers: number;
+    requiredScenarios: number;
+    recommendedScenarios: number;
+    reviewOnlyScenarios: number;
+    compiledScenarios: number;
+    partialScenarios: number;
+    notCompiledScenarios: number;
+    requiredScenarioGaps: number;
     topBlockers: string[];
 }
 export declare function formatDraftReadinessStage(summary: Pick<E2eDraftReadinessSummary, "level" | "score">): string;