@ihyeon/flow-qa 0.2.9 → 0.4.0

package/docs/architecture.md

@@ -22,7 +22,9 @@ flowchart LR
 
 **Only a defined vocabulary compiles.** The schema checks requirement references, assertions, step types, and inputs. It does not execute arbitrary code or shell commands from model output. Cases with unresolved `questions`, or steps that need an unavailable agent model, are blocked. `preconditions` describe required state; they are not an automatic setup contract.
 
-**Source and evidence stay linked.** Planning records the source flow's SHA-256 hash in the suite. `validate`, `generate`, and `run` check it again and stop when the flow changed. Each execution writes a new directory with a suite snapshot, generated tests, raw engine output, and Markdown/JSON reports. Previous evidence is not reused as a current success. `compare` permits a direct comparison only for the same suite hash and target identity (web URL or native platform/app/device); it cannot establish that external data was unchanged.
+**Source and evidence stay linked.** Planning records the source flow's SHA-256 hash in the suite. `validate`, `generate`, and `run` check it again and stop when the flow changed. Each execution writes a new directory with a suite snapshot, generated tests, raw engine output, and Markdown/JSON reports. Previous evidence is not reused as a current success. CLI `compare` permits a direct comparison only for the same suite ID, suite hash, case selection, and target identity (web URL or native platform/app/device); it cannot establish that external data was unchanged.
+
+**Selection changes execution scope, not source identity.** The full suite and source are validated and the full original snapshot/hash remain in each run directory. Selected cases and their referenced requirements form the executable subset. Partial reports carry explicit selection metadata and contain only selected results, including blocked, error, and interrupted results. UI comparisons flag partial/differing scopes; missing results never imply a pass. Selection changes require fresh UI review and cannot bypass the target or file fingerprint checks.
 
 **Targets can be configured after authoring.** A suite's optional platform distinguishes web from mobile. Omitted platform preserves legacy web suites and their hashes. Native app IDs and explicit device names belong to target settings, not generated code from the planner. UI execution checks a fingerprint of the reviewed target identity. Mobile runtime readiness is checked before device execution; configuration alone does not prove connectivity.
 

package/docs/mobile.md

@@ -9,7 +9,7 @@ Choose **Mobile** on the scenario screen, describe the flow, and generate or dow
 With the installed CLI:
 
 ```sh
-npm install -g @ihyeon/flow-qa@0.2.9
+npm install -g @ihyeon/flow-qa@0.4.0
 flow-qa init ./my-qa
 # Write your mobile flow in my-qa/flows/my-flow.md first.
 flow-qa plan flows/my-flow.md --platform mobile --prompt-only --workspace ./my-qa

package/docs/releases.md

@@ -2,6 +2,28 @@
 
 The alpha entries below are historical release records. They document earlier package versions and checks.
 
+## 0.4.0: run selected cases
+
+Adds case selection to the execution panel and repeated `--case <id>` flags to CLI runs. Selecting or clearing cases resets UI review, and execution still requires the reviewed file and target fingerprints. Unknown, duplicate, and empty selections are rejected. Selection is preserved across navigation, language changes, and polling, while a new draft starts with all cases selected.
+
+Each run retains the full original suite snapshot/hash and verifies its source, while compiling and assessing only the chosen cases. Partial reports add explicit `selection` metadata to `flow-qa/report-1`; unselected cases are neither executed nor reported as passed/skipped. Selected blocked/error/interrupted cases retain their normal meaning. History, report downloads, and comparisons display the partial scope; comparisons warn when selected case sets differ. Existing full runs and their report shape are preserved.
+
+Locally verified on macOS with Node.js 24.22.1 on 2026-11-03: 115 tests, type checking, UI syntax and design-system audit, and existing demo/UI checks passed. The dedicated Chromium smoke ran two selected cases successfully, confirmed zero requests from an excluded sentinel case, and kept a selected unresolved-question case blocked without attempts. It covered review invalidation, bulk selection, navigation/language preservation, 390px layout, original snapshot/hash preservation, and report/comparison scope. A fresh tarball install executed two full runs plus a one-case partial run (seven passing browser cases) and compared their reports. No model calls or native-device execution were used.
+
+- Package: [`@ihyeon/flow-qa@0.4.0`](https://www.npmjs.com/package/@ihyeon/flow-qa/v/0.4.0)
+- Release: [GitHub 0.4.0](https://github.com/hyeonse-swing/flow-qa/releases/tag/v0.4.0)
+
+## 0.3.0: compare recorded runs
+
+Adds a two-run comparison workspace to Run history. Select a baseline and current report, swap their direction, filter by case transition, open either detailed report, and export the complete comparison as Markdown or JSON. Cases are matched by ID, with distinct categories for passed-to-failed, failed-to-passed, other changes, equal statuses, added, and missing. Flaky, blocked, skipped, and execution-error results retain their meaning.
+
+The read-only comparison endpoint requires the same suite ID and recorded target settings, rejects ambiguous case IDs, and uses the existing safe report loader. Suite changes, reversed dates, and interrupted/error runs are explicitly flagged. Mobile comparisons also warn that recorded device selectors do not establish physical device identity. Comparisons describe saved outcomes, not proof of a regression or fixed defect. Shared Markdown hides URLs and links under the existing report policy; JSON retains recorded metadata. Selections remain across navigation, language changes, and history updates; selection changes or disappeared runs invalidate old exports, and late responses cannot replace a newer comparison. The existing report format is unchanged.
+
+Locally verified on macOS with Node.js 24.22.1 on 2026-11-02: 101 tests, type checking, UI syntax and design-system audit, existing browser/demo checks, and comparison UI checks using synthetic report fixtures. All 36 terminal-status combinations, compatibility gates, URL/link hiding, English/Korean exports, filtering, polling, navigation, 390px layout, late responses, and stale downloads were covered. A fresh tarball installation ran two real Chromium demo runs (six passing cases) and compared their saved reports. No model calls or native-device execution were used.
+
+- Package: [`@ihyeon/flow-qa@0.3.0`](https://www.npmjs.com/package/@ihyeon/flow-qa/v/0.3.0)
+- Release: [GitHub 0.3.0](https://github.com/hyeonse-swing/flow-qa/releases/tag/v0.3.0)
+
 ## 0.2.9: search run history
 
 Adds Search runs to run history, matching suite IDs, run IDs, and target names with case-insensitive partial search. Surrounding whitespace is ignored, and the search combines with the existing status filter. Clearing the query preserves the selected status. The current query stays through navigation, language changes, and automatic history updates; open reports and downloads remain available. Search covers only the most recent 50 reports already loaded by the UI.
@@ -100,7 +122,7 @@ Locally verified on macOS with Node.js 24.14.1 on 2026-10-12: 89 tests, type che
 
 A push to `main` publishes only when `package.json` changes to a new plain `X.Y.Z` version. Pull requests and pushes without a version change run verification without publishing. The package and lockfile versions must agree; the version must increase, and registry errors stop the release. Rerunning a successful publication skips the existing immutable npm version.
 
-The `publish` job in [ci.yml](../.github/workflows/ci.yml) waits for the full verification matrix on Ubuntu and macOS, then publishes through npm trusted publishing (OIDC). It uses the `latest` tag, requires the registry's `gitHead` to match the workflow commit, and checks the published package with a fresh consumer installation and three real Chromium demo cases. The commit check also applies when retrying an already published version. No npm token is stored in GitHub secrets. After the published package passes its consumer smoke check, a separate job creates the matching GitHub tag and release at the exact verified commit. It preserves existing releases and rejects a tag pointing to another commit. Rerunning an older release does not replace a newer latest release.
+The `publish` job in [ci.yml](../.github/workflows/ci.yml) waits for the full verification matrix on Ubuntu and macOS, then publishes through npm trusted publishing (OIDC). It uses the `latest` tag, requires the registry's `gitHead` to match the workflow commit, and checks the published package with a fresh consumer installation, two full Chromium demo runs plus a one-case selected run (seven cases), and comparison of their saved reports. The commit check also applies when retrying an already published version. No npm token is stored in GitHub secrets. After the published package passes its consumer smoke check, a separate job creates the matching GitHub tag and release at the exact verified commit. It preserves existing releases and rejects a tag pointing to another commit. Rerunning an older release does not replace a newer latest release.
 
 For the next release, update the release notes and current installation examples, then bump the version before committing and pushing:
 

package/docs/ui.md

@@ -7,7 +7,7 @@ Describe a user flow, review cases, and configure a target when ready to run. We
 Node.js 24.x (24.22.1 or later) or 26.12.1 or later is required. Node 25 is unsupported. Upgrade the Node runtime and restart any running UI process; updating Flow QA alone does not patch Node. Install Flow QA and start the web demo:
 
 ```sh
-npm install -g @ihyeon/flow-qa@0.2.9
+npm install -g @ihyeon/flow-qa@0.4.0
 flow-qa browser install
 flow-qa ui --demo
 ```
@@ -46,6 +46,16 @@ From version `0.2.2`, the UI requests the browser's leave-page confirmation when
 
 From version `0.2.4`, loading a suite from the library or importing valid case files also asks before replacing unsaved scenario or case content. **New JSON draft** asks before clearing unsaved JSON and keeps the scenario text. Cancel keeps the current content and selected import files. Unchanged saved cases and ordinary navigation between screens do not prompt; invalid imports still show their validation error without replacing anything.
 
+### Run selected cases
+
+From version `0.4.0`, **Scenario → Review, then run → Cases to run** lists the validated suite's cases. Initially all are selected. Use individual checkboxes, **Select all**, or **Clear selection** to choose a nonempty set. A changed selection clears the review confirmation; choose the target, inspect the selected cases, and confirm review again before execution. Selection controls are disabled during a run or another busy operation. Navigation, language changes, and polling keep the selection. Loading/importing/validating a new draft resets it to all cases. It is not saved across page reloads.
+
+The selected cases alone determine runnable/blocked counts and the run result. A selected case with unresolved questions or an unavailable required model stays blocked. Unselected cases do not execute and are not reported as passed or skipped. Flow QA does not infer prerequisites: each selected case must establish its own starting state. The source flow and full saved suite still pass existing validation and review-hash checks. Selection does not edit the saved file; the script preview and script/case downloads continue to include the full suite.
+
+The run directory keeps the complete original `suite.json` and its hash. A partial `qa-report.json` retains `flow-qa/report-1` and adds `selection: { caseIds, totalCases }`; its `cases` list contains exactly the selected IDs in original suite order. A full run, including explicitly selecting every case, omits this field as before. History, detailed reports, and Markdown exports show selected/total counts and explain that omitted cases were not assessed. Comparison exports retain each run's scope, warn on partial runs and differing selections, and describe absent cases as missing results rather than recovered defects. Invalid selection metadata is not accepted as a valid report.
+
+For the CLI, repeat `--case`: `flow-qa run cases/task-flow.json --target demo --case TC-001 --case TC-003`. Unknown, duplicate, or empty IDs are rejected before execution. The command keeps existing exit semantics: a partial pass is exit 0, a failure/block is exit 1, interrupted is 130, and invalid input is 2. If CI requires complete-suite coverage, omit `--case` and check that report selection metadata is absent.
+
 ### Expand or collapse case review
 
 From version `0.2.8`, **Expand all cases** and **Collapse all cases** control the case details in Scenario. Individual case titles still toggle their own details. Language changes and navigation preserve which cases are open; loading a new suite keeps the original first-case-open default. These controls only change the view: they do not approve cases, modify the draft, or change the JSON editor disclosure. Review confirmation is still required before execution.
@@ -62,6 +72,16 @@ From version `0.2.7`, **Run history → Run status** filters the loaded reports
 
 From version `0.2.9`, **Run history → Search runs** matches part of a suite ID, run ID, or target name. Matching ignores letter case and surrounding whitespace, and supports Korean target names. Search combines with **Run status**; clear the search to return to the selected status. The query stays through navigation, language changes, and automatic report updates, and resets on page reload. Search covers the most recent 50 reports loaded by the UI, not older records, target URLs, or individual case content. Changing the query keeps the open report and its downloads available.
 
+### Compare two runs
+
+From version `0.3.0`, open **Run history → Compare two runs**, choose a **Baseline run** and **Current run**, then compare. Both selectors use the most recent 50 loaded reports, independently of the history search and status filter. The reports must belong to the same suite ID and recorded target settings: a matching web URL, or mobile platform, app ID, and device. Target display names may differ. Mobile reports record the configured device selector, which can be a name; all mobile comparisons warn that matching selectors do not establish that the same physical device was used. Comparing does not run tests, call a model, or change case review.
+
+Cases are matched by ID. **Newly failed** means only `passed → failed`; **Now passed** means only `failed → passed`. Other status transitions, including flaky, blocked, skipped, and execution errors, appear under **Changed status**. Equal statuses are **Same status**. **Added to current** and **Missing from current** mean present in only one selected report; missing does not mean passed or deleted from the suite. Filter the visible rows by category, open either detailed report for its evidence, or download the complete comparison as Markdown or JSON. Downloads include every category, regardless of the display filter, and do not include evidence files. Markdown follows the existing report URL/link hiding policy; JSON retains the recorded target settings and case text.
+
+When suite hashes differ, the UI and exports warn that steps or expectations may have changed. These are recorded outcomes, not proof of equivalent tests, a product regression, or a fixed defect. Additional warnings identify reversed chronological order and interrupted/error runs. The comparison never treats a flaky pass as a clean pass. The existing report format remains unchanged; comparison JSON uses `flow-qa/comparison-1`.
+
+Swap reverses the baseline and current selections. Changing either selection clears the old result and downloads until a new comparison succeeds. Navigation, language changes, and history updates preserve valid selections and results; a selected run disappearing from the loaded list invalidates the comparison. Choices reset on page reload. Markdown product labels follow the selected language, while user-written case content and status codes are preserved.
+
 ### Import existing case files
 
 From version `0.2.1`, select a `flow-qa/suite-1` JSON file and, when it declares a source, its Markdown file in case review, then choose **Import case files**. The two file controls let you select files from different folders. For an error-to-test export, select `cases/error-to-test.json` and `flows/error-to-test.md`. A JSON file without a source can be imported alone or with a Markdown file.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@ihyeon/flow-qa",
-  "version": "0.2.9",
+  "version": "0.4.0",
   "license": "MIT",
   "author": "Hyeonse Im",
   "repository": {
@@ -57,7 +57,7 @@
     "check:ui": "node --check src/ui/public/app.js && node --check src/ui/public/display.js && node --check src/ui/public/i18n.js && hds-audit .",
     "demo:serve": "node examples/demo/server.ts",
     "demo:smoke": "node scripts/demo-smoke.ts",
-    "smoke:ui": "node scripts/ui-i18n-smoke.mjs"
+    "smoke:ui": "node scripts/ui-i18n-smoke.mjs && node scripts/ui-comparison-smoke.mjs && node scripts/ui-selection-smoke.mjs"
   },
   "dependencies": {
     "@ai-sdk/openai-compatible": "3.0.53",