@ihyeon/flow-qa 0.2.9 → 0.4.0

package/README.ko.md

@@ -16,7 +16,7 @@ CLI와 `127.0.0.1` 전용 웹 UI를 제공합니다. UI는 영어로 시작하
 Node.js 24.x(24.22.1 이상) 또는 26.12.1 이상이 필요합니다. 브라우저는 첫 웹 실행 전에 설치합니다.
 
 ```sh
-npm install -g @ihyeon/flow-qa@0.2.9
+npm install -g @ihyeon/flow-qa@0.4.0
 flow-qa browser install
 flow-qa ui --demo
 ```
@@ -62,6 +62,16 @@ flow-qa plan flows/my-flow.md --workspace my-qa
 
 `flow-qa doctor --workspace my-qa`로 로컬 의존성과 모델 인증용 환경 변수의 설정 여부를 확인할 수 있습니다. 키 값은 출력하지 않고 모델 서버도 호출하지 않습니다. 환경 변수가 있어도 실제 인증 성공을 확인한 것은 아닙니다.
 
+## 선택한 케이스만 실행
+
+`0.4.0`부터 **시나리오 → 검토 후 실행 → 실행할 케이스**에서 필요한 케이스만 고를 수 있습니다. 처음에는 모두 선택됩니다. 선택을 바꾸면 검토 확인을 다시 해야 하며, 저장한 원본 케이스 파일은 그대로 유지됩니다. 다른 케이스를 준비 단계로 자동 실행하지 않으므로 각 케이스에 필요한 시작 상태를 직접 구성하세요.
+
+CLI에서는 `flow-qa run cases/task-flow.json --target demo --case TC-001 --case TC-003`처럼 ID를 반복합니다. `--case`를 생략하면 전체를 실행합니다. 부분 실행의 통과는 선택한 케이스에만 해당하며, 보고서와 비교 화면에도 선택 범위를 표시합니다. 자세한 동작은 [UI 가이드](docs/ui.md#run-selected-cases)를 참고하세요.
+
+## 실행 결과 비교
+
+**실행 기록 → 실행 2건 비교**에서 같은 스위트와 대상의 두 기록을 선택하면 통과→실패, 실패→통과, 그 외 상태 변경, 동일 상태, 추가·누락 케이스를 확인할 수 있습니다. 분류별로 보거나 전체 비교를 Markdown·JSON으로 내려받으세요. 테스트 내용 변경과 불완전한 실행은 별도로 표시하며, 상태 변화만으로 제품 버그의 발생·해결을 확정하지 않습니다. 자세한 동작은 [UI 가이드](docs/ui.md#compare-two-runs)를 참고하세요.
+
 ## 모바일 대상
 
 `0.2.0`부터 사용자가 직접 설정하는 iOS·Android 대상을 지원합니다. 시나리오 화면에서 **모바일**을 선택하면 앱을 정하기 전에도 케이스를 준비할 수 있습니다. 이후 **연결 설정**에서 플랫폼·설치된 앱의 번들/패키지 ID·기기를 등록하고 실행할 때 선택합니다. 설정 저장은 앱 설치나 테스트 실행을 시작하지 않습니다.

package/README.md

@@ -16,7 +16,7 @@ Flow QA is available as a CLI and a web UI bound to `127.0.0.1`. Provide a targe
 Node.js 24.x (24.22.1 or later) or 26.12.1 or later is required. Install Chromium before the first web run:
 
 ```sh
-npm install -g @ihyeon/flow-qa@0.2.9
+npm install -g @ihyeon/flow-qa@0.4.0
 flow-qa browser install
 flow-qa ui --demo
 ```
@@ -62,6 +62,16 @@ For authenticated models, add `--key-env FLOW_QA_API_KEY` to the model command a
 
 `flow-qa doctor --workspace my-qa` checks local dependencies and whether configured model-key environment variables are present. It does not print key values or call the model server; a present key does not confirm remote authentication.
 
+## Run selected cases
+
+From `0.4.0`, choose cases under **Scenario → Review, then run → Cases to run**. Every case starts selected. Clear or change the selection, review it, then run against your target. Selecting cases never edits the saved suite or implicitly runs prerequisite cases. Each case must establish its own starting state.
+
+The CLI supports repeated IDs: `flow-qa run cases/task-flow.json --target demo --case TC-001 --case TC-003`. Omit `--case` to run the complete suite. Reports and comparisons label partial runs explicitly; a selected-run pass does not evaluate omitted cases. See the [UI guide](docs/ui.md#run-selected-cases).
+
+## Compare recorded runs
+
+In **Run history → Compare two runs**, select two reports from the same suite and target to see passed-to-failed, failed-to-passed, other status changes, unchanged, added, and missing cases. Filter the comparison or download all results as Markdown/JSON. Changed test definitions and incomplete runs carry explicit warnings; observed status changes do not prove a product defect was introduced or fixed. See the [UI guide](docs/ui.md#compare-two-runs).
+
 ## Mobile targets
 
 Version `0.2.0` adds user-configured iOS and Android targets. Choose **Mobile** in the scenario screen to prepare cases before choosing an app. Later, register the platform, installed app's bundle/package ID, and device in **Connections**, then select that target for execution. Saving settings does not install an app or start a test.

package/dist/src/case-selection.js

@@ -0,0 +1,50 @@
+export class CaseSelectionError extends Error {
+}
+/** Resolve requested IDs against the complete suite without changing its order or contents. */
+export function selectCases(suite, caseIds) {
+    if (caseIds === undefined)
+        return { suite };
+    if (!Array.isArray(caseIds) || caseIds.length === 0)
+        throw new CaseSelectionError('Select at least one case ID.');
+    const available = new Set(suite.cases.map((testCase) => testCase.id));
+    const requested = new Set();
+    for (const id of caseIds) {
+        if (typeof id !== 'string' || !available.has(id))
+            throw new CaseSelectionError(`Unknown case ID: ${String(id)}`);
+        if (requested.has(id))
+            throw new CaseSelectionError(`Duplicate selected case ID: ${id}`);
+        requested.add(id);
+    }
+    if (requested.size === suite.cases.length)
+        return { suite };
+    const cases = suite.cases.filter((testCase) => requested.has(testCase.id));
+    const requirementIds = new Set(cases.flatMap((testCase) => testCase.requirementIds));
+    return {
+        suite: { ...suite, cases, requirements: suite.requirements.filter((requirement) => requirementIds.has(requirement.id)) },
+        selection: { caseIds: cases.map((testCase) => testCase.id), totalCases: suite.cases.length },
+    };
+}
+/** Check optional additive report metadata before treating a report as comparable. */
+export function isReportSelectionValid(cases, selection) {
+    const caseIds = cases.map((item) => item.id);
+    if (caseIds.some((id) => typeof id !== 'string' || !id.trim()) || new Set(caseIds).size !== caseIds.length)
+        return false;
+    if (selection === undefined)
+        return true;
+    if (selection === null || typeof selection !== 'object' || Array.isArray(selection))
+        return false;
+    const prototype = Object.getPrototypeOf(selection);
+    if (prototype !== Object.prototype && prototype !== null)
+        return false;
+    if (Reflect.ownKeys(selection).length !== 2 || !Object.hasOwn(selection, 'caseIds') || !Object.hasOwn(selection, 'totalCases'))
+        return false;
+    const value = selection;
+    if (!Number.isInteger(value.totalCases) || value.totalCases > 100 || value.totalCases <= caseIds.length)
+        return false;
+    if (!Array.isArray(value.caseIds) || value.caseIds.length === 0 || value.caseIds.length !== caseIds.length)
+        return false;
+    if (value.caseIds.some((id) => typeof id !== 'string' || !id.trim()))
+        return false;
+    const selected = new Set(value.caseIds);
+    return selected.size === value.caseIds.length && caseIds.every((id) => selected.has(id));
+}

package/dist/src/cli.js

@@ -10,6 +10,7 @@ import { configSchema, idSchema, modelSchema, targetSchema } from "./schema.js";
 import { initWorkspace, readConfig, readSuite, sha256, verifySuiteSource, writeConfig, writeJSON } from "./workspace.js";
 import { buildPlanningPrompt, planFlow } from "./planner.js";
 import { compileSuite } from "./compiler.js";
+import { CaseSelectionError, isReportSelectionValid, selectCases } from "./case-selection.js";
 import { runSuite } from "./engine.js";
 import { renderMarkdown } from "./report.js";
 import { targetFingerprint, targetLabel } from "./targets.js";
@@ -29,6 +30,7 @@ Usage: flow-qa <command> [arguments] [options]
   validate <cases.json>               Check case structure and assertions
   generate <cases.json> [--out test.ts] Generate e2e test code
   run <cases.json> --target <name>     Run against a registered target and save a QA report
+      [--case <id> ...]               Run only the selected case IDs (repeat --case for more)
   report <run-directory>              View a saved QA report
   compare <run-a> <run-b>              Compare case results
   ui [--demo] [--port 4310]           Start the local QA UI
@@ -54,6 +56,7 @@ Local example: flow-qa ui --demo
   validate <cases.json>               케이스 구조와 검증 조건 확인
   generate <cases.json> [--out test.ts] e2e 테스트 코드 생성
   run <cases.json> --target <name>     연결한 대상에서 실행하고 QA 리포트 저장
+      [--case <id> ...]               선택한 케이스만 실행 (--case를 반복해 여러 개 선택)
   report <run-directory>              저장된 QA 리포트 보기
   compare <run-a> <run-b>             케이스별 결과 비교
   ui [--demo] [--port 4310]           로컬 QA 화면 실행
@@ -126,6 +129,7 @@ export async function main(args) {
         const { positionals, values } = parseArgs({ args, allowPositionals: true, strict: true, options: {
                 workspace: { type: 'string' }, url: { type: 'string' }, model: { type: 'string' },
                 'key-env': { type: 'string' }, out: { type: 'string' }, target: { type: 'string' },
+                case: { type: 'string', multiple: true },
                 platform: { type: 'string' }, 'bundle-id': { type: 'string' }, device: { type: 'string' },
                 'prompt-only': { type: 'boolean' }, help: { type: 'boolean', short: 'h' },
                 demo: { type: 'boolean' }, port: { type: 'string' }, 'with-deps': { type: 'boolean' }, lang: { type: 'string' },
@@ -145,7 +149,7 @@ export async function main(args) {
             (values['bundle-id'] !== undefined || values.device !== undefined) && command !== 'connect') {
             throw new Error(localized(locale, 'Platform and device options are only supported by connect and plan. See --help.', '플랫폼·기기 옵션은 connect와 plan에서만 사용할 수 있습니다. --help를 참고하세요.'));
         }
-        if (values.demo && command !== 'ui' || values.port && command !== 'ui' ||
+        if (values.case !== undefined && command !== 'run' || values.demo && command !== 'ui' || values.port && command !== 'ui' ||
             values['with-deps'] && (command !== 'browser' || arg !== 'install'))
             throw new Error(localized(locale, 'This option is not supported for the command. See --help.', '이 명령에서 사용할 수 없는 옵션입니다. --help를 참고하세요.'));
         if (command === 'ui') {
@@ -246,12 +250,24 @@ export async function main(args) {
             }
             const config = await readConfig(workspace);
             const targetName = values.target ?? (Object.keys(config.targets).length === 1 ? Object.keys(config.targets)[0] : undefined);
+            let caseIds;
+            try {
+                caseIds = selectCases(suite, values.case).selection?.caseIds;
+            }
+            catch (error) {
+                if (error instanceof CaseSelectionError)
+                    throw new Error(localizeProductMessage('Select one or more unique cases from this suite.', locale));
+                throw error;
+            }
             const controller = new AbortController();
             const interrupt = () => controller.abort();
             process.once('SIGINT', interrupt);
             process.once('SIGTERM', interrupt);
             try {
-                const { report, runDir } = await runSuite({ workspace, suite, config, targetName: required(targetName, '--target', locale), signal: controller.signal });
+                const { report, runDir } = await runSuite({ workspace, suite, config, targetName: required(targetName, '--target', locale), signal: controller.signal,
+                    ...(caseIds ? { caseIds } : {}) });
+                if (report.selection)
+                    console.log(localized(locale, `Selected run: ${report.selection.caseIds.length}/${report.selection.totalCases} cases; the full suite was not evaluated.`, `선택 실행: ${report.selection.totalCases}개 중 ${report.selection.caseIds.length}개. 전체 케이스를 검증한 결과가 아닙니다.`));
                 console.log(localized(locale, `Run result: ${report.status}\nQA report: ${resolve(runDir, 'qa-report.md')}\nRaw result: ${resolve(runDir, 'qa-report.json')}`, `실행 결과: ${report.status}\nQA 리포트: ${resolve(runDir, 'qa-report.md')}\n원본 결과: ${resolve(runDir, 'qa-report.json')}`));
                 process.exitCode = report.status === 'passed' ? 0 : report.status === 'interrupted' ? 130 : 1;
             }
@@ -283,11 +299,16 @@ export async function main(args) {
                 const { name: _targetName, ...identity } = data.target;
                 if (!targetSchema.safeParse(identity).success)
                     throw new Error(localized(locale, 'This is not a valid QA report.', '유효한 QA 리포트가 아닙니다.'));
+                if (!isReportSelectionValid(data.cases, data.selection))
+                    throw new Error(localized(locale, 'This is not a valid QA report.', '유효한 QA 리포트가 아닙니다.'));
                 return data;
             };
             const before = await load(arg), after = await load(second);
-            const comparable = before.suiteHash === after.suiteHash && targetFingerprint(before.target) === targetFingerprint(after.target);
-            console.log(comparable ? localized(locale, 'Comparing the same suite and target (check external data state separately)', '동일 명세·대상 비교 (외부 데이터 상태는 별도 확인)') : localized(locale, 'Suite or target differs; no direct regression verdict is made.', '명세 또는 대상이 달라 직접적인 회귀 판정은 하지 않습니다.'));
+            const sameSelection = JSON.stringify(before.cases.map((item) => item.id).sort()) === JSON.stringify(after.cases.map((item) => item.id).sort());
+            const comparable = before.suiteId === after.suiteId && before.suiteHash === after.suiteHash && targetFingerprint(before.target) === targetFingerprint(after.target) && sameSelection;
+            console.log(comparable ? localized(locale, 'Comparing the same suite, target and case selection (check external data state separately)', '동일 명세·대상·케이스 범위 비교 (외부 데이터 상태는 별도 확인)') : localized(locale, 'Suite, target or case selection differs; no direct regression verdict is made.', '명세·대상 또는 케이스 범위가 달라 직접적인 회귀 판정은 하지 않습니다.'));
+            if (before.selection || after.selection)
+                console.log(localized(locale, 'This comparison includes selected cases only; omitted cases were not evaluated.', '이 비교에는 선택 실행이 포함되어 있습니다. 미선택 케이스는 검증되지 않았습니다.'));
             const previous = new Map(before.cases.map((c) => [c.id, c.status]));
             const current = new Map(after.cases.map((c) => [c.id, c.status]));
             for (const id of new Set([...previous.keys(), ...current.keys()]))

package/dist/src/engine.js

@@ -4,6 +4,7 @@ import { createWriteStream } from 'node:fs';
 import { lstat, mkdir, readFile, realpath, stat, writeFile } from 'node:fs/promises';
 import path from 'node:path';
 import { fileURLToPath } from 'node:url';
+import { selectCases } from "./case-selection.js";
 import { compileSuite } from "./compiler.js";
 import { acquireMobileDevice, mobileReadiness } from "./mobile-runtime.js";
 import { failureReport, normalizeReport1, renderMarkdown, stoppedReport } from "./report.js";
@@ -158,6 +159,10 @@ function runnableSuite(suite, blocked) {
 }
 /** Run one immutable suite snapshot through the locally installed e2e CLI. */
 export async function runSuite(options) {
+    // A malformed selection must not leave a run directory behind.
+    const selected = selectCases(options.suite, options.caseIds);
+    let executionSuite = selected.suite;
+    let selection = selected.selection;
     const workspace = await realpath(options.workspace);
     const qaRoot = await ownedDirectory(workspace, '.flowqa');
     const runsRoot = await ownedDirectory(qaRoot, 'runs');
@@ -183,7 +188,7 @@ export async function runSuite(options) {
     let blocked = new Map();
     let target = { name: options.targetName, url: '' };
     let releaseMobileDevice;
-    const context = () => ({ suite: options.suite, runId, suiteHash, target, startedAt, finishedAt: new Date().toISOString(), blockedReasons: blocked });
+    const context = () => ({ suite: executionSuite, selection, runId, suiteHash, target, startedAt, finishedAt: new Date().toISOString(), blockedReasons: blocked });
     const blockAll = (suite, reason) => {
         for (const testCase of suite.cases)
             if (!blocked.has(testCase.id))
@@ -193,6 +198,7 @@ export async function runSuite(options) {
     try {
         const suite = suiteSchema.parse(options.suite);
         await verifySuiteSource(suite, workspace);
+        ({ suite: executionSuite, selection } = selectCases(suite, options.caseIds));
         const rawTarget = options.config?.targets?.[options.targetName];
         if (rawTarget && typeof rawTarget === 'object' && !Array.isArray(rawTarget)) {
             const raw = rawTarget;
@@ -200,8 +206,8 @@ export async function runSuite(options) {
                 if ((raw.platform === 'ios' || raw.platform === 'android') && typeof raw.bundleId === 'string') {
                     target = { name: options.targetName, kind: 'mobile', platform: raw.platform, bundleId: raw.bundleId, device: '' };
                 }
-                blocked = blockedCases(suite, options.config);
-                blockAll(suite, 'A mobile target requires an explicit device.');
+                blocked = blockedCases(executionSuite, options.config);
+                blockAll(executionSuite, 'A mobile target requires an explicit device.');
             }
         }
         const config = configSchema.parse(options.config);
@@ -209,23 +215,23 @@ export async function runSuite(options) {
         if (!selectedTarget)
             throw new Error(`Target not found: ${options.targetName}`);
         target = reportTarget(options.targetName, selectedTarget);
-        blocked = blockedCases(suite, config);
+        blocked = blockedCases(executionSuite, config);
         if ((suite.platform ?? 'web') !== targetKind(selectedTarget))
-            blockAll(suite, 'The suite platform does not match the selected target.');
-        const runnable = suite.cases.filter((testCase) => !blocked.has(testCase.id));
+            blockAll(executionSuite, 'The suite platform does not match the selected target.');
+        const runnable = executionSuite.cases.filter((testCase) => !blocked.has(testCase.id));
         let sdkDevice;
         if (selectedTarget.kind === 'mobile' && runnable.length > 0 && !options.signal?.aborted) {
             const readiness = await mobileReadiness(selectedTarget, options.listMobileDevices);
             if (!readiness.ready)
-                blockAll(suite, readiness.reason);
+                blockAll(executionSuite, readiness.reason);
             else {
                 sdkDevice = readiness.sdkDevice;
                 releaseMobileDevice = acquireMobileDevice(selectedTarget.platform, readiness.deviceId);
             }
             if (!releaseMobileDevice)
-                blockAll(suite, 'Selected mobile device is in use.');
+                blockAll(executionSuite, 'Selected mobile device is in use.');
         }
-        const executable = runnableSuite(suite, blocked);
+        const executable = runnableSuite(executionSuite, blocked);
         const testFile = path.join(runDir, TEST_FILE);
         const configFile = path.join(runDir, CONFIG_FILE);
         const agentAvailable = runnable.some(agentCase);

package/dist/src/i18n.js

@@ -62,6 +62,14 @@ const koreanMessages = {
     'Invalid request origin.': '요청 출처가 올바르지 않습니다.',
     'A case file name is required.': '케이스 파일 이름이 필요합니다.',
     'A run ID is required.': '실행 ID가 필요합니다.',
+    'Both run IDs are required.': '비교할 두 실행 ID가 필요합니다.',
+    'A comparison needs two different runs.': '서로 다른 두 실행을 선택해야 합니다.',
+    'Both reports must belong to the same suite.': '두 리포트의 케이스 모음이 같아야 합니다.',
+    'Both reports must use the same target.': '두 리포트의 실행 대상이 같아야 합니다.',
+    'Both reports need a non-empty suite hash.': '두 리포트에 비어 있지 않은 케이스 모음 해시가 있어야 합니다.',
+    'Both reports need unique, non-empty case IDs.': '두 리포트의 케이스 ID는 비어 있지 않고 중복되지 않아야 합니다.',
+    'The report selection does not match its recorded cases.': '보고서의 선택 범위가 기록된 케이스와 일치하지 않습니다.',
+    'Select one or more unique cases from this suite.': '이 케이스 모음에서 중복 없이 하나 이상의 케이스를 선택하세요.',
     'Invalid evidence selection.': '증거 파일 선택이 올바르지 않습니다.',
     'The request is not allowed.': '허용되지 않은 요청입니다.',
     'Invalid request token.': '요청 토큰이 올바르지 않습니다.',

package/dist/src/report.js

@@ -1,8 +1,21 @@
 import { realpathSync } from 'node:fs';
 import path from 'node:path';
 import { expectedChecks } from "./compiler.js";
+import { isReportSelectionValid } from "./case-selection.js";
 import { localized, localizeProductMessage } from "./i18n.js";
 import { targetLabel } from "./targets.js";
+function scopedCases(context) {
+    if (!context.selection)
+        return context.suite.cases;
+    const selected = new Set(context.selection.caseIds);
+    const cases = context.suite.cases.filter((testCase) => selected.has(testCase.id));
+    if (!isReportSelectionValid(cases, context.selection))
+        throw new Error('Report selection does not match the current suite.');
+    return cases;
+}
+function reportSelection(context) {
+    return context.selection ? { selection: { caseIds: [...context.selection.caseIds], totalCases: context.selection.totalCases } } : {};
+}
 function object(value, label) {
     if (value === null || typeof value !== 'object' || Array.isArray(value))
         throw new Error(`Invalid raw report field: ${label}.`);
@@ -151,7 +164,8 @@ export function normalizeReport1(raw, context) {
     const rawResults = array(run.results, 'run.results');
     const rawErrors = array(run.errors, 'run.errors').map(rawError);
     const sourceName = path.basename(context.testFile);
-    const knownTitles = new Map(context.suite.cases.map((testCase) => [`[${testCase.id}] ${testCase.title}`, testCase.id]));
+    const selectedCases = scopedCases(context);
+    const knownTitles = new Map(selectedCases.map((testCase) => [`[${testCase.id}] ${testCase.title}`, testCase.id]));
     const mapped = new Map();
     for (const value of rawResults) {
         const result = object(value, 'result');
@@ -185,7 +199,7 @@ export function normalizeReport1(raw, context) {
         mapped.set(id, { status, attempts, ...(skip === undefined ? {} : { reason: string(skip.reason, 'result.skip.reason') }) });
     }
     const errors = rawErrors.map((error) => `${error.code}: ${error.message}`);
-    const cases = context.suite.cases.map((testCase) => {
+    const cases = selectedCases.map((testCase) => {
         const expected = expectedChecks(testCase);
         const result = mapped.get(testCase.id);
         const blockedReason = context.blockedReasons.get(testCase.id);
@@ -220,7 +234,7 @@ export function normalizeReport1(raw, context) {
     return {
         schemaVersion: 'flow-qa/report-1', runId: context.runId, suiteId: context.suite.id,
         suiteHash: context.suiteHash, target: context.target, startedAt: context.startedAt,
-        finishedAt: context.finishedAt, status, cases, errors, usage: reportUsage,
+        finishedAt: context.finishedAt, status, cases, errors, usage: reportUsage, ...reportSelection(context),
     };
 }
 /** Keep every recorded attempt when the adapter stops a still-running child. */
@@ -236,13 +250,13 @@ export function failureReport(context, status, error) {
         schemaVersion: 'flow-qa/report-1', runId: context.runId, suiteId: context.suite.id,
         suiteHash: context.suiteHash, target: context.target, startedAt: context.startedAt,
         finishedAt: context.finishedAt, status,
-        cases: context.suite.cases.map((testCase) => ({
+        cases: scopedCases(context).map((testCase) => ({
             id: testCase.id, title: testCase.title,
             status: context.blockedReasons.has(testCase.id) ? 'blocked' : 'error',
             expected: expectedChecks(testCase), attempts: [],
             reason: context.blockedReasons.get(testCase.id) ?? error,
         })),
-        errors: [error],
+        errors: [error], ...reportSelection(context),
     };
 }
 function safeText(value, locale) {
@@ -277,6 +291,7 @@ export function renderMarkdown(report, locale = 'en') {
             ? safe(targetLabel(report.target))
             : label('(URL available in the local JSON report)', '(주소는 로컬 JSON 보고서에서 확인)')}`,
         `- ${label('Status', '상태')}: ${safe(report.status)}`,
+        ...(report.selection ? [`- ${label('Scope', '범위')}: ${label(`Selected ${report.selection.caseIds.length} / ${report.selection.totalCases} cases; full suite not evaluated.`, `선택 ${report.selection.caseIds.length} / 전체 ${report.selection.totalCases}개 케이스; 전체 케이스 모음은 평가하지 않았습니다.`)}`] : []),
         `- ${label('Started', '시작')}: ${safe(report.startedAt)}`,
         `- ${label('Finished', '종료')}: ${safe(report.finishedAt)}`,
         `- ${label('Results', '결과')}: ${label('passed', '통과')} ${counts.get('passed') ?? 0}, ${label('failed', '실패')} ${counts.get('failed') ?? 0}, ${label('flaky', '재시도 후 통과')} ${counts.get('flaky') ?? 0}, ${label('blocked', '차단')} ${counts.get('blocked') ?? 0}, ${label('skipped', '건너뜀')} ${counts.get('skipped') ?? 0}, ${label('error', '오류')} ${counts.get('error') ?? 0}`,

package/dist/src/run-comparison.js

@@ -0,0 +1,165 @@
+import { localized } from "./i18n.js";
+import { isReportSelectionValid } from "./case-selection.js";
+import { targetFingerprint, targetLabel } from "./targets.js";
+export const comparisonCategories = ['newly-failed', 'now-passed', 'changed', 'unchanged', 'added', 'missing'];
+export class ComparisonError extends Error {
+    status;
+    constructor(status, message) { super(message); this.status = status; }
+}
+function validateCaseIds(report) {
+    const cases = new Map();
+    for (const item of report.cases) {
+        if (!item.id.trim() || cases.has(item.id)) {
+            throw new ComparisonError(409, 'Both reports need unique, non-empty case IDs.');
+        }
+        cases.set(item.id, item);
+    }
+    return cases;
+}
+function runSummary(report) {
+    return {
+        runId: report.runId, suiteId: report.suiteId, suiteHash: report.suiteHash,
+        target: { ...report.target }, startedAt: report.startedAt,
+        finishedAt: report.finishedAt, status: report.status,
+        ...(report.selection ? { selection: { caseIds: [...report.selection.caseIds], totalCases: report.selection.totalCases } } : {}),
+    };
+}
+function caseSnapshot(item) {
+    if (!item)
+        return null;
+    return {
+        title: item.title, status: item.status, expected: [...item.expected],
+        ...(item.reason === undefined ? {} : { reason: item.reason }),
+    };
+}
+function category(baseline, current) {
+    if (!baseline)
+        return 'added';
+    if (!current)
+        return 'missing';
+    if (baseline.status === current.status)
+        return 'unchanged';
+    if (baseline.status === 'passed' && current.status === 'failed')
+        return 'newly-failed';
+    if (baseline.status === 'failed' && current.status === 'passed')
+        return 'now-passed';
+    return 'changed';
+}
+/** Compare only recorded final case statuses and report metadata. Never inspect attempts or artifacts. */
+export function compareReports(baseline, current) {
+    if (baseline.runId === current.runId)
+        throw new ComparisonError(400, 'A comparison needs two different runs.');
+    if (baseline.suiteId !== current.suiteId)
+        throw new ComparisonError(409, 'Both reports must belong to the same suite.');
+    if (targetFingerprint(baseline.target) !== targetFingerprint(current.target)) {
+        throw new ComparisonError(409, 'Both reports must use the same target.');
+    }
+    if (!baseline.suiteHash.trim() || !current.suiteHash.trim()) {
+        throw new ComparisonError(409, 'Both reports need a non-empty suite hash.');
+    }
+    const before = validateCaseIds(baseline);
+    const after = validateCaseIds(current);
+    if (!isReportSelectionValid(baseline.cases, baseline.selection) || !isReportSelectionValid(current.cases, current.selection)) {
+        throw new ComparisonError(409, 'The report selection does not match its recorded cases.');
+    }
+    const warnings = [];
+    if (baseline.suiteHash !== current.suiteHash)
+        warnings.push('suite-changed');
+    if (baseline.selection || current.selection) {
+        warnings.push('partial-run');
+        if (before.size !== after.size || [...before.keys()].some((id) => !after.has(id)))
+            warnings.push('selection-changed');
+    }
+    if (Date.parse(current.startedAt) < Date.parse(baseline.startedAt))
+        warnings.push('reverse-order');
+    if (baseline.status === 'interrupted' || baseline.status === 'error' || current.status === 'interrupted' || current.status === 'error') {
+        warnings.push('incomplete-run');
+    }
+    if (baseline.target.kind === 'mobile')
+        warnings.push('mobile-identity-unverified');
+    const counts = Object.fromEntries(comparisonCategories.map((value) => [value, 0]));
+    const cases = [...new Set([...before.keys(), ...after.keys()])].sort().map((id) => {
+        const baselineCase = before.get(id);
+        const currentCase = after.get(id);
+        const transition = category(baselineCase, currentCase);
+        counts[transition]++;
+        return { id, category: transition, baseline: caseSnapshot(baselineCase), current: caseSnapshot(currentCase) };
+    });
+    return { schemaVersion: 'flow-qa/comparison-1', baseline: runSummary(baseline), current: runSummary(current), warnings, counts, cases };
+}
+function safeInline(value, locale) {
+    return value.replace(/(?:https?:\/\/|file:\/\/)[^\s<>"'\])}]+/gi, localized(locale, '[URL hidden]', '[URL 숨김]'))
+        .replace(/\[[^\]]*\]\([^)]*\)/g, localized(locale, '[link hidden]', '[링크 숨김]'))
+        .replace(/[\x00-\x1f\x7f\u0085\u2028\u2029]/g, (character) => {
+        if (character === '\n')
+            return '\\n';
+        if (character === '\r')
+            return '\\r';
+        if (character === '\t')
+            return '\\t';
+        return `\\u${character.charCodeAt(0).toString(16).padStart(4, '0')}`;
+    }).replace(/&/g, '&amp;').replace(/</g, '&lt;').replace(/>/g, '&gt;')
+        .replace(/([\\`*_{}\[\]()#+.!|~:])/g, '\\$1');
+}
+export function renderComparisonMarkdown(comparison, locale = 'en') {
+    const label = (english, korean) => localized(locale, english, korean);
+    const safe = (value) => safeInline(value, locale);
+    const run = (name, item) => [
+        `## ${name}`,
+        '',
+        `- ${label('Run ID', '실행 ID')}: ${safe(item.runId)}`,
+        `- ${label('Suite ID', '케이스 모음 ID')}: ${safe(item.suiteId)}`,
+        `- ${label('Suite hash', '케이스 모음 해시')}: ${safe(item.suiteHash)}`,
+        `- ${label('Target', '대상')}: ${safe(item.target.name)} ${item.target.kind === 'mobile'
+            ? safe(targetLabel(item.target))
+            : label('(URL available in the local JSON comparison)', '(주소는 로컬 JSON 비교 결과에서 확인)')}`,
+        `- ${label('Started', '시작')}: ${safe(item.startedAt)}`,
+        `- ${label('Finished', '종료')}: ${safe(item.finishedAt)}`,
+        `- ${label('Status', '상태')}: ${safe(item.status)}`,
+        ...(item.selection ? [`- ${label('Scope', '범위')}: ${label(`Selected ${item.selection.caseIds.length} / ${item.selection.totalCases} cases; full suite not evaluated.`, `선택 ${item.selection.caseIds.length} / 전체 ${item.selection.totalCases}개 케이스; 전체 케이스 모음은 평가하지 않았습니다.`)}`] : []),
+        '',
+    ];
+    const warningText = {
+        'suite-changed': label('Steps or expectations may differ because the suite changed; these outcomes do not prove a regression or fix.', '케이스 모음이 변경되어 단계나 기대 결과가 다를 수 있습니다. 이 결과만으로 회귀나 수정이 입증되지는 않습니다.'),
+        'partial-run': label('At least one run evaluated only selected cases; the full suite was not evaluated.', '하나 이상의 실행에서 선택한 케이스만 평가했습니다. 전체 케이스 모음은 평가하지 않았습니다.'),
+        'selection-changed': label('The runs selected different case sets; added or missing entries may reflect scope rather than a changed outcome.', '실행마다 선택한 케이스가 다릅니다. 추가 또는 누락 항목은 결과 변화가 아닌 실행 범위 차이일 수 있습니다.'),
+        'reverse-order': label('The selected current run started before the baseline run.', '선택한 현재 실행이 기준 실행보다 먼저 시작됐습니다.'),
+        'incomplete-run': label('At least one run ended with error or interruption; case results may be incomplete.', '하나 이상의 실행이 오류 또는 중단으로 끝나 케이스 결과가 불완전할 수 있습니다.'),
+        'mobile-identity-unverified': label('The recorded app and device settings match, but these reports do not verify that both runs used the same physical device.', '기록된 앱과 기기 설정은 같지만, 두 실행이 같은 실제 기기에서 진행됐는지는 이 리포트로 확인할 수 없습니다.'),
+    };
+    const lines = [
+        `# ${label('QA run comparison', 'QA 실행 비교')}`,
+        '',
+        ...run(label('Baseline', '기준 실행'), comparison.baseline),
+        ...run(label('Current', '현재 실행'), comparison.current),
+        `## ${label('Transition counts', '상태 전이 개수')}`,
+        '',
+        ...comparisonCategories.map((category) => `- ${category}: ${comparison.counts[category]}`),
+        '',
+        `## ${label('Warnings', '주의 사항')}`,
+        '',
+        ...(comparison.warnings.length
+            ? comparison.warnings.map((warning) => `- ${warning}: ${warningText[warning]}`)
+            : [`- ${label('None', '없음')}`]),
+        '',
+        label('These categories describe observed status transitions only. They do not establish the cause of a failure or prove that a defect was fixed.', '이 분류는 관찰된 상태 전이만 나타냅니다. 실패 원인이나 결함 수정 여부를 입증하지 않습니다.'),
+        '',
+        `## ${label('Cases', '케이스')}`,
+        '',
+    ];
+    for (const item of comparison.cases) {
+        lines.push(`### ${safe(item.id)} — ${item.category}`, '');
+        for (const [name, entry] of [[label('Baseline', '기준 실행'), item.baseline], [label('Current', '현재 실행'), item.current]]) {
+            if (!entry) {
+                lines.push(`- ${name}: ${label('Absent', '없음')}`);
+                continue;
+            }
+            lines.push(`- ${name}: ${safe(entry.status)} · ${safe(entry.title)}`);
+            lines.push(`  - ${label('Expected', '기대 결과')}: ${entry.expected.length ? entry.expected.map(safe).join('; ') : label('None', '없음')}`);
+            if (entry.reason !== undefined)
+                lines.push(`  - ${label('Reason', '실행 상태 근거')}: ${safe(entry.reason)}`);
+        }
+        lines.push('');
+    }
+    return lines.join('\n');
+}