@ihyeon/flow-qa 0.6.0 → 0.7.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.ko.md +1 -1
- package/README.md +1 -1
- package/dist/src/ui/public/app.js +107 -17
- package/dist/src/ui/public/i18n.js +6 -4
- package/dist/src/ui/public/index.html +3 -3
- package/dist/src/ui/public/style.css +7 -0
- package/dist/src/ui/server.js +25 -5
- package/docs/releases.md +11 -0
- package/docs/ui.md +3 -3
- package/package.json +2 -2
package/README.ko.md
CHANGED
|
@@ -72,7 +72,7 @@ CLI에서는 `flow-qa run cases/task-flow.json --target demo --case TC-001 --cas
|
|
|
72
72
|
|
|
73
73
|
## 실행 결과 비교
|
|
74
74
|
|
|
75
|
-
`0.6.0`부터 **실행 기록**에서 저장된 전체 보고서를 스위트 ID·실행 ID·대상 이름으로 검색하고 실행 상태로 함께 좁힐 수 있습니다. 결과는 50건씩 표시하며 **더 보기**로 이전 기록을 불러옵니다. **실행 기록 새로고침**은 검색·상태 조건을 유지한 채 목록을 갱신합니다. 오래된 보고서도 다운로드와 실패 케이스 준비를 사용할 수 있습니다. 자세한 동작은 [기록 검색·페이지 조회](docs/ui.md#search-run-history)를 참고하세요.
|
|
75
|
+
`0.6.0`부터 **실행 기록**에서 저장된 전체 보고서를 스위트 ID·실행 ID·대상 이름으로 검색하고 실행 상태로 함께 좁힐 수 있습니다. 결과는 50건씩 표시하며 **더 보기**로 이전 기록을 불러옵니다. **실행 기록 새로고침**은 검색·상태 조건을 유지한 채 목록을 갱신합니다. 오래된 보고서도 다운로드와 실패 케이스 준비를 사용할 수 있습니다. `0.7.0`부터 각 기록의 **기준으로 선택**·**현재로 선택**으로 이전 실행끼리 비교할 수 있으며, 검색이나 새로고침 후에도 선택을 유지합니다. 자세한 동작은 [기록 검색·페이지 조회](docs/ui.md#search-run-history)를 참고하세요.
|
|
76
76
|
|
|
77
77
|
**실행 기록 → 실행 2건 비교**에서 같은 스위트와 대상의 두 기록을 선택하면 통과→실패, 실패→통과, 그 외 상태 변경, 동일 상태, 추가·누락 케이스를 확인할 수 있습니다. 분류별로 보거나 전체 비교를 Markdown·JSON으로 내려받으세요. 테스트 내용 변경과 불완전한 실행은 별도로 표시하며, 상태 변화만으로 제품 버그의 발생·해결을 확정하지 않습니다. 자세한 동작은 [UI 가이드](docs/ui.md#compare-two-runs)를 참고하세요.
|
|
78
78
|
|
package/README.md
CHANGED
|
@@ -72,7 +72,7 @@ From `0.5.0`, open a detailed report and choose **Prepare failed/error cases** t
|
|
|
72
72
|
|
|
73
73
|
## Compare recorded runs
|
|
74
74
|
|
|
75
|
-
From `0.6.0`, **Run history** searches all valid saved reports by suite ID, run ID, or target name, combined with the run-status filter. Results load 50 at a time; use **Load more** for older results and **Refresh history** to update the list while keeping your filters. Older reports support the same downloads and failed-case preparation. See [history search and paging](docs/ui.md#search-run-history).
|
|
75
|
+
From `0.6.0`, **Run history** searches all valid saved reports by suite ID, run ID, or target name, combined with the run-status filter. Results load 50 at a time; use **Load more** for older results and **Refresh history** to update the list while keeping your filters. Older reports support the same downloads and failed-case preparation. From `0.7.0`, choose **Use as baseline** and **Use as current** on any history rows to compare older runs, retaining the pair through searches and refreshes. See [history search and paging](docs/ui.md#search-run-history).
|
|
76
76
|
|
|
77
77
|
In **Run history → Compare two runs**, select two reports from the same suite and target to see passed-to-failed, failed-to-passed, other status changes, unchanged, added, and missing cases. Filter the comparison or download all results as Markdown/JSON. Changed test definitions and incomplete runs carry explicit warnings; observed status changes do not prove a product defect was introduced or fixed. See the [UI guide](docs/ui.md#compare-two-runs).
|
|
78
78
|
|
|
@@ -4,7 +4,7 @@
|
|
|
4
4
|
const i18n = window.FlowQAI18n;
|
|
5
5
|
const t = i18n.t;
|
|
6
6
|
const names = { scenario: 'nav.scenario', library: 'nav.library', history: 'nav.history', settings: 'nav.settings' };
|
|
7
|
-
const app = { state: null, connected: false, displayTargets: [], draft: null, selectedCaseIds: new Set(), script: '', savedFile: '', savedHash: '', origin: 'none', sourceFlow: '', boundSource: null, stale: false, dirty: false, busy: false, planning: false, planRequest: 0, stopping: false, report: null, reportMarkdown: '', selectedRun: '', rerunOrigin: null, rerunTargetMismatch: false, navigationVersion: 0, comparison: null, comparisonRequest: 0, comparisonBusy: false, comparisonStatus: 'comparison.selectBoth', history: { runs: [], nextCursor: null, pagesLoaded: 0, loaded: false, loading: false, request: 0, debounce: null, kind: null, error: null, updatesAvailable: false, pendingRecentUpdate: false }, view: 'scenario' };
|
|
7
|
+
const app = { state: null, connected: false, displayTargets: [], draft: null, selectedCaseIds: new Set(), script: '', savedFile: '', savedHash: '', origin: 'none', sourceFlow: '', boundSource: null, stale: false, dirty: false, busy: false, planning: false, planRequest: 0, stopping: false, report: null, reportMarkdown: '', selectedRun: '', rerunOrigin: null, rerunTargetMismatch: false, navigationVersion: 0, comparison: null, comparisonRequest: 0, comparisonBusy: false, comparisonStatus: 'comparison.selectBoth', comparisonPins: { baseline: null, current: null }, comparisonSelectionGeneration: 0, comparisonValidation: null, history: { runs: [], nextCursor: null, pagesLoaded: 0, loaded: false, loading: false, request: 0, debounce: null, kind: null, error: null, updatesAvailable: false, pendingRecentUpdate: false }, view: 'scenario' };
|
|
8
8
|
|
|
9
9
|
function el(tag, className, value) {
|
|
10
10
|
const node = document.createElement(tag);
|
|
@@ -38,8 +38,10 @@
|
|
|
38
38
|
try { result = await response.json(); } catch { throw localizedError('app.responseUnreadable', { status: response.status }); }
|
|
39
39
|
if (!response.ok) {
|
|
40
40
|
const issues = Array.isArray(result?.issues) ? result.issues.slice(0, 8).map((issue) => `${issue.path}: ${issue.message}`).join('\n') : '';
|
|
41
|
-
|
|
42
|
-
|
|
41
|
+
const error = !result?.error && !issues ? localizedError('app.requestFailed', { status: response.status })
|
|
42
|
+
: new Error(`${result?.error || t('app.requestFailed', { status: response.status })}${issues ? `\n${issues}` : ''}`);
|
|
43
|
+
error.status = response.status;
|
|
44
|
+
throw error;
|
|
43
45
|
}
|
|
44
46
|
return result;
|
|
45
47
|
}
|
|
@@ -282,10 +284,20 @@
|
|
|
282
284
|
: app.state?.active ? 'app.historyActiveEmpty' : 'app.historyEmpty';
|
|
283
285
|
host.append(el('p', 'empty', t(key)));
|
|
284
286
|
}
|
|
287
|
+
const pair = comparisonPair();
|
|
285
288
|
for (const run of history.runs) {
|
|
286
|
-
const row = el('div', 'list-row
|
|
289
|
+
const row = el('div', 'list-row history-run-row'); row.dataset.runId = run.runId;
|
|
290
|
+
const copy = el('div'); copy.append(el('strong', '', `${run.suiteId} · ${i18n.status(run.status)}`), el('small', '', t('app.runMeta', { date: formatDate(run.startedAt), target: targetDisplay(run.target), passed: run.passedCount ?? 0, total: run.caseCount ?? 0, id: run.runId })));
|
|
287
291
|
const scope = selectionScope(run.selection); if (scope) copy.append(el('small', 'selection-scope', scope));
|
|
288
|
-
const
|
|
292
|
+
const actions = el('div', 'run-row-actions');
|
|
293
|
+
const button = el('button', 'button', t('app.viewReport')); button.type = 'button'; button.dataset.action = 'view-report'; button.disabled = run.status === 'running' || run.status === 'stopping'; button.addEventListener('click', () => loadReport(run.runId)); actions.append(button);
|
|
294
|
+
for (const [role, label] of [['baseline', 'comparison.useBaseline'], ['current', 'comparison.useCurrent']]) {
|
|
295
|
+
const select = el('button', 'button quiet', t(label)); select.type = 'button'; select.dataset.action = role === 'baseline' ? 'compare-baseline' : 'compare-current';
|
|
296
|
+
select.setAttribute('aria-pressed', String(pair[role] === run.runId));
|
|
297
|
+
select.disabled = run.status === 'running' || run.status === 'stopping';
|
|
298
|
+
select.addEventListener('click', () => selectComparisonRun(role, run)); actions.append(select);
|
|
299
|
+
}
|
|
300
|
+
row.append(copy, actions); host.append(row);
|
|
289
301
|
}
|
|
290
302
|
const messages = [];
|
|
291
303
|
if (history.loaded) messages.push(t('history.pageCount', { count: history.runs.length }));
|
|
@@ -320,6 +332,7 @@
|
|
|
320
332
|
history.loaded = true;
|
|
321
333
|
history.error = null;
|
|
322
334
|
if (!append) history.updatesAvailable = false;
|
|
335
|
+
renderComparisonOptions();
|
|
323
336
|
} catch {
|
|
324
337
|
if (request !== history.request) return;
|
|
325
338
|
history.error = append ? 'more' : 'first';
|
|
@@ -341,7 +354,7 @@
|
|
|
341
354
|
const request = ++history.request;
|
|
342
355
|
history.runs = []; history.nextCursor = null; history.pagesLoaded = 0; history.loaded = false;
|
|
343
356
|
history.loading = true; history.kind = 'first'; history.error = null; history.updatesAvailable = false; history.pendingRecentUpdate = false;
|
|
344
|
-
renderHistory();
|
|
357
|
+
renderHistory(); renderComparisonOptions();
|
|
345
358
|
const filters = historyFilters();
|
|
346
359
|
if (delay) history.debounce = setTimeout(() => { history.debounce = null; void fetchHistoryPage(request, filters); }, delay);
|
|
347
360
|
else void fetchHistoryPage(request, filters);
|
|
@@ -367,7 +380,18 @@
|
|
|
367
380
|
const comparisonCategoryKeys = { 'newly-failed': 'comparison.newlyFailed', 'now-passed': 'comparison.nowPassed', changed: 'comparison.changed', unchanged: 'comparison.unchanged', added: 'comparison.added', missing: 'comparison.missing' };
|
|
368
381
|
const comparisonWarningKeys = { 'suite-changed': 'comparison.warningSuite', 'reverse-order': 'comparison.warningReverse', 'incomplete-run': 'comparison.warningIncomplete', 'mobile-identity-unverified': 'comparison.warningMobileIdentity', 'partial-run': 'comparison.warningPartial', 'selection-changed': 'comparison.warningSelectionChanged' };
|
|
369
382
|
function comparisonPair() { return { baseline: $('compare-baseline').value, current: $('compare-current').value }; }
|
|
370
|
-
function
|
|
383
|
+
function comparisonRun(runId) {
|
|
384
|
+
return (app.state?.runs || []).find((run) => run.runId === runId)
|
|
385
|
+
|| app.history.runs.find((run) => run.runId === runId)
|
|
386
|
+
|| Object.values(app.comparisonPins).find((run) => run?.runId === runId);
|
|
387
|
+
}
|
|
388
|
+
function comparisonValid() {
|
|
389
|
+
const { baseline, current } = comparisonPair();
|
|
390
|
+
return !!baseline && !!current && baseline !== current && [baseline, current].every((id) => {
|
|
391
|
+
const run = comparisonRun(id);
|
|
392
|
+
return run && run.status !== 'running' && run.status !== 'stopping';
|
|
393
|
+
});
|
|
394
|
+
}
|
|
371
395
|
function comparisonStatus(key, params = {}, error = false) {
|
|
372
396
|
app.comparisonStatus = key; app.comparisonStatusParams = params;
|
|
373
397
|
const node = $('comparison-status'); node.textContent = t(key, params); node.classList.toggle('report-errors', error);
|
|
@@ -385,23 +409,78 @@
|
|
|
385
409
|
$('comparison-summary').replaceChildren(); $('comparison-warnings').replaceChildren(); $('comparison-content').replaceChildren();
|
|
386
410
|
comparisonStatus('comparison.selectBoth'); comparisonControls();
|
|
387
411
|
}
|
|
412
|
+
function selectComparisonRun(role, run) {
|
|
413
|
+
if (run && (run.status === 'running' || run.status === 'stopping')) return;
|
|
414
|
+
const select = $(role === 'baseline' ? 'compare-baseline' : 'compare-current');
|
|
415
|
+
const runId = run?.runId || '';
|
|
416
|
+
if (select.value === runId && (app.comparisonPins[role]?.runId || '') === runId) return;
|
|
417
|
+
app.comparisonPins[role] = run || null;
|
|
418
|
+
app.comparisonSelectionGeneration += 1;
|
|
419
|
+
renderComparisonOptions();
|
|
420
|
+
select.value = runId;
|
|
421
|
+
clearComparison(); renderHistory();
|
|
422
|
+
}
|
|
388
423
|
function renderComparisonOptions() {
|
|
389
|
-
const runs =
|
|
390
|
-
|
|
424
|
+
const runs = new Map();
|
|
425
|
+
for (const run of [...(app.state?.runs || []).slice(0, 50), ...app.history.runs, ...Object.values(app.comparisonPins)]) {
|
|
426
|
+
if (run?.runId && !runs.has(run.runId)) runs.set(run.runId, run);
|
|
427
|
+
}
|
|
391
428
|
for (const id of ['compare-baseline', 'compare-current']) {
|
|
392
429
|
const select = $(id); const selected = select.value;
|
|
393
430
|
select.replaceChildren(el('option', '', t('comparison.choose'))); select.firstChild.value = '';
|
|
394
|
-
for (const run of runs) {
|
|
431
|
+
for (const run of runs.values()) {
|
|
395
432
|
const option = el('option', '', `${t('comparison.runLabel', { id: run.runId, shortId: run.runId.slice(-8), date: formatDate(run.startedAt), suite: run.suiteId, target: targetDisplay(run.target), status: i18n.status(run.status) })}${selectionScope(run.selection) ? ` · ${selectionScope(run.selection)}` : ''}`);
|
|
396
|
-
option.value = run.runId; select.append(option);
|
|
433
|
+
option.value = run.runId; option.disabled = run.status === 'running' || run.status === 'stopping'; select.append(option);
|
|
397
434
|
}
|
|
398
|
-
select.value = runs.
|
|
399
|
-
if (selected && !select.value) missing = true;
|
|
435
|
+
select.value = runs.has(selected) ? selected : '';
|
|
400
436
|
}
|
|
401
|
-
if (
|
|
402
|
-
if (!runs.length && !app.comparison) comparisonStatus('comparison.noRuns');
|
|
437
|
+
if (!runs.size && !app.comparison) comparisonStatus('comparison.noRuns');
|
|
403
438
|
comparisonControls();
|
|
404
439
|
}
|
|
440
|
+
async function validateComparisonSelection(state) {
|
|
441
|
+
if (app.comparisonValidation) return app.comparisonValidation;
|
|
442
|
+
const pair = comparisonPair();
|
|
443
|
+
const generation = app.comparisonSelectionGeneration;
|
|
444
|
+
const recent = new Set((state.runs || []).map((run) => run.runId));
|
|
445
|
+
const ids = [...new Set(Object.values(pair).filter((id) => id && !recent.has(id)))];
|
|
446
|
+
if (!ids.length) return;
|
|
447
|
+
const validation = Promise.all(ids.map(async (runId) => {
|
|
448
|
+
try {
|
|
449
|
+
const result = await api(`/api/run-summary?runId=${encodeURIComponent(runId)}`);
|
|
450
|
+
if (result?.run?.runId !== runId) throw new Error('Invalid run summary response.');
|
|
451
|
+
return { runId, run: result.run };
|
|
452
|
+
} catch (error) { return { runId, error }; }
|
|
453
|
+
}));
|
|
454
|
+
app.comparisonValidation = validation;
|
|
455
|
+
try {
|
|
456
|
+
const outcomes = await validation;
|
|
457
|
+
const selected = comparisonPair();
|
|
458
|
+
if (generation !== app.comparisonSelectionGeneration || selected.baseline !== pair.baseline || selected.current !== pair.current) return;
|
|
459
|
+
let missing = false;
|
|
460
|
+
let updated = false;
|
|
461
|
+
for (const outcome of outcomes) {
|
|
462
|
+
if (outcome.error?.status === 404) {
|
|
463
|
+
for (const role of ['baseline', 'current']) {
|
|
464
|
+
if (selected[role] === outcome.runId) {
|
|
465
|
+
$(role === 'baseline' ? 'compare-baseline' : 'compare-current').value = '';
|
|
466
|
+
app.comparisonPins[role] = null;
|
|
467
|
+
missing = true;
|
|
468
|
+
}
|
|
469
|
+
}
|
|
470
|
+
app.history.runs = app.history.runs.filter((run) => run.runId !== outcome.runId);
|
|
471
|
+
} else if (outcome.run) {
|
|
472
|
+
for (const role of ['baseline', 'current']) if (selected[role] === outcome.runId) app.comparisonPins[role] = outcome.run;
|
|
473
|
+
updated = true;
|
|
474
|
+
}
|
|
475
|
+
}
|
|
476
|
+
if (missing) {
|
|
477
|
+
app.comparisonSelectionGeneration += 1;
|
|
478
|
+
clearComparison();
|
|
479
|
+
if (app.history.loading) refreshHistory(); else renderHistory();
|
|
480
|
+
}
|
|
481
|
+
if (missing || updated) renderComparisonOptions();
|
|
482
|
+
} finally { if (app.comparisonValidation === validation) app.comparisonValidation = null; }
|
|
483
|
+
}
|
|
405
484
|
async function compareRuns() {
|
|
406
485
|
if (!comparisonValid()) return;
|
|
407
486
|
const { baseline, current } = comparisonPair(); const locale = i18n.locale(); const request = ++app.comparisonRequest;
|
|
@@ -478,6 +557,7 @@
|
|
|
478
557
|
if (full || initialize) { targetOptions(); modelForms(); }
|
|
479
558
|
if (previousActive?.kind === 'run' && !state.active) { if (state.lastError) notice(state.lastError, true); else noticeKey('app.runEnded'); }
|
|
480
559
|
if (state.lastError && state.lastError !== app.lastError) { app.lastError = state.lastError; notice(state.lastError, true); }
|
|
560
|
+
await validateComparisonSelection(state);
|
|
481
561
|
}
|
|
482
562
|
async function work(fn) { if (app.busy) return; app.busy = true; readiness(); notice(''); try { await fn(); } catch (error) { showError(error); } finally { app.busy = false; readiness(); } }
|
|
483
563
|
async function loadSuite(file) { await work(async () => { const result = await api(`/api/suite?file=${encodeURIComponent(file)}`); if (!confirmScenarioReplacement('app.confirmReplaceDraft')) return; $('flow-input').value = result.flow || ''; setDraft(result.suite, result.script, 'library', result.flow || '', file, result.suiteHash || ''); noticeKey('app.suiteLoaded'); view('scenario'); }); }
|
|
@@ -660,9 +740,19 @@
|
|
|
660
740
|
$('history-status-filter').addEventListener('change', () => refreshHistory());
|
|
661
741
|
$('history-refresh').addEventListener('click', () => refreshHistory());
|
|
662
742
|
$('history-load-more').addEventListener('click', loadMoreHistory);
|
|
663
|
-
for (const id of ['compare-baseline', 'compare-current']) $(id).addEventListener('change',
|
|
743
|
+
for (const [id, role] of [['compare-baseline', 'baseline'], ['compare-current', 'current']]) $(id).addEventListener('change', () => {
|
|
744
|
+
const run = comparisonRun($(id).value);
|
|
745
|
+
selectComparisonRun(role, run || null);
|
|
746
|
+
});
|
|
664
747
|
$('compare-btn').addEventListener('click', compareRuns);
|
|
665
|
-
$('compare-swap').addEventListener('click', () => {
|
|
748
|
+
$('compare-swap').addEventListener('click', () => {
|
|
749
|
+
const pair = comparisonPair();
|
|
750
|
+
app.comparisonPins = { baseline: comparisonRun(pair.current), current: comparisonRun(pair.baseline) };
|
|
751
|
+
app.comparisonSelectionGeneration += 1;
|
|
752
|
+
renderComparisonOptions();
|
|
753
|
+
$('compare-baseline').value = pair.current; $('compare-current').value = pair.baseline;
|
|
754
|
+
clearComparison(); renderHistory();
|
|
755
|
+
});
|
|
666
756
|
$('comparison-filter').addEventListener('change', renderComparison);
|
|
667
757
|
$('comparison-md').addEventListener('click', downloadComparisonMarkdown);
|
|
668
758
|
$('comparison-json').addEventListener('click', () => { if (app.comparison && comparisonValid()) { const { baseline, current } = comparisonPair(); download(`${baseline}-to-${current}.json`, JSON.stringify(app.comparison, null, 2), 'application/json'); } });
|
|
@@ -84,7 +84,7 @@
|
|
|
84
84
|
'history.eyebrow': ['03 / RUNS', '03 / 실행'],
|
|
85
85
|
'history.intro': ['Only runs that actually started appear here. Once results are saved, you can view the detailed report.', '실제로 시작된 실행만 표시합니다. 결과가 저장되면 상세 보고서를 볼 수 있습니다.'],
|
|
86
86
|
'history.title': ['Runs', '실행 목록'],
|
|
87
|
-
'history.help': ['
|
|
87
|
+
'history.help': ['View a completed report or select a run as the comparison baseline or current run.', '완료된 보고서를 보거나 실행을 비교 기준 또는 현재 실행으로 선택하세요.'],
|
|
88
88
|
'history.searchLabel': ['Search runs', '실행 기록 검색'],
|
|
89
89
|
'history.searchPlaceholder': ['Suite ID, run ID, or target name', '스위트 ID, 실행 ID 또는 대상 이름'],
|
|
90
90
|
'history.statusFilter': ['Run status', '실행 상태'],
|
|
@@ -100,12 +100,14 @@
|
|
|
100
100
|
'history.loadMoreError': ['Could not load more runs. Retry loading more.', '실행 기록을 더 불러오지 못했습니다. 다시 시도하세요.'],
|
|
101
101
|
'history.updatesAvailable': ['Recent runs changed. Refresh history to update this list.', '최근 실행 기록이 변경됐습니다. 실행 기록을 새로고침해 목록을 갱신하세요.'],
|
|
102
102
|
'comparison.title': ['Compare two runs', '실행 2건 비교'],
|
|
103
|
-
'comparison.help': ['
|
|
103
|
+
'comparison.help': ['Select a baseline and current run from history or the lists below. Inspect detailed reports for evidence.', '실행 기록이나 아래 목록에서 기준 실행과 현재 실행을 선택하세요. 증거는 상세 보고서에서 확인하세요.'],
|
|
104
104
|
'comparison.baseline': ['Baseline run', '기준 실행'],
|
|
105
105
|
'comparison.current': ['Current run', '현재 실행'],
|
|
106
106
|
'comparison.choose': ['Choose a run', '실행 선택'],
|
|
107
107
|
'comparison.compare': ['Compare', '비교하기'],
|
|
108
108
|
'comparison.swap': ['Swap runs', '실행 순서 바꾸기'],
|
|
109
|
+
'comparison.useBaseline': ['Use as baseline', '기준으로 선택'],
|
|
110
|
+
'comparison.useCurrent': ['Use as current', '현재로 선택'],
|
|
109
111
|
'comparison.filter': ['Show cases', '케이스 표시'],
|
|
110
112
|
'comparison.all': ['All categories', '전체 분류'],
|
|
111
113
|
'comparison.newlyFailed': ['Newly failed', '새로 실패'],
|
|
@@ -119,9 +121,9 @@
|
|
|
119
121
|
'comparison.openBaseline': ['View baseline report', '기준 실행 보고서 보기'],
|
|
120
122
|
'comparison.openCurrent': ['View current report', '현재 실행 보고서 보기'],
|
|
121
123
|
'comparison.selectBoth': ['Select two different runs to compare.', '서로 다른 실행 2건을 선택하세요.'],
|
|
122
|
-
'comparison.noRuns': ['No
|
|
124
|
+
'comparison.noRuns': ['No runs available to compare.', '비교할 실행이 없습니다.'],
|
|
123
125
|
'comparison.loading': ['Comparing selected runs…', '선택한 실행을 비교하고 있습니다…'],
|
|
124
|
-
'comparison.limit': ['
|
|
126
|
+
'comparison.limit': ['Search or load more in Run history to select from all saved runs. The lists include the 50 most recent runs, loaded history, and your selections. Comparison counts describe observed statuses; they do not establish failure cause or prove a defect was fixed.', '실행 기록을 검색하거나 더 불러와 저장된 전체 실행 중에서 선택하세요. 아래 목록에는 최근 50건, 불러온 기록, 현재 선택이 표시됩니다. 비교 수치는 관찰된 상태만 나타내며 실패 원인이나 결함 수정 여부를 입증하지 않습니다.'],
|
|
125
127
|
'comparison.ready': ['Comparison ready. Categories describe observed case status changes.', '비교 결과가 준비됐습니다. 분류는 관찰된 케이스 상태 차이입니다.'],
|
|
126
128
|
'comparison.noCases': ['No cases in this category.', '이 분류에 해당하는 케이스가 없습니다.'],
|
|
127
129
|
'comparison.warningSuite': ['Suite content changed. Steps or expectations may differ, so these counts do not establish equivalent regressions or fixes.', '스위트 내용이 바뀌었습니다. 단계나 기대 결과가 다를 수 있으므로 이 수치만으로 같은 조건의 회귀나 수정이라고 판단할 수 없습니다.'],
|
|
@@ -93,11 +93,11 @@
|
|
|
93
93
|
</section>
|
|
94
94
|
<section id="view-history" class="view" hidden>
|
|
95
95
|
<div class="page-heading"><p class="eyebrow" data-i18n="history.eyebrow">03 / RUNS</p><h1 data-i18n="nav.history">Run history</h1><p data-i18n="history.intro">Only runs that actually started appear here. Once results are saved, you can view the detailed report.</p></div>
|
|
96
|
-
<section class="list-section"><div class="section-head"><span class="section-number">01</span><div><h2 data-i18n="history.title">Runs</h2><p data-i18n="history.help">
|
|
96
|
+
<section class="list-section"><div class="section-head"><span class="section-number">01</span><div><h2 data-i18n="history.title">Runs</h2><p data-i18n="history.help">View a completed report or select a run as the comparison baseline or current run.</p></div></div><div class="section-body form-stack"><label for="history-search" data-i18n="history.searchLabel">Search runs</label><input id="history-search" type="search" maxlength="200" aria-controls="history-list" placeholder="Suite ID, run ID, or target name" data-i18n-placeholder="history.searchPlaceholder"><label for="history-status-filter" data-i18n="history.statusFilter">Run status</label><select id="history-status-filter" aria-controls="history-list"><option value="" data-i18n="history.statusAll">All statuses</option><option value="passed" data-i18n="status.passed">passed</option><option value="failed" data-i18n="status.failed">failed</option><option value="blocked" data-i18n="status.blocked">blocked</option><option value="error" data-i18n="status.error">error</option><option value="interrupted" data-i18n="status.interrupted">interrupted</option></select><div id="history-list" class="list-body" aria-busy="false"></div><p id="history-page-status" class="hint" role="status" aria-live="polite"></p><div class="actions"><button type="button" id="history-load-more" class="button" data-i18n="history.loadMore" hidden>Load more</button><button type="button" id="history-refresh" class="button quiet" data-i18n="history.refresh">Refresh history</button></div></div></section>
|
|
97
97
|
<section id="comparison-panel" class="comparison-panel" aria-labelledby="comparison-title">
|
|
98
|
-
<div class="section-head"><span class="section-number">02</span><div><h2 id="comparison-title" data-i18n="comparison.title">Compare two runs</h2><p data-i18n="comparison.help">
|
|
98
|
+
<div class="section-head"><span class="section-number">02</span><div><h2 id="comparison-title" data-i18n="comparison.title">Compare two runs</h2><p data-i18n="comparison.help">Select a baseline and current run from history or the lists below. Inspect detailed reports for evidence.</p></div></div>
|
|
99
99
|
<div class="section-body">
|
|
100
|
-
<p class="hint" data-i18n="comparison.limit">
|
|
100
|
+
<p class="hint" data-i18n="comparison.limit">Search or load more in Run history to select from all saved runs. The lists include the 50 most recent runs, loaded history, and your selections. Comparison counts describe observed statuses; they do not establish failure cause or prove a defect was fixed.</p>
|
|
101
101
|
<div class="comparison-selects"><div><label for="compare-baseline" data-i18n="comparison.baseline">Baseline run</label><select id="compare-baseline" aria-controls="comparison-content"><option value="" data-i18n="comparison.choose">Choose a run</option></select></div><div><label for="compare-current" data-i18n="comparison.current">Current run</label><select id="compare-current" aria-controls="comparison-content"><option value="" data-i18n="comparison.choose">Choose a run</option></select></div></div>
|
|
102
102
|
<div class="actions comparison-actions"><button type="button" id="compare-btn" class="button primary" data-i18n="comparison.compare" disabled>Compare</button><button type="button" id="compare-swap" class="button quiet" data-i18n="comparison.swap" disabled>Swap runs</button></div>
|
|
103
103
|
<p id="comparison-status" class="hint" role="status" aria-live="polite"></p>
|
|
@@ -147,6 +147,11 @@ button:focus-visible, input:focus-visible, select:focus-visible, textarea:focus-
|
|
|
147
147
|
.list-row > div { min-width: 0; }
|
|
148
148
|
.list-row strong { display: block; font-size: 15px; overflow-wrap: anywhere; }
|
|
149
149
|
.list-row small { display: block; margin-top: 6px; color: var(--muted); font-size: 12px; line-height: 1.55; overflow-wrap: anywhere; }
|
|
150
|
+
.history-run-row { flex-wrap: wrap; }
|
|
151
|
+
.history-run-row > div:first-child { flex: 1 1 260px; }
|
|
152
|
+
.run-row-actions { display: flex; flex-wrap: wrap; justify-content: flex-end; gap: 8px; }
|
|
153
|
+
.run-row-actions .button[aria-pressed='true'] { border-color: var(--accent-solid); background: var(--accent-solid); color: var(--on-accent); opacity: 1; }
|
|
154
|
+
.run-row-actions .button[aria-pressed='true']:hover:enabled { color: var(--on-accent); filter: brightness(.91); }
|
|
150
155
|
.empty { max-width: 50ch; margin: 0; padding: 32px 0; color: var(--muted); line-height: 1.7; }
|
|
151
156
|
|
|
152
157
|
.comparison-selects { display: grid; grid-template-columns: repeat(2, minmax(0, 1fr)); gap: 18px; }
|
|
@@ -218,6 +223,8 @@ button:focus-visible, input:focus-visible, select:focus-visible, textarea:focus-
|
|
|
218
223
|
.execution-grid { grid-template-columns: 1fr; gap: 16px; }
|
|
219
224
|
.run-bar { align-items: start; flex-direction: column; }
|
|
220
225
|
.list-row { align-items: start; flex-direction: column; gap: 12px; }
|
|
226
|
+
.history-run-row > div:first-child { flex: 0 1 auto; width: 100%; }
|
|
227
|
+
.run-row-actions { justify-content: flex-start; }
|
|
221
228
|
.comparison-selects { grid-template-columns: 1fr; }
|
|
222
229
|
.comparison-count { padding: 9px 5px; }
|
|
223
230
|
.comparison-count strong { font-size: 19px; }
|
package/dist/src/ui/server.js
CHANGED
|
@@ -426,6 +426,12 @@ export async function startUIServer(options) {
|
|
|
426
426
|
}
|
|
427
427
|
return candidates.sort((a, b) => b.mtime - a.mtime || (a.id < b.id ? -1 : a.id > b.id ? 1 : 0));
|
|
428
428
|
}
|
|
429
|
+
function runSummary(item) {
|
|
430
|
+
return { runId: item.runId, suiteId: item.suiteId, status: item.status, startedAt: item.startedAt,
|
|
431
|
+
finishedAt: item.finishedAt, caseCount: item.cases.length,
|
|
432
|
+
passedCount: item.cases.filter((entry) => entry.status === 'passed' || entry.status === 'flaky').length,
|
|
433
|
+
target: item.target, ...(item.selection ? { selection: item.selection } : {}) };
|
|
434
|
+
}
|
|
429
435
|
async function runSummaries(limit, query = '', status = '', after) {
|
|
430
436
|
const history = [];
|
|
431
437
|
for (const candidate of await runCandidates()) {
|
|
@@ -437,11 +443,7 @@ export async function startUIServer(options) {
|
|
|
437
443
|
continue;
|
|
438
444
|
if (query && ![item.suiteId, item.runId, item.target.name].some((field) => field.toLowerCase().includes(query)))
|
|
439
445
|
continue;
|
|
440
|
-
|
|
441
|
-
finishedAt: item.finishedAt, caseCount: item.cases.length,
|
|
442
|
-
passedCount: item.cases.filter((entry) => entry.status === 'passed' || entry.status === 'flaky').length,
|
|
443
|
-
target: item.target, ...(item.selection ? { selection: item.selection } : {}) };
|
|
444
|
-
history.push({ candidate, summary });
|
|
446
|
+
history.push({ candidate, summary: runSummary(item) });
|
|
445
447
|
if (history.length === limit)
|
|
446
448
|
break;
|
|
447
449
|
}
|
|
@@ -521,6 +523,24 @@ export async function startUIServer(options) {
|
|
|
521
523
|
nextCursor: rows.length > limit ? encodeRunCursor(runs.at(-1).candidate, filterHash) : null });
|
|
522
524
|
return;
|
|
523
525
|
}
|
|
526
|
+
if (url.pathname === '/api/run-summary') {
|
|
527
|
+
if (url.searchParams.getAll('runId').length !== 1)
|
|
528
|
+
invalid('A run ID is required.');
|
|
529
|
+
const runId = url.searchParams.get('runId');
|
|
530
|
+
if (!RUN_ID.test(runId))
|
|
531
|
+
invalid('Invalid run ID.');
|
|
532
|
+
let item;
|
|
533
|
+
try {
|
|
534
|
+
({ report: item } = await report(runId, false));
|
|
535
|
+
}
|
|
536
|
+
catch (error) {
|
|
537
|
+
if (error instanceof HTTPError && error.status === 400)
|
|
538
|
+
notFound();
|
|
539
|
+
throw error;
|
|
540
|
+
}
|
|
541
|
+
sendJSON(response, 200, { run: runSummary(item) });
|
|
542
|
+
return;
|
|
543
|
+
}
|
|
524
544
|
if (url.pathname === '/api/suite') {
|
|
525
545
|
const file = url.searchParams.get('file');
|
|
526
546
|
if (!file)
|
package/docs/releases.md
CHANGED
|
@@ -2,6 +2,17 @@
|
|
|
2
2
|
|
|
3
3
|
The alpha entries below are historical release records. They document earlier package versions and checks.
|
|
4
4
|
|
|
5
|
+
## 0.7.0: compare archived runs directly from history
|
|
6
|
+
|
|
7
|
+
History rows now offer **Use as baseline** and **Use as current**, so older runs found by search or paging can be compared. The chosen pair stays available through query/status changes, history refresh, polling, navigation, and language changes. Selected row actions expose their pressed state, and changing a selection or swapping roles clears the old comparison and exports until compared again.
|
|
8
|
+
|
|
9
|
+
A read-only summary lookup validates selected runs outside the recent 50 using the existing safe JSON report loader; comparison does not require a saved Markdown file. A confirmed deleted, invalid, or unsafe report clears its selection and comparison. Temporary request failures preserve the pair, and stale validation responses cannot change a newer selection. Choosing reports preserves the scenario draft, review state, and open detailed report, and never starts QA execution.
|
|
10
|
+
|
|
11
|
+
Locally verified on macOS with Node.js 24.22.1 on 2026-11-06: 118 tests, type checking, UI syntax/design-system audit, all existing UI checks, and the real Chromium demo passed. A dedicated browser smoke used 61 synthetic records to verify comparison beyond the latest 50, JSON-only reports, row selection, filtering/paging, polling, navigation/language preservation, swaps, temporary failures, stale missing-file responses after changing away and back to the same selection, deleted archived files, and a 390px layout. A fresh tarball installation ran seven passing browser cases and verified archived summary lookup and comparison. No model calls, external logins, or native-device runs were used.
|
|
12
|
+
|
|
13
|
+
- Package: [`@ihyeon/flow-qa@0.7.0`](https://www.npmjs.com/package/@ihyeon/flow-qa/v/0.7.0)
|
|
14
|
+
- Release: [GitHub 0.7.0](https://github.com/hyeonse-swing/flow-qa/releases/tag/v0.7.0)
|
|
15
|
+
|
|
5
16
|
## 0.6.0: search and page through the full run history
|
|
6
17
|
|
|
7
18
|
Run-history search and status filters now cover every valid saved report instead of only the latest 50. **Load more** appends up to 50 older matches, and **Refresh history** restarts the list with the same filters. Loaded counts, end-of-results, loading, and retry messages make the current scope explicit. Opening an older report supports the existing downloads and failed/error-case preparation.
|
package/docs/ui.md
CHANGED
|
@@ -76,7 +76,7 @@ Results load 50 at a time, newest first by run-directory modification time, with
|
|
|
76
76
|
|
|
77
77
|
Changing search or status restarts at the first page. The query, status, and loaded pages stay through navigation and language changes, and reset on page reload. The open report, its downloads, and scenario draft stay intact. A delayed response from an older query cannot replace newer results. Older reports support the same report downloads and failed/error-case preparation.
|
|
78
78
|
|
|
79
|
-
When recent run records change, a single loaded page refreshes automatically. If several pages are loaded, the UI keeps them and shows an update hint. **Refresh history** restarts at the first page with the current filters and also picks up external edits or deletions of older records. Pagination is a live view, not a frozen snapshot: newly added records appear when refreshed, and reports changed externally while paging may move. Invalid, incomplete, or unsafe report files are excluded.
|
|
79
|
+
When recent run records change, a single loaded page refreshes automatically. If several pages are loaded, the UI keeps them and shows an update hint. **Refresh history** restarts at the first page with the current filters and also picks up external edits or deletions of older records. Pagination is a live view, not a frozen snapshot: newly added records appear when refreshed, and reports changed externally while paging may move. Invalid, incomplete, or unsafe report files are excluded. Use the row actions to select any listed report for comparison, including older results.
|
|
80
80
|
|
|
81
81
|
### Prepare failed cases
|
|
82
82
|
|
|
@@ -90,13 +90,13 @@ The preparation notice identifies the original run. The next execution creates a
|
|
|
90
90
|
|
|
91
91
|
### Compare two runs
|
|
92
92
|
|
|
93
|
-
From version `0.3.0`, open **Run history → Compare two runs**, choose a **Baseline run** and **Current run**, then compare.
|
|
93
|
+
From version `0.3.0`, open **Run history → Compare two runs**, choose a **Baseline run** and **Current run**, then compare. From `0.7.0`, each history row offers **Use as baseline** and **Use as current**. Search or load older pages to choose any saved run, then press **Compare**. The selectors include the most recent 50 reports, currently loaded history rows, and the two selected runs. Selected runs stay available when searches or filters hide their rows. The reports must belong to the same suite ID and recorded target settings: a matching web URL, or mobile platform, app ID, and device. Target display names may differ. Mobile reports record the configured device selector, which can be a name; all mobile comparisons warn that matching selectors do not establish that the same physical device was used. Comparing does not run tests, call a model, or change case review.
|
|
94
94
|
|
|
95
95
|
Cases are matched by ID. **Newly failed** means only `passed → failed`; **Now passed** means only `failed → passed`. Other status transitions, including flaky, blocked, skipped, and execution errors, appear under **Changed status**. Equal statuses are **Same status**. **Added to current** and **Missing from current** mean present in only one selected report; missing does not mean passed or deleted from the suite. Filter the visible rows by category, open either detailed report for its evidence, or download the complete comparison as Markdown or JSON. Downloads include every category, regardless of the display filter, and do not include evidence files. Markdown follows the existing report URL/link hiding policy; JSON retains the recorded target settings and case text.
|
|
96
96
|
|
|
97
97
|
When suite hashes differ, the UI and exports warn that steps or expectations may have changed. These are recorded outcomes, not proof of equivalent tests, a product regression, or a fixed defect. Additional warnings identify reversed chronological order and interrupted/error runs. The comparison never treats a flaky pass as a clean pass. The existing report format remains unchanged; comparison JSON uses `flow-qa/comparison-1`.
|
|
98
98
|
|
|
99
|
-
Swap reverses the baseline and current selections. Changing either selection clears the old result and downloads until a new comparison succeeds. Navigation, language changes, and history updates preserve valid selections and results; a selected run
|
|
99
|
+
Swap reverses the baseline and current selections. Changing either selection clears the old result and downloads until a new comparison succeeds. Navigation, language changes, and history updates preserve valid selections and results; leaving the latest 50 or the visible history page does not clear a selected run. Background checks validate selected older records directly; a confirmed missing, invalid, or unsafe report clears that selection and invalidates the comparison. Temporary request failures preserve the pair for the next check. Choices reset on page reload. Markdown product labels follow the selected language, while user-written case content and status codes are preserved.
|
|
100
100
|
|
|
101
101
|
### Import existing case files
|
|
102
102
|
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@ihyeon/flow-qa",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.7.0",
|
|
4
4
|
"license": "MIT",
|
|
5
5
|
"author": "Hyeonse Im",
|
|
6
6
|
"repository": {
|
|
@@ -57,7 +57,7 @@
|
|
|
57
57
|
"check:ui": "node --check src/ui/public/app.js && node --check src/ui/public/display.js && node --check src/ui/public/i18n.js && hds-audit .",
|
|
58
58
|
"demo:serve": "node examples/demo/server.ts",
|
|
59
59
|
"demo:smoke": "node scripts/demo-smoke.ts",
|
|
60
|
-
"smoke:ui": "node scripts/ui-i18n-smoke.mjs && node scripts/ui-comparison-smoke.mjs && node scripts/ui-selection-smoke.mjs && node scripts/ui-rerun-smoke.mjs && node scripts/ui-history-smoke.mjs"
|
|
60
|
+
"smoke:ui": "node scripts/ui-i18n-smoke.mjs && node scripts/ui-comparison-smoke.mjs && node scripts/ui-selection-smoke.mjs && node scripts/ui-rerun-smoke.mjs && node scripts/ui-history-smoke.mjs && node scripts/ui-archive-comparison-smoke.mjs"
|
|
61
61
|
},
|
|
62
62
|
"dependencies": {
|
|
63
63
|
"@ai-sdk/openai-compatible": "3.0.53",
|