@ihyeon/flow-qa 0.6.0 → 0.8.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.ko.md +4 -2
- package/README.md +4 -2
- package/dist/src/ui/public/app.js +163 -26
- package/dist/src/ui/public/i18n.js +16 -4
- package/dist/src/ui/public/index.html +4 -4
- package/dist/src/ui/public/style.css +14 -0
- package/dist/src/ui/server.js +25 -5
- package/docs/mobile.md +1 -1
- package/docs/releases.md +22 -0
- package/docs/ui.md +14 -4
- package/package.json +2 -2
package/README.ko.md
CHANGED
|
@@ -16,7 +16,7 @@ CLI와 `127.0.0.1` 전용 웹 UI를 제공합니다. UI는 영어로 시작하
|
|
|
16
16
|
Node.js 24.x(24.22.1 이상) 또는 26.12.1 이상이 필요합니다. 브라우저는 첫 웹 실행 전에 설치합니다.
|
|
17
17
|
|
|
18
18
|
```sh
|
|
19
|
-
npm install -g @ihyeon/flow-qa@0.
|
|
19
|
+
npm install -g @ihyeon/flow-qa@0.8.0
|
|
20
20
|
flow-qa browser install
|
|
21
21
|
flow-qa ui --demo
|
|
22
22
|
```
|
|
@@ -70,9 +70,11 @@ CLI에서는 `flow-qa run cases/task-flow.json --target demo --case TC-001 --cas
|
|
|
70
70
|
|
|
71
71
|
`0.5.0`부터 상세 보고서에서 **실패·오류 케이스 준비**를 누르면 실행 당시와 같은 저장된 케이스를 불러와 `실패`와 `실행 오류`만 선택합니다. 준비 후 케이스와 대상을 다시 검토하고 직접 실행하세요. 케이스·원본 파일이 없거나 바뀌었으면 준비를 중단하며, 실행 대상이 없거나 바뀌었으면 다시 선택해야 합니다. 자세한 동작은 [실패 케이스 준비](docs/ui.md#prepare-failed-cases)를 참고하세요.
|
|
72
72
|
|
|
73
|
+
`0.8.0`부터 **상세 보고서**에서 케이스·오류 내용을 검색하고 상태별로 좁히거나 **실패·오류만** 볼 수 있습니다. 표시 건수와 보고서 전체 케이스 수를 구분하며, 다운로드와 실패·오류 재실행 준비는 보고서 전체 범위를 유지합니다. 자세한 동작은 [보고서 케이스 탐색](docs/ui.md#explore-cases-in-a-report)을 참고하세요.
|
|
74
|
+
|
|
73
75
|
## 실행 결과 비교
|
|
74
76
|
|
|
75
|
-
`0.6.0`부터 **실행 기록**에서 저장된 전체 보고서를 스위트 ID·실행 ID·대상 이름으로 검색하고 실행 상태로 함께 좁힐 수 있습니다. 결과는 50건씩 표시하며 **더 보기**로 이전 기록을 불러옵니다. **실행 기록 새로고침**은 검색·상태 조건을 유지한 채 목록을 갱신합니다. 오래된 보고서도 다운로드와 실패 케이스 준비를 사용할 수 있습니다. 자세한 동작은 [기록 검색·페이지 조회](docs/ui.md#search-run-history)를 참고하세요.
|
|
77
|
+
`0.6.0`부터 **실행 기록**에서 저장된 전체 보고서를 스위트 ID·실행 ID·대상 이름으로 검색하고 실행 상태로 함께 좁힐 수 있습니다. 결과는 50건씩 표시하며 **더 보기**로 이전 기록을 불러옵니다. **실행 기록 새로고침**은 검색·상태 조건을 유지한 채 목록을 갱신합니다. 오래된 보고서도 다운로드와 실패 케이스 준비를 사용할 수 있습니다. `0.7.0`부터 각 기록의 **기준으로 선택**·**현재로 선택**으로 이전 실행끼리 비교할 수 있으며, 검색이나 새로고침 후에도 선택을 유지합니다. 자세한 동작은 [기록 검색·페이지 조회](docs/ui.md#search-run-history)를 참고하세요.
|
|
76
78
|
|
|
77
79
|
**실행 기록 → 실행 2건 비교**에서 같은 스위트와 대상의 두 기록을 선택하면 통과→실패, 실패→통과, 그 외 상태 변경, 동일 상태, 추가·누락 케이스를 확인할 수 있습니다. 분류별로 보거나 전체 비교를 Markdown·JSON으로 내려받으세요. 테스트 내용 변경과 불완전한 실행은 별도로 표시하며, 상태 변화만으로 제품 버그의 발생·해결을 확정하지 않습니다. 자세한 동작은 [UI 가이드](docs/ui.md#compare-two-runs)를 참고하세요.
|
|
78
80
|
|
package/README.md
CHANGED
|
@@ -16,7 +16,7 @@ Flow QA is available as a CLI and a web UI bound to `127.0.0.1`. Provide a targe
|
|
|
16
16
|
Node.js 24.x (24.22.1 or later) or 26.12.1 or later is required. Install Chromium before the first web run:
|
|
17
17
|
|
|
18
18
|
```sh
|
|
19
|
-
npm install -g @ihyeon/flow-qa@0.
|
|
19
|
+
npm install -g @ihyeon/flow-qa@0.8.0
|
|
20
20
|
flow-qa browser install
|
|
21
21
|
flow-qa ui --demo
|
|
22
22
|
```
|
|
@@ -70,9 +70,11 @@ The CLI supports repeated IDs: `flow-qa run cases/task-flow.json --target demo -
|
|
|
70
70
|
|
|
71
71
|
From `0.5.0`, open a detailed report and choose **Prepare failed/error cases** to load its unchanged saved suite with only `failed` and `error` cases selected. Preparation never starts a run: review the cases and target, then run explicitly. Changed or missing case/source files block preparation; a changed or missing target must be selected again. See [preparing failed cases](docs/ui.md#prepare-failed-cases).
|
|
72
72
|
|
|
73
|
+
From `0.8.0`, **Detailed report** supports case/error-text search, status filters, and **Failed/error only**. Counts distinguish displayed matches from all recorded cases. Report downloads and failed/error preparation retain the full report scope. See [report case exploration](docs/ui.md#explore-cases-in-a-report).
|
|
74
|
+
|
|
73
75
|
## Compare recorded runs
|
|
74
76
|
|
|
75
|
-
From `0.6.0`, **Run history** searches all valid saved reports by suite ID, run ID, or target name, combined with the run-status filter. Results load 50 at a time; use **Load more** for older results and **Refresh history** to update the list while keeping your filters. Older reports support the same downloads and failed-case preparation. See [history search and paging](docs/ui.md#search-run-history).
|
|
77
|
+
From `0.6.0`, **Run history** searches all valid saved reports by suite ID, run ID, or target name, combined with the run-status filter. Results load 50 at a time; use **Load more** for older results and **Refresh history** to update the list while keeping your filters. Older reports support the same downloads and failed-case preparation. From `0.7.0`, choose **Use as baseline** and **Use as current** on any history rows to compare older runs, retaining the pair through searches and refreshes. See [history search and paging](docs/ui.md#search-run-history).
|
|
76
78
|
|
|
77
79
|
In **Run history → Compare two runs**, select two reports from the same suite and target to see passed-to-failed, failed-to-passed, other status changes, unchanged, added, and missing cases. Filter the comparison or download all results as Markdown/JSON. Changed test definitions and incomplete runs carry explicit warnings; observed status changes do not prove a product defect was introduced or fixed. See the [UI guide](docs/ui.md#compare-two-runs).
|
|
78
80
|
|
|
@@ -4,7 +4,7 @@
|
|
|
4
4
|
const i18n = window.FlowQAI18n;
|
|
5
5
|
const t = i18n.t;
|
|
6
6
|
const names = { scenario: 'nav.scenario', library: 'nav.library', history: 'nav.history', settings: 'nav.settings' };
|
|
7
|
-
const app = { state: null, connected: false, displayTargets: [], draft: null, selectedCaseIds: new Set(), script: '', savedFile: '', savedHash: '', origin: 'none', sourceFlow: '', boundSource: null, stale: false, dirty: false, busy: false, planning: false, planRequest: 0, stopping: false, report: null, reportMarkdown: '', selectedRun: '', rerunOrigin: null, rerunTargetMismatch: false, navigationVersion: 0, comparison: null, comparisonRequest: 0, comparisonBusy: false, comparisonStatus: 'comparison.selectBoth', history: { runs: [], nextCursor: null, pagesLoaded: 0, loaded: false, loading: false, request: 0, debounce: null, kind: null, error: null, updatesAvailable: false, pendingRecentUpdate: false }, view: 'scenario' };
|
|
7
|
+
const app = { state: null, connected: false, displayTargets: [], draft: null, selectedCaseIds: new Set(), script: '', savedFile: '', savedHash: '', origin: 'none', sourceFlow: '', boundSource: null, stale: false, dirty: false, busy: false, planning: false, planRequest: 0, stopping: false, report: null, reportMarkdown: '', selectedRun: '', reportFilter: { search: '', status: 'all' }, rerunOrigin: null, rerunTargetMismatch: false, navigationVersion: 0, comparison: null, comparisonRequest: 0, comparisonBusy: false, comparisonStatus: 'comparison.selectBoth', comparisonPins: { baseline: null, current: null }, comparisonSelectionGeneration: 0, comparisonValidation: null, history: { runs: [], nextCursor: null, pagesLoaded: 0, loaded: false, loading: false, request: 0, debounce: null, kind: null, error: null, updatesAvailable: false, pendingRecentUpdate: false }, view: 'scenario' };
|
|
8
8
|
|
|
9
9
|
function el(tag, className, value) {
|
|
10
10
|
const node = document.createElement(tag);
|
|
@@ -38,8 +38,10 @@
|
|
|
38
38
|
try { result = await response.json(); } catch { throw localizedError('app.responseUnreadable', { status: response.status }); }
|
|
39
39
|
if (!response.ok) {
|
|
40
40
|
const issues = Array.isArray(result?.issues) ? result.issues.slice(0, 8).map((issue) => `${issue.path}: ${issue.message}`).join('\n') : '';
|
|
41
|
-
|
|
42
|
-
|
|
41
|
+
const error = !result?.error && !issues ? localizedError('app.requestFailed', { status: response.status })
|
|
42
|
+
: new Error(`${result?.error || t('app.requestFailed', { status: response.status })}${issues ? `\n${issues}` : ''}`);
|
|
43
|
+
error.status = response.status;
|
|
44
|
+
throw error;
|
|
43
45
|
}
|
|
44
46
|
return result;
|
|
45
47
|
}
|
|
@@ -282,10 +284,20 @@
|
|
|
282
284
|
: app.state?.active ? 'app.historyActiveEmpty' : 'app.historyEmpty';
|
|
283
285
|
host.append(el('p', 'empty', t(key)));
|
|
284
286
|
}
|
|
287
|
+
const pair = comparisonPair();
|
|
285
288
|
for (const run of history.runs) {
|
|
286
|
-
const row = el('div', 'list-row
|
|
289
|
+
const row = el('div', 'list-row history-run-row'); row.dataset.runId = run.runId;
|
|
290
|
+
const copy = el('div'); copy.append(el('strong', '', `${run.suiteId} · ${i18n.status(run.status)}`), el('small', '', t('app.runMeta', { date: formatDate(run.startedAt), target: targetDisplay(run.target), passed: run.passedCount ?? 0, total: run.caseCount ?? 0, id: run.runId })));
|
|
287
291
|
const scope = selectionScope(run.selection); if (scope) copy.append(el('small', 'selection-scope', scope));
|
|
288
|
-
const
|
|
292
|
+
const actions = el('div', 'run-row-actions');
|
|
293
|
+
const button = el('button', 'button', t('app.viewReport')); button.type = 'button'; button.dataset.action = 'view-report'; button.disabled = run.status === 'running' || run.status === 'stopping'; button.addEventListener('click', () => loadReport(run.runId)); actions.append(button);
|
|
294
|
+
for (const [role, label] of [['baseline', 'comparison.useBaseline'], ['current', 'comparison.useCurrent']]) {
|
|
295
|
+
const select = el('button', 'button quiet', t(label)); select.type = 'button'; select.dataset.action = role === 'baseline' ? 'compare-baseline' : 'compare-current';
|
|
296
|
+
select.setAttribute('aria-pressed', String(pair[role] === run.runId));
|
|
297
|
+
select.disabled = run.status === 'running' || run.status === 'stopping';
|
|
298
|
+
select.addEventListener('click', () => selectComparisonRun(role, run)); actions.append(select);
|
|
299
|
+
}
|
|
300
|
+
row.append(copy, actions); host.append(row);
|
|
289
301
|
}
|
|
290
302
|
const messages = [];
|
|
291
303
|
if (history.loaded) messages.push(t('history.pageCount', { count: history.runs.length }));
|
|
@@ -320,6 +332,7 @@
|
|
|
320
332
|
history.loaded = true;
|
|
321
333
|
history.error = null;
|
|
322
334
|
if (!append) history.updatesAvailable = false;
|
|
335
|
+
renderComparisonOptions();
|
|
323
336
|
} catch {
|
|
324
337
|
if (request !== history.request) return;
|
|
325
338
|
history.error = append ? 'more' : 'first';
|
|
@@ -341,7 +354,7 @@
|
|
|
341
354
|
const request = ++history.request;
|
|
342
355
|
history.runs = []; history.nextCursor = null; history.pagesLoaded = 0; history.loaded = false;
|
|
343
356
|
history.loading = true; history.kind = 'first'; history.error = null; history.updatesAvailable = false; history.pendingRecentUpdate = false;
|
|
344
|
-
renderHistory();
|
|
357
|
+
renderHistory(); renderComparisonOptions();
|
|
345
358
|
const filters = historyFilters();
|
|
346
359
|
if (delay) history.debounce = setTimeout(() => { history.debounce = null; void fetchHistoryPage(request, filters); }, delay);
|
|
347
360
|
else void fetchHistoryPage(request, filters);
|
|
@@ -367,7 +380,18 @@
|
|
|
367
380
|
const comparisonCategoryKeys = { 'newly-failed': 'comparison.newlyFailed', 'now-passed': 'comparison.nowPassed', changed: 'comparison.changed', unchanged: 'comparison.unchanged', added: 'comparison.added', missing: 'comparison.missing' };
|
|
368
381
|
const comparisonWarningKeys = { 'suite-changed': 'comparison.warningSuite', 'reverse-order': 'comparison.warningReverse', 'incomplete-run': 'comparison.warningIncomplete', 'mobile-identity-unverified': 'comparison.warningMobileIdentity', 'partial-run': 'comparison.warningPartial', 'selection-changed': 'comparison.warningSelectionChanged' };
|
|
369
382
|
function comparisonPair() { return { baseline: $('compare-baseline').value, current: $('compare-current').value }; }
|
|
370
|
-
function
|
|
383
|
+
function comparisonRun(runId) {
|
|
384
|
+
return (app.state?.runs || []).find((run) => run.runId === runId)
|
|
385
|
+
|| app.history.runs.find((run) => run.runId === runId)
|
|
386
|
+
|| Object.values(app.comparisonPins).find((run) => run?.runId === runId);
|
|
387
|
+
}
|
|
388
|
+
function comparisonValid() {
|
|
389
|
+
const { baseline, current } = comparisonPair();
|
|
390
|
+
return !!baseline && !!current && baseline !== current && [baseline, current].every((id) => {
|
|
391
|
+
const run = comparisonRun(id);
|
|
392
|
+
return run && run.status !== 'running' && run.status !== 'stopping';
|
|
393
|
+
});
|
|
394
|
+
}
|
|
371
395
|
function comparisonStatus(key, params = {}, error = false) {
|
|
372
396
|
app.comparisonStatus = key; app.comparisonStatusParams = params;
|
|
373
397
|
const node = $('comparison-status'); node.textContent = t(key, params); node.classList.toggle('report-errors', error);
|
|
@@ -385,23 +409,78 @@
|
|
|
385
409
|
$('comparison-summary').replaceChildren(); $('comparison-warnings').replaceChildren(); $('comparison-content').replaceChildren();
|
|
386
410
|
comparisonStatus('comparison.selectBoth'); comparisonControls();
|
|
387
411
|
}
|
|
412
|
+
function selectComparisonRun(role, run) {
|
|
413
|
+
if (run && (run.status === 'running' || run.status === 'stopping')) return;
|
|
414
|
+
const select = $(role === 'baseline' ? 'compare-baseline' : 'compare-current');
|
|
415
|
+
const runId = run?.runId || '';
|
|
416
|
+
if (select.value === runId && (app.comparisonPins[role]?.runId || '') === runId) return;
|
|
417
|
+
app.comparisonPins[role] = run || null;
|
|
418
|
+
app.comparisonSelectionGeneration += 1;
|
|
419
|
+
renderComparisonOptions();
|
|
420
|
+
select.value = runId;
|
|
421
|
+
clearComparison(); renderHistory();
|
|
422
|
+
}
|
|
388
423
|
function renderComparisonOptions() {
|
|
389
|
-
const runs =
|
|
390
|
-
|
|
424
|
+
const runs = new Map();
|
|
425
|
+
for (const run of [...(app.state?.runs || []).slice(0, 50), ...app.history.runs, ...Object.values(app.comparisonPins)]) {
|
|
426
|
+
if (run?.runId && !runs.has(run.runId)) runs.set(run.runId, run);
|
|
427
|
+
}
|
|
391
428
|
for (const id of ['compare-baseline', 'compare-current']) {
|
|
392
429
|
const select = $(id); const selected = select.value;
|
|
393
430
|
select.replaceChildren(el('option', '', t('comparison.choose'))); select.firstChild.value = '';
|
|
394
|
-
for (const run of runs) {
|
|
431
|
+
for (const run of runs.values()) {
|
|
395
432
|
const option = el('option', '', `${t('comparison.runLabel', { id: run.runId, shortId: run.runId.slice(-8), date: formatDate(run.startedAt), suite: run.suiteId, target: targetDisplay(run.target), status: i18n.status(run.status) })}${selectionScope(run.selection) ? ` · ${selectionScope(run.selection)}` : ''}`);
|
|
396
|
-
option.value = run.runId; select.append(option);
|
|
433
|
+
option.value = run.runId; option.disabled = run.status === 'running' || run.status === 'stopping'; select.append(option);
|
|
397
434
|
}
|
|
398
|
-
select.value = runs.
|
|
399
|
-
if (selected && !select.value) missing = true;
|
|
435
|
+
select.value = runs.has(selected) ? selected : '';
|
|
400
436
|
}
|
|
401
|
-
if (
|
|
402
|
-
if (!runs.length && !app.comparison) comparisonStatus('comparison.noRuns');
|
|
437
|
+
if (!runs.size && !app.comparison) comparisonStatus('comparison.noRuns');
|
|
403
438
|
comparisonControls();
|
|
404
439
|
}
|
|
440
|
+
async function validateComparisonSelection(state) {
|
|
441
|
+
if (app.comparisonValidation) return app.comparisonValidation;
|
|
442
|
+
const pair = comparisonPair();
|
|
443
|
+
const generation = app.comparisonSelectionGeneration;
|
|
444
|
+
const recent = new Set((state.runs || []).map((run) => run.runId));
|
|
445
|
+
const ids = [...new Set(Object.values(pair).filter((id) => id && !recent.has(id)))];
|
|
446
|
+
if (!ids.length) return;
|
|
447
|
+
const validation = Promise.all(ids.map(async (runId) => {
|
|
448
|
+
try {
|
|
449
|
+
const result = await api(`/api/run-summary?runId=${encodeURIComponent(runId)}`);
|
|
450
|
+
if (result?.run?.runId !== runId) throw new Error('Invalid run summary response.');
|
|
451
|
+
return { runId, run: result.run };
|
|
452
|
+
} catch (error) { return { runId, error }; }
|
|
453
|
+
}));
|
|
454
|
+
app.comparisonValidation = validation;
|
|
455
|
+
try {
|
|
456
|
+
const outcomes = await validation;
|
|
457
|
+
const selected = comparisonPair();
|
|
458
|
+
if (generation !== app.comparisonSelectionGeneration || selected.baseline !== pair.baseline || selected.current !== pair.current) return;
|
|
459
|
+
let missing = false;
|
|
460
|
+
let updated = false;
|
|
461
|
+
for (const outcome of outcomes) {
|
|
462
|
+
if (outcome.error?.status === 404) {
|
|
463
|
+
for (const role of ['baseline', 'current']) {
|
|
464
|
+
if (selected[role] === outcome.runId) {
|
|
465
|
+
$(role === 'baseline' ? 'compare-baseline' : 'compare-current').value = '';
|
|
466
|
+
app.comparisonPins[role] = null;
|
|
467
|
+
missing = true;
|
|
468
|
+
}
|
|
469
|
+
}
|
|
470
|
+
app.history.runs = app.history.runs.filter((run) => run.runId !== outcome.runId);
|
|
471
|
+
} else if (outcome.run) {
|
|
472
|
+
for (const role of ['baseline', 'current']) if (selected[role] === outcome.runId) app.comparisonPins[role] = outcome.run;
|
|
473
|
+
updated = true;
|
|
474
|
+
}
|
|
475
|
+
}
|
|
476
|
+
if (missing) {
|
|
477
|
+
app.comparisonSelectionGeneration += 1;
|
|
478
|
+
clearComparison();
|
|
479
|
+
if (app.history.loading) refreshHistory(); else renderHistory();
|
|
480
|
+
}
|
|
481
|
+
if (missing || updated) renderComparisonOptions();
|
|
482
|
+
} finally { if (app.comparisonValidation === validation) app.comparisonValidation = null; }
|
|
483
|
+
}
|
|
405
484
|
async function compareRuns() {
|
|
406
485
|
if (!comparisonValid()) return;
|
|
407
486
|
const { baseline, current } = comparisonPair(); const locale = i18n.locale(); const request = ++app.comparisonRequest;
|
|
@@ -478,6 +557,7 @@
|
|
|
478
557
|
if (full || initialize) { targetOptions(); modelForms(); }
|
|
479
558
|
if (previousActive?.kind === 'run' && !state.active) { if (state.lastError) notice(state.lastError, true); else noticeKey('app.runEnded'); }
|
|
480
559
|
if (state.lastError && state.lastError !== app.lastError) { app.lastError = state.lastError; notice(state.lastError, true); }
|
|
560
|
+
await validateComparisonSelection(state);
|
|
481
561
|
}
|
|
482
562
|
async function work(fn) { if (app.busy) return; app.busy = true; readiness(); notice(''); try { await fn(); } catch (error) { showError(error); } finally { app.busy = false; readiness(); } }
|
|
483
563
|
async function loadSuite(file) { await work(async () => { const result = await api(`/api/suite?file=${encodeURIComponent(file)}`); if (!confirmScenarioReplacement('app.confirmReplaceDraft')) return; $('flow-input').value = result.flow || ''; setDraft(result.suite, result.script, 'library', result.flow || '', file, result.suiteHash || ''); noticeKey('app.suiteLoaded'); view('scenario'); }); }
|
|
@@ -556,7 +636,7 @@
|
|
|
556
636
|
catch (error) { showError(error); }
|
|
557
637
|
finally { app.stopping = false; readiness(); }
|
|
558
638
|
}
|
|
559
|
-
async function loadReport(runId) { await work(async () => { const result = await api(`/api/report?runId=${encodeURIComponent(runId)}`); app.report = result.report; app.reportMarkdown = result.markdown; app.selectedRun = runId; renderReport(); readiness(); $('report-panel').hidden = false; $('report-panel').scrollIntoView({ behavior: 'smooth', block: 'start' }); }); }
|
|
639
|
+
async function loadReport(runId) { await work(async () => { const result = await api(`/api/report?runId=${encodeURIComponent(runId)}`); if (app.selectedRun !== runId) app.reportFilter = { search: '', status: 'all' }; app.report = result.report; app.reportMarkdown = result.markdown; app.selectedRun = runId; renderReport(); readiness(); $('report-panel').hidden = false; $('report-panel').scrollIntoView({ behavior: 'smooth', block: 'start' }); }); }
|
|
560
640
|
async function prepareFailedCases() { await work(async () => {
|
|
561
641
|
const runId = app.selectedRun;
|
|
562
642
|
const report = app.report;
|
|
@@ -591,14 +671,48 @@
|
|
|
591
671
|
download(`${runId}.md`, result.markdown, 'text/markdown');
|
|
592
672
|
} catch (error) { if (runId === app.selectedRun && locale === i18n.locale()) showError(error); }
|
|
593
673
|
}
|
|
594
|
-
|
|
595
|
-
|
|
596
|
-
|
|
597
|
-
const
|
|
598
|
-
|
|
599
|
-
|
|
600
|
-
|
|
601
|
-
|
|
674
|
+
const reportStatuses = ['all', 'failed-error', 'passed', 'failed', 'error', 'blocked', 'flaky', 'skipped'];
|
|
675
|
+
function reportCaseMatchesSearch(item, query) {
|
|
676
|
+
if (!query) return true;
|
|
677
|
+
const values = [item.id, item.title, item.reason, ...(item.expected || [])];
|
|
678
|
+
for (const attempt of item.attempts || []) {
|
|
679
|
+
if (attempt.error) values.push(attempt.error.code, attempt.error.message);
|
|
680
|
+
for (const error of attempt.secondaryErrors || []) values.push(error.code, error.message);
|
|
681
|
+
for (const step of attempt.steps || []) {
|
|
682
|
+
values.push(step.label);
|
|
683
|
+
if (step.error) values.push(step.error.code, step.error.message);
|
|
684
|
+
}
|
|
685
|
+
}
|
|
686
|
+
return values.some((value) => String(value ?? '').toLowerCase().includes(query));
|
|
687
|
+
}
|
|
688
|
+
function renderReportFilters() {
|
|
689
|
+
const cases = app.report?.cases || [];
|
|
690
|
+
const counts = Object.fromEntries(reportStatuses.map((status) => [status, 0]));
|
|
691
|
+
counts.all = cases.length;
|
|
692
|
+
for (const item of cases) {
|
|
693
|
+
if (Object.hasOwn(counts, item.status)) counts[item.status] += 1;
|
|
694
|
+
if (item.status === 'failed' || item.status === 'error') counts['failed-error'] += 1;
|
|
695
|
+
}
|
|
696
|
+
for (const option of $('report-case-status').options) {
|
|
697
|
+
const label = option.value === 'all' ? t('report.filterAll') : option.value === 'failed-error' ? t('report.filterFailedError') : i18n.status(option.value);
|
|
698
|
+
option.textContent = `${label} (${counts[option.value]})`;
|
|
699
|
+
}
|
|
700
|
+
if ($('report-case-search').value !== app.reportFilter.search) $('report-case-search').value = app.reportFilter.search;
|
|
701
|
+
$('report-case-status').value = app.reportFilter.status;
|
|
702
|
+
$('report-case-reset').disabled = !app.reportFilter.search && app.reportFilter.status === 'all';
|
|
703
|
+
return cases;
|
|
704
|
+
}
|
|
705
|
+
function renderReportCases() {
|
|
706
|
+
if (!app.report) return;
|
|
707
|
+
const cases = renderReportFilters();
|
|
708
|
+
const query = app.reportFilter.search.trim().toLowerCase();
|
|
709
|
+
const status = app.reportFilter.status;
|
|
710
|
+
const shown = cases.filter((item) => (status === 'all' || (status === 'failed-error' ? item.status === 'failed' || item.status === 'error' : item.status === status)) && reportCaseMatchesSearch(item, query));
|
|
711
|
+
$('report-case-count').textContent = t('report.caseCount', { shown: shown.length, total: cases.length });
|
|
712
|
+
const host = $('report-case-list'); replace(host);
|
|
713
|
+
if (!cases.length) { host.append(el('p', 'empty', t('report.empty'))); return; }
|
|
714
|
+
if (!shown.length) { host.append(el('p', 'empty', t('report.noMatches'))); return; }
|
|
715
|
+
for (const item of shown) {
|
|
602
716
|
const section = el('section', 'case-report'); section.append(el('h3', '', `${item.id} · ${item.title} · ${i18n.status(item.status)}`));
|
|
603
717
|
if (item.reason) section.append(el('p', '', t('app.reason', { reason: item.reason })));
|
|
604
718
|
section.append(el('p', '', t('app.expected'))); const expected = el('ul', 'compact-list'); for (const check of item.expected || []) expected.append(el('li', '', check)); section.append(expected);
|
|
@@ -622,6 +736,16 @@
|
|
|
622
736
|
host.append(section);
|
|
623
737
|
}
|
|
624
738
|
}
|
|
739
|
+
function renderReport() {
|
|
740
|
+
const report = app.report; if (!report) return;
|
|
741
|
+
$('report-heading').textContent = `${report.runId} · ${i18n.status(report.status)} · ${targetDisplay(report.target)}`;
|
|
742
|
+
const host = $('report-content'); replace(host);
|
|
743
|
+
const meta = el('div', 'report-meta'); for (const value of [t('app.reportStatus', { status: i18n.status(report.status) }), t('app.reportTarget', { target: targetDisplay(report.target) }), t('app.reportStarted', { date: formatDate(report.startedAt) }), t('app.reportFinished', { date: formatDate(report.finishedAt) })]) meta.append(el('span', '', value)); const scope = selectionScope(report.selection); if (scope) meta.append(el('span', 'selection-scope', scope)); host.append(meta);
|
|
744
|
+
if (report.usage) host.append(el('p', 'hint', t('app.usage', { tokens: report.usage.modelTokens ?? t('app.unavailable'), cost: typeof report.usage.estimatedCostUsd === 'number' ? `$${report.usage.estimatedCostUsd.toFixed(4)}` : t('app.unavailable') })));
|
|
745
|
+
if (report.errors?.length) host.append(el('p', 'report-errors', report.errors.join('\n')));
|
|
746
|
+
const caseList = el('div'); caseList.id = 'report-case-list'; host.append(caseList);
|
|
747
|
+
renderReportCases();
|
|
748
|
+
}
|
|
625
749
|
async function saveModel(event) { event.preventDefault(); const form = event.currentTarget; await work(async () => { const values = new FormData(form); const body = { role: form.dataset.role, baseURL: String(values.get('baseURL') || '').trim(), model: String(values.get('model') || '').trim() }; const apiKeyEnv = String(values.get('apiKeyEnv') || '').trim(); if (apiKeyEnv) body.apiKeyEnv = apiKeyEnv; await api('/api/models', body); await refresh(true); noticeKey('app.modelSaved'); }); }
|
|
626
750
|
async function clearModel(role) { await work(async () => { await api('/api/models', { role, clear: true }); await refresh(true); noticeKey('app.modelCleared'); }); }
|
|
627
751
|
function targetFields() { const mobile = $('target-kind').value !== 'web'; for (const [id, required] of [['target-url-field', !mobile], ['target-bundle-field', mobile], ['target-device-field', mobile]]) { const field = $(id); field.hidden = !required; field.querySelector('input').required = required; } }
|
|
@@ -660,9 +784,19 @@
|
|
|
660
784
|
$('history-status-filter').addEventListener('change', () => refreshHistory());
|
|
661
785
|
$('history-refresh').addEventListener('click', () => refreshHistory());
|
|
662
786
|
$('history-load-more').addEventListener('click', loadMoreHistory);
|
|
663
|
-
for (const id of ['compare-baseline', 'compare-current']) $(id).addEventListener('change',
|
|
787
|
+
for (const [id, role] of [['compare-baseline', 'baseline'], ['compare-current', 'current']]) $(id).addEventListener('change', () => {
|
|
788
|
+
const run = comparisonRun($(id).value);
|
|
789
|
+
selectComparisonRun(role, run || null);
|
|
790
|
+
});
|
|
664
791
|
$('compare-btn').addEventListener('click', compareRuns);
|
|
665
|
-
$('compare-swap').addEventListener('click', () => {
|
|
792
|
+
$('compare-swap').addEventListener('click', () => {
|
|
793
|
+
const pair = comparisonPair();
|
|
794
|
+
app.comparisonPins = { baseline: comparisonRun(pair.current), current: comparisonRun(pair.baseline) };
|
|
795
|
+
app.comparisonSelectionGeneration += 1;
|
|
796
|
+
renderComparisonOptions();
|
|
797
|
+
$('compare-baseline').value = pair.current; $('compare-current').value = pair.baseline;
|
|
798
|
+
clearComparison(); renderHistory();
|
|
799
|
+
});
|
|
666
800
|
$('comparison-filter').addEventListener('change', renderComparison);
|
|
667
801
|
$('comparison-md').addEventListener('click', downloadComparisonMarkdown);
|
|
668
802
|
$('comparison-json').addEventListener('click', () => { if (app.comparison && comparisonValid()) { const { baseline, current } = comparisonPair(); download(`${baseline}-to-${current}.json`, JSON.stringify(app.comparison, null, 2), 'application/json'); } });
|
|
@@ -699,6 +833,9 @@
|
|
|
699
833
|
$('report-md').addEventListener('click', downloadReportMarkdown);
|
|
700
834
|
$('report-json').addEventListener('click', () => { if (app.report) download(`${app.selectedRun}.json`, JSON.stringify(app.report, null, 2), 'application/json'); });
|
|
701
835
|
$('report-rerun').addEventListener('click', prepareFailedCases);
|
|
836
|
+
$('report-case-search').addEventListener('input', (event) => { app.reportFilter.search = event.currentTarget.value; renderReportCases(); });
|
|
837
|
+
$('report-case-status').addEventListener('change', (event) => { app.reportFilter.status = event.currentTarget.value; renderReportCases(); });
|
|
838
|
+
$('report-case-reset').addEventListener('click', () => { app.reportFilter = { search: '', status: 'all' }; renderReportCases(); });
|
|
702
839
|
$('target-form').addEventListener('submit', saveTarget);
|
|
703
840
|
$('target-kind').addEventListener('change', targetFields); targetFields();
|
|
704
841
|
$('language-select').addEventListener('change', (event) => i18n.setLocale(event.currentTarget.value));
|
|
@@ -84,7 +84,7 @@
|
|
|
84
84
|
'history.eyebrow': ['03 / RUNS', '03 / 실행'],
|
|
85
85
|
'history.intro': ['Only runs that actually started appear here. Once results are saved, you can view the detailed report.', '실제로 시작된 실행만 표시합니다. 결과가 저장되면 상세 보고서를 볼 수 있습니다.'],
|
|
86
86
|
'history.title': ['Runs', '실행 목록'],
|
|
87
|
-
'history.help': ['
|
|
87
|
+
'history.help': ['View a completed report or select a run as the comparison baseline or current run.', '완료된 보고서를 보거나 실행을 비교 기준 또는 현재 실행으로 선택하세요.'],
|
|
88
88
|
'history.searchLabel': ['Search runs', '실행 기록 검색'],
|
|
89
89
|
'history.searchPlaceholder': ['Suite ID, run ID, or target name', '스위트 ID, 실행 ID 또는 대상 이름'],
|
|
90
90
|
'history.statusFilter': ['Run status', '실행 상태'],
|
|
@@ -100,12 +100,14 @@
|
|
|
100
100
|
'history.loadMoreError': ['Could not load more runs. Retry loading more.', '실행 기록을 더 불러오지 못했습니다. 다시 시도하세요.'],
|
|
101
101
|
'history.updatesAvailable': ['Recent runs changed. Refresh history to update this list.', '최근 실행 기록이 변경됐습니다. 실행 기록을 새로고침해 목록을 갱신하세요.'],
|
|
102
102
|
'comparison.title': ['Compare two runs', '실행 2건 비교'],
|
|
103
|
-
'comparison.help': ['
|
|
103
|
+
'comparison.help': ['Select a baseline and current run from history or the lists below. Inspect detailed reports for evidence.', '실행 기록이나 아래 목록에서 기준 실행과 현재 실행을 선택하세요. 증거는 상세 보고서에서 확인하세요.'],
|
|
104
104
|
'comparison.baseline': ['Baseline run', '기준 실행'],
|
|
105
105
|
'comparison.current': ['Current run', '현재 실행'],
|
|
106
106
|
'comparison.choose': ['Choose a run', '실행 선택'],
|
|
107
107
|
'comparison.compare': ['Compare', '비교하기'],
|
|
108
108
|
'comparison.swap': ['Swap runs', '실행 순서 바꾸기'],
|
|
109
|
+
'comparison.useBaseline': ['Use as baseline', '기준으로 선택'],
|
|
110
|
+
'comparison.useCurrent': ['Use as current', '현재로 선택'],
|
|
109
111
|
'comparison.filter': ['Show cases', '케이스 표시'],
|
|
110
112
|
'comparison.all': ['All categories', '전체 분류'],
|
|
111
113
|
'comparison.newlyFailed': ['Newly failed', '새로 실패'],
|
|
@@ -119,9 +121,9 @@
|
|
|
119
121
|
'comparison.openBaseline': ['View baseline report', '기준 실행 보고서 보기'],
|
|
120
122
|
'comparison.openCurrent': ['View current report', '현재 실행 보고서 보기'],
|
|
121
123
|
'comparison.selectBoth': ['Select two different runs to compare.', '서로 다른 실행 2건을 선택하세요.'],
|
|
122
|
-
'comparison.noRuns': ['No
|
|
124
|
+
'comparison.noRuns': ['No runs available to compare.', '비교할 실행이 없습니다.'],
|
|
123
125
|
'comparison.loading': ['Comparing selected runs…', '선택한 실행을 비교하고 있습니다…'],
|
|
124
|
-
'comparison.limit': ['
|
|
126
|
+
'comparison.limit': ['Search or load more in Run history to select from all saved runs. The lists include the 50 most recent runs, loaded history, and your selections. Comparison counts describe observed statuses; they do not establish failure cause or prove a defect was fixed.', '실행 기록을 검색하거나 더 불러와 저장된 전체 실행 중에서 선택하세요. 아래 목록에는 최근 50건, 불러온 기록, 현재 선택이 표시됩니다. 비교 수치는 관찰된 상태만 나타내며 실패 원인이나 결함 수정 여부를 입증하지 않습니다.'],
|
|
125
127
|
'comparison.ready': ['Comparison ready. Categories describe observed case status changes.', '비교 결과가 준비됐습니다. 분류는 관찰된 케이스 상태 차이입니다.'],
|
|
126
128
|
'comparison.noCases': ['No cases in this category.', '이 분류에 해당하는 케이스가 없습니다.'],
|
|
127
129
|
'comparison.warningSuite': ['Suite content changed. Steps or expectations may differ, so these counts do not establish equivalent regressions or fixes.', '스위트 내용이 바뀌었습니다. 단계나 기대 결과가 다를 수 있으므로 이 수치만으로 같은 조건의 회귀나 수정이라고 판단할 수 없습니다.'],
|
|
@@ -141,6 +143,16 @@
|
|
|
141
143
|
'comparison.error': ['Comparison failed: {message}', '비교 실패: {message}'],
|
|
142
144
|
'comparison.runLabel': ['{date} · {suite} · {shortId} · {target} · {status} · {id}', '{date} · {suite} · {shortId} · {target} · {status} · {id}'],
|
|
143
145
|
'history.report': ['Detailed report', '상세 보고서'],
|
|
146
|
+
'report.searchLabel': ['Search cases', '케이스 검색'],
|
|
147
|
+
'report.searchPlaceholder': ['Search ID, title, expected result, or errors', 'ID, 제목, 기대 결과 또는 오류 검색'],
|
|
148
|
+
'report.statusLabel': ['Case status', '케이스 상태'],
|
|
149
|
+
'report.filterAll': ['All statuses', '전체 상태'],
|
|
150
|
+
'report.filterFailedError': ['Failed/error only', '실패·오류만'],
|
|
151
|
+
'report.reset': ['Reset filters', '필터 초기화'],
|
|
152
|
+
'report.caseCount': ['Showing {shown} of {total} report cases', '보고서 케이스 {total}개 중 {shown}개 표시'],
|
|
153
|
+
'report.empty': ['This report has no cases.', '이 보고서에는 케이스가 없습니다.'],
|
|
154
|
+
'report.noMatches': ['No cases match these filters.', '필터에 맞는 케이스가 없습니다.'],
|
|
155
|
+
'report.filterHint': ['Filters only change the displayed cases. Downloads include all cases, and failed/error preparation uses all recorded failed/error cases regardless of visibility.', '필터는 표시되는 케이스에만 적용됩니다. 다운로드에는 모든 케이스가 포함되고, 실패·오류 준비에는 표시 여부와 관계없이 기록된 모든 실패·오류 케이스가 사용됩니다.'],
|
|
144
156
|
'history.prepareFailed': ['Prepare failed/error cases', '실패·오류 케이스 준비'],
|
|
145
157
|
'history.prepareHint': ['Only failed and error cases are selected. Passed, flaky, blocked, and skipped cases are excluded. Preparation never starts a run.', '실패·오류 케이스만 선택합니다. 통과·간헐 실패·차단·건너뜀 케이스는 제외됩니다. 준비만으로 실행은 시작되지 않습니다.'],
|
|
146
158
|
'history.downloadMarkdown': ['Download Markdown', 'Markdown 다운로드'],
|
|
@@ -93,18 +93,18 @@
|
|
|
93
93
|
</section>
|
|
94
94
|
<section id="view-history" class="view" hidden>
|
|
95
95
|
<div class="page-heading"><p class="eyebrow" data-i18n="history.eyebrow">03 / RUNS</p><h1 data-i18n="nav.history">Run history</h1><p data-i18n="history.intro">Only runs that actually started appear here. Once results are saved, you can view the detailed report.</p></div>
|
|
96
|
-
<section class="list-section"><div class="section-head"><span class="section-number">01</span><div><h2 data-i18n="history.title">Runs</h2><p data-i18n="history.help">
|
|
96
|
+
<section class="list-section"><div class="section-head"><span class="section-number">01</span><div><h2 data-i18n="history.title">Runs</h2><p data-i18n="history.help">View a completed report or select a run as the comparison baseline or current run.</p></div></div><div class="section-body form-stack"><label for="history-search" data-i18n="history.searchLabel">Search runs</label><input id="history-search" type="search" maxlength="200" aria-controls="history-list" placeholder="Suite ID, run ID, or target name" data-i18n-placeholder="history.searchPlaceholder"><label for="history-status-filter" data-i18n="history.statusFilter">Run status</label><select id="history-status-filter" aria-controls="history-list"><option value="" data-i18n="history.statusAll">All statuses</option><option value="passed" data-i18n="status.passed">passed</option><option value="failed" data-i18n="status.failed">failed</option><option value="blocked" data-i18n="status.blocked">blocked</option><option value="error" data-i18n="status.error">error</option><option value="interrupted" data-i18n="status.interrupted">interrupted</option></select><div id="history-list" class="list-body" aria-busy="false"></div><p id="history-page-status" class="hint" role="status" aria-live="polite"></p><div class="actions"><button type="button" id="history-load-more" class="button" data-i18n="history.loadMore" hidden>Load more</button><button type="button" id="history-refresh" class="button quiet" data-i18n="history.refresh">Refresh history</button></div></div></section>
|
|
97
97
|
<section id="comparison-panel" class="comparison-panel" aria-labelledby="comparison-title">
|
|
98
|
-
<div class="section-head"><span class="section-number">02</span><div><h2 id="comparison-title" data-i18n="comparison.title">Compare two runs</h2><p data-i18n="comparison.help">
|
|
98
|
+
<div class="section-head"><span class="section-number">02</span><div><h2 id="comparison-title" data-i18n="comparison.title">Compare two runs</h2><p data-i18n="comparison.help">Select a baseline and current run from history or the lists below. Inspect detailed reports for evidence.</p></div></div>
|
|
99
99
|
<div class="section-body">
|
|
100
|
-
<p class="hint" data-i18n="comparison.limit">
|
|
100
|
+
<p class="hint" data-i18n="comparison.limit">Search or load more in Run history to select from all saved runs. The lists include the 50 most recent runs, loaded history, and your selections. Comparison counts describe observed statuses; they do not establish failure cause or prove a defect was fixed.</p>
|
|
101
101
|
<div class="comparison-selects"><div><label for="compare-baseline" data-i18n="comparison.baseline">Baseline run</label><select id="compare-baseline" aria-controls="comparison-content"><option value="" data-i18n="comparison.choose">Choose a run</option></select></div><div><label for="compare-current" data-i18n="comparison.current">Current run</label><select id="compare-current" aria-controls="comparison-content"><option value="" data-i18n="comparison.choose">Choose a run</option></select></div></div>
|
|
102
102
|
<div class="actions comparison-actions"><button type="button" id="compare-btn" class="button primary" data-i18n="comparison.compare" disabled>Compare</button><button type="button" id="compare-swap" class="button quiet" data-i18n="comparison.swap" disabled>Swap runs</button></div>
|
|
103
103
|
<p id="comparison-status" class="hint" role="status" aria-live="polite"></p>
|
|
104
104
|
<div id="comparison-results" hidden><div id="comparison-warnings"></div><div id="comparison-summary"></div><div class="comparison-filter-row"><label for="comparison-filter" data-i18n="comparison.filter">Show cases</label><select id="comparison-filter" aria-controls="comparison-content"><option value="" data-i18n="comparison.all">All categories</option><option value="newly-failed" data-i18n="comparison.newlyFailed">Newly failed</option><option value="now-passed" data-i18n="comparison.nowPassed">Now passed</option><option value="changed" data-i18n="comparison.changed">Changed status</option><option value="unchanged" data-i18n="comparison.unchanged">Same status</option><option value="added" data-i18n="comparison.added">Added to current</option><option value="missing" data-i18n="comparison.missing">Missing from current</option></select></div><div class="actions comparison-actions"><button type="button" id="comparison-md" class="button" data-i18n="comparison.downloadMarkdown" disabled>Download comparison Markdown</button><button type="button" id="comparison-json" class="button quiet" data-i18n="comparison.downloadJson" disabled>Download comparison JSON</button><button type="button" id="compare-open-baseline" class="button quiet" data-i18n="comparison.openBaseline" disabled>View baseline report</button><button type="button" id="compare-open-current" class="button quiet" data-i18n="comparison.openCurrent" disabled>View current report</button></div><div id="comparison-content"></div></div>
|
|
105
105
|
</div>
|
|
106
106
|
</section>
|
|
107
|
-
<section id="report-panel" class="report-panel" hidden><div class="section-head"><span class="section-number">03</span><div><h2 data-i18n="history.report">Detailed report</h2><p id="report-heading"></p></div></div><div class="section-body"><div class="actions"><button type="button" id="report-md" class="button" data-i18n="history.downloadMarkdown">Download Markdown</button><button type="button" id="report-json" class="button quiet" data-i18n="history.downloadJson">Download JSON</button><button type="button" id="report-rerun" class="button primary" disabled data-i18n="history.prepareFailed">Prepare failed/error cases</button></div><p class="hint" data-i18n="history.prepareHint">Passed, flaky, blocked, and skipped cases are excluded. Preparation does not start a run.</p><p class="hint" data-i18n="history.evidenceHint">The report summarizes results. Open or download screenshots and recordings from each attempt. Evidence files are not included in report downloads.</p><div id="report-content"></div></div></section>
|
|
107
|
+
<section id="report-panel" class="report-panel" hidden><div class="section-head"><span class="section-number">03</span><div><h2 data-i18n="history.report">Detailed report</h2><p id="report-heading"></p></div></div><div class="section-body"><div class="actions"><button type="button" id="report-md" class="button" data-i18n="history.downloadMarkdown">Download Markdown</button><button type="button" id="report-json" class="button quiet" data-i18n="history.downloadJson">Download JSON</button><button type="button" id="report-rerun" class="button primary" disabled data-i18n="history.prepareFailed">Prepare failed/error cases</button></div><p class="hint" data-i18n="history.prepareHint">Passed, flaky, blocked, and skipped cases are excluded. Preparation does not start a run.</p><p class="hint" data-i18n="history.evidenceHint">The report summarizes results. Open or download screenshots and recordings from each attempt. Evidence files are not included in report downloads.</p><div class="report-case-filters"><div><label for="report-case-search" data-i18n="report.searchLabel">Search cases</label><input id="report-case-search" type="search" aria-controls="report-case-list" placeholder="Search ID, title, expected result, or errors" data-i18n-placeholder="report.searchPlaceholder"></div><div><label for="report-case-status" data-i18n="report.statusLabel">Case status</label><select id="report-case-status" aria-controls="report-case-list"><option value="all">All</option><option value="failed-error">Failed/error only</option><option value="passed">passed</option><option value="failed">failed</option><option value="error">error</option><option value="blocked">blocked</option><option value="flaky">flaky</option><option value="skipped">skipped</option></select></div><button type="button" id="report-case-reset" class="button quiet" data-i18n="report.reset">Reset filters</button></div><p id="report-case-count" class="hint" role="status" aria-live="polite"></p><p class="hint" data-i18n="report.filterHint">Filters only change the displayed cases. Downloads include all cases, and failed/error preparation uses all recorded failed/error cases regardless of visibility.</p><div id="report-content"></div></div></section>
|
|
108
108
|
</section>
|
|
109
109
|
<section id="view-settings" class="view" hidden>
|
|
110
110
|
<div class="page-heading"><p class="eyebrow" data-i18n="settings.eyebrow">04 / CONNECTIONS</p><h1 data-i18n="nav.settings">Connections</h1><p data-i18n="settings.intro">Configure run targets and models separately. Key values are read only from server environment variables.</p></div>
|
|
@@ -105,6 +105,11 @@ button:focus-visible, input:focus-visible, select:focus-visible, textarea:focus-
|
|
|
105
105
|
.runtime-panel > p { margin: 0; overflow-wrap: anywhere; }
|
|
106
106
|
#case-readiness { margin: 0 0 12px; }
|
|
107
107
|
#report-content { margin-top: 20px; }
|
|
108
|
+
.report-case-filters { display: grid; grid-template-columns: minmax(0, 2fr) minmax(0, 1fr) auto; align-items: end; gap: 12px; margin-top: 24px; }
|
|
109
|
+
.report-case-filters > div { min-width: 0; }
|
|
110
|
+
.report-case-filters label { display: block; margin-bottom: 8px; font-size: 13px; font-weight: 750; }
|
|
111
|
+
.report-case-filters input, .report-case-filters select { display: block; width: 100%; min-width: 0; min-height: 44px; padding: 10px 12px; border: 1px solid var(--line); border-radius: var(--radius-control); background: var(--paper); color: var(--ink); }
|
|
112
|
+
#report-case-count { margin: 12px 0 4px; }
|
|
108
113
|
.blocked-title { margin-top: 21px !important; color: var(--accent-text); }
|
|
109
114
|
.draft-actions { display: flex; align-items: center; flex-wrap: wrap; gap: 8px 14px; margin: calc(var(--space-factor) * 16px) 0; }
|
|
110
115
|
.draft-actions span { color: var(--muted); font-size: 12px; }
|
|
@@ -147,6 +152,11 @@ button:focus-visible, input:focus-visible, select:focus-visible, textarea:focus-
|
|
|
147
152
|
.list-row > div { min-width: 0; }
|
|
148
153
|
.list-row strong { display: block; font-size: 15px; overflow-wrap: anywhere; }
|
|
149
154
|
.list-row small { display: block; margin-top: 6px; color: var(--muted); font-size: 12px; line-height: 1.55; overflow-wrap: anywhere; }
|
|
155
|
+
.history-run-row { flex-wrap: wrap; }
|
|
156
|
+
.history-run-row > div:first-child { flex: 1 1 260px; }
|
|
157
|
+
.run-row-actions { display: flex; flex-wrap: wrap; justify-content: flex-end; gap: 8px; }
|
|
158
|
+
.run-row-actions .button[aria-pressed='true'] { border-color: var(--accent-solid); background: var(--accent-solid); color: var(--on-accent); opacity: 1; }
|
|
159
|
+
.run-row-actions .button[aria-pressed='true']:hover:enabled { color: var(--on-accent); filter: brightness(.91); }
|
|
150
160
|
.empty { max-width: 50ch; margin: 0; padding: 32px 0; color: var(--muted); line-height: 1.7; }
|
|
151
161
|
|
|
152
162
|
.comparison-selects { display: grid; grid-template-columns: repeat(2, minmax(0, 1fr)); gap: 18px; }
|
|
@@ -208,6 +218,8 @@ button:focus-visible, input:focus-visible, select:focus-visible, textarea:focus-
|
|
|
208
218
|
.settings-grid { grid-template-columns: 1fr; }
|
|
209
219
|
}
|
|
210
220
|
@media (max-width: 550px) {
|
|
221
|
+
.report-case-filters { grid-template-columns: 1fr; }
|
|
222
|
+
.report-case-filters .button { justify-self: start; }
|
|
211
223
|
.page-heading h1 { font-size: clamp(36px, 10vw, 46px); }
|
|
212
224
|
.page-heading > p:last-child { font-size: 14px; }
|
|
213
225
|
.topline-right { display: none; }
|
|
@@ -218,6 +230,8 @@ button:focus-visible, input:focus-visible, select:focus-visible, textarea:focus-
|
|
|
218
230
|
.execution-grid { grid-template-columns: 1fr; gap: 16px; }
|
|
219
231
|
.run-bar { align-items: start; flex-direction: column; }
|
|
220
232
|
.list-row { align-items: start; flex-direction: column; gap: 12px; }
|
|
233
|
+
.history-run-row > div:first-child { flex: 0 1 auto; width: 100%; }
|
|
234
|
+
.run-row-actions { justify-content: flex-start; }
|
|
221
235
|
.comparison-selects { grid-template-columns: 1fr; }
|
|
222
236
|
.comparison-count { padding: 9px 5px; }
|
|
223
237
|
.comparison-count strong { font-size: 19px; }
|
package/dist/src/ui/server.js
CHANGED
|
@@ -426,6 +426,12 @@ export async function startUIServer(options) {
|
|
|
426
426
|
}
|
|
427
427
|
return candidates.sort((a, b) => b.mtime - a.mtime || (a.id < b.id ? -1 : a.id > b.id ? 1 : 0));
|
|
428
428
|
}
|
|
429
|
+
function runSummary(item) {
|
|
430
|
+
return { runId: item.runId, suiteId: item.suiteId, status: item.status, startedAt: item.startedAt,
|
|
431
|
+
finishedAt: item.finishedAt, caseCount: item.cases.length,
|
|
432
|
+
passedCount: item.cases.filter((entry) => entry.status === 'passed' || entry.status === 'flaky').length,
|
|
433
|
+
target: item.target, ...(item.selection ? { selection: item.selection } : {}) };
|
|
434
|
+
}
|
|
429
435
|
async function runSummaries(limit, query = '', status = '', after) {
|
|
430
436
|
const history = [];
|
|
431
437
|
for (const candidate of await runCandidates()) {
|
|
@@ -437,11 +443,7 @@ export async function startUIServer(options) {
|
|
|
437
443
|
continue;
|
|
438
444
|
if (query && ![item.suiteId, item.runId, item.target.name].some((field) => field.toLowerCase().includes(query)))
|
|
439
445
|
continue;
|
|
440
|
-
|
|
441
|
-
finishedAt: item.finishedAt, caseCount: item.cases.length,
|
|
442
|
-
passedCount: item.cases.filter((entry) => entry.status === 'passed' || entry.status === 'flaky').length,
|
|
443
|
-
target: item.target, ...(item.selection ? { selection: item.selection } : {}) };
|
|
444
|
-
history.push({ candidate, summary });
|
|
446
|
+
history.push({ candidate, summary: runSummary(item) });
|
|
445
447
|
if (history.length === limit)
|
|
446
448
|
break;
|
|
447
449
|
}
|
|
@@ -521,6 +523,24 @@ export async function startUIServer(options) {
|
|
|
521
523
|
nextCursor: rows.length > limit ? encodeRunCursor(runs.at(-1).candidate, filterHash) : null });
|
|
522
524
|
return;
|
|
523
525
|
}
|
|
526
|
+
if (url.pathname === '/api/run-summary') {
|
|
527
|
+
if (url.searchParams.getAll('runId').length !== 1)
|
|
528
|
+
invalid('A run ID is required.');
|
|
529
|
+
const runId = url.searchParams.get('runId');
|
|
530
|
+
if (!RUN_ID.test(runId))
|
|
531
|
+
invalid('Invalid run ID.');
|
|
532
|
+
let item;
|
|
533
|
+
try {
|
|
534
|
+
({ report: item } = await report(runId, false));
|
|
535
|
+
}
|
|
536
|
+
catch (error) {
|
|
537
|
+
if (error instanceof HTTPError && error.status === 400)
|
|
538
|
+
notFound();
|
|
539
|
+
throw error;
|
|
540
|
+
}
|
|
541
|
+
sendJSON(response, 200, { run: runSummary(item) });
|
|
542
|
+
return;
|
|
543
|
+
}
|
|
524
544
|
if (url.pathname === '/api/suite') {
|
|
525
545
|
const file = url.searchParams.get('file');
|
|
526
546
|
if (!file)
|
package/docs/mobile.md
CHANGED
|
@@ -9,7 +9,7 @@ Choose **Mobile** on the scenario screen, describe the flow, and generate or dow
|
|
|
9
9
|
With the installed CLI:
|
|
10
10
|
|
|
11
11
|
```sh
|
|
12
|
-
npm install -g @ihyeon/flow-qa@0.
|
|
12
|
+
npm install -g @ihyeon/flow-qa@0.8.0
|
|
13
13
|
flow-qa init ./my-qa
|
|
14
14
|
# Write your mobile flow in my-qa/flows/my-flow.md first.
|
|
15
15
|
flow-qa plan flows/my-flow.md --platform mobile --prompt-only --workspace ./my-qa
|
package/docs/releases.md
CHANGED
|
@@ -2,6 +2,28 @@
|
|
|
2
2
|
|
|
3
3
|
The alpha entries below are historical release records. They document earlier package versions and checks.
|
|
4
4
|
|
|
5
|
+
## 0.8.0: explore cases and errors within a report
|
|
6
|
+
|
|
7
|
+
Detailed reports now support case-content search and status filters, including **Failed/error only**. Search covers IDs, titles, reasons, expectations, attempt/secondary errors, and step labels/errors. Status choices show counts from the complete open report, while the match count describes displayed cases. Partial-run scope and report-level warnings remain visible; empty reports are distinguished from filters with no matches.
|
|
8
|
+
|
|
9
|
+
Filters persist through navigation, language changes, polling, and reopening the same report. Opening a different report successfully resets them; a failed load preserves the existing view. Downloads and failed/error preparation continue to use every recorded case, and evidence URLs retain original attempt/artifact indices. No report, execution, or API schema changes are introduced. Installation examples now use this release.
|
|
10
|
+
|
|
11
|
+
Locally verified on macOS with Node.js 24.22.1 on 2026-11-06: 118 tests, type checking, UI syntax/design-system audit, all existing UI checks, and the real Chromium demo passed. A dedicated browser smoke covered all six case statuses, case/expectation/error/step search, intersecting filters, complete Markdown/JSON exports, original evidence indices, full failed/error preparation scope, same/new/unreadable report transitions, partial and empty reports, draft/review preservation, English/Korean, and a 390px layout. A fresh tarball installation ran seven passing browser cases and exercised the installed report filter UI and complete JSON download. No model calls, external logins, or native-device runs were used.
|
|
12
|
+
|
|
13
|
+
- Package: [`@ihyeon/flow-qa@0.8.0`](https://www.npmjs.com/package/@ihyeon/flow-qa/v/0.8.0)
|
|
14
|
+
- Release: [GitHub 0.8.0](https://github.com/hyeonse-swing/flow-qa/releases/tag/v0.8.0)
|
|
15
|
+
|
|
16
|
+
## 0.7.0: compare archived runs directly from history
|
|
17
|
+
|
|
18
|
+
History rows now offer **Use as baseline** and **Use as current**, so older runs found by search or paging can be compared. The chosen pair stays available through query/status changes, history refresh, polling, navigation, and language changes. Selected row actions expose their pressed state, and changing a selection or swapping roles clears the old comparison and exports until compared again.
|
|
19
|
+
|
|
20
|
+
A read-only summary lookup validates selected runs outside the recent 50 using the existing safe JSON report loader; comparison does not require a saved Markdown file. A confirmed deleted, invalid, or unsafe report clears its selection and comparison. Temporary request failures preserve the pair, and stale validation responses cannot change a newer selection. Choosing reports preserves the scenario draft, review state, and open detailed report, and never starts QA execution.
|
|
21
|
+
|
|
22
|
+
Locally verified on macOS with Node.js 24.22.1 on 2026-11-06: 118 tests, type checking, UI syntax/design-system audit, all existing UI checks, and the real Chromium demo passed. A dedicated browser smoke used 61 synthetic records to verify comparison beyond the latest 50, JSON-only reports, row selection, filtering/paging, polling, navigation/language preservation, swaps, temporary failures, stale missing-file responses after changing away and back to the same selection, deleted archived files, and a 390px layout. A fresh tarball installation ran seven passing browser cases and verified archived summary lookup and comparison. No model calls, external logins, or native-device runs were used.
|
|
23
|
+
|
|
24
|
+
- Package: [`@ihyeon/flow-qa@0.7.0`](https://www.npmjs.com/package/@ihyeon/flow-qa/v/0.7.0)
|
|
25
|
+
- Release: [GitHub 0.7.0](https://github.com/hyeonse-swing/flow-qa/releases/tag/v0.7.0)
|
|
26
|
+
|
|
5
27
|
## 0.6.0: search and page through the full run history
|
|
6
28
|
|
|
7
29
|
Run-history search and status filters now cover every valid saved report instead of only the latest 50. **Load more** appends up to 50 older matches, and **Refresh history** restarts the list with the same filters. Loaded counts, end-of-results, loading, and retry messages make the current scope explicit. Opening an older report supports the existing downloads and failed/error-case preparation.
|
package/docs/ui.md
CHANGED
|
@@ -7,7 +7,7 @@ Describe a user flow, review cases, and configure a target when ready to run. We
|
|
|
7
7
|
Node.js 24.x (24.22.1 or later) or 26.12.1 or later is required. Node 25 is unsupported. Upgrade the Node runtime and restart any running UI process; updating Flow QA alone does not patch Node. Install Flow QA and start the web demo:
|
|
8
8
|
|
|
9
9
|
```sh
|
|
10
|
-
npm install -g @ihyeon/flow-qa@0.
|
|
10
|
+
npm install -g @ihyeon/flow-qa@0.8.0
|
|
11
11
|
flow-qa browser install
|
|
12
12
|
flow-qa ui --demo
|
|
13
13
|
```
|
|
@@ -76,7 +76,17 @@ Results load 50 at a time, newest first by run-directory modification time, with
|
|
|
76
76
|
|
|
77
77
|
Changing search or status restarts at the first page. The query, status, and loaded pages stay through navigation and language changes, and reset on page reload. The open report, its downloads, and scenario draft stay intact. A delayed response from an older query cannot replace newer results. Older reports support the same report downloads and failed/error-case preparation.
|
|
78
78
|
|
|
79
|
-
When recent run records change, a single loaded page refreshes automatically. If several pages are loaded, the UI keeps them and shows an update hint. **Refresh history** restarts at the first page with the current filters and also picks up external edits or deletions of older records. Pagination is a live view, not a frozen snapshot: newly added records appear when refreshed, and reports changed externally while paging may move. Invalid, incomplete, or unsafe report files are excluded.
|
|
79
|
+
When recent run records change, a single loaded page refreshes automatically. If several pages are loaded, the UI keeps them and shows an update hint. **Refresh history** restarts at the first page with the current filters and also picks up external edits or deletions of older records. Pagination is a live view, not a frozen snapshot: newly added records appear when refreshed, and reports changed externally while paging may move. Invalid, incomplete, or unsafe report files are excluded. Use the row actions to select any listed report for comparison, including older results.
|
|
80
|
+
|
|
81
|
+
### Explore cases in a report
|
|
82
|
+
|
|
83
|
+
From `0.8.0`, open **Run history → Detailed report** and use **Search cases** with **Case status** to narrow the displayed results. Search ignores case and surrounding whitespace, and matches case IDs, titles, reasons, expected results, attempt errors, secondary errors, step labels, and step errors. User-written Korean text is supported. Artifact paths and translated status labels are not search fields.
|
|
84
|
+
|
|
85
|
+
**Failed/error only** shows those two recorded statuses; the other status choices distinguish passed, failed, error, blocked, flaky, and skipped. Status-option counts describe the entire open report, while the displayed count shows matches out of that report's recorded cases. A partial run's omitted cases are not included in those counts; its selected/full-suite scope remains visible separately. Report-level warnings remain visible even when no case matches. **Reset filters** clears both search and status.
|
|
86
|
+
|
|
87
|
+
Filters stay while navigating, changing language, refreshing history/settings, or reopening the same report. Successfully opening another report resets them; a failed report load preserves the current report and filters. Filters reset on page reload. Evidence links still refer to each case's original attempt and artifact.
|
|
88
|
+
|
|
89
|
+
These controls change only displayed cases. Markdown and JSON downloads contain every recorded case, and **Prepare failed/error cases** still prepares every recorded failed/error case from the report, even if hidden by the current filters. Filtering does not alter a saved report, the scenario draft, review confirmation, or execution selection.
|
|
80
90
|
|
|
81
91
|
### Prepare failed cases
|
|
82
92
|
|
|
@@ -90,13 +100,13 @@ The preparation notice identifies the original run. The next execution creates a
|
|
|
90
100
|
|
|
91
101
|
### Compare two runs
|
|
92
102
|
|
|
93
|
-
From version `0.3.0`, open **Run history → Compare two runs**, choose a **Baseline run** and **Current run**, then compare.
|
|
103
|
+
From version `0.3.0`, open **Run history → Compare two runs**, choose a **Baseline run** and **Current run**, then compare. From `0.7.0`, each history row offers **Use as baseline** and **Use as current**. Search or load older pages to choose any saved run, then press **Compare**. The selectors include the most recent 50 reports, currently loaded history rows, and the two selected runs. Selected runs stay available when searches or filters hide their rows. The reports must belong to the same suite ID and recorded target settings: a matching web URL, or mobile platform, app ID, and device. Target display names may differ. Mobile reports record the configured device selector, which can be a name; all mobile comparisons warn that matching selectors do not establish that the same physical device was used. Comparing does not run tests, call a model, or change case review.
|
|
94
104
|
|
|
95
105
|
Cases are matched by ID. **Newly failed** means only `passed → failed`; **Now passed** means only `failed → passed`. Other status transitions, including flaky, blocked, skipped, and execution errors, appear under **Changed status**. Equal statuses are **Same status**. **Added to current** and **Missing from current** mean present in only one selected report; missing does not mean passed or deleted from the suite. Filter the visible rows by category, open either detailed report for its evidence, or download the complete comparison as Markdown or JSON. Downloads include every category, regardless of the display filter, and do not include evidence files. Markdown follows the existing report URL/link hiding policy; JSON retains the recorded target settings and case text.
|
|
96
106
|
|
|
97
107
|
When suite hashes differ, the UI and exports warn that steps or expectations may have changed. These are recorded outcomes, not proof of equivalent tests, a product regression, or a fixed defect. Additional warnings identify reversed chronological order and interrupted/error runs. The comparison never treats a flaky pass as a clean pass. The existing report format remains unchanged; comparison JSON uses `flow-qa/comparison-1`.
|
|
98
108
|
|
|
99
|
-
Swap reverses the baseline and current selections. Changing either selection clears the old result and downloads until a new comparison succeeds. Navigation, language changes, and history updates preserve valid selections and results; a selected run
|
|
109
|
+
Swap reverses the baseline and current selections. Changing either selection clears the old result and downloads until a new comparison succeeds. Navigation, language changes, and history updates preserve valid selections and results; leaving the latest 50 or the visible history page does not clear a selected run. Background checks validate selected older records directly; a confirmed missing, invalid, or unsafe report clears that selection and invalidates the comparison. Temporary request failures preserve the pair for the next check. Choices reset on page reload. Markdown product labels follow the selected language, while user-written case content and status codes are preserved.
|
|
100
110
|
|
|
101
111
|
### Import existing case files
|
|
102
112
|
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@ihyeon/flow-qa",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.8.0",
|
|
4
4
|
"license": "MIT",
|
|
5
5
|
"author": "Hyeonse Im",
|
|
6
6
|
"repository": {
|
|
@@ -57,7 +57,7 @@
|
|
|
57
57
|
"check:ui": "node --check src/ui/public/app.js && node --check src/ui/public/display.js && node --check src/ui/public/i18n.js && hds-audit .",
|
|
58
58
|
"demo:serve": "node examples/demo/server.ts",
|
|
59
59
|
"demo:smoke": "node scripts/demo-smoke.ts",
|
|
60
|
-
"smoke:ui": "node scripts/ui-i18n-smoke.mjs && node scripts/ui-comparison-smoke.mjs && node scripts/ui-selection-smoke.mjs && node scripts/ui-rerun-smoke.mjs && node scripts/ui-history-smoke.mjs"
|
|
60
|
+
"smoke:ui": "node scripts/ui-i18n-smoke.mjs && node scripts/ui-comparison-smoke.mjs && node scripts/ui-selection-smoke.mjs && node scripts/ui-rerun-smoke.mjs && node scripts/ui-history-smoke.mjs && node scripts/ui-archive-comparison-smoke.mjs && node scripts/ui-report-explorer-smoke.mjs"
|
|
61
61
|
},
|
|
62
62
|
"dependencies": {
|
|
63
63
|
"@ai-sdk/openai-compatible": "3.0.53",
|