verikun 0.16.0 → 0.18.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -104,7 +104,7 @@ vk screenshot # -> ./.verikun/screen.png
104
104
  | Command | Description |
105
105
  |---|---|
106
106
  | `ai <file> [--model m] [--max-cost-usd n] [--timeout dur] [--cost-override in/out] [--effort e] [--package pkg] [--app-build id] [--server url] [--show-plan] [--recompile] [--json]` | Run a plain-English test: compile it to a deterministic plan once, replay it model-free, and self-heal failures via the model. Needs `ANTHROPIC_API_KEY` or `OPENAI_API_KEY` (per model), or no key with `--model codex-cli` / `cursor-cli` (a logged-in `codex` / `cursor-agent` CLI). See [AI](#ai--natural-language-tests). |
107
- | `suite <dir> [--app <id>] [--name n] [--server url] [--json]` (+ all `ai` flags) | Run every `*.md` in `<dir>` as one sequential suite with an overview report and a non-zero exit on failure — the CI gate. See [Suites](#suites--run-a-directory-of-tests). |
107
+ | `suite <dir> [--app <id>] [--name n] [--retries n] [--server url] [--json]` (+ all `ai` flags) | Run every `*.md` in `<dir>` as one sequential suite with an overview report and a non-zero exit on failure — the CI gate. See [Suites](#suites--run-a-directory-of-tests). |
108
108
 
109
109
  ### Remote
110
110
  | Command | Description |
@@ -122,7 +122,7 @@ vk screenshot # -> ./.verikun/screen.png
122
122
  |---|---|
123
123
  | `run start [name] [--force]` | Begin a named run. One auto-starts on the first action if you don't. |
124
124
  | `run status` | Show the active run and its recorded steps. |
125
- | `run archive [name]` | Write JUnit + HTML report to `./.verikun/runs/<id>/`; exits non-zero if any step failed. |
125
+ | `run archive [name] [--no-logs]` | Write JUnit + HTML report to `./.verikun/runs/<id>/`; exits non-zero if any step failed. Captures `artifacts/logcat.txt` by default. |
126
126
  | `run clear` | Discard the active run without a report. |
127
127
 
128
128
  ## Test runs & reports
@@ -130,10 +130,11 @@ vk screenshot # -> ./.verikun/screen.png
130
130
  Actions are recorded into a **test run** — one auto-starts on the first action
131
131
  (set `VERIKUN_NO_RUN=1` to disable). Every command becomes a step with its
132
132
  timing, the selector + identifier it resolved through, and pass/fail; a failing
133
- step also captures a screenshot **and** the UI hierarchy of the page. When a
134
- step fails you can additionally run `vk log <package>` to pull the device logs —
135
- that step records the logs **into the same run**, so the crash trace shows up in
136
- the report alongside the failure.
133
+ step also captures a screenshot **and** the UI hierarchy of the page. Device
134
+ logs are captured automatically at archive time into `artifacts/logcat.txt`
135
+ (session-scoped when possible). You can still run `vk log <package>` mid-run to
136
+ pull a snapshot into a step for the report; `--no-logs` / `VERIKUN_NO_LOGS`
137
+ skips the archive dump on green runs (failures still capture).
137
138
 
138
139
  `vk run archive` finalizes the run into `./.verikun/runs/<id>/`:
139
140
 
@@ -141,8 +142,11 @@ the report alongside the failure.
141
142
  for failed assertions, `<error>` for environment errors, and the resolved
142
143
  identifier in `<system-out>`. Drops straight into CI.
143
144
  - **`report.html`** — a self-contained report: every step, the identifiers used,
144
- any screenshots taken, the screenshot + hierarchy of any failed page, and any
145
- device logs captured via `vk log`.
145
+ any screenshots taken, the screenshot + hierarchy of any failed page, a link to
146
+ the full device log, an app-scoped log accordion when the run launched an app,
147
+ and any per-step logs from `vk log`.
148
+ - **`artifacts/logcat.txt`** — full device log for the run window (default).
149
+ - **`artifacts/logcat-app.txt`** — app-scoped log (when a package/bundle was launched).
146
150
  - **`run.json`** — the raw recording.
147
151
 
148
152
  `vk run archive` exits non-zero when the run contained failures, so the same
@@ -287,12 +291,29 @@ vk suite tests/ --app com.example.app --server "$VERIKUN_SERVER" # remote devi
287
291
  - **Each test is a full `vk ai` run** — plan cache, self-healing, cost budget, and
288
292
  its own archived JUnit + HTML report under `./.verikun/runs/<id>/`. A test that
289
293
  fails (or errors) doesn't stop the suite; the rest still run.
290
- - **But a broken *environment* does stop it.** If a test dies from an environment
291
- error (exit 3tool gone, device unplugged, server unreachable), the toolchain is
292
- re-probed; only if it is *still* broken does the suite abort. That re-probe matters:
293
- a transient `uiautomator` dump failure also exits 3, and shouldn't vaporize a
294
- 20-test run. Continuing on a genuinely dead box just produces one identical red row
295
- per remaining test noise that reads exactly like a mass regression.
294
+ - **`--retries N` recovers from flakes.** A failed test is re-run up to N times
295
+ (default `0`opt-in, so CI cost/time stay predictable). If a later attempt
296
+ passes, the suite exits `0` and the flake is a **warning**, not a hard failure.
297
+ Failed attempt archives stay linked from the suite overview (`attempts` on the
298
+ test row + a `warnings` list on the manifest), so flakiness remains visible.
299
+ Cost and duration sum across attempts.
300
+ - **What earns a retry:** anything that might come out differently — a flaky
301
+ selector, a wedged app, and **a broken environment**, including a `vk server`
302
+ connection dropping mid-suite. The bias is intentional: an attempt costs one test,
303
+ giving up costs the whole suite plus a human rerunning it. Environment retries wait
304
+ a little longer each time (an outage that survives the health probe usually needs
305
+ seconds, not milliseconds) and each one lands in `warnings`, so riding out a wobble
306
+ is never silent. Exactly two failures are never retried, because a rerun cannot
307
+ change them: a **budget abort** (each attempt gets its own ceiling, so it would just
308
+ re-abort having spent twice) and a **usage error** (exit `2` — an unreadable test
309
+ file, a payload the server refuses).
310
+ - **But a broken *environment* does stop it, once the attempts are gone.** If a test
311
+ dies from an environment error (exit 3 — tool gone, device unplugged, server
312
+ unreachable), the toolchain is re-probed; only if it is *still* broken **and** no
313
+ retries remain does the suite abort. That re-probe matters: a transient
314
+ `uiautomator` dump failure also exits 3, and shouldn't vaporize a 20-test run.
315
+ Continuing on a genuinely dead box just produces one identical red row per remaining
316
+ test — noise that reads exactly like a mass regression.
296
317
  - **The suite writes an overview** to `./.verikun/suites/<id>/`:
297
318
  - **`index.json`** — a stable, `schemaVersion`ed manifest: per-test pass/fail,
298
319
  steps, model repairs, cost, duration, and the run id, plus suite totals. This
@@ -300,17 +321,21 @@ vk suite tests/ --app com.example.app --server "$VERIKUN_SERVER" # remote devi
300
321
  it (see the [CI recipe](#ci-recipe)) instead of verikun growing upload plugins.
301
322
  On an abort it also carries `aborted: {reason, notRun}`; the not-run tests get
302
323
  **no rows and no place in `totals`**, so `passed + failed === tests` still holds
303
- and nothing downstream mistakes a skipped test for a regression.
324
+ and nothing downstream mistakes a skipped test for a regression. Retried flakes
325
+ add `flaky` / `attempts` on the test row and suite-level `warnings` (additive;
326
+ `schemaVersion` stays `1`).
304
327
  - **`index.html`** — a summary page linking every test's `report.html`, with a
305
- banner naming the not-run tests when the suite aborted.
306
- - **Exit code is the CI gate:** `0` all green · `1` a test failed · `2` bad/empty
307
- directory · `3` environment (the provider or the device toolchain is unavailable,
308
- or the box broke mid-run). The `1`-vs-`3` split is the point: `1` is a regression
309
- to investigate, `3` is a machine to fix. All `ai` flags (`--model`,
310
- `--max-cost-usd`, `--timeout`, …) apply to every test; both the provider
311
- (`ANTHROPIC_API_KEY` / `OPENAI_API_KEY`, or the `codex` / `cursor-agent` CLI for
312
- `--model codex-cli` / `cursor-cli`) **and** the device toolchain (`adb` / `idb` +
313
- a resolvable device) are checked up front, before anything is compiled.
328
+ banner naming the not-run tests when the suite aborted, and a warnings banner
329
+ when a flake recovered on retry (prior failed attempts stay linked).
330
+ - **Exit code is the CI gate:** `0` all green (including flakes that recovered with
331
+ `--retries`) · `1` a test failed · `2` bad/empty directory · `3` environment (the
332
+ provider or the device toolchain is unavailable, or the box broke mid-run). The
333
+ `1`-vs-`3` split is the point: `1` is a regression to investigate, `3` is a
334
+ machine to fix. All `ai` flags (`--model`, `--max-cost-usd`, `--timeout`, …)
335
+ apply to every test; both the provider (`ANTHROPIC_API_KEY` / `OPENAI_API_KEY`,
336
+ or the `codex` / `cursor-agent` CLI for `--model codex-cli` / `cursor-cli`)
337
+ **and** the device toolchain (`adb` / `idb` + a resolvable device) are checked up
338
+ front, before anything is compiled.
314
339
 
315
340
  ## Remote devices — `vk server`
316
341
 
@@ -112,7 +112,7 @@ function createRemoteBackend(opts, health) {
112
112
  const execRaw = async (req, record) => {
113
113
  const res = await t.postJson('/v1/exec', req, EXEC_TIMEOUT_MS);
114
114
  if (record && res.step)
115
- opts.onStep?.(res.step, decodeArtifacts(res.artifacts));
115
+ opts.onStep?.(res.step, decodeArtifacts(res.artifacts), res.logStart);
116
116
  return { code: res.code, error: res.error ? (0, rpc_1.rebuildError)(res.error) : undefined };
117
117
  };
118
118
  return {
@@ -121,6 +121,10 @@ function createRemoteBackend(opts, health) {
121
121
  const res = await t.postJson('/v1/elements', {}, ELEMENTS_TIMEOUT_MS);
122
122
  return res.elements;
123
123
  },
124
+ async getLogs(logOpts = {}) {
125
+ const res = await t.postJson('/v1/logs', logOpts, ELEMENTS_TIMEOUT_MS);
126
+ return res.logs ?? '';
127
+ },
124
128
  async install(appPath) {
125
129
  // v1 remote installs are single-file uploads; the extension is the only thing
126
130
  // the client tells the server about the artifact (never a path).
package/dist/args.js CHANGED
@@ -42,6 +42,7 @@ const BOOLEAN = new Set([
42
42
  'recompile',
43
43
  'no-cache',
44
44
  'no-restart',
45
+ 'no-logs',
45
46
  'allow-install',
46
47
  'allow-unsafe-anonymous',
47
48
  // Selector state modifiers (STATE_ATTRS in ui/selector.ts) and their negations.
package/dist/cli.js CHANGED
@@ -975,16 +975,47 @@ function cmdRun(positionals, flags, platform, device) {
975
975
  case 'archive':
976
976
  case 'finish':
977
977
  case 'save': {
978
- const { dir, xmlPath, htmlPath, state } = run_1.Recorder.archive(positionals[1]);
978
+ const noLogs = (0, args_1.flagBool)(flags, 'no-logs');
979
+ // Best-effort: archive-time log capture needs a device. Prefer the run's
980
+ // bound serial/platform so a multi-device host hits the right one. A
981
+ // missing/broken toolchain must not prevent sealing the report.
982
+ let fetchLogs;
983
+ const active = run_1.Recorder.status();
984
+ const hasFailures = !!active?.steps.some((s) => s.status !== 'passed');
985
+ if ((0, run_1.wantsArchiveLogs)(hasFailures, noLogs)) {
986
+ try {
987
+ const plat = active?.platform === 'ios' || active?.platform === 'android'
988
+ ? active.platform
989
+ : platform;
990
+ const driver = (0, drivers_1.getDriver)(plat, active?.device || device);
991
+ fetchLogs = (opts) => driver.getLogs(opts);
992
+ }
993
+ catch (e) {
994
+ (0, output_1.err)(`[verikun] archive log capture unavailable (${e.message})`);
995
+ }
996
+ }
997
+ const { dir, xmlPath, htmlPath, state } = run_1.Recorder.archive(positionals[1], { noLogs, fetchLogs });
979
998
  const { passed, failed } = tally(state.steps);
980
999
  if (asJson) {
981
- (0, output_1.json)({ archived: dir, report: htmlPath, junit: xmlPath, steps: state.steps.length, passed, failed });
1000
+ (0, output_1.json)({
1001
+ archived: dir,
1002
+ report: htmlPath,
1003
+ junit: xmlPath,
1004
+ steps: state.steps.length,
1005
+ passed,
1006
+ failed,
1007
+ ...(state.logFile ? { logFile: state.logFile } : {}),
1008
+ });
982
1009
  }
983
1010
  else {
984
1011
  (0, output_1.out)(dir); // primary result: the archived run directory
985
1012
  (0, output_1.err)(`archived '${state.name}': ${state.steps.length} step(s), ${passed} passed, ${failed} failed/error`);
986
1013
  (0, output_1.err)(` JUnit: ${xmlPath}`);
987
1014
  (0, output_1.err)(` HTML: ${htmlPath}`);
1015
+ if (state.logFile)
1016
+ (0, output_1.err)(` Logs: ${(0, node_path_1.join)(dir, state.logFile)}`);
1017
+ if (state.appLogFile)
1018
+ (0, output_1.err)(` App: ${(0, node_path_1.join)(dir, state.appLogFile)}`);
988
1019
  }
989
1020
  // Exit non-zero when the run contained failures, so CI can gate on it.
990
1021
  return failed > 0 ? 1 : 0;
@@ -1280,6 +1311,7 @@ async function resolveBackend(platform, device, flags) {
1280
1311
  backend: {
1281
1312
  exec: (command, positionals, f) => executeOutcome(command, positionals, f, driver),
1282
1313
  getElements: () => driver.getElements(),
1314
+ getLogs: (opts) => driver.getLogs(opts),
1283
1315
  install: (appPath) => driver.install(appPath),
1284
1316
  reset: (appId) => {
1285
1317
  assertSafeAppId(appId);
@@ -1300,8 +1332,9 @@ async function resolveBackend(platform, device, flags) {
1300
1332
  url: server,
1301
1333
  authKey: (0, args_1.flagStr)(flags, 'auth-key') || process.env.VERIKUN_SERVER_AUTH_KEY || undefined,
1302
1334
  // Each remote step is spliced into the local active run so the archived report
1303
- // is identical to a local run's.
1304
- onStep: (step, artifacts) => run_1.Recorder.appendForeignStep(step, artifacts, runCtx),
1335
+ // is identical to a local run's. logStart travels from the server's device clock
1336
+ // so archive-time / vk log scoping works without a local driver.
1337
+ onStep: (step, artifacts, logStart) => run_1.Recorder.appendForeignStep(step, artifacts, { ...runCtx, logStart }),
1305
1338
  };
1306
1339
  const health = await (0, remote_1.pingServer)(opts); // fails fast (exit 3) on a bad URL or key
1307
1340
  runCtx = { platform: health.platform, device: health.serial };
@@ -1325,6 +1358,45 @@ async function resolveBackend(platform, device, flags) {
1325
1358
  remote: { url: server, version: health.version },
1326
1359
  };
1327
1360
  }
1361
+ /**
1362
+ * Best-effort archive-time device-log capture via an ExecBackend (local driver or
1363
+ * remote `/v1/logs`). Writes `artifacts/logcat.txt` onto the active run so a later
1364
+ * `Recorder.archive()` finds `logFile` already set. Never throws — a gone device
1365
+ * must not prevent sealing the report.
1366
+ */
1367
+ async function prefetchArchiveLogs(backend, noLogs = false) {
1368
+ const state = run_1.Recorder.status();
1369
+ if (!state)
1370
+ return;
1371
+ const hasFailures = state.steps.some((s) => s.status !== 'passed');
1372
+ if (!(0, run_1.wantsArchiveLogs)(hasFailures, noLogs))
1373
+ return;
1374
+ if (!backend.getLogs)
1375
+ return;
1376
+ // Skip only when both artifacts are already attached (e.g. a prior prefetch).
1377
+ if (state.logFile && state.appLogFile)
1378
+ return;
1379
+ const window = (0, run_1.archiveLogWindow)(state);
1380
+ try {
1381
+ const full = state.logFile ? undefined : await backend.getLogs(window);
1382
+ const appId = (0, run_1.inferRunAppId)(state);
1383
+ let app;
1384
+ if (appId && !state.appLogFile) {
1385
+ try {
1386
+ app = await backend.getLogs({ ...window, appId, scopedOnly: true });
1387
+ }
1388
+ catch (e) {
1389
+ (0, output_1.err)(`[verikun] could not capture archive app logs (${e.message})`);
1390
+ }
1391
+ }
1392
+ if (full !== undefined || (app !== undefined && app !== '')) {
1393
+ run_1.Recorder.attachArchiveLogs(full, app);
1394
+ }
1395
+ }
1396
+ catch (e) {
1397
+ (0, output_1.err)(`[verikun] could not capture archive device logs (${e.message})`);
1398
+ }
1399
+ }
1328
1400
  /**
1329
1401
  * Run one natural-language test through a backend and return DATA — no stdout
1330
1402
  * writes (stdout stays the caller's one result; progress streams to stderr).
@@ -1365,10 +1437,14 @@ async function runAiTest(file, opts, backend, platform, device) {
1365
1437
  if (existing && existing.steps.length > 0) {
1366
1438
  // Seal the pre-existing run into the archive instead of letting start(force=true)
1367
1439
  // discard it — a manual in-progress run should never be silently lost.
1440
+ await prefetchArchiveLogs(backend);
1368
1441
  const sealed = run_1.Recorder.archive();
1369
1442
  (0, output_1.err)(`[ai] archived the active run ('${existing.name}', ${existing.steps.length} step(s)) → ${sealed.dir}`);
1370
1443
  }
1371
1444
  const started = run_1.Recorder.start(`ai: ${(0, node_path_1.basename)(file)}`, platform, device, true);
1445
+ // Prefer --package when the prose never launches by id (rare but possible).
1446
+ if (opts.pkg)
1447
+ run_1.Recorder.annotateRun({ appId: opts.pkg });
1372
1448
  // Suppress per-step `out()` so stdout stays the one final result; progress -> stderr.
1373
1449
  const prevQuiet = (0, output_1.setOutputQuiet)(true);
1374
1450
  let result;
@@ -1395,6 +1471,7 @@ async function runAiTest(file, opts, backend, platform, device) {
1395
1471
  // to roll over. Then let the error map to an exit code as usual.
1396
1472
  run_1.Recorder.annotateRun({ ai: { ok: false, cost: cost.summaryLine(), modelRepairs: 0, improvements: [] } });
1397
1473
  try {
1474
+ await prefetchArchiveLogs(backend);
1398
1475
  run_1.Recorder.archive();
1399
1476
  }
1400
1477
  catch (sealErr) {
@@ -1422,6 +1499,7 @@ async function runAiTest(file, opts, backend, platform, device) {
1422
1499
  run_1.Recorder.annotateRun({
1423
1500
  ai: { ok: result.ok, cost: costLine, modelRepairs: result.modelRepairs, improvements: result.improvements },
1424
1501
  });
1502
+ await prefetchArchiveLogs(backend);
1425
1503
  const { dir, xmlPath, htmlPath, state } = run_1.Recorder.archive();
1426
1504
  const status = result.ok
1427
1505
  ? 'PASS'
@@ -1537,7 +1615,7 @@ async function cmdInstall(positionals, flags) {
1537
1615
  async function cmdSuiteEntry(positionals, flags) {
1538
1616
  const dirArg = positionals[0];
1539
1617
  if (!dirArg)
1540
- throw new errors_1.CliError('Usage: verikun suite <dir> [--app <id>] [--server url] [--name n] [--json]', 2);
1618
+ throw new errors_1.CliError('Usage: verikun suite <dir> [--app <id>] [--server url] [--name n] [--retries n] [--json]', 2);
1541
1619
  const opts = parseAiOptions(flags);
1542
1620
  // Pre-flight the provider BEFORE touching any device/server: every test needs it
1543
1621
  // to compile (on a cache miss) or to repair at runtime.
@@ -1708,11 +1786,23 @@ async function executeForServer(command, positionals, flags, driver, platform) {
1708
1786
  catch {
1709
1787
  /* surfaced by the command handler below */
1710
1788
  }
1789
+ // Sample the device clock up front so the caller's run can set logStart (the
1790
+ // ephemeral recorder never persists RunState). Best-effort — empty/unavailable
1791
+ // just means archive / vk log fall back to last-N.
1792
+ let logStart;
1793
+ try {
1794
+ const t = driver.deviceTime();
1795
+ if (t)
1796
+ logStart = t;
1797
+ }
1798
+ catch {
1799
+ /* device clock unavailable */
1800
+ }
1711
1801
  const recorder = run_1.Recorder.beginEphemeralStep(command, positionals, flags, platform, serial);
1712
1802
  const ctx = { driver, platform, device: serial, positionals, flags, record: recorder };
1713
1803
  const outcome = await runRecorded(command, ctx, recorder, driver);
1714
1804
  const { step, artifacts } = recorder.takeEphemeral();
1715
- return { ...outcome, step, artifacts };
1805
+ return { ...outcome, step, artifacts, ...(logStart ? { logStart } : {}) };
1716
1806
  }
1717
1807
  /**
1718
1808
  * Run one already-parsed command for the CLI / `batch`: dispatch meta-commands,
@@ -1849,14 +1939,19 @@ AI (run a natural-language test — compile once, replay model-free, self-heal)
1849
1939
  codex-cli | cursor-cli.
1850
1940
 
1851
1941
  SUITE (run a directory of natural-language tests as one gated suite)
1852
- suite <dir> [--app <id>] [--name n] [--json] (+ all \`ai\` flags, incl. --server)
1942
+ suite <dir> [--app <id>] [--name n] [--retries n] [--json]
1943
+ (+ all \`ai\` flags, incl. --server)
1853
1944
  Run every *.md in <dir> (lexicographic order —
1854
1945
  prefix 01-, 02- to sequence; README.md skipped)
1855
1946
  through \`vk ai\`. With --app, app data is reset
1856
- between tests (iOS: force-stop). Writes a suite
1857
- overview to ./.verikun/suites/<id>/{index.json,
1858
- index.html} linking each test's report. Exits 1
1859
- if any test failed the CI gate.
1947
+ between tests (iOS: force-stop). --retries N
1948
+ re-runs a failed test up to N times; a later
1949
+ pass recovers the suite (exit 0) and surfaces a
1950
+ warning, keeping failed-attempt evidence in the
1951
+ report. Writes a suite overview to
1952
+ ./.verikun/suites/<id>/{index.json, index.html}
1953
+ linking each test's report. Exits 1 if any test
1954
+ failed — the CI gate.
1860
1955
 
1861
1956
  SERVER (expose a locally-connected device to remote verikun clients)
1862
1957
  server [--bind addr] [--port n] [--auth-key k] [--allow-install]
@@ -1879,7 +1974,10 @@ ENVIRONMENT
1879
1974
  TEST RUNS (actions are recorded; a run auto-starts on first action)
1880
1975
  run start [name] [--force] Begin a named run (else one starts implicitly)
1881
1976
  run status Show the active run, its device/session, and steps
1882
- run archive [name] Write JUnit + HTML report, move to ./.verikun/runs/<id>/
1977
+ run archive [name] [--no-logs] Write JUnit + HTML report, move to ./.verikun/runs/<id>/
1978
+ Captures artifacts/logcat.txt by default (session-scoped);
1979
+ --no-logs / VERIKUN_NO_LOGS skips on green runs (failures
1980
+ still capture). Capture is best-effort and never blocks archive.
1883
1981
  run clear Discard the active run with no report
1884
1982
  An implicit run auto-closes (archives) and rolls over on a device change, a
1885
1983
  VERIKUN_SESSION change, or VERIKUN_RUN_IDLE_MIN minutes idle (default 30; 0 off).
@@ -356,15 +356,36 @@ class AdbDriver {
356
356
  args.push('-t', String(n));
357
357
  }
358
358
  if (opts.appId) {
359
- // Scope to the app's process when it's alive. If pidof is empty the app
360
- // isn't running (likely crashed) fall through to a system-wide dump so
361
- // the FATAL EXCEPTION, still in the crash buffer, is not missed.
362
- const pid = this.shell(['pidof', opts.appId]).trim().split(/\s+/)[0];
363
- if (pid)
364
- args.push(`--pid=${pid}`);
359
+ // Prefer --uid: it survives process death/restart (crash traces stay under the
360
+ // package's uid). Fall back to --pid for a live process when uid isn't known.
361
+ // When neither works: vk log falls through to system-wide so a FATAL EXCEPTION
362
+ // isn't missed; archive accordion passes scopedOnly to keep the dump empty.
363
+ const uid = this.packageUid(opts.appId);
364
+ if (uid) {
365
+ args.push(`--uid=${uid}`);
366
+ }
367
+ else {
368
+ const pid = this.shell(['pidof', opts.appId]).trim().split(/\s+/)[0];
369
+ if (pid)
370
+ args.push(`--pid=${pid}`);
371
+ else if (opts.scopedOnly)
372
+ return '';
373
+ }
365
374
  }
366
375
  return this.shell(args, 15000);
367
376
  }
377
+ /** Android userId for an installed package, or '' if unknown. Used to scope
378
+ * logcat across process restarts (unlike pidof, which only sees a live process). */
379
+ packageUid(appId) {
380
+ try {
381
+ const out = this.shell(['dumpsys', 'package', appId], 10000);
382
+ const m = /\buserId=(\d+)\b/.exec(out);
383
+ return m?.[1] ?? '';
384
+ }
385
+ catch {
386
+ return '';
387
+ }
388
+ }
368
389
  deviceTime() {
369
390
  // logcat's default timestamp is MM-DD HH:MM:SS.mmm in the device's LOCAL time.
370
391
  // Sample it from the device clock with a space-free format (so no device-shell
@@ -433,6 +433,11 @@ class IdbDriver {
433
433
  const proc = opts.appId.split('.').pop() || opts.appId;
434
434
  args.push('--predicate', `process CONTAINS "${proc}"`);
435
435
  }
436
+ else if (opts.scopedOnly) {
437
+ // scopedOnly without an appId can't mean anything on iOS — empty rather than
438
+ // a full dump (mirrors Android's "couldn't scope" behaviour).
439
+ return '';
440
+ }
436
441
  const out = (0, exec_1.runText)(XCRUN, args, { timeout: 20000 }).stdout;
437
442
  if (opts.since)
438
443
  return out; // the whole session window
package/dist/report.js CHANGED
@@ -95,14 +95,20 @@ function toJUnitXml(run) {
95
95
  .join('\n');
96
96
  const suiteAttrs = `name="${xmlAttr(run.name)}" tests="${c.tests}" failures="${c.failures}" ` +
97
97
  `errors="${c.errors}" time="${suiteTime}" timestamp="${xmlAttr(run.startedAt)}"`;
98
+ const suiteExtras = [];
99
+ if (run.logFile)
100
+ suiteExtras.push(`device log: ${run.logFile}`);
101
+ if (run.ai) {
102
+ suiteExtras.push('vk ai: ' +
103
+ run.ai.cost +
104
+ (run.ai.improvements.length ? '\nSuggested improvements:\n' + run.ai.improvements.join('\n') : ''));
105
+ }
98
106
  return (`<?xml version="1.0" encoding="UTF-8"?>\n` +
99
107
  `<testsuites name="verikun" tests="${c.tests}" failures="${c.failures}" errors="${c.errors}" time="${suiteTime}">\n` +
100
108
  `<testsuite ${suiteAttrs}>\n` +
101
109
  `${cases}\n` +
102
- (run.ai
103
- ? ` <system-out>${xmlText('vk ai: ' +
104
- run.ai.cost +
105
- (run.ai.improvements.length ? '\nSuggested improvements:\n' + run.ai.improvements.join('\n') : ''))}</system-out>\n`
110
+ (suiteExtras.length
111
+ ? ` <system-out>${xmlText(suiteExtras.join('\n'))}</system-out>\n`
106
112
  : '') +
107
113
  `</testsuite>\n</testsuites>\n`);
108
114
  }
@@ -118,6 +124,7 @@ const STYLE = `
118
124
  .summary { display:flex; gap:8px; flex-wrap:wrap; align-items:center; margin-bottom: 20px; }
119
125
  .chip { font-weight:600; font-size:13px; padding:4px 10px; border-radius:999px; color:#fff; }
120
126
  .chip.pass{background:var(--pass)} .chip.fail{background:var(--fail)} .chip.err{background:var(--err)}
127
+ .chip.warn{background:var(--err)}
121
128
  .chip.muted{ background:#eaeef2; color:var(--muted); }
122
129
  ol.steps { list-style:none; margin:0; padding:0; }
123
130
  li.step { background:#fff; border:1px solid var(--line); border-left-width:4px; border-radius:8px; margin-bottom:10px; padding:12px 14px; }
@@ -135,6 +142,10 @@ const STYLE = `
135
142
  .msg.fail { color:var(--fail); }
136
143
  img.shot { display:block; margin-top:10px; max-width:300px; max-height:520px; border:1px solid var(--line); border-radius:6px; }
137
144
  details { margin-top:8px; }
145
+ details.run-log { margin-top:20px; background:#fff; border:1px solid var(--line); border-radius:8px; padding:10px 14px; }
146
+ details.run-log > summary { font-weight:600; color:#1f2328; }
147
+ details.run-log > summary a { font-weight:400; }
148
+ details.run-log pre { max-height:480px; }
138
149
  summary { cursor:pointer; color:var(--muted); font-size:13px; }
139
150
  pre { background:#0d1117; color:#e6edf3; padding:12px; border-radius:6px; overflow:auto; font-size:12px; line-height:1.45; max-height:360px; }
140
151
  .aibox { background:#fff; border:1px solid var(--line); border-radius:8px; padding:12px 14px; margin-bottom:20px; font-size:13px; }
@@ -206,19 +217,40 @@ const SUITE_STYLE = `
206
217
  table.tests td.num { text-align:right; font-variant-numeric:tabular-nums; white-space:nowrap; }
207
218
  table.tests a { color:inherit; }
208
219
  .fail-reason { color:var(--fail); font-size:12px; margin-top:2px; }
220
+ .flake-note { color:var(--err); font-size:12px; margin-top:2px; }
221
+ .attempts { margin-top:4px; font-size:12px; color:var(--muted); }
222
+ .attempts a { color:var(--fail); }
209
223
  .aborted { background:#fff4e5; border:1px solid #f0b429; border-radius:8px; padding:12px 14px; margin:0 0 14px; font-size:13px; }
210
224
  .aborted strong { color:#8a5300; }
211
225
  .aborted ul { margin:6px 0 0; padding-left:20px; color:var(--muted); }
226
+ .warnings { background:#fff8c5; border:1px solid #d4a72c; border-radius:8px; padding:12px 14px; margin:0 0 14px; font-size:13px; }
227
+ .warnings strong { color:#7d4e00; }
228
+ .warnings ul { margin:6px 0 0; padding-left:20px; color:var(--muted); }
212
229
  `;
230
+ function suiteAttemptLinks(attempts, linkBase) {
231
+ const links = attempts
232
+ .map((a, i) => {
233
+ const label = `attempt ${i + 1}`;
234
+ if (!a.id)
235
+ return htmlEsc(label);
236
+ return `<a href="${htmlEsc(`${linkBase}runs/${encodeURIComponent(a.id)}/report.html`)}">${htmlEsc(label)}</a>`;
237
+ })
238
+ .join(', ');
239
+ return `<div class="attempts">prior failed: ${links}</div>`;
240
+ }
213
241
  function suiteTestRow(t, linkBase) {
214
242
  // A test that errored before its run started (id '') has no report to link.
215
243
  const label = t.id
216
244
  ? `<a href="${htmlEsc(`${linkBase}runs/${encodeURIComponent(t.id)}/report.html`)}">${htmlEsc(t.name)}</a>`
217
245
  : htmlEsc(t.name);
218
246
  const failure = t.failure ? `<div class="fail-reason">${htmlEsc(t.failure)}</div>` : '';
247
+ const flake = t.flaky ? `<div class="flake-note">passed on retry (flake)</div>` : '';
248
+ const prior = t.attempts?.length ? suiteAttemptLinks(t.attempts, linkBase) : '';
249
+ const status = t.flaky ? 'FLAKY' : t.ok ? 'PASS' : 'FAIL';
250
+ const statusClass = t.ok ? 'passed' : 'failed';
219
251
  return ` <tr>
220
- <td><span class="st ${t.ok ? 'passed' : 'failed'}">${t.ok ? 'PASS' : 'FAIL'}</span></td>
221
- <td>${label}${failure}</td>
252
+ <td><span class="st ${statusClass}">${status}</span></td>
253
+ <td>${label}${flake}${failure}${prior}</td>
222
254
  <td class="num">${t.passedSteps}/${t.steps}${t.failedSteps ? ` (${t.failedSteps} failed)` : ''}</td>
223
255
  <td class="num">${t.modelRepairs || ''}</td>
224
256
  <td class="num">$${t.costUsd.toFixed(4)}</td>
@@ -233,9 +265,11 @@ function suiteTestRow(t, linkBase) {
233
265
  function toSuiteHtml(suite, opts = {}) {
234
266
  const linkBase = opts.linkBase ?? '../../';
235
267
  const t = suite.totals;
268
+ const flaky = suite.tests.filter((x) => x.flaky).length;
236
269
  const chips = [
237
270
  `<span class="chip pass">${t.passed} passed</span>`,
238
271
  t.failed ? `<span class="chip fail">${t.failed} failed</span>` : '',
272
+ flaky ? `<span class="chip warn">${flaky} flaky</span>` : '',
239
273
  suite.aborted ? `<span class="chip fail">ABORTED</span>` : '',
240
274
  `<span class="chip muted">${t.tests} tests &middot; ${t.steps} steps &middot; ${fmtDuration(t.durationMs)} &middot; $${t.costUsd.toFixed(4)}</span>`,
241
275
  ]
@@ -250,6 +284,13 @@ function toSuiteHtml(suite, opts = {}) {
250
284
  ${suite.aborted.notRun.length
251
285
  ? ` <ul>${suite.aborted.notRun.map((f) => `<li>${htmlEsc(f)} — not run</li>`).join('')}</ul>\n`
252
286
  : ''} </div>
287
+ `
288
+ : '';
289
+ const warningsBanner = suite.warnings?.length
290
+ ? ` <div class="warnings">
291
+ <strong>Warnings</strong>
292
+ <ul>${suite.warnings.map((w) => `<li>${htmlEsc(w)}</li>`).join('')}</ul>
293
+ </div>
253
294
  `
254
295
  : '';
255
296
  const metaBits = [
@@ -274,7 +315,7 @@ ${suite.aborted.notRun.length
274
315
  <div class="summary">
275
316
  ${chips}
276
317
  </div>
277
- ${abortedBanner} <table class="tests">
318
+ ${abortedBanner}${warningsBanner} <table class="tests">
278
319
  <thead><tr><th></th><th>Test</th><th>Steps</th><th>Repairs</th><th>Cost</th><th>Duration</th></tr></thead>
279
320
  <tbody>
280
321
  ${suite.tests.map((x) => suiteTestRow(x, linkBase)).join('\n')}
@@ -285,7 +326,12 @@ ${suite.tests.map((x) => suiteTestRow(x, linkBase)).join('\n')}
285
326
  </html>
286
327
  `;
287
328
  }
288
- function toHtml(run) {
329
+ /**
330
+ * @param opts.appLog app-scoped logcat body for the bottom accordion (kept out
331
+ * of RunState / run.json — pass the file contents when writing
332
+ * report.html). The full device dump stays a meta-row file link.
333
+ */
334
+ function toHtml(run, opts = {}) {
289
335
  const c = counts(run);
290
336
  const chips = [
291
337
  `<span class="chip pass">${c.passed} passed</span>`,
@@ -301,7 +347,20 @@ function toHtml(run) {
301
347
  `started ${htmlEsc(run.startedAt)}`,
302
348
  run.finishedAt ? `finished ${htmlEsc(run.finishedAt)}` : '',
303
349
  run.implicit ? 'implicit run' : '',
350
+ run.logFile ? `<a href="${htmlEsc(run.logFile)}">device log</a>` : '',
304
351
  ].filter(Boolean);
352
+ // Bottom accordion: app-scoped dump only. The noisy full device log stays a
353
+ // download via the meta link — same shape on every framework (package/uid scope).
354
+ const appLabel = run.appId ? ` for ${htmlEsc(run.appId)}` : '';
355
+ const appFileLink = run.appLogFile
356
+ ? ` (<a href="${htmlEsc(run.appLogFile)}">${htmlEsc(run.appLogFile)}</a>)`
357
+ : '';
358
+ const appLogPanel = opts.appLog !== undefined && opts.appLog !== ''
359
+ ? `\n <details class="run-log">
360
+ <summary>App log${appLabel}${appFileLink}</summary>
361
+ <pre>${htmlEsc(opts.appLog)}</pre>
362
+ </details>`
363
+ : '';
305
364
  return `<!doctype html>
306
365
  <html lang="en">
307
366
  <head>
@@ -320,7 +379,7 @@ function toHtml(run) {
320
379
  ${run.ai ? aiPanelHtml(run.ai) : ''}
321
380
  <ol class="steps">
322
381
  ${run.steps.map(stepHtml).join('\n ')}
323
- </ol>
382
+ </ol>${appLogPanel}
324
383
  </div>
325
384
  </body>
326
385
  </html>
package/dist/run.js CHANGED
@@ -1,10 +1,14 @@
1
1
  "use strict";
2
2
  Object.defineProperty(exports, "__esModule", { value: true });
3
- exports.Recorder = void 0;
3
+ exports.Recorder = exports.RUN_APP_LOG_ARTIFACT = exports.RUN_LOG_ARTIFACT = void 0;
4
4
  exports.isRecordable = isRecordable;
5
+ exports.wantsArchiveLogs = wantsArchiveLogs;
6
+ exports.archiveLogWindow = archiveLogWindow;
7
+ exports.inferRunAppId = inferRunAppId;
5
8
  exports.runId = runId;
6
9
  exports.uniqueDir = uniqueDir;
7
10
  exports.rolloverReason = rolloverReason;
11
+ exports.rolloverLogsSameDevice = rolloverLogsSameDevice;
8
12
  exports.stepName = stepName;
9
13
  const node_fs_1 = require("node:fs");
10
14
  const node_path_1 = require("node:path");
@@ -32,6 +36,58 @@ function isRecordable(command) {
32
36
  }
33
37
  const HIERARCHY_CAP = 24000; // chars of failure hierarchy kept inline in run.json
34
38
  const LOG_CAP = 50000; // chars of device logs kept inline in run.json (tail-kept)
39
+ /** Max chars written to the archive-time `artifacts/logcat.txt` (tail-kept). Larger
40
+ * than the per-step LOG_CAP because the file is not inlined into run.json. */
41
+ const LOG_FILE_CAP = 512_000;
42
+ /** When no session `logStart` marker exists, archive captures this many trailing lines. */
43
+ const ARCHIVE_LOG_LINES = 5000;
44
+ exports.RUN_LOG_ARTIFACT = 'artifacts/logcat.txt';
45
+ exports.RUN_APP_LOG_ARTIFACT = 'artifacts/logcat-app.txt';
46
+ const APP_LIFECYCLE = new Set(['launch', 'open', 'stop', 'clear']);
47
+ /** Whether archive-time device-log capture should run. Default on; `--no-logs` /
48
+ * `VERIKUN_NO_LOGS` opts out of *green* runs only — a failed run always captures
49
+ * (best-effort), because that is when the log is load-bearing. Exported for tests. */
50
+ function wantsArchiveLogs(hasFailures, noLogsFlag = false) {
51
+ if (hasFailures)
52
+ return true;
53
+ if (noLogsFlag)
54
+ return false;
55
+ if (process.env.VERIKUN_NO_LOGS)
56
+ return false;
57
+ return true;
58
+ }
59
+ /** Window passed to `getLogs` at archive time: session-scoped when `logStart` is
60
+ * known, otherwise a bounded trailing dump. Exported for tests. */
61
+ function archiveLogWindow(state) {
62
+ if (state.logStart)
63
+ return { since: state.logStart };
64
+ return { lines: ARCHIVE_LOG_LINES };
65
+ }
66
+ /** Package / bundle id the run exercised, if we can tell. Prefer an explicit
67
+ * `state.appId` (set on launch/open/stop/clear); otherwise recover from those
68
+ * steps' names. Exported for tests. */
69
+ function inferRunAppId(state) {
70
+ if (state.appId && /^[A-Za-z0-9._-]+$/.test(state.appId))
71
+ return state.appId;
72
+ for (let i = state.steps.length - 1; i >= 0; i--) {
73
+ const s = state.steps[i];
74
+ if (!APP_LIFECYCLE.has(s.command))
75
+ continue;
76
+ // stepName is `launch com.foo` / `clear com.foo` (no flags in the name).
77
+ const m = /^(?:launch|open|stop|clear)\s+([A-Za-z0-9._-]+)\s*$/.exec(s.name);
78
+ if (m)
79
+ return m[1];
80
+ }
81
+ return undefined;
82
+ }
83
+ function truncateLogFile(text) {
84
+ if (text.length <= LOG_FILE_CAP)
85
+ return text;
86
+ return '…(truncated)\n' + text.slice(-LOG_FILE_CAP);
87
+ }
88
+ function runHasFailures(state) {
89
+ return state.steps.some((s) => s.status !== 'passed');
90
+ }
35
91
  // --- paths & persistence --------------------------------------------------
36
92
  const activeDir = () => (0, node_path_1.join)((0, output_1.artifactDir)(), 'run');
37
93
  const archiveBase = () => (0, node_path_1.join)((0, output_1.artifactDir)(), 'runs');
@@ -123,6 +179,14 @@ function rolloverReason(state, serial, session) {
123
179
  return `idle for ${fmtAge(ageMs(state))} (>${idle}m)`;
124
180
  return null;
125
181
  }
182
+ /** Whether the incoming step's driver is safe to use for archive-time logs of the
183
+ * run being sealed. On a device-change rollover the driver is already pointed at
184
+ * the *new* serial — pulling logcat from it would attribute the wrong device's
185
+ * output to the old run. Idle/session rollover on the same device is fine.
186
+ * Exported for tests. */
187
+ function rolloverLogsSameDevice(runDevice, newSerial) {
188
+ return !runDevice || !newSerial || runDevice === newSerial;
189
+ }
126
190
  function stepName(command, positionals, flags) {
127
191
  const p = positionals;
128
192
  const at = typeof flags['at'] === 'string' ? flags['at'] : undefined;
@@ -187,7 +251,13 @@ class Recorder {
187
251
  const reason = rolloverReason(state, serial, session);
188
252
  if (reason) {
189
253
  try {
190
- const dest = Recorder.seal(state, dir);
254
+ // Only reuse this step's driver when it still targets the run's device.
255
+ // A device-change rollover must not write the new device's logcat into
256
+ // the old run's archive (vk run archive binds a driver to state.device).
257
+ const fetchLogs = driver && rolloverLogsSameDevice(state.device, serial)
258
+ ? (opts) => driver.getLogs(opts)
259
+ : undefined;
260
+ const dest = Recorder.seal(state, dir, { fetchLogs });
191
261
  (0, output_1.err)(`[verikun] previous run '${state.name}' (${state.steps.length} step(s)) auto-closed → ${dest} (${reason}); starting a fresh run`);
192
262
  state = null;
193
263
  rolledOver = true;
@@ -220,6 +290,11 @@ class Recorder {
220
290
  if (!state.session && session)
221
291
  state.session = session;
222
292
  }
293
+ // Remember the app under test from lifecycle commands so archive can scope
294
+ // the accordion log without requiring a separate flag.
295
+ if (APP_LIFECYCLE.has(command) && positionals[0] && /^[A-Za-z0-9._-]+$/.test(positionals[0])) {
296
+ state.appId = positionals[0];
297
+ }
223
298
  // Anchor the log window at the session's first step (covers both implicit
224
299
  // creation and an explicit `vk run start`, which records no marker itself).
225
300
  // Best-effort — a missing marker just means `vk log` falls back to last-N.
@@ -386,18 +461,87 @@ class Recorder {
386
461
  this.state.updatedAt = nowIso();
387
462
  saveState(this.dir, this.state);
388
463
  }
389
- /** Finalize a run: write reports next to it, then move it into ./.verikun/runs/<id>/. */
390
- static seal(state, dir) {
464
+ /** Finalize a run: optionally capture device logs, write reports next to it, then
465
+ * move it into ./.verikun/runs/<id>/. Log capture is best-effort — a device that
466
+ * is gone (often why the run failed) must never prevent sealing the report. */
467
+ static seal(state, dir, opts = {}) {
468
+ Recorder.maybeCaptureArchiveLogs(state, dir, opts);
391
469
  state.finishedAt = nowIso();
392
470
  state.updatedAt = nowIso();
393
471
  saveState(dir, state);
394
472
  (0, node_fs_1.writeFileSync)((0, node_path_1.join)(dir, 'report.xml'), (0, report_1.toJUnitXml)(state));
395
- (0, node_fs_1.writeFileSync)((0, node_path_1.join)(dir, 'report.html'), (0, report_1.toHtml)(state));
473
+ // Accordion embeds the *app*-scoped dump (readable); the full device dump stays
474
+ // a file link in the meta row. Missing file is fine — capture is best-effort.
475
+ let appLog;
476
+ if (state.appLogFile) {
477
+ try {
478
+ appLog = (0, node_fs_1.readFileSync)((0, node_path_1.join)(dir, state.appLogFile), 'utf8');
479
+ }
480
+ catch {
481
+ /* report still writes */
482
+ }
483
+ }
484
+ (0, node_fs_1.writeFileSync)((0, node_path_1.join)(dir, 'report.html'), (0, report_1.toHtml)(state, { appLog }));
396
485
  (0, node_fs_1.mkdirSync)(archiveBase(), { recursive: true });
397
486
  const dest = uniqueDir((0, node_path_1.join)(archiveBase(), state.id));
398
487
  (0, node_fs_1.renameSync)(dir, dest);
399
488
  return dest;
400
489
  }
490
+ /** Write archive-time log artifacts on the ACTIVE run without sealing. Used by
491
+ * remote (`--server`) callers that fetch asynchronously before `archive()`.
492
+ * Pass `full` / `app` only for the pieces you have — omitted sides are left alone. */
493
+ static attachArchiveLogs(full, app) {
494
+ const dir = activeDir();
495
+ const state = loadState(dir);
496
+ if (!state)
497
+ return;
498
+ if (full !== undefined) {
499
+ Recorder.writeArchiveLog(state, dir, full, exports.RUN_LOG_ARTIFACT, 'logFile');
500
+ }
501
+ if (app !== undefined && app !== '') {
502
+ if (!state.appId)
503
+ state.appId = inferRunAppId(state);
504
+ Recorder.writeArchiveLog(state, dir, app, exports.RUN_APP_LOG_ARTIFACT, 'appLogFile');
505
+ }
506
+ saveState(dir, state);
507
+ }
508
+ static writeArchiveLog(state, dir, text, rel, field) {
509
+ const body = truncateLogFile(text);
510
+ (0, node_fs_1.mkdirSync)((0, node_path_1.join)(dir, 'artifacts'), { recursive: true });
511
+ (0, node_fs_1.writeFileSync)((0, node_path_1.join)(dir, rel), body);
512
+ state[field] = rel;
513
+ }
514
+ static maybeCaptureArchiveLogs(state, dir, opts) {
515
+ if (!wantsArchiveLogs(runHasFailures(state), opts.noLogs))
516
+ return;
517
+ if (!opts.fetchLogs)
518
+ return;
519
+ const window = archiveLogWindow(state);
520
+ if (!state.logFile) {
521
+ try {
522
+ const text = opts.fetchLogs(window);
523
+ if (text !== undefined && text !== null) {
524
+ Recorder.writeArchiveLog(state, dir, text, exports.RUN_LOG_ARTIFACT, 'logFile');
525
+ }
526
+ }
527
+ catch (e) {
528
+ (0, output_1.err)(`[verikun] could not capture archive device logs (${e.message})`);
529
+ }
530
+ }
531
+ const appId = inferRunAppId(state);
532
+ if (appId)
533
+ state.appId = appId;
534
+ if (appId && !state.appLogFile) {
535
+ try {
536
+ const text = opts.fetchLogs({ ...window, appId, scopedOnly: true });
537
+ if (text)
538
+ Recorder.writeArchiveLog(state, dir, text, exports.RUN_APP_LOG_ARTIFACT, 'appLogFile');
539
+ }
540
+ catch (e) {
541
+ (0, output_1.err)(`[verikun] could not capture archive app logs (${e.message})`);
542
+ }
543
+ }
544
+ }
401
545
  /** One-line context summary for `vk run status`. */
402
546
  static contextLine(state) {
403
547
  const bits = [];
@@ -428,6 +572,8 @@ class Recorder {
428
572
  * a local one. Re-indexes the step, writes its artifact buffers under the new
429
573
  * index, and rewrites the step's artifact references to match. Creates an
430
574
  * implicit run first if none is active (parity with beginStep's auto-start).
575
+ * `ctx.logStart` (device-clock marker from the server) is recorded on the first
576
+ * splice so archive-time / `vk log` scoping works the same as a local run.
431
577
  */
432
578
  static appendForeignStep(step, artifacts = {}, ctx = {}) {
433
579
  if (process.env.VERIKUN_NO_RUN)
@@ -448,6 +594,17 @@ class Recorder {
448
594
  };
449
595
  (0, output_1.err)('[verikun] recording test run (implicit) — archive: `vk run archive` · discard: `vk run clear`');
450
596
  }
597
+ // Anchor the log window from the server's device clock (beginEphemeralStep
598
+ // never persists state, so the marker has to travel on the wire).
599
+ if (!state.logStart && ctx.logStart)
600
+ state.logStart = ctx.logStart;
601
+ // Mirror local beginStep: remember the app from lifecycle steps so archive
602
+ // can scope the accordion without a local launch having set state.appId.
603
+ if (APP_LIFECYCLE.has(step.command)) {
604
+ const m = /^(?:launch|open|stop|clear)\s+([A-Za-z0-9._-]+)\s*$/.exec(step.name);
605
+ if (m)
606
+ state.appId = m[1];
607
+ }
451
608
  const index = state.steps.length;
452
609
  const spliced = { ...step, index };
453
610
  for (const [rel, buf] of Object.entries(artifacts)) {
@@ -521,8 +678,10 @@ class Recorder {
521
678
  (0, node_fs_1.rmSync)(dir, { recursive: true, force: true });
522
679
  return existing;
523
680
  }
524
- /** Write JUnit + HTML reports and move the run into ./.verikun/runs/<id>/. */
525
- static archive(name) {
681
+ /** Write JUnit + HTML reports and move the run into ./.verikun/runs/<id>/.
682
+ * By default captures a bounded device-log tail into `artifacts/logcat.txt`
683
+ * (see wantsArchiveLogs / ArchiveLogOpts). */
684
+ static archive(name, opts = {}) {
526
685
  const dir = activeDir();
527
686
  if (!(0, node_fs_1.existsSync)(statePath(dir))) {
528
687
  throw new errors_1.CliError('No active test run to archive. Run an action first, or `vk run start`.', 1);
@@ -532,7 +691,7 @@ class Recorder {
532
691
  throw new errors_1.CliError('Active run state is unreadable (.verikun/run/run.json is corrupt).', 3);
533
692
  if (name)
534
693
  state.name = name;
535
- const dest = Recorder.seal(state, dir);
694
+ const dest = Recorder.seal(state, dir, opts);
536
695
  return { dir: dest, xmlPath: (0, node_path_1.join)(dest, 'report.xml'), htmlPath: (0, node_path_1.join)(dest, 'report.html'), state };
537
696
  }
538
697
  }
package/dist/server.js CHANGED
@@ -11,6 +11,7 @@
11
11
  // through the SAME validateNode gate that guards `vk ai` model repairs, so only
12
12
  // KNOWN_COMMANDS action verbs execute — never `ui`/`log`, never a shell. The
13
13
  // server's driver is fixed at startup; client flags can never repoint the device.
14
+ // Archive-time log capture uses the dedicated /v1/logs endpoint instead.
14
15
  // - /v1/install is a privileged management verb: auth PLUS --allow-install, body
15
16
  // streamed to a server-generated temp path (the client supplies only an
16
17
  // allowlisted extension — never a path), optional sha256 verification.
@@ -154,13 +155,14 @@ function buildServer(config) {
154
155
  if (node.type !== 'command')
155
156
  throw new HttpError(400, 'rejected: not a command leaf');
156
157
  const t0 = Date.now();
157
- const { code, error, step, artifacts } = await (0, cli_1.executeForServer)(node.command, node.positionals, (0, ir_1.leafToFlags)(node), config.driver, config.platform);
158
+ const { code, error, step, artifacts, logStart } = await (0, cli_1.executeForServer)(node.command, node.positionals, (0, ir_1.leafToFlags)(node), config.driver, config.platform);
158
159
  (0, output_1.err)(`[server] exec ${node.command} ${node.positionals.join(' ')} → exit ${code} (${Date.now() - t0}ms)`);
159
160
  const payload = {
160
161
  code,
161
162
  ...(error ? { error: (0, rpc_1.describeError)(error) } : {}),
162
163
  ...(step ? { step } : {}),
163
164
  ...(artifacts && Object.keys(artifacts).length ? { artifacts: encodeArtifacts(artifacts) } : {}),
165
+ ...(logStart ? { logStart } : {}),
164
166
  };
165
167
  sendJson(res, 200, payload);
166
168
  }
@@ -169,6 +171,45 @@ function buildServer(config) {
169
171
  const elements = config.driver.getElements(); // CliError(3) on dump failure → 500 below
170
172
  sendJson(res, 200, { elements });
171
173
  }
174
+ async function handleLogs(req, res) {
175
+ const body = await readBody(req, EXEC_BODY_CAP);
176
+ let parsed = {};
177
+ if (body.length) {
178
+ try {
179
+ parsed = JSON.parse(body.toString('utf8'));
180
+ }
181
+ catch {
182
+ throw new HttpError(400, 'invalid JSON body');
183
+ }
184
+ }
185
+ // Mirror the driver's --since charset gate so a remote caller cannot inject
186
+ // into the device shell via a crafted marker (see AdbDriver.getLogs).
187
+ if (parsed.since !== undefined && parsed.since !== null) {
188
+ if (typeof parsed.since !== 'string' || !/^[0-9 :.\-]+$/.test(parsed.since)) {
189
+ throw new HttpError(400, `invalid since: only a logcat timestamp (digits, space, '-', ':', '.') is allowed`);
190
+ }
191
+ }
192
+ const lines = parsed.lines === undefined || parsed.lines === null
193
+ ? undefined
194
+ : typeof parsed.lines === 'number' && Number.isFinite(parsed.lines) && parsed.lines > 0
195
+ ? Math.floor(parsed.lines)
196
+ : undefined;
197
+ const appId = parsed.appId === undefined || parsed.appId === null
198
+ ? undefined
199
+ : typeof parsed.appId === 'string' && /^[A-Za-z0-9_.-]+$/.test(parsed.appId)
200
+ ? parsed.appId
201
+ : (() => {
202
+ throw new HttpError(400, `invalid appId '${String(parsed.appId)}'`);
203
+ })();
204
+ const logs = config.driver.getLogs({
205
+ ...(lines !== undefined ? { lines } : {}),
206
+ ...(parsed.since ? { since: parsed.since } : {}),
207
+ ...(appId ? { appId } : {}),
208
+ ...(parsed.scopedOnly ? { scopedOnly: true } : {}),
209
+ });
210
+ const payload = { logs };
211
+ sendJson(res, 200, payload);
212
+ }
172
213
  async function handleInstall(req, res) {
173
214
  const ext = String(req.headers['x-verikun-ext'] ?? '').toLowerCase();
174
215
  if (ext !== 'apk' && ext !== 'ipa') {
@@ -256,6 +297,8 @@ function buildServer(config) {
256
297
  return deviceEndpoint(() => handleExec(req, res));
257
298
  if (req.method === 'POST' && path === '/v1/elements')
258
299
  return deviceEndpoint(() => handleElements(req, res));
300
+ if (req.method === 'POST' && path === '/v1/logs')
301
+ return deviceEndpoint(() => handleLogs(req, res));
259
302
  if (req.method === 'POST' && path === '/v1/install') {
260
303
  if (!config.allowInstall) {
261
304
  throw new HttpError(403, 'install is disabled on this server (start it with --allow-install)', 3);
package/dist/suite.js CHANGED
@@ -13,6 +13,8 @@ Object.defineProperty(exports, "__esModule", { value: true });
13
13
  exports.sortTestFiles = sortTestFiles;
14
14
  exports.listTestFiles = listTestFiles;
15
15
  exports.toSuiteResult = toSuiteResult;
16
+ exports.toSuiteAttempt = toSuiteAttempt;
17
+ exports.mergeSuiteAttempts = mergeSuiteAttempts;
16
18
  exports.cmdSuite = cmdSuite;
17
19
  const node_fs_1 = require("node:fs");
18
20
  const node_path_1 = require("node:path");
@@ -100,6 +102,64 @@ function toSuiteResult(file, r, durationMs) {
100
102
  ...(r.ok ? {} : { failure: failure ?? 'failed' }),
101
103
  };
102
104
  }
105
+ /** Compact one attempt for the `attempts` evidence array (pure). */
106
+ function toSuiteAttempt(r) {
107
+ return {
108
+ id: r.id,
109
+ ok: r.ok,
110
+ durationMs: r.durationMs,
111
+ costUsd: r.costUsd,
112
+ ...(r.failure ? { failure: r.failure } : {}),
113
+ };
114
+ }
115
+ /**
116
+ * Merge a sequence of attempt rows into the final suite row: primary `id` is the last
117
+ * attempt (winning green, or last red), cost/duration/repairs sum across attempts, and
118
+ * prior attempts are retained as flake evidence.
119
+ */
120
+ function mergeSuiteAttempts(attempts) {
121
+ if (attempts.length === 0)
122
+ throw new Error('mergeSuiteAttempts: empty');
123
+ const last = attempts[attempts.length - 1];
124
+ if (attempts.length === 1)
125
+ return last;
126
+ const round = (n) => Number(n.toFixed(4));
127
+ const prior = attempts.slice(0, -1).map(toSuiteAttempt);
128
+ const flaky = last.ok && prior.some((a) => !a.ok);
129
+ return {
130
+ ...last,
131
+ durationMs: attempts.reduce((a, t) => a + t.durationMs, 0),
132
+ costUsd: round(attempts.reduce((a, t) => a + t.costUsd, 0)),
133
+ modelRepairs: attempts.reduce((a, t) => a + t.modelRepairs, 0),
134
+ attempts: prior,
135
+ ...(flaky ? { flaky: true } : {}),
136
+ };
137
+ }
138
+ // What --retries will and won't spend an attempt on. The bias is deliberate and
139
+ // asymmetric: a retry costs one test, while giving up costs the whole suite plus a
140
+ // human rerunning it. So the rule is *retry unless a rerun provably cannot change the
141
+ // outcome* — the two predicates below are the only "provably" cases, everything else
142
+ // (flaky selector, wedged app, a wobbling network to `vk server`) earns another go.
143
+ /** Budget aborts won't heal on retry: each attempt gets its own cost ceiling, so a
144
+ * rerun just re-aborts at the same place having spent the money twice. */
145
+ function isRetryable(r) {
146
+ return !r.ok && !r.abortedForBudget;
147
+ }
148
+ /** A thrown USAGE error (exit 2) is the one throw a rerun cannot change — an unreadable
149
+ * test file, a payload the server refuses, a flag it doesn't understand. Everything
150
+ * else, including every environment error, is retried while attempts remain. */
151
+ function isRetryableThrow(e) {
152
+ return !(e instanceof errors_1.CliError && e.exitCode === 2);
153
+ }
154
+ function parseRetries(flags) {
155
+ const n = (0, args_1.flagNum)(flags, 'retries');
156
+ if (n === undefined)
157
+ return 0;
158
+ if (!Number.isInteger(n) || n < 0) {
159
+ throw new errors_1.CliError(`--retries must be a non-negative integer, got '${n}'`, 2);
160
+ }
161
+ return n;
162
+ }
103
163
  async function cmdSuite(dirArg, flags, deps) {
104
164
  const dir = (0, node_path_1.resolve)(process.cwd(), dirArg);
105
165
  if (!(0, node_fs_1.existsSync)(dir) || !(0, node_fs_1.statSync)(dir).isDirectory()) {
@@ -109,69 +169,133 @@ async function cmdSuite(dirArg, flags, deps) {
109
169
  if (files.length === 0) {
110
170
  throw new errors_1.CliError(`suite: no test files (*.md) in '${dirArg}'`, 2);
111
171
  }
172
+ const retries = parseRetries(flags);
112
173
  const suiteId = (0, run_1.runId)();
113
174
  const name = (0, args_1.flagStr)(flags, 'name') || (0, node_path_1.basename)(dir);
114
175
  const startedAt = new Date().toISOString();
115
- (0, output_1.err)(`[suite] '${name}': ${files.length} test(s) from ${dirArg} (${deps.platform}${deps.device ? ` · ${deps.device}` : ''})`);
176
+ (0, output_1.err)(`[suite] '${name}': ${files.length} test(s) from ${dirArg} (${deps.platform}${deps.device ? ` · ${deps.device}` : ''})${retries > 0 ? ` · up to ${retries} retry(ies) on failure` : ''}`);
116
177
  const results = [];
178
+ const warnings = [];
117
179
  let aborted;
180
+ async function resetApp(label) {
181
+ // Returns the abort reason when the suite should stop (confirmed env break during reset).
182
+ if (!deps.reset)
183
+ return undefined;
184
+ try {
185
+ await deps.reset();
186
+ (0, output_1.err)(`[suite] app state reset${label}`);
187
+ return undefined;
188
+ }
189
+ catch (e) {
190
+ // A reset that failed because the BOX is broken means nothing after it is
191
+ // trustworthy — but only if a re-probe agrees. Otherwise surface and continue:
192
+ // a flaky reset should not zero out the whole suite, and the test itself will
193
+ // fail loudly if the stale state actually matters.
194
+ const broken = (0, errors_1.isEnvError)(e) ? await stillBroken(deps) : undefined;
195
+ if (broken)
196
+ return broken;
197
+ (0, output_1.err)(`[suite] reset failed (${e.message}) — continuing`);
198
+ return undefined;
199
+ }
200
+ }
201
+ /** A confirmed env break with attempts left: say so, pause, and let the loop retry.
202
+ * The pause matters — the failures this rides out (a server restart, a wifi drop, a
203
+ * USB re-enumeration) clear in seconds, and retrying into the same dead socket
204
+ * immediately would burn every attempt inside the outage. */
205
+ async function noteEnvRetry(file, attempt, reason) {
206
+ const warn = `${file}: environment error on attempt ${attempt + 1} (${reason}) — retried`;
207
+ warnings.push(warn);
208
+ (0, output_1.err)(`[suite] WARN ${warn}`);
209
+ await sleep((deps.probeRetryMs ?? PROBE_RETRY_MS) * (attempt + 1));
210
+ }
118
211
  for (let i = 0; i < files.length && !aborted; i++) {
119
212
  const file = files[i];
120
213
  (0, output_1.err)(`[suite] ── (${i + 1}/${files.length}) ${file} ──`);
121
- if (deps.reset) {
214
+ const attemptRows = [];
215
+ for (let attempt = 0; attempt <= retries; attempt++) {
216
+ // The last attempt is where a retryable failure becomes the verdict: a confirmed
217
+ // env break aborts the suite, anything else stands as this test's failed row.
218
+ const lastAttempt = attempt === retries;
219
+ if (attempt > 0)
220
+ (0, output_1.err)(`[suite] retry ${attempt}/${retries} for ${file}`);
221
+ // Re-isolate before EVERY attempt — between tests and between retries alike.
222
+ const resetBreak = await resetApp(attempt > 0 ? ' (retry)' : '');
223
+ if (resetBreak) {
224
+ if (!lastAttempt) {
225
+ await noteEnvRetry(file, attempt, `reset failed: ${resetBreak}`);
226
+ continue;
227
+ }
228
+ // With no attempt row this test never ran, so notRun starts at the CURRENT file.
229
+ aborted = {
230
+ reason: `reset failed: ${resetBreak}`,
231
+ notRun: files.slice(attemptRows.length ? i + 1 : i),
232
+ };
233
+ break;
234
+ }
235
+ const t0 = Date.now();
122
236
  try {
123
- await deps.reset();
124
- (0, output_1.err)('[suite] app state reset');
237
+ const r = await deps.runTest((0, node_path_1.join)(dir, file));
238
+ attemptRows.push(toSuiteResult(file, r, Date.now() - t0));
239
+ if (r.abortedForEnv) {
240
+ const broken = await stillBroken(deps);
241
+ if (broken) {
242
+ if (!lastAttempt) {
243
+ // Even a CONFIRMED break is worth an attempt: the probe window is a couple
244
+ // of seconds, which a server restart outlives — and aborting costs the run.
245
+ await noteEnvRetry(file, attempt, broken);
246
+ continue;
247
+ }
248
+ aborted = { reason: broken, notRun: files.slice(i + 1) };
249
+ break;
250
+ }
251
+ // Transient env blip: retryable like any other failure.
252
+ }
253
+ if (r.ok || !isRetryable(r) || lastAttempt)
254
+ break;
125
255
  }
126
256
  catch (e) {
127
- // A reset that failed because the BOX is broken means nothing after it is
128
- // trustworthybut only if a re-probe agrees. Otherwise surface and continue:
129
- // a flaky reset should not zero out the whole suite, and the test itself will
130
- // fail loudly if the stale state actually matters.
257
+ // A test that THREW (device gone, server unreachable, bad file) still becomes a
258
+ // failed row one broken test must not vaporize the suite report for the tests
259
+ // that already ran. Out of attempts, a confirmed env break stops the suite.
260
+ const msg = e instanceof Error ? e.message : String(e);
261
+ (0, output_1.err)(`[suite] ${file} errored: ${msg}`);
262
+ attemptRows.push({
263
+ id: '',
264
+ file,
265
+ name: (0, node_path_1.basename)(file, (0, node_path_1.extname)(file)),
266
+ ok: false,
267
+ durationMs: Date.now() - t0,
268
+ costUsd: 0,
269
+ steps: 0,
270
+ passedSteps: 0,
271
+ failedSteps: 0,
272
+ modelRepairs: 0,
273
+ failure: msg.split('\n')[0],
274
+ });
131
275
  const broken = (0, errors_1.isEnvError)(e) ? await stillBroken(deps) : undefined;
132
- if (broken) {
133
- // This test never ran, so it gets no row — notRun starts at the CURRENT file.
134
- aborted = { reason: `reset failed: ${broken}`, notRun: files.slice(i) };
276
+ if (lastAttempt) {
277
+ if (broken)
278
+ aborted = { reason: broken, notRun: files.slice(i + 1) };
135
279
  break;
136
280
  }
137
- (0, output_1.err)(`[suite] reset failed (${e.message}) — continuing`);
138
- }
139
- }
140
- const t0 = Date.now();
141
- try {
142
- const r = await deps.runTest((0, node_path_1.join)(dir, file));
143
- results.push(toSuiteResult(file, r, Date.now() - t0));
144
- // The test itself reported an environment abort (exit 3 mid-plan). Same rule:
145
- // fatal only if the box is still broken. This test HAS a row and a real report,
146
- // so notRun starts after it.
147
- if (r.abortedForEnv) {
148
- const broken = await stillBroken(deps);
281
+ if (!isRetryableThrow(e))
282
+ break;
149
283
  if (broken)
150
- aborted = { reason: broken, notRun: files.slice(i + 1) };
284
+ await noteEnvRetry(file, attempt, broken);
151
285
  }
152
286
  }
153
- catch (e) {
154
- // A test that THREW (device gone, server unreachable, bad file) still becomes a
155
- // failed row — one broken test must not vaporize the suite report for the tests
156
- // that already ran. But if it threw because the environment is gone, stop.
157
- const msg = e instanceof Error ? e.message : String(e);
158
- (0, output_1.err)(`[suite] ${file} errored: ${msg}`);
159
- results.push({
160
- id: '',
161
- file,
162
- name: (0, node_path_1.basename)(file, (0, node_path_1.extname)(file)),
163
- ok: false,
164
- durationMs: Date.now() - t0,
165
- costUsd: 0,
166
- steps: 0,
167
- passedSteps: 0,
168
- failedSteps: 0,
169
- modelRepairs: 0,
170
- failure: msg.split('\n')[0],
171
- });
172
- const broken = (0, errors_1.isEnvError)(e) ? await stillBroken(deps) : undefined;
173
- if (broken)
174
- aborted = { reason: broken, notRun: files.slice(i + 1) };
287
+ if (attemptRows.length === 0) {
288
+ // Every attempt was blocked by a failing reset, so the test never ran and gets no
289
+ // row — `aborted.notRun` (set above) already names it. Nothing to merge.
290
+ break;
291
+ }
292
+ const merged = mergeSuiteAttempts(attemptRows);
293
+ results.push(merged);
294
+ if (merged.flaky) {
295
+ const n = merged.attempts?.length ?? 0;
296
+ const warn = `${file} passed on retry after ${n} failed attempt${n === 1 ? '' : 's'}`;
297
+ warnings.push(warn);
298
+ (0, output_1.err)(`[suite] WARN ${warn}`);
175
299
  }
176
300
  }
177
301
  if (aborted) {
@@ -189,6 +313,7 @@ async function cmdSuite(dirArg, flags, deps) {
189
313
  totals: (0, report_1.suiteTotals)(results),
190
314
  tests: results,
191
315
  ...(aborted ? { aborted } : {}),
316
+ ...(warnings.length ? { warnings } : {}),
192
317
  };
193
318
  // .verikun/suites/<id>/ sits beside .verikun/runs/<id>/, so index.html reaches a
194
319
  // test report at ../../runs/<id>/report.html — the linkBase below.
@@ -198,8 +323,12 @@ async function cmdSuite(dirArg, flags, deps) {
198
323
  (0, node_fs_1.writeFileSync)((0, node_path_1.join)(outDir, 'index.html'), (0, report_1.toSuiteHtml)(suite, { linkBase: '../../' }));
199
324
  const t = suite.totals;
200
325
  (0, output_1.err)(`[suite] ${t.passed}/${t.tests} passed · ${t.steps} steps · $${t.costUsd.toFixed(4)} · ${(t.durationMs / 1000).toFixed(1)}s`);
201
- for (const r of results)
202
- (0, output_1.err)(` ${r.ok ? 'PASS' : 'FAIL'} ${r.file}${r.failure ? ` — ${r.failure}` : ''}`);
326
+ for (const r of results) {
327
+ const tag = r.flaky ? 'FLAKY' : r.ok ? 'PASS' : 'FAIL';
328
+ (0, output_1.err)(` ${tag} ${r.file}${r.failure ? ` — ${r.failure}` : r.flaky ? ' — passed on retry' : ''}`);
329
+ }
330
+ if (warnings.length)
331
+ (0, output_1.err)(`[suite] ${warnings.length} warning(s)`);
203
332
  (0, output_1.err)(`[suite] overview: ${(0, node_path_1.join)(outDir, 'index.html')}`);
204
333
  if ((0, args_1.flagBool)(flags, 'json'))
205
334
  (0, output_1.json)(suite);
@@ -207,6 +336,7 @@ async function cmdSuite(dirArg, flags, deps) {
207
336
  (0, output_1.out)(outDir); // primary machine result: the suite directory
208
337
  // The CI gate: any failed test fails the invocation (mirrors `vk run archive`). An
209
338
  // environment abort exits 3 instead, so CI can tell "the runner is broken" from "the
210
- // app regressed" — the whole point of stopping early.
339
+ // app regressed" — the whole point of stopping early. A flake that recovered is ok
340
+ // (exit 0) with a warning — that is the whole point of --retries.
211
341
  return aborted ? 3 : t.failed > 0 ? 1 : 0;
212
342
  }
package/dist/version.js CHANGED
@@ -3,4 +3,4 @@ Object.defineProperty(exports, "__esModule", { value: true });
3
3
  exports.VERSION = void 0;
4
4
  // GENERATED by scripts/gen-version.mjs from package.json's "version" at build time
5
5
  // (the `prebuild` script). Do NOT edit by hand; bump package.json instead.
6
- exports.VERSION = '0.16.0';
6
+ exports.VERSION = '0.18.0';
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "verikun",
3
- "version": "0.16.0",
3
+ "version": "0.18.0",
4
4
  "description": "Drive Android emulators/devices and iOS simulators for AI agents: tap, type, swipe, screenshot, and inspect the UI hierarchy by semantic identifiers — like Puppeteer for native apps.",
5
5
  "keywords": [
6
6
  "android",