verikun 0.7.0 → 0.9.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -1,15 +1,12 @@
1
1
  # verikun
2
2
 
3
- Drive a connected Android device/emulator or iOS simulator the way
4
- Puppeteer drives a browser — **tap, type, swipe, screenshot**, and most
5
- importantly **inspect the UI hierarchy by semantic identifiers** so an AI agent
6
- can act and then *verify* what happened.
3
+ > **Agent-driven, natural-language mobile tests — during agent development or in CI.** Self-healing and self-improving, with cost caps and test reports.
7
4
 
8
- It is a thin, deterministic, zero-runtime-dependency wrapper over `adb` (Android)
9
- and `idb` + `xcrun simctl` (iOS) that turns the raw `uiautomator` dump (Android)
10
- or `idb ui describe-all` accessibility tree (iOS) into a compact, token-efficient
11
- list of meaningful elements addressable by `resource-id` / accessibility id,
12
- visible text, accessibility label, or class.
5
+ - **Agent CLI** `vk <command>`: one-shot commands to inspect the screen as a semantic tree (or screenshot) and act on it.
6
+ - **Puppeteer for native mobile** — a thin wrapper over native Android and iOS automation runners with zero runtime dependencies.
7
+ - **Natural-language tests** — `vk ai <file>`: runs plain-English tests, compiled once and replayed model-free (~$0), calling a model only to self-heal a drifted step.
8
+ - **Self-improving** the agent runner will provide prescriptive improvements to existing scripts to help stabilise flakiness for future runs.
9
+ - **CI-ready** `vk suite` runs a folder of tests as one gated pass/fail run; `vk server` exposes a real device over an authenticated tunnel so a disposable CI runner (no phone attached) can still drive it.
13
10
 
14
11
  ```
15
12
  $ vk ui
@@ -23,6 +20,12 @@ $ vk tap @sign_in_btn
23
20
  tapped [3] Button "Sign in" @sign_in_btn (540,1020) tap
24
21
  ```
25
22
 
23
+ ## Skill/plugin instead of MCP
24
+
25
+ verikun ships as a skill and plugin, not an MCP server, and that is deliberate. A skill lets us **guide the agent on how to use verikun** — when to inspect the hierarchy, what to assert, which command fits the step, and how to read the result back. That domain knowledge travels with the tool, so the agent drives the device *well*, not just correctly.
26
+
27
+ There is also no need for an MCP here: verikun runs locally with all its dependencies, and the agent calls it through the plain `vk` CLI — no shared session, data, or authentication to broker.
28
+
26
29
  ## Install
27
30
 
28
31
  Requires Node ≥ 18 and the Android platform-tools (`adb`) on your `PATH`.
@@ -44,6 +47,16 @@ This repo doubles as a Claude Code [plugin marketplace](https://code.claude.com/
44
47
 
45
48
  The plugin ships the **skill**; the `vk` **CLI** is a separate Node package — install it with `npm install -g verikun` (see [Install](#install) above) so `vk` lands on your `PATH`. The compiled `dist/` is gitignored, so it isn't bundled into the installed plugin.
46
49
 
50
+ ### Install the skill for other agents (Cursor, Copilot, Windsurf, …)
51
+
52
+ Not using Claude Code? The skill is a plain [`SKILL.md`](.claude/skills/verikun/SKILL.md) with `name`/`description` frontmatter, so [`vercel-labs/skills`](https://github.com/vercel-labs/skills) can install it into any of the 70+ agents it supports. Install the `vk` CLI (see [Install](#install) above), then pull the skill straight from this repo:
53
+
54
+ ```sh
55
+ npx skills add ddikman/verikun --skill verikun # pick your agent when prompted
56
+ ```
57
+
58
+ Add `--agent cursor` (or `windsurf`, `github-copilot`, `opencode`, …) to target one directly, and `-g` to install it globally instead of into the current project. As with the plugin, this installs the **skill** only — the `vk` **CLI** still comes from `npm install -g verikun`.
59
+
47
60
  ## Quick start
48
61
 
49
62
  ```sh
@@ -90,7 +103,7 @@ vk screenshot # -> ./.verikun/screen.png
90
103
  ### AI
91
104
  | Command | Description |
92
105
  |---|---|
93
- | `ai <file> [--model m] [--max-cost-usd n] [--timeout dur] [--cost-override in/out] [--effort e] [--package pkg] [--app-build id] [--server url] [--show-plan] [--recompile] [--json]` | Run a plain-English test: compile it to a deterministic plan once, replay it model-free, and self-heal failures via the model. Needs `ANTHROPIC_API_KEY` or `OPENAI_API_KEY` (per model). See [AI](#ai--natural-language-tests). |
106
+ | `ai <file> [--model m] [--max-cost-usd n] [--timeout dur] [--cost-override in/out] [--effort e] [--package pkg] [--app-build id] [--server url] [--show-plan] [--recompile] [--json]` | Run a plain-English test: compile it to a deterministic plan once, replay it model-free, and self-heal failures via the model. Needs `ANTHROPIC_API_KEY` or `OPENAI_API_KEY` (per model), or no key with `--model codex-cli` (a logged-in `codex` CLI). See [AI](#ai--natural-language-tests). |
94
107
  | `suite <dir> [--app <id>] [--name n] [--server url] [--json]` (+ all `ai` flags) | Run every `*.md` in `<dir>` as one sequential suite with an overview report and a non-zero exit on failure — the CI gate. See [Suites](#suites--run-a-directory-of-tests). |
95
108
 
96
109
  ### Remote
@@ -197,7 +210,9 @@ printf 'launch com.example.app\nassert @home_tab\nrun archive smoke\n' | vk batc
197
210
  with no model calls on the happy path**. The model is woken only to *repair* a step
198
211
  whose selector stops resolving; a green run persists the repaired plan, so the next
199
212
  run is free again. That is what keeps a CI suite's steady-state token cost near zero.
200
- Needs `ANTHROPIC_API_KEY` (Claude models) or `OPENAI_API_KEY` (OpenAI models).
213
+ Needs `ANTHROPIC_API_KEY` (Claude models) or `OPENAI_API_KEY` (OpenAI models) — or **no
214
+ key** with `--model codex-cli`, which drives an already-logged-in `codex` CLI off your
215
+ ChatGPT subscription (`codex login` once; verikun just needs the binary on PATH).
201
216
 
202
217
  ```sh
203
218
  # onboarding.md (plain English):
@@ -226,12 +241,19 @@ a hard iteration cap and stop early if the screen stops changing.
226
241
  repair can't spend or hang without limit. `--cost-override <input/output>` overrides
227
242
  the bundled per-1M price table if it drifts.
228
243
  - **`--model`** picks the model and its provider — Anthropic (`claude-haiku-4-5` ·
229
- `claude-sonnet-4-6` (default) · `claude-opus-4-8` · `claude-fable-5`) or OpenAI
244
+ `claude-sonnet-4-6` (default) · `claude-opus-4-8` · `claude-fable-5`), OpenAI
230
245
  (`gpt-5.4-mini` · `gpt-5.4` · `gpt-5.5`), each read from its own key
231
- (`ANTHROPIC_API_KEY` / `OPENAI_API_KEY`); **`--recompile`** ignores the cache.
246
+ (`ANTHROPIC_API_KEY` / `OPENAI_API_KEY`), or the CLI backend **`codex-cli`** (no key
247
+ the logged-in `codex` binary; spend is on your subscription, so its cost line is `$0` and
248
+ `--max-cost-usd` / `--cost-override` are no-ops); **`--recompile`** ignores the cache.
232
249
  - An `ai` run records like any other flow, so it produces the same JUnit + HTML report —
233
250
  with the cost line and any **suggested test improvements** (workarounds the model
234
251
  applied, which you can fold back into the prose to stabilize the test and cut tokens).
252
+ - **Review screenshots are inserted automatically.** The compiler adds `screenshot` steps
253
+ around transitions and inside loops, so the report carries a before/after visual trail
254
+ for post-run review. They are dumped for humans, never read back by the model (no token
255
+ cost on replay), and never gate the test — a capture that hiccups is logged and skipped,
256
+ not a failure.
235
257
 
236
258
  ## Suites — run a directory of tests
237
259
 
@@ -260,8 +282,8 @@ vk suite tests/ --app com.example.app --server "$VERIKUN_SERVER" # remote devi
260
282
  - **`index.html`** — a summary page linking every test's `report.html`.
261
283
  - **Exit code is the CI gate:** `1` if any test failed, `0` all green, `2` bad/empty
262
284
  directory. All `ai` flags (`--model`, `--max-cost-usd`, `--timeout`, …) apply to
263
- every test; the model's key (`ANTHROPIC_API_KEY` / `OPENAI_API_KEY`) is checked
264
- up front.
285
+ every test; the provider (`ANTHROPIC_API_KEY` / `OPENAI_API_KEY`, or the `codex` CLI for
286
+ `--model codex-cli`) is checked up front.
265
287
 
266
288
  ## Remote devices — `vk server`
267
289
 
@@ -424,6 +446,13 @@ safely resample (palette, 16-bit, interlaced) are written through untouched, so
424
446
  screenshot is never corrupted — only sometimes left full-size (noted on stderr).
425
447
  Failure-evidence captures in test-run reports stay full-resolution for debugging.
426
448
 
449
+ **Read-back vs evidence.** The downscaling above matters when an agent *reads a
450
+ screenshot back into its context* to decide the next action — that is the token cost to
451
+ manage. A screenshot taken purely as **report evidence and never read back** costs nothing
452
+ at runtime, so driving a flow to a report should capture liberally around transitions.
453
+ `vk ai` does this automatically (see [AI](#ai--natural-language-tests)); when driving by
454
+ hand, `vk screenshot` around each screen change and leave the PNG in the report.
455
+
427
456
  ## How it works
428
457
 
429
458
  ```
@@ -0,0 +1,242 @@
1
+ "use strict";
2
+ Object.defineProperty(exports, "__esModule", { value: true });
3
+ exports.CliProvider = exports.CODEX_SPEC = void 0;
4
+ exports.schemaInstruction = schemaInstruction;
5
+ exports.extractJson = extractJson;
6
+ const node_fs_1 = require("node:fs");
7
+ const node_path_1 = require("node:path");
8
+ const node_os_1 = require("node:os");
9
+ const errors_1 = require("../errors");
10
+ const format_1 = require("../ui/format");
11
+ const ir_1 = require("./ir");
12
+ const grammar_1 = require("./grammar");
13
+ const openai_1 = require("./openai");
14
+ const exec_1 = require("../exec");
15
+ // The CLI-agent provider: instead of an HTTP API + API key, drive an already-authenticated
16
+ // coding-agent CLI (codex / cursor-agent) as a one-shot text->JSON transformer. This lets a
17
+ // user compile/repair `vk ai` tests off their existing ChatGPT/Cursor SUBSCRIPTION — the CLI
18
+ // carries its own login, so verikun needs no key (just the binary on PATH). A sibling to
19
+ // claude.ts/openai.ts behind the same AgentProvider seam; like openai.ts is one class
20
+ // parameterized by baseUrl, this is one class parameterized by a CliAgentSpec per binary.
21
+ //
22
+ // Structured output: these CLIs are not HTTP endpoints, so there is no output_config /
23
+ // response_format. codex enforces a schema natively (--output-schema); a CLI without one
24
+ // gets the schema injected into the prompt. Either way parsePlan/validateNode (engine.ts)
25
+ // stays the execution trust boundary — ir.ts documents this exact "parse path when
26
+ // structured output is unavailable", so a malformed/hallucinated result is still rejected.
27
+ //
28
+ // These CLIs are AGENTS (tools, a working dir, a coding-oriented system prompt), so a
29
+ // forceful preamble + a read-only sandbox (per spec) coerce them into a pure transform that
30
+ // never touches the repo. The model runs here ONLY on compile + repair, never on replay.
31
+ // An agentic CLI compile is far slower than an HTTP call, and runText's 30s default would
32
+ // kill it mid-think. This is the per-invocation wall-clock cap (a hung spawn is killed and
33
+ // mapped to exit 3 by exec.ts); the engine's --timeout still bounds the whole run BETWEEN calls.
34
+ const DEFAULT_REQUEST_TIMEOUT_MS = 180_000;
35
+ const PREAMBLE = 'You are being used as a pure text-to-JSON transformer, NOT a coding assistant. ' +
36
+ 'Do NOT read, write, or edit any files. Do NOT run shell commands or use any tools. ' +
37
+ 'Do NOT explain, summarize, or add any commentary. Respond with ONLY a single JSON ' +
38
+ 'object as your final message, exactly matching the specification below.';
39
+ /** codex (OpenAI Codex CLI): non-interactive `codex exec` with NATIVE JSON-schema output
40
+ * (--output-schema) — the cleanest CLI path. Runs read-only in a neutral dir so it can't touch
41
+ * the verikun tree; the final (schema-shaped) message is written to --output-last-message, which
42
+ * we read back (deterministic, unlike parsing stdout, whose decoration is version-dependent). */
43
+ exports.CODEX_SPEC = {
44
+ id: 'codex',
45
+ bin: 'codex',
46
+ schema: 'file',
47
+ // codex's --output-schema is OpenAI strict Structured Outputs — adapt the shared ir.ts schema
48
+ // the same way openai.ts does (all keys required, optionals made nullable, additionalProperties
49
+ // false). parsePlan tolerates the resulting nulls (package/platform → undefined).
50
+ encodeSchema: openai_1.toStrictSchema,
51
+ usesOutputFile: true,
52
+ buildArgs(prompt, { schemaFile, outFile, cwd, model }) {
53
+ const args = [
54
+ 'exec',
55
+ '--skip-git-repo-check', // don't require (or scan) a git repo
56
+ '--cd', cwd, // root the agent in a neutral temp dir, not the verikun working tree
57
+ '--sandbox', 'read-only', // hard backstop: the agent cannot write anything
58
+ '--ephemeral', // don't persist session files for a stateless transform
59
+ ];
60
+ if (schemaFile)
61
+ args.push('--output-schema', schemaFile); // constrain the final message to the schema
62
+ if (outFile)
63
+ args.push('--output-last-message', outFile); // final message -> file we read back
64
+ if (model)
65
+ args.push('--model', model);
66
+ args.push(prompt); // prompt is the trailing positional
67
+ return args;
68
+ },
69
+ rawText: (stdout) => stdout, // fallback only; the message is read from --output-last-message
70
+ loginHint: 'run `codex login` to sign in with your ChatGPT subscription (no API key needed)',
71
+ };
72
+ // Collision-free temp-file names within a process without needing Math.random() (which the
73
+ // plan-cache/version paths keep deterministic); pid + a counter is enough.
74
+ let tempCounter = 0;
75
+ class CliProvider {
76
+ opts;
77
+ run;
78
+ baseTmp;
79
+ timeoutMs;
80
+ constructor(opts) {
81
+ this.opts = opts;
82
+ this.run = opts.runImpl ?? exec_1.runText;
83
+ this.baseTmp = opts.tmpDir ?? (0, node_os_1.tmpdir)();
84
+ this.timeoutMs = opts.requestTimeoutMs ?? DEFAULT_REQUEST_TIMEOUT_MS;
85
+ }
86
+ async compile(input) {
87
+ const parts = [];
88
+ if (input.pkg)
89
+ parts.push(`App package: ${input.pkg}`);
90
+ parts.push(`Platform: ${input.platform}`);
91
+ if (input.seed) {
92
+ parts.push('A plan compiled for a PREVIOUS build of this app follows. Reuse it where the test still holds; ' +
93
+ 'change only what the test now requires. PRIOR PLAN:\n' +
94
+ JSON.stringify(input.seed, null, 2));
95
+ }
96
+ parts.push('NATURAL-LANGUAGE TEST:\n' + input.nl);
97
+ const json = this.call(grammar_1.GRAMMAR, parts.join('\n\n'), ir_1.PLAN_JSON_SCHEMA);
98
+ // usage:{} — a CLI is billed to the user's subscription, not per token, so cost is $0
99
+ // (documented no-op for --max-cost-usd). The run is still bounded by maxRepairs + --timeout.
100
+ return { plan: (0, ir_1.parsePlan)(json), usage: {} };
101
+ }
102
+ async repair(ctx) {
103
+ const parts = ['FAILED STEP: ' + JSON.stringify(ctx.failedStep), 'FAILURE: ' + ctx.reason];
104
+ if (ctx.candidates && ctx.candidates.length) {
105
+ parts.push(`The selector matched ${ctx.candidates.length} elements (ambiguous) — pick a more specific selector for the SAME intended element, or give_up if none of them is it.`);
106
+ }
107
+ parts.push('CURRENT SCREEN:\n' + (0, format_1.formatCompact)(ctx.hierarchy));
108
+ const json = this.call(grammar_1.REPAIR_GRAMMAR, parts.join('\n\n'), ir_1.REPAIR_DECISION_JSON_SCHEMA);
109
+ const decision = (json ?? {});
110
+ if (decision.decision === 'give_up') {
111
+ return {
112
+ replaceStep: null,
113
+ declineReason: decision.reason?.trim() || 'no element on the current screen matches the step intent',
114
+ usage: {},
115
+ };
116
+ }
117
+ // Hand the proposed leaf back UNVALIDATED — engine.ts validates every repair against the
118
+ // grammar before splicing (it is the execution trust boundary), exactly like the API providers.
119
+ return { replaceStep: (decision.step ?? null), usage: {} };
120
+ }
121
+ /** Spawn the CLI once and return the parsed JSON object it produced. Synchronous (spawnSync);
122
+ * the async method wrappers satisfy the Promise-returning AgentProvider seam. */
123
+ call(system, user, schema) {
124
+ const spec = this.opts.spec;
125
+ const promptParts = [PREAMBLE, system];
126
+ if (spec.schema === 'prompt')
127
+ promptParts.push(schemaInstruction(schema));
128
+ promptParts.push(user);
129
+ const prompt = promptParts.join('\n\n');
130
+ let schemaFile;
131
+ let outFile;
132
+ try {
133
+ if (spec.schema === 'file') {
134
+ const encoded = spec.encodeSchema ? spec.encodeSchema(schema) : schema;
135
+ schemaFile = this.writeTemp('schema', '.json', JSON.stringify(encoded));
136
+ }
137
+ if (spec.usesOutputFile)
138
+ outFile = this.tempPath('out', '.txt'); // path only; the CLI writes it
139
+ const args = spec.buildArgs(prompt, { schemaFile, outFile, cwd: this.baseTmp, model: this.opts.model });
140
+ // runText throws CliError(exit 3) for ENOENT / timeout / spawn failure — let it propagate.
141
+ const res = this.run(spec.bin, args, { timeout: this.timeoutMs, cwd: this.baseTmp });
142
+ if (res.code !== 0) {
143
+ // Lead with the CLI's own stderr — it carries the real reason (usage limit, auth, a bad
144
+ // flag). Only fall back to the login hint when stderr said nothing, so we don't
145
+ // mis-suggest a re-login for e.g. a quota error.
146
+ const detail = tail(res.stderr);
147
+ const suffix = detail ? `: ${detail}` : ` — ${spec.loginHint}`;
148
+ throw new errors_1.CliError(`\`${spec.bin}\` exited ${res.code}${suffix}`, 3);
149
+ }
150
+ // Prefer the message file (deterministic); fall back to stdout if the CLI wrote nothing there.
151
+ const fromFile = outFile ? readIfExists(outFile).trim() : '';
152
+ const text = fromFile || spec.rawText(res.stdout).trim();
153
+ if (!text)
154
+ throw new errors_1.CliError(`\`${spec.bin}\` returned an empty response.`, 1);
155
+ return extractJson(text);
156
+ }
157
+ finally {
158
+ for (const f of [schemaFile, outFile]) {
159
+ if (!f)
160
+ continue;
161
+ try {
162
+ (0, node_fs_1.unlinkSync)(f);
163
+ }
164
+ catch {
165
+ /* best-effort cleanup — a leftover temp file is harmless */
166
+ }
167
+ }
168
+ }
169
+ }
170
+ tempPath(kind, ext) {
171
+ return (0, node_path_1.join)(this.baseTmp, `verikun-${this.opts.spec.id}-${kind}-${process.pid}-${tempCounter++}${ext}`);
172
+ }
173
+ writeTemp(kind, ext, content) {
174
+ const file = this.tempPath(kind, ext);
175
+ (0, node_fs_1.writeFileSync)(file, content, 'utf8');
176
+ return file;
177
+ }
178
+ }
179
+ exports.CliProvider = CliProvider;
180
+ /** Read a file, returning '' if it does not exist / can't be read — lets the message-file path
181
+ * fall back to stdout when a CLI didn't populate --output-last-message. */
182
+ function readIfExists(path) {
183
+ try {
184
+ return (0, node_fs_1.readFileSync)(path, 'utf8');
185
+ }
186
+ catch {
187
+ return '';
188
+ }
189
+ }
190
+ /** For a CLI with no native schema flag (schema:'prompt'): describe the required output shape
191
+ * inline. parsePlan/validateNode still re-checks whatever comes back. */
192
+ function schemaInstruction(schema) {
193
+ return ('Your entire response MUST be a single JSON object matching this JSON Schema exactly, ' +
194
+ 'with no prose and no code fences:\n' + JSON.stringify(schema));
195
+ }
196
+ /** Tolerantly pull a JSON object out of a CLI's stdout. codex's --output-schema output is
197
+ * already clean JSON; a schema-in-prompt CLI may wrap it in ```fences``` or a sentence. The
198
+ * brace scanner is string/escape aware, so it finds the object even inside a fence or after a
199
+ * "Here is the plan:" preamble. Throws CliError(exit 1) on failure — parsePlan is still the gate. */
200
+ function extractJson(text) {
201
+ const candidate = firstBalancedObject(text) ?? text.trim();
202
+ try {
203
+ return JSON.parse(candidate);
204
+ }
205
+ catch {
206
+ throw new errors_1.CliError('the CLI provider did not return parseable JSON.', 1);
207
+ }
208
+ }
209
+ /** The first balanced `{...}` in `text`, honoring string literals + backslash escapes so a
210
+ * brace inside a JSON string value doesn't throw off the depth count. null if there is none. */
211
+ function firstBalancedObject(text) {
212
+ const start = text.indexOf('{');
213
+ if (start < 0)
214
+ return null;
215
+ let depth = 0;
216
+ let inStr = false;
217
+ let esc = false;
218
+ for (let i = start; i < text.length; i++) {
219
+ const c = text[i];
220
+ if (inStr) {
221
+ if (esc)
222
+ esc = false;
223
+ else if (c === '\\')
224
+ esc = true;
225
+ else if (c === '"')
226
+ inStr = false;
227
+ continue;
228
+ }
229
+ if (c === '"')
230
+ inStr = true;
231
+ else if (c === '{')
232
+ depth++;
233
+ else if (c === '}' && --depth === 0)
234
+ return text.slice(start, i + 1);
235
+ }
236
+ return null;
237
+ }
238
+ /** A trimmed, size-capped tail of a CLI's stderr for error messages ('' when it wrote nothing). */
239
+ function tail(stderr, n = 500) {
240
+ const t = stderr.trim();
241
+ return t.length > n ? '…' + t.slice(-n) : t;
242
+ }
@@ -21,6 +21,11 @@ const MODELS = {
21
21
  'gpt-5.4-mini': { input: 0.75, output: 4.5, provider: 'openai' },
22
22
  'gpt-5.4': { input: 2.5, output: 15, provider: 'openai' },
23
23
  'gpt-5.5': { input: 5, output: 30, provider: 'openai' },
24
+ // CLI-agent backend: billed to the user's ChatGPT subscription via the `codex` CLI, not per
25
+ // token — so price is $0 and --max-cost-usd/--cost-override are inert no-ops (the run is
26
+ // bounded by maxRepairs + --timeout instead). Named `codex-cli` to read clearly as "the CLI"
27
+ // and to avoid colliding with cursor's own `gpt-5.x-codex` model aliases.
28
+ 'codex-cli': { input: 0, output: 0, provider: 'codex' },
24
29
  };
25
30
  exports.MODEL_PRICES = Object.fromEntries(Object.entries(MODELS).map(([m, s]) => [m, { input: s.input, output: s.output }]));
26
31
  exports.ALLOWED_MODELS = Object.keys(MODELS);
@@ -8,6 +8,11 @@ const ir_1 = require("./ir");
8
8
  const describe = (leaf) => [leaf.command, ...leaf.positionals, ...leaf.flags.map((f) => (f.value === 'true' ? `--${f.name}` : `--${f.name} ${f.value}`))]
9
9
  .join(' ')
10
10
  .trim();
11
+ /** A screenshot leaf is best-effort review evidence, not a gate. The `vk ai`
12
+ * grammar has the model sprinkle them around transitions, so a capture that
13
+ * fails (a device hiccup on screencap) must never turn a green run red — see
14
+ * the guard in execLeaf. */
15
+ const isScreenshotLeaf = (leaf) => leaf.command === 'screenshot' || leaf.command === 'shot';
11
16
  /** A structural fingerprint of the screen: sorted id+text+type set. Used for the
12
17
  * loop no-progress check — deliberately NOT the raw hierarchy (its node ordering
13
18
  * is nondeterministic between identical states, which would false-trip). */
@@ -129,6 +134,19 @@ async function runPlan(plan, deps) {
129
134
  }
130
135
  if (outcome.code === 0)
131
136
  return { status: 'ok' };
137
+ // A review screenshot is best-effort evidence, never a gate. The grammar has the
138
+ // model insert them liberally around transitions, so a capture that fails (a
139
+ // device hiccup on screencap) must not fail an otherwise-green run. Log it,
140
+ // downgrade the just-recorded failed step to a clean pass (markLastStepHealed
141
+ // sets status=passed/exitCode=0 and drops the failure evidence) so the report and
142
+ // JUnit stay consistent with the green run, then continue. Scoped to screenshot/
143
+ // shot — every other command's failure stays terminal.
144
+ if (isScreenshotLeaf(current)) {
145
+ const why = outcome.error ? outcome.error.message.split('\n')[0] : `exited ${outcome.code}`;
146
+ deps.log(`[ai] ${where}: screenshot capture failed (${why}) — continuing (best-effort review screenshot)`);
147
+ deps.markHealed?.(`screenshot capture failed (${why}) — skipped (best-effort)`);
148
+ return { status: 'ok' };
149
+ }
132
150
  if (isHealable(outcome)) {
133
151
  return { status: 'fail', where, reason: `unresolved after ${maxRepairs} repair attempt(s): ${outcome.error.message.split('\n')[0]}` };
134
152
  }
@@ -54,7 +54,12 @@ RULES:
54
54
  - assert is for VERIFICATION only and is terminal — never use it as a step you expect to
55
55
  fail. Put genuinely-optional UI behind if-present.
56
56
  - Prefer resource-id / accessibility selectors over visible text where possible.
57
- - Translate the test literally and minimally; do not invent steps the prose does not imply.`;
57
+ - Translate the test literally and minimally: do not invent ACTION steps (tap/text/swipe/key/assert)
58
+ the prose does not imply. The ONE exception is screenshot — insert screenshot steps liberally as
59
+ post-run review evidence: after each screen transition (launch, a navigation tap, a submit, a
60
+ swipe/scroll) AND inside if-present/repeat bodies, so a failing branch or loop iteration is visible
61
+ in the report. Screenshots never affect the result; err toward too many. They are dumped into the
62
+ report for humans and never read back, so they are free on replay.`;
58
63
  exports.REPAIR_GRAMMAR = `A single step in a verikun plan failed to resolve its selector against the live screen
59
64
  (shown below). Decide between two outcomes — and be STRICT:
60
65
 
package/dist/cli.js CHANGED
@@ -40,6 +40,7 @@ exports.healNote = healNote;
40
40
  exports.parseDuration = parseDuration;
41
41
  exports.waitWindowMs = waitWindowMs;
42
42
  exports.waitNote = waitNote;
43
+ exports.formatDeviceTable = formatDeviceTable;
43
44
  exports.confineToCwd = confineToCwd;
44
45
  exports.assertSafeAppId = assertSafeAppId;
45
46
  exports.chooseLogOpts = chooseLogOpts;
@@ -62,6 +63,7 @@ const image_1 = require("./image");
62
63
  const engine_1 = require("./agent/engine");
63
64
  const claude_1 = require("./agent/claude");
64
65
  const openai_1 = require("./agent/openai");
66
+ const cli_provider_1 = require("./agent/cli-provider");
65
67
  const cache_1 = require("./agent/cache");
66
68
  const cost_1 = require("./agent/cost");
67
69
  const remote_1 = require("./agent/remote");
@@ -202,13 +204,34 @@ function cmdDevices(ctx) {
202
204
  (0, output_1.err)('No devices found.');
203
205
  return 0;
204
206
  }
205
- for (const d of allDevices) {
206
- (0, output_1.out)([d.platform, d.serial, d.state, d.model ?? '', d.product ? `(${d.product})` : '', d.note ? `[${d.note}]` : '']
207
- .filter(Boolean)
208
- .join('\t'));
209
- }
207
+ for (const line of formatDeviceTable(allDevices))
208
+ (0, output_1.out)(line);
210
209
  return 0;
211
210
  }
211
+ /**
212
+ * Render the device list as an aligned, headed table (header line first, then one
213
+ * line per device). Optional columns (MODEL/PRODUCT/NOTE) are dropped when no device
214
+ * populates them; every shown cell is padded to its column width so columns line up
215
+ * regardless of which cells are empty — the previous `.filter(Boolean).join('\t')`
216
+ * dropped empty cells, sliding later cells into earlier tab stops. Exported for unit
217
+ * testing.
218
+ */
219
+ function formatDeviceTable(devices) {
220
+ const columns = [
221
+ { header: 'PLATFORM', get: (d) => d.platform },
222
+ { header: 'SERIAL', get: (d) => d.serial },
223
+ { header: 'STATE', get: (d) => d.state },
224
+ { header: 'MODEL', get: (d) => d.model ?? '', optional: true },
225
+ { header: 'PRODUCT', get: (d) => d.product ?? '', optional: true },
226
+ { header: 'NOTE', get: (d) => d.note ?? '', optional: true },
227
+ ];
228
+ // Drop optional columns that no device populates (e.g. NOTE for an Android-only list).
229
+ const shown = columns.filter((c) => !c.optional || devices.some((d) => c.get(d) !== ''));
230
+ const rows = [shown.map((c) => c.header), ...devices.map((d) => shown.map((c) => c.get(d)))];
231
+ const widths = shown.map((_, i) => Math.max(...rows.map((r) => r[i].length)));
232
+ // Pad every cell except the last shown column (no trailing whitespace); join with 2 spaces.
233
+ return rows.map((r) => r.map((cell, i) => (i === r.length - 1 ? cell : cell.padEnd(widths[i]))).join(' ').trimEnd());
234
+ }
212
235
  function cmdDoctor(ctx) {
213
236
  if (ctx.platform === 'ios') {
214
237
  try {
@@ -961,20 +984,45 @@ function readAiTest(file) {
961
984
  throw new errors_1.CliError(`ai: '${file}' is empty`, 2);
962
985
  return nl;
963
986
  }
964
- /** The env var carrying the API key for a model's provider (per-provider keys). */
965
- function keyEnvFor(model) {
966
- return (0, cost_1.providerFor)(model) === 'openai' ? 'OPENAI_API_KEY' : 'ANTHROPIC_API_KEY';
987
+ /** Is the backend for `model` usable right now? HTTP providers need their API key in env;
988
+ * a CLI provider needs its binary on PATH (auth lives in the CLI's own login, not an env key). */
989
+ function providerAvailable(model) {
990
+ switch ((0, cost_1.providerFor)(model)) {
991
+ case 'openai':
992
+ return !!process.env.OPENAI_API_KEY;
993
+ case 'codex':
994
+ return (0, exec_1.commandExists)(cli_provider_1.CODEX_SPEC.bin);
995
+ default:
996
+ return !!process.env.ANTHROPIC_API_KEY;
997
+ }
998
+ }
999
+ /** What's missing when a provider is unavailable — the tail of the preflight error message. */
1000
+ function providerRequirement(model) {
1001
+ switch ((0, cost_1.providerFor)(model)) {
1002
+ case 'openai':
1003
+ return 'OPENAI_API_KEY is not set';
1004
+ case 'codex':
1005
+ return `the \`${cli_provider_1.CODEX_SPEC.bin}\` CLI was not found on PATH — install it and run \`codex login\` (ChatGPT subscription, no API key)`;
1006
+ default:
1007
+ return 'ANTHROPIC_API_KEY is not set';
1008
+ }
967
1009
  }
968
- /** Route the model to its backend; each provider reads its own key. A missing key
969
- * means no provider (compile/repair unavailable) the same graceful degradation
970
- * as before. */
1010
+ /** Route the model to its backend. HTTP providers read their own key; a CLI provider shells
1011
+ * out to its logged-in binary. Unavailable → null (compile/repair off), the same graceful
1012
+ * degradation as before: a cached plan can still replay for free without any provider. */
971
1013
  function makeProvider(opts) {
972
- const apiKey = process.env[keyEnvFor(opts.model)];
973
- if (!apiKey)
974
- return null;
975
- return (0, cost_1.providerFor)(opts.model) === 'openai'
976
- ? new openai_1.OpenAiProvider({ model: opts.model, apiKey, effort: opts.effort })
977
- : new claude_1.ClaudeProvider({ model: opts.model, apiKey, effort: opts.effort });
1014
+ switch ((0, cost_1.providerFor)(opts.model)) {
1015
+ case 'openai': {
1016
+ const apiKey = process.env.OPENAI_API_KEY;
1017
+ return apiKey ? new openai_1.OpenAiProvider({ model: opts.model, apiKey, effort: opts.effort }) : null;
1018
+ }
1019
+ case 'codex':
1020
+ return (0, exec_1.commandExists)(cli_provider_1.CODEX_SPEC.bin) ? new cli_provider_1.CliProvider({ spec: cli_provider_1.CODEX_SPEC }) : null;
1021
+ default: {
1022
+ const apiKey = process.env.ANTHROPIC_API_KEY;
1023
+ return apiKey ? new claude_1.ClaudeProvider({ model: opts.model, apiKey, effort: opts.effort }) : null;
1024
+ }
1025
+ }
978
1026
  }
979
1027
  /** Obtain the plan: a cache hit (free) or a compile (pays tokens; may seed from a
980
1028
  * prior build's plan to avoid a full recompile). The fresh compile is cached right
@@ -987,7 +1035,7 @@ async function obtainPlan(key, file, opts, cost, provider) {
987
1035
  return { plan: cached.plan, cached: true };
988
1036
  }
989
1037
  if (!provider) {
990
- throw new errors_1.CliError(`${keyEnvFor(opts.model)} is not set \`vk ai\` needs it to compile the test (model ${opts.model}). Set it and retry.`, 3);
1038
+ throw new errors_1.CliError(`${providerRequirement(opts.model)} — needed to compile the test (model ${opts.model}).`, 3);
991
1039
  }
992
1040
  const seed = (0, cache_1.findSeed)(key);
993
1041
  if (seed)
@@ -1058,7 +1106,7 @@ async function runAiTest(file, opts, backend, platform, device) {
1058
1106
  const { plan, cached } = await obtainPlan(key, file, opts, cost, provider);
1059
1107
  // Running needs the provider for repair-on-failure; a cache hit with no key can't repair.
1060
1108
  if (!provider) {
1061
- throw new errors_1.CliError(`${keyEnvFor(opts.model)} is not set \`vk ai\` needs it to repair a failing step at runtime (model ${opts.model}).`, 3);
1109
+ throw new errors_1.CliError(`${providerRequirement(opts.model)} — needed to repair a failing step at runtime (model ${opts.model}).`, 3);
1062
1110
  }
1063
1111
  // The budget is a TOTAL-run ceiling: if the compile alone already crossed it, abort
1064
1112
  // before running. A cache hit spends nothing, so a free replay is still allowed.
@@ -1244,10 +1292,10 @@ async function cmdSuiteEntry(positionals, flags) {
1244
1292
  if (!dirArg)
1245
1293
  throw new errors_1.CliError('Usage: verikun suite <dir> [--app <id>] [--server url] [--name n] [--json]', 2);
1246
1294
  const opts = parseAiOptions(flags);
1247
- // Pre-flight the model key BEFORE touching any device/server: every test needs it
1295
+ // Pre-flight the provider BEFORE touching any device/server: every test needs it
1248
1296
  // to compile (on a cache miss) or to repair at runtime.
1249
- if (!process.env[keyEnvFor(opts.model)]) {
1250
- throw new errors_1.CliError(`${keyEnvFor(opts.model)} is not set \`vk suite\` needs it to compile/repair tests (model ${opts.model}).`, 3);
1297
+ if (!providerAvailable(opts.model)) {
1298
+ throw new errors_1.CliError(`${providerRequirement(opts.model)} — needed to compile/repair tests (model ${opts.model}).`, 3);
1251
1299
  }
1252
1300
  const reqPlatform = platformFromFlags(flags);
1253
1301
  const { backend, platform, device } = await resolveBackend(reqPlatform, deviceFromFlags(flags, reqPlatform), flags);
@@ -1538,13 +1586,16 @@ AI (run a natural-language test — compile once, replay model-free, self-heal)
1538
1586
  path. The model is woken only to repair a step
1539
1587
  that fails to resolve; a green run persists the
1540
1588
  (repaired) plan so the next run is free. Needs
1541
- ANTHROPIC_API_KEY (Claude) or OPENAI_API_KEY
1542
- (gpt-5.x). Progress -> stderr; the report path ->
1589
+ ANTHROPIC_API_KEY (Claude), OPENAI_API_KEY (gpt-5.x),
1590
+ or a logged-in agent CLI (--model codex-cli uses your
1591
+ 'codex login' ChatGPT subscription — no API key; cost
1592
+ is $0 so --max-cost-usd/--cost-override are no-ops).
1593
+ Progress -> stderr; the report path ->
1543
1594
  stdout. --show-plan prints the compiled IR without
1544
1595
  running; --recompile ignores the cache.
1545
1596
  Models: claude-haiku-4-5 | claude-sonnet-4-6
1546
1597
  (default) | claude-opus-4-8 | claude-fable-5 |
1547
- gpt-5.4-mini | gpt-5.4 | gpt-5.5.
1598
+ gpt-5.4-mini | gpt-5.4 | gpt-5.5 | codex-cli.
1548
1599
 
1549
1600
  SUITE (run a directory of natural-language tests as one gated suite)
1550
1601
  suite <dir> [--app <id>] [--name n] [--json] (+ all \`ai\` flags, incl. --server)
package/dist/exec.js CHANGED
@@ -1,8 +1,11 @@
1
1
  "use strict";
2
2
  Object.defineProperty(exports, "__esModule", { value: true });
3
3
  exports.runText = runText;
4
+ exports.commandExists = commandExists;
4
5
  exports.runBinary = runBinary;
5
6
  const node_child_process_1 = require("node:child_process");
7
+ const node_fs_1 = require("node:fs");
8
+ const node_path_1 = require("node:path");
6
9
  const errors_1 = require("./errors");
7
10
  const MAX_BUFFER = 64 * 1024 * 1024; // screenshots can be a few MB
8
11
  function describeError(cmd, args, err) {
@@ -14,18 +17,49 @@ function describeError(cmd, args, err) {
14
17
  }
15
18
  return new errors_1.CliError(`Failed to run '${cmd}': ${err.message}`, 3);
16
19
  }
17
- /** Run a command and capture stdout/stderr as UTF-8 text. */
20
+ /** Run a command and capture stdout/stderr as UTF-8 text. `cwd` runs it rooted elsewhere
21
+ * (the CLI-agent providers run in a neutral temp dir so they never touch the working tree). */
18
22
  function runText(cmd, args, opts = {}) {
19
23
  const r = (0, node_child_process_1.spawnSync)(cmd, args, {
20
24
  encoding: 'utf8',
21
25
  timeout: opts.timeout ?? 30000,
22
26
  input: opts.input,
27
+ cwd: opts.cwd,
23
28
  maxBuffer: MAX_BUFFER,
24
29
  });
25
30
  if (r.error)
26
31
  throw describeError(cmd, args, r.error);
27
32
  return { code: r.status ?? 0, stdout: r.stdout ?? '', stderr: r.stderr ?? '' };
28
33
  }
34
+ /** Is `bin` an executable on PATH (or a direct path to one)? Used to decide a CLI provider is
35
+ * available without invoking the agent — a cheap, pure PATH scan (no spawn, no runtime dep). */
36
+ function commandExists(bin) {
37
+ if (bin.includes('/') || bin.includes('\\'))
38
+ return isExecutableFile(bin);
39
+ const isWin = process.platform === 'win32';
40
+ const exts = isWin ? (process.env.PATHEXT ?? '.EXE;.CMD;.BAT;.COM').split(';') : [''];
41
+ for (const dir of (process.env.PATH ?? '').split(node_path_1.delimiter)) {
42
+ if (!dir)
43
+ continue;
44
+ for (const ext of exts)
45
+ if (isExecutableFile((0, node_path_1.join)(dir, bin + ext)))
46
+ return true;
47
+ }
48
+ return false;
49
+ }
50
+ function isExecutableFile(p) {
51
+ try {
52
+ if (!(0, node_fs_1.statSync)(p).isFile())
53
+ return false;
54
+ if (process.platform === 'win32')
55
+ return true; // Windows has no X bit; a matching file is enough
56
+ (0, node_fs_1.accessSync)(p, node_fs_1.constants.X_OK);
57
+ return true;
58
+ }
59
+ catch {
60
+ return false;
61
+ }
62
+ }
29
63
  /** Run a command and capture stdout as raw bytes (e.g. PNG screenshots). */
30
64
  function runBinary(cmd, args, opts = {}) {
31
65
  const r = (0, node_child_process_1.spawnSync)(cmd, args, {
package/dist/version.js CHANGED
@@ -3,4 +3,4 @@ Object.defineProperty(exports, "__esModule", { value: true });
3
3
  exports.VERSION = void 0;
4
4
  // GENERATED by scripts/gen-version.mjs from package.json's "version" at build time
5
5
  // (the `prebuild` script). Do NOT edit by hand; bump package.json instead.
6
- exports.VERSION = '0.7.0';
6
+ exports.VERSION = '0.9.0';
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "verikun",
3
- "version": "0.7.0",
3
+ "version": "0.9.0",
4
4
  "description": "Drive Android emulators/devices and iOS simulators for AI agents: tap, type, swipe, screenshot, and inspect the UI hierarchy by semantic identifiers — like Puppeteer for native apps.",
5
5
  "keywords": [
6
6
  "android",