verikun 0.9.0 → 0.11.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +25 -9
- package/dist/agent/claude.js +6 -0
- package/dist/agent/cli-provider.js +69 -1
- package/dist/agent/cost.js +6 -4
- package/dist/agent/engine.js +332 -42
- package/dist/agent/grammar.js +73 -6
- package/dist/agent/ir.js +220 -46
- package/dist/agent/lint.js +77 -0
- package/dist/agent/openai.js +6 -0
- package/dist/bin/verikun.js +0 -0
- package/dist/cli.js +104 -40
- package/dist/ui/selector.js +17 -1
- package/dist/version.js +1 -1
- package/package.json +1 -1
- package/dist/drivers/simctl.js +0 -156
package/README.md
CHANGED
|
@@ -103,7 +103,7 @@ vk screenshot # -> ./.verikun/screen.png
|
|
|
103
103
|
### AI
|
|
104
104
|
| Command | Description |
|
|
105
105
|
|---|---|
|
|
106
|
-
| `ai <file> [--model m] [--max-cost-usd n] [--timeout dur] [--cost-override in/out] [--effort e] [--package pkg] [--app-build id] [--server url] [--show-plan] [--recompile] [--json]` | Run a plain-English test: compile it to a deterministic plan once, replay it model-free, and self-heal failures via the model. Needs `ANTHROPIC_API_KEY` or `OPENAI_API_KEY` (per model), or no key with `--model codex-cli` (a logged-in `codex` CLI). See [AI](#ai--natural-language-tests). |
|
|
106
|
+
| `ai <file> [--model m] [--max-cost-usd n] [--timeout dur] [--cost-override in/out] [--effort e] [--package pkg] [--app-build id] [--server url] [--show-plan] [--recompile] [--json]` | Run a plain-English test: compile it to a deterministic plan once, replay it model-free, and self-heal failures via the model. Needs `ANTHROPIC_API_KEY` or `OPENAI_API_KEY` (per model), or no key with `--model codex-cli` / `cursor-cli` (a logged-in `codex` / `cursor-agent` CLI). See [AI](#ai--natural-language-tests). |
|
|
107
107
|
| `suite <dir> [--app <id>] [--name n] [--server url] [--json]` (+ all `ai` flags) | Run every `*.md` in `<dir>` as one sequential suite with an overview report and a non-zero exit on failure — the CI gate. See [Suites](#suites--run-a-directory-of-tests). |
|
|
108
108
|
|
|
109
109
|
### Remote
|
|
@@ -211,8 +211,9 @@ with no model calls on the happy path**. The model is woken only to *repair* a s
|
|
|
211
211
|
whose selector stops resolving; a green run persists the repaired plan, so the next
|
|
212
212
|
run is free again. That is what keeps a CI suite's steady-state token cost near zero.
|
|
213
213
|
Needs `ANTHROPIC_API_KEY` (Claude models) or `OPENAI_API_KEY` (OpenAI models) — or **no
|
|
214
|
-
key** with `--model codex-cli`, which
|
|
215
|
-
|
|
214
|
+
key** with `--model codex-cli` / `--model cursor-cli`, which drive an already-logged-in
|
|
215
|
+
`codex` or `cursor-agent` CLI off your ChatGPT / Cursor subscription (log in once;
|
|
216
|
+
verikun just needs the binary on PATH).
|
|
216
217
|
|
|
217
218
|
```sh
|
|
218
219
|
# onboarding.md (plain English):
|
|
@@ -232,6 +233,14 @@ like permission dialogs) and **bounded loops** (`repeat … until`, e.g. scroll
|
|
|
232
233
|
row appears) — control flow a flat [`batch`](#batch) script can't express. Loops carry
|
|
233
234
|
a hard iteration cap and stop early if the screen stops changing.
|
|
234
235
|
|
|
236
|
+
An `if-present` guard **waits for its selector to settle** before deciding the optional UI
|
|
237
|
+
isn't there, so a dialog that animates in a beat after the transition is still caught. The
|
|
238
|
+
window guarantees at least two looks at the screen (wall clock alone isn't a usable unit —
|
|
239
|
+
a UI dump ranges from ~200ms on a fast phone to ~2.5s on an emulator), so an absent guard
|
|
240
|
+
costs about one extra dump. `VERIKUN_GUARD_SETTLE_MS` tunes it; `0` restores the old
|
|
241
|
+
single-shot probe. A loop's own exit check never pays this window — it's absent on every
|
|
242
|
+
iteration by construction, which is what makes it a loop.
|
|
243
|
+
|
|
235
244
|
- **Progress streams to stderr** (so a CI job never goes silent); **stdout is the
|
|
236
245
|
report path** (or a JSON summary with `--json`). The compiled plan is logged to the
|
|
237
246
|
run before it executes, for troubleshooting.
|
|
@@ -243,9 +252,12 @@ a hard iteration cap and stop early if the screen stops changing.
|
|
|
243
252
|
- **`--model`** picks the model and its provider — Anthropic (`claude-haiku-4-5` ·
|
|
244
253
|
`claude-sonnet-4-6` (default) · `claude-opus-4-8` · `claude-fable-5`), OpenAI
|
|
245
254
|
(`gpt-5.4-mini` · `gpt-5.4` · `gpt-5.5`), each read from its own key
|
|
246
|
-
(`ANTHROPIC_API_KEY` / `OPENAI_API_KEY`), or
|
|
247
|
-
|
|
248
|
-
`--max-cost-usd` /
|
|
255
|
+
(`ANTHROPIC_API_KEY` / `OPENAI_API_KEY`), or a CLI backend — **`codex-cli`** (the
|
|
256
|
+
logged-in `codex` binary) or **`cursor-cli`** (`cursor-agent`) — which need no key at
|
|
257
|
+
all: spend is on your subscription, so their cost line is `$0` and `--max-cost-usd` /
|
|
258
|
+
`--cost-override` are no-ops. Each CLI picks its own underlying model, and runs
|
|
259
|
+
read-only in a scratch directory so it never touches your working tree.
|
|
260
|
+
**`--recompile`** ignores the cache.
|
|
249
261
|
- An `ai` run records like any other flow, so it produces the same JUnit + HTML report —
|
|
250
262
|
with the cost line and any **suggested test improvements** (workarounds the model
|
|
251
263
|
applied, which you can fold back into the prose to stabilize the test and cut tokens).
|
|
@@ -282,8 +294,8 @@ vk suite tests/ --app com.example.app --server "$VERIKUN_SERVER" # remote devi
|
|
|
282
294
|
- **`index.html`** — a summary page linking every test's `report.html`.
|
|
283
295
|
- **Exit code is the CI gate:** `1` if any test failed, `0` all green, `2` bad/empty
|
|
284
296
|
directory. All `ai` flags (`--model`, `--max-cost-usd`, `--timeout`, …) apply to
|
|
285
|
-
every test; the provider (`ANTHROPIC_API_KEY` / `OPENAI_API_KEY`, or the `codex`
|
|
286
|
-
`--model codex-cli`) is checked up front.
|
|
297
|
+
every test; the provider (`ANTHROPIC_API_KEY` / `OPENAI_API_KEY`, or the `codex` /
|
|
298
|
+
`cursor-agent` CLI for `--model codex-cli` / `cursor-cli`) is checked up front.
|
|
287
299
|
|
|
288
300
|
## Remote devices — `vk server`
|
|
289
301
|
|
|
@@ -363,7 +375,11 @@ class:Button simplified type ("Button") or full class ("android.widget.Button
|
|
|
363
375
|
```
|
|
364
376
|
|
|
365
377
|
Modifiers: `--contains` makes text/desc matches substring-based; `--index N`
|
|
366
|
-
selects the Nth match (0-based) when a selector intentionally matches several
|
|
378
|
+
selects the Nth match (0-based) when a selector intentionally matches several;
|
|
379
|
+
`--enabled` matches only a control that is **actionable right now** — use it for a
|
|
380
|
+
Submit/Check button the app disables until a form is valid, since such a button is
|
|
381
|
+
present long before it is usable and tapping presence taps a dead control (with
|
|
382
|
+
auto-wait this reads as "wait until it is pressable").
|
|
367
383
|
If a selector for an action matches more than one element and no `--index` is
|
|
368
384
|
given, the command fails with exit code 2 and lists the candidates — it never
|
|
369
385
|
taps a guess.
|
package/dist/agent/claude.js
CHANGED
|
@@ -35,6 +35,12 @@ class ClaudeProvider {
|
|
|
35
35
|
JSON.stringify(input.seed, null, 2));
|
|
36
36
|
}
|
|
37
37
|
parts.push('NATURAL-LANGUAGE TEST:\n' + input.nl);
|
|
38
|
+
if (input.retryFeedback) {
|
|
39
|
+
// Last, so it is the freshest thing in context: a previous compile of this same
|
|
40
|
+
// test lost something the prose stated. Naming it beats hoping the retry differs.
|
|
41
|
+
parts.push('YOUR PREVIOUS ATTEMPT AT THIS TEST WAS REJECTED. Fix this and emit the whole plan again:\n' +
|
|
42
|
+
input.retryFeedback);
|
|
43
|
+
}
|
|
38
44
|
const { json, usage } = await this.call(grammar_1.GRAMMAR, parts.join('\n\n'), ir_1.PLAN_JSON_SCHEMA, 8192);
|
|
39
45
|
return { plan: (0, ir_1.parsePlan)(json), usage };
|
|
40
46
|
}
|
|
@@ -1,6 +1,7 @@
|
|
|
1
1
|
"use strict";
|
|
2
2
|
Object.defineProperty(exports, "__esModule", { value: true });
|
|
3
|
-
exports.CliProvider = exports.CODEX_SPEC = void 0;
|
|
3
|
+
exports.CliProvider = exports.CURSOR_SPEC = exports.CODEX_SPEC = void 0;
|
|
4
|
+
exports.cursorResultText = cursorResultText;
|
|
4
5
|
exports.schemaInstruction = schemaInstruction;
|
|
5
6
|
exports.extractJson = extractJson;
|
|
6
7
|
const node_fs_1 = require("node:fs");
|
|
@@ -69,6 +70,67 @@ exports.CODEX_SPEC = {
|
|
|
69
70
|
rawText: (stdout) => stdout, // fallback only; the message is read from --output-last-message
|
|
70
71
|
loginHint: 'run `codex login` to sign in with your ChatGPT subscription (no API key needed)',
|
|
71
72
|
};
|
|
73
|
+
/** cursor-agent (Cursor CLI): non-interactive `--print` with a JSON envelope on stdout. Unlike
|
|
74
|
+
* codex it has NO schema flag of any kind, so the schema is injected into the prompt and
|
|
75
|
+
* extractJson/parsePlan do the rest. Two flags are load-bearing beyond the obvious ones:
|
|
76
|
+
* `--trust`, without which a headless call dies at a "Workspace Trust Required" gate and exits 1
|
|
77
|
+
* before the model ever runs; and `--mode ask`, cursor's documented read-only Q&A mode, which
|
|
78
|
+
* stands in for codex's `--sandbox read-only` (cursor's own `--sandbox` only takes enabled/
|
|
79
|
+
* disabled). Plain `--print` "has access to all tools, including write and shell", so `--mode ask`
|
|
80
|
+
* plus the neutral `--workspace` are what keep this a pure transform — and we deliberately never
|
|
81
|
+
* pass --force/--yolo/--approve-mcps. */
|
|
82
|
+
exports.CURSOR_SPEC = {
|
|
83
|
+
id: 'cursor',
|
|
84
|
+
bin: 'cursor-agent',
|
|
85
|
+
schema: 'prompt', // no native schema flag — schemaInstruction() injects it into the prompt
|
|
86
|
+
usesOutputFile: false, // stdout is the only channel out; no --output-last-message equivalent
|
|
87
|
+
buildArgs(prompt, { cwd, model }) {
|
|
88
|
+
const args = [
|
|
89
|
+
'--print', // non-interactive one-shot
|
|
90
|
+
'--output-format', 'json', // a stable envelope instead of TTY-decorated text
|
|
91
|
+
'--mode', 'ask', // read-only Q&A mode: no edits, no shell
|
|
92
|
+
'--workspace', cwd, // root the agent in a neutral temp dir, not the verikun working tree
|
|
93
|
+
'--trust', // required: else headless stops at the workspace-trust prompt
|
|
94
|
+
];
|
|
95
|
+
if (model)
|
|
96
|
+
args.push('--model', model);
|
|
97
|
+
args.push(prompt); // prompt is the trailing positional
|
|
98
|
+
return args;
|
|
99
|
+
},
|
|
100
|
+
rawText: cursorResultText,
|
|
101
|
+
loginHint: 'run `cursor-agent login` to sign in with your Cursor subscription (no API key needed)',
|
|
102
|
+
};
|
|
103
|
+
/** Peel the model's final message out of cursor-agent's `--output-format json` envelope:
|
|
104
|
+
* `{type:"result", subtype:"success", is_error:false, result:"<final text>", …}`.
|
|
105
|
+
* Two things a plain `JSON.parse(s).result` would get wrong:
|
|
106
|
+
* - cursor can exit 0 while reporting failure via `is_error:true` (e.g. it hit a turn limit), which
|
|
107
|
+
* CliProvider's exit-code check cannot see. Left alone, the error prose would flow into
|
|
108
|
+
* extractJson and surface as a misleading "did not return parseable JSON", so map it to the same
|
|
109
|
+
* exit 3 a non-zero exit gets.
|
|
110
|
+
* - an envelope shape drift (or a future default of --output-format text) falls back to the raw
|
|
111
|
+
* stdout, so extractJson's tolerant scan still gets a chance instead of failing outright. */
|
|
112
|
+
function cursorResultText(stdout) {
|
|
113
|
+
let envelope;
|
|
114
|
+
try {
|
|
115
|
+
const parsed = JSON.parse(stdout.trim());
|
|
116
|
+
if (parsed && typeof parsed === 'object' && !Array.isArray(parsed))
|
|
117
|
+
envelope = parsed;
|
|
118
|
+
}
|
|
119
|
+
catch {
|
|
120
|
+
/* not JSON at all (e.g. --output-format text) — fall through to the raw stdout */
|
|
121
|
+
}
|
|
122
|
+
if (!envelope)
|
|
123
|
+
return stdout;
|
|
124
|
+
if (envelope.is_error === true) {
|
|
125
|
+
const detail = typeof envelope.result === 'string' ? tail(envelope.result) : '';
|
|
126
|
+
throw new errors_1.CliError(`\`cursor-agent\` reported an error: ${detail || '(no detail)'}`, 3);
|
|
127
|
+
}
|
|
128
|
+
// Only peel when this really IS the envelope. Parsing alone isn't enough: if cursor ever returns
|
|
129
|
+
// the plan object bare (or renames the field), treating any JSON object as an envelope would
|
|
130
|
+
// blank it to '' and report "returned an empty response" — so anything without a string
|
|
131
|
+
// `result` falls through to the raw stdout, where extractJson can still find the object.
|
|
132
|
+
return typeof envelope.result === 'string' ? envelope.result : stdout;
|
|
133
|
+
}
|
|
72
134
|
// Collision-free temp-file names within a process without needing Math.random() (which the
|
|
73
135
|
// plan-cache/version paths keep deterministic); pid + a counter is enough.
|
|
74
136
|
let tempCounter = 0;
|
|
@@ -94,6 +156,12 @@ class CliProvider {
|
|
|
94
156
|
JSON.stringify(input.seed, null, 2));
|
|
95
157
|
}
|
|
96
158
|
parts.push('NATURAL-LANGUAGE TEST:\n' + input.nl);
|
|
159
|
+
if (input.retryFeedback) {
|
|
160
|
+
// Last, so it is the freshest thing in context: a previous compile of this same
|
|
161
|
+
// test lost something the prose stated. Naming it beats hoping the retry differs.
|
|
162
|
+
parts.push('YOUR PREVIOUS ATTEMPT AT THIS TEST WAS REJECTED. Fix this and emit the whole plan again:\n' +
|
|
163
|
+
input.retryFeedback);
|
|
164
|
+
}
|
|
97
165
|
const json = this.call(grammar_1.GRAMMAR, parts.join('\n\n'), ir_1.PLAN_JSON_SCHEMA);
|
|
98
166
|
// usage:{} — a CLI is billed to the user's subscription, not per token, so cost is $0
|
|
99
167
|
// (documented no-op for --max-cost-usd). The run is still bounded by maxRepairs + --timeout.
|
package/dist/agent/cost.js
CHANGED
|
@@ -21,11 +21,13 @@ const MODELS = {
|
|
|
21
21
|
'gpt-5.4-mini': { input: 0.75, output: 4.5, provider: 'openai' },
|
|
22
22
|
'gpt-5.4': { input: 2.5, output: 15, provider: 'openai' },
|
|
23
23
|
'gpt-5.5': { input: 5, output: 30, provider: 'openai' },
|
|
24
|
-
// CLI-agent
|
|
25
|
-
// token — so price is $0 and --max-cost-usd/--cost-override are inert no-ops (the
|
|
26
|
-
// bounded by maxRepairs + --timeout instead).
|
|
27
|
-
// and
|
|
24
|
+
// CLI-agent backends: billed to the user's ChatGPT/Cursor subscription via an already-logged-in
|
|
25
|
+
// CLI, not per token — so price is $0 and --max-cost-usd/--cost-override are inert no-ops (the
|
|
26
|
+
// run is bounded by maxRepairs + --timeout instead). The `-cli` suffix reads clearly as "the
|
|
27
|
+
// CLI" and keeps these from colliding with the CLIs' own model aliases — cursor in particular
|
|
28
|
+
// offers `gpt-5.3-codex`, `gpt-5.4-high`, `claude-opus-4-8-thinking-high` and friends.
|
|
28
29
|
'codex-cli': { input: 0, output: 0, provider: 'codex' },
|
|
30
|
+
'cursor-cli': { input: 0, output: 0, provider: 'cursor' },
|
|
29
31
|
};
|
|
30
32
|
exports.MODEL_PRICES = Object.fromEntries(Object.entries(MODELS).map(([m, s]) => [m, { input: s.input, output: s.output }]));
|
|
31
33
|
exports.ALLOWED_MODELS = Object.keys(MODELS);
|